PGI and DataMINX Data Transfer Service
Hi, Apologies if you get this more than once - Dave tried to email it to the list earlier today and it appears not to have got through so trying from my account to see if this makes a difference. Mario -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ---------- Forwarded message ---------- Date: Tue, 27 Oct 2009 11:48:40 -0000 From: david.meredith@stfc.ac.uk Hi folks, Can I introduce our DataMINX project (http://code.google.com/p/dtsproject ). We are currently involved with a data transfer service for supporting a range of different protocols and would like to adopt and potentially contribute to a relevant OGF spec if appropriate. DMI and JSDL file staging have their strengths and weaknesses in this area, and perhaps there's an opportunity to exploit these strengths within each standard in some way (Steve, hope you don't mind me re-using your words here). DataMINX is a UK-Australia collaboration currently involving the UK NGS, OMII-UK, ARCS and Intersect. May I also point you to our wiki (http://code.google.com/p/dtsproject/wiki/DMI_JSDL_Issues ) where we have put together a summary of some JSDL/DMI issues for use in a bulk data transfer type service. The wiki presents a couple of draft proposals (DMIB and JSDL Data Transfer). Please note both are merely rough proposals with the aim of capturing requirements and for compare/contrast/creating-conjecture (hopefully lots to discuss). For brevity, I have summarised the proposals below, but please do refer to the wiki as there are lots of considerations/requirements and discussion points. Kind regards, Dave Meredith (on behalf of the DataMinx team) DMIB The first proposal ('DMIB' where we have used 'B' to indicate Bulk-transfer) re-uses the existing DMI elements in a new document-literal wrapped style message model. This does not modify the existing DMI (except for the addition of some new xsd:any extensions), rather, it wraps together the SourceDEPR and SinkDEPR elements (both wsa:EndpointReferenceTypeS) and the dmi:TransferRequirements into a single wrapping element. This 'wrapping' element can be defined multiple times in a single request packet in order to define a bulk transfer. (The dmi:DataLocations, dmi:Data elements are not modified and are re-used as per the dmi spec). Semantically, this appears to be similar to defining multiple source and target JSDL DataStaging elements. JSDL Data Transfer The second draft proposal, defines a <DataTransfer> element (rather than DataStaging). This is used to define some extra elements and modifies the format of the <DataStaging> element to be more suited to bulk transfers, since staging data to/from an intermediary compute resource may not appropriate here (implementation detail). Additions include: a repositioned Credential element, new abstract URIProperties element, client defined sub-transfer id (is used in conjunction with the overall bulk transfer id for drilling down to determine status of sub-transfers), added a modified dmi:TransferRequirements/ element that adds the jsdl:CreationFlag/. -----Original Message----- From: Steve Crouch [mailto:s.crouch@omii.ac.uk] Sent: 26 October 2009 10:18 To: Morris Riedel Cc: Meredith, David (STFC,DL,ESC); gerson galang; Alex Arana; Peter Turner; Mario Antonioletti; Shahbaz Memon Subject: PGI and DataMINX Data Transfer Service Hi Morris, You may recall from the PGI telecon we had on the 2nd October we discussed a set of JSDL requirements for datastaging that have come out of the Australian DataMINX project (including Credential and additional 'URI properties' elements within Source and Target elements). As a recap, they have secured funding to to develop a scalable, production-level, heavy-use Data Transfer Service for deployment, and wish to adopt the relevant OGF standard(s) for the service and contribute to these standards. They're also currently pursuing requirements for an OGSA-DMI rendering within the DMI WG. I've cc'd Peter Turner, PI of the DataMINX project, David Meredith from STFC (UK), Gerson Galang from ARCS, and Alex Arana from Intersect who are involved in the project. Previously, David's been heavily involved in developing the JSDL parameter sweep extension. They can elaborate with far greater precision what is required here. I've also cc'd Mario Antonioletti and Shahbaz Memon from the OGSA-DMI WG, where some progress has been made already within DMI. What we've realised is that both DMI and JSDL have their strengths and weaknesses in this area, and perhaps there's an opportunity to exploit these strengths within each standard in some way. I recall you were planning to come up with a JSDL profile that includes the requirements we discussed e.g. from SRM (amongst others). Are you aware at this stage what your plans are for developing the profile? If it's ok, if you could let us know your plans for the profile and how we could be involved that would be great. Many thanks, Cheers Steve -- Dr Stephen Crouch, Software Architect, OMII-UK, School of Electronics and Computer Science, Room 4067, Level 4, Building 32, University of Southampton, Highfield, Southampton SO17 1BJ Tel: +44 (0)23 8059 8787 EMail: s.crouch@omii.ac.uk Fax: +44 (0)23 8059 3045 WWW: http://www.ecs.soton.ac.uk/~stc -- Scanned by iCritical.
participants (1)
-
Mario Antonioletti