Hi, First of all my apologies but I completely forgot that we were scheduled to have a call last Monday - it somehow fell off my outlook calendar, still no excuse. We will now have the call this Friday at 11am UK time (GMT+1) which makes it 8pm in Australia. Details as below: Agenda ====== - Agenda bashing - Progress update - MAA determine if proposed changes can be passed off as errata. - AL determine if existing spec can transfer multiple directories living on the same host. - SM WSRF OGSA-DMI rendering. - Other stuff ... - Planning Please let me know if anything else should be added to the list. Call details ============ Participant access code: 23057332 then # Primary dial-in number: 0870 2407821 or if dialling internationally +44 2078193600 Or local access alternatives: USA West Coast 4089616553 USA East Coast 7183541169 Germany (0)6950070811 Finland (0)969379595 Australia 0282239343 Mario +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+ -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
Hi folks, Further to Mario's email, a couple of useful links for Friday's meeting: 1) The proposal for a wrapped message request is at the following link (for transferring multiple files and directories in one atomic request by wrapping multiple src-sink DEPRs): http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai n/resources/dmi-messagesProposals.xsd An example at: http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai n/resources/dmi-messagesProposals.xml 2) For reference, the second link points to the current DataMINX schema which adopts more of a JSDL/HPC file staging approach (using multiple src and target URIs rather than DEPRs). http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai n/resources/minx-dts.xsd An example at: http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai n/resources/MINX-DataTransferRequest-Example.xml To recap, in DataMinx we need to: a) Define a transfer consisting of multiple [src-sink] declarations in a single request document where the data may reside on different hosts. b) Define all messages in a WS-I document-literal-wrapped format (since we are using a RESTtful style WS and our messages are routed via JMS queues to appropriate worker nodes who will perform the transfer). Not sure how this would fit with the WSRF factory-instance approach? dave. -----Original Message----- From: ogsa-dmi-wg-bounces@ogf.org [mailto:ogsa-dmi-wg-bounces@ogf.org] On Behalf Of Mario Antonioletti Sent: 15 July 2009 16:24 To: OGSA-DMI Subject: [ogsa-dmi-wg] Rescheduled Call for this Friday. Hi, First of all my apologies but I completely forgot that we were scheduled to have a call last Monday - it somehow fell off my outlook calendar, still no excuse. We will now have the call this Friday at 11am UK time (GMT+1) which makes it 8pm in Australia. Details as below: Agenda ====== - Agenda bashing - Progress update - MAA determine if proposed changes can be passed off as errata. - AL determine if existing spec can transfer multiple directories living on the same host. - SM WSRF OGSA-DMI rendering. - Other stuff ... - Planning Please let me know if anything else should be added to the list. Call details ============ Participant access code: 23057332 then # Primary dial-in number: 0870 2407821 or if dialling internationally +44 2078193600 Or local access alternatives: USA West Coast 4089616553 USA East Coast 7183541169 Germany (0)6950070811 Finland (0)969379595 Australia 0282239343 Mario +----------------------------------------------------------------------- + |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +----------------------------------------------------------------------- + -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg -- Scanned by iCritical.
Hi Dave,
a) Define a transfer consisting of multiple [src-sink] declarations in a single request document where the data may reside on different hosts.
- Assuming if DTS accepting multiple sub-transfers as an atomic operation, - what about getting status of individual transfer, because this might be possible that some transfers are Started/failed/Done at some point. - This is not only related to Status, the client requires InstanceAttributes (StartTime, State, CompletionTime, Attempts, BytesTransferred) for each of the sub-transfer - Consider if client cancels or aborts any sub-transfer I think this information is hardly be projected in the case of multiple sub-transfers grouped in a single large transfer instance. IMHO DTS port type can define operations to prepare/send multiple transfers at once, and as a result the client gets a set of transfer ids to track individual ones. Cheers, Shahbaz
-----Original Message----- From: ogsa-dmi-wg-bounces@ogf.org [mailto:ogsa-dmi-wg-bounces@ogf.org] On Behalf Of Mario Antonioletti Sent: 15 July 2009 16:24 To: OGSA-DMI Subject: [ogsa-dmi-wg] Rescheduled Call for this Friday.
Hi, First of all my apologies but I completely forgot that we were scheduled to have a call last Monday - it somehow fell off my outlook calendar, still no excuse.
We will now have the call this Friday at 11am UK time (GMT+1) which makes it 8pm in Australia. Details as below:
Agenda ======
- Agenda bashing - Progress update - MAA determine if proposed changes can be passed off as errata. - AL determine if existing spec can transfer multiple directories living on the same host. - SM WSRF OGSA-DMI rendering. - Other stuff ... - Planning
Please let me know if anything else should be added to the list.
Call details ============ Participant access code: 23057332 then #
Primary dial-in number: 0870 2407821 or if dialling internationally +44 2078193600 Or local access alternatives: USA West Coast 4089616553 USA East Coast 7183541169 Germany (0)6950070811 Finland (0)969379595 Australia 0282239343
Mario
+----------------------------------------------------------------------- + |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +----------------------------------------------------------------------- + -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
-- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg -- Scanned by iCritical. -- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg
------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
Hi Shabaz, Thanks for the comments. Some responses interleaved below. Cheers, Dave
a) Define a transfer consisting of multiple [src-sink] declarations in a single request document where the data may reside on different hosts.
- Assuming if DTS accepting multiple sub-transfers as an atomic operation
I guess this could be specified by something like the DMI undo strategy, i.e. best effort (transfer as many sub-transfers as possible), all or nothing (fail on first sub-transfer failure).
- what about getting status of individual transfer, because this might be possible that some transfers are Started/failed/Done at some point.
Could each sub transfer should have a separate ID.
- This is not only related to Status, the client requires InstanceAttributes (StartTime, State, CompletionTime, Attempts, BytesTransferred) for each of the sub-transfer
Yes, I agree, e.g. use embedded dmi:TransferRequirements for each sub transfer (see line 118 in the example at http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/main/re... )
- Consider if client cancels or aborts any sub-transfer I think this information is hardly be projected in the case of multiple sub-transfers grouped in a single large transfer instance.
Would this be possible if each of the sub-transfers has an ID also thus returning a set of transfer ids (your next bullet).
IMHO DTS port type can define operations to prepare/send multiple transfers at once, and as a result the client gets a set of transfer ids to track individual ones.
I think that this is what I am trying to propose in the example/proposal - i.e. use something like the wrapped dmi message example to define the 'multi-transfer' port type. The aim is to prevent preventing the need to contact the WS repeatedly for a set of transfers. Unless I have missed something, are there any existing examples of this multi-transfer port type ? -----Original Message----- From: shahbaz.memon@gmail.com [mailto:shahbaz.memon@gmail.com] On Behalf Of Shahbaz Memon Sent: 16 July 2009 13:29 To: Meredith, DJ (David) Cc: Mario Antonioletti; OGSA-DMI Subject: Re: [ogsa-dmi-wg] Rescheduled Call for this Friday. Hi Dave,
a) Define a transfer consisting of multiple [src-sink] declarations in a single request document where the data may reside on different hosts.
- Assuming if DTS accepting multiple sub-transfers as an atomic operation, - what about getting status of individual transfer, because this might be possible that some transfers are Started/failed/Done at some point. - This is not only related to Status, the client requires InstanceAttributes (StartTime, State, CompletionTime, Attempts, BytesTransferred) for each of the sub-transfer - Consider if client cancels or aborts any sub-transfer I think this information is hardly be projected in the case of multiple sub-transfers grouped in a single large transfer instance. IMHO DTS port type can define operations to prepare/send multiple transfers at once, and as a result the client gets a set of transfer ids to track individual ones. Cheers, Shahbaz
-----Original Message----- From: ogsa-dmi-wg-bounces@ogf.org [mailto:ogsa-dmi-wg-bounces@ogf.org] On Behalf Of Mario Antonioletti Sent: 15 July 2009 16:24 To: OGSA-DMI Subject: [ogsa-dmi-wg] Rescheduled Call for this Friday.
Hi, First of all my apologies but I completely forgot that we were scheduled to have a call last Monday - it somehow fell off my outlook calendar, still no excuse.
We will now have the call this Friday at 11am UK time (GMT+1) which makes it 8pm in Australia. Details as below:
Agenda ======
- Agenda bashing - Progress update - MAA determine if proposed changes can be passed off as errata. - AL determine if existing spec can transfer multiple directories living on the same host. - SM WSRF OGSA-DMI rendering. - Other stuff ... - Planning
Please let me know if anything else should be added to the list.
Call details ============ Participant access code: 23057332 then #
Primary dial-in number: 0870 2407821 or if dialling internationally +44 2078193600 Or local access alternatives: USA West Coast 4089616553 USA East Coast 7183541169 Germany (0)6950070811 Finland (0)969379595 Australia 0282239343
Mario
+----------------------------------------------------------------------- + |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +----------------------------------------------------------------------- + -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
-- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg -- Scanned by iCritical. -- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg
------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
-- Scanned by iCritical.
Hi Shabaz, Dave's right, we plan to use the DMI undo strategy to handle mutiple transfers. The plan is for us to give more information to users depending on the subtransfers that have successfully finished or failed. The DTS will make the decision based on the undo strategy that the user has specified in his job. The schema is still a work in progress and there will still be a few more iterations before we freeze modifications to it. My task now is to specify a schema that actually imports from JSDL and OGSA-DMI and define our own elements if they have not been defined anywhere else. Cheers, Gerson On Fri, Jul 17, 2009 at 12:41 AM, Meredith, DJ (David) < david.meredith@stfc.ac.uk> wrote:
Hi Shabaz,
Thanks for the comments. Some responses interleaved below.
Cheers, Dave
a) Define a transfer consisting of multiple [src-sink] declarations in a single request document where the data may reside on different hosts.
- Assuming if DTS accepting multiple sub-transfers as an atomic operation
I guess this could be specified by something like the DMI undo strategy, i.e. best effort (transfer as many sub-transfers as possible), all or nothing (fail on first sub-transfer failure).
- what about getting status of individual transfer, because this might be possible that some transfers are Started/failed/Done at some point.
Could each sub transfer should have a separate ID.
- This is not only related to Status, the client requires InstanceAttributes (StartTime, State, CompletionTime, Attempts, BytesTransferred) for each of the sub-transfer
Yes, I agree, e.g. use embedded dmi:TransferRequirements for each sub transfer (see line 118 in the example at http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/main/re...)
- Consider if client cancels or aborts any sub-transfer I think this information is hardly be projected in the case of multiple sub-transfers grouped in a single large transfer instance.
Would this be possible if each of the sub-transfers has an ID also thus returning a set of transfer ids (your next bullet).
IMHO DTS port type can define operations to prepare/send multiple transfers at once, and as a result the client gets a set of transfer ids to track individual ones.
I think that this is what I am trying to propose in the example/proposal - i.e. use something like the wrapped dmi message example to define the 'multi-transfer' port type. The aim is to prevent preventing the need to contact the WS repeatedly for a set of transfers. Unless I have missed something, are there any existing examples of this multi-transfer port type ?
-----Original Message----- From: shahbaz.memon@gmail.com [mailto:shahbaz.memon@gmail.com] On Behalf Of Shahbaz Memon Sent: 16 July 2009 13:29 To: Meredith, DJ (David) Cc: Mario Antonioletti; OGSA-DMI Subject: Re: [ogsa-dmi-wg] Rescheduled Call for this Friday.
Hi Dave,
a) Define a transfer consisting of multiple [src-sink] declarations in a single request document where the data may reside on different hosts.
- Assuming if DTS accepting multiple sub-transfers as an atomic operation, - what about getting status of individual transfer, because this might be possible that some transfers are Started/failed/Done at some point. - This is not only related to Status, the client requires InstanceAttributes (StartTime, State, CompletionTime, Attempts, BytesTransferred) for each of the sub-transfer - Consider if client cancels or aborts any sub-transfer I think this information is hardly be projected in the case of multiple sub-transfers grouped in a single large transfer instance.
IMHO DTS port type can define operations to prepare/send multiple transfers at once, and as a result the client gets a set of transfer ids to track individual ones.
Cheers,
Shahbaz
-----Original Message----- From: ogsa-dmi-wg-bounces@ogf.org [mailto:ogsa-dmi-wg-bounces@ogf.org] On Behalf Of Mario Antonioletti Sent: 15 July 2009 16:24 To: OGSA-DMI Subject: [ogsa-dmi-wg] Rescheduled Call for this Friday.
Hi, First of all my apologies but I completely forgot that we were scheduled to have a call last Monday - it somehow fell off my outlook calendar, still no excuse.
We will now have the call this Friday at 11am UK time (GMT+1) which makes it 8pm in Australia. Details as below:
Agenda ======
- Agenda bashing - Progress update - MAA determine if proposed changes can be passed off as errata. - AL determine if existing spec can transfer multiple directories living on the same host. - SM WSRF OGSA-DMI rendering. - Other stuff ... - Planning
Please let me know if anything else should be added to the list.
Call details ============ Participant access code: 23057332 then #
Primary dial-in number: 0870 2407821 or if dialling internationally +44 2078193600 Or local access alternatives: USA West Coast 4089616553 USA East Coast 7183541169 Germany (0)6950070811 Finland (0)969379595 Australia 0282239343
Mario
+----------------------------------------------------------------------- + |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/<http://www.epcc.ed.ac.uk/%7Emario/> | +----------------------------------------------------------------------- + -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
-- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg -- Scanned by iCritical. -- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
-- Scanned by iCritical. -- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg
Hi guys, For those who just started reading the mail exchanges and had issues finding the links that Dave sent. Here are the working links.. http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/main/re... http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/main/re... http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/main/re... Sorry, I had to move stuff around today and unclutter that directory so I could only keep the schema we're using on the project in the resources directory. I also uploaded new schemas which now reuse the types/elements defined in OGSA-DMI and JSDL. Cheers, Gerson On Thu, Jul 16, 2009 at 7:04 PM, Meredith, DJ (David) < david.meredith@stfc.ac.uk> wrote:
Hi folks,
Further to Mario's email, a couple of useful links for Friday's meeting:
1) The proposal for a wrapped message request is at the following link (for transferring multiple files and directories in one atomic request by wrapping multiple src-sink DEPRs): http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai n/resources/dmi-messagesProposals.xsd<http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai%0An/resources/dmi-messagesProposals.xsd> An example at: http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai n/resources/dmi-messagesProposals.xml<http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai%0An/resources/dmi-messagesProposals.xml>
2) For reference, the second link points to the current DataMINX schema which adopts more of a JSDL/HPC file staging approach (using multiple src and target URIs rather than DEPRs). http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai n/resources/minx-dts.xsd<http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai%0An/resources/minx-dts.xsd> An example at: http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai n/resources/MINX-DataTransferRequest-Example.xml<http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai%0An/resources/MINX-DataTransferRequest-Example.xml>
To recap, in DataMinx we need to: a) Define a transfer consisting of multiple [src-sink] declarations in a single request document where the data may reside on different hosts. b) Define all messages in a WS-I document-literal-wrapped format (since we are using a RESTtful style WS and our messages are routed via JMS queues to appropriate worker nodes who will perform the transfer). Not sure how this would fit with the WSRF factory-instance approach?
dave.
-----Original Message----- From: ogsa-dmi-wg-bounces@ogf.org [mailto:ogsa-dmi-wg-bounces@ogf.org] On Behalf Of Mario Antonioletti Sent: 15 July 2009 16:24 To: OGSA-DMI Subject: [ogsa-dmi-wg] Rescheduled Call for this Friday.
Hi, First of all my apologies but I completely forgot that we were scheduled to have a call last Monday - it somehow fell off my outlook calendar, still no excuse.
We will now have the call this Friday at 11am UK time (GMT+1) which makes it 8pm in Australia. Details as below:
Agenda ======
- Agenda bashing - Progress update - MAA determine if proposed changes can be passed off as errata. - AL determine if existing spec can transfer multiple directories living on the same host. - SM WSRF OGSA-DMI rendering. - Other stuff ... - Planning
Please let me know if anything else should be added to the list.
Call details ============ Participant access code: 23057332 then #
Primary dial-in number: 0870 2407821 or if dialling internationally +44 2078193600 Or local access alternatives: USA West Coast 4089616553 USA East Coast 7183541169 Germany (0)6950070811 Finland (0)969379595 Australia 0282239343
Mario
+----------------------------------------------------------------------- + |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/<http://www.epcc.ed.ac.uk/%7Emario/> | +----------------------------------------------------------------------- + -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
-- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg -- Scanned by iCritical. -- ogsa-dmi-wg mailing list ogsa-dmi-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-dmi-wg
participants (4)
-
Gerson Galang -
Mario Antonioletti -
Meredith, DJ (David) -
Shahbaz Memon