December 2007 - ogsa-dmi-wg

Aggregation Proposal
by Ravi Madduri 19 Dec '07

19 Dec '07

All: Please find attached my aggregation proposal Please comment -- Ravi K Madduri The Globus Alliance | Argonne National Laboratory | University of Chicago http://www-unix.mcs.anl.gov/~madduri

3 2

Retries Proposal
by Ravi Madduri 19 Dec '07

19 Dec '07

Hi Please find attached the proposal for retrying a transfer in case of a failure. I made two separate documents for the two proposals I had. I am still working on the Transfer aggregation proposal. Please comment -- Ravi K Madduri The Globus Alliance | Argonne National Laboratory | University of Chicago http://www-unix.mcs.anl.gov/~madduri

3 5

Telcon Minutes
by Mario Antonioletti 13 Dec '07

13 Dec '07

DMI Telcon - 12/12/07 ===================== Attendees: Mario Antonioletti, EPCC Steven Newhouse, Microsoft Michel Drescher, Fujitsu Ravi Madduri, ANL Agenda: 1. IPR Notice 2. Agenda Bashing 3. Ravi's modified proposal 4. Planning 5. AOB +--- 1. IPR Notice 2. Agenda Bashing Michel proposed an extra agenda item to discuss the updating of the spec. 3. Ravi's Proposals Ravi circulated his Retries Proposal - the aggregation one still needs to be done. Removed the status state transition - state transition now takes place when it Fails. After a number of retries it goes into a failed state. InstanceAttributes now has a Retries attribute. Michel had some comments: Want MaxAttempts to be optional. If it is not specified it is one. Need to place in a question mark on the MaxAttempts element in the TransferRequirements complex element. Need to semantically link EndNoLaterThan with MaxAttemts - compliance for both values should go in the text. EndNoLaterThan takes precedence. The alignment of MaxAttempts and Retries - Ravi's proposal states that if the values are equal then the transfer has to fail - mixing semantics of MaxAttempts and Retries - which is a bit confusing.. The Retries is how many failed attempts have been had - Steven suggested renaming Retries to Attempt First time it is set to 1. When it fails and it tries again if it's equal to MaxAttempts then it fails unless the EndNoLaterThan is violated first. Michel suggests using postiveIntegers for the data type for Attempt and MaxAttempts. Ravi going to work on the aggregation proposal and send out to the list. 3b. Updating the spec. For version 43 of the spec Michel wanted to make some updates - have 6 & 7 comments, some changes and delete some things from the spec. Section 3.3 - Steven added text - would like to accept it. Fine. Two comments on p10 from Michel - do not want to wait for a wide specification of URIs for protocols from OGSA. Steven suggested taking the comment out of the document but adding as a tracker for using a standard for URI protocol identifiers from OGSA. Removed the undo strategy comment from the same page. Removed Mario's comment on p13. The sourceMode and sinkMode are removed - Mario's comment goes with it. Section 4.2.1.3 - Ravi stated "Can these include like parallelStreams, TCP Buffer size etc ?" Michel would like it removed. Ravi asks for an extensibility point. Steven has no objection but they are protocol dependent parameters and there had been a concern about having it in a protocol neutral part. Michel fine about having an extensibility and if an implementation does not understand it it can throw an Notsatisfiable fault. Consensus: fine to add an extensibility element. Michel has added one. Remove Ravi's comment. Accept typo fixes on p16 and section 5 that Ravi spotted. Section 5.1.2 - Michel's DTI failure text - accepted. Ravi's comment about having an enumeration of strings - the value of the status. Discussion as to whether we can borrow from the state models proposed in SAGA and/or BES for jobs. Steven had taken action from the meeting in Seattle to look at the SAGA and BES models but it has not been done. May have the same states but used in different models they may have different semantics for each model. Ravi thought that BES used to have 3 overlapping states - not sure if it makes sense to re-use it but would have to check. Steven agrees - people might assume that the states mean the same thing across all specs. Michel - concepts and semantics are similar but would feel uneasy to align - might be an interesting topic for OGSA discussion. Would prefer to start with our own DMI state and re-examine the state alignment at some later point. Not clear if it makes sense to align all of this. We can put a tracker for the state model for a future version of the spec to try and harmonise future state models. For now we make our own enumeration of strings - not URIs. Remove Mario's comment from p28. 4. Planning Michel will wrangle the XML Schema and WSDL - draw out a common bit and then come up with WS-I and WSRF renderings of these. The WSRF rendering must include the WS-I rendering because of the base faults - will follow the ByteIO model for doing this. Will put up a new version of the doc and then Ravi can add to the spec. Retries could go into the spec directly. Aggregation can go to the list and then we can have a look at it. If there are no objections then it can go into the spec too. Complete the spec and review the spec in the new year. Have a telcon if there is a need otherwise agrees Ravi's changes on the list. 5. AOB None. A Merry Xmas and a happy new year to all. +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

1 0

New draft 44 of the functional specification
by Michel Drescher 12 Dec '07

12 Dec '07

Folks, please find draft 44 of the spec on Gridforge: https:// forge.gridforum.org/sf/go/doc14200?nav=1 (also attached for people's convenience). I accepted changes and deleted comments as agreed on today's call. I altered the text as agreed on today's call, leaving those changes unaccepted for everybody to have a peck on my changes. Cheers, Michel

1 0

DMI Telcon
by Mario Antonioletti 12 Dec '07

12 Dec '07

Hi, A short reminder that we will be having our usual DMI telcon today: Date: 12th December 2007 When: 9am (Pacific) 11am (Eastern) 5pm (UK) PIN: 2719140 USA: +1 866 500 6738 (Toll: +1-203-480-8000) UK: +44 118 909 2000 a plausible agenda for this meeting will be: Agenda: 1. IPR Notice 2. Agenda Bashing 3. Ravi's modified proposals? 4. Planning 5. AOB Can change the above agenda on the call as required. Mario +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

1 0

DMI Telcon Minutes
by Mario Antonioletti 06 Dec '07

06 Dec '07

OGSA DMI Telcon 05/12/07 ======================== Attendees: Mario Antonioletti, EPCC Allen Luniewski, IBM Steven Newhouse, Microsoft Michel Drescher, Fujitsu Ravi Madduri, ANL Agenda: 1. IPR Notice 2. Previous Minutes and Action Review 3. Agenda Bashing 4. Issues Arising 5. Spec Progress 6. AOB +--- Start off by looking at Ravi's proposals - Ravi likes and is in agreement with the comments that were made. Dealing with aggregations: RFT originally started with one file transfers but this needed to be changed for staging where more than one file had to be transferred at a time. Having separate DEPRs for every file that needs to be transferred in the proposal was seen as being somewhat worrying. Should have a logical representation for the files in a single DEPR. Issue then moved on to whether multiple files can or should be represented in a DEPR - should there be the notion of an aggregate in the DEPR? A counter argument was proposed that DEPRs are opaque things - not XML containers that have associated semantics - produced by the source and sink and, other than the stuff that DMI define, the contents should not interpreted by the DTF/DTI. Having a DEPR representing multiple files may cause problems and introduces ugly protocol dependencies. A DEPR is an abstract thing that is only interpreted at the source and sink. However, the DTF is made aware of the contents through negotiation with the source and sink. Within this model an aggregate is represented by the source and sink and out of scope for DMI. So, if you wanted to transfer a directory the source/sink would specify a URL that gridFTP understands, a / at the end of the URL, which is the interpreted as a directory. This could be done using existing attributes in DMI. This information would be encoded in the URL by the source/sink. Aggregation becomes an implementation detail. Consensus reached: aggregation can already be addressed with the current spec. Ravi is to modify his existing proposal to take this discussion into account and it will be discussed at the next call. On Transfer Retries: Should there be another state for transfer retries or could a retry counter attribute be used instead? Consensus reached: go from the existing states failed -> scheduled -> active - no new state and have a transfer retries counter attribute in the DTI. Ravi to modify his proposal to reflect this and discuss at the next call. Michel's proposal - communicating a transfer failure from the DTI. Could have a fault bound to Start or Activate operations or if the client sets a scheduled data transfer - communicate it back through the state - client can poll to find this out if the data transfer could not be instantiated. It was thought that the fault bound model was a bit restrictive and that the state bound model was more general. Putting it in a failed state and putting information as to why it failed would be a good thing as it would make it easier to implement. Have a single point of contact for the client - service has gone into the failed state and discover what went wrong. Could find out whether it was a syntactical problem or a protocol problem, etc. Consensus reached: drop the fault based communication to have a state based communication instead. Michel to modify his proposal accordingly and move this text into the spec. Michel gets the write token on the spec until this is completed. Steven and Ravi have made modifications to the spec - there were no major objections. DONM: 12/12/07, 9am (Pacific) 11am (CENTRAL) 5pm (UK). +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

1 0

Proposal on fault communication for the DTI
by Michel Drescher 06 Dec '07

06 Dec '07

Folks, please find attached my proposal on how DMI could model the fault communication for the DTI. I deliberately did not fully write the proposal as a finished text for the specification. I first wanted to make clear why I designed the proposal as it is. If we agree to go ahead with whatever solution, we then still can add the solution to the spec, and bang our heads together on the words. :) Cheers, Michel

4 7

Proposals
by Ravi Madduri 05 Dec '07

05 Dec '07

Hi Please find attached my proposals to include more than one source/sink pairs in the RequestDataTransferInstance and another proposal to include the retry information in the transfer request. I can talk about the proposal more in tomorrow's call Thanks -- Ravi K Madduri The Globus Alliance | Argonne National Laboratory | University of Chicago http://www-unix.mcs.anl.gov/~madduri

4 3

DMI Telcon
by Mario Antonioletti 05 Dec '07

05 Dec '07

Hi, A short reminder that we will be having a DMI telcon today. **NOTE** the slightly earlier start time. Date: 5th December 2007 When: 8:30am (Pacific) 10:30am (Eastern) 4:30pm (UK) PIN: 2719140 USA: +1 866 500 6738 (Toll: +1-203-480-8000) UK: +44 118 909 2000 a plausible agenda for this meeting will be: Agenda: 1. IPR Notice 2. Previous Minutes and Action Review 3. Agenda Bashing 4. Ravi's modifications and proposal(s) 5. Michel's proposal 6. Spec Progress 7. AOB Can change the above agenda on the call as required. Mario +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

1 0

Another new spec version uploaded
by Steven Newhouse 05 Dec '07

05 Dec '07

All, I've added in the non-normative text (as section 3.3) proposed previously and discussed on the last call to the latest version of the specification and placed the new version on gridforge. Steven

1 0