***SPAM*** proposed ActivityCredential and DataStaging extensions for HPC Profile
Folks, I have just uploaded the two initial DRAFTS of the Activity Credential Extension and the Data Staging Extension for the HPC Profile. http://forge.ggf.org/sf/docman/do/listDocuments/projects.ogsa-hpcp-wg/docman .root.drafts As per our discussions last week (thurs) on the conference call, we intend to push forward with the interop demo at SC 2007 based on a combination of the HPC Basic Profile (with X.509 user certs and/or username/password) and these two proposed extensions. We will spend the majority of the next call (Thurs Sept 27) on these two extensions. Please read them ASAP and comment in email (to the mailing list). Thanks, Marty
A few comments on the draft: 1. Rename the document to File Staging as opposed to Data Staging. We are explicit about files throughout the document and do not address tructured data. 2. FileSystemName - I'd vote for ignoring it if set without generating faults. If we are going to honour it then we have to support the JSDL FileSystem model which would add to the complexity IMHO. 3. FileName - this is the local relative filename or directory for recursive copying. I think this should be relative to the users HOME directory (or equivalent) or the WorkingDirectory element of HPCA if specified. Not sure if that is mentioned anywhere. 4. CreationFlag - Where in the JSDL spec does it mention the 'dontOverwrite' error? This JSDL definition seems OK to me. 5. Staging Schedule - Supporting this exposes too much of the cluster architecture IMHO. As it's a request and can be ignored I would ignore it! 6. File Staging failure. Good questions. Failure during staging in I would suggest - Once any failure occurs stop any other transfers, cleanup all files, and go to failure state before the job starts. On stageout - attempt ALL stage out activities, do not execute any delete on termination of failed file transfers, go to failed state reporting failed transfers once all transfers have been attempted. 7. Security. One thing to consider (we've had many discussion in DMI-WG about this - you might want to look at the current spec draft) is what to do if different credentials are needed to get different files. Can the activity credential specify multiple tokens? How could I associate these tokens with a particular file source/target action? My reading of the activity credential profile is that there is no way to associate the (single?) credential with a particular action. I may need one credential to start the job, another to get my files, and a third to put the files after job execution. Some things to discuss at the next call. Steven
A couple of quick comments: Steven Newhouse wrote:
A few comments on the draft:
1. Rename the document to File Staging as opposed to Data Staging. We are explicit about files throughout the document and do not address tructured data.
2. FileSystemName – I’d vote for ignoring it if set without generating faults. If we are going to honour it then we have to support the JSDL FileSystem model which would add to the complexity IMHO.
If you choose not to support FileSystemName and it is present then generating a fault is the approach in line with the JSDL specification: "All elements present in a JSDL document MUST be satisfied ..."
3. FileName – this is the local relative filename or directory for recursive copying. I think this should be relative to the users HOME directory (or equivalent) or the WorkingDirectory element of HPCA if specified. Not sure if that is mentioned anywhere.
Since you don't plan to support filesystems this should be relative to the working directory. This is mentioned in the JSDL spec. -- Andreas Savva
participants (3)
-
Andreas Savva -
Marty Humphrey -
Steven Newhouse