A few comments on the draft:
1.
Rename the document to File Staging as opposed to Data Staging.
We are explicit about files throughout the document and do not address
tructured data.
2.
FileSystemName – I’d vote for ignoring it if set without
generating faults. If we are going to honour it then we have to support the JSDL
FileSystem model which would add to the complexity IMHO.
3.
FileName – this is the local relative filename or directory for
recursive copying. I think this should be relative to the users HOME directory
(or equivalent) or the WorkingDirectory element of HPCA if specified. Not sure
if that is mentioned anywhere.
4.
CreationFlag – Where in the JSDL spec does it mention the
‘dontOverwrite’ error? This JSDL definition seems OK to me.
5.
Staging Schedule – Supporting this exposes too much of the
cluster architecture IMHO. As it’s a request and can be ignored I would ignore
it!
6.
File Staging failure. Good questions. Failure during staging in
I would suggest – Once any failure occurs stop any other transfers, cleanup all
files, and go to failure state before the job starts. On stageout – attempt ALL
stage out activities, do not execute any delete on termination of failed file
transfers, go to failed state reporting failed transfers once all transfers
have been attempted.
7.
Security. One thing to consider (we’ve had many discussion in
DMI-WG about this – you might want to look at the current spec draft) is what
to do if different credentials are needed to get different files. Can the
activity credential specify multiple tokens? How could I associate these tokens
with a particular file source/target action? My reading of the activity
credential profile is that there is no way to associate the (single?)
credential with a particular action. I may need one credential to start the
job, another to get my files, and a third to put the files after job execution.
Some things to discuss at the
next call.
Steven