Hi,
Some thoughts: - most of the batch systems were designed to work with network shared file system. However one can imagine a situation when the home file
- i would prefer to keep this interface as simple as possible and thus handle the only case that can not be handled without interaction with DRMS: staging file from submission host to execution host, as the execution host is usually not know before a job starts.
This brings the new file staging approach closer to what we had in DRMAAv1. The unknown execution host is a very valid argument, since we implicitly assume that staging happens before job start. Your research also shows that at LSF can only copy from / to the submission host, which definitely kills the idea of free server transfers. I can live with that.
- in order to keep the interface really simply i would assume that file names (not necessary the absolute paths) are the same both on execution host and submission host. (again if not, the user can do easily some workaround by copying/moving the file on submission host).
Interesting. So the idea is to stage only whole directories ? What is if I only want to move the STDIN file, and nothing else ?
So my proposition of the DRMAA staging interface looks as follows: split "fileTransfers" attribute into two attributes (also of the OrderedStringList type): - stageInFiles - stageOutFiles which are simple list of files to be staged-in/staged-out (no URLs, only paths). The paths can be relative (to current working directory on submission host, and job working directory on execution host).
I like that, despite the fact that I would like to see single file / wildcard support as discussed last time in the phone call. Let's find some agreement in todays phone call. Best, Peter.