Hi JP and Gerson, during this call http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting2... we agreed that the attributes TmpDir, ScratchDir and DataDir part of the Computing Resource (e.g., a batch system) should be removed: http://forge.ogf.org/sf/sfmain/do/go/artf6196 the values can be discovered by the job in the worker node environment. Do you confirm that this is ok with you? Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
Sergio, I disagree. A remote user wanting to submit and run a job has to know something about the execution environment in order to even be able to use TmpDir, ScratchDir, and DataDir. First, even if they send a probe job to discover those paths, what will the job look at to discover these directories? Some environment variable that points to these directories, or something else? What environment variable does it look it? Are we implying some other environment variable standard so that it can discover these paths? BTW, we have talked about standard execution environment variables, which I think is a good thing, but I believe GLUE should be flexible enough to allow remove users to discover these paths even when standard variables aren't there. I don't see a reason to tie GLUE to standard environments. The TeraGrid users standard environment variables, AND we're going to publish the paths those variables point to thru information services so that users can construct jobs that use these paths (without launching probe jobs). Regards, JP On May 8, 2008, at 9:17 AM, Sergio Andreozzi wrote:
Hi JP and Gerson,
during this call http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting2...
we agreed that the attributes TmpDir, ScratchDir and DataDir part of the Computing Resource (e.g., a batch system) should be removed:
http://forge.ogf.org/sf/sfmain/do/go/artf6196
the values can be discovered by the job in the worker node environment.
Do you confirm that this is ok with you?
Cheers, Sergio
-- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
JP Navarro wrote:
I disagree. A remote user wanting to submit and run a job has to know something about the execution environment in order to even be able to use TmpDir, ScratchDir, and DataDir. First, even if they send a probe job to discover those paths, what will the job look at to discover these directories? Some environment variable that points to these directories, or something else? What environment variable does it look it? Are we implying some other environment variable standard so that it can discover these paths?
in OGF, there is this document that motivates the request for removal of the mentioned attributes: http://forge.ogf.org/sf/go/doc15010
BTW, we have talked about standard execution environment variables, which I think is a good thing, but I believe GLUE should be flexible enough to allow remove users to discover these paths even when standard variables aren't there. I don't see a reason to tie GLUE to standard environments.
I understand your point. Let's wait for the comment fro Laurence which promoted this document. Cheers, Sergio
The TeraGrid users standard environment variables, AND we're going to publish the paths those variables point to thru information services so that users can construct jobs that use these paths (without launching probe jobs).
Regards,
JP
On May 8, 2008, at 9:17 AM, Sergio Andreozzi wrote:
Hi JP and Gerson,
during this call http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting2...
we agreed that the attributes TmpDir, ScratchDir and DataDir part of the Computing Resource (e.g., a batch system) should be removed:
http://forge.ogf.org/sf/sfmain/do/go/artf6196
the values can be discovered by the job in the worker node environment.
Do you confirm that this is ok with you?
Cheers, Sergio
-- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
-- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
Hi JP, I am pretty sure that we have had this discussion before. My point of view is that only information needed to make the job arrive on the cluster should be exposed. Any information that the job needs when it is on the cluster should be discovered locally. For our use cases the idea of preparing a job to run on a particular cluster and then sending it directly there does not exist. If these attributes are important to your user community now then we will have to put them in the schema, however, I would hope that in the future when we have a local discovery method, we could remove them. Laurence Sergio Andreozzi wrote:
JP Navarro wrote:
I disagree. A remote user wanting to submit and run a job has to know something about the execution environment in order to even be able to use TmpDir, ScratchDir, and DataDir. First, even if they send a probe job to discover those paths, what will the job look at to discover these directories? Some environment variable that points to these directories, or something else? What environment variable does it look it? Are we implying some other environment variable standard so that it can discover these paths?
in OGF, there is this document that motivates the request for removal of the mentioned attributes: http://forge.ogf.org/sf/go/doc15010
BTW, we have talked about standard execution environment variables, which I think is a good thing, but I believe GLUE should be flexible enough to allow remove users to discover these paths even when standard variables aren't there. I don't see a reason to tie GLUE to standard environments.
I understand your point. Let's wait for the comment fro Laurence which promoted this document.
Cheers, Sergio
The TeraGrid users standard environment variables, AND we're going to publish the paths those variables point to thru information services so that users can construct jobs that use these paths (without launching probe jobs).
Regards,
JP
On May 8, 2008, at 9:17 AM, Sergio Andreozzi wrote:
Hi JP and Gerson,
during this call http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting2...
we agreed that the attributes TmpDir, ScratchDir and DataDir part of the Computing Resource (e.g., a batch system) should be removed:
http://forge.ogf.org/sf/sfmain/do/go/artf6196
the values can be discovered by the job in the worker node environment.
Do you confirm that this is ok with you?
Cheers, Sergio
-- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
Hi JP, can you refine these definitions? And also, please, try to make clear the differences between these 3 directories (do we need all of them?) TmpDir: The absolute path of a temporary directory shared across execution environment instances (i.e., worker nodes). This directory must be available to programs using the normal file access primitives (open/read/write/close) and may be used as shared space by programs running on multiple hosts (e.g. MPI) ScratchDir: The absolute path for a shared directory available for application data. Typically a POSIX accessible transient disk space shared between the execution environment instances. It may be used by MPI applications or to store intermediate files that need further processing by local jobs or as staging area, specially if the execution environment instances have no internet connectivity DataDir: ... Cheers, Sergio JP Navarro wrote:
Sergio,
I disagree. A remote user wanting to submit and run a job has to know something about the execution environment in order to even be able to use TmpDir, ScratchDir, and DataDir. First, even if they send a probe job to discover those paths, what will the job look at to discover these directories? Some environment variable that points to these directories, or something else? What environment variable does it look it? Are we implying some other environment variable standard so that it can discover these paths?
BTW, we have talked about standard execution environment variables, which I think is a good thing, but I believe GLUE should be flexible enough to allow remove users to discover these paths even when standard variables aren't there. I don't see a reason to tie GLUE to standard environments.
The TeraGrid users standard environment variables, AND we're going to publish the paths those variables point to thru information services so that users can construct jobs that use these paths (without launching probe jobs).
Regards,
JP
On May 8, 2008, at 9:17 AM, Sergio Andreozzi wrote:
Hi JP and Gerson,
during this call http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting2...
we agreed that the attributes TmpDir, ScratchDir and DataDir part of the Computing Resource (e.g., a batch system) should be removed:
http://forge.ogf.org/sf/sfmain/do/go/artf6196
the values can be discovered by the job in the worker node environment.
Do you confirm that this is ok with you?
Cheers, Sergio
-- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
-- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
participants (3)
-
JP Navarro -
Laurence Field -
Sergio Andreozzi