RE: [drmaa-wg] making a custom file path for output
Hi Hrabri, we discussed this issue at some length at GGF13 and were unsure as to how to proceed. We would like to be able to reference labeled attributes, but, I think we're still discussing how to implement this issue in the xsd schema depending on current tooling! I think, unfortunately, our choice of how we implement this in xsd will influence how we specify how to do this in the spec. Clearly this is not ideal and should be the other way round! Watch the JSDL space and let us know how you do this in the end. Cheers and take care, Ali On Tue, 5 Apr 2005, Rajic, Hrabri wrote:
Hi Ali,
Placeholders would be special variables like home directory or job id ($ sign prefixed below), which DRMAA could substitute in job attributes like the output path:
drmaa_set_attribute( $jt, DRMAA_OUTPUT_PATH, $DRMAA_HOME_DIR/MY_APP/my_job_output.$DRMAA_JOB_ID.out"
-Hrabri
-----Original Message----- From: Ali Anjomshoaa [mailto:ali@epcc.ed.ac.uk] Sent: Tuesday, April 05, 2005 10:59 AM To: Rajic, Hrabri Cc: Andreas Haas; Tim Harsch; DRMAA Working Group Subject: RE: [drmaa-wg] making a custom file path for output
Hi Guys,
If you could give me a quick synopsis of what you consider placeholders and in what context, perhaps I can help from the JSDL perspective.
Cheers,
Ali
On Tue, 5 Apr 2005, Rajic, Hrabri wrote:
Andreas,
I would suggest we approach the question as two issues:
1) what are the global placeholders in DRMAA 2) in which (string typed) job attributes the placeholders could be used
Here are few questions that go with the above:
How is JDSL handling placeholders - nice to know? Do all major DRM systems support placeholders well? Do we need a new error code indicating failure in resolving a placeholder? Any other errors needed?
-Hrabri
-----Original Message----- From: owner-drmaa-wg@ggf.org [mailto:owner-drmaa-wg@ggf.org] On Behalf Of Andreas Haas Sent: Tuesday, April 05, 2005 7:37 AM To: Tim Harsch Cc: DRMAA Working Group Subject: Re: [drmaa-wg] making a custom file path for output
Hi Tim,
I fully agree there is a need for this. Please see
http://forge.gridforum.org/tracker/?aid=805
I raised it's priority to P2. I believe that issue must be solved with resubitted DRMAA spec.
Regards, Andreas
On Mon, 4 Apr 2005, Tim Harsch wrote:
I'm sorry I wasn't clear. I guess the heart of my question is: How can create an arbitrarty template for the file and path name? Specifically what constant tells the DRM to insert job id into that filename template.
Said in Perl psuedo form:
drmaa_set_attribute( $jt, $DRMAA_OUTPUT_PATH + ":/tmp/my_job_output.$DRMAA_JOB_ID.out" );
# with that code job outputs look like /tmp/my_job_output.1.out /tmp/my_job_output.2.out /tmp/my_job_output.3.out
# without that code job outputs look like /home/harsch/script_name.o1 /home/harsch/script_name.o2 /home/harsch/script_name.o3
Is there a DRMAA_JOB_ID constant?
----- Original Message ----- From: "Rajic, Hrabri" <hrabri.rajic@intel.com> To: "DRMAA Working Group" <drmaa-wg@gridforum.org> Sent: Monday, April 04, 2005 12:02 PM Subject: RE: [drmaa-wg] making a custom file path for output
There is a simple solution, provided it works for you: to rely on the shell to supply the home directory path instead of DRMAA.
-Hrabri
-----Original Message----- From: owner-drmaa-wg@ggf.org [mailto:owner-drmaa-wg@ggf.org] On Behalf Of Tim Harsch Sent: Monday, April 04, 2005 1:19 PM To: DRMAA Working Group Subject: [drmaa-wg] making a custom file path for output
Hi all, I have a requirement for the app I'm writing ( using Schedule::DRMAAc ) which I'm trying to implement.
Normally the output files from jobs go to home directory and are of the form <jobname>.o<jobid>
I want to move the jobs to a subdir in /tmp, but retain the <jobid> in the filename. I tried this:
( $job_error, $diagnosis ) = drmaa_set_attribute( $jt, $DRMAA_OUTPUT_PATH, ":" . $tmpdir . '/' . __PACKAGE__ . '.' . $DRMAA_PLACEHOLDER_INCR . ".out" );
... but PLACEHOLDER is only for parametric jobs, I've been scanning the docs and have not found a placeholder for job id. Any ideas on how I could accomplish this task?
A similar task would be to move the job output path without changing the filename format at all. How could I do that?
--
---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali@epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------
-- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali@epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------
In GRAM, we have a notion of "substitution variables" which we have reduced in scope for WS-GRAM (web services GRAM in GT 4.0) to only be referenced in certain string values and only consisting of a fixed set of service-defined values. (In previous versions, we allowed the user to define their own substitution mappings.) I have been debating whether there is some reason JSDL needs to support this mechanism for use in GRAM. For values that are static, we can expose the service-defined values as resource properties and just require the client to have prefetched them and "written them in" where they would have used a variable reference. I do not see this as a problem since I never bought the idea that a job description must be "portable" in any sense. The biggest drawback to requiring clients to handle the write-in is when there are values that change on a per-job basis. Then, you need some chatty protocol to establish the job session, fetch its job-specific values, write them in, and then transmit the fully concrete job description. Does JSDL need a way to have free variable references in some of its string values, e.g. references that are understood in the context of a particular service and which are inherently not captured by the lexical scope of the JSDL instance document itself? If these variable names were URI/URNs, there would not be any semantic ambiguity... the consumer of the document would know whether they are capable of rewriting the references to make a meaningful JSDL document. karl On Apr 06, Ali Anjomshoaa loaded a tape reading:
Hi Hrabri,
we discussed this issue at some length at GGF13 and were unsure as to how to proceed. We would like to be able to reference labeled attributes, but, I think we're still discussing how to implement this issue in the xsd schema depending on current tooling!
I think, unfortunately, our choice of how we implement this in xsd will influence how we specify how to do this in the spec. Clearly this is not ideal and should be the other way round!
Watch the JSDL space and let us know how you do this in the end.
Cheers and take care,
Ali
-- Karl Czajkowski karlcz@univa.com
I should have added, in trying to understand the relationship between variable substitution and term referencing that Ali is implying: is it really better to try to reference elements from outside the JSDL document and "override" their values than to embed some reference language in the values? What would go in the JSDL doc itself? Empty or dummy values? Or am I missing the point entirely? karl On Apr 06, Karl Czajkowski loaded a tape reading: ...
Does JSDL need a way to have free variable references in some of its string values, e.g. references that are understood in the context of a particular service and which are inherently not captured by the lexical scope of the JSDL instance document itself? If these variable names were URI/URNs, there would not be any semantic ambiguity... the consumer of the document would know whether they are capable of rewriting the references to make a meaningful JSDL document. ... On Apr 06, Ali Anjomshoaa loaded a tape reading:
Hi Hrabri,
we discussed this issue at some length at GGF13 and were unsure as to how to proceed. We would like to be able to reference labeled attributes, but, I think we're still discussing how to implement this issue in the xsd schema depending on current tooling!
...
Ali
-- Karl Czajkowski karlcz@univa.com
Ali Anjomshoaa wrote:
we discussed this issue at some length at GGF13 and were unsure as to how to proceed. We would like to be able to reference labeled attributes, but, I think we're still discussing how to implement this issue in the xsd schema depending on current tooling!
I think, unfortunately, our choice of how we implement this in xsd will influence how we specify how to do this in the spec. Clearly this is not ideal and should be the other way round!
On Tue, 5 Apr 2005, Rajic, Hrabri wrote:
Placeholders would be special variables like home directory or job id ($ sign prefixed below), which DRMAA could substitute in job attributes like the output path:
drmaa_set_attribute( $jt, DRMAA_OUTPUT_PATH, $DRMAA_HOME_DIR/MY_APP/my_job_output.$DRMAA_JOB_ID.out"
Hmm, I have to say that I don't like arbitrary substitutions in XML content, especially in styles like this. (It's one of the bits I do not care for about Ant, BTW.) But the way I'd handle such a problem is through something like submitting the job through a service that applied an XSLT substitution to the document before producing a doc with "ground terms". If you see what I mean. In the specific case of wanting to have file arguments given to the application made relative to one of our virtual filesystem concepts, there is an argument to be made for some improvement of the JSDL spec. At the moment, the only mechanisms we have for arguments are "relative to the job working directory" or "absolute", neither of which are great for this use-case. (The same argument applies to environment variables, of course.) Does this brain-dump indicate the sort of area where we need to address some kind of solution? Or are there other cases which will need more refinement before we work out how to fix it all? Donal.
participants (3)
-
Ali Anjomshoaa -
Donal K. Fellows -
Karl Czajkowski