RE: [jsdl-wg] application abstraction in JSDL (fwd)
Donal,
Ali Anjomshoaa wrote:
Fwding bounced mail...
(Alexander, if you're going to be writing much about this, please join the mailing list; it simplifies things greatly.)
Sorry. I've subscribed now.
"Lukichev, Alexander S" <alexander.s.lukichev@intel.com>
Our team is currently develops an application that will use JSDL documents as input to invoke jobs in a distributed system. I'd like to comment if possible some aspects of the latest JSDL schema.
When you say "latest schema", is this pre- or post- the recent work done at the face-to-face meeting? It is my impression that many of these issues you raise were actually addressed there... :^) FYI, you should refer to the document at https://forge.gridforum.org/docman2/ViewCategory.php?group_id=1 22&category_id=868 which I believe is the most recent version of the spec.
Well, I refer this one.
Currently to describe the job to run you can specify: a) executable name b) arguments c) environment variables
This could be not enough configurable because executables can reside at different places on the different systems, arguments can be passed in different ways to close applications on different systems, etc. Thus JSDL contains pretty enough information for locating the suitable server, the located server could fail to run the requested application because of lack of configuration information.
Actually, there is no need to specify the executable name at all. An alternative (that we would encourage people to use where possible) is the "application name", and that provides the necessary abstraction.
That's right.
We also support an extensible set of application "types" which will allow for a much more sophisticated range of behaviour.
The schema limits the possible values to several _types_ of applications like scripts, java classes, sql queries, etc. But how e.g. fortran compilation could be specified? Imagine that you don't know the details about the location of the fortran compiler and the compiler vendor.
We can look at the UNICORE experience and try to configure the particular applications using the following entities:
a) fields (i.e. partucular file names, argument values, e.g. "source=a.c", "size=1234")
I see this as being a different interpretation on the Argument element (which would require using a non-JSDL type for now, though this will make for a powerful thing to do once we've got JSDL 1.0 out of the door.)
OK, so non-JSDL Argument could be a solution.
b) options (i.e. names for common modes of application work; though setting this mode could differ from system to system, e.g. "debug=on" results in passing "-g" to some compilers) c) contexts (modes for application invocations, e.g. "profiliration")
Thess would be probably done with extension elements in the Application element (unless of course they are more like Resources, when they'd go somewhere else). Anyone else want to comment?
d) variations (variations of expected results, e.g. "overwrite" for file operations)
I don't currently see how the current DataStaging element's CreationFlag and DeleteOnTermination sub-elements are insufficient. Please explain further.
Well, file operations can include "copy folder" or "untar file" and can be expressed as applications. So such variations can exist.
It would be nice if some kind of such specification (probably with another entities but with the idea of abstraction of application) is described in the JSDL spec.
I believe it is. Aside from the very standard kinds of execution scheme (like "run this executable") we also have some more sophisticated ideas like mechanisms for invokation of Java programs, running of SQL queries or even invokation of webservices. However, one of the things we still need to do is to fill out the (non-normative?) specification of how these things will be done.
Hope this helps.
Thanks. Alexander
Donal.
Hiya, I've done some big cropping to the email. See comments below below:
The schema limits the possible values to several _types_ of applications like scripts, java classes, sql queries, etc. But how e.g. fortran compilation could be specified? Imagine that you don't know the details about the location of the fortran compiler and the compiler vendor.
The schema uses a union of NMTOKEN and a set of enumerations (as you list above). Therefor you can either use one of the pre-defined enumerations or derive your own set of NMTOKENS that can be used here. The enumerated tokens were the application types that we have already identified as being general enough to make part of the core schema. Perhapses something like "FortranCompile" could go into a specific extension for your use of jsdl or into a compiling extension set along with "CCompile" and "JavaCompile". Personally I prefer the latter. At some stage we may decide that some of these extensions are general enough to become part of the core jsdl (say version 1.3 spec) in which case it could be added. Hopefully this gives enough scope for you to achieve what you need.
We can look at the UNICORE experience and try to configure the particular applications using the following entities:
a) fields (i.e. partucular file names, argument values, e.g. "source=a.c", "size=1234")
I see this as being a different interpretation on the Argument element (which would require using a non-JSDL type for now, though this will make for a powerful thing to do once we've got JSDL 1.0 out of the door.)
OK, so non-JSDL Argument could be a solution.
Again this could go along with the ApplicationTypeEnumeration extensions. You could specify for a new type how to interpret the Argument elements and/or add extension elements that deal with these. I don't see jsdl 1.0 being the final version and see that as people come up with new extension sets that are useful to a lot of people the jsdl group may choose to add them into the core spec for future releases. Or just support certain core extensions.
d) variations (variations of expected results, e.g. "overwrite" for file operations)
I don't currently see how the current DataStaging element's CreationFlag and DeleteOnTermination sub-elements are insufficient. Please explain further.
Well, file operations can include "copy folder" or "untar file" and can be expressed as applications. So such variations can exist.
Hmm, this would suggest that having the ability to extend the DataStaging element was a good idea. Yes I think you've got a point here. So that's my vote for an xsd:any in the DataStaging element. We steered away from putting "copy folder" or "untar file" in the core spec as so many DRM systems didn't support this. But it would make a useful extension set. Hope that helps, steve.. -- ------------------------------------------------------------------------ Dr A. Stephen McGough ------------------------------------------------------------------------ Research Associate, Imperial College London, Department of Computing, 180 Queen's Gate, London SW7 2BZ, UK tel: +44 (0)207-594-8310 fax: +44 (0)207-581-8024 ------------------------------------------------------------------------ Assistant Warden, West Wing, Beit Hall, Imperial College, Prince Consort Road, London, SW7 2BB tel: +44 (0)207-594-9910 ------------------------------------------------------------------------
A Stephen McGough wrote:
Hmm, this would suggest that having the ability to extend the DataStaging element was a good idea. Yes I think you've got a point here. So that's my vote for an xsd:any in the DataStaging element. We steered away from putting "copy folder" or "untar file" in the core spec as so many DRM systems didn't support this. But it would make a useful extension set.
FWIW, I'd do that as special URLs, perhaps like this: untar:ftp://foo.bar.com/pub/data.tar It's possible to do some *really* horrible things with URL syntax. :^D Donal.
Personally I'd go for a "tarThis" element. I hate these overloading of URL thingies. steve.. Donal K. Fellows wrote:
A Stephen McGough wrote:
Hmm, this would suggest that having the ability to extend the DataStaging element was a good idea. Yes I think you've got a point here. So that's my vote for an xsd:any in the DataStaging element. We steered away from putting "copy folder" or "untar file" in the core spec as so many DRM systems didn't support this. But it would make a useful extension set.
FWIW, I'd do that as special URLs, perhaps like this: untar:ftp://foo.bar.com/pub/data.tar
It's possible to do some *really* horrible things with URL syntax. :^D
Donal.
-- ------------------------------------------------------------------------ Dr A. Stephen McGough ------------------------------------------------------------------------ Research Associate, Imperial College London, Department of Computing, 180 Queen's Gate, London SW7 2BZ, UK tel: +44 (0)207-594-8310 fax: +44 (0)207-581-8024 ------------------------------------------------------------------------ Assistant Warden, West Wing, Beit Hall, Imperial College, Prince Consort Road, London, SW7 2BB tel: +44 (0)207-594-9910 ------------------------------------------------------------------------
participants (3)
-
A Stephen McGough -
Donal K. Fellows -
Lukichev, Alexander S