Proposed Extension draft document
Folks, here's a write-up of the parameter sweep extension. If possible, I'd like to have a discussion about it on the upcoming call. Cheers, Michel -- Michel <dot> Drescher <at> uk <dot> fujitsu <dot> com Fujitsu Laboratories of Europe +44 20 8606 4834
Michel Drescher wrote:
here's a write-up of the parameter sweep extension.
If possible, I'd like to have a discussion about it on the upcoming call.
A few comments. It's possible to structure the Assignment stuff so that the xsi:type arguments may be omitted. The key is that you have to declare elements and a substitution group as well. Given that, there's no ambiguity in the resulting documents and tooling like Axis does the right thing, generating an inheritance hierarchy. Your example (Appendix 2) omits required SweepGroup elements. This leads me to ask which is actually definitive about your proposal; the example or the schema/spec. It would be nice to be able to attach some kind of way of restricting the space being swept; I might want to sweep an even grid of points excepting omitting a few where I know there are singularities (e.g. the origin, a fairly common case!) Will it be possible to sweep over non-string content (e.g. element content)? Like this, it would be easy to sweep over, say, EPRs... Donal.
Donal K. Fellows wrote:
Michel Drescher wrote:
here's a write-up of the parameter sweep extension.
If possible, I'd like to have a discussion about it on the upcoming call.
A few comments.
It's possible to structure the Assignment stuff so that the xsi:type arguments may be omitted. The key is that you have to declare elements and a substitution group as well. Given that, there's no ambiguity in the resulting documents and tooling like Axis does the right thing, generating an inheritance hierarchy.
Yes I know. Both alternatives -- abstract types and substitutionGroups -- are functionally equivalent. So whichever fits best current tooling (the draft is in quite early stage, though) should be used in the extension. I have no preference, really. On the other hand, the substitutionGroup technique requires you to define a default assignment function. I think there is no default assignment function that would be applicable here.
Your example (Appendix 2) omits required SweepGroup elements. This leads me to ask which is actually definitive about your proposal; the example or the schema/spec.
Actually, it doesn't. I tried to write the draft proposal in the style of element information sets, but I obviously failed. ;-) In fact, the schema came into existence first; I hacked the document yesterday -- which seem quite obvious regarding the language used. ;-) Hence the example does not omit the SweepGroup element -- "SweepGroup" is an XML Schema group that does not render element start and end tags. Again, if tooling does not support that very well if at all, it is a matter of seconds to change that to a plain old contain er element.
It would be nice to be able to attach some kind of way of restricting the space being swept; I might want to sweep an even grid of points excepting omitting a few where I know there are singularities (e.g. the origin, a fairly common case!)
I think I know what you mean. Let's talk about it (and have some schema fun?) today at FLE.
Will it be possible to sweep over non-string content (e.g. element content)? Like this, it would be easy to sweep over, say, EPRs...
The "Parameter" element's value is a XPath expression. So it may evaluate to an attribute's value or element value. If the assignment function is chosen wisely, then it yields strings that contain XML snippets. This way, you can sweep over EPRs... As I said, the draft proposal is in a fairly early stage. While this implies a nice amount of work yet uncompleted it also has nice opportunities, like "plug in your use case here!(TM)" ;-) Cheers, Michel -- Michel <dot> Drescher <at> uk <dot> fujitsu <dot> com Fujitsu Laboratories of Europe +44 20 8606 4834
Michel Drescher wrote:
Yes I know. Both alternatives -- abstract types and substitutionGroups -- are functionally equivalent. So whichever fits best current tooling (the draft is in quite early stage, though) should be used in the extension. I have no preference, really.
On the other hand, the substitutionGroup technique requires you to define a default assignment function. I think there is no default assignment function that would be applicable here.
Actually, it doesn't. If the default is an abstract element, then people have to use the subtypes/elements instead. And it tools up correctly with at least one (Axis 1.3).
Hence the example does not omit the SweepGroup element -- "SweepGroup" is an XML Schema group that does not render element start and end tags. Again, if tooling does not support that very well if at all, it is a matter of seconds to change that to a plain old contain er element.
Hmm, I'd be tempted to leave the type out then. Cuts confusion for us bears of little brain. :-)
The "Parameter" element's value is a XPath expression. So it may evaluate to an attribute's value or element value. If the assignment function is chosen wisely, then it yields strings that contain XML snippets. This way, you can sweep over EPRs...
I'd rather define the assignment values themselves to be XML chunks otherwise there's the whole question of how much quoting has to be done to assemble a string containing (in infoset space) <> characters. Donal.
Donal K. Fellows wrote:
Michel Drescher wrote:
Yes I know. Both alternatives -- abstract types and substitutionGroups -- are functionally equivalent. So whichever fits best current tooling (the draft is in quite early stage, though) should be used in the extension. I have no preference, really.
On the other hand, the substitutionGroup technique requires you to define a default assignment function. I think there is no default assignment function that would be applicable here.
Actually, it doesn't. If the default is an abstract element, then people have to use the subtypes/elements instead. And it tools up correctly with at least one (Axis 1.3).
Yes, interesting alternative. Will explore this further.
Hence the example does not omit the SweepGroup element -- "SweepGroup" is an XML Schema group that does not render element start and end tags. Again, if tooling does not support that very well if at all, it is a matter of seconds to change that to a plain old contain er element.
Hmm, I'd be tempted to leave the type out then. Cuts confusion for us bears of little brain. :-)
CannotParseException - you are opting for using a container element (i.e. explicit start and end tag?)
The "Parameter" element's value is a XPath expression. So it may evaluate to an attribute's value or element value. If the assignment function is chosen wisely, then it yields strings that contain XML snippets. This way, you can sweep over EPRs...
I'd rather define the assignment values themselves to be XML chunks otherwise there's the whole question of how much quoting has to be done to assemble a string containing (in infoset space) <> characters.
Good point. Thought about this - which is the appropriate type? "xsd:anySimpleType"? Cheers, Michel -- Michel <dot> Drescher <at> uk <dot> fujitsu <dot> com Fujitsu Laboratories of Europe +44 20 8606 4834
After a quick look I am curious about - the meaning, if any, of the initial values of any elements that will be replaced by the 'sweep' declarations in the jsdl document. Should these be left empty, should they be the first invocation, etc? - more interesting substitutions, e.g., how to generate output filenames specific for each iteration. A fairly common approach is to add a suffix from the iteration counter. How would it work with 'sweep'? What about the n-dimensional case when you'd probably want to combine values from the nested 'sweep's? - the explanation of the difference between 'embedded' and 'embracing' - the example. I think a small set of simpler examples would probably be better at this point. Also, not specific to this proposal, we should talk about whether we want to encourage people to define their own outermost tags (ParameterSweep) that encapsulate JobDefinition; or whether we want to define a generic 'ComplexJobDefinition' (or some such element) that encapsulates (possibly multiple) JobDefinition elements together with this kind of extensions; or whether to encourage people to put stuff inside JobDefinition (and after JobDescription). I've seen people do all of the above, so some statement on what is preferable might help all around. Andreas Michel Drescher wrote:
Folks,
here's a write-up of the parameter sweep extension.
If possible, I'd like to have a discussion about it on the upcoming call.
Cheers, Michel
-- Andreas Savva Fujitsu Laboratories Ltd
Michel, All, In a discussion with a GridSAM user who has been asking for parameter sweep functionality for some time a use case came up that I'd not thought about before and it's not clear how we could do it with the current extension. He has a use case where he wants to perform a parameter sweep using a program which takes a configuration file as the only input. The filename is always the same (lets call it input.data) and the program has no way to specify a different file name. I can't see at the moment a way to deal with this without altering the file staging elements which then places a dependency between file staging and parameter sweep - not sure if this is a good idea. I suppose it could be done by placing a wrapper around the program... again not sure if this is good. Any thoughts? steve.. Michel Drescher wrote:
Folks,
here's a write-up of the parameter sweep extension.
If possible, I'd like to have a discussion about it on the upcoming call.
Cheers, Michel
-- ------------------------------------------------------------------------ Dr A. Stephen McGough http://www.doc.ic.ac.uk/~asm ------------------------------------------------------------------------ Technical Coordinator, London e-Science Centre, Imperial College London, Department of Computing, 180 Queen's Gate, London SW7 2BZ, UK tel: +44 (0)207-594-8409 fax: +44 (0)207-581-8024 ------------------------------------------------------------------------
A S McGough wrote:
He has a use case where he wants to perform a parameter sweep using a program which takes a configuration file as the only input. The filename is always the same (lets call it input.data) and the program has no way to specify a different file name.
I can't see at the moment a way to deal with this without altering the file staging elements which then places a dependency between file staging and parameter sweep - not sure if this is a good idea. I suppose it could be done by placing a wrapper around the program... again not sure if this is good.
To me that sounds like what he needs is a way of applying some kind of transformation to a template file that is then staged to the job instance. Either that or a wrapper that takes as arguments the varying parameters, sticks them into the config file, and then runs the wrapped program. Conceptually they are about the same thing, I know, but the difference is whether it is some kind of little workflow that does the parameter application, or whether it is the job itself. Another alternative would be to add a staging extension that allows the staging source to be within the JSDL document itself in some way. IIRC, BPEL has done something similar for "constant" arguments. With that, applying changes through Michel's mechanism would be much easier. Donal.
participants (4)
-
A S McGough -
Andreas Savva -
Donal K. Fellows -
Michel Drescher