jsdl sweep over values within input files
hi, Some notes following the JSDL-WG session 1. It was generally agreed that sweeping over values defined within input files should be 'in scope' of the JSDL parameter sweep extensions. However, needs some more clarification. Two different / opposing options were discussed which determine whether sweeping over values within input files is in scope or out of scope of the JSDL Sweep extensions: Option 1) out of scope All input files should be generated 'up-front' and staged to the working dir (or to different JSDL file systems) by 'some other process'. This would mean the Sweep extensions could be used as they currently exist (i.e. by sweeping over the name/location of the existing generated files). In this scenario however, the JSDL-Sweep extensions would *not* have any use in defining the parametric values that are substituted into the input files (e.g. via loops/value lists). If this is preferred, then this would have to be clearly stated within the JSDL documentation, and I would also argue that sweeping over values within input files would then be 'out of scope' of the JSDL parameter sweep extensions. Option 2) in scope The JSDL Sweep document *should* be used to define the values of parametric variables that are substituted into the input files (e.g. via loops/value lists). In this scenario, sweeping over values in input files would be in scope' of the Sweep extensions. If this is preferred, then it appeared that the use of the ${placeholder} variables that exist within the input files and referred to in the JSDL Sweep document could be applied. How a JSDL sweep document names/reference the input files needs clarification (e.g. abstract extension points akin to the loop/value functions?). regards, dave ------------------------ Dr David Meredith STFC - eScience Centre Grid Technology Group Daresbury Laboratory Warrington Cheshire WA4 4AD Tel: 01925 603762(Direct Line) email: d.j.meredith@dl.ac.uk
Meredith, DJ (David) wrote:
Option 1) ‘out of scope’ [...] Option 2) ‘in scope’ [...]
According to what I heard in the session, the option that was most likely to be favoured was a hybrid between the two. In particular, though there would be a parameter sweep where this feature was formally out of scope, there would also be a standard extension to this which would enable the generation of substituted files. There are advantages to doing this. For one thing, it is going to be easier to implement the core parameter sweep correctly as it has "fewer moving parts". For another thing, this should also mean that the "extension to an extension" should be small and focussed; it will not need to say anything about the sweep and value generation functions, and can instead focus on filename operations and how to name regions of files for substitution. This was just my impression, but it was a positive impression and it is something I think you shouldn't be too down about. We *do* understand the real requirement for these things (and definitely in clearer focus than before too, so the meeting was worthwhile). Donal.
Hi Donal, David, I second Donals executive summary. Moreover, I would state that param sweeps over values in input files is definitely in scope for our work in the JSDL-WG. However, I would for reasons Donal nicely sketched, favour having this done in an extension to the extension. The details on how to do this are yet to be sorted out, but my take on the session was to agree that this sort of problem is in scope for further normative description and support. Cheers, Michel On 5 Jun 2008, at 22:20, Donal K. Fellows wrote:
Meredith, DJ (David) wrote:
Option 1) ‘out of scope’ [...] Option 2) ‘in scope’ [...]
According to what I heard in the session, the option that was most likely to be favoured was a hybrid between the two. In particular, though there would be a parameter sweep where this feature was formally out of scope, there would also be a standard extension to this which would enable the generation of substituted files.
There are advantages to doing this. For one thing, it is going to be easier to implement the core parameter sweep correctly as it has "fewer moving parts". For another thing, this should also mean that the "extension to an extension" should be small and focussed; it will not need to say anything about the sweep and value generation functions, and can instead focus on filename operations and how to name regions of files for substitution.
This was just my impression, but it was a positive impression and it is something I think you shouldn't be too down about. We *do* understand the real requirement for these things (and definitely in clearer focus than before too, so the meeting was worthwhile).
Donal. -- jsdl-wg mailing list jsdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/jsdl-wg
-- Michel Drescher Fujitsu Laboratories of Europe, Ltd. Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE Reg. No. 4153469 +44 20 8606 4834 Michel.Drescher@uk.fujitsu.com
I summarized the consensus at the end of the session and the minutes note that the proposal - is in scope - should be structured as an extension to the parameter sweep spec (for the reasons Donal outlined) We also agreed that work should start and run together/in parallel to the existing spec and to make any changes necessary (undecided) to support this proposal as an extension We left open the possibility of putting this extension in the same document as the parameter sweep spec. We had a good discussion in the parameter sweep session. Thanks to everyone who took part. Andreas Btw, it might be clearer to drop the word 'extension' from the main parameter sweep spec since it is not strictly speaking jsdl-specific. Michel Drescher wrote:
Hi Donal, David,
I second Donals executive summary.
Moreover, I would state that param sweeps over values in input files is definitely in scope for our work in the JSDL-WG. However, I would for reasons Donal nicely sketched, favour having this done in an extension to the extension.
The details on how to do this are yet to be sorted out, but my take on the session was to agree that this sort of problem is in scope for further normative description and support.
Cheers, Michel
On 5 Jun 2008, at 22:20, Donal K. Fellows wrote:
Meredith, DJ (David) wrote:
Option 1) ‘out of scope’ [...] Option 2) ‘in scope’ [...]
According to what I heard in the session, the option that was most likely to be favoured was a hybrid between the two. In particular, though there would be a parameter sweep where this feature was formally out of scope, there would also be a standard extension to this which would enable the generation of substituted files.
There are advantages to doing this. For one thing, it is going to be easier to implement the core parameter sweep correctly as it has "fewer moving parts". For another thing, this should also mean that the "extension to an extension" should be small and focussed; it will not need to say anything about the sweep and value generation functions, and can instead focus on filename operations and how to name regions of files for substitution.
This was just my impression, but it was a positive impression and it is something I think you shouldn't be too down about. We *do* understand the real requirement for these things (and definitely in clearer focus than before too, so the meeting was worthwhile).
Donal. -- jsdl-wg mailing list jsdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/jsdl-wg
-- Michel Drescher Fujitsu Laboratories of Europe, Ltd. Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE Reg. No. 4153469
+44 20 8606 4834 Michel.Drescher@uk.fujitsu.com
-- jsdl-wg mailing list jsdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/jsdl-wg
-- Andreas Savva Fujitsu Laboratories Ltd.
participants (4)
-
Andreas Savva -
Donal K. Fellows -
Meredith, DJ (David) -
Michel Drescher