Karl, To use the example in the teleconference, all I want to do is be able to make the distinction between the following statements in the resource description : 1. X CPUs any way you can 2. X CPUs on the same machine 3. X CPUS on the same machine and that's all that machine should have and similarly for memory, for example. I take a very narrow, perhaps simplistic view, that this is merely syntax. And so find it very hard to accept arguments that effectively come down to saying that making the distinction between statement 1 and 2 is good and making the distinction between statements 2 and 3 is not. Andreas Karl Czajkowski wrote:
I thought a little bit more about this problem Andreas has and I think I jumped to a solution too quickly.
Presumably, notions like whether you are getting "bare metal" or "virtual machines" or "posix process startup" is indicated in the application type. So, the only remaining question is whether there is some cross-cutting option that could change the behavior for a given application model.
Let me give a sort of weak "proof by contradiction". Assuming there is some cross-cutting concept here, I think it would be called something like "allocation model" and it has two enumerated values I can think of right now:
"total" allocations are what Andreas wants for his provisioning of node(s) for OS images.
"partial" allocations are what Chris wants for placement of jobs onto shares resources.
The difference is that the partial allocation says "job is allocated an abstract resource R1 which satisfies the resource selection criteria and MAY be part of a larger resource R2". The total allocation says that "job is allocated a resource R and that's that".
Here's the contradiction: the difference between these is really subtle and has to do with the "opacity" of the resource abstraction (the so called "job container") within which the job is run. If the job container is opaque enough, the job cannot tell whether it got a total or partial allocation: a scheduler for POSIX jobs could use system partitioning controls to create what looks like a smaller SMP out of a larger SMP, for example! However, a typical POSIX shared resource has a transparent job container where jobs can detect the larger, shared resource's characteristics. In addition, whether the allocation is advisory or enforced is another related but separate issue. I do not see how we can generically model either of these aspects of allocation without citing behavioral aspects of a particular application type's environment.
For me, one question remains: do we think it is in scope to define different types of resource abstraction for the posix application type? In other words, does the JSDL posix application element need a way to express whether the apparent machine (visible using posix system interfaces) is allowed to be "larger" than the allocation given to the job or not? I am not sure there is a need. The scientific community has been using the transparent abstraction for years, where the system MAY be larger and it is best for the application to "avert its eyes". :-) Besides, we've already punted on how the application learns what its allocation is and how the application processes get started. This allocation opacity issue seems quite metaphysical in comparison...
I definitely think it is out of scope whether an "opaque" allocation is using real "bare hardware" or some sort of virtualized machine, as the distinction comes down to very different QoS concerns like processor and memory performance. This should be handled through extensions and additional context like WS-Agreement, etc.
karl
-- Andreas Savva Fujitsu Laboratories Ltd