Soonwook Hwang wrote:
I was wondering whether you meant "divided single apps" by "parameter sweep types of apps." If that is the case, I don't think that it's less common than "multiple coupled apps," and that it's a rare application. I think that it's in fact the kind of applications that will perhaps benefit most from exploiting the Grid with the current limitations in network bandwidth and latercy between WAN interconnections. As we see in case of Condor, Nimrod and other systems. there are many existing systems aiming at tackling "parameter sweep apps." In addition, we might be able to consider a collection of single job submitted from different job managers to EPS/CSG to be a kind (?) of parameter sweep apps that EPS/CSG needs to make mapping plans. In brief, the point I am trying to make here is that when it comes to the design of EPS/CSG interface and protocol, having the matching/scheduling of parameter sweep type of apps (i.e., a set of independent tasks of job type) in mind is important as well.
As far as I can tell, there are two major classes of non-trivial grid usage here. One is the "we have this complex plan of work to carry out" (i.e. a workflow) and the other is "we want to do this long list of fairly simple things" (i.e. a parameter sweep). These two major classes of system have very different properties; in the workflow, the main desire is to coordinate what may be very large atomic jobs to carry out some larger task and the resource-selection plan is focussed on maximizing the likelihood of success while minimizing the cost, and in the parameter sweep the focus is no longer on the individual executions but on the maximizing of "job throughput" and resource-selection can be done more on the basis of "who is free for another work packet". Indeed I suspect that the best approach might involve the use of a CSG to select a set of worker containers, with very little planning at all. Oh well, that would at least demonstrate the need for having the CSG as an entity separate to the EPS... Donal.