March 2006 - ogsa-wg - lists.ogf.org

FW: [cddlm] Revised CDL for BLAST slides (Re: Proposed agenda for March 15th call)
by Milojicic, Dejan S (HP Labs) 15 Mar '06

15 Mar '06

Hi Hiro and the Teams, We discussed your presentation at the CDDLM meeting. We have questions about the following aspects: 1. The presented comparison is a little bit of apples-to-oranges. We are comparing a job submission with the deployment tool. What exactly do we want to accomplish? For example, should we provide a list of "JSDL document does not include" as an equivalent to slide 6 in the last presentation you have sent? 2. You start from a radically simple case, which does not emphasize any of the CDL features: dependencies, lazy evaluation, naming, etc. You do not even have the need to deploy the BLAST application, it is stored in a well know location on the local system. Should we not expand it for where there is a need for deployment? 3. Is this example good enough for you? What about other more complex scenarios? Do you plan to introduce them. I think for balanced evaluation, you may want to address two scenarios upfront and how they are addressed by both tools. In a summary, how should we take it further? Should we extend the example you had with what CDL does? Should we target the F2F for this purpose? Thanks, Dejan. Also, I include the follow-up message From Jun Tatemura, which further clarifies: I went through the newer version of CDLforBLAST slides. My understanding on this slides is that they expect CDDLM/CDL to deploy (configure) - application binary (installation of blast binary and required environment) - file system (mount file system) - database (setup -- I don't know details on what must be set up...) - environment variables and let JSDL do the rest. If so, what we can do would be: (1) first, we want to confirm that this is the line they want to draw (--> your question 1 comes here. It this what they mean?) (2) if we agree on the line, then we would like to have more information on the requirement (and/or assumed environment) to deploy the above so that we can design CDDLM components for this use case. (--> your questions 2,3 are aligned with this - how much we should extend the deployment requirements from the base case (where everything is already available)) (3) Once we have components, we can write CDL and discuss how CDL and JSDL interact. Thanks, Jun > -----Original Message----- > From: owner-cddlm-wg(a)ggf.org [mailto:owner-cddlm-wg@ggf.org] On Behalf > Of Hiro Kishimoto > Sent: Tuesday, March 14, 2006 9:18 PM > To: OGSA WG; cddlm-wg(a)ggf.org > Subject: [cddlm] Revised CDL for BLAST slides (Re: Proposed agenda for > March 15th call) > > Hi all, > > I've added Chris Smith's comments to CDL for BLAST slides, Let's > review this at this OGSA-WG call tomorrow. > > It is also uploaded to gridforge; > https://forge.gridforum.org/projects/ogsa-wg/document/CDL_for_BLAST > > Thanks, > ---- > Hiro Kishimoto > > Hiro Kishimoto wrote: > > Hi all, > > > > The following is a proposed agenda for OGSA-WG telecon on > March 15th > > Wednesday from 4pm to 6pm (CST). > > > > Dial-in numbers for Wednesday: > > Free: +1-888-452-0308 > > Intl/Toll: +1-484-644-0505 > > PIN: 71815 > > > > Screen share service will be provided. > > URL: http://ogsa.glance.net > > Session key: 0315 > > See more explanation: > > > http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/06/msg00077.ht > > ml > > > > 1) Early discussion (20 min) > > Note taker assignment > > Roll call > > Telecon minutes approval (if available) > > Agenda bashing > > > > 2) April F2F meeting update (20 min) > > > > F2F meeting venue is Intel@Santa Clara & Fujitsu@Sunnyvale. > > The updated session schedule is uploaded to GridForge: > > > https://forge.gridforum.org/projects/ogsa-wg/document/200604F2F_sessio > > n > > > > In order to have general EMS discussion before > CDDLM session, > > is it possible to switch these two sessions? > > a) joint CDDLM-WG session (Apr. 5 Wed. 1:30-4:30pm) > > b) EMS architecture session (Apr. 6 Thu. 9am-12:30pm) > > > > 3) joint CDDLM-WG discussion (60 min) > > > > From the minutes of joint F2F meeting in Athens; > > > >> ACTION: Steven Newhouse to write up a first draft of a few more > >> scenarios on the relationship between JSDL and CDL. > >> - To be included in the current EMS scenarios document. > - To be > >> delivered by mid-March > >> - Review on an OGSA call and continue refinement > >> > >> ** Whiteboard list produced during the discussion > >> > >> - Duplication of JSDL in CDL? > >> - Directory/user creation > >> - What is the difference between [job / application / > container] > >> configuration] > >> - Where is line between JSDL and CDL and is it policy specific > >> - Job lifecycle including deployment and start/stop > > > > > > Steven will send out revised version of his > "interaction between > > CDDLM & JSDL" scenarios before the call. > > > > Revised "CDL for BLAST" slides attached. > > > > 4) Wrap up (10 min) > > AOB > > > > <NEXT CALL> > > March 20 (Mon): OGSA-HPC profile (Marvin) March 22 (Wed): Security > > Architecture (Takuya and Frank) March 27 (Mon): EMS scenarios, > > Steven's scenarios review, > > Information Model, Ellen's CIM profile > review March 29 > > (Wed): F2F preparation > > > > Telecon schedule: > > > https://forge.gridforum.org/projects/ogsa-wg/document/telecon_schedule > > / > > >

1 0

RE: [ogsa-wg] Paper proposing "evolutionary vertical design efforts"
by Subramaniam, Ravi 15 Mar '06

15 Mar '06

Hi Marvin, Thanks for documenting and explaining what you see as the basic HPC case. I agree with your observations and what you have listed are what most, if not all HPC cluster systems I know of, implement. I think there is something else that may be apparent. If I may suggest that the bulleted list with the exception of the "11th (?)" bullet would nicely fit into the BES situation. If I look at the cluster you describe (i.e. the client perspective with queued, running and finished) it seem to me that that would be a BES container (at least from an interface perspective and where the scheduler that you are submitting the jobs to is the abstract representation of the BES "container"). We had discussed this recursive container concept in the EMS discussion in OGSA V1.0. This seems especially true since you advocate that in this simple HPC case a "user" would not have visibility into the granular elements of what makes a cluster and the operations would essentially be within this "black box". Would it make sense to add most of these notions to the BES discussion and the scheduler aspects would fit in RSS (where they could provide a basic RSS interface)? Putting these together with some minor additional specs may be the basic HPC profile. Am I missing something? Thanks! Ravi ________________________________ From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On Behalf Of Marvin Theimer Sent: Monday, March 13, 2006 2:42 PM To: Ian Foster; ogsa-wg(a)ggf.org Cc: Marvin Theimer Subject: RE: [ogsa-wg] Paper proposing "evolutionary vertical design efforts" Hi; Ian, you are correct that I view job submission to a cluster as being one of the simplest, and hence most basic, HPC use cases to start with. Or, to be slightly more general, I view job submission to a "black box" that can run jobs - be it a cluster or an SMP or an SGI NUMA machine or what-have-you - as being the simplest and hence most basic HPC use case to start with. The key distinction for me is that the internals of the "box" are for the most part not visible to the client, at least as far as submitting and running compute jobs is concerned. There may well be a separate interface for dealing with things like system management, but I want to explicitly separate those things out in order to allow for use of "boxes" that might be managed by proprietary means or by means obeying standards that a particular job submission client is unfamiliar with. I think the use case that Ravi Subramaniam posted to this mailing list back on 2/17 is a good one to start a discussion around. However, I'd like to present it from a different point-of-view than he did. The manner in which the use case is currently presented emphasizes all the capabilities and services needed to handle the fully general case of submitting a batch job to a computing utility/service. That's a great way of producing a taxonomy against which any given system or design can be compared to see what it has to offer. I would argue that the next step is to ask what's the simplest subset that represents a useful system/design and how should one categorize the various capabilities and services he has identified so as to arrive at meaningful components that can be selectively used to obtain progressively more capable systems. Another useful exercise to do is to examine existing job scheduling systems in order to understand what they provide. Since in the real world we will have to deal with the legacy of existing systems it will be important to understand how they relate to the use cases we explore. In the same vein, it will be important to take into account and understand other existing infrastructures that people use that are related to HPC use cases. I'm thinking of things like security infrastructures, directory services, and so forth. From the point-of-view of managing complexity and reducing total-cost-of-ownership, it will be important to understand the extent to which existing infrastructure and services can be reused rather than reinvented. To kick off a discussion around the topic of a minimalist HPC use case, I present a straw man description of such below and then present a first attempt at categorizing various areas of extension. The categorization of extension areas is not meant to be complete or even all that carefully thought-out as far as componentization boundaries are concerned; it is merely meant to be a first contribution to get the discussion going. A basic HPC use case: Compute cluster embedded within an organization. * This is your basic batch job scheduling scenario. Only a very basic state transition diagram is visible to the client, with the following states for a job: queued, running, finished. Additional states -- and associated state transition request operations and functionality -- are not supported. Examples of additional states and associated functionality include suspension of jobs and migration of jobs. * Only "standard" resources can be described, for example: number of cpus/nodes needed, memory requirements, disk requirements, etc. (think resources that are describable by JSDL). * Once a job has been submitted it can be cancelled, but its resource requests can't be modified. * A distributed file system is accessible from client desktop machines and client file servers, as well as compute nodes of the compute cluster. This implies that no data staging is required, that programs can be (for the most part) executed from existing file system locations, and that no program "provisioning" is required (since you can execute them from wherever they are already installed). Thus in this use case all data transfer and program installation operations are the responsibility of the user. * Users already have accounts within the existing security infrastructure (e.g. Kerberos). They would like to use these and not have to create/manage additional authentication/authorization credentials (at least at the level that is visible to them). * The job scheduling service resides at a well-known network name and it is aware of the compute cluster and its resources by "private" means (e.g. it runs on the head node of the cluster and employs private means to monitor and control the resources of the cluster). This implies that there is no need for any sort of directory services for finding the compute cluster or the resources it represents other than basic DNS. * Compute cluster system management is opaque to users and is the concern of the compute cluster's owners. This implies that system management is not part of the compute cluster's public job scheduling interface. This also implies that there is no need for a logging interface to the service. I assume that application-level logging can be done by means of libraries that write to client files; i.e. that there is no need for any sort of special system support for logging. * A simple polling-based interface is the simplest form of interface to something like a job scheduling service. However, a simple call-back notification interface is a very useful addition that potentially provides substantial performance benefits since it can enable the avoidance of lots of unnecessary network traffic. Only job state changes result in notification messages. * There are no notions of fault tolerance. Jobs that fail must be resubmitted by the client. Neither the cluster head node nor its compute nodes are fault tolerant. I do expect the client software to return an indication of failure-due-system-fault when appropriate. (Note that this may also occur when things like network partitions occur.) * One does need some notion of how to deal with orphaned resources and jobs. The notion of job lifetime and post-expiration garbage collection is a natural approach here. * The scheduling service provides a fixed set of scheduling policies, with only a few basic choices (or maybe even just one), such as FIFO or round-robin. There is no notion, in general, of SLAs (which are a form of scheduling policy). * Enough information must be returned to the client when a job finishes to enable basic accounting functionality. This means things like total wall-clock time the job ran and a summary of resources used. There is not a need for the interface to support any sort of grouping of accounting information. That is, jobs do not need to be associated with projects, groups, or other accounting entities and the job scheduling service is not responsible for tracking accounting information across such entities. As long as basic resource utilization information is returnable for each job, accounting can be done externally to the job scheduling service. I do assume that jobs can be uniquely identified by some means and can be uniquely associated with some principal entity existing in the overall system, such as a user name. * Just as there is no notion of requiring the job scheduling service to track any but the most basic job-level accounting information, there is no notion of the service enforcing quotas on jobs. * Although it is generally useful to separate the notions of resource reservation from resource usage (e.g. to enable interactive and debugging use of resources), it is not a necessity for the most basic of job scheduling services. * There is no notion of tying multiple jobs together, either to support things like dependency graphs or to support things like workflows. Such capabilities must be implemented by clients of the job scheduling service. Interesting extension areas: * Additional scheduling policies o Weighted fair-share, ... o Multiple queues o SLAs o ... * Extended resource descriptions o Additional resource types, such as GPUs o Additional types of compute resources, such as desktop computers o Condor-style class ads * Extended job descriptions (as returned to requesting clients and sys admins) * Additional classes of security credentials * Reservations separated from execution o Enabling interactive and debugging jobs o Support for multiple competing schedulers (incl. desktop cycle stealing and market-based approaches to scheduling compute resources) * Ability to modify jobs during their existence * Fault tolerance o Automatic rescheduling of jobs that failed due to system faults o Highly available resources: This is partly a policy statement by a scheduling service about its characteristics and partly the ability to rebind clients to migrated service endpoints * Extended state transition diagrams and associated functionalities o Job suspension o Job migration o ... * Accounting & quotas * Operating on arrays of jobs * Meta-schedulers, multiple schedulers, and ecologies and hierarchies of multiple schedulers o Meta-schedulers * Hierarchical job scheduling with a meta-scheduler as the only entry point; forwarding jobs to the meta-scheduler from other subsidiary schedulers o Condor-style matchmaking * Directory services o Using existing directory services o Abstract directory service interface(s) * Data transfer topics o Application data staging * Naming * Efficiency * Convenience * Cleanup o Program staging/provisioning * Description * Installation * Cleanup Marvin. ________________________________ From: Ian Foster [mailto:foster@mcs.anl.gov] Sent: Monday, February 20, 2006 9:20 AM To: Marvin Theimer; ogsa-wg(a)ggf.org Cc: Marvin Theimer; Savas Parastatidis; Tony Hey; Marty Humphrey; gcf(a)grids.ucs.indiana.edu Subject: Re: [ogsa-wg] Paper proposing "evolutionary vertical design efforts" Dear All: The most important thing to understand at this point (IMHO) is the scope of this "HPC use case," as this will determine just how minimal we can be. I get the impression that the principal goal may be "job submission to a cluster." Is that correct? How do we start to circumscribe the scope more explicitly? Ian. At 05:45 AM 2/16/2006 -0800, Marvin Theimer wrote: Enclosed is a paper that advocates an additional set of activities that the authors believe that the OGSA working groups should engage in. Broadly speaking, the OGSA and related working groups are already doing a bunch of important things: * There is broad exploration of the big picture, including enumeration of use cases, taxonomy of areas, identification of research issues, etc. * There is work going on in each of the horizontal areas that have been identified, such as EMS, data services, etc. * There is working going around individual specifications, such as BES, JSDL, etc. Given that individual specifications are beginning to come to fruition, the authors believe it is time to also start defining vertical profilesthat precisely describe how groups of individual specifications should be employed to implement specific use cases in an interoperable manner. The authors also believe that the process of defining these profiles offers an opportunity to close the design loopby relating the various on-going protocol and standards efforts back to the use cases in a very concrete manner. This provides an end-to-end setting in which to identify holes and issues that might require additional protocols and/or (incremental) changes to existing protocols. The paper introduces both the general notion of doing focused vertical design effortsand then focuses on a specific vertical design effort, namely a minimal HPC design. The paper derives a specific HPC design in a first principlesmanner since the authors believe that this increases the chances of identifying issues. As a consequence, existing specifications and the activities of existing working groups are not mentioned and this paper is not an attempt to actually define a specifications profile. Also, the absence of references to existing work is not meant to imply that such work is in any way irrelevant or inappropriate. The paper should be viewed as a first abstract attempt to propose a new kind of activity within OGSA. The expectation is that future open discussions and publications will explore the concrete details of such a proposal. This paper was recently sent to a few key individuals in order to get feedback from them before submitting it to the wider GGF community. Unfortunately that process took longer than intended and some members of the community may have already seen a copy of the paper without knowing the context within it was written. This email should hopefully dispel any misconceptions that may have occurred. For those people who will be around on for the F2F meetings on Friday, Marvin Theimer will be giving a talk on the contents of this paper at a time and place to be announced. Marvin Theimer, Savas Parastatidis, Tony Hey, Marty Humphrey, Geoffrey Fox _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org <http://www.globus.org/>

2 1

Action item from mondays call
by Andrew Grimshaw 15 Mar '06

15 Mar '06

All, On the call Monday night I was asked to provide a few paragraphs on provisioning versus instance instantiation and "currying". To me this is pretty straight-forward. I provision an application on my computer by installing it. CDL is the tool for that. Every time I want to run the program, I create a separate application instance - JSDL/BES is the right tool for that. With respect to whether a particular input is bound to the application (as came up in the call). More formally: Assume that a program is a function L(a,b,c,d, .) that maps the inputs to some set of outputs. (We'll assume that the programs are deterministic.) If we bind the formal parameter "a" with a particular value (say a particular input file), then we have effectively created a new function L'(b,c,d, ..). L and L' are two different functions. In lambda calculus this is called "currying" - I've also seen it called "reification" in other contexts. In terms of programs like BLAST. BLAST basically takes an input sequence database and an input sequence and generates a set of scores - usually the top k scores. . (There are many other parameters, but lets focus on the two major ones.) The scores are generated by comparing the input sequence against each of the target sequences in the database. If we "bind" BLAST to a particular target database - then we have effectively created a new BLAST, BLASTBound_to_the_particular_database. It is not the same function any more. In terms of the discussion we've been having on the differences of the roles of CDL and JSDL/BES I think if it this way. If I want to be able to run BLAST on a machine - I may use (via perhaps the program manager) CDL to "provision" BLAST in the BES container. To run BLAST with a particular set of input files, then I use JSDL/BES. If I am going to be running a large number of "BLAST jobs" against a particular DB, I may want to "provision" BLASTBound_to_the_particular_database , using CDL, and then run a set of comparisons against particular sequences using JSDL/BES. Andrew Grimshaw Professor of Computer Science University of Virginia 434-982-2204 grimshaw(a)cs.virginia.edu

1 0

Webcast thoughts
by Andrew Grimshaw 15 Mar '06

15 Mar '06

All, I'd like to start an email discussion in advance of the F2F on the webcast content. While we do have an initial set of slides - I think we need to make sure that we design the talk for our expected audience - rather than based on the material we already have in the deck. When Dave I went to MS our slide deck was not technical enough - we had the wrong talk with us, and so had to improvise. So, one of the things I'd like to know is: Who is our expected audience? Is it technical people, if so, are they web services aware?, are they applications developers? Or are they mostly managers and planners. Or do we have to try and make everyone happy. Any insight? A Andrew Grimshaw Professor of Computer Science University of Virginia 434-982-2204 grimshaw(a)cs.virginia.edu

1 0

Proposed agenda for March 15th call
by Hiro Kishimoto 15 Mar '06

15 Mar '06

Hi all, The following is a proposed agenda for OGSA-WG telecon on March 15th Wednesday from 4pm to 6pm (CST). Dial-in numbers for Wednesday: Free: +1-888-452-0308 Intl/Toll: +1-484-644-0505 PIN: 71815 Screen share service will be provided. URL: http://ogsa.glance.net Session key: 0315 See more explanation: http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/06/msg00077.html 1) Early discussion (20 min) Note taker assignment Roll call Telecon minutes approval (if available) Agenda bashing 2) April F2F meeting update (20 min) F2F meeting venue is Intel@Santa Clara & Fujitsu@Sunnyvale. The updated session schedule is uploaded to GridForge: https://forge.gridforum.org/projects/ogsa-wg/document/200604F2F_session In order to have general EMS discussion before CDDLM session, is it possible to switch these two sessions? a) joint CDDLM-WG session (Apr. 5 Wed. 1:30-4:30pm) b) EMS architecture session (Apr. 6 Thu. 9am-12:30pm) 3) joint CDDLM-WG discussion (60 min) From the minutes of joint F2F meeting in Athens; > ACTION: Steven Newhouse to write up a first draft of a few more > scenarios on the relationship between JSDL and CDL. > - To be included in the current EMS scenarios document. > - To be delivered by mid-March > - Review on an OGSA call and continue refinement > > ** Whiteboard list produced during the discussion > > - Duplication of JSDL in CDL? > - Directory/user creation > - What is the difference between > [job / application / container] configuration] > - Where is line between JSDL and CDL and is it policy specific > - Job lifecycle including deployment and start/stop Steven will send out revised version of his "interaction between CDDLM & JSDL" scenarios before the call. Revised "CDL for BLAST" slides attached. 4) Wrap up (10 min) AOB <NEXT CALL> March 20 (Mon): OGSA-HPC profile (Marvin) March 22 (Wed): Security Architecture (Takuya and Frank) March 27 (Mon): EMS scenarios, Steven's scenarios review, Information Model, Ellen's CIM profile review March 29 (Wed): F2F preparation Telecon schedule: https://forge.gridforum.org/projects/ogsa-wg/document/telecon_schedule/ -- Hiro Kishimoto

1 1

Minutes from the March 13 telecon
by Treadwell, Jem 14 Mar '06

14 Mar '06

Hi, the minutes from Monday's OGSA telecon are attached and in GridForge at http://tinyurl.com/fgkmf. - Jem ________________________________ Jem Treadwell Hewlett-Packard Company 6000 Irwin Road Mount Laurel, NJ 08054 Phone: 856-638-6021 Fax: 856-638-6190 E-mail: Jem.Treadwell(a)hp.com <mailto:Jem.Treadwell@hp.com>

1 0

RE: [ogsa-wg] Paper proposing "evolutionary vertical design efforts"
by Carl Kesselman 14 Mar '06

14 Mar '06

Hi, While I have no wish to engage in the "what is a Grid" argument, there are some elements of your base use case that I would be concerned about. Specifically, the assumption that the submission in into a "local cluster" on which there is an existing account may lead one to a solution that may not generalize to the solution to the case of submission across autonomous policy domains. I would also argue that ignoring issues of fault tolerance from the beginning is also problematic. One must at least design operations that are restartable (for example at most once submission semantics). I would finally suggest that while examining existing job schedule systems is a good thing to do, we should also examine existing remote submission systems (dare I say Grid systems). The basic HPC use case is one in which there is a significant amount implementation and usage experience. Thanks, Carl ________________________________ From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On Behalf Of Marvin Theimer Sent: Monday, March 13, 2006 2:42 PM To: Ian Foster; ogsa-wg(a)ggf.org Cc: Marvin Theimer Subject: RE: [ogsa-wg] Paper proposing "evolutionary vertical design efforts" Hi; Ian, you are correct that I view job submission to a cluster as being one of the simplest, and hence most basic, HPC use cases to start with. Or, to be slightly more general, I view job submission to a "black box" that can run jobs - be it a cluster or an SMP or an SGI NUMA machine or what-have-you - as being the simplest and hence most basic HPC use case to start with. The key distinction for me is that the internals of the "box" are for the most part not visible to the client, at least as far as submitting and running compute jobs is concerned. There may well be a separate interface for dealing with things like system management, but I want to explicitly separate those things out in order to allow for use of "boxes" that might be managed by proprietary means or by means obeying standards that a particular job submission client is unfamiliar with. I think the use case that Ravi Subramaniam posted to this mailing list back on 2/17 is a good one to start a discussion around. However, I'd like to present it from a different point-of-view than he did. The manner in which the use case is currently presented emphasizes all the capabilities and services needed to handle the fully general case of submitting a batch job to a computing utility/service. That's a great way of producing a taxonomy against which any given system or design can be compared to see what it has to offer. I would argue that the next step is to ask what's the simplest subset that represents a useful system/design and how should one categorize the various capabilities and services he has identified so as to arrive at meaningful components that can be selectively used to obtain progressively more capable systems. Another useful exercise to do is to examine existing job scheduling systems in order to understand what they provide. Since in the real world we will have to deal with the legacy of existing systems it will be important to understand how they relate to the use cases we explore. In the same vein, it will be important to take into account and understand other existing infrastructures that people use that are related to HPC use cases. I'm thinking of things like security infrastructures, directory services, and so forth. From the point-of-view of managing complexity and reducing total-cost-of-ownership, it will be important to understand the extent to which existing infrastructure and services can be reused rather than reinvented. To kick off a discussion around the topic of a minimalist HPC use case, I present a straw man description of such below and then present a first attempt at categorizing various areas of extension. The categorization of extension areas is not meant to be complete or even all that carefully thought-out as far as componentization boundaries are concerned; it is merely meant to be a first contribution to get the discussion going. A basic HPC use case: Compute cluster embedded within an organization. * This is your basic batch job scheduling scenario. Only a very basic state transition diagram is visible to the client, with the following states for a job: queued, running, finished. Additional states -- and associated state transition request operations and functionality -- are not supported. Examples of additional states and associated functionality include suspension of jobs and migration of jobs. * Only "standard" resources can be described, for example: number of cpus/nodes needed, memory requirements, disk requirements, etc. (think resources that are describable by JSDL). * Once a job has been submitted it can be cancelled, but its resource requests can't be modified. * A distributed file system is accessible from client desktop machines and client file servers, as well as compute nodes of the compute cluster. This implies that no data staging is required, that programs can be (for the most part) executed from existing file system locations, and that no program "provisioning" is required (since you can execute them from wherever they are already installed). Thus in this use case all data transfer and program installation operations are the responsibility of the user. * Users already have accounts within the existing security infrastructure (e.g. Kerberos). They would like to use these and not have to create/manage additional authentication/authorization credentials (at least at the level that is visible to them). * The job scheduling service resides at a well-known network name and it is aware of the compute cluster and its resources by "private" means (e.g. it runs on the head node of the cluster and employs private means to monitor and control the resources of the cluster). This implies that there is no need for any sort of directory services for finding the compute cluster or the resources it represents other than basic DNS. * Compute cluster system management is opaque to users and is the concern of the compute cluster's owners. This implies that system management is not part of the compute cluster's public job scheduling interface. This also implies that there is no need for a logging interface to the service. I assume that application-level logging can be done by means of libraries that write to client files; i.e. that there is no need for any sort of special system support for logging. * A simple polling-based interface is the simplest form of interface to something like a job scheduling service. However, a simple call-back notification interface is a very useful addition that potentially provides substantial performance benefits since it can enable the avoidance of lots of unnecessary network traffic. Only job state changes result in notification messages. * There are no notions of fault tolerance. Jobs that fail must be resubmitted by the client. Neither the cluster head node nor its compute nodes are fault tolerant. I do expect the client software to return an indication of failure-due-system-fault when appropriate. (Note that this may also occur when things like network partitions occur.) * One does need some notion of how to deal with orphaned resources and jobs. The notion of job lifetime and post-expiration garbage collection is a natural approach here. * The scheduling service provides a fixed set of scheduling policies, with only a few basic choices (or maybe even just one), such as FIFO or round-robin. There is no notion, in general, of SLAs (which are a form of scheduling policy). * Enough information must be returned to the client when a job finishes to enable basic accounting functionality. This means things like total wall-clock time the job ran and a summary of resources used. There is not a need for the interface to support any sort of grouping of accounting information. That is, jobs do not need to be associated with projects, groups, or other accounting entities and the job scheduling service is not responsible for tracking accounting information across such entities. As long as basic resource utilization information is returnable for each job, accounting can be done externally to the job scheduling service. I do assume that jobs can be uniquely identified by some means and can be uniquely associated with some principal entity existing in the overall system, such as a user name. * Just as there is no notion of requiring the job scheduling service to track any but the most basic job-level accounting information, there is no notion of the service enforcing quotas on jobs. * Although it is generally useful to separate the notions of resource reservation from resource usage (e.g. to enable interactive and debugging use of resources), it is not a necessity for the most basic of job scheduling services. * There is no notion of tying multiple jobs together, either to support things like dependency graphs or to support things like workflows. Such capabilities must be implemented by clients of the job scheduling service. Interesting extension areas: * Additional scheduling policies o Weighted fair-share, ... o Multiple queues o SLAs o ... * Extended resource descriptions o Additional resource types, such as GPUs o Additional types of compute resources, such as desktop computers o Condor-style class ads * Extended job descriptions (as returned to requesting clients and sys admins) * Additional classes of security credentials * Reservations separated from execution o Enabling interactive and debugging jobs o Support for multiple competing schedulers (incl. desktop cycle stealing and market-based approaches to scheduling compute resources) * Ability to modify jobs during their existence * Fault tolerance o Automatic rescheduling of jobs that failed due to system faults o Highly available resources: This is partly a policy statement by a scheduling service about its characteristics and partly the ability to rebind clients to migrated service endpoints * Extended state transition diagrams and associated functionalities o Job suspension o Job migration o ... * Accounting & quotas * Operating on arrays of jobs * Meta-schedulers, multiple schedulers, and ecologies and hierarchies of multiple schedulers o Meta-schedulers * Hierarchical job scheduling with a meta-scheduler as the only entry point; forwarding jobs to the meta-scheduler from other subsidiary schedulers o Condor-style matchmaking * Directory services o Using existing directory services o Abstract directory service interface(s) * Data transfer topics o Application data staging * Naming * Efficiency * Convenience * Cleanup o Program staging/provisioning * Description * Installation * Cleanup Marvin. ________________________________ From: Ian Foster [mailto:foster@mcs.anl.gov] Sent: Monday, February 20, 2006 9:20 AM To: Marvin Theimer; ogsa-wg(a)ggf.org Cc: Marvin Theimer; Savas Parastatidis; Tony Hey; Marty Humphrey; gcf(a)grids.ucs.indiana.edu Subject: Re: [ogsa-wg] Paper proposing "evolutionary vertical design efforts" Dear All: The most important thing to understand at this point (IMHO) is the scope of this "HPC use case," as this will determine just how minimal we can be. I get the impression that the principal goal may be "job submission to a cluster." Is that correct? How do we start to circumscribe the scope more explicitly? Ian. At 05:45 AM 2/16/2006 -0800, Marvin Theimer wrote: Enclosed is a paper that advocates an additional set of activities that the authors believe that the OGSA working groups should engage in. Broadly speaking, the OGSA and related working groups are already doing a bunch of important things: * There is broad exploration of the big picture, including enumeration of use cases, taxonomy of areas, identification of research issues, etc. * There is work going on in each of the horizontal areas that have been identified, such as EMS, data services, etc. * There is working going around individual specifications, such as BES, JSDL, etc. Given that individual specifications are beginning to come to fruition, the authors believe it is time to also start defining vertical profilesthat precisely describe how groups of individual specifications should be employed to implement specific use cases in an interoperable manner. The authors also believe that the process of defining these profiles offers an opportunity to close the design loopby relating the various on-going protocol and standards efforts back to the use cases in a very concrete manner. This provides an end-to-end setting in which to identify holes and issues that might require additional protocols and/or (incremental) changes to existing protocols. The paper introduces both the general notion of doing focused vertical design effortsand then focuses on a specific vertical design effort, namely a minimal HPC design. The paper derives a specific HPC design in a first principlesmanner since the authors believe that this increases the chances of identifying issues. As a consequence, existing specifications and the activities of existing working groups are not mentioned and this paper is not an attempt to actually define a specifications profile. Also, the absence of references to existing work is not meant to imply that such work is in any way irrelevant or inappropriate. The paper should be viewed as a first abstract attempt to propose a new kind of activity within OGSA. The expectation is that future open discussions and publications will explore the concrete details of such a proposal. This paper was recently sent to a few key individuals in order to get feedback from them before submitting it to the wider GGF community. Unfortunately that process took longer than intended and some members of the community may have already seen a copy of the paper without knowing the context within it was written. This email should hopefully dispel any misconceptions that may have occurred. For those people who will be around on for the F2F meetings on Friday, Marvin Theimer will be giving a talk on the contents of this paper at a time and place to be announced. Marvin Theimer, Savas Parastatidis, Tony Hey, Marty Humphrey, Geoffrey Fox _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org <http://www.globus.org/>

2 1

Re: [ogsa-wg] Paper proposing "evolutionary vertical design efforts"
by Ian Foster 13 Mar '06

13 Mar '06

Dear All: The most important thing to understand at this point (IMHO) is the scope of this "HPC use case," as this will determine just how minimal we can be. I get the impression that the principal goal may be "job submission to a cluster." Is that correct? How do we start to circumscribe the scope more explicitly? Ian. At 05:45 AM 2/16/2006 -0800, Marvin Theimer wrote: >Enclosed is a paper that advocates an additional set of activities that >the authors believe that the OGSA working groups should engage in. > > > >Broadly speaking, the OGSA and related working groups are already doing a >bunch of important things: > >· There is broad exploration of the big picture, including >enumeration of use cases, taxonomy of areas, identification of research >issues, etc. > >· There is work going on in each of the horizontal areas that have >been identified, such as EMS, data services, etc. > >· There is working going around individual specifications, such as >BES, JSDL, etc. > > > >Given that individual specifications are beginning to come to fruition, >the authors believe it is time to also start defining vertical >profilesthat precisely describe how groups of individual specifications >should be employed to implement specific use cases in an interoperable >manner. The authors also believe that the process of defining these >profiles offers an opportunity to close the design loopby relating the >various on-going protocol and standards efforts back to the use cases in a >very concrete manner. This provides an end-to-end setting in which to >identify holes and issues that might require additional protocols and/or >(incremental) changes to existing protocols. The paper introduces both >the general notion of doing focused vertical design effortsand then >focuses on a specific vertical design effort, namely a minimal HPC design. > > > >The paper derives a specific HPC design in a first principlesmanner since >the authors believe that this increases the chances of identifying >issues. As a consequence, existing specifications and the activities of >existing working groups are not mentioned and this paper is not an attempt >to actually define a specifications profile. Also, the absence of >references to existing work is not meant to imply that such work is in any >way irrelevant or inappropriate. The paper should be viewed as a first >abstract attempt to propose a new kind of activity within OGSA. The >expectation is that future open discussions and publications will explore >the concrete details of such a proposal. > > > >This paper was recently sent to a few key individuals in order to get >feedback from them before submitting it to the wider GGF >community. Unfortunately that process took longer than intended and some >members of the community may have already seen a copy of the paper without >knowing the context within it was written. This email should hopefully >dispel any misconceptions that may have occurred. > > > >For those people who will be around on for the F2F meetings on Friday, >Marvin Theimer will be giving a talk on the contents of this paper at a >time and place to be announced. > > > >Marvin Theimer, Savas Parastatidis, Tony Hey, Marty Humphrey, Geoffrey Fox > > _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org

2 1

Proposed agenda for March 13th call
by Hiro Kishimoto 13 Mar '06

13 Mar '06

Hi all, The following is a proposed agenda for OGSA-WG telecon on March 13th. Monday from 4pm to 6pm (CST). The dial-in number for Monday; US: +1 718 3541071 (New York) or +1 408 9616509 (San Jose) UK: +44 (0)207 3655269 (London) Japan: +81 (0)3 3570 8225 (Tokyo) PIN: 4371991 Screen share service will be provided. URL: http://ogsa.glance.net Session key: 0313 See more explanation: http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/06/msg00077.html 1) Early discussion (10 min) Note taker assignment Roll call Telecon minutes approval (March 8th) https://forge.gridforum.org/projects/ogsa-wg/document/minutes-20060308/en/1 OGSA WSRF BP 1.0 status update (Tom) Agenda bashing 2) EMS scenarios review (60 min) Steven sent out his deployment and provisioning scenarios to the list. Thanks Steven!! http://www-unix.gridforum.org/mail_archive/ogsa-wg/2006/03/msg00024.html > ACTION: Steven Newhouse to write up a first draft of a few more > scenarios on the relationship between JSDL and CDL. > - To be included in the current EMS scenarios document. > - To be delivered by mid-March > - Review on an OGSA call and continue refinement Previous F2F meeting minutes; https://forge.gridforum.org/projects/ogsa-wg/document/ggf16-f2f-minutes-200… https://forge.gridforum.org/projects/ogsa-wg/document/ogsa-f2f-minutes-2006… https://forge.gridforum.org/projects/ogsa-wg/document/cddlm-acs-joint-sessi… EMS module diagram reference by Stuart (under discussion) https://forge.gridforum.org/projects/ogsa-wg/document/GGF15-F2F-OGSA-CDDLM-… BLAST JSDL document example by Chris Smith https://forge.gridforum.org/projects/ogsa-wg/document/jsdl_blast_instance/e… EMS architecture scenario document https://forge.gridforum.org/projects/ogsa-wg/document/draft-ggf-ogsa-ems-sc… 3) April F2F meeting venue and agenda (30 min) >From March 8th minutes; > Discussed possible venues in the Bay area. There was a clear > preference for doing the meeting in the Sunnyvale area. It is > expected that the site can be confirmed next week. > Actions: > - Jay will look into IBM facilities in Almaden and Santa Theresa. > - Hiro/Andreas to confirm whether the availability or not of the > Fujitsu facility > - Hiro to ask Ravi whether the Intel facility is available Proposed agenda review https://forge.gridforum.org/projects/ogsa-wg/document/200604F2F_session Survey result as of March 7th (20 answers). https://forge.gridforum.org/projects/ogsa-wg/document/200604_Survey 4) Wrap up (10 min) AOB <NEXT CALL> March 15 (Wed): CDDLM-WG joint call (Dejan and Steven) March 20 (Mon): OGSA-HPC profile (Marvin) March 22 (Wed): Security Architecture (Takuya and Frank) March 27 (Mon): EMS scenarios, Steven's scenarios review, Information Model, Ellen's CIM profile review Telecon schedule: https://forge.gridforum.org/projects/ogsa-wg/document/telecon_schedule/ -- Hiro Kishimoto

1 1

OGSA-EMS Telecon
by Steven Newhouse 13 Mar '06

13 Mar '06

All, Following on from the discussions at the F2F in Athens here are some deployment and provisioning scenarios for discussion on Monday's telecon. Steven -- ---------------------------------------------------------------- Dr Steven Newhouse Tel:+44 (0)2380 598789 Director, Open Middleware Infrastructure Institute-UK (OMII-UK) c/o Suite 6005, Faraday Building (B21), Highfield Campus, Southampton University, Highfield, Southampton, SO17 1BJ, UK

2 2