- ogsa-wg - lists.ogf.org

GT 4.0 GRAM docs for input to BES discussions
by Karl Czajkowski 10 Mar '05

10 Mar '05

Here is a compact summary of the GT 4.0 WS-GRAM interface and links to further documentation. Please start by reading the documentation unless you are already an expert on GRAM, XSD, WSDL, and WSRF concepts. --------------------------------------------------------- GT 4.0 WS-GRAM documentation Note, these documents are in draft form... 1) GRAM Key Concepts http://www-unix.globus.org/toolkit/docs/development/4.0-drafts/execution/ke… 2) WS-GRAM Approach http://www-unix.globus.org/toolkit/docs/development/4.0-drafts/execution/ke… 3) Semantics and syntax of WSDL http://www-unix.globus.org/toolkit/docs/development/4.0-drafts/execution/ws… 4) Job Description Language http://www-unix.globus.org/toolkit/docs/development/4.0-drafts/execution/ws… 5) Links to more WS-GRAM docs than you can shake a stick at http://www-unix.globus.org/toolkit/docs/development/4.0-drafts/execution/ws… Because this is already a lot of documentation, the following is a terse overview of the WSDL rendering of the core WS-GRAM job discovery, submission, monitoring, and cancellation interface. --------------------------------------------------------- PORTTYPE ManagedJobFactoryPortType The Managed Job Factory Resource (MJFR) represents one localized job scheduler or compute element. (In GT 4.0, there is a separate MJFR for each deployed local scheduler adapter on a host.) It is a WSRF style resource with a resource properties document to represent the overall status and capabilities of the local compute element: <managedJobFactoryResourceProperties> <localResourceManager>xsd:string</localResourceManager> <globusLocation>xsd:string</globusLocation> <hostCPUType>xsd:string</hostCPUType>? <hostManufacturer>xsd:string</hostManufacturer>? <hostOSName>xsd:string</hostOSName>? <hostOSVersion>xsd:string</hostOSVersion>? <scratchBaseDirectory>xsd:string</scratchBaseDirectory>? <delegationFactoryEndpoint>wsa:EndpointReferenceType</delegationFactoryEndpoint> <stagingDelegationFactoryEndpoint>wsa:EndpointReferenceType</stagingDelegationFactoryEndpoint>? <condorArchitecture>xsd:string</condorArchitecture>? <condorOS>xsd:string</condorOS>? <gluece:GLUECE> <gluece:Cluster Name=xsd:string UniqueID=xsd:string InformationServiceURL=xsd:anyURI> <SubCluster/>* xsd:any##other* </gluece:Cluster>* <ComputingElement> <Info/>? <State/>? <Policy/>? <Job/>* <AccessControlBase/>? xsd:any##other* </ComputingElement>* xsd:any##other* </gluece:GLUECE>? <gluece:GLUECESummary/>? <ServiceMetaDataInfo> <startTime>xsd:dateTime</startTime> <version>xsd:string</version> </ServiceMetaDataInfo> </managedJobFactoryResourceProperties> These properties allow inspection of the underlying compute platform using an XSD rendering of the GLUE schema: gluece:GLUECE, gluece:GLUECESummary (Please see the GLUE schema for more information, using ad-hoc GRAM properties: hostCPUType, hostManufacturer, hostOSName, hostOSVersion while a few provide introspection on the GRAM deployment itself: localResourceManager, globusLocation, scratchBaseDirectory, ServiceMetaDataInfo The two EPRs are used by a client to discover where to delegate credentials that will be referenced by future job submissions: delegationFactoryEndpoint, stagingDelegationFactoryEndpoint. ---------- OPERATION job:createManagedJob Request creation of a Managed Executable Job Resource whose EPR will be returned in the response. INPUT message: createManagedJobInputMessage has one part: <createManagedJob> <InitialTerminationTime>xsd:dateTime</InitialTerminationTime>? <JobID>wsa:AttributedURI</JobID>? <wsnt:Subscribe></wsnt:Subscribe>? <desc:job> ... </desc:job> </createManagedJob> The optional JobID element is used to request idempotent invocation semantics in a binding-independent manner. The optional wsnt:Subscribe element is used to request automatic subscription to the newly created Managed Job. This call can also create a Managed Multi-Job Resource, i.e. a co-allocated job spread across multiple WS-GRAM hosts, because the job element is actually in an XSD choice with a multijob element. OUTPUT message: createManagedJobOutputMessage has one part: <createManagedJobResponse> <NewTerminationTime>xsd:dateTime</NewTerminationTime> <CurrentTime>xsd:dateTime</CurrentTime> <managedJobEndpoint>wsa:EndpointReferenceType</managedJobEndpoint> <subscriptionEndpoint>wsa:EndpointReferenceType</subscriptionEndpoint>? </createManagedJobResponse> The optional subscriptionEndpoint is returned if ----------- Other operations, composed from the WSRF service environment, are: [From WS-ResourceProperties] -- get access to factory status information GetResourceProperty QueryResourceProperties GetMultipleResourceProperties --------------------------------------------------------- PORTTYPE ManagedExecutableJobPortType A Managed Executable Job Resource (MEJR) represents one job that has already been submitted by a client. It is a WSRF style resource with a resource properties document to represent status of the job: <managedExecutableJobResourceProperties> <stdoutURL>xsd:anyURI</stdoutURL>? <stderrURL>xsd:anyURI</stderrURL>? <credentialPath>xsd:string</credentialPath>? <exitCode>xsd:int<exitCode/>? <serviceLevelAgreement> <desc:job> ... </desc:job> </serviceLevelAgreement> <Capacity>xsd:int</Capacity> <userSubject>xsd:string</userSubject> <fault/> <TopicExpressionDialects>xsd:anyURI</TopicExpressionDialects> <Topic Dialect=xsd:anyURI> xsd:any? </Topic>+ <TerminationTime>xsd:dateTime</TerminationTime> <localUserId>xsd:string</localUserId> <CurrentTime>xsd:dateTime</CurrentTime> <holding>xsd:boolean</holding> <RegistrantData>xsd:base64Binary</RegistrantData> <RendezvousCompleted>xsd:boolean</RendezvousCompleted> <FixedTopicSet>xsd:boolean</FixedTopicSet> <state>Unsubmitted|StageIn|Pending|Active|Suspended|StageOut|Cleanup|Done|Failed</state> </managedExecutableJobResourceProperties> These properties relate to job output file management: stdoutURL, stderrURL delegated credential management: credentialPath parallel task rendezvous (for MPICH-G2): Capacity, RegistrantData, RendezvousCompleted job status: exitCode, fault, holding, state job introspection: credentialPath, serviceLevelAgreement, localUserId for WSRF introspection: TopicExpressionDialects, Topic, FixedTopicSet, TerminationTime, CurrentTime. ---------- OPERATION exec:release Releases job from hold state. The hold state is an optional behavior selected in the job description to prevent post-execution file deletions (clean-up) from occuring while a remote client is still attempting to access the files. The release operation permits the normal clean-up to occur. INPUT message: releaseInputMessage has one (empty) part: <release/> OUTPUT message: releaseOutputMessage has one (empty) part: <releaseResponse/> ----------- In addition, a rendezvous provides an additional operation: [From GT4 rendezvous manager type] -- support for bootstrapping MPICH-G2 etc. register Other operations, composed from the WSRF service environment, are: [From WS-ResourceLifetime] -- schedule termination of a job SetTerminationTime Destroy [From WS-ResourceProperties] -- get access to job status information GetResourceProperty QueryResourceProperties GetMultipleResourceProperties [From WS-BaseNotification] -- subscribe for job status notifications Subscribe GetCurrentMessage --------------------------------------------------------- Job description document syntax for use in creating a Managed Executable Job Resource: <job> <factoryEndpoint>wsa:EndpointReferenceType</factoryEndpoint>? <jobCredentialEndpoint>wsa:EndpointReferenceType</jobCredentialEndpoint>? <stagingCredentialEndpoint>wsa:EndpointReferenceType</stagingCredentialEndpoint>? <localUserId> ... </localUserId> [0..1] <holdState> ... </holdState> [0..1] <executable>xsd:string</executable>? <directory>xsd:string</directory>? <argument>xsd:string</argument>* <environment> <name>xsd:string</name> <value>xsd:string</value> </environment>* <stdin>xsd:string</stdin>? <stdout>xsd:string</stdout>? <stderr>xsd:string</stderr>? <count>xsd:positiveInteger</count>? <libraryPath>xsd:string</libraryPath>* <hostCount>xsd:positiveInteger</hostCount>? <project>xsd:string</project>? <queue>xsd:string</queue>? <maxTime>xsd:long</maxTime>? <maxWallTime>xsd:long</maxWallTime>? <maxCpuTime>xsd:long</maxCpuTime>? <maxMemory>xsd:nonNegativeInteger</maxMemory>? <minMemory>xsd:nonNegativeInteger</minMemory>? <jobType>mpi|single|multiple|condor</jobType>? <fileStageIn>rft:TransferRequestType</fileStageIn>? <fileStageOut>rft:TransferRequestType</fileStageOut>? <fileCleanUp>rft:DeleteRequestType</fileCleanUp>? <extensions>xsd:any##other</extensions>? </job> An extended form of this syntax consists of an array of the above descriptions to define a "multi-job". -- Karl Czajkowski karlcz(a)univa.com

1 0

GGF13 OGSA session #5
by Hiro Kishimoto 10 Mar '05

10 Mar '05

Hi all, At today's OGSA-WG teleconference, we agree to use OGSA-WG session #5 (March 15 (Tue) 11am-12:30pm Korea time = 8-9:30pm CST) for EGA feedback discussion. We are doing our best to prepare GGF13 but unlikely to make "basic data profile" discussion ready by March 15. We agreed to skip it at GGF13 On the other hand, John Tollefsrud, GGF liaison to the Enterprise Grid Alliance (EGA), asks us to review and answer EGA's feedback on OGSA documents. UK eScience is a member of EGA and Steven Newhouse will be represent EGA at the session. Also another EGA rep in US can dial-in. Attached is their feedback document. Please have a look. Thanks, ---- Hiro Kishimoto

1 0

DRMAA view on BES
by Rajic, Hrabri 09 Mar '05

09 Mar '05

While the DRMAA group members would like to stay on the sidelines regarding the WS infrastructure, JSDL, and WS-Agreement issues we feel that OGSA-BES members could save some time when discussing task/job/whatever management, control, and status portTypes to look into section 6.3.8 on page 26 of the DRMAA IDL binding document. If this is something that the initiators have in mind to discuss at OGSA-BES BoF we could contribute a very (ultra?) short presentation. Greetings -Hrabri

1 0

Regrets for this evenings telecon (3/9)
by Tom Maguire 09 Mar '05

09 Mar '05

Need to tidy up a few things around the house before travel to Korea, Tom Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. – Antoine de Saint-Exupery T o m M a g u i r e STSM, On Demand Architecture Poughkeepsie, NY 12601

1 0

Globus Alliance views on BES
by Karl Czajkowski 09 Mar '05

09 Mar '05

Hi, sorry for the delay but we've put together some documents summarizing our Globus Alliance view on what the important issues and constraints are for a successful Basic Execution Service standard. Please see the attached documents: 1. Approach -- a high-level survey of our concerns 2. Rendering -- more detailed thoughts about interface designs 3. Roadmap -- simple thoughts on a phased approach Later, I will send an additional note with links to Globus Toolkit 4.0 draft documents for our WS-GRAM job management component, and a brief overview of the service interface for those who want to think in WSDL and not read a bunch of documents. This probably will not be ready for another 12-18 hours. Please feel free to respond on the list or privately with comments about agreement, disagreement, or relative priorities! I will try to incorporate any feedback I hear into planning for my ultra short presentation at the BoF in Seoul. Thanks, karl -- Karl Czajkowski karlcz(a)univa.com

1 1

Proposed agenda for OGSA ByteIO BOF
by Dave Berry 09 Mar '05

09 Mar '05

All, Here is a draft agenda for Monday's ByteIO BOF. (I note Ian's concern about WG names and assume this will be thrashed out by the GFSG/ADs). Presentation of GGF IP Rules Pass around sign-up sheet Background & Use Cases (Dave) Discussion Proposed interface (Mark) Discussion Charter discussion Seven questions Existing WG or new WG? Milestones Stakeholders, co-chairs Scheduling of work, calls etc. Best wishes, Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

Proposed agenda for March 9th call
by Hiro Kishimoto 09 Mar '05

09 Mar '05

Hi all, The following is a proposed agenda for OGSA-WG telecon on March 9th Wednesday from 5pm to 7pm (CST). This is the last one before GGF13. Dial-in numbers for Wednesday: Free: +1-888-452-0308 Intl/Toll: +1-484-644-0505 PIN: 71815 Screen share service will be provided. URL: http://ogsa.glance.net Session key: 0309 See more explanation: http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/06/msg00077.html 1) Early discussion (10 min) Note taker assignment Roll call Telecon minutes approval (if available) Agenda bashing 2) GGF13 session preparation (30 min) Confirmation of owner/note taker/guests for each session - OGSA session plan https://forge.gridforum.org/projects/ogsa-wg/document/GGF13_session_plan - GGF13 schedule http://tinyurl.com/4cz98 - Basic EM profile discussion (Hiro will send out proposal before the call) 3) roadmap discussion (if available) 4) Wrap up (10 min) AOB Mar. 13-16 GGF13 in Seoul ---- Hiro Kishimoto

1 1

Base Profile Issue: Query RP
by David Snelling 09 Mar '05

09 Mar '05

Folks, I would like to open an issue on this. It appears that Query resource Properties was made mandatory in the Base Profile. I believe that give the state and number of query technologies possible and the existence of two other mechanisms to access RPs, this should be optional. Thanks to Andreas for checking the minutes for me to ascertain if this issue was actually discussed. It was not and therefore, I feel it is safe to open it. Begin forwarded message: > From: Andreas Savva <andreas.savva(a)jp.fujitsu.com> > Date: 9 March 2005 7:19:16 GMT > To: David Snelling <David.Snelling(a)uk.fujitsu.com> > Subject: Re: Favour > > I also looked at my initial notes and I couldn't find a mention of > this. I think this is a problem with my notes (the minute taker was > broken at the time). Sorry. > > Is there anything else you want me to do? > > Andreas Savva wrote: >> I had a look at the f2f minutes but there is no mention of this.... >> Sorry. >> My notes from the first day were very bad as I had a very hard time >> staying focused. Terrible jet lag in the afternoon. >> I'll have a look through the raw notes which I have back home and let >> you know if I find anything. >> David Snelling wrote: >>> Andreas, >>> >>> Can you find (in the minutes) any info on the discussion at the F2f >>> as to why the QueryRP was made manditory? >>> > > -- > Andreas Savva > Fujitsu Laboratories Ltd > > -- Take care: Dr. David Snelling < David . Snelling . UK . Fujitsu . com > Fujitsu Laboratories of Europe Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE +44-208-606-4649 (Office) +44-208-606-4539 (Fax) +44-7768-807526 (Mobile)

1 0

Proposed (DRAFT) Agenda for Basic EM BOF
by Andrew Grimshaw 09 Mar '05

09 Mar '05

The objective of the BOF is to form the OGSA Basic Execution Management Service Working Group (OGSA-BEM). The output of the OGSA-BEM will fit into the overall OGSA EMS architecture, as initially described in the OGSA V1 Document. It is expected that the OGSA EMS architecture will evolve over time - and that evolution will eventually need to be incorporated. Such incorporation may not take place in this working group - depending on whether OGSA-BEM completes before changes are made in the overall OGSA-EMS. Proposed Agenda Presentation of GGF IP Rules Pass around sign-up sheet Background on OGSA EMS Basic Execution Management - A Focus on Minimal Services Draft Service Containers description Charter discussion Should charter be extended to include "Actors"? Should the charter be extended to include any form of agreements? Milestones Scheduling of calls

4 7

RE: [ogsa-wg] A thought
by Von Reich, Jeffrin 08 Mar '05

08 Mar '05

Well, There is nothing like sense of humor for doing a self analysis :). for that maybe more coffee also might be required ( and more note taking too :) ) . take care, Jeffrin -----Original Message----- From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On Behalf Of Ian Foster Sent: Monday, March 07, 2005 5:12 PM To: Andrew Grimshaw; ogsa-wg(a)ggf.org Subject: Re: [ogsa-wg] A thought and consumed countless gallons of coffee!! At 06:55 PM 3/7/2005 -0500, Andrew Grimshaw wrote: Since Brussels the OGSA working group (including design teams) have had three face-to-face meetings - in Washington DC, Chicago, and London. They have - spent over eighty hours on group teleconferences - flown over 200,000 miles to face-to-face meetings - spent over 100 person nights in hotel rooms - eaten countless muffins and sweet rolls - gained more kilos than we can count _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org <http://www.globus.org/>

1 0