October 2006 - ogsa-hpcp-wg

Agenda for Friday's Call
by Richard Ciapala 20 Oct '06

20 Oct '06

Hi, During tomorrow's conf call, maybe we can take another step forward with regards to coordinating the demos that show off the HPC Basic Profile web service implementations. As we have discussed on previous calls, the rough plan has been to balance some level of consistency across server-side implementations while allowing participants to build their own clients. Though in some cases participants may partner to build metascheduling or workflow scenarios, we also discussed a "base" demo where any participant can execute an activity on any other participant's cluster, manage the activity and retrieve output. If we agree to have this base demo, besides the individual HPC Basic Profile web service implementations, each participant would be asked to provide the following: * An application hosted on their cluster * The JSDL to execute that application * A consistent way to specify the name of the application in the JSDL so it can be forwarded to the correct cluster (for those who will do forwarding scenarios) * A HTTP directory from where the results of the application is produced. Possibly in the format of (http://address/output/application/user) With that said, an agenda for tomorrow's call could be: * Should we move forward with the base demo? * How should the application be specified in the JSDL for forwarding scenarios? * Use existing JSDL ApplicationName (which is not in HPC Basic Profile but isn't precluded from being used) * HPCProfileApplication\@name attribute (which is in the HPC Basic Profile) * Other? * Open marketing issues * WSDL/XSD changes Thanks

3 5

Discuss Open SC06 Interop Marketing Issues
by Richard Ciapala 20 Oct '06

20 Oct '06

When: Tuesday, October 24, 2006 8:00 AM-9:00 AM (GMT-08:00) Pacific Time (US & Canada); Tijuana. Where: Conf Call *~*~*~*~*~*~*~*~*~* Toll Numbers: USA 1-203-480-8000 TollFree Numbers: USA 866-500-6738 Participant Passcode: 231867

1 0

Re: [ogsa-hpcp-wg] Some Question from EGEE project
by Glenn Wasson 20 Oct '06

20 Oct '06

Alessandro, Let me take a shot at your questions and others can chime in if they want. 1) I'm not sure that the WS-Addressing issue was resolved. Some services require WSA headers, some do not. Of course, all services use EPRs in message bodies to refer to jobs, but I don't think that's what you are asking. I guess there are also some clients that automatically put WSA info in the headers of their outgoing messages. However, if your service doesn't actually need this info, I guess you can ignore it. The security story in general is that clients will communicate with services using https and WS-Security UsernameTokens will be used (each service and/or user has their own username and password). 2) the details of the demo are still be worked out 3) I'm not sure exactly which version of the WSDL we want to use. The last version that Peter sent out (that had the minOccurs=0 changes in it) is the one I intend to use, but I don't know if other changes will occur between now and SC. These changes are really just corrections to the WSDL and so I suspect everyone will want to use the latest. Perhaps on the call this Friday, we can discuss when to "hold" the WSDL. 4) I use doc/literal. I believe everyone else I've interop'd with so far does too. Anyone using something else? Glenn _____ From: ogsa-hpcp-wg-bounces(a)ogf.org [mailto:ogsa-hpcp-wg-bounces@ogf.org] On Behalf Of Alessandro Maraschini Sent: Thursday, October 19, 2006 12:56 PM To: ogsa-hpcp-wg(a)ogf.org Cc: ogsa-bes-wg(a)ogf.org Subject: [ogsa-hpcp-wg] Some Question from EGEE project Hi all, After a brief discussion with Steven Newhouse at EGEE06, we decided to try and participate in the SC06 interoperability DEMO with two more services coming from the european project EGEE (http://www.eu-egee.org/) CREAM & WMPROXY: those WS-based services have been working for several months now; over the last few weeks we have been trying to integrate our work with BES and JSDL implementation. Yet we still have some unsolved questions, I hope you would be so kind to answer them: 1) WS-Addressing and WS-Security According to what I read, the client will send SOAP messages including WS-addressing headers while the server might not support WS-Addressing at all and just ignore WS-addressing headers, is that correct? What about WS-security on server side? Is that a suggestion rather than a to-be-implemented feature? 2) Is there available any [template] client that will be used to call services during the demo? How will be the demo structured? Which is the client used for querying the service? 3) Can we consider the BES-factory wsdl as definitive? There had been some complains during the week, yet only minimal parts aparently are going to be changed. Can we assume the last WSDL is the one that will be used for the demo? 4) Binding and style/encoding Regarding the Binding, are we free to use both DOCUMENT/LITERAL and RPC/LITERAL or is there some restriction? cheers, Alessandro -- ----- Alessandro Maraschini Grid R&D Group Government and Institutions Division DATAMAT S.p.A. - a Finmeccanica company Via Laurentina, 760 - I - 00143 Rome - Italy http://www.datamat.it mailto: alessandro.maraschini(a)datamat.it phone: +39 06 5027 4501 (direct) +39 06 5027 2211 (secretary) fax: +39 06 5027 4330

2 1

Problems
by Mark Morgan 19 Oct '06

19 Oct '06

I think that there are additional problems with the WSDL and XSD for bes-factory in that the result types that have Fault elements inside all assume a cardinatlity of [1, 1] when it seems clear to me that the type should be of cardinality [0, 1]. -Mark -- Mark Morgan Research Scientist Department of Computer Science University of Virginia http://www.cs.virginia.edu mmm2a(a)virginia.edu (434) 982-2047

4 10

Answers to HPCP Marketing Questions
by Richard Ciapala 19 Oct '06

19 Oct '06

John/Joel, Here are answers to the first set of questions from your previous email. Everyone, Please feel free to comment and help fill out the "Who is involved?" section. Thanks --- What is the HPC Basic Profile? The HPC Basic Profile is a proposed standard that is used to describe how a particular set of specifications are composed to enable interoperability between HPC systems for a specific set of operations. It consists of references to existing specifications, along with specific clarifications of the contents of those specifications, restrictions on the use of those specifications, and references to any normative extensions to those specifications. Examples of the operations include the submission of tasks to a batch job scheduler, the retrieval of a task's execution status and the retrieval of information about an HPC system's resources. Specifications that are referenced by the profile include the WS-I Basic Profile, the Job Submission Description Language (an XML language for describing tasks to be run) and the Basic Execution Service (a Web service definition of the operations). Why do people care? The HPC Basic Profile enables interoperability between HPC systems from different academic institutions, open source communities and vendors within the scope of the operations defined in the specifications it references. It leverages standard Web services protocols and development environments. It enables organizations to more easily integrate HPC systems into their existing environments, interface with HPC systems at remote locations and leverage generic HPC utilities. The profile also allows application and middleware software providers to target multiple HPC systems via a single protocol. Who are involved? The HPC Basic Profile SC06 Demo working group is part of the Open Grid Forum. The following is a list of the participants and details of their HPC Basic Profile implementation: Participant Web Service Tooling (OS) Scheduler (OS) Globus ?? ?? Fujitsu ?? ?? Platform Computing GSOAP (Linux) LSF (Linux) Microsoft Windows Communication Framework (Windows) Compute Cluster Server (Windows) Tokyo Institute of Technology MS Web Services Enhancements (Windows) SGE (Linux) Unicore ?? ?? University of Virginia MS Web Services Enhancements (Windows) Compute Cluster Server (Windows) University of Virginia MS Web Services Enhancements (Windows) PBS (Linux) University of Virginia Jetty/Axis (Linux) ?? NOTE: This is the list from the wiki and will change. Needs to include each participants target OS, web service tools and scheduler. What are we showing? The SC06 interop demonstration shows interoperability using the HPC Basic Profile between multiple 3rd party schedulers and Web service platforms. Each participant's HPC Basic Profile Web service fronts a compute cluster that hosts one or more applications. Demo stations are setup at booths throughout the conference where tasks are submitted or forwarded via the HPC Basic Profile to the compute cluster that hosts the application requested in the task.

3 2

Some Question from EGEE project
by Alessandro Maraschini 19 Oct '06

19 Oct '06

Hi all, After a brief discussion with Steven Newhouse at EGEE06, we decided to try and participate in the SC06 interoperability DEMO with two more services coming from the european project EGEE (http://www.eu-egee.org/) CREAM & WMPROXY: those WS-based services have been working for several months now; over the last few weeks we have been trying to integrate our work with BES and JSDL implementation. Yet we still have some unsolved questions, I hope you would be so kind to answer them: 1) WS-Addressing and WS-Security According to what I read, the client will send SOAP messages including WS-addressing headers while the server might not support WS-Addressing at all and just ignore WS-addressing headers, is that correct? What about WS-security on server side? Is that a suggestion rather than a to-be-implemented feature? 2) Is there available any [template] client that will be used to call services during the demo? How will be the demo structured? Which is the client used for querying the service? 3) Can we consider the BES-factory wsdl as definitive? There had been some complains during the week, yet only minimal parts aparently are going to be changed. Can we assume the last WSDL is the one that will be used for the demo? 4) Binding and style/encoding Regarding the Binding, are we free to use both DOCUMENT/LITERAL and RPC/LITERAL or is there some restriction? cheers, Alessandro -- ----- Alessandro Maraschini Grid R&D Group Government and Institutions Division DATAMAT S.p.A. - a Finmeccanica company Via Laurentina, 760 I - 00143 Rome Italy http://www.datamat.it mailto: alessandro.maraschini(a)datamat.it phone: +39 06 5027 4501 (direct) +39 06 5027 2211 (secretary) fax: +39 06 5027 4330

1 0

ActivityStatusType in bes-factory.xsd
by Christopher Smith 19 Oct '06

19 Oct '06

Hi all, Given the text in the bes specification (v26), the type of the elements within ActivityStatusType should not be xsd:anyType, but should use the definitions as follows. This is in line with the extensible state model as described in the specification: <xsd:complexType name="ActivityStateType"> <xsd:sequence> <xsd:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/> </xsd:sequence> </xsd:complexType> <xsd:complexType name="ActivityStatusType"> <xsd:choice> <xsd:element name="Pending" type="bes-factory:ActivityStateType"/> <xsd:element name="Running" type="bes-factory:ActivityStateType"/> <xsd:element name="Cancelled" type="bes-factory:ActivityStateType"/> <xsd:element name="Failed" type="bes-factory:ActivityStateType"/> <xsd:element name="Finished" type="bes-factory:ActivityStateType"/> </xsd:choice> </xsd:complexType> Um ... how can we track these issues, and (more importantly) quickly get some closure on them in order to update our implementations? -- Chris

2 2

To many 'any' in factory attributes document
by Christopher Smith 18 Oct '06

18 Oct '06

Hi all, I don't know if others are having this issue, but my tooling doesn't like all the xsd:any that show up in the FactoryResourceAttributesDocumentType. The problem is this: BasicFactoryResourceAttributesDocumentType has: - stuff1 - stuff2 - ... - xsd:any Then the FactoryResourceAttributesDocumentType has: - stuff1 (from extension) - stuff2 (from extension) - ... - xsd:any (from extension) - otherstuff1 - otherstuff2 - ... - xsd:any That makes two xsd:any in one complex type, which is one too many in my opinion. Can we perhaps refactor this type? I've also noticed that I can only provide one OperatingSystem, CPUArchitecture, etc, etc, which doesn't allow me to list the elements of my cluster. How about something like: <xsd:complexType name="ResourceAttributesDocumentType"> <xsd:sequence> <xsd:element name="OperatingSystem" type="jsdl:OperatingSystem_Type" minOccurs="0"/> <xsd:element name="CPUArchitecture" type="jsdl:CPUArchitecture_Type" minOccurs="0"/> <xsd:element name="CPUCount" type="xsd:unsignedInt" minOccurs="0"/> <xsd:element name="CPUSpeed" type="xsd:unsignedLong" minOccurs="0"/> <xsd:element name="PhysicalMemory" type="xsd:unsignedLong" minOccurs="0"/> <xsd:element name="VirtualMemory" type="xsd:unsignedLong" minOccurs="0"/> <xsd:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/> </xsd:sequence> <xsd:attribute name="name" type="xsd:string" use="required"/> </xsd:complexType> <xsd:element name="ResourceAttributesDocument" type="ResourceAttributesDocumentType/> <xsd:complexType name="FactoryResourceAttributesDocumentType"> <xsd:complexContent> <xsd:sequence> <xsd:element ref="ResourceAttributesDocument" minOccurs="0" maxOccurs="unbounded"/> <xsd:element name="IsAcceptingNewActivities" type="xsd:boolean" minOccurs="1" maxOccurs ="1"/> <xsd:element name="CommonName" type="xsd:string" minOccurs="0" maxOccurs ="1"/> <xsd:element name="LongDescription" type="xsd:string" minOccurs="0" maxOccurs ="1"/> <xsd:element name="TotalNumberOfActivities" type="xsd:unsignedInt" minOccurs="1" maxOccurs ="1"/> <xsd:element name="LocalResourceManagerType" type="xsd:QName"/> <xsd:element name="ActivityReference" type="wsa:EndpointReferenceType" minOccurs="0" maxOccurs="unbounded"/> <xsd:element name="ContainedResourceAttributes" type="xsd:anyType" minOccurs="0" maxOccurs="unbounded"/> <xsd:element name="NamingProfile" type="xsd:QName" minOccurs="1" maxOccurs ="1"/> <xsd:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/> </xsd:sequence> </xsd:extension> </xsd:complexContent> </xsd:complexType> Even if we don't refactor like I suggested, we need to control the xsd:any's. -- Chris

3 3

UVA SC06 interop endpoint
by Glenn Wasson 18 Oct '06

18 Oct '06

Fellow interopers: I mentioned on the call that I have 2 endpoints available, but one of them is still a little raw, so I'll just give out the URL for one and the second will follow next week. The URL is https://wincluster1.cs.virginia.edu/HPCP/HPCPService.asmx The service is a .NET2.0/WSE3 frontend and MS CCS backend. The CA cert for the host is available at: https://www.pki.virginia.edu/ca/rootcert/rootcert.cer Here's an example JSDL doc that can be used: <ActivityDocument> <JobDefinition xmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl"> <JobDescription> <JobIdentification> <JobName>JobName</JobName> <JobProject>JobProject</JobProject> </JobIdentification> <Application> <ApplicationName>ipconfig</ApplicationName> <HPCProfileApplication xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" name="TaskName" xmlns="http://schemas.ggf.org/jsdl/2006/07/jsdl-hpcp"> <Executable>ipconfig.exe</Executable> <Argument>/all</Argument> <Output>ipconfig.txt</Output> <Error>ipconfig_err.txt</Error> <WorkingDirectory>\\wincluster1\head_node_share\Dev\HPC Profile\HPCP_Out</WorkingDirectory> </HPCProfileApplication> </Application> <Resources> <TotalCPUCount> <UpperBoundedRange>1</UpperBoundedRange> <LowerBoundedRange>1</LowerBoundedRange> </TotalCPUCount> </Resources> </JobDescription> </JobDefinition> </ActivityDocument> *** You can view the resulting output at http://wincluster1.cs.virginia.edu/HPCP-Out/ipconfig.txt All methods are available on this service (though the return from GetFactoryAttributesDocument is a bit sparse). Also, any errors should return SOAPFaults, though their format may be idiosyncratic to my implementation. Glenn

3 4

SC06 Demo Marketing Conf Call - 10/17
by Richard Ciapala 17 Oct '06

17 Oct '06

Participants on the call * Glenn Wasson * John Ehrig * Chris Smith * Mark Linesch * Michel Drescher * Steven Newhouse * Peter Lane * Ming Xu * Susanne Balle * Joel Replogle Notes * We need a volunteer to create the technical description and diagram * We will start using the wiki for the marketing collateral * Besides PR efforts, OGF will announce the demo for SC06 via a broadcast email and on their web site Participant WorkItems * Rich - Send email to working group with answers to following by EOD Tuesday: o What is the HPC Profile? o Why do people care? o Who are involved? o What are we showing (from a general perspective)? * Everyone - Please send to John Ehrig (jehrig(a)inventures.com) and hpcp-wg mailing list the following information: o Marketing contact to coordinate PR (i.e. get quotes) o Booth #'s and booth owner contact names o Timeslots for dedicated HPCP presentation, if possible o Plans for adoption (from commercial and non-commercial participants) * John - Drive responses to the above questions * Chris - Put together slides for participants to use for booth presentations (available closer to SC06) * Chris - Ask a few customers whether they are willing to provide quotes * John - Work on common placard for participants booths

1 0