- ogsa-hpcp-wg - lists.ogf.org

Thoughts turn to SC08
by Steven Newhouse 15 Oct '08

15 Oct '08

And as October comes thoughts turn to SC08... For the last two years the HPCP-WG has organized interoperability experiments around the HPC Basic Profile. This year we can demonstrate the use of this OGF recommended standard as it enters production use in products and projects. During OGF 24 we discussed some options for activities at SC08: * Deploy compliant implementations. Record those organizations present at SC08 on the group's wiki page * Interoperability tests around the File Staging profile. Use as the basis of writing and submitting the experience document to the specification from proposed to recommended. Need to define which (of the optional) file staging protocols we should use the resources available for testing the endpoints. * Implementing and using the Application Template. This provides a mechanism to reduce the amount of endpoint specific information a client has to provide to start an activity on a resource. This information is provided (transparently) server side by the endpoint. There is a draft specification. The SC08 experience could be used to refine it. * GLUE. We could use GLUE (if we can agree on a simple way to provide the information and what information to provide) to guide some simple resource selection between HPCBP endpoints. This could be combined with the Application Template to some simple brokering between jobs. Comments welcome on these topics! If there is sufficient interest I would suggest a call next week on Thursday 9th October at 0900 Pacific. Steven

4 7

HPCBP SC07 Flyer
by Steven Newhouse 08 Oct '08

08 Oct '08

All, This is the SC07 flyer. As per today's call Marty will be coordinating the updates needed to turn it into the SC08 flyer. Please send all updates & additions to him. Steven

1 0

SC08 Demo Call
by Steven Newhouse 08 Oct '08

08 Oct '08

Present: Marty Humphrey Mark Morgan Joel Replogle Stephen McGough Steven Newhouse Chris Smith Vesso Novov Bill Nitzberg Agenda 1. Available implementations * Software: GridSAM, Genesis II, Platform, Microsoft, UVA - Service on WinHPC Server, UVA - Service on PBS, Altair consumes a HPCBP endpoint * SC08 Presence: Genesis II (people no space), Platform (looking for demo time), Altair (space but no people yet), Microsoft (space and people), GridSAM (no space, maybe people) 2. File Staging Support Genesis II - Supports SCP but could do others UVa - Support needs TBC Altair - Client side file staging. Supports 'everything' if the client & services installed. Platform - Thinks BES++ supports FTP. Microsoft - Support FTPS with an extension for SCP GridSAM - Supports FTPS, SFTP, HTTP, WEBDAV, GridFTP. Will look to see if SCP can be supported. SCP may be the common protocol, but not critical as we can do pairwise testing. 3. Application Template Fairly straight forward to implement. Put a standard 'JSDL' on the wiki page for a number of applications. The localized information is provided server side: * sleep * executes something locally * executes something locally with input & output data files. Open source applications available on Windows & Linux. Find data sets to provide short run times. 4. GLUE What aspects of GLUE might we want to use? We could just return a GLUE document. * Do we want to specify what we want to return for a BES endpoint? * How can we use the returned GLUE elements to select a resource or something? Look to return (minimally) * Execution environment * Application environment Probably leave out the share aspect in the first pass. Need to separate out what we might do now for SC08 against what we do long term. 5. SC08 Demo * Do pairwise interaction to verify the file staging specification for the experience document * Have application templates that people can use * Record material on the OGF wiki page 6. SC08 BOF Wednesday evening. Marty will ping the group for possible involvement once the plans for the BOF crystallizes. The BOF could cover current status, discuss current work (file staging, application template) and future work (GLUE). 7. AOB * Promotion - updating the one pager Action: * Have a call (next Wednesday) where we see what the GLUE schema does! * Wiki page for SC08 * SC08 demo call in two weeks time * Provide update details to Marty on one pager.

1 0

HPCBP and BES 1.0 are now complete
by Steven Newhouse 18 Sep '08

18 Sep '08

All, At OGF24 this week, the Standards Council reviewed the experimental documents relating to the: * HPC Basic Profile 1.0 * HPC Profile Application Extension 1.0 * Job Submission Description Language 1.0 * Basic Execution Service 1.0 With the evidence contained within the experimental documents relating to the implementation and interop experiences, these specifications were moved from the 'provisional' to 'full' OGF Recommendations. Many thanks to the OGF community for their engagement in this activity over the years. The documents and their status on the OGF website will be updated over the next few days. Steven

1 0

OGF24 HPC Profile WG Session Tues Sept 16 9:00am
by Marty Humphrey 09 Sep '08

09 Sep '08

Folks, We will be having an HPC Profile WG Session at OGF24 on Tuesday Sept 16 at 9:00am. This is an important point for the working group, so we encourage everyone to attend! Topics of discussion include: . Quick update on the status of the specifications/documents already submitted to the OGF (HPC Basic Profile, Interop Experiences with the HPC Basic Profile, HPC File Staging Profile, Advanced Filter) . Quick update on the open-source implementations (e.g., BES++, UVa WinHPCP, etc.) . Discussion of current activities/drafts (e.g., use of GLUE, Kerberos profile, Application Template, . Discussion of uptake/outreach activities (e.g., Supercomputing 2008 Show Floor, Supercomputing 2008 Birds-of-a-Feather, etc.) See you there! -- Marty Marty Humphrey Associate Professor Department of Computer Science University of Virginia

1 0

Public Comments Sought: ISV Remote Computing Usage Primer
by Steven Newhouse 28 Aug '08

28 Aug '08

All, The Independent Software Vendor Remote Computing Usage Primer (http://www.ogf.org/Public_Comment_Docs/Documents/2008-07/ISV-V93.pdf) is currently in public comment. Abstract below: This document provides information to the Independent Software Vendor (ISV) community as to how specifications from the Open Grid Forum and other standard setting organizations (SSO) can be used to support Remote Computing scenarios. It does not define any standards or technical recommendations. Please read and make public comments by the 1st September 2008 through http://www.ogf.org/gf/docs/comment.php?id=271 Many thanks, Steven & Andrew

1 0

File staging faults
by Piotr Domagalski 26 Aug '08

26 Aug '08

Hi! Hopefully, this is the last thing that bothers me: 1. As far as I can see, there no such thing as an XML Schema for HPC File Staging Profile. Is that right? 2. Assuming that #1 is true, I'm preparing a schema for me to ease handling of the specified elements. The faults seem a bit unclear to me, though. What are their types? Are they all xsd:string? What about UnsupportedProtocolFault and DeleteOnTerminationFault from which description you can read that they could have a <File> subelement? Do they solely have this subelement or are they mixed-type so that they can contain both string and <File> subelement, e.g.: <DeleteOnTerminationFault> something bad happend <File>ftp://foo.com/bar.txt</File> </DeleteOnTerminationFault> Thanks for any clarifications! -- Piotr Domagalski

4 3

ActivityStatusType inconsistency
by Piotr Domagalski 25 Aug '08

25 Aug '08

Hi all! In the examples in latest HPC File Staging Profile (https://forge.gridforum.org/sf/go/doc15024?nav=1) the state of activity is a value of bes-factory:ActivityStatus/bes-factory:State whereas according to the latest BES XML Schema (https://forge.gridforum.org/sf/go/doc15062?nav=1) it is put into an attribute of the bes-factory:State element. -- Piotr Domagalski

2 2

Re: [ogsa-hpcp-wg] [SAGA-RG] Hosted applications
by Steven Newhouse 25 Aug '08

25 Aug '08

There is also a bit of overlap here with the Application Template done in the HPC Profile WG. http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.ogsa-hpcp… The main goal of this spec is to reduce the amount of info the user has to specify in a JSDL document and let the system admin provide (server side) the details needed to run a particular admin on a local cluster. Steven > -----Original Message----- > From: saga-rg-bounces(a)ogf.org [mailto:saga-rg-bounces@ogf.org] On > Behalf Of Malcolm Illingworth > Sent: Monday, August 25, 2008 10:15 AM > To: 'SAGA RG' > Subject: Re: [SAGA-RG] Hosted applications > > Hi Thilo, > > I'm afraid its rather a contrived example. This would execute a bash > script > using the bash environment. > > The idea is that UNICORE supports "applications as a service". For > example, > imagine an application installed at a large number of sites. Different > sites > might install the application in different locations, eg for local > operating > reasons. To avoid the user having to know the location at each site, > UNICORE > can expose the application as a "service" at each site. So, another > contrived example could be as follows: > > <jsdl:JobDescription> > <jsdl:Application> > <jsdl:ApplicationName>CPMD</jsdl:ApplicationName> > <jsdl:ApplicationVersion>3.1.16</jsdl:ApplicationVersion> > <jsdl1:POSIXApplication > xmlns:jsdl1="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix"> > <jsdl1:Environment > name="INPUT_DATA">mydata.dat</jsdl1:Environment> > </jsdl1:POSIXApplication> > </jsdl:Application> > </jsdl:JobDescription> > > This would run a CPMD instance, without the user having to know the > exact or > even relative location of the application. > > Hope this makes things a bit clearer? > > Malcolm. > (ps perhaps this is overlapping with the service discovery API?) > > > -----Original Message----- > From: Thilo Kielmann [mailto:kielmann@cs.vu.nl] > Sent: 25 August 2008 17:39 > To: Malcolm Illingworth > Subject: Re: [SAGA-RG] Hosted applications > > Malcolm, > > I am not fully understanding your example. > Can you please add what would actually be executed given this piece of > JSDL? > > I am wondering whether a relative path would be "good enough" or what > else > would be missing? > The "other end of the spectrum" would be to use the RPC package but > this > might be overkill. > > Curious... > > > Thilo > > On Mon, Aug 25, 2008 at 02:35:25PM +0100, Malcolm Illingworth wrote: > > From: Malcolm Illingworth <malcolm(a)epcc.ed.ac.uk> > > To: 'SAGA RG' <saga-rg(a)ogf.org> > > Subject: [SAGA-RG] Hosted applications > > > > Hi, > > > > My apologies if this is referring to an old discussion, but here goes > ... > > > > UNICORE supports the concept of an "abstract" hosted application. > This > > means that an application is installed on a remote execution host, > and > > can be referred to via a name, rather than a path to the executable. > > In JSDL this would look something like: > > > > ================================================== > > <jsdl:JobDescription> > > <jsdl:Application> > > <jsdl:ApplicationName>Bash shell</jsdl:ApplicationName> > > <jsdl:ApplicationVersion>3.1.16</jsdl:ApplicationVersion> > > <jsdl1:POSIXApplication > > xmlns:jsdl1="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix"> > > <jsdl1:Environment > name="SOURCE">input.sh</jsdl1:Environment> > > </jsdl1:POSIXApplication> > > </jsdl:Application> > > </jsdl:JobDescription> > > ================================================== > > > > Any thoughts of how I could support this at the SAGA level? The SAGA > > job description defines an EXECUTABLE property, but I'm assuming this > > has to be an actual path. > > > > Regards, > > Malcolm. > > > > ---------------------------------------------------- |epcc| - > > > > Malcolm Illingworth > > EPCC, University of Edinburgh > > James Clerk Maxwell Building > > Mayfield Road E-mail: malcolm(a)epcc.ed.ac.uk > > Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 > > United Kingdom Fax: + 44 (0) 131 650 6555 > > > > ------------------------------------------------------------- > > > > -- > > saga-rg mailing list > > saga-rg(a)ogf.org > > http://www.ogf.org/mailman/listinfo/saga-rg > > > > -- > Thilo Kielmann > http://www.cs.vu.nl/~kielmann/ > > -- > saga-rg mailing list > saga-rg(a)ogf.org > http://www.ogf.org/mailman/listinfo/saga-rg

2 1

greeting from Leitao Guo
by Guo Leitao 20 Aug '08

20 Aug '08

hi everyone, This is Leitao Guo from China Mobile Research Institute. I'm engaged in the research of high performance computing and distributed database. I subscribed the mailinglist of OGSA-HPCP-WG, but didn't receive any email. I don't know whether there is something wrong with my subscription, or this group is not active any more. Can anyone tell me the work status of this group? Many thanks! Leitao 2008.8.20

3 4