June 2008 - ogsa-hpcp-wg

FW: [Fwd: FIle Staging Extensions Comment]
by Steven Newhouse 03 Jul '08

03 Jul '08

> -----Original Message----- > From: Mark Morgan [mailto:mmm2a@virginia.edu] > Sent: Monday, June 23, 2008 6:40 PM > To: humphrey(a)cs.virginia.edu; Steven Newhouse; csmith(a)platform.com; > Richard Ciapala > Subject: [Fwd: FIle Staging Extensions Comment] > > I wasn't able to post this to the HPCP mailing list. > > -------- Forwarded Message -------- > > From: ogsa-hpcp-wg-owner(a)ogf.org > > To: mmm2a(a)virginia.edu > > Subject: FIle Staging Extensions Comment > > Date: Mon, 23 Jun 2008 12:38:31 -0500 > > > > You are not allowed to post to this mailing list, and your message > has > > been automatically rejected. If you think that your messages are > > being rejected in error, contact the mailing list owner at > > ogsa-hpcp-wg-owner(a)ogf.org. > > > > email message attachment > > > -------- Forwarded Message -------- > > > From: Mark Morgan <mmm2a(a)virginia.edu> > > > To: ogsa-hpcp-wg(a)ogf.org > > > Subject: FIle Staging Extensions Comment > > > Date: Mon, 23 Jun 2008 13:36:30 -0400 > > > > > > I recently went to implement some of the file staging extensions > that > > > the HPC group has described. In particular I was looking to > implement > > > SCP and I figured that if I was going to implement that spec., it > made > > > sense to do it according to a standard rather than to come up with > > > something new and non-interoperable. However, in the process of > > > implementing the specification, I came across a minor nitpick that > I > > > have concerns about. Specifically, if I read the specification > > > correctly, I believe that it is the case that a BES container is > > > required by the spec. to fault if a user passes authentication > > > information in the URI rather than using the file staging > extensions. > > > I'm confused as to why the group decided to prohibit a valid form > of > > > extension to the JSDL and BES specification. I understand that > that > > > method of passing information may not be the best choice or most > secure > > > choice, but I would think that it makes more sense for the HPC > group to > > > simply standardize on their way of passing authentication > information > > > rather than prohibit other, in my opinion, equally valid > authentication > > > mechanisms. The fact of the matter is that Genesis II has, in the > past, > > > allowed authentication information to be passed in the File staging > > > URIs. At this point, it seems like I have the choice of forgoing > > > backwards compatibility in favor of HPCP File staging extensions > > > compliance, or forgoing HPCP File staging compliance in favor of > > > backward compatibility where I really see no good reason why both > > > couldn't be achieved. > > > > > > I'm not trying to cause trouble here, I'm simply curious as to the > > > reasons for this decision and specifically wanted to mention this > in > > > case the HPC group hadn't thought of this potential issue before. > > > > > > -Mark > > > >

6 8

File Staging Profile published as GFD 135
by Steven Newhouse 29 Jun '08

29 Jun '08

Finally, the HPCBP File Staging profile has been published: http://www.ogf.org/documents/GFD.135.pdf Enjoy! Steven

1 0

Re: [ogsa-hpcp-wg] [OGSA-BES-WG] Activity Delegation Interfaces and Protocols
by Steven Newhouse 04 Jun '08

04 Jun '08

This was certainly too short a notice for me to attend. There has been discussion in the HPCP WG on generating a Kerberos profile for the HPCBP that would support the delegation scenario through ticket forwarding. Can you please post information relating to this discussion to the HPCP mailing list when it becomes available? Thanks, Steven > -----Original Message----- > From: ogsa-bes-wg-bounces(a)ogf.org [mailto:ogsa-bes-wg-bounces@ogf.org] > On Behalf Of Alexander Papaspyrou > Sent: Tuesday, June 03, 2008 3:15 PM > To: Open Grid Forum - OGSA-RSS-WG; Open Grid Forum - GSA-RG; ogsa-bes- > wg(a)ogf.org > Subject: [OGSA-BES-WG] Activity Delegation Interfaces and Protocols > > ----- Apologies for duplicates from the cross-post ----- > > > Folks, > > for quite a while now, the topic of "Activity Instance Schema" has > been floating around in JSDL. Basically, it boils down to what is used > to describe the concretization of a something that has been initially > defined by a JobDefinition template and submitted to a handling entity > (such as a BES container or a scheduler). > > One of the use cases that has been extensively discussed in the past > JSDL telecons was the delegation of a job (or, to be more general) of > an activity after its initial submission. That is, how to ensure a > submitted, but not running activity can be delegated in a simple, > coherent and safe way from one scheduler to another. > > We would like to kick off discussion during today's session on this > topic by starting with a very simple delegation protocol. Herefor, I'd > like to invite everyone to join us today at 3:45pm in Sarria. Although > we'll also discuss other issues (such as the applicability of SPARQL > to CSG), there will be plenty of time to chew on this new thing. > > Looking forward to a vivid discussion, albeit a very last-minute > notice. > > Cheers, > Alexander > > -- > Dipl.-Inform. Alexander Papaspyrou > http://ds.e-technik.uni-dortmund.de/~alexp > > Robotics Research Institute phone : +49(231)755-5058 > Information Technology Section fax : +49(231)755-3251 > Dortmund University of Technology, Germany > -- > ogsa-bes-wg mailing list > ogsa-bes-wg(a)ogf.org > http://www.ogf.org/mailman/listinfo/ogsa-bes-wg

1 0

Re: [ogsa-hpcp-wg] [ogsa-wg] Grid Command Line interfaces
by Hugo Mills 04 Jun '08

04 Jun '08

On Wed, Jun 04, 2008 at 02:48:25PM +0100, Omer F. Rana wrote: > Andre: I thought the aim of SAGA was to provide a programmatic > API rather than a command line tools interface? If you're a shell programmer (as I tend to be these days), the command-line tools *are* the API binding for the language... Hugo. -- Hugo Mills <hugo(a)soton.ac.uk | hugo(a)omii.ac.uk> OMII-UK, Southampton, SO18 1NP, UK

1 0

Re: [ogsa-hpcp-wg] [ogsa-wg] Grid Command Line interfaces
by Steven Newhouse 04 Jun '08

04 Jun '08

> Doesn't DRMAA already do this? > DRMAA and SAGA both only define programmatic APIs at present. Steven

1 0

Re: [ogsa-hpcp-wg] [ogsa-wg] Grid Command Line interfaces
by Steven Newhouse 04 Jun '08

04 Jun '08

It may make sense to define common tools for job: Submit Status Terminate I'm not sure what broader interest we would have to do generic SAGA commands. Steven > -----Original Message----- > From: Andre Merzky [mailto:andre@merzky.net] > Sent: Wednesday, June 04, 2008 2:48 PM > To: Steven Newhouse > Cc: Andre Merzky; ogsa-wg(a)ogf.org; ogsa-hpcp-wg(a)ogf.org > Subject: Re: [ogsa-wg] Grid Command Line interfaces > > Quoting [Steven Newhouse] (Jun 04 2008): > > > > > is a SAGA command line binding something you would > > > conider worth pursuing? We actually started to do > > > something like that, in a pet project... > > > > Do you mean the ability to implement any defined command > > line interface using the SAGA APIs? (i.e. internal to the > > command) Or To define a set of command line tools to cover > > elements of the SAGA API? > > The latter. For example, for the SAGA call > > class saga::filesystem::file > { > void copy (saga::url src, saga::url tgt, it flags); > } > > define the command line tool > > saga_file_copy [flags] <src> <tgt> > > flags: > session related flags > -s|--session <s> run command in session s > -c|--context <c> use context c > > operational flags > -a|--async=<Sync|Async|Task> > use async mode Sync, Async or Task > default is Sync > call specific flags > -r|--recursive copy recursively > -o|--overwrite overwrite target if exists > ... > > So, the command line tools would basically reflect what we > define in the SAGA API spec, with a set of flags which are > consistent for all command line tools such defined. > > > A session could look like: > > # saga_create_context --name=my_context --type=UserPass --user=anon > <prompts for password> > > # saga_create_session --name my_session --add_context=my_context > > # /bin/date | saga_file_cat --session=my_session --write > gsiftp://localhost/tmp/in.dat > > # saga_file_copy --session=my_session gsiftp://localhost/tmp/in.dat > gsiftp://remotehost/tmp/out.dat > > # saga_file_cat --session=my_session gsiftp://remotehost/tmp/out.dat > Wed Jun 4 14:43:27 CEST 2008 > > > or, with some default assumptions of course (default session > and context): > > > # /bin/date | saga_file_cat --write gsiftp://localhost/tmp/in.dat > # saga_file_copy gsiftp://localhost/tmp/in.dat > gsiftp://remotehost/tmp/out.dat > # saga_file_cat gsiftp://remotehost/tmp/out.dat > Wed Jun 4 14:43:27 CEST 2008 > > > Best, Andre. > > PS.: As for option (a) of yours: yes, that would be trivial to > implement in SAGA :-) Well, at least it would be easy (one > needs to add some magick for state management, to keep track > of async ops and security credentials between separate calls > to different tools. > > > Steven > > > > -- > Nothing is ever easy.

1 0

Feedback about HPCBP and BES
by Steven Newhouse 04 Jun '08

04 Jun '08

All, These are my combined notes from the HPCP-WG (which discussed current and future extensions) and the GIN session that was providing feedback on BES/HPCBP/JSDL specifications. Hopefully, we can use this to drive future BES and HPCP WG activities. Steven In no particular order: * File Staging Extension: Should support EPRs in the Source & Target elements in addition to just URIs. This will enable data transfers from end points (such as ByteIO, DMI, etc) * File Staging Extension: Current model assumes that the HPCBP service pulls data into the container. A use case is to get something back to the client so that they can upload/download files into the job workspace. * GLUE Profiling: Now that GLUE is entering public comment we should look at how we can use this within BES to either replace the current BES information model or define a port type to provide GLUE information. * Attribute Support: How information relating to a user's attributes can be provided with the job request... for instance SAML attributes document for UNICORE job submission through BES. * State Change Request: If the BES state model is extended to support (say) suspend and resume how does the user initiate this state change? It was observed that the BES-Activity port exists in BES but is not used. This could provide an opportunity to use this port type to provide: * Operations to upload/download files into an activity specific file space * Provide a state change request to the activity (i.e. suspend) as part of an extension to the BES state model * Renew/provide a credential for access to external resources from the BES container

1 0

Re: [ogsa-hpcp-wg] [ogsa-wg] Grid Command Line interfaces
by Steven Newhouse 04 Jun '08

04 Jun '08

> is a SAGA command line binding something you would conider > worth pursuing? We actually started to do something like > that, in a pet project... Do you mean the ability to implement any defined command line interface using the SAGA APIs? (i.e. internal to the command) Or To define a set of command line tools to cover elements of the SAGA API? Steven

1 0

HPCBP-WG Session Notes
by Steven Newhouse 04 Jun '08

04 Jun '08

HPCP Extensions Session Presenter: Steven Newhouse Note Taker: Stephen McGough Where are the specifications for public comment? - GridForge - But GridForge is broken - This is known problem - Will there be extra time for this? - Clock only starts when they appear on GridForge, also there's a 60 day period. Other extensions? - SAML extension (but person to champion this is not present) - Extend State model - what are the rules for doing this? (hold / suspend job) - BES defines how to extend state model - But how do I extend the interface provided for the EPR to change the job state - Could just define a new port type which will not break current compatibility - Can inspect a BES service to define what extensions are available - Problem: which state changes are allowed: which ones are allowed? - same as previous question? - return to pending state? - Not currently allowed through BES - State diagram is too restrictive - If you allow state transition which is not in BES then you're not BES compliant - What is your use case? - If your data transfer fails go back from failed to staging out state later for retry - Would it be better to stay in running and sub-state that? - JSDL extensions: - Access control - Notification of activity - Is this Activity schema work? - Terminal states - how long does job stay in this state. Extension to JSDL - Write this up as an extension and submit it - Where to bring this up? BES/JSDL/HPCP - Here for now and will figure out where it should end up

1 0

HPCP WG at OGF23
by Steven Newhouse 04 Jun '08

04 Jun '08

All, Attached at the slides from today's session. I'll be sending out some notes later today. Steven

1 0