- ogsa-d-wg - lists.ogf.org

FW: [ogsa-wg] Initial draft of HPC profile working group charter
by Dave Berry 05 May '06

05 May '06

Marvin's message bounced. _____ From: Marvin Theimer [mailto:theimer@microsoft.com] Sent: 05 May 2006 16:15 To: Dave Berry; ogsa-wg(a)ggf.org; dmis-bof(a)ggf.org; ogsa-d-wg(a)gridforum.org Subject: RE: [ogsa-wg] Initial draft of HPC profile working group charter Hi; You are right that most of the discussion so far has been about job submission and not about the related need to get data from client to execution subsystem(s) and back again. But since the goal of the group is to define a profile covering the end-to-end vertical use cases, I know that we will need to tackle the topic of data staging at some point. As with JSDL and BES's relation to the job submission parts of the work, I'm hoping we will be able to substantially leverage other existing work, such as that of the data DMIS groups, as appropriate. I'll make that more explicit in the next revision of the draft charter. Thanks for pointing this matter out, Marvin. _____ From: Dave Berry [mailto:daveb@nesc.ac.uk] Sent: Friday, May 05, 2006 7:14 AM To: Marvin Theimer; ogsa-wg(a)ggf.org; dmis-bof(a)ggf.org; ogsa-d-wg(a)gridforum.org Subject: RE: [ogsa-wg] Initial draft of HPC profile working group charter Hi, All the discussion I've seen about this profile so far has focussed on specifying and submitting jobs, but the proposed charter says that this is only one of two areas of interest; the other being data staging. The immediate question is what link will this profile work have to the Data Movement Interface Specification group? Will the profile refer to the DMIS spec in the same way that it will refer to JSDL and BES? (Of course, JSDL also covers data staging to some extent). Work is also continuing on the data aspects of the OGSA architecture, in the OGSA Data WG. I would encourage the HPC profile group to engage with the OGSA Data group and the DMIS group concerning how to fit data staging into the overall architecture. Unfortunately I won't be in Tokyo next week but it would be useful if members of these groups could get together there to agree a working relationship. Best wishes, Dave. _____ From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On Behalf Of Marvin Theimer Sent: 05 May 2006 01:48 To: ogsa-wg(a)ggf.org Subject: [ogsa-wg] Initial draft of HPC profile working group charter Hi; Enclosed is an initial draft of the OGSA HPC profile working group charter. This will be brought up for discussion at the HPC profile BoF at the upcoming GGF17 conferenece in Tokyo. In the mean time, please feel free to comment on the draft via email to this mailing list. Thanks, Marvin.

1 0

RE: [ogsa-wg] Initial draft of HPC profile working group charter
by Dave Berry 05 May '06

05 May '06

Hi, All the discussion I've seen about this profile so far has focussed on specifying and submitting jobs, but the proposed charter says that this is only one of two areas of interest; the other being data staging. The immediate question is what link will this profile work have to the Data Movement Interface Specification group? Will the profile refer to the DMIS spec in the same way that it will refer to JSDL and BES? (Of course, JSDL also covers data staging to some extent). Work is also continuing on the data aspects of the OGSA architecture, in the OGSA Data WG. I would encourage the HPC profile group to engage with the OGSA Data group and the DMIS group concerning how to fit data staging into the overall architecture. Unfortunately I won't be in Tokyo next week but it would be useful if members of these groups could get together there to agree a working relationship. Best wishes, Dave. _____ From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On Behalf Of Marvin Theimer Sent: 05 May 2006 01:48 To: ogsa-wg(a)ggf.org Subject: [ogsa-wg] Initial draft of HPC profile working group charter Hi; Enclosed is an initial draft of the OGSA HPC profile working group charter. This will be brought up for discussion at the HPC profile BoF at the upcoming GGF17 conferenece in Tokyo. In the mean time, please feel free to comment on the draft via email to this mailing list. Thanks, Marvin.

1 0

Next meeting May 17th
by Dave Berry 03 May '06

03 May '06

All, Please note that there will be no OGSA Data meeting this week or next week. The next meeting is scheduled for May 17th. By that date we should have completed several current tasks, including: - The latest version of the architecture document, to include all current revisions, to be on GridForge. - The scenarios document to reference interfaces from the architecture document. - Draft scenarios for Grid file system, provenance, and metadata catalogs. I attach the minutes of last week's meeting, which you can also find at http://tinyurl.com/ecfan. Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

This week's telcon will start 30 minutes earlier
by Dave Berry 24 Apr '06

24 Apr '06

The next teleconference of the OGSA Data Architecture working group will be held on Wednesday, April 16th, 2006 at the EARLIER time of 7:30am Pacific / 10:30am Eastern / 3:30pm UK / 4:30pm Europe. Here is the teleconference information: UK dialin: 0845 245 0224 US dialin: 1 866 838 0046 Intl dialin: +44 1452 542309 Passcode: 275359 Draft agenda for the April 26th teleconference: 1) Early discussion (5 minutes) Note taker assignment Roll call Minutes Approval (http://tinyurl.com/rcmab) Agenda bashing 2) Action report (5 minutes) - Dave to ask Allen about starting the call earlier next week [Done!] - Dave to draft slides for the OGSA webcast and send them to the list [Done] - Chris to volunteer to be on the panel for the OGSA webcast [Done] - Dave to chase Neil re ByteIO [Done] - Allen to talk to Tom Maguire about EMC's view of SRM. - Chris to draft GFS scenario - Stephen & Dave to review scenarios doc from a storage perspective. - Stephen & Dave to add interfaces to the scenarios doc. - Dave to draft metadata catalog scenario & services (cf. SRB) - Stephen to add a paragraph to the architecture document about reasons for naming, if still needed after Allen's revisions. - Ann to check the Access section for suitability for file replication. - Neil to update the ByteIO subsection; then Allen to put latest version online. - Section authors review security aspects of their sections. - Allen/Dave to contact David Martin & Hiro Kishimoto to get reviewers & expert participation. 3) Progress report and issues arising (30 minutes) - Storage in the architecture document - Replication management - Adding interfaces in scenarios - Metadata operations - Transfer vs. Access 4) Planning for GGF17 (5 minutes) Do we go ahead with the planned session? 6) Task Planning (10 minutes) We need to schedule work and discussion of the following: 1. Review of latest data arch doc - Need ByteIO input, then Allen to put latest version online. 2. Interfaces for arch doc - can we arrange a F2F? 3. Interfaces for scenarios doc - Stephen & Dave. 4. Review architecture from storage perspective (including security) - Allen 4b. Review scenarios from storage perspective - Stephen & Dave. 5. New GFS scenario - Chris. 6. New Provenance scenario - Stephen 7. Metadata catalog scenario & services (cf. SRB) - Dave? 7) Wrap up (5 minutes) Topics for future meetings Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

Slides for OGSA webcast
by Dave Berry 21 Apr '06

21 Apr '06

I've added a couple of slides about the data services to the presentation for next week's OGSA webcast. You can see the set at http://tinyurl.com/gu4d3. If you have any comments, plase let me know before Monday. <http://tinyurl.com/gu4d3> . Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

Agenda for tomorrow's telcon
by Dave Berry 18 Apr '06

18 Apr '06

The next teleconference of the OGSA Data Architecture working group will be held on Wednesday, April 19th, 2006 at the usual time of 8:00am Pacific / 11:00am Eastern / 4:00pm UK / 5:00pm Europe. Here is the teleconference information: UK dialin: 0845 245 0224 US dialin: 1 866 838 0046 Intl dialin: +44 1452 542309 Passcode: 275359 Draft agenda for the April 19th teleconference: 1) Early discussion (5 minutes) Note taker assignment Roll call Minutes Approval Agenda bashing 2) Action report (5 minutes) - Allen to review architecture document for more places where we need to mention storage. - Stephen & Dave to review scenarios doc from a storage perspective. - Stephen & Dave to add interfaces to the scenarios doc. - Stephen to draft GFS scenario. - Stephen to draft provenance scenario. - Dave to draft metadata catalog scenario & services (cf. SRB) - Allen to talk to Tom Maguire about EMC's view of SRM. - Dave to forward old message about interface description. - Stephen to add a paragraph to the architecture document about reasons for naming, if still needed after Allen's revisions. - Ann to check the Access section for suitability for file replication. - Neil to update the ByteIO subsection; then Allen to put latest version online. - Section authors review security aspects of their sections. - Allen/Dave to contact David Martin & Hiro Kishimoto to get reviewers & expert participation. 3) Progress report and issues arising (15 minutes) - Storage in the architecture document - Adding interfaces in scenarios - Provenance scenario - Operations to add data elements to replication systems - Metadata operations 4) Planning for GGF17 (10 minutes) Allen has booked a slot for the Friday morning. Chris is hoping to be there, as will Neil. 5) Slides for OGSA webcast (10 minutes) I've been asked to provide data slides for next week's OGSA webcast. The current draft for the webcast slides can be found at http://tinyurl.com/o5v8u. 6) Planning (10 minutes) We need to schedule work and discussion of the following: 1. Review of latest data arch doc - Need ByteIO input, then Allen to put latest version online. 2. Interfaces for arch doc - can we arrange a F2F? 3. Interfaces for scenarios doc - Stephen & Dave. 4a. Review architecture from storage perspective (including security) - Allen 4b. Review scenarios from storage perspective - Stephen & Dave. 5. New GFS scenario - Stephen. 6. New Provenance scenario - Stephen 8. Discussion of DAIS (and ByteIO?) specs - due Apr 10th. 9. Metadata catalog scenario & services (cf. SRB) - Dave? 7) Wrap up (5 minutes) Topics for future meetings Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

Storage in the Data Architecture
by Allen Luniewski 11 Apr '06

11 Apr '06

On last week's teleconference I took an action to look over the architecture document to see if there were places where we needed to consider storage more than the document currently does. I have just completed a quick scan of the document and we appear to be in good shape. I only detect the following places where we may need to say more: 1. In Naming (section 3.1) we may want to augment the table with some storage specific entities that need to be named. 2. Section 9 (Caching) needs to discuss where the data being cached is stored while on the caching server. It is not clear to me if this should be a file (system) or something from storage. If the cache is perceived to be short lived then one might argue that this is just temporary storage and nothing need be said to the cache service to tell it where to put such transient data. However, if the cache is perceived to be long lived, or even if the amount of cached data will be large, the owner of the cache may have a desire to control where the data goes. I am inclined to believe that the latter is the more likely case. So, I think that there is a clear hole here in the description. 3. In the Creation of Federations (section 11.1), we need, as for caching, a means to tell the federation service where it should keep any data that it generates in support of the federation. Since this data could be long lived, we will inevitably get into storage and storage concepts such as lifetime management. 4. Finally, the caching item above raises the whole question of temporary data used by any service in the environment. One could argue that the users of the grid system have a vested interest in understanding and controlling where this temporary data lives, especially if the data is either large or relatively long lived. If you go down this path, it is far from clear to me how one meets this need without either adding undue complexity to every factory(-like) operation or mandating a management interface to every service to allow control of the location of the service's temporary data. Personally, I have a slight tendency to not attempt to mandate something in this area. I am inclined to make a note in the architecture that there is an issue here that every service must address but that it is up to the service designer (?) to make the call about how to deal with the issue. Normal market forces would then determine if that decision was the proper one. I don't claim that these are all of the places but it is, hopefully, a good start. Allen

1 0

OGSA standards for naming, replication & registries
by Dave Berry 11 Apr '06

11 Apr '06

All, I've just been in a discussion reviewing the OGSA Data Scenarios and some questions arose regarding the OGSA naming services. In particular, we were looking at the replication section of the draft OGSA Data Architecture, which includes the text given below. My first question is: will the first three operations listed below be provided by the OGSA naming activity? To put this very succinctly, will a replica catalog be simply an instance of a resolution service defined by the OGSA-naming WG? My second question is: will the OGSA-naming WG define resolution services that associated arbitrary metadata with names? The reason we ask is that the discussion about RNS/WS-Directory seems to require that a directory service can associate attributes with entries in the directory. If this is so, will the same requirement apply to resolution services for abstract names -> addresses? If these resolution services are managing attributes, do they allow queries over these attributes? If so, what will be the difference between a resolution service and a general registry service or metadata catalog? Here is the text referred to: ------------- The interface to a replica catalogue or replica location service (ReplicaCatalog) may include the following operations: * Operations to create or update mappings between globally unique names and physical addresses for data items * Query operations to discover the physical addresses of data items associated with a globally unique name * Query operations to discover the globally unique name(s) associated with a physical address of a data item * Operations to associate attributes with globally unique names or physical addresses of data items * Query operations to discover globally unique names or physical addresses of data items with specified attributes ---------- Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

Telcon minutes, 5th April 2006
by Dave Berry 05 Apr '06

05 Apr '06

Attached and at http://tinyurl.com/lpp6v. Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA Tel: +44 131 651 4039

1 0

FW: [ogsa-wg] HP/IBM/Intel/MSFT roadmap
by Dave Berry 05 Apr '06

05 Apr '06

This is the white paper that was mentioned on today's telcon. Dave. -----Original Message----- From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On Behalf Of Hiro Kishimoto Sent: 15 March 2006 21:46 To: OGSA WG Subject: [ogsa-wg] HP/IBM/Intel/MSFT roadmap Hi all, We need to know what they say; "Web services management standards convergence announcement" http://devresource.hp.com/drc/specifications/wsm/index.jsp -- Hiro Kishimoto

1 0