July 2006 - ogsa-d-wg - lists.ogf.org

Minutes; Next telcon Aug 16th
by Dave Berry 27 Jul '06

27 Jul '06

The minutes of this week's telcon are attached and on Gridforge at http://tinyurl.com/rnskb. The next telcon will be on August 16th. Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA Tel: +44 131 651 4039

1 0

Agenda for July 26th
by Dave Berry 24 Jul '06

24 Jul '06

The next teleconference of the OGSA Data Architecture working group will be on Wednesday 26th July at the usual time of 8:00am Pacific / 11:00am Eastern / 4:00pm UK / 5:00pm Europe. We will discuss the comments we received from the OGSA F2F and review the scenarios document. Here is the teleconference information: UK dialin: 0845 245 0224 US dialin: 1 866 838 0046 Intl dialin: +44 1452 542309 Passcode: 275359 Draft agenda for the July 26th teleconference: 1) Early discussion (5 minutes) Note taker assignment Roll call Minutes Approval (http://tinyurl.com/q33dc) Agenda bashing 2) Action report (5 minutes) - Dave to mail OGSA WG list to look at our docs for F2F [Done] - Dave to prepare some draft slides for OGSA WG F2F and send to list for review [Done]. - Mario to talk to Neil about OGSA support for factory ops. - ?? Create or reuse scenario slides for main OGSA slide set. - Chris to send out second draft of GFS scenario. - All WG members to review latest version of architecture document. - Stephen to carry on chasing Luc about the provenance scenario. - Stephen & Dave to review scenarios doc from a storage perspective. - Dave to draft metadata catalog scenario & services (cf. SRB) - Ann to check the Access section for suitability for file replication. 3) Feedback from OGSA F2F (20 minutes) Discussion of the comments we received from our presentation to the OGSA F2F and how we should respond. 4) Review of scenarios document (20 minutes) In particular the more recent scenarios. 5) Wrap up (5 minutes) Topics for future meetings Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

Notes from yesterday's F2F data session
by Dave Berry 18 Jul '06

18 Jul '06

These are notes I made from the comments made during yesterday's presentation. I stress that we encourage people to review the documents and give us feedback. I also think it is urgent that we produce a common subset of scenarios combining EMS & Data services. 1. All references to "profiles" in the presentation should have been to "further normative specifications" - which may include profiles where appropriate. The primary point is that we need to encourage key specs and to incorporate them into the OGSA Roadmap. 2. The slides did not clarify that the "Architecture" takes a toolbox approach, rather than a predefined relationship between the services. 3. The diagram showing the relationships between interfaces, services & resources should allow multiple resources per service. It should use the same style as the later diagrams (i.e. showing services & resources separately rather than as concentric layers). 3a. The Architecture and Services documents should use the same diagram styles - probably the one used in the OGSA 1.5 document. 4. The "Basic Structure" diagram should add a "Storage Access" interface (or perhaps just a sink/source interface for data transfer). Also, "Other Data Services" should be renamed to "Data Management" - there is no intention to distinguish these services per se from the other "Data Management" services in the diagram. Note: This diagram also appears in the OGSA 1.5 document. 5. The "Composite Entities" diagram should include a "federation management interface" and a "cache management interface". I got the impression that the diagram was not successful in showing that the federation and cache services exposed the same interfaces as their "supplier" services; it might be better to repeat these interfaces on the federation and cache services rather than using the dotted line. Note: This diagram also appears in the OGSA 1.5 document. 6. An alternative implementation of the Data Pipelining scenario could use StreamableByteIO between the Visualisation service and the stored animations. It might be worth giving this example to show the different ways that the services can be used to implement a solution. 7. The "Bringing data online" scenario should clarify why data transfer is used instead of data access. Perhaps the phrase "read files" should be changed to "transfer files". 8. In the "Data Staging" scenario, the "Parameter Space Exploration Service" should probably be replaced by a script engine or workflow enactment engine. 9. There was a discussion about the meaning of the term "policy". It was suggested that the sample "policies" listed on the slides were better described as sample parameters for policies. The Data Architecture document attempts to use the term policy consistently and we'd be happy to take guidance on improving this presentation. 10. The slide about "Common Properties" was floating an idea. The comments from the group were that if these properties were to be generalised, then the ParentDataResource should be an EPR instead of a URI and should allow multiple parents. Also, the "Sensitivity" properties were basically an expression of the consistency model, which in the general case could be more complex (as indeed we have noted elsewhere in the Data Architecture document). 11. Some of the functionality of the WS-DAI (optional) CoreResourceList interface will be provided by the WS-Naming facility - specifically the the functionality of the Resolve operation. RNS is not the appropriate reference here because it deals in strings instead of URIs. The other operation provided by the CoreResourceList interface, GetDataResourceList, does not belong with WS-Naming (and perhaps would better fit with the possibly-generic properties on the previous slide). 12. DFDL was mentioned in the discussion about "OGSA Data gaps", as a possible mechanism for describing file "schemas". In fact the slide was simply referring to POSIX-style "stat" information but we do see DFDL as a potential description mechanism. The current section on "Data Description" in the Data Architecture document probably needs some fleshing out to consider possible languages in more detail. 13. There was some discussion of what is meant by a "data service". The terminology used in the Data Architecture document is that "Data Services" are the subset of the OGSA services that deal with data and hence include services for data transfer, data access, data replication, storage management, etc. There is an alternative body of opinion that data access services should be terminologically privileged, i.e. that the term "data service" should mean a data access service. 14. Regarding the Information Model, Jay mentioned that a key piece of information is whether a data resource is logically "near" another resource. This is of use to schedulers, workflow enactments and applications trying to choose which data resource to use, among other examples. It isn't clear how to incorporate this information into an information model (although this should not stop us developing other aspects of that model). I encourage everyone to review the Data Architecture and Data Services documents. These documents will be released under the OGSA brand and we are at a good stage to take on board any comments. Best wishes, Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

Next telcon July 26th
by Dave Berry 12 Jul '06

12 Jul '06

The next telcon of the OGSA Data WG will be in two weeks time. I attach the minutes of today's meeting, which are also on GridForge: http://tinyurl.com/q33dc. Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

Today's Agenda
by Dave Berry 12 Jul '06

12 Jul '06

The next teleconference of the OGSA Data Architecture working group will be today at the usual time of 8:00am Pacific / 11:00am Eastern / 4:00pm UK / 5:00pm Europe. We will prepare for the OGSA F2F next week and review the latest draft of the architecture document, which can be found at https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.ogsa- d-wg/docman.root.working_drafts/doc13621/2 Here is the teleconference information: UK dialin: 0845 245 0224 US dialin: 1 866 838 0046 Intl dialin: +44 1452 542309 Passcode: 275359 Draft agenda for the July 12th teleconference: 1) Early discussion (5 minutes) Note taker assignment Roll call Minutes Approval (http://tinyurl.com/nbekd) Agenda bashing 2) Action report (5 minutes) - Chris to send out second draft of GFS scenario when agreement reached with GFS WG. - Dave to revise Data Access section. [Done] - Dave to ask Hiro and David Martin to review the document, advising them that parts are still being worked on. [Done] - All WG members to review latest version of architecture document. - Stephen to ask Luc's provenance team why they have chosen a separate provenance store. [Done] - Stephen to carry on chasing Luc about the provenance scenario. - Stephen & Dave to review scenarios doc from a storage perspective. - Dave to draft metadata catalog scenario & services (cf. SRB) - Ann to check the Access section for suitability for file replication. - Section authors review security aspects of their sections. 3) Planning for F2F (20 minutes) Content for the Data session - e.g. interaction with execution stack, naming, generic properties 4) Review of architecture document (25 minutes) Changes to data access section, naming, file description 5) Wrap up (5 minutes) Topics for future meetings - July 5: No call? - July 12: ? Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA +44 131 651 4039

1 0

FW: Data Provenance Scenario
by Stephen Davey 11 Jul '06

11 Jul '06

Hi, I have an action: "Stephen to ask Luc's provenance team why they have chosen a separate provenance store." Luc's answer is below. Basically he says that it is a logically separate provenance store, but that it isn't necessarily physically separate. I'll not be able to make the telcon again this week - apologies. Cheers, Stephen. -------------------------- Stephen Davey, National e-Science Centre, Edinburgh, UK. -----Original Message----- From: Luc Moreau [mailto:L.Moreau@ecs.soton.ac.uk] Sent: 10 July 2006 12:38 To: Stephen Davey Subject: Re: Data Provenance Scenario Stephen, Sorry I forgot to reply to your message. In our deployment of the OTM, the provenance store is physically separated from the data. However, agreed with you, in other applicants you could see the provenance store as an extra port on your data storage, with data and process documentation all stored in a common physical storage. Luc Professor Luc Moreau Electronics and Computer Science tel: +44 23 8059 4487 University of Southampton fax: +44 23 8059 2865 Southampton SO17 1BJ email: l.moreau(a)ecs.soton.ac.uk United Kingdom http://www.ecs.soton.ac.uk/~lavm

1 0

RE: [ogsa-d-wg] New version of architecture document
by Dave Berry 03 Jul '06

03 Jul '06

It occurs to me that some of the issues covered by the WS-DAI properties are perhaps worthy of more general discussion in the architecture: 1. Derived data resources. WS-DAI has properties that show whether updates to a child are reflected in the parent and vice-versa. I think we mention derived data in passing; we could say more about it and treat it more consistently. 2. Concurrent access and local transaction support. ByteIO deliberately says nothing about concurrent access; we should at least mention this. Possible we should consider these issues across all data services? The discussion of distributed transactions in 3.9 should mention the WS-DAI support for local transactions. 3. The WS-DAI CoreResource interface handles mapping from abstract names to addresses. We need to understand how this relates to the OGSA naming scheme. 4. WS-DAI defines properties for the name of a resource and a human-readable description of the resource. These seem reasonably generic properties - we (or even the OGSA WG) might want to consider whether they are more widely useful. (Or perhaps we should ask the DAIS WG to explain why they are necessary in WS-DAI if they are not needed anywhere else). Finally, I note that section 11.3 mentions enhanced access operations that would ease the federation of a data resource. This is worth a forward mention from the data access section itself. Dave. -----Original Message----- From: owner-ogsa-d-wg(a)ggf.org [mailto:owner-ogsa-d-wg@ggf.org] On Behalf Of Dave Berry Sent: 29 June 2006 23:34 To: ogsa-d-wg(a)gridforum.org Subject: [ogsa-d-wg] New version of architecture document I've uploaded a new version of the architecture document - https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.ogsa- d-wg/docman.root.working_drafts/doc13621/2. Changes: - I've moved some generic description out of the data access section into the introductory sections. - I've tidied the data access section. At least, I think it's tidier; please could the authors take a look and tell me what I've messed up? - I've made the discussion of policies and agreements more consistent throughout the document. This includes moving a list of example policies from the federation section into section 2. - I've tidied the glossary, removing unused items and ensuring that certain other terms are used consistently in the text. Note: where I've copied chunks of text from one place to another, I switched off change tracking because such large deletions and additions don't tell you much. Instead, I've highlighted the text in yellow where it now appears. Still to do (at least): - Add a discussion of interfaces to the data caching section. - Ditto for the replication section. - Revise the metadata catalog section. - Tidy the appendix list of standards. I'm about to go on holiday for a week, so these tasks won't get done for a while. Dave Berry Deputy Director, Research & E-infrastructure Development National e-Science Centre, 15 South College Street Edinburgh, EH8 9AA Tel: +44 131 651 4039

2 1