November 2005 - ogsa-d-wg

RE: [ogsa-d-wg] Telcon Agenda for 30th November: Scenarios document
by Peter Kunszt 30 Nov '05

30 Nov '05

hi dave sorry i won't be able to atted today after all - i have to be at home by 5pm. cheers peter > -----Original Message----- > From: owner-ogsa-d-wg(a)ggf.org > [mailto:owner-ogsa-d-wg@ggf.org] On Behalf Of Dave Berry > Sent: 27 November 2005 16:58 > To: ogsa-d-wg(a)ggf.org > Subject: [ogsa-d-wg] Telcon Agenda for 30th November: > Scenarios document > > Dear All, > > The next teleconference of the OGSA Data Architecture working > group will be held on Wednesday, November 30th, 2005 at the > usual time of 8:00am Pacific / 11:00am Eastern / 4:00pm UK / > 5:00pm Europe. > > In this telcon we will review the Scenarios document. Please > read this document and bring issues to the call. > > As a reminder, here is the teleconference information: > > UK dialin: 0845 245 0224 > US dialin: 1 866 838 0046 > Intl dialin: +44 1452 542309 > > Passcode: 369786 > > > Agenda for the November 30th teleconference: > > 1) Early discussion > Note taker assignment > Roll call > Minutes Approval (9th November: http://tinyurl.com/cktqe, 16th > November: http://tinyurl.com/9hpnr) > Agenda bashing > > 2) Action report > > - All to discuss plans for GGF16 OGSA Data meetings & F2F. > - Dave to find out about status of WS-Agreement post GGF15. > - Peter & Stephen to add first draft of Storage Management scenario. > > - Ann to update replication section. > - Everyone to comment on draft RNS specification. > - Ann to check the Access section for suitability for file > replication. > - Mario to update the Access section and make text available > for comment. > - Section authors to add security notes to their sections. > - Dave to revamp section 3 of the architecture document. > - Everyone to talk with your contacts to get more > participation in this WG. > - Allen/Dave to contact David Martin & Hiro Kishimoto to get > reviewers & expert participation (after internal reviews). > > > 3) Progess & planning update (5 minutes) > > Update on document writing, plans for telcons and F2F. > > 4) Review of Scenarios document (40 minutes) > > Issues that we should consider include, but are not limited to: > > - Matching scenarios against service descriptions > - Completeness > - Level of detail > - Security implications > - Consistency between scenarios > - Identify where work is needed on metadata or on naming conventions > > 5) Wrap up (5 minutes) > > Next meetings: > > Dec 7th: No meeting? (Allen on holiday) > Dec 14th: Naming, RNS & GFS architecture > Dec 21st: Full Data Architecture doc review > Dec 28th: No meeting. Christmas hols. > > > Dave Berry > Research Manager, National e-Science Centre > 15 South College St., Edinburgh > Tel: +44 131 6514039 > >

1 0

Data scenarios v0.7
by Stephen Davey 29 Nov '05

29 Nov '05

Hi everyone, Attached is my latest scenarios document - version 0.7 which includes Allen's comments on v0.6. It also has a Storage scenario from Peter and I have made a start at a Personal Data Service scenario, and a bit of a Data Discovery scenario. A copy is also on GridForge - https://forge.gridforum.org/docman2/ViewCategory.php?group_id=150&catego ry_id=1033 . I wonder if we should try to ensure that there is a least one scenario that is focussed on each of the main sections in the Data Architecture document? The architecture sections and related scenarios are currently as follows: Data Description - ??? Data Transfer - data transfer scenario, data staging scenario. Data Access - personal data service scenario?? Storage Resource Management - storage scenario. Cache Services - ??? Data Replication - data replication scenario. Data Federation - data integration scenario, personal data service scenario. Metadata Catalogue & Registries - data discovery scenario? Personal Data Service - personal data service scenario. Transformational Services - ??? Data Streams - ??? Also mentioned in the Data Architecture document are: Naming - personal data service scenario?? Metadata - data discovery scenario?? Security Transactions Sessions Execution Management Services (EMS) Hope people get time to read it before the telcon tomorrow, cheers, Stephen. ---------------------------------------------------------------------- Stephen Davey, NextGrid Software Architect, National e-Science Centre, 15 South College St., Edinburgh, EH8 9AA, UK Tel: +44 131 6 509820

1 0

Re: [ogsa-d-wg] Telcon Agenda, 16th November - Storage management section review
by Peter Kunszt 27 Nov '05

27 Nov '05

hi all please find attached a new storage chapter version (chapter 7). i've tried to work in the comments by allen and stephen. some comments/points to discuss further: - comment by allen on tying this chapter 'back to the QoS' discussion earlier in the doc - i could not find the corresponding place. in the versions i was looking at QoS was only mentioned marginally. which passage did you refer to? - i removed the reference to encryption. it is true that encryption algorithms, key management etc can be considered an external service to the data storage resource. it is up to the client to put encrypted data into the storage space, not necessarily up to the storage to provide the facility (which is a nice way to dodge this difficult topic :-) i hope you agree. - I decided to also remove the 'space partition' property. it was too complicated and the same can be achieved by a set of spaces provided by the storage instead. if the user requests a space which is too large, the storage resource may simply negotiate a smaller one, and the user may ask for more of those. this pushes the management of a set of spaces to the client which is probably the right thing to do (SOA). comments? - about security semantics of file space types (ie ones having a file namespace): allen asks whether this should be defined here or by the security group. i would argue that it should be defined here as they are tied to storage security semantics. the ogsa security groups deal with generic security aspects and semantics for authentication authorization and accounting. but it is up to each specific service to provide the corresponding semantics using these tokens. for example in a file space the principals provided through ws-security (authz) can be used to set and check the ACLs. so in summary: the security semantics of data services are up to the data services to define. this is of course only my opinion based on my current understanding and i'm happy to be convinced otherwise. - costing of storage. to define the cost of a resource sounds like a generic problem which i believe should be discussed in a dedicated group of the OGSA design. is this already done? i've simply put a forward reference to it. - i have (re-)added the chapters on security and transfer. the security discussion is new and has the word discussion in its title. it contains some additional aspects to consider that were not in the chapter yet. the transfer chapter got dropped from the last version for no good reason so i put it back.. if we decided to drop it for a reason, please remind me. sorry for the delay, peter ps. i'll try to do the use cases as well before the phone con, i got very good input from stephen already. -- ------ CERN, 1211 Geneva 23, Switzerland

1 0

Telcon Agenda for 30th November: Scenarios document
by Dave Berry 27 Nov '05

27 Nov '05

Dear All, The next teleconference of the OGSA Data Architecture working group will be held on Wednesday, November 30th, 2005 at the usual time of 8:00am Pacific / 11:00am Eastern / 4:00pm UK / 5:00pm Europe. In this telcon we will review the Scenarios document. Please read this document and bring issues to the call. As a reminder, here is the teleconference information: UK dialin: 0845 245 0224 US dialin: 1 866 838 0046 Intl dialin: +44 1452 542309 Passcode: 369786 Agenda for the November 30th teleconference: 1) Early discussion Note taker assignment Roll call Minutes Approval (9th November: http://tinyurl.com/cktqe, 16th November: http://tinyurl.com/9hpnr) Agenda bashing 2) Action report - All to discuss plans for GGF16 OGSA Data meetings & F2F. - Dave to find out about status of WS-Agreement post GGF15. - Peter & Stephen to add first draft of Storage Management scenario. - Ann to update replication section. - Everyone to comment on draft RNS specification. - Ann to check the Access section for suitability for file replication. - Mario to update the Access section and make text available for comment. - Section authors to add security notes to their sections. - Dave to revamp section 3 of the architecture document. - Everyone to talk with your contacts to get more participation in this WG. - Allen/Dave to contact David Martin & Hiro Kishimoto to get reviewers & expert participation (after internal reviews). 3) Progess & planning update (5 minutes) Update on document writing, plans for telcons and F2F. 4) Review of Scenarios document (40 minutes) Issues that we should consider include, but are not limited to: - Matching scenarios against service descriptions - Completeness - Level of detail - Security implications - Consistency between scenarios - Identify where work is needed on metadata or on naming conventions 5) Wrap up (5 minutes) Next meetings: Dec 7th: No meeting? (Allen on holiday) Dec 14th: Naming, RNS & GFS architecture Dec 21st: Full Data Architecture doc review Dec 28th: No meeting. Christmas hols. Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

Re: [ogsa-naming-wg] Data architecture requirements on naming
by Dave Berry 23 Nov '05

23 Nov '05

Dave's e-mail bounced. -----Original Message----- In-Reply-To: <5D8E46F5AA2D8A428667B8787DEF216CE1DA46(a)hendrix.nesc.ed.ac.uk> References: <5D8E46F5AA2D8A428667B8787DEF216CE1DA46(a)hendrix.nesc.ed.ac.uk> Mime-Version: 1.0 (Apple Message framework v623) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <dc887e7aaa502353086548c5a6cc3e70(a)UK.Fujitsu.com> Content-Transfer-Encoding: 7bit Cc: <ogsa-naming-wg(a)ggf.org>, "ogsa-wg" <ogsa-wg(a)gridforum.org>, <ogsa-d-wg(a)gridforum.org> From: David Snelling <David.Snelling(a)UK.Fujitsu.com> Subject: Re: [ogsa-naming-wg] Data architecture requirements on naming Date: Wed, 23 Nov 2005 08:41:36 +0000 To: "Dave Berry" <daveb(a)nesc.ac.uk> Dave, Good points and more issues for the Naming WG to chew on. My point of view: Of your options, I don't think we should run down the policy route. It would have to be an open ended tag that might have a couple of predefined policies, fastest, most secure, cheapest, ... But there will be others: "any one will do except from that useless repository at XXX". There is also the possibility that a single EPR can contain multiple resolutions. See Tom proposal for wsdl bindings. Also, recall that in OGSI we had the Locator as the return value from a resolver. It contained other names (GSHs) and well as a number of (GSRs). On 22 Nov 2005, at 10:29, Dave Berry wrote: > Dear All, > > Recent discussions on the OGSA data architecture have raised two use > cases that may affect the naming discussion. Both involve the mapping > of one "abstract" (for some meaning of "abstract") name to multiple > possible "concrete" names. A further step is then required to choose > the best "concrete" name, according to the policies required by the > client. We are wondering whether the WS-Name/EPR resolution services > can be modified to support these functionalities. > > Note: here, "abstract" names may or may not have the same meaning as > abstract names in the proposed WS-Names spec. This is an issue that > needs clarification. > > Case 1 is when we are dealing with replicated data. It is common to > have an "abstract" name that represents a given file, table or query > result that is stored someone in the system. This is then resolved to > one of multiple copies. The client typically wants the copy with the > best response time (which will be some function of latency and > bandwidth). > > Case 2 is when we are dealing with data stored at a certain location > and > the data may be accessed by several different protocols (e.g. GridFTP, > HTTP, ...). The client (or transfer service) wants to map the > "abstract" name to the fastest protocol that it can handle. > > In each case, we could take one of two approaches: > > (A) resolve the "abstract" name to a list of "concrete" names, and use > a > separate selection service to choose among them > > (B) pass the appropriate policy specification to the resolver service, > which will then return the appropriate "concrete" name. > > Note: (B) could be considered (and even implemented) as a composition > of > the resolver and selection services in (A). > > Would it make sense to specify either one of these functionalities as > part of the resolver services considered in WS-Naming? E.g. the > operations could return a list instead of a single EPR (option A) or > they could return a single EPR as now but take a policy argument > (option > B). > > Best wishes, > > Dave Berry > Research Manager, National e-Science Centre > 15 South College St., Edinburgh > Tel: +44 131 6514039 > > -- Take care: Dr. David Snelling < David . Snelling . UK . Fujitsu . com > Fujitsu Laboratories of Europe Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE +44-208-606-4649 (Office) +44-208-606-4539 (Fax) +44-7768-807526 (Mobile)

1 0

W3C Efficient XML Interchange WG
by Mario Antonioletti 23 Nov '05

23 Nov '05

Hi, This could be of interest to this WG: http://www.w3.org/XML/EXI The XML Binary Characterization Working Group has conducted work during one year to gather information about uses cases where the overhead of generating, parsing, transmitting, storing, or accessing XML-based data may be deemed too great for a particular application, characterizing the properties that XML provides as well as those that are required by the use cases, and establishing measurements to help judge whether XML 1.x and alternate encodings provide the required properties... More info at the link above ... I think that this should be of interest to this WG. Mario +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

1 0

RE: [ogsa-d-wg] Data architecture requirements on naming
by Peter Kunszt 22 Nov '05

22 Nov '05

hi dave in our experience, case (B) is not being used by clients. the reason is that the policies that are put in place to find the 'best' concrete replica are very application specific and are usually implemented by the applications themselves. what turned out to be much more useful was to present the list of replicas when asked for it with some measured qualifyers that the grid can provide, i.e. predicted access cost, latency, time to wait (queues) or whatever is available so that the application can resolve the 'best' replica based on any model/policy they may want to apply. it turned out that implementing a generic policy enforcer is very difficult and what you end up doing is to provide a python interpreter so that the applications can write their own policy scripts... hope this helps, cheers, peter > -----Original Message----- > From: owner-ogsa-d-wg(a)ggf.org > [mailto:owner-ogsa-d-wg@ggf.org] On Behalf Of Dave Berry > Sent: 22 November 2005 11:30 > To: ogsa-naming-wg(a)ggf.org; ogsa-wg; ogsa-d-wg(a)gridforum.org > Subject: [ogsa-d-wg] Data architecture requirements on naming > > Dear All, > > Recent discussions on the OGSA data architecture have raised > two use cases that may affect the naming discussion. Both > involve the mapping of one "abstract" (for some meaning of > "abstract") name to multiple possible "concrete" names. A > further step is then required to choose the best "concrete" > name, according to the policies required by the client. We > are wondering whether the WS-Name/EPR resolution services can > be modified to support these functionalities. > > Note: here, "abstract" names may or may not have the same > meaning as abstract names in the proposed WS-Names spec. > This is an issue that needs clarification. > > Case 1 is when we are dealing with replicated data. It is > common to have an "abstract" name that represents a given > file, table or query result that is stored someone in the > system. This is then resolved to one of multiple copies. > The client typically wants the copy with the best response > time (which will be some function of latency and bandwidth). > > Case 2 is when we are dealing with data stored at a certain > location and the data may be accessed by several different > protocols (e.g. GridFTP, HTTP, ...). The client (or transfer > service) wants to map the "abstract" name to the fastest > protocol that it can handle. > > In each case, we could take one of two approaches: > > (A) resolve the "abstract" name to a list of "concrete" > names, and use a separate selection service to choose among them > > (B) pass the appropriate policy specification to the resolver > service, which will then return the appropriate "concrete" name. > > Note: (B) could be considered (and even implemented) as a > composition of the resolver and selection services in (A). > > Would it make sense to specify either one of these > functionalities as part of the resolver services considered > in WS-Naming? E.g. the operations could return a list > instead of a single EPR (option A) or they could return a > single EPR as now but take a policy argument (option B). > > Best wishes, > > Dave Berry > Research Manager, National e-Science Centre > 15 South College St., Edinburgh > Tel: +44 131 6514039 > >

1 0

Data architecture requirements on naming
by Dave Berry 22 Nov '05

22 Nov '05

Dear All, Recent discussions on the OGSA data architecture have raised two use cases that may affect the naming discussion. Both involve the mapping of one "abstract" (for some meaning of "abstract") name to multiple possible "concrete" names. A further step is then required to choose the best "concrete" name, according to the policies required by the client. We are wondering whether the WS-Name/EPR resolution services can be modified to support these functionalities. Note: here, "abstract" names may or may not have the same meaning as abstract names in the proposed WS-Names spec. This is an issue that needs clarification. Case 1 is when we are dealing with replicated data. It is common to have an "abstract" name that represents a given file, table or query result that is stored someone in the system. This is then resolved to one of multiple copies. The client typically wants the copy with the best response time (which will be some function of latency and bandwidth). Case 2 is when we are dealing with data stored at a certain location and the data may be accessed by several different protocols (e.g. GridFTP, HTTP, ...). The client (or transfer service) wants to map the "abstract" name to the fastest protocol that it can handle. In each case, we could take one of two approaches: (A) resolve the "abstract" name to a list of "concrete" names, and use a separate selection service to choose among them (B) pass the appropriate policy specification to the resolver service, which will then return the appropriate "concrete" name. Note: (B) could be considered (and even implemented) as a composition of the resolver and selection services in (A). Would it make sense to specify either one of these functionalities as part of the resolver services considered in WS-Naming? E.g. the operations could return a list instead of a single EPR (option A) or they could return a single EPR as now but take a policy argument (option B). Best wishes, Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

Telcon Agenda, 16th November - Storage management section review
by Dave Berry 16 Nov '05

16 Nov '05

Dear All, The next teleconference of the OGSA Data Architecture working group will be held on Wednesday, November 16th, 2005 at the usual time of 8:00am Pacific / 11:00am Eastern / 4:00pm UK / 5:00pm Europe. In this telcon we will review the Storage Management section and any associated scenarios. Please read this section and the scenarios and bring issues to the call. As a reminder, here is the teleconference information: UK dialin: 0845 245 0224 US dialin: 1 866 838 0046 Intl dialin: +44 1452 542309 Passcode: 369786 Agenda for the November 16th teleconference: 1) Early discussion Note taker assignment Roll call Agenda bashing 2) Action report - Ann to update replication section. - Stephen to update replication scenarios. - Allen to advise Stephen on how to address database replication in the scenarios document. - Everyone to comment on draft RNS specification. - Ann to check the Access section for suitability for file replication. - Mario to update the Access section and make text available for comment. - Section authors to add security notes to their sections. - Dave to revamp section 3 of the architecture document. - Everyone to talk with your contacts to get more participation in this WG. - Allen/Dave to contact David Martin & Hiro Kishimoto to get reviewers & expert participation (after internal reviews). 3) Progess & planning update (5 minutes) Update on document writing, plans for telcons and F2F. 4) Review of Storage Management section and associated scenarios (40 minutes) Issues that we should consider include, but are not limited to: - Matching scenarios against service descriptions - Level of detail - Security implications - Consistency with other sections - Identify where work is needed on metadata or on naming conventions - Graceful degradation 5) Wrap up (5 minutes) Next meetings: Nov 23rd: No meeting? (Thanksgiving, Dave travelling) Nov 30th: Scenarios document Dec 7th: ? (Allen on holiday) Dec 14th: ? Dec 21st: ? Possible topics include naming, GFS architecture document, further reviews Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

2 1

OGSA Data replication scenarios - v0.6
by Stephen Davey 16 Nov '05

16 Nov '05

Hi there, Following on from the replication ideas talked about at the OGSA-D telcon last week I have updated the scenarios document, in particular the replication scenarios in sections 3.4.1 & 3.4.2. (See attached doc v0.6). There was some discussion as to whether the scenarios should be incorporated into the main OGSA Data Architecture document. I think they should perhaps be kept separate. The OGSA Data Architecture document would then be well over 100 pages if they were included, which would be getting rather a lot. Plus the OGSA Data Architecture document talks about the individual architectural components and services, whereas the scenarios document gives examples of how these components can fit together in particular cases. But the odd picture in the Data Architecture document might also be very useful, so we could perhaps include say one of the replication (functional) diagrams and maybe one of the data integration pictures. Cheers, Stephen. ---------------------------------------------------------------------- Stephen Davey, NextGrid Software Architect, National e-Science Centre, 15 South College St., Edinburgh, EH8 9AA, UK Tel: +44 131 6 509820

2 1