- ogsa-d-wg - lists.ogf.org

The Bigger Picture.
by Mario Antonioletti 23 May '05

23 May '05

Hi, I think one thing that has been lacking a bit in the OGSA-D WG so far has been the bigger picture trying to tie some of the concepts together. A long time ago I remember seeing the concept maps that were produced for the WS architecture (see: http://www.w3.org/TR/ws-arch/) and I quite liked these. I don't know what tool they used but I found something similar, if not the same. See: http://cmap.ihmc.us/ It's a java application and I think, though I've not looked too closely, is free. I have tried to create a concept map for the data architecture and have included an exported image as an attachment to this email as well as the cmap file that contains the source to the diagram in case anyone wants to play with it. It was very easy to generate and change and I believe the cmaps can be shared so they could be developed in a collaborative fashion. I think it helps to ellicit detail from the model, e.g. to what does the data description bind to: the virtualization or the service? Does the idea of having a data set defined in the data transfer make sense? i.e. I think of a data set as a disconnected piece of data that has been materialised outside the data resource and this would only make sense if it were staged. Do we benefit at all from making the distinction between a simple and aggregated data service? etc, etc, etc. Note that I am not advocating that these should be included in the document, nor that everything should go in one diagram (as is currently). I think that the diagrams can be nested (maybe). Is this a worthwhile thing to do? If the answer is yes can it be done so the diagrams are shared? Thoughts? I will try to distribute something on data access by the close of day today (UK time) albeit it'll be far from complete. Mario +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

3 3

Storage chapter first go
by Peter Kunszt 22 May '05

22 May '05

hi everyone, as promised in the phonecon a week ago, here is a first shot at describing the ideas around a data storage resource. peter -- CERN IT-Grid Middleware

1 0

Draft agenda for next week's F2F
by Dave Berry 18 May '05

18 May '05

All, Please find attached a draft agenda for the F2F meeting on May 26th, based on the discussion in today's telcon. We can revise this agenda as necessary, either via e-mail beforehand or on the day. Please can the authors of each document section prepare a couple of slides for session 1? The idea of this session is to highlight issues that require input from the rest of the architecture, or that will affect the overall design. Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

OGSA-D telcon agenda for May 18 - F2F planning
by Dave Berry 16 May '05

16 May '05

Dear all, This week's telcon will be on Wednesday (18th May), at the usual time of 8:00am PDT / 11:00am EDT / 4:00pm BST / 5:00pm CEST. PIN for all calls: 632 065 UK Dial In: 0845 245 0224 US Dial In: 1866 838 0046 International Dial In: +44 1452 542309 Here is a provisional agenda for this week. 1) Early discussion (5 min) Note taker assignment Roll call Minutes approval (attached, http://tinyurl.com/b4qxv) Agenda bashing 2) Action report (5 min) - Dave & Allen to add GGF working drafts to agenda for May 18 call [Done] - Dave & Allen to finalize session plan at May 18 call and send in session requests ASAP after that. - Dave & Allen to make F2F agenda discussion to be part of the May 18 call [Done] - Dave & Allen to publish approximate start and end times for the F2F ASAP to allow people to finalize travel plans. - Dave & Allen to make sure the F2F on May 26 should include a session to discuss how to identify data and how to get the bits into and out of storage. - Peter to write a first cut at the Storage section of the architecture document. - Dave to try to arrange data transfer discussion. - Mario to further update the access section. - Dave to approach transaction RG re GGF session? 3) Document status (15 min) - Reports from contributors 4) WG status (15 min) - GGF working drafts - GGF session plan 5) F2F agenda (15 min) - ByteIO - How to identify data and how to get the bits into and out of storage - Document structure 6) Wrap up (5 min) AOB DONM Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

OGSA-D telcon agenda for May 11 - Storage Management
by Dave Berry 07 May '05

07 May '05

Dear all, Here are the planned topics for upcoming telcons: May 11 - Data storage May 18 - Document review & plan for F2F May 25 - no call? May 26 - F2F, Imperial College London This week's telcon will be on Wednesday (11th May), at the usual time of 8:00am PDT / 11:00am EDT / 4:00pm BST / 5:00pm CEST. PIN for all calls: 632 065 UK Dial In: 0845 245 0224 US Dial In: 1866 838 0046 International Dial In: +44 1452 542309 Here is a provisional agenda for this week. 1) Early discussion (5 min) Note taker assignment Roll call Minutes approval (attached, http://tinyurl.com/d8fvd) Agenda bashing 2) Action report (5 min) - Dave to try to arrange data transfer discussion. - Allen to mail Steve about adding Allen and Dave as GridForge admins [Done] - Mario to further update the access section. - Dave to approach transaction RG re GGF session? 3) WG status (5 min) 4) F2F update (5 min) 5) Technical content (35 min) - Storage Management 6) Wrap up (5 min) AOB DONM Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

Data Subset Replication Use Cases
by Allen Luniewski 04 May '05

04 May '05

At the April 27 Data Architecture call, I volunteered to write a use case for replication of subsets of data. Below is my attempt to capture the basic idea in a few examples.. Allen Data Subset Replication Replication of entire objects (e.g., a file, an entire database) is a natural, and obvious, place to start considering replication. However there is a real need to replicate subsets of data. Here are a number of motivating examples (use cases): Company A has an employee database containing all information about its employees. The database is multiple terabytes in size and updates are frequent. Suppose that the payroll information is contained in a single table in that database. The payroll department needs to have fast access to the payroll table. This table is only a few tens of gigabytes in size and updates are infrequent. Replicating just this subset reduces storage consumption on the payroll system, reduces the bandwidth used to maintain the replica and reduces the processing power used to create and handle updates to the payroll system. This is an example of replicating a single table instead of an entire database. A Life Sciences example. Suppose that there is a large file that describes the entire human genome held on some server at UCLA. This file is multiple terabytes in size. A researcher in Paris desires to perform some computation on genes 14 through 19. For efficiency the data being processed must be at a server in Paris. Instead of expending considerable resource to move the entire file, relatively small amounts of resource are used to move just that portion of the file containing genes 14 through 19. Now suppose that a second researcher in Paris desires to perform a computation on genes 18 and 19. Instead of moving the entire file, or even moving the subset of the file containing genes 18 and 19, that researcher can reuse the partial replica already held in Parsis since it contains genes 14 through 19 and the genes of interest are a subset of those. A final example from the database world. Suppose that a multi-site hospital keeps a database of its patients and that the database contains all patient information including voluminous information such as x-ray images and MRI scans. Thus it is very large. One of the hospitals in the system, located in Boston, is very specialized - it only sees local elderly cancer patients. This hospital needs to have a replica of the database that contains only those patients - replicating the entire database is, as above, a waste of precious resources. So the hospital needs a replica that contains only those patient records that match a query that might look something like: "Patient.age > 65 AND DISTANCE(Patient.address, "Boston") < '50 miles' AND (Patient.illnesses INCLUDES "cancer" OR Patient.past Illnesses INCLUDES "cancer")"

1 0

Data Access Scope.
by Mario Antonioletti 04 May '05

04 May '05

Hi, I was thinking round some comments that Allen wrote on the section I distributed yesterday and became somewhat confused as to what the scope of data access is for this ogsa data architecture document. I make some assertions below and some of the consequences - any thoughts? What should be ruled in scope and what is out of scope for this version of the document? Mario --- Assertions: 1. a data service represents: i) A single data resource (the data resource may be arbitrarily complex, e.g. it could be a federation but the data service has a single point of contact with the data resource) ii) More than one data resource iii) More than one data service iv) A combination of data resources and data services 2. the data service's data access capability allows: i) Read only access. ii) Read and Write access. 3. data obtained by a data service through a data resource is: i) Returned directly back to the client ii) Materialised and kept at the service side, exposed through another (or the same) data service. iii) Data is delivered to a third party, the client gets a status notification. Consequences: 1. For cases 1 (ii)-(iv) require some naming scheme for the service to determine what data resource/service can be targetted assuming that a message will only ever target a single resource. Most of the entities will have names that already exist. Question: can/should a message be able to target multiple resources? 2. For cases 1 (iii) and (iv) require some service-to-service access mechanism. Is this in scope for data access. 3. For case 2 (ii) Write access will complicate things by a *lot*. Things get even more complicated if cases 1 (ii)-(iv) which may require to operate in some transactional context. 4. For case 3 (i) is kind of weak for transferring large amounts of data. Are cases 3 (ii) and (iii) in scope for data access? You could have an operation that specifies another end point (which need not be a service) to which data should be delivered. Is this in scope? +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

1 0

Data Access.
by Mario Antonioletti 03 May '05

03 May '05

Hi, Ok. Have not that much time to do much with this. It is still very tentative. Would welcome any feedback - constructive or destructive - or even better contributions if people want to pick up the pen. In essence have written a brief introduction, stolen some stuff from the DAIS specs, re-used the DAIS stuff I wrote on a previous occassion but not really updated it. Stole - whole - with slight modificaitons some stuff that Mark and Andrew wrote for GGF13. Still requires much work but it can be a starting point for discussion at the call tomorrow. Mario +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

1 0

Data: telcon agenda for May 4 - WS-DAI & ByteIO
by Dave Berry 29 Apr '05

29 Apr '05

Dear all, Here are the planned topics for upcoming telcons: May 4 - Data access (WS-DAI and ByteIO) May 11 - Data storage May 18 - Document review & plan for F2F This week's telcon will be at the usual time of Wednesday (4th May), at 8:00am PDT / 11:00am EDT / 4:00pm BST / 5:00pm CEST. PIN for all calls: 632 065 UK Dial In: 0845 245 0224 US Dial In: 1866 838 0046 International Dial In: +44 1452 542309 Here is a provisional agenda for this week. 1) Early discussion (5 min) Note taker assignment Roll call Minutes approval (attached) Agenda bashing 2) Action report (5 min) - Fred to send notes on how to use Majordomo to Allen and Dave. - Dave to try to arrange data transfer discussion. - Allen to write up naming use cases for passing to OGSA WG (after review in our WG). - Ann and Allen to clean up the draft and mail it around. 3) WG status (5 min) 4) F2F planning (5 min) 5) Technical content (35 min) - WS-DAI - ByteIO 6) Wrap up (5 min) AOB DONM Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

Fw: draft of section on replication
by Allen Luniewski 28 Apr '05

28 Apr '05

Ann's note bounced. Allen ----- Forwarded by Allen Luniewski/Almaden/IBM on 04/27/2005 04:29 PM ----- Ann Chervenak <annc(a)ISI.EDU> 04/27/2005 04:21 PM To Dave Berry <daveb(a)nesc.ac.uk> cc ogsa-d-wg(a)gridforum.org, Dave Pearson <Dave.Pearson(a)oracle.com>, Mario Antonioletti <mario(a)epcc.ed.ac.uk>, Martin Westhead <M.Westhead(a)epcc.ed.ac.uk>, gtr(a)ast.cam.ac.uk, chappella(a)battelle.org, arun(a)sdsc.edu, Jane Xu/San Jose/IBM@IBMUS, o.tatebe(a)aist.go.jp, Yves.Denneulin(a)imag.fr, ppadala(a)cise.ufl.edu, ivm(a)fnal.gov, jalameda(a)ncsa.uiuc.edu, Cecile Madsen/Santa Teresa/IBM@IBMUS, allcock(a)mcs.anl.gov, Allen Luniewski <luniew(a)almaden.ibm.com>, Malcolm Atkinson <mpa(a)nesc.ac.uk>, Susan Malaika/Santa Teresa/IBM@IBMUS, "Peter Clarke (UCL)" <clarke(a)hep.ucl.ac.uk>, David E Martin/Chicago/IBM@IBMUS, Peter Kunszt <pkunszt(a)mail.cern.ch>, Ian.Bird(a)cern.ch, Erwin.Laure(a)cern.ch, Gavin.Mccance(a)cern.ch, Scott Oster <oster(a)bmi.osu.edu>, Stephen Langella <langella(a)bmi.osu.edu>, Shannon Hastings <hastings(a)bmi.osu.edu>, Manuel Pereira/Almaden/IBM@IBMUS, Ted Anderson/Almaden/IBM@IBMUS, Leo Luan <luan(a)almaden.ibm.com>, "Mark M. Morgan" <mmm2a(a)cs.virginia.edu>, Ann Chervenak <annc(a)ISI.EDU> Subject draft of section on replication Hi, The draft of the section on replication that Allen and I have been working on is attached. We have included a few comments about outstanding issues. Your comments are welcome. Ann

1 0