- ogsa-d-wg - lists.ogf.org

what needs naming in the Data Architecture
by Stephen Davey 15 Dec '05

15 Dec '05

Hi everyone, As part of the RNS & GFS discussion today I wonder if it would be useful to think of all the things in the data architecture that might need naming, and then to check that these would be covered by the 2 specs. Below is my list of all things vaguely data related that I could think of. Is it planned to also discuss the Naming section (3.2) of the Data Architecture document? Cheers, Stephen. caches containers (of resources) data bytes data streams database tables databases defaults digital libraries endpoint references (EPRs) file directories file locations files identities languages locales metadata namespaces naming schemes networks people policies properties queries query result row sets references registries replicas repositories resource locations resources roles schema schema mappings services storage (space) transactions transformations transport protocols user defined entities vocabularies workflows --------------------------------------------------------- Stephen Davey, NextGrid Software Architect, National e-Science Centre, 15 South College St., Edinburgh, EH8 9AA, UK Tel: +44 131 6 509820

2 1

Draft architecture document of Grid file system
by Osamu Tatebe 14 Dec '05

14 Dec '05

Hi all, Attached is a draft document of architecture of Grid file system. It is based on discussion at GGF15 GFS-WG session. Basically it describes functionality of Grid file system, and scenarios to realize these functionalities in service oriented architecture. I would like to improve this document by the next GGF16. It is far from a complete document, but the reason why I send this preliminary document to the mailing list now is that I cannot participate in OGSA-D teleconference today (or tomorrow). I hope this can help and provide some idea and input to OGSA data document. PS. I requested one 90-min session for VFDS (Virtual Filesystem Directory Service) and architecture discussion at GGF16. Thanks, Osamu

1 0

Telcon agenda for 14th December - Naming & GFS architecture
by Dave Berry 13 Dec '05

13 Dec '05

Dear All, The next teleconference of the OGSA Data Architecture working group will be held on Wednesday, December 14th, 2005 at the usual time of 8:00am Pacific / 11:00am Eastern / 4:00pm UK / 5:00pm Europe. In this telcon we will discuss how the RNS proposal and the GFS architecture document affect our work. Please read these document and bring issues to the call. As a reminder, here is the teleconference information: UK dialin: 0845 245 0224 US dialin: 1 866 838 0046 Intl dialin: +44 1452 542309 Passcode: 369786 Agenda for the December 14th teleconference: 1) Early discussion Note taker assignment Roll call Minutes Approval (http://tinyurl.com/avudp) Agenda bashing 2) Action report - Dave to mail references to documents for next meeting. - Chris to talk to GFS about telcon on 14th and a possible GGF16 session. - Dave to ask Ann whether she will be at Sunnyvale - Dave to find out about status of WS-Agreement post GGF15. - Ann to update replication section. - Everyone to comment on draft RNS specification and send comments to the list. - Ann to check the Access section for suitability for file replication. - Mario to update the Access section and make text available for comment. - Section authors to add security notes to their sections. - Dave to revamp section 3 of the architecture document. - Everyone to talk with your contacts to get more participation in this WG. - Allen/Dave to contact David Martin & Hiro Kishimoto to get reviewers & expert participation (after internal reviews). 3) Progess & planning update (5 minutes) Update on document writing, plans for telcons and F2F. 4) Naming and RNS (20 minutes) Does this meet our needs? Can we use it "as is"? Do we have particular needs that we need to pass on? 5) GFS Architecture document (20 minutes) Could this be a use case for our generic architecture? What matches and what doesn't? 6) Wrap up (5 minutes) Next meetings: Dec 21st: Full Data Architecture doc review Dec 28th: No meeting. Jan 4th: No meeting. Jan 11th: ? Jan 16th-23rd: OGSA F2F in Sunnyvale Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

3 2

RNS spec
by Stephen Davey 12 Dec '05

12 Dec '05

Hi everyone, I've had a read of the RNS spec (Nov 2005 version), and made a short summary of it - see below. My own thoughts are as follows: - As suggested by the use of "Human Interface Names", I think that an RNS service would be very useful for user/client GUIs etc. But it does still seem to be quite file application oriented. - Would it be possible to customise the view of the namespace for different users? (Like for example in the way that everyone has their internet bookmarks/favourites arranged in folders of their own devising.) Perhaps one could build up one's own view as a series of federated namespace services that you have discovered. Although one could equally do this by building up a collection of data services perhaps? - The authors of the spec identify several issues that are not yet tackled including consistency and coherency (between redundant and delegated RNS services) and access control lists. These could be quite challenging! - I'm not sure how this all links in with WS-Names and it's "resolve" operation for example. WS-Naming isn't mentioned at all in the spec. Speaking to Malcolm Atkinson last week, he was wondering how data that was generated on the fly and yet to be materialised could be stored/named by the Resource Namespace Service. For example, data that is given by performing a query on a data service or resource, such as http://www.upmystreet.com/map/l/EH1.html?mapNest=1&mapSess=L1VLL2ZpbmRte W5lYXJlc3QvbW90b3JzL3R5cmUtZGVhbGVycy1hbmQtcmVwYWlycy9yZXN1bHRzL2luLw%3D %3D . I guess that because this is a valid URI it can be represented as an EPR and hence named within the RNS service? Cheers, Stephen. ------------------------------------------------------------------------ ---------------------------------------- For RNS spec see https://forge.gridforum.org/docman2/ViewCategory.php?group_id=115&catego ry_id=369 . The specification has 2 parts: Resource Namespace Service (RNS) - this is the main part of the spec. Resource Endpoint Resolution Service (RNS Resolver) - a companion service to RNS. Resource Namespace Service (RNS) - Hierarchically managed namespace for federated & virtualised data, services or any grid resources. - Looks like a directory tree (of entities). - WSRF compliant. - Revised abstraction of the Virtual Filesystem Directory Service (VFDS). - Has 3 types of names; Human Interface Names, (optional) Logical References, Endpoint References. - The optional Logical References (i.e. abstract names) can be used as an additional level of indirection to link the human names to the EPRs, with the assistance of an RNS Resolver or any other EPR (Name) Resolution Service. - Note that mapping information and associated pointer handles for directly mapped human interface name to endpoint references are not exposed by RNS and are therefore only used internally by RNS. - RNS features an extensible design allowing normative profile specifications, such as OGSA Basic Profiles, to define a standard set of resource properties for specific instantiations of the namespace service - deleteProperty, insertProperty, listProperties, updateProperty. - Specification does not address any data related namespace requirements. - RNS is comprised of two fundamental namespace components: virtual directories and junctions. - A virtual directory is an RNS entry that is represented as a non-leaf node in the hierarchical namespace tree. - A junction is an RNS entry that interconnects a reference to an existing resource into the global namespace. - Referral junctions are junctions that are used to graft RNS namespaces - allow federation of independent domains of control. - A logical reference junction is a junction that contains a context unique (potentially global) logical name. If the logical name is expressed as an EPR then a resolver service must be contained in the EPRs WS-Address. - RNS defines a stateful resource referred to as an IteratorContext. (Allows part of the directory list to be returned based on the current iterators index.) - Operations defined are: create, delete, list, lookup, update, (move, rename, mkdir), createIteratorContext, getIteratorContext. Resource Endpoint Resolution Service - Also known as RNS Resolver, provides resolution of logical references - i.e. an abstract name to address/EPR resolution service. - Independent of the RNS but may have the same service URL (but different port type). - RNS Resolver is composed of the following operations: 1) An operation for resolving logical names to endpoint references. 2) Operations for creating, removing, and updating logical references [and EPRs]. General considerations: - The following issues are not explored yet in the document: security, replication, consistency and coherency (between redundant and delegated RNS services), access control lists, root-level names and resolution services, and discovery. ---------------------------------------------------------------------- Stephen Davey, NextGrid Software Architect, National e-Science Centre, 15 South College St., Edinburgh, EH8 9AA, UK Tel: +44 131 6 509820

1 0

Survey for OGSA-WG F2F meeting in January 2006
by Hiro Kishimoto 12 Dec '05

12 Dec '05

Hi all, and sorry for the cross-posting. As you already know, OGSA-WG and fellow WGs are planning to have F2F meeting in January 2006. The meeting will be held in the week of January 16 from Monday morning to Friday noon at Fujitsu campus in Sunnyvale (SF Bay Area). Please let us know your plan for attendance by answering the following on-line survey. Deadline is EoB December 21st. http://www.zoomerang.com/survey.zgi?p=WEB224URW95UZU Session owners: Please provide detailed session agenda, I will incorporate your in the meeting schedule. The latest proposed agenda for January OGSA-WG and the fellow WGs F2F meeting is uploaded to the GridForge. https://forge.gridforum.org/projects/ogsa-wg/document/2006Jan-OGSA-F2F-agen… Thanks, -- Hiro Kishimoto

1 0

Release 3.0 of OGSA-DQP now available.
by Mario Antonioletti 09 Dec '05

09 Dec '05

On behalf of the OGSA-DQP team: Hi All It is a pleasure to announce the third release of OGSA-DQP (Open Grid Services Architecture Distributed Query Processor). OGSA-DQP allows queries to be evaluated over multiple distributed relational databases, where the queries may include calls to web services. The software architecture accesses relational databases using OGSA-DAI (http://www.ogsadai.org.uk) and supports both pipelined and partitioned parallelism. More details about OGSA-DQP can be found at: http://www.ogsadai.org.uk/dqp/ The OGSA-DQP Coordinator Service can now be deployed on to either WSRF or WS-I platforms, using either OGSA-DAI WSRF 2.1 or OGSA-DAI WS-I 2.1 to access the databases referenced in queries. The query evaluation service only requires a WS-I platform, which minimises requirements and allows a simple deployment to be made using WAR files. Furthermore, the query compiler and optimiser can now be built using Cygwin, allowing the coordinator service to be deployed on Windows as well as Linux. This new release also contains some major improvements in query compilation/optimisation and evaluation with respect to performance, multiple bug fixes and a new command line client. A gzipped tarball and winzipped distribution of the source code can be obtained from: http://www.ogsadai.org.uk/downloads If you have previously downloaded OGSA-DAI or OGSA-DQP then the same user name and password will work, otherwise please register to obtain the username/password at: http://www.ogsadai.org.uk/download.cgi?OGSA-DQP This software is released under the OGSA-DAI Project license which is available at http://www.ogsadai.org.uk/releases/dqp.php. We are happy to receive feedback. Please send any queries or bugs to the OGSA-DAI User Mailing list users(a)ogsadai.org.uk (note that you need to be registered to post to this list, to register see: http://www.ogsadai.org.uk/support/list.php) or the support help desk (support(a)ogsadai.org.uk) OGSA-DQP was designed and developed at the Universities of Manchester and Newcastle, UK. It was funded by the UK e-Science programme through the OGSA-DAI, DAIT and myGrid projects. The OGSA-DQP Project Team +-----------------------------------------------------------------------+ |Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | |Tel:0131 650 5141|mario@epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/ | +-----------------------------------------------------------------------+

1 0

Next telcons
by Dave Berry 05 Dec '05

05 Dec '05

Dear all, There will not be an OGSA-D telcon this week. Next week (Dec 14th) we will discuss naming, RNS and the GFS architecture document. I assume the current version of the RNS spec is the one on GridForge at https://forge.gridforum.org/projects/gfs-wg/document/Resource_Namespace_ Service_Final_Draft/en/2. The latest version of the GFS architecture document I have found is the one from GGF14 (attached). The following week (Dec 21st) we will review the architecture document in full. Please finish any rewrites you are currently tasked with and send them to Allen by the 16th. (This applies to me too!). Best wishes, Dave.

1 0

GGF16 sessions
by Dave Berry 05 Dec '05

05 Dec '05

Dear all, I have requested two 90-minute OGSA-D sessions at GGF16. One is for us to present an overview of the architecture (services and scenarios). The other is for an in-depth discussion. Best wishes, Dave.

1 0

RE: [ogsa-d-wg] Telcon Agenda for 30th November: Scenarios document
by Peter Kunszt 30 Nov '05

30 Nov '05

hi dave sorry i won't be able to atted today after all - i have to be at home by 5pm. cheers peter > -----Original Message----- > From: owner-ogsa-d-wg(a)ggf.org > [mailto:owner-ogsa-d-wg@ggf.org] On Behalf Of Dave Berry > Sent: 27 November 2005 16:58 > To: ogsa-d-wg(a)ggf.org > Subject: [ogsa-d-wg] Telcon Agenda for 30th November: > Scenarios document > > Dear All, > > The next teleconference of the OGSA Data Architecture working > group will be held on Wednesday, November 30th, 2005 at the > usual time of 8:00am Pacific / 11:00am Eastern / 4:00pm UK / > 5:00pm Europe. > > In this telcon we will review the Scenarios document. Please > read this document and bring issues to the call. > > As a reminder, here is the teleconference information: > > UK dialin: 0845 245 0224 > US dialin: 1 866 838 0046 > Intl dialin: +44 1452 542309 > > Passcode: 369786 > > > Agenda for the November 30th teleconference: > > 1) Early discussion > Note taker assignment > Roll call > Minutes Approval (9th November: http://tinyurl.com/cktqe, 16th > November: http://tinyurl.com/9hpnr) > Agenda bashing > > 2) Action report > > - All to discuss plans for GGF16 OGSA Data meetings & F2F. > - Dave to find out about status of WS-Agreement post GGF15. > - Peter & Stephen to add first draft of Storage Management scenario. > > - Ann to update replication section. > - Everyone to comment on draft RNS specification. > - Ann to check the Access section for suitability for file > replication. > - Mario to update the Access section and make text available > for comment. > - Section authors to add security notes to their sections. > - Dave to revamp section 3 of the architecture document. > - Everyone to talk with your contacts to get more > participation in this WG. > - Allen/Dave to contact David Martin & Hiro Kishimoto to get > reviewers & expert participation (after internal reviews). > > > 3) Progess & planning update (5 minutes) > > Update on document writing, plans for telcons and F2F. > > 4) Review of Scenarios document (40 minutes) > > Issues that we should consider include, but are not limited to: > > - Matching scenarios against service descriptions > - Completeness > - Level of detail > - Security implications > - Consistency between scenarios > - Identify where work is needed on metadata or on naming conventions > > 5) Wrap up (5 minutes) > > Next meetings: > > Dec 7th: No meeting? (Allen on holiday) > Dec 14th: Naming, RNS & GFS architecture > Dec 21st: Full Data Architecture doc review > Dec 28th: No meeting. Christmas hols. > > > Dave Berry > Research Manager, National e-Science Centre > 15 South College St., Edinburgh > Tel: +44 131 6514039 > >

1 0

Data scenarios v0.7
by Stephen Davey 29 Nov '05

29 Nov '05

Hi everyone, Attached is my latest scenarios document - version 0.7 which includes Allen's comments on v0.6. It also has a Storage scenario from Peter and I have made a start at a Personal Data Service scenario, and a bit of a Data Discovery scenario. A copy is also on GridForge - https://forge.gridforum.org/docman2/ViewCategory.php?group_id=150&catego ry_id=1033 . I wonder if we should try to ensure that there is a least one scenario that is focussed on each of the main sections in the Data Architecture document? The architecture sections and related scenarios are currently as follows: Data Description - ??? Data Transfer - data transfer scenario, data staging scenario. Data Access - personal data service scenario?? Storage Resource Management - storage scenario. Cache Services - ??? Data Replication - data replication scenario. Data Federation - data integration scenario, personal data service scenario. Metadata Catalogue & Registries - data discovery scenario? Personal Data Service - personal data service scenario. Transformational Services - ??? Data Streams - ??? Also mentioned in the Data Architecture document are: Naming - personal data service scenario?? Metadata - data discovery scenario?? Security Transactions Sessions Execution Management Services (EMS) Hope people get time to read it before the telcon tomorrow, cheers, Stephen. ---------------------------------------------------------------------- Stephen Davey, NextGrid Software Architect, National e-Science Centre, 15 South College St., Edinburgh, EH8 9AA, UK Tel: +44 131 6 509820

1 0