May 2006 - cddlm-wg - lists.ogf.org

CVS changes at sourceforge
by Steve Loughran 15 May '06

15 May '06

You all may have been lucky enough to avoid noticing, but we here have been very aware that sourceforge CVS has been down for a week. It is now back up, but different: http://sourceforge.net/docs/E04 The obvious change is that the new host of the project is deploy.cvs.sourceforge.net instead of just cvs.sourceforge.net The repository went down with filesystem problems, and rather than identify the moment things went bad sourceforge have restored the repository from a few days before, we think about may 7. I will merge in all changes from right up to the moment the repository went down, because our CI build system has a snapshot of just before it went offline. I can't do it yet as it wont log me in, so allow a few days for things to stabilise before trying yourself. Try not to make any drastic changes to your existing codebase as you will have to merge it in (I have a build.xml <copy> operation that does a byte-for-byte comparison to help). -steve

1 0

canonicalization/serialization standards
by Hiro Kishimoto 15 May '06

15 May '06

Hi Jun, As I told you at GGF17, the following is the text from OGSA WSRF BP 1.0. Good and bad canonicalization/serialization standards for XML doc. > Several specifications defining canonicalization/serialization > algorithms have been developed: Canonical XML [XML-C14N], Exclusive > XML Canonicalization [XML-Exc-C14N] and > [SchemaCentric-XML-C14N]. The first two specifications have > well known issues that make them unsuitable for serialization (except > in the most limited of use cases). [SchemaCentric-XML-C14N] > addresses these issues. For a complete discussion of the limitations > of Canonical XML and Exclusive XML Canonicalization see > [SchemaCentric-XML-C14N] section 1.1. [SchemaCentric-XML-C14N] B. Atkinson, S. Aissi, M. Hondo (ed.), Schema Centric XML Canonicalization, OASIS Committee Specification, 10 July 2002. http://www.uddi.org/pubs/SchemaCentricCanonicalization-20020710.htm [XML-C14N] J. Boyer (ed.) Canonical XML W3C Recommendation, March 2001 http://www.w3.org/TR/2001/REC-xml-c14n-20010315 http://www.ietf.org/rfc/rfc3076.txt [XML-Exc-C14N] J. Boyer, D. Eastlake, and J. Reagle (ed.) Exclusive XML Canonicalization W3C Candidate Recommendation, 12 February 2002 http://www.w3.org/TR/2002/CR-xml-exc-c14n-20020212 -- Hiro Kishimoto

1 0

Notes from the regular meeting on 5/10
by Milojicic, Dejan S (HP Labs) 10 May '06

10 May '06

Attendees: Ayla, Guilherme, Flavio, and Dejan. Jun in Japan at GGF, Stuart and Steve were missing. Ayla has question on how they are performing CDL component model validation. They just finished testing, and they see that Satish is testing against their implementation. Satish: their deployment is still not done. Not the one CDL that Stuart has proposed but another one that UFCG has provided. Jun was also trying to run some tests from Tokyo. It also appears that NEC server is down. UFCG has 100% complete testing of component model and would like to start testing against other implementations. Will use the approach that Satish has proposed. Wrt deployment APIs, some tests were missing, so UFCG has implemented only some of them since it was hard to automate them. Satish (NEC): Completed all the tests agains their own implementation, but not much progress wrt other systems. Sent an error log to Steve yesterday. Still need to check what is exactly the problem. Probably will also set the demo for GGF17, will share that demo, and put it online so that everybody can see that. The important piece missing was / at the end of the URL. This was missing in Steve's implementation, then Steve pointed this out, after that Satish was able to run test property etc. Some of the basic tests were otherwise failing. Thanks, Dejan.

3 2

second pass at jun's slides
by Steve Loughran 10 May '06

10 May '06

hello all I've updated the slides with the latest test results from my system, and tweaked some other things -put in current interop test results and URL to full report -some textual commentary in the notes page expands on the details. -added '$Vendor Test Results' on each test result slides for $Vendor={hp,ucfg,nec} -made the TODO page an HP TODO page that lists things I have to do -made the issues page purely one about communal testing issues, things that would be of interest to anyone doing interop testing 1. we need better access to each other's logs. Otherwise you have to try and get the other teams to rummage in their I am actually publishing a log dir: http://deployapi.iseran.com:8080/logs/ However, on the public endpoint, log4J is not logging their, even though it does on my staging system. Some research needed there, but a fix should go in before the end of the week, CVS-problems notwithstanding. 2. Failing tests cannot distinguish why tests failed, because we dont have standard fault codes. Example: imagine the test to submit a CDL doc with an incomplete non-lazy reference. This must not enter the running state, so initalize must fail. But without an error code test, you cannot distinguish "failed with invalid reference" from "failed because the XML parser is broken". These generates false positives. There is no fix for this, other than standard fault codes. 3. having continuous integration servers redeploying the servers and rerunning the tests all the time would help the other teams. For example, if my work box ran the hp client interop tests every half hour, the published results would always be up to date, and the cycle time for anyone testing their server would be much shorter. -steve

2 2

RE: [cddlm] Basic POSIX Component & template draft
by Dave Berry 08 May '06

08 May '06

Hi, It sounds like you're having interesting discussions. What concerns me is how does this work relate to: - the hierarchical naming level of the OGSA naming system? - the work of the Grid File System working group? - ByteIO? - the OGSA Data Architecture? Dave. > -----Original Message----- > From: owner-cddlm-wg(a)ggf.org [mailto:owner-cddlm-wg@ggf.org] > On Behalf Of Hiro Kishimoto > Sent: 08 May 2006 14:56 > To: Steve Loughran > Cc: OGSA WG; cddlm-wg(a)ggf.org > Subject: Re: [cddlm] Basic POSIX Component & template draft > > Hi Steve, > > Your file & filesystem components are very good. They are > what I am looking for. One more question I have in my mind is > how to set user-id, group-id, and permissions to deployed files. > > Can you call in OGSA-EMS session on Wednesday? > It starts 5:45pm JST = 9:45am UK. > > The dial-in number for this session; > US: +1 718 3541071 (New York) or > +1 408 9616509 (San Jose) > UK: +44 (0)207 3655269 (London) > Japan: +81 (0)3 3570 8225 (Tokyo) > PIN: 4371991 > > Thanks, > ---- > Hiro Kishimoto > > Steve Loughran wrote: > > Hiro Kishimoto wrote: > >> Hi all, > >> > >> Jun Tatemura and I took a home work assignment about "Basic POSIX > >> Component" for BLAST application deployment. > >> > >> Attached is my first-cut draft and hope to discuss at > GGF17 OGSA EMS > >> session. > >> Please have a look and give your feedbacks before or at the GGF17. > >> Your feedbacks are very important. > >> > >> Thanks and see you in Tokyo! > > > > > > This is really good, and I do agree, we do need a set of > foundational > > things to deploy. > > > > One thing I would argue, having learned the lessons of both Ant and > > SmartFrog, is that a consistent model of file system > components comes > > first. That is, we need components to model > > > > -files (with a liveness test that verifies the file is > present) -paths > > (an ordered list of files/directories) -directories (with > components > > that can create a dir on deployment, optionally to delete > it and its > > contents on termination) -wildcarded sets of files (**/*.csv) > > -temporary files and directories -cached downloads -text files > > > > I'm attaching the PDF file describing the current smartfrog set of > > components that do this. The underlying way that this is > done is that > > all components that consider themselves to be sources of > > filenames/paths set the attribute absolutePath at runtime. They > > usually also export an RMI interface for RPC-style > operations, along side their RESTy state. > > > > A standard set of filesystem components lets you include the > > operations to create and clean up directories, temporary files and > > other housekeeping operations into the workflows of a deployment. > > > > Looking at the Posix proposal, I'd suggest starting with > the reusable > > set of filesystem types, including a <fs:mount> component > that could > > mount a local or remote filesystem, and which would then pass the > > local location to interested parties as the > filesystem:absolutePath attribute. > > > > You absolutely don't need to introduce a new shared datatype > > FileSystemName, because that implies some kind of alternate > > naming/locating scheme outside of what CDL and the runtime > has built in. > > Dynamic CDL references are sufficient to crosslink > information inside > > the graph of deployed things, and offer flexibility as well > as ease of > > management (you see them when you walk the graph). > > > > First, we'd need filesystem types: > > > > <tmp cdl:extends="fs:TempFileSystem"> > > <fs:Description> ... </fs:Description> > > <fs:DiskSpace> > > <fs:LowerBoundedRange>10737418240.0</fs:LowerBoundedRange> > > </fs:DiskSpace> > > </tmp> > > > > <home cdl:extends="fs:Directory"> > > <fs:Description>Chris's home directory</fs:Description> > > <fs:dir>/home/csmith</fs:dir> > > </home> > > > > When deployed, both of these would add absolutePath as an > attribute, > > so that tmp/absolutePath woudl resolve to, say, tmp/work01234/ and > > home:absolutePath to /home/csmith. > > > > > > You'd then mount a blast dir, say on an nfs filesys whose URL you > > provide > > > > <blastfs cdl:extends="fs:RemoteFileSystem"> > > <fs:url>nfs://filestore/csmith/blast</fs:url> > > <fs:mountUser>csmith</fs:mountUser> > > </tmp> > > > > The database file is under here, and you declare that it > must exist. > > Deployment will fail > > if it is absent: > > > > <db cdl:extends="fs:File"> > > <fs:dir cdl:ref="/tmp/absolutePath" cdl:lazy="true" /> > > <fs:filename>db/ncbiblast/est</fs:filename> > > <fs:filemustexist>true</fs:filemustexist> > > </db> > > > > > > To use it, just refer to its path > > > > <blast:database cdl:ref="/db/absolutePath" cdl:lazy="true"/> > > > > Paths could be represented by some list-like element, whose entries > > are evaluated at deploy time and then a platform-specific > path created > > as a result > > > > <path cdl:extends="fs:Path"> > > <pathentry cdl:ref="/blastfs/absolutePath" cdl:lazy="true" /> > > <pathentry cdl:ref="/home/absolutePath" cdl:lazy="true"/> </path> > > > > On Windows /path/absolutePath would become something like > > "\\filestore\csmith\blast;c:\documents and settings\csmith" > > while on unix it could be > > "/nfs/filestore/csmith/blast:/home/csmith" > > > > This information now goes down to the jsdl component > > > > <jsdl-posix:Environment name="PATH" > > cdl:ref="path:absolutePath" cdl:lazy="true"/> > > <jsdl-posix:Environment name="TMPDIR" > > cdl:ref="tmp/absolutePath" cdl:lazy="true"/> > > > > > > -Steve > > > > > > > > > > > > > > > >

2 1

Basic POSIX Component & template draft
by Hiro Kishimoto 08 May '06

08 May '06

Hi all, Jun Tatemura and I took a home work assignment about "Basic POSIX Component" for BLAST application deployment. Attached is my first-cut draft and hope to discuss at GGF17 OGSA EMS session. Please have a look and give your feedbacks before or at the GGF17. Your feedbacks are very important. Thanks and see you in Tokyo! -- Hiro Kishimoto

2 3

HP interop client results now up online
by Steve Loughran 04 May '06

04 May '06

I've got my build set up to generate test reports when run against the server and then to publish the results: http://people.apache.org/~stevel/cddlm/interop/ Ourgrid is offline right now, NEC's endpoint is throwing a runtime exception in a handler whenever I talk to it. For some reason, Luntbuild refuses to check out and compile the Alpine bit of the source tree. If I could fix that, I could have it rerun and republish these tests every hour. Until then, I'll do it by hand at most once a day. -steve

1 0

first pass at filling out jun's slides
by Steve Loughran 03 May '06

03 May '06

Jun, here is my first quick fill in of these slides. -steve

3 2

Notes from 5/3 (ACTION ITEMS REVIEW HIRO'S DOCUMENT; SEND % OF COMPLETENESS OF REF. IMPLEM. AND INTEROPERABILITY TESTS).
by Milojicic, Dejan S (HP Labs) 03 May '06

03 May '06

Attendees: Ayla, Flavio, Guilherme, Satish, Steve, Hiro, Jun, and Dejan. 1. Interoperability Satish tried talking to Steve's endpoint, he is able to ping it but was not able to run the test. His SOAP headers were not able to talk to his endpoint, so Satish is changing his tests. Steve will publish his logs so that they can debug it together. UFCG: Guilherme performed some tests on Satish's endpoints (1-12 passed), he could not run one other, will send it to Satish. Flavio is also working on component model testing and discovered some changes to implementation, it took more time to do testing for the component model. UFCG completeness: CDL-99%, Component Model, almost at the same point 60-65%, Deployment API: 100%. Steve: (against his own implementation) CDL 90% (the only things do not work wrt name space processing and no tests for expressions); component model is irrelevant; deployment APIs (against his own system, everything up to deployment is implemented, WS-Notifications not addressed, so a little bit less than 50%). The major issue is that nobody can talk to his end point at the moment. Satish: Interoperability, Deployment APIs 90%; 80% of component model. Next he will work with Steve to see why his endpoint does not work. 2. Hiro's document SDD Technical committee called for the submission for specification and they will have a meeting in May 15 in Chicago. They requested the submission for the previous week for documents. They asked for GGF to give the specification for consideration. By the end of this week or Monday/Tue next week he is looking for the feedback. Actually, the tentative deadline for sending the document is Monday, so please send any comments before that. Next F2F will be before GGF18, late July/August time frame. Hiro invites suggestions for the place/time. 3. Jun's document Please update the most recent presentation that Steve has sent, Dejan will use this information to summarize the status in Hiro's document to SDD. Thanks, Dejan.

1 0

RE: [cddlm] FW: OASIS SDD-TC
by Milojicic, Dejan S (HP Labs) 03 May '06

03 May '06

Hi, Attached is the PDF, some people had problems opening it with OpenOffice. Thanks, Dejan. > -----Original Message----- > From: Hiro Kishimoto [mailto:hiro.kishimoto@jp.fujitsu.com] > Sent: Monday, May 01, 2006 2:56 PM > To: Milojicic, Dejan S (HP Labs) > Cc: cddlm-wg(a)ggf.org > Subject: Re: [cddlm] FW: OASIS SDD-TC > > Thanks Dejan, > > Attached is a draft for SDD and ACS/CDDLM/JSDL discussion. > I value your feedbacks. > ---- > Hiro Kishimoto > > Milojicic, Dejan S (HP Labs) wrote: > > Hi Team, > > > > Hiro will join us next meeting to discuss the topic > addressed in the > > email below (SDD and ACS/CDDLM/JSDL). This will be a good > opportunity > > to ask Hiro about his position wrt interoperability and > > standardization, as well as about other topics. BTW Hiro is > the member > > of the CDDLM mailing list, so you can start any discussion > using email. > > > > Best regards, > > > > Dejan. > > > >> -----Original Message----- > >> From: Hiro Kishimoto [mailto:hiro.kishimoto@jp.fujitsu.com] > >> Sent: Sunday, April 30, 2006 11:22 PM > >> To: Milojicic, Dejan S (HP Labs) > >> Cc: Stuart Schaefer; Jun Tatemura > >> Subject: Re: OASIS SDD-TC > >> > >> Thanks Dejan, > >> > >> I agree with you. Please discuss at your meeting. I can join to > >> answer > > > >> any question from your group if you like. > >> > >> It is May 3rd Wednesday 6am PDT (= 10pm JST), right? > >> > >> Thanks again, > >> ---- > >> Hiro Kishimoto > >> > >> Milojicic, Dejan S (HP Labs) wrote: > >>> Thanks Hiro, > >>> > >>> I personally think that this is a good idea. Please let > us discuss > >>> this as a team at the next Wednesday meeting and I will > get back to > >>> you thereafter. Again, personally I think that this is the right > >>> approach, but we are used to make decisions as a team and I > >> prefer to > >>> do it this way again. > >>> > >>> Best regards, > >>> > >>> Dejan. > >>> > >>>> -----Original Message----- > >>>> From: Hiro Kishimoto [mailto:hiro.kishimoto@jp.fujitsu.com] > >>>> Sent: Saturday, April 29, 2006 5:39 PM > >>>> To: Dejan Milojicic; Stuart Schaefer; Jun Tatemura > >>>> Subject: OASIS SDD-TC > >>>> > >>>> Hi Dejan, Stuart, and Jun, > >>>> > >>>> As you may already heard form Jun, OASIS SDD-TC (they are > >> working on > >>>> closely related technology; solution installation) is now > >> calling for > >>>> submission. > >>>> > >>>> http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sdd > >>>> http://lists.oasis-open.org/archives/sdd/200604/msg00027.html > >>>> http://www.oasis-open.org/committees/download.php/17882/oasis- > >>>> sdd-requirements-v1.1.doc > >>>> > >>>> Although their focus is non-grid domain, the last thing I > >> would see > >>>> is GGF and OASIS will publish two incompatible deployment > >> standards > >>>> and thus impose a unnecessary burden on both end users and > >> software > >>>> providers. > >>>> > >>>> In order to avoid such worst case scenario, I think GGF should > >>>> provide related GGF specifications for SDD-TC and ask them to > >>>> consider. > >>>> It does not mean CDDLM-WG should work with SDD-TC and modify or > >>>> update your specs to meet their needs. Instead, just ask > SDD-TC to > >>>> make their spec compatible with CDDLM and another GGF's specs. > >>>> > >>>> The following is the list of GGF's specs we would like to offer; > >>>> > >>>> (1) Application Contents Service Specification 1.0 > >>>> (2) CDDLM XML-CDL > >>>> (3) CDDLM component model > >>>> (4) Job Submission Description Language 1.0 > >>>> (SDD-TC also covers required resource description) > >>>> > >>>> We exclude CDDLM Deployment API since SDD-TC does not > >> cover interface > >>>> specification. > >>>> > >>>> It is very important for OGSA and GGF in general to keep > >> consistent > >>>> with wider Web service community. For this reason, ACS-WG > >> and JSDL-WG > >>>> already agree to the above plan and we would like to hear your > >>>> thoughts. > >>>> > >>>> Thanks, > >>>> -- > >>>> Hiro Kishimoto > >>>> > >>>> > >>> > >> > > > > > > >

1 0