May 2006 - cddlm-wg - lists.ogf.org

Notes from the meeting on 5/31/2006
by Milojicic, Dejan S (HP Labs) 07 Jun '06

07 Jun '06

Attendees: Satish (NEC), Flavio, Guilherme (UFCG), Dejan (HP) Agenda: 1. Update: UFCG: no significant progress with component model because they missing files from others, from everyone. ACTION item: get files from Satish, Steve, and Stuart, so that UFCG can make some progress. (Satish: some of these cases, such as switch and delegate still haven't been implemented in the parser, so right now he does not have these files, will probably work on these this week and send them to Flavio. Others (STUART AND STEVE): PLEASE REPLY BY EMAIL WHEN YOU WILL PROVIDE THE FILES). Flavio will explicitly request these files. Guilherme testing against the NEC endpoint broke again, not sure what is it about. The interval was 2 minutes. Satish requested to get the sample tests and possibly trace. Satish was testing from inside and did not get problems. NEC: Wrt UFCG, he did deployment test using CDL that Flavio sent a few weeks ago, is trying to reproduce the problems and it is failing but still does not understand what it is. The problem is with not being able to deployment components on each other's components. Deployment APIs are 75% on HP endpoint, but not wrt component model. Asked Steve with how to resolve the issue with him not being able to talk to other endpoints, but still didn't get response. 2. We received feedback on the CDL spec and Jun should take action on it. Jun, can you please let us know approximately when you will be able to address the questions raised. 3. EVERYONE: PLEASE TRY TO ATTEND NEXT MEETING. WE ARE CLOSE TO OUR MILESTONE TO COMPLETE TESTING SO THAT WE CAN START WRITING INFORMAL DOCUMENT SUMMARIZING INTEROPERABILITY! Thanks, Dejan.

5 6

updated deployapi tests
by Steve Loughran 26 May '06

26 May '06

hello, I've just checked in some changes to the deploy API -added some more tests related to initialisation (what I was working on ) -moved some tests down into the informative category, like the one that says 'you must be able to create more than one application simultaneously', as that is not actually part of the spec. I note that I dont have any tests yet for the resolve operation. that is a mistake I will have to correct. If anyone can define some tests next week,feel free to do so and stick them in the document. I am busy running the latest version of my client against the hp and the ourgrid endpoint. the NEC endpoint is not responding to me, either the firewall is blocking incoming calls or the endpoint is not there. The endpoint is successfuly demonstrating that my client stack does not implement timeouts, which is something I will have to fix. When I get back from my vacation we can start on the interop document. Something like -copy of the ogsa-dai interoperability intro -test methodology -different implementations -summary of tests (table) -test result summary (appendices containing the <junitreport> results or similar. Findings -we can demo interop between two implementations -things like WSRF, base faults and WS-N are problematic as they arent consistently implemented -SOAP stacks throw SOAPFaults when they feel like it, even if the spec mandates BaseFaults -Without a rigorous taxonomy of which fault to throw for which problem, its hard to be sure that different implementations fail for the expected reasons on any test that expectgs failure. -security. Its hard to turn security on during interop testing. -system load. testing clients can overload the server. -how do you test WS-Notification works if you are calling from behind a firewall? -discuss on what is the best model client-side. High latency links show the limitations of RPC as a metaphor, but what alternatives are there. State machine? Model the remote resources with proxy classes that cache (and refresh) properties? -unclear semantics of wsrf:Destroy in a fault tolerant world. Future enhancements to the deploy API that testing raises -the hostname hint is confusing and impossible to test. If you can't test it works, lets drop it. -it would be good to have an option on addFile to declare the URL to which a document 'pretends' to live -an alternate notification mechanism (jabber/xmpp?) that goes through firewalls. -WSRF property to list all wsrf properties on a resource Thoughts? -steve

1 0

Notes from the CDDLM meeting on 5/24/2006
by Milojicic, Dejan S (HP Labs) 24 May '06

24 May '06

Attendees: Flavio, Guilherme, Steve, Satish, Dejan 1. Update UFCG: Running the deployment API tests against Satish's endpoint. Tests ran for an hour, 36 tests. Some tests are failing, as documented in a separate email. This week, Flavio is implementing a component model test, also currently we did not have CDLs for these endpoints. Flavio requested them, mailing list may not be working well, one email from Monday has not arrived. HP: Endpoint is up and running, some things are working, others are failing. WS-Notification related things are failing, everything else should be working. Can talk to UFCG and vice versa. He is logging all failures, has this working on his machine, but does not work on others. Next step is t sit down with other endpoints and see why this is failing. We should coordinate this time, since it is time-consuming. The other things is going back to CDL implementation. That has been brought up at the mailing list. NEC: Completed around 50% tests against HP's endpoint, yesterday sent email to Flavio. Will work with him this week to complete this model component test and work with Steve to find out what exactly the problem was with the portal. That's it. 2. Steve: Everyone is pleased that we are making progress. This is obvious on the weekly basis. WS-Notification and remaining pieces of CDL are of highest concern at this moment. Guilherme: within a month we will have a better interoperability. We still need to modify CDL in order to have interoperability. Email was sent about important differences of CDL. Satish: complete deployment tests at least against UFCG until the end of this week, the next thing is to make sure that Steve's tests can talk to NEC endpoint. Common CDL is the main thing that everyone is talking about. Dejan: can we have the target that we more or less are done by the end of June. Dejan and Steve will be absent for the big part of July, so we are shooting to also complete the interoperability informational document in the same time frame. We will start writing this document within 2-3 weeks from now. Dejan will check with Hiro whether we can get one slot at the OGSA F2F to present our interoperability results. Steve: Thinking about turning on password security on his endpoint. Simple HTTP password authentication. Will it break on other sides? This will be the first test. Thanks, Dejan.

1 0

CDL parser - Import with Namespace doubts
by Rakesh Sahu 23 May '06

23 May '06

Hi Our CDL parser implementation is complete and is able to successfully pass all the test cases. But we still have some questions about "import with namespaces" part of the spec. As per the specs the namespace should be applied only to the toplevel element under cdl:configuration. This can cause problems during reference value resolutions. For now we are ignoring the namespaces during the lookup but this will cause problems in complex cases where there may be multiple imports and elements with the same name. My take is that normal XML namespace scoping rules should be applied - the namespace declaration should explicitly be applied to the top element and to all elements within the content of that element. Thanks Rakesh

2 1

Re: [cddlm] Sourceforge and a new CDL resolution bug in my implementation.
by Guilherme Mauro Germoglio Barbosa / Projeto Ourgrid - Projeto Ourgrid/LSD 22 May '06

22 May '06

Hi Steve, I think your implementation is correct (not the test). In the specification, the inheritance of elements algorithm (7.2.2.1) states: /7.2.2.1 Inheritance of Elements Inheritance of elements from a node n' to a node n is defined as follows: 1 Let N an empty node list. 2 For each child element e' of n' from the first element to the last element: 2.1 If the node n has child elements E = {e1, e2,...} that have the same name as e' 2.1.1 If N does not contain an element that has the same name as e': *2.1.1.1 For each element ei in E in the document order, append ei to the end of N. _Inherit attributes from e' to ei._* 2.2 Otherwise, append e' to the end of N. 3 For each child element e of the node n from the first element to the last element: 3.1 if the element e' does not have a child element that has the same name as e, append e to the end of N. 4 Replace n's children with the nodes in N./ And your code, based on what you said on the e-mail, does what was underlined above. Guilherme Steve Loughran wrote: > Sourceforge is back up, but you need to check out everything again in > a new directory tree. > > > Incidenally I just found a little thing my CDL impl. can't resolve > > > <cdl:cdl> > <cdl:configuration> > > <component> > <entry cdl:ref="todo"/> > </component> > > <c2 cdl:extends="component"> > <entry>something</entry> > </c2> > > </cdl:configuration> > <cdl:system> > <example cdl:extends="c2" /> > </cdl:system> > </cdl:cdl> > > This fails on my impl, with the entry being undefined. > > <example xmlns=""> > <entry cdl:ref="todo">something</entry> > </example> > > I'm clearly not scrubbing cdl:ref stuff when I extend an element, when > in reality I should strip that and cdl:refroot. > > This is now a new test: > /org/ggf/cddlm/files/cdl/valid/set_02_references/cddlm-cdl-2005-02-0023.xml > > > Its interesting to see that I can still find problems in my code while > writing more CDL files. If anyone has more example CDL Files, > especially those that work on their system, please add them as CDL > tests just to give us as broad a set of > tests as possible. > > -Steve > > > >

3 3

GGF17 Report
by Jun Tatemura 19 May '06

19 May '06

Dear CDDLM team, In the session, I have presented - current document status + CDL public comments - interoperability status - interoperability demo (NEC client to access NEC/UFCG/HP portals) followed by discussion for OGSA/EMS - basic POSIX components for EMS lead by Hiro. Please find the slides and note (by keisuke fukui) attached. Best Regards, Jun Tatemura /// CDDLM: /// Jun Tatemura+ about 10 attendees - CDL - Deployment API - Component Model Document Status - Two specs published - One spec in public comment period -- CDDLM CDL (- May 26) CDL Document Clarifications [1] Import namespace [2Typical use case of @cdl:lazy Question - Then what will happenwhen the user assigns a value to a property with @cdl:lazy? - Is this allowed? Answer - You (the user ) can assign a value - The reference to the property is resolved at deployment time, anyway - How this is interpreted by the component depends on the component (application specific) - Clarifying statements will be added to the spec Hiro: These two comments are answered already on the tracker? Jun: The first one is done. Not yet for the second. will do after the session. 9:15 Interoperability: Three Specs to Test CDL Interoperability Definition CDL Test Environment Toolkit for CDL Component Tests Test Document CDL Test Status Hiro: Do you think standard XML normalization doesn't solve the problem? Jun: An Example. When doing reference the same prefix can mean different namespace depending on the Hiro: This is a common problem to XML document. Jun: Yes. That's why this is not mentioned in CDL spec itself. The implementation needs to care to do correct namespace interpretation. Hiro: Is to compare many implementation can generate equivalent document? Jun: Yes. Hiro: Is there any test planned for deployment API? Jun: It addresses to component model, while CDL is more general. The test doesn't cover deployment API usage of CDDLM implementation. CDDLM Interoperability Our Test Approach Deployment API test 9:35 Deployment API Test Demo - NEC (Muse) - UFCG (Muse) - HP (There are less problems between NEC and UFCG because of this) Component Model Test HP Reference Implementation HP Testing Status UFCG Reference Implementation. Testing Issues - Security: need to restrict compoenents to deploy. - Need well known compoenenets & <cdl:import> URLs. - Without standardised failure codes for certain faults (e.g. unresolved reference) failure tests just test for failure Underlying spec issues - WS-A is complex - Faulting: SOAP Faults vs BaseFaults. SOAP 9:45 (So this is the status. Questions?) Hiro: What about the implementation from Stuart. Jun: It will come later. It might present new underlying problems since it is the Windows implementation. Hiro: Understood that CDL test doesn't use real application. Jun: Applications used for to verify the component model don't do real task. Document named "Component Test Model" is explained. (Question about application and component continues.) Keisuke: Is that box in the diagram a compoent resource or XML fragment? Jun: It is a compoenent resource. It can be Java code, but it is implementation dependent. 9:55 // Basic POSIX component for EMS // Hiro Kishimoto The Basic POSIX component is a generic component usable to deploy POSIX applications. Deployment Infrastructure BLAST CDL Document Deploy Component Jay: I don't want the use to specify all parameters when writing scripts to do transfer, copy, set attribute, and so on. Jun: Script needs to be somehow parameterized to use the value from CDL. Basic POISIX component - Identified "gap" between family of CDDLM specs and EMS services - Provide services -- Application binary installation -- File system mount -- Database setup (in case of BLAST) - Provide output (will be incorpoated in JSDL document) -- Application path -- Environment variables CDL document includes - 1. Application binary installation - 2. File system mount Hiro: I think Filesystem mount will be of value for many. Is there any plan to privde it? Jun: Not yet, I think. Basic POSIX template - Companion template to Basic posix component Next steps - Jointly develop BPC and BPT spec -- OGSA EMS & CDDLM-WG - More example than BLAST -- OGSA EMS & JSDL-WG - Put Job Manageer in this deployment picture - Put ACS in this picture - Advanced Common Component Hiro: How the components can be stored in ACS? Keisuke: One of the goal of ACS is to bundle a set of files for a task as a unit. When retrieving contents, one can do that at the granularity of the contents inside an archive. I can imagine components can be archived and later retrieve each. So I think it is possible with the current ACS at this moment. 10:20 End of session

2 1

Notes
by Milojicic, Dejan S (HP Labs) 18 May '06

18 May '06

Attended: Ayla, Flavio, Guilherme, Satish, and Dejan (Steve excused himself; Jun appears to be sick) Status and Plans for next week ----------------------------- Ayla, Flavio, Guilherme: finished component model last week, and now finished missing deployment API testing. They tested against Steve's portal, have completed 45%. Last week had slightly less but added more tests. With Satish have some problems with failures. Added some sleeps in between tests, to reduce stress on the Tomcat server on. Still the server sometimes goes down, they will work offline with Satish on this. Plans for this week: perform component model tests against other endpoints. Stuart has specified some CDL, and components and they will work on this. Also continue to work on deployment APIs against Steve and Satish's endpoint. From the email from Ayla, going into more details: **Implementation status: CDL - 100% done. Deployment API - 100% done. Component Model - 100% done. **CDL test case: Test cases 58/59 (98.3%). **Deployment API test case: Test with our portal (100%, 4 tests remaining were implemented) - it was 88.88% Test with NEC portal (14/36=38.88%. The server could not stand due to the load) Test with HP portal (45.16%) Stuart: Focusing internally on cleanup of the implementation that is not working. Still needs some time to sync-up with Jun. Still can't parse all test cases and when they fix some of them other problems pop up. Deployment APIs is up and running. 20% passing. Started testing on other end-point with others, but gave up, still focusing on internal. Still not have endpoints, working with IT to make it happen. For the next week, if he can not digest CDL, he will post it to Jun and the group. Satish: component model and CDL complete 100%. Received CDL from Guilherme and will complete tests against UFCG implementation. 30% have been completed against HP's portal, but non component model so far. Regarding the system load on Tomcat, after creating 8 or 9 systems, the JVM can not take any more load on Tomcat and after initializing 3 or 4 systems, the system has been fully loaded. Need to work on how to resolve this issue. Need to complete deployment tests against UFCG portal and what are the CDL tests that they need. Will also help Stuart. Report from GGF17 ---------------- GGF17: Satish spoke to Jun for about 2 minuets. GGF went very well and he could show the interoperability demo, wanted to show portal implementation and a lot of other things but he did not have too much time. Did the demo for 15 minutes on interoperability. Hiro's submission to SDD ------------------------ Hiro presented submission of combined ACS, CDDLM, and JSDL to SDD. Presentation went well, there was an extensive set of slides presented. It is unclear how it will move forward. Hiro will keep us in the loop. Overall sense of where they are with completeness of Interoperability --------------------------------------------------------------------- (In the sense of completeness of tests against one other endpoint) UFCG: 30-40% Some tests are not suitable for interoperability. However, the tests are good. We may want to declare some tests as not required for interoperability, make it very clear. NEC: Around 40%, the issue is the same as Ayla described, using your own test, but without the CDL files from other implementation, you can not run any of the tests. We should have a single common CDL file that each implementation can support so that we can freely use the tests against any target. Softricity: ~20%. Action items: ------------- 1. Jun to report from GGF 2. Satish and Ayla will make an informal proposal how to decide on which tests go to interoperability and how to have consolidated CDLs. Thanks, Dejan.

3 2

Interoperability testing problems
by Ayla Debora Dantas de Souza - Projeto Ourgrid 18 May '06

18 May '06

Hi all, while performing interoperability tests, we have noticed that it is not possible to test others endpoint simply changing the Portal EPR. Besides, we must also use CDLs that work on the given implementation, and configure our tests with those CDLs. This happens because CodeBase and CommandPath are used in different ways between the implementations, as the Component Model allows that. Therefore, in order to perform some of the tests, we need CDLs that work on the each others implementation, or we need in the Test Plans a definition of a CDL that should work in all of them. For now, we have interchanged some CDLs with Satish to test the NEC implementation. Is this the correct way to proceed? We were not sure if this is a good way to test interoperability. Besides, we wanted to know what is the definition of interoperability we will use in order to show that two different implementations interoperate. Thanks, Ayla

2 2

OASIS Solution Deployment Descriptor TC
by Hiro Kishimoto 18 May '06

18 May '06

Hi all, OASIS OASIS Solution Deployment Descriptor TC is developing XML schema for deployment related metadata of multiple software packages (they call solution). Their scope is widely overlapped with CDDLM, ACS, JSDL, and OGSA-EMS. In order to prevent from undesirable divergence, Keisuke Fukui (ACS-WG co-chair) and I have attended to SDD TC F2F meeting this week, introduced our CDDLM/ACS/JSDL specs and ask them to utilize these GGF specifications if appropriate. http://www.oasis-open.org/committees/download.php/18091/ggf-sdd-submission.… http://www.oasis-open.org/committees/download.php/18167/GGF-spec-submission… The F2F meeting went very well and SDD TC agreed that although they chose IUDD V2 specification as their base specification, they will align and converge with ACS/JSDL/CDDLM specs in long team and also short term. It is very good news for both GGF and OASIS. http://www.oasis-open.org/committees/download.php/18210/SDDF2F.html http://www.oasis-open.org/committees/download.php/18208/Strategy%20for%20De… http://www.oasis-open.org/committees/download.php/18209/Deployment%20Descri… If time permits, I will report out and answer questions on this new thrilling opportunity at Thursday's OGSA-WG weekly call. Thanks, -- Hiro Kishimoto

1 0

Sourceforge and a new CDL resolution bug in my implementation.
by Steve Loughran 16 May '06

16 May '06

Sourceforge is back up, but you need to check out everything again in a new directory tree. Incidenally I just found a little thing my CDL impl. can't resolve <cdl:cdl> <cdl:configuration> <component> <entry cdl:ref="todo"/> </component> <c2 cdl:extends="component"> <entry>something</entry> </c2> </cdl:configuration> <cdl:system> <example cdl:extends="c2" /> </cdl:system> </cdl:cdl> This fails on my impl, with the entry being undefined. <example xmlns=""> <entry cdl:ref="todo">something</entry> </example> I'm clearly not scrubbing cdl:ref stuff when I extend an element, when in reality I should strip that and cdl:refroot. This is now a new test: /org/ggf/cddlm/files/cdl/valid/set_02_references/cddlm-cdl-2005-02-0023.xml Its interesting to see that I can still find problems in my code while writing more CDL files. If anyone has more example CDL Files, especially those that work on their system, please add them as CDL tests just to give us as broad a set of tests as possible. -Steve

1 0