I have some comments as well, -inline Tino Vazquez wrote:
Hi Richard,
comments interspersed,
On Thu, Oct 1, 2009 at 10:40 PM, Richard Davies <richard.davies@elastichosts.com> wrote:
First of all, congratulations to Ignacio, Tino and the UCM team on producing the first implementation of an OCCI draft - real progress and good work! I wasn't on the call, but have had a quick look through the links which Ignacio sent.
Thanks a lot. *blush*
Working with the OpenNebula for the past few days, its great progress on the first OCCI implementation. :-)
One thing which this brings home to me is that there is still significant work to tie down unresolved areas the OCCI draft so that implementations will actually be interoperable.
This isn't to criticize the UCM team at all - they have done a good job working from a draft specification and invented plausible mechanisms to fill gaps where they have found them.
Nevertheless, I can see quite a few areas in the their draft specs which clearly (& necessarily) go beyond what the Sep 21 specification draft (the last which I read) had actually defined.
Indeed, we took the liberty of implementing things that weren't present in the OCCI specification in order to dynamize the discussions. In any case, we don't regard our implementation as final (not even close) and we are open to make the neccessary changes to comply as much as possible with the specification (always taking our possibilities into account).
Taking one example from their site:
<COMPUTE> <NAME>MyCompute</NAME> <STORAGE> <SWAP size="1024" dev="sda2"/> <DISK image="ab5c9770-7ade-012c-f1d5-00254bd6f386" dev="sda1"/> <FS size="512" format="ext3" dev="sda3"/> </STORAGE> <NETWORK> <NIC network="23" ip="192.168.0.9"/> </NETWORK> </COMPUTE>
This uses XML nesting to specify when storage and network resources and linked into a compute resource, and has solved several issues such as the local devices to be used for the storage resources and naming of interfaces.
By contrast, the Sep 21 specification suggests having separate entries for each resource, and linking them together using mechanisms similar to the HTTP Link header (with no example given). The spec doesn't yet say anything about how local devices should be specified.
I agree that the specification should indicate how to link storage and network to virtual machines more clearly.
The OCCI document was initially intended to be laid out like an atom doc. One big long document, in its rudimentary form. This brought up issues with any restful implementation and the impacts of document dynamic data (attribute values) and the ability to take advantage of web caches. The notion of links were introduced to mediate restful principles, but there was a great assumption that everyone's interpretation of linking and attribute layout was consistent to atom implementation. This information never made it into the occi documentation. here is a sample of the atom from wikipedia <entry> <title>Atom-Powered Robots Run Amok</title> <link href="http://example.org/2003/12/13/atom03" /> <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id> <updated>2003-12-13T18:30:02Z</updated> <summary>Some text.</summary> </entry> This is Richard Davies' version from an email in April: <server> <id>decca5a5-8952-4004-9793-cdbbf05c3c63</id> <title>Debian GNU/Linux 5.0 Virtual Appliance</title> <summary>Base installation of Debian GNU/Linux 5.0</summary> <cpu>2</cpu> <mem>4Gb</mem> <disk id="file1" href="virtual-disk.vmdk" size="148251374"/> <nic>2</nic> <state>RUNNING</state> <meter rate="0.10" currency="USD" unit="hours">35.27</meter> <monitor type="cpu">75.2%</monitor> <monitor type="mem">1059374258</monitor> <storage id="4696b561-a253-42b4-bd27-7aa4950e0a60"/> <network id="45a73b80-c957-4ae1-97c6-b70652eba1d1"/> </server> Here is Sam Johnston's version from one of the docs showing links: <entry> <id>urn:uuid:47bb7df8-587e-47fa-bd89-6f2f81c14b19</id> <title>Virtual Machine #1</title> <summary>Sample Compute Resource</summary> <updated>2009-05-04T09:52:37Z</updated> <link href="/47bb7df8-587e-47fa-bd89-6f2f81c14b19" /> <link rel="http://purl.org/occi/storage#device" href="urn:uuid:4cc8cf62-69a4-4650-9e8c-7d4c516884df" title="Hard Drive"/> <link rel="http://purl.org/occi/network#interface" href="urn:uuid:dc88b244-145f-49e4-be7c-0880dcad42e9" title="Internet Connection"/> <link rel="http://purl.org/occi/network#interface" href="urn:uuid:253d83dd-e417-4e1f-9958-8c0a63120475" title="Private Network"/> </entry> In these examples, you can clearly see each occi attribute is commonly defined in its own XML element, more like a property than an attribute. The link is clearly represented differently, in an alternate form. The use of alternate forms would not be used for occi in order to reduce complexity. It looks like the linking adds a significantly more complexity to the document and the client. My Opinion on this matter: I think there is confusion on the how terminology is applied to the occi document and technologies like XML. An occi attribute is NOT an XML attribute, at least not yet. In both Sam's and Richard's examples, neither clearly shows that occi attributes are defined as XML elements or XML attributes. In both Richard's and SAM's forms, the application of occi attributes to XML are inconsistent leading to significant confusion on how to implement. This may be a mute point for the next few weeks, since we are moving forward with a text representation in the occi specification.
Not sure what you mean by local devices, could you please elaborate more on this?
Another example: UCM do not use any extra management verbs. http://www.ogf.org/pipermail/occi-wg/2009-September/001235.html By contrast the Sep 21 spec hints that some will exist, but does not define any of them.
You are rigth, the verbs should be specified. We chose another mechanism using updates through PUT to achieve the same thing (more RESTful in my opinion, but also suffering from it, as changing a state is not always guaranteed)
Whether the implementation is restful will be determined by the use and control of web caches. I don't think you can get away with RESTful principles without managing document/data consistency at the web caches. Once cache management is implemented it shouldn't be too difficult to improve document/data consistency. The more I look into RESTful principles and implementations, I seem to be drawing to the conclusion RESTful principles has little to no value for small documents as seen here with occi and other web apis. I'm also beginning to challenge the gains of implementing web cache controls in comparison to native occi request traffic loads.
My feeling is that the main areas where the UCM team have had to innovate their own solutions are very similar to the list of areas which I had previously highlighted as needing resolution in my mail: http://www.ogf.org/pipermail/occi-wg/2009-September/001215.html
However, I'd also be keen for the UCM team themselves to write down a list of the areas where they believe that they had to go beyond the actual written OCCI drafts to produce something workable.
From our experience, the spec needs to define clearly the states that a COMPUTE resource can be in. From the last conf call the general feeling was that it would be very difficult to agree on one state machine diagram. I vouche for trying to agree on a minimal one, or at least, as Sam suggested, have a registry were the states are well defined. We will be then be happy to implement the mangement verbs instead of relying in the PUT method we currently use.
Other issue is aligned with something you point out in your mail, that is, the attributes of the resources. We completely agree with the idea of adding units (as OVF does), although I don't know which will be the best to do so (a string separated by a space with "numbers units", or an extra attribute for each attribute that needs units to be defined?). This will make the spec more flexible.
I think units are the best bet, they do not exclude using raw numeric values. I do have a concern using raw numeric values with large volumes. Volumes in terms of TB, PB or EB, may not be able to be generated or parsed by many web tools, libraries and languages.
Also, the state should be part of the COMPUTE attributes (in fact, we added it in our implementation because we needed it to update the COMPUTE state).
About the decission of theformat of the representations I think it will be shooting our own foot to ask implementators to support each three formats, even if we do so for interoperabilities sake. We need to chose between one or take into consideration Sam's proposal of using HTTP headers for the metadata.
I believe we should pick ONE format for now. I believe Thijs acting as chairman, mandated text only for this version of the document. I'm not convinced Sam's proposal is the one we should be using. If RESTful principals appear to be impracticable or do not provide any gains and if Sam's proposal brings us to rpc efficiencies, shouldn't we move to a more standard RPC or SOA/SOAP mechanism instead of reinventing the wheel ?
And I'd be equally keen for Sam to take a careful look through the UCM links below, identify every place that the UCM implementation doesn't look like what he had expected and tighten the draft in these areas to either the UCM solution or whatever he had intended to write but not yet written.
Cheers,
Richard.
Regards,
-Tino
cheers, gary
Ignacio Martin Llorente wrote:
Hi,
Because we are now (at 16:00 CEST) discussing the OpenNebula OCCI implementation, I am sending some links with relevant info:
- OpenNebula OCCI API Specification: http://www.opennebula.org/doku.php?id=documentation:rel1.4:occidd - Usage Guide: http://www.opennebula.org/doku.php?id=documentation:rel1.4:occiug
Cheers,
-- Ignacio M. Llorente, Full Professor (Catedratico): http://dsa-research.org/llorente DSA Research Group: web http://dsa-research.org and blog http://blog.dsa-research.org OpenNebula Open Source Toolkit for Cloud Computing: http://www.OpenNebula.org RESERVOIR European Project in Cloud Computing: http://www.reservoir-fp7.eu
-- Constantino Vázquez, Grid Technology Engineer/Researcher: http://www.dsa-research.org/tinova DSA Research Group: http://dsa-research.org Globus GridWay Metascheduler: http://www.GridWay.org OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg