Re: [occi-wg] confusion about status of link / headers
Adrian, We don't vote... we discuss our way to consensus and escalate if we fail. Anyway I've had enough for one day/lifetime and I've got a house full of furniture to build tonight. I do wonder how many people here have actually bothered to read the spec and understand the issues though - if you haven't already then please do so now and speak up if anything doesn't make sense. Sent from my iPhone On 20/10/2009, at 18:44, Adrian Cole <adrian@jclouds.org> wrote:
As we aren't introducing new information, I think we are at a point where a vote can take place. This issue isn't infinitely complex, and reasonable to model in choices. OCCI desires to be a community process, so let's see where we fall on this.
Do you agree? -Adrian
On Tue, Oct 20, 2009 at 9:39 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Sam,
On Tue, Oct 20, 2009 at 5:31 PM, Sam Johnston <samj@samj.net> wrote:
Alexis,
Sweeping, unjustified statements are intensely frustrating, not to mention unhelpful in the absence of a coherent alternative. If people choose not to disclose conflicts I happen to be aware of then you bet I'll disclose them for them - nothing personal about it.
I don't know what 'conflicts' you are referring to, or how these relate to the matter at hand. If by 'sweeping, unjustified' you refer to the word 'unprecedented' then I shall justify it by pointing out that - at least as I currently understand it - you are proposing use of headers to a *greater* extent than has been done before and to cover a *greater* set of use cases.
Cutting to the chase, both Microsoft and Amazon are using headers for metadata to the same extent as what has been proposed.
Really - to the "same" extent? That seems contrary to what has been asserted on this thread previously.
What premature optimisation? *Not* using XML? Do we really need to drag that carcass out again? Surely Adrian's performance tests put the last nail in that particular coffin.
Premature optimisation = making performance a critical design concern when we want an API.
I have noted Adrian's points as have others.
The question really is about whether we want to be an interface/ protocol like HTTP or some mongrel hybrid of that along with a set of XML, JSON and/or text formats for everything we ever intend to model (knowing that work has already been done by others).
Work that has been done by others and shown to work, is a good thing.
alexis
Sent from my iPhone
On 20/10/2009, at 18:10, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Sam
Cut the personal stuff please.
Re 'unproven', I think the issues people are having are:
* The extent, scope and purpose of your application of headers to metadata is unprecedented - as I understand it. Nobody is saying it has not been done in other and restricted cases.
* Premature optimisation.
alexis
On Tue, Oct 20, 2009 at 4:44 PM, Sam Johnston <samj@samj.net> wrote:
Gary,
Please be specific with your objections so as I can have some hope of answering them... "several other emails" does not suffice as to the best of my knowledge there are no outstanding issues that I have not comprehensively and convincingly dealt with.
Your repeated assertions than HTTP headers are... what is it now... "unproven"... are demonstrably unfounded (refer to the various cloud APIs referred to by myself and Adrian, including both MS Azure and Amazon S3, as well as the history of the Internet in general) and given the frequency, bordering on FUD.
Regarding performance, OCCI should strive to be as efficient as possible as it primarily targets public cloud providers who operate at arbitrarily large "cloud" scale. The existing specification is *extremely* efficient and alternatives would at least want to be in the same neighbourhood. Efficiency may be an inconvenient factor for whatever your alternative is, but it's critical for many of our use cases.
Finally, you reject OVF without providing any alternative beyond building a replacement from scratch ourselves, after our second and final deadline has just sailed by no less. Have you considered the implementation cost of this? And in the same breath you accuse my proposal of being "unproven"? Give me a break...
Correct me if I'm wrong but the way I see it is that you want to revert the last 2-3 months of my work because you have an investment in an old version of the spec. With each day I spend on OCCI costing me well over €1,000 excuse me for being pissed about someon e with such a blatantly obvious conflict attacking my work and refusing to provide a coherent argument, particularly with no "camera ready" alternative anywhere in sight.
Sent from my iPhone
On 20/10/2009, at 16:51, Gary Mazz <garymazzaferro@gmail.com> wrote:
I've understood Sam's proposal from the beginning. I have had issue with from the beginning.
Although I find it very interesting and valuable in terms of economy, I have issue with it as the OCCI's primary interface implementation for a few reasons which have been outlined in several other emails.
It appears that Sam has created a new technology, a lower cost RPC mechanism, which is unproven for both small and wide scale deployment. For interoperability, IMO, occi should focus on proven mainstream technologies as the primary interfaces. I am not suggesting, shelving or not supporting Sam's proposal, based on market acceptance, it may prove to be one of many occi interface implementations.
As for cost saving in interface technologies, I do not buy into the idea that the occi traffic will be so overwhelming that there is a requirement to maximally optimize http requests and responses. IMO, in comparison to images and consumer data occi traffic will be insignificant. I think we are worrying about spilled milk while the house is burning.
There has been some discussion on supporting technologies including OVF. Whether occi supports OVF has not been fully discussed, examined or any consensus reached. I believe adopting technologies like OVF will only lead to mapping issues between occi attributes and the external formats creating a long term management headache.
cheers, gary
Alexis Richardson wrote: > OK > > Is Sam's statement something that we can achieve consensus > around? > > a > > > On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> > wrote: > >> Alexis, >> >> Seems our posts crossed in the mail. I hope the last one clears >> things up >> but the fact that you're using OVF and XML/JSON interchangeably >> suggests >> that you [are possibly one of many who] don't grok the >> proposal. >> Worse, it >> seems you've got it back to front in thinking of the "core" as >> metadata, >> which is not at all what I had intended. >> >> The focus is on the resources and their various >> representations. >> The >> metadata (that is, data about data) essentially provides >> whatever is >> necessary to "glue" these resources together. It is basically >> using HTTP as >> the meta-model rather than an envelope format like Atom/SOAP >> or, >> worse, a >> meta-model that we have to create ourselves from scratch. >> >> So it's more like: >> >> 1) OCCI is compatible with ALL existing formats, eg OVF >> >> 2) Existing formats are used are used for ALL representations >> >> 3) ALL additional metadata should be in the headers >> >> 4) ALL of the core stuff should be in the body (that is, the >> metadata in the >> headers is just there to support the data and wire it up in >> useful >> ways) >> >> I think the confusion stems from your assuming that there is >> still >> XML/JSON/Atom/etc. representations. There aren't. HTTP headers >> provide a >> single, performant, interoperable metadata channel that has >> already been >> both standardised and extensively implemented. >> >> If you've followed me this far then you should have realised >> that >> every HTTP >> user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, >> etc.) >> is already >> a basic OCCI client out-of-the-box. The result is that 5-line >> scripts like >> demo.py are functional with no dependence on client libraries >> (that would >> absolutely be necessary if you were to start trying to define >> your >> own >> formats). >> >> Sam >> >> PS I'm sorry my "bunk" comment came across as referring to you >> personally - >> it was directed at the argument. >> >> On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson >> <alexis.richardson@gmail.com> wrote: >> >>> I think what Sam is proposing is the following: >>> >>> 1) If OCCI can handle 'any' data representation, eg OVF, XML, >>> JSON, >>> then it needs some core / common model. Otherwise: there can >>> be no >>> defined behaviour / interop >>> >>> 2) This core / common model is ALL metadata >>> >>> 3) ALL this metadata should be in the headers >>> >>> 4) ALL the non core stuff eg OVF payload, XML representation, >>> should be in >>> body >>> >>> Sam - is that right? >>> >>> a >>> >>> >>> On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> >>> wrote: >>> >>>> On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX >>>> <andrewx.edmonds@intel.com> wrote: >>>> >>>>> To that point I'd ask - If we're talking about meta-data >>>>> that >>>>> infers >>>>> there >>>>> is some data to accompany it. Where are the examples of this >>>>> data? This >>>>> would help in forming the full picture. >>>>> >>>> Here, from the core spec (using OVF as an example, but >>>> could be >>>> anything). >>>> Request is green, response metadata is yellow & response >>>> data/ >>>> payload is >>>> orange: >>>> >>>> >>>>> GET /us-east/webapp/vm01 HTTP/1.1 >>>>> User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0 >>>>> Host: cloud.example.com >>>>> Accept: */* >>>>> >>>> < HTTP/1.1 200 OK >>>> < Date: Sat, 10 Oct 2009 12:56:51 GMT >>>> >>>> < Content-Type: application/ovf >>>> < Link: </us-east/webapp/vm01;start>; >>>> >>>> < rel="http://purl.org/occi/action/start"; >>>> < title="Start" >>>> >>>> < Link: </us-east/webapp/build.pdf>; >>>> < rel="related"; >>>> >>>> < title="Documentation"; >>>> < type="application/pdf" >>>> >>>> < Category: compute; >>>> < label="Compute Resource"; >>>> >>>> < scheme="http://purl.org/occi/kind/" >>>> < Server: occi-server/1.0 (linux) OCCI/1.0 >>>> >>>> < Connection: close >>>> < >>>> < <?xml version="1.0" encoding="UTF-8"?> >>>> >>>> < <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema- >>>> instance" >>>> < xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/ >>>> 1" >>>> >>>> < xmlns="http://schemas.dmtf.org/ovf/envelope/1" >>>> < xml:lang="en-US"> >>>> >>>> ... >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: occi-wg-bounces@ogf.org [mailto:occi-wg- >>>>> bounces@ogf.org] On >>>>> Behalf >>>>> Of Alexis Richardson >>>>> Sent: 20 October 2009 09:49 >>>>> To: Sam Johnston >>>>> Cc: Tim Bray; occi-wg@ogf.org >>>>> Subject: Re: [occi-wg] confusion about status of link / >>>>> headers >>>>> >>>>> Sam, >>>>> >>>>> Please don't throw around words like "bunk" willy nilly. >>>>> >>>>> I have just had a look at >>>>> http://www.rackspacecloud.com/cloud_hosting_products/ >>>>> servers/ >>>>> api >>>>> >>>>> I can see some limited use of headers for metadata in a few >>>>> places. >>>>> That's never been controversial. But the understanding >>>>> that I >>>>> have of >>>>> your proposal is to use headers wholesale for all metadata. >>>>> Rackspace >>>>> don't appear to do that, unless I have misunderstood their >>>>> document. >>>>> I also may have misunderstood your proposal. If nobody >>>>> understands >>>>> your proposal except you, then it will not be possible to >>>>> gain >>>>> consensus around it. >>>>> >>>>> Are you saying "OCCI metadata should be more like Rackspace >>>>> metadata"? >>>>> >>>>> alexis >>>>> >>>>> >>>>> On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston >>>>> <samj@samj.net> >>>>> wrote: >>>>> >>>>>> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson >>>>>> <alexis.richardson@gmail.com> wrote: >>>>>> >>>>>>> What about compute clouds? >>>>>>> >>>>>> Rackspace Cloud API for a start: >>>>>> >>>>>> >>>>>>> HTTP/1.1 204 No Content >>>>>>> Date: Mon, 12 Nov 2007 15:32:21 GMT >>>>>>> Server: Apache >>>>>>> X-Server-Management-Url: >>>>>>> https://servers.api.rackspacecloud.com/v1.0/35428 >>>>>>> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>>>>> X-CDN-Management-Url: >>>>>>> https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>>>>> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb >>>>>>> Content-Length: 0 >>>>>>> Content-Type: text/plain; charset=UTF-8 >>>>>>> >>>>>> Microsoft Azure uses headers for "attributes" as well, in >>>>>> the >>>>>> same >>>>>> way >>>>>> as I >>>>>> had originally proposed: >>>>>> >>>>>> x-ms-meta-name: value Returns a metadata value for the >>>>>> container. >>>>>> >>>>>> However prefer that the header names are static/opaque >>>>>> rather >>>>>> than >>>>>> using >>>>>> a >>>>>> template and parsing them: >>>>>> >>>>>> >>>>>>> Attribute: name=value >>>>>>> >>>>>> This to me is cleaner and more self-explanatory, plus it is >>>>>> easily >>>>>> collapsed >>>>>> ala: >>>>>> >>>>>> >>>>>>> Attribute: name1=value1, name2=value2, ... namen=valuen >>>>>>> >>>>>> Anyway suffice to say that claims this is somehow >>>>>> "experimental" are >>>>>> bunk. >>>>>> >>>>>> Sam >>>>>> >>>>>> >>>>>>> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org >>>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>>> Here are options for metadata used in some of the major >>>>>>>> storage >>>>>>>> clouds >>>>>>>> FWIW: >>>>>>>> >>>>>>>> S3, Rackspace, EMC Atmos, Azure - Headers >>>>>>>> Nirvanix - query params in, xml entity out >>>>>>>> Mezeo - entity >>>>>>>> >>>>>>>> Of the ones using headers, S3, Rackspace and Azure use >>>>>>>> prefix with >>>>>>>> values stored as-us. Atmos joins all metadata together >>>>>>>> into >>>>>>>> one >>>>>>>> header, making parsing trivial (split /,/), but >>>>>>>> necessary. >>>>>>>> >>>>>>>> The most expensive option of the above is entity, where >>>>>>>> each >>>>>>>> metadata >>>>>>>> value is a separate GET. However, entities allow binary >>>>>>>> metadata >>>>>>>> and >>>>>>>> zero restrictions on it, which may be useful. >>>>>>>> >>>>>>>> In jclouds, we time parsing of response values. A simple >>>>>>>> XML doc >>>>>>>> with >>>>>>>> only several elements written in SAX takes a few ms to >>>>>>>> parse. My >>>>>>>> log >>>>>>>> files are not precise enough to find the overhead in >>>>>>>> parsing >>>>>>>> headers: >>>>>>>> they always start and finish within the same millisecond. >>>>>>>> >>>>>>>> I hope this background helps, and also helps explain >>>>>>>> why I'm >>>>>>>> vocal >>>>>>>> on >>>>>>>> such topics such as headers vs entities :) >>>>>>>> >>>>>>>> Cheers, >>>>>>>> -Adrian >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston >>>>>>>> <samj@samj.net> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >>>>>>>>> <garymazzaferro@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> The http header and key/value pairs need to parsed >>>>>>>>>> also, >>>>>>>>>> there >>>>>>>>>> is >>>>>>>>>> no >>>>>>>>>> free >>>>>>>>>> ride here. >>>>>>>>>> >>>>>>>>> Every HTTP library I have ever used parses HTTP >>>>>>>>> headers and >>>>>>>>> puts >>>>>>>>> them >>>>>>>>> in a >>>>>>>>> nice hash for you ready to consume. If we go for "Name: >>>>>>>>> Value" >>>>>>>>> then >>>>>>>>> that's >>>>>>>>> all there is to it. If we go for "Attribute: >>>>>>>>> name=value" as >>>>>>>>> is >>>>>>>>> currently >>>>>>>>> proposed (which is arguably cleaner, follows cookies' >>>>>>>>> "prior art" >>>>>>>>> and >>>>>>>>> avoids >>>>>>>>> Amazon's prefix hack) then you just have to split on >>>>>>>>> '='. >>>>>>>>> >>>>>>>>> To illustrate how clean this is by example: >>>>>>>>> >>>>>>>>> >>>>>>>>>> #!/usr/bin/python >>>>>>>>>> import urllib2 >>>>>>>>>> response = urllib2.urlopen('http://cloud.example.com/ >>>>>>>>>> myvm') >>>>>>>>>> representation = response.read() >>>>>>>>>> metadata = response.info() >>>>>>>>>> print metadata['occi-compute-cores'] >>>>>>>>>> >>>>>>>>> As soon as you start talking about payloads you have to >>>>>>>>> fire up a >>>>>>>>> parser >>>>>>>>> (JSON/XML/Atom/etc.) or write your own (previous text >>>>>>>>> rendering) >>>>>>>>> which >>>>>>>>> is >>>>>>>>> significantly more work to do at both design and run >>>>>>>>> times. >>>>>>>>> Not >>>>>>>>> to >>>>>>>>> mention >>>>>>>>> more work for us to do now and more scope for >>>>>>>>> interoperability >>>>>>>>> problems. >>>>>>>>> >>>>>>>>> Sam >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> occi-wg mailing list >>>>>>>>> occi-wg@ogf.org >>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> occi-wg mailing list >>>>>>>> occi-wg@ogf.org >>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>>> >>>>>>>> >>>>>> >>>>> _______________________________________________ >>>>> occi-wg mailing list >>>>> occi-wg@ogf.org >>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>> --- >>>>> ---------------------------------------------------------- >>>>> Intel Ireland Limited (Branch) >>>>> Collinstown Industrial Park, Leixlip, County Kildare, >>>>> Ireland >>>>> Registered Number: E902934 >>>>> >>>>> This e-mail and any attachments may contain confidential >>>>> material for >>>>> the sole use of the intended recipient(s). Any review or >>>>> distribution >>>>> by others is strictly prohibited. If you are not the >>>>> intended >>>>> recipient, please contact the sender and delete all copies. >>>>> >>>> >> > _______________________________________________ > occi-wg mailing list > occi-wg@ogf.org > http://www.ogf.org/mailman/listinfo/occi-wg > >
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
Ahh, ok (sorry... still an OCCI newbie). Disregard my last call to arms. -Adrian On Tue, Oct 20, 2009 at 9:56 AM, Sam Johnston <samj@samj.net> wrote:
Adrian,
We don't vote... we discuss our way to consensus and escalate if we fail.
Anyway I've had enough for one day/lifetime and I've got a house full of furniture to build tonight. I do wonder how many people here have actually bothered to read the spec and understand the issues though - if you haven't already then please do so now and speak up if anything doesn't make sense.
Sent from my iPhone
On 20/10/2009, at 18:44, Adrian Cole <adrian@jclouds.org> wrote:
As we aren't introducing new information, I think we are at a point where a vote can take place. This issue isn't infinitely complex, and reasonable to model in choices. OCCI desires to be a community process, so let's see where we fall on this.
Do you agree? -Adrian
On Tue, Oct 20, 2009 at 9:39 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Sam,
On Tue, Oct 20, 2009 at 5:31 PM, Sam Johnston <samj@samj.net> wrote:
Alexis,
Sweeping, unjustified statements are intensely frustrating, not to mention unhelpful in the absence of a coherent alternative. If people choose not to disclose conflicts I happen to be aware of then you bet I'll disclose them for them - nothing personal about it.
I don't know what 'conflicts' you are referring to, or how these relate to the matter at hand. If by 'sweeping, unjustified' you refer to the word 'unprecedented' then I shall justify it by pointing out that - at least as I currently understand it - you are proposing use of headers to a *greater* extent than has been done before and to cover a *greater* set of use cases.
Cutting to the chase, both Microsoft and Amazon are using headers for metadata to the same extent as what has been proposed.
Really - to the "same" extent? That seems contrary to what has been asserted on this thread previously.
What premature optimisation? *Not* using XML? Do we really need to drag that carcass out again? Surely Adrian's performance tests put the last nail in that particular coffin.
Premature optimisation = making performance a critical design concern when we want an API.
I have noted Adrian's points as have others.
The question really is about whether we want to be an interface/ protocol like HTTP or some mongrel hybrid of that along with a set of XML, JSON and/or text formats for everything we ever intend to model (knowing that work has already been done by others).
Work that has been done by others and shown to work, is a good thing.
alexis
Sent from my iPhone
On 20/10/2009, at 18:10, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Sam
Cut the personal stuff please.
Re 'unproven', I think the issues people are having are:
* The extent, scope and purpose of your application of headers to metadata is unprecedented - as I understand it. Nobody is saying it has not been done in other and restricted cases.
* Premature optimisation.
alexis
On Tue, Oct 20, 2009 at 4:44 PM, Sam Johnston <samj@samj.net> wrote:
Gary,
Please be specific with your objections so as I can have some hope of answering them... "several other emails" does not suffice as to the best of my knowledge there are no outstanding issues that I have not comprehensively and convincingly dealt with.
Your repeated assertions than HTTP headers are... what is it now... "unproven"... are demonstrably unfounded (refer to the various cloud APIs referred to by myself and Adrian, including both MS Azure and Amazon S3, as well as the history of the Internet in general) and given the frequency, bordering on FUD.
Regarding performance, OCCI should strive to be as efficient as possible as it primarily targets public cloud providers who operate at arbitrarily large "cloud" scale. The existing specification is *extremely* efficient and alternatives would at least want to be in the same neighbourhood. Efficiency may be an inconvenient factor for whatever your alternative is, but it's critical for many of our use cases.
Finally, you reject OVF without providing any alternative beyond building a replacement from scratch ourselves, after our second and final deadline has just sailed by no less. Have you considered the implementation cost of this? And in the same breath you accuse my proposal of being "unproven"? Give me a break...
Correct me if I'm wrong but the way I see it is that you want to revert the last 2-3 months of my work because you have an investment in an old version of the spec. With each day I spend on OCCI costing me well over €1,000 excuse me for being pissed about someon e with such a blatantly obvious conflict attacking my work and refusing to provide a coherent argument, particularly with no "camera ready" alternative anywhere in sight.
Sent from my iPhone
On 20/10/2009, at 16:51, Gary Mazz <garymazzaferro@gmail.com> wrote:
> I've understood Sam's proposal from the beginning. I have had > issue > with from the beginning. > > Although I find it very interesting and valuable in terms of > economy, I have issue with it as the OCCI's primary interface > implementation for a few reasons which have been outlined in > several > other emails. > > It appears that Sam has created a new technology, a lower cost > RPC > mechanism, which is unproven for both small and wide scale > deployment. For interoperability, IMO, occi should focus on > proven > mainstream technologies as the primary interfaces. I am not > suggesting, shelving or not supporting Sam's proposal, based on > market acceptance, it may prove to be one of many occi interface > implementations. > > As for cost saving in interface technologies, I do not buy into > the > idea that the occi traffic will be so overwhelming that there > is a > requirement to maximally optimize http requests and responses. > IMO, > in comparison to images and consumer data occi traffic will be > insignificant. I think we are worrying about spilled milk while > the > house is burning. > > There has been some discussion on supporting technologies > including > OVF. Whether occi supports OVF has not been fully discussed, > examined or any consensus reached. I believe adopting > technologies > like OVF will only lead to mapping issues between occi attributes > and the external formats creating a long term management > headache. > > cheers, > gary > > > Alexis Richardson wrote: >> OK >> >> Is Sam's statement something that we can achieve consensus >> around? >> >> a >> >> >> On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> >> wrote: >> >>> Alexis, >>> >>> Seems our posts crossed in the mail. I hope the last one clears >>> things up >>> but the fact that you're using OVF and XML/JSON interchangeably >>> suggests >>> that you [are possibly one of many who] don't grok the >>> proposal. >>> Worse, it >>> seems you've got it back to front in thinking of the "core" as >>> metadata, >>> which is not at all what I had intended. >>> >>> The focus is on the resources and their various >>> representations. >>> The >>> metadata (that is, data about data) essentially provides >>> whatever is >>> necessary to "glue" these resources together. It is basically >>> using HTTP as >>> the meta-model rather than an envelope format like Atom/SOAP >>> or, >>> worse, a >>> meta-model that we have to create ourselves from scratch. >>> >>> So it's more like: >>> >>> 1) OCCI is compatible with ALL existing formats, eg OVF >>> >>> 2) Existing formats are used are used for ALL representations >>> >>> 3) ALL additional metadata should be in the headers >>> >>> 4) ALL of the core stuff should be in the body (that is, the >>> metadata in the >>> headers is just there to support the data and wire it up in >>> useful >>> ways) >>> >>> I think the confusion stems from your assuming that there is >>> still >>> XML/JSON/Atom/etc. representations. There aren't. HTTP headers >>> provide a >>> single, performant, interoperable metadata channel that has >>> already been >>> both standardised and extensively implemented. >>> >>> If you've followed me this far then you should have realised >>> that >>> every HTTP >>> user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, >>> etc.) >>> is already >>> a basic OCCI client out-of-the-box. The result is that 5-line >>> scripts like >>> demo.py are functional with no dependence on client libraries >>> (that would >>> absolutely be necessary if you were to start trying to define >>> your >>> own >>> formats). >>> >>> Sam >>> >>> PS I'm sorry my "bunk" comment came across as referring to you >>> personally - >>> it was directed at the argument. >>> >>> On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson >>> <alexis.richardson@gmail.com> wrote: >>> >>>> I think what Sam is proposing is the following: >>>> >>>> 1) If OCCI can handle 'any' data representation, eg OVF, XML, >>>> JSON, >>>> then it needs some core / common model. Otherwise: there can >>>> be no >>>> defined behaviour / interop >>>> >>>> 2) This core / common model is ALL metadata >>>> >>>> 3) ALL this metadata should be in the headers >>>> >>>> 4) ALL the non core stuff eg OVF payload, XML representation, >>>> should be in >>>> body >>>> >>>> Sam - is that right? >>>> >>>> a >>>> >>>> >>>> On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> >>>> wrote: >>>> >>>>> On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX >>>>> <andrewx.edmonds@intel.com> wrote: >>>>> >>>>>> To that point I'd ask - If we're talking about meta-data >>>>>> that >>>>>> infers >>>>>> there >>>>>> is some data to accompany it. Where are the examples of this >>>>>> data? This >>>>>> would help in forming the full picture. >>>>>> >>>>> Here, from the core spec (using OVF as an example, but >>>>> could be >>>>> anything). >>>>> Request is green, response metadata is yellow & response >>>>> data/ >>>>> payload is >>>>> orange: >>>>> >>>>> >>>>>> GET /us-east/webapp/vm01 HTTP/1.1 >>>>>> User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0 >>>>>> Host: cloud.example.com >>>>>> Accept: */* >>>>>> >>>>> < HTTP/1.1 200 OK >>>>> < Date: Sat, 10 Oct 2009 12:56:51 GMT >>>>> >>>>> < Content-Type: application/ovf >>>>> < Link: </us-east/webapp/vm01;start>; >>>>> >>>>> < rel="http://purl.org/occi/action/start"; >>>>> < title="Start" >>>>> >>>>> < Link: </us-east/webapp/build.pdf>; >>>>> < rel="related"; >>>>> >>>>> < title="Documentation"; >>>>> < type="application/pdf" >>>>> >>>>> < Category: compute; >>>>> < label="Compute Resource"; >>>>> >>>>> < scheme="http://purl.org/occi/kind/" >>>>> < Server: occi-server/1.0 (linux) OCCI/1.0 >>>>> >>>>> < Connection: close >>>>> < >>>>> < <?xml version="1.0" encoding="UTF-8"?> >>>>> >>>>> < <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema- >>>>> instance" >>>>> < xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/ >>>>> 1" >>>>> >>>>> < xmlns="http://schemas.dmtf.org/ovf/envelope/1" >>>>> < xml:lang="en-US"> >>>>> >>>>> ... >>>>> >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: occi-wg-bounces@ogf.org [mailto:occi-wg- >>>>>> bounces@ogf.org] On >>>>>> Behalf >>>>>> Of Alexis Richardson >>>>>> Sent: 20 October 2009 09:49 >>>>>> To: Sam Johnston >>>>>> Cc: Tim Bray; occi-wg@ogf.org >>>>>> Subject: Re: [occi-wg] confusion about status of link / >>>>>> headers >>>>>> >>>>>> Sam, >>>>>> >>>>>> Please don't throw around words like "bunk" willy nilly. >>>>>> >>>>>> I have just had a look at >>>>>> http://www.rackspacecloud.com/cloud_hosting_products/ >>>>>> servers/ >>>>>> api >>>>>> >>>>>> I can see some limited use of headers for metadata in a few >>>>>> places. >>>>>> That's never been controversial. But the understanding >>>>>> that I >>>>>> have of >>>>>> your proposal is to use headers wholesale for all metadata. >>>>>> Rackspace >>>>>> don't appear to do that, unless I have misunderstood their >>>>>> document. >>>>>> I also may have misunderstood your proposal. If nobody >>>>>> understands >>>>>> your proposal except you, then it will not be possible to >>>>>> gain >>>>>> consensus around it. >>>>>> >>>>>> Are you saying "OCCI metadata should be more like Rackspace >>>>>> metadata"? >>>>>> >>>>>> alexis >>>>>> >>>>>> >>>>>> On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston >>>>>> <samj@samj.net> >>>>>> wrote: >>>>>> >>>>>>> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson >>>>>>> <alexis.richardson@gmail.com> wrote: >>>>>>> >>>>>>>> What about compute clouds? >>>>>>>> >>>>>>> Rackspace Cloud API for a start: >>>>>>> >>>>>>> >>>>>>>> HTTP/1.1 204 No Content >>>>>>>> Date: Mon, 12 Nov 2007 15:32:21 GMT >>>>>>>> Server: Apache >>>>>>>> X-Server-Management-Url: >>>>>>>> https://servers.api.rackspacecloud.com/v1.0/35428 >>>>>>>> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>>>>>> X-CDN-Management-Url: >>>>>>>> https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>>>>>> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb >>>>>>>> Content-Length: 0 >>>>>>>> Content-Type: text/plain; charset=UTF-8 >>>>>>>> >>>>>>> Microsoft Azure uses headers for "attributes" as well, in >>>>>>> the >>>>>>> same >>>>>>> way >>>>>>> as I >>>>>>> had originally proposed: >>>>>>> >>>>>>> x-ms-meta-name: value Returns a metadata value for the >>>>>>> container. >>>>>>> >>>>>>> However prefer that the header names are static/opaque >>>>>>> rather >>>>>>> than >>>>>>> using >>>>>>> a >>>>>>> template and parsing them: >>>>>>> >>>>>>> >>>>>>>> Attribute: name=value >>>>>>>> >>>>>>> This to me is cleaner and more self-explanatory, plus it is >>>>>>> easily >>>>>>> collapsed >>>>>>> ala: >>>>>>> >>>>>>> >>>>>>>> Attribute: name1=value1, name2=value2, ... namen=valuen >>>>>>>> >>>>>>> Anyway suffice to say that claims this is somehow >>>>>>> "experimental" are >>>>>>> bunk. >>>>>>> >>>>>>> Sam >>>>>>> >>>>>>> >>>>>>>> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org >>>>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Here are options for metadata used in some of the major >>>>>>>>> storage >>>>>>>>> clouds >>>>>>>>> FWIW: >>>>>>>>> >>>>>>>>> S3, Rackspace, EMC Atmos, Azure - Headers >>>>>>>>> Nirvanix - query params in, xml entity out >>>>>>>>> Mezeo - entity >>>>>>>>> >>>>>>>>> Of the ones using headers, S3, Rackspace and Azure use >>>>>>>>> prefix with >>>>>>>>> values stored as-us. Atmos joins all metadata together >>>>>>>>> into >>>>>>>>> one >>>>>>>>> header, making parsing trivial (split /,/), but >>>>>>>>> necessary. >>>>>>>>> >>>>>>>>> The most expensive option of the above is entity, where >>>>>>>>> each >>>>>>>>> metadata >>>>>>>>> value is a separate GET. However, entities allow binary >>>>>>>>> metadata >>>>>>>>> and >>>>>>>>> zero restrictions on it, which may be useful. >>>>>>>>> >>>>>>>>> In jclouds, we time parsing of response values. A simple >>>>>>>>> XML doc >>>>>>>>> with >>>>>>>>> only several elements written in SAX takes a few ms to >>>>>>>>> parse. My >>>>>>>>> log >>>>>>>>> files are not precise enough to find the overhead in >>>>>>>>> parsing >>>>>>>>> headers: >>>>>>>>> they always start and finish within the same millisecond. >>>>>>>>> >>>>>>>>> I hope this background helps, and also helps explain >>>>>>>>> why I'm >>>>>>>>> vocal >>>>>>>>> on >>>>>>>>> such topics such as headers vs entities :) >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> -Adrian >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston >>>>>>>>> <samj@samj.net> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >>>>>>>>>> <garymazzaferro@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> The http header and key/value pairs need to parsed >>>>>>>>>>> also, >>>>>>>>>>> there >>>>>>>>>>> is >>>>>>>>>>> no >>>>>>>>>>> free >>>>>>>>>>> ride here. >>>>>>>>>>> >>>>>>>>>> Every HTTP library I have ever used parses HTTP >>>>>>>>>> headers and >>>>>>>>>> puts >>>>>>>>>> them >>>>>>>>>> in a >>>>>>>>>> nice hash for you ready to consume. If we go for "Name: >>>>>>>>>> Value" >>>>>>>>>> then >>>>>>>>>> that's >>>>>>>>>> all there is to it. If we go for "Attribute: >>>>>>>>>> name=value" as >>>>>>>>>> is >>>>>>>>>> currently >>>>>>>>>> proposed (which is arguably cleaner, follows cookies' >>>>>>>>>> "prior art" >>>>>>>>>> and >>>>>>>>>> avoids >>>>>>>>>> Amazon's prefix hack) then you just have to split on >>>>>>>>>> '='. >>>>>>>>>> >>>>>>>>>> To illustrate how clean this is by example: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> #!/usr/bin/python >>>>>>>>>>> import urllib2 >>>>>>>>>>> response = urllib2.urlopen('http://cloud.example.com/ >>>>>>>>>>> myvm') >>>>>>>>>>> representation = response.read() >>>>>>>>>>> metadata = response.info() >>>>>>>>>>> print metadata['occi-compute-cores'] >>>>>>>>>>> >>>>>>>>>> As soon as you start talking about payloads you have to >>>>>>>>>> fire up a >>>>>>>>>> parser >>>>>>>>>> (JSON/XML/Atom/etc.) or write your own (previous text >>>>>>>>>> rendering) >>>>>>>>>> which >>>>>>>>>> is >>>>>>>>>> significantly more work to do at both design and run >>>>>>>>>> times. >>>>>>>>>> Not >>>>>>>>>> to >>>>>>>>>> mention >>>>>>>>>> more work for us to do now and more scope for >>>>>>>>>> interoperability >>>>>>>>>> problems. >>>>>>>>>> >>>>>>>>>> Sam >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> occi-wg mailing list >>>>>>>>>> occi-wg@ogf.org >>>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> occi-wg mailing list >>>>>>>>> occi-wg@ogf.org >>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> occi-wg mailing list >>>>>> occi-wg@ogf.org >>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>> --- >>>>>> ---------------------------------------------------------- >>>>>> Intel Ireland Limited (Branch) >>>>>> Collinstown Industrial Park, Leixlip, County Kildare, >>>>>> Ireland >>>>>> Registered Number: E902934 >>>>>> >>>>>> This e-mail and any attachments may contain confidential >>>>>> material for >>>>>> the sole use of the intended recipient(s). Any review or >>>>>> distribution >>>>>> by others is strictly prohibited. If you are not the >>>>>> intended >>>>>> recipient, please contact the sender and delete all copies. >>>>>> >>>>> >>> >> _______________________________________________ >> occi-wg mailing list >> occi-wg@ogf.org >> http://www.ogf.org/mailman/listinfo/occi-wg >> >> >
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
At IETF meetings, I've seen the chairs ask people to Hum for one proposal or the other - indicating both number of proponents and their strength of conviction... ;-) -- mark Adrian Cole wrote:
Ahh, ok (sorry... still an OCCI newbie). Disregard my last call to arms.
-Adrian
On Tue, Oct 20, 2009 at 9:56 AM, Sam Johnston <samj@samj.net> wrote:
Adrian,
We don't vote... we discuss our way to consensus and escalate if we fail.
Anyway I've had enough for one day/lifetime and I've got a house full of furniture to build tonight. I do wonder how many people here have actually bothered to read the spec and understand the issues though - if you haven't already then please do so now and speak up if anything doesn't make sense.
Sent from my iPhone
On 20/10/2009, at 18:44, Adrian Cole <adrian@jclouds.org> wrote:
As we aren't introducing new information, I think we are at a point where a vote can take place. This issue isn't infinitely complex, and reasonable to model in choices. OCCI desires to be a community process, so let's see where we fall on this.
Do you agree? -Adrian
On Tue, Oct 20, 2009 at 9:39 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Sam,
On Tue, Oct 20, 2009 at 5:31 PM, Sam Johnston <samj@samj.net> wrote:
Alexis,
Sweeping, unjustified statements are intensely frustrating, not to mention unhelpful in the absence of a coherent alternative. If people choose not to disclose conflicts I happen to be aware of then you bet I'll disclose them for them - nothing personal about it.
I don't know what 'conflicts' you are referring to, or how these relate to the matter at hand. If by 'sweeping, unjustified' you refer to the word 'unprecedented' then I shall justify it by pointing out that - at least as I currently understand it - you are proposing use of headers to a *greater* extent than has been done before and to cover a *greater* set of use cases.
Cutting to the chase, both Microsoft and Amazon are using headers for metadata to the same extent as what has been proposed.
Really - to the "same" extent? That seems contrary to what has been asserted on this thread previously.
What premature optimisation? *Not* using XML? Do we really need to drag that carcass out again? Surely Adrian's performance tests put the last nail in that particular coffin.
Premature optimisation = making performance a critical design concern when we want an API.
I have noted Adrian's points as have others.
The question really is about whether we want to be an interface/ protocol like HTTP or some mongrel hybrid of that along with a set of XML, JSON and/or text formats for everything we ever intend to model (knowing that work has already been done by others).
Work that has been done by others and shown to work, is a good thing.
alexis
Sent from my iPhone
On 20/10/2009, at 18:10, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Sam
Cut the personal stuff please.
Re 'unproven', I think the issues people are having are:
* The extent, scope and purpose of your application of headers to metadata is unprecedented - as I understand it. Nobody is saying it has not been done in other and restricted cases.
* Premature optimisation.
alexis
On Tue, Oct 20, 2009 at 4:44 PM, Sam Johnston <samj@samj.net> wrote:
> Gary, > > Please be specific with your objections so as I can have some > hope of > answering them... "several other emails" does not suffice as to > the > best of my knowledge there are no outstanding issues that I have > not > comprehensively and convincingly dealt with. > > Your repeated assertions than HTTP headers are... what is it > now... > "unproven"... are demonstrably unfounded (refer to the various > cloud > APIs referred to by myself and Adrian, including both MS Azure and > Amazon S3, as well as the history of the Internet in general) and > given the frequency, bordering on FUD. > > Regarding performance, OCCI should strive to be as efficient as > possible as it primarily targets public cloud providers who > operate > at > arbitrarily large "cloud" scale. The existing specification is > *extremely* efficient and alternatives would at least want to be > in > the same neighbourhood. Efficiency may be an inconvenient factor > for > whatever your alternative is, but it's critical for many of our > use > cases. > > Finally, you reject OVF without providing any alternative beyond > building a replacement from scratch ourselves, after our second > and > final deadline has just sailed by no less. Have you considered the > implementation cost of this? And in the same breath you accuse my > proposal of being "unproven"? Give me a break... > > Correct me if I'm wrong but the way I see it is that you want to > revert the last 2-3 months of my work because you have an > investment > in an old version of the spec. With each day I spend on OCCI > costing > me well over €1,000 excuse me for being pissed about someon > e with > such a blatantly obvious conflict attacking my work and refusing > to > provide a coherent argument, particularly with no "camera ready" > alternative anywhere in sight. > > Sent from my iPhone > > On 20/10/2009, at 16:51, Gary Mazz <garymazzaferro@gmail.com> > wrote: > > >> I've understood Sam's proposal from the beginning. I have had >> issue >> with from the beginning. >> >> Although I find it very interesting and valuable in terms of >> economy, I have issue with it as the OCCI's primary interface >> implementation for a few reasons which have been outlined in >> several >> other emails. >> >> It appears that Sam has created a new technology, a lower cost >> RPC >> mechanism, which is unproven for both small and wide scale >> deployment. For interoperability, IMO, occi should focus on >> proven >> mainstream technologies as the primary interfaces. I am not >> suggesting, shelving or not supporting Sam's proposal, based on >> market acceptance, it may prove to be one of many occi interface >> implementations. >> >> As for cost saving in interface technologies, I do not buy into >> the >> idea that the occi traffic will be so overwhelming that there >> is a >> requirement to maximally optimize http requests and responses. >> IMO, >> in comparison to images and consumer data occi traffic will be >> insignificant. I think we are worrying about spilled milk while >> the >> house is burning. >> >> There has been some discussion on supporting technologies >> including >> OVF. Whether occi supports OVF has not been fully discussed, >> examined or any consensus reached. I believe adopting >> technologies >> like OVF will only lead to mapping issues between occi attributes >> and the external formats creating a long term management >> headache. >> >> cheers, >> gary >> >> >> Alexis Richardson wrote: >> >>> OK >>> >>> Is Sam's statement something that we can achieve consensus >>> around? >>> >>> a >>> >>> >>> On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> >>> wrote: >>> >>> >>>> Alexis, >>>> >>>> Seems our posts crossed in the mail. I hope the last one clears >>>> things up >>>> but the fact that you're using OVF and XML/JSON interchangeably >>>> suggests >>>> that you [are possibly one of many who] don't grok the >>>> proposal. >>>> Worse, it >>>> seems you've got it back to front in thinking of the "core" as >>>> metadata, >>>> which is not at all what I had intended. >>>> >>>> The focus is on the resources and their various >>>> representations. >>>> The >>>> metadata (that is, data about data) essentially provides >>>> whatever is >>>> necessary to "glue" these resources together. It is basically >>>> using HTTP as >>>> the meta-model rather than an envelope format like Atom/SOAP >>>> or, >>>> worse, a >>>> meta-model that we have to create ourselves from scratch. >>>> >>>> So it's more like: >>>> >>>> 1) OCCI is compatible with ALL existing formats, eg OVF >>>> >>>> 2) Existing formats are used are used for ALL representations >>>> >>>> 3) ALL additional metadata should be in the headers >>>> >>>> 4) ALL of the core stuff should be in the body (that is, the >>>> metadata in the >>>> headers is just there to support the data and wire it up in >>>> useful >>>> ways) >>>> >>>> I think the confusion stems from your assuming that there is >>>> still >>>> XML/JSON/Atom/etc. representations. There aren't. HTTP headers >>>> provide a >>>> single, performant, interoperable metadata channel that has >>>> already been >>>> both standardised and extensively implemented. >>>> >>>> If you've followed me this far then you should have realised >>>> that >>>> every HTTP >>>> user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, >>>> etc.) >>>> is already >>>> a basic OCCI client out-of-the-box. The result is that 5-line >>>> scripts like >>>> demo.py are functional with no dependence on client libraries >>>> (that would >>>> absolutely be necessary if you were to start trying to define >>>> your >>>> own >>>> formats). >>>> >>>> Sam >>>> >>>> PS I'm sorry my "bunk" comment came across as referring to you >>>> personally - >>>> it was directed at the argument. >>>> >>>> On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson >>>> <alexis.richardson@gmail.com> wrote: >>>> >>>> >>>>> I think what Sam is proposing is the following: >>>>> >>>>> 1) If OCCI can handle 'any' data representation, eg OVF, XML, >>>>> JSON, >>>>> then it needs some core / common model. Otherwise: there can >>>>> be no >>>>> defined behaviour / interop >>>>> >>>>> 2) This core / common model is ALL metadata >>>>> >>>>> 3) ALL this metadata should be in the headers >>>>> >>>>> 4) ALL the non core stuff eg OVF payload, XML representation, >>>>> should be in >>>>> body >>>>> >>>>> Sam - is that right? >>>>> >>>>> a >>>>> >>>>> >>>>> On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> >>>>> wrote: >>>>> >>>>> >>>>>> On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX >>>>>> <andrewx.edmonds@intel.com> wrote: >>>>>> >>>>>> >>>>>>> To that point I'd ask - If we're talking about meta-data >>>>>>> that >>>>>>> infers >>>>>>> there >>>>>>> is some data to accompany it. Where are the examples of this >>>>>>> data? This >>>>>>> would help in forming the full picture. >>>>>>> >>>>>>> >>>>>> Here, from the core spec (using OVF as an example, but >>>>>> could be >>>>>> anything). >>>>>> Request is green, response metadata is yellow & response >>>>>> data/ >>>>>> payload is >>>>>> orange: >>>>>> >>>>>> >>>>>> >>>>>>> GET /us-east/webapp/vm01 HTTP/1.1 >>>>>>> User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0 >>>>>>> Host: cloud.example.com >>>>>>> Accept: */* >>>>>>> >>>>>>> >>>>>> < HTTP/1.1 200 OK >>>>>> < Date: Sat, 10 Oct 2009 12:56:51 GMT >>>>>> >>>>>> < Content-Type: application/ovf >>>>>> < Link: </us-east/webapp/vm01;start>; >>>>>> >>>>>> < rel="http://purl.org/occi/action/start"; >>>>>> < title="Start" >>>>>> >>>>>> < Link: </us-east/webapp/build.pdf>; >>>>>> < rel="related"; >>>>>> >>>>>> < title="Documentation"; >>>>>> < type="application/pdf" >>>>>> >>>>>> < Category: compute; >>>>>> < label="Compute Resource"; >>>>>> >>>>>> < scheme="http://purl.org/occi/kind/" >>>>>> < Server: occi-server/1.0 (linux) OCCI/1.0 >>>>>> >>>>>> < Connection: close >>>>>> < >>>>>> < <?xml version="1.0" encoding="UTF-8"?> >>>>>> >>>>>> < <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema- >>>>>> instance" >>>>>> < xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/ >>>>>> 1" >>>>>> >>>>>> < xmlns="http://schemas.dmtf.org/ovf/envelope/1" >>>>>> < xml:lang="en-US"> >>>>>> >>>>>> ... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: occi-wg-bounces@ogf.org [mailto:occi-wg- >>>>>>> bounces@ogf.org] On >>>>>>> Behalf >>>>>>> Of Alexis Richardson >>>>>>> Sent: 20 October 2009 09:49 >>>>>>> To: Sam Johnston >>>>>>> Cc: Tim Bray; occi-wg@ogf.org >>>>>>> Subject: Re: [occi-wg] confusion about status of link / >>>>>>> headers >>>>>>> >>>>>>> Sam, >>>>>>> >>>>>>> Please don't throw around words like "bunk" willy nilly. >>>>>>> >>>>>>> I have just had a look at >>>>>>> http://www.rackspacecloud.com/cloud_hosting_products/ >>>>>>> servers/ >>>>>>> api >>>>>>> >>>>>>> I can see some limited use of headers for metadata in a few >>>>>>> places. >>>>>>> That's never been controversial. But the understanding >>>>>>> that I >>>>>>> have of >>>>>>> your proposal is to use headers wholesale for all metadata. >>>>>>> Rackspace >>>>>>> don't appear to do that, unless I have misunderstood their >>>>>>> document. >>>>>>> I also may have misunderstood your proposal. If nobody >>>>>>> understands >>>>>>> your proposal except you, then it will not be possible to >>>>>>> gain >>>>>>> consensus around it. >>>>>>> >>>>>>> Are you saying "OCCI metadata should be more like Rackspace >>>>>>> metadata"? >>>>>>> >>>>>>> alexis >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston >>>>>>> <samj@samj.net> >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>>> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson >>>>>>>> <alexis.richardson@gmail.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>>> What about compute clouds? >>>>>>>>> >>>>>>>>> >>>>>>>> Rackspace Cloud API for a start: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> HTTP/1.1 204 No Content >>>>>>>>> Date: Mon, 12 Nov 2007 15:32:21 GMT >>>>>>>>> Server: Apache >>>>>>>>> X-Server-Management-Url: >>>>>>>>> https://servers.api.rackspacecloud.com/v1.0/35428 >>>>>>>>> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>>>>>>> X-CDN-Management-Url: >>>>>>>>> https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>>>>>>> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb >>>>>>>>> Content-Length: 0 >>>>>>>>> Content-Type: text/plain; charset=UTF-8 >>>>>>>>> >>>>>>>>> >>>>>>>> Microsoft Azure uses headers for "attributes" as well, in >>>>>>>> the >>>>>>>> same >>>>>>>> way >>>>>>>> as I >>>>>>>> had originally proposed: >>>>>>>> >>>>>>>> x-ms-meta-name: value Returns a metadata value for the >>>>>>>> container. >>>>>>>> >>>>>>>> However prefer that the header names are static/opaque >>>>>>>> rather >>>>>>>> than >>>>>>>> using >>>>>>>> a >>>>>>>> template and parsing them: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Attribute: name=value >>>>>>>>> >>>>>>>>> >>>>>>>> This to me is cleaner and more self-explanatory, plus it is >>>>>>>> easily >>>>>>>> collapsed >>>>>>>> ala: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Attribute: name1=value1, name2=value2, ... namen=valuen >>>>>>>>> >>>>>>>>> >>>>>>>> Anyway suffice to say that claims this is somehow >>>>>>>> "experimental" are >>>>>>>> bunk. >>>>>>>> >>>>>>>> Sam >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> Here are options for metadata used in some of the major >>>>>>>>>> storage >>>>>>>>>> clouds >>>>>>>>>> FWIW: >>>>>>>>>> >>>>>>>>>> S3, Rackspace, EMC Atmos, Azure - Headers >>>>>>>>>> Nirvanix - query params in, xml entity out >>>>>>>>>> Mezeo - entity >>>>>>>>>> >>>>>>>>>> Of the ones using headers, S3, Rackspace and Azure use >>>>>>>>>> prefix with >>>>>>>>>> values stored as-us. Atmos joins all metadata together >>>>>>>>>> into >>>>>>>>>> one >>>>>>>>>> header, making parsing trivial (split /,/), but >>>>>>>>>> necessary. >>>>>>>>>> >>>>>>>>>> The most expensive option of the above is entity, where >>>>>>>>>> each >>>>>>>>>> metadata >>>>>>>>>> value is a separate GET. However, entities allow binary >>>>>>>>>> metadata >>>>>>>>>> and >>>>>>>>>> zero restrictions on it, which may be useful. >>>>>>>>>> >>>>>>>>>> In jclouds, we time parsing of response values. A simple >>>>>>>>>> XML doc >>>>>>>>>> with >>>>>>>>>> only several elements written in SAX takes a few ms to >>>>>>>>>> parse. My >>>>>>>>>> log >>>>>>>>>> files are not precise enough to find the overhead in >>>>>>>>>> parsing >>>>>>>>>> headers: >>>>>>>>>> they always start and finish within the same millisecond. >>>>>>>>>> >>>>>>>>>> I hope this background helps, and also helps explain >>>>>>>>>> why I'm >>>>>>>>>> vocal >>>>>>>>>> on >>>>>>>>>> such topics such as headers vs entities :) >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> -Adrian >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston >>>>>>>>>> <samj@samj.net> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >>>>>>>>>>> <garymazzaferro@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> The http header and key/value pairs need to parsed >>>>>>>>>>>> also, >>>>>>>>>>>> there >>>>>>>>>>>> is >>>>>>>>>>>> no >>>>>>>>>>>> free >>>>>>>>>>>> ride here. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> Every HTTP library I have ever used parses HTTP >>>>>>>>>>> headers and >>>>>>>>>>> puts >>>>>>>>>>> them >>>>>>>>>>> in a >>>>>>>>>>> nice hash for you ready to consume. If we go for "Name: >>>>>>>>>>> Value" >>>>>>>>>>> then >>>>>>>>>>> that's >>>>>>>>>>> all there is to it. If we go for "Attribute: >>>>>>>>>>> name=value" as >>>>>>>>>>> is >>>>>>>>>>> currently >>>>>>>>>>> proposed (which is arguably cleaner, follows cookies' >>>>>>>>>>> "prior art" >>>>>>>>>>> and >>>>>>>>>>> avoids >>>>>>>>>>> Amazon's prefix hack) then you just have to split on >>>>>>>>>>> '='. >>>>>>>>>>> >>>>>>>>>>> To illustrate how clean this is by example: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> #!/usr/bin/python >>>>>>>>>>>> import urllib2 >>>>>>>>>>>> response = urllib2.urlopen('http://cloud.example.com/ >>>>>>>>>>>> myvm') >>>>>>>>>>>> representation = response.read() >>>>>>>>>>>> metadata = response.info() >>>>>>>>>>>> print metadata['occi-compute-cores'] >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> As soon as you start talking about payloads you have to >>>>>>>>>>> fire up a >>>>>>>>>>> parser >>>>>>>>>>> (JSON/XML/Atom/etc.) or write your own (previous text >>>>>>>>>>> rendering) >>>>>>>>>>> which >>>>>>>>>>> is >>>>>>>>>>> significantly more work to do at both design and run >>>>>>>>>>> times. >>>>>>>>>>> Not >>>>>>>>>>> to >>>>>>>>>>> mention >>>>>>>>>>> more work for us to do now and more scope for >>>>>>>>>>> interoperability >>>>>>>>>>> problems. >>>>>>>>>>> >>>>>>>>>>> Sam >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> occi-wg mailing list >>>>>>>>>>> occi-wg@ogf.org >>>>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> occi-wg mailing list >>>>>>>>>> occi-wg@ogf.org >>>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>> _______________________________________________ >>>>>>> occi-wg mailing list >>>>>>> occi-wg@ogf.org >>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>> --- >>>>>>> ---------------------------------------------------------- >>>>>>> Intel Ireland Limited (Branch) >>>>>>> Collinstown Industrial Park, Leixlip, County Kildare, >>>>>>> Ireland >>>>>>> Registered Number: E902934 >>>>>>> >>>>>>> This e-mail and any attachments may contain confidential >>>>>>> material for >>>>>>> the sole use of the intended recipient(s). Any review or >>>>>>> distribution >>>>>>> by others is strictly prohibited. If you are not the >>>>>>> intended >>>>>>> recipient, please contact the sender and delete all copies. >>>>>>> >>>>>>> >>> _______________________________________________ >>> occi-wg mailing list >>> occi-wg@ogf.org >>> http://www.ogf.org/mailman/listinfo/occi-wg >>> >>> >>>
occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
-- <http://www.sun.com> * Mark A. Carlson * Sr. Architect *Systems Group* Phone x69559 / 303-223-6139 Email Mark.Carlson@Sun.COM
Does anyone get to sing along? On Tue, Oct 20, 2009 at 7:50 PM, Mark A. Carlson <Mark.Carlson@sun.com>wrote:
At IETF meetings, I've seen the chairs ask people to Hum for one proposal or the other - indicating both number of proponents and their strength of conviction... ;-)
-- mark
Adrian Cole wrote:
Ahh, ok (sorry... still an OCCI newbie). Disregard my last call to arms.
-Adrian
On Tue, Oct 20, 2009 at 9:56 AM, Sam Johnston <samj@samj.net> <samj@samj.net> wrote:
Adrian,
We don't vote... we discuss our way to consensus and escalate if we fail.
Anyway I've had enough for one day/lifetime and I've got a house full of furniture to build tonight. I do wonder how many people here have actually bothered to read the spec and understand the issues though - if you haven't already then please do so now and speak up if anything doesn't make sense.
Sent from my iPhone
On 20/10/2009, at 18:44, Adrian Cole <adrian@jclouds.org> <adrian@jclouds.org> wrote:
As we aren't introducing new information, I think we are at a point where a vote can take place. This issue isn't infinitely complex, and reasonable to model in choices. OCCI desires to be a community process, so let's see where we fall on this.
Do you agree? -Adrian
On Tue, Oct 20, 2009 at 9:39 AM, Alexis Richardson<alexis.richardson@gmail.com> <alexis.richardson@gmail.com> wrote:
Sam,
On Tue, Oct 20, 2009 at 5:31 PM, Sam Johnston <samj@samj.net> <samj@samj.net> wrote:
Alexis,
Sweeping, unjustified statements are intensely frustrating, not to mention unhelpful in the absence of a coherent alternative. If people choose not to disclose conflicts I happen to be aware of then you bet I'll disclose them for them - nothing personal about it.
I don't know what 'conflicts' you are referring to, or how these relate to the matter at hand. If by 'sweeping, unjustified' you refer to the word 'unprecedented' then I shall justify it by pointing out that - at least as I currently understand it - you are proposing use of headers to a *greater* extent than has been done before and to cover a *greater* set of use cases.
Cutting to the chase, both Microsoft and Amazon are using headers for metadata to the same extent as what has been proposed.
Really - to the "same" extent? That seems contrary to what has been asserted on this thread previously.
What premature optimisation? *Not* using XML? Do we really need to drag that carcass out again? Surely Adrian's performance tests put the last nail in that particular coffin.
Premature optimisation = making performance a critical design concern when we want an API.
I have noted Adrian's points as have others.
The question really is about whether we want to be an interface/ protocol like HTTP or some mongrel hybrid of that along with a set of XML, JSON and/or text formats for everything we ever intend to model (knowing that work has already been done by others).
Work that has been done by others and shown to work, is a good thing.
alexis
Sent from my iPhone
On 20/10/2009, at 18:10, Alexis Richardson<alexis.richardson@gmail.com> <alexis.richardson@gmail.com> wrote:
Sam
Cut the personal stuff please.
Re 'unproven', I think the issues people are having are:
* The extent, scope and purpose of your application of headers to metadata is unprecedented - as I understand it. Nobody is saying it has not been done in other and restricted cases.
* Premature optimisation.
alexis
On Tue, Oct 20, 2009 at 4:44 PM, Sam Johnston <samj@samj.net> <samj@samj.net> wrote:
Gary,
Please be specific with your objections so as I can have some hope of answering them... "several other emails" does not suffice as to the best of my knowledge there are no outstanding issues that I have not comprehensively and convincingly dealt with.
Your repeated assertions than HTTP headers are... what is it now... "unproven"... are demonstrably unfounded (refer to the various cloud APIs referred to by myself and Adrian, including both MS Azure and Amazon S3, as well as the history of the Internet in general) and given the frequency, bordering on FUD.
Regarding performance, OCCI should strive to be as efficient as possible as it primarily targets public cloud providers who operate at arbitrarily large "cloud" scale. The existing specification is *extremely* efficient and alternatives would at least want to be in the same neighbourhood. Efficiency may be an inconvenient factor for whatever your alternative is, but it's critical for many of our use cases.
Finally, you reject OVF without providing any alternative beyond building a replacement from scratch ourselves, after our second and final deadline has just sailed by no less. Have you considered the implementation cost of this? And in the same breath you accuse my proposal of being "unproven"? Give me a break...
Correct me if I'm wrong but the way I see it is that you want to revert the last 2-3 months of my work because you have an investment in an old version of the spec. With each day I spend on OCCI costing me well over €1,000 excuse me for being pissed about someon e with such a blatantly obvious conflict attacking my work and refusing to provide a coherent argument, particularly with no "camera ready" alternative anywhere in sight.
Sent from my iPhone
On 20/10/2009, at 16:51, Gary Mazz <garymazzaferro@gmail.com> <garymazzaferro@gmail.com> wrote:
I've understood Sam's proposal from the beginning. I have had issue with from the beginning.
Although I find it very interesting and valuable in terms of economy, I have issue with it as the OCCI's primary interface implementation for a few reasons which have been outlined in several other emails.
It appears that Sam has created a new technology, a lower cost RPC mechanism, which is unproven for both small and wide scale deployment. For interoperability, IMO, occi should focus on proven mainstream technologies as the primary interfaces. I am not suggesting, shelving or not supporting Sam's proposal, based on market acceptance, it may prove to be one of many occi interface implementations.
As for cost saving in interface technologies, I do not buy into the idea that the occi traffic will be so overwhelming that there is a requirement to maximally optimize http requests and responses. IMO, in comparison to images and consumer data occi traffic will be insignificant. I think we are worrying about spilled milk while the house is burning.
There has been some discussion on supporting technologies including OVF. Whether occi supports OVF has not been fully discussed, examined or any consensus reached. I believe adopting technologies like OVF will only lead to mapping issues between occi attributes and the external formats creating a long term management headache.
cheers, gary
Alexis Richardson wrote:
OK
Is Sam's statement something that we can achieve consensus around?
a
On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> <samj@samj.net> wrote:
Alexis,
Seems our posts crossed in the mail. I hope the last one clears things up but the fact that you're using OVF and XML/JSON interchangeably suggests that you [are possibly one of many who] don't grok the proposal. Worse, it seems you've got it back to front in thinking of the "core" as metadata, which is not at all what I had intended.
The focus is on the resources and their various representations. The metadata (that is, data about data) essentially provides whatever is necessary to "glue" these resources together. It is basically using HTTP as the meta-model rather than an envelope format like Atom/SOAP or, worse, a meta-model that we have to create ourselves from scratch.
So it's more like:
1) OCCI is compatible with ALL existing formats, eg OVF
2) Existing formats are used are used for ALL representations
3) ALL additional metadata should be in the headers
4) ALL of the core stuff should be in the body (that is, the metadata in the headers is just there to support the data and wire it up in useful ways)
I think the confusion stems from your assuming that there is still XML/JSON/Atom/etc. representations. There aren't. HTTP headers provide a single, performant, interoperable metadata channel that has already been both standardised and extensively implemented.
If you've followed me this far then you should have realised that every HTTP user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, etc.) is already a basic OCCI client out-of-the-box. The result is that 5-line scripts like demo.py are functional with no dependence on client libraries (that would absolutely be necessary if you were to start trying to define your own formats).
Sam
PS I'm sorry my "bunk" comment came across as referring to you personally - it was directed at the argument.
On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson<alexis.richardson@gmail.com> <alexis.richardson@gmail.com> wrote:
I think what Sam is proposing is the following:
1) If OCCI can handle 'any' data representation, eg OVF, XML, JSON, then it needs some core / common model. Otherwise: there can be no defined behaviour / interop
2) This core / common model is ALL metadata
3) ALL this metadata should be in the headers
4) ALL the non core stuff eg OVF payload, XML representation, should be in body
Sam - is that right?
a
On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> <samj@samj.net> wrote:
On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX<andrewx.edmonds@intel.com> <andrewx.edmonds@intel.com> wrote:
To that point I'd ask - If we're talking about meta-data that infers there is some data to accompany it. Where are the examples of this data? This would help in forming the full picture.
Here, from the core spec (using OVF as an example, but could be anything). Request is green, response metadata is yellow & response data/ payload is orange:
GET /us-east/webapp/vm01 HTTP/1.1 User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0 Host: cloud.example.com Accept: */*
< HTTP/1.1 200 OK < Date: Sat, 10 Oct 2009 12:56:51 GMT
< Content-Type: application/ovf < Link: </us-east/webapp/vm01;start>;
< rel="http://purl.org/occi/action/start" <http://purl.org/occi/action/start>; < title="Start"
< Link: </us-east/webapp/build.pdf>; < rel="related";
< title="Documentation"; < type="application/pdf"
< Category: compute; < label="Compute Resource";
< scheme="http://purl.org/occi/kind/" <http://purl.org/occi/kind/> < Server: occi-server/1.0 (linux) OCCI/1.0
< Connection: close < < <?xml version="1.0" encoding="UTF-8"?>
< <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema- instance" <http://www.w3.org/2001/XMLSchema-instance> < xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/ 1" <http://schemas.dmtf.org/ovf/envelope/1>
< xmlns="http://schemas.dmtf.org/ovf/envelope/1" <http://schemas.dmtf.org/ovf/envelope/1> < xml:lang="en-US">
...
-----Original Message----- From: occi-wg-bounces@ogf.org [mailto:occi-wg <occi-wg>-bounces@ogf.org] On Behalf Of Alexis Richardson Sent: 20 October 2009 09:49 To: Sam Johnston Cc: Tim Bray; occi-wg@ogf.org Subject: Re: [occi-wg] confusion about status of link / headers
Sam,
Please don't throw around words like "bunk" willy nilly.
I have just had a look athttp://www.rackspacecloud.com/cloud_hosting_products/ servers/ api
I can see some limited use of headers for metadata in a few places. That's never been controversial. But the understanding that I have of your proposal is to use headers wholesale for all metadata. Rackspace don't appear to do that, unless I have misunderstood their document. I also may have misunderstood your proposal. If nobody understands your proposal except you, then it will not be possible to gain consensus around it.
Are you saying "OCCI metadata should be more like Rackspace metadata"?
alexis
On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston<samj@samj.net> <samj@samj.net> wrote:
On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson<alexis.richardson@gmail.com> <alexis.richardson@gmail.com> wrote:
What about compute clouds?
Rackspace Cloud API for a start:
HTTP/1.1 204 No Content Date: Mon, 12 Nov 2007 15:32:21 GMT Server: Apache X-Server-Management-Url:https://servers.api.rackspacecloud.com/v1.0/35428 X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-CDN-Management-Url:https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb Content-Length: 0 Content-Type: text/plain; charset=UTF-8
Microsoft Azure uses headers for "attributes" as well, in the same way as I had originally proposed:
x-ms-meta-name: value Returns a metadata value for the container.
However prefer that the header names are static/opaque rather than using a template and parsing them:
Attribute: name=value
This to me is cleaner and more self-explanatory, plus it is easily collapsed ala:
Attribute: name1=value1, name2=value2, ... namen=valuen
Anyway suffice to say that claims this is somehow "experimental" are bunk.
Sam
On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org
wrote:
Here are options for metadata used in some of the major storage clouds FWIW:
S3, Rackspace, EMC Atmos, Azure - Headers Nirvanix - query params in, xml entity out Mezeo - entity
Of the ones using headers, S3, Rackspace and Azure use prefix with values stored as-us. Atmos joins all metadata together into one header, making parsing trivial (split /,/), but necessary.
The most expensive option of the above is entity, where each metadata value is a separate GET. However, entities allow binary metadata and zero restrictions on it, which may be useful.
In jclouds, we time parsing of response values. A simple XML doc with only several elements written in SAX takes a few ms to parse. My log files are not precise enough to find the overhead in parsing headers: they always start and finish within the same millisecond.
I hope this background helps, and also helps explain why I'm vocal on such topics such as headers vs entities :)
Cheers, -Adrian
On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston<samj@samj.net> <samj@samj.net> wrote:
On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro<garymazzaferro@gmail.com> <garymazzaferro@gmail.com> wrote:
The http header and key/value pairs need to parsed also, there is no free ride here.
Every HTTP library I have ever used parses HTTP headers and puts them in a nice hash for you ready to consume. If we go for "Name: Value" then that's all there is to it. If we go for "Attribute: name=value" as is currently proposed (which is arguably cleaner, follows cookies' "prior art" and avoids Amazon's prefix hack) then you just have to split on '='.
To illustrate how clean this is by example:
#!/usr/bin/python import urllib2 response = urllib2.urlopen('http://cloud.example.com/ myvm') representation = response.read() metadata = response.info() print metadata['occi-compute-cores']
As soon as you start talking about payloads you have to fire up a parser (JSON/XML/Atom/etc.) or write your own (previous text rendering) which is significantly more work to do at both design and run times. Not to mention more work for us to do now and more scope for interoperability problems.
Sam
_______________________________________________ occi-wg mailing listocci-wg@ogf.orghttp://www.ogf.org/mailman/listinfo/occi-wg
_______________________________________________ occi-wg mailing listocci-wg@ogf.orghttp://www.ogf.org/mailman/listinfo/occi-wg
_______________________________________________ occi-wg mailing listocci-wg@ogf.orghttp://www.ogf.org/mailman/listinfo/occi-wg --- ---------------------------------------------------------- Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
_______________________________________________ occi-wg mailing listocci-wg@ogf.orghttp://www.ogf.org/mailman/listinfo/occi-wg
_______________________________________________ occi-wg mailing listocci-wg@ogf.orghttp://www.ogf.org/mailman/listinfo/occi-wg
_______________________________________________ occi-wg mailing listocci-wg@ogf.orghttp://www.ogf.org/mailman/listinfo/occi-wg
-- <http://www.sun.com> * Mark A. Carlson * Sr. Architect
*Systems Group* Phone x69559 / 303-223-6139 Email Mark.Carlson@Sun.COM
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
participants (4)
-
Adrian Cole -
Alexis Richardson -
Mark A. Carlson -
Sam Johnston