At IETF meetings, I've seen the chairs ask people to Hum for one proposal or the other - indicating both number of proponents and their strength of conviction... ;-) -- mark Adrian Cole wrote:
Ahh, ok (sorry... still an OCCI newbie). Disregard my last call to arms.
-Adrian
On Tue, Oct 20, 2009 at 9:56 AM, Sam Johnston <samj@samj.net> wrote:
Adrian,
We don't vote... we discuss our way to consensus and escalate if we fail.
Anyway I've had enough for one day/lifetime and I've got a house full of furniture to build tonight. I do wonder how many people here have actually bothered to read the spec and understand the issues though - if you haven't already then please do so now and speak up if anything doesn't make sense.
Sent from my iPhone
On 20/10/2009, at 18:44, Adrian Cole <adrian@jclouds.org> wrote:
As we aren't introducing new information, I think we are at a point where a vote can take place. This issue isn't infinitely complex, and reasonable to model in choices. OCCI desires to be a community process, so let's see where we fall on this.
Do you agree? -Adrian
On Tue, Oct 20, 2009 at 9:39 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Sam,
On Tue, Oct 20, 2009 at 5:31 PM, Sam Johnston <samj@samj.net> wrote:
Alexis,
Sweeping, unjustified statements are intensely frustrating, not to mention unhelpful in the absence of a coherent alternative. If people choose not to disclose conflicts I happen to be aware of then you bet I'll disclose them for them - nothing personal about it.
I don't know what 'conflicts' you are referring to, or how these relate to the matter at hand. If by 'sweeping, unjustified' you refer to the word 'unprecedented' then I shall justify it by pointing out that - at least as I currently understand it - you are proposing use of headers to a *greater* extent than has been done before and to cover a *greater* set of use cases.
Cutting to the chase, both Microsoft and Amazon are using headers for metadata to the same extent as what has been proposed.
Really - to the "same" extent? That seems contrary to what has been asserted on this thread previously.
What premature optimisation? *Not* using XML? Do we really need to drag that carcass out again? Surely Adrian's performance tests put the last nail in that particular coffin.
Premature optimisation = making performance a critical design concern when we want an API.
I have noted Adrian's points as have others.
The question really is about whether we want to be an interface/ protocol like HTTP or some mongrel hybrid of that along with a set of XML, JSON and/or text formats for everything we ever intend to model (knowing that work has already been done by others).
Work that has been done by others and shown to work, is a good thing.
alexis
Sent from my iPhone
On 20/10/2009, at 18:10, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Sam
Cut the personal stuff please.
Re 'unproven', I think the issues people are having are:
* The extent, scope and purpose of your application of headers to metadata is unprecedented - as I understand it. Nobody is saying it has not been done in other and restricted cases.
* Premature optimisation.
alexis
On Tue, Oct 20, 2009 at 4:44 PM, Sam Johnston <samj@samj.net> wrote:
> Gary, > > Please be specific with your objections so as I can have some > hope of > answering them... "several other emails" does not suffice as to > the > best of my knowledge there are no outstanding issues that I have > not > comprehensively and convincingly dealt with. > > Your repeated assertions than HTTP headers are... what is it > now... > "unproven"... are demonstrably unfounded (refer to the various > cloud > APIs referred to by myself and Adrian, including both MS Azure and > Amazon S3, as well as the history of the Internet in general) and > given the frequency, bordering on FUD. > > Regarding performance, OCCI should strive to be as efficient as > possible as it primarily targets public cloud providers who > operate > at > arbitrarily large "cloud" scale. The existing specification is > *extremely* efficient and alternatives would at least want to be > in > the same neighbourhood. Efficiency may be an inconvenient factor > for > whatever your alternative is, but it's critical for many of our > use > cases. > > Finally, you reject OVF without providing any alternative beyond > building a replacement from scratch ourselves, after our second > and > final deadline has just sailed by no less. Have you considered the > implementation cost of this? And in the same breath you accuse my > proposal of being "unproven"? Give me a break... > > Correct me if I'm wrong but the way I see it is that you want to > revert the last 2-3 months of my work because you have an > investment > in an old version of the spec. With each day I spend on OCCI > costing > me well over €1,000 excuse me for being pissed about someon > e with > such a blatantly obvious conflict attacking my work and refusing > to > provide a coherent argument, particularly with no "camera ready" > alternative anywhere in sight. > > Sent from my iPhone > > On 20/10/2009, at 16:51, Gary Mazz <garymazzaferro@gmail.com> > wrote: > > >> I've understood Sam's proposal from the beginning. I have had >> issue >> with from the beginning. >> >> Although I find it very interesting and valuable in terms of >> economy, I have issue with it as the OCCI's primary interface >> implementation for a few reasons which have been outlined in >> several >> other emails. >> >> It appears that Sam has created a new technology, a lower cost >> RPC >> mechanism, which is unproven for both small and wide scale >> deployment. For interoperability, IMO, occi should focus on >> proven >> mainstream technologies as the primary interfaces. I am not >> suggesting, shelving or not supporting Sam's proposal, based on >> market acceptance, it may prove to be one of many occi interface >> implementations. >> >> As for cost saving in interface technologies, I do not buy into >> the >> idea that the occi traffic will be so overwhelming that there >> is a >> requirement to maximally optimize http requests and responses. >> IMO, >> in comparison to images and consumer data occi traffic will be >> insignificant. I think we are worrying about spilled milk while >> the >> house is burning. >> >> There has been some discussion on supporting technologies >> including >> OVF. Whether occi supports OVF has not been fully discussed, >> examined or any consensus reached. I believe adopting >> technologies >> like OVF will only lead to mapping issues between occi attributes >> and the external formats creating a long term management >> headache. >> >> cheers, >> gary >> >> >> Alexis Richardson wrote: >> >>> OK >>> >>> Is Sam's statement something that we can achieve consensus >>> around? >>> >>> a >>> >>> >>> On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> >>> wrote: >>> >>> >>>> Alexis, >>>> >>>> Seems our posts crossed in the mail. I hope the last one clears >>>> things up >>>> but the fact that you're using OVF and XML/JSON interchangeably >>>> suggests >>>> that you [are possibly one of many who] don't grok the >>>> proposal. >>>> Worse, it >>>> seems you've got it back to front in thinking of the "core" as >>>> metadata, >>>> which is not at all what I had intended. >>>> >>>> The focus is on the resources and their various >>>> representations. >>>> The >>>> metadata (that is, data about data) essentially provides >>>> whatever is >>>> necessary to "glue" these resources together. It is basically >>>> using HTTP as >>>> the meta-model rather than an envelope format like Atom/SOAP >>>> or, >>>> worse, a >>>> meta-model that we have to create ourselves from scratch. >>>> >>>> So it's more like: >>>> >>>> 1) OCCI is compatible with ALL existing formats, eg OVF >>>> >>>> 2) Existing formats are used are used for ALL representations >>>> >>>> 3) ALL additional metadata should be in the headers >>>> >>>> 4) ALL of the core stuff should be in the body (that is, the >>>> metadata in the >>>> headers is just there to support the data and wire it up in >>>> useful >>>> ways) >>>> >>>> I think the confusion stems from your assuming that there is >>>> still >>>> XML/JSON/Atom/etc. representations. There aren't. HTTP headers >>>> provide a >>>> single, performant, interoperable metadata channel that has >>>> already been >>>> both standardised and extensively implemented. >>>> >>>> If you've followed me this far then you should have realised >>>> that >>>> every HTTP >>>> user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, >>>> etc.) >>>> is already >>>> a basic OCCI client out-of-the-box. The result is that 5-line >>>> scripts like >>>> demo.py are functional with no dependence on client libraries >>>> (that would >>>> absolutely be necessary if you were to start trying to define >>>> your >>>> own >>>> formats). >>>> >>>> Sam >>>> >>>> PS I'm sorry my "bunk" comment came across as referring to you >>>> personally - >>>> it was directed at the argument. >>>> >>>> On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson >>>> <alexis.richardson@gmail.com> wrote: >>>> >>>> >>>>> I think what Sam is proposing is the following: >>>>> >>>>> 1) If OCCI can handle 'any' data representation, eg OVF, XML, >>>>> JSON, >>>>> then it needs some core / common model. Otherwise: there can >>>>> be no >>>>> defined behaviour / interop >>>>> >>>>> 2) This core / common model is ALL metadata >>>>> >>>>> 3) ALL this metadata should be in the headers >>>>> >>>>> 4) ALL the non core stuff eg OVF payload, XML representation, >>>>> should be in >>>>> body >>>>> >>>>> Sam - is that right? >>>>> >>>>> a >>>>> >>>>> >>>>> On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> >>>>> wrote: >>>>> >>>>> >>>>>> On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX >>>>>> <andrewx.edmonds@intel.com> wrote: >>>>>> >>>>>> >>>>>>> To that point I'd ask - If we're talking about meta-data >>>>>>> that >>>>>>> infers >>>>>>> there >>>>>>> is some data to accompany it. Where are the examples of this >>>>>>> data? This >>>>>>> would help in forming the full picture. >>>>>>> >>>>>>> >>>>>> Here, from the core spec (using OVF as an example, but >>>>>> could be >>>>>> anything). >>>>>> Request is green, response metadata is yellow & response >>>>>> data/ >>>>>> payload is >>>>>> orange: >>>>>> >>>>>> >>>>>> >>>>>>> GET /us-east/webapp/vm01 HTTP/1.1 >>>>>>> User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0 >>>>>>> Host: cloud.example.com >>>>>>> Accept: */* >>>>>>> >>>>>>> >>>>>> < HTTP/1.1 200 OK >>>>>> < Date: Sat, 10 Oct 2009 12:56:51 GMT >>>>>> >>>>>> < Content-Type: application/ovf >>>>>> < Link: </us-east/webapp/vm01;start>; >>>>>> >>>>>> < rel="http://purl.org/occi/action/start"; >>>>>> < title="Start" >>>>>> >>>>>> < Link: </us-east/webapp/build.pdf>; >>>>>> < rel="related"; >>>>>> >>>>>> < title="Documentation"; >>>>>> < type="application/pdf" >>>>>> >>>>>> < Category: compute; >>>>>> < label="Compute Resource"; >>>>>> >>>>>> < scheme="http://purl.org/occi/kind/" >>>>>> < Server: occi-server/1.0 (linux) OCCI/1.0 >>>>>> >>>>>> < Connection: close >>>>>> < >>>>>> < <?xml version="1.0" encoding="UTF-8"?> >>>>>> >>>>>> < <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema- >>>>>> instance" >>>>>> < xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/ >>>>>> 1" >>>>>> >>>>>> < xmlns="http://schemas.dmtf.org/ovf/envelope/1" >>>>>> < xml:lang="en-US"> >>>>>> >>>>>> ... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: occi-wg-bounces@ogf.org [mailto:occi-wg- >>>>>>> bounces@ogf.org] On >>>>>>> Behalf >>>>>>> Of Alexis Richardson >>>>>>> Sent: 20 October 2009 09:49 >>>>>>> To: Sam Johnston >>>>>>> Cc: Tim Bray; occi-wg@ogf.org >>>>>>> Subject: Re: [occi-wg] confusion about status of link / >>>>>>> headers >>>>>>> >>>>>>> Sam, >>>>>>> >>>>>>> Please don't throw around words like "bunk" willy nilly. >>>>>>> >>>>>>> I have just had a look at >>>>>>> http://www.rackspacecloud.com/cloud_hosting_products/ >>>>>>> servers/ >>>>>>> api >>>>>>> >>>>>>> I can see some limited use of headers for metadata in a few >>>>>>> places. >>>>>>> That's never been controversial. But the understanding >>>>>>> that I >>>>>>> have of >>>>>>> your proposal is to use headers wholesale for all metadata. >>>>>>> Rackspace >>>>>>> don't appear to do that, unless I have misunderstood their >>>>>>> document. >>>>>>> I also may have misunderstood your proposal. If nobody >>>>>>> understands >>>>>>> your proposal except you, then it will not be possible to >>>>>>> gain >>>>>>> consensus around it. >>>>>>> >>>>>>> Are you saying "OCCI metadata should be more like Rackspace >>>>>>> metadata"? >>>>>>> >>>>>>> alexis >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston >>>>>>> <samj@samj.net> >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>>> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson >>>>>>>> <alexis.richardson@gmail.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>>> What about compute clouds? >>>>>>>>> >>>>>>>>> >>>>>>>> Rackspace Cloud API for a start: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> HTTP/1.1 204 No Content >>>>>>>>> Date: Mon, 12 Nov 2007 15:32:21 GMT >>>>>>>>> Server: Apache >>>>>>>>> X-Server-Management-Url: >>>>>>>>> https://servers.api.rackspacecloud.com/v1.0/35428 >>>>>>>>> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>>>>>>> X-CDN-Management-Url: >>>>>>>>> https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>>>>>>> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb >>>>>>>>> Content-Length: 0 >>>>>>>>> Content-Type: text/plain; charset=UTF-8 >>>>>>>>> >>>>>>>>> >>>>>>>> Microsoft Azure uses headers for "attributes" as well, in >>>>>>>> the >>>>>>>> same >>>>>>>> way >>>>>>>> as I >>>>>>>> had originally proposed: >>>>>>>> >>>>>>>> x-ms-meta-name: value Returns a metadata value for the >>>>>>>> container. >>>>>>>> >>>>>>>> However prefer that the header names are static/opaque >>>>>>>> rather >>>>>>>> than >>>>>>>> using >>>>>>>> a >>>>>>>> template and parsing them: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Attribute: name=value >>>>>>>>> >>>>>>>>> >>>>>>>> This to me is cleaner and more self-explanatory, plus it is >>>>>>>> easily >>>>>>>> collapsed >>>>>>>> ala: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Attribute: name1=value1, name2=value2, ... namen=valuen >>>>>>>>> >>>>>>>>> >>>>>>>> Anyway suffice to say that claims this is somehow >>>>>>>> "experimental" are >>>>>>>> bunk. >>>>>>>> >>>>>>>> Sam >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> Here are options for metadata used in some of the major >>>>>>>>>> storage >>>>>>>>>> clouds >>>>>>>>>> FWIW: >>>>>>>>>> >>>>>>>>>> S3, Rackspace, EMC Atmos, Azure - Headers >>>>>>>>>> Nirvanix - query params in, xml entity out >>>>>>>>>> Mezeo - entity >>>>>>>>>> >>>>>>>>>> Of the ones using headers, S3, Rackspace and Azure use >>>>>>>>>> prefix with >>>>>>>>>> values stored as-us. Atmos joins all metadata together >>>>>>>>>> into >>>>>>>>>> one >>>>>>>>>> header, making parsing trivial (split /,/), but >>>>>>>>>> necessary. >>>>>>>>>> >>>>>>>>>> The most expensive option of the above is entity, where >>>>>>>>>> each >>>>>>>>>> metadata >>>>>>>>>> value is a separate GET. However, entities allow binary >>>>>>>>>> metadata >>>>>>>>>> and >>>>>>>>>> zero restrictions on it, which may be useful. >>>>>>>>>> >>>>>>>>>> In jclouds, we time parsing of response values. A simple >>>>>>>>>> XML doc >>>>>>>>>> with >>>>>>>>>> only several elements written in SAX takes a few ms to >>>>>>>>>> parse. My >>>>>>>>>> log >>>>>>>>>> files are not precise enough to find the overhead in >>>>>>>>>> parsing >>>>>>>>>> headers: >>>>>>>>>> they always start and finish within the same millisecond. >>>>>>>>>> >>>>>>>>>> I hope this background helps, and also helps explain >>>>>>>>>> why I'm >>>>>>>>>> vocal >>>>>>>>>> on >>>>>>>>>> such topics such as headers vs entities :) >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> -Adrian >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston >>>>>>>>>> <samj@samj.net> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >>>>>>>>>>> <garymazzaferro@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> The http header and key/value pairs need to parsed >>>>>>>>>>>> also, >>>>>>>>>>>> there >>>>>>>>>>>> is >>>>>>>>>>>> no >>>>>>>>>>>> free >>>>>>>>>>>> ride here. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> Every HTTP library I have ever used parses HTTP >>>>>>>>>>> headers and >>>>>>>>>>> puts >>>>>>>>>>> them >>>>>>>>>>> in a >>>>>>>>>>> nice hash for you ready to consume. If we go for "Name: >>>>>>>>>>> Value" >>>>>>>>>>> then >>>>>>>>>>> that's >>>>>>>>>>> all there is to it. If we go for "Attribute: >>>>>>>>>>> name=value" as >>>>>>>>>>> is >>>>>>>>>>> currently >>>>>>>>>>> proposed (which is arguably cleaner, follows cookies' >>>>>>>>>>> "prior art" >>>>>>>>>>> and >>>>>>>>>>> avoids >>>>>>>>>>> Amazon's prefix hack) then you just have to split on >>>>>>>>>>> '='. >>>>>>>>>>> >>>>>>>>>>> To illustrate how clean this is by example: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> #!/usr/bin/python >>>>>>>>>>>> import urllib2 >>>>>>>>>>>> response = urllib2.urlopen('http://cloud.example.com/ >>>>>>>>>>>> myvm') >>>>>>>>>>>> representation = response.read() >>>>>>>>>>>> metadata = response.info() >>>>>>>>>>>> print metadata['occi-compute-cores'] >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> As soon as you start talking about payloads you have to >>>>>>>>>>> fire up a >>>>>>>>>>> parser >>>>>>>>>>> (JSON/XML/Atom/etc.) or write your own (previous text >>>>>>>>>>> rendering) >>>>>>>>>>> which >>>>>>>>>>> is >>>>>>>>>>> significantly more work to do at both design and run >>>>>>>>>>> times. >>>>>>>>>>> Not >>>>>>>>>>> to >>>>>>>>>>> mention >>>>>>>>>>> more work for us to do now and more scope for >>>>>>>>>>> interoperability >>>>>>>>>>> problems. >>>>>>>>>>> >>>>>>>>>>> Sam >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> occi-wg mailing list >>>>>>>>>>> occi-wg@ogf.org >>>>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> occi-wg mailing list >>>>>>>>>> occi-wg@ogf.org >>>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>> _______________________________________________ >>>>>>> occi-wg mailing list >>>>>>> occi-wg@ogf.org >>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>>> --- >>>>>>> ---------------------------------------------------------- >>>>>>> Intel Ireland Limited (Branch) >>>>>>> Collinstown Industrial Park, Leixlip, County Kildare, >>>>>>> Ireland >>>>>>> Registered Number: E902934 >>>>>>> >>>>>>> This e-mail and any attachments may contain confidential >>>>>>> material for >>>>>>> the sole use of the intended recipient(s). Any review or >>>>>>> distribution >>>>>>> by others is strictly prohibited. If you are not the >>>>>>> intended >>>>>>> recipient, please contact the sender and delete all copies. >>>>>>> >>>>>>> >>> _______________________________________________ >>> occi-wg mailing list >>> occi-wg@ogf.org >>> http://www.ogf.org/mailman/listinfo/occi-wg >>> >>> >>>
occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
-- <http://www.sun.com> * Mark A. Carlson * Sr. Architect *Systems Group* Phone x69559 / 303-223-6139 Email Mark.Carlson@Sun.COM