Please don't throw around words like "bunk" willy nilly.

I tell you what... I'll resist the urge to declare your arguments "bunk" if you resist the urge to declare my proposals "experimental", "new technology", not "prior art" and whatever other weasel words are being tossed around to imply that they are somehow risky. To quote McNealy for the second time today “If everybody believes in your strategy you have zero chance of [success]" - change that matters is always controversial.

I have just had a look at
http://www.rackspacecloud.com/cloud_hosting_products/servers/api

I can see some limited use of headers for metadata in a few places.
That's never been controversial. But the understanding that I have of
your proposal is to use headers wholesale for all metadata. Rackspace
don't appear to do that, unless I have misunderstood their document.

Rackspace have taken on the creation of both a format and a protocol combined (note the absence of renderings like OVF). That's wrong for many reasons, not the least of which include creating significant extra work for yourself, not playing nice with other standards, raising the bar for implementers on both sides of the interface, creating more opportunities for interoperability problems... I could go on. However it suited them well because their task was to expose the relatively limited functionality of a single implementation, not to satisfy the needs of many. Also, whether they use headers for a single attribute or all of them is irrelevant - it still proves headers are safe for such interfaces.

We made a conscious decision early on to embrace existing standards such as OVF and I've provided a way for us to allow them to coexist and compete on merit by steering well clear of the HTTP entity-body. The non-Atom alternative requires us to follow their example by developing both protocol and format, with the additional hurdle that our format must also be compatible and compete with others like OVF. It's a mess - I know because I spent quite some time trying to make it work. Without changing (e.g. complicating) the model you've got different representations for the same resource telling different stories which is just plain wrong.

In any case, cloud storage services including both Azure and S3 (which can certainly be considered cloud infrastructure) do use HTTP headers wholesale for metadata. HTTP intends for headers to be used for metadata too, and provides for arbitrary extension to suit.

I also may have misunderstood your proposal. If nobody understands
your proposal except you, then it will not be possible to gain
consensus around it.

If indeed people don't understand the proposal then that is something we need to work on. It's very intuitive for anyone who has worked with web technologies:

Instances of objects we model are "resources" which are abstract & can't be retrieved directly (e.g. you can't pull a physical machine over an API, yet)
You can however GET "representations" of resources in one or more existing standard formats (e.g. OVF, VMDK, etc.)
Some representations will be better than others at telling the "full story" while others may be quite lossy (ala JPEG).
The non-Atom alternative is to create an extremely lossy representation that must be retrieved in addition to the "native" representation (e.g. OVF), which includes information not available in other representations, and which may conflict with other representation e.g. current resources allocated (which may be very dynamic and differ from the saved state).
Metadata including the organisation of the resources into categories and an interconnected graph, as well as owner information, related resources, possible state transitions, etc. needs to be captured somewhere.
HTTP provides a perfectly good metadata channel in the form of HTTP headers, whereby a single CRLF character (e.g. a blank line) delineates the metadata from the data itself.
HTTP also provides a convenient way to access the metadata alone (e.g. HTTP HEAD). This is useful for tasks like enumeration/discovery which don't require clients to have the representation itself.
HTTP also provides many optimisations for common tasks such as COPYing templates and MOVEing resources between clouds.

Basically my proposal is to, wherever possible, use HTTP exactly as it was originally intended, deviating only where we absolutely have to.

Are you saying "OCCI metadata should be more like Rackspace metadata"?

No, just that Rackspace (among others) validate the use of HTTP headers in cloud APIs.

Sam

On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> wrote:
> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson
> <alexis.richardson@gmail.com> wrote:
>>
>> What about compute clouds?
>
> Rackspace Cloud API for a start:
>
>> HTTP/1.1 204 No Content
>> Date: Mon, 12 Nov 2007 15:32:21 GMT
>> Server: Apache
>> X-Server-Management-Url: https://servers.api.rackspacecloud.com/v1.0/35428
>> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4
>> X-CDN-Management-Url: https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4
>> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
>> Content-Length: 0
>> Content-Type: text/plain; charset=UTF-8
>
>
> Microsoft Azure uses headers for "attributes" as well, in the same way as I
> had originally proposed:
>
> x-ms-meta-name: value Returns a metadata value for the container.
>
> However prefer that the header names are static/opaque rather than using a
> template and parsing them:
>
>> Attribute: name=value
>
> This to me is cleaner and more self-explanatory, plus it is easily collapsed
> ala:
>
>> Attribute: name1=value1, name2=value2, ... namen=valuen
>
> Anyway suffice to say that claims this is somehow "experimental" are bunk.
>
> Sam
>
>> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org> wrote:
>> > Here are options for metadata used in some of the major storage clouds
>> > FWIW:
>> >
>> > S3, Rackspace, EMC Atmos, Azure - Headers
>> > Nirvanix - query params in, xml entity out
>> > Mezeo - entity
>> >
>> > Of the ones using headers, S3, Rackspace and Azure use prefix with
>> > values stored as-us. Atmos joins all metadata together into one
>> > header, making parsing trivial (split /,/), but necessary.
>> >
>> > The most expensive option of the above is entity, where each metadata
>> > value is a separate GET. However, entities allow binary metadata and
>> > zero restrictions on it, which may be useful.
>> >
>> > In jclouds, we time parsing of response values. A simple XML doc with
>> > only several elements written in SAX takes a few ms to parse. My log
>> > files are not precise enough to find the overhead in parsing headers:
>> > they always start and finish within the same millisecond.
>> >
>> > I hope this background helps, and also helps explain why I'm vocal on
>> > such topics such as headers vs entities :)
>> >
>> > Cheers,
>> > -Adrian
>> >
>> >
>> > On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> wrote:
>> >> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro
>> >> <garymazzaferro@gmail.com>
>> >> wrote:
>> >>
>> >>> The http header and key/value pairs need to parsed also, there is no
>> >>> free
>> >>> ride here.
>> >>
>> >> Every HTTP library I have ever used parses HTTP headers and puts them
>> >> in a
>> >> nice hash for you ready to consume. If we go for "Name: Value" then
>> >> that's
>> >> all there is to it. If we go for "Attribute: name=value" as is
>> >> currently
>> >> proposed (which is arguably cleaner, follows cookies' "prior art" and
>> >> avoids
>> >> Amazon's prefix hack) then you just have to split on '='.
>> >>
>> >> To illustrate how clean this is by example:
>> >>
>> >>> #!/usr/bin/python
>> >>> import urllib2
>> >>> response = urllib2.urlopen('http://cloud.example.com/myvm')
>> >>> representation = response.read()
>> >>> metadata = response.info()
>> >>> print metadata['occi-compute-cores']
>> >>
>> >> As soon as you start talking about payloads you have to fire up a
>> >> parser
>> >> (JSON/XML/Atom/etc.) or write your own (previous text rendering) which
>> >> is
>> >> significantly more work to do at both design and run times. Not to
>> >> mention
>> >> more work for us to do now and more scope for interoperability
>> >> problems.
>> >>
>> >> Sam
>> >>
>> >>
>> >> _______________________________________________
>> >> occi-wg mailing list
>> >> occi-wg@ogf.org
>> >> http://www.ogf.org/mailman/listinfo/occi-wg
>> >>
>> >>
>> > _______________________________________________
>> > occi-wg mailing list
>> > occi-wg@ogf.org
>> > http://www.ogf.org/mailman/listinfo/occi-wg
>> >
>
>