Alexis, I do not believe we can set aside the OGF27 unapproved unilateral change to the document. It is a fundamental problem in operation. These are my suggestions: 1) Place the document in a format where each interface can have its own life cycle. 2) Create a template for adjunct documents 3) Limit check-in to the repository to two people and one backup. 4) Recruit an editor and a backup editor. This spec isn't that big. 5) Provide an incubator for emerging technologies and proposals for inclusion in the specification. 6) Form a public mailing list purposed for emerging technologies, proposal discussions and voting 7) Provide a proposal document number for each submitted document 8) Before the document can be considered for review, it must be in the adjunct documents template -gary Alexis Richardson wrote:

Leaving aside the when and how in relation to OGF 27 specifically, we cannot claim that unilateral changes are supported by the group if they are not supported by the group. I am very concerned about this. We of course don't want to ignore innovative proposals, but we need to build consensus around them before inclusion in the spec.

Suggestions for a good way forward are solicited..

alexis

On Mon, Oct 19, 2009 at 4:19 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

That is a good point, a better question would be how did it get into the spec presented like it was the preferred method. Probably because there is no single editor and anyone can change the documents anyway they feel fit.

I don't think what Sam is working on is out of scope for OCCI, it was unintended to support multiple interfaces. Sam seems to be running with this one, driving OCCI to the lowest level of the HTTP protocols, essentially creating a new technology, untried on multiple levels in the internet infrastructure.

My issue with it is it was placed in the spec at the last second before OGF 27, other implementations were remove in lieu of this one, it was placed in the spec irrespective of a group consensus and SNIA, a strategic partner, publicly announcing they would NOT support this interface. However, this does not preclude the rest of this group to continue with the original concept of OCCI information in HTTP entities (content body). For maintainability, this does force the document to take on a new format of separating the implementations from reference model (we need one first). Interface implementations should fall into adjunct documents. This specification model has been successfully executed by numerous standards bodies.

cheers,

Gary

Alexis Richardson wrote:

...

Sam & group,

I just saw this tweet: http://twitter.com/samj/statuses/4991958514

You say that "HTTP has an out-of-band metadata channel in the form of headers. #occi's using Link: as a flexible, lightweight RDF alternative".

I'm a bit confused here.... I thought this was still under discussion. What am I missing?

alexis _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

On 2009-10-19, at 9:21 AM, Alexis Richardson wrote:

Gary

Thanks. That strikes me as a fairly complex process.

Does anyone have any alternative suggestions? We need a simple model for reaching consensus here, that grows the community and adoption.

In practice, I've had experience with three processes; ISO, W3C/Oasis, and IETF process. ISO is institutional voting, with complex threshold rules. W3C and Oasis individual members vote. Of course, this means you have to define who's a member and thus gets a vote. In the W3C, you argue for a while and then the chair (co-chairs usually) assert what the consensus is. Informally consensus is considered to be the absence of sustained intense reasonable resistance. If you disagree you appeal to the Area Director, the IESG, the IAB and eventually the Internet Society (I may have that appeal chain out of order). I prefer the IETF model but all have been observed to work. -Tim

On 2009-10-19, at 9:42 AM, Alexis Richardson wrote:

...

In practice, I've had experience with three processes; ... ... In the W3C, you argue for a while and then the chair (co-chairs usually) assert what the consensus is. Informally consensus is considered to be the absence of sustained intense reasonable resistance. If you disagree you appeal to the Area Director, the IESG, the IAB and eventually the Internet Society (I may have that appeal chain out of order).

Did you mean 'IETF' for this last item?

Urgh, right of course. Sorry -T

Well it sounds like at least three people, including myself, prefer the IETF model. Any other views? alexis On Mon, Oct 19, 2009 at 5:50 PM, Sam Johnston <samj@samj.net> wrote:

On Mon, Oct 19, 2009 at 6:42 PM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

Tim

Thank-you.  Quick question below...

On Mon, Oct 19, 2009 at 5:37 PM, Tim Bray <Tim.Bray@sun.com> wrote:

...

...

Does anyone have any alternative suggestions?  We need a simple model for reaching consensus here, that grows the community and adoption.

In practice, I've had experience with three processes; ... ...  In the W3C, you argue for a while and then the chair (co-chairs usually) assert what the consensus is.  Informally consensus is considered to be the absence of sustained intense reasonable resistance.  If you disagree you appeal to the Area Director, the IESG, the IAB and eventually the Internet Society (I may have that appeal chain out of order).

Did you mean 'IETF' for this last item?

Yes. Note that it's also my strong preference to follow the IETF's example, whereby discussion focusing on the technical merits of each alternative would continue until rough consensus is reached (as called by the chairs) with an appeal chain through the OGF in the unlikely event that it is needed.

The key thing is to stay focused on the technical pros and cons and leave all the other cruft (such as unhelpful REST religious debates) at the door.

Sam

On Mon, Oct 19, 2009 at 6:03 PM, Sam Johnston <samj@samj.net> wrote:

On Mon, Oct 19, 2009 at 6:53 PM, Alexis Richardson

Trying to build a standard from scratch is like trying to work out what colour to paint the bikeshed, as evidenced by discussions like this.

Yes, when we formed OCCI we agreed to minimise invention of new technology - obviously this is a 'judgement call'. The chairs should apply this principle when facilitating consensus. alexis

On Mon, Oct 19, 2009 at 10:46 PM, Sam Johnston <samj@samj.net> wrote:

On Mon, Oct 19, 2009 at 7:47 PM, Alexis Richardson

I think it's best you stick to calling the consensus based on discussions,

That's the plan I'd like: Discussions - on this list - about clear changes to a specific draft.

which hopefully you will also be contributing to (there's no harm in wearing both hats if you keep the roles separate).

Ha, I hope so.

Such a "test" is highly subjective and easily [ab]used to short circuit consensus and/or suppress ideas you don't personally understand or appreciate.

Yes, our current process is too open to such abuses.

Case in point is the unjustified claim that using HTTP headers for metadata is somehow experimental "new technology" when it was explicitly defined for this purpose by RFC2068 over a decade ago and used extensively since:

...

Entity-header fields define optional metainformation about the entity-body or, if no body is present, about the resource identified by the request.

If applying this to more metadata has been such a good idea for a decade, why wasn't it adopted much more? Answer - we don't know. This absence of certainty appears to have been a legitimate source of concern for many people.

Conversely the creation of a domain-specific language for each and every resource that we need to represent (at least 3 for infrastructure, 5-10+ for platforms and an infinite number for applications) and somehow keeping that in sync with authorative "native" representations like OVF is *far* more experimental, error prone and ultimately likely to fail.

We are doing infrastructure, and basing it as much as possible on prior art. alexis

On Mon, Oct 19, 2009 at 11:33 PM, Sam Johnston <samj@samj.net> wrote:

On Tue, Oct 20, 2009 at 12:07 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

If applying this to more metadata has been such a good idea for a decade, why wasn't it adopted much more?  Answer - we don't know. This absence of certainty appears to have been a legitimate source of concern for many people.

Likely because the information on the web was until now useful for humans but opaque to computers. As we're taking it to the next level with cloud, semweb, etc. our demands on the aging infrastructure are increasing and features such as the Link: header which were originally specified in HTTP back in 1997 are being revitalised.

By all means revitalise - but through a separate RFC showing the general pattern - then suggest applying it to OCCI and other specs.

Would it make you feel more comfortable if I were to tell you that Amazon have been using HTTP headers for arbitrary metadata for years? "In REST, user metadata keys must begin with "x-amz-meta-" to distinguish them as custom HTTP headers."

No it would not. Using some of this for S3 does not prove that it works for managing running images.

...

We are doing infrastructure, and basing it as much as possible on prior art.

We are doing infrastructure today but my proposed roadmap has provision for both platform and application layers tomorrow (which is why I went to great lengths to separate out OCCI Core after the fact). Sure we should take one step at a time but confining ourselves to infrastructure is not seeing the forest for the trees. Furthermore, the standard as it stands today can trivially cater for any kind of resource you throw at it because the data channel completely clean (there are no Atom or SOAP headers, XML, JSON or other formats that need parsing - everything you need is right there in your existing HTTP user agent).

This does not matter if we don't have consensus.

It's taken me 6 months pretty much full time to crack this particular nut and I'd very much appreciate if you were to give this work the consideration it deserves.

The consideration that it deserves is: LOTS. That is the problem.

As for "prior art", were that the case we'd have just rubber stamped one of many existing infrastructure APIs (and in doing so delivered one vendor a huge competitive advantage over all the others).

This is a non sequitur. alexis

On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro <garymazzaferro@gmail.com>wrote: The http header and key/value pairs need to parsed also, there is no free

ride here.

Every HTTP library I have ever used parses HTTP headers and puts them in a nice hash for you ready to consume. If we go for "Name: Value" then that's all there is to it. If we go for "Attribute: name=value" as is currently proposed (which is arguably cleaner, follows cookies' "prior art" and avoids Amazon's prefix hack) then you just have to split on '='. To illustrate how clean this is by example: #!/usr/bin/python

import urllib2 response = urllib2.urlopen('http://cloud.example.com/myvm') representation = response.read() metadata = response.info() print metadata['occi-compute-cores']

As soon as you start talking about payloads you have to fire up a parser (JSON/XML/Atom/etc.) or write your own (previous text rendering) which is * significantly* more work to do at both design and run times. Not to mention more work for us to do now and more scope for interoperability problems. Sam

What about compute clouds? On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org> wrote:

Here are options for metadata used in some of the major storage clouds FWIW:

S3, Rackspace, EMC Atmos, Azure - Headers Nirvanix - query params in, xml entity out Mezeo - entity

Of the ones using headers, S3, Rackspace and Azure use prefix with values stored as-us.  Atmos joins all metadata together into one header, making parsing trivial (split /,/), but necessary.

The most expensive option of the above is entity, where each metadata value is a separate GET.  However, entities allow binary metadata and zero restrictions on it, which may be useful.

In jclouds, we time parsing of response values.  A simple XML doc with only several elements written in SAX takes a few ms to parse.  My log files are not precise enough to find the overhead in parsing headers: they always start and finish within the same millisecond.

I hope this background helps, and also helps explain why I'm vocal on such topics such as headers vs entities :)

Cheers, -Adrian

On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro <garymazzaferro@gmail.com> wrote:

...

The http header and key/value pairs need to parsed also, there is no free ride here.

Every HTTP library I have ever used parses HTTP headers and puts them in a nice hash for you ready to consume. If we go for "Name: Value" then that's all there is to it. If we go for "Attribute: name=value" as is currently proposed (which is arguably cleaner, follows cookies' "prior art" and avoids Amazon's prefix hack) then you just have to split on '='.

To illustrate how clean this is by example:

...

#!/usr/bin/python import urllib2 response = urllib2.urlopen('http://cloud.example.com/myvm') representation = response.read() metadata = response.info() print metadata['occi-compute-cores']

As soon as you start talking about payloads you have to fire up a parser (JSON/XML/Atom/etc.) or write your own (previous text rendering) which is significantly more work to do at both design and run times. Not to mention more work for us to do now and more scope for interoperability problems.

Sam

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

Sam, Please don't throw around words like "bunk" willy nilly. I have just had a look at http://www.rackspacecloud.com/cloud_hosting_products/servers/api I can see some limited use of headers for metadata in a few places. That's never been controversial. But the understanding that I have of your proposal is to use headers wholesale for all metadata. Rackspace don't appear to do that, unless I have misunderstood their document. I also may have misunderstood your proposal. If nobody understands your proposal except you, then it will not be possible to gain consensus around it. Are you saying "OCCI metadata should be more like Rackspace metadata"? alexis On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> wrote:

On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

What about compute clouds?

Rackspace Cloud API for a start:

...

HTTP/1.1 204 No Content Date: Mon, 12 Nov 2007 15:32:21 GMT Server: Apache X-Server-Management-Url: https://servers.api.rackspacecloud.com/v1.0/35428 X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-CDN-Management-Url: https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb Content-Length: 0 Content-Type: text/plain; charset=UTF-8

Microsoft Azure uses headers for "attributes" as well, in the same way as I had originally proposed:

x-ms-meta-name: value     Returns a metadata value for the container.

However prefer that the header names are static/opaque rather than using a template and parsing them:

...

Attribute: name=value

This to me is cleaner and more self-explanatory, plus it is easily collapsed ala:

...

Attribute: name1=value1, name2=value2, ... namen=valuen

Anyway suffice to say that claims this is somehow "experimental" are bunk.

Sam

...

On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org> wrote:

...

Here are options for metadata used in some of the major storage clouds FWIW:

S3, Rackspace, EMC Atmos, Azure - Headers Nirvanix - query params in, xml entity out Mezeo - entity

Of the ones using headers, S3, Rackspace and Azure use prefix with values stored as-us.  Atmos joins all metadata together into one header, making parsing trivial (split /,/), but necessary.

The most expensive option of the above is entity, where each metadata value is a separate GET.  However, entities allow binary metadata and zero restrictions on it, which may be useful.

In jclouds, we time parsing of response values.  A simple XML doc with only several elements written in SAX takes a few ms to parse.  My log files are not precise enough to find the overhead in parsing headers: they always start and finish within the same millisecond.

I hope this background helps, and also helps explain why I'm vocal on such topics such as headers vs entities :)

Cheers, -Adrian

On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro <garymazzaferro@gmail.com> wrote:

...

The http header and key/value pairs need to parsed also, there is no free ride here.

Every HTTP library I have ever used parses HTTP headers and puts them in a nice hash for you ready to consume. If we go for "Name: Value" then that's all there is to it. If we go for "Attribute: name=value" as is currently proposed (which is arguably cleaner, follows cookies' "prior art" and avoids Amazon's prefix hack) then you just have to split on '='.

To illustrate how clean this is by example:

...

#!/usr/bin/python import urllib2 response = urllib2.urlopen('http://cloud.example.com/myvm') representation = response.read() metadata = response.info() print metadata['occi-compute-cores']

As soon as you start talking about payloads you have to fire up a parser (JSON/XML/Atom/etc.) or write your own (previous text rendering) which is significantly more work to do at both design and run times. Not to mention more work for us to do now and more scope for interoperability problems.

Sam

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX < andrewx.edmonds@intel.com> wrote:

To that point I'd ask - If we're talking about meta-data that infers there is some data to accompany it. Where are the examples of this data? This would help in forming the full picture.

Here, from the core spec <http://occi.googlecode.com/hg/docs/occi-core.html>(using OVF as an example, but could be anything). Request is green, response metadata is yellow & response data/payload is orange:

GET /us-east/webapp/vm01 HTTP/1.1 User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0 Host: cloud.example.com Accept: */*

< HTTP/1.1 200 OK < Date: Sat, 10 Oct 2009 12:56:51 GMT < Content-Type: application/ovf < Link: </us-east/webapp/vm01;start>; < rel="http://purl.org/occi/action/start"; < title="Start" < Link: </us-east/webapp/build.pdf>; < rel="related"; < title="Documentation"; < type="application/pdf" < Category: compute; < label="Compute Resource"; < scheme="http://purl.org/occi/kind/" < Server: occi-server/1.0 (linux) OCCI/1.0 < Connection: close < < <?xml version="1.0" encoding="UTF-8"?> < <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" < xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" < xmlns="http://schemas.dmtf.org/ovf/envelope/1" < xml:lang="en-US"> ...

-----Original Message----- From: occi-wg-bounces@ogf.org [mailto:occi-wg-bounces@ogf.org] On Behalf Of Alexis Richardson Sent: 20 October 2009 09:49 To: Sam Johnston Cc: Tim Bray; occi-wg@ogf.org Subject: Re: [occi-wg] confusion about status of link / headers

Sam,

Please don't throw around words like "bunk" willy nilly.

I have just had a look at http://www.rackspacecloud.com/cloud_hosting_products/servers/api

I can see some limited use of headers for metadata in a few places. That's never been controversial. But the understanding that I have of your proposal is to use headers wholesale for all metadata. Rackspace don't appear to do that, unless I have misunderstood their document. I also may have misunderstood your proposal. If nobody understands your proposal except you, then it will not be possible to gain consensus around it.

Are you saying "OCCI metadata should be more like Rackspace metadata"?

alexis

On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

What about compute clouds?

Rackspace Cloud API for a start:

...

HTTP/1.1 204 No Content Date: Mon, 12 Nov 2007 15:32:21 GMT Server: Apache X-Server-Management-Url: https://servers.api.rackspacecloud.com/v1.0/35428 X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-CDN-Management-Url: https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb Content-Length: 0 Content-Type: text/plain; charset=UTF-8

Microsoft Azure uses headers for "attributes" as well, in the same way as I had originally proposed:

x-ms-meta-name: value Returns a metadata value for the container.

However prefer that the header names are static/opaque rather than using a template and parsing them:

...

Attribute: name=value

This to me is cleaner and more self-explanatory, plus it is easily

collapsed > ala: > >> Attribute: name1=value1, name2=value2, ... namen=valuen > > Anyway suffice to say that claims this is somehow "experimental" are bunk. > > Sam > >> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole &lt;adrian@jclouds.org> wrote: >> > Here are options for metadata used in some of the major storage clouds >> > FWIW: >> > >> > S3, Rackspace, EMC Atmos, Azure - Headers >> > Nirvanix - query params in, xml entity out >> > Mezeo - entity >> > >> > Of the ones using headers, S3, Rackspace and Azure use prefix with >> > values stored as-us. Atmos joins all metadata together into one >> > header, making parsing trivial (split /,/), but necessary. >> > >> > The most expensive option of the above is entity, where each metadata >> > value is a separate GET. However, entities allow binary metadata and >> > zero restrictions on it, which may be useful. >> > >> > In jclouds, we time parsing of response values. A simple XML doc with >> > only several elements written in SAX takes a few ms to parse. My log >> > files are not precise enough to find the overhead in parsing headers: >> > they always start and finish within the same millisecond. >> > >> > I hope this background helps, and also helps explain why I'm vocal on >> > such topics such as headers vs entities :) >> > >> > Cheers, >> > -Adrian >> > >> > >> > On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston &lt;samj@samj.net> wrote: >> >> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >> >> &lt;garymazzaferro@gmail.com> >> >> wrote: >> >> >> >>> The http header and key/value pairs need to parsed also, there is no >> >>> free >> >>> ride here. >> >> >> >> Every HTTP library I have ever used parses HTTP headers and puts them >> >> in a >> >> nice hash for you ready to consume. If we go for "Name: Value" then >> >> that's >> >> all there is to it. If we go for "Attribute: name=value" as is >> >> currently >> >> proposed (which is arguably cleaner, follows cookies' "prior art" and >> >> avoids >> >> Amazon's prefix hack) then you just have to split on '='. >> >> >> >> To illustrate how clean this is by example: >> >> >> >>> #!/usr/bin/python >> >>> import urllib2 >> >>> response = urllib2.urlopen('http://cloud.example.com/myvm') >> >>> representation = response.read() >> >>> metadata = response.info() >> >>> print metadata['occi-compute-cores'] >> >> >> >> As soon as you start talking about payloads you have to fire up a >> >> parser >> >> (JSON/XML/Atom/etc.) or write your own (previous text rendering) which >> >> is >> >> significantly more work to do at both design and run times. Not to >> >> mention >> >> more work for us to do now and more scope for interoperability >> >> problems. >> >> >> >> Sam >> >> >> >> >> >> _______________________________________________ >> >> occi-wg mailing list >> >> occi-wg@ogf.org >> >> http://www.ogf.org/mailman/listinfo/occi-wg >> >> >> >> >> > _______________________________________________ >> > occi-wg mailing list >> > occi-wg@ogf.org >> > http://www.ogf.org/mailman/listinfo/occi-wg >> > > > _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg ------------------------------------------------------------- Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

Alexis, Seems our posts crossed in the mail. I hope the last one clears things up but the fact that you're using OVF and XML/JSON interchangeably suggests that you [are possibly one of many who] don't grok the proposal. Worse, it seems you've got it back to front in thinking of the "core" as metadata, which is not at all what I had intended. The focus is on the resources and their various representations. The metadata (that is, data about data) essentially provides whatever is necessary to "glue" these resources together. It is basically using HTTP as the meta-model rather than an envelope format like Atom/SOAP or, worse, a meta-model that we have to create ourselves from scratch. So it's more like: 1) OCCI is compatible with ALL existing formats, eg OVF 2) Existing formats are used are used for ALL representations 3) ALL additional metadata should be in the headers 4) ALL of the *core* stuff should be in the body (that is, the metadata in the headers is just there to support the data and wire it up in useful ways) I think the confusion stems from your assuming that there is still XML/JSON/Atom/etc. representations. There aren't. HTTP headers provide a single, performant, interoperable metadata channel that has already been both standardised and extensively implemented. If you've followed me this far then you should have realised that every HTTP user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, etc.) is already a basic OCCI client out-of-the-box. The result is that 5-line scripts like demo.py <http://occi.googlecode.com/hg/python/examples/demo.py> are functional with no dependence on client libraries (that would absolutely be necessary if you were to start trying to define your own formats). Sam PS I'm sorry my "bunk" comment came across as referring to you personally - it was directed at the argument. On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson < alexis.richardson@gmail.com> wrote:

I think what Sam is proposing is the following:

1) If OCCI can handle 'any' data representation, eg OVF, XML, JSON, then it needs some core / common model. Otherwise: there can be no defined behaviour / interop

2) This core / common model is ALL metadata

3) ALL this metadata should be in the headers

4) ALL the non core stuff eg OVF payload, XML representation, should be in body

Sam - is that right?

a

...

On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX <andrewx.edmonds@intel.com> wrote:

...

To that point I'd ask - If we're talking about meta-data that infers

...

...

is some data to accompany it. Where are the examples of this data? This would help in forming the full picture.

Here, from the core spec (using OVF as an example, but could be anything). Request is green, response metadata is yellow & response data/payload is orange:

...

GET /us-east/webapp/vm01 HTTP/1.1 User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0

...

Host: cloud.example.com Accept: */*

...

< HTTP/1.1 200 OK < Date: Sat, 10 Oct 2009 12:56:51 GMT

< Content-Type: application/ovf < Link: </us-east/webapp/vm01;start>;

< rel="http://purl.org/occi/action/start"; < title="Start"

< Link: </us-east/webapp/build.pdf>; < rel="related";

< title="Documentation"; < type="application/pdf"

< Category: compute; < label="Compute Resource";

< scheme="http://purl.org/occi/kind/" < Server: occi-server/1.0 (linux) OCCI/1.0

< Connection: close < < <?xml version="1.0" encoding="UTF-8"?>

< <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" < xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"

< xmlns="http://schemas.dmtf.org/ovf/envelope/1" < xml:lang="en-US">

...

...

-----Original Message----- From: occi-wg-bounces@ogf.org [mailto:occi-wg-bounces@ogf.org] On

Behalf

...

Of Alexis Richardson Sent: 20 October 2009 09:49 To: Sam Johnston Cc: Tim Bray; occi-wg@ogf.org Subject: Re: [occi-wg] confusion about status of link / headers

Sam,

Please don't throw around words like "bunk" willy nilly.

I have just had a look at http://www.rackspacecloud.com/cloud_hosting_products/servers/api

I can see some limited use of headers for metadata in a few places. That's never been controversial. But the understanding that I have of your proposal is to use headers wholesale for all metadata. Rackspace don't appear to do that, unless I have misunderstood their document. I also may have misunderstood your proposal. If nobody understands your proposal except you, then it will not be possible to gain consensus around it.

Are you saying "OCCI metadata should be more like Rackspace metadata"?

alexis

On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

What about compute clouds?

Rackspace Cloud API for a start:

...

HTTP/1.1 204 No Content Date: Mon, 12 Nov 2007 15:32:21 GMT Server: Apache X-Server-Management-Url: https://servers.api.rackspacecloud.com/v1.0/35428 X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-CDN-Management-Url: https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb Content-Length: 0 Content-Type: text/plain; charset=UTF-8

Microsoft Azure uses headers for "attributes" as well, in the same way as I had originally proposed:

x-ms-meta-name: value Returns a metadata value for the container.

However prefer that the header names are static/opaque rather than using a template and parsing them:

...

Attribute: name=value

This to me is cleaner and more self-explanatory, plus it is easily collapsed ala:

...

Attribute: name1=value1, name2=value2, ... namen=valuen

Anyway suffice to say that claims this is somehow "experimental" are bunk.

Sam

...

On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org> wrote:

...

Here are options for metadata used in some of the major storage clouds FWIW:

S3, Rackspace, EMC Atmos, Azure - Headers Nirvanix - query params in, xml entity out Mezeo - entity

Of the ones using headers, S3, Rackspace and Azure use prefix with values stored as-us. Atmos joins all metadata together into one header, making parsing trivial (split /,/), but necessary.

The most expensive option of the above is entity, where each

...

...

...

...

...

value is a separate GET. However, entities allow binary metadata and zero restrictions on it, which may be useful.

In jclouds, we time parsing of response values. A simple XML doc with only several elements written in SAX takes a few ms to parse. My log files are not precise enough to find the overhead in parsing

...

...

...

...

...

they always start and finish within the same millisecond.

I hope this background helps, and also helps explain why I'm vocal on such topics such as headers vs entities :)

Cheers, -Adrian

On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> wrote: > On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro > <garymazzaferro@gmail.com> > wrote: > >> The http header and key/value pairs need to parsed also, there is >> no >> free >> ride here. > > Every HTTP library I have ever used parses HTTP headers and puts > them > in a > nice hash for you ready to consume. If we go for "Name: Value"

On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> wrote: there metadata headers: then

...

...

...

...

...

> that's > all there is to it. If we go for "Attribute: name=value" as is > currently > proposed (which is arguably cleaner, follows cookies' "prior art" > and > avoids > Amazon's prefix hack) then you just have to split on '='. > > To illustrate how clean this is by example: > >> #!/usr/bin/python >> import urllib2 >> response = urllib2.urlopen('http://cloud.example.com/myvm') >> representation = response.read() >> metadata = response.info() >> print metadata['occi-compute-cores'] > > As soon as you start talking about payloads you have to fire up a > parser > (JSON/XML/Atom/etc.) or write your own (previous text rendering) > which > is > significantly more work to do at both design and run times. Not to > mention > more work for us to do now and more scope for interoperability > problems. > > Sam > > > _______________________________________________ > occi-wg mailing list > occi-wg@ogf.org > http://www.ogf.org/mailman/listinfo/occi-wg > > _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg ------------------------------------------------------------- Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

I've understood Sam's proposal from the beginning. I have had issue with from the beginning. Although I find it very interesting and valuable in terms of economy, I have issue with it as the OCCI's primary interface implementation for a few reasons which have been outlined in several other emails. It appears that Sam has created a new technology, a lower cost RPC mechanism, which is unproven for both small and wide scale deployment. For interoperability, IMO, occi should focus on proven mainstream technologies as the primary interfaces. I am not suggesting, shelving or not supporting Sam's proposal, based on market acceptance, it may prove to be one of many occi interface implementations. As for cost saving in interface technologies, I do not buy into the idea that the occi traffic will be so overwhelming that there is a requirement to maximally optimize http requests and responses. IMO, in comparison to images and consumer data occi traffic will be insignificant. I think we are worrying about spilled milk while the house is burning. There has been some discussion on supporting technologies including OVF. Whether occi supports OVF has not been fully discussed, examined or any consensus reached. I believe adopting technologies like OVF will only lead to mapping issues between occi attributes and the external formats creating a long term management headache. cheers, gary Alexis Richardson wrote:

OK

Is Sam's statement something that we can achieve consensus around?

a

On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> wrote:

...

Alexis,

Seems our posts crossed in the mail. I hope the last one clears things up but the fact that you're using OVF and XML/JSON interchangeably suggests that you [are possibly one of many who] don't grok the proposal. Worse, it seems you've got it back to front in thinking of the "core" as metadata, which is not at all what I had intended.

The focus is on the resources and their various representations. The metadata (that is, data about data) essentially provides whatever is necessary to "glue" these resources together. It is basically using HTTP as the meta-model rather than an envelope format like Atom/SOAP or, worse, a meta-model that we have to create ourselves from scratch.

So it's more like:

1) OCCI is compatible with ALL existing formats, eg OVF

2) Existing formats are used are used for ALL representations

3) ALL additional metadata should be in the headers

4) ALL of the core stuff should be in the body (that is, the metadata in the headers is just there to support the data and wire it up in useful ways)

I think the confusion stems from your assuming that there is still XML/JSON/Atom/etc. representations. There aren't. HTTP headers provide a single, performant, interoperable metadata channel that has already been both standardised and extensively implemented.

If you've followed me this far then you should have realised that every HTTP user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, etc.) is already a basic OCCI client out-of-the-box. The result is that 5-line scripts like demo.py are functional with no dependence on client libraries (that would absolutely be necessary if you were to start trying to define your own formats).

Sam

PS I'm sorry my "bunk" comment came across as referring to you personally - it was directed at the argument.

On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

I think what Sam is proposing is the following:

1) If OCCI can handle 'any' data representation, eg OVF, XML, JSON, then it needs some core / common model. Otherwise: there can be no defined behaviour / interop

2) This core / common model is ALL metadata

3) ALL this metadata should be in the headers

4) ALL the non core stuff eg OVF payload, XML representation, should be in body

Sam - is that right?

a

On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX <andrewx.edmonds@intel.com> wrote:

...

To that point I'd ask - If we're talking about meta-data that infers there is some data to accompany it. Where are the examples of this data? This would help in forming the full picture.

Here, from the core spec (using OVF as an example, but could be anything). Request is green, response metadata is yellow & response data/payload is orange:

...

GET /us-east/webapp/vm01 HTTP/1.1 User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0

Host: cloud.example.com Accept: */*

< HTTP/1.1 200 OK < Date: Sat, 10 Oct 2009 12:56:51 GMT

< Content-Type: application/ovf < Link: </us-east/webapp/vm01;start>;

< rel="http://purl.org/occi/action/start"; < title="Start"

< Link: </us-east/webapp/build.pdf>; < rel="related";

< title="Documentation"; < type="application/pdf"

< Category: compute; < label="Compute Resource";

< scheme="http://purl.org/occi/kind/" < Server: occi-server/1.0 (linux) OCCI/1.0

< Connection: close < < <?xml version="1.0" encoding="UTF-8"?>

< <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" < xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"

< xmlns="http://schemas.dmtf.org/ovf/envelope/1" < xml:lang="en-US">

...

...

-----Original Message----- From: occi-wg-bounces@ogf.org [mailto:occi-wg-bounces@ogf.org] On Behalf Of Alexis Richardson Sent: 20 October 2009 09:49 To: Sam Johnston Cc: Tim Bray; occi-wg@ogf.org Subject: Re: [occi-wg] confusion about status of link / headers

Sam,

Please don't throw around words like "bunk" willy nilly.

I have just had a look at http://www.rackspacecloud.com/cloud_hosting_products/servers/api

I can see some limited use of headers for metadata in a few places. That's never been controversial. But the understanding that I have of your proposal is to use headers wholesale for all metadata. Rackspace don't appear to do that, unless I have misunderstood their document. I also may have misunderstood your proposal. If nobody understands your proposal except you, then it will not be possible to gain consensus around it.

Are you saying "OCCI metadata should be more like Rackspace metadata"?

alexis

On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

> What about compute clouds? > Rackspace Cloud API for a start:

> HTTP/1.1 204 No Content > Date: Mon, 12 Nov 2007 15:32:21 GMT > Server: Apache > X-Server-Management-Url: > https://servers.api.rackspacecloud.com/v1.0/35428 > X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 > X-CDN-Management-Url: > https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 > X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb > Content-Length: 0 > Content-Type: text/plain; charset=UTF-8 > Microsoft Azure uses headers for "attributes" as well, in the same way as I had originally proposed:

x-ms-meta-name: value Returns a metadata value for the container.

However prefer that the header names are static/opaque rather than using a template and parsing them:

> Attribute: name=value > This to me is cleaner and more self-explanatory, plus it is easily collapsed ala:

> Attribute: name1=value1, name2=value2, ... namen=valuen > Anyway suffice to say that claims this is somehow "experimental" are bunk.

Sam

> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org> > wrote: > >> Here are options for metadata used in some of the major storage >> clouds >> FWIW: >> >> S3, Rackspace, EMC Atmos, Azure - Headers >> Nirvanix - query params in, xml entity out >> Mezeo - entity >> >> Of the ones using headers, S3, Rackspace and Azure use prefix with >> values stored as-us. Atmos joins all metadata together into one >> header, making parsing trivial (split /,/), but necessary. >> >> The most expensive option of the above is entity, where each >> metadata >> value is a separate GET. However, entities allow binary metadata >> and >> zero restrictions on it, which may be useful. >> >> In jclouds, we time parsing of response values. A simple XML doc >> with >> only several elements written in SAX takes a few ms to parse. My >> log >> files are not precise enough to find the overhead in parsing >> headers: >> they always start and finish within the same millisecond. >> >> I hope this background helps, and also helps explain why I'm vocal >> on >> such topics such as headers vs entities :) >> >> Cheers, >> -Adrian >> >> >> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> >> wrote: >> >>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >>> <garymazzaferro@gmail.com> >>> wrote: >>> >>> >>>> The http header and key/value pairs need to parsed also, there >>>> is >>>> no >>>> free >>>> ride here. >>>> >>> Every HTTP library I have ever used parses HTTP headers and puts >>> them >>> in a >>> nice hash for you ready to consume. If we go for "Name: Value" >>> then >>> that's >>> all there is to it. If we go for "Attribute: name=value" as is >>> currently >>> proposed (which is arguably cleaner, follows cookies' "prior art" >>> and >>> avoids >>> Amazon's prefix hack) then you just have to split on '='. >>> >>> To illustrate how clean this is by example: >>> >>> >>>> #!/usr/bin/python >>>> import urllib2 >>>> response = urllib2.urlopen('http://cloud.example.com/myvm') >>>> representation = response.read() >>>> metadata = response.info() >>>> print metadata['occi-compute-cores'] >>>> >>> As soon as you start talking about payloads you have to fire up a >>> parser >>> (JSON/XML/Atom/etc.) or write your own (previous text rendering) >>> which >>> is >>> significantly more work to do at both design and run times. Not >>> to >>> mention >>> more work for us to do now and more scope for interoperability >>> problems. >>> >>> Sam >>> >>> >>> _______________________________________________ >>> occi-wg mailing list >>> occi-wg@ogf.org >>> http://www.ogf.org/mailman/listinfo/occi-wg >>> >>> >>> >> _______________________________________________ >> occi-wg mailing list >> occi-wg@ogf.org >> http://www.ogf.org/mailman/listinfo/occi-wg >> >>

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg ------------------------------------------------------------- Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

Sam Cut the personal stuff please. Re 'unproven', I think the issues people are having are: * The extent, scope and purpose of your application of headers to metadata is unprecedented - as I understand it. Nobody is saying it has not been done in other and restricted cases. * Premature optimisation. alexis On Tue, Oct 20, 2009 at 4:44 PM, Sam Johnston <samj@samj.net> wrote:

Gary,

Please be specific with your objections so as I can have some hope of answering them... "several other emails" does not suffice as to the best of my knowledge there are no outstanding issues that I have not comprehensively and convincingly dealt with.

Your repeated assertions than HTTP headers are... what is it now... "unproven"... are demonstrably unfounded (refer to the various cloud APIs referred to by myself and Adrian, including both MS Azure and Amazon S3, as well as the history of the Internet in general) and given the frequency, bordering on FUD.

Regarding performance, OCCI should strive to be as efficient as possible as it primarily targets public cloud providers who operate at arbitrarily large "cloud" scale. The existing specification is *extremely* efficient and alternatives would at least want to be in the same neighbourhood. Efficiency may be an inconvenient factor for whatever your alternative is, but it's critical for many of our use cases.

Finally, you reject OVF without providing any alternative beyond building a replacement from scratch ourselves, after our second and final deadline has just sailed by no less. Have you considered the implementation cost of this? And in the same breath you accuse my proposal of being "unproven"? Give me a break...

Correct me if I'm wrong but the way I see it is that you want to revert the last 2-3 months of my work because you have an investment in an old version of the spec. With each day I spend on OCCI costing me well over €1,000 excuse me for being pissed about someone with such a blatantly obvious conflict attacking my work and refusing to provide a coherent argument, particularly with no "camera ready" alternative anywhere in sight.

Sent from my iPhone

On 20/10/2009, at 16:51, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

I've understood Sam's proposal from the beginning. I have had issue with from the beginning.

Although I find it very interesting and valuable in terms of economy, I have issue with it as the OCCI's primary interface implementation for a few reasons which have been outlined in several other emails.

It appears that Sam has created a new technology, a lower cost RPC mechanism, which is unproven for both small and wide scale deployment. For interoperability, IMO,  occi should focus on proven mainstream technologies as the primary interfaces. I am not suggesting, shelving or not supporting Sam's proposal, based on market acceptance, it may prove to be one of many occi interface implementations.

As for cost saving in interface technologies, I do not buy into the idea that the occi traffic will be so overwhelming that there is a requirement to maximally optimize http requests and responses. IMO, in comparison to images and consumer data occi traffic will be insignificant. I think we are worrying about spilled milk while the house is burning.

There has been some discussion on supporting technologies including OVF. Whether occi supports OVF has not been fully discussed, examined or any consensus reached. I believe adopting technologies like OVF will only lead to mapping issues between occi attributes and the external formats creating a long term management headache.

cheers, gary

Alexis Richardson wrote:

...

OK

Is Sam's statement something that we can achieve consensus around?

a

On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> wrote:

...

Alexis,

Seems our posts crossed in the mail. I hope the last one clears things up but the fact that you're using OVF and XML/JSON interchangeably suggests that you [are possibly one of many who] don't grok the proposal. Worse, it seems you've got it back to front in thinking of the "core" as metadata, which is not at all what I had intended.

The focus is on the resources and their various representations. The metadata (that is, data about data) essentially provides whatever is necessary to "glue" these resources together. It is basically using HTTP as the meta-model rather than an envelope format like Atom/SOAP or, worse, a meta-model that we have to create ourselves from scratch.

So it's more like:

1) OCCI is compatible with ALL existing formats, eg OVF

2) Existing formats are used are used for ALL representations

3) ALL additional metadata should be in the headers

4) ALL of the core stuff should be in the body (that is, the metadata in the headers is just there to support the data and wire it up in useful ways)

I think the confusion stems from your assuming that there is still XML/JSON/Atom/etc. representations. There aren't. HTTP headers provide a single, performant, interoperable metadata channel that has already been both standardised and extensively implemented.

If you've followed me this far then you should have realised that every HTTP user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, etc.) is already a basic OCCI client out-of-the-box. The result is that 5-line scripts like demo.py are functional with no dependence on client libraries (that would absolutely be necessary if you were to start trying to define your own formats).

Sam

PS I'm sorry my "bunk" comment came across as referring to you personally - it was directed at the argument.

On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

I think what Sam is proposing is the following:

1) If OCCI can handle 'any' data representation, eg OVF, XML, JSON, then it needs some core / common model.  Otherwise: there can be no defined behaviour / interop

2) This core / common model is ALL metadata

3) ALL this metadata should be in the headers

4) ALL the non core stuff eg OVF payload, XML representation, should be in body

Sam - is that right?

a

On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX <andrewx.edmonds@intel.com> wrote:

> To that point I'd ask - If we're talking about meta-data that > infers > there > is some data to accompany it. Where are the examples of this > data? This > would help in forming the full picture. > Here, from the core spec (using OVF as an example, but could be anything). Request is green, response metadata is yellow & response data/ payload is orange:

> GET /us-east/webapp/vm01 HTTP/1.1 > User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0 >          Host: cloud.example.com > Accept: */* > < HTTP/1.1 200 OK < Date: Sat, 10 Oct 2009 12:56:51 GMT

< Content-Type: application/ovf < Link: </us-east/webapp/vm01;start>;

<       rel="http://purl.org/occi/action/start"; <       title="Start"

< Link: </us-east/webapp/build.pdf>; <       rel="related";

<       title="Documentation"; <       type="application/pdf"

< Category: compute; <       label="Compute Resource";

<       scheme="http://purl.org/occi/kind/" < Server: occi-server/1.0 (linux) OCCI/1.0

< Connection: close < < <?xml version="1.0" encoding="UTF-8"?>

< <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" <           xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"

<           xmlns="http://schemas.dmtf.org/ovf/envelope/1" <           xml:lang="en-US">

...

> -----Original Message----- > From: occi-wg-bounces@ogf.org [mailto:occi-wg-bounces@ogf.org] On > Behalf > Of Alexis Richardson > Sent: 20 October 2009 09:49 > To: Sam Johnston > Cc: Tim Bray; occi-wg@ogf.org > Subject: Re: [occi-wg] confusion about status of link / headers > > Sam, > > Please don't throw around words like "bunk" willy nilly. > > I have just had a look at > http://www.rackspacecloud.com/cloud_hosting_products/servers/api > > I can see some limited use of headers for metadata in a few > places. > That's never been controversial.  But the understanding that I > have of > your proposal is to use headers wholesale for all metadata. > Rackspace > don't appear to do that, unless I have misunderstood their > document. > I also may have misunderstood your proposal.  If nobody > understands > your proposal except you, then it will not be possible to gain > consensus around it. > > Are you saying "OCCI metadata should be more like Rackspace > metadata"? > > alexis > > > On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> > wrote: > >> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson >> <alexis.richardson@gmail.com> wrote: >> >>> What about compute clouds? >>> >> Rackspace Cloud API for a start: >> >> >>> HTTP/1.1 204 No Content >>> Date: Mon, 12 Nov 2007 15:32:21 GMT >>> Server: Apache >>> X-Server-Management-Url: >>> https://servers.api.rackspacecloud.com/v1.0/35428 >>> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>> X-CDN-Management-Url: >>> https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb >>> Content-Length: 0 >>> Content-Type: text/plain; charset=UTF-8 >>> >> Microsoft Azure uses headers for "attributes" as well, in the >> same >> way >> as I >> had originally proposed: >> >> x-ms-meta-name: value     Returns a metadata value for the >> container. >> >> However prefer that the header names are static/opaque rather >> than >> using >> a >> template and parsing them: >> >> >>> Attribute: name=value >>> >> This to me is cleaner and more self-explanatory, plus it is >> easily >> collapsed >> ala: >> >> >>> Attribute: name1=value1, name2=value2, ... namen=valuen >>> >> Anyway suffice to say that claims this is somehow >> "experimental" are >> bunk. >> >> Sam >> >> >>> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org >>> > >>> wrote: >>> >>>> Here are options for metadata used in some of the major >>>> storage >>>> clouds >>>> FWIW: >>>> >>>> S3, Rackspace, EMC Atmos, Azure - Headers >>>> Nirvanix - query params in, xml entity out >>>> Mezeo - entity >>>> >>>> Of the ones using headers, S3, Rackspace and Azure use >>>> prefix with >>>> values stored as-us.  Atmos joins all metadata together into >>>> one >>>> header, making parsing trivial (split /,/), but necessary. >>>> >>>> The most expensive option of the above is entity, where each >>>> metadata >>>> value is a separate GET.  However, entities allow binary >>>> metadata >>>> and >>>> zero restrictions on it, which may be useful. >>>> >>>> In jclouds, we time parsing of response values.  A simple >>>> XML doc >>>> with >>>> only several elements written in SAX takes a few ms to >>>> parse.  My >>>> log >>>> files are not precise enough to find the overhead in parsing >>>> headers: >>>> they always start and finish within the same millisecond. >>>> >>>> I hope this background helps, and also helps explain why I'm >>>> vocal >>>> on >>>> such topics such as headers vs entities :) >>>> >>>> Cheers, >>>> -Adrian >>>> >>>> >>>> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> >>>> wrote: >>>> >>>>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >>>>> <garymazzaferro@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>>> The http header and key/value pairs need to parsed also, >>>>>> there >>>>>> is >>>>>> no >>>>>> free >>>>>> ride here. >>>>>> >>>>> Every HTTP library I have ever used parses HTTP headers and >>>>> puts >>>>> them >>>>> in a >>>>> nice hash for you ready to consume. If we go for "Name: >>>>> Value" >>>>> then >>>>> that's >>>>> all there is to it. If we go for "Attribute: name=value" as >>>>> is >>>>> currently >>>>> proposed (which is arguably cleaner, follows cookies' >>>>> "prior art" >>>>> and >>>>> avoids >>>>> Amazon's prefix hack) then you just have to split on '='. >>>>> >>>>> To illustrate how clean this is by example: >>>>> >>>>> >>>>>> #!/usr/bin/python >>>>>> import urllib2 >>>>>> response = urllib2.urlopen('http://cloud.example.com/myvm') >>>>>> representation = response.read() >>>>>> metadata = response.info() >>>>>> print metadata['occi-compute-cores'] >>>>>> >>>>> As soon as you start talking about payloads you have to >>>>> fire up a >>>>> parser >>>>> (JSON/XML/Atom/etc.) or write your own (previous text >>>>> rendering) >>>>> which >>>>> is >>>>> significantly more work to do at both design and run times. >>>>> Not >>>>> to >>>>> mention >>>>> more work for us to do now and more scope for >>>>> interoperability >>>>> problems. >>>>> >>>>> Sam >>>>> >>>>> >>>>> _______________________________________________ >>>>> occi-wg mailing list >>>>> occi-wg@ogf.org >>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> occi-wg mailing list >>>> occi-wg@ogf.org >>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>> >>>> >> > _______________________________________________ > occi-wg mailing list > occi-wg@ogf.org > http://www.ogf.org/mailman/listinfo/occi-wg > ------------------------------------------------------------- > Intel Ireland Limited (Branch) > Collinstown Industrial Park, Leixlip, County Kildare, Ireland > Registered Number: E902934 > > This e-mail and any attachments may contain confidential > material for > the sole use of the intended recipient(s). Any review or > distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. >

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

What metrics are we comparing against and how can I help? I'd like to help remove roadblocks and give more information, even if it means spending my evening writing a test scenario. It is clear that you don't like this approach, but it is unclear what is condemning about this approach and how the older approach is preferred. Is it our internal status quo? (How much weight to we want to give to prior decisions when new information or approaches surface?) Is it impact to current users of OCCI? (How many users are actively developing against the OCCI spec? How much work are we creating for them when we tune the API?) Is it maturity? (How many years cloud experience do our technology choices require? Is 3 years too short?) Is it adoption (external status-quo)? (what percentage of current cloud adoption makes one choice too scary?) Is it that efficiency doesn't matter? (if so, when does it? how do we measure this?) Is it that later rewrites are not impactful? (if so, why is change so impactful now?) Is it that we are constraining something by using headers? (If so, what? How are other clouds dealing with that limitation?) Is it that we don't have budget for change? (What are we sacrificing by having this discussion?) I know you have very good reasons for your position. I'm sorry to burden you with answering these questions, but you are vocal in your opposition to this where others aren't so much. If you can help us understand what exactly is riding under your preference, we can all benefit and actually action those concerns. If we understand your concerns and find that option X doesn't in fact address them, then I think we are all better off. At the very least, we are dealing with more objective comparisons that take the weight of your arguments seriously. I hope this helps, -Adrian On Tue, Oct 20, 2009 at 2:15 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

Sam,

I will no longer address this issue under these terms. If you can't remember my statements, go back and look them up. This argument below is full of supposition. The onus is not on me to justify why it should not be admitted, but on you to to prove its value under the terms you outline below as well as others including a case study. Prove there is a need for a new technology for occi.  Where are the numbers ? Prove the technology will work in the large environments.

I keep repeating myself here, but again. I am not advocating to discard this interface. I am advocating this is not the only interface.  I sure this interface may be very valuable in the future once it trusted and proven to work on a global scale.

gary

Sam Johnston wrote:

...

Gary,

Please be specific with your objections so as I can have some hope of answering them... "several other emails" does not suffice as to the best of my knowledge there are no outstanding issues that I have not comprehensively and convincingly dealt with.

Your repeated assertions than HTTP headers are... what is it now... "unproven"... are demonstrably unfounded (refer to the various cloud APIs referred to by myself and Adrian, including both MS Azure and Amazon S3, as well as the history of the Internet in general) and given the frequency, bordering on FUD.

Regarding performance, OCCI should strive to be as efficient as possible as it primarily targets public cloud providers who operate at arbitrarily large "cloud" scale. The existing specification is *extremely* efficient and alternatives would at least want to be in the same neighbourhood. Efficiency may be an inconvenient factor for whatever your alternative is, but it's critical for many of our use cases.

Finally, you reject OVF without providing any alternative beyond building a replacement from scratch ourselves, after our second and final deadline has just sailed by no less. Have you considered the implementation cost of this? And in the same breath you accuse my proposal of being "unproven"? Give me a break...

Correct me if I'm wrong but the way I see it is that you want to revert the last 2-3 months of my work because you have an investment in an old version of the spec. With each day I spend on OCCI costing me well over €1,000 excuse me for being pissed about someone with such a blatantly obvious conflict attacking my work and refusing to provide a coherent argument, particularly with no "camera ready" alternative anywhere in sight.

Sent from my iPhone

On 20/10/2009, at 16:51, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

I've understood Sam's proposal from the beginning. I have had issue with from the beginning.

Although I find it very interesting and valuable in terms of economy, I have issue with it as the OCCI's primary interface implementation for a few reasons which have been outlined in several other emails.

It appears that Sam has created a new technology, a lower cost RPC mechanism, which is unproven for both small and wide scale deployment. For interoperability, IMO,  occi should focus on proven mainstream technologies as the primary interfaces. I am not suggesting, shelving or not supporting Sam's proposal, based on market acceptance, it may prove to be one of many occi interface implementations.

As for cost saving in interface technologies, I do not buy into the idea that the occi traffic will be so overwhelming that there is a requirement to maximally optimize http requests and responses. IMO, in comparison to images and consumer data occi traffic will be insignificant. I think we are worrying about spilled milk while the house is burning.

There has been some discussion on supporting technologies including OVF. Whether occi supports OVF has not been fully discussed, examined or any consensus reached. I believe adopting technologies like OVF will only lead to mapping issues between occi attributes and the external formats creating a long term management headache.

cheers, gary

Alexis Richardson wrote:

...

OK

Is Sam's statement something that we can achieve consensus around?

a

On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> wrote:

...

Alexis,

Seems our posts crossed in the mail. I hope the last one clears things up but the fact that you're using OVF and XML/JSON interchangeably suggests that you [are possibly one of many who] don't grok the proposal. Worse, it seems you've got it back to front in thinking of the "core" as metadata, which is not at all what I had intended.

The focus is on the resources and their various representations. The metadata (that is, data about data) essentially provides whatever is necessary to "glue" these resources together. It is basically using HTTP as the meta-model rather than an envelope format like Atom/SOAP or, worse, a meta-model that we have to create ourselves from scratch.

So it's more like:

1) OCCI is compatible with ALL existing formats, eg OVF

2) Existing formats are used are used for ALL representations

3) ALL additional metadata should be in the headers

4) ALL of the core stuff should be in the body (that is, the metadata in the headers is just there to support the data and wire it up in useful ways)

I think the confusion stems from your assuming that there is still XML/JSON/Atom/etc. representations. There aren't. HTTP headers provide a single, performant, interoperable metadata channel that has already been both standardised and extensively implemented.

If you've followed me this far then you should have realised that every HTTP user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, etc.) is already a basic OCCI client out-of-the-box. The result is that 5-line scripts like demo.py are functional with no dependence on client libraries (that would absolutely be necessary if you were to start trying to define your own formats).

Sam

PS I'm sorry my "bunk" comment came across as referring to you personally - it was directed at the argument.

On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

I think what Sam is proposing is the following:

1) If OCCI can handle 'any' data representation, eg OVF, XML, JSON, then it needs some core / common model.  Otherwise: there can be no defined behaviour / interop

2) This core / common model is ALL metadata

3) ALL this metadata should be in the headers

4) ALL the non core stuff eg OVF payload, XML representation, should be in body

Sam - is that right?

a

On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> wrote:

> On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX > <andrewx.edmonds@intel.com> wrote: > > >> To that point I'd ask - If we're talking about meta-data that >> infers >> there >> is some data to accompany it. Where are the examples of this >> data? This >> would help in forming the full picture. >> >> > Here, from the core spec (using OVF as an example, but could be > anything). > Request is green, response metadata is yellow & response data/ > payload is > orange: > > > >> GET /us-east/webapp/vm01 HTTP/1.1 >> User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0 >>          Host: cloud.example.com >> Accept: */* >> >> > < HTTP/1.1 200 OK > < Date: Sat, 10 Oct 2009 12:56:51 GMT > > < Content-Type: application/ovf > < Link: </us-east/webapp/vm01;start>; > > <       rel="http://purl.org/occi/action/start"; > <       title="Start" > > < Link: </us-east/webapp/build.pdf>; > <       rel="related"; > > <       title="Documentation"; > <       type="application/pdf" > > < Category: compute; > <       label="Compute Resource"; > > <       scheme="http://purl.org/occi/kind/" > < Server: occi-server/1.0 (linux) OCCI/1.0 > > < Connection: close > < > < <?xml version="1.0" encoding="UTF-8"?> > > < <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > <           xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" > > <           xmlns="http://schemas.dmtf.org/ovf/envelope/1" > <           xml:lang="en-US"> > > ... > > > > >> -----Original Message----- >> From: occi-wg-bounces@ogf.org [mailto:occi-wg-bounces@ogf.org] On >> Behalf >> Of Alexis Richardson >> Sent: 20 October 2009 09:49 >> To: Sam Johnston >> Cc: Tim Bray; occi-wg@ogf.org >> Subject: Re: [occi-wg] confusion about status of link / headers >> >> Sam, >> >> Please don't throw around words like "bunk" willy nilly. >> >> I have just had a look at >> http://www.rackspacecloud.com/cloud_hosting_products/servers/api >> >> I can see some limited use of headers for metadata in a few >> places. >> That's never been controversial.  But the understanding that I >> have of >> your proposal is to use headers wholesale for all metadata. >> Rackspace >> don't appear to do that, unless I have misunderstood their >> document. >> I also may have misunderstood your proposal.  If nobody >> understands >> your proposal except you, then it will not be possible to gain >> consensus around it. >> >> Are you saying "OCCI metadata should be more like Rackspace >> metadata"? >> >> alexis >> >> >> On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> >> wrote: >> >> >>> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson >>> <alexis.richardson@gmail.com> wrote: >>> >>> >>>> What about compute clouds? >>>> >>>> >>> Rackspace Cloud API for a start: >>> >>> >>> >>>> HTTP/1.1 204 No Content >>>> Date: Mon, 12 Nov 2007 15:32:21 GMT >>>> Server: Apache >>>> X-Server-Management-Url: >>>> https://servers.api.rackspacecloud.com/v1.0/35428 >>>> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>> X-CDN-Management-Url: >>>> https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 >>>> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb >>>> Content-Length: 0 >>>> Content-Type: text/plain; charset=UTF-8 >>>> >>>> >>> Microsoft Azure uses headers for "attributes" as well, in the >>> same >>> way >>> as I >>> had originally proposed: >>> >>> x-ms-meta-name: value     Returns a metadata value for the >>> container. >>> >>> However prefer that the header names are static/opaque rather >>> than >>> using >>> a >>> template and parsing them: >>> >>> >>> >>>> Attribute: name=value >>>> >>>> >>> This to me is cleaner and more self-explanatory, plus it is >>> easily >>> collapsed >>> ala: >>> >>> >>> >>>> Attribute: name1=value1, name2=value2, ... namen=valuen >>>> >>>> >>> Anyway suffice to say that claims this is somehow >>> "experimental" are >>> bunk. >>> >>> Sam >>> >>> >>> >>>> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org >>>> >>>> wrote: >>>> >>>> >>>>> Here are options for metadata used in some of the major >>>>> storage >>>>> clouds >>>>> FWIW: >>>>> >>>>> S3, Rackspace, EMC Atmos, Azure - Headers >>>>> Nirvanix - query params in, xml entity out >>>>> Mezeo - entity >>>>> >>>>> Of the ones using headers, S3, Rackspace and Azure use >>>>> prefix with >>>>> values stored as-us.  Atmos joins all metadata together into >>>>> one >>>>> header, making parsing trivial (split /,/), but necessary. >>>>> >>>>> The most expensive option of the above is entity, where each >>>>> metadata >>>>> value is a separate GET.  However, entities allow binary >>>>> metadata >>>>> and >>>>> zero restrictions on it, which may be useful. >>>>> >>>>> In jclouds, we time parsing of response values.  A simple >>>>> XML doc >>>>> with >>>>> only several elements written in SAX takes a few ms to >>>>> parse.  My >>>>> log >>>>> files are not precise enough to find the overhead in parsing >>>>> headers: >>>>> they always start and finish within the same millisecond. >>>>> >>>>> I hope this background helps, and also helps explain why I'm >>>>> vocal >>>>> on >>>>> such topics such as headers vs entities :) >>>>> >>>>> Cheers, >>>>> -Adrian >>>>> >>>>> >>>>> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> >>>>> wrote: >>>>> >>>>> >>>>>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >>>>>> <garymazzaferro@gmail.com> >>>>>> wrote: >>>>>> >>>>>> >>>>>> >>>>>>> The http header and key/value pairs need to parsed also, >>>>>>> there >>>>>>> is >>>>>>> no >>>>>>> free >>>>>>> ride here. >>>>>>> >>>>>>> >>>>>> Every HTTP library I have ever used parses HTTP headers and >>>>>> puts >>>>>> them >>>>>> in a >>>>>> nice hash for you ready to consume. If we go for "Name: >>>>>> Value" >>>>>> then >>>>>> that's >>>>>> all there is to it. If we go for "Attribute: name=value" as >>>>>> is >>>>>> currently >>>>>> proposed (which is arguably cleaner, follows cookies' >>>>>> "prior art" >>>>>> and >>>>>> avoids >>>>>> Amazon's prefix hack) then you just have to split on '='. >>>>>> >>>>>> To illustrate how clean this is by example: >>>>>> >>>>>> >>>>>> >>>>>>> #!/usr/bin/python >>>>>>> import urllib2 >>>>>>> response = urllib2.urlopen('http://cloud.example.com/myvm') >>>>>>> representation = response.read() >>>>>>> metadata = response.info() >>>>>>> print metadata['occi-compute-cores'] >>>>>>> >>>>>>> >>>>>> As soon as you start talking about payloads you have to >>>>>> fire up a >>>>>> parser >>>>>> (JSON/XML/Atom/etc.) or write your own (previous text >>>>>> rendering) >>>>>> which >>>>>> is >>>>>> significantly more work to do at both design and run times. >>>>>> Not >>>>>> to >>>>>> mention >>>>>> more work for us to do now and more scope for >>>>>> interoperability >>>>>> problems. >>>>>> >>>>>> Sam >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> occi-wg mailing list >>>>>> occi-wg@ogf.org >>>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> occi-wg mailing list >>>>> occi-wg@ogf.org >>>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>>> >>>>> >>>>> >> _______________________________________________ >> occi-wg mailing list >> occi-wg@ogf.org >> http://www.ogf.org/mailman/listinfo/occi-wg >> ------------------------------------------------------------- >> Intel Ireland Limited (Branch) >> Collinstown Industrial Park, Leixlip, County Kildare, Ireland >> Registered Number: E902934 >> >> This e-mail and any attachments may contain confidential >> material for >> the sole use of the intended recipient(s). Any review or >> distribution >> by others is strictly prohibited. If you are not the intended >> recipient, please contact the sender and delete all copies. >> >>

---

occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

Look Sam, If you feel that this is the best way for move occi forward, go for it. I'm iust trying to preserve the integrity of the work and support it properly in this group. I will be rolling out an occi interface or occi like interface publicly by Q1'10. I will not be deploying this proposed protocol until it is proven to work in a global environment. If that means abandoning occi, well, I'll abandon occi. I personally, feel I cannot and will not risk a customer's business or the livelihood of partners and their families on a single opinion and unproven technology. When you get real numbers from a global time study, I'll be glad to review them and support your efforts. Enjoy your night off. cheers gary Sam Johnston wrote:

Gary,

And here I was thinking I was getting the night off...

I am advocating this is not the only interface.

It's not the only interface - there's currently this one for machines and XHTML5 for users (and masochistic machines who like to parse XML). However it is the only interface users would have to implement (rather than having to code XML, JSON *and* text, which was the case before).

Anyway I see Adrian has kindly stepped up to discuss the details with you so I can get back to writing PyUnit unit tests.

Sam

Adrian, These issues are business issues and not appropriate discussion for this group. The issues about this technology have been brought up by myself and others. I'm done discussing this issue. If you trust S3, go for it.:-) -gary Adrian Cole wrote:

Hi, Gary.

I attended the AWS enterprise meetup in LA last week. The number of objects they mentioned that are stored in S3 is currently 70 billion and the service has been running for over 3.5 years. Granted that their "global story" is shorter, but they do 2 years presence in the EU. All 70 billion objects use headers for metadata.

What metadata system is more proven then this? What are the technical risks of using headers? Are there clients or systems that are unable to pass http headers, but are able to pass entities?

None of this matters, if the concern is the code impact to your application. However, I'd like to know the details of the broader concern you may be implying.

Thanks, -Adrian

On Tue, Oct 20, 2009 at 4:24 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

Look Sam,

If you feel that this is the best way for move occi forward, go for it. I'm iust trying to preserve the integrity of the work and support it properly in this group. I will be rolling out an occi interface or occi like interface publicly by Q1'10. I will not be deploying this proposed protocol until it is proven to work in a global environment. If that means abandoning occi, well, I'll abandon occi.

I personally, feel I cannot and will not risk a customer's business or the livelihood of partners and their families on a single opinion and unproven technology. When you get real numbers from a global time study, I'll be glad to review them and support your efforts.

Enjoy your night off.

cheers gary

Sam Johnston wrote:

...

Gary,

And here I was thinking I was getting the night off...

I am advocating this is not the only interface.

It's not the only interface - there's currently this one for machines and XHTML5 for users (and masochistic machines who like to parse XML). However it is the only interface users would have to implement (rather than having to code XML, JSON *and* text, which was the case before).

Anyway I see Adrian has kindly stepped up to discuss the details with you so I can get back to writing PyUnit unit tests.

Sam

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

Hi Andre, I read Adrian's email in regards to the a tested history of stuffing attributes into http headers. I'm not debating that issue. It does work, or else http wouldn't. What is not proven and my issue is this is an unproven technology for management applications. We have 2 new items here, a new and unproven management protocol and now a new and unproven networking protocol. Selling one is hard enough when questions like is it proven in a 5-9s environment on a global scale. Adopting both is far too risky for serious enterprise adoption. Believing it can happen, is just unfamiliarity with the market. My issues are not driven by technology for technologies sake. And there are places for that, like apache incubators. My concerns are derived from practical business issues including technology risk management. Honestly, I'm have run out of road on this issue, and have more pressing commitments to spend my time on. Andre, I think many in this group has lost perspective and is caught up in rhetoric. Its up to the proposer's to justify and provide evidence that this is a viable technology that can be deployed. Not the other way around. Proposer should noted whether the target environment and application is it intended for a web browser technology, as life critical application technology; is it indented to support reliably as "best of breed" in the global enterprise. -gary Andre Merzky wrote:

Quoting [Gary Mazz] (Oct 20 2009):

...

Adrian,

These issues are business issues and not appropriate discussion for this group. The issues about this technology have been brought up by myself and others. I'm done discussing this issue. If you trust S3, go for it.:-)

Uhm, I think Adrian is trying to make a technical point, i.e. is citing prior art. If you consider the S3 prior art invalid, ok, I am sure he'll listen to your arguments, and he and Sam will try to find another example...

Andre.

PS.: You know, it is great to see a group with strong individuals - but I would not like to chair it... ;-)

...

-gary

Adrian Cole wrote:

...

Hi, Gary.

I attended the AWS enterprise meetup in LA last week. The number of objects they mentioned that are stored in S3 is currently 70 billion and the service has been running for over 3.5 years. Granted that their "global story" is shorter, but they do 2 years presence in the EU. All 70 billion objects use headers for metadata.

What metadata system is more proven then this? What are the technical risks of using headers? Are there clients or systems that are unable to pass http headers, but are able to pass entities?

None of this matters, if the concern is the code impact to your application. However, I'd like to know the details of the broader concern you may be implying.

Thanks, -Adrian

On Tue, Oct 20, 2009 at 4:24 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

Look Sam,

If you feel that this is the best way for move occi forward, go for it. I'm iust trying to preserve the integrity of the work and support it properly in this group. I will be rolling out an occi interface or occi like interface publicly by Q1'10. I will not be deploying this proposed protocol until it is proven to work in a global environment. If that means abandoning occi, well, I'll abandon occi.

I personally, feel I cannot and will not risk a customer's business or the livelihood of partners and their families on a single opinion and unproven technology. When you get real numbers from a global time study, I'll be glad to review them and support your efforts.

Enjoy your night off.

cheers gary

Sam Johnston wrote:

...

Gary,

And here I was thinking I was getting the night off...

I am advocating this is not the only interface.

It's not the only interface - there's currently this one for machines and XHTML5 for users (and masochistic machines who like to parse XML). However it is the only interface users would have to implement (rather than having to code XML, JSON *and* text, which was the case before).

Anyway I see Adrian has kindly stepped up to discuss the details with you so I can get back to writing PyUnit unit tests.

Sam

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

All, This thread unfortunately (or fortunately depending how you look at it) dropped off the list after this point and continued through the night while I was (thankfully) fast asleep. Today's a new day and rather than risk wasting another I don't plan to spend more time on it at this point as I am, like Adrian <http://twitter.com/jclouds/status/5037108001>, fast approaching the end of my tether. FWIW people who should know <http://mike.mykanjo.co.uk/> do agree with this approach <http://twitter.com/jclouds/status/5030884020> and the general feedback we have been receiving off-list has been overwhelmingly positive - some examples: "from your presentation I think OCCI is great stuff! I'm interested to see what OCCI looks like, as it seems high debate may change it" "I'm absolutely amazed at the progress you've made if this is how OCCI forums run. It's a pretty cool API." "I just read through the spec. It's actually surprisingly simple, which is good. I really like the clean RESTful interface." "I think you guys have done a great job -- the spec is extremely straightforward and intuitive for anyone familiar with the usual HTTP verbs." I could go on but you get the point - despite the unfortunate tone of this thread things are looking good for us, provided we avoid being derailed by what are relatively trivial issues (relative to, say, actually delivering a complete spec in a timely fashion that people believe in sufficiently as to implement it). Sam On Wed, Oct 21, 2009 at 2:31 AM, Gary Mazz <garymazzaferro@gmail.com> wrote:

Hi Andre,

I read Adrian's email in regards to the a tested history of stuffing attributes into http headers. I'm not debating that issue. It does work, or else http wouldn't.

What is not proven and my issue is this is an unproven technology for management applications. We have 2 new items here, a new and unproven management protocol and now a new and unproven networking protocol. Selling one is hard enough when questions like is it proven in a 5-9s environment on a global scale. Adopting both is far too risky for serious enterprise adoption. Believing it can happen, is just unfamiliarity with the market.

My issues are not driven by technology for technologies sake. And there are places for that, like apache incubators. My concerns are derived from practical business issues including technology risk management.

Honestly, I'm have run out of road on this issue, and have more pressing commitments to spend my time on.

Andre, I think many in this group has lost perspective and is caught up in rhetoric. Its up to the proposer's to justify and provide evidence that this is a viable technology that can be deployed. Not the other way around. Proposer should noted whether the target environment and application is it intended for a web browser technology, as life critical application technology; is it indented to support reliably as "best of breed" in the global enterprise.

-gary

...

Quoting [Gary Mazz] (Oct 20 2009):

...

Adrian,

These issues are business issues and not appropriate discussion for this group. The issues about this technology have been brought up by myself and others. I'm done discussing this issue. If you trust S3, go for it.:-)

Uhm, I think Adrian is trying to make a technical point, i.e. is citing prior art. If you consider the S3 prior art invalid, ok, I am sure he'll listen to your arguments, and he and Sam will try to find another example...

Andre.

PS.: You know, it is great to see a group with strong individuals - but I would not like to chair it... ;-)

...

-gary

Adrian Cole wrote:

...

Hi, Gary.

I attended the AWS enterprise meetup in LA last week. The number of objects they mentioned that are stored in S3 is currently 70 billion and the service has been running for over 3.5 years. Granted that their "global story" is shorter, but they do 2 years presence in the EU. All 70 billion objects use headers for metadata.

What metadata system is more proven then this? What are the technical risks of using headers? Are there clients or systems that are unable to pass http headers, but are able to pass entities?

None of this matters, if the concern is the code impact to your application. However, I'd like to know the details of the broader concern you may be implying.

Thanks, -Adrian

On Tue, Oct 20, 2009 at 4:24 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

Look Sam,

If you feel that this is the best way for move occi forward, go for it. I'm iust trying to preserve the integrity of the work and support it properly in this group. I will be rolling out an occi interface or occi like interface publicly by Q1'10. I will not be deploying this

Andre Merzky wrote: proposed

...

...

...

...

protocol until it is proven to work in a global environment. If that means abandoning occi, well, I'll abandon occi.

I personally, feel I cannot and will not risk a customer's business or the livelihood of partners and their families on a single opinion and unproven technology. When you get real numbers from a global time study, I'll be glad to review them and support your efforts.

Enjoy your night off.

cheers gary

Sam Johnston wrote:

...

Gary,

And here I was thinking I was getting the night off...

I am advocating this is not the only interface.

It's not the only interface - there's currently this one for machines and XHTML5 for users (and masochistic machines who like to parse XML). However it is the only interface users would have to implement (rather than having to code XML, JSON *and* text, which was the case before).

Anyway I see Adrian has kindly stepped up to discuss the details with you so I can get back to writing PyUnit unit tests.

Sam

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

I don't think we should create an interface that is innately limiting in the name of "mainstream." I've been told more then once that OCCI is currently for compute, but is made for other services. I argue that scale is a relevant concern, unless we feel like having these same discussions later, in haste, and in reaction to breaking points. I agree that start/stops don't create a high burden of requests: You could probably write a crappy cgi script and it would still perform well enough. However, state pings, metadata queries, etc, will certainly happen in some frequency. Anything we do in the future, by extension or otherwise, will cause more traffic and higher need for scale. I don't think history will condemn us for either pragmatically considering efficiency or using headers. my 2p. -Adrian On Tue, Oct 20, 2009 at 7:51 AM, Gary Mazz <garymazzaferro@gmail.com> wrote:

I've understood Sam's proposal from the beginning. I have had issue with from the beginning.

Although I find it very interesting and valuable in terms of economy, I have issue with it as the OCCI's primary interface implementation for a few reasons which have been outlined in several other emails.

It appears that Sam has created a new technology, a lower cost RPC mechanism, which is unproven for both small and wide scale deployment. For interoperability, IMO,  occi should focus on proven mainstream technologies as the primary interfaces. I am not suggesting, shelving or not supporting Sam's proposal, based on market acceptance, it may prove to be one of many occi interface implementations.

As for cost saving in interface technologies, I do not buy into the idea that the occi traffic will be so overwhelming that there is a requirement to maximally optimize http requests and responses. IMO, in comparison to images and consumer data occi traffic will be insignificant. I think we are worrying about spilled milk while the house is burning.

There has been some discussion on supporting technologies including OVF. Whether occi supports OVF has not been fully discussed, examined or any consensus reached. I believe adopting technologies like OVF will only lead to mapping issues between occi attributes and the external formats creating a long term management headache.

cheers, gary

Alexis Richardson wrote:

...

OK

Is Sam's statement something that we can achieve consensus around?

a

On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net> wrote:

...

Alexis,

Seems our posts crossed in the mail. I hope the last one clears things up but the fact that you're using OVF and XML/JSON interchangeably suggests that you [are possibly one of many who] don't grok the proposal. Worse, it seems you've got it back to front in thinking of the "core" as metadata, which is not at all what I had intended.

The focus is on the resources and their various representations. The metadata (that is, data about data) essentially provides whatever is necessary to "glue" these resources together. It is basically using HTTP as the meta-model rather than an envelope format like Atom/SOAP or, worse, a meta-model that we have to create ourselves from scratch.

So it's more like:

1) OCCI is compatible with ALL existing formats, eg OVF

2) Existing formats are used are used for ALL representations

3) ALL additional metadata should be in the headers

4) ALL of the core stuff should be in the body (that is, the metadata in the headers is just there to support the data and wire it up in useful ways)

I think the confusion stems from your assuming that there is still XML/JSON/Atom/etc. representations. There aren't. HTTP headers provide a single, performant, interoperable metadata channel that has already been both standardised and extensively implemented.

If you've followed me this far then you should have realised that every HTTP user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, etc.) is already a basic OCCI client out-of-the-box. The result is that 5-line scripts like demo.py are functional with no dependence on client libraries (that would absolutely be necessary if you were to start trying to define your own formats).

Sam

PS I'm sorry my "bunk" comment came across as referring to you personally - it was directed at the argument.

On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

I think what Sam is proposing is the following:

1) If OCCI can handle 'any' data representation, eg OVF, XML, JSON, then it needs some core / common model.  Otherwise: there can be no defined behaviour / interop

2) This core / common model is ALL metadata

3) ALL this metadata should be in the headers

4) ALL the non core stuff eg OVF payload, XML representation, should be in body

Sam - is that right?

a

On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX <andrewx.edmonds@intel.com> wrote:

...

To that point I'd ask - If we're talking about meta-data that infers there is some data to accompany it. Where are the examples of this data? This would help in forming the full picture.

Here, from the core spec (using OVF as an example, but could be anything). Request is green, response metadata is yellow & response data/payload is orange:

...

GET /us-east/webapp/vm01 HTTP/1.1 User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0

Host: cloud.example.com Accept: */*

< HTTP/1.1 200 OK < Date: Sat, 10 Oct 2009 12:56:51 GMT

< Content-Type: application/ovf < Link: </us-east/webapp/vm01;start>;

<       rel="http://purl.org/occi/action/start"; <       title="Start"

< Link: </us-east/webapp/build.pdf>; <       rel="related";

<       title="Documentation"; <       type="application/pdf"

< Category: compute; <       label="Compute Resource";

<       scheme="http://purl.org/occi/kind/" < Server: occi-server/1.0 (linux) OCCI/1.0

< Connection: close < < <?xml version="1.0" encoding="UTF-8"?>

< <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" <           xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"

<           xmlns="http://schemas.dmtf.org/ovf/envelope/1" <           xml:lang="en-US">

...

...

-----Original Message----- From: occi-wg-bounces@ogf.org [mailto:occi-wg-bounces@ogf.org] On Behalf Of Alexis Richardson Sent: 20 October 2009 09:49 To: Sam Johnston Cc: Tim Bray; occi-wg@ogf.org Subject: Re: [occi-wg] confusion about status of link / headers

Sam,

Please don't throw around words like "bunk" willy nilly.

I have just had a look at http://www.rackspacecloud.com/cloud_hosting_products/servers/api

I can see some limited use of headers for metadata in a few places. That's never been controversial.  But the understanding that I have of your proposal is to use headers wholesale for all metadata.  Rackspace don't appear to do that, unless I have misunderstood their document. I also may have misunderstood your proposal.  If nobody understands your proposal except you, then it will not be possible to gain consensus around it.

Are you saying "OCCI metadata should be more like Rackspace metadata"?

alexis

On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> wrote:

> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson > <alexis.richardson@gmail.com> wrote: > >> What about compute clouds? >> > Rackspace Cloud API for a start: > > >> HTTP/1.1 204 No Content >> Date: Mon, 12 Nov 2007 15:32:21 GMT >> Server: Apache >> X-Server-Management-Url: >> https://servers.api.rackspacecloud.com/v1.0/35428 >> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 >> X-CDN-Management-Url: >> https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 >> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb >> Content-Length: 0 >> Content-Type: text/plain; charset=UTF-8 >> > Microsoft Azure uses headers for "attributes" as well, in the same > way > as I > had originally proposed: > > x-ms-meta-name: value     Returns a metadata value for the container. > > However prefer that the header names are static/opaque rather than > using > a > template and parsing them: > > >> Attribute: name=value >> > This to me is cleaner and more self-explanatory, plus it is easily > collapsed > ala: > > >> Attribute: name1=value1, name2=value2, ... namen=valuen >> > Anyway suffice to say that claims this is somehow "experimental" are > bunk. > > Sam > > >> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org> >> wrote: >> >>> Here are options for metadata used in some of the major storage >>> clouds >>> FWIW: >>> >>> S3, Rackspace, EMC Atmos, Azure - Headers >>> Nirvanix - query params in, xml entity out >>> Mezeo - entity >>> >>> Of the ones using headers, S3, Rackspace and Azure use prefix with >>> values stored as-us.  Atmos joins all metadata together into one >>> header, making parsing trivial (split /,/), but necessary. >>> >>> The most expensive option of the above is entity, where each >>> metadata >>> value is a separate GET.  However, entities allow binary metadata >>> and >>> zero restrictions on it, which may be useful. >>> >>> In jclouds, we time parsing of response values.  A simple XML doc >>> with >>> only several elements written in SAX takes a few ms to parse.  My >>> log >>> files are not precise enough to find the overhead in parsing >>> headers: >>> they always start and finish within the same millisecond. >>> >>> I hope this background helps, and also helps explain why I'm vocal >>> on >>> such topics such as headers vs entities :) >>> >>> Cheers, >>> -Adrian >>> >>> >>> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> >>> wrote: >>> >>>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >>>> <garymazzaferro@gmail.com> >>>> wrote: >>>> >>>> >>>>> The http header and key/value pairs need to parsed also, there >>>>> is >>>>> no >>>>> free >>>>> ride here. >>>>> >>>> Every HTTP library I have ever used parses HTTP headers and puts >>>> them >>>> in a >>>> nice hash for you ready to consume. If we go for "Name: Value" >>>> then >>>> that's >>>> all there is to it. If we go for "Attribute: name=value" as is >>>> currently >>>> proposed (which is arguably cleaner, follows cookies' "prior art" >>>> and >>>> avoids >>>> Amazon's prefix hack) then you just have to split on '='. >>>> >>>> To illustrate how clean this is by example: >>>> >>>> >>>>> #!/usr/bin/python >>>>> import urllib2 >>>>> response = urllib2.urlopen('http://cloud.example.com/myvm') >>>>> representation = response.read() >>>>> metadata = response.info() >>>>> print metadata['occi-compute-cores'] >>>>> >>>> As soon as you start talking about payloads you have to fire up a >>>> parser >>>> (JSON/XML/Atom/etc.) or write your own (previous text rendering) >>>> which >>>> is >>>> significantly more work to do at both design and run times. Not >>>> to >>>> mention >>>> more work for us to do now and more scope for interoperability >>>> problems. >>>> >>>> Sam >>>> >>>> >>>> _______________________________________________ >>>> occi-wg mailing list >>>> occi-wg@ogf.org >>>> http://www.ogf.org/mailman/listinfo/occi-wg >>>> >>>> >>>> >>> _______________________________________________ >>> occi-wg mailing list >>> occi-wg@ogf.org >>> http://www.ogf.org/mailman/listinfo/occi-wg >>> >>> > _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg ------------------------------------------------------------- Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

On Tue, Oct 20, 2009 at 10:49 AM, Alexis Richardson < alexis.richardson@gmail.com> wrote: Please don't throw around words like "bunk" willy nilly.

I tell you what... I'll resist the urge to declare your arguments "bunk" if you resist the urge to declare my proposals "experimental", "new technology", not "prior art" and whatever other weasel words are being tossed around to imply that they are somehow risky. To quote McNealy for the second time today “If everybody believes in your strategy you have zero chance of [success]" - change that matters is always controversial.

I have just had a look at http://www.rackspacecloud.com/cloud_hosting_products/servers/api

I can see some limited use of headers for metadata in a few places. That's never been controversial. But the understanding that I have of your proposal is to use headers wholesale for all metadata. Rackspace don't appear to do that, unless I have misunderstood their document.

Rackspace have taken on the creation of both a format and a protocol combined (note the absence of renderings like OVF). That's wrong for many reasons, not the least of which include creating significant extra work for yourself, not playing nice with other standards, raising the bar for implementers on both sides of the interface, creating more opportunities for interoperability problems... I could go on. However it suited them well because their task was to expose the relatively limited functionality of a single implementation, not to satisfy the needs of many. Also, whether they use headers for a single attribute or all of them is irrelevant - it still proves headers are safe for such interfaces. We made a conscious decision early on to embrace existing standards such as OVF and I've provided a way for us to allow them to coexist and compete on merit by steering well clear of the HTTP entity-body. The non-Atom alternative requires us to follow their example by developing both protocol and format, with the additional hurdle that our format must also be compatible and compete with others like OVF. It's a mess - I know because I spent quite some time trying to make it work. Without changing (e.g. complicating) the model you've got different representations for the same resource telling different stories which is just plain wrong. In any case, cloud storage services including both Azure and S3 (which can certainly be considered cloud infrastructure) do use HTTP headers wholesale for metadata. HTTP intends for headers to be used for metadata too, and provides for arbitrary extension to suit. I also may have misunderstood your proposal. If nobody understands

your proposal except you, then it will not be possible to gain consensus around it.

If indeed people don't understand the proposal then that *is* something we need to work on. It's very intuitive for anyone who has worked with web technologies: - Instances of objects we model are "resources" which are abstract & can't be retrieved directly (e.g. you can't pull a physical machine over an API, yet) - You can however GET "representations" of resources in one or more existing standard formats (e.g. OVF, VMDK, etc.) - Some representations will be better than others at telling the "full story" while others may be quite lossy (ala JPEG). - The non-Atom alternative is to create an extremely lossy representation that must be retrieved in addition to the "native" representation (e.g. OVF), which includes information not available in other representations, and which may conflict with other representation e.g. current resources allocated (which may be very dynamic and differ from the saved state). - Metadata including the organisation of the resources into categories and an interconnected graph, as well as owner information, related resources, possible state transitions, etc. needs to be captured somewhere. - HTTP provides a perfectly good metadata channel in the form of HTTP headers, whereby a single CRLF character (e.g. a blank line) delineates the metadata from the data itself. - HTTP also provides a convenient way to access the metadata alone (e.g. HTTP HEAD). This is useful for tasks like enumeration/discovery which don't require clients to have the representation itself. - HTTP also provides many optimisations for common tasks such as COPYing templates and MOVEing resources between clouds. Basically my proposal is to, wherever possible, use HTTP exactly as it was originally intended, deviating only where we absolutely have to. Are you saying "OCCI metadata should be more like Rackspace metadata"?

No, just that Rackspace (among others) validate the use of HTTP headers in cloud APIs. Sam On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

What about compute clouds?

Rackspace Cloud API for a start:

...

HTTP/1.1 204 No Content Date: Mon, 12 Nov 2007 15:32:21 GMT Server: Apache X-Server-Management-Url: https://servers.api.rackspacecloud.com/v1.0/35428 X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-CDN-Management-Url: https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4 X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb Content-Length: 0 Content-Type: text/plain; charset=UTF-8

Microsoft Azure uses headers for "attributes" as well, in the same way as I had originally proposed:

x-ms-meta-name: value Returns a metadata value for the container.

However prefer that the header names are static/opaque rather than using a template and parsing them:

...

Attribute: name=value

This to me is cleaner and more self-explanatory, plus it is easily

collapsed > ala: > >> Attribute: name1=value1, name2=value2, ... namen=valuen > > Anyway suffice to say that claims this is somehow "experimental" are bunk. > > Sam > >> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole &lt;adrian@jclouds.org> wrote: >> > Here are options for metadata used in some of the major storage clouds >> > FWIW: >> > >> > S3, Rackspace, EMC Atmos, Azure - Headers >> > Nirvanix - query params in, xml entity out >> > Mezeo - entity >> > >> > Of the ones using headers, S3, Rackspace and Azure use prefix with >> > values stored as-us. Atmos joins all metadata together into one >> > header, making parsing trivial (split /,/), but necessary. >> > >> > The most expensive option of the above is entity, where each metadata >> > value is a separate GET. However, entities allow binary metadata and >> > zero restrictions on it, which may be useful. >> > >> > In jclouds, we time parsing of response values. A simple XML doc with >> > only several elements written in SAX takes a few ms to parse. My log >> > files are not precise enough to find the overhead in parsing headers: >> > they always start and finish within the same millisecond. >> > >> > I hope this background helps, and also helps explain why I'm vocal on >> > such topics such as headers vs entities :) >> > >> > Cheers, >> > -Adrian >> > >> > >> > On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston &lt;samj@samj.net> wrote: >> >> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro >> >> &lt;garymazzaferro@gmail.com> >> >> wrote: >> >> >> >>> The http header and key/value pairs need to parsed also, there is no >> >>> free >> >>> ride here. >> >> >> >> Every HTTP library I have ever used parses HTTP headers and puts them >> >> in a >> >> nice hash for you ready to consume. If we go for "Name: Value" then >> >> that's >> >> all there is to it. If we go for "Attribute: name=value" as is >> >> currently >> >> proposed (which is arguably cleaner, follows cookies' "prior art" and >> >> avoids >> >> Amazon's prefix hack) then you just have to split on '='. >> >> >> >> To illustrate how clean this is by example: >> >> >> >>> #!/usr/bin/python >> >>> import urllib2 >> >>> response = urllib2.urlopen('http://cloud.example.com/myvm') >> >>> representation = response.read() >> >>> metadata = response.info() >> >>> print metadata['occi-compute-cores'] >> >> >> >> As soon as you start talking about payloads you have to fire up a >> >> parser >> >> (JSON/XML/Atom/etc.) or write your own (previous text rendering) which >> >> is >> >> significantly more work to do at both design and run times. Not to >> >> mention >> >> more work for us to do now and more scope for interoperability >> >> problems. >> >> >> >> Sam >> >> >> >> >> >> _______________________________________________ >> >> occi-wg mailing list >> >> occi-wg@ogf.org >> >> http://www.ogf.org/mailman/listinfo/occi-wg >> >> >> >> >> > _______________________________________________ >> > occi-wg mailing list >> > occi-wg@ogf.org >> > http://www.ogf.org/mailman/listinfo/occi-wg >> > > >

Adrian, On Tue, Oct 20, 2009 at 7:19 AM, Adrian Cole <adrian@jclouds.org> wrote:

Here are options for metadata used in some of the major storage clouds FWIW:

S3, Rackspace, EMC Atmos, Azure - Headers Nirvanix - query params in, xml entity out Mezeo - entity

Thanks - this is great information.

Of the ones using headers, S3, Rackspace and Azure use prefix with values stored as-us. Atmos joins all metadata together into one header, making parsing trivial (split /,/), but necessary.

If you use something like "Attribute: name=value" then HTTP specifies that this can be collapsed into a single "Attribute: name1=value1, name2=value2" header (',' is used to separate headers while ';' separates header components).

The most expensive option of the above is entity, where each metadata value is a separate GET. However, entities allow binary metadata and zero restrictions on it, which may be useful.

In such cases it is probably better to use Link: headers. For example, we can advertise a console screenshot in an image/* format using something like: Link: </myvm.png>; rel="http://purl.org/occi/relation#console" The same approach is currently used to advertise SSH/RDP/etc. access too.

In jclouds, we time parsing of response values. A simple XML doc with only several elements written in SAX takes a few ms to parse. My log files are not precise enough to find the overhead in parsing headers: they always start and finish within the same millisecond.

While unsurprising it's good to have some numbers to back up the assumption that headers are more performant... I haven't pushed this point before now because I didn't have the evidence.

I hope this background helps, and also helps explain why I'm vocal on such topics such as headers vs entities :)

Sure - it's great to have you on board. Sam On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj@samj.net> wrote:

...

On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro < garymazzaferro@gmail.com> wrote:

...

The http header and key/value pairs need to parsed also, there is no free ride here.

Every HTTP library I have ever used parses HTTP headers and puts them in a nice hash for you ready to consume. If we go for "Name: Value" then that's all there is to it. If we go for "Attribute: name=value" as is currently proposed (which is arguably cleaner, follows cookies' "prior art" and avoids Amazon's prefix hack) then you just have to split on '='.

To illustrate how clean this is by example:

...

#!/usr/bin/python import urllib2 response = urllib2.urlopen('http://cloud.example.com/myvm') representation = response.read() metadata = response.info() print metadata['occi-compute-cores']

As soon as you start talking about payloads you have to fire up a parser (JSON/XML/Atom/etc.) or write your own (previous text rendering) which is significantly more work to do at both design and run times. Not to mention more work for us to do now and more scope for interoperability problems.

Sam

...

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

On Tue, Oct 20, 2009 at 12:52 AM, gary mazzaferro <garymazzaferro@gmail.com>wrote:

Agreed, consensus and discussion is need to follow though on decisions. The lack of discussion due the parties not engaging is not considered consensus.

If people choose not to participate in the discussions now then they have only themselves to blame when they are led by the nose by the dominant vendor. Right now I see cloud computing turning into a re-run of the last few decades with a single vendor controlling the standards that matter. We're in the best position to provide an alternative and even then I wonder if we're able to deliver (unfortunately some have already made up their mind on that subject) given there are so many agendas to align. In any case referring back to Tim's informal definition of consensus as the "absence of sustained intense reasonable resistance", silence *is* consensus... basically "speak now or forever hold your peace". If it is your intention to provide "reasonable resistance" to the current proposal then please do so by way of a "camera ready" alternative and a rationale as to why it is superior. Sam

On 2009-10-19, at 9:53 AM, Alexis Richardson wrote:

Well it sounds like at least three people, including myself, prefer the IETF model.

Well, filling in some details then... In an IETF WG there are typically one or two co-chairs and then one or two co-editors. Typically the editors produce a sequence of drafts; they get to make lots of judgment calls, particularly on wording, and to include their own solutions to issues that haven't really been discussed yet. Then when things get intense, the chairs eventually decree what the consensus is and the editors do that. It's really helpful that at any one time, there's one internet-draft which is the subject of discussion. When I was chairing I used to demand that suggested changes come with complete "camera-ready" deltas aimed at the current draft. In the Atom WG we had a wiki which was mostly used to construct these change proposals. -Tim

On Mon, Oct 19, 2009 at 6:12 PM, Tim Bray <Tim.Bray@sun.com> wrote:

It's really helpful that at any one time, there's one internet-draft which is the subject of discussion.

I think this the key point - qua drafting at least. I have also observed it to work well in other groups. It enforces consensus by discussion. Conversely I am not in favour of consensus by DVCS algorithms.

When I was chairing I used to demand that suggested changes come with complete "camera-ready" deltas aimed at the current draft.

This helps the above. alexis

For what its worth, OGF is modeled after IETF, and the IETF process on reaching consensus should be applied. From http://www.ogf.org/documents/GFD.2.pdf : "The GGF intends to emulate, as appropriate, the Internet Engineering Task Force (IETF, www.ietf.org) and to support and complement the Internet Standards Process as outlined in [1]. is therefore advantageous that the GGF structure and process closely mirror those of the IETF. " [1] Bradner, S., “IETF Working Group Guidelines and Procedures,” RFC 2418, September 1998. http://www.ietf.org/rfc/rfc2418.txt Hope that helps, Andre. Quoting [Tim Bray] (Oct 19 2009):

On 2009-10-19, at 9:21 AM, Alexis Richardson wrote:

...

Gary

Thanks. That strikes me as a fairly complex process.

Does anyone have any alternative suggestions? We need a simple model for reaching consensus here, that grows the community and adoption.

In practice, I've had experience with three processes; ISO, W3C/Oasis, and IETF process. ISO is institutional voting, with complex threshold rules. W3C and Oasis individual members vote. Of course, this means you have to define who's a member and thus gets a vote. In the W3C, you argue for a while and then the chair (co-chairs usually) assert what the consensus is. Informally consensus is considered to be the absence of sustained intense reasonable resistance. If you disagree you appeal to the Area Director, the IESG, the IAB and eventually the Internet Society (I may have that appeal chain out of order). I prefer the IETF model but all have been observed to work. -Tim _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

-- Nothing is ever easy.

Guys, I am interested in one thing only here -- a simple model for reaching consensus here, that grows the community and adoption. Once I have canvassed opinion on this I shall have a chat with the other chairs about it. I am NOT calling into question anyone's intentions or agenda. I do wish to AVOID discussion of specific technical points at this time. alexis On Mon, Oct 19, 2009 at 5:44 PM, Sam Johnston <samj@samj.net> wrote:

Alexis,

Agreed. Gary that's an incredible amount of bureaucracy you've managed to squeeze into one email, and all over a debate as to which side of the CRLF character the metadata belongs on (following a religious war over <> vs {} no less). At this stage finishing the document must be our number one priority (having just watched our final deadline sail by with OGF 27) and reverting anything (in this case months of my work) cannot be considered going forward however you look at it.

If I understand well you have implemented an ancient version of the spec in a commercial product and will stop at nothing to prevent any changes being made, despite my having discussed this issue at length both on- and off-list starting August/September. Conversely, as a representative of end users I assure you I have no agenda beyond releasing the highest quality protocol possible in a timely fashion so as to remain relevant (something we are already struggling with).

How about we focus on the technical merits of the alternative(s)? HTTP provides a perfectly good metadata channel that is extremely well tested (claims to the contrary are FUD, per above) and means we can delicately sidestep the entire issue of creating our own representations (with the associated interoperability problems and the additional cost of having to implement multiple formats). Bear in mind that there are existing (generally open) standards for representations of everything from virtual machines to calendar entries and users can choose the most appropriate for the task - if we stray in-band then we'll need to do work for each and every object that we ever intend to represent (note that HTTP does no such thing).

Sam

On Mon, Oct 19, 2009 at 6:21 PM, Alexis Richardson <alexis.richardson@gmail.com> wrote:

...

Gary

Thanks.  That strikes me as a fairly complex process.

Does anyone have any alternative suggestions?  We need a simple model for reaching consensus here, that grows the community and adoption.

alexis

On Mon, Oct 19, 2009 at 5:10 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

Alexis,

I do not believe we can set aside the OGF27 unapproved unilateral change to the document. It is a fundamental  problem in operation.

These are my suggestions:

1) Place the document in a format where each interface can have its own life cycle. 2) Create a template for adjunct documents 3) Limit check-in to the repository to two people and one backup. 4) Recruit an editor and a backup editor. This spec isn't that big. 5) Provide an incubator for emerging technologies and proposals for inclusion in the specification. 6) Form a public mailing list purposed for emerging technologies, proposal discussions and voting 7) Provide a proposal document number for each submitted document 8) Before the document can be considered for review, it must be in the adjunct documents template

-gary

Alexis Richardson wrote:

...

Leaving aside the when and how in relation to OGF 27 specifically, we cannot claim that unilateral changes are supported by the group if they are not supported by the group.  I am very concerned about this. We of course don't want to ignore innovative proposals, but we need to build consensus around them before inclusion in the spec.

Suggestions for a good way forward are solicited..

alexis

On Mon, Oct 19, 2009 at 4:19 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

That is a good point, a better question would be how did it get into the spec presented like it was the preferred method. Probably because there is no single editor and anyone can change the documents anyway they feel fit.

I don't think what Sam is working on is out of scope for OCCI,  it was unintended to support multiple interfaces. Sam seems to be running with this one, driving OCCI to the lowest level of the HTTP protocols, essentially creating a new technology, untried on multiple levels in the internet infrastructure.

My issue with it is it was placed in the spec at the last second before OGF 27,  other implementations were remove in lieu of this one, it was placed in the spec irrespective of a group consensus and SNIA, a strategic partner, publicly announcing they would NOT support this interface. However, this does not preclude the rest of this group to continue with the original concept of OCCI information in HTTP entities (content body). For maintainability, this does force the document to take on a new format of separating  the implementations from  reference model (we need one first).  Interface implementations should fall into adjunct documents. This specification model has been successfully executed by numerous standards bodies.

cheers,

Gary

Alexis Richardson wrote:

...

Sam & group,

I just saw this tweet: http://twitter.com/samj/statuses/4991958514

You say that "HTTP has an out-of-band metadata channel in the form of headers. #occi's using Link: as a flexible, lightweight RDF alternative".

I'm a bit confused here.... I thought this was still under discussion.  What am I missing?

alexis _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

Andre, A face-to-face is a great idea. Although I'm spending a lot of time in Zürich these days, Paris may be another good option as I know Alexis in particular is fairly busy and there are flights from Dublin for Andy but it's much of a muchness. Sam On Mon, Oct 19, 2009 at 7:08 PM, Andre Merzky <andre@merzky.net> wrote:

Quoting [Alexis Richardson] (Oct 19 2009):

...

Gary

Thanks. That strikes me as a fairly complex process.

Does anyone have any alternative suggestions? We need a simple model for reaching consensus here, that grows the community and adoption.

Suggestion:

- agree on one editing backend (google docs/mercury) (like now)

- anybody can edit (after registration) (like now)

- before publication or presentation, require a one-week final call on mailing list. Only allow presentation as OCCI draft if group agrees. Otherwise, require label 'my suggestion for OCCI draft' or similar.

Note that OGF requires a final call on the list before submitting to the OGF editor anyway. Submitted documents MUST represent group consensus, i.e. must represent the group majority.

BTW: Voting on issues is explicitely encouraged: IMHO, you disuss issues to death otherwise ("running code and rough consensus"). So:

- discuss issue - consider all alternatives / implementations - if discussion stalls: vote on the issue - *stick* *to* *the* *decision* ;-) - only re-open the discussion IFF - a new alternative emerges - external or internal dependencies of the issue change - implementation is impossible

While I am on it, one more suggestion on procedures in OCCI: Other OGF groups have made tremendous progress in face-to-face meetings, such as during the OGF events, but also outside. The OCCI group members don't seem to massively attend OGF events, really - probably because the group is much more diverse than other OGF groups (which is a good thing!). Anyway, I think you guys should consider a meeting in person: lock a couple of key players in a room for a couple of days, and *decide* on things.

I'd be happy to help to organize a F2F. In fact, I checked with Thilo from the Vrije Universiteit in Amsterdam (VU), and he'd be willing to host, anytime (*).

So, please think about that, and let me know if that would help to settle the open technical issues in OCCI in the near future.

Cheers, Andre.

(*) It is not that VU is contributing to OCCI, but Amsterdam is very convenient to reach for most people, and cheap for Europeans, as EasyJet goes there from many locations. Also, the VU is located about 15 minutes from the Airport (one train stop + 5 min walk). Also, VU is able to cover the cost for the venue. Accomodation is affordable and nearby. I'll rather be silent about Dutch food though... ;-)

...

alexis

...

Alexis,

I do not believe we can set aside the OGF27 unapproved unilateral change to the document. It is a fundamental problem in operation.

These are my suggestions:

1) Place the document in a format where each interface can have its own

...

cycle. 2) Create a template for adjunct documents 3) Limit check-in to the repository to two people and one backup. 4) Recruit an editor and a backup editor. This spec isn't that big. 5) Provide an incubator for emerging technologies and proposals for inclusion in the specification. 6) Form a public mailing list purposed for emerging technologies,

...

discussions and voting 7) Provide a proposal document number for each submitted document 8) Before the document can be considered for review, it must be in the adjunct documents template

-gary

Alexis Richardson wrote:

...

Leaving aside the when and how in relation to OGF 27 specifically, we cannot claim that unilateral changes are supported by the group if they are not supported by the group. I am very concerned about this. We of course don't want to ignore innovative proposals, but we need to build consensus around them before inclusion in the spec.

Suggestions for a good way forward are solicited..

alexis

On Mon, Oct 19, 2009 at 4:19 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

That is a good point, a better question would be how did it get into

...

...

...

spec presented like it was the preferred method. Probably because

...

...

...

is no single editor and anyone can change the documents anyway they feel fit.

I don't think what Sam is working on is out of scope for OCCI, it was unintended to support multiple interfaces. Sam seems to be running with this one, driving OCCI to the lowest level of the HTTP protocols, essentially creating a new technology, untried on multiple levels in the internet infrastructure.

My issue with it is it was placed in the spec at the last second before OGF 27, other implementations were remove in lieu of this one, it was

...

...

...

in the spec irrespective of a group consensus and SNIA, a strategic

...

...

...

publicly announcing they would NOT support this interface. However,

...

...

...

does not preclude the rest of this group to continue with the original concept of OCCI information in HTTP entities (content body). For maintainability, this does force the document to take on a new

On Mon, Oct 19, 2009 at 5:10 PM, Gary Mazz <garymazzaferro@gmail.com> wrote: life proposal the there placed partner, this format

...

...

...

of separating the implementations from reference model (we need one first). Interface implementations should fall into adjunct documents. This specification model has been successfully executed by numerous standards bodies.

cheers,

Gary

Alexis Richardson wrote:

...

Sam & group,

I just saw this tweet: http://twitter.com/samj/statuses/4991958514

You say that "HTTP has an out-of-band metadata channel in the form

of

...

headers. #occi's using Link: as a flexible, lightweight RDF alternative".

I'm a bit confused here.... I thought this was still under discussion. What am I missing?

alexis _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

-- Nothing is ever easy. _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

I also support the F2F meeting and can likely facilitate such a meeting here on Intel's Leixlip site. -----Original Message----- From: occi-wg-bounces@ogf.org [mailto:occi-wg-bounces@ogf.org] On Behalf Of Andre Merzky Sent: 19 October 2009 18:08 To: Alexis Richardson Cc: occi-wg@ogf.org Subject: Re: [occi-wg] confusion about status of link / headers Quoting [Alexis Richardson] (Oct 19 2009):

Gary

Thanks. That strikes me as a fairly complex process.

Does anyone have any alternative suggestions? We need a simple model for reaching consensus here, that grows the community and adoption.

Suggestion: - agree on one editing backend (google docs/mercury) (like now) - anybody can edit (after registration) (like now) - before publication or presentation, require a one-week final call on mailing list. Only allow presentation as OCCI draft if group agrees. Otherwise, require label 'my suggestion for OCCI draft' or similar. Note that OGF requires a final call on the list before submitting to the OGF editor anyway. Submitted documents MUST represent group consensus, i.e. must represent the group majority. BTW: Voting on issues is explicitely encouraged: IMHO, you disuss issues to death otherwise ("running code and rough consensus"). So: - discuss issue - consider all alternatives / implementations - if discussion stalls: vote on the issue - *stick* *to* *the* *decision* ;-) - only re-open the discussion IFF - a new alternative emerges - external or internal dependencies of the issue change - implementation is impossible While I am on it, one more suggestion on procedures in OCCI: Other OGF groups have made tremendous progress in face-to-face meetings, such as during the OGF events, but also outside. The OCCI group members don't seem to massively attend OGF events, really - probably because the group is much more diverse than other OGF groups (which is a good thing!). Anyway, I think you guys should consider a meeting in person: lock a couple of key players in a room for a couple of days, and *decide* on things. I'd be happy to help to organize a F2F. In fact, I checked with Thilo from the Vrije Universiteit in Amsterdam (VU), and he'd be willing to host, anytime (*). So, please think about that, and let me know if that would help to settle the open technical issues in OCCI in the near future. Cheers, Andre. (*) It is not that VU is contributing to OCCI, but Amsterdam is very convenient to reach for most people, and cheap for Europeans, as EasyJet goes there from many locations. Also, the VU is located about 15 minutes from the Airport (one train stop + 5 min walk). Also, VU is able to cover the cost for the venue. Accomodation is affordable and nearby. I'll rather be silent about Dutch food though... ;-)

alexis

On Mon, Oct 19, 2009 at 5:10 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

Alexis,

I do not believe we can set aside the OGF27 unapproved unilateral change to the document. It is a fundamental problem in operation.

These are my suggestions:

1) Place the document in a format where each interface can have its own life cycle. 2) Create a template for adjunct documents 3) Limit check-in to the repository to two people and one backup. 4) Recruit an editor and a backup editor. This spec isn't that big. 5) Provide an incubator for emerging technologies and proposals for inclusion in the specification. 6) Form a public mailing list purposed for emerging technologies, proposal discussions and voting 7) Provide a proposal document number for each submitted document 8) Before the document can be considered for review, it must be in the adjunct documents template

-gary

Alexis Richardson wrote:

...

Leaving aside the when and how in relation to OGF 27 specifically, we cannot claim that unilateral changes are supported by the group if they are not supported by the group. I am very concerned about this. We of course don't want to ignore innovative proposals, but we need to build consensus around them before inclusion in the spec.

Suggestions for a good way forward are solicited..

alexis

On Mon, Oct 19, 2009 at 4:19 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

That is a good point, a better question would be how did it get into the spec presented like it was the preferred method. Probably because there is no single editor and anyone can change the documents anyway they feel fit.

I don't think what Sam is working on is out of scope for OCCI, it was unintended to support multiple interfaces. Sam seems to be running with this one, driving OCCI to the lowest level of the HTTP protocols, essentially creating a new technology, untried on multiple levels in the internet infrastructure.

My issue with it is it was placed in the spec at the last second before OGF 27, other implementations were remove in lieu of this one, it was placed in the spec irrespective of a group consensus and SNIA, a strategic partner, publicly announcing they would NOT support this interface. However, this does not preclude the rest of this group to continue with the original concept of OCCI information in HTTP entities (content body). For maintainability, this does force the document to take on a new format of separating the implementations from reference model (we need one first). Interface implementations should fall into adjunct documents. This specification model has been successfully executed by numerous standards bodies.

cheers,

Gary

Alexis Richardson wrote:

...

Sam & group,

I just saw this tweet: http://twitter.com/samj/statuses/4991958514

You say that "HTTP has an out-of-band metadata channel in the form of headers. #occi's using Link: as a flexible, lightweight RDF alternative".

I'm a bit confused here.... I thought this was still under discussion. What am I missing?

alexis _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

-- Nothing is ever easy. _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg ------------------------------------------------------------- Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

Quoting [Alexis Richardson] (Oct 19 2009):

On Mon, Oct 19, 2009 at 6:08 PM, Andre Merzky <andre@merzky.net> wrote:

...

Quoting [Alexis Richardson] (Oct 19 2009):

...

Does anyone have any alternative suggestions?  We need a simple model for reaching consensus here, that grows the community and adoption.

Suggestion:

 - agree on one editing backend (google docs/mercury) (like now)

 - anybody can edit (after registration) (like now)

Thanks Andre. I agree with most of your points but not the above point, unless you mean "anyone can work on an edit and present it". But, I think the model where the draft can be changed at any time by anyone has led to difficulties and confusion (certainly on my part) as to what is "going on".

I see your point. In other groups, however, the above model worked fine, and having an editor which signes patches to documents would rather be a roadblock than anything else. I mean, the revision control systems all allow branches and tags etc - so it is not like any changes there are hewn in stone. Good processes are in place for software, and the same are applicable for documents in my experiences. Like: Sam could create a branch OCCI-SAM for editing, which get (partially) merged into OCCI-DRAFT after discussion. Restricting write access to SVN will slow down editing for sure, IMHO. Anyway, this is just my opinion - don't want to impose anything, as I don't edit the doc myself, really... Best, Andre.

alexis

...

 - before publication or presentation, require a one-week final    call on mailing list.  Only allow presentation as OCCI draft    if group agrees.  Otherwise, require label 'my suggestion for    OCCI draft' or similar.

Note that OGF requires a final call on the list before submitting to the OGF editor anyway.  Submitted documents MUST represent group consensus, i.e. must represent the group majority.

BTW: Voting on issues is explicitely encouraged: IMHO, you disuss issues to death otherwise ("running code and rough consensus").  So:

 - discuss issue  - consider all alternatives / implementations  - if discussion stalls: vote on the issue  - *stick* *to* *the* *decision* ;-)  - only re-open the discussion IFF    - a new alternative emerges    - external or internal dependencies of the issue change    - implementation is impossible

While I am on it, one more suggestion on procedures in OCCI: Other OGF groups have made tremendous progress in face-to-face meetings, such as during the OGF events, but also outside.  The OCCI group members don't seem to massively attend OGF events, really - probably because the group is much more diverse than other OGF groups (which is a good thing!).  Anyway, I think you guys should consider a meeting in person: lock a couple of key players in a room for a couple of days, and *decide* on things.

I'd be happy to help to organize a F2F.  In fact, I checked with Thilo from the Vrije Universiteit in Amsterdam (VU), and he'd be willing to host, anytime (*).

So, please think about that, and let me know if that would help to settle the open technical issues in OCCI in the near future.

Cheers, Andre.

(*) It is not that VU is contributing to OCCI, but Amsterdam is very convenient to reach for most people, and cheap for Europeans, as EasyJet goes there from many locations.  Also, the VU is located about 15 minutes from the Airport (one train stop + 5 min walk). Also, VU is able to cover the cost for the venue.  Accomodation is affordable and nearby.  I'll rather be silent about Dutch food though... ;-)

...

alexis

On Mon, Oct 19, 2009 at 5:10 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

Alexis,

I do not believe we can set aside the OGF27 unapproved unilateral change to the document. It is a fundamental  problem in operation.

These are my suggestions:

1) Place the document in a format where each interface can have its own life cycle. 2) Create a template for adjunct documents 3) Limit check-in to the repository to two people and one backup. 4) Recruit an editor and a backup editor. This spec isn't that big. 5) Provide an incubator for emerging technologies and proposals for inclusion in the specification. 6) Form a public mailing list purposed for emerging technologies, proposal discussions and voting 7) Provide a proposal document number for each submitted document 8) Before the document can be considered for review, it must be in the adjunct documents template

-gary

Alexis Richardson wrote:

...

Leaving aside the when and how in relation to OGF 27 specifically, we cannot claim that unilateral changes are supported by the group if they are not supported by the group.  I am very concerned about this. We of course don't want to ignore innovative proposals, but we need to build consensus around them before inclusion in the spec.

Suggestions for a good way forward are solicited..

alexis

On Mon, Oct 19, 2009 at 4:19 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

...

That is a good point, a better question would be how did it get into the spec presented like it was the preferred method. Probably because there is no single editor and anyone can change the documents anyway they feel fit.

I don't think what Sam is working on is out of scope for OCCI,  it was unintended to support multiple interfaces. Sam seems to be running with this one, driving OCCI to the lowest level of the HTTP protocols, essentially creating a new technology, untried on multiple levels in the internet infrastructure.

My issue with it is it was placed in the spec at the last second before OGF 27,  other implementations were remove in lieu of this one, it was placed in the spec irrespective of a group consensus and SNIA, a strategic partner, publicly announcing they would NOT support this interface. However, this does not preclude the rest of this group to continue with the original concept of OCCI information in HTTP entities (content body). For maintainability, this does force the document to take on a new format of separating  the implementations from  reference model (we need one first).  Interface implementations should fall into adjunct documents. This specification model has been successfully executed by numerous standards bodies.

cheers,

Gary

Alexis Richardson wrote:

> > Sam & group, > > I just saw this tweet: http://twitter.com/samj/statuses/4991958514 > > You say that "HTTP has an out-of-band metadata channel in the form of > headers. #occi's using Link: as a flexible, lightweight RDF > alternative". > > I'm a bit confused here.... I thought this was still under discussion. >  What am I missing? > > alexis > _______________________________________________ > occi-wg mailing list > occi-wg@ogf.org > http://www.ogf.org/mailman/listinfo/occi-wg > > >

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

-- Nothing is ever easy.

-- Nothing is ever easy.

Quoting [Alexis Richardson] (Oct 19 2009):

Personally, I don't mind people creating and editing a branch, so long as the merge algorithm involves acute, chair-mediated, community consensus at specific points in time.

alexis

Agree A. -- Nothing is ever easy.