Ahh, ok (sorry... still an OCCI newbie).  Disregard my last call to arms.

-Adrian

On Tue, Oct 20, 2009 at 9:56 AM, Sam Johnston <samj@samj.net> wrote:

Adrian,

We don't vote... we discuss our way to consensus and escalate if we
fail.

Anyway I've had enough for one day/lifetime and I've got a house full
of furniture to build tonight. I do wonder how many people here have
actually bothered to read the spec and understand the issues though -
if you haven't already then please do so now and speak up if anything
doesn't make sense.

Sent from my iPhone

On 20/10/2009, at 18:44, Adrian Cole <adrian@jclouds.org> wrote:

As we aren't introducing new information, I think we are at a point
where a vote can take place.  This issue isn't infinitely complex, and
reasonable to model in choices.  OCCI desires to be a community
process, so let's see where we fall on this.

Do you agree?
-Adrian

On Tue, Oct 20, 2009 at 9:39 AM, Alexis Richardson
<alexis.richardson@gmail.com> wrote:

Sam,

On Tue, Oct 20, 2009 at 5:31 PM, Sam Johnston <samj@samj.net> wrote:

Alexis,

Sweeping, unjustified statements are intensely frustrating, not to
mention unhelpful in the absence of a coherent alternative. If
people
choose not to disclose conflicts I happen to be aware of then you
bet
I'll disclose them for them - nothing personal about it.

I don't know what 'conflicts' you are referring to, or how these
relate to the matter at hand.  If by 'sweeping, unjustified' you
refer
to the word 'unprecedented' then I shall justify it by pointing out
that - at least as I currently understand it - you are proposing use
of headers to a *greater* extent than has been done before and to
cover a *greater* set of use cases.

Cutting to the chase, both Microsoft and Amazon are using headers
for
metadata to the same extent as what has been proposed.

Really - to the "same" extent?  That seems contrary to what has been
asserted on this thread previously.

What premature optimisation? *Not* using XML? Do we really need to
drag that carcass out again? Surely Adrian's performance tests put
the
last nail in that particular coffin.

Premature optimisation = making performance a critical design concern
when we want an API.

I have noted Adrian's points as have others.

The question really is about whether we want to be an interface/
protocol like HTTP or some mongrel hybrid of that along with a set
of
XML, JSON and/or text formats for everything we ever intend to model
(knowing that work has already been done by others).

Work that has been done by others and shown to work, is a good thing.

alexis

Sent from my iPhone

On 20/10/2009, at 18:10, Alexis Richardson
<alexis.richardson@gmail.com> wrote:

Sam

Cut the personal stuff please.

Re 'unproven', I think the issues people are having are:

* The extent, scope and purpose of your application of headers to
metadata is unprecedented - as I understand it.  Nobody is saying
it
has not been done in other and restricted cases.

* Premature optimisation.

alexis

On Tue, Oct 20, 2009 at 4:44 PM, Sam Johnston <samj@samj.net>
wrote:

Gary,

Please be specific with your objections so as I can have some
hope of
answering them... "several other emails" does not suffice as to
the
best of my knowledge there are no outstanding issues that I have
not
comprehensively and convincingly dealt with.

Your repeated assertions than HTTP headers are... what is it
now...
"unproven"... are demonstrably unfounded (refer to the various
cloud
APIs referred to by myself and Adrian, including both MS Azure and
Amazon S3, as well as the history of the Internet in general) and
given the frequency, bordering on FUD.

Regarding performance, OCCI should strive to be as efficient as
possible as it primarily targets public cloud providers who
operate
at
arbitrarily large "cloud" scale. The existing specification is
*extremely* efficient and alternatives would at least want to be
in
the same neighbourhood. Efficiency may be an inconvenient factor
for
whatever your alternative is, but it's critical for many of our
use
cases.

Finally, you reject OVF without providing any alternative beyond
building a replacement from scratch ourselves, after our second
and
final deadline has just sailed by no less. Have you considered the
implementation cost of this? And in the same breath you accuse my
proposal of being "unproven"? Give me a break...

Correct me if I'm wrong but the way I see it is that you want to
revert the last 2-3 months of my work because you have an
investment
in an old version of the spec. With each day I spend on OCCI
costing
me well over €1,000 excuse me for being pissed about someon
e with
such a blatantly obvious conflict attacking my work and refusing
to
provide a coherent argument, particularly with no "camera ready"
alternative anywhere in sight.

Sent from my iPhone

On 20/10/2009, at 16:51, Gary Mazz <garymazzaferro@gmail.com>
wrote:

I've understood Sam's proposal from the beginning. I have had
issue
with from the beginning.

Although I find it very interesting and valuable in terms of
economy, I have issue with it as the OCCI's primary interface
implementation for a few reasons which have been outlined in
several
other emails.

It appears that Sam has created a new technology, a lower cost
RPC
mechanism, which is unproven for both small and wide scale
deployment. For interoperability, IMO,  occi should focus on
proven
mainstream technologies as the primary interfaces. I am not
suggesting, shelving or not supporting Sam's proposal, based on
market acceptance, it may prove to be one of many occi interface
implementations.

As for cost saving in interface technologies, I do not buy into
the
idea that the occi traffic will be so overwhelming that there
is a
requirement to maximally optimize http requests and responses.
IMO,
in comparison to images and consumer data occi traffic will be
insignificant. I think we are worrying about spilled milk while
the
house is burning.

There has been some discussion on supporting technologies
including
OVF. Whether occi supports OVF has not been fully discussed,
examined or any consensus reached. I believe adopting
technologies
like OVF will only lead to mapping issues between occi attributes
and the external formats creating a long term management
headache.

cheers,
gary

Alexis Richardson wrote:

OK

Is Sam's statement something that we can achieve consensus
around?

a

On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj@samj.net>
wrote:

Alexis,

Seems our posts crossed in the mail. I hope the last one clears
things up
but the fact that you're using OVF and XML/JSON interchangeably
suggests
that you [are possibly one of many who] don't grok the
proposal.
Worse, it
seems you've got it back to front in thinking of the "core" as
metadata,
which is not at all what I had intended.

The focus is on the resources and their various
representations.
The
metadata (that is, data about data) essentially provides
whatever is
necessary to "glue" these resources together. It is basically
using HTTP as
the meta-model rather than an envelope format like Atom/SOAP
or,
worse, a
meta-model that we have to create ourselves from scratch.

So it's more like:

1) OCCI is compatible with ALL existing formats, eg OVF

2) Existing formats are used are used for ALL representations

3) ALL additional metadata should be in the headers

4) ALL of the core stuff should be in the body (that is, the
metadata in the
headers is just there to support the data and wire it up in
useful
ways)

I think the confusion stems from your assuming that there is
still
XML/JSON/Atom/etc. representations. There aren't. HTTP headers
provide a
single, performant, interoperable metadata channel that has
already been
both standardised and extensively implemented.

If you've followed me this far then you should have realised
that
every HTTP
user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl,
etc.)
is already
a basic OCCI client out-of-the-box. The result is that 5-line
scripts like
demo.py are functional with no dependence on client libraries
(that would
absolutely be necessary if you were to start trying to define
your
own
formats).

Sam

PS I'm sorry my "bunk" comment came across as referring to you
personally -
it was directed at the argument.

On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson
<alexis.richardson@gmail.com> wrote:

I think what Sam is proposing is the following:

1) If OCCI can handle 'any' data representation, eg OVF, XML,
JSON,
then it needs some core / common model.  Otherwise: there can
be no
defined behaviour / interop

2) This core / common model is ALL metadata

3) ALL this metadata should be in the headers

4) ALL the non core stuff eg OVF payload, XML representation,
should be in
body

Sam - is that right?

a

On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj@samj.net>
wrote:

On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX
<andrewx.edmonds@intel.com> wrote:

To that point I'd ask - If we're talking about meta-data
that
infers
there
is some data to accompany it. Where are the examples of this
data? This
would help in forming the full picture.

Here, from the core spec (using OVF as an example, but
could be
anything).
Request is green, response metadata is yellow & response
data/
payload is
orange:

GET /us-east/webapp/vm01 HTTP/1.1
User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0
       Host: cloud.example.com
Accept: */*

< HTTP/1.1 200 OK
< Date: Sat, 10 Oct 2009 12:56:51 GMT

< Content-Type: application/ovf
< Link: </us-east/webapp/vm01;start>;

<       rel="http://purl.org/occi/action/start";
<       title="Start"

< Link: </us-east/webapp/build.pdf>;
<       rel="related";

<       title="Documentation";
<       type="application/pdf"

< Category: compute;
<       label="Compute Resource";

<       scheme="http://purl.org/occi/kind/"
< Server: occi-server/1.0 (linux) OCCI/1.0

< Connection: close
<
< <?xml version="1.0" encoding="UTF-8"?>

< <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-
instance"
<           xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/
1"

<           xmlns="http://schemas.dmtf.org/ovf/envelope/1"
<           xml:lang="en-US">

...

-----Original Message-----
From: occi-wg-bounces@ogf.org [mailto:occi-wg-
bounces@ogf.org] On
Behalf
Of Alexis Richardson
Sent: 20 October 2009 09:49
To: Sam Johnston
Cc: Tim Bray; occi-wg@ogf.org
Subject: Re: [occi-wg] confusion about status of link /
headers

Sam,

Please don't throw around words like "bunk" willy nilly.

I have just had a look at
http://www.rackspacecloud.com/cloud_hosting_products/
servers/
api

I can see some limited use of headers for metadata in a few
places.
That's never been controversial.  But the understanding
that I
have of
your proposal is to use headers wholesale for all metadata.
Rackspace
don't appear to do that, unless I have misunderstood their
document.
I also may have misunderstood your proposal.  If nobody
understands
your proposal except you, then it will not be possible to
gain
consensus around it.

Are you saying "OCCI metadata should be more like Rackspace
metadata"?

alexis

On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston
<samj@samj.net>
wrote:

On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson
<alexis.richardson@gmail.com> wrote:

What about compute clouds?

Rackspace Cloud API for a start:

HTTP/1.1 204 No Content
Date: Mon, 12 Nov 2007 15:32:21 GMT
Server: Apache
X-Server-Management-Url:
https://servers.api.rackspacecloud.com/v1.0/35428
X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4
X-CDN-Management-Url:
https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
Content-Length: 0
Content-Type: text/plain; charset=UTF-8

Microsoft Azure uses headers for "attributes" as well, in
the
same
way
as I
had originally proposed:

x-ms-meta-name: value     Returns a metadata value for the
container.

However prefer that the header names are static/opaque
rather
than
using
a
template and parsing them:

Attribute: name=value

This to me is cleaner and more self-explanatory, plus it is
easily
collapsed
ala:

Attribute: name1=value1, name2=value2, ... namen=valuen

Anyway suffice to say that claims this is somehow
"experimental" are
bunk.

Sam

On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian@jclouds.org

wrote:

Here are options for metadata used in some of the major
storage
clouds
FWIW:

S3, Rackspace, EMC Atmos, Azure - Headers
Nirvanix - query params in, xml entity out
Mezeo - entity

Of the ones using headers, S3, Rackspace and Azure use
prefix with
values stored as-us.  Atmos joins all metadata together
into
one
header, making parsing trivial (split /,/), but
necessary.

The most expensive option of the above is entity, where
each
metadata
value is a separate GET.  However, entities allow binary
metadata
and
zero restrictions on it, which may be useful.

In jclouds, we time parsing of response values.  A simple
XML doc
with
only several elements written in SAX takes a few ms to
parse.  My
log
files are not precise enough to find the overhead in
parsing
headers:
they always start and finish within the same millisecond.

I hope this background helps, and also helps explain
why I'm
vocal
on
such topics such as headers vs entities :)

Cheers,
-Adrian

On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston
<samj@samj.net>
wrote:

On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro
<garymazzaferro@gmail.com>
wrote:

The http header and key/value pairs need to parsed
also,
there
is
no
free
ride here.

Every HTTP library I have ever used parses HTTP
headers and
puts
them
in a
nice hash for you ready to consume. If we go for "Name:
Value"
then
that's
all there is to it. If we go for "Attribute:
name=value" as
is
currently
proposed (which is arguably cleaner, follows cookies'
"prior art"
and
avoids
Amazon's prefix hack) then you just have to split on
'='.

To illustrate how clean this is by example:

#!/usr/bin/python
import urllib2
response = urllib2.urlopen('http://cloud.example.com/
myvm')
representation = response.read()
metadata = response.info()
print metadata['occi-compute-cores']

As soon as you start talking about payloads you have to
fire up a
parser
(JSON/XML/Atom/etc.) or write your own (previous text
rendering)
which
is
significantly more work to do at both design and run
times.
Not
to
mention
more work for us to do now and more scope for
interoperability
problems.

Sam

_______________________________________________
occi-wg mailing list
occi-wg@ogf.org
http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________
occi-wg mailing list
occi-wg@ogf.org
http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________
occi-wg mailing list
occi-wg@ogf.org
http://www.ogf.org/mailman/listinfo/occi-wg
---
----------------------------------------------------------
Intel Ireland Limited (Branch)
Collinstown Industrial Park, Leixlip, County Kildare,
Ireland
Registered Number: E902934

This e-mail and any attachments may contain confidential
material for
the sole use of the intended recipient(s). Any review or
distribution
by others is strictly prohibited. If you are not the
intended
recipient, please contact the sender and delete all copies.

_______________________________________________
occi-wg mailing list
occi-wg@ogf.org
http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________
occi-wg mailing list
occi-wg@ogf.org
http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________
occi-wg mailing list
occi-wg@ogf.org
http://www.ogf.org/mailman/listinfo/occi-wg