Format: 4 distinct decisions
All, I've just have a good phone conversation with Alexis, in which we agreed that the current formats discussion is actually addressing 4 independent questions simultaneously. I'd like to set these out individually, and get the group's opinion on each independent question individually rather than discussing them in combination as we have often done to date. Please can people follow up with their preference for each of decisions 1-4 independently. I would particularly appreciate the opinions of Randy and Tim as the other two cloud vendors involved in our mailing list, and of anyone else who may end up betting part of their business on OCCI. Cheers, Richard. =========================== Decision 1: Should OCCI specify wire format(s) or an abstract model: a) The OCCI API will be defined in terms of a specific rendering of the nouns, verbs and attributes to the actual bytes transferred on the wire. b) The OCCI API will be defined in terms of the abstract model (both nouns, verbs and attributes, and possibly also meta-model around these). Any implementation of this model is OCCI-compliant, regardless of the bytes on the wire. Perspective of myself and Alexis: OCCI should specify wire format(s), since this is the only way to guarantee in-practice interoperability between OCCI-compliant clouds. If OCCI will specify a wire format, then: Decision 2: Should OCCI-compliance require implementation of a single core wire format (any others are optional) or multiple core wire formats: a) Exactly one wire format is specified as core. A cloud must implement this alone to claim OCCI-compliance. Additional wire formats may be specified, but these are optional alternative renderings for easier integration with various external systems and cannot be relied upon for interoperability. b) Multiple wire formats as specified as core. A cloud must implement all of these to claim OCCI-compliance. Perspective of myself and Alexis: OCCI should have a single core wire format, since this is sufficient to ensure interoperability between OCCI-compliant clouds and minimizes the burden of OCCI-compliance. Additional alternative wire formats may be specified where these will aid integration with different types of system, but these will not be required to claim OCCI-compliance. These alternative wire formats may take different views on decision 3 and 4 from the core (e.g GData XML, GData JSON, Plain JSON and Plain XML can all coexist as alternative wire formats carrying the same nouns, verbs and attributes). Decision 3: For the core wire format(s), what meta-model should be adopted in rendering the nouns, verbs and attributes (i.e. a framework/style within which the nouns, verbs and attributes are rendered and any framework services are added): a) Minimal/no meta-model: Nouns, verbs and attributes are rendered as directly as possible over HTTP in a very basic REST style. b) GData meta-model: Nouns, verbs and attributes are rendered as GData objects and carried according to GData conventions with GData services. c) Other? Important*: This is an independent decision from decision 4 on syntax Perspective of myself and Alexis: For the core wire format, OCCI should minimize the meta model so that the specification of the core wire format is as small as possible, and hence interoperability between OCCI-compliant clouds is easiest to demonstrate, test and debug. Decision 4: For the core wire format(s), is XML or JSON the better syntax ('angle brackets vs. curly brackets') a) XML b) JSON Important*: This is an independent decision from decision 3 on meta-model Perspective of myself and Alexis: No strong view. We lean towards JSON, since it feels easier to make a tight specification and hence to easier to guarantee interoperability. Tightly specified use of XML might be equally good. * Decisions 3 and 4 are independent. Sam's GData XML http://www.ogf.org/pipermail/occi-wg/2009-May/000381.html is 3b with 4a. My Simple JSON http://www.ogf.org/pipermail/occi-wg/2009-May/000523.html is 3a with 4b. But equally it would be possible to specify GData JSON (3b, 4b) or Simple XML (3a, 4a).
Thijs has asked me to split out the four decisions into separate e-mail threads, hoping to separate the discussion of each independent question. So here goes - please try to separately your replies to each question and reply to the appropriate threads: Decision 1: Should OCCI specify wire format(s) or an abstract model: a) The OCCI API will be defined in terms of a specific rendering of the nouns, verbs and attributes to the actual bytes transferred on the wire. b) The OCCI API will be defined in terms of the abstract model (both nouns, verbs and attributes, and possibly also meta-model around these). Any implementation of this model is OCCI-compliant, regardless of the bytes on the wire. Perspective of myself and Alexis: OCCI should specify wire format(s), since this is the only way to guarantee in-practice interoperability between OCCI-compliant clouds.
So this one is a bit weird to me - even if you specific some on-the-wire format it still has to conform to a model of some sorts - even if it's implicit. Perhaps the following refinements below might be of help: Meta-model - model constraints OCCI: have one, if any, and simple see [1]. Model - what you can express in a graph, constrained by meta-model OCCI: have one and simple. Uses the collected nouns, verbs and attributes in [1]. Model instance - a user created graph based on concepts described in the model OCCI: a library in a particular language that creates an in-memory model and outputs a serialisation and performs the reverse. Model serialisation - on-the-wire format OCCI: defer to Richard's "Format 2" email thread. HTH, Andy [1] http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/NounsVerbsAnd... -----Original Message----- From: occi-wg-bounces@ogf.org [mailto:occi-wg-bounces@ogf.org] On Behalf Of Richard Davies Sent: 13 May 2009 15:40 To: occi-wg@ogf.org Subject: [occi-wg] Format 1: Wire format or abstract model Thijs has asked me to split out the four decisions into separate e-mail threads, hoping to separate the discussion of each independent question. So here goes - please try to separately your replies to each question and reply to the appropriate threads: Decision 1: Should OCCI specify wire format(s) or an abstract model: a) The OCCI API will be defined in terms of a specific rendering of the nouns, verbs and attributes to the actual bytes transferred on the wire. b) The OCCI API will be defined in terms of the abstract model (both nouns, verbs and attributes, and possibly also meta-model around these). Any implementation of this model is OCCI-compliant, regardless of the bytes on the wire. Perspective of myself and Alexis: OCCI should specify wire format(s), since this is the only way to guarantee in-practice interoperability between OCCI-compliant clouds. _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg ------------------------------------------------------------- Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
"Edmonds, AndrewX" <andrewx.edmonds@intel.com> writes:
So this one is a bit weird to me - even if you specific some on-the-wire format it still has to conform to a model of some sorts - even if it's implicit.
My understanding of the two options is either OCCI is just a model without a rendering, or OCCI is a model together with a rendering. Clearly a rendering implies a model, as you say, and defining a rendering doesn't prevent us making the model explicit in a more abstract sense. Cheers, Chris.
So sounds like the two options are pretty much the same. I'd be in favour of explicating the model. -----Original Message----- From: Chris Webb [mailto:chris.webb@elastichosts.com] Sent: 13 May 2009 17:09 To: Edmonds, AndrewX Cc: Richard Davies; occi-wg@ogf.org Subject: Re: [occi-wg] Format 1: Wire format or abstract model "Edmonds, AndrewX" <andrewx.edmonds@intel.com> writes:
So this one is a bit weird to me - even if you specific some on-the-wire format it still has to conform to a model of some sorts - even if it's implicit.
My understanding of the two options is either OCCI is just a model without a rendering, or OCCI is a model together with a rendering. Clearly a rendering implies a model, as you say, and defining a rendering doesn't prevent us making the model explicit in a more abstract sense. Cheers, Chris. ------------------------------------------------------------- Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
On Wed, May 13, 2009 at 6:08 PM, Chris Webb <chris.webb@elastichosts.com>wrote:
"Edmonds, AndrewX" <andrewx.edmonds@intel.com> writes:
So this one is a bit weird to me - even if you specific some on-the-wire format it still has to conform to a model of some sorts - even if it's implicit.
My understanding of the two options is either OCCI is just a model without a rendering, or OCCI is a model together with a rendering. Clearly a rendering implies a model, as you say, and defining a rendering doesn't prevent us making the model explicit in a more abstract sense.
Right. I'd be surprised if the consensus were to be that we didn't need to have both. The more interesting question for me is whether we simply map our model to something that already exists (AtomPub) or build the whole thing from scratch - which appears easy but isn't. I well understand the traditional process of formally documenting a model and then mechanically cranking out a wire protocol but the devil's in the details, especially at scale (where a single enterprise can have as many resources as ElasticHosts or even Amazon have in total). As an example, the introduction of ETags in GData 2.0 saved these guys<http://globelogger.com/2008/11/gdata-now-compliant-with-atompub.html>2 million calls a day - that sort of thing translates directly to wasted money when you have engineers sitting around waiting and lose infastructure agility (for example, machines losing sales because they take longer to respond to load changes). As a more concrete example, say I want to implement a desktop client for managing any OCCI-compliant service. I'm going want to maintain a local database of resources and at launch I'm going to want to check if anything's changed and ideally retrieve only what has (it's an enterprise remember so the initial load of millions of objects could take tens of minutes). How do we implement that, and do I want to bother knowing that AtomPub already does it <http://tools.ietf.org/html/rfc5023#section-9.5>? Sam
If OCCI will specify a wire format, then: Decision 2: Should OCCI-compliance require implementation of a single core wire format (any others are optional) or multiple core wire formats: a) Exactly one wire format is specified as core. A cloud must implement this alone to claim OCCI-compliance. Additional wire formats may be specified, but these are optional alternative renderings for easier integration with various external systems and cannot be relied upon for interoperability. b) Multiple wire formats as specified as core. A cloud must implement all of these to claim OCCI-compliance. Perspective of myself and Alexis: OCCI should have a single core wire format, since this is sufficient to ensure interoperability between OCCI-compliant clouds and minimizes the burden of OCCI-compliance. Additional alternative wire formats may be specified where these will aid integration with different types of system, but these will not be required to claim OCCI-compliance. These alternative wire formats may take different views on decision 3 and 4 from the core (e.g GData XML, GData JSON, Plain JSON and Plain XML can all coexist as alternative wire formats carrying the same nouns, verbs and attributes).
Richard, I am and have pretty much always been for option (a) - one core format for interop with supporting formats for integration. More than half of our use cases are actually integration, which is to say that interop is the exception rather than the rule. This makes sense when you consider that every cloud deployment needs integration but far fewer will ever actually need/want to move. That being the case I think there is an extremely strong case for making multiple formats as easy to support as possible (enter XSLT). There's an option missing which has been pushed on the list: "(c) Exactly one wire format. Period." Tim Bray's in that camp which is enough for me to say that it should be represented, even though I don't necessarily agree with the logic. Sam On Wed, May 13, 2009 at 4:40 PM, Richard Davies < richard.davies@elastichosts.com> wrote:
If OCCI will specify a wire format, then:
Decision 2: Should OCCI-compliance require implementation of a single core wire format (any others are optional) or multiple core wire formats: a) Exactly one wire format is specified as core. A cloud must implement this alone to claim OCCI-compliance. Additional wire formats may be specified, but these are optional alternative renderings for easier integration with various external systems and cannot be relied upon for interoperability. b) Multiple wire formats as specified as core. A cloud must implement all of these to claim OCCI-compliance.
Perspective of myself and Alexis: OCCI should have a single core wire format, since this is sufficient to ensure interoperability between OCCI-compliant clouds and minimizes the burden of OCCI-compliance. Additional alternative wire formats may be specified where these will aid integration with different types of system, but these will not be required to claim OCCI-compliance. These alternative wire formats may take different views on decision 3 and 4 from the core (e.g GData XML, GData JSON, Plain JSON and Plain XML can all coexist as alternative wire formats carrying the same nouns, verbs and attributes). _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
Decision 3: For the core wire format(s), what meta-model should be adopted in rendering the nouns, verbs and attributes (i.e. a framework/style within which the nouns, verbs and attributes are rendered and any framework services are added): a) Minimal/no meta-model: Nouns, verbs and attributes are rendered as directly as possible over HTTP in a very basic REST style. b) GData meta-model: Nouns, verbs and attributes are rendered as GData objects and carried according to GData conventions with GData services. c) Other? Important*: This is an independent decision from decision 4 on syntax Perspective of myself and Alexis: For the core wire format, OCCI should minimize the meta model so that the specification of the core wire format is as small as possible, and hence interoperability between OCCI-compliant clouds is easiest to demonstrate, test and debug.
To add a bit more to my perspective on this: * I am very interested in what Sam is proposing * But do not understand it * I don't understand how it is consistent with HATEOAS but look forward to being shown * I would love to hear Tim B's view on this area On Wed, May 13, 2009 at 3:41 PM, Richard Davies <richard.davies@elastichosts.com> wrote:
Decision 3: For the core wire format(s), what meta-model should be adopted in rendering the nouns, verbs and attributes (i.e. a framework/style within which the nouns, verbs and attributes are rendered and any framework services are added): a) Minimal/no meta-model: Nouns, verbs and attributes are rendered as directly as possible over HTTP in a very basic REST style. b) GData meta-model: Nouns, verbs and attributes are rendered as GData objects and carried according to GData conventions with GData services. c) Other?
Important*: This is an independent decision from decision 4 on syntax
Perspective of myself and Alexis: For the core wire format, OCCI should minimize the meta model so that the specification of the core wire format is as small as possible, and hence interoperability between OCCI-compliant clouds is easiest to demonstrate, test and debug. _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
Ok first let's degooglify this, especially now we know Microsoft are behind it as well: On Wed, May 13, 2009 at 4:41 PM, Richard Davies < richard.davies@elastichosts.com> wrote:
Decision 3: For the core wire format(s), what meta-model should be adopted in rendering the nouns, verbs and attributes (i.e. a framework/style within which the nouns, verbs and attributes are rendered and any framework services are added): a) Minimal/no meta-model: Nouns, verbs and attributes are rendered as directly as possible over HTTP in a very basic REST style. b) *AtomPub* meta-model: Nouns, verbs and attributes are rendered as *Atom * objects and carried according to *Atom* conventions with *AtomPub*services. c) Other?
For a) you still need a meta-model, you may just end up writing one yourself.
Perspective of myself and Alexis: For the core wire format, OCCI should minimize the meta model so that the specification of the core wire format is as small as possible, and hence interoperability between OCCI-compliant clouds is easiest to demonstrate, test and debug.
It's a leap of faith to claim the one which is "as small as possible" is also "easiest to demonstrate, test and debug" - this will rarely be the case (framing, error checking, documentation, etc. always adds [good] fat). I agree though with the assertion that "the core wire format [should be] as small as possible", and would go so far as to say that even resources should be extensions (there's no reason people shouldn't be able to use OCCI for bare hypervisors or storage and network devices - an all-in-one protocol is sure to cause problems there). That is, the core protocol should provide a skeleton and nothing more... OCCI without resources would be like HTTP without the verbs where implementors would take only the ones they need (as is basically the case today), including ones they define themselves (like EH's IP addresses that I maintain are unnecessary). A useful data point is that Microsoft started going down something like the POX (Plain Old XML) path with web3s<http://dev.live.com/livedata/web3s.htm#_Toc165288985>and this<http://www.25hoursaday.com/weblog/2008/02/28/WindowsLivePlatformNewsMicrosoftStandardizesOnAtomPubForWebServicesAndOtherStories.aspx>is what they had to say about their about face adoption of AtomPub: *The fact is when we listened to the community of Web developers the
feedback was overwhelmingly clear that people would prefer if we worked together with the community to make AtomPub work for the scenarios we felt it wasn’t suited for than Microsoft creating a competing proprietary protocol.*
Here's what one innocent bystander <http://www.markbaker.ca/blog/about/> had to say (better than I would have):
*I’m confident this is for the best. In addition to Atom/APP being existing standards (with an accompanying abundance of existing tooling), Microsoft will also gain the evolutionary advantages of the hypermedia as the engine of application state constraint, which Web3S opted to replace with a schema-driven application model. Kudos to everybody involved in that decision.*
The world doesn't need YAFP (yet another protocol) - it's got more than it can handle already - and even if it did I don't think this is the best group to define it (hello IETF)... besides we've got better/more important things to be doing than reinventing wheels. As such I'd definitely go for (b) (almost) every time. If we had a small, static set of stuff to do (think Twitter) then I might go for (a), but we don't and we can't even hope to predict everything people will want/need to do (besides, Twitter supports Atom, JSON and POX for most operations). Sam
Decision 4: For the core wire format(s), is XML or JSON the better syntax ('angle brackets vs. curly brackets') a) XML b) JSON Important*: This is an independent decision from decision 3 on meta-model Perspective of myself and Alexis: No strong view. We lean towards JSON, since it feels easier to make a tight specification and hence to easier to guarantee interoperability. Tightly specified use of XML might be equally good.
My 2c on this one: * I worry that allowing users to extend the core on the wire protocol by having something as extensible as XML, will lead them to do so in ways that break interop * I want something where it is really totally bogglingly obvious when the data deviates from the allowed format I note that eg HTTP has a very narrow core for interop, but people have come up with lots of ways to extend its usefulness by convention eg AtomPub On Wed, May 13, 2009 at 3:42 PM, Richard Davies <richard.davies@elastichosts.com> wrote:
Decision 4: For the core wire format(s), is XML or JSON the better syntax ('angle brackets vs. curly brackets') a) XML b) JSON
Important*: This is an independent decision from decision 3 on meta-model
Perspective of myself and Alexis: No strong view. We lean towards JSON, since it feels easier to make a tight specification and hence to easier to guarantee interoperability. Tightly specified use of XML might be equally good. _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
Alexis, All of these points actually work in favour of XML - comments inline: On Wed, May 13, 2009 at 4:50 PM, Alexis Richardson < alexis.richardson@gmail.com> wrote:
My 2c on this one:
* I worry that allowing users to extend the core on the wire protocol by having something as extensible as XML, will lead them to do so in ways that break interop
I think we all [need to] accept that users must be able to extend the protocol and in order to do so safely XML offers namespaces (among other things) - in fact we need to do the same ourselves unless we're expecting to deliver everything (billing, SLAs, monitoring, etc.) at the same time in one bundle. Most alternatives (including JSON) put everyone in a room together and still expect to be able to work out who farted when things go wrong.
* I want something where it is really totally bogglingly obvious when the data deviates from the allowed format
Tightly specified XML is trivial to burn into any one of a number of schema formats (e.g. RELAX NG) and then you've got on- and off-line validators which can tell you exactly which line was broken and why. This is marvellous for debugging and was used extensively to beat the reference implementation into compliance with Atom 1.0, without having to rely on talking to another implementation which may or may not be right. Most alternatives (including JSON) have no such functionality... at least not yet.
I note that eg HTTP has a very narrow core for interop, but people have come up with lots of ways to extend its usefulness by convention eg AtomPub
You realise of course that this works because HTTP adds a small amount of structure to an arbitrary/opaque data channel? Atom does the same, just adding the bare minimum of XML structure in the process... Sam
On Wed, May 13, 2009 at 3:42 PM, Richard Davies <richard.davies@elastichosts.com> wrote:
Decision 4: For the core wire format(s), is XML or JSON the better syntax ('angle brackets vs. curly brackets') a) XML b) JSON
Important*: This is an independent decision from decision 3 on meta-model
Perspective of myself and Alexis: No strong view. We lean towards JSON, since it feels easier to make a tight specification and hence to easier to guarantee interoperability. Tightly specified use of XML might be equally good. _______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
participants (5)
-
Alexis Richardson -
Chris Webb -
Edmonds, AndrewX -
Richard Davies -
Sam Johnston