On repainting the bikeshed at the 11th hour...
Morning all, So I've been on the road for the last half a dozen hours watching somewhat of a debate has kick off as to whether we should continue to use angle brackets (XML) to dice our data or start from scratch with curly braces (JSON). Sure it's understandable given the previous mess<http://stage.vambenepe.com/archives/700>(whereby literally dozens of different "standards" were born only to end up dead and dying) but that's essentially what it boils down to and it does feel a lot like we've taken a trip back to 2006<http://www.oreillynet.com/xml/blog/2006/12/and_the_winner_of_the_json_vs.html>. As one person pointed out in private it's clear there is now a good deal of backlash against "XML/SOAP/complexity" - apparently enough to consider wiping the slate clean of many years of hard (albeit thus far largely futile) work. I'm not particularly religious about the matter either way (my only vested interest is in the time I've volunteered to date and what little I have left in which to deliver results) but I do think the model is *far* more important and worthy of detailed discussion than the protocol. I've seen neither XML nor Atom impinge on anything we've needed to codify so far and once you have to reach for a library (as you do in both cases) the machines could talk pig latin to each other for all we care. The primary (and only valid IMO) objection to XML is that it tends to be abused more often than not, but zero examples of such abuse in our designs have been given and we've agreed to limit ourselves to an essentially flat structure (that must be able to be cleanly represented in alternative formats including JSON and TXT) so this is a non-issue anyway. Looking at the practicalities, XML is very mature with plenty of expertise and tools (from editors to application accelerators/firewalls) available for it while JSON is a rapidly developing new kid on the block (or is at least perceived to be by enterprise users - and perception/marketing is half of the battle). There are a lot of things we can do today with XML that we can't [and may never be able to] do with JSON such as transparently embedding supporting formats like OVF (which like it or not is going to have its time in the sun), transforming it to whatever format(s) we like in a mechanical, platform independent fashion and serialising, encrypting and signing it (for storage, queueing, etc). Atom itself is already arguably the single most successful "cloud" standard with Google using it for pretty much every public facing interface (protocol buffers are used under the covers for efficiency) and every feed on the web is using it (or something like it). There is a good deal of overlap with whatever domain specific language we might have invented ourselves (I know because I tried this before [re]discovering Atom) and it is a better fit than you might first think - feel free to go through the exercise if you need to see for yourself. In terms of the future it's also far more extensible in that adding new resources et al to a domain specific language would almost certainly require us to reconvene to create OCCI 2.0, while extensions can be safely implemented (and validated) without risk to the rest of the spec; this is not something I see as being particularly well handled with JSON now or any time soon. Furthermore, things we currently take for granted behind the scenes with XML (like being able to pass around a DOM object and have different components update it as necessary before handing the result to a renderer) are not possible unless you happen to be writing JavaScript (one of the only languages where JSON is a first class citizen rather than a third party extension). Given the current plan is to support multiple formats (even if we only actually require one of them) while maintaining the ability to move losslessly between them, there is nothing stopping us from shifting our weight to the other foot if and when JSON matures. The web never mandated the use of a single format so I don't see why we should start now (though telling implementors they must support all of them is a tall order). In terms of reaching a loose consensus I think the EH/GG guys will be satisfied if we can get clean[er] transforms, Sun ought to be happy if we follow their methodologies if not their exact protocol (e.g. single entry point, "buttons" for state changes, etc.), Google may well get on board if we make it easy for them and I'll be satisfied if I don't have to throw away a substantial base of work already completed as well as the opportunity to use the myriad existing XML/Atom/GData tools and clients (remembering a domain specific language starts with exactly zero implementations). Of course if someone *does* have a particularly strong fetish for curly braces and cares to rewrite the spec as well as a reference implementation and a bunch of transforms by next week then be my guest. Sam
Sam, Painting this as a mere difference between angle brackets or curly braces trivializes or avoids the central problem: the very tools you cite as advantages of XML are, in my opinion, merely symptoms of the complexity inherent in using XML, especially in the so-called "enterprise environment". The question of maturity seems to be entirely about those same tools, rather than about any concerns about JSON being a moving target (it isn't). Sun has provided a compelling example of how to use JSON in the construction of a simple, RESTful cloud interface, yet all your arguments against it are nebulous. What _specific_ thing is it you believe meets these criteria: 1) Is a concrete requirement of the core interface 2) Is possible with XML (without dragging in the very baggage that has sunk so many other "enterprise" protocols based on it) 3) Is not possible with JSON I am not talking nice-to-haves, here. I mean real, protocol can't function without it stuff, because if the core protocol _can_ function without a feature, why on earth would you put it in? I recommend the following exercise: divide the features into "definitely need right now", "might need at some point", "definitely won't need". Discard all but the first category in creating the core protocol, compare the result to the Sun API, smile with recognition, adopt it as the base, move on. Ben On Wed, May 6, 2009 at 4:42 PM, Sam Johnston <samj@samj.net> wrote:
Morning all,
So I've been on the road for the last half a dozen hours watching somewhat of a debate has kick off as to whether we should continue to use angle brackets (XML) to dice our data or start from scratch with curly braces (JSON). Sure it's understandable given the previous mess<http://stage.vambenepe.com/archives/700>(whereby literally dozens of different "standards" were born only to end up dead and dying) but that's essentially what it boils down to and it does feel a lot like we've taken a trip back to 2006<http://www.oreillynet.com/xml/blog/2006/12/and_the_winner_of_the_json_vs.html>. As one person pointed out in private it's clear there is now a good deal of backlash against "XML/SOAP/complexity" - apparently enough to consider wiping the slate clean of many years of hard (albeit thus far largely futile) work.
I'm not particularly religious about the matter either way (my only vested interest is in the time I've volunteered to date and what little I have left in which to deliver results) but I do think the model is *far* more important and worthy of detailed discussion than the protocol. I've seen neither XML nor Atom impinge on anything we've needed to codify so far and once you have to reach for a library (as you do in both cases) the machines could talk pig latin to each other for all we care. The primary (and only valid IMO) objection to XML is that it tends to be abused more often than not, but zero examples of such abuse in our designs have been given and we've agreed to limit ourselves to an essentially flat structure (that must be able to be cleanly represented in alternative formats including JSON and TXT) so this is a non-issue anyway.
Looking at the practicalities, XML is very mature with plenty of expertise and tools (from editors to application accelerators/firewalls) available for it while JSON is a rapidly developing new kid on the block (or is at least perceived to be by enterprise users - and perception/marketing is half of the battle). There are a lot of things we can do today with XML that we can't [and may never be able to] do with JSON such as transparently embedding supporting formats like OVF (which like it or not is going to have its time in the sun), transforming it to whatever format(s) we like in a mechanical, platform independent fashion and serialising, encrypting and signing it (for storage, queueing, etc).
Atom itself is already arguably the single most successful "cloud" standard with Google using it for pretty much every public facing interface (protocol buffers are used under the covers for efficiency) and every feed on the web is using it (or something like it). There is a good deal of overlap with whatever domain specific language we might have invented ourselves (I know because I tried this before [re]discovering Atom) and it is a better fit than you might first think - feel free to go through the exercise if you need to see for yourself. In terms of the future it's also far more extensible in that adding new resources et al to a domain specific language would almost certainly require us to reconvene to create OCCI 2.0, while extensions can be safely implemented (and validated) without risk to the rest of the spec; this is not something I see as being particularly well handled with JSON now or any time soon. Furthermore, things we currently take for granted behind the scenes with XML (like being able to pass around a DOM object and have different components update it as necessary before handing the result to a renderer) are not possible unless you happen to be writing JavaScript (one of the only languages where JSON is a first class citizen rather than a third party extension).
Given the current plan is to support multiple formats (even if we only actually require one of them) while maintaining the ability to move losslessly between them, there is nothing stopping us from shifting our weight to the other foot if and when JSON matures. The web never mandated the use of a single format so I don't see why we should start now (though telling implementors they must support all of them is a tall order). In terms of reaching a loose consensus I think the EH/GG guys will be satisfied if we can get clean[er] transforms, Sun ought to be happy if we follow their methodologies if not their exact protocol (e.g. single entry point, "buttons" for state changes, etc.), Google may well get on board if we make it easy for them and I'll be satisfied if I don't have to throw away a substantial base of work already completed as well as the opportunity to use the myriad existing XML/Atom/GData tools and clients (remembering a domain specific language starts with exactly zero implementations).
Of course if someone *does* have a particularly strong fetish for curly braces and cares to rewrite the spec as well as a reference implementation and a bunch of transforms by next week then be my guest.
Sam
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
Morning Sam, On Thu, May 7, 2009 at 12:42 AM, Sam Johnston <samj@samj.net> wrote:
Morning all,
So I've been on the road for the last half a dozen hours watching somewhat of a debate has kick off as to whether we should continue to use angle brackets (XML) to dice our data or start from scratch with curly braces (JSON).
We did agree to come back to the data formats, so I don't think we are starting from scratch.
I'm not particularly religious about the matter either way (my only vested interest is in the time I've volunteered to date and what little I have left in which to deliver results) but I do think the model is far more important and worthy of detailed discussion than the protocol.
+1 The model is really important. But that does not mean the formats discussion is a bikeshed discussion. Well, maybe it is. I am not sure. But several strong advocates have emerged for JSON which at least is consistent with our approach of keeping data formats flat and simple. So let's work this out.
I've seen neither XML nor Atom impinge on anything we've needed to codify so far and once you have to reach for a library (as you do in both cases) the machines could talk pig latin to each other for all we care.
Now there's an API I would like to see ;-) http://3.bp.blogspot.com/_w55opxuxeX8/RgBBr8FpH0I/AAAAAAAAAQs/W3OwpT_LRF8/s3...
The primary (and only valid IMO) objection to XML is that it tends to be abused more often than not, but zero examples of such abuse in our designs have been given and we've agreed to limit ourselves to an essentially flat structure (that must be able to be cleanly represented in alternative formats including JSON and TXT) so this is a non-issue anyway.
I'd like to hear more about why Sun moved off XML onto JSON. That said, the issue is not whether we abuse our own design, it's about whether we make it easy for end users to do so. That could be worse.
Looking at the practicalities, XML is very mature with plenty of expertise and tools (from editors to application accelerators/firewalls) available for it while JSON is a rapidly developing new kid on the block (or is at least perceived to be by enterprise users - and perception/marketing is half of the battle).
Personally I disagree with this. JSON is not rapidly developing. I have not encountered any enterprise that objects to JSON in my own experience with RabbitMQ, possibly because enterprise users are also (usually) humans.
There are a lot of things we can do today with XML that we can't [and may never be able to] do with JSON such as transparently embedding supporting formats like OVF (which like it or not is going to have its time in the sun), transforming it to whatever format(s) we like in a mechanical, platform independent fashion and serialising, encrypting and signing it (for storage, queueing, etc).
I think this is the point we need to consider. Our charter talks about playing nicely with OVF. Richard --- how would you feel about OVF-JSON? I note that the OVF spec XML appears to be flat. Perhaps a machine could rewrite it as JSON ;-) I am only half joking. We did this with AMQP, which condensed the spec down to just a few pages, see e.g. http://hg.rabbitmq.com/rabbitmq-codegen/file/1e6ad9a9cd37/amqp-0.8.json
Atom itself is already arguably the single most successful "cloud" standard with Google using it for pretty much every public facing interface (protocol buffers are used under the covers for efficiency) and every feed on the web is using it (or something like it).
This argument carries weight if we need FEEDS for OCCI. I have a hunch that feeds would be useful but it would really help if you could make the argument that they are necessary. You also made the point that Atom gives us a way of dealing with metadata and it would be helpful to spell that out again.
There is a good deal of overlap with whatever domain specific language we might have invented ourselves (I know because I tried this before [re]discovering Atom) and it is a better fit than you might first think - feel free to go through the exercise if you need to see for yourself.
It would be much easier if you took us through the exercise ;-)
In terms of the future it's also far more extensible in that adding new resources et al to a domain specific language would almost certainly require us to reconvene to create OCCI 2.0, while extensions can be safely implemented (and validated) without risk to the rest of the spec; this is not something I see as being particularly well handled with JSON now or any time soon. Furthermore, things we currently take for granted behind the scenes with XML (like being able to pass around a DOM object and have different components update it as necessary before handing the result to a renderer) are not possible unless you happen to be writing JavaScript (one of the only languages where JSON is a first class citizen rather than a third party extension).
Sure ... but why is it necessary to mess with DOM objects. I am not saying you are wrong, I am just trying to understand what use case this solves. I have no problem with reconvening for OCCI 2.0 or 1.1 ... but that won't happen unless 1.0 is good.
Given the current plan is to support multiple formats (even if we only actually require one of them) while maintaining the ability to move losslessly between them, there is nothing stopping us from shifting our weight to the other foot if and when JSON matures.
I think Tim has come forward with a strong counterargument to our initial view. As I understand it, he makes the point that we as a community do not have a strong basis for understanding interop which casts doubt on the value of standards work at this time. Yet, by giving people a way to see if interop can be achieved in any sense at all, based on some common and open interface, we can deliver value. In order to make interop maximally likely, one format is desirable. Allowing format flexibility *at this time* could compromise the main goal of being able to show interop using an open interface, which is the justification of OCCI.
The web never mandated the use of a single format so I don't see why we should start now (though telling implementors they must support all of them is a tall order). In terms of reaching a loose consensus I think the EH/GG guys will be satisfied if we can get clean[er] transforms, Sun ought to be happy if we follow their methodologies if not their exact protocol (e.g. single entry point, "buttons" for state changes, etc.), Google may well get on board if we make it easy for them and I'll be satisfied if I don't have to throw away a substantial base of work already completed as well as the opportunity to use the myriad existing XML/Atom/GData tools and clients (remembering a domain specific language starts with exactly zero implementations).
Well arguably some folks will be satisfied if we don't throw away a substantial base of work done by Sun ;-) Please can you elaborate re Google. GData supports JSON.
Of course if someone does have a particularly strong fetish for curly braces and cares to rewrite the spec as well as a reference implementation and a bunch of transforms by next week then be my guest.
It's not just about curly braces. It's about lowering the total cost of interop. alexis
Sam
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
On 5/7/09, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Morning Sam,
On Thu, May 7, 2009 at 12:42 AM, Sam Johnston <samj@samj.net> wrote:
Morning all,
So I've been on the road for the last half a dozen hours watching somewhat of a debate has kick off as to whether we should continue to use angle brackets (XML) to dice our data or start from scratch with curly braces (JSON).
We did agree to come back to the data formats, so I don't think we are starting from scratch.
Agreed but that was before we had an almost complete spec, a rough reference implementation complete with mechanical transforms to the requested formats and the start of a HTML interface for good measure. For JSON we have nothing and I'm on a flight to the cote d'azur for my birthday so you're on your own if you insist on making changes.
I'm not particularly religious about the matter either way (my only vested interest is in the time I've volunteered to date and what little I have left in which to deliver results) but I do think the model is far more important and worthy of detailed discussion than the protocol.
+1
The model is really important. But that does not mean the formats discussion is a bikeshed discussion. Well, maybe it is. I am not sure. But several strong advocates have emerged for JSON which at least is consistent with our approach of keeping data formats flat and simple. So let's work this out.
Given the way we use XML it is 100% religious, and unfortunate given JSON does not in any way cater for playing nice with other standards efforts nor transforming to multiple formats, among other things.
I've seen neither XML nor Atom impinge on anything we've needed to codify so far and once you have to reach for a library (as you do in both cases) the machines could talk pig latin to each other for all we care.
Now there's an API I would like to see ;-)
http://3.bp.blogspot.com/_w55opxuxeX8/RgBBr8FpH0I/AAAAAAAAAQs/W3OwpT_LRF8/s3...
Can't see that on the iPhone unfortunately :(
The primary (and only valid IMO) objection to XML is that it tends to be abused more often than not, but zero examples of such abuse in our designs have been given and we've agreed to limit ourselves to an essentially flat structure (that must be able to be cleanly represented in alternative formats including JSON and TXT) so this is a non-issue anyway.
I'd like to hear more about why Sun moved off XML onto JSON.
Q-Layer was always JSON wasn't it?
That said, the issue is not whether we abuse our own design, it's about whether we make it easy for end users to do so. That could be worse.
Looking at the practicalities, XML is very mature with plenty of expertise and tools (from editors to application accelerators/firewalls) available for it while JSON is a rapidly developing new kid on the block (or is at least perceived to be by enterprise users - and perception/marketing is half of the battle).
Personally I disagree with this. JSON is not rapidly developing. I have not encountered any enterprise that objects to JSON in my own experience with RabbitMQ, possibly because enterprise users are also (usually) humans.
The schema is a WiP for a start, and an essential enterprise feature.
There are a lot of things we can do today with XML that we can't [and may never be able to] do with JSON such as transparently embedding supporting formats like OVF (which like it or not is going to have its time in the sun), transforming it to whatever format(s) we like in a mechanical, platform independent fashion and serialising, encrypting and signing it (for storage, queueing, etc).
I think this is the point we need to consider.
Our charter talks about playing nicely with OVF.
Our success *requires* playing nice with OVF. And SNIA. And probably a bunch of other stuff in the future. Still zero answers on this absolute requirement from the JSON junkies too.
Richard --- how would you feel about OVF-JSON? I note that the OVF spec XML appears to be flat. Perhaps a machine could rewrite it as JSON ;-)
I am only half joking. We did this with AMQP, which condensed the spec down to just a few pages, see e.g. http://hg.rabbitmq.com/rabbitmq-codegen/file/1e6ad9a9cd37/amqp-0.8.json
Our spec will be concise anyway given how we use XML in the lightest way possible.
Atom itself is already arguably the single most successful "cloud" standard with Google using it for pretty much every public facing interface (protocol buffers are used under the covers for efficiency) and every feed on the web is using it (or something like it).
This argument carries weight if we need FEEDS for OCCI. I have a hunch that feeds would be useful but it would really help if you could make the argument that they are necessary. You also made the point that Atom gives us a way of dealing with metadata and it would be helpful to spell that out again.
Seeing the world as a bunch of feeds works better than you might think... look at Google's great example (which, unlike a lotof the other stuff we've been talkig about is in production on a massive scale).
There is a good deal of overlap with whatever domain specific language we might have invented ourselves (I know because I tried this before [re]discovering Atom) and it is a better fit than you might first think - feel free to go through the exercise if you need to see for yourself.
It would be much easier if you took us through the exercise ;-)
Will have to find some Internet in the south for that but sure, good idea.
In terms of the future it's also far more extensible in that adding new resources et al to a domain specific language would almost certainly require us to reconvene to create OCCI 2.0, while extensions can be safely implemented (and validated) without risk to the rest of the spec; this is not something I see as being particularly well handled with JSON now or any time soon. Furthermore, things we currently take for granted behind the scenes with XML (like being able to pass around a DOM object and have different components update it as necessary before handing the result to a renderer) are not possible unless you happen to be writing JavaScript (one of the only languages where JSON is a first class citizen rather than a third party extension).
Sure ... but why is it necessary to mess with DOM objects. I am not saying you are wrong, I am just trying to understand what use case this solves. I have no problem with reconvening for OCCI 2.0 or 1.1 ... but that won't happen unless 1.0 is good.
Cloud architectures are complex with many differs t extensions having to contribute to answers. JSON does not lend itself to this kind of thing so it's actually (a lot) more work for implementors. Suck it and see if you like.
Given the current plan is to support multiple formats (even if we only actually require one of them) while maintaining the ability to move losslessly between them, there is nothing stopping us from shifting our weight to the other foot if and when JSON matures.
I think Tim has come forward with a strong counterargument to our initial view. As I understand it, he makes the point that we as a community do not have a strong basis for understanding interop which casts doubt on the value of standards work at this time. Yet, by giving people a way to see if interop can be achieved in any sense at all, based on some common and open interface, we can deliver value. In order to make interop maximally likely, one format is desirable. Allowing format flexibility *at this time* could compromise the main goal of being able to show interop using an open interface, which is the justification of OCCI.
That's only true if you lack an automated way of supporting multiple formats, and we don't. I'm all for being nice to the users first and implementors second, even as someone with a foot in both camps.
The web never mandated the use of a single format so I don't see why we should start now (though telling implementors they must support all of them is a tall order). In terms of reaching a loose consensus I think the EH/GG guys will be satisfied if we can get clean[er] transforms, Sun ought to be happy if we follow their methodologies if not their exact protocol (e.g. single entry point, "buttons" for state changes, etc.), Google may well get on board if we make it easy for them and I'll be satisfied if I don't have to throw away a substantial base of work already completed as well as the opportunity to use the myriad existing XML/Atom/GData tools and clients (remembering a domain specific language starts with exactly zero implementations).
Well arguably some folks will be satisfied if we don't throw away a substantial base of work done by Sun ;-)
Please can you elaborate re Google. GData supports JSON.
GData clearly uses a standard XML to JSON XSLT stylesheet to do that... The result is pretty damn ugly but still very useable.
Of course if someone does have a particularly strong fetish for curly braces and cares to rewrite the spec as well as a reference implementation and a bunch of transforms by next week then be my guest.
It's not just about curly braces. It's about lowering the total cost of interop.
Right, and I reckon it rises without XML. Anyway taking off... gotta run Sam on iPhone (on plane)
alexis
Sam
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
Sam -- for the record, I find blanket assertions such as "an essential enterprise feature" to be problematic. I'm about as enterprisey as anyone could possibly be, and you are definitely *not* speaking on my behalf here. I've no doubt that you are speaking for someone or something that relates to your experience, and an alternative phrasing such as "an essential feature for the enterprise customers I work with" would clearly articulate that. But I"m here to tell you -- there's a large and valid population of folk who self-identify as "enterprise" who disagree with much of your reasoning. Here are some examples: -- "The schema is a WiP for a start, and an essential enterprise feature". No, it is not. In my world, we are busy trying to learn many lessons from the Web, only some of which fit within the scope of the term REST. One of the most important lessons we are trying to internalise and apply is that a priori schemas are sometimes much more trouble than they are worth. In that world, among other things, there are any number of examples of cases where I am trying to figure out how to *store my data* without an a priori schema, because that confers on me all sorts of advantages which seem to move me an inch or two towards the holy grail of disposable software. I can assure you that, in those sorts of contexts, the idea of going to those lengths, only to move data to and from those schema-less data stores via "standard" mechanisms that impose one of its own can provoke gales of (slightly bitter) laughter. -- "Our success *requires* playing nice with OVF. And SNIA. And probably a bunch of other stuff in the future" followed then by "Our spec will be concise anyway given how we use XML in the lightest way possible". It worries me that you may not see the contradiction lurking in those two statements. At best, these statements have a high probability of both being true only at a given point in time; one quite early in the life cycle of OCCI (which, of course, is where we are). Over time, it is *very* unlikely (verging on the impossible, IMO) that both statements can *remain* true. This is because the imperative underlying each of these statements is in direct conflict with the other -- "playing nice with others" is in direct conflict with "concise". We all seem to be in agreement about the merits of concise -- the disagreement revolves around the degree of "playing nice with others", and / or preparing for playing nice with others. I think there is a strong argument to be made for making a deliberate decision, at this point in the lifecycle, to *not* make playing nice with others a goal. Explicitly -- IOW, making it a non-goal. Instead, focus on what I thought this group was chartered to do -- the simplest possible common representation of what's already out there, working. The "others" be damned. -- "Cloud architectures are complex with many differs t extensions having to contribute to answers. JSON does not lend itself to this kind of thing so it's actually (a lot) more work for implementors" That sounds, to this "enterprise voice", like a feature, not a bug. Constraints are good. I like constraints. You have not yet convinced me that the "extensions" you refer to here are critical to the core of the aforementioned simplest possible common representation of what's already out there, working. Instead, the impression I get is that these extensions (or even the possibility of having them) serves primarily the "play nice with others" (non) goal. Since I don't think worrying about playing nice with others is a good idea, imposing a contstraint that makes it harder to do seems like a good idea to me. Helps focus on the core goal -- distractions fall away. In my experience, that's a design approach that, in turn, produces *very* robust artefacts -- ones on which it is subsequently relatively easy to build more stuff (to play with other, for example). Doing things the other way round (trying to anticipate and account for possible "extensions"), in my experience, too often leads to fragile, brittle designs. This argument, OTOH: -- "Agreed but that was before we had an almost complete spec, a rough reference implementation complete with mechanical transforms to the requested formats and the start of a HTML interface for good measure. For JSON we have nothing" ... is quite compelling, and in my view. And that may outweigh all other concerns, given the desire to hold the date. Dunno. Mark Masterson Enterprise architect, troublemaker CSC Financial Services EMEA | m: +49.172.6163412 | Internal blog: Schloss Masterson | Public blog: Process Perfection (http://jroller.com/MasterMark/) | mmasterson@csc.com | www.csc.com. CSC • This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind CSC to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose ï CSC Computer Sciences Limited • Registered Office: Royal Pavilion, Wellesley Road, Aldershot, Hampshire, GU11 1PZ, UK • Registered in England No: 0963578
Sam, You are consistently switching back and forth between specification issues and implementation issues in such a way that it is unclear if you recognize the importance of keeping them independent. That a _current_ implementation uses a number of XML features in a certain way that is convenient for its implementers does not mean we should force the use of XML on all implementers. Further, the notion that we can adopt XML, in large part for those vague, enterprise users, and manage, against all evidence, to _not_ pull in all the XML bells and whistles (meaning: keep it lightweight) boggles the mind. If the OCCI charter requires it to do such things, the charter should be changed. I am not religiously against XML, I am religiously in favor of simplicity. Ben On Thu, May 7, 2009 at 6:21 AM, Sam Johnston <samj@samj.net> wrote:
On 5/7/09, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Morning Sam,
On Thu, May 7, 2009 at 12:42 AM, Sam Johnston <samj@samj.net> wrote:
Morning all,
So I've been on the road for the last half a dozen hours watching somewhat of a debate has kick off as to whether we should continue to use angle brackets (XML) to dice our data or start from scratch with curly braces (JSON).
We did agree to come back to the data formats, so I don't think we are starting from scratch.
Agreed but that was before we had an almost complete spec, a rough reference implementation complete with mechanical transforms to the requested formats and the start of a HTML interface for good measure. For JSON we have nothing and I'm on a flight to the cote d'azur for my birthday so you're on your own if you insist on making changes.
I'm not particularly religious about the matter either way (my only vested interest is in the time I've volunteered to date and what little I have left in which to deliver results) but I do think the model is far more important and worthy of detailed discussion than the protocol.
+1
The model is really important. But that does not mean the formats discussion is a bikeshed discussion. Well, maybe it is. I am not sure. But several strong advocates have emerged for JSON which at least is consistent with our approach of keeping data formats flat and simple. So let's work this out.
Given the way we use XML it is 100% religious, and unfortunate given JSON does not in any way cater for playing nice with other standards efforts nor transforming to multiple formats, among other things.
I've seen neither XML nor Atom impinge on anything we've needed to codify so far and once you have to reach for a library (as you do in both cases) the machines could talk pig latin to each other for all we care.
Now there's an API I would like to see ;-)
http://3.bp.blogspot.com/_w55opxuxeX8/RgBBr8FpH0I/AAAAAAAAAQs/W3OwpT_LRF8/s3...
Can't see that on the iPhone unfortunately :(
The primary (and only valid IMO) objection to XML is that it tends to be abused more often than not, but zero examples of such abuse in our designs have been given and we've agreed to limit ourselves to an essentially flat structure (that must be able to be cleanly represented in alternative formats including JSON and TXT) so this is a non-issue anyway.
I'd like to hear more about why Sun moved off XML onto JSON.
Q-Layer was always JSON wasn't it?
That said, the issue is not whether we abuse our own design, it's about whether we make it easy for end users to do so. That could be worse.
Looking at the practicalities, XML is very mature with plenty of expertise and tools (from editors to application accelerators/firewalls) available for it while JSON is a rapidly developing new kid on the block (or is at least perceived to be by enterprise users - and perception/marketing is half of the battle).
Personally I disagree with this. JSON is not rapidly developing. I have not encountered any enterprise that objects to JSON in my own experience with RabbitMQ, possibly because enterprise users are also (usually) humans.
The schema is a WiP for a start, and an essential enterprise feature.
There are a lot of things we can do today with XML that we can't [and may never be able to] do with JSON such as transparently embedding supporting formats like OVF (which like it or not is going to have its time in the sun), transforming it to whatever format(s) we like in a mechanical, platform independent fashion and serialising, encrypting and signing it (for storage, queueing, etc).
I think this is the point we need to consider.
Our charter talks about playing nicely with OVF.
Our success *requires* playing nice with OVF. And SNIA. And probably a bunch of other stuff in the future. Still zero answers on this absolute requirement from the JSON junkies too.
Richard --- how would you feel about OVF-JSON? I note that the OVF spec XML appears to be flat. Perhaps a machine could rewrite it as JSON ;-)
I am only half joking. We did this with AMQP, which condensed the spec down to just a few pages, see e.g. http://hg.rabbitmq.com/rabbitmq-codegen/file/1e6ad9a9cd37/amqp-0.8.json
Our spec will be concise anyway given how we use XML in the lightest way possible.
Atom itself is already arguably the single most successful "cloud" standard with Google using it for pretty much every public facing interface (protocol buffers are used under the covers for efficiency) and every feed on the web is using it (or something like it).
This argument carries weight if we need FEEDS for OCCI. I have a hunch that feeds would be useful but it would really help if you could make the argument that they are necessary. You also made the point that Atom gives us a way of dealing with metadata and it would be helpful to spell that out again.
Seeing the world as a bunch of feeds works better than you might think... look at Google's great example (which, unlike a lotof the other stuff we've been talkig about is in production on a massive scale).
There is a good deal of overlap with whatever domain specific language we might have invented ourselves (I know because I tried this before [re]discovering Atom) and it is a better fit than you might first think - feel free to go through the exercise if you need to see for yourself.
It would be much easier if you took us through the exercise ;-)
Will have to find some Internet in the south for that but sure, good idea.
In terms of the future it's also far more extensible in that adding new resources et al to a domain specific language would almost certainly require us to reconvene to create OCCI 2.0, while extensions can be safely implemented (and validated) without risk to the rest of the spec; this is not something I see as being particularly well handled with JSON now or any time soon. Furthermore, things we currently take for granted behind the scenes with XML (like being able to pass around a DOM object and have different components update it as necessary before handing the result to a renderer) are not possible unless you happen to
be
writing JavaScript (one of the only languages where JSON is a first class citizen rather than a third party extension).
Sure ... but why is it necessary to mess with DOM objects. I am not saying you are wrong, I am just trying to understand what use case this solves. I have no problem with reconvening for OCCI 2.0 or 1.1 ... but that won't happen unless 1.0 is good.
Cloud architectures are complex with many differs t extensions having to contribute to answers. JSON does not lend itself to this kind of thing so it's actually (a lot) more work for implementors. Suck it and see if you like.
Given the current plan is to support multiple formats (even if we only actually require one of them) while maintaining the ability to move losslessly between them, there is nothing stopping us from shifting our weight to the other foot if and when JSON matures.
I think Tim has come forward with a strong counterargument to our initial view. As I understand it, he makes the point that we as a community do not have a strong basis for understanding interop which casts doubt on the value of standards work at this time. Yet, by giving people a way to see if interop can be achieved in any sense at all, based on some common and open interface, we can deliver value. In order to make interop maximally likely, one format is desirable. Allowing format flexibility *at this time* could compromise the main goal of being able to show interop using an open interface, which is the justification of OCCI.
That's only true if you lack an automated way of supporting multiple formats, and we don't. I'm all for being nice to the users first and implementors second, even as someone with a foot in both camps.
The web never mandated the use of a single format so I don't see why we should start now
(though
telling implementors they must support all of them is a tall order). In terms of reaching a loose consensus I think the EH/GG guys will be satisfied if we can get clean[er] transforms, Sun ought to be happy if we follow their methodologies if not their exact protocol (e.g. single entry point, "buttons" for state changes, etc.), Google may well get on board if we make it easy for them and I'll be satisfied if I don't have to throw away a substantial base of work already completed as well as the opportunity to use the myriad existing XML/Atom/GData tools and clients (remembering a domain specific language starts with exactly zero implementations).
Well arguably some folks will be satisfied if we don't throw away a substantial base of work done by Sun ;-)
Please can you elaborate re Google. GData supports JSON.
GData clearly uses a standard XML to JSON XSLT stylesheet to do that... The result is pretty damn ugly but still very useable.
Of course if someone does have a particularly strong fetish for curly braces and cares to rewrite the spec as well as a reference implementation and
a
bunch of transforms by next week then be my guest.
It's not just about curly braces. It's about lowering the total cost of interop.
Right, and I reckon it rises without XML.
Anyway taking off... gotta run
Sam on iPhone (on plane)
alexis
Sam
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
Ben, An unimplementable specification is as useless as teats on a bull... I do think it's important that we bear the developers in mind where possible as if it's easy to develop for it's more likely to be used extensively. I'm going to assume you know what atom looks like up close and personal as I'm confined to an iPhone for now so my demoing ability is limited... if we leave the content element well alone then it is no more complicated than any XML based domain specific language we would create, especially if we keep it flat and/or limit ourselves to what is possible in text and json. Further, these alternative transforms need be nothing more than convenience formats for writing shell scripts and web interfaces... and by providing ready made transforms they need be no more work for implementors. If an implementor wants to render direct to PDF for reporting then so be it. If they choose not to provide this extra functionality then let the market decide. So long as we focus on one primary format we're fine. Sam On 5/7/09, Benjamin Black <b@b3k.us> wrote:
Sam,
You are consistently switching back and forth between specification issues and implementation issues in such a way that it is unclear if you recognize the importance of keeping them independent. That a _current_ implementation uses a number of XML features in a certain way that is convenient for its implementers does not mean we should force the use of XML on all implementers. Further, the notion that we can adopt XML, in large part for those vague, enterprise users, and manage, against all evidence, to _not_ pull in all the XML bells and whistles (meaning: keep it lightweight) boggles the mind. If the OCCI charter requires it to do such things, the charter should be changed.
I am not religiously against XML, I am religiously in favor of simplicity.
Ben
On Thu, May 7, 2009 at 6:21 AM, Sam Johnston <samj@samj.net> wrote:
On 5/7/09, Alexis Richardson <alexis.richardson@gmail.com> wrote:
Morning Sam,
On Thu, May 7, 2009 at 12:42 AM, Sam Johnston <samj@samj.net> wrote:
Morning all,
So I've been on the road for the last half a dozen hours watching somewhat of a debate has kick off as to whether we should continue to use angle brackets (XML) to dice our data or start from scratch with curly braces (JSON).
We did agree to come back to the data formats, so I don't think we are starting from scratch.
Agreed but that was before we had an almost complete spec, a rough reference implementation complete with mechanical transforms to the requested formats and the start of a HTML interface for good measure. For JSON we have nothing and I'm on a flight to the cote d'azur for my birthday so you're on your own if you insist on making changes.
I'm not particularly religious about the matter either way (my only vested interest is in the time I've volunteered to date and what little I have left in which to deliver results) but I do think the model is far more important and worthy of detailed discussion than the protocol.
+1
The model is really important. But that does not mean the formats discussion is a bikeshed discussion. Well, maybe it is. I am not sure. But several strong advocates have emerged for JSON which at least is consistent with our approach of keeping data formats flat and simple. So let's work this out.
Given the way we use XML it is 100% religious, and unfortunate given JSON does not in any way cater for playing nice with other standards efforts nor transforming to multiple formats, among other things.
I've seen neither XML nor Atom impinge on anything we've needed to codify so far and once you have to reach for a library (as you do in both cases) the machines could talk pig latin to each other for all we care.
Now there's an API I would like to see ;-)
http://3.bp.blogspot.com/_w55opxuxeX8/RgBBr8FpH0I/AAAAAAAAAQs/W3OwpT_LRF8/s3...
Can't see that on the iPhone unfortunately :(
The primary (and only valid IMO) objection to XML is that it tends to be abused more often than not, but zero examples of such abuse in our designs have been given and we've agreed to limit ourselves to an essentially flat structure (that must be able to be cleanly represented in alternative formats including JSON and TXT) so this is a non-issue anyway.
I'd like to hear more about why Sun moved off XML onto JSON.
Q-Layer was always JSON wasn't it?
That said, the issue is not whether we abuse our own design, it's about whether we make it easy for end users to do so. That could be worse.
Looking at the practicalities, XML is very mature with plenty of expertise and tools (from editors to application accelerators/firewalls) available for it while JSON is a rapidly developing new kid on the block (or is at least perceived to be by enterprise users - and perception/marketing is half of the battle).
Personally I disagree with this. JSON is not rapidly developing. I have not encountered any enterprise that objects to JSON in my own experience with RabbitMQ, possibly because enterprise users are also (usually) humans.
The schema is a WiP for a start, and an essential enterprise feature.
There are a lot of things we can do today with XML that we can't [and may never be able to] do with JSON such as transparently embedding supporting formats like OVF (which like it or not is going to have its time in the sun), transforming it to whatever format(s) we like in a mechanical, platform independent fashion and serialising, encrypting and signing it (for storage, queueing, etc).
I think this is the point we need to consider.
Our charter talks about playing nicely with OVF.
Our success *requires* playing nice with OVF. And SNIA. And probably a bunch of other stuff in the future. Still zero answers on this absolute requirement from the JSON junkies too.
Richard --- how would you feel about OVF-JSON? I note that the OVF spec XML appears to be flat. Perhaps a machine could rewrite it as JSON ;-)
I am only half joking. We did this with AMQP, which condensed the spec down to just a few pages, see e.g. http://hg.rabbitmq.com/rabbitmq-codegen/file/1e6ad9a9cd37/amqp-0.8.json
Our spec will be concise anyway given how we use XML in the lightest way possible.
Atom itself is already arguably the single most successful "cloud" standard with Google using it for pretty much every public facing interface (protocol buffers are used under the covers for efficiency) and every feed on the web is using it (or something like it).
This argument carries weight if we need FEEDS for OCCI. I have a hunch that feeds would be useful but it would really help if you could make the argument that they are necessary. You also made the point that Atom gives us a way of dealing with metadata and it would be helpful to spell that out again.
Seeing the world as a bunch of feeds works better than you might think... look at Google's great example (which, unlike a lotof the other stuff we've been talkig about is in production on a massive scale).
There is a good deal of overlap with whatever domain specific language we might have invented ourselves (I know because I tried this before [re]discovering Atom) and it is a better fit than you might first think - feel free to go through the exercise if you need to see for yourself.
It would be much easier if you took us through the exercise ;-)
Will have to find some Internet in the south for that but sure, good idea.
In terms of the future it's also far more extensible in that adding new resources et al to a domain specific language would almost certainly require us to reconvene to create OCCI 2.0, while extensions can be safely implemented (and validated) without risk to the rest of the spec; this is not something I see as being particularly well handled with JSON now or any time soon. Furthermore, things we currently take for granted behind the scenes with XML (like being able to pass around a DOM object and have different components update it as necessary before handing the result to a renderer) are not possible unless you happen to
be
writing JavaScript (one of the only languages where JSON is a first class citizen rather than a third party extension).
Sure ... but why is it necessary to mess with DOM objects. I am not saying you are wrong, I am just trying to understand what use case this solves. I have no problem with reconvening for OCCI 2.0 or 1.1 ... but that won't happen unless 1.0 is good.
Cloud architectures are complex with many differs t extensions having to contribute to answers. JSON does not lend itself to this kind of thing so it's actually (a lot) more work for implementors. Suck it and see if you like.
Given the current plan is to support multiple formats (even if we only actually require one of them) while maintaining the ability to move losslessly between them, there is nothing stopping us from shifting our weight to the other foot if and when JSON matures.
I think Tim has come forward with a strong counterargument to our initial view. As I understand it, he makes the point that we as a community do not have a strong basis for understanding interop which casts doubt on the value of standards work at this time. Yet, by giving people a way to see if interop can be achieved in any sense at all, based on some common and open interface, we can deliver value. In order to make interop maximally likely, one format is desirable. Allowing format flexibility *at this time* could compromise the main goal of being able to show interop using an open interface, which is the justification of OCCI.
That's only true if you lack an automated way of supporting multiple formats, and we don't. I'm all for being nice to the users first and implementors second, even as someone with a foot in both camps.
The web never mandated the use of a single format so I don't see why we should start now
(though
telling implementors they must support all of them is a tall order). In terms of reaching a loose consensus I think the EH/GG guys will be satisfied if we can get clean[er] transforms, Sun ought to be happy if we follow their methodologies if not their exact protocol (e.g. single entry point, "buttons" for state changes, etc.), Google may well get on board if we make it easy for them and I'll be satisfied if I don't have to throw away a substantial base of work already completed as well as the opportunity to use the myriad existing XML/Atom/GData tools and clients (remembering a domain specific language starts with exactly zero implementations).
Well arguably some folks will be satisfied if we don't throw away a substantial base of work done by Sun ;-)
Please can you elaborate re Google. GData supports JSON.
GData clearly uses a standard XML to JSON XSLT stylesheet to do that... The result is pretty damn ugly but still very useable.
Of course if someone does have a particularly strong fetish for curly braces and cares to rewrite the spec as well as a reference implementation and
a
bunch of transforms by next week then be my guest.
It's not just about curly braces. It's about lowering the total cost of interop.
Right, and I reckon it rises without XML.
Anyway taking off... gotta run
Sam on iPhone (on plane)
alexis
Sam
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg
On Thu, May 7, 2009 at 2:42 PM, Sam Johnston <samj@samj.net> wrote:
Ben,
An unimplementable specification is as useless as teats on a bull... I do think it's important that we bear the developers in mind where possible as if it's easy to develop for it's more likely to be used extensively.
I agree, yet don't understand how it applies to anything I've said. Are you really suggesting that using JSON would result in a specification that is "unimplementable"?
I'm going to assume you know what atom looks like up close and personal as I'm confined to an iPhone for now so my demoing ability is limited... if we leave the content element well alone then it is no more complicated than any XML based domain specific language we would create, especially if we keep it flat and/or limit ourselves to what is possible in text and json.
Further, these alternative transforms need be nothing more than convenience formats for writing shell scripts and web interfaces...
And we end up right back in the space I described in an earlier email: either we force all implementations to implement all formats or we guarantee the production of incompatible, fully compliant implementations. Riding the fence just produces a sore bottom. Make a choice.
and by providing ready made transforms they need be no more work for implementors. If an implementor wants to render direct to PDF for reporting then so be it. If they choose not to provide this extra functionality then let the market decide. So long as we focus on one primary format we're fine.
Sam
"If we don't do all the complex stuff for which XML-based systems are notorious, we'll be fine". There is a first time for everything. I'm not going to hold my breath, though. Ben
On 5/8/09, Benjamin Black <b@b3k.us> wrote:
On Thu, May 7, 2009 at 2:42 PM, Sam Johnston <samj@samj.net> wrote:
Ben,
An unimplementable specification is as useless as teats on a bull... I do think it's important that we bear the developers in mind where possible as if it's easy to develop for it's more likely to be used extensively.
I agree, yet don't understand how it applies to anything I've said. Are you really suggesting that using JSON would result in a specification that is "unimplementable"?
No, I'm just saying that the two things need not be firewalled. Separating the protool from the model is justifiable however.
I'm going to assume you know what atom looks like up close and personal as I'm confined to an iPhone for now so my demoing ability is limited... if we leave the content element well alone then it is no more complicated than any XML based domain specific language we would create, especially if we keep it flat and/or limit ourselves to what is possible in text and json.
Further, these alternative transforms need be nothing more than convenience formats for writing shell scripts and web interfaces...
And we end up right back in the space I described in an earlier email: either we force all implementations to implement all formats or we guarantee the production of incompatible, fully compliant implementations. Riding the fence just produces a sore bottom. Make a choice.
No we don't - provided they implement the main format the rest is just icing on the cake. A single line cron job won't understand json either but that's not to say we should drop to the lowest common denominator, or that we shouldn't support this use case given we can trivially.
and by providing ready made transforms they need be no more work for implementors. If an implementor wants to render direct to PDF for reporting then so be it. If they choose not to provide this extra functionality then let the market decide. So long as we focus on one primary format we're fine.
Sam
"If we don't do all the complex stuff for which XML-based systems are notorious, we'll be fine". There is a first time for everything. I'm not going to hold my breath, though.
Have you seen the XML samples in the wiki or that generated by http://occiteat.appspot.com? Granted I need to add actuators/controllers and attributes like <mem>4096</mem> but you can already see the structure is clean. Sam
On May 7, 2009, at 6:21 AM, Sam Johnston wrote:
I'd like to hear more about why Sun moved off XML onto JSON.
Q-Layer was always JSON wasn't it?
I've already stated that Q-layer was not the issue. Our cloud-storage implementation came first, long before we acquired Q-layer. The initial implementation of the storage service was in JAX-B which gives you both XML and JSON for free. We have a lot of people with net- protocol experience in the shop, and we just don't buy into this multiple-formats thing; it adds complexity, loses clarity, and isn't how the Internet or the Web work. So we decided to go with just JSON, and the code I'm personally writing (Ruby interface to the q-layer back-end) is JSON-only and will stay that way. Why JSON? Because all our data elements are numbers or short chunks of text, do not involve fancy formatting, and fit naturally into lists and dicts. Nesting is non-zero but also not deep (a data center has clusters, clusters have machines). JSON had everything we needed and nothing we didn't need. Plain text makes the nesting hard. XML is overkill. -T
On 5/8/09, Tim Bray <Tim.Bray@sun.com> wrote:
On May 7, 2009, at 6:21 AM, Sam Johnston wrote:
I'd like to hear more about why Sun moved off XML onto JSON.
Q-Layer was always JSON wasn't it?
I've already stated that Q-layer was not the issue. Our cloud-storage implementation came first, long before we acquired Q-layer. The initial implementation of the storage service was in JAX-B which gives you both XML and JSON for free. We have a lot of people with net- protocol experience in the shop, and we just don't buy into this multiple-formats thing; it adds complexity, loses clarity, and isn't how the Internet or the Web work. So we decided to go with just JSON, and the code I'm personally writing (Ruby interface to the q-layer back-end) is JSON-only and will stay that way.
Ok so I don't expect to see a cron job talking XML just as I don't expect to see management software using plain text. Perhaps it's marginally more convenient for web developers to get a JSON serialisation rather than having to fire up parser objects too but these are very different use cases that can be trivially supported in a mechanical, cross-platform fashion by basing on XML. If they didn't exist then JSON may well be a more interesting option for us, but they do and we're happy to support them. Your opinion about choosing one format and sticking with it is understandable, but is not necessarily shared by everyone. The web does have content negotiation and it is used... if you can then why not? It's another potential differentiator and it need not impact interoperability if done properly.
Why JSON? Because all our data elements are numbers or short chunks of text, do not involve fancy formatting, and fit naturally into lists and dicts. Nesting is non-zero but also not deep (a data center has clusters, clusters have machines). JSON had everything we needed and nothing we didn't need. Plain text makes the nesting hard. XML is overkill.
Sure, for your requirements. We do need to be more flexible in order to deliver interoperability without stifling innovation as well as playing nice with others (like SNIA who will be joining us on next week's call and wantingto hear about potential integration points). Thanks for your continued interest/input - it's great to see such passionate discussion ongoing, and the vast majority of it is useful. I'm going to have to get some shut eye... Sam on iPhone Sam
participants (5)
-
Alexis Richardson -
Benjamin Black -
Mark Masterson -
Sam Johnston -
Tim Bray