- occi-wg - lists.ogf.org

OCCI MC - State Machine Diagram
by Sam Johnston 14 May '09

14 May '09

Afternoon all, I have created a diagram (attached) of what I think the absolute minimum core states need to be... essentially boiling them down to "STOPPED" and "ACTIVE" with "START" and "STOP" being the only requisite actuators. Transitional "STOPPING" and "STARTING" states are optional. I believe states should be completely unambiguous so I don't particularly like the DMTF model ("stopped" and "active" machines are also "defined" but this is a separate state) and that also rules out vague terminology like "inactive" (which could mean both "stopped" and "suspended"). I've got an optional RESTART actuator which takes you from "ACTIVE" to "STARTING" as well as an optional "SUSPEND" and "RESUME" cycle which takes you via the transitional "SUSPENDING" and "RESUMING" states between "ACTIVE" and "SUSPENDED". DMTF have another "paused' state but I wonder whether this needs to be a state in its own right or if it's an attribute of "suspended". That's a rhetorical question - we could spend all week discussing nuances but for now I want to make sure we agree on the absolutely minimalist core functionality. Other states can be added via a live registry which we can pre-populate with a view to guiding innovation without stifling it. This has been uploaded to the wiki: http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/StateModel Sam

17 89

(industry noise on CC protocols) cross post from a cloud group
by Alexis Richardson 14 May '09

14 May '09

I thought this might be of interest -- ---------- Forwarded message ---------- From: Gervas Douglas <gervas.douglas(a)gmail.com> Date: Thu, May 14, 2009 at 12:13 AM Subject: [cloudcomputing-tech] Brogden on protocols for CC services To: cloudcomputing-tech(a)yahoogroups.com <<Call it grid computing, cloud computing or software as a service, distributed computing is on a lot of developers minds recently. There are so many different ways in which people can use the "cloud" that there would appear to be no possibility of a single "framework" for cloud based services. This has not prevented a lot of developer discussion and fear that the "cloud" will somehow become a closed realm of proprietary protocols as opposed to an open-source utopia. (See for example the open cloud manifesto.) It seems to me that a discussion of the wide range of protocols and architecture in grid/cloud/distributed computing is in order. I am going to discuss some of them in order of increasing complexity. Bandwidth and storage with REST Little change in architecture is required when an enterprise uses cloud facillities to supplement owned server bandwidth to handle unusual demands. As I discussed in this article, making use of cloud bandwidth simply requires redirecting requests to a cloud URL. This follows the REST principle that when you want the current state of a resource, the internal details of how you get it are unimportant. No special software beyond a HTTP server is required on the cloud server side. The high bandwidth and storage capacities combined with low cost and simple application make this the simplest way to take advantage of the cloud. Remote Procedure Calls with XML-RPC and SOAP The concept of a Remote Procedure Call has been around for years and forms the basic technology for much network computing. The architecure gives the programmer an interface which lets you program as if the method will actually be executed on your computer but will actually run elsewhere. The technology known as XML-RPC was the pioneer for the idea of formatting Remote Procedure Calls in XML messages. This simple yet powerful concept for supplying variables to call a method on a remote computer is still in use. However, some developers wanted more, so we got SOAP. Originally SOAP stood for Simple Object Access Protocol but it certainly is not simple and has little requirement to use objects. The inescapable disadvantage of XML encoded messages is that they are very inefficient in terms of actual data transmission and CPU time required to parse the message. On the plus side, by using text XML documents, developers can paper over the incompatibilities between different hardware and software in distributed systems. Furthermore everything that is transmitted is human readable and it is easy to create or read messages with a variety of techniques. As the evolution of SOAP shows, it is easy to add capabilities to XML formatted messages without seriously impacting existing applications. Your SOAP or XML-RPC server on the cloud side will require custom programming but it will be a familiar environment, typically JAVA with an SQL server and huge huge disk capacity. Cloud vendors such as Amazon are rushing such capabilities into service. RPC between objects with CORBA The Common Object Requesting Broker Architecture or CORBA is an early effort at a standard to enable object interactions within a network. It uses an interface definition language (IDL) to define the face these objects present and an object request broker (ORB) to negotiate the exchange of information about objects. CORBA makes a major effort to ensure that the IDL can be supported in many languages and operating systems. The Internet Inter-Orb Protocol (IIOP) is the protocol by which ORBs communicate between heterogeneous mixes of computer systems using TCP/IP. It should not surprise you that object oriented languages like Java are easy to map to IDL. Support for IIOP has been part of the Java standard library since 1998. In spite of the fact that there have been many cycles of popular "latest thing" buzz-words since CORBA was introduced, it continues to be used in network computing with strong and mature industry support. The Object Management Group is the non-profit industry consortium responsible for maintaining the standard. Version 1.0 was released in 1991 and we are currently at version 3.1 released in 2004 so you can see that this is a refined and mature standard. You can get quite an argument among developers as to whether CORBA and/or SOAP is too complex for the future of cloud computing and SOA. Application specific protocols A recent news article indicated that Microsoft's "cloud services operating system" called Azure, will eventually allow the use of the "Tabular Data Streams" protocol in communicating with SQL Data Services, the cloud version of SQL. It appears that customers really wanted to enable cloud relational database operations similar to existing in-house applications rather than translating requests to REST style. I think it is likely we will see a lot more cloud services that use application specific protocols. Resources REST versus SOAP - the REST story REST versus SOAP - the SOAP story Home page for the "open cloud manifesto" Wikipedia history of Tabular Data Streams protocol>> You can find this at: http://searchsoa.techtarget.com/tip/0,289483,sid26_gci1355134_mem1,00.html?… Gervas __._,_.___ Messages in this topic (1) Reply (via web post) | Start a new topic Messages | Files | Photos | Links | Database | Polls | Members | Calendar Change settings via the Web (Yahoo! ID required) Change settings via email: Switch delivery to Daily Digest | Switch format to Traditional Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe Recent Activity 1 New Members Visit Your Group Give Back Yahoo! for Good Get inspired by a good cause. Y! Toolbar Get it Free! easy 1-click access to your groups. Yahoo! Groups Start a group in 3 easy steps. Connect with others. . __,_._,___

1 0

Networks: attributes and verbs
by Richard Davies 14 May '09

14 May '09

All, For me, the largest gap in the nouns, verbs and attributes is with regards to networks. Here are some thoughts on capabilities which I believe we need and options to implement these - feedback please! Basics ------ Networks will be a top level noun, along with servers and storage. Users will link a server to a network to specify that the server is attached to that network. There will be one special network presenting the public internet, and users can create additional private networks for themselves. The open questions are: Public static IPs ----------------- On all public clouds, customers can purchase static IPs (e.g. Amazon Elastic IP) for uses where a constant server location is helpful (e.g. typical web hosting). We'll need a means for customers to create, list and destroy the public static IPs which they own. Two options: a) Public static IPs are first-class nouns b) They are listed inside the public internet network I'd favour a), which is the case with most public clouds today. Active networks --------------- At its most basic, a network should behave like a plain ethernet switch - it provides no services at all, and simply connects all the servers which attach to it. Servers are free to chose their own IP addresses, etc. There are a number of active services which are possible on a network: - Central DHCP server - Bridge to a physical VLAN (e.g. containing physical dedicated servers in colocation with the cloud provider) - Load balancer between several web servers on a private network across to a single IP on the public internet. - etc. Two options: a) The network object itself can optionally provide these. They are configured using attributes and verbs on the network. b) There are separate 'appliance' objects which provide these services, and are linked onto the network just as a server would be (e.g. a 'DHCP server' appliance and or 'load balancer' appliance). a) feels lighter-weight, but I suspect b) is more powerful. As such the choice depends on how far OCCI wants to go down this route. Linking to a network -------------------- Finally, there are a few attributes which can specified on the link when a server is linked to a network: - The physical interface on the server (nic:0, nic:1, etc.) - Port firewalling rules (e.g. connect server to the internet network, but only allow port 80 inbound) - IP firewalling rules (e.g. connect server to the private network, and allow it to communicate on 192.168.0.0-23 but no others) - Local DHCP (e.g. the server operating system is internally configured to DHCP. I am connecting it to the public internet, and want it to appear with my assigned public static IP. Please send a DHCP response with that specific IP address). I think that all of these are best implemented as attributes on the link.

8 17

Content Negotiation: Alternative Formats
by Sam Johnston 13 May '09

13 May '09

Ok so let's try to get some clarity around the various alternative formats and use sensible names for them now on - we can do a lot better than "Sam's GData XML" vs "My Simple JSON". Here's a few that come to mind: - *AtomPub* <http://en.wikipedia.org/wiki/Atom_%28standard%29>: Atom meets REST or "*a simple HTTP-based protocol for creating and updating web resources*" - *CSV <http://en.wikipedia.org/wiki/Comma-separated_values>*: Comma Separated Variable ala GoGrid, useful for reporting, asset registries, etc. - *[X]HTML <http://en.wikipedia.org/wiki/HTML>*: Web interface for end users, optionally with semantic markup - *JSON* <http://en.wikipedia.org/wiki/JSON>: JavaScript Object Notation or "*a lightweight computer data interchange format*" - *PDF <http://en.wikipedia.org/wiki/PDF>*: Portable Documentation Format, useful for documentation and reporting - *POX* <http://en.wikipedia.org/wiki/Plain_Old_XML>: Plain Old XML - *TXT*: Plain Text API (e.g. "KEY VALUE", "X|Y|Z", etc.) I'm hoping we agree to one required core format and then leave it up to implementors to decide what they feel like rendering. Sam ---------- Forwarded message ---------- From: Richard Davies <richard.davies(a)elastichosts.com> Date: Wed, May 13, 2009 at 2:47 PM Subject: [occi-wg] Format: 4 distinct decisions To: occi-wg(a)ogf.org * Decisions 3 and 4 are independent. Sam's GData XML http://www.ogf.org/pipermail/occi-wg/2009-May/000381.html is 3b with 4a. My Simple JSON http://www.ogf.org/pipermail/occi-wg/2009-May/000523.html is 3a with 4b. But equally it would be possible to specify GData JSON (3b, 4b) or Simple XML (3a, 4a).

2 2

Format: 4 distinct decisions
by Richard Davies 13 May '09

13 May '09

All, I've just have a good phone conversation with Alexis, in which we agreed that the current formats discussion is actually addressing 4 independent questions simultaneously. I'd like to set these out individually, and get the group's opinion on each independent question individually rather than discussing them in combination as we have often done to date. Please can people follow up with their preference for each of decisions 1-4 independently. I would particularly appreciate the opinions of Randy and Tim as the other two cloud vendors involved in our mailing list, and of anyone else who may end up betting part of their business on OCCI. Cheers, Richard. =========================== Decision 1: Should OCCI specify wire format(s) or an abstract model: a) The OCCI API will be defined in terms of a specific rendering of the nouns, verbs and attributes to the actual bytes transferred on the wire. b) The OCCI API will be defined in terms of the abstract model (both nouns, verbs and attributes, and possibly also meta-model around these). Any implementation of this model is OCCI-compliant, regardless of the bytes on the wire. Perspective of myself and Alexis: OCCI should specify wire format(s), since this is the only way to guarantee in-practice interoperability between OCCI-compliant clouds. If OCCI will specify a wire format, then: Decision 2: Should OCCI-compliance require implementation of a single core wire format (any others are optional) or multiple core wire formats: a) Exactly one wire format is specified as core. A cloud must implement this alone to claim OCCI-compliance. Additional wire formats may be specified, but these are optional alternative renderings for easier integration with various external systems and cannot be relied upon for interoperability. b) Multiple wire formats as specified as core. A cloud must implement all of these to claim OCCI-compliance. Perspective of myself and Alexis: OCCI should have a single core wire format, since this is sufficient to ensure interoperability between OCCI-compliant clouds and minimizes the burden of OCCI-compliance. Additional alternative wire formats may be specified where these will aid integration with different types of system, but these will not be required to claim OCCI-compliance. These alternative wire formats may take different views on decision 3 and 4 from the core (e.g GData XML, GData JSON, Plain JSON and Plain XML can all coexist as alternative wire formats carrying the same nouns, verbs and attributes). Decision 3: For the core wire format(s), what meta-model should be adopted in rendering the nouns, verbs and attributes (i.e. a framework/style within which the nouns, verbs and attributes are rendered and any framework services are added): a) Minimal/no meta-model: Nouns, verbs and attributes are rendered as directly as possible over HTTP in a very basic REST style. b) GData meta-model: Nouns, verbs and attributes are rendered as GData objects and carried according to GData conventions with GData services. c) Other? Important*: This is an independent decision from decision 4 on syntax Perspective of myself and Alexis: For the core wire format, OCCI should minimize the meta model so that the specification of the core wire format is as small as possible, and hence interoperability between OCCI-compliant clouds is easiest to demonstrate, test and debug. Decision 4: For the core wire format(s), is XML or JSON the better syntax ('angle brackets vs. curly brackets') a) XML b) JSON Important*: This is an independent decision from decision 3 on meta-model Perspective of myself and Alexis: No strong view. We lean towards JSON, since it feels easier to make a tight specification and hence to easier to guarantee interoperability. Tightly specified use of XML might be equally good. * Decisions 3 and 4 are independent. Sam's GData XML http://www.ogf.org/pipermail/occi-wg/2009-May/000381.html is 3b with 4a. My Simple JSON http://www.ogf.org/pipermail/occi-wg/2009-May/000523.html is 3a with 4b. But equally it would be possible to specify GData JSON (3b, 4b) or Simple XML (3a, 4a).

5 13

Process: Format decisions made and to come
by Richard Davies 13 May '09

13 May '09

Thijs, Ignacio, Alexis and Sam, As the four coordinators of the OCCI working group, I would like to ask you to clarify the process going forward for our format discussions. It has been a good if controversial discussion, with many decent points raised on all sides. On the other hand, it continues to take up a lot of the bandwidth of this list. As the coordinators, I would like you to jointly state which decisions you consider to have been made (which we should therefore cease to generate noise debating) and which decisions are still open (and hence worth spending time contributing to). For the open decisions, I'd ideally like a rough idea of how and by when we intend to resolve them, so that it is clear when the list will move on to focus primarily on other issues. Thanks in advance, Richard.

2 1

Microsoft: "Atom Publishing Protocol (AtomPub) as the future direction"
by Sam Johnston 13 May '09

13 May '09

Afternoon all, I'm surprised this slipped by before but while re-reading Joe Gregorio's recent controversial rant <http://bitworking.org/news/425/atompub-is-a-failure> on AtomPub I spotted this comment<http://bitworking.org/news/425/atompub-is-a-failure#X2>which explains: > Microsoft is migrating all of its Live Services APIs<http://msdn.microsoft.com/en-us/library/bb264574.aspx>into a single unified AtomPub API in the Live > Framework <http://dev.live.com/liveframework/>. It appears they will also > open up the ability for third parties to do the same. > > I don't think the JSON argument is such a big deal. Google Data APIs > support output in JSON and RSS formats. Microsoft's Live Framework supports > full CRUD with JSON, AtomPub, and POX, as well as output in RSS. Under the > hood, they are simply offering a JSON encoding of AtomPub. > Interesting choice of "POX" as in "Plain Old XML", thus avoiding the XML-RPC/SOAP stigma... but we digress... this post<http://dev.live.com/blogs/devlive/archive/2008/02/27/213.aspx>from a Microsoft VP leaves little to the imagination: > Atom Publishing Protocol (AtomPub) as the future direction > > Microsoft is making a large investment in unifying our developer platform > protocols for services on the open, standards-based Atom format (RFC 4287<http://www.ietf.org/rfc/rfc4287.txt>) > and the Atom Publishing Protocol (RFC 5023<http://www.ietf.org/rfc/rfc5023.txt> > ). At MIX we are enabling several new Live services with AtomPub endpoints > which enable any HTTP-aware application to easily consume Atom feeds of > photos and for unstructured application storage (see below for more > details). Or you can use any Atom-aware public tools or libraries, such as .NET > WCF Syndication <http://msdn2.microsoft.com/en-us/library/bb412202.aspx>to read or write these cloud service-based feeds.<snip> > > The intent for these early, experimental releases are to gather valuable > feedback from the community around our idiomatic and freely licensed > extensions to AtomPub which deal with important service scenarios, such as > URL formats, nested directories, image streams, and service metadata. You > can read more about this on the Project Astoria team blog<http://blogs.msdn.com/astoriateam/archive/2008/02/13/atompub-support-in-the…> > . > The developer docs <http://msdn.microsoft.com/en-us/library/dd137315.aspx>further explain: > AtomPub was chosen as the base-level interaction protocol for CRUD > operations on Live Framework resources. Using a > community-supported interpretation of Representational State Transfer (REST) > and its interaction model for exchanging feed-based representations enables > a variety of basic AtomPub clients, applications, and tools to interact with > the Live Framework. > It's worth noting that in a "big switch in direction" and "huge endorsement for AtomPub " "Microsoft dropp[ed] Web3S for AtomPub for Windows Live! service APIs<http://www.xlml.com/aehso/2008/02/29/microsoft-dropping-web3s-for-atompub-f…>". This is interesting because it was a move from away from a "simple XML" type rendering (example <http://dev.live.com/livedata/web3s.htm#_Toc165288985>) previously proposed here for OCCI towards AtomPub with a light, standardised structure. Apparently Salesforce are getting in on the game too, at least in so far as their integrations with Google<http://wiki.developerforce.com/index.php/Google_Data_API_Toolkit>go. According to Bill de hÓra<http://www.dehora.net/journal/2008/06/24/salesforcecom-and-google-integrati…> : > One upside is that adopting Atom protocol<http://wiki.apexdevnet.com/index.php/Google_Data_API_Toolkit>and the associated HTTP-like-it-oughta-be approach a should help > salesforce.com stop creating multiple incompatible APIs on top of the many > (~18?) they have already (apparently these APIs account for 45% of sf.com's > transactions). Or at least slow that kind of unneccessary churn down. > Salesforce's API has been around for so long that it's still SOAPy but I would not be surprised to see this change at some point... an open standard that delivers the extensions that Google and Microsoft felt necessary to add could well be all it takes to get them on board too (while they don't offer infrastructure, they do offer platform services and that's the logical progression for OCCI). Sam

5 10

OCCI concall (today!) - Wednesday 13th, 2009 @ 5pm CET
by Thijs Metsch 13 May '09

13 May '09

Hi folks, Time again for our weekly concall. Sorry for the late invitation, I'm quite busy and hardly have time to catch up on emails. Our Agenda is set to: 1. Discussion with SNIA 2. Preparation for OGF 26 3. Timeline discussion 4. API discussions 4a) formats 4b) nouns, verbs, attributes Especially for the first point I would love to see some nice discussions! Feel free to add your points to the agenda in the Wiki add: http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/Concall20090… For all the dial-in options and more details about timezones and so on please visit: http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/Teleconferen… All the best, -Thijs -- Thijs Metsch Tel: +49 (0)941 3075-122 (x60122) http://blogs.sun.com/intheclouds Software Engineer Grid Computing Sun Microsystems GmbH Dr.-Leo-Ritter-Str. 7 mailto:thijs.metsch@sun.com D-93049 Regensburg http://www.sun.com

2 1

Re: [occi-wg] OCCI MC - State Machine Diagram
by Andre Merzky 13 May '09

13 May '09

Sorry for the *very* late reply... Quoting [Michael Richardson] (Apr 20 2009): > > >>>>> "Andre" == Andre Merzky <andre(a)merzky.net> writes: > Andre> For example, assume that I have a client which implements the > Andre> core specification only, thus only knows the STOPPED, ACTIVE > Andre> and SUSPENDED states (your original figure). What is that > Andre> client supposed to do if the backend reports an PAUSED state? > > How did it get to that state? > > Since the client didn't put it into that state, it means that an > operator or error must have forced this, so it's an error. Life does > not always go as planned. It is dangerous to assume that the client is the only legimitate entity which operates on the machine state diagram. The system admin, internal events, other clients, etc, may all have reason to trigger state changes, and this would not represent errors. > If the client does not do PAUSED state, the system shouldn't get into > that state. see above. > The client, upon seeing a state that it does not understand could > attempt to recover by requesting a transition to primary state that it > does understand. If the client does not know the state, it cannot determine what a valid state transition would be. > Andre> BTW, I agree with Krishna's point that ENTRY and EXIT points > Andre> are useful. > > Where are they attached? ENTRY point is attached to all states which can be reached initially, i.e. on creation. EXIT point is attached to all states which cannot be left, i.e. no transition exists to leave that state. Best, Andre. -- Nothing is ever easy.

2 1

thought on interoperability vs integration
by Alexis Richardson 13 May '09

13 May '09

Hi all, Thanks for a thought-provoking week of emails on the OCCI-WG list. Especially thanks to Sam, Richard, Ben and Tim for laying out a lot of the issues. One link that I found useful was this: http://www.tbray.org/ongoing/When/200x/2009/03/16/Sun-Cloud where we find the following statement: --- if Cloud technology is going to take off, there’ll have to be a competitive ecosystem; so that when you bet on a service provider, if the relationship doesn’t work out there’s a way to take your business to another provider with relatively little operational pain. Put another way: no lock-in. ... I got all excited about this back in January at that Cloud Interop session. Anant Jhingran, an IBM VIP, spoke up and said “Customers don’t want interoperability, they want integration.” ... “Bzzzzzzzzzt! Wrong!” I thought. But then I realized he was only half wrong; anyone going down this road needs integration and interoperability. --- What we have been discussing is "for customers". So it is about both of these things: * Interoperability ("interop") * Integration This made me realise that our OCCI discussions have correctly been about both issues. But, incorrectly, we have been commingling the two. For example a person will say "you need X for interop" and the reply will be "but you need Y" when in fact Y is for integration. And vice versa. This is a problem for us because it leads to confusion. But it's a symptom of a larger problem which is that interop and integration have opposite requirements. * Interop is about reducing the entropy in a general purpose system down to some level of behavioural invariance. This minimises the cost of meeting the defined behaviour correctly, and minimises the risk of different behaviours breaking interop. This is in turn about having a defined behaviour (in some format) and minimising its scope. * Integration is about minimising the frictions between (a) the general purpose system and (b) any specific purpose external system that may be coupled to it. It is also, often, about maximising the number of specific systems (or "features") that may be connected. Since we don't know ex ante what those are, this tends to maximise scope (eg "feature creep"). Too much specificity is the same as complexity. Because interop requires minimal scope, and integration pushes for maximal scope, they are in tension. They are BOTH about simplicity. Simplicity cannot be invoked on its own, as a reason for preferring interop over integration: * Interop is about simplicity of definition * Integration is about simplicity of end use BUT * Interop simplicity is ex ante * Integration simplicity is ex post We cannot predict all ex post issues, but we can try and make simple definitions ex ante. I argue below this means we have to have ONE definition. So let's look at interop first: It's really important that interop be based on defined behaviour. Or it cannot be verified. Lack of verifiability is almost always a symptom that something is opaque, complex and ambiguous, which will break interop in subtle ways that cannot be seen in advance (due to the opacity). This later leads to brittleness when interop and integration are attempted in practice; and that leads to expensive patching at all levels, ... which is one thing that WS-* exhibits. NOTE 1 on WS-* ---- IMO this was not accidental, and due to focussing on solving integration before solving interop. IIRC, the first WS-* was an Interop committee to fix minor vendor mismatches and versioning issues in prior WS protocols such as WSDL and SOAP. The formalisms of WSDL and SOAP were not precise enough to spot these issues upfront, so they were left until it was too late. Now let's look at the implications of having a definition of behaviour: You cannot define a system using multiple definitions. You have to have one definition, preferably in a formalism that admits of automatic conformance testing. In the case of data formats, this leads us to three possible choices: 1) We remove data formats from the interop profile. They are not part of interop. Data interop is excluded. Data payloads are opaque blobs. OR 2) We have one data format that can be defined unambiguously. OR 3) We have multiple formats and derive them from a single unambiguous definition using canonical, verifiable, automatable, mappings. This definition can either be in a new format which we invent (option 3a) or it can be in an existing format (option 3b). I am going to rule out option (3a) on grounds of time and due to the prevalence of existing candidates. Also, I think choice (1) is a complete cop-out -- I don't see how we can claim 'useful' interop without data interop. This leaves options (2) and (3b). BOTH of these options require one format. By occam's razor, option 2 is preferable on grounds of simplicity - option (3b) does not add anything except one mapping for every extra format that is based on the core definition. Such complexity MUST be deferred as long as possible, and MAY be pushed into integration use cases. The latter is preferable. Complexity MAY be added later if necessary and then, reluctantly. As a general observation if you have complexity (entropy) in a system, it is very hard to take out. It may permeate the system at all levels. Removal of the complexity is known as 'complete refactoring'. This is a bad outcome. Even worse is when refactoring cannot be done for commercial reasons, eg because the systems are in production. Then the complexity must be encapsulated and hidden. Attempts to wrap complex behaviours in simpler packaging usually fail. NOTE 2 on WS-* ---- This tried to do too much and ended up trying to wrap the complexity in simple packaging. This usually fails to work as we have seen. Another general observation: * Integration requires interop, ie. the use of a common general interop model; otherwise it is piecemeal and pointless. Example - the old 'integration brokers' that had N*N connections to maintain, and which got replaced by N connections to a common system. * But you can have interop without integration - it just means 'you have a smaller audience'. This is fine because you can always grow your audience by integrating more case with the interoperating core. It is easier to do that when the interoperating core is programmatically simple (as in low entropy, small code blocks, easy to test conformance to the definition). I would like to add some observations from the world of AMQP... AMQP-1) It is a protocol with one simple common data format - it gets that right. We leave it to integration products and services to support data formats at the edge (eg "ESBs"). OCCI should not be like an ESB - that is for products and services in the integration business. AMQP-2) That AMQP data format is not XML - see below for more thoughts on that. XML can be carried as an attached payload (just as it can be in the JSON case btw). AMQP-3) The 0-8 and 0-9 specs took 18 months of production use to show up the many very tiny interop bugs. We used those to create 0-9-1 which does have interop (we have tested this) and is only 40 pages long. This would not have been possible with a complex spec. It would not have been possible with multiple data formats. AMQP-4) The 0-10 spec was focussed on supporting a lot of use cases eg "integrate with multiple transport formats including SCTP and TCP and UDP" and adding lots of features at all levels (eg JMS, support for hardware, ..). That spec is really complicated and nearly 300 pages long. Some great new ideas are in it, but it's long and in my own opinion not supportive of two interoperating implementations. AMQP-5) All these painful lessons have taken the AMQP working group to a much happier place with AMQP 1.0 which tries to simplify everything by taking stuff out that is not needed for interop, plus refactoring (see above my comments on how removing entropy is hard) and clean-up. All of the above has taken time because we did not learn from WS-*. We did too much too fast and confused interop with integration. We are back on track now. Now to the issue of data formats. I have already argued that FOR INTEROP, there must be one definition. I argued that the best way to do this is via a suitable single format. We can support as many recommended ways as we like FOR INTEGRATION ... and they can be evolved over time. Here is my 2c on XML. XML-1) XML lets you do too much - because of namespaces and xsd it is in effect multiple formats. This is bad - we want a single, testable, constrained definition for data interop. XML-2) To enforce compliance with a simple XML definition, you need to have an extra definition of a well formed 'small' document. But creating a new definition of a data format in XML is equivalent to defining a new data format, the same as 'option 3a' above. But that option was ruled out above, on grounds of time constraint.. provided that a suitable alternative exists already (see JSON claims below). NOTE 3 on WS-* ---- IMHO a third reason why WS-* failed to be simple enough to get happy, quick and wide adoption, is that (XML-1) issue left too much data integration in the hands of customers, because vendors could not produce useful products when XML could be all things to all people. By not delivering on data integration, it became hard to deliver on the promise of integration at all. And recall that lowering integration costs was the selling point... So let's get INTEROP right and then do INTEGRATION. Interop requires: * Model - yes * Data format - one definition * Metadata - ? tbd As an aside - I think that GData is the nuts but it is also really an *integration techonology*. Now, here are some claims about JSON: JSON-1) Sun have demonstrated that it is plausible as a data model for a cloud API. That makes it plausible for us to ask: can it be used as the core definition for interop? JSON-2) It is lower entropy than XML. This makes it easy to test conformance. JSON-3) This means we do things the right way round -- simpler 'ex ante' choices make it EASIER for us to extend and enhance the protocol on a by need basis. For example carrying OVF payloads, or other integration points. Many will be done by users. So my recommendation is that the best way to avoid WS-* outcomes is A) Use one format for interop. Do interop first. Integration later. B) *At least for now* and *during the working draft* stage, to use JSON C) Other formats, for now, are "integration". But we do interop first. OK.... I have to run off. I wrote this down in one go and don't have time to review it. Apologies for any mistakes in the presentation. What do you all think? alexis

7 18