- occi-wg - lists.ogf.org

Test suite for OCCI
by Sam Johnston 26 Oct '09

26 Oct '09

Morning all, Last for tonight I promise... I (well, and a bunch of people much smarter than myself) figure the best way to make sure that implementations are interoperable is to develop a comprehensive test suite from the specification. Accordingly I've been looking at the various options and having written the earlier reference implementation in Python (for Google App Engine) I figure PyUnit is as good a candidate as ever. It helps that virtually all systems have python either preinstalled or at least available for download nowadays and that there's a Google App Engine harness for it ( gaeunit.py <http://code.google.com/p/gaeunit/>). This should allow anyone (including end users) to point it at their implementations and test compliance without having to download anything. I say "should" because I'm not sure if GAE supports custom HTTP methods yet (e.g. PUT, OPTIONS and more obscure ones like COPY and MOVE), but I'll soon find out. Before I spend too much more time looking into this I wanted to give you all an opportunity to comment, particularly if you have better ideas? Do any of you think this (and test suites in general) are a particularly good or bad idea? Sam

4 5

OCCI JavaScript libarary
by Sam Johnston 26 Oct '09

26 Oct '09

Ok so there was one more thing (and this is definitely the last one as it's late already and we gained an hour today)... Mike Kelly <http://mike.mykanjo.co.uk/> was kind enough to whip up an example Javascript <http://jsbin.com/eyobo> proving the concept of a web-based OCCI client and I've taken him up on his offer to convert it to an includable script (e.g. occi.js). It would be great to see this extended such that the user (web) interface can be interactive, asynchronous, and take advantage of the many Web 2.0 libraries available today as there's no reason for web consoles to remain proprietary. Fortunately all the methods we need <http://msdn.microsoft.com/en-us/library/ms536648(VS.85).aspx> are supported along with the ability to get/set arbitrary headers, though it probably wouldn't hurt to also support [something like] X-HTTP-Method-Override<http://code.google.com/apis/gdata/docs/2.0/basics.html#UpdatingEntry> as a fallback for paranoid networks. I anticipate that there will be two primary interfaces - a machine interface (native representations) and an optional user interface (XHTML5, doubling as XML for those who prefer it). The Javascript library would allow the [otherwise static] user interface to be "animated" through asynchronous connection back to the machine interface. This [should have] allowed us to sidestep the formats debate, keep implementations simple, and still provide for arbitrarily rich (yet standardised) web interfaces. Everybody wins (or so the theory goes). Sam

1 0

Moving Forward
by Andre Merzky 22 Oct '09

22 Oct '09

Hi all, Proposal: Stop discussing the attribut topic (*), *at* *all*, and postpone a resolution to either a phone call, or the face to face meeting. From what I have seen, the technical disagreements are minor - the concerns about applicability of the technology are major. That is nothing which you will be able to resolve by mail as it seems, so, rather then getting bogged down on that topic, try to move on. My understanding of OCCI may be limited, but I think that none of the other open tickets and discussion items[1] would be affected by postponing the attribute-in-header issue. I am confident that you *will* find a compromise, one way or the other, and, from what I understand, either solution will work toward an implementable standard, and either solution will be considered as sub-optimal by some. Thats life, and, IMHO, acceptable. Best, Andre (wearing my AD hat). (*) IIUC, this is what the last 124 mails have been all about [1] http://code.google.com/p/occi/issues/list -- Nothing is ever easy.

1 0

A few important clarifications...
by Sam Johnston 22 Oct '09

22 Oct '09

Morning all, I wanted to write to you quickly to clarify a few random but important points following my having been accused of "fraud and negligence", "misrepresentation", "anarchy" and various other unsavoury things in (mostly) off-list threads. - I have no agenda beyond making open cloud a reality. Before you go accusing me of anything take a second to remind yourself that I represent enterprise end users, don't have a "horse in this race" and ultimately just want to deliver the highest quality API possible, as soon as possible so we have something to "sell". My only vested interest is that I've now spent well over €100,000 of my own time into OCCI and want to see it a success. - I'll do whatever it takes to make it happen, but I don't plan to die trying<http://economictimes.indiatimes.com/infotech/software/SAP-India-MD-Ranjan-D…>. Recently I've been working through 03h38 every day (when the Internet at the company apartment goes down for a multiple-hour reboot) and am at it again before 7. What that means is that you can expect me to be somewhat more zen going forward - if you troll I'll treat you like a troll and likely encourage others to do the same. Same applies if you present arguments that cannot be proven or disproven, dodge the point, or otherwise fail to take us forward. - Nothing has been "publicly represented as unilaterally adopted by this group" and likely nothing will be for some time to come. Whenever I talk about our work in public I use terms like "the current thinking" or "my preferred approach". You should do the same until *at least* such time as we have a complete draft which has been accepted by the group and submitted to the OGF pipeline for public comment. - While "informally consensus is considered to be the absence of sustained intense reasonable resistance", there can be no consensus if there is no proposal. We're working slightly differently from the IETF, where contributors work alone or in small groups to develop Internet-Drafts which are then refined by committee, but we need to be able to work efficiently and quickly and don't have the resources (e.g. time) to discuss every point in detail. - We don't vote. We're not a membership organisation and anyone can join at any time - as such it is impossible to hold "fair" ballots like our more traditional counterparts such as DMTF (where a $12k membership gets you a vote that is very valuable in terms of influencing the outcome). - We're open and transparent so unless you have a damn good reason for taking/keeping your topic off-list then please resist the urge to do so. If you do have something (e.g. your identity) that you want to keep secret then I'll respect your privacy but don't be surprised if I ask you if it really needs to stay off-list (yes, I'll ask first). - Instead we seek consensus which means we thrash out issues on the list with a view to finding the best way forward. Try to keep the discussions focused and provided this is done properly we won't have to revisit decisions. The chairs act as umpires, calling the consensus where necessary, and there is an escalation path through the OGF channels (which I hope we won't have to use). - I'm busy pulling information and ideas from many sources to create a sufficiently complete proposal as to allow us to *start* building consensus. If there is some aspect you don't like then the single best way to be heard is to prepare a "camera ready" alternative *yourself* and explain specifically and in detail why it's superior. That is to say, if you want something done and know how to do it then you have both the motive and means to do it yourself. I don't have to like your idea for it to achieve consensus but I'm not a "work for hire" resource for realising ideas I don't agree with. As always I am available to discuss whatever's concerning you; OCCI, the process, getting involved, accessing the repository, the weather... both on- and off-list, by phone/skype/chat, as you want, generally between 7am and midnight CEST. Anyway I've got to go get me one of these Magic Mouses as my trackpad just died a day out of warranty, but after that let's get back to work. Cheers, Sam

1 0

Addressing the Challenge of Cloud-Computing Interoperability
by Sam Johnston 21 Oct '09

21 Oct '09

Here's a reasonably good summary of where things are at with cloud interop. Sam *Addressing the Challenge of Cloud-Computing Interoperability*<http://www.computer.org/portal/web/computingnow/archive/news031> *by George Lawton* Cloud computing—in which users work online with resources based on providers' servers, rather than on their own computers—is becoming increasingly popular. However, cloud computing is still relatively new, so proponents and vendors have not yet developed interoperability between platforms. Some industry observers say this could hurt the technology's growth. For example, companies can't necessarily move their data and applications if they find another cloud platform they like better than the one they are using. Also, some enterprises want a best-of-breed option, employing different cloud platforms for different applications. In some cases, they want to use data and applications across the platforms. "What folks are mostly interested in is that the applications themselves are not locked into a particular platform," said Sam Charington, vice-president of product management of cloud-computing vendor Appistry. In addition, they want to utilize their security, management, identity, and other tools across the different cloud platforms they utilize. Many businesses want interoperability between their in-house infrastructure and the cloud, noted Forrester Research market analyst James Staten. They might want to use an in-house application to process data in the cloud, they might want to use a cloud-based application to process in-house data, or they might want to use applications or tools that will run both in-house and on the cloud. Other key issues include having a single sign-on for users who access multiple cloud platforms, deploying and provisioning resources from the platforms with a single management tool, enabling a service hosted on one platform to automatically call a service hosted by another, and having a private cloud application seamlessly obtain resources from a public cloud when necessary. In response to this demand, several standards groups and industry consortia are developing specifications and best practices to enable cloud interoperability. Likewise, several vendors have released products that allow some degree of interoperability. Key Issues Cloud applications work via virtualization. They run in a provider's server on a virtual machine (VM). A single server contains multiple VMs, enabling cloud providers to efficiently offer applications to numerous customers. In a virtualization system, the hypervisor allocates the host machine's resources to each virtualized operating system or to each program running on a virtualized OS. Many cloud platforms don’t interoperate because they use distinct hypervisor and VM technologies, and they store and configure operating systems and applications differently. The platforms also use various security standards and management interfaces. Amazon has its Amazon Elastic Compute Cloud, Citrix works with the XenServer, Linux supports the Kernel-Based Virtual Machine (KVM), Microsoft uses the Hyper-V, and VMware utilizes the ESX and ESXi hypervisors. Each hypervisor supports a different VM format, and the formats are not natively interoperable. A problem today is determining exactly where cloud-interoperability standards are needed. To help address this, the Distributed Management Task Force (DMTF), an industry consortium that develops and promotes systems-management-related standards, has formed the Open Cloud Standards Incubator (www.dmtf.org/about/cloud-incubator) The Open Group's Cloud Work Group (www.opengroup.org/cloudcomputing) is conducting meetings among members to create a document describing business-related requirements and concerns for cloud computing, including interoperability. In the process, they are looking at best practices needed to address the concerns. Workload Movement Workload movement is the ability of an organization to automatically move data, applications, and server configurations from its own systems to a public cloud. To enable workload movement, the DMTF has created the Open Virtualization Format (OVF) specification. The OVF standard provides an intermediary format for VM images. It lets an organization create a VM instance on top of one hypervisor and then export it to the OVF so that it can be run by another hypervisor. OVF supporters include Citrix Systems, IBM, Microsoft, Oracle and VMware, but not Amazon, which is a major cloud-computing provider. Companies such as Appistry and 3Tera are exploring ways of using middleware to enable enterprise applications that will work on multiple cloud platforms. The Open Cloud Consortium (www.opencloudconsortium.org) is working on a standard that, like OVF, would create an intermediary format that would make it easier to migrate distributed data and applications—typically used to process large amounts of data—across cloud platforms. According to OCC chair Robert Grossman, the consortium will release its first cloud-interoperability standards in six to 12 months. Security and Identity Companies want to apply their existing security and user-identity-management tools to applications running on different cloud platforms. Security and identity management are critical to organizations and are a particular concern when data and applications reside on the cloud, noted Dave Lounsbury, vice president of collaboration services at the Open Group, an industry consortium that develops enterprise-oriented information-interoperability standards. Cloud-computing adoption will be limited until these issues are addressed, he said. The Open Group's Jericho Forum<http://www.opengroup.org/jericho/cloud_cube_model_v1.0.pdf>is studying security- and identity-related cloud-computing issues. In part, they are looking at ways to use security- and identity-related applications across cloud platforms. The Cloud Security Alliance, another industry consortium, is working on recommendations for best practices regarding cloud-computing security, including interoperability-related issues<http://www.cloudsecurityalliance.org/guidance> . Interoperability between the Cloud and the Enterprise Organizations need to automatically provision services, manage VM instances, and work with both cloud-based and enterprise-based applications using a single tool set that can function across existing programs and multiple cloud providers. Efforts are under way to solve this problem. For example, the Open Grid Forum (www.ogf.org) an industry group, is working on the Open Cloud Computing Interface, which would provide an API for managing different cloud platforms. In a similar move, ServePath, a hosting-services provider, has released its GoGrid API. The API should be easy to adopt because it's based on existing standards, not proprietary technology, said ServePath technology evangelist Michael Sheehan. Several vendors—including Appistry, AppZero, and 3Tera—have created suites of development and deployment platforms that make it easy to write a program once and deploy it on one of many cloud environments. In essence, these suites provide a layer of abstraction between the programmer and the cloud platforms. Developers create applications for this intermediate layer, which then supports and manages multiple hypervisors or external cloud platforms. These products would let applications run either internally or in the cloud. Cloudy Future The US government recently began looking for proposals to implement cloud-computing interoperability on its systems. "Usually we think of the federal government as being 10 years behind," said DMTF president Winston Bumpus, "but in this case, we are looking at the government as a leader." A major wild card in the cloud-interoperability effort is Amazon, whose strong market position has enabled it to work with its own technology and not participate in standards efforts. However, Sheehan predicted that Amazon eventually will have to support interoperability to attract more corporate business. At this early stage in cloud computing's evolution, Forrester's Staten said, premature standardization could stifle innovation and the technology's development. And in today’s relatively new cloud-computing marketplace, he added, there’s little incentive for vendors to cooperate with one another. Bumpus, on the other hand, said there are good reasons for interoperability, which is why numerous organizations are beginning to work on it. ------------------------------ George Lawton is a freelance technology writer based in Monte Rio, California. Contact him at glawton(a)glawton.com..

1 0

Supporting standard formats
by Sam Johnston 21 Oct '09

21 Oct '09

Afternoon all, I've been looking at the various formats available for our representations and have added the following ones to OCCI Infrastructure<http://occi.googlecode.com/hg/docs/occi-infrastructure.html>. Now before you savage me for not including your favourite format or including one you don't like, bear in mind that this is just to give people *suggestions* as to formats which may be suitable, not *recommendations* as to which ones we prefer - that's for the market to decide. The exact wording is "The following representation formats are identified (but not necessarily recommended) as possible candidates for this kind". Sam *Table 4. Compute Representations* NameTypeSpecificationCitrix Xen Virtual Appliance (XVA) [deprecated] application/octet-streamVHD<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…>Open Virtualisation Appliance (OVA)application/ova+xmlOVF<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…>Open Virtualisation Format (OVF)application/ovf+xmlOVF<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…> *Table 6. Network Representations* NameTypeSpecificationN/AN/AN/A *Table 8. Storage Representations* NameTypeSpecificationISO image formatapplication/x-iso9660-imageISO<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…>Microsoft Virtual Hard Diskapplication/octet-streamVHD<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…>QEMU qcow2 image formatapplication/x-qemu-qcow2QCOW2<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…>Raw disk format (IMA, IMG, RAW, etc.)application/octet-streamRAW<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…>Virtualbox Virtual Disk Image [undocumented]application/octet-streamVDI<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…>VMware Virtual Disk Formatapplication/octet-streamVMDK<http://occi.googlecode.com/hg/docs/occi-infrastructure.html#infrastructure-…>

2 1

still can't get to concall
by Gary Mazz 21 Oct '09

21 Oct '09

Hi, have to rewrite a proposal ... gary

1 0

can't make concall today...
by Thijs Metsch 21 Oct '09

21 Oct '09

Hi, Sorry I can't make the call today. If needed Andy can open the line... Cheers, -Thijs -- Thijs Metsch Tel: +49 (0)941 3075-122 (x60122) http://blogs.sun.com/intheclouds http://www.twitter.com/befreax Software Engineer Cloud, Grid and Virtualization Sun Microsystems GmbH Dr.-Leo-Ritter-Str. 7 mailto:thijs.metsch@sun.com D-93049 Regensburg http://www.sun.com

2 1

William Vambenepe: Missing out on the OCCI fun
by Sam Johnston 21 Oct '09

21 Oct '09

Thankyou to William Vambenepe, Tim Bray and other seasoned standards boffins for their advice and contributions from the sidelines. I understand and appreciate their preference to work at a "meta level" and value whatever feedback they care to provide, including this insightful post that's hot off the press: Missing out on the OCCI fun As a recovering “design by committee” offender I have to be careful when lurking near standards groups mailing list, for fear my instincts may take over and I might join the fray. But tonight a few tweets containing alluring words like “header” and “metadata”<http://twitter.com/samj/status/5037202759>got the better of me and sent me plowing through a long and heated discussion thread<http://www.ogf.org/pipermail/occi-wg/2009-October/thread.html#1315>in the OGF OCCI mailing list archive. I found the discussion fascinating, both from a technical perspective and a theatrical perspective. Technically, the discussion is about whether to use HTTP headers to carry “metadata” (by which I think they mean everything that’s not part of the business payload, e.g. an OVF document or other domain-specific payload). I don’t have enough context on the specific proposal to care to express my opinion on its merits, but what I find very interesting is that this shines another light on the age-old issue of how to carry non-payload info when designing a protocol. Whatever you call these data fields, you have to specify (by decreasing order of architectural importance): - How you deal with unknown fields: mustUnderstand or mustIgnore semantics. - How you keep them apart (prevent two people defining fields by the same name, telling different versions apart). - How you parse their content (and are they all parsed in the same manner or is it specific to each field). - Where they go. SOAP provides one set of answers. - They go at the top of the XML doc, in a section called the SOAP header. - They are XML-formatted. - They are namespace-qualified. - You can tag each one with a mustUnderstand attribute to force any consumer who doesn’t understand them to fault. You may agree or not with the approach SOAP took, but it’s important to realize that at its core SOAP is just this: the answer (in the form of the SOAP processing model) to these simple questions (here is more about the SOAP processing model and the abuses it has suffered<http://stage.vambenepe.com/archives/118>if you’re interested). WSDL is something else. The WS-* stack is also something else. It’s probably too late to rescue SOAP from these associations, but I wanted to point this out for the record. Whatever you answer to the four “non-payload data fields” questions above, there are many practical concerns that you have to consider when validating your proposal. They may not all be relevant to your use case, but then explicitly decide that they are not. They are things like: - Performance - Ability to process in a stream-based system - Ease of development (tool support, runtime accessibility…) - Ease of debugging - Field length limitations - Security - Ability to structure the data in the fields - Ability to use different transports (way overplayed in SOAP, but not totally irrelevant either) - Ability to survive intermediaries / proxies Now leaving the technology aside, this OCCI email thread is also interesting from a human and organizational perspective. Another take on the good old Commedia dell standarte <http://stage.vambenepe.com/archives/96>. Again, I don’t have enough context in the history of this specific group to have an opinion about the dynamics. I’ll just say that things are a bit more “free-flowing” than when people like my friend Dave Snelling were in charge in OGF. In any case, it’s great that the debate is taking place in public. If it had been a closed discussion they probably would not have benefited from Tim Bray dropping in to share his experience. On the plus side, they would have avoided my pontifications…

1 1

Some (thankyou) thoughts from OGF27
by Thijs Metsch 21 Oct '09

21 Oct '09

Hi, as said OGF27 was great. The session turned out to be visited pretty good (another Cloud session was in parallel - that doesn't help much) But we had around 65-70 people aboard. Attached are some photos from the sessions. Also from the closing remarks done by Craig who had OCCI also on his presentation. OCCI also appeared on many other slides of people who I didn't know :-) So that is good for us! If you wanna know who is on the photos feel free to ask. Among them is Craig Lee (while talking in the OCCI sessions - thumbs up :-) and of cause the attendees in the room! You can find the slides of all presentations here: http://ogf.org/gf/event_schedule/index.php?id=1742 Thanks again to all of you for making this happen, -Thijs -- Thijs Metsch Tel: +49 (0)941 3075-122 (x60122) http://blogs.sun.com/intheclouds http://www.twitter.com/befreax Software Engineer Cloud, Grid and Virtualization Sun Microsystems GmbH Dr.-Leo-Ritter-Str. 7 mailto:thijs.metsch@sun.com D-93049 Regensburg http://www.sun.com

2 1