April 2009 - occi-wg - lists.ogf.org

APIs for cloud computing
by Alexis Richardson 17 Apr '09

17 Apr '09

All, Please see the following matrix contributed by Patrick Kerpan from CohesiveFT. http://spreadsheets.google.com/ccc?key=pGccO5mv6yH8Y4wV1ZAJrbQ It lists operations that might appear in a cloud services API, and how some of the known APIs implement those operations (if at all). The idea is to be a reference point for understanding OCCI needs. It is not a substitute for work to create OCCI. If you wish to have write access to this document, please contact Pat directly. alexis

5 13

Re: [occi-wg] Nouns and Verbs (was: Syntax of OCCI API) - state model
by Thijs Metsch 17 Apr '09

17 Apr '09

Hi, On Fri, 2009-04-17 at 13:46 +0100, Chris Webb wrote: > of VMs within a cloud for load-balancing is something that should > surely be completely user-transparent. Is this intended for some other > purpose?) I think it might be useful to know the exact state at sometime. Maybe we an split is up: It only get in migration state when the users triggered the operation. It stays in running when it is a load-balacing operation. > Also, I don't follow what your 'installed' and 'starting' states are > for. > Surely our virtual machine is either running or not: we can't > distinguish > from outside whether a guest OS has fully started or not, even if we > believe > such a distinction is well-defined. I tried to summarize the states in the wiki page :-) Cheers, -Thijs -- Thijs Metsch Tel: +49 (0)941 3075-122 (x60122) http://blogs.sun.com/intheclouds Software Engineer Grid Computing Sun Microsystems GmbH Dr.-Leo-Ritter-Str. 7 mailto:thijs.metsch@sun.com D-93049 Regensburg http://www.sun.com

4 6

edits to patrick's google doc
by Rich Wolski 17 Apr '09

17 Apr '09

Hi All, I've edited Patrick's spreadsheet to make it a little more up to date. Cheers, Rich

2 1

Semantics of OCCI API: nouns and verbs
by Chris Webb 17 Apr '09

17 Apr '09

The single strongest point we want to make is that an API should be simple. We're convinced that an API covering infrastructure can be formulated on a handful of straightforward objects with no more than around fifteen or twenty operations in total. We're also convinced that these operations can be very simple and straightforward, with syntax which could easily be typed by hand, but this is a later issue once we agree on the nouns and verbs themselves. In the spirit of 'put up or shut up', here's a suggestion of what the complete set of nouns and verbs might be, based on what we have in our existing API. Nouns: drives - block devices to attach to virtual machines. The general case here is clearly behaviour like Amazon EBS - drives are persistent and exist independently of virtual machines. A virtual machine can mount several drives, and conversely a drive (like a CD image) can in principle be mounted from several virtual machines. GoGrid 'one drive per server' behaviour is clearly a subset of this, and Amazon's filesystem initialised from an AMI at boot can be represented by throwaway copies of a drive (or copy-on-write) instead of writing back to the drive. resources - things like static IP addresses and VLANs which need to be reserved for use by virtual machines. guests - virtual servers booted from and accessing drives. Our guests exist as objects only when they are running, similar to Amazon's instances, but it may be more general to allow guests in stopped and suspended states in addition, as GoGrid currently do. Verbs: drives list (list accessible drives) --- info (list drive properties) create (takes properties including size; give parent drive for snapshot) destroy (delete existing drive) set (takes properties including size to resize) read (read data of given size with given offset in drive) write (write data at given offset in drive) image (copy data from one drive to another) resources list (list allocated resources) --- info (detail on an allocated resource) create (to allocate a resource such as a static IP address) destroy (to release a resource) guests list (list virtual machines) --- info (list virtual machine properties) create (takes simple description, e.g. including attached drives and network interfaces) set (updates configuration from new description where possible) destroy (hard-kill virtual machine) reset (send reset to virtual machine) shutdown (gracefully shut down virtual machine) [perhaps also stop, start, suspend for stopped guests?] Does anyone think we've missed anything? Comparison with draft API on the wiki ===================================== The draft API on the OCCI wiki currently appears to offer nouns for servers, storage devices, network interfaces with the ability to create, retrieve, update and delete any noun and 6 verbs for machine control. Our suggestion above is very similar, but more explicit and descriptive. In particular: - list, info, create, destroy, set are equivalent to CRUD; - added drive operations: read, write, image - resize operation handled by passing the new size to drive set; - nouns for network resources (e.g. static IPs, VLANs) rather than network interfaces[1]; - similar set of machine operations, assuming we want to handle stopped guests. [1] We believe that interfaces are simply one aspect of a server's configuration, and that the nouns that matter are the objects such as static IPs which are "owned" by a customer and hence can be configured onto the server. Cheers, Chris.

10 20

Syntax of OCCI API
by Chris Webb 16 Apr '09

16 Apr '09

Moving from semantics to syntax, I've been reviewing the draft API design which Sam has posted on the wiki at http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/APIDesign Sam - congratulations on getting the ball running and something up for us to comment on! What follows may sound like criticism, but I agree with the majority of your design goals. The beauty of the best existing APIs (e.g. ElasticHosts, GoGrid) is that they are very simple to use and to implement. They have a small number of nouns and verbs (see my previous mail) and also very straightforward syntax: typical commands take only ~100 bytes and could be typed by hand. I'm concerned that the draft API is heading towards syntax with ~1000s of bytes of a relatively complex XML schema - this is a consequence of "Design Pattern A" following the Google GData approach. http://wiki.cloudcommunity.org/wiki/Open_Cloud_Computing_Interface [I can't find this on the OGF wiki - where is it?] http://www.ogf.org/pipermail/capi-bof/2009-March/000138.html I'd like to revisit Sam's original email to the CAPI list on design goals---I did not see any response from others at the time---and propose "Design Pattern B" following the ElasticHosts and GoGrid approach. I'm happy to expand on this on the wiki. I propose splitting the existing API Design page into several parts: - Design goals (as Sam's original mail and my comments below) - Nouns and verbs (i.e. semantics of objects; independent of syntax) - Design Pattern A (Sam's syntax in style of Google GData) - Design Pattern B (syntax in style of ElasticHosts/GoGrid) Do people agree? Revisiting design goals ======================= > Design goals > > * Clarity making it easy to implement and consume (15-20 calls max) > * Based on Hypertext Transfer Protocol (HTTP) > * Using Representational State Transfer (REST) (with HATEOAS?) > * Requires Transport Layer Security (TLS) for integrity and > confidentiality > * Relies on HTTP for security (rather than signing requests which caused > problems for Amazon): > o HTTP Basic access authentication > o TLS Client certificate authentication > o OAuth We agree with all of these. I think the authentication should be via the Authorization: header with a set of supported methods announced by WWW-Authenticate: in the standard HTTP way. I think it's outside the scope of this standard to specify which methods must be supported, although we would probably implement plain and cram-md5 as a minimum. > * Single URL entry point for trivial client configuration and support of > multiple different APIs (e.g. http://api.example.com/occi) I agree that all API calls should be under a single stem. I would separate the ~15 verbs to separate leaf URIs under that stem for simplicity of each individual call, as you already do with machine operations, so that the CRUD operations also each have their own leaf URIs as opposed to operating on the object URI directly. (This is the style of ElasticHosts verbs create, info, set, destroy on drives, guests and resources, e.g. POST /guests/create POST /drives/create POST /guests/GUEST/destroy etc.) > * Single structured data format (e.g. XML vs JSON vs ???) I disagree here. In the spirit of making the API simple for users with different platforms and languages, I recommend multiple skins distinguished by HTTP Content-Type. (ElasticHosts today support text and JSON with plans for XML; GoGrid support JSON, XML and CSV; Sun support JSON.) > * XML responses (with XML namespaces for extensibility) > o Atom and/or AtomPub format (for extensive client library support?) ala > Google GData APIs The key mistake is using complicated data structures: either complex XML schemas (c.f. existing draft, which already nests 7 levels of indentation!) or simply lists of properties of lists, etc. (c.f. Sun's cloud API). In both the ElasticHosts and GoGrid APIs, we've been very disciplined in making each of our verbs take and return a very simple data structure: either a simple list of uuids or key/value pairs of properties for a single object. This makes parsing trivial in any language, and makes it easy for us to support receiving and generating the same data with different content types (XML, JSON, text, etc. as above). > * Sensible, transparent versioning - only one API URL for all clients > (perhaps using HTTP content negotiation) Content negotiation is for agreeing the content-type (i.e. syntax) not anything higher level. The content type of (say) JSON is application/json or text/json, not something OCCI-specific, so this looks like an abuse of mechanism to me. I'm open to suggestions about how this can be done cleanly in a way that isn't cumbersome for end-users, though. > * Random UUIDs (version 4) for internal object identification (opaque is > good for security) e.g. {f47ac10b-58cc-4372-a567-0e02b2c3d479} I support UUIDs. They can be allocated in a distributed manner without locking or worries about collisions, and we use them throughout our API. However, some providers may want to issue numeric ids, etc., so instead of constraining identifiers to UUID syntax, perhaps any standard need only specify that object IDs are a unique string from a given alphabet ([a-z0-9-]+ say: best to avoid whitespace and syntactically-significant characters), which are issued by the provider when the object is created. > * WebDAV interface for native OS GUI manipulation? (e.g. drag VM from > library folder to cloud to start, delete to stop, etc.) > o WebDAV adds collections et al in a "standard" fashion > o Interesting implications for getting files into and out of the cloud I think this is something outside the scope of an HTTP API standard, even if it's a cute feature for a cloud provider! > Features > > * Object persistence optional (e.g. Servers in "stopped" state rather than > ephemeral) > o Allows for "lite" implementations such as those burnt into BIOSes. This is an issue of semantics rather than syntax, and I tackled it in my "nouns and verbs" post. Today APIs vary (ElasticHosts, Amazon have ephemeral servers; GoGrid have persistent servers). I lean towards an API supporting persistent servers for greater generality than ElasticHosts currently offer. > Objects > > * Workload: initially virtual machines (e.g. OVF), potentially later > applications (e.g. Ruby gem, Python egg, Java WAR/EAR) > o Can be identified by Internet media type (e.g. application/ovf+xml) > * Container: initially hypervisor (e.g. Hyper-V/KVM/VMware/Xen), > potentially later depending on workload (e.g. Java app server for Java > WAR/EAR) My concern here is that we do not descend into complicated data structures (see above). For example OVF has a significantly complex XML schema in itself. If all we need to do is specify simple parameters like megabytes of memory or gigabytes of disk space then we do not need OVF to do this. Design pattern B ================ In light of these thoughts, I'd propose an alternative "Design Pattern B" in the style of the ElasticHosts <url:http://www.elastichosts.com/products/api> and GoGrid <url:http://wiki.gogrid.com/wiki/index.php/API> APIs, with one URL per verb and much simpler syntax. Essentially, we would agree a small set of ~3 nouns and ~15 verbs as per my other mail. Each verb should take or return a very simple data structure: either a simple list of uuids or key/value pairs of properties for a single object. This data structure could be expressed in a choice of formats (XML, JSON, text, etc.) and would take ~100 bytes or less in any of these. Shall I go ahead, split out the API design wiki page as I described and expand this description of design pattern B? Best wishes, Chris.

3 6

OCCI concall Wednesday 4pm CET
by Thijs Metsch 15 Apr '09

15 Apr '09

Hi @all, I'm pleased to announce our first (from now on) weekly concall. It will take place next Wednesday @ 4pm CET. The agenda is currently set to: 1. Introduction 2. Use cases and Requirements 3. Technical API discussion. If you like more topics or have more ideas feel free to add them to our issue tracker at our sourceforge page: http://forge.ogf.org/sf/tracker/do/listTrackers/projects.occi-wg/tracker The passcode and the dial-in options are below. All the best, -Thijs Passcode: Participant: 9032568 Dial in numbers: Country Toll Numbers Freephone/Toll Free Number ARGENTINA 0800-777-0463 AUSTRALIA ADELAIDE: 61-8-8121-4868 1-800-249-288 AUSTRALIA BRISBANE: 61-7-3102-0970 1-800-249-288 AUSTRALIA CANBERRA: 61-2-6100-1970 1-800-249-288 AUSTRALIA MELBOURNE: 61-3-9010-7739 1-800-249-288 AUSTRALIA PERTH: 61-8-9467-5249 1-800-249-288 AUSTRALIA SYDNEY: 61-2-8205-8125 1-800-249-288 AUSTRIA 43-1-92-86-506 0800-005-029 BELGIUM 32-1-150-0314 0800-4-8680 BRAZIL 0800-7610674 CHILE 1230-020-2867 CHINA* 86-400-810-4766 10800-712-1433 10800-120-1433 COLOMBIA 01800-9-156430 CZECH REPUBLIC 420-2-25-98-56-54 800-700-173 DENMARK 45-7014-0280 8088-6132 ESTONIA 800-011-1089 FINLAND Land Line: 106-33-146 0-800-1-10100 FINLAND Mobile: 09-106-33-146 0-800-1-10100 FRANCE LYON: 33-4-26-69-12-81 080-563-9647 FRANCE MARSEILLE: 33-4-86-06-00-81 080-563-9647 FRANCE PARIS: 33-1-70-70-74-20 080-563-9647 GERMANY 49-69-2222-2566 0800-000-3441 GREECE 30-80-1-100-0683 00800-12-6973 HONG KONG 852-2286-5731 800-930-705 HUNGARY 06-800-18013 INDIA 000-800-852-1266 INDONESIA 001-803-011-3787 IRELAND 353-1-247-5253 1800-932-145 ISRAEL 1-80-9214916 ITALY 39-02-3600-3642 800-986-570 JAPAN OSAKA: 81-6-7739-4769 0034-800-400828 JAPAN TOKYO: 81-3-5539-5189 0034-800-400828 LATVIA 8000-3025 LUXEMBOURG 352-27-000-1360 MALAYSIA 1-800-80-2812 MEXICO 001-866-627-0574 NETHERLANDS 31-20-718-8533 0800-020-1392 NEW ZEALAND 64-9-970-4769 0800-449-823 NORWAY 47-21-59-00-59 800-15414 PANAMA 011-001-800-5072129 PERU 0800-53733 PHILIPPINES 63-2-858-3715 POLAND 00-800-1212021 PORTUGAL 8008-14061 RUSSIA 8-10-8002-9683011 SINGAPORE 65-6883-9228 800-120-4662 SLOVAK REPUBLIC 421-2-322-422-21 SOUTH AFRICA 080-09-80416 SOUTH KOREA 82-2-6744-1081 00798-14800-6860 SPAIN 34-91-414-62-98 800-099-810 SWEDEN 46-8-505-78-524 0200-890-106 SWITZERLAND 41-44-580-4389 0800-001-523 TAIWAN 886-2-2795-7377 00801-137-766 THAILAND 001-800-1206-65656 UNITED KINGDOM BIRMINGHAM: 44-121-210-9021 0808-238-6025 UNITED KINGDOM GLASGOW: 44-141-202-3221 0808-238-6025 UNITED KINGDOM LEEDS: 44-113-301-2121 0808-238-6025 UNITED KINGDOM LONDON: 44-20-7075-3246 0808-238-6025 UNITED KINGDOM MANCHESTER: 44-161-601-1421 0808-238-6025 URUGUAY 000-413-598-3415 USA 1-203-418-3122 866-692-3163 VENEZUELA 0800-1-00-3733 -- Thijs Metsch Tel: +49 (0)941 3075-122 (x60122) http://blogs.sun.com/intheclouds Software Engineer Grid Computing Sun Microsystems GmbH Dr.-Leo-Ritter-Str. 7 mailto:thijs.metsch@sun.com D-93049 Regensburg http://www.sun.com

1 0

Scheduling parameters
by Mehdi Sheikhalishahi 15 Apr '09

15 Apr '09

Dear All: In the API Specification, Do you address the scheduling parameters for Advance Reservation, Co-allocation and others? -- Best Regards, Mehdi Sheikhalishahi

8 22

Requirements from SLA@SOI
by Ignacio Martin Llorente 14 Apr '09

14 Apr '09

Hi Andy, Why don't we move the General API Requirements and General Model Requirements sections that you added at http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/APIDesign to the Use Cases and Requirements page at http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/UseCasesAndR… (see last section) Regards -- Ignacio M. Llorente, Full Professor (Catedratico): web http://dsa-research.org/llorente and blog http://imllorente.dsa-research.org/ DSA Research Group: web http://dsa-research.org and blog http://blog.dsa-research.org Globus GridWay Metascheduler: http://www.GridWay.org OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org

2 1

The OCCI-WG infrastructure
by Thijs Metsch 14 Apr '09

14 Apr '09

Hi @all, I'm pleased to announce that our infrastructure is set up and running at http://forge.ogf.org. Out project is called occi-wg and can be found here: http://forge.ogf.org/sf/projects/occi-wg. Please feel free to join. We will take care that your account will be added to the project as soon as possible. Since we left the BoF status by last week, I'm also announcing our new mailing-list occi-wg(a)ogf.org. Feel free to join (http://www.ogf.org/mailman/listinfo/occi-wg) but I will also send out invitations to all people who are subscribed to this mailing-list. All the best & and looking forward to work with you now that all things have been set up - and we can focus on the creation of our API! -Thijs -- Thijs Metsch Tel: +49 (0)941 3075-122 (x60122) http://blogs.sun.com/intheclouds Software Engineer Grid Computing Sun Microsystems GmbH Dr.-Leo-Ritter-Str. 7 mailto:thijs.metsch@sun.com D-93049 Regensburg http://www.sun.com

1 0