Sam, thanks for the brain dump. I got the top level part of it... I'm still little unsure of the model, specifically how property relationships are handled. Lets say each virtual port can have a collection of UNICAST, MULTICAST or BROADCAST type MAC addresses assigned. The mac address collection is represented as a set of links referencing the MAC address entities (port properties). Can the MAC addresses "back reference" the Virtual port instance ? How would this be represented in json ? I guess I'm looking for the mapping to json. Hint Hint Note: I'm not hoping for a discussion of JSON vs xml. We have xml today, so I'm using it for discussion points.. If the decision is to move to json at some point, we need to map the current xml properties to json. I have also included a matrix of network properties for devices and protocols. The matrix is a compilation of several different standards and operating system capabilities. The matrix is only about 15% complete. It's very ugly right now, I'll be completing most of it this week. This is not written in stone and is all open for debate. BTW, the matrix is in excel 2008 xml format. My TODO list for the matrix: We need a way to describe, set and monitor the QOS of a virtual port when the physical port is shared by several VMs or other execution environment. Should we use the same QOS policy model for load balancing across 802.3ad style aggregated links. Additionally, need a model to reconcile TCP Offload engines, iSCSI and HBAs that combine those protocols on multiple ports. cheers -gary Linking Example (the brackets are my comments/narratives): [Network Port Resource MAC example] <entry> <id>urn:uuid:253d83dd-e417-4e1f-88888-8c0a63120475</id> [Network Port Resource Instance Identifier] <title>Network Port Resource #1 for VM #1</title> <summary>Sample Network Port Resource #1</summary> <updated>2009-05-04T09:52:37Z</updated> <link href="/253d83dd-e417-4e1f-9958-8c0a63120471" /> [Network Port MAC Address Instance Identifier (UNICAST)] <link href="/253d83dd-e417-4e1f-9958-8c0a63120472" /> [Network Port MAC Address Instance Identifier (UNICAST)] <link href="/253d83dd-e417-4e1f-9958-8c0a63120473" /> [Network Port MAC Address Instance Identifier (MULTICAST)] <link href="/253d83dd-e417-4e1f-9958-8c0a63120474" /> [Network Port MAC Address Instance Identifier (MULTICAST)] <link href="/253d83dd-e417-4e1f-9958-8c0a63120475" /> [Network Port MAC Address Instance Identifier (BROADCAST)] <link href="/253d83dd-e417-4e1f-9958-8c0a63120476" /> [Network Port MAC Address Instance Identifier (NOT CONFIG'D)] </entry> <entry> <id>urn:uuid:253d83dd-e417-4e1f-9958-8c0a63120471</id> [MAC Address Instance Identifier] <title>Network Port MAC Address</title> <summary>Network Port MAC Address</summary> <topology>Wireless 20TB Future</topology> <addresstype>UNICAST</addresstype> <address>123108098234098</address> <address_nick_name>Private</address_nick_name> <link href="/253d83dd-e417-4e1f-88888-8c0a63120475" /> [Back Link to Network Port Resource Instance Identifier] <updated>2009-05-04T09:52:37Z</updated> </entry> <entry> <id>urn:uuid:253d83dd-e417-4e1f-9958-8c0a63120472</id> [MAC Address Instance Identifier] <title>Network Port MAC Address</title> <summary>Network Port MAC Address</summary> <topology>Wireless 20TB Future</topology> <addresstype>UNICAST</addresstype> <address>123abcdef</address> <address_nick_name>Family</address_nick_name> <link href="/253d83dd-e417-4e1f-88888-8c0a63120475" /> [Back Link to Network Port Resource Instance Identifier] <updated>2009-05-04T09:52:37Z</updated> </entry> <entry> <id>urn:uuid:253d83dd-e417-4e1f-9958-8c0a63120473</id> [MAC Address Instance Identifier] <title>Network Port MAC Address</title> <summary>Network Port MAC Address</summary> <topology>Wireless 20TB Future</topology> <addresstype>MULTICAST</addresstype> <address>ABDF777cdef</address> <address_nick_name>Hulu Channels</address_nick_name> <link href="/253d83dd-e417-4e1f-88888-8c0a63120475" /> [Back Link to Network Port Resource Instance Identifier] <updated>2009-05-04T09:52:37Z</updated> </entry> <entry> <id>urn:uuid:253d83dd-e417-4e1f-9958-8c0a63120474</id> [MAC Address Instance Identifier] <title>Network Port MAC Address</title> <summary>Network Port MAC Address</summary> <topology>Wireless 20TB Future</topology> <addresstype>MULTICAST</addresstype> <address>WOW-12345ALL</address> <address_nick_name>WOW Raids</address_nick_name> <link href="/253d83dd-e417-4e1f-88888-8c0a63120475" /> [Back Link to Network Port Resource Instance Identifier] <updated>2009-05-04T09:52:37Z</updated> </entry> <entry> <id>urn:uuid:253d83dd-e417-4e1f-9958-8c0a63120475</id> [MAC Address Instance Identifier] <title>Network Port MAC Address</title> <summary>Network Port MAC Address</summary> <topology>Wireless 20TB Future</topology> <addresstype>BROADCAST</addresstype> <address>FFFFFFFFFF</address> <address_nick_name>NATO Alerts</address_nick_name> <link href="/253d83dd-e417-4e1f-88888-8c0a63120475" /> [Back Link to Network Port Resource Instance Identifier] <updated>2009-05-04T09:52:37Z</updated> </entry> <entry> <id>urn:uuid:253d83dd-e417-4e1f-9958-8c0a63120476</id> [NOT CONFIG'D MAC Address Instance Identifier] <title>Network Port MAC Address</title> <summary>Network Port MAC Address</summary> <topology>Wireless 20TB Future</topology> <addresstype>NOT CONFIG'D</addresstype> <address></address> <address_nick_name>Any Nick Name Policy</address_nick_name> <link href="/253d83dd-e417-4e1f-88888-8c0a63120475" /> [Back Link to Network Port Resource Instance Identifier] <updated>2009-05-04T09:52:37Z</updated> </entry> address - with a MAC address assigned. We start adding interfaces so we Sam Johnston wrote:

Gary,

On Sun, May 17, 2009 at 3:17 AM, Gary Mazz <garymazzaferro@gmail.com <mailto:garymazzaferro@gmail.com>> wrote:

Hi,

Can we see a concrete example of top level storage and network rendering with one layer of decomposition ? I'd like to know what it will look like, especially with links applied.

The reference implementation at http://occitest.appspot.com/ currently generates valid Atom <http://validator.w3.org/feed/check.cgi?url=http%3A%2F%2Foccitest.appspot.com> and while it has basic links between resources, it lacks things like the local identifier (e.g. "eth0" or "sda") because there are a number of ways to skin this particular cat:

* add them where possible as generic attributes in the main occi namespace (only really applies to the local identifier, but it does generally make sense to give a local handle to any resource): <link ... occi:localname="eth0" network:address="1.2.3.4" /> * add them as specific attributes in the extension namespaces (e.g. network:interface, storage:device) <link ... network:interface="eth0" network:address="1.2.3.4" /> * add them as elements under the link element <link ... > <network:interface>eth0</network:interface> <network:address type="ipv4>1.2.3.4</network:address> </link> * some combination ( <link ... occi:localname="eth0"> <network:address type="ipv4>1.2.3.4</network:address> </link>

I got the idea of having them as elements under the link element from Microsoft's Astoria team blog <http://blogs.msdn.com/astoriateam/>, specifically the post on Related entries and feeds: links and link expansion <http://blogs.msdn.com/astoriateam/archive/2008/02/18/related-entries-and-feeds-links-and-link-expansion.aspx>. The suggestion has been expanded and is approved of <http://blogs.msdn.com/astoriateam/archive/2008/02/18/related-entries-and-feeds-links-and-link-expansion.aspx#8573352> by Joe Gregorio/Google too.

This is _very_ relevant for performant/scalable serialisation/migration of infrastructure between providers (e.g. portability)... it means essentially that we can follow Tim's suggestion of using links to advertise e.g. OVF versions of resources, but then when required we can also dereference and safely include the content inline under the link element:

<link href="http://example.com/810a39a0-f8a4-49f7-8844-0399b4499c6f.ovf"> <content type="application/ovf+xml"> <!-- OVF content here --> </content> </link>

Obviously one would only dereference links when requested to do so, and even if they were dereferenced (e.g. by clicking "Save As..." on your cloud) they wouldn't get in the way of simpler implementations that don't care about them. It's about as clean & elegant as we're ever going to get if we're expecting portability.

Incidentally currently storage and networking devices don't link to anything (though they could - a virtual network pointing back at a physical one or a logical volume pointing back at a physical RAID). The compute resources are more interesting because they point to both. If you're having trouble retrieving the feed in its current form I've attached it for you.

Sam

2009/5/18 Richard Davies <richard.davies@elastichosts.com>

I would also like to see the current AtomPub example extended with a few actual attributes on the nouns (e.g. cpu cores, memory size, storage size) and showing a few example actuators (e.g. server start/stop, etc) since I understand that they will also appear in the feed.

Ok so for attributes I have thus far been adding them directly to the atom <entry> elements to keep things nice and flat and the <content> free for other purposes (e.g. expanding on a simple OCCI description of a compute or network resource with DMTF/OVF and SNIA standard descriptors respectively). For example: <entry>

<id>...</id> <compute:cores>4</compute:cores> <compute:memory>16Gb</compute:memory> ... </entry>

If you're worried about burning 1's and 0's on namespaces then you can trim them back: <entry>

<id>...</id> <c:cores>4</c:cores> <c:memory>16Gb</c:memory> ... </entry>

For the state transition links you need a few things: - The link itself (URL) - A machine-readable identifier for the link (e.g. http://purl.org/occi/state#start) - A human-friendly identifier for the link in requested/local language (e.g. "Take Snapshot") How this looks in atom: <link title="Start" rel="http://purl.org/occi/state#start" href="

http://example.com/{uuid}/state/start" />

Note that anybody can create new transitions but those that may need to be interoperable would end up in our registries. Cheers, Sam (sending offline) Gary Mazz wrote:

...

Hi,

Can we see a concrete example of top level storage and network rendering with one layer of decomposition ? I'd like to know what it will look like, especially with links applied.

cheers

-gary

---

occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

I finished my PhD thesis on a RESTful model for grid computing a few years ago. I did some work on formalizing XML-based descriptors, including units/scaling. You may find that some of the material in Appendix B and C is useful: http://crystal.harvard.edu/~ijstokes/thesis/RESTfulGrid-StokesRees-Thesis.pd... I envisioned two options: i) units were specified in attributes on an element: <memory type="GB"> 2 </memory> ii) units were specified in-line and needed to be parsed: <memory> 2 GB </memory> At the time I felt that "type" provided a more generic modifier on a value than "units", and would allow things like: <start type="USDate">5/21/2009</start> or <start type="EUDate">21/5/2009</start> I provide 5 sets of "type" modifiers: time, frequency, metric, binary, transfer. No rocket science, but perhaps providing an cut-down portion of the schemas will be helpful, so I include them below. Ian <xs:simpleType name="TimeTypeList"> <xs:enumeration value="fs"/> <xs:enumeration value="ps"/> <xs:enumeration value="ns"/> <xs:enumeration value="us"/> <xs:enumeration value="ms"/> <xs:enumeration value="s"/> <xs:enumeration value="sec"/> <xs:enumeration value="second"/> <xs:enumeration value="min"/> <xs:enumeration value="minute"/> <xs:enumeration value="h"/> <xs:enumeration value="hr"/> <xs:enumeration value="hour"/> <xs:enumeration value="d"/> <xs:enumeration value="day"/> <xs:enumeration value="wk"/> <xs:enumeration value="week"/> <xs:enumeration value="mon"/> <xs:enumeration value="month"/> <xs:enumeration value="yr"/> <xs:enumeration value="year"/> <xs:simpleType name="MetricTypeList"> <xs:enumeration value="f"/> <xs:enumeration value="p"/> <xs:enumeration value="n"/> <xs:enumeration value="u"/> <xs:enumeration value="m"/> <xs:enumeration value="K"/> <xs:enumeration value="M"/> <xs:enumeration value="G"/> <xs:enumeration value="T"/> <xs:enumeration value="P"/> <xs:simpleType name="FrequencyTypeList"> <xs:enumeration value="Hz"/> <xs:enumeration value="KHz"/> <xs:enumeration value="MHz"/> <xs:enumeration value="GHz"/> <xs:enumeration value="THz"/> <xs:simpleType name="BinaryTypeList"> <xs:enumeration value="B"/> <xs:enumeration value="KB"/> <xs:enumeration value="MB"/> <xs:enumeration value="GB"/> <xs:enumeration value="TB"/> <xs:enumeration value="PB"/> <xs:simpleType name="TransferTypeList"> <xs:enumeration value="Kb/s"/> <xs:enumeration value="Mb/s"/> <xs:enumeration value="Gb/s"/> <xs:enumeration value="Tb/s"/> <xs:enumeration value="Pb/s"/>

On Wed, May 27, 2009 at 12:32 AM, Benjamin Black <b@b3k.us> wrote:

a 4th option, which i rather prefer since the units stuff tends to be relevant to and consumed by humans via UI rather than machines via API, is not to use units at all.

<memory>2147483648</memory>

either of the above is far easier to transform to and from non-XML representations, in my experience, with the latter being zero effort. a couple extra bytes won't harm us and we adhere to my first engineering rule: the best solution to a problem is not to have it in the first place.

Andy and I spent a few hours on the phone tonight getting ourselves aligned and this was basically the conclusion we came to as well (though we were talking about choosing e.g. megabytes for memory, gigabytes for disk and gigahertz for processors). If we want/need to add units later we always can but they do then to make implementations fugly too. Sam

Just as an fyi, media folks work in "bits" -gary Andre Merzky wrote:

Quoting [Sam Johnston] (May 26 2009):

...

...

a 4th option, which i rather prefer since the units stuff tends to be relevant to and consumed by humans via UI rather than machines via API, is not to use units at all. <memory>2147483648</memory> either of the above is far easier to transform to and from non-XML representations, in my experience, with the latter being zero effort. a couple extra bytes won't harm us and we adhere to my first engineering rule: the best solution to a problem is not to have it in the first place.

Andy and I spent a few hours on the phone tonight getting ourselves aligned and this was basically the conclusion we came to as well (though we were talking about choosing e.g. megabytes for memory, gigabytes for disk and gigahertz for processors).

I think that is a great compromise: simple format, + human readable.

Andre.

Andre Merzky wrote:

Oh well... - you can't make everybody happy. At the end one needs to decide on one of the options, and either way, just getting rid of units (by defining them as fixed) seems like a good solution. As others stated: a UI can always represent a more suitable version...

We work in different domains, clearly. For me, and the work I've done full time for the past 7 years, and hundreds of other people doing grid computing (which I am definitely saying is different from the requirements of cloud computing folks) there is a lot of manual creation of descriptor files, so "raw" data useability in terms of creation and in terms of readability is an important factor. It would not be unreasonable to have a "basic" and "advanced" profile for properties, where basic use a fixed/unitary unit, while "advanced" use a type/scale modifier for the units, or support unit conversion. Designing a standard so it is easy to write JavaScript web pages for it doesn't seem like it should be a dominant concern. Anyway, I only chimed in because someone asked about application of units to some of the properties. Ian

I think units are a very important issue. More significant than atom/json/xlm discussions. I think someone pushed a satellite into mars over a foot/meter discrepancy. What if its an e911 or other emergency application running on a cloud. It really helps to reduce operational risk with a page of text in a spec. -gary Andre Merzky wrote:

Oh well... - you can't make everybody happy. At the end one needs to decide on one of the options, and either way, just getting rid of units (by defining them as fixed) seems like a good solution. As others stated: a UI can always represent a more suitable version...

A

Quoting [Gary Mazz] (May 26 2009):

...

Just as an fyi, media folks work in "bits"

-gary

Andre Merzky wrote:

...

Quoting [Sam Johnston] (May 26 2009):

...

...

a 4th option, which i rather prefer since the units stuff tends to be relevant to and consumed by humans via UI rather than machines via API, is not to use units at all. <memory>2147483648</memory> either of the above is far easier to transform to and from non-XML representations, in my experience, with the latter being zero effort. a couple extra bytes won't harm us and we adhere to my first engineering rule: the best solution to a problem is not to have it in the first place.

Andy and I spent a few hours on the phone tonight getting ourselves aligned and this was basically the conclusion we came to as well (though we were talking about choosing e.g. megabytes for memory, gigabytes for disk and gigahertz for processors).

I think that is a great compromise: simple format, + human readable.

Andre.

Quoting [Gary Mazz] (May 26 2009):

I think units are a very important issue. More significant than atom/json/xlm discussions. I think someone pushed a satellite into mars over a foot/meter discrepancy.

What if its an e911 or other emergency application running on a cloud. It really helps to reduce operational risk with a page of text in a spec.

I am not saying they are not important - but it does not matter on *which* one you agree, as long as we agree on something, and the spec is clear about that... Andre.

-gary

Andre Merzky wrote:

...

Oh well... - you can't make everybody happy. At the end one needs to decide on one of the options, and either way, just getting rid of units (by defining them as fixed) seems like a good solution. As others stated: a UI can always represent a more suitable version...

A

Quoting [Gary Mazz] (May 26 2009):

...

Just as an fyi, media folks work in "bits"

-gary

Andre Merzky wrote:

...

Quoting [Sam Johnston] (May 26 2009):

...

...

a 4th option, which i rather prefer since the units stuff tends to be relevant to and consumed by humans via UI rather than machines via API, is not to use units at all. <memory>2147483648</memory> either of the above is far easier to transform to and from non-XML representations, in my experience, with the latter being zero effort. a couple extra bytes won't harm us and we adhere to my first engineering rule: the best solution to a problem is not to have it in the first place.

Andy and I spent a few hours on the phone tonight getting ourselves aligned and this was basically the conclusion we came to as well (though we were talking about choosing e.g. megabytes for memory, gigabytes for disk and gigahertz for processors).

I think that is a great compromise: simple format, + human readable.

Andre.

-- Nothing is ever easy.

honestly, folks. "native units of an industry"? _this_ is a level of navel contemplation indicative of not enough real problems to solve. On Tue, May 26, 2009 at 7:17 PM, Gary Mazz <garymazzaferro@gmail.com> wrote:

Andre,

Maybe I've misinterpreted what was proposed? I though if a particular industry infrastructure, like broadcast and telecom, natively uses a terminology like bits, the spec won't support their native terminology. For example cable channels, satellites and mpeg is commonly referred to bits.

By not supporting the native units of an industry, you infer that is not an industry your specification recognizes. I'm sure that's not the intent, but that is how it may be interpreted.

-gary

Andre Merzky wrote:

...

Quoting [Gary Mazz] (May 26 2009):

...

I think units are a very important issue. More significant than atom/json/xlm discussions. I think someone pushed a satellite into mars over a foot/meter discrepancy.

What if its an e911 or other emergency application running on a cloud. It really helps to reduce operational risk with a page of text in a spec.

I am not saying they are not important - but it does not matter on *which* one you agree, as long as we agree on something, and the spec is clear about that...

Andre.

...

-gary

Andre Merzky wrote:

...

Oh well... - you can't make everybody happy. At the end one needs to decide on one of the options, and either way, just getting rid of units (by defining them as fixed) seems like a good solution. As others stated: a UI can always represent a more suitable version...

A

Quoting [Gary Mazz] (May 26 2009):

...

Just as an fyi, media folks work in "bits"

-gary

Andre Merzky wrote:

...

Quoting [Sam Johnston] (May 26 2009):

>> a 4th option, which i rather prefer since the units stuff tends to be >> relevant to and consumed by humans via UI rather than machines via >> API, >> is not to use units at all. >> <memory>2147483648</memory> >> either of the above is far easier to transform to and from non-XML >> representations, in my experience, with the latter being zero effort. >> a couple extra bytes won't harm us and we adhere to my first >> engineering rule: the best solution to a problem is not to have it in >> the first place. >> >> >> > Andy and I spent a few hours on the phone tonight getting ourselves > aligned and this was basically the conclusion we came to as well > (though we were talking about choosing e.g. megabytes for memory, > gigabytes for disk and gigahertz for processors). > > > I think that is a great compromise: simple format, + human readable.

Andre.

_______________________________________________ occi-wg mailing list occi-wg@ogf.org http://www.ogf.org/mailman/listinfo/occi-wg

So, now I can order a new 10gigE card ? Opps, sorry a 125Mb/s ethernet card, or is that 10bit encoding ... a 100MB/s ethernet card, I forgot, they have compression on the circuit so, I'm sending text, thats 8:1 compression and I'm sending binary thats 2:1 but binary only 20% of the traffic and 10 bit encoding so that's a 720MB/s ethernet card. We can find the same issue in fibre channel and infiniband Enough factitiousness... I've only had 4 hrs of sleep in 3 days. Obviously common sense must be applied. There are common industry terminologies which should be respected. The DMTF spec calls out their CIM models with represent items in units other than bytes. Presenting information in a form alien to the industry and role of the user only isolates the user from the technology, adoption is a critical. Companies spend a lot of money training their people both formally and on the job. If I was assessing a product to roll out. And, the product has different terminology than what my team was trained for, I would take a hard look to see how it affected my business processes. I would need my staff to communicate in common terms, I don't need a product that will drive up my operational costs; cause me to retrain and reduce my production efficiency. I have made that decision against purchasing a product for that identical reason. I'm not going to argue not representing everything in bytes. If there is consensus everything is one unit, go for it. I'll vote -1.You could win me over if you can find one of the cloud apis or network product marketed that allocates network bandwidth in bytes per second. The real issue here is nearly everyone has a different set of requirements. Design decisions are normally driven by them. Since occi is racing to a goal, hard requirements are lacking. Everything works out over time. -gary Andre Merzky wrote:

Quoting [Gary Mazz] (May 26 2009):

...

Andre,

Maybe I've misinterpreted what was proposed? I though if a particular industry infrastructure, like broadcast and telecom, natively uses a terminology like bits, the spec won't support their native terminology. For example cable channels, satellites and mpeg is commonly referred to bits.

By not supporting the native units of an industry, you infer that is not an industry your specification recognizes. I'm sure that's not the intent, but that is how it may be interpreted.

No, I think you understood correctly. Still, you can't make all happy if you want to have a well defined and finite! set of units.

The units are supposed to be applied to disk space, memory size, and network bandwidth only - is the industry you are refering to specifying network bandwith in MBit/second? I doubt it.

Where would you stop? Would you allow bit/minute? TB/decade? That would mean free form units, yet another registry, and yet another set of conditions where implementations will fail to interop.

Instead agreeing on a well known, fundamental unit which can trivially be converted to other, domain specific units, seems like an excellent and *simple* thing. IMHO, we should do as was proposed: either use always bytes (or always bits if you want), or always KB, or MB, or [GB for disk, MB for memory, MB/s for network].

I don't think some industry will feel ignored when we specify disk space in gigabytes...

Cheers, Andre.

...

-gary

Andre Merzky wrote:

...

Quoting [Gary Mazz] (May 26 2009):

...

I think units are a very important issue. More significant than atom/json/xlm discussions. I think someone pushed a satellite into mars over a foot/meter discrepancy.

What if its an e911 or other emergency application running on a cloud. It really helps to reduce operational risk with a page of text in a spec.

I am not saying they are not important - but it does not matter on *which* one you agree, as long as we agree on something, and the spec is clear about that...

Andre.

...

-gary

Andre Merzky wrote:

...

Oh well... - you can't make everybody happy. At the end one needs to decide on one of the options, and either way, just getting rid of units (by defining them as fixed) seems like a good solution. As others stated: a UI can always represent a more suitable version...

A

Quoting [Gary Mazz] (May 26 2009):

...

Just as an fyi, media folks work in "bits"

-gary

Andre Merzky wrote:

> Quoting [Sam Johnston] (May 26 2009): > > > > >>> a 4th option, which i rather prefer since the units stuff tends to be >>> relevant to and consumed by humans via UI rather than machines via >>> API, >>> is not to use units at all. >>> <memory>2147483648</memory> >>> either of the above is far easier to transform to and from non-XML >>> representations, in my experience, with the latter being zero effort. >>> a couple extra bytes won't harm us and we adhere to my first >>> engineering rule: the best solution to a problem is not to have it in >>> the first place. >>> >>> >>> >>> >> Andy and I spent a few hours on the phone tonight getting ourselves >> aligned and this was basically the conclusion we came to as well >> (though we were talking about choosing e.g. megabytes for memory, >> gigabytes for disk and gigahertz for processors). >> >> >> >> > I think that is a great compromise: simple format, + human > readable. > > Andre. > > > > > > >