- dfdl-wg - lists.ogf.org

Array storage order attribute - was RE: [dfdl-wg] How to handle m ulti-dimensional arrays - version 2
by Mike Beckerle 03 Mar '05

03 Mar '05

First to address this tiny issue: > > One other minor point - if the order of x and y in the DFDL > file is important (as it is in the example), do we need a > <dfdl:array storageOrder="firstDimensionChangesFirst"> > option? OR can we just list y first and then x? > I thought about using listing order. I decided that people want to list the dimensions in the order they think of them. In this case x first, then y, irrespective of what the storage order is. Then express that representational tweak separately. We could make it a default that lastDimensionChangesFirst then you'd only need to express this attribute if you've listed the dimensions backward, or you've got column-major data. I'm opposed to most default settings for rep properties. I'm with the XML philosophy of being explicit about everything every time. But in this case it seems ok. ...mikeb

1 0

RE: [dfdl-wg] How to handle multi-dimensional arrays - version 2
by Myers, James D 03 Mar '05

03 Mar '05

> > > Here's a slightly different formulation of the multi-dimension stuff. > > 1) no longer dictates the XSD for representing the array. > This cuts both ways since you no longer really have an XSD > model for multi-dimensional arrays. That is. It is up to the > author of the DFDL Schema to insure the needed information > about the array (coordinates of each element) make it in to > the logical model in a useful way. I didn't realize we were proposing to extend the XML schema to have a multidimensional array type, versus providing a way for DFDL to read and internally represent a multidimensional array. The latter seems descriptive and the former prescriptive. > > 2) I added in the complexity of calculating the array size, > actually the lower and upper bounds of each dimension, > dynamically based on data. This makes the example more real. > > This still works out pretty well. I'm still pondering whether > I like this better or not. I'm thinking about perhaps some > sort of pseudo attributes which are guaranteed to be put into > XML if you actually render to XML, but where a DFDL API-based > implementation can choose not to realize them. I think this example removes some of the prescriptive nature of the first one, but I'd like to be able to format my array however I want, e.g. as <row><elem>3</elem><elem>2</elem></row> <row><elem>5</elem><elem>6</elem></row> ... Or even <states><state>Alabama</state><state>Alaska</state></states> <population><pop>34.2</pop><pop>10.6</pop></population> .... (an array containing state names, population and other data, perhaps serialized in the file as all info for each state together). If DFDL could separate the reading of such an array from how it is output in the schema, I could do any of this. Having multiple layers is a start - DFDL reads the array in to something that is addressable along the lines Mike proposes and then the contents of that layer are referenced via xpath to provide values in some structure I define in XSD. The only piece missing (I think) is that we haven't yet defined how to access iterators, i.e. if I have an element <elem minoccurs="1" maxoccurs = "5"> , how can I say that element n (n = 1...5) has dfdl:runtimevalue <a x="n" y="1">, which would put just the first column of a into the element sequence. If, in Mike's example, I could define the x and y dimensions independent of an array-reading context, just so I can use them in value references for dfdl:runtimevalue elements, I think we'd be all set. This type of capability would allow all sorts of useful things - including the array to set of vectors conversions outlined here as well as subsampling, expansion/contraction of sprase arrays (where the array is stored as a sequence of x,y, value triples for only nonzero elements), etc. One other minor point - if the order of x and y in the DFDL file is important (as it is in the example), do we need a <dfdl:array storageOrder="firstDimensionChangesFirst"> option? OR can we just list y first and then x? Jim

1 0

How to handle multi-dimensional arrays - version 2
by mike.beckerle＠ascentialsoftware.com 03 Mar '05

03 Mar '05

Here's a slightly different formulation of the multi-dimension stuff. 1) no longer dictates the XSD for representing the array. This cuts both ways since you no longer really have an XSD model for multi-dimensional arrays. That is. It is up to the author of the DFDL Schema to insure the needed information about the array (coordinates of each element) make it in to the logical model in a useful way. 2) I added in the complexity of calculating the array size, actually the lower and upper bounds of each dimension, dynamically based on data. This makes the example more real. This still works out pretty well. I'm still pondering whether I like this better or not. I'm thinking about perhaps some sort of pseudo attributes which are guaranteed to be put into XML if you actually render to XML, but where a DFDL API-based implementation can choose not to realize them. Your comments appreciated. ...mikeb

1 0

RE: [dfdl-wg] How to handle multi-dimensional arrays
by mike.beckerle＠ascentialsoftware.com 02 Mar '05

02 Mar '05

I think I get your points Jim. Responding stream of consciousness interlineated below. Later points build on earlier ones, so read it all before typing up any reactions to earlier ones. ...mike > > This approach seems to dictate how the user will represent > the array in XML (dictating their schema) rather than just > describing how to pick up the right content. Which I think we > agreed is a bad thing. Your concern seems to be with the requirment that the element have attributes which make the position of the element explicit. I suppose this is only needed if you want to access the array with some sort of multi-dimensional X-path like expression. We chose this attribute mechanism because it seemed the most compact thing we could come up with that was consistent with xpath-like expressions that could select a specific element of the array. We could make it flexible so that you could use sub-elements to hold the positions, not just attributes, or you could even leave them out. If you leave them out then I don't see a way to both tolerate storage-order variations and provide index-based access within the XSD/XML framework. However, if you are not translating into XML, but using an API to access the data where that API is supplied by a DFDL-aware library and has direct support for multi-dimensional arrays, then you certainly do not want lots of storage for a numeric array being taken up representing redundant attributes that just hold the coordinate positions. Hence, these dimension attributes really should be hidden, and not parts (elements or attributes) of the logical data model. > That's not to say this isn't a > reasonable way to represent a multidim array in XML, just > that having DFDL go look at the attributes outside the > annotation and requiring this formatting in the output XML > doesn't seem right. > Point taken. However, can you suggest what might be alternatives for concrete output XML for a multidim array? I'm kind of stuck in the rut of this one currently. Everything else I look at feels awkward in comparison, but this may just be failure of imagination. > In > <dfdl:dataFormat arrayStorageOrder="@y @x"> > > it seems that DFDL only needs the dimension sizes for its own > purposes and we could probably use our referencing mechanism > to get them (i.e. if I read the two dimension ints earlier > and want to reference them for the array sizes) - maybe some > kind of <dfdl:runtimeoccurs> elements for the n dimensions? (BTW: since the earlier mailing, in our prototype this rep property has been renamed to arrayDimensions, since it serves to identify the attributes which provide the dimensions of the element. The order of the elements in the list continues to be used to specify the storage order.) > > To allow the kind of output XML you propose, we probably need > something new to allow you to loop. I don't see why we need a loop. Well not yet anyway. I can clearly see needing to write expressions that need to know the coordinates of the current element's position in the array. I was thinking of that as using xpath-like expressions that would refer to the "dimension attributes". I.e., if I have a 2d array with axes x and y, then ./@x is this element's x-axis position, and ./@y is this element's y-axis position. In a single-dimensional case with unnamed axis, then ./position() is the X-path way of doing this. So far I think the DFDL system can do the iterations, you just have to write the "loop body" which is called over and over again. E.g., this would let you do stuff like combine two conforming arrays of scalars to create an array of tuples. Another important example is that it lets you do stuff like have the null-indicators for an array of nullable values be stored in a separate array of individual flag bits. I'm rather engaged in trying to see how far we can get without putting in any sort of looping or recursion constructs currently since with layering and the ability to write expressions that refer to the "current element's position" or its multidimensional analogue, and to dimension things using the sizes of other things as parameters I think we can do very powerful things quite easily but still declaratively. I'm not philosophically opposed to having more power, I just don't want to put it in until the need is clear. I haven't seen any examples yet where an explicit loop, not an implied loop, is truly needed. > If the only place we need > looping is for multidimensional arrays (and the special case > of one dim), perhaps we can do something similar to what you > propose and essentially have the array mechanism define some > loop variables that can be referenced (a dfdl layer?). I suppose one could use a layer to hide the dimension attributes from the logical model. If the layer was named "rep", then the arrayDimensions property would be "./rep/@y ./rep/@x", and they wouldn't appear in the actual XML logical model of the data. You just wouldn't then have any X-path like way of indexing the resulting logical structure without digging into the rep. It would be up to you whether you want them in the logical structure or not. A layer is perhaps not abstract enough. You don't really want these things to be realized at all, not in a layer nor in the logical model. I'll give this some more thought and there will be some subsequent proposal. Along the lines of your #currentcursorvalue# thing below. The idea is you get an expression that accesses the current position on any dimension of a multi-dimensional array, and it's up to you if you want to populate an attribute subelement or otherwise with it or not. > don't have a full proposal thought out, but imagine defining > an array as a stream that can be referenced using multiple > dimensions rather than a single cursor, and having a > mechanism so that the current value of the cursor(s) are > available to the user. > > So, from the Reference.xsd example, we might want to have an > attribute that shows the x value of the xdata elements > analogous to the multidim > example: > > <xs:element name="xdata" type="xs:float" > maxOccurs="unbounded"> > <xs:annotation> > <xs:appinfo> > > <dfdl:runtimeoccurs>../x</dfdl:runtimeoccurs> > </xs:appinfo> > </xs:annotation> > <xs:attribute name="x"> > <dfdl:runtimevalue = #currentcursorvalue#/> > </xs:attribute> > </xs:element> > > where #currentcursorvalue# is something we have not yet made > available for output (or is this available via xpath - the > position of the current element in a sequence?). Yes in xpath position() for a 1d array fills this role. > This would > change the Reference.xml example output to have elements like > > <xdata x="1">2.78</xdata> > <xdata x="2">3.14</xdata> > > Right. So the user chooses whether to tuck the position into an attribute or not. > So, if I can summarize/rephrase, I think we should keep the > mechanism for single or multidim arrays separate from how the > output is displayed, but I like the idea of making the > current cursor(s) available for use, which I don't think > we've done yet. And having a real multidimension construct > rather than calculating them from a flat cursor is probably a > requirement for scientific use, so some multidim analog of > dfdl:runtimeoccurs is needed. > Ah, runtimeoccurs. My example was fixed length so this didn't come up. I have a dfdl:lengthCalc expression, and also dfdl:storedLength path property in the prototype today. These need to generalize for multidimensional stuff. I'll put that in the next version. ...mikeb

1 0

RE: [dfdl-wg] How to handle multi-dimensional arrays
by Myers, James D 02 Mar '05

02 Mar '05

Mike, Some random quick comments - hopefully not to terse or stream-of-conciousness to be incomprehensible: This approach seems to dictate how the user will represent the array in XML (dictating their schema) rather than just describing how to pick up the right content. Which I think we agreed is a bad thing. That's not to say this isn't a reasonable way to represent a multidim array in XML, just that having DFDL go look at the attributes outside the annotation and requiring this formatting in the output XML doesn't seem right. In <dfdl:dataFormat arrayStorageOrder="@y @x"> it seems that DFDL only needs the dimension sizes for its own purposes and we could probably use our referencing mechanism to get them (i.e. if I read the two dimension ints earlier and want to reference them for the array sizes) - maybe some kind of <dfdl:runtimeoccurs> elements for the n dimensions? To allow the kind of output XML you propose, we probably need something new to allow you to loop. If the only place we need looping is for multidimensional arrays (and the special case of one dim), perhaps we can do something similar to what you propose and essentially have the array mechanism define some loop variables that can be referenced (a dfdl layer?). I don't have a full proposal thought out, but imagine defining an array as a stream that can be referenced using multiple dimensions rather than a single cursor, and having a mechanism so that the current value of the cursor(s) are available to the user. So, from the Reference.xsd example, we might want to have an attribute that shows the x value of the xdata elements analogous to the multidim example: <xs:element name="xdata" type="xs:float" maxOccurs="unbounded"> <xs:annotation> <xs:appinfo> <dfdl:runtimeoccurs>../x</dfdl:runtimeoccurs> </xs:appinfo> </xs:annotation> <xs:attribute name="x"> <dfdl:runtimevalue = #currentcursorvalue#/> </xs:attribute> </xs:element> where #currentcursorvalue# is something we have not yet made available for output (or is this available via xpath - the position of the current element in a sequence?). This would change the Reference.xml example output to have elements like <xdata x="1">2.78</xdata> <xdata x="2">3.14</xdata> So, if I can summarize/rephrase, I think we should keep the mechanism for single or multidim arrays separate from how the output is displayed, but I like the idea of making the current cursor(s) available for use, which I don't think we've done yet. And having a real multidimension construct rather than calculating them from a flat cursor is probably a requirement for scientific use, so some multidim analog of dfdl:runtimeoccurs is needed. Jim -----Original Message----- From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] On Behalf Of mike.beckerle(a)ascentialsoftware.com Sent: Friday, February 18, 2005 4:22 PM To: dfdl-wg(a)gridforum.org Subject: [dfdl-wg] How to handle multi-dimensional arrays We have come up with an approach to how to represent multi-dimensional arrays within XSD-described XML. The attached test file (.xml) and DFDL Schema (.dfdl.xsd) illustrate the proposed solution. The proposal does not require any changes to XSD, XML or any other special constructs outside of a single dfdl annotation to specify the storage order of the representation. I'm pretty happy with how this works out. We can handle arrays with different storage orders, like fortran style column-major vs. more common row-major, and it dovetails nicely with XPath expressions and the XSD data model. Schema validation can really do something for you, like tell you if you have all the elements of the array (if it's fixed size), and that you don't have multiple elements occupying the same array location. Those interested in multi-dimensional array support please give this some consideration. That said, I'm departing on vacation for a week, so I'll toss this out there for people to look at, but I won't be able to interact with you all on it until I get back. ...mikeb

1 0

RE: [dfdl-wg] How to handle multi-dimensional arrays
by Mike Beckerle 02 Mar '05

02 Mar '05

Thanks for the supportive comments Steve, Anyone else? Particularly those of you closer to the scientific-computing community. Would you care to comment on the multi-dimensional arrays proposal that I sent out a bit more than a week ago? ...mikeb > -----Original Message----- > From: Steve Hanson [mailto:smh@uk.ibm.com] > Sent: Wednesday, March 02, 2005 9:23 AM > To: mike.beckerle(a)ascentialsoftware.com > Cc: dfdl-wg(a)gridforum.org; owner-dfdl-wg(a)ggf.org > Subject: Re: [dfdl-wg] How to handle multi-dimensional arrays > > Hi Mike > > I will confess to having not given a great deal of thought to > multi-dim arrays in the past, but what you propose looks to be sound. > > Regards, Steve > > Steve Hanson > WebSphere Business Integration Brokers, > IBM Hursley, England > Internet: smh(a)uk.ibm.com > Phone (+44)/(0) 1962-815848 --------------- > > We have come up with an approach to how to represent > multi-dimensional arrays within XSD-described XML. The > attached test file (.xml) and DFDL Schema (.dfdl.xsd) > illustrate the proposed solution. > > The proposal does not require any changes to XSD, XML or any > other special constructs outside of a single dfdl annotation > to specify the storage order of the representation. > > I'm pretty happy with how this works out. We can handle > arrays with different storage orders, like fortran style > column-major vs. more common row-major, and it dovetails > nicely with XPath expressions and the XSD data model. Schema > validation can really do something for you, like tell you if > you have all the elements of the array (if it's fixed size), > and that you don't have multiple elements occupying the same > array location. > > Those interested in multi-dimensional array support please > give this some consideration. > > That said, I'm departing on vacation for a week, so I'll toss > this out there for people to look at, but I won't be able to > interact with you all on it until I get back. > > ...mikeb > > [attachment "test2dArray.xml" deleted by Steve > Hanson/UK/IBM] [attachment "test2dArray.dfdl.xsd" deleted by > Steve Hanson/UK/IBM] >

1 0

RE: [dfdl-wg] alternate syntax for DFDL annotations
by Mike Beckerle 02 Mar '05

02 Mar '05

Sorry for missing our call today, and thanks to everyone for the rapid feedback on the non-native attributes approach. I think there's a consensus here that non-native attributes are going too far, and that if we can just make the appinfo annotations a bit more dense we'll be happy. So , for now, we're not going to do non-native attributes in our DFDL prototype, I'll make a note about the possiblility in an appendix of the doc though. None of the tools I have will validate them, and that's a big drawback. Right now, our prototype handles small examples with annotations that look like this: <xs:element name="foo" type="xs:int"> <xs:annotation> <xs:appinfo source="http://dataformat.org/"> <dfdl:dataFormat byteOrder="bigEndian" charset="UTF-8">  <dfdl:separator>,</dfdl:separator>  </dfdl:dataFormat> </xs:appinfo> </xs:annotation> </xs:element> Note that if you use appinfo annotations then you don't need namespace qualification on the attribute names. We also allow for "extension" declarations in that the dfdl:dataFormat element is the head of a XSD substitution group, So you can also define an extension like ascl:binaryIntegerFormat below: <xs:element name="foo" type="xs:int"> <xs:annotation> <xs:appinfo source="http://dataformat.org/"> <ascl:binaryIntegerFormat byteOrder="bigEndian"/> </xs:appinfo> </xs:annotation> </xs:element> The ascl:binaryIntegerFormat is a narrowing. It doesn't allow any properties to be specified that don't make sense for a binary integer. It's in a different namespace to illustrate that these could be defined by a DFDL user in their DFDL schema, not just by the dfdl implementation itself. The basic dfdl:dataFormat element allows any and all representation properties. Whether we supply standard library elements which are narrowings of it, or let those be defined by end-users is something we can discuss. This all comes for free by just using XSD properly. It adds almost nothing to the implementation. ...mikeb > -----Original Message----- > From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] On > Behalf Of Steve Hanson > Sent: Wednesday, March 02, 2005 1:08 PM > To: Myers, James D > Cc: dfdl-wg(a)gridforum.org; owner-dfdl-wg(a)ggf.org > Subject: RE: [dfdl-wg] alternate syntax for DFDL annotations > > > > > > I can't say I am over keen on this, because > - you will end up with odd rules where some properties can be > attributes and some can't, which will confuse users. > - you have to invent errors and/or overide behaviour when you > get a clash between attribute and annotation > - (minor) it makes the job of any software that instantiates > a dfdl memory model harder > > If we do adopt this then #2 is preferable. > > Regards, Steve > > Steve Hanson > WebSphere Business Integration Brokers, > IBM Hursley, England > Internet: smh(a)uk.ibm.com > Phone (+44)/(0) 1962-815848 > > > > > "Myers, James D" > > <jim.myers(a)pnl.go > > v> > To > Sent by: dfdl-wg(a)gridforum.org > > owner-dfdl-wg@ggf > cc > .org > > > Subject > RE: [dfdl-wg] > alternate syntax for > 02/03/2005 17:35 DFDL annotations > > > > > > > > > > > > > > > > > > > I agree - I think I like #2. I know at one point in the past > we had complaints when we tried to use attributes with > namespaces - it may just have been that parsers weren't > dealing with them correctly early on. > Whatever it was, option 2 would provide a work around as well > as helping us keep a single 'canonical' form for DFDL. > > Jim > > > -----Original Message----- > > From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] > On Behalf > > Of Martin Westhead > > Sent: Wednesday, March 02, 2005 12:16 PM > > To: Mike Beckerle > > Cc: dfdl-wg(a)gridforum.org > > Subject: Re: [dfdl-wg] alternate syntax for DFDL annotations > > > > > > Hi Mike, > > > > This is interesting I wasn't aware you could do this. My > concern would > > be if it turned out the schema tools couldn't handle it but I tried > > XML Spy which seems to just ignore the extra annotations. > > > > So I'd be happy to use them. As you point out we will need > both these > > and annotations so we are left with a choice. Either we: > > > > 1. Specify some (most) dfdl additions in this non-native > attribute > > form and use annotations when required or > > > > 2. We specify everything as annotations but allow this form as a > > systematic short hand. > > > > I think I prefer (2). The sort of thing I have in mind is > making the > > annotation syntax look like: > > > > <xs:element name="foo" type="xs:string"> > > <annotation> > > <appinfo> > > <dfdl:attributes dfdl:repType="text" > > dfdl:charset="UTF-8" > > dfdl:repLength="10"/> > > </appinfo> > > <annotation > > </xs:element> > > > > And then this be directly equivalent (and trivially transformed > > into/from) what you had before: > > > > <xs:element name="foo" > > type="xs:string" > > dfdl:repType="text" > > dfdl:charset="UTF-8" > > dfdl:repLength="10"/> > > > > I'm not sure if it matters that much but it seems like it might be > > neater if everything could be represented in the same form. > > > > Also there may be times when you would like to use the > annotation form > > because you have a bock of stuff that could be reused > multiple times. > > Once your annotations have become attributes in the main > schema they > > are more intimately tied to it. > > > > Cheers, > > > > Martin > > > > > > > > Mike Beckerle wrote: > > > Up til now we've considered DFDL annotations only as inside the > > > appinfo context. However, we should consider whether we should use > > non-native > > > attributes as well or > > > > > > as an alternative: E.g., simple DFDL rep properties could also be > > > expressed like this: > > > > > > <xs:element name="foo" > > > > > > type="xs:string" > > > > > > dfdl:repType="text" > > > > > > dfdl:charset="UTF-8" > > > > > > dfdl:repLength="10"/> > > > > > > > > > > > > This has the advantages of compactness, and is a fully > > supported way > > > of > > > extending XML Schema. That is, using non-native attributes is a > > > supported extension idiom. This won't handle things that > > really need the > > > syntactic support of element structure to express their > complexity, > > > e.g., things like specifying text delimiters with quoting and > > > escape-sequence specifications. For those we'll still need > > to open an > > > appinfo annotation up. However, for basic things like > > byteOrder and such > > > it is far more attractive syntactically to use non-native > > attributes > > > than appinfo annotations. > > > > > > > > > Comments? > > > > > > ...mikeb > > > > > > > > > >

1 0

RE: [dfdl-wg] Conference Call Today - notes
by Myers, James D 02 Mar '05

02 Mar '05

Sorry for the late FYI on this: If we have a meeting in Toronto (anywhere outside the US), PNNL staff will need 45-60 days to get clearance for the foreign travel. That would mean making decisions about dates and specific location as soon as possible for us to be involved. (I would hope to have at least one of us and probably two of us there.) Jim > -----Original Message----- > From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] On > Behalf Of mike.beckerle(a)ascentialsoftware.com > Sent: Wednesday, February 16, 2005 12:45 PM > To: dfdl-wg(a)gridforum.org > Subject: [dfdl-wg] Conference Call Today - notes > > > > DFDL-WG Conference call 2005-02-16 > > Attendees: Suman Kalia, Steve Hanson, Susan Malaika - IBM, > Tara Talbot - PNNL, Mike Beckerle - Ascential Software > > IBM offered to host a F2F meeting in mid to late April, or > early May at their Toronto facility. Seems like a good time > frame. Gives Ascential folks time time to get prototype "in > play", and leaves time before the June GGF > (Chicago) to get some things written up. > > MikeB and Suman to schedule time to work on combined model > and list of representation properties: week of Feb 28th. > > Ascential prototype: hoping to have simplest examples working > by end of this week. (No promises). Things like 2 binary > integers. MikeB to post current XSD for dfdl properties and a > few of our test files for the prototype. > > Susan asked about whether implementations of things like BFD > are available. Tara to investigate whether BFD is publicly > available now. > > We discussed when to start recruiting Oracle, Microsoft, and > other participants, since we will need them. We agreed that > our hope would be the June GGF though it really depends on > having an initial document representing some sort of > converged consensus of the current team on current directions. > > > Mike Beckerle > Ascential > >

2 1

Fw: [dfdl-wg] alternate syntax for DFDL annotations
by Suman Kalia 02 Mar '05

02 Mar '05

Hi Mike -- We will be creating XML schema to model our annotations and the attributes applicable to the representation of a particular element/type will be grouped within the complex type of the annotation schema. My preference will be use annotation approach consistently in DFDL. Moreover the attributes defined in the compact form are not easily accessible from the XML Schema APIs, you have to look into the DOMNode associated with the XSDElementDeclaration to get access to such attributes; it makes the programming task a little harder. Having said that SOAP encoded schemas use this compact form to identify the type and dimensions of the array types; some folks like this approach and some hate it.. Suman Kalia IBM Toronto Lab WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 03/02/2005 12:38 PM ----- Mike Beckerle <mike.beckerle(a)ascentialsoftware.com> Sent by: owner-dfdl-wg(a)ggf.org 03/01/2005 03:53 PM To dfdl-wg(a)gridforum.org cc Subject [dfdl-wg] alternate syntax for DFDL annotations Up til now we've considered DFDL annotations only as inside the appinfo context. However, we should consider whether we should use non-native attributes as well or as an alternative: E.g., simple DFDL rep properties could also be expressed like this: <xs:element name="foo" type="xs:string" dfdl:repType="text" dfdl:charset="UTF-8" dfdl:repLength="10"/> This has the advantages of compactness, and is a fully supported way of extending XML Schema. That is, using non-native attributes is a supported extension idiom. This won't handle things that really need the syntactic support of element structure to express their complexity, e.g., things like specifying text delimiters with quoting and escape-sequence specifications. For those we'll still need to open an appinfo annotation up. However, for basic things like byteOrder and such it is far more attractive syntactically to use non-native attributes than appinfo annotations. Comments? ...mikeb

1 0

RE: [dfdl-wg] alternate syntax for DFDL annotations
by Myers, James D 02 Mar '05

02 Mar '05

I agree - I think I like #2. I know at one point in the past we had complaints when we tried to use attributes with namespaces - it may just have been that parsers weren't dealing with them correctly early on. Whatever it was, option 2 would provide a work around as well as helping us keep a single 'canonical' form for DFDL. Jim > -----Original Message----- > From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] On > Behalf Of Martin Westhead > Sent: Wednesday, March 02, 2005 12:16 PM > To: Mike Beckerle > Cc: dfdl-wg(a)gridforum.org > Subject: Re: [dfdl-wg] alternate syntax for DFDL annotations > > > Hi Mike, > > This is interesting I wasn't aware you could do this. My > concern would > be if it turned out the schema tools couldn't handle it but I > tried XML > Spy which seems to just ignore the extra annotations. > > So I'd be happy to use them. As you point out we will need both these > and annotations so we are left with a choice. Either we: > > 1. Specify some (most) dfdl additions in this non-native attribute > form and use annotations when required or > > 2. We specify everything as annotations but allow this form as a > systematic short hand. > > I think I prefer (2). The sort of thing I have in mind is making the > annotation syntax look like: > > <xs:element name="foo" type="xs:string"> > <annotation> > <appinfo> > <dfdl:attributes dfdl:repType="text" > dfdl:charset="UTF-8" > dfdl:repLength="10"/> > </appinfo> > <annotation > </xs:element> > > And then this be directly equivalent (and trivially transformed > into/from) what you had before: > > <xs:element name="foo" > type="xs:string" > dfdl:repType="text" > dfdl:charset="UTF-8" > dfdl:repLength="10"/> > > I'm not sure if it matters that much but it seems like it might be > neater if everything could be represented in the same form. > > Also there may be times when you would like to use the > annotation form > because you have a bock of stuff that could be reused multiple times. > Once your annotations have become attributes in the main > schema they are > more intimately tied to it. > > Cheers, > > Martin > > > > Mike Beckerle wrote: > > Up til now we've considered DFDL annotations only as inside the > > appinfo > > context. However, we should consider whether we should use > non-native > > attributes as well or > > > > as an alternative: E.g., simple DFDL rep properties could also be > > expressed like this: > > > > <xs:element name="foo" > > > > type="xs:string" > > > > dfdl:repType="text" > > > > dfdl:charset="UTF-8" > > > > dfdl:repLength="10"/> > > > > > > > > This has the advantages of compactness, and is a fully > supported way > > of > > extending XML Schema. That is, using non-native attributes is a > > supported extension idiom. This won't handle things that > really need the > > syntactic support of element structure to express their complexity, > > e.g., things like specifying text delimiters with quoting and > > escape-sequence specifications. For those we'll still need > to open an > > appinfo annotation up. However, for basic things like > byteOrder and such > > it is far more attractive syntactically to use non-native > attributes > > than appinfo annotations. > > > > > > Comments? > > > > ...mikeb > > > >

2 1