- dfdl-wg - lists.ogf.org

RE: [dfdl-wg] Conference Call Today - notes
by Myers, James D 02 Mar '05

02 Mar '05

Sorry for the late FYI on this: If we have a meeting in Toronto (anywhere outside the US), PNNL staff will need 45-60 days to get clearance for the foreign travel. That would mean making decisions about dates and specific location as soon as possible for us to be involved. (I would hope to have at least one of us and probably two of us there.) Jim > -----Original Message----- > From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] On > Behalf Of mike.beckerle(a)ascentialsoftware.com > Sent: Wednesday, February 16, 2005 12:45 PM > To: dfdl-wg(a)gridforum.org > Subject: [dfdl-wg] Conference Call Today - notes > > > > DFDL-WG Conference call 2005-02-16 > > Attendees: Suman Kalia, Steve Hanson, Susan Malaika - IBM, > Tara Talbot - PNNL, Mike Beckerle - Ascential Software > > IBM offered to host a F2F meeting in mid to late April, or > early May at their Toronto facility. Seems like a good time > frame. Gives Ascential folks time time to get prototype "in > play", and leaves time before the June GGF > (Chicago) to get some things written up. > > MikeB and Suman to schedule time to work on combined model > and list of representation properties: week of Feb 28th. > > Ascential prototype: hoping to have simplest examples working > by end of this week. (No promises). Things like 2 binary > integers. MikeB to post current XSD for dfdl properties and a > few of our test files for the prototype. > > Susan asked about whether implementations of things like BFD > are available. Tara to investigate whether BFD is publicly > available now. > > We discussed when to start recruiting Oracle, Microsoft, and > other participants, since we will need them. We agreed that > our hope would be the June GGF though it really depends on > having an initial document representing some sort of > converged consensus of the current team on current directions. > > > Mike Beckerle > Ascential > >

2 1

Fw: [dfdl-wg] alternate syntax for DFDL annotations
by Suman Kalia 02 Mar '05

02 Mar '05

Hi Mike -- We will be creating XML schema to model our annotations and the attributes applicable to the representation of a particular element/type will be grouped within the complex type of the annotation schema. My preference will be use annotation approach consistently in DFDL. Moreover the attributes defined in the compact form are not easily accessible from the XML Schema APIs, you have to look into the DOMNode associated with the XSDElementDeclaration to get access to such attributes; it makes the programming task a little harder. Having said that SOAP encoded schemas use this compact form to identify the type and dimensions of the array types; some folks like this approach and some hate it.. Suman Kalia IBM Toronto Lab WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 03/02/2005 12:38 PM ----- Mike Beckerle <mike.beckerle(a)ascentialsoftware.com> Sent by: owner-dfdl-wg(a)ggf.org 03/01/2005 03:53 PM To dfdl-wg(a)gridforum.org cc Subject [dfdl-wg] alternate syntax for DFDL annotations Up til now we've considered DFDL annotations only as inside the appinfo context. However, we should consider whether we should use non-native attributes as well or as an alternative: E.g., simple DFDL rep properties could also be expressed like this: <xs:element name="foo" type="xs:string" dfdl:repType="text" dfdl:charset="UTF-8" dfdl:repLength="10"/> This has the advantages of compactness, and is a fully supported way of extending XML Schema. That is, using non-native attributes is a supported extension idiom. This won't handle things that really need the syntactic support of element structure to express their complexity, e.g., things like specifying text delimiters with quoting and escape-sequence specifications. For those we'll still need to open an appinfo annotation up. However, for basic things like byteOrder and such it is far more attractive syntactically to use non-native attributes than appinfo annotations. Comments? ...mikeb

1 0

RE: [dfdl-wg] alternate syntax for DFDL annotations
by Myers, James D 02 Mar '05

02 Mar '05

I agree - I think I like #2. I know at one point in the past we had complaints when we tried to use attributes with namespaces - it may just have been that parsers weren't dealing with them correctly early on. Whatever it was, option 2 would provide a work around as well as helping us keep a single 'canonical' form for DFDL. Jim > -----Original Message----- > From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] On > Behalf Of Martin Westhead > Sent: Wednesday, March 02, 2005 12:16 PM > To: Mike Beckerle > Cc: dfdl-wg(a)gridforum.org > Subject: Re: [dfdl-wg] alternate syntax for DFDL annotations > > > Hi Mike, > > This is interesting I wasn't aware you could do this. My > concern would > be if it turned out the schema tools couldn't handle it but I > tried XML > Spy which seems to just ignore the extra annotations. > > So I'd be happy to use them. As you point out we will need both these > and annotations so we are left with a choice. Either we: > > 1. Specify some (most) dfdl additions in this non-native attribute > form and use annotations when required or > > 2. We specify everything as annotations but allow this form as a > systematic short hand. > > I think I prefer (2). The sort of thing I have in mind is making the > annotation syntax look like: > > <xs:element name="foo" type="xs:string"> > <annotation> > <appinfo> > <dfdl:attributes dfdl:repType="text" > dfdl:charset="UTF-8" > dfdl:repLength="10"/> > </appinfo> > <annotation > </xs:element> > > And then this be directly equivalent (and trivially transformed > into/from) what you had before: > > <xs:element name="foo" > type="xs:string" > dfdl:repType="text" > dfdl:charset="UTF-8" > dfdl:repLength="10"/> > > I'm not sure if it matters that much but it seems like it might be > neater if everything could be represented in the same form. > > Also there may be times when you would like to use the > annotation form > because you have a bock of stuff that could be reused multiple times. > Once your annotations have become attributes in the main > schema they are > more intimately tied to it. > > Cheers, > > Martin > > > > Mike Beckerle wrote: > > Up til now we've considered DFDL annotations only as inside the > > appinfo > > context. However, we should consider whether we should use > non-native > > attributes as well or > > > > as an alternative: E.g., simple DFDL rep properties could also be > > expressed like this: > > > > <xs:element name="foo" > > > > type="xs:string" > > > > dfdl:repType="text" > > > > dfdl:charset="UTF-8" > > > > dfdl:repLength="10"/> > > > > > > > > This has the advantages of compactness, and is a fully > supported way > > of > > extending XML Schema. That is, using non-native attributes is a > > supported extension idiom. This won't handle things that > really need the > > syntactic support of element structure to express their complexity, > > e.g., things like specifying text delimiters with quoting and > > escape-sequence specifications. For those we'll still need > to open an > > appinfo annotation up. However, for basic things like > byteOrder and such > > it is far more attractive syntactically to use non-native > attributes > > than appinfo annotations. > > > > > > Comments? > > > > ...mikeb > > > >

2 1

DFDL call notes
by Myers, James D 02 Mar '05

02 Mar '05

FYI - quick notes about a short call today - Steve, Tara, and I hung out on the DFDL call today for about 20 minutes to see if we'd get a quorum. We didn't. We noted that the availability of the prototype from Ascential seems to be a pre-requisite for further discussions about the reader/writer/filter/function ideas and resolving the issues around transform versus sub-type centric approaches. (I'm glad its not me in the critical path this time!) In any case, I think we agreed that we need the prototype, or some description of it, soon to get the discussion moving again and make another productive f-2-f possible. Jim

1 0

alternate syntax for DFDL annotations
by Mike Beckerle 02 Mar '05

02 Mar '05

Up til now we've considered DFDL annotations only as inside the appinfo context. However, we should consider whether we should use non-native attributes as well or as an alternative: E.g., simple DFDL rep properties could also be expressed like this: <xs:element name="foo" type="xs:string" dfdl:repType="text" dfdl:charset="UTF-8" dfdl:repLength="10"/> This has the advantages of compactness, and is a fully supported way of extending XML Schema. That is, using non-native attributes is a supported extension idiom. This won't handle things that really need the syntactic support of element structure to express their complexity, e.g., things like specifying text delimiters with quoting and escape-sequence specifications. For those we'll still need to open an appinfo annotation up. However, for basic things like byteOrder and such it is far more attractive syntactically to use non-native attributes than appinfo annotations. Comments? ...mikeb

2 1

How to handle multi-dimensional arrays
by mike.beckerle＠ascentialsoftware.com 02 Mar '05

02 Mar '05

We have come up with an approach to how to represent multi-dimensional arrays within XSD-described XML. The attached test file (.xml) and DFDL Schema (.dfdl.xsd) illustrate the proposed solution. The proposal does not require any changes to XSD, XML or any other special constructs outside of a single dfdl annotation to specify the storage order of the representation. I'm pretty happy with how this works out. We can handle arrays with different storage orders, like fortran style column-major vs. more common row-major, and it dovetails nicely with XPath expressions and the XSD data model. Schema validation can really do something for you, like tell you if you have all the elements of the array (if it's fixed size), and that you don't have multiple elements occupying the same array location. Those interested in multi-dimensional array support please give this some consideration. That said, I'm departing on vacation for a week, so I'll toss this out there for people to look at, but I won't be able to interact with you all on it until I get back. ...mikeb

2 1

BFD Parser Engine
by Talbott, Tara D 22 Feb '05

22 Feb '05

During the last meeting Susan inquired about the implementation of the BFD parser. Here is a link to the BFD homepage which describes the project and specifications, http://collaboratory.emsl.pnl.gov/sam/bfd/. The parser engine is available from that page, but we are still investigating the status of the open source license so the source is not yet available, I will notifiy you when it is. Thanks, Tara

1 0

RE: [dfdl-wg] DFDL Schema and test case format for forthcoming pr ototype
by Mike Beckerle 16 Feb '05

16 Feb '05

>From the one test case I included you can't see this, but the <data kind=...> element allows whitespace insertion and ignores all whitespace. E.g., <data kind="bits"> 0 1000000 10000000 00000000 00000000 0 1000001 01010100 00000000 00000000 0 0111111 11001100 11001100 11001100 </data> Is accepted as is: <data kind="hex"> 0095 35 0000000000000048 222038109f e2e4d5d940d3c5c140c9d540 000f 0095 35 0000000951500501 088322722f d2e4d9e3e94b4bf0d9f64b4b 002f </data> We did this since most hex dump tools will space things out a bit, and the spaces help clarity. There's also a <data kind="text"> variant which is different since it is whitespace preserving. ...mikeb > -----Original Message----- > From: mike.beckerle(a)ascentialsoftware.com > [mailto:mike.beckerle@ascentialsoftware.com] > Sent: Wednesday, February 16, 2005 2:06 PM > To: dfdl-wg(a)gridforum.org > Subject: [dfdl-wg] DFDL Schema and test case format for > forthcoming prototype > > > I posted schemas and a single test case file up at grid forge > as a way of letting people know a little more about what > we're up to at Ascential. > Here's the zip file of them: > > http://forge.gridforum.org/projects/dfdl-wg/document/ascl-info > -impl-2005-02- > 16.zip/en/1. > > It's a grid forge thing that this will download as a > ".zip.unknown" file, requiring you to rename it. > > We've written a few more test files than are included here, > covering Jim Myers's examples from last summer, and then > some, and they validate as XML under a few different > validators, but nothing is running yet to process them. The > enclosed example in the zip file is the one we're hoping to > make work this week. > > So the simple way to understand our prototype is that it runs > tests like these, and is otherwise intended to be easy to > change as the specs evolve, and is focused on simplicity and > clarity, not speed. > > Our goal with these testCase files is that you can get a hex > dump of some interesting data, paste it into a test case > document like this, and thereby easily create self-contained tests. > > Here's the README.txt contents: > > This directory contains test files in the xml/tests > subdirectory, and common schemas in the xsd subdirectory. > > xml/tests/testSimpleBinaryIntegers1.dfdl.xsd - DFDL annotated > XML schema for very simple data. 2 binary integers > xml/tests/testSimpleBinaryIntegers1.xml - test case file. > References the schema, includes hexified binary data, and > result data for comparison. > > xsd/dfdl.xsd and .html - the schema for the DFDL annotations > xsd/testCase.xsd and .html - the schema for the test files > > The html documentation is generated by a tool we use called > Oxygen XML. > > XMLSchema.xsd - These four are used in validating DFDL > annotated schemas to insure they are valid XSD. > XMLSchema.dtd - These are all available from the web, but are > included here for speed reasons. > datatypes.xsd > datatypes.dtd >

1 0

DFDL Schema and test case format for forthcoming prototype
by mike.beckerle＠ascentialsoftware.com 16 Feb '05

16 Feb '05

I posted schemas and a single test case file up at grid forge as a way of letting people know a little more about what we're up to at Ascential. Here's the zip file of them: http://forge.gridforum.org/projects/dfdl-wg/document/ascl-info-impl-2005-02- 16.zip/en/1. It's a grid forge thing that this will download as a ".zip.unknown" file, requiring you to rename it. We've written a few more test files than are included here, covering Jim Myers's examples from last summer, and then some, and they validate as XML under a few different validators, but nothing is running yet to process them. The enclosed example in the zip file is the one we're hoping to make work this week. So the simple way to understand our prototype is that it runs tests like these, and is otherwise intended to be easy to change as the specs evolve, and is focused on simplicity and clarity, not speed. Our goal with these testCase files is that you can get a hex dump of some interesting data, paste it into a test case document like this, and thereby easily create self-contained tests. Here's the README.txt contents: This directory contains test files in the xml/tests subdirectory, and common schemas in the xsd subdirectory. xml/tests/testSimpleBinaryIntegers1.dfdl.xsd - DFDL annotated XML schema for very simple data. 2 binary integers xml/tests/testSimpleBinaryIntegers1.xml - test case file. References the schema, includes hexified binary data, and result data for comparison. xsd/dfdl.xsd and .html - the schema for the DFDL annotations xsd/testCase.xsd and .html - the schema for the test files The html documentation is generated by a tool we use called Oxygen XML. XMLSchema.xsd - These four are used in validating DFDL annotated schemas to insure they are valid XSD. XMLSchema.dtd - These are all available from the web, but are included here for speed reasons. datatypes.xsd datatypes.dtd

1 0

Reposting of minutes from Dec 2004 F2F
by Mike Beckerle 16 Feb '05

16 Feb '05

Susan said she couldn't find these minutes on gridforge. When I doublechecked, I couldn't find them either so I reposted them. This URL works for me. http://forge.gridforum.org/projects/dfdl-wg/document/ggf-dfdl-wg-2004-12-06- meeting-summary002_/en/1.

1 0