- dfdl-wg - lists.ogf.org

Logistics for upcoming F2F: Location and Hotels
by mike.beckerle＠ascentialsoftware.com 20 Apr '05

20 Apr '05

<http://tinyurl.com/9t8oq> http://tinyurl.com/9t8oq is a URL to a mapquest page giving the location of our building. From there you can search for nearby hotels. I'm told the Wyndham is both nice and is quite nearby and has IBM rates for IBMers. You will need to rent a car at the airport. ...mike beckerle

1 0

Notes from call 2005-04-13
by mike.beckerle＠ascentialsoftware.com 14 Apr '05

14 Apr '05

Here's my notes: DFDL WG Semi-Weekly Call 2005-04-13 Attendees Jim Meyers Tara Talbot Steve Hanson Mike Beckerle Karen Loughran Agenda 1) Scope of the v1.0 specification (not to be confused with "scoping of annotations") 2) Discussion of scoping of annotations based on Ascential posting. 3) Planning of agenda for upcomign F2F meeting on May 11.12, 13, at Ascential in Westborough, MA Suggested topics so far: - Review of TD Model and extension to cover DFDL properties - Examples from the Ascl prototype - go through several line by line to see what we do and don't like about them- F2F Who's coming: Mike Beckerle Jim Meyers Tara Talbot Suman Kalia Steve Hanson Who's not coming: Martin Westhead Karen Loughran Scope of v1.0 Spec. - Steve Hanson - don't bite off too much for v1.0 - QSAM formats with data split over other pieces. - Encrypted sections and how we handle those. - DFDL Validation i.e., check that the binary data is valid according to the DFDL Schema. (It's alot of work to do a validating parser. EDIFACT, X12 and such require more complex validation than just. Things like field a if present means field b should NOT be present.) Jim Meyers - multi-layer stuff is what supports those things. Scoping rules discussion - Defer discussion - Agenda for F2F - - model and parser behavior aspects: rules for default values, null handling, how to figure out how to figure ot the length precedence rules, etc. - open content - tagged, but some of the tags are unknown. HL7 people put extra fields into a group. Happens with every format though, not just tags. Polymorphic chunks. xsd:all - unordered data. - PNNL project and Jaxme implementation - here's what's on the ground. Focus should be on how that's changing the language. - Scoping of annotations - extensibility mechanisms (low prio) - multi-dimensional arrays (low prio) - reader/writier/filter/function - topic - don't bother with a deep dive on any implementation.

1 0

RE: [dfdl-wg] DFDL subset of XML schema
by mike.beckerle＠ascentialsoftware.com 13 Apr '05

13 Apr '05

Hi Steve, Yes, we're not really proposing that subset for the "final" language, rather we're trying to be as minimalist as we can up front to facilitate the prototype. I think it's fine if the prototype is a subset of what we specify for v1.0. I just don't want it to be inconsistent with it. The prototype is "non-normative" in w3c lingo, so there will be two documents. One describing what the prototype does and implements, and the draft standard document can be different. Per your point 2 below. Let's split this into "single top level global element" and the others. The others cause no issue far as I can tell. In fact in our prototype it turns out that we don't even see any difference between an element with anonymous type and an element reference. The code just sees an element with name, type, etc. So those are easy to support. Attributes are easily supported also. We need to add a flag bit to our prototype that's all. The single top level issue is a tiny bit deeper. In XML you can get away with more than one global top level element because the documents are always tagged to make it unambiguous which one of the possible global top level elements describes the file. In DFDL we need specific information about which of the possible global element declarations applies to the actual file since there may be nothing in the data which makes it clear. We could do this with an annotation that indicates "this is the one that actually applies to the file". What we did in the prototype is just require there be only one global element declaration to make this unambiguous. The primary reason we left out element references is to be minimal. There's nothing you can do with them that you can't do with a type definition and an ordinary element declaration, so they seem simply unnecessary, and that solved our ambiguity problem too. Resusable groups - yes these are easily supported also modulo that they can have separate minOccurs/maxOccurs at point of use. Again I think our prototype never even sees them. The EMF XSD library essentially forward substitutes them for us so our code never deals with them. Right now we'd miss any additional min/max occurs information though, so that is a bug. Re: hexbinary - I don't understand your use of hexbinary. Can you clarify? Other simple types: yes we could put all of the date types in. We just chose to keep it minimal. I left out the obscure 'date fragment' types because I've never seen data containing things like that, but it's a very minor thing. If you think we need them, then we need them. However I would argue against putting in things for the sake of having more of XSD "covered". Substitution groups - I agree these could be a pseudo choice construct, but I prefer to make an XSD subset and be explicit about it being a subset rather than go for a way to assign meaning to everything in XSD. -----Original Message----- From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] On Behalf Of Steve Hanson Sent: Wednesday, April 13, 2005 6:59 AM To: dfdl-wg(a)gridforum.org Subject: [dfdl-wg] DFDL subset of XML schema Mike, looking at your proposal working draft, I don't agree with the DFDL subset you are proposing. I think it is too restrictive. Specifically: 1) xsd:all - we have discussed this as part of the unordered mail exchange last week so I think we now agree this is needed. 2) Single top-level global element, global attributes, element references, attribute references. This prevents re-use. 3) Reusable groups. Ditto. 4) Simple type hexBinary. This is the MRM model's default mapping for binary data. 5) Other simple types. Some of these could be discussed - eg, the date restrictions. 6) Substitution groups. We basically treat these a choice in non-XML data. But I would be ok with deferring support post 1.0. Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848

2 1

Call Today
by mike.beckerle＠ascentialsoftware.com 13 Apr '05

13 Apr '05

Reminder: we have our semi-weekly call today. Here's some agenda items I've gathered from emails and such: 1) Scope of the v1.0 specification (not to be confused with "scoping of annotations") 2) Discussion of scoping of annotations based on Ascential posting. 3) Planning of agenda for upcomign F2F meeting on May 11.12, 13, at Ascential in Westborough, MA Suggested topics so far: - Review of TD Model and extension to cover DFDL properties - Examples from the Ascl prototype - go through several line by line to see what we do and don't like about them- Talk to you at 9am US.PDT, Noon US.EDT, 17h00 UK Mike Beckerle Ascential Software

2 1

DFDL subset of XML schema
by Steve Hanson 13 Apr '05

13 Apr '05

Mike, looking at your proposal working draft, I don't agree with the DFDL subset you are proposing. I think it is too restrictive. Specifically: 1) xsd:all - we have discussed this as part of the unordered mail exchange last week so I think we now agree this is needed. 2) Single top-level global element, global attributes, element references, attribute references. This prevents re-use. 3) Reusable groups. Ditto. 4) Simple type hexBinary. This is the MRM model's default mapping for binary data. 5) Other simple types. Some of these could be discussed - eg, the date restrictions. 6) Substitution groups. We basically treat these a choice in non-XML data. But I would be ok with deferring support post 1.0. Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848

1 0

Conference call this week?
by mike.beckerle＠ascentialsoftware.com 11 Apr '05

11 Apr '05

Do we have our semi-weekly call this week? For no explicable reason (i.e., user error) all the entries for this have dropped off my calendar so I'm not sure if it's this week or next. ...mikeb Mike Beckerle Ascential Software

2 1

RE: [dfdl-wg] May Face to Face Meeting
by Myers, James D 11 Apr '05

11 Apr '05

Works for me... Jim -----Original Message----- From: owner-dfdl-wg(a)ggf.org [mailto:owner-dfdl-wg@ggf.org] On Behalf Of mike.beckerle(a)ascentialsoftware.com Sent: Friday, April 08, 2005 12:49 PM To: dfdl-wg(a)gridforum.org Subject: [dfdl-wg] May Face to Face Meeting We tried to coordinate Toronto, but it was just a bad week at IBM facilities there, so rather than move the dates we've decided to go with Plan B, which is to hold the F2F meeting at Ascential in Massachusetts instead. This also simplifies life for those for whom international travel is more complicated. Here's my proposed details: Dates: May 11, 12, 13-to mid afternoon. That's a wednesday, thursday, friday. Location: Ascential Westborough, MA (www.ascential.com) Please respond if you can/can't attend, or really-really want to, but can't make these dates work. For specific topics on the agenda it will be possible for us to arrange conference-calls. As we work out the agenda we'll see if this is needed.

1 0

May Face to Face Meeting
by mike.beckerle＠ascentialsoftware.com 08 Apr '05

08 Apr '05

We tried to coordinate Toronto, but it was just a bad week at IBM facilities there, so rather than move the dates we've decided to go with Plan B, which is to hold the F2F meeting at Ascential in Massachusetts instead. This also simplifies life for those for whom international travel is more complicated. Here's my proposed details: Dates: May 11, 12, 13-to mid afternoon. That's a wednesday, thursday, friday. Location: Ascential Westborough, MA (www.ascential.com <http://www.ascential.com> ). Please respond if you can/can't attend, or really-really want to, but can't make these dates work. For specific topics on the agenda it will be possible for us to arrange conference-calls. As we work out the agenda we'll see if this is needed.

2 1

binary xml document from w3c
by mike.beckerle＠ascentialsoftware.com 07 Apr '05

07 Apr '05

http://www.w3.org/TR/2005/NOTE-xbc-characterization-20050331/ <http://www.w3.org/TR/2005/NOTE-xbc-characterization-20050331/> This is dated 31 March 2005. This is just FYI. Since I just ran across this. ...mikeb

1 0

tagged data examples
by mike.beckerle＠ascentialsoftware.com 07 Apr '05

07 Apr '05

DFDLers, Suman and I were discussing a particular data format problem that I undertook to solve in DFDL. I thought it would be good to bring the discussion to the whole group. The problem is tagged data fields. Many computer messaging formats use things like this for some or all of their data fields. Here are two records containing a first and last name in a tagged format: fname:Tim!LName:Stewart; LASTNAME:Smith!firstName:Tom; Notes: the tags have varying forms, i.e., fname, firstname firstName, FIRSTNAME, FNAME, all are accepted as the tag for the first name field. The definition here is that it is case insensitive and either fname or firstname forms. Similar for lastname. Also the tagged fields can appear in any order, and are optional. Here's my test file showing how the XML comes out: (I've attached these as files also in case the email system hammers them.) This is testTaggedData1.xml <?xml version="1.0" encoding="iso-8859-1"?>  <dfdlTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://dataformat.org/testCase" xmlns:tc="http://dataformat.org/testCase" xsi:schemaLocation="http://dataformat.org/testCase ../../xsd/testCase.xsd http://dataformat.org/tests testTaggedData1.dfdl.xsd"> <inputTest>  <data kind="text">fname:Tim!LName:Stewart; LASTNAME:Smith!firstName:Tom; </data> <dfdlSchema file="testTaggedData1.dfdl.xsd"/> <tc:xmlResult xmlns="http://dataformat.org/tests"> <myData> <custInfo> <firstName>Tim</firstName> <lastName>Stewart</lastName> </custInfo> <custInfo> <firstName>Tom</firstName> <lastName>Smith</lastName> </custInfo> </myData> </tc:xmlResult> </inputTest> </dfdlTest> Now, the DFDL itself There are 3 variants here. I'll start with the simplest one. You get a very simple DFDL for this if you assume you can have (a) a way to specify the values of initiators, terminators, and separators as regular expressions (b) support for xsd:all groups. This is testTaggedData3.dfdl.xsd (ok, this one has long lines, so the email system is sure to hammer it, so I won't inline it here.) I think this particular example is pretty straightforward. However, I have two other example DFDL schemas for this which make fewer assumptions. testTaggedData2.dfdl.xsd still allows one to specify regular expressions for the initiator rep property, but does not allow use of xsd:all. Which is a construct I *was* trying to avoid because, well, it's complicated and feels non-primitive. I think you'll agree the complexity goes up significantly. testTaggedData1.dfdl.xsd eliminates specifying the initiator at all, and specifies the tags by way of an additional field hidden in a hidden-layer which has value constrained by an XSD pattern facet to match a specific regular expression. It also does not use xsd:all. My summary from going through this exercise: We need both xsd:all support, and regular expressions for initiators, and all delimiters. I'm happy that one can express these things without needing these constructs, but tagged representations are too commonplace for this much complex construction to be required. The complex constructions I used in testTaggedData1 and testTaggedData2 would only be needed if the tags were complex formatted entities the format of which couldn't be handled by a regular expression. Note that if the tags are actually not case insensitive, but are really fixed strings, then there is no need for the regular expression capability. I'm not sure where we should draw the line here. I'm comfortable with xsd:all support, and plain strings as delimiters or with regexps as delimiters. ...mikeb

2 1