March 2013 - dfdl-wg - lists.ogf.org

OGF DFDL WG Call Minutes 2013-03-05
by Steve Hanson 05 Mar '13

05 Mar '13

Please find minutes from the above call on Redmine at http://redmine.ogf.org/dmsf_files/13014?download= Errata document is now at v012 here: http://redmine.ogf.org/dmsf_files/8104?download= Action 140 errata addendum at v001 here: http://redmine.ogf.org/dmsf_files/13017?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] dfdl:calendarPattern does not seem to do anything
by Garriss Jr., James P. 05 Mar '13

05 Mar '13

I've observed the same problem that Roger observed, and I'm using xs:date. Consider this element: <xsd:element dfdl:calendarCheckPolicy="strict" dfdl:calendarPattern="dd MMM yyyy" dfdl:calendarPatternKind="explicit" ibmDfdlExtn:sampleValue="2001-01-01" name="BadDate" type="xsd:date" /> It seems to me that the input date must exactly match the pattern given (the check policy is strict). If not, then there should be a parse error. Yet MBTK happily converts this bad input: 9 Feb 13 To <BadDate xsi:type="xs:date">13-02-11</BadDate> So either I don't understand the purpose of calendarPattern, or it's not quite working right. (Maybe someone could test this on Daffodil?) From: Steve Hanson [mailto:smh@uk.ibm.com] Sent: Tuesday, March 05, 2013 4:46 AM To: Costello, Roger L. Cc: David Hardcastle; Cranford, Jonathan W.; Garriss Jr., James P. Subject: Re: dfdl:calendarPattern does not seem to do anything Roger, Change your data type from xs:string to xs:date, xs:time or xs:dateTime, make sure that dfdl:calendarPatternKind is 'explicit' and then dfdl:calendarPattern will work. This is all covered in the DFDL spec section 13.11. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group<http://www.ogf.org/dfdl/> IBM SWG, Hursley, UK smh(a)uk.ibm.com<mailto:smh@uk.ibm.com> tel:+44-1962-815848 From: "Costello, Roger L." <costello(a)mitre.org<mailto:costello@mitre.org>> To: David Hardcastle/UK/IBM@IBMGB, Steve Hanson/UK/IBM@IBMGB, Cc: "Cranford, Jonathan W." <jcranford(a)mitre.org<mailto:jcranford@mitre.org>>, "Garriss Jr., James P." <jgarriss(a)mitre.org<mailto:jgarriss@mitre.org>> Date: 04/03/2013 18:21 Subject: dfdl:calendarPattern does not seem to do anything ________________________________ Hello Steve and David, Here I use the dfdl:calendarPattern for expressing the day of a month: <xsd:element name="Day" type="xsd:string" dfdl:calendarCheckPolicy="strict" dfdl:calendarPattern="dd" dfdl:calendarPatternKind="explicit" /> Note that the pattern says that the input data should consist of 2 digits (dd). However, when I parse this data: xx or this data: 123 the parser says that the data is fine. (Checking the "Validate data against schema" button has no effect) I conclude the dfdl:calendarPattern does not verify that the data conforms to the specified pattern. Further, I would have thought that the specified pattern would inform the parser when it has obtained a token, i.e., I shouldn't have to specify a dfdl:terminator on the element since the pattern should inform the parser. Alas, that is not the case. If I don't specify a terminator (or higher up, a separator) then the parser just keeps consuming characters until it gets to some previously defined terminator or separator. As far as I can tell, dfdl:calendarPattern is non-functional. Thoughts? /Roger Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

ISO 8601 restricted profile and optional time zones
by Steve Hanson 05 Mar '13

05 Mar '13

I'd like to discuss allowing time zone to be optional with the 'I' calendar pattern symbol. It looks like the ISO 8601 restricted profile insists that time zone appears if there are time components. See below. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 05/03/2013 10:52 ----- From: Steve Hanson/UK/IBM To: "Costello, Roger L." <costello(a)mitre.org>, Cc: David Hardcastle/UK/IBM@IBMGB, "Cranford, Jonathan W." <jcranford(a)mitre.org>, "Garriss Jr., James P." <jgarriss(a)mitre.org> Date: 05/03/2013 10:51 Subject: Re: MBTK doesn't seem to support xsd:time correctly Roger, XML Schema spec talks about what is allowed in the value space of the XML Schema types, ie, what DFDL creates in its infoset. When parsing, DFDL uses a calendar pattern to identify what calendar components appear in the data, using dfdl:calendarPatternKind and dfdl:calendarPattern properties. If the pattern contains fractional seconds symbols ('SSS'), then at least one fractional second must appear in the data else it is an error. If the pattern contains timezone symbols (eg, 'Z') then a time zone must appear in the data else it is an error. If you have data where the appearance of fractional seconds and/or time zone is optional, you might be able to use the calendar pattern 'I' symbol instead. This allows the data to match the time component of any of the 'granularities' in the ISO 8601 restricted profile ( http://www.w3.org/TR/NOTE-datetime) This will work if your data is of the form HH:mm[:ss[.SSS]][TZD]. Does your data match the ISO8601 restricted profile? Testing the 'I' symbol this today I think there is a bug in MBTK where it only allows either no fractional seconds at all or exactly 3 fractional seconds (it should allow any number). I will raise a defect. Also, although MBTK allows time zone to be missing for 'I', I am not sure this is permitted by the restricted profile which seems to insist on time zone being present. I will raise this with the DFDL Working Group, as it seems to me that allowing time zone to be optional is useful. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: "Costello, Roger L." <costello(a)mitre.org> To: David Hardcastle/UK/IBM@IBMGB, Steve Hanson/UK/IBM@IBMGB, Cc: "Cranford, Jonathan W." <jcranford(a)mitre.org>, "Garriss Jr., James P." <jgarriss(a)mitre.org> Date: 04/03/2013 18:24 Subject: MBTK doesn't seem to support xsd:time correctly Hello Steve and David, Here I declare a time element of type xsd:time: <xsd:element name="Time" type="xsd:time"/> According to the XML Schema specification the xsd:time datatype can optionally specify fractional digits, e.g.: 11:30:30.333 +0500 and timezone is optional: 11:30:30 However, if I use fractional digits I get an error and if I omit timezone I get an error. I looked at the DFDL spec and I don't see any mention about not allowing fractional seconds or requiring timezone. Is this a bug in MBTK? /Roger Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 2

unordered sequence with constrained occurrences
by Garriss Jr., James P. 05 Mar '13

05 Mar '13

Suppose text data has 3 constructs: a, b, and c. * a must occur 1 time * b can occur 0 or 1 time * c can occur any number of times, 0 or more These 3 constructs can appear in any order. So these are valid inputs: abc a bcccca But these are not: ccbcc abbc abcabc Can data like this be modeled with DFDL? The unordered sequence can be modeled with a data array, like this: Array (0 to unbounded) Sequence a (0 to 1) b (0 to 1) c (0 to 1) /Sequence /Array But I don't know how to constrain the total number of occurrences. Appreciate any ideas!

1 0

OGF DFDL WG Call Agenda 2013-03-05
by Steve Hanson 04 Mar '13

04 Mar '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13014?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Can I ignore data I don't want in DFDL?
by Garriss Jr., James P. 01 Mar '13

01 Mar '13

Suppose I am using DFDL to parse email headers. Suppose the RFC only allows 3 headers: To, From, Subject. DFDL can handle this, no problem. But suppose I get an email that includes a 4th header, one I have not planned for (i.e., have not included in the DFDL schema), don't care about, and don't want in the infoset. Like so: From: <john(a)doe.com> To: <jane(a)doe.com> Keywords: sales <-- this line should be ignored! Subject: Latest sales figures Can DFDL handle this? Does it have a mechanism for allowing me to ignore (and thus drop) data I haven't planned for and don't care about?

2 4

Representing multiple spaces
by Garriss Jr., James P. 01 Mar '13

01 Mar '13

Suppose I have a terminator that can be multiple spaces, whether 0 spaces, 1 space, 2 spaces, or more spaces. No other types of whitespace allowed, just spaces. Because there's this entity: %WSP*; I assumed there would also be this entity: %ES*; But there's not. Why not? How would I represent this terminator? TIA

3 5

Can I prevent data from getting into the infoset?
by Garriss Jr., James P. 01 Mar '13

01 Mar '13

Suppose I have an email address with 3 parts, a display name, a local part, and a domain. "James Garriss" <jgarriss(a)mitre.org> Display name = James Garriss Local part = jgarriss Domain = mitre.org DFDL is capable of parsing and validating these as three elements. No problem. But let's suppose that I don't want one of these elements, the display name, to make it into the infoset. I just want to drop it on the floor after I parse it. Question #1: Can I do that? If so, how? Alternatively, suppose I want it in the infoset, but then I want to programmatically remove it from the infoset *before* I unparse it. Question #2: Is that possible? If so, how? TIA

2 1

Re: [DFDL-WG] Can I prevent data from getting into the infoset?
by Garriss Jr., James P. 01 Mar '13

01 Mar '13

Hang on. Is dfdl:hiddenGroupRef the answer to my question below? From: Garriss Jr., James P. Sent: Friday, March 01, 2013 7:43 AM To: dfdl-wg(a)ogf.org Subject: Can I prevent data from getting into the infoset? Suppose I have an email address with 3 parts, a display name, a local part, and a domain. "James Garriss" <jgarriss(a)mitre.org<mailto:jgarriss@mitre.org>> Display name = James Garriss Local part = jgarriss Domain = mitre.org DFDL is capable of parsing and validating these as three elements. No problem. But let's suppose that I don't want one of these elements, the display name, to make it into the infoset. I just want to drop it on the floor after I parse it. Question #1: Can I do that? If so, how? Alternatively, suppose I want it in the infoset, but then I want to programmatically remove it from the infoset *before* I unparse it. Question #2: Is that possible? If so, how? TIA

1 0

is this legal
by Mike Beckerle 01 Mar '13

01 Mar '13

I can't find clarity on this: <sequence> <element name="foo" type="int" dfdl:lengthKind="explicit" dfdl:length="1"/> <element name="bar" type="int" dfdl:lengthKind="explicit" dfdl:length="1"/> <element name="foo" type="int" dfdl:lengthKind="explicit" dfdl:length="1"/> <element name="bar" type="int" dfdl:lengthKind="explicit" dfdl:length="1"/> </sequence> Is this allowed? If so, then the XPaths for accessing the 2nd foo would be foo[2], and the path "foo" would be ambiguous or could be treated as identifying an array. In which case one could do an expression dfdl:occursCount("foo") and get back 2 ?? Or am I completely missing the boat here? -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com

4 4