Re: [DFDL-WG] dfdl:calendarPattern does not seem to do anything
I've observed the same problem that Roger observed, and I'm using xs:date. Consider this element: <xsd:element dfdl:calendarCheckPolicy="strict" dfdl:calendarPattern="dd MMM yyyy" dfdl:calendarPatternKind="explicit" ibmDfdlExtn:sampleValue="2001-01-01" name="BadDate" type="xsd:date" /> It seems to me that the input date must exactly match the pattern given (the check policy is strict). If not, then there should be a parse error. Yet MBTK happily converts this bad input: 9 Feb 13 To <BadDate xsi:type="xs:date">13-02-11</BadDate> So either I don't understand the purpose of calendarPattern, or it's not quite working right. (Maybe someone could test this on Daffodil?) From: Steve Hanson [mailto:smh@uk.ibm.com] Sent: Tuesday, March 05, 2013 4:46 AM To: Costello, Roger L. Cc: David Hardcastle; Cranford, Jonathan W.; Garriss Jr., James P. Subject: Re: dfdl:calendarPattern does not seem to do anything Roger, Change your data type from xs:string to xs:date, xs:time or xs:dateTime, make sure that dfdl:calendarPatternKind is 'explicit' and then dfdl:calendarPattern will work. This is all covered in the DFDL spec section 13.11. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group<http://www.ogf.org/dfdl/> IBM SWG, Hursley, UK smh@uk.ibm.com<mailto:smh@uk.ibm.com> tel:+44-1962-815848 From: "Costello, Roger L." <costello@mitre.org<mailto:costello@mitre.org>> To: David Hardcastle/UK/IBM@IBMGB, Steve Hanson/UK/IBM@IBMGB, Cc: "Cranford, Jonathan W." <jcranford@mitre.org<mailto:jcranford@mitre.org>>, "Garriss Jr., James P." <jgarriss@mitre.org<mailto:jgarriss@mitre.org>> Date: 04/03/2013 18:21 Subject: dfdl:calendarPattern does not seem to do anything ________________________________ Hello Steve and David, Here I use the dfdl:calendarPattern for expressing the day of a month: <xsd:element name="Day" type="xsd:string" dfdl:calendarCheckPolicy="strict" dfdl:calendarPattern="dd" dfdl:calendarPatternKind="explicit" /> Note that the pattern says that the input data should consist of 2 digits (dd). However, when I parse this data: xx or this data: 123 the parser says that the data is fine. (Checking the "Validate data against schema" button has no effect) I conclude the dfdl:calendarPattern does not verify that the data conforms to the specified pattern. Further, I would have thought that the specified pattern would inform the parser when it has obtained a token, i.e., I shouldn't have to specify a dfdl:terminator on the element since the pattern should inform the parser. Alas, that is not the case. If I don't specify a terminator (or higher up, a separator) then the parser just keeps consuming characters until it gets to some previously defined terminator or separator. As far as I can tell, dfdl:calendarPattern is non-functional. Thoughts? /Roger Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
James, In your case DFDL is actually using the calendar pattern, but you are encountering a bug somewhere either in IBM DFDL or in the ICU library we use for date/time parsing. If I correct your date to '09 Feb 2013' to match the pattern, then it works ok and gives '2013-02-09' in the infoset. We are currently in dialog with ICU on this subject, as it turns out that there is some leniency in their processing even in strict mode. We have raised a ticket with them to clarify exactly what this leniency is. But even if that explained why '9' is accepted for dd and '13' is accepted for yyyy, that does not explain why the infoset munged this into '13-02-11'. I have raised a defect. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: "Garriss Jr., James P." <jgarriss@mitre.org> To: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Date: 05/03/2013 15:31 Subject: Re: [DFDL-WG] dfdl:calendarPattern does not seem to do anything Sent by: dfdl-wg-bounces@ogf.org I’ve observed the same problem that Roger observed, and I’m using xs:date. Consider this element: <xsd:element dfdl:calendarCheckPolicy="strict" dfdl:calendarPattern="dd MMM yyyy" dfdl:calendarPatternKind="explicit" ibmDfdlExtn:sampleValue="2001-01-01" name="BadDate" type ="xsd:date" /> It seems to me that the input date must exactly match the pattern given (the check policy is strict). If not, then there should be a parse error. Yet MBTK happily converts this bad input: 9 Feb 13 To <BadDate xsi:type="xs:date">13-02-11</BadDate> So either I don’t understand the purpose of calendarPattern, or it’s not quite working right. (Maybe someone could test this on Daffodil?) From: Steve Hanson [mailto:smh@uk.ibm.com] Sent: Tuesday, March 05, 2013 4:46 AM To: Costello, Roger L. Cc: David Hardcastle; Cranford, Jonathan W.; Garriss Jr., James P. Subject: Re: dfdl:calendarPattern does not seem to do anything Roger, Change your data type from xs:string to xs:date, xs:time or xs:dateTime, make sure that dfdl:calendarPatternKind is 'explicit' and then dfdl:calendarPattern will work. This is all covered in the DFDL spec section 13.11. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: "Costello, Roger L." <costello@mitre.org> To: David Hardcastle/UK/IBM@IBMGB, Steve Hanson/UK/IBM@IBMGB, Cc: "Cranford, Jonathan W." <jcranford@mitre.org>, "Garriss Jr., James P." <jgarriss@mitre.org> Date: 04/03/2013 18:21 Subject: dfdl:calendarPattern does not seem to do anything Hello Steve and David, Here I use the dfdl:calendarPattern for expressing the day of a month: <xsd:element name="Day" type="xsd:string" dfdl:calendarCheckPolicy="strict" dfdl:calendarPattern="dd" dfdl:calendarPatternKind="explicit" /> Note that the pattern says that the input data should consist of 2 digits (dd). However, when I parse this data: xx or this data: 123 the parser says that the data is fine. (Checking the "Validate data against schema" button has no effect) I conclude the dfdl:calendarPattern does not verify that the data conforms to the specified pattern. Further, I would have thought that the specified pattern would inform the parser when it has obtained a token, i.e., I shouldn't have to specify a dfdl:terminator on the element since the pattern should inform the parser. Alas, that is not the case. If I don't specify a terminator (or higher up, a separator) then the parser just keeps consuming characters until it gets to some previously defined terminator or separator. As far as I can tell, dfdl:calendarPattern is non-functional. Thoughts? /Roger Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (2)
-
Garriss Jr., James P. -
Steve Hanson