- dfdl-wg - lists.ogf.org

action 224: add section for implementation defined limits
by Cranford, Jonathan W. 24 Oct '13

24 Oct '13

All, Action item 224 was raised two weeks ago during the WG call. 224 Add section for implementation defined limits (All) 3/9: Several places in the spec cite this, should be grouped. Currently partially listed in section 2.6. Also note distinction between 'implementation defined' and 'implementation dependent'. Check spec for correct usage. Resolve during public comment. The action item was created based on a comment I made during the call, so I thought it'd be good to provide an example of the distinction I was trying to make. The W3C XProc specification does a great job of differentiating between implementation-defined and implementation-dependent features, with a convenient list of each in the appendix. Appendix A (http://www.w3.org/TR/xproc/#conformance) contains the following text. Conformant processors must implement all of the features described in this specification except those that are explicitly identified as optional. Some aspects of processor behavior are not completely specified; those features are either implementation-dependent<http://www.w3.org/TR/xproc/#dt-implementation-dependent> or implementation-defined<http://www.w3.org/TR/xproc/#dt-implementation-defined>. [Definition: An implementation-dependent feature is one where the implementation has discretion in how it is performed. Implementations are not required to document or explain how implementation-dependent<http://www.w3.org/TR/xproc/#dt-implementation-dependent> features are performed.] [Definition: An implementation-defined feature is one where the implementation has discretion in how it is performed. Conformant implementations must document how implementation-defined<http://www.w3.org/TR/xproc/#dt-implementation-defined> features are performed.] Section A.1 then lists all the implementation-defined features, and section A.2 lists all the implementation-dependent features. I think the XProc spec provides a great example to follow on two counts. First, it formally distinguishes between implementation-defined and implementation-dependent features. The choice of terms isn't nearly as important as the distinction itself, of course: implementations must document how certain features are implemented. In the DFDL realm, section 2.6 lists some implementation limits which always constitute schema definition errors; surely these are the types of details that must be documented by any DFDL implementation. Using terminology such as "implementation-defined" and "implementation-dependent" would flag these types of documentation requirements for implementations within the specification. Second, all the implementation-defined and implementation-dependent features are listed in one place in the specification. I think doing the same in the DFDL spec would provide a great resource for DFDL implementers. Comments? If everyone agrees, I don't mind taking the action to search through the document looking for candidates for inclusion in such a list. Sincerely, -- Jonathan W. Cranford Senior Information Systems Engineer The MITRE Corporation (http://www.mitre.org)

2 3

OGF DFDL WG Call Agenda 2013-10-23
by Steve Hanson 22 Oct '13

22 Oct '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13148?download= Note call is on Wednesday this week despite what agenda says! Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Fw: Public comment 116 - Japanese CCSID 943
by Steve Hanson 22 Oct '13

22 Oct '13

Suggested words added to the public comment. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 08/10/2013 10:08 ----- From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 07/10/2013 19:52 Subject: Re: [DFDL-WG] Public comment 116 - Japanese CCSID 943 I suggest that 100% ascii compatible is too strong a statement and we can dial this back. Example: utf-8 will not handle asciiTandemModified style as that style uses the most-significant bit for the 0x80-0x89 negative sign + digit. That is, asciiTandemModified style is making use of extended-ascii codepoints, and there are many flavors of extended ascii with many different assignments of characters to codepoints for this range 80-89. So the question arises of who's in charge, the characters, or the codepoints? I.e., does the zoned sign style even care about the encoding, or does it translate the associated codepoints without concern for their mappings to actual characters. I think the codepoints should be in charge. So long as you have an encoding where all the characters used by your zoned decimals (as digits, or digits with overpunched signs) have legal codepoints in that encoding, you are good to go. If in one charset encoding one of the characters is assigned the euro symbol, and in other charset encodings that same codepoint is assigned some other character, then it should not matter so long as the codepoint doesn't cause a decode error. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Mon, Oct 7, 2013 at 9:30 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: Please see below discussion of the issue raised by public comment 116. While there is a simple workaround, the real issue is that ccsid 943 is in daily use in Japan for zoned decimals and DFDL does not support that. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 07/10/2013 14:05 ----- ICU library treats the ccsid '943' as ICU ibm-943_P130-1999 which is not 100% ASCII compatible due to two code points being different - 0x5C and 0x7E. There is another encoding ICU ibm-943_P15A-2003 which is ASCII compatible, commonly called Shift_JIS. The difference is one that you are probably familiar with - the backslash is replaced by the Yen symbol. Here is the extract from ICU converter site: Internal Converter Name IBM IANA ibm-943_P15A-2003 Shift_JIS MS_Kanji csShiftJIS windows-31j csWindows31J ibm-943_P130-1999 ibm-943 This causes DFDL to reject the ccsid '943' when used on a zoned decimal on the grounds that it is not ASCII-compatible. Why does DFDL do this? It's because it is being safe. So far, we have identified 4 different 'overpunching' schemes for ASCII zoned decimals, and there might well be one or two more used by less common machine architectures: asciiStandard: ASCII characters '0123456789' (0x30-0x39) and 'pqrstuvwxy' (0x70-0x79) for negative sign punch. asciiTranslatedEBCDIC: ASCII characters '{ABCDEFGHI' (0x7B, 0x41-0x49) and '}JKLMNOPQR' (0x7D, 0x4A-0x52) for negative sign punch. asciiCARealiaModified: ASCII characters '0123456789' (0x30-0x39) and '<SP>!"#$%&'()' (0x20-0x29) for negative sign punch. asciiTandemModified: ASCII characters '0123456789' (0x30-0x39) and control characters 0x80-0x89 for negative sign punch. In case other schemes are discovered that use different byte range for negative sign punch (eg, 0x50 to 0x59), the DFDL specification has said that ASCII zoned decimals must be in a 100% ASCII compatible encoding. What we can observe is that ibm-943_P130-1999 is actually safe for representing ASCII zoned decimals in all the above schemes, because the 0x5C and 0x7E characters do not match any of the ranges of bytes used by the schemes. And we can also observe that (apart from the special case of asciiTranslatedEBCDIC) the overpunching schemes simply use the bits in the first nibble of the byte, so any new scheme we discover is very unlikely to affect 0x5C or 0x7E. So, the DFDL specification *could* be changed to treat ibm-943_P130-1999 as ASCII compatible for zoned decimals. The alternative is to use a work around whereby in a DFDL schema that models a data stream in ccsid 943, the default is 943 but zoned decimals override this and use Shift_JIS. When this workaround was discussed with a Japanese user of IBM DFDL, the reaction was "Why do I have to go to that trouble? It should just work in 943." Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Rescheduled: OGF DFDL WG Call (23 Oct 16:00 GDT in DM2046)
by Steve Hanson 21 Oct '13

21 Oct '13

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg Rescheduling as Mike and Jonathan can't make Tuesday this week

1 0

How do I preserve order in an unordered list?
by Garriss Jr., James P. 16 Oct '13

16 Oct '13

(Sent on behalf of Roger Costello) Hi Folks, I am trying out the sequenceKind="unordered" property. I created this simple test: <xs:element name="Test"> <xs:complexType> <xs:sequence dfdl:sequenceKind="unordered" dfdl:separator="%SP;"> <xs:element name="A" type="xs:string" dfdl:initiator="A:" /> <xs:element name="B" type="xs:string" dfdl:initiator="B:" /> </xs:sequence> </xs:complexType> </xs:element> That schema says the input data must consist of A:___ and B:___, in any order. Here is sample input: B:Cat A:Dog I processed the input using the schema and here is the result that I got from Daffodil: <Test> <A>Dog</A> <B>Cat</B> </Test> Notice that the order of the input data changed in the result XML. This was quite surprising to me. Upon consulting the DFDL specification, it appears that the exhibited behavior is expected: ... a DFDL processor must sort the members of an unordered group into schema order when parsing. QUESTIONS: 1. Is this really the desired behavior? I would not expect parsing to alter the order of any data. From my XML Schema experience, I would be shocked if an XML Schema validator altered the order of markup in XML instances simply because the XML Schema specified <all> (unordered sequence). 2. If we grant that this really is the desired behavior, then how do I create an unordered sequence in which DFDL parsing preserves the order of the data? In the above example, if the input data lists B:Cat first and A:Dog second, then how do I get that order preserved in the result XML? /Roger

4 5

Fw: [wg-all] New documents in public comment - last day!
by Steve Hanson 15 Oct '13

15 Oct '13

Last day for public comments on the 3 DFDL documents below... Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 15/10/2013 16:16 ----- From: Greg Newby <gbnewby(a)alaska.edu> To: All WG <wg-all(a)ogf.org>, Date: 16/09/2013 19:37 Subject: [wg-all] New documents in public comment Sent by: wg-all-bounces(a)ogf.org Dear OGF Community: New documents have begun public comment. Please consider reading and providing feedback. Brief affirmations are very useful to demonstrate community input and support. More detailed analysis is, of course, welcome. All documents in public comment are accessible here: http://redmine.ogf.org/projects/editor-pubcom/boards Note that this is an updated location, using OGF's new Redmine tracker system. Anyone may register for a Redmine username. These documents are newly available for public comment: * DFDL v1.0 Revision, recommendation track, via DFDL-WG * DFDL v1.0 Experience, experimental track, via DFDL-WG * DFDL v1.0 Rev. Experience, experimental track, via DFDL-WG This document was previously made available for public comment, and is still available for comment: * Grid Certificate Profile, recommendation track, via CAOPS-WG -- Greg Newby, OGF Standards Editor Dr. Gregory Newby, Director of the Arctic Region Supercomputing Center Univ of Alaska Fairbanks-909 Koyukuk Dr-PO Box 756020-Fairbanks-AK 99775-6020 e: gbnewby(a)alaska.edu v: 907-450-8663 f: 907-450-8603 w: people.arsc.edu/~newby -- The wg-all mailing list is an umbrella list for all OGF mailing lists. If you are subscribed to any OGF mailing list, you will receive mails to wg-all. wg-all mailing list wg-all(a)ogf.org https://www.ogf.org/mailman/listinfo/wg-all Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Cancelled: OGF DFDL WG Call
by Steve Hanson 15 Oct '13

15 Oct '13

Mike not available

1 0

OGF DFDL WG Call Agenda 2013-10-15
by Steve Hanson 15 Oct '13

15 Oct '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13138?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Minutes 2013-10-08
by Steve Hanson 08 Oct '13

08 Oct '13

Please find minutes from the above call at http://redmine.ogf.org/dmsf_files/13136?download= Public Comment The following set are now in Public Comment until 15th October. Revised DFDL 1.0 Specification: https://redmine.ogf.org/issues/117 DFDL Experience Document 1 (Spec Errata): https://redmine.ogf.org/issues/118 DFDL Experience Document 2 (Action 140): https://redmine.ogf.org/issues/119 Please download, review and create comments as necessary. Only 1 week left! Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

bit order - proposed new erratum - response to public comment 43, action item 234
by Mike Beckerle 08 Oct '13

08 Oct '13

Attached is a MS-Word document containing my first cut at an errata to describe adding a new bitOrder property to DFDL. It is more complicated than I would like. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy<http://www.ogf.org/About/abt_policies.php>

2 1