- dfdl-wg - lists.ogf.org

IBM DFDL error messages
by Steve Hanson 31 Oct '13

31 Oct '13

IBM is sharing the error messages that it issues for schema definition errors in order to assist other DFDL implementations. Attached are the IBM DFDL model component and validator component error messages. Between them they should cover all SDEs, except those caused by violation of XML 1.0 and base XML Schema 1.0 rules - we just echo what we get back from the EMF code. Notice that the messages all have a consistent structure for the text. However considerable rationalisation is possible if one combines error messages that are essentially the same and use appropriate inserts. If the DFDL WG agrees that is the right thing to do then IBM can do this and re-distribute. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

OGF DFDL WG Call Minutes 2013-10-31
by Steve Hanson 31 Oct '13

31 Oct '13

Please find minutes from the above call at http://redmine.ogf.org/dmsf_files/13150?download= Public Comment for the revised DFDL 1.0 specification is now finished. Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Agenda 2013-10-31
by Steve Hanson 30 Oct '13

30 Oct '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13150?download= Note call is on Thursday this week at 3pm UK (11am Eastern). Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Confirmed: OGF DFDL WG Call (31 Oct 15:00 GMT in DE3S09)
by Steve Hanson 30 Oct '13

30 Oct '13

Confirmed: OGF DFDL WG Call 31/10/2013 - (Repeats) Location: DE3S09 Steve Hanson has confirmed this meeting Repeats This entry repeats Required: jcranford(a)mitre.org, mbeckerle(a)tresys.com, mbeckerle.dfdl(a)gmail.com, Suman Kalia/Toronto/IBM@IBMCA, Tim Kimber/UK/IBM@IBMGB Optional: dfdl-wg(a)ogf.org, Stephanie Fetzer/Charlotte/IBM@IBMUS OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg

1 0

Rescheduled: OGF DFDL WG Call (31 Oct 15:00 GMT in DE3S09)
by Steve Hanson 25 Oct '13

25 Oct '13

Rescheduled: OGF DFDL WG Call 31/10/2013 - (Repeats) Location: DE3S09 Steve Hanson has rescheduled this meeting. You have not added this entry to your calendar. Repeats This entry repeats Required: jcranford(a)mitre.org, mbeckerle(a)tresys.com, mbeckerle.dfdl(a)gmail.com, Suman Kalia/Toronto/IBM@IBMCA, Tim Kimber/UK/IBM@IBMGB Optional: dfdl-wg(a)ogf.org, Stephanie Fetzer/Charlotte/IBM@IBMUS OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg

1 0

FW: [Datascience-list] NIST Data Science Symposium RESCHEDULED to March 4-5, 2014
by Cranford, Jonathan W. 24 Oct '13

24 Oct '13

FYI. -Jonathan -----Original Message----- From: datascience-list-bounces(a)nist.gov [mailto:datascience-list-bounces@nist.gov] On Behalf Of Greenberg, Craig S Sent: Thursday, October 24, 2013 8:57 AM To: datascience-list Cc: datascience Subject: [Datascience-list] NIST Data Science Symposium RESCHEDULED to March 4-5, 2014 Dear Data Science Community, The NIST Data Science Symposium has been rescheduled to March 4-5, 2014. The difficult decision to reschedule the event is due to several unforeseen circumstances. We hope that you will be available to participate in the rescheduled symposium, and we are deeply sorry for any inconvenience this may cause. Abstracts that have been submitted will still be considered, and updated versions as well as new submissions will be welcome. For those who already registered to attend the symposium, your registration will remain active. If you need to withdraw your submitted abstract or cancel your registration, please send an email to datascience(a)nist.gov. More information will be made available in the near future. In the meantime, please feel free to send any questions you have to datascience(a)nist.gov.

1 0

OGF DFDL WG Call Minutes 2013-10-23
by Steve Hanson 24 Oct '13

24 Oct '13

Please find minutes from the above call at http://redmine.ogf.org/dmsf_files/13148?download= Public Comment for the revised DFDL 1.0 specification is now finished. Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

action 224: add section for implementation defined limits
by Cranford, Jonathan W. 24 Oct '13

24 Oct '13

All, Action item 224 was raised two weeks ago during the WG call. 224 Add section for implementation defined limits (All) 3/9: Several places in the spec cite this, should be grouped. Currently partially listed in section 2.6. Also note distinction between 'implementation defined' and 'implementation dependent'. Check spec for correct usage. Resolve during public comment. The action item was created based on a comment I made during the call, so I thought it'd be good to provide an example of the distinction I was trying to make. The W3C XProc specification does a great job of differentiating between implementation-defined and implementation-dependent features, with a convenient list of each in the appendix. Appendix A (http://www.w3.org/TR/xproc/#conformance) contains the following text. Conformant processors must implement all of the features described in this specification except those that are explicitly identified as optional. Some aspects of processor behavior are not completely specified; those features are either implementation-dependent<http://www.w3.org/TR/xproc/#dt-implementation-dependent> or implementation-defined<http://www.w3.org/TR/xproc/#dt-implementation-defined>. [Definition: An implementation-dependent feature is one where the implementation has discretion in how it is performed. Implementations are not required to document or explain how implementation-dependent<http://www.w3.org/TR/xproc/#dt-implementation-dependent> features are performed.] [Definition: An implementation-defined feature is one where the implementation has discretion in how it is performed. Conformant implementations must document how implementation-defined<http://www.w3.org/TR/xproc/#dt-implementation-defined> features are performed.] Section A.1 then lists all the implementation-defined features, and section A.2 lists all the implementation-dependent features. I think the XProc spec provides a great example to follow on two counts. First, it formally distinguishes between implementation-defined and implementation-dependent features. The choice of terms isn't nearly as important as the distinction itself, of course: implementations must document how certain features are implemented. In the DFDL realm, section 2.6 lists some implementation limits which always constitute schema definition errors; surely these are the types of details that must be documented by any DFDL implementation. Using terminology such as "implementation-defined" and "implementation-dependent" would flag these types of documentation requirements for implementations within the specification. Second, all the implementation-defined and implementation-dependent features are listed in one place in the specification. I think doing the same in the DFDL spec would provide a great resource for DFDL implementers. Comments? If everyone agrees, I don't mind taking the action to search through the document looking for candidates for inclusion in such a list. Sincerely, -- Jonathan W. Cranford Senior Information Systems Engineer The MITRE Corporation (http://www.mitre.org)

2 3

OGF DFDL WG Call Agenda 2013-10-23
by Steve Hanson 22 Oct '13

22 Oct '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13148?download= Note call is on Wednesday this week despite what agenda says! Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Fw: Public comment 116 - Japanese CCSID 943
by Steve Hanson 22 Oct '13

22 Oct '13

Suggested words added to the public comment. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 08/10/2013 10:08 ----- From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 07/10/2013 19:52 Subject: Re: [DFDL-WG] Public comment 116 - Japanese CCSID 943 I suggest that 100% ascii compatible is too strong a statement and we can dial this back. Example: utf-8 will not handle asciiTandemModified style as that style uses the most-significant bit for the 0x80-0x89 negative sign + digit. That is, asciiTandemModified style is making use of extended-ascii codepoints, and there are many flavors of extended ascii with many different assignments of characters to codepoints for this range 80-89. So the question arises of who's in charge, the characters, or the codepoints? I.e., does the zoned sign style even care about the encoding, or does it translate the associated codepoints without concern for their mappings to actual characters. I think the codepoints should be in charge. So long as you have an encoding where all the characters used by your zoned decimals (as digits, or digits with overpunched signs) have legal codepoints in that encoding, you are good to go. If in one charset encoding one of the characters is assigned the euro symbol, and in other charset encodings that same codepoint is assigned some other character, then it should not matter so long as the codepoint doesn't cause a decode error. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Mon, Oct 7, 2013 at 9:30 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: Please see below discussion of the issue raised by public comment 116. While there is a simple workaround, the real issue is that ccsid 943 is in daily use in Japan for zoned decimals and DFDL does not support that. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 07/10/2013 14:05 ----- ICU library treats the ccsid '943' as ICU ibm-943_P130-1999 which is not 100% ASCII compatible due to two code points being different - 0x5C and 0x7E. There is another encoding ICU ibm-943_P15A-2003 which is ASCII compatible, commonly called Shift_JIS. The difference is one that you are probably familiar with - the backslash is replaced by the Yen symbol. Here is the extract from ICU converter site: Internal Converter Name IBM IANA ibm-943_P15A-2003 Shift_JIS MS_Kanji csShiftJIS windows-31j csWindows31J ibm-943_P130-1999 ibm-943 This causes DFDL to reject the ccsid '943' when used on a zoned decimal on the grounds that it is not ASCII-compatible. Why does DFDL do this? It's because it is being safe. So far, we have identified 4 different 'overpunching' schemes for ASCII zoned decimals, and there might well be one or two more used by less common machine architectures: asciiStandard: ASCII characters '0123456789' (0x30-0x39) and 'pqrstuvwxy' (0x70-0x79) for negative sign punch. asciiTranslatedEBCDIC: ASCII characters '{ABCDEFGHI' (0x7B, 0x41-0x49) and '}JKLMNOPQR' (0x7D, 0x4A-0x52) for negative sign punch. asciiCARealiaModified: ASCII characters '0123456789' (0x30-0x39) and '<SP>!"#$%&'()' (0x20-0x29) for negative sign punch. asciiTandemModified: ASCII characters '0123456789' (0x30-0x39) and control characters 0x80-0x89 for negative sign punch. In case other schemes are discovered that use different byte range for negative sign punch (eg, 0x50 to 0x59), the DFDL specification has said that ASCII zoned decimals must be in a 100% ASCII compatible encoding. What we can observe is that ibm-943_P130-1999 is actually safe for representing ASCII zoned decimals in all the above schemes, because the 0x5C and 0x7E characters do not match any of the ranges of bytes used by the schemes. And we can also observe that (apart from the special case of asciiTranslatedEBCDIC) the overpunching schemes simply use the bits in the first nibble of the byte, so any new scheme we discover is very unlikely to affect 0x5C or 0x7E. So, the DFDL specification *could* be changed to treat ibm-943_P130-1999 as ASCII compatible for zoned decimals. The alternative is to use a work around whereby in a DFDL schema that models a data stream in ccsid 943, the default is 943 but zoned decimals override this and use Shift_JIS. When this workaround was discussed with a Japanese user of IBM DFDL, the reaction was "Why do I have to go to that trouble? It should just work in 943." Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0