- dfdl-wg - lists.ogf.org

Re: [DFDL-WG] Proposed Errata: New function dfdl:stringLiteralToString
by Steve Hanson 12 Dec '12

12 Dec '12

Should name be dfdl:stringFromStringLiteral to mirror existing dfdl:stringLiteralFromString (or should the latter be dfdl:stringToStringLiteral - follows the Java toString() pattern) ? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Date: 12/12/2012 17:20 Subject: Re: [DFDL-WG] Proposed Errata: New function dfdl:stringLiteralToString Mike, should we disallow DFDL Character Class entities also ? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: dfdl-wg(a)ogf.org, Date: 12/12/2012 15:40 Subject: [DFDL-WG] Proposed Errata: New function dfdl:stringLiteralToString Sent by: dfdl-wg-bounces(a)ogf.org Proposed Errata: The following function is provided to assist in creating data that contains characters expressed using DFDL String Literal syntax. dfdl:stringLiteralToString($arg) Returns a string constructed from the $arg string argument. If $arg contains syntax matching DFDL Character Entites syntax, then the corresponding characters are used in the result. Any characters in $arg not matching the DFDL Character Entities syntax remain unchanged in the result. Note that DFDL Byte Value Entities are not allowed. It is a schema definition error if $arg contains syntax matching DFDL Byte Value Entities syntax. Use this function when you need to create a value which contains characters for which DFDL Character Entities are needed. An example is to create data containing the NUL (character code 0) codepoint. This character code is not allowed in XML documents, including DFDL Schemas; hence, it must be specified using a DFDL Character Entity. Within a DFDL Expression, use this function to obtain a string containing this character. Example: dfdl:outputValueCalc='{ fn:concat(../x, dfdl:stringLiteralToString("%NUL;")) }' -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412 -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

OGF DFDL WG Call Minutes 2012-12-11
by Steve Hanson 12 Dec '12

12 Dec '12

Please find minutes from the above call on Redmine at http://redmine.ogf.org/dmsf_files/12977?download= Errata document is now at v011 (draft) here: http://redmine.ogf.org/dmsf_files/8104?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Proposed Errata: New function dfdl:stringLiteralToString
by Mike Beckerle 12 Dec '12

12 Dec '12

Proposed Errata: The following function is provided to assist in creating data that contains characters expressed using DFDL String Literal syntax. dfdl:stringLiteralToString($arg) Returns a string constructed from the $arg string argument. If $arg contains syntax matching DFDL Character Entites syntax, then the corresponding characters are used in the result. Any characters in $arg not matching the DFDL Character Entities syntax remain unchanged in the result. Note that DFDL Byte Value Entities are not allowed. It is a schema definition error if $arg contains syntax matching DFDL Byte Value Entities syntax. Use this function when you need to create a value which contains characters for which DFDL Character Entities are needed. An example is to create data containing the NUL (character code 0) codepoint. This character code is not allowed in XML documents, including DFDL Schemas; hence, it must be specified using a DFDL Character Entity. Within a DFDL Expression, use this function to obtain a string containing this character. Example: dfdl:outputValueCalc='{ fn:concat(../x, dfdl:stringLiteralToString("%NUL;")) }' -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

2 2

clarification needed: choice children have separator from surrounding sequence or not?
by Mike Beckerle 12 Dec '12

12 Dec '12

Consider this textual delimited DFDL schema fragment: <sequence dfdl:separator="/" dfdl:separatorPosition="prefix"> <element name="a" type="xs:string"/> <choice dfdl:initiator="(" dfdl:terminator=")"> <element name="foo" maxOcccurs="10" type="xs:int" dfdl:occursCountKind="parsed" dfdl:terminator=";"/> <element name="bar" type="xs:string" /> </choice> </sequence> Is the separator of the enclosing sequence expected to appear before 'foo' and 'bar' elements inside the choice? If you consider this data '/abc/(/1;/2;/3;/4;)' versus this data '/abc/(1;2;3;4;)' We have to have the / after the abc to find the end of element 'a'. The choice's initiator won't be used as terminating markup. However, then when we have instances of the children of the choice like 'foo' elements, do those have the prefix separator / inherited from the surrounding sequence outside the choice, or do they behave as if surrounded by an inner implied sequence which has no separator? I'm convincing myself that the latter must be the interpretation, but I wanted to see what others think. ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

2 1

Clarification/Issue - is UCS-2 an encoding
by Mike Beckerle 12 Dec '12

12 Dec '12

The encoding name "UCS-2" is used several times in the specification. This is not an IANA encoding name. What is intended is ISO-10646-UCS-2. A note in the IANA character set list notes that this encoding doesn't specify byte order but that it is needed. However, there are no endian variants for this encoding as there are with UTF-16 (UTF-16BE and UTF-16LE). I suggest we remove all mention of UCS-2 from the spec, and use UTF-16 instead. The byte-order issues for UTF-16 are already discussed. This requires changing the description of utf16Width property as follows: Section 11: utf16Width Replace paragraph begining with "Specifies..." by this: Specifies whether the encoding 'UTF-16' should be treated as a fixed or variable-width encoding. 'UTF-16' is potentially a variable width encoding using either 2-byte codepoints, or matched surrogate-pair codepoints containing two adjacent 2-byte codepoints that are combined to compute the character code. However, it is historically common for users to specify 'UTF-16' when they mean the fixed-width subset which does not allow use of the surrogate pairs. When utf16Width='variable', then surrogate pairs are expected and assumed to encode a single character. When utf16Width='fixed', then surrogate pair codepoints are treated as individual character codes. Section 12.3.7.1.1 Character Width Remove "(e.g., UCS-2)" from table, replace with "(e.g., UTF-16 with dfdl:utf16Width='fixed')". Replace "variable (e.g., Shift_JIS, UTF-8, UTF-16)" with "variable (e.g., Shift_JIS, UTF-8, UTF-16 with dfdl:utf16Width='variable')" -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

2 1

Proposed Errata 2.95 change: Mandatory Alignment for Textual Data
by Mike Beckerle 12 Dec '12

12 Dec '12

This proposed errata either replaces/updates 2.95, or cancels that and is a new one. Section 12.1.1 is amended. The table of explicit alignments, table 14, is modified. The column for Text is dropped. A new section is added: Mandatory Alignment for Textual Data. We use the term textual data to describe data with dfdl:representation="text", as well as data being matched to delimiters (parsing) or output as delimiters (unparsing), and data being matched to regular expressions (parsing only - as in a dfdl:assert with testKind='pattern'). Textual data has mandatory alignment that is character-set-encoding dependent. That is, these mandates come from the character set specified by the dfdl:encoding property. When processing textual data, it is a schema definition error if the dfdl:alignment and dfdl:alignmentUnits properties are used to specify alignment that is not a multiple of the encoding-required mandatory alignment. If the data is not aligned to the proper boundary for the encoding when textual data is processed, then bits are skipped (parsing) or filled from dfdl:fillByte (unparsing) to achieve the mandatory alignment. All character set encodings except those listed specifically below have mandatory alignment of 8-bit/1-byte. For encoding US-ASCII-7bit-packed, the alignment is 1-bit (textual data in this encoding may appear on any bit boundary, i.e., no byte alignment is required). TBD: Other encodings...ECMA-6bit, etc. -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

1 0

AUTO: Sunil Dandamudi is out of the office (returning 12/13/2012)
by Sunil Dandamudi 12 Dec '12

12 Dec '12

I am out of the office until 12/13/2012. Note: This is an automated response to your message "dfdl-wg Digest, Vol 78, Issue 12" sent on 12/12/2012 3:54:44. This is the only notification you will receive while this person is away.

1 0

Re: [DFDL-WG] needed: signed bit fields
by Steve Hanson 12 Dec '12

12 Dec '12

I think we are all agreed on the behaviour so I'll update errata 2.90. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM To: Steve Hanson/UK/IBM@IBMGB, Date: 12/12/2012 09:44 Subject: Re: [DFDL-WG] needed: signed bit fields I vote for a schema definition error. In the unlikely event that 0 and -1 are the desired values, xs:unsignedInt can be used and a simple XPath expression can negate the resulting value. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Steve Hanson/UK/IBM@IBMGB To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Cc: dfdl-wg(a)ogf.org Date: 11/12/2012 17:23 Subject: Re: [DFDL-WG] needed: signed bit fields Sent by: dfdl-wg-bounces(a)ogf.org I believe the correct interpretation is 0 and -1. Not keen on re-interpreting this as 0 and 1. I am ok with a schema definition error. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: dfdl-wg(a)ogf.org Date: 11/12/2012 16:45 Subject: Re: [DFDL-WG] needed: signed bit fields Doesn't seem to say what happens if type is signed, but length in bits is 1. Can we make this an SDE please, since it can't be interpreted as a twos-complement integer? Alternatively, can we clarify that if the length is 1, the bit is interpreted as if there was a sign bit of 0? When people write: <element name="myBit" type="xs:int" dfdl:length="1"/> I am pretty sure they aren't expecting values of 0 or -1. They're expecting 0 or 1. ...mikeb On Tue, Dec 11, 2012 at 3:46 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: Errata 2.90. Sections 12.3, 12.3.7.2. Additionally allow lengthUnits 'bits' to apply to binary signed integer types, to support the modeling of signed integer bit fields in the C language. The physical bits are interpreted as a two's complement integer. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: dfdl-wg(a)ogf.org, Date: 10/12/2012 21:42 Subject: [DFDL-WG] needed: signed bit fields Sent by: dfdl-wg-bounces(a)ogf.org current DFDL v1.0 spec says bit fields are all unsigned integers up to length 64. I am modeling data that uses lots of twos-complement 24-bit long and other length signed integers on various bit-boundaries. Was there a reason to leave signed twos-complement out for bit fields (other than perhaps just we thought we might get away with it?) The only corner case I can think of is if you make a 1-bit wide signed bit field. This should be a Schema Defintion Error I believe, because twos-complement isn't defined unless you have at one sign bit, and at least 1 mantissa bit. ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412 -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

needed: signed bit fields
by Mike Beckerle 11 Dec '12

11 Dec '12

current DFDL v1.0 spec says bit fields are all unsigned integers up to length 64. I am modeling data that uses lots of twos-complement 24-bit long and other length signed integers on various bit-boundaries. Was there a reason to leave signed twos-complement out for bit fields (other than perhaps just we thought we might get away with it?) The only corner case I can think of is if you make a 1-bit wide signed bit field. This should be a Schema Defintion Error I believe, because twos-complement isn't defined unless you have at one sign bit, and at least 1 mantissa bit. ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

2 3

Invitation: OGF DFDL Working Group weekly call - extra (12 Dec 14:00 GMT)
by Steve Hanson 11 Dec '12

11 Dec '12

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl GridForge DFDL: http://forge.ogf.org/projects/dfdl-wg/

1 0