- dfdl-wg - lists.ogf.org

Re: [DFDL-WG] needed: signed bit fields
by Steve Hanson 12 Dec '12

12 Dec '12

I think we are all agreed on the behaviour so I'll update errata 2.90. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM To: Steve Hanson/UK/IBM@IBMGB, Date: 12/12/2012 09:44 Subject: Re: [DFDL-WG] needed: signed bit fields I vote for a schema definition error. In the unlikely event that 0 and -1 are the desired values, xs:unsignedInt can be used and a simple XPath expression can negate the resulting value. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Steve Hanson/UK/IBM@IBMGB To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Cc: dfdl-wg(a)ogf.org Date: 11/12/2012 17:23 Subject: Re: [DFDL-WG] needed: signed bit fields Sent by: dfdl-wg-bounces(a)ogf.org I believe the correct interpretation is 0 and -1. Not keen on re-interpreting this as 0 and 1. I am ok with a schema definition error. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: dfdl-wg(a)ogf.org Date: 11/12/2012 16:45 Subject: Re: [DFDL-WG] needed: signed bit fields Doesn't seem to say what happens if type is signed, but length in bits is 1. Can we make this an SDE please, since it can't be interpreted as a twos-complement integer? Alternatively, can we clarify that if the length is 1, the bit is interpreted as if there was a sign bit of 0? When people write: <element name="myBit" type="xs:int" dfdl:length="1"/> I am pretty sure they aren't expecting values of 0 or -1. They're expecting 0 or 1. ...mikeb On Tue, Dec 11, 2012 at 3:46 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: Errata 2.90. Sections 12.3, 12.3.7.2. Additionally allow lengthUnits 'bits' to apply to binary signed integer types, to support the modeling of signed integer bit fields in the C language. The physical bits are interpreted as a two's complement integer. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: dfdl-wg(a)ogf.org, Date: 10/12/2012 21:42 Subject: [DFDL-WG] needed: signed bit fields Sent by: dfdl-wg-bounces(a)ogf.org current DFDL v1.0 spec says bit fields are all unsigned integers up to length 64. I am modeling data that uses lots of twos-complement 24-bit long and other length signed integers on various bit-boundaries. Was there a reason to leave signed twos-complement out for bit fields (other than perhaps just we thought we might get away with it?) The only corner case I can think of is if you make a 1-bit wide signed bit field. This should be a Schema Defintion Error I believe, because twos-complement isn't defined unless you have at one sign bit, and at least 1 mantissa bit. ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412 -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

needed: signed bit fields
by Mike Beckerle 11 Dec '12

11 Dec '12

current DFDL v1.0 spec says bit fields are all unsigned integers up to length 64. I am modeling data that uses lots of twos-complement 24-bit long and other length signed integers on various bit-boundaries. Was there a reason to leave signed twos-complement out for bit fields (other than perhaps just we thought we might get away with it?) The only corner case I can think of is if you make a 1-bit wide signed bit field. This should be a Schema Defintion Error I believe, because twos-complement isn't defined unless you have at one sign bit, and at least 1 mantissa bit. ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

2 3

Invitation: OGF DFDL Working Group weekly call - extra (12 Dec 14:00 GMT)
by Steve Hanson 11 Dec '12

11 Dec '12

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl GridForge DFDL: http://forge.ogf.org/projects/dfdl-wg/

1 0

OGF DFDL WG Call Agenda 2012-12-11
by Steve Hanson 11 Dec '12

11 Dec '12

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/12977?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Action 185: Text number rounding and ICU
by Steve Hanson 11 Dec '12

11 Dec '12

As described in action 185, the DFDL spec behaviour for the DFDL properties that control text number rounding was derived from the documented behaviour for ICU4J. However the documentation is not correct and the ICU4J behaviour is as described for ICU4C. An ICU ticket was raised to clarify the position, and the intended behaviour is the observed behaviour. That affects the DFDL text number rounding properties as follows: textNumberRounding Enum Specifies how rounding is controlled during unparsing. Valid values ‘pattern', 'explicit' When dfdl:textNumberRep is 'standard' this property only applies when dfdl:textStandardBase is 10. If 'pattern' then rounding takes place according to the pattern. A rounding increment may be specified in the dfdl:textNumberPattern using digits '1' though '9', otherwise rounding is to the width of the pattern. The rounding mode is always 'roundHalfEven'. If 'explicit' then the rounding increment is specified by the dfdl:textNumberRoundingIncrement property, and any digits '1' through '9' in the dfdl:textNumberPattern are treated as digit '0'. The rounding mode is specified by the dfdl:textRoundingMode property. To disable rounding, use 'explicit' in conjunction with 'roundUnnecessary' for the dfdl:textNumberRoundingMode. If rounding is disabled then any excess precision is treated as a processing error. Annotation: dfdl:element, dfdl:simpleType textNumberRoundingMode Enum Specifies how rounding occurs during unparsing, when dfdl:textNumberRounding is 'explicit'. When dfdl:textNumberRep is 'standard' this property only applies when dfdl:textStandardBase is 10. To switch off rounding, use 'roundUnnecessary'. Valid values ‘roundCeiling’, ‘roundFloor’, ‘roundDown’, ‘roundUp’, ‘roundHalfEven’, ‘roundHalfDown’, ‘roundHalfUp', 'roundUnnecessary' Annotation: dfdl:element, dfdl:simpleType textNumberRoundingIncrement Double Specifies the rounding increment to use during unparsing, when dfdl:textNumberRounding is 'explicit'. When dfdl:textNumberRep is 'standard' this property only applies when dfdl:textStandardBase is 10. To switch off rounding, use 0.0. A negative value is a schema definition error. Annotation: dfdl:element, dfdl:simpleType Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: Action 183 - chicken-and-egg situation with lengths given by expressions
by Steve Hanson 11 Dec '12

11 Dec '12

The email thread which provided the material for the discussion which led to: 2.100. Section 12.3.1. State that when unparsing an element with lengthKind ‘explicit’ and where length is an expression, then the data in the Infoset is treated as variable length and not fixed length. The behaviour is the same as lengthKind ‘prefixed’. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 16/11/2012 16:04 ----- From: Tim Kimber/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org, Date: 11/09/2012 12:39 Subject: Re: [DFDL-WG] Action 183 - chicken-and-egg situation with lengths given by expressions Sent by: dfdl-wg-bounces(a)ogf.org Good point. The problem is that lengthKind-'explicit' is being used for two things: a) a length that is static b) a length that is calculated ...so the DFDL serializer must assume that the expression needs to be evaluated. For occursCountKind we have separate values for 'fixed' and 'expression'. If we did not, then occursCountKind would have the same problem except that it would affect defaulting rather than padding. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Steve Hanson/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org, Date: 11/09/2012 12:27 Subject: [DFDL-WG] Action 183 - chicken-and-egg situation with lengths given by expressions Sent by: dfdl-wg-bounces(a)ogf.org This mail is on the expected behaviour of the DFDL unparser when writing out a 'data' element the length of which is held in an earlier 'len' element. There are several scenarios, some straightforward and some that exhibit a chicken-and-egg behaviour. The principle of what happens is understood, the action is to make sure that the behaviour is explained in enough detail in the spec to enable implementations to be consistent. (Note - IBM DFDL does not yet support outputVaueCalc so has not hit this yet). Scenarios follow. The 'data' element shown is simple, but the same principles apply if it is complex. 1) 'len' is set from infoset - 'len' can be set in augmented infoset - No issue as 'data's length expression may be evaluated <xsd:element name="message1"> <xsd:complexType> <xsd:sequence> <xsd:element name="len" type="xsd:int" dfdl:lengthKind="explicit" dfdl:length="2" /> <xsd:element name="data" type="xsd:string" dfdl:length="{/message1/len}" dfdl:lengthKind="explicit" /> </xsd:sequence> </xsd:complexType> </xsd:element> 2) 'len' is set using outputValueCalc with fixed expression - When 'len's outputValueCalc is encountered, it can be evaluated then and there - 'len' can be set in augmented infoset - No issue as 'data's length expression may be evaluated <xsd:element name="message1"> <xsd:complexType> <xsd:sequence> <xsd:element name="len" type="xsd:int" dfdl:outputValueCalc="{10}" dfdl:lengthKind="explicit" dfdl:length="2" /> <xsd:element name="data" type="xsd:string" dfdl:length="{/message1/len}" dfdl:lengthKind="explicit" /> </xsd:sequence> </xsd:complexType> </xsd:element> 3) 'len' is set using outputValueCalc with reference 'data' (unpadded) - When 'len's outputValueCalc is encountered, it can not yet be evaluated as it depends on the length of 'data' - 'len' can not yet be set in augmented infoset - Problem as 'data's length expression can not be evaluated - But we do know the unpadded length of 'data' so 'len's outputValueCalc can now be evaluated - In turn this means that 'data's length expression can now be evaluated <xsd:element name="message1"> <xsd:complexType> <xsd:sequence> <xsd:element name="len" type="xsd:int" dfdl:outputValueCalc="{dfdl:unpaddedLength(/message1/data)}" dfdl:lengthKind="explicit" dfdl:length="2" /> <xsd:element name="data" type="xsd:string" dfdl:length="{/message1/len}" dfdl:lengthKind="explicit" /> </xsd:sequence> </xsd:complexType> </xsd:element> 4) 'len' is set using outputValueCalc with reference 'data' (padded) - When 'len's outputValueCalc is encountered, it can not yet be evaluated as it depends on the length of 'data' - 'len' can not yet be set in augmented infoset - Problem as 'data's length expression can not be evaluated - We don't know the padded length of 'data' because we don't know 'len' - Problem: 'data's length expression can never be evaluated <xsd:element name="message1"> <xsd:complexType> <xsd:sequence> <xsd:element name="len" type="xsd:int" dfdl:outputValueCalc="{dfdl:representationLength(/message1/data)}" dfdl:lengthKind="explicit" dfdl:length="2" /> <xsd:element name="data" type="xsd:string" dfdl:length="{/message1/len}" dfdl:lengthKind="explicit" /> </xsd:sequence> </xsd:complexType> </xsd:element> Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org Date: 04/09/2012 17:31 Subject: Fw: Behaviour for lengthKind 'endOfparent' is still not fully specified DFDL WG call 4th Sept 2012: 1) Agreed that for binary data, only xs:hexBinary and packed/BCD allowed to have endOfParent 2) Agreed this is the correct behaviour when filling to a known length 3) Agreed this is the correct behaviour when filling to a known length 4) Agreed this is the correct behaviour when filling to a known length It was noted that lengthKind 'explicit' on the parent may not result in a known length if the length is an expression. This is an example of a more general chicken-and-egg situation with lengths given by expressions, for which outputValueCalc and DFDL functions unpaddedLength() were added can be used. Action raised to ensure that the behaviour of an implementation is fully defined by the spec. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 04/09/2012 17:24 ----- From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org <dfdl-wg(a)ogf.org> Date: 04/09/2012 14:08 Subject: Behaviour for lengthKind 'endOfparent' is still not fully specified Noted when I reviewed latest spec - endOfParent and unparsing is not fully thought through. The spec today says that I can use endOfParent with binary data. There is a restriction in section 12.3.8, but it only applies when an element is endOfParent and its parent is lengthKind delimited. There are a couple of cases to consider: 1) Binary data of restricted length (see list in other email "proposed clarification/narrowing - delimited binary data should decimal"). I don't think it makes sense to allow these. We don't allow these binary reps for delimited. 2) Text data of variable length when unparsing. Box scenario. If the data in the infoset is shorter than the space in the box, what we do? I think we should pad to box length with appropriate padChar, according to justification, as that is effectively a 'specified length'. Error if textPadKind is 'none'. Use parent's lengthUnits. 3) HexBinary data of variable length when unparsing. Box scenario. If the data in the infoset is shorter than the space in the box, what we do? I think we should right-pad to box length with fill byte, as that is effectively a 'specified length'. 4) Packed/BCD binary data of variable length when unparsing. Box scenario. If the data in the infoset is shorter than the space in the box, what we do? I think we should pad to box length with zero bytes, according to justification, as that is effectively a 'specified length'. (Must be zero bytes and not fill byte as must be numeric in order to be parsed). In relation to 2 - 4, note that lengthKind 'endOfParent' can only be used with a parent lengthKind of 'explicit', 'pattern', 'prefixed' or 'endOfParent' or a choice with choiceLengthKind 'explicit', so the box scenario when unparsing therefore occurs only when lengthKind is 'explicit' or choiceLengthKind is 'explicit' - these are the cases when the length is known. Also note that when there are nested 'endOfParent' elements (which is allowed) then all padding must be done on the simple element (ie, the innermost element), to ensure that what is output can be parsed. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Unparsing - Pad to "big enough length to fit..." - can it be achieved in DFDL?
by Mike Beckerle 11 Dec '12

11 Dec '12

I have a format where fields are supposed to be padded so they all have the length of the longest one. I.e., something like this with fields terminated by | chars FIELD1|FIELD2 |ST|PH | ABC |CAPE COD|MA|888-999-0000| DEFG |BOSTON |MA|- | I believe I can warn if all the participants in a column of this tabular thing don't have the same length. I would just set a variable with the length of the first row, then on subsequent rows (parsed with a different element in the schema) I would verify that the length equals what is stored in the variable by way of an assertion. I believe this is what recoverableError kind is for actually, when you have redundant information you want to warn about. The problem I have is unparsing. How in DFDL I could determine how wide to pad each field before unparsing it. The spec requires me to measure the width of the fields (including that quasi-header row which is first, and measure how long each field is, then pad them to that maximum. I believe there is no way to do this currently in DFDL, because there is no way that an expression can refer to more than one record, nor any way to iterate or operate over sets of data. Instead an application must be written which can use DFDL v1.0 but the application has to provide some of the smarts. In this case, the application would have to walk the infoset and determine the maximum length for each such field, then provide these length maximums to the DFDL schema by way of externally set variables. Is there any other way? Second issue: In addition, the spec of my format requires the total maximum length for all the fields of a row to be at most 72 characters long. When unparsing, we don't evaluate assertions. So how can I test this? -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

2 2

Re: [DFDL-WG] DFDL character entities in DFDL expressions
by Mike Beckerle 05 Dec '12

05 Dec '12

Well, yes, I think we're discussing exactly how that should work. No matter what, we do need to be clear that string values created in DFDL's expression language can contain these XML-illegal characters, since they are allowed in DFDL's infoset. This means that DFDL implementations can only re-use an existing XPath implementation to create their DFDL expression language implementation to the extent that it does NOT enforce the XML-illegal characters restrictions all over the place. I am currently working with standard Saxon-B XPath and will report back. But let's be optimistic. The question then is just what is the solution to creating a string-literal including these characters. That cannot be done without some beyond-XML mechanism. DFDL has a string-literal notation for expressing these characters, so we either say that string literals in the expression language can use the DFDL character and numeric entities, or we can do something more 'library like', and provide a function which interprets the string-literal notation, and isolate the implementation concerns a bit. As a language embedded in XML schema, we already straddle the fence of two somewhat inconsistent language environments. E.g., the literals one can use as the value of the default attribute on an element declaration cannot use DFDL character entities, as this is a purely XML Schema construct. Similarly, the regular expressions one can use for the XML schema pattern facet are more restrictive than the DFDL regular expressions one can use in a dfdl:assert, or a dfdl:lengthKind='pattern'. So, it's acceptable to me to say that expressions also have some split where the dfdl-specific aspects, like the dfdl character and numeric entities notation, is isolated in a sub-construct. On Wed, Dec 5, 2012 at 6:37 AM, Andrew Coleman <andrew_coleman(a)uk.ibm.com>wrote: > No, casting a hexBinary to a string will just write out the octets - i.e. > the string will be '00'. > > XPath itself has no mechanism for interpreting entity references or > character references. Its hosting language (XQuery or XSLT/XML) provides > this. Since DFDL is XML, wouldn't that provide a mechanism? > > Regards, > - Andy > > __________________________________________ > Andrew Coleman > WebSphere Message Broker Development > IBM Hursley Park > > > > > From: Steve Hanson/UK/IBM > To: Tim Kimber/UK/IBM@IBMGB, > Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org, Mike Beckerle < > mbeckerle.dfdl(a)gmail.com>, Andrew Coleman/UK/IBM@IBMGB > Date: 05/12/2012 11:06 > Subject: Re: [DFDL-WG] DFDL character entities in DFDL expressions > ------------------------------ > > > Aren't XPath facilities sufficient here? > > outputValueCalc="{ if (fn:string-length(../s) lt 64) then > fn:concat(../s, xs:string(xs:hexBinary('00'))) else ../s }" > > Regards > > Steve Hanson > Architect, Data Format Description Language (DFDL) > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK* > **smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:+44-1962-815848 > > > > > From: Tim Kimber/UK/IBM@IBMGB > To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, > Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org > Date: 05/12/2012 10:51 > Subject: Re: [DFDL-WG] DFDL character entities in DFDL expressions > Sent by: dfdl-wg-bounces(a)ogf.org > ------------------------------ > > > > I think the restriction was aimed at avoiding things like this: > > outputValueCalc="{ if (fn:string-length(../s) lt 64) then > fn:concat(../s, '%#rFF;') else ../s }" > > I agree that a total ban is too restrictive. My personal preference would > be for the dfdl:string() function because it makes the usage of > DFDL-specific features obvious in the DFDL expression. But what would be > the return type of dfdl:string()? It it returned a sequence of characters > then the raw byte entity ( %#rnn; ) would still need to be disallowed. > regards, > > Tim Kimber, DFDL Team, > Hursley, UK > Internet: kimbert(a)uk.ibm.com > Tel. 01962-816742 > Internal tel. 37246742 > > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> > To: dfdl-wg(a)ogf.org, > Date: 04/12/2012 23:36 > Subject: [DFDL-WG] DFDL character entities in DFDL expressions > Sent by: dfdl-wg-bounces(a)ogf.org > ------------------------------ > > > > > We currently have this language in the spec: > > "Within an expression, a string is never interpreted as a DFDL string > literal." > > To me this means one cannot use DFDL character entities in an expression. > > However, I need to do this: > > outputValueCalc="{ if (fn:string-length(../s) lt 64) then > fn:concat(../s, '%NUL;') else ../s }" > > Basically, I need to append a NUL on the end of the string in the output > value case. > > Unless I can put a %NUL; into an expression and have it interpreted as a > DFDL String literal, I am not sure how I can achieve this. > > At minimum I need a new DFDL function which might be an alternate string > constructor, such as dfdl:string('....') which interprets the argument as > something where the contents are to be scanned for DFDL character entities > and they are substituted so that the resulting string can contain the > characters that are disallowed in XML. (like NUL) > > -- > Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies > Tel: 781-330-0412 > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > *https://www.ogf.org/mailman/listinfo/dfdl-wg*<https://www.ogf.org/mailman/listinfo/dfdl-wg> > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > https://www.ogf.org/mailman/listinfo/dfdl-wg > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

2 2

DFDL character entities in DFDL expressions
by Mike Beckerle 05 Dec '12

05 Dec '12

We currently have this language in the spec: "Within an expression, a string is never interpreted as a DFDL string literal." To me this means one cannot use DFDL character entities in an expression. However, I need to do this: outputValueCalc="{ if (fn:string-length(../s) lt 64) then fn:concat(../s, '%NUL;') else ../s }" Basically, I need to append a NUL on the end of the string in the output value case. Unless I can put a %NUL; into an expression and have it interpreted as a DFDL String literal, I am not sure how I can achieve this. At minimum I need a new DFDL function which might be an alternate string constructor, such as dfdl:string('....') which interprets the argument as something where the contents are to be scanned for DFDL character entities and they are substituted so that the resulting string can contain the characters that are disallowed in XML. (like NUL) -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

3 2

Clarification/Fix: Glossary entries about scanning
by Mike Beckerle 04 Dec '12

04 Dec '12

Definition of "scan" assumed in text throughout the spec encompasses both scanning for delimiters, and scanning for matches to regular expressions. So these glossary changes are needed to clarify that delimiter scanning is a specialized sub-class of scanning generally. Delimiter Scanning: Remove subphrase ", or simply scanning for short. Scanning also..." replace with "Delimiter scanning also ..." Scan: Remove the word "bytes". Insert after "...and terminators." this text ", or matches to regular expressions." -- Mike Beckerle | OGF DFDL WG Co-Chair | Tresys Technologies Tel: 781-330-0412

1 0