DFDL: Calculated Values section rewrite
My deliverable for the next draft (32) was to revise the calculated values section. I have rewritten it and the draft is attached. Of note: I have removed the troublesome "outputLengthCalc" property, as I no longer see a critical need for it. One of the examples computes the number of occurrences for a byte array, and I think that mechanism along with alignment is sufficient to handle the troublesome cases I was considering where size of padding had to be dynamically computed. The section now consists of a short table of definition, followed by illustrative examples, however, I think this is fine. The semantics of inputValueCalc and outputValueCalc aren't really that complex. They can be described in a paragraph. It is the motivation for them that is complicated, so I think examples in the spec, while sometimes considered problematic, are OK in this case.
Mike A couple of comments on the using derived/representation nomenclature with OutputValueCalc and a couple of minor corrections. However I thought the Length Prefix example a bit odd as I would have expected a single HexBinary element rather than an array of bytes. And that got me thinking why isn't it just LengthKind=prefixed but I discovered that there is no proper description of prefixed. Was it intended to cover different kinds of length prefix or is it assumed to be 2 (or 4) bytes? Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: "Mike Beckerle" <mbeckerle.dfdl@gmail.com> To: <dfdl-wg@ogf.org> Date: 26/03/2008 16:39 Subject: [DFDL-WG] DFDL: Calculated Values section rewrite My deliverable for the next draft (32) was to revise the calculated values section. I have rewritten it and the draft is attached. Of note: I have removed the troublesome ?outputLengthCalc? property, as I no longer see a critical need for it. One of the examples computes the number of occurrences for a byte array, and I think that mechanism along with alignment is sufficient to handle the troublesome cases I was considering where size of padding had to be dynamically computed. The section now consists of a short table of definition, followed by illustrative examples, however, I think this is fine. The semantics of inputValueCalc and outputValueCalc aren?t really that complex. They can be described in a paragraph. It is the motivation for them that is complicated, so I think examples in the spec, while sometimes considered problematic, are OK in this case. [attachment "calculated-value-properties.doc" deleted by Alan Powell/UK/IBM] -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: no proper description of prefixed. Recall that last Nov we discussed lengthKind='prefixed'. See http://www.ogf.org/pipermail/dfdl-wg/2007-November/000667.html This eliminates the keywords for prefix types and lengths of prefix fields. We just introduce predefined types like dfdl:prefix2 and dfdl:prefix4 which are very keyword-ish names. I guess this material is pending for inclusion into the spec still. Now back to our regularly scheduled email reply... I've attached a revised calculated value properties doc which incorporates suggestions from Alan, specifically, there's a new example, which is a 2-d array with nrows, ncols fields stored before it. I think this motivates the connection between outputValueCalc, and number of occurrences and occurs calculations nicely. .mikeb _____ From: Alan Powell [mailto:alan_powell@uk.ibm.com] Sent: Thursday, March 27, 2008 2:02 PM To: mbeckerle.dfdl@gmail.com Cc: dfdl-wg@ogf.org; dfdl-wg-bounces@ogf.org Subject: Re: [DFDL-WG] DFDL: Calculated Values section rewrite Mike A couple of comments on the using derived/representation nomenclature with OutputValueCalc and a couple of minor corrections. However I thought the Length Prefix example a bit odd as I would have expected a single HexBinary element rather than an array of bytes. And that got me thinking why isn't it just LengthKind=prefixed but I discovered that there is no proper description of prefixed. Was it intended to cover different kinds of length prefix or is it assumed to be 2 (or 4) bytes? Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: "Mike Beckerle" <mbeckerle.dfdl@gmail.com> To: <dfdl-wg@ogf.org> Date: 26/03/2008 16:39 Subject: [DFDL-WG] DFDL: Calculated Values section rewrite _____ My deliverable for the next draft (32) was to revise the calculated values section. I have rewritten it and the draft is attached. Of note: I have removed the troublesome "outputLengthCalc" property, as I no longer see a critical need for it. One of the examples computes the number of occurrences for a byte array, and I think that mechanism along with alignment is sufficient to handle the troublesome cases I was considering where size of padding had to be dynamically computed. The section now consists of a short table of definition, followed by illustrative examples, however, I think this is fine. The semantics of inputValueCalc and outputValueCalc aren't really that complex. They can be described in a paragraph. It is the motivation for them that is complicated, so I think examples in the spec, while sometimes considered problematic, are OK in this case. [attachment "calculated-value-properties.doc" deleted by Alan Powell/UK/IBM] -- dfdl-wg mailing list dfdl-wg@ogf.org <http://www.ogf.org/mailman/listinfo/dfdl-wg> http://www.ogf.org/mailman/listinfo/dfdl-wg _____ Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Comments from Ian and Steve: 1) Is it a schema error if default/fixed attributes specified along with dfdl:inputValueCalc and/or dfdl:outputValueCalc? 2) 2D array. - dfdl:occurrences() is not in the expression language in draft 031. Is something missing from the language, or should the XPath count() function be used here? - $dfdl:occurrences - $ not needed - dfdl:outputValueCalc for 'ncols' - we think the else clause should be '..\rows[1]\cols' 3) PD date - As an example, it's perhaps not a good one as we have decimalCalendarFormat as a property of a decimal number, which is intended to do exactly this - As it stands, it will need updating to reflect latest decimal supplement Regards, Steve Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh@uk.ibm.com Phone (+44)/(0) 1962-815848 "Mike Beckerle" <mbeckerle.dfdl@gmail.com> Sent by: dfdl-wg-bounces@ogf.org 09/04/2008 02:44 Please respond to mbeckerle.dfdl@gmail.com To Alan Powell/UK/IBM@IBMGB cc dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org Subject Re: [DFDL-WG] DFDL: Calculated Values section rewrite I?ve attached a revised calculated value properties doc which incorporates suggestions from Alan, specifically, there?s a new example, which is a 2-d array with nrows, ncols fields stored before it. I think this motivates the connection between outputValueCalc, and number of occurrences and occurs calculations nicely. ?mikeb From: Alan Powell [mailto:alan_powell@uk.ibm.com] Sent: Thursday, March 27, 2008 2:02 PM To: mbeckerle.dfdl@gmail.com Cc: dfdl-wg@ogf.org; dfdl-wg-bounces@ogf.org Subject: Re: [DFDL-WG] DFDL: Calculated Values section rewrite Mike A couple of comments on the using derived/representation nomenclature with OutputValueCalc and a couple of minor corrections. However I thought the Length Prefix example a bit odd as I would have expected a single HexBinary element rather than an array of bytes. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: "Mike Beckerle" <mbeckerle.dfdl@gmail.com> To: <dfdl-wg@ogf.org> Date: 26/03/2008 16:39 Subject: [DFDL-WG] DFDL: Calculated Values section rewrite My deliverable for the next draft (32) was to revise the calculated values section. I have rewritten it and the draft is attached. Of note: I have removed the troublesome ?outputLengthCalc? property, as I no longer see a critical need for it. One of the examples computes the number of occurrences for a byte array, and I think that mechanism along with alignment is sufficient to handle the troublesome cases I was considering where size of padding had to be dynamically computed. The section now consists of a short table of definition, followed by illustrative examples, however, I think this is fine. The semantics of inputValueCalc and outputValueCalc aren?t really that complex. They can be described in a paragraph. It is the motivation for them that is complicated, so I think examples in the spec, while sometimes considered problematic, are OK in this case. [attachment "calculated-value-properties.doc" deleted by Alan Powell/UK/IBM] -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Hmm. We're in this situation where DFDL is so rich with features that anything those features don't handle will by definition be an obscure corner case thereby hard to motivate. I can change this example to use three single byte binary integers instead of 6 packed decimal digits. This is certainly a feasible binary data format, though it's nothing I've ever seen. It also makes the example orthogonal to the decimal changes, so generally is more on point. Attachment is new example plus the other improvements suggested. _____ From: Steve Hanson [mailto:smh@uk.ibm.com] Sent: Wednesday, April 09, 2008 9:29 AM To: mbeckerle.dfdl@gmail.com Cc: dfdl-wg@ogf.org Subject: Re: [DFDL-WG] DFDL: Calculated Values section rewrite Comments from Ian and Steve: 1) Is it a schema error if default/fixed attributes specified along with dfdl:inputValueCalc and/or dfdl:outputValueCalc? 2) 2D array. - dfdl:occurrences() is not in the expression language in draft 031. Is something missing from the language, or should the XPath count() function be used here? - $dfdl:occurrences - $ not needed - dfdl:outputValueCalc for 'ncols' - we think the else clause should be '..\rows[1]\cols' 3) PD date - As an example, it's perhaps not a good one as we have decimalCalendarFormat as a property of a decimal number, which is intended to do exactly this - As it stands, it will need updating to reflect latest decimal supplement Regards, Steve Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh@uk.ibm.com Phone (+44)/(0) 1962-815848 "Mike Beckerle" <mbeckerle.dfdl@gmail.com> Sent by: dfdl-wg-bounces@ogf.org 09/04/2008 02:44 Please respond to mbeckerle.dfdl@gmail.com To Alan Powell/UK/IBM@IBMGB cc dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org Subject Re: [DFDL-WG] DFDL: Calculated Values section rewrite I've attached a revised calculated value properties doc which incorporates suggestions from Alan, specifically, there's a new example, which is a 2-d array with nrows, ncols fields stored before it. I think this motivates the connection between outputValueCalc, and number of occurrences and occurs calculations nicely. .mikeb _____ From: Alan Powell [mailto:alan_powell@uk.ibm.com] Sent: Thursday, March 27, 2008 2:02 PM To: mbeckerle.dfdl@gmail.com Cc: dfdl-wg@ogf.org; dfdl-wg-bounces@ogf.org Subject: Re: [DFDL-WG] DFDL: Calculated Values section rewrite Mike A couple of comments on the using derived/representation nomenclature with OutputValueCalc and a couple of minor corrections. However I thought the Length Prefix example a bit odd as I would have expected a single HexBinary element rather than an array of bytes. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: "Mike Beckerle" <mbeckerle.dfdl@gmail.com> To: <dfdl-wg@ogf.org> Date: 26/03/2008 16:39 Subject: [DFDL-WG] DFDL: Calculated Values section rewrite _____ My deliverable for the next draft (32) was to revise the calculated values section. I have rewritten it and the draft is attached. Of note: I have removed the troublesome "outputLengthCalc" property, as I no longer see a critical need for it. One of the examples computes the number of occurrences for a byte array, and I think that mechanism along with alignment is sufficient to handle the troublesome cases I was considering where size of padding had to be dynamically computed. The section now consists of a short table of definition, followed by illustrative examples, however, I think this is fine. The semantics of inputValueCalc and outputValueCalc aren't really that complex. They can be described in a paragraph. It is the motivation for them that is complicated, so I think examples in the spec, while sometimes considered problematic, are OK in this case. [attachment "calculated-value-properties.doc" deleted by Alan Powell/UK/IBM] -- dfdl-wg mailing list dfdl-wg@ogf.org <http://www.ogf.org/mailman/listinfo/dfdl-wg> http://www.ogf.org/mailman/listinfo/dfdl-wg _____ Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg _____ Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Mike I have added this to the spec doc with some minor changes shown in attached. Two new questions I assume in/outvalueCalc are not allow on arrays dfdl:hidden prevents elements appearing in the infoset on parsing but there isn't an equivalent to prevent infoset elements appearing in the data stream on unparsing. We have assumed that inputvalueCalc prevents output but is that always the case? (Steve you need to add InputValueCalc to unparsing property precedence). If inputValueCalc is sufficient to prevent output couldn't outputvalueCalc prevent input rather than hidden? Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: "Mike Beckerle" <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB Cc: dfdl-wg@ogf.org Date: 09/04/2008 15:03 Subject: Re: [DFDL-WG] DFDL: Calculated Values section rewrite Hmm. We?re in this situation where DFDL is so rich with features that anything those features don?t handle will by definition be an obscure corner case thereby hard to motivate. I can change this example to use three single byte binary integers instead of 6 packed decimal digits. This is certainly a feasible binary data format, though it?s nothing I?ve ever seen. It also makes the example orthogonal to the decimal changes, so generally is more on point. Attachment is new example plus the other improvements suggested. From: Steve Hanson [mailto:smh@uk.ibm.com] Sent: Wednesday, April 09, 2008 9:29 AM To: mbeckerle.dfdl@gmail.com Cc: dfdl-wg@ogf.org Subject: Re: [DFDL-WG] DFDL: Calculated Values section rewrite Comments from Ian and Steve: 1) Is it a schema error if default/fixed attributes specified along with dfdl:inputValueCalc and/or dfdl:outputValueCalc? 2) 2D array. - dfdl:occurrences() is not in the expression language in draft 031. Is something missing from the language, or should the XPath count() function be used here? - $dfdl:occurrences - $ not needed - dfdl:outputValueCalc for 'ncols' - we think the else clause should be '..\rows[1]\cols' 3) PD date - As an example, it's perhaps not a good one as we have decimalCalendarFormat as a property of a decimal number, which is intended to do exactly this - As it stands, it will need updating to reflect latest decimal supplement Regards, Steve Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh@uk.ibm.com Phone (+44)/(0) 1962-815848 "Mike Beckerle" <mbeckerle.dfdl@gmail.com> Sent by: dfdl-wg-bounces@ogf.org 09/04/2008 02:44 Please respond to mbeckerle.dfdl@gmail.com To Alan Powell/UK/IBM@IBMGB cc dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org Subject Re: [DFDL-WG] DFDL: Calculated Values section rewrite I?ve attached a revised calculated value properties doc which incorporates suggestions from Alan, specifically, there?s a new example, which is a 2-d array with nrows, ncols fields stored before it. I think this motivates the connection between outputValueCalc, and number of occurrences and occurs calculations nicely. ?mikeb From: Alan Powell [mailto:alan_powell@uk.ibm.com] Sent: Thursday, March 27, 2008 2:02 PM To: mbeckerle.dfdl@gmail.com Cc: dfdl-wg@ogf.org; dfdl-wg-bounces@ogf.org Subject: Re: [DFDL-WG] DFDL: Calculated Values section rewrite Mike A couple of comments on the using derived/representation nomenclature with OutputValueCalc and a couple of minor corrections. However I thought the Length Prefix example a bit odd as I would have expected a single HexBinary element rather than an array of bytes. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: "Mike Beckerle" <mbeckerle.dfdl@gmail.com> To: <dfdl-wg@ogf.org> Date: 26/03/2008 16:39 Subject: [DFDL-WG] DFDL: Calculated Values section rewrite My deliverable for the next draft (32) was to revise the calculated values section. I have rewritten it and the draft is attached. Of note: I have removed the troublesome ?outputLengthCalc? property, as I no longer see a critical need for it. One of the examples computes the number of occurrences for a byte array, and I think that mechanism along with alignment is sufficient to handle the troublesome cases I was considering where size of padding had to be dynamically computed. The section now consists of a short table of definition, followed by illustrative examples, however, I think this is fine. The semantics of inputValueCalc and outputValueCalc aren?t really that complex. They can be described in a paragraph. It is the motivation for them that is complicated, so I think examples in the spec, while sometimes considered problematic, are OK in this case. [attachment "calculated-value-properties.doc" deleted by Alan Powell/UK/IBM] -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU [attachment "calculated-value-properties-v3.doc" deleted by Alan Powell/UK/IBM] -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (3)
-
Alan Powell -
Mike Beckerle -
Steve Hanson