[wg-all] Repeating groups in DFDL
For anyone that was at the 19/07 phone meeting, this is what Adam Fox and I were trying to find a solution to. If anyone knows of a solution, please contact me or adam.fox@nrl.navy.mil. ---------------------------------------------------------------------------------------------------- Please see the attached xml schema while reading this message. The problem we want to solve is how to represent a structure that can have a variable amount of occurences (even zero), and the amount of repeats is controlled by a one bit field, which we will call"repeatBit". The repeating structure in this case is "A1". The element before it, "presentBit", determines if there is at least one occurence of A1. If presentBit is zero, then A1 has zero occurences. If presentBit is one, then A1 has AT LEAST one occurence. The first element in A1 is "repeatBit". What repeatBit does is tells us if there will be another occurence of A1 after the current occurence. When repeatBit is zero, then that means we will read through the rest of A1, then that is the last occurence of A1. As long as repeatBit continues to be one however, we will read through the rest of A1, then start a new occurence. I have included all the known required DFDL notation. Please let me know what is missing. EXAMPLE INPUT (results are in base 10): NOTE: Not all bits are used in these examples. Unused bits should be ignored for the purpose of the examples. 1) 0111 0101 presentBit = 0 aFieldAfterA1 = 3 2) 1011 1010 presentBit = 1 A1 -repeatBit = 0 -someField = 3 aFieldAfterA1 = 2 3) 1101 0110 0000 0000 presentBit = 1 A1 -repeatBit = 1 -someField = 1 A1 -repeatBit = 0 -someField = 3 aFieldAfterA1 = 0
A couple of clarifications if I may....
- In your example, you have someField as being of simple type, can it be
complex type in your real format?
- You said "If presentBit is zero, then A1 has zero occurences". But in
your xsd you have A1 as (implied) minOccurs="1". I think you meant
minOccurs="0"?
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From:
Ryan Farrell
Great example Ryan, We've spent quite a bit of time over the last several months on textual formats with these kinds of characteristics, which I would describe as "occurrances determined by parsing" - that is, you look at the data to determine "is there another element or not" one by one as they are parsed. We need to make sure the properties are there to make this work for the binary case as well. I actually believe we cannot handle this format right now. We don't have a property which says to look at the current element to determine whether to expect a subsequent element, as a way of determining occurrence counts. Next week we'll add resolving this example to the workgroup actions list. ...mikeb Mike Beckerle Senior Technology Leader/Manager Deloitte Managed Analytics 100 Fifth Avenue, Waltham, MA 02451 Tel/Direct: +1 781 330 0412 | Fax: +1 866 253 3006 www.deloitte.com -----Original Message----- From: dfdl-wg-bounces@ogf.org [mailto:dfdl-wg-bounces@ogf.org] On Behalf Of Ryan Farrell Sent: Tuesday, July 19, 2011 2:12 PM To: dfdl-wg@ogf.org Subject: [DFDL-WG] [wg-all] Repeating groups in DFDL For anyone that was at the 19/07 phone meeting, this is what Adam Fox and I were trying to find a solution to. If anyone knows of a solution, please contact me or adam.fox@nrl.navy.mil. ---------------------------------------------------------------------------- ------------------------ Please see the attached xml schema while reading this message. The problem we want to solve is how to represent a structure that can have a variable amount of occurences (even zero), and the amount of repeats is controlled by a one bit field, which we will call"repeatBit". The repeating structure in this case is "A1". The element before it, "presentBit", determines if there is at least one occurence of A1. If presentBit is zero, then A1 has zero occurences. If presentBit is one, then A1 has AT LEAST one occurence. The first element in A1 is "repeatBit". What repeatBit does is tells us if there will be another occurence of A1 after the current occurence. When repeatBit is zero, then that means we will read through the rest of A1, then that is the last occurence of A1. As long as repeatBit continues to be one however, we will read through the rest of A1, then start a new occurence. I have included all the known required DFDL notation. Please let me know what is missing. EXAMPLE INPUT (results are in base 10): NOTE: Not all bits are used in these examples. Unused bits should be ignored for the purpose of the examples. 1) 0111 0101 presentBit = 0 aFieldAfterA1 = 3 2) 1011 1010 presentBit = 1 A1 -repeatBit = 0 -someField = 3 aFieldAfterA1 = 2 3) 1101 0110 0000 0000 presentBit = 1 A1 -repeatBit = 1 -someField = 1 A1 -repeatBit = 0 -someField = 3 aFieldAfterA1 = 0
Yes we need a friendly way of saying 'I am the last item' when using
occursCountKind is 'parsed'.
But with what we have today, could we use a discriminator on the repeating
element that fails when the indicator of the previous repeat was 0 ? So
speculation starts to parse the thing after the last repeat on the
assumption it is another repeat and then fails and backtracks.
dfdl:assert{if dfdl:position() eq 1 then true else if
'./present[dfdl:position()-1]' eq 0 then false else true}
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From:
"Mike Beckerle"
From our call today, I suggest this schema, based on the assertion suggestion from steve/tim, and adding in the unparser/output direction calculations, and putting this together to match Ryan's example schema. I'm not certain on the xpath expressions syntax. I may have number of ".." moves wrong (at least).
I added outputValueCalc properties to the flag bits, I added
occursCountKind='parsed' to the array, and the assertion to the repeatBit
element. I put in the check so that a zero length array is possible based on
the presentBit as well.
Hi Mike
That looks pretty good. A couple of minor changes in bold red (I don't
think there's a length() function).
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From:
"Mike Beckerle"
participants (3)
-
Mike Beckerle
-
Ryan Farrell
-
Steve Hanson