Updated below to reflect comments from WG call: 16.6 Forward Progress Requirement To prevent an infinite loop, the parsing of an array that is potentially unbounded must terminate when the following are true: * The occurrence is a point of uncertainty; * The position in the data does not move during the parsing of the occurrence (including any associated Separator, PrefixSeparator or PostfixSeparator region); * The occurrence is known-to-exist with empty representation or nil representation. An array is potentially unbounded if any of the following are true: * dfdl:occursCountKind is 'stopValue' * dfdl:occursCountKind is 'parsed' * dfdl:occursCountKind is 'implicit' and XSDL maxOccurs is unbounded When dfdl:occursCountKind 'stopValue' this results in a processing error because the stop value will never be encountered. Further, to prevent unnecessary consumption of resources for large bounded values of XSDL maxOccurs, the parsing of an array must terminate when the following are true: * dfdl:occursCountKind is 'implicit'; * The occurrence is a point of uncertainty; * The position in the data does not move during the parsing of the occurrence (including any associated Separator, PrefixSeparator or PostfixSeparator region); * The occurrence is known-to-exist with empty representation. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 17/02/2014 15:54 ----- From: Steve Hanson/UK/IBM To: Tim Kimber/UK/IBM@IBMGB, Cc: dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org Date: 03/02/2014 14:54 Subject: Re: [DFDL-WG] Fw: Spec question: 16.6 Forward Progress Requirement Tim 'Optional occurrence' is already defined in the Glossary with a different meaning. I've spent some time thinking through this issue again, and I'd prefer not to say that this is a processing error. Although net effect is the same we have to be very careful that the words here are consistent with section 9. Note that we don't have to worry about known-not-to-exist (missing/absent) as this terminates the array anyway by definition. The circumstance we are really discussing here is known-to-exist. 16.6 Forward Progress Requirement The parsing of an array that is potentially unbounded terminates when the position in the data does not move during the parsing of an occurrence that is a point of uncertainty (including any associated Separator, PrefixSeparator or PostfixSeparator region), even if the occurrence is known-to-exist. This is to prevent an infinite loop. An array is potentially unbounded if any of the following are true: * dfdl:occursCountKind is 'stopValue' * dfdl:occursCountKind is 'parsed' * dfdl:occursCountKind is 'implicit' and XSDL maxOccurs is unbounded When dfdl:occursCountKind 'stopValue' this results in a processing error because the stop value will never be encountered. Note: When dfdl:occursCountKind is 'implicit' and XSDL maxOccurs is not unbounded, and the position in the data does not move during the parsing of an occurrence that is a point of uncertainty (including any associated Separator, PrefixSeparator or PostfixSeparator region), and the occurrence is known-to-exist, and the occurrence does not give rise to a value in the infoset, implementations may terminate the parsing of the array. This is to prevent the unnecessary consumption of resources for large values of XSDL maxOccurs. Note that the last (bounded) paragraph above goes back on what we said about nil representation. For consistency with section 9, nil should not terminate the array. This might result in excessive resource consumption, but that is necessary to create the infoset implied by the schema. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM@IBMGB To: dfdl-wg@ogf.org, Date: 29/01/2014 15:12 Subject: [DFDL-WG] Fw: Spec question: 16.6 Forward Progress Requirement Sent by: dfdl-wg-bounces@ogf.org Proposed revised wording: 16.6 Forward Progress Requirement When parsing an optional occurrence of an array element, if the element is found to have a zero-length representation and its associated separator region is also zero-length then it is a processing error. An optional occurrence is defined as follows: - any occurrence of an element with occursCountKind='implicit' and index > XSDL minOccurs - any occurrence of an element with occursCountKind='parsed' or 'stopValue' If occursCountKind is 'stopValue' then the element will not be a point of uncertainty and the processing error will invalidate the entire array. For occursCountKind='implicit' or 'parsed' the processing error will terminate the array and will thus avoid either redundant processing or an infinite loop ( when XSDL maxOccurs='unbounded'). Feel free to propose better terminology. I'm not sure whether 'optional occurrence' has been used elsewhere in the specification with a different meaning. The final paragraph is conveying important information but can be reworded if necessary. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 ----- Forwarded by Tim Kimber/UK/IBM on 29/01/2014 14:42 ----- From: Tim Kimber/UK/IBM To: dfdl-wg@ogf.org, Date: 28/01/2014 14:24 Subject: Spec question: 16.6 Forward Progress Requirement 16.6 Forward Progress Requirement It is a processing error when an array is potentially unbounded and the position in the data does not move during the parsing of an occurrence of the element including any associated separator. This is to prevent an infinite loop. An array is potentially unbounded if any of the following are true: * dfdl:occursCountKind is 'parsed' or 'stopValue' * dfdl:occursCountKind is 'implicit' and XSDL maxOccurs is 'unbounded' Q1: Is this paragraph intended to a) safely terminate the parsing of the array, allowing the parser to proceed with whatever follows in the DFDL schema or b) cause the parser to discard the entire array along with everything that was parsed within the nearest point of uncertainty? If b) then I think a Schema Definition Error would be more appropriate ( because backing out the entire array implies that none of the occurrences were well-formed and some later branch might be the correct option ). Although... Q2: If the array has occursCountKind='stopValue' then none of the occurrences is an actual point of uncertainty because only stopValue can terminate the array. If paragraph 16.1 is invoked then a processing error will be thrown. But the occurrence is not a point of uncertainty so the entire array will be treated as badly-formed. In other words, behaviour will be as described in option b) above. I think that is correct in this case because the array's stopValue was never found. Q3: Should this infinite-loop detection be performed only after at least minOccurs occurrences of the array have been processed successfully? Until then, I think an empty occurrence should cause the default value to be put into the info set. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU