Re: [DFDL-WG] DFDL pointer & offset proposal

Actually, the target of the indirection is just like an element or model-group, except without alignment regions. Ugly to have to repeat the whole grammar for that. Perhaps we can get away with just documenting that the alignment regions must be zero length when target of an indirection? I did not try to put this modified grammar into my commented copy of the proposal (attached). The "connection" to the indirect pointer or offset is not something that the grammar can express. Attached * my review (with new comments that start with "V4:"), * my first cut of a DFDL schema for TIFF v6.0, * a copy of the TIFF v6.0 spec. I wasn't actually able to pull off TIFF. Comments in the schema discuss why, which is due to TIFF being a linked list, and I'm not able to terminate the chain. There's maybe a trick using negative offsets, but I don't want to consider that yet, as those are entirely problematic. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy http://www.ogf.org/About/abt_policies.php On Fri, Jul 24, 2020 at 7:03 AM Steve Hanson wrote:

Bradd, Mike - as discussed on WG call please review my version of the grammar for next call.

The key issue is how to link the *target* of the indirection into the main grammar, as it is dis-contiguous.

Regards

Steve Hanson

IBM Hybrid Integration, Hursley, UK Architect, *IBM DFDL* http://www.ibm.com/developerworks/library/se-dfdl/index.html Co-Chair, *OGF DFDL Working Group* http://www.ogf.org/dfdl/ *smh@uk.ibm.com* tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday

From: Bradd Kadlecik/Poughkeepsie/IBM To: Steve Hanson/UK/IBM Cc: DFDL-WG , "dfdl-wg" , Mike Beckerle Date: 23/07/2020 14:52 Subject: Re: [EXTERNAL] Re: [DFDL-WG] DFDL pointer & offset proposal ------------------------------

I incorporated some of your grammar changes but had a few modifications of my own we can discuss further in case I'm not understanding some of the subtleties. I think there may be a case for a nil value with indirection however as well. I've updated the document to include the changes as version 4.

Regards,

*Bradd Kadlecik* z/TPF Development ------------------------------ *Phone:* 1-845-433-1573 * E-mail:* *braddk@us.ibm.com* 2455 South Rd Poughkeepsie, NY 12601-5400 United States

From: Steve Hanson/UK/IBM To: Bradd Kadlecik/Poughkeepsie/IBM@IBMUS Cc: DFDL-WG , "dfdl-wg" , Mike Beckerle Date: 07/10/2020 08:11 AM Subject: Re: [EXTERNAL] Re: [DFDL-WG] DFDL pointer & offset proposal ------------------------------

I think 'indirect' in property names should be 'indirection', so I will assume that in what follows.

I am not happy with 'indirectionEmptyValue' or 'indirectionMissingValue'. Both 'empty' and 'missing' are concepts with well-defined meanings in DFDL. How about 'indirectionInactiveValue' or better 'indirectionUnusedValue'?

My stab at an updated grammar below (changes in blue). There is no concept of an empty indirection value or a nil indirection value, so we don't need those clauses.

I also think we can simplify the EnclosedElement construct (changes in red )

Bradd I know you were struggling with how to link the *target* of the indirection into the main grammar, as it is dis-contiguous. I think we can achieve that by incorporating the target(s) into the Document as a whole.

If we think that the concept of an indirection can only ever happen inside a complex type, then we could move IndirectionRep from SimpleElement/ComplexElement to EnclosedElement.

I don't think it is worth getting more specific with IndirectElement. Specifically, it has no LeadingAlignment or TrailingAlignment, but to reflect that requires duplicating most of the rest of the grammar.

You have not disallowed lengthKind 'prefixed' for the 'indirectionType', so that is included in the grammar.

*Productions*

Document = DocumentElement IndirectElement*

DocumentElement = SimpleElement | ComplexElement

SimpleElement = SimpleLiteralNilElementRep | SimpleEmptyElementRep | SimpleNormalRep | IndirectionRep SimpleEnclosedElement = SimpleElement | AbsentElementRep

ComplexElement = ComplexLiteralNilElementRep | ComplexNormalRep | ComplexEmptyElementRep | IndirectionRep ComplexEnclosedElement = ComplexElement | AbsentElementRep

EnclosedElement = SimpleElement | ComplexElement | AbsentElementRep

IndirectElement = SimpleElement | ComplexElement

AbsentElementRep = *Absent*

IndirectionRep = IndirectionLeftFraming PrefixLength IndirectionContent IndirectionRightFraming

IndirectionLeftFraming = LeadingAlignment IndirectionRightFraming = TrailingAlignment

IndirectionContent = *LeftPadding* [ IndirectionValue* | IndirectionUnusedValue *] RightPadOrFill

IndirectionValue = *SimpleValue*

SimpleEmptyElementRep = EmptyElementLeftFraming EmptyElementRightFraming ComplexEmptyElementRep = EmptyElementLeftFraming EmptyElementRightFraming

EmptyElementLeftFraming = LeadingAlignment *EmptyElementInitiator * PrefixLength EmptyElementRightFraming = *EmptyElementTerminator* TrailingAlignment

SimpleLiteralNilElementRep = NilElementLeftFraming [*NilLiteralCharacters* | NilElementLiteralContent] NilElementRightFraming ComplexLiteralNilElementRep = NilElementLeftFraming *NilLiteralValue* NilElementRightFraming

NilElementLeftFraming = LeadingAlignment *NilElementInitiator * PrefixLength NilElementRightFraming = *NilElementTerminator* TrailingAlignment

NilElementLiteralContent = *LeftPadding* *NilLiteralValue *RightPadOrFill

SimpleNormalRep = LeftFraming PrefixLength SimpleContent RightFraming ComplexNormalRep = LeftFraming PrefixLength ComplexContent *ElementUnused* RightFraming

LeftFraming = LeadingAlignment *Initiator* RightFraming = *Terminator* TrailingAlignment

PrefixLength = SimpleContent | PrefixPrefixLength SimpleContent PrefixPrefixLength = SimpleContent

SimpleContent = *LeftPadding* [ *NilLogicalValue* | *SimpleValue* ] RightPadOrFill

ComplexContent = Sequence | Choice

Sequence = LeftFraming SequenceContent RightFraming SequenceContent = [ *PrefixSeparator * EnclosedContent [ *Separator* EnclosedContent ]* *PostfixSeparator* ]

Choice = LeftFraming ChoiceContent RightFraming ChoiceContent = [ EnclosedContent ] *ChoiceUnused*

EnclosedContent = [ EnclosedElement | Array | Sequence | Choice ]

Array = [ EnclosedElement [ *Separator* EnclosedElement ]* [ *Separator* StopValue] ]

StopValue = SimpleElement

LeadingAlignment = *LeadingSkip* *AlignmentFill* TrailingAlignment = *TrailingSkip* RightPadOrFill = *RightPadding* | *RightFill* | *RightPadding RightFill*

Regards

Steve Hanson

IBM Hybrid Integration, Hursley, UK Architect, *IBM DFDL* http://www.ibm.com/developerworks/library/se-dfdl/index.html Co-Chair, *OGF DFDL Working Group* http://www.ogf.org/dfdl/ *smh@uk.ibm.com* tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday

From: Bradd Kadlecik/Poughkeepsie/IBM To: Steve Hanson/UK/IBM Cc: DFDL-WG , "dfdl-wg" , Mike Beckerle Date: 09/07/2020 15:09 Subject: Re: [EXTERNAL] Re: [DFDL-WG] DFDL pointer & offset proposal ------------------------------

I have updated the document proposal with proposed DFDL grammar changes: [attachment "DFDL_Indirection_v3.docx" deleted by Bradd Kadlecik/Poughkeepsie/IBM]

Regards,

*Bradd Kadlecik* z/TPF Development ------------------------------ *Phone:* 1-845-433-1573 * E-mail:* *braddk@us.ibm.com* 2455 South Rd Poughkeepsie, NY 12601-5400 United States

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU