I have not looked at TDML before but it does look interesting. I'm sure the z/TPF DFDL parser will likely need a way to use this in the future so that we can validate the behavior. Having an expected binary output for pointers does present a problem but the way I'd likely try to do it is by using something we've started using internally. We have tooling that runs an unparse with JSON, XML, BSON, etc as input and creates a binary file. We also can run a parse using a binary file as input to create whatever document format desired. With pointer support this didn't work of course so we made a simple routine in the parser that can create a contiguous blob that exchanges all pointer addresses for offsets. This is called before filing out the binary. When reading in the binary we can call another routine that exchanges all offsets with the pointer address. There's an internal switch to treat all pointers as offsets when either creating the infoset or when creating the data stream from it. This may also be a function users find useful to be able to file down such data and then use it again after reading it back in.

In the end this isn't really validating the pointer data but the behavior between offsets and pointers should be fairly consistent.


Regards,

Bradd Kadlecik
z/TPF Development
Phone: 1-845-433-1573
E-mail: braddk@us.ibm.com
2455 South Rd
Poughkeepsie, NY 12601-5400
United States


Inactive hide details for Mike Beckerle ---03/03/2021 10:43:53 AM---I merged your PR - squashed the 5 commits into one, so thatMike Beckerle ---03/03/2021 10:43:53 AM---I merged your PR - squashed the 5 commits into one, so that the document appears to be just added.

From: Mike Beckerle <mbeckerle.dfdl@gmail.com>
To: Bradd Kadlecik <braddk@us.ibm.com>
Cc: DFDL-WG <dfdl-wg@ogf.org>, Steve Hanson <smh@uk.ibm.com>
Date: 03/03/2021 10:43 AM
Subject: [EXTERNAL] Re: Contributing to github




I merged your PR - squashed the 5 commits into one, so that the document appears to be just added. You can delete the branch if you want, or keep for future use. I was thinking about implementing this feature recently, and I am trying to come

I merged your PR - squashed the 5 commits into one, so that the document appears to be just added.
You can delete the branch if you want, or keep for future use.

I was thinking about implementing this feature recently, and I am trying to come up with a test rig for it.
I'd like to somehow adapt the TDML test language to support testing these concepts in some uniform way where we can all agree on
what the results of a test should be, and exchange such tests.

I think you have to be able to define multiple data "documents" in the TDML test case, and arrange for the "address" of each of them to be somehow embeddable in other ones to form a tree of pointed-to data blocks.

Offsets will be easier, as you can do it all in the same single data document.

I'm curious what you can share about how to test Brad.

Thoughts?



Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com
Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy



On Thu, Feb 18, 2021 at 5:45 PM Bradd Kadlecik <braddk@us.ibm.com> wrote:
Regards,

Bradd Kadlecik
z/TPF Development
Phone: 1-845-433-1573
E-mail: braddk@us.ibm.com
2455 South Rd
Poughkeepsie, NY 12601-5400
United States
Phone: 1-845-433-1573
E-mail: braddk@us.ibm.com
2455 South Rd
Poughkeepsie, NY 12601-5400
United States
Phone: 1-845-433-1573
E-mail: braddk@us.ibm.com
2455 South Rd
Poughkeepsie, NY 12601-5400
United States