newVariableInstance direction property discussion - PCAP example
For our next conference call, I said I would walk through the Apache Daffodil newVariableInstance feature, and how for unparsing we had to add a direction property to variables so they can be evaluated with forward referencing for unparsing. The code I'd like to discuss is here: https://github.com/DFDLSchemas/PCAP/blob/cb9ee66aa565d982f01130918bcd785def5... The defineVariable that is the best motivating example is defined at line 104, and is used at line 213. This is a simple common-subexpression elimination, but it is used to avoid redundant calls to dfdl:valueLength in the dfdl:outputValueCalc of the stored length field named InclLen. There is a second example of using newVariableInstance at unparse time in this schema also, with the variable definitions starting at line 115, and usage in the group defined starting at line 320. This usage is far more complex, and in hindsight I think a DFDL parse of PCAP data should not be creating IP addresses in the Infoset that look like "1.2.3.4", because it requires all this complexity to unparse from that back to the binary representation. However, it is a good exercise in how the newVariableInstance feature works. -mikeb Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>
Hi, We are currently working on supporting newVariableInstance in DFDL4S, so I'd like attend the next conference call and tune in on this discussion. Can anyone help me, and pointing me to the next call(s) schedule/coordinates? I've noticed that there was a move to Github, and I couldn't locate this information. --Marcos On Mon, Apr 12, 2021 at 7:26 PM Mike Beckerle <mbeckerle.dfdl@gmail.com> wrote:
For our next conference call, I said I would walk through the Apache Daffodil newVariableInstance feature, and how for unparsing we had to add a direction property to variables so they can be evaluated with forward referencing for unparsing.
The code I'd like to discuss is here:
https://github.com/DFDLSchemas/PCAP/blob/cb9ee66aa565d982f01130918bcd785def5...
The defineVariable that is the best motivating example is defined at line 104, and is used at line 213. This is a simple common-subexpression elimination, but it is used to avoid redundant calls to dfdl:valueLength in the dfdl:outputValueCalc of the stored length field named InclLen.
There is a second example of using newVariableInstance at unparse time in this schema also, with the variable definitions starting at line 115, and usage in the group defined starting at line 320. This usage is far more complex, and in hindsight I think a DFDL parse of PCAP data should not be creating IP addresses in the Infoset that look like "1.2.3.4", because it requires all this complexity to unparse from that back to the binary representation. However, it is a good exercise in how the newVariableInstance feature works.
-mikeb
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy
-- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
-- HE Space for ESA - European Space Agency Marcos Bento Mission Analysis Software Engineer System Support Division Earth Observation Projects Department Directorate of Earth Observation Programmes ESTEC Keplerlaan 1, PO Box 299 NL-2200 AG Noordwijk, The Netherlands marcos.bento@esa.int | www.esa.int T +31 71 565 3749 This message is intended only for the recipient(s) named above. It may contain proprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo@esa.int).
Hi Marcos Your email is in our DFDL WG call calendar invite, which also has all the links; are you not receiving them? Next call is Thurs 15/4 @ 4pm UK. OGF DFDL WG call dial-in details. https://ibm.webex.com/meet/smh GitHub DFDL: https://github.com/OpenGridForum/DFDL Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh@uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday From: "Marcos Bento (external)" <Marcos.Bento@esa.int> To: DFDL-WG <dfdl-wg@ogf.org> Date: 13/04/2021 08:54 Subject: [EXTERNAL] Re: [DFDL-WG] newVariableInstance direction property discussion - PCAP example Sent by: "dfdl-wg" <dfdl-wg-bounces@ogf.org> Hi, We are currently working on supporting newVariableInstance in DFDL4S, so I'd like attend the next conference call and tune in on this discussion. Can anyone help me, and pointing me to the next call(s) schedule/coordinates? I've noticed that there was a move to Github, and I couldn't locate this information. --Marcos On Mon, Apr 12, 2021 at 7:26 PM Mike Beckerle <mbeckerle.dfdl@gmail.com> wrote:
For our next conference call, I said I would walk through the Apache
Daffodil newVariableInstance feature, and how for unparsing we had to add a direction property to variables so they can be evaluated with forward referencing for unparsing.
The code I'd like to discuss is here:
https://github.com/DFDLSchemas/PCAP/blob/cb9ee66aa565d982f01130918bcd785def5...
The defineVariable that is the best motivating example is defined at
line 104, and is used at line 213. This is a simple common-subexpression elimination, but it is used to avoid redundant calls to dfdl:valueLength in the dfdl:outputValueCalc of the stored length field named InclLen.
There is a second example of using newVariableInstance at unparse time
in this schema also, with the variable definitions starting at line 115, and usage in the group defined starting at line 320. This usage is far more complex, and in hindsight I think a DFDL parse of PCAP data should not be creating IP addresses in the Infoset that look like "1.2.3.4", because it requires all this complexity to unparse from that back to the binary representation. However, it is a good exercise in how the newVariableInstance feature works.
-mikeb
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense |
www.owlcyberdefense.com
Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy
-- dfdl-wg mailing list dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg -- HE Space for ESA - European Space Agency Marcos Bento Mission Analysis Software Engineer System Support Division Earth Observation Projects Department Directorate of Earth Observation Programmes ESTEC Keplerlaan 1, PO Box 299 NL-2200 AG Noordwijk, The Netherlands marcos.bento@esa.int | www.esa.int T +31 71 565 3749 This message is intended only for the recipient(s) named above. It may contain proprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo@esa.int). -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Hi Steve, Thanks for the information; I've added it to my calendar. Maybe I'm missing something, but I usually only get an email with the minutes of the last call (after the fact, obviously). In the past I remember receiving a number of invites/ics, but it has been a while since I get any such email with an actual call/invite/ics. --Marcos On Tue, Apr 13, 2021 at 10:10 AM Steve Hanson <smh@uk.ibm.com> wrote:
Hi Marcos
Your email is in our DFDL WG call calendar invite, which also has all the links; are you not receiving them? Next call is Thurs 15/4 @ 4pm UK.
OGF DFDL WG call dial-in details. https://ibm.webex.com/meet/smh
GitHub DFDL: https://github.com/OpenGridForum/DFDL
Regards
Steve Hanson
IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh@uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday
From: "Marcos Bento (external)" <Marcos.Bento@esa.int> To: DFDL-WG <dfdl-wg@ogf.org> Date: 13/04/2021 08:54 Subject: [EXTERNAL] Re: [DFDL-WG] newVariableInstance direction property discussion - PCAP example Sent by: "dfdl-wg" <dfdl-wg-bounces@ogf.org>
________________________________
Hi,
We are currently working on supporting newVariableInstance in DFDL4S, so I'd like attend the next conference call and tune in on this discussion.
Can anyone help me, and pointing me to the next call(s) schedule/coordinates? I've noticed that there was a move to Github, and I couldn't locate this information.
--Marcos
On Mon, Apr 12, 2021 at 7:26 PM Mike Beckerle <mbeckerle.dfdl@gmail.com> wrote:
For our next conference call, I said I would walk through the Apache Daffodil newVariableInstance feature, and how for unparsing we had to add a direction property to variables so they can be evaluated with forward referencing for unparsing.
The code I'd like to discuss is here:
https://github.com/DFDLSchemas/PCAP/blob/cb9ee66aa565d982f01130918bcd785def5...
The defineVariable that is the best motivating example is defined at line 104, and is used at line 213. This is a simple common-subexpression elimination, but it is used to avoid redundant calls to dfdl:valueLength in the dfdl:outputValueCalc of the stored length field named InclLen.
There is a second example of using newVariableInstance at unparse time in this schema also, with the variable definitions starting at line 115, and usage in the group defined starting at line 320. This usage is far more complex, and in hindsight I think a DFDL parse of PCAP data should not be creating IP addresses in the Infoset that look like "1.2.3.4", because it requires all this complexity to unparse from that back to the binary representation. However, it is a good exercise in how the newVariableInstance feature works.
-mikeb
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy
-- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
-- HE Space for ESA - European Space Agency Marcos Bento Mission Analysis Software Engineer System Support Division Earth Observation Projects Department Directorate of Earth Observation Programmes
ESTEC Keplerlaan 1, PO Box 299 NL-2200 AG Noordwijk, The Netherlands marcos.bento@esa.int | www.esa.int T +31 71 565 3749
This message is intended only for the recipient(s) named above. It may contain proprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo@esa.int).
-- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
This message is intended only for the recipient(s) named above. It may contain proprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo@esa.int).
participants (3)
-
Marcos Bento (external) -
Mike Beckerle -
Steve Hanson