- dfdl-wg - lists.ogf.org

Rescheduled: OGF DFDL Working Group call (15 Mar 14:00 GMT in IBM: Hursley DE3S09)
by Steve Hanson 13 Mar '16

13 Mar '16

Calls for 2016. OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 Netherlands Toll-Free 0-800-363-6036 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg DFDL Getting Started: http://www.ibm.com/developerworks/library/se-dfdl/ DFDL 1.0 Spec: http://www.ogf.org/documents/GFD.207.pdf US DST has started bt UK DST has not, so changing the time to keep the US time the same.

1 0

interesting bit-order variation
by Mike Beckerle 09 Mar '16

09 Mar '16

So back when we were working on bit order, we decided that 3 of 4 combinations actually exist in the real world: byte order bigEndian bit order mostSignificantBitFirst byte order littleEndian bit order mostSignificantBitFirst byte order littleEndian bit order leastSignificantBitFirst I have, surprise surprise, run into a data format specification called Asterix, which is public, and which uses the fourth combination. This spec is all big-endian, except when they discuss the bits within a byte they present them in order from most-significant bit first, to least,...... but the labels they assign are least-significant first, so for example they explain the bits of a byte starting by explaining bit 8, then bit 7, etc. down to bit 1. For example: (There's an image below. If you can't see it. it's from page 24 of the spec, URL for spec is below) [image: Inline image 2] Translating this into DFDL with mostSignificantBitFirst is possible. It's just rather inconvenient. you have to take the data item size into account. Some items are 16-bit words so the bits start from bit-16 first, some things are 32-bits and start from bit 32, etc. etc. You have to subtract from the word size to get the "mostSignificantBitFirst" bit position. The spec is relatively modern in that it is byte/octet oriented. Uses 8-bits per character, and pays attention to not spanning byte boundaries with fields except for 2's complement integers, of which a few are 14-bits, i.e., not a multiple of 8-bits long. It's a big spec, but the part I have reviewed is called "cat062" and it is available here: http://www.eurocontrol.int/sites/default/files/content/documents/nm/asterix… ...mike Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

Re: [DFDL-WG] Clarification for nil processing and zero length
by Steve Hanson 04 Mar '16

04 Mar '16

No, because for a complex element to be nil the only allowable value is empty string, so WSP* is too permissive. Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Andrew Edwards/UK/IBM To: Steve Hanson/UK/IBM@IBMGB Cc: Alex Wood1/UK/IBM@IBMGB, Mark Frost/UK/IBM@IBMGB, dfdl-wg(a)ogf.org Date: 04/03/2016 13:06 Subject: Re: Clarification for nil processing and zero length I agree with the proposed change. Would it also apply to 9.3.2.2 for complex elements? That has very similar words. Cheers, Andy Edwards Software Engineer E-mail: andy.edwards(a)uk.ibm.com Snail-mail: MP211, Hursley park, Hursley, WINCHESTER, Hants, SO21 2JN Phone (internal): 247222 Phone (external): 44-1962-817222 Desk: DE3 F16 The Feynman problem solving Algorithm: 1) Write down the problem 2) Think real hard 3) Write down the answer -- Murray Gell-mann in the NY Times ----- Original message ----- From: Steve Hanson/UK/IBM To: DFDL-WG <dfdl-wg(a)ogf.org> Cc: Alex Wood1/UK/IBM@IBMGB, Andrew Edwards/UK/IBM@IBMGB, Mark Frost/UK/IBM Subject: Clarification for nil processing and zero length Date: Fri, Mar 4, 2016 11:35 AM The DFDL 1.0 spec current says: 9.3.2.1 Simple element If the result is length zero as described above, the representation is then established by checking, in order for: 1. nil representation (if %ES; is a literal nil value). 2. empty representation. 3. normal representation (xs:string or xs:hexBinary only) 4. absent representation (if none of the prior representations apply). But should bullet 1 be: 1. nil representation (if either %ES; or %WSP*; on its own is a literal nil value). I added a test to IBM DFDL and found that setting dfdl:nilKind="literalValue" & dfdl:nilValue="%WSP*;" did not match an element value of empty string. That surprised me, and I think the IBM DFDL code is strictly implementing bullet 1. Using "%WSP*;" is useful for allowing zero or more white space to mean <nil>. I could use "%WSP+; %ES" to achieve the same goal but I'm not sure that was the intent here. Quick response appreciated. Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Clarification for nil processing and zero length
by Andrew Edwards 04 Mar '16

04 Mar '16

1 0

[DFDL4S]: Results of OGF DFDL WG Call Agenda 2016-02-16
by Michele Zundo 03 Mar '16

03 Mar '16

Dear Steve/Mike, here few considerations regarding the 2 points in the MoM: 1) Regarding the evaluation of the XPath expressions in the dfdl:discriminator tests: > From: "Rui Mestre (DME)" <rui.mestre(a)deimos.com.pt> > > Our implementation of the DFDL parser is based on a slightly different interpretation of the way the paths are applied. > Our approach is based on the fact that the discriminator expression is not being evaluated at the level of the element > containing the discriminator but instead at the level of the type containing such element. > In the example at hand the type TypeMasterChId is defined with an element "Transfer_Frame_Version" followed by a "choice" among several alternative elements Spacecraft_Id_V*. So the decision of which "choice" branch to choose is done while "located" at the root level of TypeMasterChId. While parsing the sub-components of TypeMasterChId the several branches of the "choice" construct are evaluated (in order) and the first one evaluating to "true" is the one selected. Hence the relative path is written in reference to the top level of the type. Therefore in such cases the paths start with "./" instead of "../". > 2) dfdl:assert vs dmx:assertExpression The question from Steve: “Also you said that you saw no need to implement DFDL assert - so what are the dmx:assertExpression attributes doing? “ >> (Maurizio De Bartolomei) : as far as I understood, the DFDL assert shall be used to check that data is well formed (i.e. the data is compliant to a given structure and can be parsed) and not for data validation (i.e. inspecting the content of a given element and comparing against an expected value). >> The dmx:assertExpression does not check that the data is well formed but only checks the value of a given item against an expected value, if the value is different a message is returned. >> This can be improved by either changing the expression name (i.e. dmx:checkExpression) or investigate if DFDL provides already suitable expressions for content validation without loosing the compactness and clarity of dmx:assertExpression). > From: "Rui Mestre (DME)" <rui.mestre(a)deimos.com.pt> > > Regarding the use of dmx:assertExpression (as opposed to dfdl:assert) we fully agree with Maurizio's answer above > About his statement "(...)or investigate if DFDL provides already suitable expressions for content validation without loosing the compactness and clarity of dmx:assertExpression" we had in fact identified an alternative: a possibility is to use XML Schema facets to capture the assert expression by a restriction to the data type. > However that approach does not cover the semantics currently available in our "custom" operators (in,inrange, et al.). So until the DFDL standard is updated to contain the corresponding semantics it would not be possible to completely replace the dmx:assertExpression with a standard DFDL (or in this case XML) construct. > > ******* > >>> >>>>> >>>>> From: Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> >>>>> Subject: Re: OGF DFDL WG Call Agenda 2016-02-16 >>>>> Date: 1 March 2016 at 19:12:13 GMT+1 >>>>> To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> >>>>> Cc: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Mike Beckerle <mbeckerle(a)tresys.com <mailto:mbeckerle@tresys.com>> >>>>> >>>>> Hi Michele >>>>> >>>>> Here is an example of where I do not think the relative path is correct. The discriminator expression should be {../Transfer_Frame_Version in [0]}. In infoset terms, Spacecraft_Id_V1 is a sibling of Transfer_Frame_Version. >>>>> >>>>> Also you said that you saw no need to implement DFDL assert - so what are the dmx:assertExpression attributes doing? >>>>> >>>>> >>>>> <xs:complexType name="TypeMasterChId"> >>>>> <xs:sequence> >>>>> <xs:element name="Transfer_Frame_Version" type="xs:int" dfdl:lengthKind="explicit" dfdl:lengthUnits="bits" dfdl:length="2" dmx:representation="Integer16" >>>>> dmx:assertExpression="../Transfer_Frame_Version eq 1" dmx:assertMessage="Invalid TF version detected." dmx:assertPropagate="true"/> >>>>> <xs:choice> >>>>> <xs:element name="Spacecraft_Id_V1" type="xs:int" dfdl:lengthKind="explicit" dfdl:lengthUnits="bits" dfdl:length="10" dmx:representation="Integer32" > >>>>> <xs:annotation> >>>>> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> >>>>> <dfdl:discriminator test="{./Transfer_Frame_Version in [0]}" /> >>>>> </xs:appinfo> >>>>> </xs:annotation> >>>>> </xs:element> >>>>> <xs:element name="Spacecraft_Id_V2" type="xs:int" dfdl:lengthKind="explicit" dfdl:lengthUnits="bits" dfdl:length="8" dmx:representation="UInteger8" >>>>> dmx:assertExpression="../Spacecraft_Id_V2 in [82, 83, 84]" dmx:assertMessage="Invalid Spacecraft ID detected."> >>>>> <xs:annotation> >>>>> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> >>>>> <dfdl:discriminator test="{true}" /> >>>>> </xs:appinfo> >>>>> </xs:annotation> >>>>> </xs:element> >>>>> </xs:choice> >>>>> </xs:sequence> >>>>> </xs:complexType> >>>>> >>>>> >>>>> >>>>> Regards >>>>> >>>>> Steve Hanson >>>>> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK >>>>> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> >>>>> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> >>>>> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> >>>>> tel:+44-1962-815848 <tel:+44-1962-815848> >>>>> mob:+44-7717-378890 >>>>> >> >> -- ----------------------------------------- Michele Zundo Head of Ground System Definition and Verification Office EOP-PEP European Space Agency, ESTEC e-mail: michele.zundo(a)esa.int This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email.

2 1

Re: [DFDL-WG] OGF DFDL WG Call Agenda 2016-02-16
by Michele Zundo 02 Mar '16

02 Mar '16

Dear Steve, here the excel sheet. > On 01 Mar 2016, at 19:12 , Steve Hanson <smh(a)uk.ibm.com> wrote: > > Hi Michele > > Please can you send the original spreadsheet ? I want to add another column to record the discussion on the WG call. > Regards > > Steve Hanson > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > ----------------------------------------- Michele Zundo Head of Ground System Definition and Verification Office EOP-PEP European Space Agency, ESTEC e-mail: michele.zundo(a)esa.int This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email.

2 1

OGF DFDL WG Call Minutes 2016-03-01
by Steve Hanson 01 Mar '16

01 Mar '16

Please find minutes from the above call at https://redmine.ogf.org/dmsf_files/13519?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] OGF DFDL WG Call Agenda 2016-02-16
by Steve Hanson 01 Mar '16

01 Mar '16

Hi Michele Please can you send the original spreadsheet ? I want to add another column to record the discussion on the WG call. Here is an example of where I do not think the relative path is correct. The discriminator expression should be {../Transfer_Frame_Version in [0]}. In infoset terms, Spacecraft_Id_V1 is a sibling of Transfer_Frame_Version. Also you said that you saw no need to implement DFDL assert - so what are the dmx:assertExpression attributes doing? <xs:complexType name="TypeMasterChId"> <xs:sequence> <xs:element name="Transfer_Frame_Version" type="xs:int" dfdl:lengthKind="explicit" dfdl:lengthUnits="bits" dfdl:length="2" dmx:representation="Integer16" dmx:assertExpression ="../Transfer_Frame_Version eq 1" dmx:assertMessage="Invalid TF version detected." dmx:assertPropagate="true"/> <xs:choice> <xs:element name="Spacecraft_Id_V1" type="xs:int" dfdl:lengthKind="explicit" dfdl:lengthUnits="bits" dfdl:length="10" dmx:representation="Integer32" > <xs:annotation> <xs:appinfo source=" http://www.ogf.org/dfdl/dfdl-1.0/"> <dfdl:discriminator test="{./Transfer_Frame_Version in [0]}" /> </xs:appinfo> </xs:annotation> </xs:element> <xs:element name="Spacecraft_Id_V2" type="xs:int" dfdl:lengthKind="explicit" dfdl:lengthUnits="bits" dfdl:length="8" dmx:representation="UInteger8" dmx:assertExpression ="../Spacecraft_Id_V2 in [82, 83, 84]" dmx:assertMessage="Invalid Spacecraft ID detected."> <xs:annotation> <xs:appinfo source=" http://www.ogf.org/dfdl/dfdl-1.0/"> <dfdl:discriminator test="{true}" /> </xs:appinfo> </xs:annotation> </xs:element> </xs:choice> </xs:sequence> </xs:complexType> Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Steve Hanson/UK/IBM To: Michele Zundo <michele.zundo(a)esa.int> Cc: DFDL-WG <dfdl-wg(a)ogf.org>, Mike Beckerle <mbeckerle(a)tresys.com> Date: 17/02/2016 08:58 Subject: Re: OGF DFDL WG Call Agenda 2016-02-16 Thanks Michele, we will discuss on the next WG call. Are you able to join? It is on 1st March. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM Integration Bus, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Michele Zundo <michele.zundo(a)esa.int> To: Steve Hanson/UK/IBM@IBMGB Cc: DFDL-WG <dfdl-wg(a)ogf.org>, Mike Beckerle <mbeckerle(a)tresys.com> Date: 16/02/2016 15:09 Subject: Re: OGF DFDL WG Call Agenda 2016-02-16 Dear Steve/Mike, we eventually discussed with our contractor the points raised last summer on the use of DFDL schema included in S2G tool (and now in our underlying parser DLDL4S. The disposition we agreed is recorded for information in the attached pdf. It relates to one of the point of latest minutes. Regards This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. On 16 Feb 2016, at 13:58 , Steve Hanson <smh(a)uk.ibm.com> wrote: Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13517?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----------------------------------------- Michele Zundo Head of Ground System Definition and Verification Office EOP-PEP European Space Agency, ESTEC e-mail: michele.zundo(a)esa.int Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Examples of import of named DFDL format definitions
by Mike Beckerle 01 Mar '16

01 Mar '16

Michele, Attached find a zip file containing two examples. One imports a named format definition in a namespace. This is using a reusable format donated to OGF by IBM named "GeneralPurposeFormat". This is probably the way you would want to do things for DFDL4S. The other example is FYI, just illustrating another way we do this which includes a named schema that is in a no-namespace file, so that the named format ends up in the target namespace of whatever it is being included into. Honestly at this point I'm not sure why we did it this way for our testing stuff, but it's there as an example for completeness sake, or perhaps so as to show how this might work in an implementation that doesn't support XML namespaces at all. The schemaLocation attributes give paths relative to the classpath. At least that's how it works in Daffodil. I hope this is helpful and saves you some time. ...mikeb Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> On Tue, Feb 16, 2016 at 10:07 AM, Michele Zundo <michele.zundo(a)esa.int> wrote: > Dear Steve/Mike, > > we eventually discussed with our contractor the points raised last summer > on the use of > DFDL schema included in S2G tool (and now in our underlying parser DLDL4S. > > The disposition we agreed is recorded for information in the attached pdf. > > It relates to one of the point of latest minutes. > > Regards > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > On 16 Feb 2016, at 13:58 , Steve Hanson <smh(a)uk.ibm.com> wrote: > > Please find agenda for call on Redmine at > https://redmine.ogf.org/dmsf_files/13517?download= > > Regards > > Steve Hanson > Architect, IBM Data Format Description Language (DFDL) > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > *smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:+44-1962-815848 > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int > > > > > > > > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > https://www.ogf.org/mailman/listinfo/dfdl-wg >

1 0

Fwd: OGF DFDL WG Call Agenda 2016-03-01
by Michele Zundo 01 Mar '16

01 Mar '16

> Begin forwarded message: > > From: Michele Zundo <michele.zundo(a)esa.int> > Subject: Re: OGF DFDL WG Call Agenda 2016-03-01 > Date: 1 March 2016 at 15:47:13 GMT+1 > To: Steve Hanson <smh(a)uk.ibm.com> > > do you have a webex ? > > >> On 29 Feb 2016, at 18:00 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: >> >> Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13519?download= <https://redmine.ogf.org/dmsf_files/13519?download=> >> >> Regards >> >> Steve Hanson >> Architect, IBM Data Format Description Language (DFDL) >> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> >> IBM SWG, Hursley, UK >> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> >> tel:+44-1962-815848 <tel:+44-1962-815848> >> > ----------------------------------------- Michele Zundo Head of Ground System Definition and Verification Office EOP-PEP European Space Agency, ESTEC e-mail: michele.zundo(a)esa.int This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email.

1 0