September 2012 - dfdl-wg

Re: [DFDL-WG] Issue 140 spinoff - proposed revised grammar
by Mike Beckerle 19 Nov '12

19 Nov '12

Attached revised grammar 005 - proposed as "to become an errata". On Mon, Jul 30, 2012 at 6:29 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: > A couple of further revisions. > > > > Regards > > Steve Hanson > Architect, Data Format Description Language (DFDL) > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK* > **smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:+44-1962-815848 > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> > To: Steve Hanson/UK/IBM@IBMGB > Cc: Tim Kimber/UK/IBM@IBMGB > Date: 25/07/2012 16:49 > Subject: Re: Issue 140 spinoff - proposed revised grammar > ------------------------------ > > > > Revised per call today. For your consideration. > > ...mikeb > > On Wed, Jul 11, 2012 at 7:39 AM, Steve Hanson <*smh(a)uk.ibm.com*<smh(a)uk.ibm.com>> > wrote: > I've made some updates which I think solves the complex empty issue, ties > the grammar more closely to the action 140 rep definitions, and addresses > Tim's content v value distinction. > > Let me know what you think. > > > > Regards > > Steve Hanson > Architect, Data Format Description Language (DFDL) > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK* > **smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:*+44-1962-815848* <%2B44-1962-815848> > > > > From: Mike Beckerle <*mbeckerle.dfdl(a)gmail.com*<mbeckerle.dfdl(a)gmail.com> > > > To: Steve Hanson/UK/IBM@IBMGB, Tim Kimber/UK/IBM@IBMGB > Date: 10/07/2012 16:25 > Subject: Re: Issue 140 spinoff - proposed revised grammar > ------------------------------ > > > > > Accept-all-changes on that document before you try to read it. Or use the > attached. > > On Tue, Jul 10, 2012 at 11:19 AM, Mike Beckerle <*mbeckerle.dfdl(a)gmail.com > * <mbeckerle.dfdl(a)gmail.com>> wrote: > I took a crack at revising the grammar to something more accurate, more > useful. It's bigger, but not 3 times as big. > > I like it, but then I'm an implementor, not a user of this spec trying to > devise a schema. Our audience is both however. > > I am curious what you think. > > One thing it cannot capture is the complexity of complex empty's, i.e., > the fact that a complex empty requires one to sometimes descend recursively > first, and then check for empty. From the grammar you would think you can > always check for this first. > > Fact is, you have to descend recursively in order to pop back up, and have > the representation be found to match what I call EmptyElement. But the > grammar can't show that. (well, maybe it could... I'll think about that > further.) > > ...mike > > -- > Mike Beckerle | OGF DFDL WG Co-Chair > Tel: *781-330-0412* <781-330-0412> > > > > > -- > Mike Beckerle | OGF DFDL WG Co-Chair > Tel: *781-330-0412* <781-330-0412> > [attachment "grammar-revision.docx" deleted by Steve Hanson/UK/IBM] > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > > -- > Mike Beckerle | OGF DFDL WG Co-Chair > Tel: 781-330-0412 > [attachment "grammar-revision-003.doc" deleted by Steve Hanson/UK/IBM] > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

2 5

Re: [DFDL-WG] proposed clarification/narrowing - no twos-complement decimal, integer or nonNegativeInteger
by Steve Hanson 19 Nov '12

19 Nov '12

Agreed on DFDL WG call 4th Sept to continue to support twos-complement decimal, integer or nonNegativeInteger, but to make the maximum implementation defined. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Date: 15/08/2012 13:10 Subject: Re: [DFDL-WG] proposed clarification/narrowing - no twos-complement decimal, integer or nonNegativeInteger Mike Having thought about this since our call yesterday, I am uncomfortable with the proposed narrowing. It means DFDL can not be guaranteed to handle the two's complement result of Java BigInteger.toByteArray(). I don't have a concrete use case of two's complement data > 8 bytes but I am sure that someone in the scientific community will need it at some point. Hence why it is allowed in the spec today. If we do allow it, then we have to decide what happens during unparsing and lengthKind is 'prefixed'. The equivalent of Table 16 for lengthKind 'pattern' is needed for lengthKind 'prefixed', as we discussed. That would imply that the unparsing behaviour would be 'Minimum number of bytes to represent significant digits and sign'. In other words, a two's complement binary integer is created that is just long enough to represent the infoset value. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: dfdl-wg(a)ogf.org Date: 14/08/2012 19:33 Subject: [DFDL-WG] proposed clarification/narrowing - no twos-complement decimal, integer or nonNegativeInteger Sent by: dfdl-wg-bounces(a)ogf.org The spec is not clear on whether decimal, integer, and nonNegativeInteger support representation='binary' with binaryNumberRep='binary' (aka twos-complement). Since these types do not have any size limit, such a capability allows arbitrarily large twos-complement integers, or perhaps it would be an implementation defined limit. I believe the right choice is that it should be disallowed. That is, only binaryNumberRep of 'bcd' or 'packed' should be allowed for these types. These types have always been about decimal-style representations to me. This keeps twos-complement only for the types long, int, short, byte and their unsigned counterparts. -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412 -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

Next DFDL WG meetings
by Steve Hanson 28 Sep '12

28 Sep '12

Tues 2nd Oct - at later time 20:00 UK (15:00 Eastern) Tues 9th Oct - at later time 20:00 UK (15:00 Eastern) Regards Steve Hanson Strategy, Common Transformation & DFDL Co-Chair, OGF DFDL WG IBM SWG, Hursley, UK, smh(a)uk.ibm.com, tel +44-(0)1962-815848

1 0

OGF DFDL WG Call Minutes 2012-09-28
by Steve Hanson 28 Sep '12

28 Sep '12

Please find minutes from the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Errata document is now at v010 here: http://forge.gridforum.org/sf/go/doc16280?nav=1 Regards Steve Hanson Architect, DFDL, IBM SWG Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Rescheduled: OGF DFDL Working Group weekly call (2 Oct 20:00 GDT)
by Steve Hanson 28 Sep '12

28 Sep '12

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl GridForge DFDL: http://forge.ogf.org/projects/dfdl-wg/ Moved to later time

1 0

Rescheduled: OGF DFDL Working Group weekly call (28 Sep 14:00 GDT in IBM Hursley DE2K09)
by smh＠uk.ibm.com 26 Sep '12

26 Sep '12

Rescheduled: OGF DFDL Working Group weekly call 28/09/2012 - (Repeats) Chair: Steve Hanson/UK/IBM Sent By: Steve_Hanson/UK/IBM%IBMGB Location: IBM Hursley DE2K09 Steve_Hanson Steve Hanson has rescheduled this meeting. You have not yet responded. Repeats This entry repeats Required: mbeckerle.dfdl(a)gmail.com, Stephanie Fetzer/Charlotte/IBM@IBMUS, Suman Kalia/Toronto/IBM@IBMCA, Tim Kimber/UK/IBM@IBMGB Optional: a.mumfor(a)radium.ncsc.mil, boyd.fletcher(a)us.army.mil, bradley.r.sexton(a)gmail.com, dfdl-wg(a)ogf.org, jcranford(a)mitre.org, marcos.bento(a)deimos.com.pt, remcgrat(a)illinois.edu OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl GridForge DFDL: http://forge.ogf.org/projects/dfdl-wg/

1 0

OGF DFDL WG Call Agenda 2012-09-26 - day later than normal
by Steve Hanson 25 Sep '12

25 Sep '12

Please find agenda for call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… . Note that the call has been put back a day, and the time brought forward to 13:30 UK (8:30 Eastern). Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Rescheduled: OGF DFDL Working Group weekly call (26 Sep 13:30 GDT in IBM Hursley DE2K09)
by Steve Hanson 24 Sep '12

24 Sep '12

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl GridForge DFDL: http://forge.ogf.org/projects/dfdl-wg/ Rescheduling as Steve is not available on Tuesday

1 0

OGF DFDL WG Call Minutes 2012-09-18 & follow-up
by Steve Hanson 20 Sep '12

20 Sep '12

Please find minutes from the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Errata document is now at v010 here: http://forge.gridforum.org/sf/go/doc16280?nav=1 Regards Steve Hanson Architect, DFDL, IBM SWG Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] Specify field based on the beggining of the
by Miguel Vieira 19 Sep '12

19 Sep '12

Well yes this structure is ok. The problem is that my next fields will continue after field 4. For instance field5 would start right after the end of field4. Going back to my data description: Header: 512Bytes; (bytes 1-512) Field1: 21504 bytes (21KB); (bytes 513-21016) Field2: 5Bytes; (bytes 21017-21021) Field3: 1Byte; (bytes 21021-21022) Field4: 3 Bytes - Stating at byte 316 counting from the beginning of the file; (bytes 316-318) Field5:10Bytes; (bytes 319-318) ...; bytes (319-...) Basically I would need to do something like fseek function in C language :) (http://www.cplusplus.com/reference/clibrary/cstdio/fseek/) Message: 1 Date: Tue, 18 Sep 2012 09:20:26 +0100 From: Tim Kimber <KIMBERT(a)uk.ibm.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Subject: Re: [DFDL-WG] Specify field based on the beggining of the binary file Message-ID: <OF826A1ADE.83551CD7-ON80257A7D.002D3AE6-80257A7D.002DD59A(a)uk.ibm.com> Content-Type: text/plain; charset="utf-8" Hi Miguel, I work with Steve, and we were discussing this a moment ago. Would it be acceptable to get this structure in the info set: Document ( complex element ) |---Header ( complex element ) |---|----Field4 ( simple element ) |---Body |---|---Field1 ( simple element ) |---|---Field2 ( simple element ) |---|---Field3 ( simple element ) If you really need to get the entire Header field as a BLOB *and* get Field4 as a separate field then you could achieve that using inputValueCalc. The value of Field4 would be calculated from the BLOB or string value of Header using XPath string operations. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Steve Hanson/UK/IBM@IBMGB To: Miguel Vieira <carlos.vieira(a)gmv.com>, Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 17/09/2012 18:29 Subject: Re: [DFDL-WG] Specify field based on the beggining of the binary file Sent by: dfdl-wg-bounces(a)ogf.org It sounds like you need two DFDL schemas and two parse operations. The first schema models the whole message. It takes the whole data stream as input and creates an infoset for Header, Field1, Field2, Field3. The second schema models just Field 4 using leadingSkip and trailingSkip to position correctly. It takes the Header from the first infoset as input, and creates an infoset for Field4. <xs:element name="Field4" type="xs:hexBinary" dfdl:lengthKind="explicit" dfdl:length="3" dfdl:lengthUnits="bytes" dfdl:leadingSkip="316" dfdl:trailingSkip="193" dfdl:alignmentUnits="bytes" /> Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Miguel Vieira <carlos.vieira(a)gmv.com> To: Steve Hanson/UK/IBM@IBMGB Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 17/09/2012 18:04 Subject: RE: [DFDL-WG] Specify field based on the beggining of the binary file Hi Steve! I need them both. From: Steve Hanson [mailto:smh@uk.ibm.com] Sent: segunda-feira, 17 de Setembro de 2012 18:00 To: Miguel Vieira Cc: dfdl-wg(a)ogf.org Subject: RE: [DFDL-WG] Specify field based on the beggining of the binary file Is Field4 the only part of the Header that is of interest? Do you want the full Header (as a 512 byte BLOB) and Field4 in the infoset? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Miguel Vieira <carlos.vieira(a)gmv.com> To: Steve Hanson/UK/IBM@IBMGB Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 17/09/2012 16:06 Subject: RE: [DFDL-WG] Specify field based on the beggining of the binary file First of all: Thanks! J My binary data specification is: Header: 512Bytes; Field1: 21504 bytes (21KB); Field2: 5Bytes; Field3: 1Byte; Field4: 3 Bytes ?Stating at byte 316 counting from the beginning of the file. This means that Field4 is placed inside the Header field. ?leadingSkipBytes? or ?alignment? would do, but can you specify the place you start counting? Thanks. Miguel From: Steve Hanson [mailto:smh@uk.ibm.com] Sent: segunda-feira, 17 de Setembro de 2012 15:41 To: Miguel Vieira Cc: dfdl-wg(a)ogf.org Subject: Re: [DFDL-WG] Specify field based on the beggining of the binary file Miguel, please can you send us a graphic of the data in question, I want to make sure I have understood your data correctly. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Miguel Vieira <carlos.vieira(a)gmv.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 17/09/2012 15:03 Subject: [DFDL-WG] Specify field based on the beggining of the binary file Sent by: dfdl-wg-bounces(a)ogf.org Hi! I have a binary file specification that specifies some fields referenced by the beginning of the binary info. For instance imagine that there is a field with 1024Bytes and the next one starts at byte 512 counting from the beginning of the info. Is it possible to do that using dfdl? I?ve seen dfdl:floating, but I can?t find much info on that. Thanks. P Please consider the environment before printing this e-mail. This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informaci?n clasificada por su emisor como confidencial en el marco de su Sistema de Gesti?n de Seguridad de la Informaci?n siendo para uso exclusivo del destinatario, quedando prohibida su divulgaci?n copia o distribuci?n a terceros sin la autorizaci?n expresa del remitente. Si Vd. ha recibido este mensaje err?neamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboraci?n. Esta mensagem, incluindo qualquer ficheiro anexo, pode conter informa??o confidencial, de acordo com nosso Sistema de Gest?o de Seguran?a da Informa??o, sendo para uso exclusivo do destinat?rio e estando proibida a sua divulga??o, c?pia ou distribui??o a terceiros sem autoriza??o expressa do remetente da mesma. Se recebeu esta mensagem por engano, por favor avise de imediato o remetente e apague-a. Obrigado pela sua colabora??o. -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU P Please consider the environment before printing this e-mail. This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacin clasificada por su emisor como confidencial en el marco de su Sistema de Gestin de Seguridad de la Informacin siendo para uso exclusivo del destinatario, quedando prohibida su divulgacin copia o distribucin a terceros sin la autorizacin expresa del remitente. Si Vd. ha recibido este mensaje errneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracin. Esta mensagem, incluindo qualquer ficheiro anexo, pode conter informao confidencial, de acordo com nosso Sistema de Gesto de Segurana da Informao, sendo para uso exclusivo do destinatrio e estando proibida a sua divulgao, cpia ou distribuio a terceiros sem autorizao expressa do remetente da mesma. Se recebeu esta mensagem por engano, por favor avise de imediato o remetente e apague-a. Obrigado pela sua colaborao. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU P Please consider the environment before printing this e-mail. This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacin clasificada por su emisor como confidencial en el marco de su Sistema de Gestin de Seguridad de la Informacin siendo para uso exclusivo del destinatario, quedando prohibida su divulgacin copia o distribucin a terceros sin la autorizacin expresa del remitente. Si Vd. ha recibido este mensaje errneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracin. Esta mensagem, incluindo qualquer ficheiro anexo, pode conter informao confidencial, de acordo com nosso Sistema de Gesto de Segurana da Informao, sendo para uso exclusivo do destinatrio e estando proibida a sua divulgao, cpia ou distribuio a terceiros sem autorizao expressa do remetente da mesma. Se recebeu esta mensagem por engano, por favor avise de imediato o remetente e apague-a. Obrigado pela sua colaborao. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

4 3