September 2013 - dfdl-wg

Re: [DFDL-WG] OGF DFDL WG Call Agenda 2013-09-17
by Steve Hanson 17 Sep '13

17 Sep '13

The call is at 16:00 UK which is 11am Eastern (5 hours difference). Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: "David E. Martin" <martinde(a)northwestern.edu> To: "Sill, Alan" <alan.sill(a)ttu.edu>, Cc: Steve Hanson/UK/IBM@IBMGB, "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, "Cranford, Jonathan W." <jcranford(a)mitre.org>, Mike Beckerle <mbeckerle(a)tresys.com>, Greg Newby <gbnewby(a)alaska.edu> Date: 17/09/2013 15:19 Subject: Re: OGF DFDL WG Call Agenda 2013-09-17 I believe the call is at 11AM CDT. -David On 9/17/13 9:05 AM, Sill, Alan wrote: > I'm connected now to the conference numb er your provided. > > Alan > > On Sep 17, 2013, at 3:23 PM, Steve Hanson <smh(a)uk.ibm.com> > wrote: > >> Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13123?download= >> >> Regards >> >> Steve Hanson >> Architect, IBM Data Format Description Language (DFDL) >> Co-Chair, OGF DFDL Working Group >> IBM SWG, Hursley, UK >> smh(a)uk.ibm.com >> tel:+44-1962-815848 >> Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

OGF DFDL WG Call Agenda 2013-09-17
by Steve Hanson 17 Sep '13

17 Sep '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13123?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

2 1

Re: [DFDL-WG] Latest draft of DFDL 1.0 specification
by Sill, Alan 16 Sep '13

16 Sep '13

Thanks, Greg. Steve, that link to the Editor project is http://redmine.ogf.org/projects/editor You may be interested in pointing your members to the overall description of the different types of OGF documents and their use, along with various other useful links including copies of the document template that include the latest IPR boilerplate, at the link below. http://redmine.ogf.org/projects/editor/wiki/About_OGF_Documents I recommend that everyone involved read this page. As we are meeting for the GFSG this coming Sunday in Madrid, getting something in front of us that we can act on then, at least to get the review process started, would be the most expeditious way to proceed. Hope this helps, Alan On Sep 12, 2013, at 1:07 PM, Greg Newby <gbnewby(a)alaska.edu> wrote: > Understood, and no problem for pre-allocating a number. Would you > please make a new tracker in the Editor pipeline, so I can put the number > there? That's where we keep track of assigned numbers. It's ok > if you don't add the draft document yet. > > The idea of an expedited review process seems reasonable to me. Once > we have the document, we can bring that suggestion to the GFSG for > their decision on how to handle it. I'm sure they'll be receptive. > > -- Greg > > On Thu, Sep 12, 2013 at 09:52:23AM +0100, Steve Hanson wrote: >> Greg >> >> GFD.152 implies that there is a 15-day GFSG review followed by a 60-day >> public comment period. We had a public comment period for GFD.174, and the >> changes in the revision are based on actual experience from implementers >> and users. I would question whether the public comment period is >> necessary for the revision, and request that the revision just undergoes >> the 15-day GFSG review. There will no doubt be a few more errata as >> implementations progress further, and so we anticipate one more revision >> at some point in the future, and that is the revision that would move to >> full Grid Recommendation. >> >> We are on a tight schedule and want to publish by end of September. We >> have users eagerly awaiting the appearance of the revision, and to publish >> an internal WG draft is not appropriate (IBM product infocenters embed the >> HTML rendering of the spec so needs to be official document). >> >> We requested the GFD number simply so that Mike Beckerle can complete the >> document edits. The document pushes the boundaries of MS Word and I think >> Mike also uses an additional plugin, so edits are best done by him. >> >> Regards >> >> Steve Hanson >> Architect, IBM Data Format Description Language (DFDL) >> Co-Chair, OGF DFDL Working Group >> IBM SWG, Hursley, UK >> smh(a)uk.ibm.com >> tel:+44-1962-815848 >> >> >> >> From: Greg Newby <gbnewby(a)alaska.edu> >> To: Steve Hanson/UK/IBM@IBMGB, >> Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" >> <dfdl-wg(a)ogf.org>, "David E. Martin" <martinde(a)northwestern.edu> >> Date: 11/09/2013 21:07 >> Subject: Re: Latest draft of DFDL 1.0 specification >> >> >> >> Steve, >> >> Is this document in the editor pipeline somewhere? I'm >> not seeing it: >> https://redmine.ogf.org/projects/editor/issues >> >> Usually we wait until a document has undergone most of the >> review process (per GFD #152) before assigning a GFD number. >> If you have a reason to need one sooner, we can allocate >> one sooner. The usual practice, though, is to wait until >> publication is imminent. >> >> We're looking forward to these revisions to #174. >> -- Greg >> >> On Wed, Sep 11, 2013 at 06:08:36PM +0100, Steve Hanson wrote: >>> Alan, Greg >>> >>> Please can you allocate us a new GFD number so we can complete the >>> document? >>> >>> Regards >>> >>> Steve Hanson >>> Architect, IBM Data Format Description Language (DFDL) >>> Co-Chair, OGF DFDL Working Group >>> IBM SWG, Hursley, UK >>> smh(a)uk.ibm.com >>> tel:+44-1962-815848 >>> >>> >>> >>> From: "Sill, Alan" <alan.sill(a)ttu.edu> >>> To: Steve Hanson/UK/IBM@IBMGB, >>> Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" >>> <dfdl-wg(a)ogf.org>, Greg Newby <gbnewby(a)alaska.edu>, "David E. Martin" >>> <martinde(a)northwestern.edu> >>> Date: 11/09/2013 15:49 >>> Subject: Re: Latest draft of DFDL 1.0 specification >>> >>> >>> >>> >>> On Sep 11, 2013, at 9:08 AM, Steve Hanson <smh(a)uk.ibm.com> >>> wrote: >>> >>>> When you say 'new version' what does that imply about version numbers? >> >>> The WG still considers what we are working on to be DFDL 1.0 plus >> errata. >>> Are you suggesting that the new revision is 1.1 ? >>> >>> OGF doesn't generally have a concept or framework for revision numbers >> for >>> specifications. Some working groups do number their specifications, and >> we >>> leave this to the work group to manage. >>> >>> The process of obsoleting and replacing a document does give a good >>> opportunity for changing an internally-managed revision number. >>> >>> Regarding the next DFDL call, this will be during OGF 39. I could try to >> >>> join, but that time overlaps one of the sessions that I should attend. >>> Perhaps David or Greg (or both) could attend your call at 16:00 Tues >> 17th >>> September? If so, please provide details. We'll be happy to >> communicate >>> with you as much as possible on this topic before then, of course. >>> >>> Alan >>> >>> >>> ----- Forwarded by Steve Hanson/UK/IBM on 11/09/2013 16:35 ----- >>> >>> From: Steve Hanson/UK/IBM >>> To: "Sill, Alan" <alan.sill(a)ttu.edu>, >>> Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" >>> <dfdl-wg(a)ogf.org>, Greg Newby <gbnewby(a)alaska.edu>, "David E. Martin" >>> <martinde(a)northwestern.edu> >>> Date: 11/09/2013 15:08 >>> Subject: Re: Latest draft of DFDL 1.0 specification >>> >>> >>> Alan >>> >>> Thanks for your reply. I forgot to say that the next call is on Tues >> 17th >>> September. >>> >>> I think that the WG would probably go for your 2nd suggestion, ie, >> publish >>> an updated P_REC that obsoletes GFD.174 but does not yet propose to >>> promote the spec to full recommendation status, because we know there >> are >>> a few more errata that will be discovered before the implementations are >> >>> completed. >>> >>> When you say 'new version' what does that imply about version numbers? >> The >>> WG still considers what we are working on to be DFDL 1.0 plus errata. >> Are >>> you suggesting that the new revision is 1.1 ? >>> >>> Regards >>> >>> Steve Hanson >>> Architect, IBM Data Format Description Language (DFDL) >>> Co-Chair, OGF DFDL Working Group >>> IBM SWG, Hursley, UK >>> smh(a)uk.ibm.com >>> tel:+44-1962-815848 >>> >>> >>> >>> From: "Sill, Alan" <alan.sill(a)ttu.edu> >>> To: Steve Hanson/UK/IBM@IBMGB, "David E. Martin" >>> <martinde(a)northwestern.edu>, >>> Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" >>> <dfdl-wg(a)ogf.org>, Greg Newby <gbnewby(a)alaska.edu> >>> Date: 11/09/2013 14:51 >>> Subject: Re: Latest draft of DFDL 1.0 specification >>> >>> >>> >>> Steve, >>> >>> Answers inline. Short summary: My recommendation would be to publish a >> new >>> version that obsoletes the current GFD.174, and optionally to use this >>> opportunity to migrate the specification from a P-REC to full REC >> status. >>> This would be facilitated by documenting, in any form that is convenient >> >>> including but not limited to an informational GFD, the experience gained >> >>> from implementations to date. Another option is to publish an updated >>> P_REC that obsoletes GFD.174 but does not yet propose to promote the >> spec >>> to full recommendation status. >>> >>> On Sep 11, 2013, at 6:47 AM, Steve Hanson <smh(a)uk.ibm.com> >>> wrote: >>> >>>> Alan >>>> >>>> Since the DFDL 1.0 specification was published in Feb 2011, the two >>> implementation teams (IBM and the Daffodil project) have identified a >>> number of errata in the specification. These have been recorded in an >>> errata document held on Redmine. The number of errata is currently at >>> around 190, and include both clarifications to the specification and >>> changes that affect an implementation, both major and minor. Typically >> as >>> errata have been raised, the implementation teams include any implied >>> changes in the next release of their implementation. >>> >>> Is there any documentation as to the experience gained from >>> implementations that has led to these updates? I as not as an OGF >>> requirement, but just for information. >>> >>>> Both implementation teams, and users of the two implementations, have >>> requested that the DFDL 1.0 specification is revised to include all >> errata >>> to date, so that the specification more closely reflects the >>> implementations. Accordingly all errata to date have been incorporated >>> into a new revision of the specification, which as a result has grown >> from >>> 168 pages to 234 pages. >>>> >>>> This new revision of the specification supersedes the original. >>> >>> This statement is what leads to my suggestion to publish a new document. >> >>> Note that our procedures do allow for replacement of a REC or P-REC for >>> non-normative changes that do not substantially affect compatibility of >>> implementations, but that just clarify or correct errors in the original >> >>> publication. It is the statement that the new revision supersedes the >>> original coupled with your earlier observation that major implementation >> >>> issues are addressed in your new version that causes me to suggest that >>> you pursue a new GFD that obsoletes the old one. >>> >>>> There is no implementation that exactly reflects the original as >>> published on the OGF web site, they both adhere more closely to the new >>> revision. The DFDL WG would therefore like to publish the new revision. >>> The DFDL WG also recognises that there may be comments against the new >>> revision, and that there may still be some errata undetected by the >>> implementation teams, so that a further revision may be necessary in the >> >>> future. Nonetheless it is important that the new revision in its current >> >>> form is externally visible, and not just kept as an internal working >>> document, as there are now many dozens of DFDL users, and they need an >>> up-to-date specification. In particular, IBM DFDL wants to ship the HTML >> >>> version of the new revision to IBM customers in its next release. >>>> >>>> We are looking to the OGF for guidance on how next to proceed. >>> >>> My guidance would be to publish a new version that obsoletes the old >> one, >>> and optionally to use this opportunity to advance the specification from >> >>> P-REC to REC status. Note that this is exactly the pattern that OGF >>> documents are supposed to follow in the life cycle described in GFD.152 >> -- >>> experience gained fro real-world implementation is fed back to produce a >> >>> new version of the specification, which at some point can declare itself >> >>> to be mature enough to request full REC status. >>> >>> As for further revisions, they would be handled by the procedure I >>> mentioned above: non-normative changes can be folded in as corrections >>> through the errata process. THis is controlled by the OGF editor (Greg >>> Newby) and whether to accept and publish an errata is decided >> essentially >>> entirely by his recommendation to the GFSG (Standards Council) to do so. >> >>> If the changes would affect the interoperability of implementations >>> written to the earlier spec in a substantial way, they should be handled >> >>> by the process of a new publication that obsoletes the old one, as we >> have >>> just discussed. >>> >>> The process of going from a P-REC to a REC is largely decoupled from >>> errata revisions, but as I have tried to point out, you may be in a >>> position do do this at thei point - especially if the group were to >>> publish its experiences with the spec as one or more informational >>> documents to provide a paper trail motivating the proposed changes. >> (This >>> part is your choice on how to produce the documentation, which does not >>> have to be in the form of a GFD but can and often is done this way. The >>> experiences of each group can be jointly or separately documented, at >> your >>> group's choice.) >>> >>>> Would it be possible for the OGF to join our next DFDL WG call to >>> discuss further? The call is at 16:00 UK (11:00 Eastern). >>> >>> If today, I can do this if you provide connection details. I include >> David >>> Martin in this reply in case he is available. >>> >>> Alan >>> >>>> Regards >>>> >>>> Steve Hanson >>>> Architect, IBM Data Format Description Language (DFDL) >>>> Co-Chair, OGF DFDL Working Group >>>> IBM SWG, Hursley, UK >>>> smh(a)uk.ibm.com >>>> tel:+44-1962-815848 >>>> Unless stated otherwise above: >>>> IBM United Kingdom Limited - Registered in England and Wales with >> number >>> 741598. >>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 >> >>> 3AU >>> >>> >>> >>> Unless stated otherwise above: >>> IBM United Kingdom Limited - Registered in England and Wales with number >> >>> 741598. >>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 >> 3AU >>> >>> Unless stated otherwise above: >>> IBM United Kingdom Limited - Registered in England and Wales with number >> >>> 741598. >>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 >> 3AU >> >> >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

3 5

Re: [DFDL-WG] Latest draft of DFDL 1.0 specification
by Steve Hanson 12 Sep '13

12 Sep '13

Greg GFD.152 implies that there is a 15-day GFSG review followed by a 60-day public comment period. We had a public comment period for GFD.174, and the changes in the revision are based on actual experience from implementers and users. I would question whether the public comment period is necessary for the revision, and request that the revision just undergoes the 15-day GFSG review. There will no doubt be a few more errata as implementations progress further, and so we anticipate one more revision at some point in the future, and that is the revision that would move to full Grid Recommendation. We are on a tight schedule and want to publish by end of September. We have users eagerly awaiting the appearance of the revision, and to publish an internal WG draft is not appropriate (IBM product infocenters embed the HTML rendering of the spec so needs to be official document). We requested the GFD number simply so that Mike Beckerle can complete the document edits. The document pushes the boundaries of MS Word and I think Mike also uses an additional plugin, so edits are best done by him. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Greg Newby <gbnewby(a)alaska.edu> To: Steve Hanson/UK/IBM@IBMGB, Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, "David E. Martin" <martinde(a)northwestern.edu> Date: 11/09/2013 21:07 Subject: Re: Latest draft of DFDL 1.0 specification Steve, Is this document in the editor pipeline somewhere? I'm not seeing it: https://redmine.ogf.org/projects/editor/issues Usually we wait until a document has undergone most of the review process (per GFD #152) before assigning a GFD number. If you have a reason to need one sooner, we can allocate one sooner. The usual practice, though, is to wait until publication is imminent. We're looking forward to these revisions to #174. -- Greg On Wed, Sep 11, 2013 at 06:08:36PM +0100, Steve Hanson wrote: > Alan, Greg > > Please can you allocate us a new GFD number so we can complete the > document? > > Regards > > Steve Hanson > Architect, IBM Data Format Description Language (DFDL) > Co-Chair, OGF DFDL Working Group > IBM SWG, Hursley, UK > smh(a)uk.ibm.com > tel:+44-1962-815848 > > > > From: "Sill, Alan" <alan.sill(a)ttu.edu> > To: Steve Hanson/UK/IBM@IBMGB, > Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" > <dfdl-wg(a)ogf.org>, Greg Newby <gbnewby(a)alaska.edu>, "David E. Martin" > <martinde(a)northwestern.edu> > Date: 11/09/2013 15:49 > Subject: Re: Latest draft of DFDL 1.0 specification > > > > > On Sep 11, 2013, at 9:08 AM, Steve Hanson <smh(a)uk.ibm.com> > wrote: > > > When you say 'new version' what does that imply about version numbers? > The WG still considers what we are working on to be DFDL 1.0 plus errata. > Are you suggesting that the new revision is 1.1 ? > > OGF doesn't generally have a concept or framework for revision numbers for > specifications. Some working groups do number their specifications, and we > leave this to the work group to manage. > > The process of obsoleting and replacing a document does give a good > opportunity for changing an internally-managed revision number. > > Regarding the next DFDL call, this will be during OGF 39. I could try to > join, but that time overlaps one of the sessions that I should attend. > Perhaps David or Greg (or both) could attend your call at 16:00 Tues 17th > September? If so, please provide details. We'll be happy to communicate > with you as much as possible on this topic before then, of course. > > Alan > > > ----- Forwarded by Steve Hanson/UK/IBM on 11/09/2013 16:35 ----- > > From: Steve Hanson/UK/IBM > To: "Sill, Alan" <alan.sill(a)ttu.edu>, > Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" > <dfdl-wg(a)ogf.org>, Greg Newby <gbnewby(a)alaska.edu>, "David E. Martin" > <martinde(a)northwestern.edu> > Date: 11/09/2013 15:08 > Subject: Re: Latest draft of DFDL 1.0 specification > > > Alan > > Thanks for your reply. I forgot to say that the next call is on Tues 17th > September. > > I think that the WG would probably go for your 2nd suggestion, ie, publish > an updated P_REC that obsoletes GFD.174 but does not yet propose to > promote the spec to full recommendation status, because we know there are > a few more errata that will be discovered before the implementations are > completed. > > When you say 'new version' what does that imply about version numbers? The > WG still considers what we are working on to be DFDL 1.0 plus errata. Are > you suggesting that the new revision is 1.1 ? > > Regards > > Steve Hanson > Architect, IBM Data Format Description Language (DFDL) > Co-Chair, OGF DFDL Working Group > IBM SWG, Hursley, UK > smh(a)uk.ibm.com > tel:+44-1962-815848 > > > > From: "Sill, Alan" <alan.sill(a)ttu.edu> > To: Steve Hanson/UK/IBM@IBMGB, "David E. Martin" > <martinde(a)northwestern.edu>, > Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" > <dfdl-wg(a)ogf.org>, Greg Newby <gbnewby(a)alaska.edu> > Date: 11/09/2013 14:51 > Subject: Re: Latest draft of DFDL 1.0 specification > > > > Steve, > > Answers inline. Short summary: My recommendation would be to publish a new > version that obsoletes the current GFD.174, and optionally to use this > opportunity to migrate the specification from a P-REC to full REC status. > This would be facilitated by documenting, in any form that is convenient > including but not limited to an informational GFD, the experience gained > from implementations to date. Another option is to publish an updated > P_REC that obsoletes GFD.174 but does not yet propose to promote the spec > to full recommendation status. > > On Sep 11, 2013, at 6:47 AM, Steve Hanson <smh(a)uk.ibm.com> > wrote: > > > Alan > > > > Since the DFDL 1.0 specification was published in Feb 2011, the two > implementation teams (IBM and the Daffodil project) have identified a > number of errata in the specification. These have been recorded in an > errata document held on Redmine. The number of errata is currently at > around 190, and include both clarifications to the specification and > changes that affect an implementation, both major and minor. Typically as > errata have been raised, the implementation teams include any implied > changes in the next release of their implementation. > > Is there any documentation as to the experience gained from > implementations that has led to these updates? I as not as an OGF > requirement, but just for information. > > > Both implementation teams, and users of the two implementations, have > requested that the DFDL 1.0 specification is revised to include all errata > to date, so that the specification more closely reflects the > implementations. Accordingly all errata to date have been incorporated > into a new revision of the specification, which as a result has grown from > 168 pages to 234 pages. > > > > This new revision of the specification supersedes the original. > > This statement is what leads to my suggestion to publish a new document. > Note that our procedures do allow for replacement of a REC or P-REC for > non-normative changes that do not substantially affect compatibility of > implementations, but that just clarify or correct errors in the original > publication. It is the statement that the new revision supersedes the > original coupled with your earlier observation that major implementation > issues are addressed in your new version that causes me to suggest that > you pursue a new GFD that obsoletes the old one. > > > There is no implementation that exactly reflects the original as > published on the OGF web site, they both adhere more closely to the new > revision. The DFDL WG would therefore like to publish the new revision. > The DFDL WG also recognises that there may be comments against the new > revision, and that there may still be some errata undetected by the > implementation teams, so that a further revision may be necessary in the > future. Nonetheless it is important that the new revision in its current > form is externally visible, and not just kept as an internal working > document, as there are now many dozens of DFDL users, and they need an > up-to-date specification. In particular, IBM DFDL wants to ship the HTML > version of the new revision to IBM customers in its next release. > > > > We are looking to the OGF for guidance on how next to proceed. > > My guidance would be to publish a new version that obsoletes the old one, > and optionally to use this opportunity to advance the specification from > P-REC to REC status. Note that this is exactly the pattern that OGF > documents are supposed to follow in the life cycle described in GFD.152 -- > experience gained fro real-world implementation is fed back to produce a > new version of the specification, which at some point can declare itself > to be mature enough to request full REC status. > > As for further revisions, they would be handled by the procedure I > mentioned above: non-normative changes can be folded in as corrections > through the errata process. THis is controlled by the OGF editor (Greg > Newby) and whether to accept and publish an errata is decided essentially > entirely by his recommendation to the GFSG (Standards Council) to do so. > If the changes would affect the interoperability of implementations > written to the earlier spec in a substantial way, they should be handled > by the process of a new publication that obsoletes the old one, as we have > just discussed. > > The process of going from a P-REC to a REC is largely decoupled from > errata revisions, but as I have tried to point out, you may be in a > position do do this at thei point - especially if the group were to > publish its experiences with the spec as one or more informational > documents to provide a paper trail motivating the proposed changes. (This > part is your choice on how to produce the documentation, which does not > have to be in the form of a GFD but can and often is done this way. The > experiences of each group can be jointly or separately documented, at your > group's choice.) > > > Would it be possible for the OGF to join our next DFDL WG call to > discuss further? The call is at 16:00 UK (11:00 Eastern). > > If today, I can do this if you provide connection details. I include David > Martin in this reply in case he is available. > > Alan > > > Regards > > > > Steve Hanson > > Architect, IBM Data Format Description Language (DFDL) > > Co-Chair, OGF DFDL Working Group > > IBM SWG, Hursley, UK > > smh(a)uk.ibm.com > > tel:+44-1962-815848 > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Latest draft of DFDL 1.0 specification
by Steve Hanson 11 Sep '13

11 Sep '13

Alan Thanks for your reply. I forgot to say that the next call is on Tues 17th September. I think that the WG would probably go for your 2nd suggestion, ie, publish an updated P_REC that obsoletes GFD.174 but does not yet propose to promote the spec to full recommendation status, because we know there are a few more errata that will be discovered before the implementations are completed. When you say 'new version' what does that imply about version numbers? The WG still considers what we are working on to be DFDL 1.0 plus errata. Are you suggesting that the new revision is 1.1 ? Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: "Sill, Alan" <alan.sill(a)ttu.edu> To: Steve Hanson/UK/IBM@IBMGB, "David E. Martin" <martinde(a)northwestern.edu>, Cc: "Sill, Alan" <alan.sill(a)ttu.edu>, "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Greg Newby <gbnewby(a)alaska.edu> Date: 11/09/2013 14:51 Subject: Re: Latest draft of DFDL 1.0 specification Steve, Answers inline. Short summary: My recommendation would be to publish a new version that obsoletes the current GFD.174, and optionally to use this opportunity to migrate the specification from a P-REC to full REC status. This would be facilitated by documenting, in any form that is convenient including but not limited to an informational GFD, the experience gained from implementations to date. Another option is to publish an updated P_REC that obsoletes GFD.174 but does not yet propose to promote the spec to full recommendation status. On Sep 11, 2013, at 6:47 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: > Alan > > Since the DFDL 1.0 specification was published in Feb 2011, the two implementation teams (IBM and the Daffodil project) have identified a number of errata in the specification. These have been recorded in an errata document held on Redmine. The number of errata is currently at around 190, and include both clarifications to the specification and changes that affect an implementation, both major and minor. Typically as errata have been raised, the implementation teams include any implied changes in the next release of their implementation. Is there any documentation as to the experience gained from implementations that has led to these updates? I as not as an OGF requirement, but just for information. > Both implementation teams, and users of the two implementations, have requested that the DFDL 1.0 specification is revised to include all errata to date, so that the specification more closely reflects the implementations. Accordingly all errata to date have been incorporated into a new revision of the specification, which as a result has grown from 168 pages to 234 pages. > > This new revision of the specification supersedes the original. This statement is what leads to my suggestion to publish a new document. Note that our procedures do allow for replacement of a REC or P-REC for non-normative changes that do not substantially affect compatibility of implementations, but that just clarify or correct errors in the original publication. It is the statement that the new revision supersedes the original coupled with your earlier observation that major implementation issues are addressed in your new version that causes me to suggest that you pursue a new GFD that obsoletes the old one. > There is no implementation that exactly reflects the original as published on the OGF web site, they both adhere more closely to the new revision. The DFDL WG would therefore like to publish the new revision. The DFDL WG also recognises that there may be comments against the new revision, and that there may still be some errata undetected by the implementation teams, so that a further revision may be necessary in the future. Nonetheless it is important that the new revision in its current form is externally visible, and not just kept as an internal working document, as there are now many dozens of DFDL users, and they need an up-to-date specification. In particular, IBM DFDL wants to ship the HTML version of the new revision to IBM customers in its next release. > > We are looking to the OGF for guidance on how next to proceed. My guidance would be to publish a new version that obsoletes the old one, and optionally to use this opportunity to advance the specification from P-REC to REC status. Note that this is exactly the pattern that OGF documents are supposed to follow in the life cycle described in GFD.152 -- experience gained fro real-world implementation is fed back to produce a new version of the specification, which at some point can declare itself to be mature enough to request full REC status. As for further revisions, they would be handled by the procedure I mentioned above: non-normative changes can be folded in as corrections through the errata process. THis is controlled by the OGF editor (Greg Newby) and whether to accept and publish an errata is decided essentially entirely by his recommendation to the GFSG (Standards Council) to do so. If the changes would affect the interoperability of implementations written to the earlier spec in a substantial way, they should be handled by the process of a new publication that obsoletes the old one, as we have just discussed. The process of going from a P-REC to a REC is largely decoupled from errata revisions, but as I have tried to point out, you may be in a position do do this at thei point - especially if the group were to publish its experiences with the spec as one or more informational documents to provide a paper trail motivating the proposed changes. (This part is your choice on how to produce the documentation, which does not have to be in the form of a GFD but can and often is done this way. The experiences of each group can be jointly or separately documented, at your group's choice.) > Would it be possible for the OGF to join our next DFDL WG call to discuss further? The call is at 16:00 UK (11:00 Eastern). If today, I can do this if you provide connection details. I include David Martin in this reply in case he is available. Alan > Regards > > Steve Hanson > Architect, IBM Data Format Description Language (DFDL) > Co-Chair, OGF DFDL Working Group > IBM SWG, Hursley, UK > smh(a)uk.ibm.com > tel:+44-1962-815848 > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 2

questions on setVariable
by Cranford, Jonathan W. 11 Sep '13

11 Sep '13

These questions may go towards public comment if the answers aren't straight-forward. Section references are against latest editor's draft. * Section 7.9 - 5th paragraph after first example - "A dfdl:setVariable value expression may refer to the value of this element using a relative path value '.'. " Which element? If dfdl:setVariable is attached to a simpleType, then the "element" referred to here is the element of that type, correct? But when it's attached to group reference, sequence, or choice, is it the parent element which contains the group reference, sequence, or choice? * Section 7.9 - next-to-last paragraph: "However, the order of execution among them is not specified." Wouldn't this be ambiguous, then? <xs:schema> <xs:annotation> <xs:appinfo source="http://www.ogf.org/dfdl/"> <xs:defineVariable name="bar" /> <xs:defineVariable name="baz" defaultValue="biff"/> </xs:appinfo> </xs:annotation> ... <xs:element name="foo"> <xs:annotation> <xs:appinfo source="http://www.ogf.org/dfdl/"> <xs:setVariable ref="bar" value="{$tns:baz}"/> <xs:setVariable ref="baz" value="fooey"/> </xs:appinfo> </xs:annotation> </xs:element> ... </xs:schema> If the order of execution isn't specified, isn't it ambiguous whether tns:bar gets the value of "biff" (defaultValue of tns:baz) or "fooey"? Respectfully, -- Jonathan W. Cranford Senior Information Systems Engineer The MITRE Corporation (http://www.mitre.org)

5 10

Fwd: questions on setVariable
by Mike Beckerle 11 Sep '13

11 Sep '13

Tim, When you say a "hash", is the IBM implementation implementing some sort of data-dependency analysis here? i.e., <setVariable name="x" value="{ $y }"/> <setVariable name="y" value="5"/> will execute in the opposite order because when executing the setVariable for x, it demands the value for y, and that executes the setVariable for y ?? Or is it truly just they all go into a hash table, and they come out in whatever order the hash table implementation produces them? ...mike Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy<http://www.ogf.org/About/abt_policies.php> On Wed, Sep 11, 2013 at 7:08 AM, Tim Kimber <KIMBERT(a)uk.ibm.com> wrote: > The model used by the IBM implementation supplies the variable-related > annotations for an XSD component as a hash, so the lexical order is lost. I > don't know whether it would even be possible to preserve lexical order. > Suman may know the answer to that. > > regards, > > Tim Kimber, DFDL Team, > Hursley, UK > Internet: kimbert(a)uk.ibm.com > Tel. 01962-816742 > Internal tel. 37246742 > > > > > From: Steve Hanson/UK/IBM@IBMGB > To: Suman Kalia <kalia(a)ca.ibm.com>, > Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, "dfdl-wg-bounces(a)ogf.org" > <dfdl-wg-bounces(a)ogf.org> > Date: 11/09/2013 11:26 > Subject: Re: [DFDL-WG] questions on setVariable > Sent by: dfdl-wg-bounces(a)ogf.org > ------------------------------ > > > > "I think the execution order needs to be specific within the annotations > at one annotation point." > > The problem is that action 186 allowed scenarios like setVariable > annotations simultaneously on an element ref, its global element and the > element's simple type (*). So there is no longer 'one annotation point', > there are multiple. That's when we formalised the concept of a 'resolved > set of annotations' for a component, and stated that for all annotations of > the same kind (**) in that set, the order of execution is not defined. We > also state schema authors can insert sequences to provide more precise > control over when annotations are executed. > > Several reasons for this. Is the lexical order of elements in appinfos, or > of multiple appinfos, guaranteed in XSDL? If not then an editor could > serialize appinfos or their content in any order. We also wanted to reserve > the right to add explicit timing control in the future. And it made > implementations easier. > > IBM DFDL 1.0.3 has been out in the field with this support since 1Q 2013. > Tim, can you say what the execution order is for a resolved set of > setVariables please? Or can we not tell because of the model? > > (*) Must be different variables, else SDE. > (**) Asserts and discriminators with testKind 'pattern' are a different > 'kind' from those with 'expression', for this purpose. > > Regards > > Steve Hanson > Architect, IBM Data Format Description Language (DFDL) > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK* > **smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:+44-1962-815848 > > > > From: Suman Kalia <kalia(a)ca.ibm.com> > To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, > Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, "dfdl-wg-bounces(a)ogf.org" > <dfdl-wg-bounces(a)ogf.org>, Steve Hanson/UK/IBM@IBMGB > Date: 10/09/2013 19:23 > Subject: Re: [DFDL-WG] questions on setVariable > ------------------------------ > > > > Mike - In the use case hilited by Jonathan where 2 annotations are > defined on the same point. I would suggest the execution order to be based > on the lexical order i.e. the order in which annotations appear.. > > Suman Kalia > IBM Canada Lab > WMB Toolkit Architect and Development Lead > Tel: 905-413-3923 T/L 313-3923 > Email: kalia(a)ca.ibm.com > > For info on Message broker * > ** > http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h… > *<http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h…> > > > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> > To: Steve Hanson <smh(a)uk.ibm.com>, > Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, "dfdl-wg-bounces(a)ogf.org" > <dfdl-wg-bounces(a)ogf.org> > Date: 09/10/2013 02:14 PM > Subject: Re: [DFDL-WG] questions on setVariable > Sent by: dfdl-wg-bounces(a)ogf.org > ------------------------------ > > > > If the first setVariable executes first, that will read variable baz, > which will cause the second setVariable to SDE because you cannot set after > the variable has been read. I believe this is the right semantics for this > schema, because both these setVariable annotations are written at the same > annotation point. > > If the second setVariable executes first, that will assign variable baz, > which is OK, then when the first setVariable executes, it will get the > value that was just assigned. > > While the spec says you shouldn't depend on this kind of ordering stuff, I > think it is pretty unsatisfactory if the order is unspecified. Section 9.5 > lays out the order of the different kinds of statement annotations relative > to each other, but doesn't say what happens between two setVariable > statements that appear at the same annotation point. I think the execution > order needs to be specific within the annotations at one annotation point. > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | * > www.tresys.com* <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are > subject to the *OGF Intellectual Property Policy*<http://www.ogf.org/About/abt_policies.php> > > > > On Tue, Sep 10, 2013 at 1:50 PM, Steve Hanson <*smh(a)uk.ibm.com*<smh(a)uk.ibm.com>> > wrote: > 1) I'll let Tim comment more on this but a relative expression uses as its > context the current element on the stack. So using '.' on a simple element > or its type resolves to the simple element, and using '.' on a complex > element or a group child (recursively) of a complex element resolves to the > complex element. Same deal when using the '..' parent axis, so beware - as > this example shows: > > <xs:element name="root"> > <xs:complexType> > <xs:sequence> > <xs:element name="delim" type="xs:string"/> > <xs:element name="body"> > <xs:complexType> > <xs:sequence > dfdl:separator="{../delim}"> > > ...and not {../../delim} ! This is all implied by the way XPath works. > XPath only knows about elements. > > 2) I don't see why Jonathan's example could give an SDE. The > defineVariables will always be executed first. However the result is > ambiguous, depending on the order each setVariable is executed, as 'bar' > could end up as 'biff' or 'fooey'. "Schema authors can insert sequences to > provide more precise control over when variables are set" - quote from > section 9.5.3 (errata 3.25). > > 3) To Mike's comment, the order of execution of setVariable and > newVariableInstance on the same annotation is strictly defined, see section > 9.5 (errata 3.25) > > Regards > > Steve Hanson > Architect, IBM Data Format Description Language (DFDL) > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK* > **smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:*+44-1962-815848* <%2B44-1962-815848> > > > > From: Mike Beckerle <*mbeckerle.dfdl(a)gmail.com*<mbeckerle.dfdl(a)gmail.com> > > > To: "Cranford, Jonathan W." <*jcranford(a)mitre.org*<jcranford(a)mitre.org>>, > > Cc: "*dfdl-wg(a)ogf.org* <dfdl-wg(a)ogf.org>" <*dfdl-wg(a)ogf.org*<dfdl-wg(a)ogf.org> > > > Date: 10/09/2013 18:23 > Subject: Re: [DFDL-WG] questions on setVariable > Sent by: *dfdl-wg-bounces(a)ogf.org* <dfdl-wg-bounces(a)ogf.org> > ------------------------------ > > > > Those are both public-comment-class issues. > > The issue of what does "." mean in various places is interesting. It is > referring to an infoset node, but how to explain exactly which one it is.... > > The out-of-order issue with evaluation that you mention is definitely an > issue. The schema you gave will work or SDE depending on the order of > evaluation of the statements and the language is not precise enough to say > what the order is. I think the order of the setVariable and > newVariableInstance statements at any annotation point, must be linear > first to last. The order of setVariable and newVariableInstance combined > into the resolved set of annotations, but from multiple annotation points > (e.g., an element and a global type definition it references), those can be > interleaved in any order. > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | * > www.tresys.com* <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are > subject to the *OGF Intellectual Property Policy*<http://www.ogf.org/About/abt_policies.php> > > > > On Tue, Sep 10, 2013 at 12:53 PM, Cranford, Jonathan W. <* > jcranford(a)mitre.org* <jcranford(a)mitre.org>> wrote: > These questions may go towards public comment if the answers aren't > straight-forward. Section references are against latest editor's draft. > > > · Section 7.9 - 5th paragraph after first example - "A > dfdl:setVariable value expression may refer to the value of this element > using a relative path value '.'. " > > > > Which element? > > > > If dfdl:setVariable is attached to a simpleType, then the "element" > referred to here is the element of that type, correct? > > > > But when it's attached to group reference, sequence, or choice, is it the > parent element which contains the group reference, sequence, or choice? > > > > > > · Section 7.9 - next-to-last paragraph: "However, the order of > execution among them is not specified." > > > > Wouldn't this be ambiguous, then? > > > > <xs:schema> > > <xs:annotation> > > <xs:appinfo source="*http://www.ogf.org/dfdl/*<http://www.ogf.org/dfdl/> > "> > > <xs:defineVariable name="bar" /> > > <xs:defineVariable name="baz" defaultValue="biff"/> > > </xs:appinfo> > > </xs:annotation> > > ... > > <xs:element name="foo"> > > <xs:annotation> > > <xs:appinfo source="*http://www.ogf.org/dfdl/*<http://www.ogf.org/dfdl/> > "> > > <xs:setVariable ref="bar" value="{$tns:baz}"/> > > <xs:setVariable ref="baz" value="fooey"/> > > </xs:appinfo> > > </xs:annotation> > > </xs:element> > > ... > > </xs:schema> > > > > If the order of execution isn't specified, isn't it ambiguous whether > tns:bar gets the value of "biff" (defaultValue of tns:baz) or "fooey"? > > > > > > Respectfully, > > > > -- > > Jonathan W. Cranford > > Senior Information Systems Engineer > > The MITRE Corporation (*http://www.mitre.org* <http://www.mitre.org/>) > > > > -- > dfdl-wg mailing list > *dfdl-wg(a)ogf.org* <dfdl-wg(a)ogf.org> > *https://www.ogf.org/mailman/listinfo/dfdl-wg*<https://www.ogf.org/mailman/listinfo/dfdl-wg> > -- > dfdl-wg mailing list* > **dfdl-wg(a)ogf.org* <dfdl-wg(a)ogf.org>* > **https://www.ogf.org/mailman/listinfo/dfdl-wg*<https://www.ogf.org/mailman/listinfo/dfdl-wg> > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org* > **https://www.ogf.org/mailman/listinfo/dfdl-wg*<https://www.ogf.org/mailman/listinfo/dfdl-wg> > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > https://www.ogf.org/mailman/listinfo/dfdl-wg > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > https://www.ogf.org/mailman/listinfo/dfdl-wg >

2 1

OGF DFDL WG Call Minutes 2013-09-10
by Steve Hanson 11 Sep '13

11 Sep '13

Please find minutes from the above call at http://redmine.ogf.org/dmsf_files/13109?download= Errata document is now at v013 here: http://redmine.ogf.org/dmsf_files/8104?download= Action 140 errata addendum at v001 here: http://redmine.ogf.org/dmsf_files/13017?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] Action 226 (was: First draft of appendix describing string literal matching)
by Steve Hanson 11 Sep '13

11 Sep '13

New action 226 raised to cover comment #3. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Tim Kimber/UK/IBM@IBMGB, Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Date: 03/09/2013 14:49 Subject: Re: [DFDL-WG] First draft of appendix describing string literal matching Thinking on 1 again, the combined table should be in section 6.3.1. Appendices are usually non-normative. And the syntax table for DFDL expressions is part of the main spec, as an analogy. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org, Date: 03/09/2013 09:49 Subject: Re: [DFDL-WG] First draft of appendix describing string literal matching Sent by: dfdl-wg-bounces(a)ogf.org Thanks for reviewing. 1. Let's drop tables 2 and 4 and replace with refs to the appendix, as suggested 2. Agreed 3. Good point. I think the intention of %ES; was that it should be used on its own. I don't see any point in allowing it to be a part of a non-zero-length DFDL string literal. So I think your modification to the grammar should be put into the spec. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Cc: Tim Kimber/UK/IBM@IBMGB Date: 02/09/2013 17:46 Subject: Re: First draft of appendix describing string literal matching Good description. My comments: 1) Apart from the first three rows, the grammar table is pretty much duplicating existing tables 2 and 4 in section 6.3.1. Suggest either that the table is dropped from the appendix and anything that is missing is added back into 6.3.1, or tables 2 and 4 are dropped and replaced by refs to appendix. I think the latter is preferable as everything is then in a single table. 2) There is a bug in the grammar for DfdlStringLiteral - there should not be '{' and '}' - that's expression syntax. 3) For recognising ES, you say "The string part is recognized if the data available for matching is zero-length". That's true if we insist that ES, if present, must be present on its own. I'm not sure we actually say that. If that is the intent, we should police this in the grammar. (Note IBM DFDL does not give an error if it find '%ES;abc' ). For 2) and 3) that would give: DfdlStringLiteral ::= (DfdlStringLiteralPart)+ | DfdlESEntity DfdlCharClassName ::= DfdlNLEntity | DfdlWSPEntity | DfdlWSPStarEntity | DfdlWSPPlusEntity It still needs an errata, as it is a change to the spec document. Needs references from 6.3.1. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Tim Kimber/UK/IBM@IBMGB, Steve Hanson/UK/IBM@IBMGB, Date: 30/08/2013 00:16 Subject: Re: First draft of appendix describing string literal matching I added this in current form as appendix D. Will be in draft r14.4. I did not create an erratum for this. It's a whole new section, not an error correction or clarificatino. But we can add one if we think it useful to point out this section. There are no cross references to this section currently in the document. We might find a few places we want to reference this from. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Wed, Aug 28, 2013 at 10:43 AM, Tim Kimber <KIMBERT(a)uk.ibm.com> wrote: Thanks Mike. I agree that the wording could be misinterpreted. Revised draft attached: regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Tim Kimber/UK/IBM@IBMGB, Cc: Steve Hanson/UK/IBM@IBMGB Date: 20/08/2013 17:33 Subject: Re: First draft of appendix describing string literal matching I'm not sure I agree with the algorithm in the 1.3 section for the string literal part "LiteralString". I believe this algorithm is independent of what encoding the schema itself is written in, i.e., what is on the <? xml encoding="..." ?> slug line at the top of the schema file. What you write in the schema file is read into memory, all characters are converted to unicode codepoints by way of that reading process. So these two statements in the Recognition Algorithm for LiteralString are of concern: "The characters in the DFDL schema will be encoded using the defined encoding for the schema in which they appear." I think this just muddies the waters. Elsewhere we should state that the encoding used when authoring a DFDL schema file does not affect the behavior of the schema. All schemas behave as if authored in utf-8, etc. "The recognition algorithm must be able to compare character sequences that are encoded using different encodings." To me that says if I write my schema in ebcdic, but the dfdl:encoding="ascii", that some algorithm other than mapping both into unicode codepoints first and then comparing them is needed. I don't think this is or should be true. I think the division of things into what you call string literal parts is needed due to raw byte, and due to character class entities. Outside of that I think translation of everything to unicode should be sufficient. ...mike On Thu, Aug 15, 2013 at 7:19 PM, Tim Kimber <KIMBERT(a)uk.ibm.com> wrote: Steve, Mike, Please take a look. Comments on high-level stuff like structure/level of detail are welcome. regards Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

ambiguity - mandatory alignment for text (12.1.2)
by Mike Beckerle 11 Sep '13

11 Sep '13

These two paragraphs contradict each other. When processing textual data, it is a schema definition error if the dfdl:alignment and dfdl:alignmentUnits properties are used to specify alignment that is not a multiple of the encoding-specified mandatory alignment. If the data is not aligned to the proper boundary for the encoding when textual data is processed, then bits are skipped (parsing) or filled from dfdl:fillByte (unparsing) to achieve the mandatory alignment. So is it to be SDE (conservative), or just move over to the mandatory alignment? E.g., <xs:sequence dfdl:encoding="utf-8" dfdl:alignment="7" dfdl:alignmentUnits="bits" dfdl:initiator="hello">.... The alignment property says 7 bits, but the initiator is utf-8 which requires 8 bits. So we either SDE, or we just add additional alignment bits to align the initiator text. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy<http://www.ogf.org/About/abt_policies.php>

2 1