Re: [DFDL-WG] Latest draft of DFDL 1.0 specification
Alan Thanks for your reply. I forgot to say that the next call is on Tues 17th September. I think that the WG would probably go for your 2nd suggestion, ie, publish an updated P_REC that obsoletes GFD.174 but does not yet propose to promote the spec to full recommendation status, because we know there are a few more errata that will be discovered before the implementations are completed. When you say 'new version' what does that imply about version numbers? The WG still considers what we are working on to be DFDL 1.0 plus errata. Are you suggesting that the new revision is 1.1 ? Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: "Sill, Alan" <alan.sill@ttu.edu> To: Steve Hanson/UK/IBM@IBMGB, "David E. Martin" <martinde@northwestern.edu>, Cc: "Sill, Alan" <alan.sill@ttu.edu>, "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Greg Newby <gbnewby@alaska.edu> Date: 11/09/2013 14:51 Subject: Re: Latest draft of DFDL 1.0 specification Steve, Answers inline. Short summary: My recommendation would be to publish a new version that obsoletes the current GFD.174, and optionally to use this opportunity to migrate the specification from a P-REC to full REC status. This would be facilitated by documenting, in any form that is convenient including but not limited to an informational GFD, the experience gained from implementations to date. Another option is to publish an updated P_REC that obsoletes GFD.174 but does not yet propose to promote the spec to full recommendation status. On Sep 11, 2013, at 6:47 AM, Steve Hanson <smh@uk.ibm.com> wrote:
Alan
Since the DFDL 1.0 specification was published in Feb 2011, the two implementation teams (IBM and the Daffodil project) have identified a number of errata in the specification. These have been recorded in an errata document held on Redmine. The number of errata is currently at around 190, and include both clarifications to the specification and changes that affect an implementation, both major and minor. Typically as errata have been raised, the implementation teams include any implied changes in the next release of their implementation.
Is there any documentation as to the experience gained from implementations that has led to these updates? I as not as an OGF requirement, but just for information.
Both implementation teams, and users of the two implementations, have requested that the DFDL 1.0 specification is revised to include all errata to date, so that the specification more closely reflects the implementations. Accordingly all errata to date have been incorporated into a new revision of the specification, which as a result has grown from 168 pages to 234 pages.
This new revision of the specification supersedes the original.
There is no implementation that exactly reflects the original as
This statement is what leads to my suggestion to publish a new document. Note that our procedures do allow for replacement of a REC or P-REC for non-normative changes that do not substantially affect compatibility of implementations, but that just clarify or correct errors in the original publication. It is the statement that the new revision supersedes the original coupled with your earlier observation that major implementation issues are addressed in your new version that causes me to suggest that you pursue a new GFD that obsoletes the old one. published on the OGF web site, they both adhere more closely to the new revision. The DFDL WG would therefore like to publish the new revision. The DFDL WG also recognises that there may be comments against the new revision, and that there may still be some errata undetected by the implementation teams, so that a further revision may be necessary in the future. Nonetheless it is important that the new revision in its current form is externally visible, and not just kept as an internal working document, as there are now many dozens of DFDL users, and they need an up-to-date specification. In particular, IBM DFDL wants to ship the HTML version of the new revision to IBM customers in its next release.
We are looking to the OGF for guidance on how next to proceed.
My guidance would be to publish a new version that obsoletes the old one, and optionally to use this opportunity to advance the specification from P-REC to REC status. Note that this is exactly the pattern that OGF documents are supposed to follow in the life cycle described in GFD.152 -- experience gained fro real-world implementation is fed back to produce a new version of the specification, which at some point can declare itself to be mature enough to request full REC status. As for further revisions, they would be handled by the procedure I mentioned above: non-normative changes can be folded in as corrections through the errata process. THis is controlled by the OGF editor (Greg Newby) and whether to accept and publish an errata is decided essentially entirely by his recommendation to the GFSG (Standards Council) to do so. If the changes would affect the interoperability of implementations written to the earlier spec in a substantial way, they should be handled by the process of a new publication that obsoletes the old one, as we have just discussed. The process of going from a P-REC to a REC is largely decoupled from errata revisions, but as I have tried to point out, you may be in a position do do this at thei point - especially if the group were to publish its experiences with the spec as one or more informational documents to provide a paper trail motivating the proposed changes. (This part is your choice on how to produce the documentation, which does not have to be in the form of a GFD but can and often is done this way. The experiences of each group can be jointly or separately documented, at your group's choice.)
Would it be possible for the OGF to join our next DFDL WG call to discuss further? The call is at 16:00 UK (11:00 Eastern).
Regards
Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number
If today, I can do this if you provide connection details. I include David Martin in this reply in case he is available. Alan 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
On Sep 11, 2013, at 9:08 AM, Steve Hanson <smh@uk.ibm.com> wrote:
When you say 'new version' what does that imply about version numbers? The WG still considers what we are working on to be DFDL 1.0 plus errata. Are you suggesting that the new revision is 1.1 ?
OGF doesn't generally have a concept or framework for revision numbers for specifications. Some working groups do number their specifications, and we leave this to the work group to manage. The process of obsoleting and replacing a document does give a good opportunity for changing an internally-managed revision number. Regarding the next DFDL call, this will be during OGF 39. I could try to join, but that time overlaps one of the sessions that I should attend. Perhaps David or Greg (or both) could attend your call at 16:00 Tues 17th September? If so, please provide details. We'll be happy to communicate with you as much as possible on this topic before then, of course. Alan
Alan, Greg Please can you allocate us a new GFD number so we can complete the document? Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: "Sill, Alan" <alan.sill@ttu.edu> To: Steve Hanson/UK/IBM@IBMGB, Cc: "Sill, Alan" <alan.sill@ttu.edu>, "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Greg Newby <gbnewby@alaska.edu>, "David E. Martin" <martinde@northwestern.edu> Date: 11/09/2013 15:49 Subject: Re: Latest draft of DFDL 1.0 specification On Sep 11, 2013, at 9:08 AM, Steve Hanson <smh@uk.ibm.com> wrote:
When you say 'new version' what does that imply about version numbers? The WG still considers what we are working on to be DFDL 1.0 plus errata. Are you suggesting that the new revision is 1.1 ?
OGF doesn't generally have a concept or framework for revision numbers for specifications. Some working groups do number their specifications, and we leave this to the work group to manage. The process of obsoleting and replacing a document does give a good opportunity for changing an internally-managed revision number. Regarding the next DFDL call, this will be during OGF 39. I could try to join, but that time overlaps one of the sessions that I should attend. Perhaps David or Greg (or both) could attend your call at 16:00 Tues 17th September? If so, please provide details. We'll be happy to communicate with you as much as possible on this topic before then, of course. Alan ----- Forwarded by Steve Hanson/UK/IBM on 11/09/2013 16:35 ----- From: Steve Hanson/UK/IBM To: "Sill, Alan" <alan.sill@ttu.edu>, Cc: "Sill, Alan" <alan.sill@ttu.edu>, "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Greg Newby <gbnewby@alaska.edu>, "David E. Martin" <martinde@northwestern.edu> Date: 11/09/2013 15:08 Subject: Re: Latest draft of DFDL 1.0 specification Alan Thanks for your reply. I forgot to say that the next call is on Tues 17th September. I think that the WG would probably go for your 2nd suggestion, ie, publish an updated P_REC that obsoletes GFD.174 but does not yet propose to promote the spec to full recommendation status, because we know there are a few more errata that will be discovered before the implementations are completed. When you say 'new version' what does that imply about version numbers? The WG still considers what we are working on to be DFDL 1.0 plus errata. Are you suggesting that the new revision is 1.1 ? Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: "Sill, Alan" <alan.sill@ttu.edu> To: Steve Hanson/UK/IBM@IBMGB, "David E. Martin" <martinde@northwestern.edu>, Cc: "Sill, Alan" <alan.sill@ttu.edu>, "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Greg Newby <gbnewby@alaska.edu> Date: 11/09/2013 14:51 Subject: Re: Latest draft of DFDL 1.0 specification Steve, Answers inline. Short summary: My recommendation would be to publish a new version that obsoletes the current GFD.174, and optionally to use this opportunity to migrate the specification from a P-REC to full REC status. This would be facilitated by documenting, in any form that is convenient including but not limited to an informational GFD, the experience gained from implementations to date. Another option is to publish an updated P_REC that obsoletes GFD.174 but does not yet propose to promote the spec to full recommendation status. On Sep 11, 2013, at 6:47 AM, Steve Hanson <smh@uk.ibm.com> wrote:
Alan
Since the DFDL 1.0 specification was published in Feb 2011, the two implementation teams (IBM and the Daffodil project) have identified a number of errata in the specification. These have been recorded in an errata document held on Redmine. The number of errata is currently at around 190, and include both clarifications to the specification and changes that affect an implementation, both major and minor. Typically as errata have been raised, the implementation teams include any implied changes in the next release of their implementation.
Is there any documentation as to the experience gained from implementations that has led to these updates? I as not as an OGF requirement, but just for information.
Both implementation teams, and users of the two implementations, have requested that the DFDL 1.0 specification is revised to include all errata to date, so that the specification more closely reflects the implementations. Accordingly all errata to date have been incorporated into a new revision of the specification, which as a result has grown from 168 pages to 234 pages.
This new revision of the specification supersedes the original.
There is no implementation that exactly reflects the original as
This statement is what leads to my suggestion to publish a new document. Note that our procedures do allow for replacement of a REC or P-REC for non-normative changes that do not substantially affect compatibility of implementations, but that just clarify or correct errors in the original publication. It is the statement that the new revision supersedes the original coupled with your earlier observation that major implementation issues are addressed in your new version that causes me to suggest that you pursue a new GFD that obsoletes the old one. published on the OGF web site, they both adhere more closely to the new revision. The DFDL WG would therefore like to publish the new revision. The DFDL WG also recognises that there may be comments against the new revision, and that there may still be some errata undetected by the implementation teams, so that a further revision may be necessary in the future. Nonetheless it is important that the new revision in its current form is externally visible, and not just kept as an internal working document, as there are now many dozens of DFDL users, and they need an up-to-date specification. In particular, IBM DFDL wants to ship the HTML version of the new revision to IBM customers in its next release.
We are looking to the OGF for guidance on how next to proceed.
My guidance would be to publish a new version that obsoletes the old one, and optionally to use this opportunity to advance the specification from P-REC to REC status. Note that this is exactly the pattern that OGF documents are supposed to follow in the life cycle described in GFD.152 -- experience gained fro real-world implementation is fed back to produce a new version of the specification, which at some point can declare itself to be mature enough to request full REC status. As for further revisions, they would be handled by the procedure I mentioned above: non-normative changes can be folded in as corrections through the errata process. THis is controlled by the OGF editor (Greg Newby) and whether to accept and publish an errata is decided essentially entirely by his recommendation to the GFSG (Standards Council) to do so. If the changes would affect the interoperability of implementations written to the earlier spec in a substantial way, they should be handled by the process of a new publication that obsoletes the old one, as we have just discussed. The process of going from a P-REC to a REC is largely decoupled from errata revisions, but as I have tried to point out, you may be in a position do do this at thei point - especially if the group were to publish its experiences with the spec as one or more informational documents to provide a paper trail motivating the proposed changes. (This part is your choice on how to produce the documentation, which does not have to be in the form of a GFD but can and often is done this way. The experiences of each group can be jointly or separately documented, at your group's choice.)
Would it be possible for the OGF to join our next DFDL WG call to discuss further? The call is at 16:00 UK (11:00 Eastern).
Regards
Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number
If today, I can do this if you provide connection details. I include David Martin in this reply in case he is available. Alan 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (2)
-
Sill, Alan -
Steve Hanson