- dfdl-wg - lists.ogf.org

DFDL: Minutes from OGF WG call, 23 Jan 2007 *CORRECTED*
by Ian W Parkinson 28 Jan '08

28 Jan '08

A small correction. with thanks to Simon - it was Steve (rather than Simon) who had previously attracted a reasonable audience at the OGF conference. Ian Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 23 Jan 2008 Attendees Mike Beckerle (Oco) Simon Parker (PolarLake) Ian Parkinson (IBM) Alan Powell (IBM) Apologies Steve Hanson (IBM), Suman Kalia (IBM) 1. OGF22 The DFDL session at OGF22 is now booked for the Monday afternoon, and Mike has registered to attend. Mike will present our updated status, and Alan promised to upload the last set of presented slides to GridForge so that Mike can update them. Alan asked whether we should attempt to drum up interest in the DFDL session to encourage attendence; Simon thought that advertising may not make much difference and that Steve had a reasonable audience when he presented. 2. Specification drafts Steve and Alan had previously assigned ownership of individual items from Mike's plan of contents for the next few drafts. Alan will assemble the next draft, due at the end of the month, and asked for input as soon as possible. Looking at the plan for the next, "vX+1", draft, the group reported the following status: Nulls/default/optionals - Mike reported no update. Description of schema components - Simon is still working on this. Regular expressions for lengths - Alan reported no progress. Expression language - Alan will shortly distribute a new version of the proposal for review. valueCalc - Mike is still to write this. Property precedence - Following a discussion on the call last week, please provide review comments. Mike will add this to the agenda for next week. Entities - Alan's recent proposal is to be discussed on the current call. White space handling - Discussion is ongoing, and Steve is to make a proposal. The plan calls for subsequent versions of the specification, including the following items with status: Supplements - Steve is working to update the supplements Speculative parsing - IBM has internally been discussing and reviewing WTX function, though no documentation presently exists covering this. 3. UML diagrams Simon is revising the UML diagrams which describe the DFDL schema components. The previous meeting minutes included a number of comments on these diagrams, and the group took this opportunity to look at some of those comments: "...I think it would be better to use the open source XML schema model as source model and show relationship of DFDL Annotations attached to the XSD schema model" - Mike noted that DFDL makes use of annotations on objects which are absent from the XSD schema model, and hence that it may be unnatural to base the DFDL schema model directly on the XSD model. Simon suggested that it would be cleanest to describe a modified version the XSD model including those XSD elements that we need to annotate, and use this as a basis for the DFDL model. "The current diagram suggests that 'variable definition' can both be part of a format base or as a standalone annotation (outside of a format). Is this true?" - Mike suggested that variable definitions don't have to be part of a format block: so, yes, this is true. Mike agreed to respond further to the set of comments by email. 4. Review of Entities proposal Alan has distributed a proposal covering entities in DFDL, intended to allow characters which are disallowed by XML1.0 (or XML1.1) to be included in DFDL schemas. These follow a similar syntax to XML, using % instead of & as an escape, with an additional mechanism for specifying raw data. This latter is intended to supplant the escaping mechanism described in current versions of the specification (which also uses % as an escape). The group felt that the description of the raw data entities should not be cast in terms of characters and character sets, but rather in terms of bytes. If treated as characters, schemas may need to be written when moving from single-byte to double-byte character sets; further, this incorrectly implies some codepage conversion is involved. The proposal also introduces a list of predefined names for certain common control characters. Mike asked whether these are the existing XML names - Alan replied that XML does not define names for control characters. Ian asked how we should represent the literal % character in strings given this form of escaping. The present draft of the specification uses "%%" to handle this; Simon suggested a string like "%pc;". The meeting felt that %% might be marginally preferable. Finally, the proposal defines some labels which aim to reduce the complexity of dealing with whitespace and newlines. The %NL; entity represents a newline on "the target platform" - Mike observed that DFDL presently does not have a concept of a target platform. Alan felt it important that a single DFDL schema be able to generate output documents targetted at different platforms. Mike proposed that we introduce a new property, "generatedNewLine", which describes the meaning of %NL; during unparse, and that %NL; should be tolerant of any common new line representation during parse. The group discussed whether this could instead be handled using a list of optional new line values, however this would not support schema portability. Simon suggested we introduce another new property to mean that %NL; should be the conventional new line representation on the platform on which an engine is running, however Mike pointed out that this simply requires appropriate configuration of the generatedNewLine property. %WSP; and %OWSP; are introduced to mean any whitespace, and optional whitespace. This will be useful in describing some formats which allow arbitrary whitespace, such as MIME. Mike pointed out that we could model such whitespace using hidden fields, but that these entities may make a schema clearer. PolarLake have found that only one such label is necessary, which means, "one or more whitespace characters", and that this needs only to be made available as a delimiter - Mike agreed that this label may represent a special type of delimiter rather than a general purpose entity. Alan would like to work through the potential use cases to see if we can restrict it in this fashion, and will update the proposal to specify that these relate to just one character. Simon suggested we could introduce an extra label, perhaps %WPS*; to match multiple whitespace characters. Meeting closed, 18:15 Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

DFDL: Minutes from OGF WG call, 23 Jan 2007
by Ian W Parkinson 24 Jan '08

24 Jan '08

All, Here's my minutes from yesterday's call - please let me know if I've missed anything. As a reminder: there's an archive of the dfdl-wg mailing list (including these meeting minutes) available at < http://www.ogf.org/pipermail/dfdl-wg>. Cheers, Ian Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 23 Jan 2008 Attendees Mike Beckerle (Oco) Simon Parker (PolarLake) Ian Parkinson (IBM) Alan Powell (IBM) Apologies Steve Hanson (IBM), Suman Kalia (IBM) 1. OGF22 The DFDL session at OGF22 is now booked for the Monday afternoon, and Mike has registered to attend. Mike will present our updated status, and Alan promised to upload the last set of presented slides to GridForge so that Mike can update them. Alan asked whether we should attempt to drum up interest in the DFDL session to encourage attendence; Simon thought that advertising may not make much difference and that he had a reasonable audience when he presented. 2. Specification drafts Steve and Alan had previously assigned ownership of individual items from Mike's plan of contents for the next few drafts. Alan will assemble the next draft, due at the end of the month, and asked for input as soon as possible. Looking at the plan for the next, "vX+1", draft, the group reported the following status: Nulls/default/optionals - Mike reported no update. Description of schema components - Simon is still working on this. Regular expressions for lengths - Alan reported no progress. Expression language - Alan will shortly distribute a new version of the proposal for review. valueCalc - Mike is still to write this. Property precedence - Following a discussion on the call last week, please provide review comments. Mike will add this to the agenda for next week. Entities - Alan's recent proposal is to be discussed on the current call. White space handling - Discussion is ongoing, and Steve is to make a proposal. The plan calls for subsequent versions of the specification, including the following items with status: Supplements - Steve is working to update the supplements Speculative parsing - IBM has internally been discussing and reviewing WTX function, though no documentation presently exists covering this. 3. UML diagrams Simon is revising the UML diagrams which describe the DFDL schema components. The previous meeting minutes included a number of comments on these diagrams, and the group took this opportunity to look at some of those comments: "...I think it would be better to use the open source XML schema model as source model and show relationship of DFDL Annotations attached to the XSD schema model" - Mike noted that DFDL makes use of annotations on objects which are absent from the XSD schema model, and hence that it may be unnatural to base the DFDL schema model directly on the XSD model. Simon suggested that it would be cleanest to describe a modified version the XSD model including those XSD elements that we need to annotate, and use this as a basis for the DFDL model. "The current diagram suggests that 'variable definition' can both be part of a format base or as a standalone annotation (outside of a format). Is this true?" - Mike suggested that variable definitions don't have to be part of a format block: so, yes, this is true. Mike agreed to respond further to the set of comments by email. 4. Review of Entities proposal Alan has distributed a proposal covering entities in DFDL, intended to allow characters which are disallowed by XML1.0 (or XML1.1) to be included in DFDL schemas. These follow a similar syntax to XML, using % instead of & as an escape, with an additional mechanism for specifying raw data. This latter is intended to supplant the escaping mechanism described in current versions of the specification (which also uses % as an escape). The group felt that the description of the raw data entities should not be cast in terms of characters and character sets, but rather in terms of bytes. If treated as characters, schemas may need to be written when moving from single-byte to double-byte character sets; further, this incorrectly implies some codepage conversion is involved. The proposal also introduces a list of predefined names for certain common control characters. Mike asked whether these are the existing XML names - Alan replied that XML does not define names for control characters. Ian asked how we should represent the literal % character in strings given this form of escaping. The present draft of the specification uses "%%" to handle this; Simon suggested a string like "%pc;". The meeting felt that %% might be marginally preferable. Finally, the proposal defines some labels which aim to reduce the complexity of dealing with whitespace and newlines. The %NL; entity represents a newline on "the target platform" - Mike observed that DFDL presently does not have a concept of a target platform. Alan felt it important that a single DFDL schema be able to generate output documents targetted at different platforms. Mike proposed that we introduce a new property, "generatedNewLine", which describes the meaning of %NL; during unparse, and that %NL; should be tolerant of any common new line representation during parse. The group discussed whether this could instead be handled using a list of optional new line values, however this would not support schema portability. Simon suggested we introduce another new property to mean that %NL; should be the conventional new line representation on the platform on which an engine is running, however Mike pointed out that this simply requires appropriate configuration of the generatedNewLine property. %WSP; and %OWSP; are introduced to mean any whitespace, and optional whitespace. This will be useful in describing some formats which allow arbitrary whitespace, such as MIME. Mike pointed out that we could model such whitespace using hidden fields, but that these entities may make a schema clearer. PolarLake have found that only one such label is necessary, which means, "one or more whitespace characters", and that this needs only to be made available as a delimiter - Mike agreed that this label may represent a special type of delimiter rather than a general purpose entity. Alan would like to work through the potential use cases to see if we can restrict it in this fashion, and will update the proposal to specify that these relate to just one character. Simon suggested we could introduce an extra label, perhaps %WPS*; to match multiple whitespace characters. Meeting closed, 18:15 Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: OGF DFDL WG call today 23-01-2008 - agenda
by Alan Powell 23 Jan '08

23 Jan '08

Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 ----- Forwarded by Alan Powell/UK/IBM on 23/01/2008 17:18 ----- Alan Powell/UK/IBM 23/01/2008 12:08 To DFDL cc Subject OGF DFDL WG call today 23-01-2008 - agenda Review items from previous calls. - OGF22 - Specification drafts - Expression language - Property precedence - UML for DFDL schema Discussion for this call - Review entity proposal - Steve's suggestions 1) This is related to the white space discussion around entities. One way to handle the situation where the terminator can vary is to allow the DFDL markup properties (dfdl:terminator, dfdl:separator, etc) to be lists, just like we already do for dfdl:nullValues. (IBM's WTX has this capability). 2) We've allowed the prefix of a prefixed length to be explicitly described as a non-event field using dfdl:lengthPrefixType. Should we permit this for markup properties? Instead of supplying a list of possible values, you supply a simple type with enums for the values. This could be viewed as an alternative/complementary to 1). There is a limitations - because we are using XSDL enumeration facet, we are constrained by its syntax so I don't see how we could use our own entity scheme or expressions. Also, I suspect that enums are inherently unordered so we'd need a way of saying which to use on output (use an element of simple type and use XSDL default attribute?). Lastly, we should not force a user to model an initiator as an element/type - most users just see it as a piece of text so just entering the value must still be allowed. 3) Let's say my delimiter is dynamically defined at the start of the data, like EDI allows. We would handle that in DFDL using an expression or variable. However, EDI also allows random white space to appear after the delimiter. Can our expression/entity syntaxes handle this? Does this preclude use of 1) or 2)? Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: Agenda for DFDL WG Call
by Steve Hanson 23 Jan '08

23 Jan '08

I can't make the call today as I am travelling but there are a couple of things I'd like to raise. These have come to light while I have been thinking about white space issues. 1) This is related to the white space discussion around entities. One way to handle the situation where the terminator can vary is to allow the DFDL markup properties (dfdl:terminator, dfdl:separator, etc) to be lists, just like we already do for dfdl:nullValues. (IBM's WTX has this capability). 2) We've allowed the prefix of a prefixed length to be explicitly described as a non-event field using dfdl:lengthPrefixType. Should we permit this for markup properties? Instead of supplying a list of possible values, you supply a simple type with enums for the values. This could be viewed as an alternative/complementary to 1). There is a limitations - because we are using XSDL enumeration facet, we are constrained by its syntax so I don't see how we could use our own entity scheme or expressions. Also, I suspect that enums are inherently unordered so we'd need a way of saying which to use on output (use an element of simple type and use XSDL default attribute?). Lastly, we should not force a user to model an initiator as an element/type - most users just see it as a piece of text so just entering the value must still be allowed. 3) Let's say my delimiter is dynamically defined at the start of the data, like EDI allows. We would handle that in DFDL using an expression or variable. However, EDI also allows random white space to appear after the delimiter. Can our expression/entity syntaxes handle this? Does this preclude use of 1) or 2)? Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 23/01/2008 08:27 ----- "Mike Beckerle" <mbeckerle.dfdl(a)gmail.com> Sent by: dfdl-wg-bounces(a)ogf.org 23/01/2008 04:18 To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] Agenda for DFDL WG Call I have these items for the agenda. 1) review of entities proposal 2) OGF 22 - our session is Monday 2/25 at 4:15pm. Other agenda items? ...mikeb -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Agenda for DFDL WG Call
by Mike Beckerle 23 Jan '08

23 Jan '08

I have these items for the agenda. 1) review of entities proposal 2) OGF 22 - our session is Monday 2/25 at 4:15pm. Other agenda items? ...mikeb

1 0

Re: [DFDL-WG] Action 14: Propose DFDL entity scheme
by Alan Powell 22 Jan '08

22 Jan '08

Hi Mike %NL; is a single character <LF> on those target platforms where that is the convention or <CR><LF> on others, etc. This is intended to make it easier for the same dfdl schema to parse messages from different platforms. I know we avoided target platform in DFDL and was expecting that this would cause some debate. This will be a good discussion for tomorrow's call Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 "Mike Beckerle" <mbeckerle(a)OCO-INC.COM> 22/01/2008 13:29 To Alan Powell/UK/IBM@IBMGB, <DFDL-Technical-Core%IBMGB(a)uk.ibm.com>, <dfdl-wg(a)ogf.org> cc Subject RE: [DFDL-WG] Action 14: Propose DFDL entity scheme Is the &NL; supposed to represent a single character? Or can it be a CRLF? There?s no notion of ?the target platform? in DFDL. We?ve specifically avoided this notion on purpose. So we need a separate property like newline=?&CR;&LF;? or newline=?&LF;? if we want &NL; to be meaningful, unless some other property is suitable. There are some other Unicode whitespace and Unicode line-ending characters. Do we want to include those in the definitions of WSP and NL ? I recall there are 4 line-endings in Unicode. ?mikeb From: dfdl-wg-bounces(a)ogf.org [mailto:dfdl-wg-bounces@ogf.org] On Behalf Of Alan Powell Sent: Tuesday, January 22, 2008 6:32 AM To: DFDL-Technical-Core%IBMGB(a)uk.ibm.com; dfdl-wg(a)ogf.org Subject: Re: [DFDL-WG] Action 14: Propose DFDL entity scheme All Attached is the latest proposal for DFDL 'entities' The main changes are: - No longer using XML entities as this proved to not meet all the requirements - New generic mnemonics for <NL> and others to represent the NL on the target platform. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

Re: [DFDL-WG] Action 14: Propose DFDL entity scheme
by Alan Powell 22 Jan '08

22 Jan '08

All Attached is the latest proposal for DFDL 'entities' The main changes are: - No longer using XML entities as this proved to not meet all the requirements - New generic mnemonics for <NL> and others to represent the NL on the target platform. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] DFDL property precedence - parsing - v1
by Steve Hanson 18 Jan '08

18 Jan '08

Hi Alan 1. We discussed breaking this precdence list down on a per DFDL object basis. This was predicated on the UML diagram being enhanced to show the allowable properties. 2. Yes, I should make that clearer 3. No, initiatorSeparator and initiator are disjoint, the former is specially for wildcards 4. No. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Alan Powell/UK/IBM 17/01/2008 16:42 To Steve Hanson/UK/IBM@IBMGB cc DFDL-Technical-Core, dfdl-wg(a)ogf.org Subject Re: DFDL property precedence - parsing - v1 Steve I interpret precedence to mean: the order the dfdl properties should be inspected and the order of dependency (ie prop y is only relevant if prop x is set) so As not all properties are relevant to choice, sequence etc shouldn't they be in the tree? Aren't all occursxxx properties dependent on max/minoccurs > 1? InitiatorSeparator dependent on initiator Separator dependent on Delimited? Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Steve Hanson/UK/IBM 11/01/2008 15:02 To DFDL-Technical-Core, dfdl-wg(a)ogf.org cc Subject DFDL property precedence - parsing - v1 Requires FreeMind from http://freemind.sourceforge.net/wiki/index.php/Download Parsing: [attachment "DFDL Property Precedence (parsing) 001.mm" deleted by Alan Powell/UK/IBM] Writing: tbd Notes: - Precedence is from top to bottom recursively, so the top priority is dfdl:inputValueCalc, then dfdl:byteOrder, then dfdl:encoding, then it's children top to bottom, then dfdl:occursKind (you get the idea). Questions arising: - Does dfdl:escapeScheme apply when dfdl:lengthKind = endOfData ? - Confirm that only signed logical number can have a signed rep, and only unsigned logical number can have an unsigned rep ? (change from spec 028 to 030) - Should dfdl:terminator, dfdl:separator and dfdl:occursSeparator be lists (to handle alternative delimiters) ? (viz dfdl:nullValues) - Why isn't dfdl:nullIndicatorIndex encompassed by dfdl:nullIndicatorPath ? - Use of enum 'xpath' for dfdl:nullKind.? We've used "explicit" elsewhere and decorated syntax to indicate xpath for other properties. For nullKind maybe a better enum would be "external" or "remote" ? - The advanced text delimiter supplement needs revision - most of its content appears to be in the core. Main question is on prefix v infix v postfix separators and how that capability is to be specified. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] DFDL property precedence - parsing - v1
by Alan Powell 17 Jan '08

17 Jan '08

Steve I interpret precedence to mean: the order the dfdl properties should be inspected and the order of dependency (ie prop y is only relevant if prop x is set) so As not all properties are relevant to choice, sequence etc shouldn't they be in the tree? Aren't all occursxxx properties dependent on max/minoccurs > 1? InitiatorSeparator dependent on initiator Separator dependent on Delimited? Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Steve Hanson/UK/IBM 11/01/2008 15:02 To DFDL-Technical-Core, dfdl-wg(a)ogf.org cc Subject DFDL property precedence - parsing - v1 Requires FreeMind from http://freemind.sourceforge.net/wiki/index.php/Download Parsing: [attachment "DFDL Property Precedence (parsing) 001.mm" deleted by Alan Powell/UK/IBM] Writing: tbd Notes: - Precedence is from top to bottom recursively, so the top priority is dfdl:inputValueCalc, then dfdl:byteOrder, then dfdl:encoding, then it's children top to bottom, then dfdl:occursKind (you get the idea). Questions arising: - Does dfdl:escapeScheme apply when dfdl:lengthKind = endOfData ? - Confirm that only signed logical number can have a signed rep, and only unsigned logical number can have an unsigned rep ? (change from spec 028 to 030) - Should dfdl:terminator, dfdl:separator and dfdl:occursSeparator be lists (to handle alternative delimiters) ? (viz dfdl:nullValues) - Why isn't dfdl:nullIndicatorIndex encompassed by dfdl:nullIndicatorPath ? - Use of enum 'xpath' for dfdl:nullKind.? We've used "explicit" elsewhere and decorated syntax to indicate xpath for other properties. For nullKind maybe a better enum would be "external" or "remote" ? - The advanced text delimiter supplement needs revision - most of its content appears to be in the core. Main question is on prefix v infix v postfix separators and how that capability is to be specified. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 16 Jan 2007 *CORRECTED*
by Ian W Parkinson 17 Jan '08

17 Jan '08

All, With thanks to Steve for the correction to 1(v). It is Steve, not Simon, who will be looking at whitespacing. Ian Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 16 Jan 2008 Attendees Geoff Judd (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) Alan Powell (IBM) Steve Hanson (IBM) Suman Kalia (IBM) 1. Review of Minutes from last week OGF22 - No update from Mike. Specification Drafts - Steve and Alan have distributed a version of Mike's plan for the next few versions. of the specification with work items assigned to individuals. Simon asked for clarification regarding his item "2-level description (schema components)" - this covers reviewing and updating the DFDL schema abstract data model document (distributed by Steve, with the agenda for this meeting) for inclusion in the DFDL specification, to replace and extend the existing UML diagram. Expression Language - Alan will update and distribute a new draft of the expression language proposal. Property Precedence - Steve has prepared a proposal as a Freemind mind map and would appreciate any review comments, especially with regard to the questions distributed along with the proposal. The group briefly discussed some of these, and Steve will email Mike asking for his opinion on some of these issues. Regarding dfdl:escapeScheme, and how it applies when lengthKind=end of data - the specification plan includes a review of the escapeScheme mechanism and this issue can be discussed then. While discussing the enum used for dfdl:nullKind, Simon observed that this issue has not arisen in PolarLake's parser as it does not have a well-developed concept of null. Steve felt we should perhaps remove the Advanced Text Delimiter Supplement. Entity References - Alan and Ian have been looking at this, and will prepare and distribute a revised proposal. Steve will write a proposal to cover whitespacing. (Simon Parker left the meeting) 2. Property Precedence The group discussed Steve's proposal and its relation to the specification document. It was suggested that the specification could list the properties of each object in precedence order; and further that the UML diagrams could show which properties can be used with each object. Meeting closed, 17:40 GMT --- IBM review of UML for DFDL Schema Abstract Model For convenience, here is the set of comments from IBM regarding the UML diagrams in the schema abstract model document. General: All associations originating from DFDLSchema should have black diamond instead of empty diamond because DFDL schema object would be the owner of these objects. Cardinality of relationship is not explicitly shown; I think the default cardinality as per standard is 1 which won't be correct in this case. e.g. DFDLSchema would have more than 1 global elements/ types etc.. Likewise there would be black diamond relationship from Element Declaration/Simple Type Definition/ComplexType Definition to DFDL Annotation Base. (Suman) In the open source XML Schema model, there is no explicitly defined object for element reference and group reference. Same XSDElementDeclaration or XSDGroupDefintion are used to model references. It is determined programmatically and I think it would be better to use the open source XML schema model as source model and show relationship of DFDL Annotations attached to the XSD schema model. (Suman) UML for DFDL subset of XSDL : Need cardinality adding (Steve) White v black diamond not correct (Suman) What does 'ref' mean on the link from Particle to Element Declaration ? (Steve) Link from Complex Type Definition to Particle - this can only be to a Model Group subclass (Steve) May want to add the following pointer to make it complete: complex type definition -> (base type) -> complex type definition (Sandy) I know this is probably not needed by DFDL; but may help to include it for completeness. We have it for simple types; may confuse people if it's not on compelx types. (Sandy) UML for DFDL annotations on XSDL: Wondering whether this is necessary. It's a middle step, possibly only useful to implementors (Sandy) 1st bullet - talks about a limitation in the XSDL model - I'd like Sandy to take us through this. MRM carries annotations on groups and group refs.(Steve) 1st bullet - the practical limitation appears to be that you can't override the properties of a global group on a group reference, like you can with elements. Is this going to be an issue? (Steve) 4th bullet - prohibiting dfdl:discriminator on xs:choice seems sensible (Steve) Should "variable definition" be derived from "DFDL annotation base"? (Sandy) The current diagram suggests that "variable definition" can both be part of a format base or as a standalone annotation (outside of a format). Is this true? (Sandy) Can "hidden" be viewed as a specialized element reference? (Sandy) This only implies a "base class" pointer from "hidden" to "element reference". There really isn't much benifit in doing this. It just *feels* better. Don't feel strongly. (Sandy) UML for post-scope resolution and inheritance flattening: Haven't finished reviewing it carefully. (Sandy) Inheritance is used much more often in this diagram. e.g. "Any Element Wildcard" is derived from both "Occurrence Properties" and "DFDL properties Base". Is this intentional? Feels counter-intuitive to me. Components should *have* properties, rather than *be* properties. (Sandy) Wondering whether it makes sense to combine diagrams 2 and 3. That is, a single diagram that has everything inherited/flattered, but don't attempt to remove properties that are not used at runtime. e.g. keep DFDL Schema etc. (Sandy) Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0