- dfdl-wg - lists.ogf.org

split into multiple topics - Re: [dfdl-wg] Issues: additional data types
by Mike Beckerle 06 Sep '05

06 Sep '05

I'd like to split this topic into several distinct ones: Arrays - I have a placeholder for this in the doc. Opaque and "code" types are separate. This is related also to the concept of "open content". Enums Bitfields Pointers Mike Beckerle Architect, Scalable Computing IBM Software Group Information Integration Solutions Westborough, MA "Robert E. McGrath" <mcgrath(a)ncsa.uiuc.edu> Sent by: owner-dfdl-wg(a)ggf.org 09/02/2005 03:13 PM To dfdl-wg(a)gridforum.org cc Subject [dfdl-wg] Issues: additional data types Greetings, Here is an "issue" for the DFDL: additional data types that should be considered. Please see attached. --- Robert E. McGrath National Center for Supercomputing Applications University of Illinois, Urbana-Champaign Champaign, Illinois 61820 (217)-333-6549 mcgrath(a)ncsa.uiuc.edu

4 14

Fw: Enums - Re: split into multiple topics - Re: [dfdl-wg] Issues: additional data types
by Steve Hanson 06 Sep '05

06 Sep '05

You can consider the MRM model as an IBM proprietary equivalent to DFDL. The logical model is XML schema, the physical model is our equivalent of DFDL annotations. I don't see the real world case for having xs:enums on the schema simple types whose values are the names of the 88s. The only requirement I have seen is for level 88 values to participate in validation of the byte stream, which means the xs:enums are the values of the 88s, not the names. Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 06/09/2005 19:43 ----- "Robert E. McGrath" <mcgrath(a)ncsa.uiu To c.edu> dfdl-wg(a)gridforum.org Sent by: cc owner-dfdl-wg@ggf .org Subject Re: Enums - Re: split into multiple topics - Re: [dfdl-wg] Issues: 06/09/2005 18:49 additional data types On Tuesday 06 September 2005 12:21, Steve Hanson wrote: > The MRM model COBOL importer gives the user an option to create > xs:enumeration values from level 88 values, and that is all. This is so > the MRM parser can validate a byte stream against the model. This is in > keeping with the rule that values supplied with metadata are part of the > logical model, not the physical model. What's the motivation for doing > anything more than this? Sorry, I don't know what "the logical model" vs. "the physical model" means. IMO, the goal is to have a way for a producer to generate data (e.g., with 88s as discussed below), which a consumer can interpret as either 88 or symbols, depending on the purpose of the consumer. > > Using your example that gives: > > <xs:element name="AS-COM-TRAN-ID"> > <xs:simpleType> > <xs:restriction base="xs:string"> > <xs:enumeration value="CP80"/> > <xs:enumeration value="IC40"/> > ... > </xs:restriction> > </xs:simpleType> > </xs:element> > > Regards, Steve > > Steve Hanson > WebSphere Business Integration Brokers, > IBM Hursley, England > Internet: smh(a)uk.ibm.com > Phone (+44)/(0) 1962-815848 > > > > Mike Beckerle > <beckerle(a)us.ibm. > com> To > Sent by: dfdl-wg(a)gridforum.org > owner-dfdl-wg@ggf cc > .org > Subject > Enums - Re: split into multiple > 06/09/2005 17:40 topics - Re: [dfdl-wg] Issues: > additional data types > > > > > > > > > > > > About enums. Here's starting thoughts: > > Here's a real-world example from COBOL: > > 01 AS-CPST-REC. > 06 AS-CPCOM. > 09 AS-COM-STORE-TYPE PIC X. > 09 AS-COM-STORE-NO PIC 9(05). > 09 AS-COM-TRAN-ID PIC X(04). > 88 TRAN-COUPON VALUE 'CP80'. > 88 TRAN-REVENUE VALUE 'IC40' > 'RA40'. > 88 TRAN-SALES VALUE 'IC40'. > 88 TRAN-DELIVER VALUE 'IC44'. > 88 TRAN-RENTS VALUE 'RA40' > 'RA42'. > 88 TRAN-RENT-RETURN VALUE 'RA41'. > 09 AS-COM-QUANTITY PIC S9(05). > 09 AS-COM-PART-NO PIC 9(06). > .... more fields elided .... > > Those "88" entries in there are enumerated constants. Note that for the > TRAN-REVENUE and TRAN-RENTS constants, multiple values are associated with > the same name. On reference this means that the constant matches either > value. When written, this means the first value is used. Cobol doesn't > strongly associate these enumerated values with the field to which they can > be assigned, but usually it's obvious. In this case it is the > AS-COM-TRAN-ID field which is the 4-character-long string which has the > string constants associated with it. > > This particular example is a common one. The record has variant structure > (not shown above) depending on the tag field which is this AS-COM-TRAN-ID > field. > > Working out how we want this example to work in XSD is the first step. > > I like using a hidden field here. For example: Here's a possible idea for > how this is represented in XSD: > > > <xs:element name="AS-COM-TRAN-ID">  > <xs:simpleType> > <xs:restriction base="xs:NCName"> > <xs:enumeration value="TRAN-COUPON"/> > <xs:enumeration value="TRAN-REVENUE"/> > <xs:enumeration value="TRAN-SALES"/> > <xs:enumeration value="TRAN-DELIVER"/> > <xs:enumeration value="TRAN-RENTS"/> > </xs:restriction> > </xs:simpleType> > </xs:element> > <xs:sequence> > <xs:annotation> > <xs:appinfo source="http://dataformat.org/"> > <xs:layer name="rep" type="AS-COM-TRAN-ID-repType"/>  > </xs:appinfo> > </xs:annotation> > </xs:sequence> > > <xs:simpleType name="AS-COM-TRAN-ID-repType"> > <xs:restriction base="xs:string> > <xs:enumeration value="CP80"/> > <xs:enumeration value="IC40"/>  > .... > </xs:restriction> > </xs:simpleType> > > Now in an annotation (not shown above) on the element AS-COM-TRAN-ID there > would be a dfdl:valueCalc property which would compute the value of > AS-COM-TRAN-ID based on the value of the hidden field. Symmetrically, the > 'rep' hidden field would have a dfdl:repCalc property which would give the > inverse formula for output. > > One difficulty I have with this is the notion that we're projecting into > the string type. I.e., these symbolic constants aren't names for integers, > but rather we're expressing operations on strings. In the above example the > enumerated constants actually are strings, but in other examples they would > be integers. The next tier of interpretation, i.e., where we're decidng the > variant based on the value of AS-COM-TRAN-ID would be expressed as string > comparisons which is potentially inefficient. This is part of the problem > with using XSD as our type system basis. XSD doesn't have a notion of > symbolic named constant. > > Alternative: There is the DTD named entity stuff. Does anybody want to > propose that? > > Mike Beckerle > Architect, Scalable Computing > IBM Software Group > Information Integration Solutions > Westborough, MA > > > Mike > Beckerle/Worcester/IBM@IBMUS > > Sent by: To > owner-dfdl-wg(a)ggf.org "Robert E. McGrath" > <mcgrath(a)ncsa.uiuc.edu> > cc > 09/02/2005 04:34 PM dfdl-wg(a)gridforum.org, > owner-dfdl-wg(a)ggf.org > Subject > split into multiple topics - Re: > [dfdl-wg] Issues: additional > data types > > > > > > > > > > > > > > I'd like to split this topic into several distinct ones: > > Arrays - I have a placeholder for this in the doc. > > Opaque and "code" types are separate. This is related also to the concept > of "open content". > > Enums > > Bitfields > > Pointers > > > Mike Beckerle > Architect, Scalable Computing > IBM Software Group > Information Integration Solutions > Westborough, MA > > "Robert E. McGrath" > <mcgrath(a)ncsa.uiuc.edu> > Sent by: owner-dfdl-wg(a)ggf.org > To > dfdl-wg(a)gridforum.org > 09/02/2005 03:13 PM cc > > Subject > [dfdl-wg] Issues: > additional data types > > > > > > > > > > > > > > > Greetings, > > Here is an "issue" for the DFDL: additional data types that should > be considered. > > Please see attached. > > --- > Robert E. McGrath > National Center for Supercomputing Applications > University of Illinois, Urbana-Champaign > Champaign, Illinois 61820 > (217)-333-6549 > > mcgrath(a)ncsa.uiuc.edu (See attached file: DT.htm) -- --- Robert E. McGrath National Center for Supercomputing Applications University of Illinois, Urbana-Champaign Champaign, Illinois 61820 (217)-333-6549 mcgrath(a)ncsa.uiuc.edu

1 0

Plumbing document
by Martin Westhead 06 Sep '05

06 Sep '05

Hi Folks, In the discussion at the end of the meeting we mentioned a document I wrote following the discussion at the December f2f. I don't think it made it as far a Gridforge so I have just added it at: https://forge.gridforum.org/docman2/ViewCategory.php?group_id=113&category_… It is an attempt at starting to describe compositional features of DFDL in a formal/semi-formal way. I think that as we add layering something like this is going to be essential to keep everything clear. Cheers, Martin

3 4

Re: Arrays issue - Re: [dfdl-wg] Issues: additional data types
by Jim Myers 06 Sep '05

06 Sep '05

1 0

Agenda items for tomorrow's meeting?
by Martin Westhead 06 Sep '05

06 Sep '05

Hi Folks, Does anyone have any items for tomorrow's agenda? I have: - Additional types (Bob's email) - Initial issues list (I will put this out later today) - Issues with draft (perhaps the authors or those commenting could pick some issues to discuss? If we can put them our in advance then others can review and prepare for discussion) Please send me (or the list) any topics, on the draft or otherwise, you'd like adding. Cheers, Martin

2 1

Comments on DFDL draft spec
by Steve Hanson 04 Sep '05

04 Sep '05

The draft is a big step forward for DFDL. Here are the initial comments from the IBM Hursley team. We have not reviewed all the Appendices yet, including the detailed properties list, but I am on a course all this week so I think it makes sense to send what we have so far to the group. (See attached file: IBM Hursley MRM team comments on DFDL draft specification Aug 2005.doc) Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848

1 0

Conference call updated
by Talbott, Tara D 31 Aug '05

31 Aug '05

The telecon has been updated to occur weekly so we're all set for next week. Tara

1 0

Notes on DFDL telcon 31/08/2005
by Martin Westhead 31 Aug '05

31 Aug '05

***Note: next meeting will be _next_ Wednesday (7/11/05) 9am PDT*** * Present: Tara Talbot Martin Westhead Mike Beckerle Steve Hanson Geoff Judd Bob McGrath Jim Meyers * status reports from PNNL-NCSA project - open source status, progress on features? any proposals forthcoming? Have the go ahead for the source forge project. All that is left to do is to put the code up. Layering related work is still going on. PNNL are aiming to get some of the basic ideas into the primer. * planning for upcoming GGF15 - who is planning to attend in person Jim Bob Mike Geoff (?) Sumin (?) Tara (?) * Latest posted document - how do we structure discussion to this proposal? What do we do in advance of GGF15, what do we try to accomplish F2F at the GGF meeting. Issues list maintained by Martin. Issues show be raised initially on the mailing list and we will have a live document/spreadsheet on the website tracking the current status. * GGF15 sessions - so far I've only reserved one 90 minutes session as a work session. Presumably we also need another 45min session as community outreach. (Deadline is friday to reserve.) Mike will book another session. Outreach is becoming increasingly important. * AOB Bi-weekly calls to move to weekly starting next week. Tara to book extra telcons. Primer document from Bob was discussed, there may be something before GGF15. Mike suggested that there may be material from the current draft that should end up in the primer. Discussed the status of 'DFDL formal semantics' Bob and Martin interested in what Mike has done with this since December.

1 0

FYI: XDTM: Overlap with DFDL
by Jim Myers 31 Aug '05

31 Aug '05

3 5

Telcon reminder
by Martin Westhead 31 Aug '05

31 Aug '05

Hi folks, DFDL conference call on Wednesday at 9am pacific time. Preliminary Agenda ------------------ - Defuddle source forge project update - GGF planning - Discussion of IBM draft Conference call instructions: Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call. I will be there for the first 20-30 minutes but will probably need a second note taker to help when I drop out. Cheers, Martin

1 0