checking on 'no recursion' restriction - are we sure we can live without it
DFDL v1.0 spec currently disallows recursive types. Are we sure we can live without this? Seems to me there are many formats e.g., JSON, which are popular now, and which naturally require recursion to express. There are a number of document-like formats - there's a fuzzy grey area where documents and data records overlap, and these will naturally be modeled using recursion. Even formats like EDIFACT allow segment nesting, though I'm not sure about whether recursive definitions are allowed or precluded, a generic EDIFACT parser wouldn't know any specific segment types, and would want to have a recursively defined generic segment structure. Comments? ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412
I tried to search XML schema definition for JSON but could not find it.. If you have please attach. You can build models without using recursion. Adding recursion to the spec will certainly increase complexity. From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: dfdl-wg@ogf.org Date: 04/06/2012 05:49 PM Subject: [DFDL-WG] checking on 'no recursion' restriction - are we sure we can live without it Sent by: dfdl-wg-bounces@ogf.org DFDL v1.0 spec currently disallows recursive types. Are we sure we can live without this? Seems to me there are many formats e.g., JSON, which are popular now, and which naturally require recursion to express. There are a number of document-like formats - there's a fuzzy grey area where documents and data records overlap, and these will naturally be modeled using recursion. Even formats like EDIFACT allow segment nesting, though I'm not sure about whether recursive definitions are allowed or precluded, a generic EDIFACT parser wouldn't know any specific segment types, and would want to have a recursively defined generic segment structure. Comments? ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412 -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
Recursion would help, but I suspect that you would still end up with a DFDL info set that was structured in a non-ideal way. The info set would be a tree in which all of the 'real' elements would have the same name. The JSON names would have to be carried on a child element 'jsonName'. Most clients applications would want to transform that DFDL info set into something more natural. One solution would be to add a new feature to DFDL to allow the info set element name to be calculated from the data using a DFDL expression - but then the DFDL info set would not be valid for the DFDL xsd. Looking at this another way, JSON is like XML - they are both self-defining formats allowing arbitrarily deep nesting. In both cases you can add some external rules that describe the expected structure of the document ( e.g. XSD for XML ) . I think DFDL is designed for the case where the structure is known in advance. The 'generic JSON parser' or 'generic XML parser' are solved problems. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 246742 From: Suman Kalia <kalia@ca.ibm.com> To: Mike Beckerle <mbeckerle.dfdl@gmail.com> Cc: dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org Date: 09/04/2012 12:27 Subject: Re: [DFDL-WG] checking on 'no recursion' restriction - are we sure we can live without it Sent by: dfdl-wg-bounces@ogf.org I tried to search XML schema definition for JSON but could not find it.. If you have please attach. You can build models without using recursion. Adding recursion to the spec will certainly increase complexity. From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: dfdl-wg@ogf.org Date: 04/06/2012 05:49 PM Subject: [DFDL-WG] checking on 'no recursion' restriction - are we sure we can live without it Sent by: dfdl-wg-bounces@ogf.org DFDL v1.0 spec currently disallows recursive types. Are we sure we can live without this? Seems to me there are many formats e.g., JSON, which are popular now, and which naturally require recursion to express. There are a number of document-like formats - there's a fuzzy grey area where documents and data records overlap, and these will naturally be modeled using recursion. Even formats like EDIFACT allow segment nesting, though I'm not sure about whether recursive definitions are allowed or precluded, a generic EDIFACT parser wouldn't know any specific segment types, and would want to have a recursively defined generic segment structure. Comments? ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412 -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
I take your point on JSON. We don't need to solve that problem really. At this point we have to put out a call for examples of recursive definitions in data formats, i.e., where one really needs recursion to model the data. I can imagine they exist, but have not had to use one, nor seen one. I have seen things that were Lisp S-Expression-like syntax, but they were probably flattenable, i.e., no true recursion needed. I checked EDIFACT, and they allow nesting, but they specifically cut it off at 1 level of nesting. DOCBOOK has an SGML variant, but that's so XML-ish, that I really don't see that it is DFDL's job to be able to parse that. Any other examples? On Mon, Apr 9, 2012 at 5:03 PM, Tim Kimber <KIMBERT@uk.ibm.com> wrote:
Recursion would help, but I suspect that you would still end up with a DFDL info set that was structured in a non-ideal way. The info set would be a tree in which all of the 'real' elements would have the same name. The JSON names would have to be carried on a child element 'jsonName'. Most clients applications would want to transform that DFDL info set into something more natural. One solution would be to add a new feature to DFDL to allow the info set element name to be calculated from the data using a DFDL expression - but then the DFDL info set would not be valid for the DFDL xsd.
Looking at this another way, JSON is like XML - they are both self-defining formats allowing arbitrarily deep nesting. In both cases you can add some external rules that describe the expected structure of the document ( e.g. XSD for XML ) . I think DFDL is designed for the case where the structure is known in advance. The 'generic JSON parser' or 'generic XML parser' are solved problems.
regards,
Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 246742
From: Suman Kalia <kalia@ca.ibm.com> To: Mike Beckerle <mbeckerle.dfdl@gmail.com> Cc: dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org Date: 09/04/2012 12:27 Subject: Re: [DFDL-WG] checking on 'no recursion' restriction - are we sure we can live without it Sent by: dfdl-wg-bounces@ogf.org ------------------------------
I tried to search XML schema definition for JSON but could not find it.. If you have please attach. You can build models without using recursion. Adding recursion to the spec will certainly increase complexity.
From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: dfdl-wg@ogf.org Date: 04/06/2012 05:49 PM Subject: [DFDL-WG] checking on 'no recursion' restriction - are we sure we can live without it Sent by: dfdl-wg-bounces@ogf.org ------------------------------
DFDL v1.0 spec currently disallows recursive types.
Are we sure we can live without this?
Seems to me there are many formats e.g., JSON, which are popular now, and which naturally require recursion to express.
There are a number of document-like formats - there's a fuzzy grey area where documents and data records overlap, and these will naturally be modeled using recursion.
Even formats like EDIFACT allow segment nesting, though I'm not sure about whether recursive definitions are allowed or precluded, a generic EDIFACT parser wouldn't know any specific segment types, and would want to have a recursively defined generic segment structure.
Comments?
...mikeb
-- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412 -- dfdl-wg mailing list dfdl-wg@ogf.org *https://www.ogf.org/mailman/listinfo/dfdl-wg*<https://www.ogf.org/mailman/listinfo/dfdl-wg> -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
------------------------------
* *
*Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU *
-- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
-- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412
Dear Mike, So how should a JSON mapping in DFDL look like? Cheers, Kris __ Kristoffer H Rose, ph.d. <krisrose@us.ibm.com> +1(914)784-7642 (fax -6324) IBM T.J.Watson Research Center, PO Box 704, Yorktown Heights, NY 10598 Mike Beckerle <mbeckerle.dfdl@gmail.com> 04/06/2012 05:48 PM To dfdl-wg@ogf.org, cc Kristoffer H Rose/Watson/IBM@IBMUS Subject checking on 'no recursion' restriction - are we sure we can live without it DFDL v1.0 spec currently disallows recursive types. Are we sure we can live without this? Seems to me there are many formats e.g., JSON, which are popular now, and which naturally require recursion to express. There are a number of document-like formats - there's a fuzzy grey area where documents and data records overlap, and these will naturally be modeled using recursion. Even formats like EDIFACT allow segment nesting, though I'm not sure about whether recursive definitions are allowed or precluded, a generic EDIFACT parser wouldn't know any specific segment types, and would want to have a recursively defined generic segment structure. Comments? ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412
participants (4)
-
Kristoffer H Rose -
Mike Beckerle -
Suman Kalia -
Tim Kimber