example/use-case wanted - separatorSuppressionPolicy 'trailingEmptyStrict' and 'never'
Users of Apache Daffodil have asked what these property values are for, i.e., what are idiomatic uses of this property for actual formats, or in other words, what kind of data are these properties needed for? I'd like to provide them examples of this if anyone can recall the kinds of specific use cases for these property values. I've been searching through the example DFDL schemas I can find, and none use trailingEmptyStrict or never for this property. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>
'trailingEmptyStrict' was for X12 if I recall. It is not valid to have trailing separators. 'never' I don't have an example for. Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh@uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: DFDL-WG <dfdl-wg@ogf.org> Date: 20/04/2021 15:12 Subject: [EXTERNAL] [DFDL-WG] example/use-case wanted - separatorSuppressionPolicy 'trailingEmptyStrict' and 'never' Sent by: "dfdl-wg" <dfdl-wg-bounces@ogf.org> Users of Apache Daffodil have asked what these property values are for, i.e., what are idiomatic uses of this property for actual formats, or in other words, what kind of data are these properties needed for? I'd like to provide them examples of this if anyone can recall the kinds of specific use cases for these property values. I've been searching through the example DFDL schemas I can find, and none use trailingEmptyStrict or never for this property. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
I see that X12 is the USA-used EDI message standard - analogous to the UN/EDIFACT used outside USA. I'll pass along that this was motivated by X12, though to my knowledge there is no DFDL schema generally available for any part of X12. I think for SSP 'never' I can gen-up an example pretty easily. Given what I've got below, can you think of where such formats would appear in anything real-world? The only thing I can think of is somebody taking a COBOL structure with two "occurs N times depending on ..." arrays in it, and modernizing the format into something variable-length with separators for use in a more modern application. My example for separatorSuppressionPolicy='never' is below. Assuming dfdl:occursCountKind='implicit' everywhere <sequence dfdl:separator="," dfdl:separatorSuppressionPolicy="never"> <element name="givenName" type="xs:string" minOccurs="0" maxOccurs="3"/> <element name="surname" type="xs:string" minOccurs="0"/> <element name="ph" type="xs:string" minOccurs="0" maxOccurs="6"/> </sequence> This format means there are always 10 locations separated by 9 commas. Whether something is a givenName, surname, or phone number is just determined positionally by counting the separators as the parse passes them. Well-formed instance: "michael, james,,rogers,888-888-8888,777-777-7777,,,," XML flavored DFDL infoset: <givenName>michael</givenName> <givenName>james</givenName> <surname>rogers</surname> <ph>888-888-8888</ph> <ph>777-777-7777</ph> Well-formed instance: "madonna,,,,,,,,," XML flavored equivalent infoset: <givenName>madonna</givenName> Both the above examples would unparse to exactly the input data. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> On Tue, Apr 20, 2021 at 1:08 PM Steve Hanson <smh@uk.ibm.com> wrote:
'trailingEmptyStrict' was for X12 if I recall. It is not valid to have trailing separators. 'never' I don't have an example for.
Regards
Steve Hanson
IBM Hybrid Integration, Hursley, UK Architect, *IBM DFDL* <http://www.ibm.com/developerworks/library/se-dfdl/index.html> Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> *smh@uk.ibm.com* <smh@uk.ibm.com> tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday
From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: DFDL-WG <dfdl-wg@ogf.org> Date: 20/04/2021 15:12 Subject: [EXTERNAL] [DFDL-WG] example/use-case wanted - separatorSuppressionPolicy 'trailingEmptyStrict' and 'never' Sent by: "dfdl-wg" <dfdl-wg-bounces@ogf.org> ------------------------------
Users of Apache Daffodil have asked what these property values are for, i.e., what are idiomatic uses of this property for actual formats, or in other words, what kind of data are these properties needed for?
I'd like to provide them examples of this if anyone can recall the kinds of specific use cases for these property values.
I've been searching through the example DFDL schemas I can find, and none use trailingEmptyStrict or never for this property.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | *www.owlcyberdefense.com* <http://www.owlcyberdefense.com> Please note: Contributions to the DFDL Workgroup's email discussions are subject to the *OGF Intellectual Property Policy* <http://www.ogf.org/About/abt_policies.php> -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
IBM has DFDL schemas for X12, various releases including the most common 4010, obtainable via support ticket (same as for EDIFACT). IBM has DFDL schemas for X12 HIPAA as part of App Connect for Healthcare v5. I'm sure there are formats out there where all delimiters must be present. Old teletype formats like MATIP maybe. Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh@uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson <smh@uk.ibm.com> Cc: DFDL-WG <dfdl-wg@ogf.org> Date: 20/04/2021 20:03 Subject: [EXTERNAL] Re: [DFDL-WG] example/use-case wanted - separatorSuppressionPolicy 'trailingEmptyStrict' and 'never' I see that X12 is the USA-used EDI message standard - analogous to the UN/EDIFACT used outside USA. I'll pass along that this was motivated by X12, though to my knowledge there is no DFDL schema generally available for any part of X12. ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd I see that X12 is the USA-used EDI message standard - analogous to the UN/EDIFACT used outside USA. I'll pass along that this was motivated by X12, though to my knowledge there is no DFDL schema generally available for any part of X12. I think for SSP 'never' I can gen-up an example pretty easily. Given what I've got below, can you think of where such formats would appear in anything real-world? The only thing I can think of is somebody taking a COBOL structure with two "occurs N times depending on ..." arrays in it, and modernizing the format into something variable-length with separators for use in a more modern application. My example for separatorSuppressionPolicy='never' is below. Assuming dfdl:occursCountKind='implicit' everywhere <sequence dfdl:separator="," dfdl:separatorSuppressionPolicy="never"> <element name="givenName" type="xs:string" minOccurs="0" maxOccurs="3"/> <element name="surname" type="xs:string" minOccurs="0"/> <element name="ph" type="xs:string" minOccurs="0" maxOccurs="6"/> </sequence> This format means there are always 10 locations separated by 9 commas. Whether something is a givenName, surname, or phone number is just determined positionally by counting the separators as the parse passes them. Well-formed instance: "michael, james,,rogers,888-888-8888,777-777-7777,,,," XML flavored DFDL infoset: <givenName>michael</givenName> <givenName>james</givenName> <surname>rogers</surname> <ph>888-888-8888</ph> <ph>777-777-7777</ph> Well-formed instance: "madonna,,,,,,,,," XML flavored equivalent infoset: <givenName>madonna</givenName> Both the above examples would unparse to exactly the input data. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Tue, Apr 20, 2021 at 1:08 PM Steve Hanson <smh@uk.ibm.com> wrote: 'trailingEmptyStrict' was for X12 if I recall. It is not valid to have trailing separators. 'never' I don't have an example for. Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh@uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: DFDL-WG <dfdl-wg@ogf.org> Date: 20/04/2021 15:12 Subject: [EXTERNAL] [DFDL-WG] example/use-case wanted - separatorSuppressionPolicy 'trailingEmptyStrict' and 'never' Sent by: "dfdl-wg" <dfdl-wg-bounces@ogf.org> Users of Apache Daffodil have asked what these property values are for, i.e., what are idiomatic uses of this property for actual formats, or in other words, what kind of data are these properties needed for? I'd like to provide them examples of this if anyone can recall the kinds of specific use cases for these property values. I've been searching through the example DFDL schemas I can find, and none use trailingEmptyStrict or never for this property. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (2)
-
Mike Beckerle -
Steve Hanson