Re: [DFDL-WG] newbie question
I didn't see the requirement for a pattern facet in your problem description. Feel free to use a pattern facet if you require one. If you want to perform cross-field validation then you should probably use DFDL asserts ( not discriminators ) for that. The best place to put the asserts is on the parent element, addressing ./A and ./C. You could use the exceptionType attribute on the assert (if the DFDL processor supports it) to make it throw a recoverable error instead of a processing error. That would enable invalid data to be parsed, and you would still be notified of any ( and all ) violations of the validation rules. regards, Tim Kimber, IBM Integration Bus Development (Industry Packs) Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: "mfcplus@yahoo.com" <mfcplus@yahoo.com> To: Tim Kimber/UK/IBM@IBMGB, Date: 28/03/2014 13:21 Subject: Re: [DFDL-WG] newbie question so if i am not to use a pattern, how do i validate against the alphanumeric part? also i have a conditional in the message type, should i use discriminator with xpath rxpression for that? elemA /elemB/elemC CRLF elemC/elemD/elemE CRLF elemF/elemG/elemH CRLF for the above strcuture, i have a requirement that elemC must not exist if elemA is not there, and must exist if elemA is there. sequenceC must not exist if sequenceA is not there, sequenceC must exist if sequenceA is there. just wondering how do i do that with dfdl discriminator. mant thanks again Sent from Yahoo Mail on Android. From: Tim Kimber <KIMBERT@uk.ibm.com>; To: mfcplus@yahoo.com <mfcplus@yahoo.com>; Cc: <dfdl-wg@ogf.org>; Subject: Re: [DFDL-WG] newbie question Sent: Fri, Mar 28, 2014 11:34:12 AM The key point here is that the data format is delimited, and there is a requirement to validate the parsed info set against some rules ( length constraints, in this case). I would - set terminator="/" - set lengthKind='delimited' - use xs:minLength and xs:maxLength facets to check the length ( or use a pattern facet if you must, but min/maxLength facets will use less CPU ) - switch on validation in the DFDL parser so that the length of the delimited value gets checked against the facets in the schema. Other possibilities exist ( e.g. DFDL assert to check the length ) but this is the most efficient. regards, Tim Kimber, IBM Integration Bus Development (Industry Packs) Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: "mfcplus@yahoo.com" <mfcplus@yahoo.com> To: Tim Kimber/UK/IBM@IBMGB, Date: 28/03/2014 11:25 Subject: Re: [DFDL-WG] newbie question a field that is 3-5 chars long alphanumeric, basically a \w{3,5}, it should give out parsing error if it doesnt fit by the pattern, no initiator terminator is / Sent from Yahoo Mail on Android From: Tim Kimber <KIMBERT@uk.ibm.com>; To: <dfdl-wg@ogf.org>; Subject: Re: [DFDL-WG] newbie question Sent: Fri, Mar 28, 2014 10:19:47 AM It depends on the data format. Normally a variable length field is terminated by a delimiter, so a data pattern/regex is not required. Can you describe the format in a little more detail? regards, Tim Kimber, IBM Integration Bus Development (Industry Packs) Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Bing Lu <mfcplus@yahoo.com> To: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Date: 28/03/2014 03:35 Subject: [DFDL-WG] newbie question Sent by: dfdl-wg-bounces@ogf.org what's the best dfdl approache to describe a variable length alpha numeric field? should i use simpletype restriction pattern, or use dfdl:assert testPattern? thanks in advance-- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Bing Please be aware that DFDL is not designed to be a full blown data validator or rules engine. Rather, it is a data parser that can perform *some* validation, including XSDL validation (using XSDL facets) and some simple cross-field checking (using dfdl:assert). Note that the dfdl:assert mechanism is primarily intended to help the parser make the correct decisions, and not for validating. That's partly why the DFDL expression language only uses a subset of XPath 2.0. If you need to perform complex cross-field validation (like checking that a 'total' field in a trailer record is the sum of the 'amount' fields in all the data records) then you are recommended to do that post-parse on the DFDL infoset that the parser creates, using a technology of your choice. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM@IBMGB To: "mfcplus@yahoo.com" <mfcplus@yahoo.com>, Cc: dfdl-wg@ogf.org Date: 28/03/2014 13:46 Subject: Re: [DFDL-WG] newbie question Sent by: dfdl-wg-bounces@ogf.org I didn't see the requirement for a pattern facet in your problem description. Feel free to use a pattern facet if you require one. If you want to perform cross-field validation then you should probably use DFDL asserts ( not discriminators ) for that. The best place to put the asserts is on the parent element, addressing ./A and ./C. You could use the exceptionType attribute on the assert (if the DFDL processor supports it) to make it throw a recoverable error instead of a processing error. That would enable invalid data to be parsed, and you would still be notified of any ( and all ) violations of the validation rules. regards, Tim Kimber, IBM Integration Bus Development (Industry Packs) Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: "mfcplus@yahoo.com" <mfcplus@yahoo.com> To: Tim Kimber/UK/IBM@IBMGB, Date: 28/03/2014 13:21 Subject: Re: [DFDL-WG] newbie question so if i am not to use a pattern, how do i validate against the alphanumeric part? also i have a conditional in the message type, should i use discriminator with xpath rxpression for that? elemA /elemB/elemC CRLF elemC/elemD/elemE CRLF elemF/elemG/elemH CRLF for the above strcuture, i have a requirement that elemC must not exist if elemA is not there, and must exist if elemA is there. sequenceC must not exist if sequenceA is not there, sequenceC must exist if sequenceA is there. just wondering how do i do that with dfdl discriminator. mant thanks again Sent from Yahoo Mail on Android. From: Tim Kimber <KIMBERT@uk.ibm.com>; To: mfcplus@yahoo.com <mfcplus@yahoo.com>; Cc: <dfdl-wg@ogf.org>; Subject: Re: [DFDL-WG] newbie question Sent: Fri, Mar 28, 2014 11:34:12 AM The key point here is that the data format is delimited, and there is a requirement to validate the parsed info set against some rules ( length constraints, in this case). I would - set terminator="/" - set lengthKind='delimited' - use xs:minLength and xs:maxLength facets to check the length ( or use a pattern facet if you must, but min/maxLength facets will use less CPU ) - switch on validation in the DFDL parser so that the length of the delimited value gets checked against the facets in the schema. Other possibilities exist ( e.g. DFDL assert to check the length ) but this is the most efficient. regards, Tim Kimber, IBM Integration Bus Development (Industry Packs) Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: "mfcplus@yahoo.com" <mfcplus@yahoo.com> To: Tim Kimber/UK/IBM@IBMGB, Date: 28/03/2014 11:25 Subject: Re: [DFDL-WG] newbie question a field that is 3-5 chars long alphanumeric, basically a \w{3,5}, it should give out parsing error if it doesnt fit by the pattern, no initiator terminator is / Sent from Yahoo Mail on Android From: Tim Kimber <KIMBERT@uk.ibm.com>; To: <dfdl-wg@ogf.org>; Subject: Re: [DFDL-WG] newbie question Sent: Fri, Mar 28, 2014 10:19:47 AM It depends on the data format. Normally a variable length field is terminated by a delimiter, so a data pattern/regex is not required. Can you describe the format in a little more detail? regards, Tim Kimber, IBM Integration Bus Development (Industry Packs) Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Bing Lu <mfcplus@yahoo.com> To: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Date: 28/03/2014 03:35 Subject: [DFDL-WG] newbie question Sent by: dfdl-wg-bounces@ogf.org what's the best dfdl approache to describe a variable length alpha numeric field? should i use simpletype restriction pattern, or use dfdl:assert testPattern? thanks in advance-- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (2)
-
Steve Hanson -
Tim Kimber