November 2015 - dfdl-wg - lists.ogf.org

still no need for fn:error
by Mike Beckerle 03 Aug '16

03 Aug '16

Here's the example as suggested by Jonathan of how to issue an error that I previously was suggesting needed fn:error. I'm convinced this is sufficient and we can avoid the need for fn:error now. <xs:element name="magic_number" type="ex:uint32" dfdl:byteOrder="bigEndian"> <xs:annotation> <xs:appinfo source="*http://www.ogf.org/dfdl/*<http://www.ogf.org/dfdl/dfdl-1.0/> "> <dfdl:assert test="{ (xs:unsignedInt(.) eq *dfdl:hUInt('0xa1b2c3d4')*) | (xs:unsignedInt(.) eq *dfdl:hUInt('0xd4c3b2a1')*) }" message="{ fn:concat( 'Magic number ', dfdl:hexBinary(dfdl:unsignedInt(.)), ' was not 0xA1B2C3D4 (for bigEndian) or 0xD4C3B2A1 (for littleEndian).' }" /> <dfdl:setVariable ref="ex:bOrd">{ xs:unsignedInt(.) }<dfdl:setVariable> </xs:appinfo> </xs:annotation> </xs:element> -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com

2 11

Cancelled: DFDL Working Group Call
by Steve Hanson 30 Nov '15

30 Nov '15

Mike not available, and no progress on open actions

1 0

Cancelled: DFDL Working Group Call
by Steve Hanson 16 Nov '15

16 Nov '15

Mike unavailable

1 0

Re: [DFDL-WG] EDIFACT schema - daffodil bug or non-bug
by Mike Beckerle 06 Nov '15

06 Nov '15

Bug taken: https://opensource.ncsa.illinois.edu/jira/browse/DFDL-1443 Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> On Fri, Nov 6, 2015 at 1:00 PM, Steve Hanson <smh(a)uk.ibm.com> wrote: > Mike > > I think that is a bug in Daffodil. The DFDL spec says that escapeSchemeRef > applies to simple types with text representation, so Daffodil is evaluating > the escape scheme properties prematurely. If you notice in the UNA > declaration, the simple elements at the start of the UNA that carry the > delimiters and escape character all have dfdl:escapeSchemeRef="" to avoid > tripping the check. > > Regards > > Steve Hanson > Architect, *IBM DFDL* > <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > *IBM Integration Bus* > <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, > Hursley, UK > *smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> > To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> > Date: 04/11/2015 17:17 > Subject: [DFDL-WG] EDIFACT schema - daffodil bug or non-bug > Sent by: "dfdl-wg" <dfdl-wg-bounces(a)ogf.org> > ------------------------------ > > > > > I'm trying to get EDIFACT working on Daffodil. > > I have a somewhat interesting chicken-egg problem. > > This schema uses dfdl:escapeCharacter and dfdl:escapeEscapeCharacter as > expressions. E.g., there is a top-level dfdl:defineVariable named > "EscapeChar" which has a default value, and the expression for the > dfdl:escapeCharacter property is { $ibmEdiFmt:EscapeChar }. > > The default format that is in effect for the root element has > dfdl:lengthKind='delimited'. > > When daffodil starts parsing the top level root/document element, it > enters a parser that is for delimited elements with an escape-scheme in > effect. First thing this parser does is get the escape scheme which > evaluates the expressions for escapeCharacter and escapeEscapeCharacter. > This picks up the default values for those variables and the variables are > then set as "already evaluated", as DFDL specifies that once a variable's > default value has been used, it cannot be subsequently set via > dfdl:setVariable. > > Now, when the very first UNA is encountered, that reads the various > delimiters/escapes from the data, and tries to set the variables. > > But the variables have already been evaluated, on the way into parsing the > "delimited" top level element, and the UNA element itself similarly. > > So it fails with a runtime SDE - default value has already been used. > > So the questions: > > Is this a schema bug in the EDIFACT schema, or is there a principle at > work here indicating that Daffodil cannot evaluate the escape scheme on > entry to an element of length kind delimited unless delimiters are actually > defined? > > It gets worse though. How late bound does this have to be? I can imagine > it being so late as to be after the last child element/group has been > parsed, when the parser unwinds the stack back up to the complex-type > element's tier, and only at that point, when it scans for the terminating > markup, would it then force the evaluation of the escape scheme. But that > seems difficult to implement. However, that would allow the delimiter for > the complex-type element to actually be stored within the children of that > same complex type element. But is this needed? > > One could argue that the EDIFACT schema should have > dfdl:lengthKind='implicit' on these global elements down until the UNA has > been parsed. Though I think that makes authoring schemas harder because a > user thinks of edifact stuff as "a delimited format", and is naturally just > going to want to stick dfdl:lengthKind="delimited" at global scope for all > the schema components. > > Thoughts? > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | > *www.tresys.com* <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are > subject to the *OGF Intellectual Property Policy* > <http://www.ogf.org/About/abt_policies.php> > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > https://www.ogf.org/mailman/listinfo/dfdl-wg > >

1 0

Re: [DFDL-WG] minor spec issue - Section 2: required, optional, and RFC2119 notational conventions
by Steve Hanson 06 Nov '15

06 Nov '15

Action raised. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM Integration Bus, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 26/10/2015 11:09 Subject: Re: [DFDL-WG] minor spec issue - Section 2: required, optional, and RFC2119 notational conventions Mike, I agree with your conclusion. In RFC2119 'must' and 'shall' are also synonyms, and I would like to see the DFDL spec use 'must' throughout and drop the use of 'shall' (it looks like there is only one use - in section 9.3.2.2). Fwiw I've always been suspicious of that statement in section 2, as I don't believe there has ever been a review of the uses of 'must', 'should', 'may' to ensure compliance with RFC2119. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM Systems, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 25/10/2015 21:55 Subject: [DFDL-WG] minor spec issue - Section 2: required, optional, and RFC2119 notational conventions Sent by: "dfdl-wg" <dfdl-wg-bounces(a)ogf.org> In Section 2 on notational conventions we say "The key words must, must not, required, shall, shall not, should, should not, recommended, may, may not and optional in this document are to be interpreted as described in [RFC2119]." Which is fine except for the words "required" and "optional" which we use in several different senses. Section 21 on optional features of the DFDL standard uses "optional" vs. "required" in this sense of RFC2119. But we also use "Optional Occurrence", "Optional Element" very specifically and define them in our glossary. (Along with Required Occurrence and Required Element.) So the above sentence on notational conventions we should just drop the words "optional" and "required". I looked for synonyms for required/optional. mandatory/nonmandatory and compulsory/noncompulsory are ones that we might consider using in the future. We do use mandatory now as in mandatory alignment of character set code units. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg

1 0

EDIFACT schema - daffodil bug or non-bug
by Mike Beckerle 06 Nov '15

06 Nov '15

I'm trying to get EDIFACT working on Daffodil. I have a somewhat interesting chicken-egg problem. This schema uses dfdl:escapeCharacter and dfdl:escapeEscapeCharacter as expressions. E.g., there is a top-level dfdl:defineVariable named "EscapeChar" which has a default value, and the expression for the dfdl:escapeCharacter property is { $ibmEdiFmt:EscapeChar }. The default format that is in effect for the root element has dfdl:lengthKind='delimited'. When daffodil starts parsing the top level root/document element, it enters a parser that is for delimited elements with an escape-scheme in effect. First thing this parser does is get the escape scheme which evaluates the expressions for escapeCharacter and escapeEscapeCharacter. This picks up the default values for those variables and the variables are then set as "already evaluated", as DFDL specifies that once a variable's default value has been used, it cannot be subsequently set via dfdl:setVariable. Now, when the very first UNA is encountered, that reads the various delimiters/escapes from the data, and tries to set the variables. But the variables have already been evaluated, on the way into parsing the "delimited" top level element, and the UNA element itself similarly. So it fails with a runtime SDE - default value has already been used. So the questions: Is this a schema bug in the EDIFACT schema, or is there a principle at work here indicating that Daffodil cannot evaluate the escape scheme on entry to an element of length kind delimited unless delimiters are actually defined? It gets worse though. How late bound does this have to be? I can imagine it being so late as to be after the last child element/group has been parsed, when the parser unwinds the stack back up to the complex-type element's tier, and only at that point, when it scans for the terminating markup, would it then force the evaluation of the escape scheme. But that seems difficult to implement. However, that would allow the delimiter for the complex-type element to actually be stored within the children of that same complex type element. But is this needed? One could argue that the EDIFACT schema should have dfdl:lengthKind='implicit' on these global elements down until the UNA has been parsed. Though I think that makes authoring schemas harder because a user thinks of edifact stuff as "a delimited format", and is naturally just going to want to stick dfdl:lengthKind="delimited" at global scope for all the schema components. Thoughts? Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

duplicate message re: EDIFACT
by Mike Beckerle 04 Nov '15

04 Nov '15

Please ignore first and read the second one. I tweeked it further to refine the questions being asked. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

EDIFACT schema - daffodil bug or schema bug
by Mike Beckerle 04 Nov '15

04 Nov '15

I'm trying to get EDIFACT working on Daffodil. I have a somewhat interesting chicken-egg problem. This schema uses dfdl:escapeCharacter and dfdl:escapeEscapeCharacter as expressions. E.g., there is a top-level dfdl:defineVariable named "EscapeChar" which has a default value, and the expression for the dfdl:escapeCharacter property is { $ibmEdiFmt:EscapeChar }. The default format that is in effect for the root element 'Interchange' has dfdl:lengthKind='delimited'. When daffodil starts parsing the top level root/document element, it enters a parser that is for delimited elements with an escape-scheme in effect. First thing this parser does is get the escape scheme which evaluates the expressions for escapeCharacter and escapeEscapeCharacter. This picks up the default values for those variables and the variables are then set as "already evaluated", as DFDL specifies that once a variable's default value has been used, it cannot be subsequently set via dfdl:setVariable. Now, when the very first UNA is encountered, that reads the various delimiters/escapes from the data, and tries to set the variables. But the variables have already been evaluated, on the way into parsing the "delimited" top level element. So it fails with a runtime SDE - default value has already been used for EscapeChar variable. So the question is. Is this a schema bug in the EDIFACT schema, or is there a principle at work here for DFDL implementations generally? The principle would be that a complex-typed element of length kind delimited when there is no definition of terminator nor applicable separator, that this situation is treated like dfdl:lengthKind='implicit' and the escapeScheme is therefore not relevant and the expressions "must not" be evaluated? One could argue that the EDIFACT schema should have dfdl:lengthKind='implicit' on the 'Interchange' element and any other complex type element. Thoughts? Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

OGF DFDL WG Call Minutes 2015-11-03
by Steve Hanson 03 Nov '15

03 Nov '15

Please find minutes from the above call at https://redmine.ogf.org/dmsf_files/13503?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
by Michele Zundo 02 Nov '15

02 Nov '15

Steve, I believe I do not understand this. can you make an example ? Michele > On 02 Nov 2015, at 16:33 , Steve Hanson <smh(a)uk.ibm.com> wrote: > > I was also going to add that DFDL supports xs:union on simple types, enabling dfdl:checkConstraints to be used when there are multiple ranges. > > · Unions; the memberTypes must be derived from the same simple type. DFDL annotations are not permitted on union members. > > The purpose of unions is to allow multiple constraints via facets such as multiple independent range restrictions on numbers. This enhances the ability to do rich validation of data. > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> > To: Michele Zundo <michele.zundo(a)esa.int> > Cc: Steve Hanson/UK/IBM@IBMGB, Rui Mestre <rui.mestre(a)deimos.com.pt>, "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Montserrat Piñol <mpinol(a)eopp.esa.int>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int> > Date: 02/11/2015 14:33 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > > Michele, > > I think the first thing I'd propose for the set membership test is for us to add the XPath 2.0 intersect operator to DFDL. So your discriminator becomes: > > <dfdl:discriminatortest="{fn:exists(/Packet_Primary_Header/Packet_Identification/APID intersect (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, > 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, > 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, > 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, > 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, > 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 256,257,258,259,260,261,262,263,264,265,266,267,268, > 272,273,274,275,276,277,278,279,280,281,282,283,284, > 288,289,290,291,292,293,294,295,296,297,298,299,300, > 304,305,306,307,308,309,310,311,312,313,314,315,316, > 320,321,322,323,324,325,326,327,328,329,330,331,332, > 336,337,338,339,340,341,342,343,344,345,346,347,348))}"/> > > > This is a small addition to DFDL, but completely in the spirit of sticking with XPath 2.0 wherever possible. > > ...mike beckerle > > > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> > > > On Wed, Oct 28, 2015 at 7:22 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote: > Hi Steve, > > your example below addresses the belonging to a range, how would then the > belonging to a set be checked ? Currently our usage looks like here: > > > > <xs:complexTypename="TypePacketData"> > <xs:sequence> > <xs:choice> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, > 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, > 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, > 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, > 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, > 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 256,257,258,259,260,261,262,263,264,265,266,267,268, > 272,273,274,275,276,277,278,279,280,281,282,283,284, > 288,289,290,291,292,293,294,295,296,297,298,299,300, > 304,305,306,307,308,309,310,311,312,313,314,315,316, > 320,321,322,323,324,325,326,327,328,329,330,331,332, > 336,337,338,339,340,341,342,343,344,345,346,347,348]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="MSI_Packet_Secondary_Header"type="TypePacketData_MSI"/> > <xs:element name="MSI_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [769,771,772,774,775,777,779,780,781,785,787,788,790,791,793,795,796,797]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_GPSR_Packet_Secondary_Header"type="TypePacketData_TMGPSR"/> > <xs:element name="TM_GPSR_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID in [99999]}" /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_Time_Packet_Secondary_Header"type="TypePacketData_TMT92"/> > <xs:element name="TM_Time_Packet_User_Data_Field"type="TypeUserData_TMT92"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [596,598,612,614,628,630,660,662]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_STR_Packet_Secondary_Header"type="TypePacketData_TMSTR"/> > <xs:element name="TM_STR_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [150,182]}" /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_CSW_AOCS_Packet_Secondary_Header"type="TypePacketData_TMCSWAOCS"/> > <xs:element name="TM_CSW_AOCS_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID inrange [144,149] > or /Packet_Primary_Header/Packet_Identification/APID inrange [151,181] > or /Packet_Primary_Header/Packet_Identification/APID inrange [183,239] > or /Packet_Primary_Header/Packet_Identification/APID inrange [516,575] > or /Packet_Primary_Header/Packet_Identification/APID in [592,593,594,595,597,613,629,656,657,658,659,661] > or /Packet_Primary_Header/Packet_Identification/APID inrange [599,611] > or /Packet_Primary_Header/Packet_Identification/APID inrange [615,627] > or /Packet_Primary_Header/Packet_Identification/APID inrange [631,639] > or /Packet_Primary_Header/Packet_Identification/APID inrange [663,671] > or /Packet_Primary_Header/Packet_Identification/APID in [768,770,776,778,782,783,784,792,794,798,799] > or /Packet_Primary_Header/Packet_Identification/APID inrange [1280,1295]}" > /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_HKTM_Packet_Secondary_Header"type="TypePacketData_HKTM"/> > <xs:element name="TM_HKTM_User_Data_Field"type="TypeUserData_HKTM"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID in [2047]}" > /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_IDLE_Packet_Secondary_Header"type="TypePacketData_TMIDLE" > /> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{true}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="UNKNOWN_User_Data_Field"type="TypePacketData_UNKNOWN"/> > </xs:sequence> > </xs:choice> > </xs:sequence> > </xs:complexType> > > > On 27 Oct 2015, at 18:01 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Just to clarify, dfdl:assert and dfdl:discriminator are identical when the expression returns false, the difference between the two is when the expression returns true and the parser is in a 'point of uncertainty' (eg choice) - a discriminator returning true has a positive discrimination on the point of uncertainty, an assert does not. > > I suspect that most of the time you can achieve what you want with just dfdl:checkConstraints() and simple type inheritance. Example below (syntax simplified for clarity). But I am happy to be proved wrong, so if you have an example ... ? > > <xs:simpleType name="alertType"> > <xs:restriction base="xs:unsignedInt> > <xs:minInclusive value="0"/> > <xs:maxInclusive value="999"/> > </xs:restriction> > </xs:simpleType> > > <xs:simpleType name="redAlertType"> > <xs:restriction base="alertType"> > <xs:minInclusive value="0"/> > <xs:maxInclusive value="200"/> > </xs:restriction> > </xs:simpleType> > > <xs:simpleType name="yellowAlertType"> > <xs:restriction base="alertType"> > <xs:minInclusive value="201"/> > <xs:maxInclusive value="500"/> > </xs:restriction> > </xs:simpleType> > > ... > > <xs:choice> > <xs:element name="red"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="redAlertType"> > <dfdl:discriminator test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > <xs:element name="yellow"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="yellowAlertType"> > <dfdl:discriminator test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > <xs:element name="other"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="alertType"> > <dfdl:assert test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > mob:+44-7717-378890 <tel:%2B44-7717-378890> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 27/10/2015 16:27 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Yes, exactly. This is one of the current use case, in addition we could also use the value to > implement different type of checks e.g. some of the parameters we look at have a type, > a range outside of which they are invalid and a set of other ranges that correspond to various other conditions > (e.g. red alarm or yellow alarm). > > Michele > > > > > On 27 Oct 2015, at 17:19 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > "Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G > TF schema version)." > > That sounds like you would use this in a dfdl:discriminator ? > > Yes, exactly. While this is one of the current use case, in addition we would also use the value to > implement different type of checks e.g. some of the parameters we look at have a type, > a range outside of which they are invalid and a set of other ranges that correspond to various other conditions > (e.g. red alarm or yellow alarm). > > Michele > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > mob:+44-7717-378890 <tel:%2B44-7717-378890> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 27/10/2015 16:09 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Dear Steve, > > to progress on this action: > > 1) it is our intention to “evolve” from our own dmx:assert to the dfdl:assert in order to standardise as much as possible > with the official spec provided we find a solution for what below. > > 2) we had an iteration with our developers and came to the conclusion that there is > need for both syntaxes: the existing (in the standard) dfdl:checkConstraints()and the new > one you proposed i.e. dfdl:checkRangeand dfdl:checkValue > > The main semantic difference between checkValue/checkRangeand checkConstraints > is that in one case we we need to associate a range to a quantity in a sort of static way > (i.e. the type “knowns” it has a range associated with it) while the checkvalue/checkrange > is something that you can do also if a range is not defined (see below). > > For example you could have a definition for a specification sheet of an electrical device. > The XSD type (Volts) intrinsically defined with a range 0-1000 but wanted > to check in derived schemas if value is between 100 and 150 or 200 and 250 and this information > would be used in different way. > > This could be used to check 0-1000 hard constraints to see if the the network will support it > while range 100-150 and 200-250 would check compatibility with US or European Voltages for commercial appliances. > > Note that dfdl:assert applies a condition to the parsing so that if the check fails the parsing is aborted with a severe error. > In the example above we would want to apply checkConstraint defined with a range 0-1000 in a dfdl:assert (to stop the parsing due to incorrect values) but not a checkvalue/checkrange. > Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G > TF schema version). > > Michele > > > > On 05 Oct 2015, at 16:23 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Hi Michele > > Any update from your discussion? > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 23/09/2015 16:44 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Dear Steve, > > thanks for the suggestion (same from Mike) we need to discuss this internally with the developers and look at few use cases > to see what would be the consequences/advantages/disadvantages. > > Give us a little bit of time.. > > Michele > > On 22 Sep 2015, at 19:12 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Hi Michele > > Thanks for your quick response. The WG discussed this on the call today. > > In fact, we wondered if the same result could already be achieved using dfdl:checkConstraints() and declaring the enums or range using XSD facets on xs:simpleType restrictions. > > Have you considered whether dfdl:checkConstraints() achieves what you want? > > Example: > > <xs:simpleType name="myRange"> > <xs:restriction base="xs:int> > <xs:minInclusive value="100"/> > <xs:maxInclusive value="200"/> > </xs:restriction> > </xs:simpleType> > > <xs:element name="myValue" type="myRange"> > <xs:annotation><xs:appinfo ...> > <dfdl:assert>dfdl:checkConstraints(.)</dfdl:assert> > </xs:appinfo></xs:annotation> > </xs:element> > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > Cc: Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 22/09/2015 17:21 > Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > Sent by: dfdl-wg-bounces(a)ogf.org <mailto:dfdl-wg-bounces@ogf.org> > > > > Dear Steve, > > nice to hear things are moving. > > The syntax below seems both reasonable and readable to me which is a good thing. > > It addresses the belonging to decimal groups (integer only ??) I understand. (xs:decimal) > > 1) It might be also useful (not in our application but in general) to see if it makes sense also to define > for dfdl:checkValues belongings to set of “enum” like value > > e.g. if we have a type for $val defined as (ON, OFF or STANDBY) or (MARRIED, SINGLE, WIDOWER) > can we also define a syntax that the $node belongs to it ? > > 2) for dfdl:checkRange it might make sense also to allow float numbers e.g. X between 3.1 to 9.5 > > > Michele > > > > PS I will forward this proposal to our DFDL4S developers (in copy) to get their thinking. > > > From: "Steve Hanson" <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> > Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > Date: 22 Sep 2015 16:50:13 CEST > To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>> > > > To kick start this action, here is a proposal ... builds on the precedent provided by dfdl:checkConstraints($node). > dfdl:checkValues($node, $val1, $val2, ...) Returns boolean true if the specified node value matches any of the values provided by $val1 etc. > The type of $val1 etc must be compatible with the type of $node. > > It is a schema definition error if the $node argument is a complex element. > > The number of value arguments is implementation-defined. > > dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal. > > It is a schema definition error if the $node argument is a complex element. > > > dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal. > > It is a schema definition error if the $node argument is a complex element. > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Steve Hanson/UK/IBM > To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>> > Date: 11/08/2015 16:28 > Subject: Fw: [DFDL]: First Release of DFDL4S Parser Library > > > > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > ----- Forwarded by Steve Hanson/UK/IBM on 11/08/2015 16:27 ----- > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> > Cc: Steve Hanson/UK/IBM@IBMGB, Maurizio De Bartolomei <Maurizio.De.Bartolomei(a)esa.int <mailto:Maurizio.De.Bartolomei@esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, "Rui Mestre (DME)" <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 18/05/2015 08:47 > Subject: Re: [DFDL]: First Release of DFDL4S Parser Library > > > > Thanks Mike, > > we will add this to our list to be considered/noted. > > However reading your explanation (NB I’m NOT at all an XPath expert) it seemed you > had some good reason for avoiding longer than 1 path, so I would like to avoid our DFDL4S > project results in an over-complication of the DFDL implementation/use of Xpath > unless there are other reasons/users/rationale requiring this feature. > (btw the syntax is still weird-ish: “intersect” reminds me of Venn Diagrams…) > > As a project manager I always evaluate solutions and their cost vs the benefit they bring, > and I believe the DFDL community should keep this is mind. > > Michele > > PS The syntax above anser to the question “belongs to” , would there be any way to express ranges of values then ? > > > > On 15 May 2015, at 16:24 , Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> wrote: > > Just a few comments on DFDL4S, and also thank you to Michele and team for the presentation on Tuesday. > > I think all the issues in the spreadsheet are fairly easily fixed in that they are not major changes to DFDL4S, and would bring it into much closer compliance with the DFDL spec. > > The exception is the XPath limitations where DFDL4S has gone beyond what XPath 2.0 allows and invented new syntax for expressing set membership requirements. > > So I took a look, and XPath 2.0 has a set intersect operator: ns1 intersect ns2 => ns3 > > This isn't in DFDL today, but might be usable to achieve the set membership test; however, it requires use of XPath node sequences of length greater than 1, which DFDL has avoided mostly. I say mostly as there are XPath expressions that return node sequences of length greater than 1 and those can be arguments to fn:count(...) for example. > > So far in DFDL such node sequences cannot "leak out" of the XPath expression into DFDL elements, and I think the usage in DFDL4S is similar in that these node sequences would be needed only to check for set membership, so the result is just a boolean as part of an assert/discriminator. > > We should examine whether XPath 2.0 set intersection is enough to meet the need. > > I believe the expressions would be something like: > > fn:exists( . intersect (123, 456, 789, .... many more items....) ) > > > - mike beckerle > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> > > > On Tue, May 12, 2015 at 10:45 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote: > for reference, > here a summary of the reported problem in an excel sheet. > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > #### Sentinel2X-bandTMISPTypes.xsd moved to MyAttachments Repository V3.8 (Link <notes:///802575AF0030E827/5DE5236E5AD1645685256EE0001BBADF/784AE1CEA59C046380257EAB0027C2EF>) on 24 August 2015 by Steve Hanson. > > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > ----------------------------------------- Michele Zundo Head of Ground System Definition and Verification Office EOP-PEP European Space Agency, ESTEC e-mail: michele.zundo(a)esa.int This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email.

2 1