May 2016 - dfdl-wg - lists.ogf.org

Confirmed: OGF DFDL Working Group call (24 May 15:00 GDT in IBM: Hursley DE3S09)
by Steve Hanson 23 May '16

23 May '16

Calls for 2016. OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 Netherlands Toll-Free 0-800-363-6036 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg DFDL Getting Started: http://www.ibm.com/developerworks/library/se-dfdl/ DFDL 1.0 Spec: http://www.ogf.org/documents/GFD.207.pdf Call will go ahead tomorrow, I will send out agenda later.

1 0

Re: [DFDL-WG] Action 282
by Michele Zundo 20 May '16

20 May '16

Dear Steve, sorry for the delay. We have verified internally that your proposal of extension of support of XPath and the 2 new function address our need. So from our side your proposal below is OK. When you have updated the DFDL language reference doc with the let us know. Regards > On 11 May 2016, at 9:55 , Steve Hanson <smh(a)uk.ibm.com> wrote: > > Michele & team > > Please join the DFDL WG call on 24th May so we can review the proposal and hopefully close this action. > > Thanks > > Steve Hanson > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > > > > From: Michele Zundo <michele.zundo(a)esa.int> > To: Steve Hanson/UK/IBM@IBMGB > Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int>, Montserrat Piñol <mpinol(a)eopp.esa.int>, Rui Mestre <rui.mestre(a)deimos.com.pt> > Date: 12/04/2016 16:43 > Subject: Re: [DFDL-WG] Action 282 > > > > Steve, I see this only now, > > we will not have time this nor the coming week, as I want to discuss the proposal internally. > > We will review the proposal below and come back to you. > > Michele > > > > > > On 11 Apr 2016, at 10:47 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > 282 > Does XPath have operators for checking a value is in a range? (Steve) > 12/5: Investigate whether equivalent to DFDL4S 'in' operator exists. > 23/6: Mike has found an XPath 'intersect' operator. Handles the enumeration case well, but not as convenient for ranges as DFDL4S's 'in' operator. > 11/8: Looked back at the motivating example from DFDL4S. Agreed that DFDL functions to do the equivalent of 'in' and 'inrange' would be useful if nothing can be re-used from XPath. Steve to write up a proposal taking into account different data types. > ... > 22/9: Proposal sent by Steve for new functions dfdl:checkValues(), dfdl:checkRangeInclusive(), dfdl:checkRangeExclusive(). Discussed whether both range functions needed, and whether they should be allowed for float and double. Mike noted that dfdl:checkConstraints() and simple types could be used instead of all three functions if values were static. Follow-up with DFDL4S team to see if they had thought of that. > 3/11: DFDL4S not happy with usability of dfdl:checkConstraints type, but ok with the intersection operator. Next step is to see what the DFDL4S schema would look like if rewritten to use dfdl:checkConstraints and intersection. > 5/1/16: Agreed that dfdl:checkConstraints() is not ideal as it requires creation of union simple types. Steve to go back to his proposal from 22/9/15 and rework to include use of the intersect operator. > 16/2: No progress > 1/3: Discussed with ESA team. Steve to rework his proposal from 22/9/15. > > Reworked proposal: > > a) Add XPath 2.0 'intersect' and "except" operators to the list of supported operators. > > Updated Table 57 as follows: > DFDL Expression ::= "{" Expr "}" > Expr ::= ExprSingle > ExprSingle ::= IfExpr > | OrExpr > IfExpr ::= "if" "(" Expr ")" "then" ExprSingle "else" ExprSingle > OrExpr ::= AndExpr ( "or" AndExpr )* > AndExpr ::= ComparisonExpr ( "and" ComparisonExpr )* > ComparisonExpr ::= AdditiveExpr ( (ValueComp) AdditiveExpr)? > AdditiveExpr ::= MultiplicativeExpr ( ("+" | "-") MultiplicativeExpr )* > MultiplicativeExpr ::= IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ( ("*" | "div" | "idiv" | "mod") IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> )* > IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ::= UnaryExpr <https://www.w3.org/TR/xpath20/#doc-xpath-InstanceofExpr> ( ("intersect" | "except") UnaryExpr )* > UnaryExpr ::= ("-" | "+")* ValueExpr > ValueExpr ::= PathExpr > ValueComp ::= "eq" | "ne" | "lt" | "le" | "gt" | "ge" > PathExpr ::= ("/" RelativePathExpr?) > | RelativePathExpr | FilterExpr > RelativePathExpr ::= StepExpr (("/") StepExpr)* > StepExpr ::= AxisStep > AxisStep ::= (ReverseStep | ForwardStep) Predicate? > ForwardStep ::= (ForwardAxis NodeTest) | AbbrevForwardStep > ForwardAxis ::= ("child" "::") > | ("self" "::") > AbbrevForwardStep ::= NodeTest | ContextItemExpr > ReverseStep ::= (ReverseAxis NodeTest) | AbbrevReverseStep > ReverseAxis ::= ("parent" "::") > AbbrevReverseStep ::= ".." > NodeTest ::= NameTest > NameTest ::= QName > FilterExpr ::= PrimaryExpr Predicate? > Predicate ::= "[" Expr "]" > PrimaryExpr ::= Literal | VarRef | ParenthesizedExpr | ContextItemExpr | FunctionCall > Literal ::= NumericLiteral | StringLiteral > NumericLiteral ::= IntegerLiteral | DecimalLiteral | DoubleLiteral > VarRef ::= "$" VarName > VarName ::= QName > ParenthesizedExpr ::= "(" Expr ")" > ContextItemExpr ::= "." > FunctionCall ::= QName "(" (ExprSingle ("," ExprSingle)*)? ")" > > > > > b) New DFDL functions > > dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double. > > It is a schema definition error if the $node argument is a complex element. > > dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double. > > It is a schema definition error if the $node argument is a complex element. > > > > > Regards > > Steve Hanson > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:+44-1962-815848> > mob:+44-7717-378890 > > > > From: Steve Hanson/UK/IBM > To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > Cc: "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 02/11/2015 19:51 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > > <xs:simpleType name="redAndYellowAlertType"> > <xs:union memberTypes="redAlertType yellowAlertType"/> > </xs:simpleType> > > The above is a type that allows ranges 0-200 and 201-500. > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:+44-1962-815848> > mob:+44-7717-378890 > > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 02/11/2015 16:08 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Steve, > > I believe I do not understand this. > > can you make an example ? > > Michele > > On 02 Nov 2015, at 16:33 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > I was also going to add that DFDL supports xs:union on simple types, enabling dfdl:checkConstraints to be used when there are multiple ranges. > > · Unions; the memberTypes must be derived from the same simple type. DFDL annotations are not permitted on union members. > > The purpose of unions is to allow multiple constraints via facets such as multiple independent range restrictions on numbers. This enhances the ability to do rich validation of data. > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:+44-1962-815848> > mob:+44-7717-378890 > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> > To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > Cc: Steve Hanson/UK/IBM@IBMGB, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 02/11/2015 14:33 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > > Michele, > > I think the first thing I'd propose for the set membership test is for us to add the XPath 2.0 intersect operator to DFDL. So your discriminator becomes: > > <dfdl:discriminatortest="{fn:exists(/Packet_Primary_Header/Packet_Identification/APID intersect (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, > 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, > 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, > 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, > 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, > 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 256,257,258,259,260,261,262,263,264,265,266,267,268, > 272,273,274,275,276,277,278,279,280,281,282,283,284, > 288,289,290,291,292,293,294,295,296,297,298,299,300, > 304,305,306,307,308,309,310,311,312,313,314,315,316, > 320,321,322,323,324,325,326,327,328,329,330,331,332, > 336,337,338,339,340,341,342,343,344,345,346,347,348))}"/> > > > This is a small addition to DFDL, but completely in the spirit of sticking with XPath 2.0 wherever possible. > > ...mike beckerle > > > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> > > > On Wed, Oct 28, 2015 at 7:22 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote: > Hi Steve, > > your example below addresses the belonging to a range, how would then the > belonging to a set be checked ? Currently our usage looks like here: > > > > <xs:complexTypename="TypePacketData"> > <xs:sequence> > <xs:choice> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, > 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, > 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, > 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, > 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, > 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 256,257,258,259,260,261,262,263,264,265,266,267,268, > 272,273,274,275,276,277,278,279,280,281,282,283,284, > 288,289,290,291,292,293,294,295,296,297,298,299,300, > 304,305,306,307,308,309,310,311,312,313,314,315,316, > 320,321,322,323,324,325,326,327,328,329,330,331,332, > 336,337,338,339,340,341,342,343,344,345,346,347,348]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="MSI_Packet_Secondary_Header"type="TypePacketData_MSI"/> > <xs:element name="MSI_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [769,771,772,774,775,777,779,780,781,785,787,788,790,791,793,795,796,797]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_GPSR_Packet_Secondary_Header"type="TypePacketData_TMGPSR"/> > <xs:element name="TM_GPSR_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID in [99999]}" /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_Time_Packet_Secondary_Header"type="TypePacketData_TMT92"/> > <xs:element name="TM_Time_Packet_User_Data_Field"type="TypeUserData_TMT92"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [596,598,612,614,628,630,660,662]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_STR_Packet_Secondary_Header"type="TypePacketData_TMSTR"/> > <xs:element name="TM_STR_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [150,182]}" /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_CSW_AOCS_Packet_Secondary_Header"type="TypePacketData_TMCSWAOCS"/> > <xs:element name="TM_CSW_AOCS_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID inrange [144,149] > or /Packet_Primary_Header/Packet_Identification/APID inrange [151,181] > or /Packet_Primary_Header/Packet_Identification/APID inrange [183,239] > or /Packet_Primary_Header/Packet_Identification/APID inrange [516,575] > or /Packet_Primary_Header/Packet_Identification/APID in [592,593,594,595,597,613,629,656,657,658,659,661] > or /Packet_Primary_Header/Packet_Identification/APID inrange [599,611] > or /Packet_Primary_Header/Packet_Identification/APID inrange [615,627] > or /Packet_Primary_Header/Packet_Identification/APID inrange [631,639] > or /Packet_Primary_Header/Packet_Identification/APID inrange [663,671] > or /Packet_Primary_Header/Packet_Identification/APID in [768,770,776,778,782,783,784,792,794,798,799] > or /Packet_Primary_Header/Packet_Identification/APID inrange [1280,1295]}" > /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_HKTM_Packet_Secondary_Header"type="TypePacketData_HKTM"/> > <xs:element name="TM_HKTM_User_Data_Field"type="TypeUserData_HKTM"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID in [2047]}" > /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_IDLE_Packet_Secondary_Header"type="TypePacketData_TMIDLE" > /> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{true}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="UNKNOWN_User_Data_Field"type="TypePacketData_UNKNOWN"/> > </xs:sequence> > </xs:choice> > </xs:sequence> > </xs:complexType> > > > On 27 Oct 2015, at 18:01 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Just to clarify, dfdl:assert and dfdl:discriminator are identical when the expression returns false, the difference between the two is when the expression returns true and the parser is in a 'point of uncertainty' (eg choice) - a discriminator returning true has a positive discrimination on the point of uncertainty, an assert does not. > > I suspect that most of the time you can achieve what you want with just dfdl:checkConstraints() and simple type inheritance. Example below (syntax simplified for clarity). But I am happy to be proved wrong, so if you have an example ... ? > > <xs:simpleType name="alertType"> > <xs:restriction base="xs:unsignedInt> > <xs:minInclusive value="0"/> > <xs:maxInclusive value="999"/> > </xs:restriction> > </xs:simpleType> > > <xs:simpleType name="redAlertType"> > <xs:restriction base="alertType"> > <xs:minInclusive value="0"/> > <xs:maxInclusive value="200"/> > </xs:restriction> > </xs:simpleType> > > <xs:simpleType name="yellowAlertType"> > <xs:restriction base="alertType"> > <xs:minInclusive value="201"/> > <xs:maxInclusive value="500"/> > </xs:restriction> > </xs:simpleType> > > ... > > <xs:choice> > <xs:element name="red"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="redAlertType"> > <dfdl:discriminator test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > <xs:element name="yellow"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="yellowAlertType"> > <dfdl:discriminator test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > <xs:element name="other"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="alertType"> > <dfdl:assert test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > mob:+44-7717-378890 <tel:%2B44-7717-378890> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 27/10/2015 16:27 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Yes, exactly. This is one of the current use case, in addition we could also use the value to > implement different type of checks e.g. some of the parameters we look at have a type, > a range outside of which they are invalid and a set of other ranges that correspond to various other conditions > (e.g. red alarm or yellow alarm). > > Michele > > > > > On 27 Oct 2015, at 17:19 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > "Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G > TF schema version)." > > That sounds like you would use this in a dfdl:discriminator ? > > Yes, exactly. While this is one of the current use case, in addition we would also use the value to > implement different type of checks e.g. some of the parameters we look at have a type, > a range outside of which they are invalid and a set of other ranges that correspond to various other conditions > (e.g. red alarm or yellow alarm). > > Michele > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > mob:+44-7717-378890 <tel:%2B44-7717-378890> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 27/10/2015 16:09 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Dear Steve, > > to progress on this action: > > 1) it is our intention to “evolve” from our own dmx:assert to the dfdl:assert in order to standardise as much as possible > with the official spec provided we find a solution for what below. > > 2) we had an iteration with our developers and came to the conclusion that there is > need for both syntaxes: the existing (in the standard) dfdl:checkConstraints()and the new > one you proposed i.e. dfdl:checkRangeand dfdl:checkValue > > The main semantic difference between checkValue/checkRangeand checkConstraints > is that in one case we we need to associate a range to a quantity in a sort of static way > (i.e. the type “knowns” it has a range associated with it) while the checkvalue/checkrange > is something that you can do also if a range is not defined (see below). > > For example you could have a definition for a specification sheet of an electrical device. > The XSD type (Volts) intrinsically defined with a range 0-1000 but wanted > to check in derived schemas if value is between 100 and 150 or 200 and 250 and this information > would be used in different way. > > This could be used to check 0-1000 hard constraints to see if the the network will support it > while range 100-150 and 200-250 would check compatibility with US or European Voltages for commercial appliances. > > Note that dfdl:assert applies a condition to the parsing so that if the check fails the parsing is aborted with a severe error. > In the example above we would want to apply checkConstraint defined with a range 0-1000 in a dfdl:assert (to stop the parsing due to incorrect values) but not a checkvalue/checkrange. > Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G > TF schema version). > > Michele > > > > On 05 Oct 2015, at 16:23 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Hi Michele > > Any update from your discussion? > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 23/09/2015 16:44 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Dear Steve, > > thanks for the suggestion (same from Mike) we need to discuss this internally with the developers and look at few use cases > to see what would be the consequences/advantages/disadvantages. > > Give us a little bit of time.. > > Michele > > On 22 Sep 2015, at 19:12 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Hi Michele > > Thanks for your quick response. The WG discussed this on the call today. > > In fact, we wondered if the same result could already be achieved using dfdl:checkConstraints() and declaring the enums or range using XSD facets on xs:simpleType restrictions. > > Have you considered whether dfdl:checkConstraints() achieves what you want? > > Example: > > <xs:simpleType name="myRange"> > <xs:restriction base="xs:int> > <xs:minInclusive value="100"/> > <xs:maxInclusive value="200"/> > </xs:restriction> > </xs:simpleType> > > <xs:element name="myValue" type="myRange"> > <xs:annotation><xs:appinfo ...> > <dfdl:assert>dfdl:checkConstraints(.)</dfdl:assert> > </xs:appinfo></xs:annotation> > </xs:element> > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > Cc: Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 22/09/2015 17:21 > Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > Sent by: dfdl-wg-bounces(a)ogf.org <mailto:dfdl-wg-bounces@ogf.org> > > > > Dear Steve, > > nice to hear things are moving. > > The syntax below seems both reasonable and readable to me which is a good thing. > > It addresses the belonging to decimal groups (integer only ??) I understand. (xs:decimal) > > 1) It might be also useful (not in our application but in general) to see if it makes sense also to define > for dfdl:checkValues belongings to set of “enum” like value > > e.g. if we have a type for $val defined as (ON, OFF or STANDBY) or (MARRIED, SINGLE, WIDOWER) > can we also define a syntax that the $node belongs to it ? > > 2) for dfdl:checkRange it might make sense also to allow float numbers e.g. X between 3.1 to 9.5 > > > Michele > > > > PS I will forward this proposal to our DFDL4S developers (in copy) to get their thinking. > > > From: "Steve Hanson" <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> > Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > Date: 22 Sep 2015 16:50:13 CEST > To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>> > > > To kick start this action, here is a proposal ... builds on the precedent provided by dfdl:checkConstraints($node). > dfdl:checkValues($node, $val1, $val2, ...) Returns boolean true if the specified node value matches any of the values provided by $val1 etc. > The type of $val1 etc must be compatible with the type of $node. > > It is a schema definition error if the $node argument is a complex element. > > The number of value arguments is implementation-defined. > > dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal. > > It is a schema definition error if the $node argument is a complex element. > > dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal. > > It is a schema definition error if the $node argument is a complex element. > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Steve Hanson/UK/IBM > To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>> > Date: 11/08/2015 16:28 > Subject: Fw: [DFDL]: First Release of DFDL4S Parser Library > > > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > ----- Forwarded by Steve Hanson/UK/IBM on 11/08/2015 16:27 ----- > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> > Cc: Steve Hanson/UK/IBM@IBMGB, Maurizio De Bartolomei <Maurizio.De.Bartolomei(a)esa.int <mailto:Maurizio.De.Bartolomei@esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, "Rui Mestre (DME)" <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 18/05/2015 08:47 > Subject: Re: [DFDL]: First Release of DFDL4S Parser Library > > > > Thanks Mike, > > we will add this to our list to be considered/noted. > > However reading your explanation (NB I’m NOT at all an XPath expert) it seemed you > had some good reason for avoiding longer than 1 path, so I would like to avoid our DFDL4S > project results in an over-complication of the DFDL implementation/use of Xpath > unless there are other reasons/users/rationale requiring this feature. > (btw the syntax is still weird-ish: “intersect” reminds me of Venn Diagrams…) > > As a project manager I always evaluate solutions and their cost vs the benefit they bring, > and I believe the DFDL community should keep this is mind. > > Michele > > PS The syntax above anser to the question “belongs to” , would there be any way to express ranges of values then ? > > > > On 15 May 2015, at 16:24 , Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> wrote: > > Just a few comments on DFDL4S, and also thank you to Michele and team for the presentation on Tuesday. > > I think all the issues in the spreadsheet are fairly easily fixed in that they are not major changes to DFDL4S, and would bring it into much closer compliance with the DFDL spec. > > The exception is the XPath limitations where DFDL4S has gone beyond what XPath 2.0 allows and invented new syntax for expressing set membership requirements. > > So I took a look, and XPath 2.0 has a set intersect operator: ns1 intersect ns2 => ns3 > > This isn't in DFDL today, but might be usable to achieve the set membership test; however, it requires use of XPath node sequences of length greater than 1, which DFDL has avoided mostly. I say mostly as there are XPath expressions that return node sequences of length greater than 1 and those can be arguments to fn:count(...) for example. > > So far in DFDL such node sequences cannot "leak out" of the XPath expression into DFDL elements, and I think the usage in DFDL4S is similar in that these node sequences would be needed only to check for set membership, so the result is just a boolean as part of an assert/discriminator. > > We should examine whether XPath 2.0 set intersection is enough to meet the need. > > I believe the expressions would be something like: > > fn:exists( . intersect (123, 456, 789, .... many more items....) ) > > > - mike beckerle > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> > > > On Tue, May 12, 2015 at 10:45 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote: > for reference, > here a summary of the reported problem in an excel sheet. > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > #### Sentinel2X-bandTMISPTypes.xsd moved to MyAttachments Repository V3.8 (Link <notes:///802575AF0030E827/5DE5236E5AD1645685256EE0001BBADF/784AE1CEA59C046380257EAB0027C2EF>) on 24 August 2015 by Steve Hanson. > > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU ----------------------------------------- Michele Zundo Head of Ground System Definition and Verification Office EOP-PEP European Space Agency, ESTEC e-mail: michele.zundo(a)esa.int This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email.

1 0

clarification: textBooleanTrue/FalseRep cannot be empty string? textStandardNaNRep and friends cannot be empty string.
by Mike Beckerle 19 May '16

19 May '16

We don't stipulate that textBooleanTrueRep nor textBooleanFalseRep cannot be empty string. We do stipulate that entity ES is not allowed. So I expect these properties cannot be "" meaning the same thing that "%ES;" would mean. Since we disallow the latter, we should disallow just "". Similarly, for textStandardGroupingSeparator, textStandardExponentRep, textStandardNaNRep, and textStandardInfinityRep we specify that the string literal value behaves as specified in textStandardDecimalSeparator, but we don't specify that these cannot be empty string (which is specified in textStandardDecimalSeparator, but not in the "Text Number Character Restrictions" part of that property description that is referenced from these other properties.) ES and WSP* are not allowed, so almost certainly empty string should not be allowed. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

clarifications needed?: dfdl:contentLength function and dfdl:valueLength function on empty and literal nil representations, and escaping
by Mike Beckerle 19 May '16

19 May '16

The dfdl:contentLength function is defined in terms of the SimpleContent or ComplexContent regions of the grammar. Let's just look at Simple types for a sec. We do not specify what the dfdl:contentLength is for an element of SimpleType which has the SimpleLiteralNilElementRep or SimpleEmptyElementRep. I suggest the value should be zero for SimpleEmptyElementRep. When parsing, an empty element by definition has no content. The fact that a default value might be inserted because of the empty representation should not change the fact that there was no content. When unparsing, SimpleEmptyElementRep can occur if an empty string is the value of a string-valued element, or an empty byte array is the value of a hexBinary element. The grammar is just stipulating the different treatment of initiator/terminator for these special cases of empty things. The content is length zero. But consider the round-trip scenario. We parse data to the infoset. During parsing the dfdl:contentLength of an element having SimpleEmptyElementRep is zero. A default value is inserted. Now we unparse this same infoset. The default value's representation very well may be SimpleNormalRep, with non-zero dfdl:contentLength. I claim this is ok. This is just another case where some data formats don't round trip unchanged. It does add an implementation headache, which is if the contentLength is cached on the infoset item, you need separate cache locations to be used when parsing and when unparsing. For SimpleLiteralNilElementRep, it should be the length of the NilLiteralCharacters or NilElementLiteralContent regions. (Note: there's the word "Content" implying that we think of the nil literal representation as content. ) This applies to both parsing and unparsing. For elements of complex type, I think for both ComplexLiteralNilElementRep and ComplexEmptyElementRep, the dfdl:contentLength should be zero when parsing. When unparsing, again a complex default may be created (because default values for interior elements of the complex type might be filled in as part of the augmented infoset.) and the dfdl:contentLength might not be zero if these default values have non-zero content length. Again I think this is ok. For dfdl:contentLength, we should clarify that the length should also include the contributions of any escape characters, escape-escape characters, and escapeBlockStart/End characters. (This is implied, because such characters are in the "value" regions of simple types, and value regions are always contained in the content region, but I think the clarification is still helpful. Similarly we need to clarify what dfdl:valueLength does. For SimpleEmptyElementRep the dfdl:valueLength should be zero. For SimpleLiteralNilElementRep, the dfdl:valueLength should be zero, because a nilled element has no value. The corner case of SimpleLiteralNilElementRep for a nillable simple element of type xs:string - since a literal nil representation and a string value are ambiguous, should be handled by calling dfdl:contentLength instead of dfdl:valueLength. So a nillable string element with literal nil nilValue="nil", should have dfdl:valueLength of zero, but dfdl:contentLength (in characters) of 3. Same element but not nilled, containing the string "nil" as its value, would have dfdl:valueLength of 3 (characters), and dfdl:contentLength of 3 (characters). For complex type elements, dfdl:valueLength is already defined to be the same as dfdl:contentLength. For elements that are not represented (that is, elements that have the dfdl:inputValueCalc property on them), I believe both dfdl:valueLength and dfdl:contentLength should cause an SDE, as this has to be an error on the part of the schema author. (An argument can be made that these should return zero however. See next paragraph.) Note however that these functions can be called on elements of complex type that contain elements that are not represented. Such contained non-represented elements contribute zero to the content length in all cases. (Consistency with this is why calling dfdl:valueLength or dfdl:contentLength directly on a non-represented element might want to return zero, instead of SDE.) dfdl:valueLength is already specified to exclude the length of padding characters that are trimmed/added. I believe we should explicitly state that it *includes* the length of escape, escape-escape, and escapeBlockStart/End characters. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

Re: [DFDL-WG] Action 282
by Michele Zundo 12 May '16

12 May '16

Sear Steve, I will check with the team and come back to you. > On 11 May 2016, at 9:55 , Steve Hanson <smh(a)uk.ibm.com> wrote: > > Michele & team > > Please join the DFDL WG call on 24th May so we can review the proposal and hopefully close this action. > > Thanks > > Steve Hanson > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > > > > From: Michele Zundo <michele.zundo(a)esa.int> > To: Steve Hanson/UK/IBM@IBMGB > Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int>, Montserrat Piñol <mpinol(a)eopp.esa.int>, Rui Mestre <rui.mestre(a)deimos.com.pt> > Date: 12/04/2016 16:43 > Subject: Re: [DFDL-WG] Action 282 > > > > Steve, I see this only now, > > we will not have time this nor the coming week, as I want to discuss the proposal internally. > > We will review the proposal below and come back to you. > > Michele > > > > > > On 11 Apr 2016, at 10:47 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > 282 > Does XPath have operators for checking a value is in a range? (Steve) > 12/5: Investigate whether equivalent to DFDL4S 'in' operator exists. > 23/6: Mike has found an XPath 'intersect' operator. Handles the enumeration case well, but not as convenient for ranges as DFDL4S's 'in' operator. > 11/8: Looked back at the motivating example from DFDL4S. Agreed that DFDL functions to do the equivalent of 'in' and 'inrange' would be useful if nothing can be re-used from XPath. Steve to write up a proposal taking into account different data types. > ... > 22/9: Proposal sent by Steve for new functions dfdl:checkValues(), dfdl:checkRangeInclusive(), dfdl:checkRangeExclusive(). Discussed whether both range functions needed, and whether they should be allowed for float and double. Mike noted that dfdl:checkConstraints() and simple types could be used instead of all three functions if values were static. Follow-up with DFDL4S team to see if they had thought of that. > 3/11: DFDL4S not happy with usability of dfdl:checkConstraints type, but ok with the intersection operator. Next step is to see what the DFDL4S schema would look like if rewritten to use dfdl:checkConstraints and intersection. > 5/1/16: Agreed that dfdl:checkConstraints() is not ideal as it requires creation of union simple types. Steve to go back to his proposal from 22/9/15 and rework to include use of the intersect operator. > 16/2: No progress > 1/3: Discussed with ESA team. Steve to rework his proposal from 22/9/15. > > Reworked proposal: > > a) Add XPath 2.0 'intersect' and "except" operators to the list of supported operators. > > Updated Table 57 as follows: > DFDL Expression ::= "{" Expr "}" > Expr ::= ExprSingle > ExprSingle ::= IfExpr > | OrExpr > IfExpr ::= "if" "(" Expr ")" "then" ExprSingle "else" ExprSingle > OrExpr ::= AndExpr ( "or" AndExpr )* > AndExpr ::= ComparisonExpr ( "and" ComparisonExpr )* > ComparisonExpr ::= AdditiveExpr ( (ValueComp) AdditiveExpr)? > AdditiveExpr ::= MultiplicativeExpr ( ("+" | "-") MultiplicativeExpr )* > MultiplicativeExpr ::= IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ( ("*" | "div" | "idiv" | "mod") IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> )* > IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ::= UnaryExpr <https://www.w3.org/TR/xpath20/#doc-xpath-InstanceofExpr> ( ("intersect" | "except") UnaryExpr )* > UnaryExpr ::= ("-" | "+")* ValueExpr > ValueExpr ::= PathExpr > ValueComp ::= "eq" | "ne" | "lt" | "le" | "gt" | "ge" > PathExpr ::= ("/" RelativePathExpr?) > | RelativePathExpr | FilterExpr > RelativePathExpr ::= StepExpr (("/") StepExpr)* > StepExpr ::= AxisStep > AxisStep ::= (ReverseStep | ForwardStep) Predicate? > ForwardStep ::= (ForwardAxis NodeTest) | AbbrevForwardStep > ForwardAxis ::= ("child" "::") > | ("self" "::") > AbbrevForwardStep ::= NodeTest | ContextItemExpr > ReverseStep ::= (ReverseAxis NodeTest) | AbbrevReverseStep > ReverseAxis ::= ("parent" "::") > AbbrevReverseStep ::= ".." > NodeTest ::= NameTest > NameTest ::= QName > FilterExpr ::= PrimaryExpr Predicate? > Predicate ::= "[" Expr "]" > PrimaryExpr ::= Literal | VarRef | ParenthesizedExpr | ContextItemExpr | FunctionCall > Literal ::= NumericLiteral | StringLiteral > NumericLiteral ::= IntegerLiteral | DecimalLiteral | DoubleLiteral > VarRef ::= "$" VarName > VarName ::= QName > ParenthesizedExpr ::= "(" Expr ")" > ContextItemExpr ::= "." > FunctionCall ::= QName "(" (ExprSingle ("," ExprSingle)*)? ")" > > > > > b) New DFDL functions > > dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double. > > It is a schema definition error if the $node argument is a complex element. > > dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double. > > It is a schema definition error if the $node argument is a complex element. > > > > > Regards > > Steve Hanson > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:+44-1962-815848> > mob:+44-7717-378890 > > > > From: Steve Hanson/UK/IBM > To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > Cc: "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 02/11/2015 19:51 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > > <xs:simpleType name="redAndYellowAlertType"> > <xs:union memberTypes="redAlertType yellowAlertType"/> > </xs:simpleType> > > The above is a type that allows ranges 0-200 and 201-500. > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:+44-1962-815848> > mob:+44-7717-378890 > > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 02/11/2015 16:08 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Steve, > > I believe I do not understand this. > > can you make an example ? > > Michele > > On 02 Nov 2015, at 16:33 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > I was also going to add that DFDL supports xs:union on simple types, enabling dfdl:checkConstraints to be used when there are multiple ranges. > > · Unions; the memberTypes must be derived from the same simple type. DFDL annotations are not permitted on union members. > > The purpose of unions is to allow multiple constraints via facets such as multiple independent range restrictions on numbers. This enhances the ability to do rich validation of data. > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:+44-1962-815848> > mob:+44-7717-378890 > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> > To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > Cc: Steve Hanson/UK/IBM@IBMGB, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 02/11/2015 14:33 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > > Michele, > > I think the first thing I'd propose for the set membership test is for us to add the XPath 2.0 intersect operator to DFDL. So your discriminator becomes: > > <dfdl:discriminatortest="{fn:exists(/Packet_Primary_Header/Packet_Identification/APID intersect (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, > 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, > 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, > 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, > 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, > 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 256,257,258,259,260,261,262,263,264,265,266,267,268, > 272,273,274,275,276,277,278,279,280,281,282,283,284, > 288,289,290,291,292,293,294,295,296,297,298,299,300, > 304,305,306,307,308,309,310,311,312,313,314,315,316, > 320,321,322,323,324,325,326,327,328,329,330,331,332, > 336,337,338,339,340,341,342,343,344,345,346,347,348))}"/> > > > This is a small addition to DFDL, but completely in the spirit of sticking with XPath 2.0 wherever possible. > > ...mike beckerle > > > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> > > > On Wed, Oct 28, 2015 at 7:22 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote: > Hi Steve, > > your example below addresses the belonging to a range, how would then the > belonging to a set be checked ? Currently our usage looks like here: > > > > <xs:complexTypename="TypePacketData"> > <xs:sequence> > <xs:choice> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, > 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, > 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, > 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, > 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, > 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 256,257,258,259,260,261,262,263,264,265,266,267,268, > 272,273,274,275,276,277,278,279,280,281,282,283,284, > 288,289,290,291,292,293,294,295,296,297,298,299,300, > 304,305,306,307,308,309,310,311,312,313,314,315,316, > 320,321,322,323,324,325,326,327,328,329,330,331,332, > 336,337,338,339,340,341,342,343,344,345,346,347,348]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="MSI_Packet_Secondary_Header"type="TypePacketData_MSI"/> > <xs:element name="MSI_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [769,771,772,774,775,777,779,780,781,785,787,788,790,791,793,795,796,797]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_GPSR_Packet_Secondary_Header"type="TypePacketData_TMGPSR"/> > <xs:element name="TM_GPSR_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID in [99999]}" /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_Time_Packet_Secondary_Header"type="TypePacketData_TMT92"/> > <xs:element name="TM_Time_Packet_User_Data_Field"type="TypeUserData_TMT92"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [596,598,612,614,628,630,660,662]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_STR_Packet_Secondary_Header"type="TypePacketData_TMSTR"/> > <xs:element name="TM_STR_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [150,182]}" /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_CSW_AOCS_Packet_Secondary_Header"type="TypePacketData_TMCSWAOCS"/> > <xs:element name="TM_CSW_AOCS_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID inrange [144,149] > or /Packet_Primary_Header/Packet_Identification/APID inrange [151,181] > or /Packet_Primary_Header/Packet_Identification/APID inrange [183,239] > or /Packet_Primary_Header/Packet_Identification/APID inrange [516,575] > or /Packet_Primary_Header/Packet_Identification/APID in [592,593,594,595,597,613,629,656,657,658,659,661] > or /Packet_Primary_Header/Packet_Identification/APID inrange [599,611] > or /Packet_Primary_Header/Packet_Identification/APID inrange [615,627] > or /Packet_Primary_Header/Packet_Identification/APID inrange [631,639] > or /Packet_Primary_Header/Packet_Identification/APID inrange [663,671] > or /Packet_Primary_Header/Packet_Identification/APID in [768,770,776,778,782,783,784,792,794,798,799] > or /Packet_Primary_Header/Packet_Identification/APID inrange [1280,1295]}" > /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_HKTM_Packet_Secondary_Header"type="TypePacketData_HKTM"/> > <xs:element name="TM_HKTM_User_Data_Field"type="TypeUserData_HKTM"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID in [2047]}" > /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_IDLE_Packet_Secondary_Header"type="TypePacketData_TMIDLE" > /> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{true}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="UNKNOWN_User_Data_Field"type="TypePacketData_UNKNOWN"/> > </xs:sequence> > </xs:choice> > </xs:sequence> > </xs:complexType> > > > On 27 Oct 2015, at 18:01 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Just to clarify, dfdl:assert and dfdl:discriminator are identical when the expression returns false, the difference between the two is when the expression returns true and the parser is in a 'point of uncertainty' (eg choice) - a discriminator returning true has a positive discrimination on the point of uncertainty, an assert does not. > > I suspect that most of the time you can achieve what you want with just dfdl:checkConstraints() and simple type inheritance. Example below (syntax simplified for clarity). But I am happy to be proved wrong, so if you have an example ... ? > > <xs:simpleType name="alertType"> > <xs:restriction base="xs:unsignedInt> > <xs:minInclusive value="0"/> > <xs:maxInclusive value="999"/> > </xs:restriction> > </xs:simpleType> > > <xs:simpleType name="redAlertType"> > <xs:restriction base="alertType"> > <xs:minInclusive value="0"/> > <xs:maxInclusive value="200"/> > </xs:restriction> > </xs:simpleType> > > <xs:simpleType name="yellowAlertType"> > <xs:restriction base="alertType"> > <xs:minInclusive value="201"/> > <xs:maxInclusive value="500"/> > </xs:restriction> > </xs:simpleType> > > ... > > <xs:choice> > <xs:element name="red"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="redAlertType"> > <dfdl:discriminator test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > <xs:element name="yellow"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="yellowAlertType"> > <dfdl:discriminator test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > <xs:element name="other"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="alertType"> > <dfdl:assert test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > mob:+44-7717-378890 <tel:%2B44-7717-378890> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 27/10/2015 16:27 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Yes, exactly. This is one of the current use case, in addition we could also use the value to > implement different type of checks e.g. some of the parameters we look at have a type, > a range outside of which they are invalid and a set of other ranges that correspond to various other conditions > (e.g. red alarm or yellow alarm). > > Michele > > > > > On 27 Oct 2015, at 17:19 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > "Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G > TF schema version)." > > That sounds like you would use this in a dfdl:discriminator ? > > Yes, exactly. While this is one of the current use case, in addition we would also use the value to > implement different type of checks e.g. some of the parameters we look at have a type, > a range outside of which they are invalid and a set of other ranges that correspond to various other conditions > (e.g. red alarm or yellow alarm). > > Michele > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > mob:+44-7717-378890 <tel:%2B44-7717-378890> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 27/10/2015 16:09 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Dear Steve, > > to progress on this action: > > 1) it is our intention to “evolve” from our own dmx:assert to the dfdl:assert in order to standardise as much as possible > with the official spec provided we find a solution for what below. > > 2) we had an iteration with our developers and came to the conclusion that there is > need for both syntaxes: the existing (in the standard) dfdl:checkConstraints()and the new > one you proposed i.e. dfdl:checkRangeand dfdl:checkValue > > The main semantic difference between checkValue/checkRangeand checkConstraints > is that in one case we we need to associate a range to a quantity in a sort of static way > (i.e. the type “knowns” it has a range associated with it) while the checkvalue/checkrange > is something that you can do also if a range is not defined (see below). > > For example you could have a definition for a specification sheet of an electrical device. > The XSD type (Volts) intrinsically defined with a range 0-1000 but wanted > to check in derived schemas if value is between 100 and 150 or 200 and 250 and this information > would be used in different way. > > This could be used to check 0-1000 hard constraints to see if the the network will support it > while range 100-150 and 200-250 would check compatibility with US or European Voltages for commercial appliances. > > Note that dfdl:assert applies a condition to the parsing so that if the check fails the parsing is aborted with a severe error. > In the example above we would want to apply checkConstraint defined with a range 0-1000 in a dfdl:assert (to stop the parsing due to incorrect values) but not a checkvalue/checkrange. > Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G > TF schema version). > > Michele > > > > On 05 Oct 2015, at 16:23 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Hi Michele > > Any update from your discussion? > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 23/09/2015 16:44 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Dear Steve, > > thanks for the suggestion (same from Mike) we need to discuss this internally with the developers and look at few use cases > to see what would be the consequences/advantages/disadvantages. > > Give us a little bit of time.. > > Michele > > On 22 Sep 2015, at 19:12 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Hi Michele > > Thanks for your quick response. The WG discussed this on the call today. > > In fact, we wondered if the same result could already be achieved using dfdl:checkConstraints() and declaring the enums or range using XSD facets on xs:simpleType restrictions. > > Have you considered whether dfdl:checkConstraints() achieves what you want? > > Example: > > <xs:simpleType name="myRange"> > <xs:restriction base="xs:int> > <xs:minInclusive value="100"/> > <xs:maxInclusive value="200"/> > </xs:restriction> > </xs:simpleType> > > <xs:element name="myValue" type="myRange"> > <xs:annotation><xs:appinfo ...> > <dfdl:assert>dfdl:checkConstraints(.)</dfdl:assert> > </xs:appinfo></xs:annotation> > </xs:element> > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > Cc: Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 22/09/2015 17:21 > Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > Sent by: dfdl-wg-bounces(a)ogf.org <mailto:dfdl-wg-bounces@ogf.org> > > > > Dear Steve, > > nice to hear things are moving. > > The syntax below seems both reasonable and readable to me which is a good thing. > > It addresses the belonging to decimal groups (integer only ??) I understand. (xs:decimal) > > 1) It might be also useful (not in our application but in general) to see if it makes sense also to define > for dfdl:checkValues belongings to set of “enum” like value > > e.g. if we have a type for $val defined as (ON, OFF or STANDBY) or (MARRIED, SINGLE, WIDOWER) > can we also define a syntax that the $node belongs to it ? > > 2) for dfdl:checkRange it might make sense also to allow float numbers e.g. X between 3.1 to 9.5 > > > Michele > > > > PS I will forward this proposal to our DFDL4S developers (in copy) to get their thinking. > > > From: "Steve Hanson" <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> > Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > Date: 22 Sep 2015 16:50:13 CEST > To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>> > > > To kick start this action, here is a proposal ... builds on the precedent provided by dfdl:checkConstraints($node). > dfdl:checkValues($node, $val1, $val2, ...) Returns boolean true if the specified node value matches any of the values provided by $val1 etc. > The type of $val1 etc must be compatible with the type of $node. > > It is a schema definition error if the $node argument is a complex element. > > The number of value arguments is implementation-defined. > > dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal. > > It is a schema definition error if the $node argument is a complex element. > > dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal. > > It is a schema definition error if the $node argument is a complex element. > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Steve Hanson/UK/IBM > To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>> > Date: 11/08/2015 16:28 > Subject: Fw: [DFDL]: First Release of DFDL4S Parser Library > > > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > ----- Forwarded by Steve Hanson/UK/IBM on 11/08/2015 16:27 ----- > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> > Cc: Steve Hanson/UK/IBM@IBMGB, Maurizio De Bartolomei <Maurizio.De.Bartolomei(a)esa.int <mailto:Maurizio.De.Bartolomei@esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, "Rui Mestre (DME)" <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 18/05/2015 08:47 > Subject: Re: [DFDL]: First Release of DFDL4S Parser Library > > > > Thanks Mike, > > we will add this to our list to be considered/noted. > > However reading your explanation (NB I’m NOT at all an XPath expert) it seemed you > had some good reason for avoiding longer than 1 path, so I would like to avoid our DFDL4S > project results in an over-complication of the DFDL implementation/use of Xpath > unless there are other reasons/users/rationale requiring this feature. > (btw the syntax is still weird-ish: “intersect” reminds me of Venn Diagrams…) > > As a project manager I always evaluate solutions and their cost vs the benefit they bring, > and I believe the DFDL community should keep this is mind. > > Michele > > PS The syntax above anser to the question “belongs to” , would there be any way to express ranges of values then ? > > > > On 15 May 2015, at 16:24 , Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> wrote: > > Just a few comments on DFDL4S, and also thank you to Michele and team for the presentation on Tuesday. > > I think all the issues in the spreadsheet are fairly easily fixed in that they are not major changes to DFDL4S, and would bring it into much closer compliance with the DFDL spec. > > The exception is the XPath limitations where DFDL4S has gone beyond what XPath 2.0 allows and invented new syntax for expressing set membership requirements. > > So I took a look, and XPath 2.0 has a set intersect operator: ns1 intersect ns2 => ns3 > > This isn't in DFDL today, but might be usable to achieve the set membership test; however, it requires use of XPath node sequences of length greater than 1, which DFDL has avoided mostly. I say mostly as there are XPath expressions that return node sequences of length greater than 1 and those can be arguments to fn:count(...) for example. > > So far in DFDL such node sequences cannot "leak out" of the XPath expression into DFDL elements, and I think the usage in DFDL4S is similar in that these node sequences would be needed only to check for set membership, so the result is just a boolean as part of an assert/discriminator. > > We should examine whether XPath 2.0 set intersection is enough to meet the need. > > I believe the expressions would be something like: > > fn:exists( . intersect (123, 456, 789, .... many more items....) ) > > > - mike beckerle > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> > > > On Tue, May 12, 2015 at 10:45 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote: > for reference, > here a summary of the reported problem in an excel sheet. > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > #### Sentinel2X-bandTMISPTypes.xsd moved to MyAttachments Repository V3.8 (Link <notes:///802575AF0030E827/5DE5236E5AD1645685256EE0001BBADF/784AE1CEA59C046380257EAB0027C2EF>) on 24 August 2015 by Steve Hanson. > > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU ----------------------------------------- Michele Zundo Head of Ground System Definition and Verification Office EOP-PEP European Space Agency, ESTEC e-mail: michele.zundo(a)esa.int This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email.

1 0

Re: [DFDL-WG] US Gov Open-Data Initiative
by Steve Hanson 11 May '16

11 May '16

Forgot about this for yesterday's call, will add to next call. Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 18/04/2016 08:43 Subject: Re: [DFDL-WG] US Gov Open-Data Initiative Hi Mike I had a quick read of the article at the weekend. I'll add to the agenda for the next WG call. Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 15/04/2016 14:19 Subject: [DFDL-WG] US Gov Open-Data Initiative Sent by: "dfdl-wg" <dfdl-wg-bounces(a)ogf.org> See https://fcw.com/articles/2016/04/14/open-data-bill.aspx?s=fcwdaily_150416 Seems like an opportunity for DFDL as a standard to provide "open" access to data without having to endure the costs and headaches of repeated conversion. ...mike Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Action 282
by Michele Zundo 11 May '16

11 May '16

Steve, I see this only now, we will not have time this nor the coming week, as I want to discuss the proposal internally. We will review the proposal below and come back to you. Michele > On 11 Apr 2016, at 10:47 , Steve Hanson <smh(a)uk.ibm.com> wrote: > > 282 > Does XPath have operators for checking a value is in a range? (Steve) > 12/5: Investigate whether equivalent to DFDL4S 'in' operator exists. > 23/6: Mike has found an XPath 'intersect' operator. Handles the enumeration case well, but not as convenient for ranges as DFDL4S's 'in' operator. > 11/8: Looked back at the motivating example from DFDL4S. Agreed that DFDL functions to do the equivalent of 'in' and 'inrange' would be useful if nothing can be re-used from XPath. Steve to write up a proposal taking into account different data types. > ... > 22/9: Proposal sent by Steve for new functions dfdl:checkValues(), dfdl:checkRangeInclusive(), dfdl:checkRangeExclusive(). Discussed whether both range functions needed, and whether they should be allowed for float and double. Mike noted that dfdl:checkConstraints() and simple types could be used instead of all three functions if values were static. Follow-up with DFDL4S team to see if they had thought of that. > 3/11: DFDL4S not happy with usability of dfdl:checkConstraints type, but ok with the intersection operator. Next step is to see what the DFDL4S schema would look like if rewritten to use dfdl:checkConstraints and intersection. > 5/1/16: Agreed that dfdl:checkConstraints() is not ideal as it requires creation of union simple types. Steve to go back to his proposal from 22/9/15 and rework to include use of the intersect operator. > 16/2: No progress > 1/3: Discussed with ESA team. Steve to rework his proposal from 22/9/15. > > Reworked proposal: > > a) Add XPath 2.0 'intersect' and "except" operators to the list of supported operators. > > Updated Table 57 as follows: > DFDL Expression ::= "{" Expr "}" > Expr ::= ExprSingle > ExprSingle ::= IfExpr > | OrExpr > IfExpr ::= "if" "(" Expr ")" "then" ExprSingle "else" ExprSingle > OrExpr ::= AndExpr ( "or" AndExpr )* > AndExpr ::= ComparisonExpr ( "and" ComparisonExpr )* > ComparisonExpr ::= AdditiveExpr ( (ValueComp) AdditiveExpr)? > AdditiveExpr ::= MultiplicativeExpr ( ("+" | "-") MultiplicativeExpr )* > MultiplicativeExpr ::= IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ( ("*" | "div" | "idiv" | "mod") IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> )* > IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ::= UnaryExpr <https://www.w3.org/TR/xpath20/#doc-xpath-InstanceofExpr> ( ("intersect" | "except") UnaryExpr )* > UnaryExpr ::= ("-" | "+")* ValueExpr > ValueExpr ::= PathExpr > ValueComp ::= "eq" | "ne" | "lt" | "le" | "gt" | "ge" > PathExpr ::= ("/" RelativePathExpr?) > | RelativePathExpr | FilterExpr > RelativePathExpr ::= StepExpr (("/") StepExpr)* > StepExpr ::= AxisStep > AxisStep ::= (ReverseStep | ForwardStep) Predicate? > ForwardStep ::= (ForwardAxis NodeTest) | AbbrevForwardStep > ForwardAxis ::= ("child" "::") > | ("self" "::") > AbbrevForwardStep ::= NodeTest | ContextItemExpr > ReverseStep ::= (ReverseAxis NodeTest) | AbbrevReverseStep > ReverseAxis ::= ("parent" "::") > AbbrevReverseStep ::= ".." > NodeTest ::= NameTest > NameTest ::= QName > FilterExpr ::= PrimaryExpr Predicate? > Predicate ::= "[" Expr "]" > PrimaryExpr ::= Literal | VarRef | ParenthesizedExpr | ContextItemExpr | FunctionCall > Literal ::= NumericLiteral | StringLiteral > NumericLiteral ::= IntegerLiteral | DecimalLiteral | DoubleLiteral > VarRef ::= "$" VarName > VarName ::= QName > ParenthesizedExpr ::= "(" Expr ")" > ContextItemExpr ::= "." > FunctionCall ::= QName "(" (ExprSingle ("," ExprSingle)*)? ")" > > > > b) New DFDL functions > dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double. > > It is a schema definition error if the $node argument is a complex element. > > dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double. > > It is a schema definition error if the $node argument is a complex element. > > > > > Regards > > Steve Hanson > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > > > > From: Steve Hanson/UK/IBM > To: Michele Zundo <michele.zundo(a)esa.int> > Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int>, Montserrat Piñol <mpinol(a)eopp.esa.int>, Rui Mestre <rui.mestre(a)deimos.com.pt> > Date: 02/11/2015 19:51 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > <xs:simpleType name="redAndYellowAlertType"> > <xs:union memberTypes="redAlertType yellowAlertType"/> > </xs:simpleType> > > The above is a type that allows ranges 0-200 and 201-500. > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > > > > > From: Michele Zundo <michele.zundo(a)esa.int> > To: Steve Hanson/UK/IBM@IBMGB > Cc: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int>, Montserrat Piñol <mpinol(a)eopp.esa.int>, Rui Mestre <rui.mestre(a)deimos.com.pt> > Date: 02/11/2015 16:08 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Steve, > > I believe I do not understand this. > > can you make an example ? > > Michele > > On 02 Nov 2015, at 16:33 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > I was also going to add that DFDL supports xs:union on simple types, enabling dfdl:checkConstraints to be used when there are multiple ranges. > > · Unions; the memberTypes must be derived from the same simple type. DFDL annotations are not permitted on union members. > > The purpose of unions is to allow multiple constraints via facets such as multiple independent range restrictions on numbers. This enhances the ability to do rich validation of data. > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> > To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > Cc: Steve Hanson/UK/IBM@IBMGB, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 02/11/2015 14:33 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > > Michele, > > I think the first thing I'd propose for the set membership test is for us to add the XPath 2.0 intersect operator to DFDL. So your discriminator becomes: > > <dfdl:discriminatortest="{fn:exists(/Packet_Primary_Header/Packet_Identification/APID intersect (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, > 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, > 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, > 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, > 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, > 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 256,257,258,259,260,261,262,263,264,265,266,267,268, > 272,273,274,275,276,277,278,279,280,281,282,283,284, > 288,289,290,291,292,293,294,295,296,297,298,299,300, > 304,305,306,307,308,309,310,311,312,313,314,315,316, > 320,321,322,323,324,325,326,327,328,329,330,331,332, > 336,337,338,339,340,341,342,343,344,345,346,347,348))}"/> > > > This is a small addition to DFDL, but completely in the spirit of sticking with XPath 2.0 wherever possible. > > ...mike beckerle > > > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> > > > On Wed, Oct 28, 2015 at 7:22 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote: > Hi Steve, > > your example below addresses the belonging to a range, how would then the > belonging to a set be checked ? Currently our usage looks like here: > > > > <xs:complexTypename="TypePacketData"> > <xs:sequence> > <xs:choice> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, > 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, > 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, > 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, > 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, > 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 256,257,258,259,260,261,262,263,264,265,266,267,268, > 272,273,274,275,276,277,278,279,280,281,282,283,284, > 288,289,290,291,292,293,294,295,296,297,298,299,300, > 304,305,306,307,308,309,310,311,312,313,314,315,316, > 320,321,322,323,324,325,326,327,328,329,330,331,332, > 336,337,338,339,340,341,342,343,344,345,346,347,348]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="MSI_Packet_Secondary_Header"type="TypePacketData_MSI"/> > <xs:element name="MSI_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [769,771,772,774,775,777,779,780,781,785,787,788,790,791,793,795,796,797]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_GPSR_Packet_Secondary_Header"type="TypePacketData_TMGPSR"/> > <xs:element name="TM_GPSR_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID in [99999]}" /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_Time_Packet_Secondary_Header"type="TypePacketData_TMT92"/> > <xs:element name="TM_Time_Packet_User_Data_Field"type="TypeUserData_TMT92"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [596,598,612,614,628,630,660,662]}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_STR_Packet_Secondary_Header"type="TypePacketData_TMSTR"/> > <xs:element name="TM_STR_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [150,182]}" /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_CSW_AOCS_Packet_Secondary_Header"type="TypePacketData_TMCSWAOCS"/> > <xs:element name="TM_CSW_AOCS_User_Data_Field"type="TypeUserData"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID inrange [144,149] > or /Packet_Primary_Header/Packet_Identification/APID inrange [151,181] > or /Packet_Primary_Header/Packet_Identification/APID inrange [183,239] > or /Packet_Primary_Header/Packet_Identification/APID inrange [516,575] > or /Packet_Primary_Header/Packet_Identification/APID in [592,593,594,595,597,613,629,656,657,658,659,661] > or /Packet_Primary_Header/Packet_Identification/APID inrange [599,611] > or /Packet_Primary_Header/Packet_Identification/APID inrange [615,627] > or /Packet_Primary_Header/Packet_Identification/APID inrange [631,639] > or /Packet_Primary_Header/Packet_Identification/APID inrange [663,671] > or /Packet_Primary_Header/Packet_Identification/APID in [768,770,776,778,782,783,784,792,794,798,799] > or /Packet_Primary_Header/Packet_Identification/APID inrange [1280,1295]}" > /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_HKTM_Packet_Secondary_Header"type="TypePacketData_HKTM"/> > <xs:element name="TM_HKTM_User_Data_Field"type="TypeUserData_HKTM"/> > </xs:sequence> > <xs:sequence> >  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminator > test="{/Packet_Primary_Header/Packet_Identification/APID in [2047]}" > /> > </xs:appinfo> > </xs:annotation> > <xs:element name="TM_IDLE_Packet_Secondary_Header"type="TypePacketData_TMIDLE" > /> > </xs:sequence> > <xs:sequence>  > <xs:annotation> > <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>"> > <dfdl:discriminatortest="{true}"/> > </xs:appinfo> > </xs:annotation> > <xs:element name="UNKNOWN_User_Data_Field"type="TypePacketData_UNKNOWN"/> > </xs:sequence> > </xs:choice> > </xs:sequence> > </xs:complexType> > > > On 27 Oct 2015, at 18:01 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Just to clarify, dfdl:assert and dfdl:discriminator are identical when the expression returns false, the difference between the two is when the expression returns true and the parser is in a 'point of uncertainty' (eg choice) - a discriminator returning true has a positive discrimination on the point of uncertainty, an assert does not. > > I suspect that most of the time you can achieve what you want with just dfdl:checkConstraints() and simple type inheritance. Example below (syntax simplified for clarity). But I am happy to be proved wrong, so if you have an example ... ? > > <xs:simpleType name="alertType"> > <xs:restriction base="xs:unsignedInt> > <xs:minInclusive value="0"/> > <xs:maxInclusive value="999"/> > </xs:restriction> > </xs:simpleType> > > <xs:simpleType name="redAlertType"> > <xs:restriction base="alertType"> > <xs:minInclusive value="0"/> > <xs:maxInclusive value="200"/> > </xs:restriction> > </xs:simpleType> > > <xs:simpleType name="yellowAlertType"> > <xs:restriction base="alertType"> > <xs:minInclusive value="201"/> > <xs:maxInclusive value="500"/> > </xs:restriction> > </xs:simpleType> > > ... > > <xs:choice> > <xs:element name="red"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="redAlertType"> > <dfdl:discriminator test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > <xs:element name="yellow"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="yellowAlertType"> > <dfdl:discriminator test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > <xs:element name="other"> > <xs:complexType> > <xs:sequence> > <xs:element name="alert" type="alertType"> > <dfdl:assert test="{dfdl:checkConstraints()}"> > </xs:element> > ... > </xs:sequence> > </xs:complexType> > </xs:element> > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > mob:+44-7717-378890 <tel:%2B44-7717-378890> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 27/10/2015 16:27 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Yes, exactly. This is one of the current use case, in addition we could also use the value to > implement different type of checks e.g. some of the parameters we look at have a type, > a range outside of which they are invalid and a set of other ranges that correspond to various other conditions > (e.g. red alarm or yellow alarm). > > Michele > > > > > On 27 Oct 2015, at 17:19 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > "Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G > TF schema version)." > > That sounds like you would use this in a dfdl:discriminator ? > > Yes, exactly. While this is one of the current use case, in addition we would also use the value to > implement different type of checks e.g. some of the parameters we look at have a type, > a range outside of which they are invalid and a set of other ranges that correspond to various other conditions > (e.g. red alarm or yellow alarm). > > Michele > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > mob:+44-7717-378890 <tel:%2B44-7717-378890> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 27/10/2015 16:09 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Dear Steve, > > to progress on this action: > > 1) it is our intention to “evolve” from our own dmx:assert to the dfdl:assert in order to standardise as much as possible > with the official spec provided we find a solution for what below. > > 2) we had an iteration with our developers and came to the conclusion that there is > need for both syntaxes: the existing (in the standard) dfdl:checkConstraints()and the new > one you proposed i.e. dfdl:checkRangeand dfdl:checkValue > > The main semantic difference between checkValue/checkRangeand checkConstraints > is that in one case we we need to associate a range to a quantity in a sort of static way > (i.e. the type “knowns” it has a range associated with it) while the checkvalue/checkrange > is something that you can do also if a range is not defined (see below). > > For example you could have a definition for a specification sheet of an electrical device. > The XSD type (Volts) intrinsically defined with a range 0-1000 but wanted > to check in derived schemas if value is between 100 and 150 or 200 and 250 and this information > would be used in different way. > > This could be used to check 0-1000 hard constraints to see if the the network will support it > while range 100-150 and 200-250 would check compatibility with US or European Voltages for commercial appliances. > > Note that dfdl:assert applies a condition to the parsing so that if the check fails the parsing is aborted with a severe error. > In the example above we would want to apply checkConstraint defined with a range 0-1000 in a dfdl:assert (to stop the parsing due to incorrect values) but not a checkvalue/checkrange. > Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G > TF schema version). > > Michele > > > > On 05 Oct 2015, at 16:23 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Hi Michele > > Any update from your discussion? > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Steve Hanson/UK/IBM@IBMGB > Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 23/09/2015 16:44 > Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > > > > Dear Steve, > > thanks for the suggestion (same from Mike) we need to discuss this internally with the developers and look at few use cases > to see what would be the consequences/advantages/disadvantages. > > Give us a little bit of time.. > > Michele > > On 22 Sep 2015, at 19:12 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote: > > Hi Michele > > Thanks for your quick response. The WG discussed this on the call today. > > In fact, we wondered if the same result could already be achieved using dfdl:checkConstraints() and declaring the enums or range using XSD facets on xs:simpleType restrictions. > > Have you considered whether dfdl:checkConstraints() achieves what you want? > > Example: > > <xs:simpleType name="myRange"> > <xs:restriction base="xs:int> > <xs:minInclusive value="100"/> > <xs:maxInclusive value="200"/> > </xs:restriction> > </xs:simpleType> > > <xs:element name="myValue" type="myRange"> > <xs:annotation><xs:appinfo ...> > <dfdl:assert>dfdl:checkConstraints(.)</dfdl:assert> > </xs:appinfo></xs:annotation> > </xs:element> > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > Cc: Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>> > Date: 22/09/2015 17:21 > Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > Sent by: dfdl-wg-bounces(a)ogf.org <mailto:dfdl-wg-bounces@ogf.org> > > > > Dear Steve, > > nice to hear things are moving. > > The syntax below seems both reasonable and readable to me which is a good thing. > > It addresses the belonging to decimal groups (integer only ??) I understand. (xs:decimal) > > 1) It might be also useful (not in our application but in general) to see if it makes sense also to define > for dfdl:checkValues belongings to set of “enum” like value > > e.g. if we have a type for $val defined as (ON, OFF or STANDBY) or (MARRIED, SINGLE, WIDOWER) > can we also define a syntax that the $node belongs to it ? > > 2) for dfdl:checkRange it might make sense also to allow float numbers e.g. X between 3.1 to 9.5 > > > Michele > > > > PS I will forward this proposal to our DFDL4S developers (in copy) to get their thinking. > > > From: "Steve Hanson" <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> > Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library) > Date: 22 Sep 2015 16:50:13 CEST > To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>> > > > To kick start this action, here is a proposal ... builds on the precedent provided by dfdl:checkConstraints($node). > dfdl:checkValues($node, $val1, $val2, ...) Returns boolean true if the specified node value matches any of the values provided by $val1 etc. > The type of $val1 etc must be compatible with the type of $node. > > It is a schema definition error if the $node argument is a complex element. > > The number of value arguments is implementation-defined. > > dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal. > > It is a schema definition error if the $node argument is a complex element. > > dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive. > The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal. > > It is a schema definition error if the $node argument is a complex element. > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > > > > From: Steve Hanson/UK/IBM > To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>> > Date: 11/08/2015 16:28 > Subject: Fw: [DFDL]: First Release of DFDL4S Parser Library > > > > > > > > > Regards > > Steve Hanson > Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK > smh(a)uk.ibm.com <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 <tel:%2B44-1962-815848> > ----- Forwarded by Steve Hanson/UK/IBM on 11/08/2015 16:27 ----- > > From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> > To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> > Cc: Steve Hanson/UK/IBM@IBMGB, Maurizio De Bartolomei <Maurizio.De.Bartolomei(a)esa.int <mailto:Maurizio.De.Bartolomei@esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, "Rui Mestre (DME)" <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>> > Date: 18/05/2015 08:47 > Subject: Re: [DFDL]: First Release of DFDL4S Parser Library > > > > Thanks Mike, > > we will add this to our list to be considered/noted. > > However reading your explanation (NB I’m NOT at all an XPath expert) it seemed you > had some good reason for avoiding longer than 1 path, so I would like to avoid our DFDL4S > project results in an over-complication of the DFDL implementation/use of Xpath > unless there are other reasons/users/rationale requiring this feature. > (btw the syntax is still weird-ish: “intersect” reminds me of Venn Diagrams…) > > As a project manager I always evaluate solutions and their cost vs the benefit they bring, > and I believe the DFDL community should keep this is mind. > > Michele > > PS The syntax above anser to the question “belongs to” , would there be any way to express ranges of values then ? > > > > On 15 May 2015, at 16:24 , Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> wrote: > > Just a few comments on DFDL4S, and also thank you to Michele and team for the presentation on Tuesday. > > I think all the issues in the spreadsheet are fairly easily fixed in that they are not major changes to DFDL4S, and would bring it into much closer compliance with the DFDL spec. > > The exception is the XPath limitations where DFDL4S has gone beyond what XPath 2.0 allows and invented new syntax for expressing set membership requirements. > > So I took a look, and XPath 2.0 has a set intersect operator: ns1 intersect ns2 => ns3 > > This isn't in DFDL today, but might be usable to achieve the set membership test; however, it requires use of XPath node sequences of length greater than 1, which DFDL has avoided mostly. I say mostly as there are XPath expressions that return node sequences of length greater than 1 and those can be arguments to fn:count(...) for example. > > So far in DFDL such node sequences cannot "leak out" of the XPath expression into DFDL elements, and I think the usage in DFDL4S is similar in that these node sequences would be needed only to check for set membership, so the result is just a boolean as part of an assert/discriminator. > > We should examine whether XPath 2.0 set intersection is enough to meet the need. > > I believe the expressions would be something like: > > fn:exists( . intersect (123, 456, 789, .... many more items....) ) > > > - mike beckerle > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> > > > On Tue, May 12, 2015 at 10:45 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote: > for reference, > here a summary of the reported problem in an excel sheet. > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > #### Sentinel2X-bandTMISPTypes.xsd moved to MyAttachments Repository V3.8 (Link <notes:///802575AF0030E827/5DE5236E5AD1645685256EE0001BBADF/784AE1CEA59C046380257EAB0027C2EF>) on 24 August 2015 by Steve Hanson. > > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org> > https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg> > > > > ----------------------------------------- > Michele Zundo > > Head of Ground System Definition and Verification Office > EOP-PEP > European Space Agency, ESTEC > e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int> > > > > > > > > > This message and any attachments are intended for the use of the addressee or addressees only. > The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its > content is not permitted. > If you received this message in error, please notify the sender and delete it from your system. > Emails can be altered and their integrity cannot be guaranteed by the sender. > > Please consider the environment before printing this email. > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU ----------------------------------------- Michele Zundo Head of Ground System Definition and Verification Office EOP-PEP European Space Agency, ESTEC e-mail: michele.zundo(a)esa.int This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email.

2 1

Clarification for nil processing and zero length
by Steve Hanson 11 May '16

11 May '16

The DFDL 1.0 spec current says: 9.3.2.1 Simple element If the result is length zero as described above, the representation is then established by checking, in order for: 1. nil representation (if %ES; is a literal nil value). 2. empty representation. 3. normal representation (xs:string or xs:hexBinary only) 4. absent representation (if none of the prior representations apply). But should bullet 1 be: 1. nil representation (if either %ES; or %WSP*; on its own is a literal nil value). I added a test to IBM DFDL and found that setting dfdl:nilKind="literalValue" & dfdl:nilValue="%WSP*;" did not match an element value of empty string. That surprised me, and I think the IBM DFDL code is strictly implementing bullet 1. Using "%WSP*;" is useful for allowing zero or more white space to mean <nil>. I could use "%WSP+; %ES" to achieve the same goal but I'm not sure that was the intent here. Quick response appreciated. Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 2

Asterix format - not compelling examples of bitOrder leastSignificantBitFirst but byteOrder bigEndian
by Mike Beckerle 10 May '16

10 May '16

http://www.eurocontrol.int/sites/default/files/content/documents/nm/asterix… The above is part of the Asterix spec. Some interesting data layouts are on pages: Page 28 - shows integer fields that span bytes. These are big-endian integers, however the bits are numbered in the prose LSBF. But DFDL doesn't use bit positions, it uses lengths. So that prose isn't needed. Page 30 - simple example of two single bit fields, followed by a 14-bit big-endian integer Page 35 - simple example of a single bit followed by a big-endian 15-bit integer Page 41 - String of 8 characters represented as 6-bit code points. (motivates the proposal for dfdl:character(...) function) In every case I see, it's very easy to model this data using bitOrder mostSignificantBitFirst and byteOrder bigEndian, and simply specifying the length (in bits) of the elements. It is just funny to look at the way they number the bits, and talk about the bit positions. Their numbering scheme uses bit 1 as the least-significant bit, but the order they present things starts first with the most significant bit. So in a 16 bit word, they'll start by describing "bit 16" first, which is the most significant bit first. So their bit numbers are assigned LSBF, but the "first" bit is the most-significant bit, first byte is the most significant byte (bigEndian). Now bit position numbers really don't play a role in a DFDL schema, so I'd say Asterix is really just bigEndian and mostSignificantBitFirst. There's nothing compelling here about the need for the additional combination of bigEndian with leastSignificantBitFirst bit order. So I guess I withdraw the suggestion that this combination is really needed by this Asterix format. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

OGF DFDL WG Call Minutes 2016-05-10
by Steve Hanson 10 May '16

10 May '16

Please find minutes from the above call at https://redmine.ogf.org/dmsf_files/13530?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0