Re: [DFDL-WG] Action 282

11 May 2016

      Michele & team

Please join the DFDL WG call on 24th May so we can review the proposal and 
hopefully close this action.

Thanks

Steve Hanson
IBM Integration Bus, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh@uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890

From:   Michele Zundo <michele.zundo@esa.int>
To:     Steve Hanson/UK/IBM@IBMGB
Cc:     "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Mike Beckerle 
<mbeckerle.dfdl@gmail.com>, Maurizio De Bartolomei 
<mdebartolomei@eopp.esa.int>, Montserrat Piñol <mpinol@eopp.esa.int>, Rui 
Mestre <rui.mestre@deimos.com.pt>
Date:   12/04/2016 16:43
Subject:        Re: [DFDL-WG] Action 282

Steve, I see this only now,

we will not have time this nor the coming week, as I want to discuss the 
proposal internally.

We will review the proposal below and come back to you.

Michele

On 11 Apr 2016, at 10:47 , Steve Hanson <smh@uk.ibm.com> wrote:

282
Does XPath have operators for checking a value is in a range? (Steve) 
12/5: Investigate whether equivalent to DFDL4S 'in' operator exists. 
23/6: Mike has found an XPath 'intersect' operator. Handles the 
enumeration case well, but not as convenient for ranges as DFDL4S's 'in' 
operator. 
11/8: Looked back at the motivating example from DFDL4S. Agreed that DFDL 
functions to do the equivalent of 'in' and 'inrange' would be useful if 
nothing can be re-used from XPath. Steve to write up a proposal taking 
into account different data types. 
... 
22/9: Proposal sent by Steve for new functions dfdl:checkValues(), 
dfdl:checkRangeInclusive(), dfdl:checkRangeExclusive(). Discussed whether 
both range functions needed, and whether they should be allowed for float 
and double. Mike noted that dfdl:checkConstraints() and simple types could 
be used instead of all three functions if values were static. Follow-up 
with DFDL4S team to see if they had thought of that. 
3/11: DFDL4S not happy with usability of dfdl:checkConstraints type, but 
ok with the intersection operator.  Next step is to see what the DFDL4S 
schema would look like if rewritten to use dfdl:checkConstraints and 
intersection. 
5/1/16: Agreed that dfdl:checkConstraints() is not ideal as it requires 
creation of union simple types. Steve to go back to his proposal from 
22/9/15 and rework to include use of the intersect operator. 
16/2: No progress 
1/3: Discussed with ESA team. Steve to rework his proposal from 22/9/15.

Reworked proposal: 

a) Add XPath 2.0 'intersect' and "except" operators to the list of 
supported operators. 

Updated Table 57 as follows: 
DFDL Expression 
   ::= 
"{" Expr "}" 
Expr 
   ::= 
ExprSingle 
ExprSingle 
   ::= 
 IfExpr
| OrExpr 
IfExpr 
   ::= 
"if" "(" Expr ")" "then" ExprSingle "else" ExprSingle 
OrExpr 
   ::= 
AndExpr ( "or" AndExpr )* 
AndExpr 
   ::= 
ComparisonExpr ( "and" ComparisonExpr )* 
ComparisonExpr 
   ::= 
AdditiveExpr ( (ValueComp) AdditiveExpr)? 
AdditiveExpr 
   ::= 
MultiplicativeExpr ( ("+" | "-") MultiplicativeExpr )* 
MultiplicativeExpr 
   ::= 
IntersectExceptExpr ( ("*" | "div" | "idiv" | "mod") IntersectExceptExpr 
)* 
IntersectExceptExpr 
   ::=    
 UnaryExpr ( ("intersect" | "except") UnaryExpr )* 
UnaryExpr 
   ::= 
("-" | "+")* ValueExpr 
ValueExpr 
   ::= 
PathExpr 
ValueComp 
   ::= 
"eq" | "ne" | "lt" | "le" | "gt" | "ge" 
PathExpr 
   ::= 
("/" RelativePathExpr?)
| RelativePathExpr | FilterExpr 
RelativePathExpr 
   ::= 
StepExpr (("/") StepExpr)* 
StepExpr 
   ::= 
AxisStep 
AxisStep 
   ::= 
(ReverseStep | ForwardStep) Predicate? 
ForwardStep 
   ::= 
(ForwardAxis NodeTest) | AbbrevForwardStep 
ForwardAxis 
   ::= 
("child" "::")
| ("self" "::")
AbbrevForwardStep 
   ::= 
NodeTest | ContextItemExpr 
ReverseStep 
   ::= 
(ReverseAxis NodeTest) | AbbrevReverseStep 
ReverseAxis 
   ::= 
("parent" "::")
AbbrevReverseStep 
   ::= 
".." 
NodeTest 
   ::= 
NameTest 
NameTest 
   ::= 
QName 
FilterExpr 
   ::= 
PrimaryExpr Predicate? 
Predicate 
   ::= 
"[" Expr "]" 
PrimaryExpr 
   ::= 
Literal | VarRef | ParenthesizedExpr | ContextItemExpr | FunctionCall 
Literal 
   ::= 
NumericLiteral | StringLiteral 
NumericLiteral 
   ::= 
IntegerLiteral | DecimalLiteral | DoubleLiteral 
VarRef 
   ::= 
"$" VarName 
VarName 
   ::= 
QName 
ParenthesizedExpr 
   ::= 
"(" Expr ")" 
ContextItemExpr 
   ::= 
"." 
FunctionCall 
   ::= 
QName "(" (ExprSingle ("," ExprSingle)*)? ")"

b) New DFDL functions 

dfdl:checkRangeInclusive($node, $val1, $val2) 
Returns boolean true if the specified node value is in the range given by 
$val1 and $val2, inclusive. 
The type of $val1 and $val2 must be compatible with the type of $node, and 
must be a derivative of xs:decimal,  xs:float or xs:double. 
It is a schema definition error if the $node argument is a complex 
element.

dfdl:checkRangeExclusive($node, $val1, $val2) 
Returns boolean true if the specified node value is in the range given by 
$val1 and $val2, exclusive. 
The type of $val1 and $val2 must be compatible with the type of $node, and 
must be a derivative of xs:decimal, xs:float or xs:double. 
It is a schema definition error if the $node argument is a complex 
element.

Regards

Steve Hanson
IBM Integration Bus, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh@uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890 

From:        Steve Hanson/UK/IBM 
To:        Michele Zundo <michele.zundo@esa.int> 
Cc:        "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Mike Beckerle <
mbeckerle.dfdl@gmail.com>, Maurizio De Bartolomei <
mdebartolomei@eopp.esa.int>, Montserrat Piñol <mpinol@eopp.esa.int>, Rui 
Mestre <rui.mestre@deimos.com.pt> 
Date:        02/11/2015 19:51 
Subject:        Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First 
Release of DFDL4S Parser Library) 

<xs:simpleType name="redAndYellowAlertType">
 <xs:union memberTypes="redAlertType yellowAlertType"/>
</xs:simpleType>

The above is a type that allows ranges 0-200 and 201-500. 

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM Integration Bus, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890 

From:        Michele Zundo <michele.zundo@esa.int> 
To:        Steve Hanson/UK/IBM@IBMGB 
Cc:        Mike Beckerle <mbeckerle.dfdl@gmail.com>, "dfdl-wg@ogf.org" <
dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei@eopp.esa.int>, 
Montserrat Piñol <mpinol@eopp.esa.int>, Rui Mestre <
rui.mestre@deimos.com.pt> 
Date:        02/11/2015 16:08 
Subject:        Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First 
Release of DFDL4S Parser Library) 

Steve, 

I believe I do not understand this. 

can you make an example ? 

Michele 

On 02 Nov 2015, at 16:33 , Steve Hanson <smh@uk.ibm.com> wrote: 

I was also going to add that DFDL supports xs:union on simple types, 
enabling dfdl:checkConstraints to be used when there are multiple ranges.

·        Unions; the memberTypes must be derived from the same simple 
type. DFDL annotations are not permitted on union members.

The purpose of unions is to allow multiple constraints via facets such as 
multiple independent range restrictions on numbers. This enhances the 
ability to do rich validation of data.

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM Integration Bus, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890

From:        Mike Beckerle <mbeckerle.dfdl@gmail.com>
To:        Michele Zundo <michele.zundo@esa.int>
Cc:        Steve Hanson/UK/IBM@IBMGB, Rui Mestre <rui.mestre@deimos.com.pt
...
, "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Montserrat Piñol <
mpinol@eopp.esa.int>, Maurizio De Bartolomei <mdebartolomei@eopp.esa.int>
Date:        02/11/2015 14:33
Subject:        Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First 
Release of DFDL4S Parser Library)
Michele,

I think the first thing I'd propose for the set membership test is for us 
to add the XPath 2.0 intersect operator to DFDL. So your discriminator 
becomes:

<dfdl:discriminatortest=
"{fn:exists(/Packet_Primary_Header/Packet_Identification/APID intersect 
(0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12,
     16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
     32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
     48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
     64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,
     80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
   256,257,258,259,260,261,262,263,264,265,266,267,268,
   272,273,274,275,276,277,278,279,280,281,282,283,284,
   288,289,290,291,292,293,294,295,296,297,298,299,300,
   304,305,306,307,308,309,310,311,312,313,314,315,316,
   320,321,322,323,324,325,326,327,328,329,330,331,332,
   336,337,338,339,340,341,342,343,344,345,346,347,348))}"/>

This is a small addition to DFDL, but completely in the spirit of sticking 
with XPath 2.0 wherever possible. 

...mike beckerle

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | 
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are 
subject to the OGF Intellectual Property Policy

On Wed, Oct 28, 2015 at 7:22 AM, Michele Zundo <michele.zundo@esa.int> 
wrote:
Hi Steve,

your example below addresses the belonging to a range, how would then  the
belonging to a set be checked ? Currently our usage looks like here:

<xs:complexTypename="TypePacketData">
<xs:sequence>
<xs:choice>
<xs:sequence> <!-- Choice for MSI -->
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/">
<dfdl:discriminatortest=
"{/Packet_Primary_Header/Packet_Identification/APID in [  0,  1,  2,  3, 
4,  5,  6,  7,  8,  9, 10, 11, 12,
     16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
     32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
     48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
     64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,
     80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
   256,257,258,259,260,261,262,263,264,265,266,267,268,
   272,273,274,275,276,277,278,279,280,281,282,283,284,
   288,289,290,291,292,293,294,295,296,297,298,299,300,
   304,305,306,307,308,309,310,311,312,313,314,315,316,
   320,321,322,323,324,325,326,327,328,329,330,331,332,
   336,337,338,339,340,341,342,343,344,345,346,347,348]}"/>
</xs:appinfo>
</xs:annotation>
<xs:element name="MSI_Packet_Secondary_Header"type="TypePacketData_MSI"/>
<xs:element name="MSI_User_Data_Field"type="TypeUserData"/>
</xs:sequence>
<xs:sequence> <!-- Choice for TM_GPSR -->
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/">
<dfdl:discriminatortest=
"{/Packet_Primary_Header/Packet_Identification/APID in 
[769,771,772,774,775,777,779,780,781,785,787,788,790,791,793,795,796,797]}"
/>
</xs:appinfo>
</xs:annotation>
<xs:element name="TM_GPSR_Packet_Secondary_Header"type=
"TypePacketData_TMGPSR"/>
<xs:element name="TM_GPSR_User_Data_Field"type="TypeUserData"/>
</xs:sequence>
<xs:sequence>
<!-- Choice for TM_Time_packet(9,2), Overlaps with MSI APID 0x0  Removed 
here (moved to 99999) as it is SCIENCE config-->
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/">
<dfdl:discriminator
test="{/Packet_Primary_Header/Packet_Identification/APID in [99999]}" />
</xs:appinfo>
</xs:annotation>
<xs:element name="TM_Time_Packet_Secondary_Header"type=
"TypePacketData_TMT92"/>
<xs:element name="TM_Time_Packet_User_Data_Field"type="TypeUserData_TMT92"
/>
</xs:sequence>
<xs:sequence> <!-- Choice for TM_STR -->
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/">
<dfdl:discriminatortest=
"{/Packet_Primary_Header/Packet_Identification/APID in 
[596,598,612,614,628,630,660,662]}"/>
</xs:appinfo>
</xs:annotation>
<xs:element name="TM_STR_Packet_Secondary_Header"type=
"TypePacketData_TMSTR"/>
<xs:element name="TM_STR_User_Data_Field"type="TypeUserData"/>
</xs:sequence>
<xs:sequence> <!-- Choice for TM_CSW_AOCS -->
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/">
<dfdl:discriminatortest=
"{/Packet_Primary_Header/Packet_Identification/APID in [150,182]}" />
</xs:appinfo>
</xs:annotation>
<xs:element name="TM_CSW_AOCS_Packet_Secondary_Header"type=
"TypePacketData_TMCSWAOCS"/>
<xs:element name="TM_CSW_AOCS_User_Data_Field"type="TypeUserData"/>
</xs:sequence>
<xs:sequence>
<!-- Choice for HKTM -->
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/">
<dfdl:discriminator
test="{/Packet_Primary_Header/Packet_Identification/APID inrange [144,149] 

or /Packet_Primary_Header/Packet_Identification/APID inrange [151,181]
or /Packet_Primary_Header/Packet_Identification/APID inrange [183,239]
or /Packet_Primary_Header/Packet_Identification/APID inrange [516,575]
or /Packet_Primary_Header/Packet_Identification/APID in 
[592,593,594,595,597,613,629,656,657,658,659,661]
or /Packet_Primary_Header/Packet_Identification/APID inrange [599,611]
or /Packet_Primary_Header/Packet_Identification/APID inrange [615,627]
or /Packet_Primary_Header/Packet_Identification/APID inrange [631,639]
or /Packet_Primary_Header/Packet_Identification/APID inrange [663,671]
or /Packet_Primary_Header/Packet_Identification/APID in 
[768,770,776,778,782,783,784,792,794,798,799]
or /Packet_Primary_Header/Packet_Identification/APID inrange [1280,1295]}"
/>
</xs:appinfo>
</xs:annotation>
<xs:element name="TM_HKTM_Packet_Secondary_Header"type=
"TypePacketData_HKTM"/>
<xs:element name="TM_HKTM_User_Data_Field"type="TypeUserData_HKTM"/>
</xs:sequence>
<xs:sequence>
<!-- Choice for TM_IDLE -->
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/">
<dfdl:discriminator
test="{/Packet_Primary_Header/Packet_Identification/APID in [2047]}"
/>
</xs:appinfo>
</xs:annotation>
<xs:element name="TM_IDLE_Packet_Secondary_Header"type=
"TypePacketData_TMIDLE"
/>
</xs:sequence>
<xs:sequence> <!-- Choice for UNKNOWN -->
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/">
<dfdl:discriminatortest="{true}"/>
</xs:appinfo>
</xs:annotation>
<xs:element name="UNKNOWN_User_Data_Field"type="TypePacketData_UNKNOWN"/>
</xs:sequence>
</xs:choice>
</xs:sequence>
</xs:complexType>

On 27 Oct 2015, at 18:01 , Steve Hanson <smh@uk.ibm.com> wrote:

Just to clarify, dfdl:assert and dfdl:discriminator are identical when the 
expression returns false, the difference between the two is when the 
expression returns true and the parser is in a 'point of uncertainty' (eg 
choice) - a discriminator returning true has a positive discrimination on 
the point of uncertainty, an assert does not. 

I suspect that most of the time you can achieve what you want with just 
dfdl:checkConstraints() and simple type inheritance. Example below (syntax 
simplified for clarity).  But I am happy to be proved wrong, so if you 
have an example ... ?

<xs:simpleType name="alertType">
<xs:restriction base="xs:unsignedInt>
<xs:minInclusive value="0"/>
<xs:maxInclusive value="999"/>
</xs:restriction>
</xs:simpleType>

<xs:simpleType name="redAlertType">
<xs:restriction base="alertType">
<xs:minInclusive value="0"/>
<xs:maxInclusive value="200"/>
</xs:restriction>
</xs:simpleType>

<xs:simpleType name="yellowAlertType">
<xs:restriction base="alertType">
<xs:minInclusive value="201"/>
<xs:maxInclusive value="500"/>
</xs:restriction>
</xs:simpleType>

...

<xs:choice>
 <xs:element name="red">
   <xs:complexType>
     <xs:sequence>
       <xs:element name="alert" type="redAlertType">
               <dfdl:discriminator test="{dfdl:checkConstraints()}">
       </xs:element>
       ...
     </xs:sequence>
   </xs:complexType>
 </xs:element>
 <xs:element name="yellow">
   <xs:complexType>
     <xs:sequence>
       <xs:element name="alert" type="yellowAlertType">
               <dfdl:discriminator test="{dfdl:checkConstraints()}">
       </xs:element>
       ...
     </xs:sequence>
   </xs:complexType>
 </xs:element>
 <xs:element name="other">
   <xs:complexType>
     <xs:sequence>
       <xs:element name="alert" type="alertType">
               <dfdl:assert test="{dfdl:checkConstraints()}">
       </xs:element>
       ...
     </xs:sequence>
   </xs:complexType>
 </xs:element>

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM Integration Bus, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890

From:        Michele Zundo <michele.zundo@esa.int>
To:        Steve Hanson/UK/IBM@IBMGB
Cc:        dfdl-wg@ogf.org, Maurizio De Bartolomei <
mdebartolomei@eopp.esa.int>, Montserrat Piñol <mpinol@eopp.esa.int>, Rui 
Mestre <rui.mestre@deimos.com.pt>
Date:        27/10/2015 16:27
Subject:        Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First 
Release of DFDL4S Parser Library)

Yes, exactly. This is one of the current use case, in addition we could 
also use the value to
implement different type of checks e.g. some of the parameters we look at 
have a type,
a range outside of which they are invalid and a set of other ranges that 
correspond to various other conditions
(e.g. red alarm or yellow alarm).

Michele

On 27 Oct 2015, at 17:19 , Steve Hanson <smh@uk.ibm.com> wrote:

"Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) 
would be used to do conditional checks in derived schemas outside the 
scope of a dfdl:assert (to guide the parsing among several options, like 
we do for the our S2G 
TF schema version)."

That sounds like you would use this in a dfdl:discriminator ? 

Yes, exactly. While this is one of the current use case, in addition we 
would also use the value to
implement different type of checks e.g. some of the parameters we look at 
have a type,
a range outside of which they are invalid and a set of other ranges that 
correspond to various other conditions
(e.g. red alarm or yellow alarm).

Michele

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM Integration Bus, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890

From:        Michele Zundo <michele.zundo@esa.int>
To:        Steve Hanson/UK/IBM@IBMGB
Cc:        dfdl-wg@ogf.org, Maurizio De Bartolomei <
mdebartolomei@eopp.esa.int>, Montserrat Piñol <mpinol@eopp.esa.int>, Rui 
Mestre <rui.mestre@deimos.com.pt>
Date:        27/10/2015 16:09
Subject:        Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First 
Release of DFDL4S Parser Library)

Dear Steve,

to progress on this action:

1) it is our intention to “evolve” from our own dmx:assert to the 
dfdl:assert in order to standardise as much as possible
with the official spec provided we find a solution for what below.

2) we had an iteration with our developers and came to the conclusion that 
 there is 
need for both syntaxes: the existing (in the standard) 
dfdl:checkConstraints()and the new
one you proposed i.e. dfdl:checkRangeand dfdl:checkValue

The main semantic difference  between checkValue/checkRangeand  
checkConstraints
is that in one case we we need to associate a range to a quantity in a 
sort of static way
(i.e. the type “knowns” it has a range associated with it) while the 
checkvalue/checkrange
is something that you can do also if a range is not defined (see below).

For example you could have a definition for a specification sheet of an 
electrical device.
The XSD type (Volts) intrinsically defined with a range 0-1000 but wanted
to check in derived schemas if value is between 100 and 150 or 200 and 250 
and this information
would be used in different way.

This could be used to check 0-1000 hard constraints to see if the the 
network will support it
while range 100-150 and 200-250 would check compatibility with US or 
European Voltages for commercial appliances.

Note that dfdl:assert applies a condition to the parsing so that if the 
check fails the parsing is aborted with a severe error.
In the example above we would want to apply checkConstraint defined with a 
range 0-1000 in a dfdl:assert (to stop the parsing due to incorrect 
values) but not a checkvalue/checkrange.
Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) 
would be used to do conditional checks in derived schemas outside the 
scope of a dfdl:assert (to guide the parsing among several options, like 
we do for the our S2G 
TF schema version).

Michele

On 05 Oct 2015, at 16:23 , Steve Hanson <smh@uk.ibm.com> wrote:

Hi Michele

Any update from your discussion?  

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848

From:        Michele Zundo <michele.zundo@esa.int>
To:        Steve Hanson/UK/IBM@IBMGB
Cc:        dfdl-wg@ogf.org, Rui Mestre <rui.mestre@deimos.com.pt>, 
Montserrat Piñol <mpinol@eopp.esa.int>, Maurizio De Bartolomei <
mdebartolomei@eopp.esa.int>
Date:        23/09/2015 16:44
Subject:        Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First 
Release of DFDL4S Parser Library)

Dear Steve,

thanks for the suggestion (same from Mike) we need to discuss this 
internally with the developers and look at few use cases 
to see what would be the consequences/advantages/disadvantages.

Give us a little bit of time..

Michele

On 22 Sep 2015, at 19:12 , Steve Hanson <smh@uk.ibm.com> wrote:

Hi Michele

Thanks for your quick response. The WG discussed this on the call today.

In fact, we wondered if the same result could already be achieved using 
dfdl:checkConstraints() and declaring the enums or range using XSD facets 
on xs:simpleType restrictions. 

Have you considered whether dfdl:checkConstraints() achieves what you 
want? 

Example:

<xs:simpleType name="myRange">
<xs:restriction base="xs:int>
<xs:minInclusive value="100"/>
<xs:maxInclusive value="200"/>
</xs:restriction>
</xs:simpleType>

<xs:element name="myValue" type="myRange">
<xs:annotation><xs:appinfo ...>
<dfdl:assert>dfdl:checkConstraints(.)</dfdl:assert>
</xs:appinfo></xs:annotation>
</xs:element>

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848

From:        Michele Zundo <michele.zundo@esa.int>
To:        dfdl-wg@ogf.org
Cc:        Rui Mestre <rui.mestre@deimos.com.pt>, Montserrat Piñol <
mpinol@eopp.esa.int>, Maurizio De Bartolomei <mdebartolomei@eopp.esa.int>
Date:        22/09/2015 17:21
Subject:        [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of 
DFDL4S        Parser Library)
Sent by:        dfdl-wg-bounces@ogf.org

Dear Steve,

nice to hear things are moving.

The syntax below seems both reasonable and readable to me which is a good 
thing.

It addresses the belonging to decimal groups (integer only ??) I 
understand. (xs:decimal)

1) It might be also useful (not in our application but in general) to see 
if it makes sense also to define 
for dfdl:checkValues belongings to set of “enum” like value 

e.g. if we have a type for $val defined as (ON, OFF or STANDBY) or 
(MARRIED, SINGLE, WIDOWER)
can we also define a syntax that the $node belongs to it ?

2) for dfdl:checkRange it might make sense also to allow float numbers 
e.g.  X between 3.1 to 9.5

Michele

PS I will forward this proposal to our DFDL4S developers (in copy) to get 
their thinking.

From: "Steve Hanson" <smh@uk.ibm.com>
Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S 
Parser Library)
Date: 22 Sep 2015 16:50:13 CEST
To: DFDL-WG <dfdl-wg@ogf.org>

To kick start this action, here is a proposal ... builds on the precedent 
provided by dfdl:checkConstraints($node). 
dfdl:checkValues($node, $val1, $val2, ...) 
Returns boolean true if the specified node value matches any of the values 
provided by $val1 etc. 
The type of $val1 etc must be compatible with the type of $node. 
It is a schema definition error if the $node argument is a complex 
element. 
The number of value arguments is implementation-defined.

dfdl:checkRangeInclusive($node, $val1, $val2) 
Returns boolean true if the specified node value is in the range given by 
$val1 and $val2, inclusive. 
The type of $val1 and $val2 must be compatible with the type of $node, and 
must be a derivative of xs:decimal. 
It is a schema definition error if the $node argument is a complex 
element.

dfdl:checkRangeExclusive($node, $val1, $val2) 
Returns boolean true if the specified node value is in the range given by 
$val1 and $val2, exclusive. 
The type of $val1 and $val2 must be compatible with the type of $node, and 
must be a derivative of xs:decimal. 
It is a schema definition error if the $node argument is a complex 
element.

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848

From:        Steve Hanson/UK/IBM
To:        DFDL-WG <dfdl-wg@ogf.org>
Date:        11/08/2015 16:28
Subject:        Fw: [DFDL]: First Release of DFDL4S Parser Library 

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
----- Forwarded by Steve Hanson/UK/IBM on 11/08/2015 16:27 -----

From:        Michele Zundo <michele.zundo@esa.int>
To:        Mike Beckerle <mbeckerle.dfdl@gmail.com>
Cc:        Steve Hanson/UK/IBM@IBMGB, Maurizio De Bartolomei <
Maurizio.De.Bartolomei@esa.int>, Montserrat Piñol <mpinol@eopp.esa.int>, 
"Rui Mestre (DME)" <rui.mestre@deimos.com.pt>
Date:        18/05/2015 08:47
Subject:        Re: [DFDL]: First Release of DFDL4S Parser Library

Thanks Mike,

we will add this to our list to be considered/noted.

However reading your explanation (NB I’m NOT at all an XPath expert) it 
seemed you
had some good reason for avoiding longer than 1 path, so I would like to 
avoid our DFDL4S
project results in an over-complication of the DFDL implementation/use of 
Xpath 
unless there are other reasons/users/rationale requiring this feature. 
(btw the syntax is still weird-ish:  “intersect” reminds me of Venn 
Diagrams…)

As a project manager I always evaluate solutions and their cost vs the 
benefit they bring,
and I believe the DFDL community should keep this is mind.

Michele

PS The syntax above anser to the question “belongs to” , would there be 
any way to express ranges of values then ?

On 15 May 2015, at 16:24 , Mike Beckerle <mbeckerle.dfdl@gmail.com> wrote:

Just a few comments on DFDL4S, and also thank you to Michele and team for 
the presentation on Tuesday. 

I think all the issues in the spreadsheet are fairly easily fixed in that 
they are not major changes to DFDL4S, and would bring it into much closer 
compliance with the DFDL spec.

The exception is the XPath limitations where DFDL4S has gone beyond what 
XPath 2.0 allows and invented new syntax for expressing set membership 
requirements.

So I took a look, and XPath 2.0 has a set intersect operator:  ns1 
intersect ns2 => ns3

This isn't in DFDL today, but might be usable to achieve the set 
membership test; however, it requires use of XPath node sequences of 
length greater than 1, which DFDL has avoided mostly. I say mostly as 
there are XPath expressions that return node sequences of length greater 
than 1 and those can be arguments to fn:count(...) for example.

So far in DFDL such node sequences cannot "leak out" of the XPath 
expression into DFDL elements, and I think the usage in DFDL4S is similar 
in that these node sequences would be needed only to check for set 
membership, so the result is just a boolean as part of an 
assert/discriminator.

We should examine whether XPath 2.0 set intersection is enough to meet the 
need. 

I believe the expressions would be something like:

fn:exists( . intersect (123, 456, 789, .... many more items....) )

- mike beckerle

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | 
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are 
subject to the OGF Intellectual Property Policy

On Tue, May 12, 2015 at 10:45 AM, Michele Zundo <michele.zundo@esa.int> 
wrote:
for reference, 
here a summary of the reported problem in an excel sheet.

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.

-----------------------------------------
Michele Zundo

Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo@esa.int

-----------------------------------------
Michele Zundo

Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo@esa.int

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

#### Sentinel2X-bandTMISPTypes.xsd moved to MyAttachments Repository V3.8 
(Link) on 24 August 2015 by Steve Hanson.

--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg

-----------------------------------------
Michele Zundo

Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo@esa.int

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.
--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg

-----------------------------------------
Michele Zundo

Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo@esa.int

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-----------------------------------------
Michele Zundo

Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo@esa.int

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.

-----------------------------------------
Michele Zundo

Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo@esa.int

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.

-----------------------------------------
Michele Zundo

Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo@esa.int

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.

--
 dfdl-wg mailing list
 dfdl-wg@ogf.org
 https://www.ogf.org/mailman/listinfo/dfdl-wg

----------------------------------------- 
Michele Zundo 

Head of Ground System Definition and Verification Office 
EOP-PEP 
European Space Agency, ESTEC 
e-mail: michele.zundo@esa.int 

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-----------------------------------------
Michele Zundo

Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo@esa.int

This message and any attachments are intended for the use of the addressee 
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in 
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete 
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the 
sender.

Please consider the environment before printing this email.

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU