dfdl-wg
Threads by month
- ----- 2026 -----
- January
- ----- 2025 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2004 -----
- December
- November
- 3039 discussions
24 May '16
Background for discussion on DFDL WG call today. Part of deferred action
242.
Regards
Steve Hanson
IBM Integration Bus, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh(a)uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890
----- Forwarded by Steve Hanson/UK/IBM on 24/05/2016 11:58 -----
From: Steve Hanson/UK/IBM
To: dfdl-wg(a)ogf.org
Date: 15/04/2014 15:45
Subject: Fw: [DFDL-WG] Action 242 - valueLength and contentLength
function wording
I had a quick look at the document and Steve's review comments.
The text tries to define valueLength() in terms of the info set value, or
states that it is 'calculated from' the infoset value. I believe this will
mislead many readers ( it already has done, in fact ). People just assume
that the 'value length' is 'the length of the logical value' and they
don't think any more deeply about it. So it is absolutely essential to
state clearly that this function returns the length of a region in the
data stream.
It may be simpler to define the behaviour in terms of the length of the
simpleContent or complexContent region, minus any padding.
The other point is around the bytes/characters issue. Mike's words, or
something similar, are definitely required because we don't mandate that
the encoding must be consistent throughout a complex type. Nor do we
prohibit a mixture of character and non-character content in a complex
type. And someone might even call valueLength() with lengthUnits
'characters' on a complex type that contains no character data at all.
In principle it would be possible for the value to be available while
parsing ( for elements that follow the specified element ). If we are
disallowing this then it should be stated very clearly somewhere. And
probably is already.
regards,
Tim Kimber,
IBM Integration Bus Development (Industry Packs)
Hursley, UK
Internet: kimbert(a)uk.ibm.com
Tel. 01962-816742
Internal tel. 37246742
----- Forwarded by Tim Kimber/UK/IBM on 15/04/2014 15:28 -----
From: Steve Hanson/UK/IBM@IBMGB
To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>,
Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, dfdl-wg-bounces(a)ogf.org
Date: 15/04/2014 15:01
Subject: Re: [DFDL-WG] Action 242 - valueLength and contentLength
function wording
Sent by: dfdl-wg-bounces(a)ogf.org
Review comments added:
Regards
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh(a)uk.ibm.com
tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>,
Date: 11/04/2014 14:04
Subject: Re: [DFDL-WG] Action 242 - valueLength and contentLength
function wording
Sent by: dfdl-wg-bounces(a)ogf.org
Revised Action 242 proposed changes word doc attached. I have incorporated
the discussion in this thread (I hope.) Please review.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
On Tue, Mar 25, 2014 at 10:56 AM, Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
wrote:
This language is consistent with what we say for lengthKind pattern in
section 12.3.5:
"When unparsing, the dfdl:valueLength of a complex type element when the
length units is 'characters' is computed as if the entire structure was
unparsed into a temporary data stream beginning at position 1, and then
this data stream is considered to be text in the character set encoding
specified by the dfdl:encoding property, regardless of the actual
representation of the complex type element or the elements contained
within it. The number of characters in this temporary data stream is the
value length of the complex type."
The behavior of the IBM DFDL implementation for valueLength is as
described is consistent with the above, excepting that it will not detect
a decode error, and it gives an SDE (?) if the encoding is not fixed
width.
Since we have decided not to require that a complex type element is
recursively all text all the way down, I believe we have to tolerate
implementations having different behaviors in the potentially meaningless
cases where there is binary data or encoding changes in the complex type.
So I would add to the above suggested language this:
"However, if creation of this data stream would cause an encoding error,
or parsing of this data stream as characters would cause a decoding error,
then the behavior and return value of dfdl:valueLength are implementation
dependent."
Looking at the DFDL spec, I am concerned that we never really say what we
mean by the "length of the ComplexContent region." (Last sentence before
Table 7 in section 12.3.7) Section 12.3.7.3 doesn't do it. The
dfdl:valueLength function may be the first place where we have to actually
say how the various sub-regions contribute to the ComplexContent region's
length.
I believe this is the obvious "sum of length of all contained regions",
but keep in mind that alignment region lengths will vary depending on the
starting alignment, so the length is, in general, dependent on the
position within the bit stream.
Hence when unparsing we have to specify that the dfdl:valueLength is
measured as if the ComplexContent region started at position 1 (as I did
above) so that internal alignment regions can be given meaningful lengths.
The general clarification should be added to 12.3.7.3, or to section
12.3.7 immediately before section 12.3.7.1. Something like this:
"The length of the ComplexContent region is the sum of the lengths of the
contained regions. However, note that alignment regions inside the
ComplexContent may be of different lengths depending on the
ComplexContent's starting position alignment."
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
On Mon, Mar 24, 2014 at 11:34 AM, Andrew Edwards <andy.edwards(a)uk.ibm.com>
wrote:
Steve (et al) - Resending as the last one bounced.
I'll usurp Tim and respond :)
Currently the IBM implementation insists on using a fixed-length encoding
and returns an "unsupported" error message for a variable width encoding.
With a fixed width encoding, we "do the maths" using the
bytes-per-character and the bytes written by this complex element.
HTH,
Andy
Andy Edwards - IBM Integration Bus - DFDL
Email:
andy.edwards(a)uk.ibm.com
Snail Mail:
MP211, Hursley park, Hursley, WINCHESTER, Hants, SO21 2JN
Tel int:
247222
Tel ext:
+44 (0)1962 817222
Desk:
DE3 V17
The Feynman problem solving Algorithm
1) Write down the problem
2) Think real hard
3) Write down the answer
-- Murray Gell-mann in the NY Times
Steve Hanson/UK/IBM
24/03/2014 14:52
To
"dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>,
cc
Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Andrew Edwards/UK/IBM@IBMGB
Subject
Re: [DFDL-WG] Action 242 - valueLength and contentLength function wording
Link
Note errata 3.9, my bolding:
"3.9. Section 12.3.5, 7.3.1, 7.3.2. The spec originally allows lengthKind
‘pattern’ to be used when the representation of the current element, or of
a child element, is binary, but imposes restrictions on the encoding that
can be in force.
Clarify that the encoding property must be defined for the element (else
schema definition error), and that a decoding processing error is possible
if the match of the regex encounters data that does not decode in that
encoding, dependent on the setting of encodingErrorPolicy. Remove section
12.3.5.1.
Same clarifications needed for testKind ”pattern” property for asserts and
discriminators.
For consistency, the restriction that a complex element of specified
length and lengthUnits ‘characters’ must have children that are all text
and that have the same encoding as the complex element, is dropped."
That's the restriction that I was referring to in my comment below. I can
see why it was dropped - basically the parser now just tries to decode n
characters using the complex element's encoding (and encodingErrorPolicy).
We could apply the same principle for dfdl:valueLength &
dfdl:contentLength - you build the stream from the bottom up, and then
decode it using the complex element's encoding (and encodingErrorPolicy ?)
to get the length in characters.
Note that's how unparsing for lengthKind 'prefixed' with lengthUnits
'characters' would work as well - the spec just says "For a complex
element, the length is that of the ComplexContent region" which is not
sufficient (12.3.4). Similar deal for lengthKind 'explicit' - in order to
know the size in chars of ElementUnused the unparser needs to know the
size in chars of the data first (12.3.7.3).
(Of course, for a fixed width encoding, you don't need to decode, you can
just do the maths, but for the general case you need to decode. Also just
doing the maths does not take encodingErrorPolicy into account).
Regards
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh(a)uk.ibm.com
tel:+44-1962-815848
From: Steve Hanson/UK/IBM
To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>,
Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, dfdl-wg-bounces(a)ogf.org
Date: 24/03/2014 12:55
Subject: Re: [DFDL-WG] Action 242 - valueLength and contentLength
function wording
Mike
23.5.3.1. Value length is only a function of the dfdl:encoding property if
the element has a text representation. Not sure this needs to be
(re)stated here.
23.5.3.1. "The value length is computed from the DFDL infoset value,
ignoring the dfdl:length or dfdl:textOutputMinLength property. Other DFDL
properties which affect the length of a text or binary representation are
respected, it is only an explicit length which is ignored." Last sentence
is too imprecise - should be phrased in terms of the grammar.
23.5.3.1. "If the second argument is 'characters' then the element must
have text representation and it is a schema definition error otherwise".
Yes but only for a simple type, so should be qualified.
23.5.3.1. "If the second argument, giving the length units, is
'characters', then recursively, this complex type element must have text
representation throughout all its contained elements and framing, all of
which must also use a uniform character set encoding." I can't see that
restriction elsewhere in the spec when it talks about length of
ComplexContent and lengthUnits 'characters' - I was expecting it to be in
section 12.3.4 or 12.3.7.3 which face the same issue - but it isn't. Did
we decide not to have this restriction? Without such a restriction, how
does the unparser come up with a meaningful length (unless it re-parses)?
(Tim - what does IBM DFDL do here?) What about delimiters and padding of
children that use %#r entities?
23.5.3.2. The points in 23.5.3.1 about escape characters, length as a
function of encoding, and bottom up for complex elements, apply equally to
23.5.3.2. It might be easier just to say in 23.5.3.2 that
dfdl:contentLength for complex elements is same as dfdl:valueLength, and
for simple elements differs only by the additional inclusion of
LeftPadding and RightPadOrFill regions.
Also noted in passing:
Specified length - An item has specified length when dfdl:lengthKind is
"implicit", "explicit", or "prefixed".
should be
Specified length - An element has specified length when dfdl:lengthKind is
"implicit" (simple type only), "explicit", or "prefixed".
Regards
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh(a)uk.ibm.com
tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>,
Date: 20/03/2014 17:21
Subject: [DFDL-WG] Action 242 - valueLength and contentLength
function wording
Sent by: dfdl-wg-bounces(a)ogf.org
See attached doc which is proposed revisions to section 23.5.3
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
[attachment "Action-252-DFDL-Functions-23.5.3.docx" deleted by Andrew
Edwards/UK/IBM] --
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
[attachment "Action-252-DFDL-Functions-23.5.3.docx" deleted by Steve
Hanson/UK/IBM] --
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
--
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
#### Action-252-DFDL-Functions-23.5.3.docx moved to MyAttachments
Repository V3.8 () on 30 April 2014 by Steve Hanson.
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
1
0
Fw: Action 242 - propose simplification for length units in dfdl:valuelength and dfdl:contentLength functions
by Steve Hanson 24 May '16
by Steve Hanson 24 May '16
24 May '16
Background for discussion on DFDL WG call today. Part of deferred action
242.
Regards
Steve Hanson
IBM Integration Bus, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh(a)uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890
----- Forwarded by Steve Hanson/UK/IBM on 24/05/2016 11:57 -----
From: Steve Hanson/UK/IBM
To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, dfdl-wg-bounces(a)ogf.org
Date: 28/04/2014 11:24
Subject: Re: [DFDL-WG] Action 242 - propose simplification for
length units in dfdl:valuelength and dfdl:contentLength functions
Mike
I don't think you can always ensure the result is well-defined.
The element (first argument) is silent about its lengthUnits when not
applicable to is lengthKind - specifically 'pattern', 'delimited',
'implicit' (complex) and 'endOfParent'. For the first three we need to
allow lengthUnits 'characters' to be provided as the second argument.
Note for 'endOfParent' the length can't be calculated so it should be an
SDE if the function is called against that element ?
It should be an SDE if lengthUnits 'characters' is provided as the second
argument and the representation is 'binary' or the type is hexBinary.
Regards
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh(a)uk.ibm.com
tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>,
Date: 24/04/2014 23:41
Subject: [DFDL-WG] Action 242 - propose simplification for length
units in dfdl:valuelength and dfdl:contentLength functions
Sent by: dfdl-wg-bounces(a)ogf.org
These functions currently take the desired result units as the second
argument.
It is very complicated if the length units of the element are bytes, to
ask for this length to be recast in characters due to the
fragment-of-a-character on the end problem. In some sense there is
quotient/remainder here, which means there are multiple operations here,
not just one. There is the equivalent of floor (as many characters as fit,
then fillByte) ceiling (every byte filled from some character), remainder
(how many left over bytes are there). I think this is silly unless we have
use cases driving this.
I believe this restriction matches all use-cases I can think of:
Restriction: If the element (first argument) to either valueLength or
contentLength has length units of
characters -> can ask for valueLength or contentLength in any of
characters, bytes, or bits
bytes -> can ask for valueLength or contentLength in bytes or bits
bits -> can only ask for valueLength or contentLength only in bits.
and it is an SDE otherwise.
This insures that the result of these functions is always well defined.
I understand why there is a box sized in characters and I need to know its
size in bytes, but the opposite I can't see a use case for, and it isn't
crisply defined even.
Ditto for length in bits. If the box is sized in bytes I might want that
in bits (so * 8), but if the box is sized in bits, and I ask for bytes,
what do I want if there is a fragment of a byte? ceiling? floor?
remainder?
Thoughts?
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
--
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
1
0
24 May '16
Background for discussion on DFDL WG call today. Part of deferred action
242.
Regards
Steve Hanson
IBM Integration Bus, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh(a)uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890
----- Forwarded by Steve Hanson/UK/IBM on 24/05/2016 11:58 -----
From: Steve Hanson/UK/IBM
To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, dfdl-wg-bounces(a)ogf.org
Date: 28/04/2014 11:10
Subject: Re: [DFDL-WG] dfdl:contentLength and complex types with
interior alignmentFill
Mike
This doesn't solve the problem fully, because it assumes an alignment of 0
and in practice the alignment might be some odd number that forces the
first child with alignment 2, 4 or 8 onto a different offset, and hence
the length is different. So I'm not sure whether the additional
calculation helps much.
Regards
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh(a)uk.ibm.com
tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>,
Date: 24/04/2014 22:36
Subject: [DFDL-WG] dfdl:contentLength and complex types with
interior alignmentFill
Sent by: dfdl-wg-bounces(a)ogf.org
dfdl:contentLength interacts pretty badly with alignment because
alignmentFill regions mean you cannot know the size of a complex type
element without knowing its starting position.
It is very likely that one will have outputValueCalc that wants to
evalutate dfdl:contentLength on an infoset item before the starting
position for that item is known or knowable.
To fix this we say that dfdl:contentLength(e) on complex-type infoset item
e, is computed as if element e's alignment was first put into place along
with the rest of the LeftFraming, and then the content length computed as
if the complex content was unparsed starting from after that LeftFraming
up to the final RightFraming In other words, the entire element has to be
unparsed, not just the complex content of it, and then the part of that
unparsed data that corresponds to the complex content is used as the
return value from dfdl:contentLength.
This works because element e's alignment (LeftFraming generally) is not
within the complex content for element e. It is before the complex content
of the element, so if the schema puts an explicit alignment on element e,
then it will clearly be aligned that way when it is really unparsed, but
the dfdl:contentLength function will also compute a length from the
infoset value as if that same alignment were in place.
I think that is about the best we can do.
Thoughts?
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
--
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
1
0
Please find agenda for call on Redmine at
https://redmine.ogf.org/dmsf_files/13532?download=
Regards
Steve Hanson
Architect, IBM Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh(a)uk.ibm.com
tel:+44-1962-815848
1
0
Re: [DFDL-WG] Fwd: proposal: DFDL needs additional function dfdl:characterCode
by Steve Hanson 24 May '16
by Steve Hanson 24 May '16
24 May '16
To be discussed on DFDL WG call today.
Regards
Steve Hanson
IBM Integration Bus, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh(a)uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890
From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
To: Steve Hanson/UK/IBM@IBMGB
Date: 10/05/2016 16:57
Subject: Fwd: [DFDL-WG] proposal: DFDL needs additional function
dfdl:characterCode
This is the thread that mentions dfdl:characterCode(string, pos): int, and
dfdl:character(charCodeInt): String proposed functions.
It also discusses the issue of XML-illegal characters in the XML
representation of the DFDL infoset.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
---------- Forwarded message ----------
From: Tim Kimber <KIMBERT(a)uk.ibm.com>
Date: Thu, Nov 1, 2012 at 12:00 PM
Subject: Re: [DFDL-WG] proposal: DFDL needs additional function
dfdl:characterCode
To: dfdl-wg(a)ogf.org
Correct - there is no way at all of using illegal characters in an XML
document. Not CDATA, not character entities. They simply must not appear
anywhere.
I agree with Steve that XML compatibility is not the only requirement for
a DFDL info set - we should not do anything to make it XML specific in a
way that harms it generality. To balance that, I also think that the DFDL
Working Group should be paying attention to the issues around XML
compatibility, given that DFDL is based on XML Schema and many potential
adopters of DFDL will want to know about XML compatibility.
Mike's proposal of mapping illegal characters into the Unicode Private Use
Area sounds like a reasonable approach for implementers to use.
regards,
Tim Kimber, DFDL Team,
Hursley, UK
Internet: kimbert(a)uk.ibm.com
Tel. 01962-816742
Internal tel. 37246742
From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
To: Suman Kalia <kalia(a)ca.ibm.com>,
Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org
Date: 01/11/2012 15:14
Subject: Re: [DFDL-WG] proposal: DFDL needs additional function
dfdl:characterCode
Sent by: dfdl-wg-bounces(a)ogf.org
Turns out the XML char entities are not an escape scheme for putting
illegal chars in. E.g. � is illegal even expressed that way.
The char entities are essentially an internationalization hack so you can
enter and render any legal character using only a small charset.
On Nov 1, 2012 8:48 AM, "Suman Kalia" <kalia(a)ca.ibm.com> wrote:
Shouldn't we be using entity references for XML syntactic character found
in text/binary data while creating info set and vice versa...
Suman Kalia
IBM Canada Lab
WMB Toolkit Architect and Development Lead
Tel: 905-413-3923 T/L 313-3923
Email: kalia(a)ca.ibm.com
For info on Message broker
http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h…
From: Steve Hanson <smh(a)uk.ibm.com>
To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>,
Cc: dfdl-wg(a)ogf.org
Date: 11/01/2012 07:57 AM
Subject: Re: [DFDL-WG] proposal: DFDL needs additional
function dfdl:characterCode
Sent by: dfdl-wg-bounces(a)ogf.org
>From WG call minutes 2012-10-30:
"Beyond the scope of DFDL 1.0. Assumption for now is that infoset needs
post-processing."
Mike has observed that other software systems "map the illegal characters
to/from the Unicode Private Use Area."
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh(a)uk.ibm.com
tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>
To: dfdl-wg(a)ogf.org,
Date: 04/10/2012 23:39
Subject: [DFDL-WG] proposal: DFDL needs additional function
dfdl:characterCode
Sent by: dfdl-wg-bounces(a)ogf.org
An important use case for DFDL is converting legacy data to/from XML.
XML 1.0 disallows a bunch of string characters.
If the data contains those characters, then the question arises of what to
turn them into that both preserves information content, but also is legal
in XML so that you can convert the DFDL infoset into XML without violating
XML's constraints.
The natural thing to do is create an element containing the character code
of the illegal character, as an integer.
E.g., character code U+0001 would become an element. Such as:
<ccode>1</ccode>.
This could be done using a hidden element that is a string, and the
element ccode above would have an inputValueCalc that converts the
offending character of that string into an integer.
But we need a function dfdl:characterCode(str, pos) : int
The arguments would be a string, and a position (base 1) within that
string, and the return result would be the character code of the character
in the string at that position. If pos is out of the bounds of the string
(i.e., is negative, 0, or too large), then a processing error would occur.
For unparsing the inverse function would also be needed:
dfdl:character(intArg) : string. This would return a string containing one
character whose codepoint is the intArg.
Example
Consider this data:
123<0>456<1>789<2>123l
where <0> means just one character with codepoint 0, etc.
In hex that would be 313233 00 343536 01 373839 02 313233
The best I can think of for modeling this while preserving all information
would end up with XML looking like this:
<nonXMLString>
<fragment><stringData>123</stringData></fragment>
<fragment><nonXMLChar><charCode>0</charCode></nonXMLChar></fragment>
<fragment><stringData>456</stringData></fragment>
<fragment><nonXMLChar><charCode>1</charCode></nonXMLChar></fragment>
<fragment><stringData>789</stringData></fragment>
<fragment><nonXMLChar><charCode>2</charCode></nonXMLChar></fragment>
<fragment><stringData>123</stringData></fragment>
</nonXMLString>
So our nonXMLString is of a type which is array of fragment, a fragment is
a choice of either (legal XML) stringData, or a nonXMLChar.
The nonXMLChar has a child element because it will need to convert to from
a string so will use inputValueCalc and outputValueCalc to do so, so it
needs to be a sequence so that it can have the other hidden elements
needed to pull this off.
stringData would have lengthKind="pattern" and a pattern that allows any
sequence of XML-allowed characters.
nonXMLChar would have a hidden first child element of type string of
explicit length 1 with an assertion that the string match a pattern that
is any of the illegal characters (but just one of them). The charCode
child element would inputValueCalc to get the character code of the
character. For 8 bit encodings it would be ok as a table lookup in XPath,
but for unicode..... we'd need a function that returns a character code.
If you just have one embedded illegal character, like NUL, then you could
just model it as a separator, which would simplify things considerably
(and is possible in a someday XML 1.1 future since NUL is then the only
disallowed character.)
But for XML 1.0's illegal characters, we need to be able to convert
to/from some non-string representation if we are to preserve information
content. Hence we need these additional functions.
--
Mike Beckerle | OGF DFDL WG Co-Chair
Tel: 781-330-0412
--
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
--
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg --
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
--
dfdl-wg mailing list
dfdl-wg(a)ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
1
0
Confirmed: OGF DFDL Working Group call (24 May 15:00 GDT in IBM: Hursley DE3S09)
by Steve Hanson 23 May '16
by Steve Hanson 23 May '16
23 May '16
Calls for 2016.
OGF DFDL WG weekly call dial-in details.
Passcode for Participants: 5381214
Canada Toll-Free 888-426-6840
China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS
China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS
France Toll-Free 0800-94-0558
Germany Toll-Free 0800-000-1018
India Toll-Free 000-800-100-1176
Ireland Toll-Free 1-800-943-427
Israel Toll-Free 1-809-417-783
Netherlands Toll-Free 0-800-363-6036
United Kingdom Caller Paid 0-20-30596451
United Kingdom Toll-Free 0800-368-0638
USA Caller Paid 215-861-6239
USA Toll-Free 888-426-6840
Other international numbers available - e-mail smh(a)uk.ibm.com.
OGF DFDL Home: http://www.ogf.org/dfdl
Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg
DFDL Getting Started: http://www.ibm.com/developerworks/library/se-dfdl/
DFDL 1.0 Spec: http://www.ogf.org/documents/GFD.207.pdf
Call will go ahead tomorrow, I will send out agenda later.
1
0
Dear Steve,
sorry for the delay.
We have verified internally that your proposal of extension of support of XPath and
the 2 new function address our need. So from our side your proposal below is OK.
When you have updated the DFDL language reference doc with the let us know.
Regards
> On 11 May 2016, at 9:55 , Steve Hanson <smh(a)uk.ibm.com> wrote:
>
> Michele & team
>
> Please join the DFDL WG call on 24th May so we can review the proposal and hopefully close this action.
>
> Thanks
>
> Steve Hanson
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848
> mob:+44-7717-378890
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int>, Montserrat Piñol <mpinol(a)eopp.esa.int>, Rui Mestre <rui.mestre(a)deimos.com.pt>
> Date: 12/04/2016 16:43
> Subject: Re: [DFDL-WG] Action 282
>
>
>
> Steve, I see this only now,
>
> we will not have time this nor the coming week, as I want to discuss the proposal internally.
>
> We will review the proposal below and come back to you.
>
> Michele
>
>
>
>
>
> On 11 Apr 2016, at 10:47 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
> 282
> Does XPath have operators for checking a value is in a range? (Steve)
> 12/5: Investigate whether equivalent to DFDL4S 'in' operator exists.
> 23/6: Mike has found an XPath 'intersect' operator. Handles the enumeration case well, but not as convenient for ranges as DFDL4S's 'in' operator.
> 11/8: Looked back at the motivating example from DFDL4S. Agreed that DFDL functions to do the equivalent of 'in' and 'inrange' would be useful if nothing can be re-used from XPath. Steve to write up a proposal taking into account different data types.
> ...
> 22/9: Proposal sent by Steve for new functions dfdl:checkValues(), dfdl:checkRangeInclusive(), dfdl:checkRangeExclusive(). Discussed whether both range functions needed, and whether they should be allowed for float and double. Mike noted that dfdl:checkConstraints() and simple types could be used instead of all three functions if values were static. Follow-up with DFDL4S team to see if they had thought of that.
> 3/11: DFDL4S not happy with usability of dfdl:checkConstraints type, but ok with the intersection operator. Next step is to see what the DFDL4S schema would look like if rewritten to use dfdl:checkConstraints and intersection.
> 5/1/16: Agreed that dfdl:checkConstraints() is not ideal as it requires creation of union simple types. Steve to go back to his proposal from 22/9/15 and rework to include use of the intersect operator.
> 16/2: No progress
> 1/3: Discussed with ESA team. Steve to rework his proposal from 22/9/15.
>
> Reworked proposal:
>
> a) Add XPath 2.0 'intersect' and "except" operators to the list of supported operators.
>
> Updated Table 57 as follows:
> DFDL Expression ::= "{" Expr "}"
> Expr ::= ExprSingle
> ExprSingle ::= IfExpr
> | OrExpr
> IfExpr ::= "if" "(" Expr ")" "then" ExprSingle "else" ExprSingle
> OrExpr ::= AndExpr ( "or" AndExpr )*
> AndExpr ::= ComparisonExpr ( "and" ComparisonExpr )*
> ComparisonExpr ::= AdditiveExpr ( (ValueComp) AdditiveExpr)?
> AdditiveExpr ::= MultiplicativeExpr ( ("+" | "-") MultiplicativeExpr )*
> MultiplicativeExpr ::= IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ( ("*" | "div" | "idiv" | "mod") IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> )*
> IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ::= UnaryExpr <https://www.w3.org/TR/xpath20/#doc-xpath-InstanceofExpr> ( ("intersect" | "except") UnaryExpr )*
> UnaryExpr ::= ("-" | "+")* ValueExpr
> ValueExpr ::= PathExpr
> ValueComp ::= "eq" | "ne" | "lt" | "le" | "gt" | "ge"
> PathExpr ::= ("/" RelativePathExpr?)
> | RelativePathExpr | FilterExpr
> RelativePathExpr ::= StepExpr (("/") StepExpr)*
> StepExpr ::= AxisStep
> AxisStep ::= (ReverseStep | ForwardStep) Predicate?
> ForwardStep ::= (ForwardAxis NodeTest) | AbbrevForwardStep
> ForwardAxis ::= ("child" "::")
> | ("self" "::")
> AbbrevForwardStep ::= NodeTest | ContextItemExpr
> ReverseStep ::= (ReverseAxis NodeTest) | AbbrevReverseStep
> ReverseAxis ::= ("parent" "::")
> AbbrevReverseStep ::= ".."
> NodeTest ::= NameTest
> NameTest ::= QName
> FilterExpr ::= PrimaryExpr Predicate?
> Predicate ::= "[" Expr "]"
> PrimaryExpr ::= Literal | VarRef | ParenthesizedExpr | ContextItemExpr | FunctionCall
> Literal ::= NumericLiteral | StringLiteral
> NumericLiteral ::= IntegerLiteral | DecimalLiteral | DoubleLiteral
> VarRef ::= "$" VarName
> VarName ::= QName
> ParenthesizedExpr ::= "(" Expr ")"
> ContextItemExpr ::= "."
> FunctionCall ::= QName "(" (ExprSingle ("," ExprSingle)*)? ")"
>
>
>
>
> b) New DFDL functions
>
> dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive.
> The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double.
>
> It is a schema definition error if the $node argument is a complex element.
>
> dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive.
> The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double.
>
> It is a schema definition error if the $node argument is a complex element.
>
>
>
>
> Regards
>
> Steve Hanson
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:+44-1962-815848>
> mob:+44-7717-378890
>
>
>
> From: Steve Hanson/UK/IBM
> To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> Cc: "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 02/11/2015 19:51
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
>
> <xs:simpleType name="redAndYellowAlertType">
> <xs:union memberTypes="redAlertType yellowAlertType"/>
> </xs:simpleType>
>
> The above is a type that allows ranges 0-200 and 201-500.
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:+44-1962-815848>
> mob:+44-7717-378890
>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 02/11/2015 16:08
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
> Steve,
>
> I believe I do not understand this.
>
> can you make an example ?
>
> Michele
>
> On 02 Nov 2015, at 16:33 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> I was also going to add that DFDL supports xs:union on simple types, enabling dfdl:checkConstraints to be used when there are multiple ranges.
>
> · Unions; the memberTypes must be derived from the same simple type. DFDL annotations are not permitted on union members.
>
> The purpose of unions is to allow multiple constraints via facets such as multiple independent range restrictions on numbers. This enhances the ability to do rich validation of data.
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:+44-1962-815848>
> mob:+44-7717-378890
>
>
>
> From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>
> To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> Cc: Steve Hanson/UK/IBM@IBMGB, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>
> Date: 02/11/2015 14:33
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
>
> Michele,
>
> I think the first thing I'd propose for the set membership test is for us to add the XPath 2.0 intersect operator to DFDL. So your discriminator becomes:
>
> <dfdl:discriminatortest="{fn:exists(/Packet_Primary_Header/Packet_Identification/APID intersect (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
> 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
> 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
> 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
> 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,
> 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
> 256,257,258,259,260,261,262,263,264,265,266,267,268,
> 272,273,274,275,276,277,278,279,280,281,282,283,284,
> 288,289,290,291,292,293,294,295,296,297,298,299,300,
> 304,305,306,307,308,309,310,311,312,313,314,315,316,
> 320,321,322,323,324,325,326,327,328,329,330,331,332,
> 336,337,338,339,340,341,342,343,344,345,346,347,348))}"/>
>
>
> This is a small addition to DFDL, but completely in the spirit of sticking with XPath 2.0 wherever possible.
>
> ...mike beckerle
>
>
>
>
> Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/>
> Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>
>
>
> On Wed, Oct 28, 2015 at 7:22 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote:
> Hi Steve,
>
> your example below addresses the belonging to a range, how would then the
> belonging to a set be checked ? Currently our usage looks like here:
>
>
>
> <xs:complexTypename="TypePacketData">
> <xs:sequence>
> <xs:choice>
> <xs:sequence> <!-- Choice for MSI -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
> 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
> 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
> 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
> 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,
> 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
> 256,257,258,259,260,261,262,263,264,265,266,267,268,
> 272,273,274,275,276,277,278,279,280,281,282,283,284,
> 288,289,290,291,292,293,294,295,296,297,298,299,300,
> 304,305,306,307,308,309,310,311,312,313,314,315,316,
> 320,321,322,323,324,325,326,327,328,329,330,331,332,
> 336,337,338,339,340,341,342,343,344,345,346,347,348]}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="MSI_Packet_Secondary_Header"type="TypePacketData_MSI"/>
> <xs:element name="MSI_User_Data_Field"type="TypeUserData"/>
> </xs:sequence>
> <xs:sequence> <!-- Choice for TM_GPSR -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [769,771,772,774,775,777,779,780,781,785,787,788,790,791,793,795,796,797]}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_GPSR_Packet_Secondary_Header"type="TypePacketData_TMGPSR"/>
> <xs:element name="TM_GPSR_User_Data_Field"type="TypeUserData"/>
> </xs:sequence>
> <xs:sequence>
> <!-- Choice for TM_Time_packet(9,2), Overlaps with MSI APID 0x0 Removed here (moved to 99999) as it is SCIENCE config-->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminator
> test="{/Packet_Primary_Header/Packet_Identification/APID in [99999]}" />
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_Time_Packet_Secondary_Header"type="TypePacketData_TMT92"/>
> <xs:element name="TM_Time_Packet_User_Data_Field"type="TypeUserData_TMT92"/>
> </xs:sequence>
> <xs:sequence> <!-- Choice for TM_STR -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [596,598,612,614,628,630,660,662]}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_STR_Packet_Secondary_Header"type="TypePacketData_TMSTR"/>
> <xs:element name="TM_STR_User_Data_Field"type="TypeUserData"/>
> </xs:sequence>
> <xs:sequence> <!-- Choice for TM_CSW_AOCS -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [150,182]}" />
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_CSW_AOCS_Packet_Secondary_Header"type="TypePacketData_TMCSWAOCS"/>
> <xs:element name="TM_CSW_AOCS_User_Data_Field"type="TypeUserData"/>
> </xs:sequence>
> <xs:sequence>
> <!-- Choice for HKTM -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminator
> test="{/Packet_Primary_Header/Packet_Identification/APID inrange [144,149]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [151,181]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [183,239]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [516,575]
> or /Packet_Primary_Header/Packet_Identification/APID in [592,593,594,595,597,613,629,656,657,658,659,661]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [599,611]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [615,627]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [631,639]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [663,671]
> or /Packet_Primary_Header/Packet_Identification/APID in [768,770,776,778,782,783,784,792,794,798,799]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [1280,1295]}"
> />
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_HKTM_Packet_Secondary_Header"type="TypePacketData_HKTM"/>
> <xs:element name="TM_HKTM_User_Data_Field"type="TypeUserData_HKTM"/>
> </xs:sequence>
> <xs:sequence>
> <!-- Choice for TM_IDLE -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminator
> test="{/Packet_Primary_Header/Packet_Identification/APID in [2047]}"
> />
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_IDLE_Packet_Secondary_Header"type="TypePacketData_TMIDLE"
> />
> </xs:sequence>
> <xs:sequence> <!-- Choice for UNKNOWN -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{true}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="UNKNOWN_User_Data_Field"type="TypePacketData_UNKNOWN"/>
> </xs:sequence>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
>
>
> On 27 Oct 2015, at 18:01 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> Just to clarify, dfdl:assert and dfdl:discriminator are identical when the expression returns false, the difference between the two is when the expression returns true and the parser is in a 'point of uncertainty' (eg choice) - a discriminator returning true has a positive discrimination on the point of uncertainty, an assert does not.
>
> I suspect that most of the time you can achieve what you want with just dfdl:checkConstraints() and simple type inheritance. Example below (syntax simplified for clarity). But I am happy to be proved wrong, so if you have an example ... ?
>
> <xs:simpleType name="alertType">
> <xs:restriction base="xs:unsignedInt>
> <xs:minInclusive value="0"/>
> <xs:maxInclusive value="999"/>
> </xs:restriction>
> </xs:simpleType>
>
> <xs:simpleType name="redAlertType">
> <xs:restriction base="alertType">
> <xs:minInclusive value="0"/>
> <xs:maxInclusive value="200"/>
> </xs:restriction>
> </xs:simpleType>
>
> <xs:simpleType name="yellowAlertType">
> <xs:restriction base="alertType">
> <xs:minInclusive value="201"/>
> <xs:maxInclusive value="500"/>
> </xs:restriction>
> </xs:simpleType>
>
> ...
>
> <xs:choice>
> <xs:element name="red">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="alert" type="redAlertType">
> <dfdl:discriminator test="{dfdl:checkConstraints()}">
> </xs:element>
> ...
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="yellow">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="alert" type="yellowAlertType">
> <dfdl:discriminator test="{dfdl:checkConstraints()}">
> </xs:element>
> ...
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="other">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="alert" type="alertType">
> <dfdl:assert test="{dfdl:checkConstraints()}">
> </xs:element>
> ...
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
> mob:+44-7717-378890 <tel:%2B44-7717-378890>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 27/10/2015 16:27
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
> Yes, exactly. This is one of the current use case, in addition we could also use the value to
> implement different type of checks e.g. some of the parameters we look at have a type,
> a range outside of which they are invalid and a set of other ranges that correspond to various other conditions
> (e.g. red alarm or yellow alarm).
>
> Michele
>
>
>
>
> On 27 Oct 2015, at 17:19 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> "Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G
> TF schema version)."
>
> That sounds like you would use this in a dfdl:discriminator ?
>
> Yes, exactly. While this is one of the current use case, in addition we would also use the value to
> implement different type of checks e.g. some of the parameters we look at have a type,
> a range outside of which they are invalid and a set of other ranges that correspond to various other conditions
> (e.g. red alarm or yellow alarm).
>
> Michele
>
>
>
>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
> mob:+44-7717-378890 <tel:%2B44-7717-378890>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 27/10/2015 16:09
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
> Dear Steve,
>
> to progress on this action:
>
> 1) it is our intention to “evolve” from our own dmx:assert to the dfdl:assert in order to standardise as much as possible
> with the official spec provided we find a solution for what below.
>
> 2) we had an iteration with our developers and came to the conclusion that there is
> need for both syntaxes: the existing (in the standard) dfdl:checkConstraints()and the new
> one you proposed i.e. dfdl:checkRangeand dfdl:checkValue
>
> The main semantic difference between checkValue/checkRangeand checkConstraints
> is that in one case we we need to associate a range to a quantity in a sort of static way
> (i.e. the type “knowns” it has a range associated with it) while the checkvalue/checkrange
> is something that you can do also if a range is not defined (see below).
>
> For example you could have a definition for a specification sheet of an electrical device.
> The XSD type (Volts) intrinsically defined with a range 0-1000 but wanted
> to check in derived schemas if value is between 100 and 150 or 200 and 250 and this information
> would be used in different way.
>
> This could be used to check 0-1000 hard constraints to see if the the network will support it
> while range 100-150 and 200-250 would check compatibility with US or European Voltages for commercial appliances.
>
> Note that dfdl:assert applies a condition to the parsing so that if the check fails the parsing is aborted with a severe error.
> In the example above we would want to apply checkConstraint defined with a range 0-1000 in a dfdl:assert (to stop the parsing due to incorrect values) but not a checkvalue/checkrange.
> Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G
> TF schema version).
>
> Michele
>
>
>
> On 05 Oct 2015, at 16:23 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> Hi Michele
>
> Any update from your discussion?
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>
> Date: 23/09/2015 16:44
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
> Dear Steve,
>
> thanks for the suggestion (same from Mike) we need to discuss this internally with the developers and look at few use cases
> to see what would be the consequences/advantages/disadvantages.
>
> Give us a little bit of time..
>
> Michele
>
> On 22 Sep 2015, at 19:12 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> Hi Michele
>
> Thanks for your quick response. The WG discussed this on the call today.
>
> In fact, we wondered if the same result could already be achieved using dfdl:checkConstraints() and declaring the enums or range using XSD facets on xs:simpleType restrictions.
>
> Have you considered whether dfdl:checkConstraints() achieves what you want?
>
> Example:
>
> <xs:simpleType name="myRange">
> <xs:restriction base="xs:int>
> <xs:minInclusive value="100"/>
> <xs:maxInclusive value="200"/>
> </xs:restriction>
> </xs:simpleType>
>
> <xs:element name="myValue" type="myRange">
> <xs:annotation><xs:appinfo ...>
> <dfdl:assert>dfdl:checkConstraints(.)</dfdl:assert>
> </xs:appinfo></xs:annotation>
> </xs:element>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>
> Cc: Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>
> Date: 22/09/2015 17:21
> Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
> Sent by: dfdl-wg-bounces(a)ogf.org <mailto:dfdl-wg-bounces@ogf.org>
>
>
>
> Dear Steve,
>
> nice to hear things are moving.
>
> The syntax below seems both reasonable and readable to me which is a good thing.
>
> It addresses the belonging to decimal groups (integer only ??) I understand. (xs:decimal)
>
> 1) It might be also useful (not in our application but in general) to see if it makes sense also to define
> for dfdl:checkValues belongings to set of “enum” like value
>
> e.g. if we have a type for $val defined as (ON, OFF or STANDBY) or (MARRIED, SINGLE, WIDOWER)
> can we also define a syntax that the $node belongs to it ?
>
> 2) for dfdl:checkRange it might make sense also to allow float numbers e.g. X between 3.1 to 9.5
>
>
> Michele
>
>
>
> PS I will forward this proposal to our DFDL4S developers (in copy) to get their thinking.
>
>
> From: "Steve Hanson" <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>>
> Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
> Date: 22 Sep 2015 16:50:13 CEST
> To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>
>
>
> To kick start this action, here is a proposal ... builds on the precedent provided by dfdl:checkConstraints($node).
> dfdl:checkValues($node, $val1, $val2, ...) Returns boolean true if the specified node value matches any of the values provided by $val1 etc.
> The type of $val1 etc must be compatible with the type of $node.
>
> It is a schema definition error if the $node argument is a complex element.
>
> The number of value arguments is implementation-defined.
>
> dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive.
> The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal.
>
> It is a schema definition error if the $node argument is a complex element.
>
> dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive.
> The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal.
>
> It is a schema definition error if the $node argument is a complex element.
>
>
>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
>
>
>
> From: Steve Hanson/UK/IBM
> To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>
> Date: 11/08/2015 16:28
> Subject: Fw: [DFDL]: First Release of DFDL4S Parser Library
>
>
>
>
>
>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
> ----- Forwarded by Steve Hanson/UK/IBM on 11/08/2015 16:27 -----
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>
> Cc: Steve Hanson/UK/IBM@IBMGB, Maurizio De Bartolomei <Maurizio.De.Bartolomei(a)esa.int <mailto:Maurizio.De.Bartolomei@esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, "Rui Mestre (DME)" <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 18/05/2015 08:47
> Subject: Re: [DFDL]: First Release of DFDL4S Parser Library
>
>
>
> Thanks Mike,
>
> we will add this to our list to be considered/noted.
>
> However reading your explanation (NB I’m NOT at all an XPath expert) it seemed you
> had some good reason for avoiding longer than 1 path, so I would like to avoid our DFDL4S
> project results in an over-complication of the DFDL implementation/use of Xpath
> unless there are other reasons/users/rationale requiring this feature.
> (btw the syntax is still weird-ish: “intersect” reminds me of Venn Diagrams…)
>
> As a project manager I always evaluate solutions and their cost vs the benefit they bring,
> and I believe the DFDL community should keep this is mind.
>
> Michele
>
> PS The syntax above anser to the question “belongs to” , would there be any way to express ranges of values then ?
>
>
>
> On 15 May 2015, at 16:24 , Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> wrote:
>
> Just a few comments on DFDL4S, and also thank you to Michele and team for the presentation on Tuesday.
>
> I think all the issues in the spreadsheet are fairly easily fixed in that they are not major changes to DFDL4S, and would bring it into much closer compliance with the DFDL spec.
>
> The exception is the XPath limitations where DFDL4S has gone beyond what XPath 2.0 allows and invented new syntax for expressing set membership requirements.
>
> So I took a look, and XPath 2.0 has a set intersect operator: ns1 intersect ns2 => ns3
>
> This isn't in DFDL today, but might be usable to achieve the set membership test; however, it requires use of XPath node sequences of length greater than 1, which DFDL has avoided mostly. I say mostly as there are XPath expressions that return node sequences of length greater than 1 and those can be arguments to fn:count(...) for example.
>
> So far in DFDL such node sequences cannot "leak out" of the XPath expression into DFDL elements, and I think the usage in DFDL4S is similar in that these node sequences would be needed only to check for set membership, so the result is just a boolean as part of an assert/discriminator.
>
> We should examine whether XPath 2.0 set intersection is enough to meet the need.
>
> I believe the expressions would be something like:
>
> fn:exists( . intersect (123, 456, 789, .... many more items....) )
>
>
> - mike beckerle
>
>
> Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/>
> Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>
>
>
> On Tue, May 12, 2015 at 10:45 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote:
> for reference,
> here a summary of the reported problem in an excel sheet.
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> #### Sentinel2X-bandTMISPTypes.xsd moved to MyAttachments Repository V3.8 (Link <notes:///802575AF0030E827/5DE5236E5AD1645685256EE0001BBADF/784AE1CEA59C046380257EAB0027C2EF>) on 24 August 2015 by Steve Hanson.
>
>
>
> --
> dfdl-wg mailing list
> dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>
> https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
> --
> dfdl-wg mailing list
> dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>
> https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
> --
> dfdl-wg mailing list
> dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>
> https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg>
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-----------------------------------------
Michele Zundo
Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo(a)esa.int
This message and any attachments are intended for the use of the addressee or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete it from your system.
Emails can be altered and their integrity cannot be guaranteed by the sender.
Please consider the environment before printing this email.
1
0
clarification: textBooleanTrue/FalseRep cannot be empty string? textStandardNaNRep and friends cannot be empty string.
by Mike Beckerle 19 May '16
by Mike Beckerle 19 May '16
19 May '16
We don't stipulate that textBooleanTrueRep nor textBooleanFalseRep cannot
be empty string.
We do stipulate that entity ES is not allowed. So I expect these properties
cannot be "" meaning the same thing that "%ES;" would mean. Since we
disallow the latter, we should disallow just "".
Similarly, for textStandardGroupingSeparator, textStandardExponentRep,
textStandardNaNRep, and textStandardInfinityRep we specify that the string
literal value behaves as specified in textStandardDecimalSeparator, but we
don't specify that these cannot be empty string (which is specified in
textStandardDecimalSeparator, but not in the "Text Number Character
Restrictions" part of that property description that is referenced from
these other properties.)
ES and WSP* are not allowed, so almost certainly empty string should not be
allowed.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
<http://www.ogf.org/About/abt_policies.php>
2
1
clarifications needed?: dfdl:contentLength function and dfdl:valueLength function on empty and literal nil representations, and escaping
by Mike Beckerle 19 May '16
by Mike Beckerle 19 May '16
19 May '16
The dfdl:contentLength function is defined in terms of the SimpleContent or
ComplexContent regions of the grammar.
Let's just look at Simple types for a sec.
We do not specify what the dfdl:contentLength is for an element of
SimpleType which has the SimpleLiteralNilElementRep or
SimpleEmptyElementRep.
I suggest the value should be zero for SimpleEmptyElementRep. When parsing,
an empty element by definition has no content. The fact that a default
value might be inserted because of the empty representation should not
change the fact that there was no content. When unparsing,
SimpleEmptyElementRep can occur if an empty string is the value of a
string-valued element, or an empty byte array is the value of a hexBinary
element. The grammar is just stipulating the different treatment of
initiator/terminator for these special cases of empty things. The content
is length zero.
But consider the round-trip scenario. We parse data to the infoset. During
parsing the dfdl:contentLength of an element having SimpleEmptyElementRep
is zero. A default value is inserted. Now we unparse this same infoset. The
default value's representation very well may be SimpleNormalRep, with
non-zero dfdl:contentLength.
I claim this is ok. This is just another case where some data formats don't
round trip unchanged. It does add an implementation headache, which is if
the contentLength is cached on the infoset item, you need separate cache
locations to be used when parsing and when unparsing.
For SimpleLiteralNilElementRep, it should be the length of the
NilLiteralCharacters or NilElementLiteralContent regions. (Note: there's
the word "Content" implying that we think of the nil literal representation
as content. ) This applies to both parsing and unparsing.
For elements of complex type, I think for both ComplexLiteralNilElementRep
and ComplexEmptyElementRep, the dfdl:contentLength should be zero when
parsing. When unparsing, again a complex default may be created (because
default values for interior elements of the complex type might be filled in
as part of the augmented infoset.) and the dfdl:contentLength might not be
zero if these default values have non-zero content length. Again I think
this is ok.
For dfdl:contentLength, we should clarify that the length should also
include the contributions of any escape characters, escape-escape
characters, and escapeBlockStart/End characters. (This is implied, because
such characters are in the "value" regions of simple types, and value
regions are always contained in the content region, but I think the
clarification is still helpful.
Similarly we need to clarify what dfdl:valueLength does.
For SimpleEmptyElementRep the dfdl:valueLength should be zero.
For SimpleLiteralNilElementRep, the dfdl:valueLength should be zero,
because a nilled element has no value.
The corner case of SimpleLiteralNilElementRep for a nillable simple element
of type xs:string - since a literal nil representation and a string value
are ambiguous, should be handled by calling dfdl:contentLength instead of
dfdl:valueLength. So a nillable string element with literal nil
nilValue="nil", should have dfdl:valueLength of zero, but
dfdl:contentLength (in characters) of 3. Same element but not nilled,
containing the string "nil" as its value, would have dfdl:valueLength of 3
(characters), and dfdl:contentLength of 3 (characters).
For complex type elements, dfdl:valueLength is already defined to be the
same as dfdl:contentLength.
For elements that are not represented (that is, elements that have the
dfdl:inputValueCalc property on them), I believe both dfdl:valueLength and
dfdl:contentLength should cause an SDE, as this has to be an error on the
part of the schema author. (An argument can be made that these should
return zero however. See next paragraph.)
Note however that these functions can be called on elements of complex type
that contain elements that are not represented. Such contained
non-represented elements contribute zero to the content length in all
cases. (Consistency with this is why calling dfdl:valueLength or
dfdl:contentLength directly on a non-represented element might want to
return zero, instead of SDE.)
dfdl:valueLength is already specified to exclude the length of padding
characters that are trimmed/added.
I believe we should explicitly state that it *includes* the length of
escape, escape-escape, and escapeBlockStart/End characters.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
<http://www.ogf.org/About/abt_policies.php>
2
1
Sear Steve,
I will check with the team and come back to you.
> On 11 May 2016, at 9:55 , Steve Hanson <smh(a)uk.ibm.com> wrote:
>
> Michele & team
>
> Please join the DFDL WG call on 24th May so we can review the proposal and hopefully close this action.
>
> Thanks
>
> Steve Hanson
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848
> mob:+44-7717-378890
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int>, Montserrat Piñol <mpinol(a)eopp.esa.int>, Rui Mestre <rui.mestre(a)deimos.com.pt>
> Date: 12/04/2016 16:43
> Subject: Re: [DFDL-WG] Action 282
>
>
>
> Steve, I see this only now,
>
> we will not have time this nor the coming week, as I want to discuss the proposal internally.
>
> We will review the proposal below and come back to you.
>
> Michele
>
>
>
>
>
> On 11 Apr 2016, at 10:47 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
> 282
> Does XPath have operators for checking a value is in a range? (Steve)
> 12/5: Investigate whether equivalent to DFDL4S 'in' operator exists.
> 23/6: Mike has found an XPath 'intersect' operator. Handles the enumeration case well, but not as convenient for ranges as DFDL4S's 'in' operator.
> 11/8: Looked back at the motivating example from DFDL4S. Agreed that DFDL functions to do the equivalent of 'in' and 'inrange' would be useful if nothing can be re-used from XPath. Steve to write up a proposal taking into account different data types.
> ...
> 22/9: Proposal sent by Steve for new functions dfdl:checkValues(), dfdl:checkRangeInclusive(), dfdl:checkRangeExclusive(). Discussed whether both range functions needed, and whether they should be allowed for float and double. Mike noted that dfdl:checkConstraints() and simple types could be used instead of all three functions if values were static. Follow-up with DFDL4S team to see if they had thought of that.
> 3/11: DFDL4S not happy with usability of dfdl:checkConstraints type, but ok with the intersection operator. Next step is to see what the DFDL4S schema would look like if rewritten to use dfdl:checkConstraints and intersection.
> 5/1/16: Agreed that dfdl:checkConstraints() is not ideal as it requires creation of union simple types. Steve to go back to his proposal from 22/9/15 and rework to include use of the intersect operator.
> 16/2: No progress
> 1/3: Discussed with ESA team. Steve to rework his proposal from 22/9/15.
>
> Reworked proposal:
>
> a) Add XPath 2.0 'intersect' and "except" operators to the list of supported operators.
>
> Updated Table 57 as follows:
> DFDL Expression ::= "{" Expr "}"
> Expr ::= ExprSingle
> ExprSingle ::= IfExpr
> | OrExpr
> IfExpr ::= "if" "(" Expr ")" "then" ExprSingle "else" ExprSingle
> OrExpr ::= AndExpr ( "or" AndExpr )*
> AndExpr ::= ComparisonExpr ( "and" ComparisonExpr )*
> ComparisonExpr ::= AdditiveExpr ( (ValueComp) AdditiveExpr)?
> AdditiveExpr ::= MultiplicativeExpr ( ("+" | "-") MultiplicativeExpr )*
> MultiplicativeExpr ::= IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ( ("*" | "div" | "idiv" | "mod") IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> )*
> IntersectExceptExpr <https://www.w3.org/TR/xpath20/#prod-xpath-IntersectExceptExpr> ::= UnaryExpr <https://www.w3.org/TR/xpath20/#doc-xpath-InstanceofExpr> ( ("intersect" | "except") UnaryExpr )*
> UnaryExpr ::= ("-" | "+")* ValueExpr
> ValueExpr ::= PathExpr
> ValueComp ::= "eq" | "ne" | "lt" | "le" | "gt" | "ge"
> PathExpr ::= ("/" RelativePathExpr?)
> | RelativePathExpr | FilterExpr
> RelativePathExpr ::= StepExpr (("/") StepExpr)*
> StepExpr ::= AxisStep
> AxisStep ::= (ReverseStep | ForwardStep) Predicate?
> ForwardStep ::= (ForwardAxis NodeTest) | AbbrevForwardStep
> ForwardAxis ::= ("child" "::")
> | ("self" "::")
> AbbrevForwardStep ::= NodeTest | ContextItemExpr
> ReverseStep ::= (ReverseAxis NodeTest) | AbbrevReverseStep
> ReverseAxis ::= ("parent" "::")
> AbbrevReverseStep ::= ".."
> NodeTest ::= NameTest
> NameTest ::= QName
> FilterExpr ::= PrimaryExpr Predicate?
> Predicate ::= "[" Expr "]"
> PrimaryExpr ::= Literal | VarRef | ParenthesizedExpr | ContextItemExpr | FunctionCall
> Literal ::= NumericLiteral | StringLiteral
> NumericLiteral ::= IntegerLiteral | DecimalLiteral | DoubleLiteral
> VarRef ::= "$" VarName
> VarName ::= QName
> ParenthesizedExpr ::= "(" Expr ")"
> ContextItemExpr ::= "."
> FunctionCall ::= QName "(" (ExprSingle ("," ExprSingle)*)? ")"
>
>
>
>
> b) New DFDL functions
>
> dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive.
> The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double.
>
> It is a schema definition error if the $node argument is a complex element.
>
> dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive.
> The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal, xs:float or xs:double.
>
> It is a schema definition error if the $node argument is a complex element.
>
>
>
>
> Regards
>
> Steve Hanson
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:+44-1962-815848>
> mob:+44-7717-378890
>
>
>
> From: Steve Hanson/UK/IBM
> To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> Cc: "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 02/11/2015 19:51
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
>
> <xs:simpleType name="redAndYellowAlertType">
> <xs:union memberTypes="redAlertType yellowAlertType"/>
> </xs:simpleType>
>
> The above is a type that allows ranges 0-200 and 201-500.
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:+44-1962-815848>
> mob:+44-7717-378890
>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 02/11/2015 16:08
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
> Steve,
>
> I believe I do not understand this.
>
> can you make an example ?
>
> Michele
>
> On 02 Nov 2015, at 16:33 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> I was also going to add that DFDL supports xs:union on simple types, enabling dfdl:checkConstraints to be used when there are multiple ranges.
>
> · Unions; the memberTypes must be derived from the same simple type. DFDL annotations are not permitted on union members.
>
> The purpose of unions is to allow multiple constraints via facets such as multiple independent range restrictions on numbers. This enhances the ability to do rich validation of data.
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:+44-1962-815848>
> mob:+44-7717-378890
>
>
>
> From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>
> To: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> Cc: Steve Hanson/UK/IBM@IBMGB, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, "dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>" <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>
> Date: 02/11/2015 14:33
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
>
> Michele,
>
> I think the first thing I'd propose for the set membership test is for us to add the XPath 2.0 intersect operator to DFDL. So your discriminator becomes:
>
> <dfdl:discriminatortest="{fn:exists(/Packet_Primary_Header/Packet_Identification/APID intersect (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
> 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
> 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
> 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
> 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,
> 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
> 256,257,258,259,260,261,262,263,264,265,266,267,268,
> 272,273,274,275,276,277,278,279,280,281,282,283,284,
> 288,289,290,291,292,293,294,295,296,297,298,299,300,
> 304,305,306,307,308,309,310,311,312,313,314,315,316,
> 320,321,322,323,324,325,326,327,328,329,330,331,332,
> 336,337,338,339,340,341,342,343,344,345,346,347,348))}"/>
>
>
> This is a small addition to DFDL, but completely in the spirit of sticking with XPath 2.0 wherever possible.
>
> ...mike beckerle
>
>
>
>
> Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/>
> Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>
>
>
> On Wed, Oct 28, 2015 at 7:22 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote:
> Hi Steve,
>
> your example below addresses the belonging to a range, how would then the
> belonging to a set be checked ? Currently our usage looks like here:
>
>
>
> <xs:complexTypename="TypePacketData">
> <xs:sequence>
> <xs:choice>
> <xs:sequence> <!-- Choice for MSI -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
> 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
> 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
> 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
> 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,
> 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
> 256,257,258,259,260,261,262,263,264,265,266,267,268,
> 272,273,274,275,276,277,278,279,280,281,282,283,284,
> 288,289,290,291,292,293,294,295,296,297,298,299,300,
> 304,305,306,307,308,309,310,311,312,313,314,315,316,
> 320,321,322,323,324,325,326,327,328,329,330,331,332,
> 336,337,338,339,340,341,342,343,344,345,346,347,348]}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="MSI_Packet_Secondary_Header"type="TypePacketData_MSI"/>
> <xs:element name="MSI_User_Data_Field"type="TypeUserData"/>
> </xs:sequence>
> <xs:sequence> <!-- Choice for TM_GPSR -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [769,771,772,774,775,777,779,780,781,785,787,788,790,791,793,795,796,797]}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_GPSR_Packet_Secondary_Header"type="TypePacketData_TMGPSR"/>
> <xs:element name="TM_GPSR_User_Data_Field"type="TypeUserData"/>
> </xs:sequence>
> <xs:sequence>
> <!-- Choice for TM_Time_packet(9,2), Overlaps with MSI APID 0x0 Removed here (moved to 99999) as it is SCIENCE config-->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminator
> test="{/Packet_Primary_Header/Packet_Identification/APID in [99999]}" />
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_Time_Packet_Secondary_Header"type="TypePacketData_TMT92"/>
> <xs:element name="TM_Time_Packet_User_Data_Field"type="TypeUserData_TMT92"/>
> </xs:sequence>
> <xs:sequence> <!-- Choice for TM_STR -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [596,598,612,614,628,630,660,662]}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_STR_Packet_Secondary_Header"type="TypePacketData_TMSTR"/>
> <xs:element name="TM_STR_User_Data_Field"type="TypeUserData"/>
> </xs:sequence>
> <xs:sequence> <!-- Choice for TM_CSW_AOCS -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{/Packet_Primary_Header/Packet_Identification/APID in [150,182]}" />
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_CSW_AOCS_Packet_Secondary_Header"type="TypePacketData_TMCSWAOCS"/>
> <xs:element name="TM_CSW_AOCS_User_Data_Field"type="TypeUserData"/>
> </xs:sequence>
> <xs:sequence>
> <!-- Choice for HKTM -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminator
> test="{/Packet_Primary_Header/Packet_Identification/APID inrange [144,149]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [151,181]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [183,239]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [516,575]
> or /Packet_Primary_Header/Packet_Identification/APID in [592,593,594,595,597,613,629,656,657,658,659,661]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [599,611]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [615,627]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [631,639]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [663,671]
> or /Packet_Primary_Header/Packet_Identification/APID in [768,770,776,778,782,783,784,792,794,798,799]
> or /Packet_Primary_Header/Packet_Identification/APID inrange [1280,1295]}"
> />
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_HKTM_Packet_Secondary_Header"type="TypePacketData_HKTM"/>
> <xs:element name="TM_HKTM_User_Data_Field"type="TypeUserData_HKTM"/>
> </xs:sequence>
> <xs:sequence>
> <!-- Choice for TM_IDLE -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminator
> test="{/Packet_Primary_Header/Packet_Identification/APID in [2047]}"
> />
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="TM_IDLE_Packet_Secondary_Header"type="TypePacketData_TMIDLE"
> />
> </xs:sequence>
> <xs:sequence> <!-- Choice for UNKNOWN -->
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/dfdl-1.0/ <http://www.ogf.org/dfdl/dfdl-1.0/>">
> <dfdl:discriminatortest="{true}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:element name="UNKNOWN_User_Data_Field"type="TypePacketData_UNKNOWN"/>
> </xs:sequence>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
>
>
> On 27 Oct 2015, at 18:01 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> Just to clarify, dfdl:assert and dfdl:discriminator are identical when the expression returns false, the difference between the two is when the expression returns true and the parser is in a 'point of uncertainty' (eg choice) - a discriminator returning true has a positive discrimination on the point of uncertainty, an assert does not.
>
> I suspect that most of the time you can achieve what you want with just dfdl:checkConstraints() and simple type inheritance. Example below (syntax simplified for clarity). But I am happy to be proved wrong, so if you have an example ... ?
>
> <xs:simpleType name="alertType">
> <xs:restriction base="xs:unsignedInt>
> <xs:minInclusive value="0"/>
> <xs:maxInclusive value="999"/>
> </xs:restriction>
> </xs:simpleType>
>
> <xs:simpleType name="redAlertType">
> <xs:restriction base="alertType">
> <xs:minInclusive value="0"/>
> <xs:maxInclusive value="200"/>
> </xs:restriction>
> </xs:simpleType>
>
> <xs:simpleType name="yellowAlertType">
> <xs:restriction base="alertType">
> <xs:minInclusive value="201"/>
> <xs:maxInclusive value="500"/>
> </xs:restriction>
> </xs:simpleType>
>
> ...
>
> <xs:choice>
> <xs:element name="red">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="alert" type="redAlertType">
> <dfdl:discriminator test="{dfdl:checkConstraints()}">
> </xs:element>
> ...
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="yellow">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="alert" type="yellowAlertType">
> <dfdl:discriminator test="{dfdl:checkConstraints()}">
> </xs:element>
> ...
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="other">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="alert" type="alertType">
> <dfdl:assert test="{dfdl:checkConstraints()}">
> </xs:element>
> ...
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
> mob:+44-7717-378890 <tel:%2B44-7717-378890>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 27/10/2015 16:27
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
> Yes, exactly. This is one of the current use case, in addition we could also use the value to
> implement different type of checks e.g. some of the parameters we look at have a type,
> a range outside of which they are invalid and a set of other ranges that correspond to various other conditions
> (e.g. red alarm or yellow alarm).
>
> Michele
>
>
>
>
> On 27 Oct 2015, at 17:19 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> "Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G
> TF schema version)."
>
> That sounds like you would use this in a dfdl:discriminator ?
>
> Yes, exactly. While this is one of the current use case, in addition we would also use the value to
> implement different type of checks e.g. some of the parameters we look at have a type,
> a range outside of which they are invalid and a set of other ranges that correspond to various other conditions
> (e.g. red alarm or yellow alarm).
>
> Michele
>
>
>
>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM Integration Bus <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
> mob:+44-7717-378890 <tel:%2B44-7717-378890>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 27/10/2015 16:09
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
> Dear Steve,
>
> to progress on this action:
>
> 1) it is our intention to “evolve” from our own dmx:assert to the dfdl:assert in order to standardise as much as possible
> with the official spec provided we find a solution for what below.
>
> 2) we had an iteration with our developers and came to the conclusion that there is
> need for both syntaxes: the existing (in the standard) dfdl:checkConstraints()and the new
> one you proposed i.e. dfdl:checkRangeand dfdl:checkValue
>
> The main semantic difference between checkValue/checkRangeand checkConstraints
> is that in one case we we need to associate a range to a quantity in a sort of static way
> (i.e. the type “knowns” it has a range associated with it) while the checkvalue/checkrange
> is something that you can do also if a range is not defined (see below).
>
> For example you could have a definition for a specification sheet of an electrical device.
> The XSD type (Volts) intrinsically defined with a range 0-1000 but wanted
> to check in derived schemas if value is between 100 and 150 or 200 and 250 and this information
> would be used in different way.
>
> This could be used to check 0-1000 hard constraints to see if the the network will support it
> while range 100-150 and 200-250 would check compatibility with US or European Voltages for commercial appliances.
>
> Note that dfdl:assert applies a condition to the parsing so that if the check fails the parsing is aborted with a severe error.
> In the example above we would want to apply checkConstraint defined with a range 0-1000 in a dfdl:assert (to stop the parsing due to incorrect values) but not a checkvalue/checkrange.
> Instead the checkvalue/checkrange (between 100 and 150 or 200 and 250) would be used to do conditional checks in derived schemas outside the scope of a dfdl:assert (to guide the parsing among several options, like we do for the our S2G
> TF schema version).
>
> Michele
>
>
>
> On 05 Oct 2015, at 16:23 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> Hi Michele
>
> Any update from your discussion?
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Steve Hanson/UK/IBM@IBMGB
> Cc: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>, Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>
> Date: 23/09/2015 16:44
> Subject: Re: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
>
>
>
> Dear Steve,
>
> thanks for the suggestion (same from Mike) we need to discuss this internally with the developers and look at few use cases
> to see what would be the consequences/advantages/disadvantages.
>
> Give us a little bit of time..
>
> Michele
>
> On 22 Sep 2015, at 19:12 , Steve Hanson <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>> wrote:
>
> Hi Michele
>
> Thanks for your quick response. The WG discussed this on the call today.
>
> In fact, we wondered if the same result could already be achieved using dfdl:checkConstraints() and declaring the enums or range using XSD facets on xs:simpleType restrictions.
>
> Have you considered whether dfdl:checkConstraints() achieves what you want?
>
> Example:
>
> <xs:simpleType name="myRange">
> <xs:restriction base="xs:int>
> <xs:minInclusive value="100"/>
> <xs:maxInclusive value="200"/>
> </xs:restriction>
> </xs:simpleType>
>
> <xs:element name="myValue" type="myRange">
> <xs:annotation><xs:appinfo ...>
> <dfdl:assert>dfdl:checkConstraints(.)</dfdl:assert>
> </xs:appinfo></xs:annotation>
> </xs:element>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
>
>
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>
> Cc: Rui Mestre <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, Maurizio De Bartolomei <mdebartolomei(a)eopp.esa.int <mailto:mdebartolomei@eopp.esa.int>>
> Date: 22/09/2015 17:21
> Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
> Sent by: dfdl-wg-bounces(a)ogf.org <mailto:dfdl-wg-bounces@ogf.org>
>
>
>
> Dear Steve,
>
> nice to hear things are moving.
>
> The syntax below seems both reasonable and readable to me which is a good thing.
>
> It addresses the belonging to decimal groups (integer only ??) I understand. (xs:decimal)
>
> 1) It might be also useful (not in our application but in general) to see if it makes sense also to define
> for dfdl:checkValues belongings to set of “enum” like value
>
> e.g. if we have a type for $val defined as (ON, OFF or STANDBY) or (MARRIED, SINGLE, WIDOWER)
> can we also define a syntax that the $node belongs to it ?
>
> 2) for dfdl:checkRange it might make sense also to allow float numbers e.g. X between 3.1 to 9.5
>
>
> Michele
>
>
>
> PS I will forward this proposal to our DFDL4S developers (in copy) to get their thinking.
>
>
> From: "Steve Hanson" <smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>>
> Subject: [DFDL-WG] Action 282 (was Re: Fw: [DFDL]: First Release of DFDL4S Parser Library)
> Date: 22 Sep 2015 16:50:13 CEST
> To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>
>
>
> To kick start this action, here is a proposal ... builds on the precedent provided by dfdl:checkConstraints($node).
> dfdl:checkValues($node, $val1, $val2, ...) Returns boolean true if the specified node value matches any of the values provided by $val1 etc.
> The type of $val1 etc must be compatible with the type of $node.
>
> It is a schema definition error if the $node argument is a complex element.
>
> The number of value arguments is implementation-defined.
>
> dfdl:checkRangeInclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, inclusive.
> The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal.
>
> It is a schema definition error if the $node argument is a complex element.
>
> dfdl:checkRangeExclusive($node, $val1, $val2) Returns boolean true if the specified node value is in the range given by $val1 and $val2, exclusive.
> The type of $val1 and $val2 must be compatible with the type of $node, and must be a derivative of xs:decimal.
>
> It is a schema definition error if the $node argument is a complex element.
>
>
>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
>
>
>
> From: Steve Hanson/UK/IBM
> To: DFDL-WG <dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>>
> Date: 11/08/2015 16:28
> Subject: Fw: [DFDL]: First Release of DFDL4S Parser Library
>
>
>
>
>
>
>
> Regards
>
> Steve Hanson
> Architect, IBM DFDL <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, OGF DFDL Working Group <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK
> smh(a)uk.ibm.com <mailto:smh@uk.ibm.com>
> tel:+44-1962-815848 <tel:%2B44-1962-815848>
> ----- Forwarded by Steve Hanson/UK/IBM on 11/08/2015 16:27 -----
>
> From: Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>>
> To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>>
> Cc: Steve Hanson/UK/IBM@IBMGB, Maurizio De Bartolomei <Maurizio.De.Bartolomei(a)esa.int <mailto:Maurizio.De.Bartolomei@esa.int>>, Montserrat Piñol <mpinol(a)eopp.esa.int <mailto:mpinol@eopp.esa.int>>, "Rui Mestre (DME)" <rui.mestre(a)deimos.com.pt <mailto:rui.mestre@deimos.com.pt>>
> Date: 18/05/2015 08:47
> Subject: Re: [DFDL]: First Release of DFDL4S Parser Library
>
>
>
> Thanks Mike,
>
> we will add this to our list to be considered/noted.
>
> However reading your explanation (NB I’m NOT at all an XPath expert) it seemed you
> had some good reason for avoiding longer than 1 path, so I would like to avoid our DFDL4S
> project results in an over-complication of the DFDL implementation/use of Xpath
> unless there are other reasons/users/rationale requiring this feature.
> (btw the syntax is still weird-ish: “intersect” reminds me of Venn Diagrams…)
>
> As a project manager I always evaluate solutions and their cost vs the benefit they bring,
> and I believe the DFDL community should keep this is mind.
>
> Michele
>
> PS The syntax above anser to the question “belongs to” , would there be any way to express ranges of values then ?
>
>
>
> On 15 May 2015, at 16:24 , Mike Beckerle <mbeckerle.dfdl(a)gmail.com <mailto:mbeckerle.dfdl@gmail.com>> wrote:
>
> Just a few comments on DFDL4S, and also thank you to Michele and team for the presentation on Tuesday.
>
> I think all the issues in the spreadsheet are fairly easily fixed in that they are not major changes to DFDL4S, and would bring it into much closer compliance with the DFDL spec.
>
> The exception is the XPath limitations where DFDL4S has gone beyond what XPath 2.0 allows and invented new syntax for expressing set membership requirements.
>
> So I took a look, and XPath 2.0 has a set intersect operator: ns1 intersect ns2 => ns3
>
> This isn't in DFDL today, but might be usable to achieve the set membership test; however, it requires use of XPath node sequences of length greater than 1, which DFDL has avoided mostly. I say mostly as there are XPath expressions that return node sequences of length greater than 1 and those can be arguments to fn:count(...) for example.
>
> So far in DFDL such node sequences cannot "leak out" of the XPath expression into DFDL elements, and I think the usage in DFDL4S is similar in that these node sequences would be needed only to check for set membership, so the result is just a boolean as part of an assert/discriminator.
>
> We should examine whether XPath 2.0 set intersection is enough to meet the need.
>
> I believe the expressions would be something like:
>
> fn:exists( . intersect (123, 456, 789, .... many more items....) )
>
>
> - mike beckerle
>
>
> Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com <http://www.tresys.com/>
> Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>
>
>
> On Tue, May 12, 2015 at 10:45 AM, Michele Zundo <michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>> wrote:
> for reference,
> here a summary of the reported problem in an excel sheet.
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> #### Sentinel2X-bandTMISPTypes.xsd moved to MyAttachments Repository V3.8 (Link <notes:///802575AF0030E827/5DE5236E5AD1645685256EE0001BBADF/784AE1CEA59C046380257EAB0027C2EF>) on 24 August 2015 by Steve Hanson.
>
>
>
> --
> dfdl-wg mailing list
> dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>
> https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
> --
> dfdl-wg mailing list
> dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>
> https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
> --
> dfdl-wg mailing list
> dfdl-wg(a)ogf.org <mailto:dfdl-wg@ogf.org>
> https://www.ogf.org/mailman/listinfo/dfdl-wg <https://www.ogf.org/mailman/listinfo/dfdl-wg>
>
>
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
> -----------------------------------------
> Michele Zundo
>
> Head of Ground System Definition and Verification Office
> EOP-PEP
> European Space Agency, ESTEC
> e-mail: michele.zundo(a)esa.int <mailto:michele.zundo@esa.int>
>
>
>
>
>
>
>
>
> This message and any attachments are intended for the use of the addressee or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the sender.
>
> Please consider the environment before printing this email.
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-----------------------------------------
Michele Zundo
Head of Ground System Definition and Verification Office
EOP-PEP
European Space Agency, ESTEC
e-mail: michele.zundo(a)esa.int
This message and any attachments are intended for the use of the addressee or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete it from your system.
Emails can be altered and their integrity cannot be guaranteed by the sender.
Please consider the environment before printing this email.
1
0