After checking with an XPath expert about the use of ContextItemExpr, here's a consolidated proposal for the EBNF changes. Please review for next WG call: PathExpr ::= (("/" RelativePathExpr?) | RelativePathExpr) | FilterExpr StepExpr ::= AxisStep AxisStep ::= (ReverseStep | ForwardStep) Predicate? FilterExpr ::= PrimaryExpr Predicate? AbbrevForwardStep ::= NodeTest | ContextItemExpr ParenthesizedExpr ::= "(" Expr ")" Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 11/02/2013 12:17 ----- From: Andrew Coleman/UK/IBM To: Steve Hanson/UK/IBM@IBMGB, Date: 11/02/2013 12:11 Subject: Re: Fw: [DFDL-WG] what do we allow in DFDL expressions after "/" Hi Steve, Yes, that change looks fine. It's hard to say why the original XPath grammar didn't define '.' as part of AbbrevForwardStep. There is often more than one way of specifying a production rule. Although a '.' in the middle of a path expression makes no difference to the result, it is useful to be able to tolerate this syntax in case the expression has been created by a naive code generator. Best regards, - Andy __________________________________________ Andrew Coleman WebSphere Message Broker Development IBM Hursley Park From: Steve Hanson/UK/IBM To: Andrew Coleman/UK/IBM@IBMGB, Date: 11/02/2013 09:41 Subject: Fw: [DFDL-WG] what do we allow in DFDL expressions after "/" Hi Andy Please can I trouble you for advice on an XPath 2.0 question? As you know DFDL allows a subset of XPath 2.0. In the DFDL spec we took the EBNF from the XPath 2.0 spec and simplified it for our needs (go to http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/index.jsp and search for 'relativepathexpr'). However it has been pointed out that what we have still allows too much, so we want to simplify it further. Specifically the term FilterExpr can appear in the middle of a RelativePathExpr because of the term StepExpr. A proposal is to only allow FilterExpr as the last term in a path, but that means we couldn't use ./xxx because the use of . (ContextItemExpr) is not part of AbbrevForwardStep, but only part of PrimaryExpr and hence FilterExpr. So it was suggested that the change directly below was made. AbbrevForwardStep ::= NodeTest | ContextItemExpr My question is, why didn't XPath 2.0 allow this in the first place? I am wondering if there is some subtlety going on that we break if we change AbbrevForwardStep in this manner. Thoughts? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 11/02/2013 09:24 ----- From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl@gmail.com>, Cc: dfdl-wg@ogf.org Date: 07/02/2013 11:50 Subject: Re: [DFDL-WG] what do we allow in DFDL expressions after "/" Agree that Predicate is optional, and that is the case in XPath 2.0. The bug crept into our spec because we removed an intermediate construction PredicateList which is defined as Predicate*. However I would prefer that we marked the optionality on the user of Predicate rather than on Predicate itself. AxisStep ::= (ReverseStep | ForwardStep) Predicate* PrimaryExpr returns a sequence, so you need to allow a predicate. (DFDL does not allow a sequence to be returned from a DFDL expression, but they can appear in internal stages of an expression). Also, we need to allow .[n]. FilterExpr ::= PrimaryExpr Predicate* IBM DFDL has many tests where we use ./foo and I am sure that will be being used by users. It makes it clear that the path is relative. To handle this, like you I considered adding ContextItemExpr to AbbrevForwardStep, as syntactically that seems to work, but then I wondered why XPath hadn't done it that way. I'd like to check with our XPath expert Andy Coleman to see whether there is some subtlety going on around contexts. Is there any harm in allowing () ? I'd leave that as it is. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: dfdl-wg@ogf.org Date: 06/02/2013 18:13 Subject: Re: [DFDL-WG] what do we allow in DFDL expressions after "/" The fact that in StepExpr a FilterExpr is an alternative to AxisStep is the big problem with this grammar. It allows many things to show up as path steps that we don't want to allow. At minimum, we need to change the FilterExpr production to drop Predicate, as I believe in no context do we allow such a predicate. No-op path steps like foo/./bar or ./bar or foo/. are also legal in XPath syntax technically, but I see no reason for us to allow them, and this is what we get if we simply state that in DFDL FilterExpr cannot appear inside a multi-step PathExpr. If we feel we must allow these, then I would just add ContextItemExpr as an alternative to NodeTest in AbbrevForwardStep. So, we can leave the rest of the productions as is, but with a clarifying note that FilterExpr in DFDL is limited i.e., that they cannot be used inside multi-step paths as the grammar would imply. Rather, they must appear in isolation (no "/" on either side). However, this grammar structure is very unmotivated in that case, restructuring the StepExpr and PathExpr as you have suggested below is a better choice as far as I am concerned. And we can highlight somehow (I suggest bold font) the productions or changes to productions that make it different from the XPath grammar. So here's the contemplated set of changes: drop FilterExpr as an alternative StepExpr ::= AxisStep add FilterExpr as an alternative (or we could just add PrimaryExpr instead) PathExpr ::= (("/" RelativePathExpr?) | RelativePathExpr) | FilterExpr drop predicate possibility FilterExpr ::= PrimaryExpr add "." as available forward step (I am ambivalent about this one, but I'll admit I have written "./foo" before) AbbrevForwardStep ::= NodeTest | ContextItemExpr I also think there is a bug in the grammar. Right now AxisStep requires a Predicate. The Predicate should be optional. The original XPath grammar uses this PredicateList thing to deal with zero or more predicates. We want zero or 1 predicates, Simplest to perhaps do Predicate ::= ("[" Expr "]")? The ? makes it optional. There is also a typo in the ReverseAxis production (dangling ") at the end. Lastly, do we allow () as an expression? Right now the grammar allows this, because ParenthesizedExpr is "(" Expr? ")" but I'm not sure () is meaningful in DFDL except as part of a function call with no arguments. i.e., foo() which is allowed by a separate production for function call syntax. On Wed, Feb 6, 2013 at 5:05 AM, Steve Hanson <smh@uk.ibm.com> wrote: Mike, More correctly, you mean Step[Index]/Step[Index]/Step[Index], but that's fine as that is handled by the definition of AxisStep as it includes Predicate. However removing FilterExpr entirely removes PrimaryExpr and all it brings with it, like literals and function calls and the use of "." so your change on its own simplifies things too much. What I think you would need is: StepExpr ::= FilterExpr | AxisStep PathExpr ::= (("/" RelativePathExpr?) | RelativePathExpr) | FilterExpr I can see (at least) one problem with only allowing FilterExpr to appear on its own, namely that you lose the ability to use "." in conjunction with a path, so even the above is not sufficient. Looking at what we have done in the original spec, all we did was edit the BNF to remove things we didn't support, making what resulted easily comparable to standard XPath 2.0. The resultant BNF implies several things that we do not support in DFDL, so we qualified the BNF with some notes. For example, DFDL only allows the use of Predicates to index arrays, so there is a note saying that a Predicate must result in an integer else it's an SDE. I think we should do the same for errata 2.101 - leave the BNF alone and add a new note to the list. And beef up the words at the start of section 23.4 to make it clear how the BNF should be read. Also noticed that the BNF in the spec is not stand-alone as the constructs for StringLiteral etc are not reproduced, and rely on the reader reading XPath 2.0 spec. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: dfdl-wg@ogf.org, Date: 05/02/2013 23:08 Subject: [DFDL-WG] what do we allow in DFDL expressions after "/" Sent by: dfdl-wg-bounces@ogf.org In reexamining the grammar for the DFDL expression language, I have the following question In a path Step/Step/Step[Index] What can Step be? I know it can be ".." or parent::QName or child::QName or just an NCName or QName, Those are all what the XPath grammar calls AxisStep. anything else? The grammar can be changed in a very small way from the original XPath grammar if the above are the only possibilities. the clause StepExpr ::= FilterExpr | AxisStep can just be changed to StepExpr ::= AxisStep -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU