[fix] unary lookups #3966
[fix] unary lookups #3966
Conversation
|
With this PR applied running XQTS results in
|
|
Kudos, SonarCloud Quality Gate passed! |
|
@line-o Out of interest, are these results from the W3C fixed XQTS or the living one on GitHub? |
|
@adamretter the above results are from a run against the HEAD of XQTS (-x HEAD) |
|
@line-o Before your change, for 5.3.0 release I see:
Did you refresh your version of XQTS from GitHub? I ask because we have different |
|
To ensure the latest xqts was downloaded afresh, I ran
Comparing these 5.3.0 results to @line-o's results for this PR, we can see that with this PR, eXist passes 15 more tests than before, has 3 fewer errors, and its success rate has improved by .04%. |
|
Interesting since I saw 31700+ Tests on one run but then it went down to 31557 on the subsequent run. I have no idea what could cause this. @joewiz the total amount of tests went down to 31557 after |
Okay, that's interesting, pretty sure I did the same, let me try again.. @joewiz Also just to confirm, which version of eXist-db is set in your |
|
I did change the version to |
|
@adamretter In my |
|
I had moved the first test result in a separate folder and had a look
|
@joewiz Likewise :-) So I ran it again, I now at least get the same number of tests; I must have inadvertently had some old data before - but I can't see how! I am on the HEAD of master of XQTS, and ran with My yield is (with
|
|
@dizzzz Could you please restart Appveyor when you have the chance? |
Can you show before and after figures please? |
|
Besides running XQTS on the PR and its parent commit, I wrote a query that tells us which test cases actually changed. I'll share XQTS aggregate results, then dig into the specific changes. TL/DR: The PR causes 4 tests that previously failed to pass and 3 tests that previously failed to error. XQTS aggregate resultsI ran exist-xqts-runner 3 times for each build, using the methodology developed in eXist-db/exist-xqts-runner#19. I am on macOS 11.5.1 with OpenJDK 1.8.0_302 (liberica-jdk8-full) and sbt 1.5.5. BaselineWith the
With the commit just before this PR (d064fc4):
So the baseline is pretty clear from the above. We have +/- 1 failure in 5.3.0 and the 5.4.0-SNAPSHOT commit that was this PR's parent. This PRHere are the results with this PR (checking out this PR's branch):
All the aggregate results tell us is that:
Missing is which tests previous failed and are now passing, which results are now generating errors, and what the tests now producing errors were returning before. Analysis of changes in test resultsTo meaningfully analyze the test results, we have to compare which test results changed and what the actual changes were. Here's what appears to be the actual changes between the PR's parent and the PR:
The 4 cases of fails that are now pass are unambiguous improvements. Thus, I won't dig into them - but they are clearly a major portion of the PR's contribution. Let's examine the 3 cases where failures turned into errors: UnaryLookup-011Test: <test-case name="UnaryLookup-011">
<description>Unary lookup with no context item</description>
<created by="Michael Kay" on="2014-11-27"/>
<dependency type="spec" value="XP31+ XQ31+"/>
<dependency type="feature" value="higherOrderFunctions"/>
<test>let $d := function($x) {$x + ?2} return $d(12)</test>
<result>
<error code="XPDY0002"/>
</result>
</test-case>Before:
After:
Stacktrace: UnaryLookup-016Test: <test-case name="UnaryLookup-016">
<description>No syntactic confusion with place-markers</description>
<created by="Michael Kay" on="2014-11-27"/>
<dependency type="spec" value="XP31+ XQ31+"/>
<dependency type="feature" value="higherOrderFunctions"/>
<test>(['a', 'b', 'c'], ['b', 'c', 'd'], ['e', 'f', 'b'])[contains(?1, ?, 'http://www.w3.org/2005/xpath-functions/collation/codepoint')('a')]</test>
<result>
<assert-deep-eq>['a', 'b', 'c']</assert-deep-eq>
</result>
</test-case>Before:
After:
Stacktrace: UnaryLookup-017 <test-case name="UnaryLookup-017">
<description>No syntactic confusion with place-markers</description>
<created by="Michael Kay" on="2014-11-27"/>
<dependency type="spec" value="XP31+ XQ31+"/>
<dependency type="feature" value="higherOrderFunctions"/>
<test>(['a', 'b', 'c'], ['b', 'c', 'd'], ['e', 'f', 'b'])[contains(?1, ?)('a')]</test>
<result>
<assert-deep-eq>['a', 'b', 'c']</assert-deep-eq>
</result>
</test-case>Before:
After:
Stacktrace: I hope this information helps in evaluating the PR. |
|
What do you mean by "traces" @dizzzz ? |
|
@joewiz This is a very thorough research and we are finally able to compare XQTS test results in detail. I cannot thank you enough for taking on this task. |
|
Related issue: Placeholder after arrow function #3336 |
|
In the monday call we have talked about the PR; we really want it pulled it, but these 3 stacktraces are new and should be addressed. 2 out of 3 stacktraces are probably straightforward to fix. In all the PR is a significant improvement! |
|
As I laid out, now that the test can be parsed a different issue surfaces. That is specifically the parser's inability to recognize the argument placeholder "?" In certain cases (see #3336). I strongly disagree with this blocking the merge, as both issues are unrelated. |
|
Compare "UnaryLookup-017" slightly rewritten to remove the argument placeholder: |
|
One could argue that "UnaryLookup-011" should be catched and return the correct error code with location information. |
|
I will provide a test and a fix for unary lookups, when the context item is absent. |
|
@line-o Am I correct in this summary of your response to the xqts findings described above and summarized by @dizzzz?
So once your fix to the NPE is fixed, the PR will be ready for a final review and merge, from your perspective? |
|
@joewiz Yes, that is exactly what I wanted to say :) |
The wrapping parenthesized expression will be removed in the optimisation step.
It allows a set of operations that are valid XQuery but were not parsed properly before.
Examples:
- `[1] ! ?1`
- `map{1:1} ! ?*`
- `(map{1:1}, map{1:2})[1 < ?1]`
fixes eXist-db#3491 With this PR applied the XQuery grammar tries to match exprSingle first. Unary lookups are therefore treated as exprSingle instead of the `?` being interpreted as an argument placeholder. Examples: - `([1], [1,2]) ! sum(?*)` - `map{1:1} ! count(?*)`
Map expressions use currentt map constructor syntax.
`map { $key : $value }`
The missing contextSequence resulted in an NPE. In Lookup this will now be checked first and the correct Error XPDY0002 will be thrown instead. - Add testcase to mapsLookup.xql
line-o
force-pushed
the
fix/unary-lookups
branch
from
3215564 to
5f8feb0
Compare
|
Kudos, SonarCloud Quality Gate passed! |
|
I am confident that with the commit 5f8feb0 UnaryLookup-011 will pass. |
|
XQTS test run results
|
|
By comparing the actual xqts results using the query above, we can not only be confident - we can be absolutely certain. The comparison query also reveals 4 other improvements.
These results were consistent across 3 runs of the test runner. Great job, @line-o! |
Description:
Allow unary lookups in function calls, (anywhere) in predicates and after the simple map operator.
The XQuery grammar now
?)Examples:
(map{1: 1}, map{1: 3}, map{1: 2})[1 > ?1] ! ?1Additional change: correct the syntax of a dumped map expression to use current syntax.
For the second example that produces:
Reference:
fixes #1655
fixes #3491
Relevant XQuery 3.1 specification: 3.11.3.1 Unary Lookup
Type of tests:
Added and uncommented XQSuite tests