added sh:conformanceDisallows, updated conformance-definition

[bergos]

This PR adds the sh:conformanceDisallows property and updates the definition of the conformance check. The examples are updated to cover the non-default severity levels. The section for the sh:conforms property contained a redundant definition of the conformance check that was removed.

Closes #453

• See this document rendered online here

[YoucTagh]

Would it make sense to add some test cases as well?

[VladimirAlexiev]

@bergos I searched for "disallowed" in your rendered version, and I don't understand something:
how would ValidationReport know which levels to disallow? Isn't that some info that the shape designer or invoker should specify? Where?

UPD: I've read #453 and in particular #453 (comment) (which is the basis of the voting). You say at the start:

We extended the specification to allow giving a parameter to the engine to change the severity level when a result is a violation.

• Where is that in the spec? It should be linked from the sections mentioning "disallowed"
• "Parameter to the engine" (provided dynamically) should not be the only way.
  This should relate to eg sh:shapesGraph and rsx:DataAndShapesGraphLink as mentioned in Need support for RDF datasets (i.e., graphs, quads) #183

[bergos]

@VladimirAlexiev You are right. I revisited severity section and added a note on how the conformance disallow set is defined.

[bergos]

@YoucTagh Tests would require a solution for issue #431.

[ajnelson-nist]

I only have one blocking remark on a paragraph about engines interacting with defaults. Either it's a group-discussion matter for a significant behavior in engines providing defaults, or there's just a little English cleanup to clarify that engines can provide some way for users to request different conformance levels for a single validation run.

[ajnelson-nist]

@YoucTagh Tests would require a solution for issue #431.

@bergos , it looks to me like we can actually write a test for this without #431 . The configuration of the validation run doesn't need to be included here - the result of the call-time configuration is included, by merit of sh:conformanceDisallows being present at all.

[VladimirAlexiev]

@bergos I think it's still backwards:

• https://raw.githack.com/w3c/data-shapes/issue-453/shacl12-core/index.html#conformance-definition "The set of disallowed severity levels is defined as the objects of triples with predicate sh:conformanceDisallows and the validation report as subject."
  But the conformance checking process outputs the validation report, so it cannot take the "disallowed" definitions from that report!
• instead, it should refer to https://raw.githack.com/w3c/data-shapes/issue-453/shacl12-core/index.html#conformanceDisallows and state that the "disallowed" definition from the engine is
  • used when computing conformance
  • copied into the validation report

[bergos]

@bergos , it looks to me like we can actually write a test for this without #431 . The configuration of the validation run doesn't need to be included here - the result of the call-time configuration is included, by merit of sh:conformanceDisallows being present at all.

@ajnelson-nist That would only work if we make sh:conformanceDisallows mandatory. I don't have a strong opinion on that. I guess my initial thought was that people don't stumble over a missing property in case they read the new spec, but deal with a validation report from an older engine, which would be the user's fault.

[bergos]

• https://raw.githack.com/w3c/data-shapes/issue-453/shacl12-core/index.html#conformance-definition "The set of disallowed severity levels is defined as the objects of triples with predicate sh:conformanceDisallows and the validation report as subject."
  But the conformance checking process outputs the validation report, so it cannot take the "disallowed" definitions from that report!

• instead, it should refer to https://raw.githack.com/w3c/data-shapes/issue-453/shacl12-core/index.html#conformanceDisallows and state that the "disallowed" definition from the engine is

  • used when computing conformance
  • copied into the validation report

@VladimirAlexiev The set in the validation report is a representation of the set of the engine. I think that's an implementation detail. One could write an implementation that creates the validation report before the validation and uses that to store the set. I think the spec should not cover how an engine keeps intermediate states/values and how they are copied during processing.

[bergos]

@ajnelson-nist I've merged the changes you proposed.

[ajnelson-nist]

Would it make sense to add some test cases as well?

Added in a43a352 , thank you for the reminder.

[ajnelson-nist]

Someone else should approve before merge, if not to review the text, then to check the tests I just added.

[YoucTagh]

The ex:PersonShape-age shape is used in both validation reports, once with severity sh:Warning and once with severity sh:Violation.
This shape is later defined in the specification as follows:

ex:PersonShape-age;
  a sh:PropertyShape;
  sh:path ex:age;
  sh:datatype xsd:integer;
  sh:maxCount 1;
  sh:reifierShape ex:ProvenanceShape;
  sh:reificationRequired true .

If the data graph in Example 24 is the one used for illustration, I think the validation reports are missing the reification validation results.

[ajnelson-nist]

The ex:PersonShape-age shape is used in both validation reports, once with severity sh:Warning and once with severity sh:Violation. This shape is later defined in the specification as follows:

ex:PersonShape-age;
  a sh:PropertyShape;
  sh:path ex:age;
  sh:datatype xsd:integer;
  sh:maxCount 1;
  sh:reifierShape ex:ProvenanceShape;
  sh:reificationRequired true .

If the data graph in Example 24 is the one used for illustration, I think the validation reports are missing the reification validation results.

Ah, whoops. Good catch, I forgot to search for the shape definition.

One response pertinent to the test design: I did keep the shape I wrote consistent between both tests. What I varied in the test was the presence of sh:conformanceDisallows. Here's the diff between the two test files, posting to just note mootness before line 42:

28c28
<       <conformance-disallows-001>
---
>       <conformance-disallows-002>
31c31
< <conformance-disallows-001>
---
> <conformance-disallows-002>
33,35c33,35
<   rdfs:label "Test of path sh:conformanceDisallows 001" ;
<   rdfs:comment "conformanceDisallows presence lets sh:conforms be true despite presence of an sh:Warning-severity result.  Test contents and behavior otherwise match conformance-disallows-002."@en ;
<   rdfs:seeAlso <conformance-disallows-002> ;
---
>   rdfs:label "Test of path sh:conformanceDisallows 002" ;
>   rdfs:comment "conformanceDisallows absence causes sh:conforms be false due to presence of an sh:Warning-severity result.  Test contents and behavior otherwise match conformance-disallows-001."@en ;
>   rdfs:seeAlso <conformance-disallows-001> ;
42,43c42
<       sh:conforms "true"^^xsd:boolean ;
<       sh:conformanceDisallows sh:Violation ;
---
>       sh:conforms "false"^^xsd:boolean ;

Next step:

The test discrepancy @YoucTagh noted seems to bubble up to the example that I copied. Is it a problem in the document to have two tests in unrelated sections use the same example IRI with different implementations around the example IRI?

I'm fine with adjusting the test, but do we also need to adjust the document?

[YoucTagh]

@ajnelson-nist, @bergos Could we merge this pull request and create a new issue to discuss:

The test discrepancy @YoucTagh noted seems to bubble up to the example that I copied. Is it a problem in the document to have two tests in unrelated sections use the same example IRI with different implementations around the example IRI?
I'm fine with adjusting the test, but do we also need to adjust the document?

[bergos]

@ajnelson-nist I've merged the changes you proposed.

@ajnelson-nist, @bergos Could we merge this pull request and create a new issue to discuss:

The test discrepancy @YoucTagh noted seems to bubble up to the example that I copied. Is it a problem in the document to have two tests in unrelated sections use the same example IRI with different implementations around the example IRI?
I'm fine with adjusting the test, but do we also need to adjust the document?

I don't think it's a problem to reuse IRIs for different examples. I didn't check, but I guess we already have some cases where that is done.

[bergos]

@YoucTagh, please approve the PR, if you think it can be merged.

[YoucTagh]

@bergos, currently, the first thing a user sees in the validation report section is Examples 20 and 21. In these examples, we immediately present the validation report, which indicates whether the data graph conforms to the SHACL shape graph. The report includes the conformance result (true or false) and the overall validation outcome.

However, when someone unfamiliar with the specification reviews this section for the first time, they will notice that the focus node is ex:Bob and the source shape is ex:PersonShape-age. Naturally, they will look for these elements in the earlier sections of the document since these are not redefined or clearly referenced. The issue isn’t about reusing IRIs. The validation report we provide does not accurately represent the validation of the data graph and shape graph used in the examples.

If you still don't think this is problematic, I won't block the merge.