Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial output validator from spec #214

Open
lungsin opened this issue Jan 21, 2025 · 0 comments
Open

Partial output validator from spec #214

lungsin opened this issue Jan 21, 2025 · 0 comments

Comments

@lungsin
Copy link

lungsin commented Jan 21, 2025

When writing spec, sometimes a test case is written with a certain property to lead for a certain output. For example, for a particular case, the input is designed such that the output is "yes". This additional information from the spec can be useful if it can be validated when the runner generates the tc; it would lead to a higher confidence that the solution produces a correct output when the validation passes, and early indication that at least one of the solution or the spec is buggy when the validation fails.

I'm not sure what API design that works well since it's possible that the test case consists of a bunch of queries and only some of these queries have certain properties.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant