A file Basilisk cannot parse is handled correctly in JSON and badly everywhere
else. Three separate defects share one cause: CheckOutcome::failures is a
side-channel that only the JSON renderer knows about.
[CHKARCH-CLI-OUTPUT-FAILURES] already argues this case for JSON, and its
reasoning applies verbatim to text:
Omitting it rendered [] — byte-for-byte the answer a clean file gets — for a
file that was never checked, so every consumer reading the report rather than
the exit code was told a file with a syntax error had no problems. […] a
report that contradicts it is worse than one that says nothing.
The spec section is implemented and tested only in
basilisk-cli/src/output/json.rs.
Text has no counterpart, no spec clause, and no test.
Repro
$ mkdir /tmp/bsk_syn && cd /tmp/bsk_syn
$ printf 'x: int = "no"\n' > good.py
$ printf 'obj = object()\nobj.\n' > broken.py
1. Text output drops the unanalysable file from the report and the count
$ basilisk check .
error[assignment_compatibility]: Type mismatch: `x` is annotated `int` (int) but assigned LiteralString
--> ./good.py:1:1
...
Found 1 diagnostic (1 error).
2026-08-01T06:52:51Z ERROR basilisk: error processing file path=./broken.py error=syntax error in ./broken.py: Expected an identifier at byte range 20..21
The report body has no entry for broken.py, and the summary says 1
diagnostic. --output json on the same run returns 2 entries. Text and JSON
disagree about what the run found.
The only signal in text mode is a tracing line on stderr, emitted after the
summary and outside the report — so it is lost to 2>/dev/null, to any log
filter, and to anyone reading the report rather than watching the console. With
no other diagnostics present, the text renderer prints nothing at all:
// crates/basilisk-cli/src/main.rs:426
let exit_code = if total == 0 && outcome.failures.is_empty() {
println!("{}", "All checked. No issues found.".green().bold());
0
} else if total == 0 {
0 // <- failures present: prints nothing, reports nothing
}
That branch was clearly added to stop the "All checked. No issues found." lie —
correct instinct, but silence is the other half of the same bug.
2. A user syntax error exits 3 (Internal failure) and masks exit 1
$ basilisk check . # 1 type error + 1 unparseable file
$ echo $?
3
$ rm broken.py && basilisk check . ; echo $?
1
[CHKARCH-CLI-EXITCODES] defines 3 as Internal failure. A syntax error in
the user's own Python is not internal to Basilisk — it is the most ordinary
finding a checker can have. Two consequences:
- CI that branches on the documented codes (
[CHKARCH-CLI-CI] tells consumers to
do exactly that) is told the tool broke, not that the code has problems.
Pipelines that retry or page on 3 will do so on a plain typo.
- The failure exit overrides the diagnostic exit unconditionally
(if outcome.failures.is_empty() { diagnostic_exit } else { 3 },
main.rs:360), so one
unparseable file hides the fact that real errors were found.
This reproduces on this repository: basilisk check . exits 3 because
basilisk-zed/tests/fixtures/completions.py ends in obj. — a deliberately
incomplete completion fixture. The fixture is correct and must stay invalid; the
handling of it is what is wrong.
3. The position is a byte range, not line:col
syntax error in ./broken.py: Expected an identifier at byte range 20..21
No user can act on byte 20. Every other checker reports file:2:5: invalid syntax. The offset comes straight from ruff_python_parser, which hands us a
TextRange — the line/column conversion the rest of the pipeline already does
for diagnostics is simply never applied here. The entry also carries no code and
therefore no see: URL.
Note the JSON entry anchoring at line 1, column 1 is specified behaviour and
should not change; this is about the message text.
Acceptance
A file Basilisk cannot parse is handled correctly in JSON and badly everywhere
else. Three separate defects share one cause:
CheckOutcome::failuresis aside-channel that only the JSON renderer knows about.
[CHKARCH-CLI-OUTPUT-FAILURES]already argues this case for JSON, and itsreasoning applies verbatim to text:
The spec section is implemented and tested only in
basilisk-cli/src/output/json.rs.Text has no counterpart, no spec clause, and no test.
Repro
1. Text output drops the unanalysable file from the report and the count
The report body has no entry for
broken.py, and the summary says 1diagnostic.
--output jsonon the same run returns 2 entries. Text and JSONdisagree about what the run found.
The only signal in text mode is a
tracingline on stderr, emitted after thesummary and outside the report — so it is lost to
2>/dev/null, to any logfilter, and to anyone reading the report rather than watching the console. With
no other diagnostics present, the text renderer prints nothing at all:
That branch was clearly added to stop the "All checked. No issues found." lie —
correct instinct, but silence is the other half of the same bug.
2. A user syntax error exits
3(Internal failure) and masks exit1[CHKARCH-CLI-EXITCODES]defines3as Internal failure. A syntax error inthe user's own Python is not internal to Basilisk — it is the most ordinary
finding a checker can have. Two consequences:
[CHKARCH-CLI-CI]tells consumers todo exactly that) is told the tool broke, not that the code has problems.
Pipelines that retry or page on
3will do so on a plain typo.(
if outcome.failures.is_empty() { diagnostic_exit } else { 3 },main.rs:360), so oneunparseable file hides the fact that real errors were found.
This reproduces on this repository:
basilisk check .exits3becausebasilisk-zed/tests/fixtures/completions.pyends inobj.— a deliberatelyincomplete completion fixture. The fixture is correct and must stay invalid; the
handling of it is what is wrong.
3. The position is a byte range, not
line:colNo user can act on byte 20. Every other checker reports
file:2:5: invalid syntax. The offset comes straight fromruff_python_parser, which hands us aTextRange— the line/column conversion the rest of the pipeline already doesfor diagnostics is simply never applied here. The entry also carries no code and
therefore no
see:URL.Note the JSON entry anchoring at line 1, column 1 is specified behaviour and
should not change; this is about the message text.
Acceptance
[CHKARCH-CLI-OUTPUT-FAILURES]to cover text output, and renderunanalysable files in the text report with the same standing they have in
JSON — visible in the body, counted in the summary, on stdout with the
report rather than as a stderr log line.
1matches itsseverity and keeps
3meaning what the table says. If it must staydistinct from
1, it needs its own row in[CHKARCH-CLI-EXITCODES]— nota reuse of "Internal failure".
line:colin the message.cli_binary_tests.rsJSON assertions, a text-vs-JSON agreement test on onemixed run, and an exit-code test for failure-only and failure-plus-error
trees.