feat(tcfeed): show what the scanner actually does, before asking - #125
Merged
Conversation
Refs #120. The most damning number in the experiment so far: of 24 open requests, none has ever run a scan. Nineteen sit at "waiting for the maintainer to approve the run" and five have no runs at all, because GitHub withholds workflow runs from first-time contributors. Every maintainer who looked saw a pending or red check and no output whatsoever. The offer was "here is a scanner for your pull requests" and the demonstration had never once executed. That explains nine straight declines better than any wording did, and no amount of rewriting the pitch would have touched it. So the request now carries the output of a real run against the repository, produced while the clone is already on disk and before our own files are written, so it describes their tree rather than the workflow being added to it. What it deliberately is not is a findings dump. prBody() has always refused to quote findings at somebody as if they were defects, and this project's own sample is the reason: every such claim in it was false. So this reports what the run did — files, seconds, counts, and the confidence tier the CLI itself assigns — and says plainly that none of it is a claim and that `confidence: pattern` means a regex matched and nothing more. A clean tree says so in two lines. Failure is silence: a scan that breaks, hangs or exceeds TCFEED_DEMO_TIMEOUT omits the section rather than blocking the request or guessing at a number. Verified against two real repositories: 152 files in 1.8s — 1 finding(s): 1 medium confidence: 0 evidence, 1 pattern 4 files in 1.2s — 0 finding(s) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ThreatCrush Security Scan67 finding(s) HIGH/CRITICAL: 11 | MEDIUM: 55 | LOW: 1
…and 17 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #120.
The number this is about
Of the 24 open requests, none has ever run a scan.
GitHub withholds workflow runs from first-time contributors, so every maintainer who opened one of these saw a pending or red check and no output whatsoever. The offer was "here is a scanner for your pull requests" and the demonstration had never once executed — 0 for 24.
That explains nine straight declines better than any wording did, and no amount of rewriting the pitch would have touched it. Two of the nine said as much in different words: Haven's maintainer and discord.go's both suggested a concrete finding would land better than a workflow.
What changes
The request now carries the output of a real run against the repository. It happens while the clone is already on disk, and before our own files are written, so it describes their tree rather than the workflow being added to it. The issue points at it, since the issue is what gets read first.
What it deliberately is not
A findings dump.
prBody()has always refused to quote findings at somebody as if they were defects, and this project's own sample is the reason — every such claim in it was false. Sending a maintainer a list of likely-false positives is worse than sending nothing.So it reports what the run did — files, seconds, counts, and the confidence tier the CLI itself assigns — and says plainly that none of it is a claim:
That is a fact about the scanner, not about their code, and it is the honest half of what a maintainer wanted to see.
Failure is silence. A scan that breaks, hangs, or exceeds
TCFEED_DEMO_TIMEOUT(default 120s) omits the section rather than blocking the request or guessing at a number. Exit 1 is treated as a result, not a failure — that is "findings at or above--fail-on", and the JSON is on stdout either way.Verified against real repositories
tsc --strict --noEmitclean. Both branches of the output exercised end to end via--dry-run:With findings —
dheerajshenoy/lektra:Clean tree —
Duckdevv/Slowducksort:🤖 Generated with Claude Code