Skip to content

fix(canary): prevent repeated streamed overlap in #375 - #376

Closed
CrispStrobe wants to merge 1 commit into
mainfrom
fix/375-canary-regression
Closed

fix(canary): prevent repeated streamed overlap in #375#376
CrispStrobe wants to merge 1 commit into
mainfrom
fix/375-canary-regression

Conversation

@CrispStrobe

Copy link
Copy Markdown
Owner

What changed

Canary streamed decoding now removes a conservative multi-token prefix when its token end times are already covered by the previous chunk. This catches reworded overlap that token-ID LCS cannot match, without enabling the broader fuzzy seam dedup or reverting the 73bb9b2 scheduler UAF fix. Single early tokens remain untouched.

Validation

  • cmake --build /tmp/crispasr-build-375b --target test-asr-overlap-trim canary -j2
  • test-asr-overlap-trim (19 assertions, 8 cases)

The reporter did not attach audio/model details, so live reproduction on the HP Z800 input is still needed.

@CrispStrobe

Copy link
Copy Markdown
Owner Author

Reviewed with a live reproduction, which is the thing this PR says it still
needs. I do not think this should merge as written — measured, it does not
fix the symptom, and it makes the output worse in a new way.

Reproduction

canary-1b-v2-q4_k, CPU, jfk_x12.wav (132 s = the JFK sentence 12×). A
repeating source is a good probe here because the correct answer is exactly 12
copies, so anything extra is duplication rather than a judgement call.

main reproduces the reporter's description precisely:

... ask what you can do. for your country. And so, my fellow Americans, ask not Ask not what your country ...
                       ^^^^ sentence interrupted                            ^^^^^^^^^^^^ phrase repeated

Both halves of #375 are present: "sentences are interrupted" and
"words/phrases are repeated".

What this PR does to it

The entire effect across 132 s is two deleted tokens:

main:  ask not  Ask not  what your country ...
#376:  ask not      not  what your country ...

It removes "Ask" and leaves "not". The duplication survives as
"ask not not" — still duplicated, and now ungrammatical. The two
interrupted-sentence artifacts are untouched.

So on the only end-to-end evidence available, the PR converts one visible defect
into a different visible defect and fixes nothing else.

The mechanism it declines to use already fixes this

CRISPASR_CANARY_SEAM_DEDUP=1 — the #365 fuzzy matcher already in the tree —
removes both tokens on the same file:

build at the seam
main ask not Ask not what your country…
this PR ask not not what your country…
CRISPASR_CANARY_SEAM_DEDUP=1 ask not what your country…

That is not a coincidence: the fuzzy matcher's own comment describes this exact
failure — "given a different amount of right-context an AED re-words the overlap
instead … the LCS matches nothing". "ask" vs "Ask" is a capitalisation
rewording, which is precisely what token-id LCS cannot see and normalised word
matching can.

Two more concerns with the approach

It reintroduces the thing that was already measured as harmful. The comment
directly above the insertion point says: "Deliberately NOT a plain time floor.
Trimming the overlap by timestamp alone was measured removing real speech — the
leading 'Many' … since an AED's boundary timings overshoot."
This PR adds a
plain time floor with >= 2 tokens as the mitigation. The 2-token threshold is
tuned to one anecdote, and the failure mode it guards against is silent deletion
of real speech — which is strictly worse than duplication, because the user can
see duplication and cannot see a dropped phrase.

leading_covered_multi excludes the clearest case. skip >= 2 && skip < n
returns 0 when the whole chunk is covered — i.e. a fully re-transcribed chunk,
the most certain duplicate there is, is deliberately not trimmed, while partial
overlaps are. If that is intentional safety, it is worth saying so; as written
the rule is hardest on the cases it is least sure about.

On the diagnosis

"The only Canary change in the window is 73bb9b2" is true, but it is a
file-scoped search over a 41-commit window that the reporter explicitly could
not narrow ("compilation fails in that range"). By construction that method
cannot see cross-cutting changes, and this window has six commits touching the
audio input path (f3d82d30 routes CLI + stereo decode through glint,
3739e70d clamps file parsers). I checked and there is no ggml submodule bump
in the range, so that one is out — but 73bb9b2 was selected, not bisected, and
the PR presents it as identified.

For what it's worth I think the causal story is still plausible in one direction:
before 73bb9b2 the cached encoder graph could hand adjacent chunks stale
buffers, which would make them decode identically and let the token-id LCS
match. Fixing the UAF made each chunk genuinely independent, so the AED rewords
and the LCS stops firing. That argues for a rewording-tolerant seam merge — the
fuzzy matcher — rather than a time floor.

Suggested path

  1. Don't merge as-is.
  2. Reproduce first — jfk_x12.wav above works, and the reporter can also run
    CRISPASR_CANARY_STREAM_THRESHOLD_S=99999 to force single-pass; if the
    duplication vanishes, the seam hypothesis is confirmed on their audio.
  3. The real decision is whether CRISPASR_CANARY_SEAM_DEDUP should default on
    for canary. It fixes this repro cleanly; it was gated because it dropped a
    leading "Many" on a 600 s clip. That is a corpus question, not a one-clip
    question — the honest next step is running both settings over the regression
    audio and counting insertions vs deletions, rather than adding a third
    heuristic beside the two we already have.
  4. Note that neither mechanism touches the interrupted sentence half
    ("you can do. for your country."), which is arguably the larger part of what
    the reporter is complaining about. That looks like splice punctuation, not
    dedup, and should be tracked separately rather than folded into this.

@CrispStrobe

Copy link
Copy Markdown
Owner Author

Correcting my own review above — its central evidence was invalid, and the
conclusion I drew from it does not stand.

I claimed "main reproduces the reporter's description precisely" on
jfk_x12.wav. I never checked that the same artifacts are absent on the
reporter's GOOD commit. They are not. Built 282e5d0b (their last-good) and ran
the identical file and model:

282e5d0b : 7cce22b79c7cae62f05591caa512fe6c
main     : 7cce22b79c7cae62f05591caa512fe6c

Byte-identical — including all three artifacts I quoted (ask not Ask not,
you can do. for your country., for yourself. your country.). So those are a
pre-existing seam defect present in the reporter's own working build, not the
regression. My "reproduction" reproduced the wrong thing.

Repeated on 594 s of natural German long-form (de/fleurs_600s.wav, ~74 seams,
which is much closer to the reporter's likely use than a repeating clip):

282e5d0b : 58346dcdf82aa6c1eda898cc8f4f75b7
main     : 58346dcdf82aa6c1eda898cc8f4f75b7

Also byte-identical. canary-1b-v2-q4_k, CPU, -l de.

What this means for the PR

My measured objections to the implementation still hold on their own terms —
turning ask not Ask not into ask not not is worse than leaving it, and the
skip >= 2 && skip < n rule excludes the fully-covered case. But my headline
("does not fix the symptom") was measured against the wrong symptom, so treat it
as a critique of the heuristic, not as a verdict on #375.

The real problem is the same for the PR and for my review: nobody has
reproduced the regression.
Two long-form probes across the exact bisect
endpoints show zero difference, so neither this change nor my proposed
alternative (CRISPASR_CANARY_SEAM_DEDUP=1) can be validated as fixing anything.
Merging a decode-path heuristic that silently deletes tokens, on that basis,
risks trading an unreproduced bug for a real one.

I've asked the reporter for what's needed on #375. Until that lands I'd hold this.

What is still solid

  • ggml did not move in the window — the submodule pointer is
    0714117daca2471b00e09554c7eaa74a06b0b2c5 at both endpoints, so that whole
    class is out.
  • 73bb9b2f is the only commit touching canary sources in the window, but it was
    selected by file-path filter, not bisected; the window also contains six
    commits touching the audio input path.
  • The pre-existing seam artifacts above are worth their own issue regardless of
    Regression in speech recognition with canary #375 — they are real, reproducible on demand, and CRISPASR_CANARY_SEAM_DEDUP=1
    cleanly fixes the duplication half of them.

CrispStrobe added a commit that referenced this pull request Aug 19, 2026
Two long-form probes across the reporter's exact bisect endpoints produce
byte-identical output, so neither PR #376 nor my review of it was measured
against the actual regression.

Records what is ruled out by measurement (ggml pointer identical at both
endpoints; 73bb9b2 changes nothing on either probe; generation_health is
test-only), what was never examined (six audio-input commits in the same window —
start there if the reporter's audio is compressed), and the trap that cost two
attempts: a file-path-scoped search cannot see cross-cutting changes, and I
treated a symptom on main as the regression without diffing it against the good
commit on the same file.

Also records the pre-existing seam defect found on the way — real, reproducible,
present on BOTH endpoints, and deserving its own issue rather than being folded
into #375.
@CrispStrobe

Copy link
Copy Markdown
Owner Author

Superseded — #375's root cause was the glint AAC decoder (fixed in 0e5d134), and the seam handling is being aligned with the NeMo blueprint directly.

@CrispStrobe
CrispStrobe deleted the fix/375-canary-regression branch August 19, 2026 19:13
CrispStrobe added a commit that referenced this pull request Aug 19, 2026
…tifacts gone

The long-form path was parakeet machinery wearing a canary comment: 8 s
chunks / 2 s overlap over a globally-normalized mel, merged by an LCS
prefix-drop + word-snap + splice-punctuation heuristics. The comment
cited NeMo's FrameBatchMultiTaskAED, which actually joins NON-overlapping
chunks with " ".join — and canary-1b-v2's shipped `.transcribe()` does
something else again: dynamic 30..40 s RAW-WAVEFORM chunks (size chosen
to maximize the last chunk), 1 s overlap, per-chunk PerFeatureZ
normalization, and lcs_alignment_merge_buffer (search the last 24 buffer
tokens against the next chunk's first 7, keep the matched acoustic event
once, trim BOTH sides). Audio within one 40 s chunk is a single pass.

This ports that blueprint exactly:
- core/canary_chunk_merge.h: longest_common_subsequence_merge +
  _find_optimal_chunk_size, header-only; tests/test-canary-chunk-merge.cpp
  pins both against vectors generated from the nemo 2.7.3 Python
  functions (58 assertions), so the port is validated cross-
  implementation, not against itself.
- canary_transcribe_streamed: blueprint chunking + merge; single
  detokenize + word rebuild at the end. The old path survives verbatim
  behind CRISPASR_CANARY_LEGACY_STREAM=1 as the bisection arm
  (CRISPASR_CANARY_SEAM_DEDUP only applies there).
- CLI adapter and session C-ABI both route through the library (the
  session previously single-passed ALL audio; >40 s single-pass EOSes
  early — a 59 s file lost everything past ~45 s).

Measured (canary-1b-v2-q4_k, CPU):
- jfk_x12.wav (the JFK quote x12, 132 s): 12 clean identical repetitions.
  Legacy gate reproduces "ask not Ask not" x2, "you can do. for your
  country" x2, "for yourself. your country" x2 on the same binary.
- de/fleurs_600s.wav (594 s): zero repeated 2/3/4-grams in 925 words;
  the "not in time order" warnings drop from per-seam to one 0.36 s cue
  overlap.
- de/fleurs_60s.wav: "T-Rex war. T-Rex war", "der der", "Frauen. Frauen"
  all gone.

Closes the seam-artifact thread from #365/#375 (PR #376 closed —
superseded by fixing the blueprint mismatch at the root).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant