[Bug] Whisper transcription truncated after ~16 tokens (any clip with >~4s of continuous speech)

[Anjielon]

[Bug] Whisper transcription truncated after ~16 tokens (any clip > ~4s of continuous speech)

Summary

On Linux (Strix Halo, XDNA2/NPU2), /v1/audio/transcriptions returns only the
first ~15-16 tokens of any transcription. Reproduced on the released 1.0.3
binary AND on a from-source build (same commit as main).

Reproduction (deterministic)

Italian TTS clip, 7.3 s, 20 words, 16 kHz mono. Three different clip lengths
(3.2 s / 5 s / 7.3 s) all return the SAME 50-character prefix; changing
response_format (json / verbose_json / text / srt), language, or
--ctx-len has no effect:

" Domani alle 9 accendi le luci del soggiorno e ric"   ← always 50 chars

Root cause

src/common/whisper/modeling_whisper.cpp, decode loop of generate() (and
generate_streaming()):

int watching_dog = 16;
...
if (watching_dog == 0 && allow_force_time_stamp){
    last_idx = this->_sample_in_time_stamp(logits);   // forces a timestamp
    ...

After 16 content tokens without a timestamp, the loop force-samples a timestamp
token. Whisper then treats the segment as closed and emits EOT — so any
continuous-speech segment longer than ~16 tokens (~4 s of speech) is cut.

Fix (verified on device)

Raising the watchdog to the decode maximum removes the truncation with no other
change (both occurrences):

int watching_dog = 448;

Same 7.3 s clip after the fix: 115 chars, full sentence, 5.0 s on NPU2
(Ryzen AI MAX+ 395, FW 1.1.2.65, amdxdna 0.6, kernel 7.0.8):

" Domani alle 9 accendi le luci del soggiorno e ricordami di comprare
  il latte e le uova al supermercato e se lunga."

If the watchdog exists to protect against runaway no-timestamp decoding, a
less invasive form would be to only force a timestamp when the model has
produced no content tokens either, or to make the threshold configurable.
Happy to open a PR with either form.

Environment

• FastFlowLM 1.0.3 (release tarball) and from-source (main)
• Fedora 43, kernel 7.0.8, amdxdna 0.6.0, NPU FW 1.1.2.65
• Ryzen AI MAX+ 395 (Strix Halo), memlock unlimited