Question: Diarization seems excessively slow

[logiclove]

Adding diarization change the performance from 70x to 26.3x realtime. Is this normal ?

• Logs

crispasr      | crispasr-server: transcribed 2888.2s audio in 109.93s (26.3x realtime), format=verbose_json

• Settings used:

CRISPASR_MODEL: ${CRISPASR_MODEL:-/models/cohere-transcribe.gguf}
CRISPASR_BACKEND: ${CRISPASR_BACKEND:-cohere}
CRISPASR_LANGUAGE: ${CRISPASR_LANGUAGE:-auto}
CRISPASR_EXTRA_ARGS: ${CRISPASR_EXTRA_ARGS:---vad --frequency-penalty 0.85 
  -t 12
  --lcs-dedup on --lcs-min-length 4 --split-on-punct  --lid-backend silero 
  --diarize --diarize-method pyannote --sherpa-segment-model auto 
  --diarize-embedder auto --diarize-cluster-threshold 0.5 --diarize-max-speakers 4 }

• Hardware
  lscpu

Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             48 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      24
  On-line CPU(s) list:       0-23
Vendor ID:                   AuthenticAMD
  Model name:                AMD Ryzen 9 9900X 12-Core Processor
    CPU family:              26
    Model:                   68
    Thread(s) per core:      2
    Core(s) per socket:      12
    Socket(s):               1
    Stepping:                0
    Frequency boost:         enabled
    CPU(s) scaling MHz:      32%
    CPU max MHz:             5662.0161
    CPU min MHz:             613.9540
    BogoMIPS:                8782.91
    Flags:                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm e
                             xtapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpuid_fault cpb cat_l3 cdp_l3 hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp ibrs_enhanced vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clw
                             b avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local user_shstk avx_vnni avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl vnmi avx512
                             vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid bus_lock_detect movdiri movdir64b overflow_recov succor smca fsrm avx512_vp2intersect flush_l1d amd_lbr_pmc_freeze

amd-smi

+------------------------------------------------------------------------------+
| AMD-SMI            26.5.0+2b22ab01                                           |
| amdgpu Version:    6.19.14.31400000                                          |
| ROCm Version:      7.14.0                                                    |
| VBIOS Version:     00131149                                                  |
| Platform:          Linux Baremetal                                           |
|-------------------------------------+----------------------------------------|
| BDF                        GPU-Name | Mem-Uti   Temp   UEC       Power-Usage |
| GPU  HIP-ID  OAM-ID  Partition-Mode | GFX-Uti    Fan               Mem-Usage |
|=====================================+========================================|
| 0000:03:00.0  AMD Radeon RX 9070 XT | 0 %      36 °C   0            37/317 W |
|   0       0     N/A             N/A | 7 %      0.0 %             57/16304 MB |
|-------------------------------------+----------------------------------------|

Compile cli:

RUN git clone https://github.com/CrispStrobe/CrispASR . && git checkout v0.8.24
RUN git submodule update --init --recursive
RUN cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=1 -DCRISPASR_BUILD_TESTS=OFF -DCRISPASR_FFMPEG=ON
RUN cmake --build build -j"$(nproc)" --target crispasr-cli

Am I missing some critical flags?

[logiclove]

It also seems that the diarization segments are missing. Am I using incompatible options?

{
  'task': 'transcribe', 
  'language': 'sv-SE', 
  'duration': 2888.233, 
  'text': "Voilà. Je crois que maintenant je vais avoir suffisamment de câbles. ..."
  'segments': [
    {'id': 0, 'start': 3.35, 'end': 5.69, 'text': 'Voilà. Je crois que maintenant je vais avoir suffisamment de câbles.', 'temperature': 0.0, 'avg_logprob': -0.057898, 'no_speech_prob': 0.0, 'words': [{'word': 'Voilà.', 'start': 3.35, 'end': 3.35}, {'word': 'Je', 'start': 3.35, 'end': 3.35}, {'word': 'crois', 'start': 3.35, 'end': 3.43}, {'word': 'que', 'start': 3.43, 'end': 3.59}, {'word': 'maintenant', 'start': 3.59, 'end': 3.83}, {'word': 'je', 'start': 3.83, 'end': 3.99}, {'word': 'vais', 'start': 3.99, 'end': 4.15}, {'word': 'avoir', 'start': 4.15, 'end': 4.55}, {'word': 'suffisamment', 'start': 4.55, 'end': 4.95}, {'word': 'de', 'start': 4.95, 'end': 5.27}, {'word': 'câbles.', 'start': 5.27, 'end': 5.69}], 'tokens': [6207, 13785, 1705, 8281, 615, 6941, 998, 4238, 5172, 3234, 530, 8460, 560, 522, 13979, 7427, 13785]}, 
    {'id': 1, 'start': 84.82, 'end': 91.49, 'text': 'Je te propose de.', 'temperature': 0.0, 'avg_logprob': -0.21963, 'no_speech_prob': 0.0, 'words': [{'word': 'Je', 'start': 84.82, 'end': 84.82}, {'word': 'te', 'start': 84.82, 'end': 85.06}, {'word': 'propose', 'start': 85.06, 'end': 85.46}, {'word': 'de.', 'start': 85.46, 'end': 91.49}], 'tokens': [1705, 790, 13763, 560, 13785]}, 
    {'id': 2, ...

[logiclove]

Retested with commit 3720540.

Still somewhat slow but already very significant Performance improvement from 110s => 87.5s.
It removed 22.5 seconds. In the remaining 87.5s, part of it is the transcription itself.

crispasr      | crispasr-server: slice 210/210 done (21.4s audio in 0.2s)
crispasr      | crispasr[diarize]: using cached /cache/pyannote-seg-3.0.gguf
crispasr      | pyannote_seg: loaded (4 LSTM layers, hidden=128)
crispasr      | crispasr: resolving titanet-large.gguf (~45 MB) via -m auto
crispasr      | crispasr: using cached /cache/titanet-large.gguf
crispasr      | titanet: 80 mels, 1024 channels, 192 emb_dim, 5 blocks
crispasr      | crispasr-server: transcribed 2888.2s audio in 87.48s (33.0x realtime), format=verbose_json

[CrispStrobe]

Fixed on main — thanks for the detailed report, the logs made this straightforward to reproduce.

Root cause: pyannote segmentation ran the whole file as one LSTM scan. The recurrence is sequential, so it used exactly 2 threads (one per direction) no matter what -t said — 171k timesteps on a 48-minute file. It now infers in parallel 60 s chunks: on a 2888 s file that stage went 32.8 s → 5.5 s (M1, 8 cores); your 24-core box should do better. Output is byte-identical across -t, and DER did not regress.

Heads up on what remains: with segmentation fixed, --diarize-embedder is now the dominant cost (52.8 s vs 5.5 s on that file) because it runs one unbatched forward per segment. Batching it is tracked separately — dropping --diarize-embedder is much faster today if you can live with per-pass-local speaker IDs.

Two accuracy bugs fell out of the same investigation: the powerset class table had two entries swapped, and the --diarize-max-speakers cap was overriding the speaker-count estimate rather than bounding it. End-to-end DER on VoxConverse dev went 15.74% → 7.81%. Your --diarize-max-speakers 4 was previously being used as the answer rather than a ceiling.

No flags needed — --sherpa-segment-model auto is already correct for the pyannote method despite the name.