scripts: speculative ctx should raise the tiered default, not override it

[gdevenyi]

Problem

With BONSAI_SPECULATIVE=1, the speculative-decoding branch in scripts/start_llama_server.sh overrode the context size to a fixed 16384 whenever BONSAI_CTX was unset or 0:

case "${BONSAI_CTX:-0}" in 0|"") _ctx=16384 ;; esac

This contradicts the BONSAI_CTX=0 = "auto" semantics established in #127 / common.sh:169, where 0/unset resolves to the RAM-tiered default. So a 48 GB+ machine (auto tier 131072) would get silently capped to 16384 as soon as speculative decoding was turned on.

The floor exists for a real reason (dspark re-prefills every request and drafts 1.5-2k tokens, so a tiny context truncates answers), but it should be a floor, not a hard override.

Fix

_ctx is already initialized to CTX_SIZE_DEFAULT (line 100), which honors BONSAI_CTX semantics from common.sh (explicit non-zero wins, 0/unset = RAM tier). Replace the override with a raise:

[ "$_ctx" -lt 16384 ] 2>/dev/null && _ctx=16384

Behavior (BONSAI_SPECULATIVE=1)

Auto tier (RAM)	Before	After
8192 (≤11 GB)	16384	16384 (raised)
16384 (≤23 GB)	16384	16384
32768 (≤35 GB)	16384 ❌	32768
65536 (≤71 GB)	16384 ❌	65536
131072 (>71 GB, 27B)	16384 ❌	131072

Explicit BONSAI_CTX=N (non-zero) is unchanged — it always wins. 2>/dev/null guards the -lt comparison in case _ctx is ever non-numeric.

Verified sh -n passes.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[gdevenyi]

This has exposed a bug in your llama.cpp fork. It looks like the dflash model is not properly considered in the --fit of llama.cpp such that it always crashes.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

scripts/start_llama_server.sh:112

• _ctx contains both auto-tiered and explicit values, so this also raises an explicit BONSAI_CTX=8192 to 16384. That breaks the documented contract that every explicit non-zero value wins; gate the floor on BONSAI_CTX being unset or 0.

            [ "$_ctx" -lt 16384 ] 2>/dev/null && _ctx=16384