feat(server): support transient null overrides on load - #3172
Conversation
Treat explicit null recipe options in /v1/load as transient tombstones for the saved per-model layer. Omitted options continue to use saved values, while concrete values remain request overrides. Keep ctx_size=-1 as the explicit auto value and preserve existing merge_args behavior.
bitgamma
left a comment
There was a problem hiding this comment.
there is one edge case that is probably worth addressing: tombstones are silently dropped when the model needs downloading
The rebuild runs before the download block:
// rebuild info.recipe_options with tombstones removed
info.recipe_options = RecipeOptions(info.recipe, per_model_options);
if (!model_manager_->is_model_downloaded(model_name) && !is_model_collection_recipe(info.recipe)) {
...
info = model_manager_->get_model_info(model_name); // <-- overwrites the tombstoned recipe_options
} If the model isn't already downloaded, info is re-fetched from get_model_info(), which rebuilds recipe_options from the full saved layer (including tombstoned keys). The transient null clear then has no effect — the model loads with the saved value. This is reachable because options can be saved without loading
(POST /models/{id}/options) and download state is independent of the saved options. It affects both the save_options and non-save_options paths.
Fix suggestion: move the tombstone-rebuild to after the download block (after both info reassignments), so it is applied to the final info used by router_->load_model.
Minor note: the tombstone-rebuild block and the save-write preservation block both re-read get_saved_model_options and could be factored into a small helper, but this is cosmetic.
|
Good catch. Moved the transient tombstone overlay after the download/re-fetch so it is applied to the final ModelInfo passed to the router. Kept this as a minimal ordering fix without factoring the surrounding logic. |
## Lemonade LLM Server Update
This automated PR updates Lemonade from `11.5.1` to `11.7.0`.
### Changelog
## Headline
- A new `POST /v1/models/register` endpoint registers or updates `user.*` model definitions without downloading any files.
- New `GET/POST/DELETE /v1/models/{id}/options` endpoints save, inspect, and reset per-model recipe options without loading the model, with per-load transient overrides.
- New `GET /v1/stats` and Prometheus `/metrics` endpoints report prefix-cache effectiveness and router route-switch counters.
- The built-in catalog adds the Qwen3.8-27B and NVIDIA Nemotron 3.5 Lightning 30B-A3B GGUF models and the Z-Image-Turbo image model.
- Lemonade can now be installed on Windows with `winget` and on macOS with a Homebrew cask.
## Breaking Changes
- `POST /pull` now returns 400 and registers nothing when a model definition names an unservable or contradictory deployment mode; such labels are no longer silently normalized.
- Stored `user_models.json` entries that violate the new deployment-mode label rule are now skipped at startup with an error instead of being repaired.
- Removed the deprecated `Lite Collection` and `Ultra Collection` model registry entries.
- `/v1/audio/transcriptions` now returns raw text bodies for `text`/`srt`/`vtt` instead of a JSON `{text: ...}` wrapper, and returns 400 for unsupported `response_format` values.
- The TheNoise image backend no longer accepts per-request `upscale` and `lora_dir` recipe options; `lora_dir` is now set server-wide in `config.json` and `upscale` is only honored as a passed-through request parameter.
- `lemonade launch opencode` now emits a `limit` object (`limit.context`/`limit.output`) instead of a top-level `contextWindow` field.
- The `no_broadcast` config key is replaced by an inverted `broadcast` boolean, auto-migrated on load.
- An existing but unreadable `extra_models_dir` now returns 400 on `/internal/set` instead of being silently applied.
## Lemonade Server
| Operating System | Downloads |
|------------------|-----------|
| **Windows** | [lemonade.msi](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade.msi) |
| **Ubuntu 24.04+** | [Launchpad PPA](https://launchpad.net/~lemonade-team/+archive/ubuntu/stable) |
| **Debian 13 (x86_64)** | [lemonade-server_11.7.0-debian13_amd64.deb](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server_11.7.0-debian13_amd64.deb) |
| **Debian 13 (ARM64)** | [lemonade-server_11.7.0-debian13_arm64.deb](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server_11.7.0-debian13_arm64.deb) |
| **Fedora 43 (x86_64)** | [lemonade-server-11.7.0-fc43.x86_64.rpm](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server-11.7.0-fc43.x86_64.rpm) |
| **Fedora 43 (ARM64)** | [lemonade-server-11.7.0-fc43.aarch64.rpm](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server-11.7.0-fc43.aarch64.rpm) |
| **Fedora 44 (x86_64)** | [lemonade-server-11.7.0-fc44.x86_64.rpm](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server-11.7.0-fc44.x86_64.rpm) |
| **Fedora 44 (ARM64)** | [lemonade-server-11.7.0-fc44.aarch64.rpm](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server-11.7.0-fc44.aarch64.rpm) |
| **macOS** | [Lemonade-11.7.0-Darwin.pkg](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/Lemonade-11.7.0-Darwin.pkg) |
> **Other platforms?** See our [Installation Options](https://lemonade-server.ai/docs/guide/install/) for [Docker](https://lemonade-server.ai/docs/guide/install/docker/), [Snap](https://lemonade-server.ai/docs/guide/install/ubuntu/#__tabbed_2_3), [Arch](https://lemonade-server.ai/docs/guide/install/arch/), [Debian](https://lemonade-server.ai/docs/guide/install/), and more.
## Embeddable Lemonade
Portable binaries for bundling into your own installer. Run `lemond ./` as a subprocess.
| Platform | Download |
|----------|----------|
| **Ubuntu x64** | [lemonade-embeddable-11.7.0-ubuntu-x64.tar.gz](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-embeddable-11.7.0-ubuntu-x64.tar.gz) |
| **Ubuntu arm64** | [lemonade-embeddable-11.7.0-ubuntu-arm64.tar.gz](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-embeddable-11.7.0-ubuntu-arm64.tar.gz) |
| **Windows x64** | [lemonade-embeddable-11.7.0-windows-x64.zip](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-embeddable-11.7.0-windows-x64.zip) |
| **macOS arm64** | [lemonade-embeddable-11.7.0-macos-arm64.tar.gz](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-embeddable-11.7.0-macos-arm64.tar.gz) |
---
## What's Changed
Thanks `SlawomirNowaczyk`, `abn`, `anditherobot`, `bitgamma`, `blackdeathdrow`, `bong-water-water-bong`, `fl0rianr`, `ianbmacdonald`, `iswaryaalex`, `jeremyfowers`, `kenvandine`, `mzyy94`, `osimarr`, `popey`, `ramkrishna2910`, `superm1`, `yelliver`, `zaneni6` for your awesome contributions to this release!
<details>
<summary>Click to expand changelog</summary>
* Bump TheNoise to version v0.1.2 by `bitgamma` in lemonade-sdk/lemonade#3079
* docs(backends): drop the model catalog tables from the backend reference by `jeremyfowers` in lemonade-sdk/lemonade#3103
* fix(alias): report the real reason for alias validation failures by `ramkrishna2910` in lemonade-sdk/lemonade#3111
* feat(server): add model registration endpoint by `fl0rianr` in lemonade-sdk/lemonade#3118
* Debian packaging improvements by `superm1` in lemonade-sdk/lemonade#3039
* docs: add winget and Homebrew install to homepage quickstart by `yelliver` in lemonade-sdk/lemonade#2970
* Trigger snap release candidate build from release branches by `kenvandine` in lemonade-sdk/lemonade#2908
* fix(server): compare artifacts, not commit SHAs, when checking for model updates by `blackdeathdrow` in lemonade-sdk/lemonade#3073
* fix(server): normalize thinking controls for llama.cpp by `fl0rianr` in lemonade-sdk/lemonade#3132
* fix: stop request during prefill now possible by `fl0rianr` in lemonade-sdk/lemonade#3133
* ci: stop building unused targets in the validate workflows by `jeremyfowers` in lemonade-sdk/lemonade#3052
* fix(server): harden extra models directory handling by `fl0rianr` in lemonade-sdk/lemonade#3105
* Fix #1833: Improve transcription response format compatibility by `anditherobot` in lemonade-sdk/lemonade#2088
* ci: stop pinning lite validation legs to the 128gb runner by `jeremyfowers` in lemonade-sdk/lemonade#3139
* feat(models): add an explicit `chat` label by `jeremyfowers` in lemonade-sdk/lemonade#3099
* feat(telemetry): capture prefix-cache effectiveness (cache_n, cached_tokens) and route-switch counts by `ramkrishna2910` in lemonade-sdk/lemonade#2968
* Update FLM version by `zaneni6` in lemonade-sdk/lemonade#3144
* Backend Battle Nightly Regression by `iswaryaalex` in lemonade-sdk/lemonade#3069
* Remove emoji from FAQ title by `jeremyfowers` in lemonade-sdk/lemonade#3154
* Manage per-model recipe options without loading by `jeremyfowers` in lemonade-sdk/lemonade#3110
* Enable % used context on opencode side panel by `osimarr` in lemonade-sdk/lemonade#1725
* feat(server,cli): add discovery and broadcast controls with config decoupling by `abn` in lemonade-sdk/lemonade#3135
* fix(packaging): add AppStream MetaInfo files for desktop entries by `superm1` in lemonade-sdk/lemonade#3123
* fix(flm): correctly report downloaded model size by `mzyy94` in lemonade-sdk/lemonade#3166
* Minor refactor of new gate code by `SlawomirNowaczyk` in lemonade-sdk/lemonade#2971
* update TheNoise v0.2.1 by `bitgamma` in lemonade-sdk/lemonade#3171
* fix(flm): stop re-pulling already-downloaded models on every load by `mzyy94` in lemonade-sdk/lemonade#3167
* feat: flag comment slop in a pre-commit rule, and run pre-commit in CI by `ianbmacdonald` in lemonade-sdk/lemonade#2689
* add lemonade version to benchmark results by `bitgamma` in lemonade-sdk/lemonade#3173
* feat(server): support transient null overrides on load by `fl0rianr` in lemonade-sdk/lemonade#3172
* fix(macos): reduce log directory permissions to prevent privilege esc… by `superm1` in lemonade-sdk/lemonade#2625
* docs: add pull request template by `fl0rianr` in lemonade-sdk/lemonade#2551
* fix(npu): pre-reject NPU loads when auto-tune can only reserve fallback ctx (#1151) by `bong-water-water-bong` in lemonade-sdk/lemonade#3164
* models: add Qwen3.8 and Nemotron 3.5 GGUF by `yelliver` in lemonade-sdk/lemonade#3177
* Fix pre-commit JSON validation issues by `superm1` in lemonade-sdk/lemonade#3179
* docs: fix multi-file checkpoint rendering by `yelliver` in lemonade-sdk/lemonade#3174
* Remove some MTP default flags by `bitgamma` in lemonade-sdk/lemonade#3187
* ci: wire server eviction tests into CI by `fl0rianr` in lemonade-sdk/lemonade#3015
* ci: run watchdog lifecycle tests in backend CI by `fl0rianr` in lemonade-sdk/lemonade#3021
* test: replace Python system info replicas with C++ coverage by `fl0rianr` in lemonade-sdk/lemonade#3020
* fix(server): let load args override saved args by `fl0rianr` in lemonade-sdk/lemonade#3204
* refactor(server): drop load_command from the options endpoint by `jeremyfowers` in lemonade-sdk/lemonade#3199
* fix(test): new watchdog test concurrency issue by `fl0rianr` in lemonade-sdk/lemonade#3201
* fix(pre-commit): make comment-slop runnable on Windows by `blackdeathdrow` in lemonade-sdk/lemonade#3208
* Update to ROCm 7.14 by `superm1` in lemonade-sdk/lemonade#2768
* Fix stale gfx90X-dcgpu TheRock URL mapping for gfx908/gfx90a by `kenvandine` in lemonade-sdk/lemonade#3196
* test: add concurrent chat completion coverage by `fl0rianr` in lemonade-sdk/lemonade#2060
* Fail startup when one resolved address cannot bind by `popey` in lemonade-sdk/lemonade#3197
* fix(ci): resolve test flakes across C++ store, python test harness, and workflows by `abn` in lemonade-sdk/lemonade#2805
</details>
## New Contributors
* `yelliver` made their first contribution in lemonade-sdk/lemonade#2970
* `zaneni6` made their first contribution in lemonade-sdk/lemonade#3144
* `mzyy94` made their first contribution in lemonade-sdk/lemonade#3166
**Full Changelog**: lemonade-sdk/lemonade@v11.6.0...v11.7.0
---
> Windows installers are signed. Free code signing provided by [SignPath.io](https://signpath.io), certificate by [SignPath Foundation](https://signpath.org). See our [Code Signing Policy](https://github.com/lemonade-sdk/lemonade#code-signing-policy).
### Changes
- Updated `config.yaml` version
- Updated `build.yaml` LEMONADE_VERSION
- Updated `Dockerfile` LEMONADE_VERSION
- Updated documentation files
- Updated CHANGELOG.md
### Packaging check
The update script verified this release ships both
`lemonade-embeddable-<version>-ubuntu-arm64.tar.gz` and
`...-x64.tar.gz`, which are the archives this app installs.
The image bundles a glibc closure computed with `ldd` at build time.
If this release adds a new shared-library dependency, the smoke test
will surface it as `error while loading shared libraries`.
### Release Notes
Full release notes: https://github.com/lemonade-sdk/lemonade/releases/tag/v11.7.0
---
This PR was automatically generated by the Update Lemonade workflow
Auto-merged by GitHub Actions
Summary
Add transient per-option clearing semantics to
POST /v1/load.An explicitly provided
nullnow means: ignore the saved per-model value for this load only and fall through to the lower server/default layers.This gives
/v1/loadthree distinct states for recipe options:null: ignore the saved per-model value for this load onlyctx_size: -1remains the explicit auto-sizing value and is not treated as a clear.Why
This lets clients clear individual saved options without mutating
recipe_options.json.For example, a client can temporarily clear saved
llamacpp_argswithout also losing unrelated saved settings. This is more precise than a globalignore_saved_optionsflag.Tests
Added coverage for:
nullclearmerge_argsbehaviorctx_size: -1