Skip to content

feat(server): support transient null overrides on load - #3172

Merged
bitgamma merged 2 commits into
mainfrom
fl0rianr/enable_load_without_saving
Aug 16, 2026
Merged

feat(server): support transient null overrides on load#3172
bitgamma merged 2 commits into
mainfrom
fl0rianr/enable_load_without_saving

Conversation

@fl0rianr

Copy link
Copy Markdown
Collaborator

Summary

Add transient per-option clearing semantics to POST /v1/load.

An explicitly provided null now means: ignore the saved per-model value for this load only and fall through to the lower server/default layers.

This gives /v1/load three distinct states for recipe options:

  • omitted: keep using the saved per-model value
  • null: ignore the saved per-model value for this load only
  • concrete value: override it for this load

ctx_size: -1 remains the explicit auto-sizing value and is not treated as a clear.

Why

This lets clients clear individual saved options without mutating recipe_options.json.

For example, a client can temporarily clear saved llamacpp_args without also losing unrelated saved settings. This is more precise than a global ignore_saved_options flag.

Tests

Added coverage for:

  • saved args → transient null clear
  • another saved option remains active
  • merge_args behavior
  • explicit ctx_size: -1

Treat explicit null recipe options in /v1/load as transient tombstones for
the saved per-model layer. Omitted options continue to use saved values,
while concrete values remain request overrides.

Keep ctx_size=-1 as the explicit auto value and preserve existing
merge_args behavior.
@fl0rianr
fl0rianr requested a review from bitgamma August 16, 2026 16:50
@fl0rianr fl0rianr added this to the GUI ui/ux - release blockers milestone Aug 16, 2026
@github-actions github-actions Bot added enhancement New feature or request area::api HTTP REST API surface and route handlers labels Aug 16, 2026

@bitgamma bitgamma left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is one edge case that is probably worth addressing: tombstones are silently dropped when the model needs downloading

The rebuild runs before the download block:

  // rebuild info.recipe_options with tombstones removed                                                                                                                                                                                                                                                                     
  info.recipe_options = RecipeOptions(info.recipe, per_model_options);                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                             
  if (!model_manager_->is_model_downloaded(model_name) && !is_model_collection_recipe(info.recipe)) {                                                                                                                                                                                                                        
      ...                                                                                                                                                                                                                                                                                                                    
      info = model_manager_->get_model_info(model_name);   // <-- overwrites the tombstoned recipe_options                                                                                                                                                                                                                   
  }                                                                                                                                                                                                                                                                                                                          

If the model isn't already downloaded, info is re-fetched from get_model_info(), which rebuilds recipe_options from the full saved layer (including tombstoned keys). The transient null clear then has no effect — the model loads with the saved value. This is reachable because options can be saved without loading
(POST /models/{id}/options) and download state is independent of the saved options. It affects both the save_options and non-save_options paths.

Fix suggestion: move the tombstone-rebuild to after the download block (after both info reassignments), so it is applied to the final info used by router_->load_model.

Minor note: the tombstone-rebuild block and the save-write preservation block both re-read get_saved_model_options and could be factored into a small helper, but this is cosmetic.

@fl0rianr

Copy link
Copy Markdown
Collaborator Author

Good catch. Moved the transient tombstone overlay after the download/re-fetch so it is applied to the final ModelInfo passed to the router. Kept this as a minimal ordering fix without factoring the surrounding logic.

@fl0rianr
fl0rianr requested a review from bitgamma August 16, 2026 17:25
@bitgamma
bitgamma enabled auto-merge August 16, 2026 17:32
@bitgamma
bitgamma added this pull request to the merge queue Aug 16, 2026
Merged via the queue into main with commit ad929fa Aug 16, 2026
80 checks passed
@bitgamma
bitgamma deleted the fl0rianr/enable_load_without_saving branch August 16, 2026 20:47
github-actions Bot added a commit to ShobuPrime/home-assistant-apps that referenced this pull request Aug 21, 2026
## Lemonade LLM Server Update

This automated PR updates Lemonade from `11.5.1` to `11.7.0`.

### Changelog

## Headline

- A new `POST /v1/models/register` endpoint registers or updates `user.*` model definitions without downloading any files.
- New `GET/POST/DELETE /v1/models/{id}/options` endpoints save, inspect, and reset per-model recipe options without loading the model, with per-load transient overrides.
- New `GET /v1/stats` and Prometheus `/metrics` endpoints report prefix-cache effectiveness and router route-switch counters.
- The built-in catalog adds the Qwen3.8-27B and NVIDIA Nemotron 3.5 Lightning 30B-A3B GGUF models and the Z-Image-Turbo image model.
- Lemonade can now be installed on Windows with `winget` and on macOS with a Homebrew cask.

## Breaking Changes

- `POST /pull` now returns 400 and registers nothing when a model definition names an unservable or contradictory deployment mode; such labels are no longer silently normalized.
- Stored `user_models.json` entries that violate the new deployment-mode label rule are now skipped at startup with an error instead of being repaired.
- Removed the deprecated `Lite Collection` and `Ultra Collection` model registry entries.
- `/v1/audio/transcriptions` now returns raw text bodies for `text`/`srt`/`vtt` instead of a JSON `{text: ...}` wrapper, and returns 400 for unsupported `response_format` values.
- The TheNoise image backend no longer accepts per-request `upscale` and `lora_dir` recipe options; `lora_dir` is now set server-wide in `config.json` and `upscale` is only honored as a passed-through request parameter.
- `lemonade launch opencode` now emits a `limit` object (`limit.context`/`limit.output`) instead of a top-level `contextWindow` field.
- The `no_broadcast` config key is replaced by an inverted `broadcast` boolean, auto-migrated on load.
- An existing but unreadable `extra_models_dir` now returns 400 on `/internal/set` instead of being silently applied.

## Lemonade Server

| Operating System | Downloads |
|------------------|-----------|
| **Windows** | [lemonade.msi](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade.msi) |
| **Ubuntu 24.04+** | [Launchpad PPA](https://launchpad.net/~lemonade-team/+archive/ubuntu/stable) |
| **Debian 13 (x86_64)** | [lemonade-server_11.7.0-debian13_amd64.deb](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server_11.7.0-debian13_amd64.deb) |
| **Debian 13 (ARM64)** | [lemonade-server_11.7.0-debian13_arm64.deb](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server_11.7.0-debian13_arm64.deb) |
| **Fedora 43 (x86_64)** | [lemonade-server-11.7.0-fc43.x86_64.rpm](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server-11.7.0-fc43.x86_64.rpm) |
| **Fedora 43 (ARM64)** | [lemonade-server-11.7.0-fc43.aarch64.rpm](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server-11.7.0-fc43.aarch64.rpm) |
| **Fedora 44 (x86_64)** | [lemonade-server-11.7.0-fc44.x86_64.rpm](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server-11.7.0-fc44.x86_64.rpm) |
| **Fedora 44 (ARM64)** | [lemonade-server-11.7.0-fc44.aarch64.rpm](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-server-11.7.0-fc44.aarch64.rpm) |
| **macOS** | [Lemonade-11.7.0-Darwin.pkg](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/Lemonade-11.7.0-Darwin.pkg) |

> **Other platforms?** See our [Installation Options](https://lemonade-server.ai/docs/guide/install/) for [Docker](https://lemonade-server.ai/docs/guide/install/docker/), [Snap](https://lemonade-server.ai/docs/guide/install/ubuntu/#__tabbed_2_3), [Arch](https://lemonade-server.ai/docs/guide/install/arch/), [Debian](https://lemonade-server.ai/docs/guide/install/), and more.

## Embeddable Lemonade

Portable binaries for bundling into your own installer. Run `lemond ./` as a subprocess.

| Platform | Download |
|----------|----------|
| **Ubuntu x64** | [lemonade-embeddable-11.7.0-ubuntu-x64.tar.gz](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-embeddable-11.7.0-ubuntu-x64.tar.gz) |
| **Ubuntu arm64** | [lemonade-embeddable-11.7.0-ubuntu-arm64.tar.gz](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-embeddable-11.7.0-ubuntu-arm64.tar.gz) |
| **Windows x64** | [lemonade-embeddable-11.7.0-windows-x64.zip](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-embeddable-11.7.0-windows-x64.zip) |
| **macOS arm64** | [lemonade-embeddable-11.7.0-macos-arm64.tar.gz](https://github.com/lemonade-sdk/lemonade/releases/download/v11.7.0/lemonade-embeddable-11.7.0-macos-arm64.tar.gz) |

---

## What's Changed

Thanks `SlawomirNowaczyk`, `abn`, `anditherobot`, `bitgamma`, `blackdeathdrow`, `bong-water-water-bong`, `fl0rianr`, `ianbmacdonald`, `iswaryaalex`, `jeremyfowers`, `kenvandine`, `mzyy94`, `osimarr`, `popey`, `ramkrishna2910`, `superm1`, `yelliver`, `zaneni6` for your awesome contributions to this release!

<details>
<summary>Click to expand changelog</summary>

* Bump TheNoise to version v0.1.2 by `bitgamma` in lemonade-sdk/lemonade#3079
* docs(backends): drop the model catalog tables from the backend reference by `jeremyfowers` in lemonade-sdk/lemonade#3103
* fix(alias): report the real reason for alias validation failures by `ramkrishna2910` in lemonade-sdk/lemonade#3111
* feat(server): add model registration endpoint by `fl0rianr` in lemonade-sdk/lemonade#3118
* Debian packaging improvements by `superm1` in lemonade-sdk/lemonade#3039
* docs: add winget and Homebrew install to homepage quickstart by `yelliver` in lemonade-sdk/lemonade#2970
* Trigger snap release candidate build from release branches by `kenvandine` in lemonade-sdk/lemonade#2908
* fix(server): compare artifacts, not commit SHAs, when checking for model updates by `blackdeathdrow` in lemonade-sdk/lemonade#3073
* fix(server): normalize thinking controls for llama.cpp by `fl0rianr` in lemonade-sdk/lemonade#3132
* fix: stop request during prefill now possible by `fl0rianr` in lemonade-sdk/lemonade#3133
* ci: stop building unused targets in the validate workflows by `jeremyfowers` in lemonade-sdk/lemonade#3052
* fix(server): harden extra models directory handling by `fl0rianr` in lemonade-sdk/lemonade#3105
* Fix #1833: Improve transcription response format compatibility by `anditherobot` in lemonade-sdk/lemonade#2088
* ci: stop pinning lite validation legs to the 128gb runner by `jeremyfowers` in lemonade-sdk/lemonade#3139
* feat(models): add an explicit `chat` label by `jeremyfowers` in lemonade-sdk/lemonade#3099
* feat(telemetry): capture prefix-cache effectiveness (cache_n, cached_tokens) and route-switch counts by `ramkrishna2910` in lemonade-sdk/lemonade#2968
* Update FLM version by `zaneni6` in lemonade-sdk/lemonade#3144
* Backend Battle Nightly Regression by `iswaryaalex` in lemonade-sdk/lemonade#3069
* Remove emoji from FAQ title by `jeremyfowers` in lemonade-sdk/lemonade#3154
* Manage per-model recipe options without loading by `jeremyfowers` in lemonade-sdk/lemonade#3110
* Enable % used context on opencode side panel by `osimarr` in lemonade-sdk/lemonade#1725
* feat(server,cli): add discovery and broadcast controls with config decoupling by `abn` in lemonade-sdk/lemonade#3135
* fix(packaging): add AppStream MetaInfo files for desktop entries by `superm1` in lemonade-sdk/lemonade#3123
* fix(flm): correctly report downloaded model size by `mzyy94` in lemonade-sdk/lemonade#3166
* Minor refactor of new gate code  by `SlawomirNowaczyk` in lemonade-sdk/lemonade#2971
* update TheNoise v0.2.1 by `bitgamma` in lemonade-sdk/lemonade#3171
* fix(flm): stop re-pulling already-downloaded models on every load by `mzyy94` in lemonade-sdk/lemonade#3167
* feat: flag comment slop in a pre-commit rule, and run pre-commit in CI by `ianbmacdonald` in lemonade-sdk/lemonade#2689
* add lemonade version to benchmark results by `bitgamma` in lemonade-sdk/lemonade#3173
* feat(server): support transient null overrides on load by `fl0rianr` in lemonade-sdk/lemonade#3172
* fix(macos): reduce log directory permissions to prevent privilege esc… by `superm1` in lemonade-sdk/lemonade#2625
* docs: add pull request template by `fl0rianr` in lemonade-sdk/lemonade#2551
* fix(npu): pre-reject NPU loads when auto-tune can only reserve fallback ctx (#1151) by `bong-water-water-bong` in lemonade-sdk/lemonade#3164
* models: add Qwen3.8 and Nemotron 3.5 GGUF by `yelliver` in lemonade-sdk/lemonade#3177
* Fix pre-commit JSON validation issues by `superm1` in lemonade-sdk/lemonade#3179
* docs: fix multi-file checkpoint rendering by `yelliver` in lemonade-sdk/lemonade#3174
* Remove some MTP default flags by `bitgamma` in lemonade-sdk/lemonade#3187
* ci: wire server eviction tests into CI by `fl0rianr` in lemonade-sdk/lemonade#3015
* ci: run watchdog lifecycle tests in backend CI by `fl0rianr` in lemonade-sdk/lemonade#3021
* test: replace Python system info replicas with C++ coverage by `fl0rianr` in lemonade-sdk/lemonade#3020
* fix(server): let load args override saved args by `fl0rianr` in lemonade-sdk/lemonade#3204
* refactor(server): drop load_command from the options endpoint by `jeremyfowers` in lemonade-sdk/lemonade#3199
* fix(test): new watchdog test concurrency issue by `fl0rianr` in lemonade-sdk/lemonade#3201
* fix(pre-commit): make comment-slop runnable on Windows by `blackdeathdrow` in lemonade-sdk/lemonade#3208
* Update to ROCm 7.14 by `superm1` in lemonade-sdk/lemonade#2768
* Fix stale gfx90X-dcgpu TheRock URL mapping for gfx908/gfx90a by `kenvandine` in lemonade-sdk/lemonade#3196
* test: add concurrent chat completion coverage by `fl0rianr` in lemonade-sdk/lemonade#2060
* Fail startup when one resolved address cannot bind by `popey` in lemonade-sdk/lemonade#3197
* fix(ci): resolve test flakes across C++ store, python test harness, and workflows by `abn` in lemonade-sdk/lemonade#2805

</details>

## New Contributors
* `yelliver` made their first contribution in lemonade-sdk/lemonade#2970
* `zaneni6` made their first contribution in lemonade-sdk/lemonade#3144
* `mzyy94` made their first contribution in lemonade-sdk/lemonade#3166

**Full Changelog**: lemonade-sdk/lemonade@v11.6.0...v11.7.0

---

> Windows installers are signed. Free code signing provided by [SignPath.io](https://signpath.io), certificate by [SignPath Foundation](https://signpath.org). See our [Code Signing Policy](https://github.com/lemonade-sdk/lemonade#code-signing-policy).

### Changes

- Updated `config.yaml` version
- Updated `build.yaml` LEMONADE_VERSION
- Updated `Dockerfile` LEMONADE_VERSION
- Updated documentation files
- Updated CHANGELOG.md

### Packaging check

The update script verified this release ships both
`lemonade-embeddable-<version>-ubuntu-arm64.tar.gz` and
`...-x64.tar.gz`, which are the archives this app installs.

The image bundles a glibc closure computed with `ldd` at build time.
If this release adds a new shared-library dependency, the smoke test
will surface it as `error while loading shared libraries`.

### Release Notes

Full release notes: https://github.com/lemonade-sdk/lemonade/releases/tag/v11.7.0

---

This PR was automatically generated by the Update Lemonade workflow

Auto-merged by GitHub Actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area::api HTTP REST API surface and route handlers enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants