Misc. bug: llama-server integration with Firefox's AI chatbot feature breaks with overly long queries

[chansikpark]

Name and Version

$llama-server --version
version: 6869 (851553ea6)
built with cc (GCC) 15.2.1 20250808 (Red Hat 15.2.1-1) for x86_64-redhat-linux

$snap list | grep firefox
firefox            144.0-2                         7084   latest/stable  mozilla**       -

Operating Systems

Linux

Which llama.cpp Modules Do You Know To Be Affected?

llama-server

Command Line

llama-server -m gpt-oss-20b-mxfp4.gguf

Problem Description & Steps To Reproduce

Background

Pursuant to #16722 closed by PR #16728 as per @allozaur's request.
Mozilla Firefox apparently announced the "gradual rollout" of an "AI Chatbot" feature in January this year following an earlier "initial soft launch". As at writing, the feature does not support out-of-the-box integration with llama.cpp, but can be enabled and customized in about:config with the browser.ml keys. A look at Bugzilla/Core/MachineLearning suggests explicit support is in the works (eg. BZ1970183). 🤷🏼

Issue Description

Llama-server integration with Firefox's AI chatbot feature breaks with overly long queries. Furthermore, when the AI chatbot panel is left open, llama-server appears to hang on just the first time the "Summarize Page" button is pressed and the webpage is too long. On subsequent overly long queries, llama-server appears to correctly return a 414 HTTP error code. Once the AI chatbot panel is closed after making one or more overly long requests, llama-server appears to behave correctly by returning the error code thereafter.

Issue Reproduction

NOTE: It appears that there's an annoyingly sneaky quirk in Firefox where browser.ml.chat.provider must be changed in order for browser.ml.chat.maxLength to kick in.

Integration Limitation: Summarize Long Page Without Changing Max Length

1. Run server: llama-server -m gpt-oss-20b-mxfp4.gguf
2. In about:config, set browser.ml.chat.provider to localhost:8080
3. Browse to an article and click on "Summarize page"

Errant Behaviour: Summarize Long Page After Changing Max Length

1. Restart llama-server to reset prompt cache.
2. Change browser.ml.chat.maxLength to eg. 100,000
3. Close AI chatbot panel and set browser.ml.chat.provider to blank
4. Change browser.ml.chat.provider back to localhost:8080
5. Clicking on the "Summarize page" button the first time produces no logs (even with -DCMAKE_BUILD_TYPE=Debug and llama-server --verbose)
6. Clicking on it a second and third time shows srv log_server_r: request: GET / 414 in llama-server logs each time
7. Interrupting the llama-server process hangs on "cleaning up before exit..."
8. Closing the AI chatbot panel outputs a final 414 before llama-server exits

Initial Investigation

Details

It looks like the external library cpp-httplib sets a compile-time value called CPPHTTPLIB_REQUEST_URI_MAX_LENGTH to 8192. Increasing this value allows for longer query parameters.

It looks like tools/server/utils.hpp might be the right place for a change if there is to be one.

diff --git a/tools/server/utils.hpp b/tools/server/utils.hpp
--- a/tools/server/utils.hpp
+++ b/tools/server/utils.hpp
@@ -13,6 +13,8 @@
 #define CPPHTTPLIB_FORM_URL_ENCODED_PAYLOAD_MAX_LENGTH 1048576
 // increase backlog size to avoid connection resets for >> 1 slots
 #define CPPHTTPLIB_LISTEN_BACKLOG 512
+// increase max URI length to allow for prompts in q URL parameter
+#define CPPHTTPLIB_REQUEST_URI_MAX_LENGTH 16384
 // disable Nagle's algorithm
 #define CPPHTTPLIB_TCP_NODELAY true
 #include <cpp-httplib/httplib.h>

Alternatively, this can be passed to cmake:

cmake -B build -DCMAKE_CXX_FLAGS="-DCPPHTTPLIB_REQUEST_URI_MAX_LENGTH=16384"

Unclear why nothing happens on just the first overly long query. Unclear whether problem is in Firefox or llama-server.

Also, a mildly notable Bugzilla discussion: "are Firefox AI summaries vulnerable to prompt injection?"

Causative Commits

69e9ff0 (Oct 24: q param)
0f2bbe6 (Feb 16: 414 hangs)

Relevant Log Output

main: server is listening on http://127.0.0.1:8080 - starting the main loop
srv  update_slots: all slots are idle
srv  log_server_r: request: GET / 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  update_slots: all slots are idle
srv  log_server_r: request: GET /slots 127.0.0.1 200
srv  log_server_r: request: GET /  414
srv  log_server_r: request: GET /  414
^Csrv    operator(): operator(): cleaning up before exit...
srv  log_server_r: request: GET /  414

[engrtipusultan]

Default context is default: 4096. The way you are running server.

use below command:
llama-server -m /home/tipu/AI/models/ggml-org/GPT-OSS-20B/gpt-oss-20b-mxfp4.gguf --port 8888 --jinja --reasoning-format auto --n-predict -1 --n-gpu-layers 999 --temp 1.0 --top-k 1000 --top-p 1.0 ----ctx-size 131072 --seed -1 --no-mmap --mlock --alias GPT-OSS-20B

adjust: --ctx-size 131072 if you have smaller RAM.
--repeat-penalty is not recommended in updated thread.

For your given problem.

Summary:
Executive Summary

• Covers kernel development scope, common frustrations, and merge benefits
• Highlights GPL‑compatible licensing requirement for contributed code
• Outlines the merge window and release cycle mechanics
• Describes the patch life‑cycle: creation → review → merge
• Emphasizes proper patch formatting, mailing‑list etiquette, and post‑submission follow‑up
• Warns that merging is not the end; ongoing reviewer interaction is essential
• Introduces advanced topics: Git patch management and reviewing others’ patches

Key Development Principles

• Early community involvement is critical for project planning
• Correct coding practice is paramount; common pitfalls are discussed
• Tools and mailing lists support the review and integration process
• Bug‑fixing is suggested as a first exercise for newcomers

What This Document Aims To Convey

• Linux kernel’s scale: > 8 MLOC, ~1 000 contributors per release
• Its evolution from a hobby project to a core OS component across all device types
• The open nature that invites hardware, embedded, distribution, and end‑user developers
• The distinctive, efficient community workflow that differs from proprietary development
• The necessity for newcomers to understand and respect this workflow to avoid frustration

Intended Audience & Guidance

• New kernel contributors looking to integrate code into mainline
• Developers seeking to navigate mailing lists, review comments, and merge timelines
• Teams wanting to align their contributions with community expectations

Credits

• Written by Jonathan Corbet.

[chansikpark]

Appreciate the reproduction effort!

Default context is default: 4096. The way you are running server.

Indeed. I tried to come up with a minimalistic reproduction, got sucked in by the Firefox quirk, and forgot to discuss context size looks like. By my mileage, the given article requires < 3000 tokens with the given model and parameters (ie with the max length workarounds), so increasing the context size does not solve this problem.

For your given problem.

It doesn't look like this summary is covering sections 1.4 The importance of getting code into the mainline, and 1.5 Licensing. This is as expected without increasing the relevant cpp-httplib configuration with eg

cmake -B build -DCMAKE_CXX_FLAGS="-DCPPHTTPLIB_REQUEST_URI_MAX_LENGTH=16384"

tangential

gpt-oss-20b-mxfp4.gguf

IIRC, I had some kind of issue running this quant on my Frankenchina RX 590 GME with DDR3 RAM.. 😢

--top-k 1000 --repeat-penalty 1.05

Hm. 🤔

EDIT: Looks like MXFP4 works for my setup now! Thanks for the nudge @engrtipusultan

[engrtipusultan]

Initially it looked like context issue. I personally dont use front end of llama.cpp server.
Though I think you should use recommended sampling settings. Not related for this issue.
#15396

Also for Quant unsloth Q8 is same size as mxfp4. As per them it is same quality.

Not related to this issue. This is interesting usecase of local inference. Do you know any extension that natively provides this functionality?
Also any variable in about:config for setting auth token.

[chansikpark]

Also for Quant unsloth Q8 is same size as mxfp4. As per them it is same quality.

Yet to do any kind of proper maxxxing, but last I checked the savings were pretty negligible. I will say I did not realize how fast GPT-OSS is with prompt processing on latest llama.cpp, making it quite suitable for summarizing long pages.

Not related to this issue. Do you know any extension that natively provides this functionality?

Not aware of any kind of more intensive implementation other than switching to these new AI browsers coming out, or perhaps Windowsland. But those aren't local inference..

Also any variable in about:config for setting auth token.

Good question. 🤷🏼

This is interesting usecase of local inference.

🙂‍↕️

[chansikpark]

CPPHTTPLIB_REQUEST_URI_MAX_LENGTH was introduced here: yhirose/cpp-httplib#140

Latest HTTP spec states:

https://www.rfc-editor.org/rfc/rfc9110.html#section-15.5.15

The 414 (URI Too Long) status code indicates that the server is refusing to service the request because the target URI is longer than the server is willing to interpret. This rare condition is only likely to occur when a client has improperly converted a POST request to a GET request with long query information, when the client has descended into an infinite loop of redirection (e.g., a redirected URI prefix that points to a suffix of itself) or when the server is under attack by a client attempting to exploit potential security holes.

Also, a notable article on maximum URL lengths from 2023 appears to suggest that Firefox might be okay with hammering servers with massive URLs (300K+). Presumably, this would not work well with remote llama-server access if anything in between the client and llama-server tries to parse the packets as a URL and chokes due to their own idiosyncratic length limit (eg firewalls?).

---

Also, hanging behavior reproducible without Firefox.

eg. with max length set to just 50

wget http://127.0.0.1:8080/12345678901234567890123456789012345678901234567890 --read-timeout=2 --tries=1

It appears that llama-server will wait for the entire duration of the read timeout before outputting 414 in the logs. Or rather, when the client is done waiting and somehow tries to interrupt the request, only then does llama-server try to chime in with the 414. Might be a bit different with multiple concurrent requests..

Unclear whether this is due to intentional or incomplete handling in cpp-httplib or llama-server.

In any case, it looks like there are two sub-issues here:

1. Max length is kind of low (Feature Request)
2. 414 errors should probably return immediately (Bug)

[chansikpark]

@engrtipusultan: Not sure why I couldn't get MXFP4 to work before, but praise be to the contributors that fixed it. For reference:

Summary of Article In Example With Max Length Workaround

Introduction – Linux Kernel Development Guide

---

1. Scope & Purpose

• Covers the kernel development life‑cycle: planning, coding, patching, review, and merging.
• Provides practical advice for new contributors (bug‑hunting, getting code merged, working with reviewers).
• Highlights advanced topics such as Git patch management and reviewing others’ patches.

2. Why Contribute to the Mainline Kernel

• Automatic Availability – code is shipped with all distributions; no separate downloads or driver disks.
• Community Support – bugs and security issues are found early through wide code reviews.
• Reduced Maintenance – out‑of‑tree code requires constant rewrites; mainline code follows a single change‑fix rule.
• Influence on Direction – active contributors shape kernel features and API choices.
• Better User Experience – integrated drivers work out‑of‑the‑box without special support or version mismatches.

3. Problems with Out‑of‑Tree / Binary‑Only Code

• Legal Uncertainty – many developers consider binary modules derived from the kernel, potentially violating GPLv2.
• Debugging & Support – hard to diagnose kernel crashes; community rarely helps.
• Version Friction – must ship a new binary for each kernel/ distro version; increases vendor overhead.
• Security & Quality – absence of community review leads to hidden bugs and vulnerabilities.
• Embedded‑System Misconception – a frozen kernel still requires updates; out‑of‑tree code may lag behind mainline fixes.

4. Licensing & Contribution Rules

• All code must be GPLv2‑compatible (GPLv2 or the three‑clause BSD license).
• No copyright assignment required; original ownership is retained.
• Sign‑off Requirement – each contributor must claim the code is licensed under GPL and can be distributed with the kernel.
• Only free, clear‑licensed code is accepted; anonymous or unidentifiable contributors are rejected.
• Legal questions should be addressed by a lawyer, not by mailing‑list responders.

5. Recognized Contributors

• Authored by Jonathan Corbet; improved by many kernel developers (e.g., Andreas Oskar, Andrew Morton, Jiri Kosina, etc.).
• Supported by the Linux Foundation and contributors’ community.

---

Bottom Line:
Contributing to the mainline kernel brings automatic distribution, community quality assurance, and long‑term maintenance benefits, while avoiding the licensing, support, and security pitfalls that accompany out‑of‑tree and binary‑only solutions. Follow the documented processes and licensing rules to ensure your patches are accepted and well‑maintained.

Logs from Generation

llama-server -m /mnt/Media/LLMs/gpt-oss-20b-mxfp4.gguf --jinja --temp 1.0 --top-k 100 --min-p 0 --top-p 1.0 --ctx-size 0
...
main: server is listening on http://127.0.0.1:8080 - starting the main loop
srv  update_slots: all slots are idle
srv  log_server_r: request: GET / 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  log_server_r: request: GET /slots 127.0.0.1 200
srv  update_slots: all slots are idle
srv  log_server_r: request: GET / 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  update_slots: all slots are idle
srv  log_server_r: request: GET /slots 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  params_from_: Chat format: GPT-OSS
slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
slot launch_slot_: id  0 | task 2 | processing task
slot update_slots: id  0 | task 2 | new prompt, n_ctx_slot = 131072, n_keep = 0, n_prompt_tokens = 3029
slot update_slots: id  0 | task 2 | n_past = 0, memory_seq_rm [0, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_past = 2048, n_tokens = 2048, progress = 0.676131
slot update_slots: id  0 | task 2 | n_past = 2048, memory_seq_rm [2048, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_past = 2965, n_tokens = 917, progress = 0.978871
slot update_slots: id  0 | task 2 | n_past = 2965, memory_seq_rm [2965, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_past = 3029, n_tokens = 64, progress = 1.000000
slot update_slots: id  0 | task 2 | prompt done, n_past = 3029, n_tokens = 64
slot update_slots: id  0 | task 2 | created context checkpoint 1 of 8 (pos_min = 2325, pos_max = 2964, size = 15.008 MiB)
srv  log_server_r: request: GET /slots 127.0.0.1 200
slot print_timing: id  0 | task 2 | 
prompt eval time =   20106.17 ms /  3029 tokens (    6.64 ms per token,   150.65 tokens per second)
       eval time =   64569.92 ms /   726 tokens (   88.94 ms per token,    11.24 tokens per second)
      total time =   84676.08 ms /  3755 tokens
slot      release: id  0 | task 2 | stop processing: n_past = 3754, truncated = 0
srv  update_slots: all slots are idle
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
^Csrv    operator(): operator(): cleaning up before exit...
llama_memory_breakdown_print: | memory breakdown [MiB]                    | total             free     self   model   context   compute       unaccounted |
llama_memory_breakdown_print: |   - Vulkan1 (RX 590 GME (RADV POLARIS10)) |  7936 = 17592186037917 + (14449 = 10949 +    3087 +     413) + 17592186044401 |
llama_memory_breakdown_print: |   - Host                                  |                             849 =   586 +       0 +     262                   |

[chansikpark]

The hanging behaviour can be reproduced with a minimal cpp-httplib server. Basically, one must wait for the lesser of the client's or server's (CPPHTTPLIB_SERVER_READ_TIMEOUT_SECOND) read timeout before a 414 error returns.

Minimal Logging HTTP Server

#define CPPHTTPLIB_REQUEST_URI_MAX_LENGTH 50
#define CPPHTTPLIB_SERVER_READ_TIMEOUT_SECOND 3
#include "vendor/cpp-httplib/httplib.h"

int main() {
  // HTTP
  httplib::Server svr;

  svr.Get("/hi", [](const httplib::Request &, httplib::Response &res) {
    res.set_content("Hello World!", "text/plain");
  });

  svr.set_logger([](const httplib::Request& req, const httplib::Response& res) {
    std::cout << req.method << " " << req.path << " -> " << res.status << std::endl;
  });

  svr.set_error_logger([](const httplib::Error& err, const httplib::Request* req) {
    std::cerr << httplib::to_string(err) << " while processing request";
    if (req) {
      std::cerr << ", client: " << req->get_header_value("X-Forwarded-For")
                << ", request: '" << req->method << " " << req->path << " " << req->version << "'"
                << ", host: " << req->get_header_value("Host");
    }
    std::cerr << std::endl;
  });

  svr.listen("0.0.0.0", 8080);
  return 0;
}

After stepping through the vendor code, I found that the following fixes the hanging:

diff --git a/vendor/cpp-httplib/httplib.h b/vendor/cpp-httplib/httplib.h
index db55d07e2..efcf6fe40 100644
--- a/vendor/cpp-httplib/httplib.h
+++ b/vendor/cpp-httplib/httplib.h
@@ -8308,8 +8308,6 @@ Server::process_request(Stream &strm, const std::string &remote_addr,
 
   // Check if the request URI doesn't exceed the limit
   if (req.target.size() > CPPHTTPLIB_REQUEST_URI_MAX_LENGTH) {
-    Headers dummy;
-    detail::read_headers(strm, dummy);
     res.status = StatusCode::UriTooLong_414;
     output_error_log(Error::ExceedUriMaxLength, &req);
     return write_response(strm, close_connection, req, res);

I've submitted a pull request for this upstream.

yhirose/cpp-httplib#2260

[engrtipusultan]

Also for Quant unsloth Q8 is same size as mxfp4. As per them it is same quality.

Yet to do any kind of proper maxxxing, but last I checked the savings were pretty negligible. I will say I did not realize how fast GPT-OSS is with prompt processing on latest llama.cpp, making it quite suitable for summarizing long pages.

Not related to this issue. Do you know any extension that natively provides this functionality?

Not aware of any kind of more intensive implementation other than switching to these new AI browsers coming out, or perhaps Windowsland. But those aren't local inference..

Also any variable in about:config for setting auth token.

Good question. 🤷🏼

This is interesting usecase of local inference.

🙂‍↕️
Offtopic.
@chansikpark I found one extension that looks promising. Did not test thoroughly.
github.com/n4ze3m/page-assist

[chansikpark]

github.com/n4ze3m/page-assist

Just from the docs, it look like they've got everything Firefox is giving but with RAG and Vision! I think in terms of QoL, Firefox native can shave a few milliseconds with UI integration, but it certainly sets a bar for their forthcoming implementation.

Gives me a reason to refresh on SOTA RAG. Thank you @engrtipusultan !

Given the OAI API support, I'd presume that this extension would not have the problems listed above, while providing Firefox AI integration.

Notably, the beta and nightly builds (145.0b8-1 and 146.0a1) have a place for API key somewhere in browser.ml.smartAssist.*. Couldn't figure out what those keys do though.