Prose Rewriter Web UI

A local web front end for the prose-rewriter GGUF releases: paste prose written by an LLM, get back a humanized version with meaning intact.

The UI has a built-in downloader for the models. Please run the fetch llama script to grab the llama.cpp installation.

paste → one paragraph per slot → streamed back into place

---

What you need

Python	3.10+, standard library only — nothing to pip install
llama-server	any recent llama.cpp build; bin/fetch-llama.sh downloads one
A GPU	optional. --gpu-layers 0 runs on CPU, slowly
Disk	2.2–4.7 GB per model you download
An OS	Linux, macOS or Windows 10+. Each has its own scripts in bin/

Quick start

git clone https://github.com/OrbFrontend/ProseRewriterWebUI
cd ProseRewriterWebUI

bin/fetch-llama.sh          # ~35 MB prebuilt llama-server, no toolchain needed
bin/webui.sh                # http://localhost:8090

On Windows, the same two steps under different names:

git clone https://github.com/OrbFrontend/ProseRewriterWebUI
cd ProseRewriterWebUI

bin\fetch-llama.cmd
bin\webui.cmd

The first run has no weights. Open the page, pick a model, press Download, and it lands in models/ and stays there.

Getting a llama-server

bin/fetch-llama.sh                 # prebuilt. Vulkan on Linux, Metal on macOS
bin/fetch-llama.sh --cpu           # prebuilt, CPU only, ~17 MB
bin/build-llama.sh                 # from source; uses CUDA if nvcc is present
LLAMA_SERVER=/path/to/llama-server bin/webui.sh    # one you already have

On Windows there is no build script, because there is nothing worth building: the prebuilt is a 33 MB download and the alternative is a multi-gigabyte Visual Studio install and an hour of compiling to gain a few percent.

bin\fetch-llama.cmd                 :: prebuilt, Vulkan, ~33 MB
bin\fetch-llama.cmd --cpu           :: prebuilt, CPU only, ~18 MB

or one you already have, from cmd:

set LLAMA_SERVER=C:\path\to\llama-server.exe
bin\webui.cmd

Vulkan is the default because it is the one GPU backend that does not care whose card it is. NVIDIA, AMD and Intel all expose it through the ordinary display driver, so there is no CUDA toolkit and no ROCm install between you and an offloaded model — one 35 MB tarball, unpack, run. It gives up a few percent against a native CUDA build on the same card; bin/build-llama.sh is there for when you want those back. On macOS the standard build already has Metal, so there is no choice to make.

The three models

All three are the same training pool and the same pipeline, published under AGPLv3.

Selector	Repository	Size	Notes
1.7B · Q8_0	prose-rewriter-1.7b-v1.2	2.17 GB	fastest
4B · Q4_K_M	prose-rewriter-4b-v1.2	2.72 GB	the larger model at the smaller footprint
4B · Q8_0	prose-rewriter-4b-v1.2	4.69 GB	best quality

Command line

bin/webui.sh [options]          # bin\webui.cmd [options] on Windows

  --host HOST         default 127.0.0.1; use 0.0.0.0 to let the room reach it
  --port PORT         default 8090
  --model ID          1.7b-q8 | 4b-q4km | 4b-q8
  --slots N           parallel llama.cpp sequences (default 4) - more = more VRAM usage
  --gpu-layers N      layers to offload; 0 is CPU-only (default: all)
  --threads N         CPU threads for llama.cpp
  --models-dir PATH   where the .gguf files live (default: ./models)
  --no-load           boot with nothing loaded; pick in the page

Environment: LLAMA_SERVER, PROSE_REWRITER_MODELS, PROSE_REWRITER_MODEL, SLOTS, GPU_LAYERS, HOST, PORT, and PYTHON for the interpreter the launcher scripts use. In PowerShell these are set as $env:NAME = "value".

Limitations

These are the model's, and they are not worked around here:

• Fictional prose only. Technical documentation is out of distribution.
• One paragraph per call, 80 bytes to 512 tokens. Longer degrades; split it.
• English, narrative register.
• It will not pass AI detectors. It preserves word choices and certain sentence structures, and Pangram and such will still know.
• Not an instruct model. It has one job and one prompt. There is nothing to ask it.

Licence

GNU Affero General Public License v3. The full text is in LICENSE.

The model weights this front end serves are released under the same licence and are derivatives of Qwen/Qwen3-1.7B-Base and Qwen/Qwen3-4B-Base, which are Apache-2.0; that licence is preserved and continues to apply to the base weights. llama.cpp is MIT and is not vendored into this repository. bin/fetch-llama.sh downloads its published binaries (bin\fetch-llama.cmd on Windows) and bin/build-llama.sh clones its source; both land under vendor/, which git ignores.

If you run a modified version of this as a network service, AGPL section 13 requires you to offer the corresponding source of your modifications to its users.