Needs a way of manually refining character list after script generation

[ElBiggus]

Tried a whole bunch of different LLMs to do the script generation, and even added additional text to the prompt to fix it, but I am having a great deal of trouble with character consistency. In the sample text I'm trying there is a character called John Cavendish, but sometimes he's referred to as John, sometimes it's Cavendish, sometimes it's Mr. Cavendish, etc., and I keep ending up with multiple characters in the script that are in fact the same person. The text is also narrated in first person by one of the characters in the story, but no matter what I try I end up with HASTINGS and NARRATOR, and it took some persuading to even get it to stop creating a character called I (from things like '"Blah blah blah," I replied').

In an ideal world the LLM would figure all this out for itself, and I can obviously manually fix the JSON, but adding functionality to merge characters in the UI would make it a lot simpler.

(Not really part of the issue, but is there a recommended LLM to use? I've tried gpt-oss:20b, deepseek-r1:14b, llama3.1:8b, phi4:14b, and a few others, but they all seem to have the same problems.)

[Finrandojin]

Thanks for the detailed feedback — character consistency is definitely one of the trickiest parts of the pipeline.

What's coming

LLM Review Pass (in progress): A second LLM pass that reviews the generated script in batches and fixes common issues automatically — duplicate/inconsistent character names, misattributed lines (narration tagged as dialogue or vice versa), leftover attribution tags ("said he", "I remarked"), and over-fragmented narrator entries. The goal is to catch the kinds of problems you're describing (JOHN vs CAVENDISH vs MR. CAVENDISH) before you ever need to touch the editor. This should also help with the first-person narrator issue where the LLM creates a separate character for "I".

Improved generation prompt: The latest version has a significantly reworked system prompt with stronger rules for narrator vs character separation, interleaved paragraph handling, and attribution tag stripping. This alone should reduce the number of issues that need fixing.

Customizable prompts: The prompt system has been rearchitected — defaults now live in a single editable plain-text file (default_prompts.txt) that hot-reloads without restarting the app. You can tailor the system prompt to your specific text. For example, for a first-person narrated story like yours, you could add context like:

This book is narrated in first person by Hastings. All first-person narration ("I replied", "I noticed", etc.) is NARRATOR, not a character called HASTINGS or I. The character HASTINGS only gets speaking lines when other characters would hear his words.

This kind of per-book guidance in the prompt can make a big difference for tricky texts.

LLM recommendations

For script generation, non-thinking models work best since they output clean JSON without preamble. In rough order of quality:

• Qwen3 Any 30B-A3B-Instruct or 80B-A3B-Instruct model or Qwen2.5 (14B+ recommended)
• Gemma 3 (27B) — strong JSON output and good character distinction but slower
• Llama 3.1/3.2 (8B+) — reliable but occasionally merges characters

The 8B models can work but tend to struggle more with character consistency across long texts. If you can run 14B+, the improvement is significant. Avoid thinking/reasoning models (DeepSeek-R1, etc.) unless you ban the thinking tokens — their internal reasoning gets mixed into the JSON output.

A character merge UI is still a good idea for edge cases — I'll keep this issue open for that. But hopefully between the review pass and the improved prompts, the need for manual merging will be much rarer.

[Finrandojin]

I've merged a bunch of Edit page UI improvements.

• add and delete lines (with undo button for mis-clicks)
• arrow to fit line text for overview and easy review and editing
• Reduced wasted space and made the rows wider

Let me know if there are any other improvements I can make.

[ElBiggus]

I will take a look later, but my attempt had options for "Merge" and "Split":

• Merge Combined the text and instruct fields with the row above (useful for when narration etc. got split into multiple chunks)
• Split Selecting a portion of text and clicking the button split it into two or three chunks (depending on what was selected) - useful for instances where, for example, speech direction got left between lines of dialogue, e.g. "That's good," he said. "Do it again!" could be split into one row with "That's good", one row with he said, and one row with "Do it again!" without needing to manually insert rows and copy/paste stuff.

[Finrandojin]

When you say combine you mean it's a simple append the merge candidate field contents , text and instruct to the end of the merge target? That I can do.

I'll have to think on the split implementation a bit but I can certainly see the use. this type of "he said / She said" narration is why my parser rules just omit the "he said" and go directly to the dialog. I'll play around with it and implement it sometime this weekend in DEV branch. By the way my control buttons are on the right side, is that a problem or you prefer them as in your screenshot?

[ElBiggus]

When you say combine you mean it's a simple append the merge candidate field contents , text and instruct to the end of the merge target?
Yup - it's functionally equivalent to copy/pasting the contents of a chunk into the one above it then deleting the original chunk, but one click rather than several operations.

It's a 30 year old habit of mine to put buttons to the right of the things they apply to as it sort of "makes sense" to me more - I am clicking on the buttons with my mouse which is to the right of my eyes that are looking at the text fields - plus there were already buttons in the column so it felt natural to put mine in there, but it's not the end of the world if they're somewhere else!

(I did have a larger goal of completely rebuilding the UI so that it was essentially an editable/interactive version of my "readable script" thing with "auto split/merge" behaviour (kind of like a bulleted or numbered list in a word processor does), so it was kind of a temporary stepping stone anyway.)

[Finrandojin]

if you have some screenshots or UI mock-ups and ideas on how the edit screen could work put them here I'm kinda flying by the seat of my pants here.

You're right it does kinda make sense on the right side. I'll leave the arrow on the right though, feels more natural there.

[ElBiggus]

I'll make a standalone mock-up of what I had in mind at some point tomorrow and ping it over.

[ElBiggus]

There was nothing worth watching on TV, so here's a standalone crude-but-functional mockup.

mockup-editable-script.html

• Open the page and load a script.JSON file.
• All fields are editable; text field has additional functionality:
  • Hitting Enter at any point in the field will split it into two rows at that point.
  • Deleting the bullet point at the start of the field will merge it with the row above. (It should probably be a paragraph mark instead of a bullet, but I only just thought of that and I can't be bothered to change it now.)

Additional functionality that I didn't bother implementing for demo purposes as they're self-explanatory:

• Undo.
• Editing Speaker shows auto-complete from existing entries.
• Delete row/insert blank row.

It also needs functionality for renumbering/invalidating any previously generated audio chunks to account for changes to the chunk indices when things are inserted or removed, a bunch of sanity checking, etc.

Ideally the whole thing - speaker names, text, etc. - would act like a single "document" so you can navigate through it in a more familiar/consistent/user-friendly manner (you would be amazed how many people still don't know Tab/Shift+Tab navigates between fields!), but hopefully it gives you an idea of the sort of thing I had in mind even if the implementation lacks polish.

A future "nice to have" feature would be a synchronised script/book view - I've had a few instances where I've had to refer back to the source text to fix some dialogue attributed to the wrong person or that has got otherwise mangled, and being able to jump straight to the right spot in the book would have made life much easier. It would require some modification to the initial "chop the book up and pass it to the LLM" flow, but shouldn't be too hairy.

[xenstar]

I think you can integrate this. I tried different pdf to text, none can really generate a good text.

https://github.com/datalab-to/marker