From 070352ee2548a6ea34f02f6167f9d393c8768869 Mon Sep 17 00:00:00 2001 From: Ranran Haoran Zhang Date: Sat, 25 Jul 2026 15:05:53 -0500 Subject: [PATCH] docs: correct stale prose about the pre-tokenized on-policy path Two places describe behaviour the code stopped having. The response-regeneration tutorial documents the output row as carrying `conversations`, a list of role/content messages. #808 replaced that with `text`, a decode of `input_ids`, and updated docs/cli/response_regeneration.md but not the tutorial. Both example rows are corrected, and the `metadata` blocks gain the `sampling_params` key the script emits. scripts/prepare_data.py's module docstring claims it always applies a chat template and produces a mask. Since #729, rows that already carry `input_ids` and `loss_mask` skip both steps; they are truncated and filtered only. Note the exception rather than leaving the docstring stating it unconditionally. No code or behaviour change. Signed-off-by: Ranran Haoran Zhang --- .../tutorials/response_regeneration.md | 20 +++++++------------ scripts/prepare_data.py | 4 ++++ 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/docs/user_guide/tutorials/response_regeneration.md b/docs/user_guide/tutorials/response_regeneration.md index 47488f0eb..9cd54e2da 100644 --- a/docs/user_guide/tutorials/response_regeneration.md +++ b/docs/user_guide/tutorials/response_regeneration.md @@ -86,21 +86,19 @@ The output is a JSONL file with one pre-tokenized row per target generation. `lo "primary_id": "conv-abc", "input_ids": [151644, 872, ...], "loss_mask": [0, 0, ..., 1, 1], - "conversations": [ - {"role": "user", "content": "What is the capital of France?"}, - {"role": "assistant", "content": "The capital of France is Paris."} - ], + "text": "<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\nThe capital of France is Paris.<|im_end|>", "metadata": { "idx": 0, "finish_reason": "stop", "is_tool_call": false, "usage": {...}, - "endpoint": "http://127.0.0.1:8000/v1/chat/completions" + "endpoint": "http://127.0.0.1:8000/v1/chat/completions", + "sampling_params": {...} } } ``` -Each assistant turn produces at least one row — and more when the target calls a tool, since every call is its own generation — so expect more lines than input conversations. `conversations` is a review-only twin of `input_ids`; training drops it. +Each assistant turn produces at least one row — and more when the target calls a tool, since every call is its own generation — so expect more lines than input conversations. `text` is the same tokens decoded back to a string (`tokenizer.decode`, special tokens kept), so reading it tells you exactly what `input_ids` holds. Training drops it. For multi-turn datasets, later turns include the regenerated history as context. For example, the second turn of the same conversation would be: @@ -110,18 +108,14 @@ For multi-turn datasets, later turns include the regenerated history as context. "primary_id": "conv-abc", "input_ids": [151644, 872, ...], "loss_mask": [0, 0, ..., 1, 1], - "conversations": [ - {"role": "user", "content": "What is the capital of France?"}, - {"role": "assistant", "content": "The capital of France is Paris."}, - {"role": "user", "content": "What about Germany?"}, - {"role": "assistant", "content": "The capital of Germany is Berlin."} - ], + "text": "<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\nThe capital of France is Paris.<|im_end|>\n<|im_start|>user\nWhat about Germany?<|im_end|>\n<|im_start|>assistant\nThe capital of Germany is Berlin.<|im_end|>", "metadata": { "idx": 0, "finish_reason": "stop", "is_tool_call": false, "usage": {...}, - "endpoint": "http://127.0.0.1:8000/v1/chat/completions" + "endpoint": "http://127.0.0.1:8000/v1/chat/completions", + "sampling_params": {...} } } ``` diff --git a/scripts/prepare_data.py b/scripts/prepare_data.py index cac9b5048..1fc06b359 100644 --- a/scripts/prepare_data.py +++ b/scripts/prepare_data.py @@ -7,6 +7,10 @@ 2. Produces a loss/assistant mask for each sample 3. Records token frequency statistics +Rows that already carry input_ids and loss_mask, as on-policy response +regeneration emits, skip steps 1 and 2. They are truncated to --seq-length and +filtered, but never re-tokenized or re-masked. + The output of this script is: 1. Processed dataset ready for online training or offline datagen in output_dir 2. Token frequency statistics file at token_freq_path