Skip to content

docs: align pipeline and dataset guidance - #34

Merged
bingran-you merged 1 commit into
mainfrom
bry/audit-latest-pipeline-docs
Jul 15, 2026
Merged

docs: align pipeline and dataset guidance#34
bingran-you merged 1 commit into
mainfrom
bry/audit-latest-pipeline-docs

Conversation

@bingran-you

Copy link
Copy Markdown
Contributor

Summary

  • Audit every project-facing document and update the 18 files that contain pipeline, dataset, compatibility, security, or evidence claims.
  • Make the executable contract consistent everywhere: Qwen3.5-9B student, Qwen3.5-397B-A17B teacher, OpenCode for teacher/evaluation/GRPO rollouts, one-epoch bf16 LoRA SFT, and one-epoch TRL LoRA GRPO.
  • Pin the public examples to the current 2,238-task train and 366-task eval repositories, while separating those public references from participant-corpus training and the sealed competition eval set.
  • Reclassify older Qwen3-4B, OpenEnv, soccer, and HF Jobs runs as historical evidence and scope the 16x14 result as exploratory same-domain uplift rather than competition generalization.
  • Document the canary provenance boundary, OpenEnv exposure requirements, HF Jobs topology gap, and private task-ID redaction rules.

This is the smallest appropriate change because executable code and configs already implement the target pipeline; the remaining drift was in repository documentation.

Validation

  • Existing task checks pass when relevant
  • Pipeline tests and documented CLI checks pass when relevant
  • No credentials, private eval content, checkpoints, or raw provider dumps are included
  • Dataset, model, and external dependency revisions are pinned when relevant
  • Public docs and evidence boundaries are updated when behavior or claims change
  • Compatibility changes update docs/architecture-status.md and retain existing contract coverage

Detailed checks:

  • 238 pipeline tests passed
  • Ruff and Python compilation passed for the pipeline package
  • Starting-kit structural checks and submission checks passed
  • Seven checked-in recipes validated; the full plan resolved to 2,238 train tasks, 366 eval tasks, Qwen3.5-9B, the Qwen3.5-397B-A17B teacher, and one epoch each for SFT and GRPO
  • Qwen3.5 HF Job bundle dry-run passed
  • Markdownlint passed on all 18 changed docs
  • Modified-doc external and relative link checks passed
  • Live Hugging Face verification matched all four pinned model/dataset SHAs; datasets contained 2,238/366 task.md packages and zero task.toml files
  • Diff whitespace and credential-pattern scans passed

Spend and external systems

No spendful validation. This PR uses the already-preserved public canary evidence and no GPU, provider, Daytona, or paid HF Job was launched. A paid HF Jobs run for the current Docker/OpenCode/two-GPU topology remains an explicit manual validation item.

@bingran-you
bingran-you merged commit 79218df into main Jul 15, 2026
3 checks passed
@bingran-you
bingran-you deleted the bry/audit-latest-pipeline-docs branch July 15, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant