Skip to content
This repository was archived by the owner on Aug 3, 2026. It is now read-only.

op4: pin hidden-eval seq_len validator-side (not miner) - #77

Merged
bitzic merged 1 commit into
mainfrom
fix/op4-pin-eval-seq-len
Jun 28, 2026
Merged

op4: pin hidden-eval seq_len validator-side (not miner)#77
bitzic merged 1 commit into
mainfrom
fix/op4-pin-eval-seq-len

Conversation

@bitzic

@bitzic bitzic commented Jun 28, 2026

Copy link
Copy Markdown
Contributor
  • add EVAL_SEQ_LEN=512 + pinned_eval_seq_len() single source of truth
  • every eval path pins min(EVAL_SEQ_LEN, max_seq_len): sandbox_eval, eval_in_workdir, audit (x2) — was miner-controlled cfg.max_seq_len//2
  • sandbox host re-derives seq_len from the checkpoint config and REJECTS a container that echoes a mismatched manifest seq_len (verify, not trust)
  • closes the gap where a miner enlarges max_seq_len to be scored on an easier longer-context eval than the king; also makes models comparable
  • stage 1 of the op4 causality redesign (HOSB)
  • tests: pin caps/floors; tampered-seq_len rejected; sandbox expectations
  • note: eval window shifts max_seq_len//2 -> min(512,max_seq_len); king is re-scored under the same pin each epoch so comparison stays like-with-like

- add EVAL_SEQ_LEN=512 + pinned_eval_seq_len() single source of truth
- every eval path pins min(EVAL_SEQ_LEN, max_seq_len): sandbox_eval,
  eval_in_workdir, audit (x2) — was miner-controlled cfg.max_seq_len//2
- sandbox host re-derives seq_len from the checkpoint config and REJECTS
  a container that echoes a mismatched manifest seq_len (verify, not trust)
- closes the gap where a miner enlarges max_seq_len to be scored on an
  easier longer-context eval than the king; also makes models comparable
- stage 1 of the op4 causality redesign (HOSB)
- tests: pin caps/floors; tampered-seq_len rejected; sandbox expectations
- note: eval window shifts max_seq_len//2 -> min(512,max_seq_len); king is
  re-scored under the same pin each epoch so comparison stays like-with-like

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bitzic
bitzic merged commit bc85157 into main Jun 28, 2026
4 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant