This repository was archived by the owner on Aug 3, 2026. It is now read-only.
validator: subprocess-isolate op4 canonical eval (corrected #61) - #75
Merged
Conversation
Supersedes PR#61 with its critical env-leak fixed + reconciled onto main. - extract shared _run_eval_subprocess: BOTH the canonical and patched op4 paths now run eval_in_workdir.py in a child process, so a fatal CUDA fault (illegal access / device-side assert) kills only the child and the validator rejects the bundle instead of aborting (a C++ CUDA abort can't be caught in-process). The default RALPH_SANDBOX=0 path gains crash-resilience the container only provides when enabled. - KEEPS env sanitization: the child gets env=_sanitized_env(...) (no seal privkey / wallet / tokens) + _redacted stderr. PR#61's first cut DROPPED this, silently reverting PR#70 and leaking RALPH_VALIDATOR_PRIVKEY to miner code. New test asserts no secret reaches the subprocess (the gap CI missed). - integrated with the now-merged op4 result cache + RALPH_SANDBOX dispatch (PR#61 was on a stale pre-sandbox/pre-cache base) - drop now-unused run_hidden_eval import Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… CI) The op4 canonical path now runs in a subprocess with an allowlist-only env, which (correctly) strips secrets — but it also stripped the validator's own RALPH_ALLOW_SYNTHETIC_EVAL, so the eval harness in the child fail-closed where the in-process path used synthetic (CI / testnet, no held-out shard). This broke test_op4_canonical_path_end_to_end in CI. Forward only that toggle (read by run_hidden_eval in the child; never set on mainnet => still fail-closed there; not miner-settable). Secrets + the real enforcement toggles (SKIP_HANDSHAKE, ALLOW_MOCK_ATTESTATION, TEST_MODE) stay blocked. Test asserts it's forwarded when set and absent when not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to forwarding the validator's own synthetic-eval toggle: the patched-subprocess env-sanitization test asserted it was blocked. It's still in _TRAINING_ENV_BLOCKLIST (so _sanitized_env scrubs it — the blocklist + direct scrub tests are unchanged), but _run_eval_subprocess re-adds it because the canonical eval harness in the child needs it. Update the integration test to assert it IS forwarded while secrets + the real enforcement toggles (SKIP_HANDSHAKE, ALLOW_MOCK_ATTESTATION, TEST_MODE) stay blocked. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes PR#61 with its critical env-leak fixed + reconciled onto main.