Retarget prompts to the actual validating GPU architecture (root cause) - #54
Merged
Conversation
Root-cause fix for the task-label vs hardware mismatch behind the guard-removal validator-hacks: prompts bake a GPU/SM label at build time, but a miner can validate on a different GPU than the prompt declared (e.g. "Hopper SM90" prompts run on Blackwell SM12x). The teacher is then asked for the wrong arch, adds an SM/capability guard, and it fails validation — leading to a repair that "passes" by deleting the guard. sparkproof-triton-generate now detects the validating GPU (require_supported_gpu) and, per prompt, calls build_prompts.reconcile_prompt_architecture to rewrite the SM label in the system/prompt text and set gpu_architecture to the real GPU. No-op when the declared arch matches or no GPU is detectable (dev/CPU falls back cleanly). Prints how many prompts were retargeted. Complements #51 (portable-prompt nudge) and #53 (guard-removal flag) by removing the mismatch at the source. Tests cover reconcile (retarget, no-op match, no-arch, same-label arch-field-only update).
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The root-cause fix for the dataset-quality thread (#51 prompt nudge, #53 guard-removal flag). It removes the mismatch that causes the validator-hack rather than mitigating it.
Problem
Prompts bake a GPU/SM label at build time (
gpu_architecture→sm_label). A miner can validate on a different GPU than the prompt declared — e.g.magicrails's"Hopper SM90"translation prompts were validated on Blackwell SM120 (RuntimeError: targets Hopper SM90, found SM120). The teacher, told "Hopper SM90", adds an SM guard that then fails, and the repair "passes" by deleting it.Fix
sparkproof-triton-generatenow:require_supported_gpu(args.gpu)["gpu_architecture"]), once.gpu_architecturediffers, callsbuild_prompts.reconcile_prompt_architecture— rewrites the SM label in thesystem/prompttext and setsgpu_architectureto the real GPU.Now the teacher is always prompted for the actual hardware, so the SM-guard failure class doesn't arise in the first place, and the dataset is labeled with the real arch. No-op when the declared arch already matches, or when no GPU is detectable (dev/CPU falls back cleanly, reconciliation disabled).
Tests (+4, 370 total, ruff clean)
test_arch_reconcile.py: retarget hopper→blackwell (label rewritten in system+prompt,gpu_architectureupdated), no-op on match, no-op without a declared arch, and same-label archs (h100→h200) update only the arch field.