Skip to content

Add sanity-check notebook for analyze_traps with injected localized singular mode#392

Open
charlesmartin14 wants to merge 1 commit intomasterfrom
codex/create-notebook-to-test-analyze_traps-method-r1yo86
Open

Add sanity-check notebook for analyze_traps with injected localized singular mode#392
charlesmartin14 wants to merge 1 commit intomasterfrom
codex/create-notebook-to-test-analyze_traps-method-r1yo86

Conversation

@charlesmartin14
Copy link
Copy Markdown
Member

Motivation

  • Provide a reproducible, minimal experiment to validate analyze_traps detection of a single injected localized singular mode in a dense matrix.
  • Capture expected localization and top-k mass metrics for a controlled left/right singular vector pair to catch regressions in trap detection.
  • Include a runnable example that demonstrates plotting and artifact return paths of the analyze_traps workflow.

Description

  • Add notebooks/analyze_traps_iid_gaussian_sanity.ipynb which builds a 200 x 400 IID Gaussian matrix and injects one highly localized singular vector pair and an amplified singular value.
  • Wrap the injected matrix in a simple OneLayerMatrix nn.Module, then call watcher.randomize_model and watcher.analyze_traps to detect traps and compute left_top_mass, right_top_mass, and top_5_mass metrics.
  • Compute expected metrics (top_k_mass, localization_ratio) for the injected vectors, assert close agreement with observed trap metrics, and exercise analyze_traps(plot=True) to generate plots.

Testing

  • Executed the notebook end-to-end; the embedded assertions that compare observed vs expected top-5 mass and localization passed successfully.
  • Ran analyze_traps with return_artifacts=True and plot=True in the notebook to ensure both code paths execute without error.
  • No unit test suite changes were made outside the notebook and no CI test failures were introduced.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant