Skip to content

Add optional trap variance-burden metrics to analyze_traps (trap_burden, variants)#362

Open
charlesmartin14 wants to merge 1 commit intomasterfrom
codex/update-pr361-to-extend-analyze_traps
Open

Add optional trap variance-burden metrics to analyze_traps (trap_burden, variants)#362
charlesmartin14 wants to merge 1 commit intomasterfrom
codex/update-pr361-to-extend-analyze_traps

Conversation

@charlesmartin14
Copy link
Copy Markdown
Member

Motivation

  • Provide optional, theory- and empirically-motivated trap "variance burden" metrics that quantify spectral excess, localization, and overlap structure for detected traps without changing existing defaults or column names.
  • Compute these heavier diagnostics only when explicitly requested to avoid extra cost and preserve backward compatibility with previous analyze_traps behavior.

Description

  • Added two optional arguments to analyze_traps: trap_burden (default False) and trap_burden_variant (default "top5"), and passed them through into params so default runs are unchanged.
  • Implemented compute_bulk_trap_reference_metrics to compute bulk-mode baselines (bulk_top_5_mass_mean, bulk_top_5_mass_std, bulk_ipr_mean, bulk_ipr_std) by unpermuting bulk rank-1 modes and computing top-mass and IPR per mode.
  • Extended analyze_single_trap to (when trap_burden=True) compute eval_perm, spectral_excess_abs, spectral_excess_rel, trap_ipr, ipr_lift, ipr_lift_excess_pos, top_5_lift, top_5_lift_excess_pos, log1p_top_5_lift, overlap diagnostics (ov_lam_weighted_mean, ov_lam_weighted_var, ov_rank_mean, ov_rank_std, ov_hhi, ov_participation, ov_entropy, ov_max), and burden aggregates (trap_variance_burden_ipr, trap_variance_burden_top5, and canonical trap_variance_burden selected by trap_burden_variant).
  • Added helper methods _top_mass_fraction (computes fraction of absolute mass in top 5% entries) and _compute_overlap_metrics (computes overlap distribution p_a from X = W_true.T @ W_true and returns weighted means/vars/rank stats and diagnostics).
  • Appended all new columns to the trap result schema in _trap_result_columns (no renames or reordering of existing columns).
  • Added README documentation section Trap variance burden describing both IPR and Top5 burden formulas and interpretation.

Testing

  • Added tests in tests/test_analyze_traps.py that verify backward compatibility when calling analyze_traps(..., trap_burden=False), presence of new columns when trap_burden=True, finite-valued diagnostics when traps exist, correct trap_variance_burden selection for trap_burden_variant="ipr" and "top5", and that invalid trap_burden_variant raises ValueError.
  • Ran pytest -q tests/test_analyze_traps.py in this environment and the suite completed successfully with tests skipped due to missing Torch runtime (16 skipped).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant