Add transparent NextGen rarity analysis tooling - #4
Conversation
📝 WalkthroughWalkthroughThis change adds documented NextGen rarity-analysis rules, a compatibility analyzer for dated JSON snapshots, a CLI, deterministic hashing and ranking, validation policies, synthetic fixtures, and comprehensive unittest coverage. ChangesNextGen rarity analysis
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant CLI
participant Analyzer
participant Snapshot
participant Output
Operator->>CLI: Submit snapshot path and duplicate policy
CLI->>Snapshot: Load and validate JSON
CLI->>Analyzer: Analyze normalized snapshot
Analyzer->>Analyzer: Calculate scores, ranks, and hashes
Analyzer-->>CLI: Return analysis result
CLI->>Output: Write JSON to stdout or file
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
6529bot WCAG 2.2 AA analysis - ec3001cVerdict: No WCAG findings This PR adds documentation (Markdown), a Python CLI/module, JSON test fixtures, and unit tests. There is no user-facing UI code (no HTML, JSX/TSX, components, templates, or styling) in the changed files, so WCAG 2.2 AA success criteria do not apply here. No accessibility regressions or usability barriers are in scope for this change. |
6529bot crypto security analysis - ec3001cVerdict: No security findings This PR adds a self-contained, offline Python analysis tool with no wallet, auth, signature, token, or web3 behavior. I reviewed against the crypto/security focus areas and found nothing exploitable in the changed code. Notes supporting the verdict:
One correctness caveat worth flagging for the author (not a security issue): the pinned hashes in |
6529bot i18n analysis - ec3001cVerdict: No i18n findings This PR adds developer/operator-facing artifacts only: Markdown docs and research notes, a Python CLI (
No fallback-debt note is required because no frontend surface is touched. |
6529bot general PR review - ec3001cVerdict: Needs changes Important
Nice-to-have
Suggested next steps
Note: I did not independently verify the numeric fixture values against the upstream Prompt for all review comments with AI agents |
|
/6529bot followup |
6529bot follow-up commit review - b17d3b0Verdict: Needs changes Important
Nice-to-have
Resolved since last review
Suggested next steps
Prompt for all review comments with AI agents |
6529bot follow-up commit review - b17d3b0Verdict: Needs changes Important
Nice-to-have
Resolved since last review
Suggested next steps
Note: I did not independently verify the pinned float/hash fixture values against upstream Prompt for all review comments with AI agents |
…tooling # Conflicts: # INDEX.md
|
@coderabbitai review |
|
/6529bot followup |
✅ Action performedReview finished.
|
|
/6529bot review general security media-external privacy-evidence glm-swarm |
|
/6529bot review general security media-external privacy-evidence glm-swarm |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
/6529bot followup |
|
Exact-head re-review request for independent reviewer task |
|
Exact-head re-review for independent reviewer task |
|
/6529bot followup |
|
Exact-head re-review for independent reviewer task |
|
/6529bot followup |
|
Exact-head re-review request for independent reviewer task |
|
/6529bot followup |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
scripts/rarity/__init__.py (1)
3-15: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExport
InputErrorfrom the package surface.The package exports the four analysis functions but not the exception they raise. A consumer that imports only
scripts.raritycannot catchInputErrorwithout reaching intoscripts.rarity.nextgen_compat. Add it to the public surface.♻️ Proposed export addition
from .nextgen_compat import ( + InputError, analyze_snapshot, canonical_json, load_snapshot, normalize_snapshot, ) __all__ = [ + "InputError", "analyze_snapshot", "canonical_json", "load_snapshot", "normalize_snapshot", ]🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/rarity/__init__.py` around lines 3 - 15, Export InputError from scripts.rarity by importing it alongside the existing symbols from nextgen_compat and adding "InputError" to __all__, so consumers can catch the package’s public exception directly.tests/rarity/test_nextgen_compat.py (1)
407-419: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a CLI success-path test.
The CLI tests cover exit code
1for bad data and exit code2for bad invocation. No test covers the documented exit code0with a written output file. Add a case that runsmain([str(FIXTURE), "--output", str(path)]), asserts the return value is0, and asserts the written JSON parses and matchesanalyze_snapshot.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/rarity/test_nextgen_compat.py` around lines 407 - 419, Add a success-path test alongside test_cli_distinguishes_bad_data_from_bad_invocation that invokes main with FIXTURE and an --output temporary path, verifies it returns 0, then reads and parses the output JSON and compares it with analyze_snapshot’s result for the fixture.scripts/rarity/nextgen_compat.py (1)
147-159: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winConsider rejecting duplicate JSON object keys in
load_snapshot.
json.loadkeeps the last value for a repeated object key and drops the earlier one silently.input_snapshot_sha256then commits to the parsed object, not to what the file actually contained, while the documentation calls it "the canonical hash of the raw supplied snapshot". Anobject_pairs_hookcloses that gap.♻️ Proposed duplicate-key guard
def reject_json_constant(value: str) -> None: raise InputError( f"non-finite numeric value is prohibited: JSON constant {value}" ) + def reject_duplicate_keys(pairs: list[tuple[str, Any]]) -> dict[str, Any]: + seen: set[str] = set() + for key, _ in pairs: + if key in seen: + raise InputError(f"duplicate JSON object key is prohibited: {key}") + seen.add(key) + return dict(pairs) + with Path(path).open("r", encoding="utf-8") as handle: - value = json.load(handle, parse_constant=reject_json_constant) + value = json.load( + handle, + parse_constant=reject_json_constant, + object_pairs_hook=reject_duplicate_keys, + )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/rarity/nextgen_compat.py` around lines 147 - 159, Update load_snapshot to pass an object_pairs_hook that detects repeated keys while parsing and raises InputError identifying the duplicate key, while preserving the existing UTF-8 loading, non-finite-number rejection, and root-object validation behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/rarity/nextgen_compat.py`:
- Around line 181-189: Bound recursion in _reject_non_finite_values,
_reject_precomputed_metric_fields, and _validate_citation_value with a shared
depth parameter and explicit maximum nesting limit; increment it for nested
dict/list traversal and raise InputError when exceeded, so deeply nested
snapshots produce the CLI’s concise error instead of RecursionError. Preserve
existing validation behavior for inputs within the limit.
In `@tests/rarity/test_nextgen_compat.py`:
- Around line 285-294: Remove the version-dependent sum(values) equality
assertion from test_left_fold_matches_javascript_reduce_not_python_sum; preserve
the javascript_reduce and _left_to_right_sum assertions unchanged.
---
Nitpick comments:
In `@scripts/rarity/__init__.py`:
- Around line 3-15: Export InputError from scripts.rarity by importing it
alongside the existing symbols from nextgen_compat and adding "InputError" to
__all__, so consumers can catch the package’s public exception directly.
In `@scripts/rarity/nextgen_compat.py`:
- Around line 147-159: Update load_snapshot to pass an object_pairs_hook that
detects repeated keys while parsing and raises InputError identifying the
duplicate key, while preserving the existing UTF-8 loading, non-finite-number
rejection, and root-object validation behavior.
In `@tests/rarity/test_nextgen_compat.py`:
- Around line 407-419: Add a success-path test alongside
test_cli_distinguishes_bad_data_from_bad_invocation that invokes main with
FIXTURE and an --output temporary path, verifies it returns 0, then reads and
parses the output JSON and compares it with analyze_snapshot’s result for the
fixture.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 3a41b2a6-4297-45e2-b622-590abd0e58a1
📒 Files selected for processing (11)
INDEX.mddocs/generative-trait-analysis.mdnotes/wip/2026-08-01-nextgen-rarity-analysis.mdscripts/rarity/__init__.pyscripts/rarity/analyze.pyscripts/rarity/nextgen_compat.pytests/__init__.pytests/rarity/__init__.pytests/rarity/fixtures/nextgen-compatibility.expected.jsontests/rarity/fixtures/nextgen-compatibility.jsontests/rarity/test_nextgen_compat.py
| def _reject_non_finite_values(value: Any, path: str = "snapshot") -> None: | ||
| if isinstance(value, float) and not math.isfinite(value): | ||
| raise InputError(f"non-finite numeric value is prohibited: {path}") | ||
| if isinstance(value, dict): | ||
| for key, child in value.items(): | ||
| _reject_non_finite_values(child, f"{path}.{key}") | ||
| elif isinstance(value, list): | ||
| for index, child in enumerate(value): | ||
| _reject_non_finite_values(child, f"{path}[{index}]") |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Bound the recursion depth of the input walkers.
_reject_non_finite_values, _reject_precomputed_metric_fields, and _validate_citation_value all recurse once per nesting level of the supplied JSON. A snapshot file with deeply nested arrays or objects raises RecursionError, not InputError. RecursionError is not in the except tuple in scripts/rarity/analyze.py lines 56-62, so the CLI aborts with a traceback instead of the documented concise error: message. Add an explicit depth limit and convert the overflow into InputError.
🛡️ Proposed depth bound
+MAX_INPUT_DEPTH = 64
+
+
-def _reject_non_finite_values(value: Any, path: str = "snapshot") -> None:
+def _reject_non_finite_values(
+ value: Any, path: str = "snapshot", depth: int = 0
+) -> None:
+ if depth > MAX_INPUT_DEPTH:
+ raise InputError(f"snapshot nesting exceeds {MAX_INPUT_DEPTH} levels: {path}")
if isinstance(value, float) and not math.isfinite(value):
raise InputError(f"non-finite numeric value is prohibited: {path}")
if isinstance(value, dict):
for key, child in value.items():
- _reject_non_finite_values(child, f"{path}.{key}")
+ _reject_non_finite_values(child, f"{path}.{key}", depth + 1)
elif isinstance(value, list):
for index, child in enumerate(value):
- _reject_non_finite_values(child, f"{path}[{index}]")
+ _reject_non_finite_values(child, f"{path}[{index}]", depth + 1)Apply the same depth parameter to _reject_precomputed_metric_fields and _validate_citation_value.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/rarity/nextgen_compat.py` around lines 181 - 189, Bound recursion in
_reject_non_finite_values, _reject_precomputed_metric_fields, and
_validate_citation_value with a shared depth parameter and explicit maximum
nesting limit; increment it for nested dict/list traversal and raise InputError
when exceeded, so deeply nested snapshots produce the CLI’s concise error
instead of RecursionError. Preserve existing validation behavior for inputs
within the limit.
| def test_left_fold_matches_javascript_reduce_not_python_sum(self) -> None: | ||
| values = [1e16, 1.0, -1e16, 1.0] | ||
|
|
||
| javascript_reduce = 0.0 | ||
| for value in values: | ||
| javascript_reduce += value | ||
|
|
||
| self.assertEqual(sum(values), 2.0) | ||
| self.assertEqual(javascript_reduce, 1.0) | ||
| self.assertEqual(_left_to_right_sum(values), javascript_reduce) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
Which Python version added Neumaier compensated summation to the built-in sum for floats?
💡 Result:
Python 3.12 is the version that added Neumaier compensated summation to the built-in sum function for floating-point numbers [1][2][3]. This improvement was implemented to enhance the accuracy of floating-point sums while maintaining performance [4][5][1]. Before this change, the standard sum function used a simple running total, which was less precise [5][1]. The addition was contributed by Raymond Hettinger [4][1].
Citations:
- 1: https://stackoverflow.com/questions/77252117/what-is-the-difference-between-built-in-sum-and-math-fsum-in-python-3-12
- 2: https://en.wikipedia.org/wiki/Kahan_summation_algorithm
- 3: https://anthonylloyd.github.io/blog/2023/10/09/fsum
- 4: GH-100425: Improve accuracy of builtin sum() for float inputs python/cpython#100426
- 5: Improve accuracy of builtin sum() for float inputs python/cpython#100425
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- candidate files ---'
fd -i 'test_nextgen_compat.py|2026-08-01-nextgen-rarity-analysis.md|pyproject.toml|tox.ini|\.python-version|runtime.txt' .
printf '%s\n' '--- relevant test section ---'
test_file="$(fd -i -t f 'test_nextgen_compat.py' | head -n 1)"
sed -n '260,305p' "$test_file"
printf '%s\n' '--- runtime/support declarations ---'
for f in $(fd -i -t f 'pyproject.toml|tox.ini|\.python-version|runtime.txt'); do
printf '\n### %s\n' "$f"
rg -n -i 'python|requires-python|py3|3\.[0-9]+' "$f" || true
done
printf '%s\n' '--- referenced note ---'
note="$(fd -i -t f '2026-08-01-nextgen-rarity-analysis.md' | head -n 1)"
if [ -n "$note" ]; then
sed -n '64,80p' "$note"
fi
printf '%s\n' '--- definitions and usages ---'
rg -n '_left_to_right_sum|javascript_reduce|test_left_fold_matches_javascript_reduce_not_python_sum' .Repository: 6529-Collections/6529networkmuseum
Length of output: 3988
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- standalone numeric behavior ---'
python3 - <<'PY'
import sys
values = [1e16, 1.0, -1e16, 1.0]
left_fold = 0.0
for value in values:
left_fold += value
print(f"interpreter={sys.implementation.name} {sys.version.split()[0]}")
print(f"sum={sum(values)!r}")
print(f"left_fold={left_fold!r}")
print(f"sum_equals_left_fold={sum(values) == left_fold}")
PY
printf '%s\n' '--- left-to-right implementation ---'
sed -n '75,95p' scripts/rarity/nextgen_compat.py
printf '%s\n' '--- tracked project/runtime configuration ---'
git ls-files | rg '(^|/)(pyproject\.toml|setup\.cfg|setup\.py|tox\.ini|noxfile\.py|Pipfile|poetry\.lock|\.python-version|runtime\.txt|.*requirements.*|.*Dockerfile.*|.*workflow.*\.ya?ml)$' || true
rg -n -i 'requires-python|python_requires|python-version|python[ ]*[><=~]|py3[0-9]|3\.[0-9]+' \
--glob '!notes/**' --glob '!tests/**' --glob '!scripts/**' . || trueRepository: 6529-Collections/6529networkmuseum
Length of output: 50391
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- standalone numeric behavior ---'
python3 - <<'PY'
import sys
values = [1e16, 1.0, -1e16, 1.0]
left_fold = 0.0
for value in values:
left_fold += value
print(f"interpreter={sys.implementation.name} {sys.version.split()[0]}")
print(f"sum={sum(values)!r}")
print(f"left_fold={left_fold!r}")
print(f"sum_equals_left_fold={sum(values) == left_fold}")
PY
printf '%s\n' '--- left-to-right implementation ---'
sed -n '75,95p' scripts/rarity/nextgen_compat.py
printf '%s\n' '--- tracked runtime configuration filenames ---'
git ls-files | rg '(^|/)(pyproject\.toml|setup\.cfg|setup\.py|tox\.ini|noxfile\.py|Pipfile|poetry\.lock|\.python-version|runtime\.txt|.*requirements.*|.*Dockerfile.*)$' || trueRepository: 6529-Collections/6529networkmuseum
Length of output: 997
Remove the version-specific sum() assertion. If the test runs on CPython 3.11 or earlier, sum(values) returns 1.0; CPython 3.12+ returns 2.0. The repository does not pin Python. Preserve the javascript_reduce and _left_to_right_sum assertions, and make the sum() check conditional or remove it.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/rarity/test_nextgen_compat.py` around lines 285 - 294, Remove the
version-dependent sum(values) equality assertion from
test_left_fold_matches_javascript_reduce_not_python_sum; preserve the
javascript_reduce and _left_to_right_sum assertions unchanged.
Summary
scripts/rarity/for the pinned 6529 NextGen trait-prevalence algorithmtests/rarity/INDEX.mdReview follow-up
opensea_trait_source_urlremain admissible and are not score inputs.Counter-based duplicate/frequency handling, and one-pass duplicate preservation bookkeeping.argparseinvocation misuse (2) from rejected/invalid data (1) and success (0).Math.min(...[])/math.infparity assertion, and an explicit CPython/canonical-float determinism profile and boundary test.output_sha256, so exact fixtures are not pinned to one CPython patch version while numeric hash reproducibility remains explicitly bounded.Foundation update
origin/mainat72622a670854cc489330d930136bae7318044e41..github/6529bot.ymlreview policy, governance review policy, CI workflow, and orchestration ledger.notes/research/nextgen-rarity-method.mdunchanged fromorigin/main.Source pins
6529seize-backendnextgen_tokens.tsat902557e9274f03b9851e97ef7ffac4b3c310b8a06529seize-backendnextgen_constants.tsat the same commitnextgencontract repository at73c09d1c07e405ddb9ccdd462283ab98ea68f903Validation
python scripts/bootstrap_validate.py— passed; 11 JSON files checkedpython -m unittest discover -s tests\\rarity -p 'test_*.py' -v— 11 passedpython -m py_compile ...— passedcodex-diff-check --cachedandcodex-diff-check origin/main...HEAD— passedorigin/mainSummary by CodeRabbit
New Features
Documentation
Tests