Use reserved examples and seal private artifacts#32
Merged
Conversation
blisspixel
marked this pull request as ready for review
July 18, 2026 18:03
There was a problem hiding this comment.
Pull request overview
This PR hardens recon’s public privacy boundary by migrating legacy fictional example identities to reserved synthetic namespaces (notably .invalid / .test), adding a repository-wide “retired example” hygiene gate, and ensuring private/agent validation artifacts are excluded from release source distributions.
Changes:
- Replaced legacy example targets (e.g., “Contoso/Fabrikam/Northwind/Acme”) with reserved synthetic identities across tests, golden renders, docs, examples, and agent guidance.
- Added retired-example detection/redaction in
scripts/check_validation_hygiene.py, and extended package/release checks to ensure private validation and agent work do not ship in sdists. - Updated public issue/PR templates and docs to explicitly prohibit sharing evaluated-target identities, records, tenant identifiers, corpus rows, and per-domain output in public collaboration surfaces.
Reviewed changes
Copilot reviewed 199 out of 199 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_userrealm_xml.py | Update Autodiscover XML fixtures to reserved synthetic domains. |
| tests/test_triage_candidates.py | Switch triage-candidate fixtures to .invalid examples. |
| tests/test_subdomain_enumeration_breadth.py | Replace legacy example apex in CT filter test prose. |
| tests/test_subcommands.py | Update CLI docstring example domain to .invalid. |
| tests/test_strict_mode.py | Update strict-mode TenantInfo fixture identity to synthetic. |
| tests/test_staleness_timestamps.py | Update cache staleness fixtures to .invalid domains. |
| tests/test_sources/test_userrealm.py | Update userrealm JSON/XML fixtures and lookup domains to synthetic. |
| tests/test_sources/test_oidc.py | Replace lookup domains with .invalid in OIDC source tests. |
| tests/test_siem_examples.py | Update SIEM sample-copy wording to synthetic brand naming. |
| tests/test_services_label_width.py | Update panel-rendering service label fixtures to synthetic. |
| tests/test_service_disclosure.py | Replace disclosure evidence raw_value domain to .invalid. |
| tests/test_server_cache.py | Use .invalid domains for server cache and rate-limit tests. |
| tests/test_run_corpus_path_safety.py | Update corpus/exclusion fixtures to .invalid names. |
| tests/test_resolver.py | Replace resolver test domains and display_name with synthetic examples. |
| tests/test_render_performance.py | Update performance fixtures to synthetic identities/domains. |
| tests/test_render_fuzz.py | Update fuzz fixtures to synthetic domains/display names. |
| tests/test_render_determinism.py | Update determinism fixtures to synthetic identities/domains. |
| tests/test_render_calibration_memo.py | Update leak-prevention assertions to new synthetic domains. |
| tests/test_release_readiness.py | Update private-data hygiene fixtures and redaction expectations. |
| tests/test_reference_calibration.py | Replace domains list fixtures with .invalid domains. |
| tests/test_public_collaboration_privacy.py | New tests pin public templates/docs to non-solicitation of target data. |
| tests/test_properties_resilience.py | Update resilience fixtures to .invalid domains. |
| tests/test_posture.py | Update posture fixtures to .invalid domains. |
| tests/test_posture_validation.py | Update posture validation TenantInfo fixture to synthetic. |
| tests/test_passive_default.py | Replace passive-default lookup domains and BIMI probe URLs. |
| tests/test_panel_render_snapshots.py | Update panel snapshot fixtures and docstrings to synthetic examples. |
| tests/test_panel_output_sanity.py | Update panel sanity fixtures to .invalid domains. |
| tests/test_panel_only_surfaces_json_absence.py | Update “panel-only” surface fixtures to .invalid domains. |
| tests/test_panel_disclosure.py | Update disclosure panel fixture to synthetic queried_domain. |
| tests/test_package_invariants.py | Expand sdist invariants: exclude private prefixes; scan archive text for retired vocab; add ignored-sentinel regression. |
| tests/test_onboarding_docs.py | Update onboarding-doc assertions to avoid legacy example strings. |
| tests/test_oidc_enrichment.py | Update OIDC enrichment fixtures to synthetic B2C host examples. |
| tests/test_mutation_resistance.py | Update mutation-resistance fixtures to .invalid domains. |
| tests/test_multi_cloud_rollup.py | Update multi-cloud rollup evidence fixtures to .invalid domains. |
| tests/test_motifs.py | Update motif chain and subdomain fixtures to synthetic values. |
| tests/test_merger.py | Update merger fixtures to use synthetic tenant_domains and queried_domain. |
| tests/test_merger_error_surfacing.py | Update display_name in error-surfacing fixtures to synthetic. |
| tests/test_mcp_graph_tools.py | Update MCP graph tool fixtures to .invalid domains. |
| tests/test_lookup_compare_cli.py | Update compare CLI tests to .invalid domains/snapshots. |
| tests/test_json_schema_contract.py | Update schema-contract fixture values to synthetic names/domains. |
| tests/test_insights.py | Update insight rendering fixtures to synthetic queried_domain. |
| tests/test_insights_unit.py | Update extensible-email fixtures to synthetic vendor naming. |
| tests/test_hedging_invariants.py | Update hedging invariant fixtures to synthetic b2clogin domain. |
| tests/test_google_identity.py | Update Google identity fixtures (IdP URLs/hosts) to synthetic. |
| tests/test_fusion_robustness.py | Update fusion CLI tests to .invalid domains. |
| tests/test_fusion_integration.py | Update fusion integration fixtures to synthetic domains. |
| tests/test_formatter.py | Update formatter fixtures and stderr-domain checks to synthetic. |
| tests/test_formatter_ceiling.py | Update ceiling fixtures to synthetic domains/subdomains. |
| tests/test_formatter_ceiling_boundary.py | Update boundary fixtures to .invalid domains. |
| tests/test_fingerprints.py | Update fingerprint loader and cname_target fixtures to synthetic terms. |
| tests/test_fingerprint_expansion.py | Update fingerprint expansion fixture identities to synthetic. |
| tests/test_fallback_chain.py | Update crt.sh error-string fixture to .invalid domain. |
| tests/test_exposure.py | Update exposure fixtures to .invalid queried/default domains. |
| tests/test_exposure_cli.py | Update exposure CLI sample fixture and domains to synthetic. |
| tests/test_explanation_coverage.py | Update explanation coverage fixtures to synthetic display name. |
| tests/test_explain_dag_top3.py | Update DAG top-3 renderer tests to .invalid domain. |
| tests/test_enrichment.py | Update enrichment fixtures and related-domain sets to .invalid. |
| tests/test_email_topology.py | Update email topology fixtures/commentary to synthetic org names/domains. |
| tests/test_email_security_score.py | Update email-security fixtures to .invalid queried_domain. |
| tests/test_dns_subdetectors.py | Update DNS subdetector fixtures to .invalid lookalike domains. |
| tests/test_dns_catalog_boundaries.py | Update catalog boundary lookalikes to .invalid domains. |
| tests/test_discover_cli.py | Update discover CLI fixtures/domains to synthetic. |
| tests/test_delta.py | Update delta fixtures to .invalid domains. |
| tests/test_delta_cli.py | Update delta CLI tests/domains/messages to .invalid. |
| tests/test_degraded_sources.py | Update degraded-source fixtures to .invalid domains. |
| tests/test_data_models.py | Update data-model fixtures and docstrings to synthetic identities. |
| tests/test_ct_session_summary.py | Update CT session fixtures and leak assertions to .invalid. |
| tests/test_contracts.py | Update contract validator examples (IDN / normalization) to .invalid. |
| tests/test_conformal_coverage.py | Update conformal coverage fixtures and leak assertions to synthetic domains. |
| tests/test_conflict_provenance.py | Update conflict provenance fixtures to .invalid domain. |
| tests/test_cohort_summary_cli.py | Update cohort summary fixtures to .invalid domains. |
| tests/test_clusterfuzzlite_integration.py | Update seed inputs to synthetic/IDN .invalid examples. |
| tests/test_cli_output_contracts.py | Update CLI contract fixtures/errors to .invalid domains. |
| tests/test_cli_coverage.py | Update CLI coverage fixtures (related domains, tenant domains) to synthetic. |
| tests/test_claim_contract.py | Update claim contract fixtures (subject/scope/owner) to .invalid. |
| tests/test_chain.py | Update chain resolution fixtures/domains to .invalid. |
| tests/test_cert_providers.py | Update cert provider filter fixtures to .invalid “other” domains. |
| tests/test_catalog_driven_corpus.py | Update catalog-driven corpus fixtures to synthetic apexes. |
| tests/test_cache_stateful.py | Update stateful-cache domain lists to .invalid/reserved examples. |
| tests/test_cache_inspection.py | Update cache inspection fixtures to .invalid domains. |
| tests/test_cache_forward_compat.py | Update forward-compat cache fixture identity to synthetic. |
| tests/test_cache_cross_version_compatibility.py | Update cross-version cache fixtures to synthetic domains. |
| tests/test_bimi_vmc.py | Update BIMI VMC fetch test URL to reserved synthetic host. |
| tests/test_bayesian_dag.py | Update Bayesian DAG renderer fixtures to .invalid domain. |
| tests/test_batch_stdin.py | Update stdin batch fixtures to .invalid inputs and dedupe examples. |
| tests/test_backward_compat.py | Update backward-compat fixtures to synthetic identities/domains. |
| tests/test_aggregate_state_reducer.py | Update aggregate reducer malformed-record fixtures to .invalid. |
| tests/test_absence_engine.py | Update absence-engine fixtures to synthetic org naming. |
| tests/golden_renders/panel_surface_full.txt | Update golden panel surface-full render to synthetic identity. |
| tests/golden_renders/panel_surface_default.txt | Update golden panel surface-default render to synthetic identity. |
| tests/golden_renders/panel_sparse_default.txt | Update golden sparse panel render to synthetic identity. |
| tests/golden_renders/panel_hardened_default.txt | Update golden hardened panel render to synthetic identity. |
| tests/golden_renders/panel_dense_verbose.txt | Update golden dense-verbose panel render to synthetic identity. |
| tests/golden_renders/panel_dense_full.txt | Update golden dense-full panel render to synthetic identity. |
| tests/golden_renders/panel_dense_explain.txt | Update golden dense-explain panel render to synthetic identity. |
| tests/golden_renders/panel_dense_default.txt | Update golden dense-default panel render to synthetic identity. |
| tests/golden_renders/markdown_sparse.txt | Update golden sparse markdown render to synthetic identity. |
| tests/golden_renders/markdown_rich.txt | Update golden rich markdown render to synthetic identity. |
| tests/golden_renders/markdown_dense.txt | Update golden dense markdown render to synthetic identity. |
| tests/fixtures/corpus-example.txt | Replace example corpus entries with reserved .invalid scenarios and updated guidance. |
| tests/conftest.py | Update shared fully-populated TenantInfo fixture to synthetic identity. |
| src/recon_tool/validator.py | Replace doc examples with reserved/example domains; update IDN commentary. |
| src/recon_tool/specificity.py | Update CNAME specificity corpus base labels (acme→delta). |
| src/recon_tool/sources/dns.py | Remove real-world apex examples from wildcard-DNS guard comment. |
| src/recon_tool/sources/dns_email.py | Generalize MX evidence docstring away from real domains. |
| src/recon_tool/server/posture.py | Update public API docstring examples to .invalid. |
| src/recon_tool/server/lookup.py | Update lookup tool docstring examples to .invalid. |
| src/recon_tool/server/introspection.py | Update introspection tool docstring examples to .invalid. |
| src/recon_tool/server/graph.py | Update graph tool docstring examples to .invalid. |
| src/recon_tool/server/ephemeral.py | Update ephemeral fingerprint docstring examples to synthetic vendor naming. |
| src/recon_tool/resolver.py | Update resolver module docs/comments to synthetic examples. |
| src/recon_tool/psl.py | Update PSL module examples to example.co.uk. |
| src/recon_tool/lexical.py | Update lexical helper docstrings/examples to .invalid. |
| src/recon_tool/formatter/panel.py | Update panel wording/punctuation and example domain in comments. |
| src/recon_tool/discovery.py | Update brand-label extraction docs/examples to synthetic domains. |
| src/recon_tool/data/recon-schema.json | Update schema description example to mail.example.co.uk. |
| src/recon_tool/clustering.py | Update display-name clustering examples to synthetic org names. |
| src/recon_tool/cli/shared.py | Update validation error help text example domain to example.com. |
| src/recon_tool/cli/fingerprints.py | Update help text examples to synthetic vendor naming; clarify corpus-example purpose. |
| src/recon_tool/cli/doctor.py | Update template examples from acme→delta synthetic placeholders. |
| src/recon_tool/cli/init.py | Update welcome banner and help examples to .invalid domains. |
| scripts/check_validation_hygiene.py | Add retired-example vocabulary detection and path redaction; extend violation scanning. |
| ROADMAP.md | Update roadmap text to reflect completed migration + new release/privacy gates. |
| README.md | Update hygiene guidance to mention retired-vocab gate and preserved ACME/provider definitions. |
| pyproject.toml | Add explicit sdist exclude denylist for private/agent validation paths. |
| examples/siem/splunk/README.md | Update SIEM example narrative and query domain to synthetic. |
| examples/siem/splunk/expected-splunk-event.json | Update expected Splunk event fixture to synthetic domains. |
| examples/siem/README.md | Update SIEM examples overview to reference synthetic fixture. |
| examples/siem/elastic/README.md | Update Elastic example narrative/query domain to synthetic. |
| examples/siem/elastic/expected-elastic-document.json | Update expected Elastic document fixture to synthetic domains. |
| examples/sample-output.json | Update sample output JSON to synthetic identity and domains. |
| examples/sample-batch.txt | Replace sample batch input list with reserved .invalid scenarios. |
| examples/README.md | Update examples policy text to RFC2606 .invalid boundary. |
| docs/traceability-matrix.md | Update traceability reference to renamed sdist invariant test. |
| docs/schema.md | Update schema docs examples to example.co.uk and synthetic values. |
| docs/roadmap.md | Update roadmap language to reflect completed reserved-identity migration and gates. |
| docs/recon-schema.json | Update schema description example to mail.example.co.uk. |
| docs/mcp.md | Update MCP usage examples and ephemeral fingerprint narrative to synthetic domains/names. |
| docs/legal.md | Update legal section to reflect current reserved-example usage and gate behavior. |
| docs/how-it-works.md | Update URL→apex reduction examples and explain-dag examples to .invalid. |
| docs/fingerprints.md | Update fingerprint contribution guidance to require synthetic fixtures and private-workspace live validation. |
| docs/data-handling-policy.md | Expand enforcement table: retired-vocab gate + sdist denylist + sentinel archive test. |
| docs/catalog-strategy.md | Update catalog strategy text to reflect retired-vocab gate and reserved examples. |
| docs/automation-examples.md | Update automation delta JSON example domain to .invalid. |
| docs/assurance-case.md | Update assurance-case references to renamed sdist invariant test. |
| docs/artifact-review.md | Remove legacy “bounded fictional examples” bullet from expected artifacts list. |
| CONTRIBUTING.md | Update public-report guidance and example wording to synthetic fixtures. |
| agents/windsurf/README.md | Update agent prompt example domain to .invalid. |
| agents/vscode/README.md | Update agent prompt example domain to .invalid. |
| agents/README.md | Update agent prompt example domain to .invalid. |
| agents/kiro/README.md | Update agent prompt example domain to .invalid. |
| agents/cursor/README.md | Update agent prompt example domain to .invalid. |
| agents/claude-code/skills/recon/SKILL.md | Update skill examples and sample panel to synthetic domains/names. |
| agents/claude-code/skills/recon-fingerprint-triage/SKILL.md | Update triage skill example domains to .invalid. |
| agents/claude-code/README.md | Update agent prompt example domain to .invalid. |
| AGENTS.md | Update recon usage guidance examples to .invalid domains. |
| .github/PULL_REQUEST_TEMPLATE/fingerprint.md | Harden PR template to avoid soliciting target data; add disclosure-safe evidence checklist. |
| .github/ISSUE_TEMPLATE/fingerprint_request.yml | Update placeholder vendor naming to synthetic. |
| .github/ISSUE_TEMPLATE/bug_report.yml | Update public bug-report guidance to synthetic .invalid/.test reproductions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+346
to
+347
| assert "alpha" not in out | ||
| assert "beta" not in out |
Comment on lines
60
to
62
| domain_count=8, | ||
| tenant_domains=("contoso.com", "contoso.net", "contoso.co.uk"), | ||
| tenant_domains=("alpha.invalid", "alpha.invalid", "alpha.invalid"), | ||
| services=("AWS CloudFront", "Cloudflare", "GCP Compute Engine"), |
Comment on lines
68
to
+71
| dmarc_policy="reject", | ||
| domain_count=3, | ||
| tenant_domains=("contoso.com", "contoso.onmicrosoft.com", "contoso.co.uk"), | ||
| related_domains=("api.contoso.com", "login.contoso.com"), | ||
| tenant_domains=("alpha.invalid", "alpha.onmicrosoft.com", "alpha.invalid"), | ||
| related_domains=("api.alpha.invalid", "login.alpha.invalid"), |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Privacy boundary
This change contains no evaluated-target identities, target-owned records, tenant identifiers, private corpus rows, or per-domain live output. Provider and product names plus provider-controlled detection patterns remain public product definitions.
Validation