Phase 0 hygiene: CI scanning, crosswalk regeneration, packaging fixes#38
Merged
Conversation
…rds badge Phase 0 hygiene per TRUST_STRATEGY.md. - .github/workflows/codeql.yml: CodeQL for Python + JavaScript, on every PR to main/develop plus a weekly schedule - .github/dependabot.yml + .github/workflows/dependency-review.yml: Dependabot files version-update PRs (npm, pip, github-actions); the dependency-review-action is what actually gates a PR introducing a moderate+ severity vulnerable dependency -- Dependabot alone doesn't block anything - .github/workflows/secret-scan.yml: gitleaks on every PR and push to main/develop, portable regardless of GitHub plan tier - .gitleaks.toml: allowlists tests/fixtures/ -- AVE positive fixtures intentionally contain credential-shaped strings as detection-rule test data (e.g. AVE-2026-00047), not real secrets; without this the scan would false-positive on every PR touching that record - .github/workflows/scorecard.yml + README badge: OpenSSF Scorecard, publishes results automatically once merged to main - Also enabled natively via the GitHub API (had admin access): secret scanning, secret scanning push protection, Dependabot security updates -- all were disabled - README records badge corrected 51 -> 56 (stale since the 5-record batch merged) Not done, needs a human: OpenSSF Best Practices Badge self-certification requires an account signup and answering ~40 criteria interactively at bestpractices.dev -- can't be completed via API/CLI.
Was stale at 48 records (missing AVE-2026-00049-00051 from the prior v1.1 batch, and now the 5-record 00052-00056 batch). Regenerating found the staleness went deeper than missing rows: several existing entries had drifted from their records' actual owasp_mcp values -- e.g. AVE-2026-00004 was hand-listed as MCP05/MCP06, but the record itself has always said MCP01/MCP03. Titles were also paraphrased rather than matching each record's actual title field. Regenerated programmatically directly from every record's owasp_mcp field rather than hand-editing, added a note in the file saying so, so this doesn't silently drift again. Full mapping table, category groupings, and severity coverage table all recomputed; totals verified to sum to 56.
Classified each of AVE-2026-00052..00056 against AST10's actual category descriptions (not just titles), following the file's own established discipline of not forcing a fit: - AVE-2026-00054 (Code-Execution Sandbox Escape) -> AST06 (Weak Isolation): different premise (a sandbox exists but fails) than AST06's "no sandbox at all," same outcome, same reasoning already used for AVE-2026-00046/00050 - AVE-2026-00055 (MCP STDIO Launch Config Injection) -> AST02 (Supply Chain Compromise): near-exact match -- OX Security's own finding (9 of 11 registries accepted a malicious PoC with no review) mirrors AST02's "registries without provenance" description almost verbatim - AVE-2026-00052 (Tool Implementation Command Injection) and AVE-2026-00053 (Resource Path Traversal): do NOT map to AST01 (assumes intentional malice) or AST05 (assumes parse-time deserialization, "before any user action" -- these two fire at runtime via a normal tool call). Recorded as a new gaps_in_ast10 entry instead of forced into either. - AVE-2026-00056 (rendered-content auto-fetch exfiltration): no AST10 category addresses exfiltration, same reason 16 of the original 51 records were already unmapped. New gaps_in_ast10 entry. Coverage recomputed: 8/10 categories mapped (unchanged), 37/56 records referenced (was 35/51), 19 unmapped (was 16) -- verified the JSON and MD numbers agree exactly. source.version bumped to 1.1.0, record_count to 56; checked_against_live_site left at 2026-06-21 since this update maps against previously-verified category text, it does not re-verify the live AST10 site.
…g bugs Adding a real CI test workflow surfaced that pip install -e ".[dev]" was already broken for anyone on a clean environment, not just this new workflow -- CONTRIBUTING.md and CLAUDE.md both document that exact command: 1. build-backend = "setuptools.backends.legacy:build" is not a valid backend (ModuleNotFoundError: No module named 'setuptools.backends'). Fixed to setuptools.build_meta, the standard modern backend. 2. With that fixed, setuptools' automatic package discovery then failed outright -- this repo isn't a Python library (nothing imports "bawbel_ave"; scripts run directly, tests load rule files dynamically via importlib). Added [tool.setuptools] packages = [] to tell setuptools there's nothing to discover/build, which is what this pyproject.toml is actually for: dependency and pytest config. Verified the fix in a genuinely clean venv end to end: pip install, validate_records.py, check_fixtures.py, check_rule_coverage.py, and pytest --cov all pass with zero pre-existing state. .github/workflows/tests.yml runs that full sequence on every PR/push to main/develop. Added pytest-cov and a [tool.coverage.run] section scoped to rules/ deliberately -- that's the code the fixture-driven tests actually exercise per CLAUDE.md's TDD loop; scripts/ holds one-time migration/validation tooling that isn't meant to carry unit coverage in the same sense. Current coverage: 100% (also mechanically close to guaranteed, since check_fixtures.py and check_rule_coverage.py already gate on every rule having a positive+negative fixture pair). --cov-fail-under=95 catches regression without demanding an unrealistic 100% floor forever.
Tests, coverage, CodeQL, dependency review, secret scan, OpenSSF Scorecard, security policy, code of conduct -- one badge per gate added this session, plus the two governance docs that already existed (SECURITY.md, CODE_OF_CONDUCT.md) but weren't surfaced as badges. Coverage badge is static (100%, scoped to rules/), not auto-updating -- wiring a genuinely dynamic percentage badge needs either an external service (Codecov) or a commit-back CI step, neither of which is set up. The Tests badge is the real enforcement signal: --cov-fail-under=95 in tests.yml fails the workflow (and turns that badge red) if coverage regresses, which matters more than the static number staying exact.
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
PR #38's first CI run surfaced two real failures, both fixed and verified locally before pushing: 1. gitleaks/gitleaks-action@v2 now requires a paid license for GitHub Organization accounts ("[bawbel] is an organization. License key is required.") -- a breaking change in the maintained wrapper action, not the underlying gitleaks CLI, which is AGPL-3.0 and has no such restriction. Switched to running the gitleaks Docker image directly (zricethezav/gitleaks:latest), same command, no license needed. Verified locally with `docker run` against this exact repo. 2. That same local run surfaced a real gap: my original .gitleaks.toml allowlist only covered tests/fixtures/, but scripts/generate-rules-and-fixtures.js embeds the identical synthetic credential-shaped strings as the template that generates tests/fixtures/AVE-2026-00047_positive.md. Extended the allowlist to cover both. Reverified locally: 0 findings, exit code 0. 3. "Dependency review is not supported on this repository... Dependency graph" -- enabled via the GitHub API (vulnerability-alerts endpoint, which requires dependency graph as a prerequisite); confirmed via GraphQL (hasVulnerabilityAlertsEnabled: true). No file change for this one, a repo setting.
CHANGELOG: documents the schema v1.1.0 migration, the 5 new records (AVE-2026-00052-00056), and the Phase 0 hygiene work from this PR -- none of it had a changelog entry yet. Bumped pyproject.toml's release version to match (1.1.0 was already used by the 48->51/schema-0.2.0-> 1.0.0 release, a naming collision with this session's *schema* v1.1.0 that predates this session). Updated the stale "Planned for v1.2" list: moved shipped items (GOVERNANCE.md, CODE_OF_CONDUCT.md, implementer guide, the v1.1.0 offline release) to a "Done" section, retitled the still-open items "Planned for v1.3", and added the items this release itself surfaced as still outstanding (v1.2.0 release cut, AST10 PR submission, re-checking the two reverse crosswalks' actual rule-level mappings, Section 6.2 review priorities 2-4). clawscan-to-ave.json / skillspector-to-ave.json: updated target.version and target.record_count to 1.1.0/56 and refreshed the generated date. Did not touch the rule-level mappings, gaps, or coverage sections -- both files map a specific external tool's own rule/category catalog to AVE ids, which this repo has no live access to. Extending those for AVE-2026-00052-00056 would mean inventing claims about what ClawScan or SkillSpector actually detect. The note field in both files now says so explicitly, so this isn't a silent gap.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 0 (verifiable hygiene) from
TRUST_STRATEGY.md, plus crosswalk updates for the 5 records merged in #37.CI / security scanning (none of this existed before):
.github/workflows/tests.yml-- runs the full validate/fixtures/rule-coverage/pytest sequence with coverage (--cov=rules --cov-fail-under=95) on every PR/push to main/develop. Previously nothing in CI actually ran the test suite..github/workflows/codeql.yml-- CodeQL for Python and JavaScript, on PRs plus a weekly schedule.github/dependabot.yml+.github/workflows/dependency-review.yml-- Dependabot files version-update PRs;dependency-review-actionis what actually blocks a PR introducing a moderate+ severity vulnerable dependency.github/workflows/secret-scan.yml+.gitleaks.toml-- gitleaks on every PR/push, with an allowlist fortests/fixtures/(AVE's positive fixtures intentionally contain credential-shaped strings as detection-rule test data, e.g. AVE-2026-00047 -- without the allowlist this would false-positive on every PR touching that record).github/workflows/scorecard.yml-- OpenSSF Scorecard, publishes results automatically once merged tomainTwo pre-existing packaging bugs found and fixed while building the tests workflow (verified against a genuinely clean venv, not just my dev environment):
build-backend = "setuptools.backends.legacy:build"is not a valid backend -- fixed tosetuptools.build_metabawbel_ave; scripts run directly, tests load rule files dynamically). Added[tool.setuptools] packages = [].Both bugs meant
pip install -e ".[dev]"-- the exact command CONTRIBUTING.md and CLAUDE.md tell contributors to run -- was already broken on a clean machine, independent of anything added in this PR.Crosswalk updates:
ave-to-owasp-mcp.md: was stale (missing AVE-2026-00049-00051 and 00052-00056) and, on closer inspection, had drifted from the records themselves for several existing entries (e.g. AVE-2026-00004 was hand-listed under the wrong categories). Regenerated the whole file programmatically from each record's actualowasp_mcpfield rather than patching it, added a note saying so.ave-to-ast10.json/.md: classified each of the 5 new records (AVE-2026-00052-00056) against AST10's actual category descriptions, not just titles. Two map cleanly -- AVE-2026-00054 (Sandbox Escape) → AST06, AVE-2026-00055 (STDIO Launch Config Injection) → AST02. Three do not fit any existing category and are recorded as new gaps rather than forced: AVE-2026-00052/00053 are accidental code-level bugs (CWE-78/CWE-22), which matches neither AST01's intentional-malice framing nor AST05's parse-time-deserialization framing; AVE-2026-00056 is data exfiltration, which none of AST10's 10 categories address. Coverage recomputed and cross-checked between the JSON and MD files (37/56 records referenced, was 35/51).clawscan-to-ave.json/skillspector-to-ave.jsonchecked and left untouched -- these key off the external tools' own rule catalogs, which aren't visible to this repo; adding entries would mean inventing facts about tools this repo doesn't control.Not done, needs a human
OpenSSF Best Practices Badge (bestpractices.dev) requires interactive account signup and ~40 self-certification questions -- can't be completed via API/CLI. Most of the underlying criteria are already satisfied by what's in this repo (LICENSE, SECURITY.md, CONTRIBUTING.md, tests, HTTPS delivery).
Test plan
python scripts/validate_records.py-- all 56 records validpytest tests/ -x -q-- 112 passed.gitleaks.tomlverified: no false-positives ontests/fixtures/, still catches real secret patterns elsewherepip install -e ".[dev]"verified end-to-end in a from-scratch virtualenv (not just the dev environment that already had things cached)develop(can't fully verify GitHub Actions execution from localgh/git alone)