From 8c83f1d8a72f6fb900bdbf4ac2934c6c881073f1 Mon Sep 17 00:00:00 2001 From: Felmon Fekadu Date: Tue, 14 Jul 2026 07:11:26 -0600 Subject: [PATCH] chore: modernize packaging metadata for v0.2.1 - use SPDX license expression and PEP 639 license-files metadata - require setuptools>=77 and remove deprecated license classifier - make README documentation links work from PyPI - correct stale compatibility claims about the nested-safety score shift - bump package version to 0.2.1 Verified: 194 tests, ruff, strict mypy, benchmark thresholds, warning-free sdist/wheel build, PEP 639 wheel metadata, and clean-wheel install. --- CHANGELOG.md | 21 +++++++++- COMPATIBILITY.md | 12 +++--- README.md | 48 ++++++++++++----------- pyproject.toml | 8 ++-- src/agent_reliability_harness/__init__.py | 2 +- 5 files changed, 56 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5cc2425..1d0b179 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,21 @@ adheres to [Semantic Versioning](https://semver.org/). ## [Unreleased] +## [0.2.1] - 2026-07-14 + +### Fixed + +- Modernized packaging license metadata to the PEP 639/SPDX form + (`license = "MIT"`, `license-files = ["LICENSE"]`) and removed the + deprecated license classifier; builds no longer emit setuptools license + deprecation warnings. +- Converted README documentation links to absolute GitHub URLs so they work + from the PyPI long-description renderer. +- Corrected stale compatibility language that incorrectly claimed every + v0.1 score remained identical despite the documented nested-safety-scan + fix (70.0 -> 70.83 for one sample; verdict unchanged). + + ## [0.2.0] - 2026-07-14 ### Added @@ -90,8 +105,10 @@ adheres to [Semantic Versioning](https://semver.org/). ### Compatibility -- v0.1.x trace and policy files produce identical scores, verdicts, and - finding messages (pinned by golden tests). New report fields are additive. +- v0.1.x trace and policy files remain accepted; verdicts and finding + messages are preserved. The documented nested-safety-scan fix changes one + sample score from 70.0 to 70.83 (verdict unchanged). New report fields are + additive. `arh compare` accepts v0.1.x baselines via a rule-id-agnostic fallback match. Exit codes and console markers are unchanged. See COMPATIBILITY.md. diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 4954b2f..9585d73 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -8,11 +8,13 @@ forward. - **Trace and policy files work unchanged.** The v0.1.x formats are strict subsets of schema v1; a missing `schema_version` means `"1"`. No migration step exists because none is needed. -- **Identical scores and verdicts.** New check categories (sequence, flow, - completion) are *not applicable* for policies that don't configure them, and - the score renormalizes over applicable categories only — so a v0.1.x policy - yields byte-identical scores. Pinned by `tests/test_compat_golden.py` - (sample scores 100.0 / 80.0 / 70.83, verdicts, and finding messages) and +- **Preserved verdicts; scores preserved except one documented safety fix.** + New check categories (sequence, flow, completion) are *not applicable* for + policies that do not configure them, and the score renormalizes over applicable + categories. Golden tests pin scores 100.0 / 80.0 unchanged; the unsafe sample + changes from 70.0 to 70.83 because v0.2 recursively scans nested content that + v0.1 silently skipped. All three verdicts and finding messages are preserved. + See `tests/test_compat_golden.py` and `tests/test_trajectory_rules.py::TestLegacyScoreCompatibility`. - **Finding messages** for all v0.1.x checks are unchanged (golden-tested). - **Console markers** (`[PASS]`, `[FAIL]`, `[safety ]` category blocks, diff --git a/README.md b/README.md index 61346e4..49965a0 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,7 @@ GATE: FAIL The gate fails on new error findings, pass→fail transitions, and added failing traces — resolved findings and unchanged failures don't re-alarm. Gate modes: `--fail-on regressions` (default), `failures`, `never`; add `--max-score-drop` -for score-based gating. See [docs/regression-testing.md](docs/regression-testing.md). +for score-based gating. See [docs/regression-testing.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/regression-testing.md). ## What it checks @@ -96,7 +96,7 @@ Seven deterministic categories; every finding carries a stable rule ID | flow | ARH-FLW-001..003 | ignored tool errors, retry storms, duplicate side effects | | completion | ARH-CMP-001..002 | missing final response, failure to terminate | -Full reference: [docs/rules.md](docs/rules.md). Ordering rules are a +Full reference: [docs/rules.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/rules.md). Ordering rules are a deliberate **partial order** (constraints), not an exact golden-trajectory match — agents can legitimately reach a goal via different paths. @@ -104,7 +104,7 @@ match — agents can legitimately reach a goal via different paths. `arh validate` accepts (auto-detected, or forced with `--format`): -- **arh** — the canonical JSON trace format ([TRACE-SPEC.md](TRACE-SPEC.md)); +- **arh** — the canonical JSON trace format ([TRACE-SPEC.md](https://github.com/felmonon/agent-reliability-harness/blob/main/TRACE-SPEC.md)); - **openai-chat** — OpenAI Chat Completions message lists with `tool_calls`; - **anthropic-messages** — Anthropic Messages conversations with `tool_use`/`tool_result` blocks. @@ -112,7 +112,7 @@ match — agents can legitimately reach a goal via different paths. Adapters never guess: fields a transcript format cannot carry (latency, cost, tokens) are left unset, which marks the dependent checks *not applicable* instead of silently passing them. Unparseable input is recorded in the trace's -metadata, not dropped. See [docs/adapters.md](docs/adapters.md). +metadata, not dropped. See [docs/adapters.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/adapters.md). ## Reports @@ -134,7 +134,7 @@ no timestamps, no randomness, no clock reads. The action writes JSON/Markdown/JUnit/SARIF reports, appends the Markdown summary to the workflow step summary, and fails according to the gate. It uses no secrets and is fork-safe. Full reference and SARIF/JUnit upload examples: -[docs/ci.md](docs/ci.md). +[docs/ci.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/ci.md). ## Design principles @@ -142,38 +142,40 @@ no secrets and is fork-safe. Full reference and SARIF/JUnit upload examples: application's dependency graph. - **Deterministic core.** No model calls, no network, no telemetry, no clock reads. Semantic (model-graded) evaluation is a planned, clearly separated - optional extra — never hidden inside deterministic scores ([ROADMAP.md](ROADMAP.md)). + optional extra — never hidden inside deterministic scores ([ROADMAP.md](https://github.com/felmonon/agent-reliability-harness/blob/main/ROADMAP.md)). - **Additive schema evolution.** v0.1.x traces, policies, and baselines work unchanged; unknown major schema versions are rejected loudly - ([COMPATIBILITY.md](COMPATIBILITY.md)). + ([COMPATIBILITY.md](https://github.com/felmonon/agent-reliability-harness/blob/main/COMPATIBILITY.md)). - **Evidence over claims.** The benchmark suite contains 34 seeded cases (28 expected-fail, 6 expected-pass controls) and measures detection: currently 34/34 correct, precision 1.0, recall 1.0, 0 false positives/negatives, byte-identical repeat runs, ~0.03 ms per trace. Those numbers cover *seeded, deterministically detectable* failures only — scope - and limits are documented in [BENCHMARK-METHODOLOGY.md](BENCHMARK-METHODOLOGY.md), - results in [BENCHMARK-RESULTS.md](BENCHMARK-RESULTS.md). + and limits are documented in [BENCHMARK-METHODOLOGY.md](https://github.com/felmonon/agent-reliability-harness/blob/main/BENCHMARK-METHODOLOGY.md), + results in [BENCHMARK-RESULTS.md](https://github.com/felmonon/agent-reliability-harness/blob/main/BENCHMARK-RESULTS.md). ## Documentation | Doc | What it covers | |---|---| -| [docs/quickstart.md](docs/quickstart.md) | Five minutes from install to a regression gate | -| [docs/concepts.md](docs/concepts.md) | Traces, policies, findings, scores, baselines | -| [docs/policy-cookbook.md](docs/policy-cookbook.md) | Copy-paste recipes for every rule type | -| [docs/adapters.md](docs/adapters.md) | OpenAI/Anthropic transcript ingestion, field mapping | -| [docs/regression-testing.md](docs/regression-testing.md) | Baselines, fingerprints, gates | -| [docs/ci.md](docs/ci.md) | GitHub Action reference and workflows | -| [docs/rules.md](docs/rules.md) | Every rule ID with remediation | -| [docs/troubleshooting.md](docs/troubleshooting.md) | Error messages and fixes | -| [TRACE-SPEC.md](TRACE-SPEC.md) / [POLICY-SPEC.md](POLICY-SPEC.md) | Normative formats | -| [ARCHITECTURE.md](ARCHITECTURE.md) / [DECISIONS.md](DECISIONS.md) | Design and ADRs | +| [docs/quickstart.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/quickstart.md) | Five minutes from install to a regression gate | +| [docs/concepts.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/concepts.md) | Traces, policies, findings, scores, baselines | +| [docs/policy-cookbook.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/policy-cookbook.md) | Copy-paste recipes for every rule type | +| [docs/adapters.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/adapters.md) | OpenAI/Anthropic transcript ingestion, field mapping | +| [docs/regression-testing.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/regression-testing.md) | Baselines, fingerprints, gates | +| [docs/ci.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/ci.md) | GitHub Action reference and workflows | +| [docs/rules.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/rules.md) | Every rule ID with remediation | +| [docs/troubleshooting.md](https://github.com/felmonon/agent-reliability-harness/blob/main/docs/troubleshooting.md) | Error messages and fixes | +| [TRACE-SPEC.md](https://github.com/felmonon/agent-reliability-harness/blob/main/TRACE-SPEC.md) / [POLICY-SPEC.md](https://github.com/felmonon/agent-reliability-harness/blob/main/POLICY-SPEC.md) | Normative formats | +| [ARCHITECTURE.md](https://github.com/felmonon/agent-reliability-harness/blob/main/ARCHITECTURE.md) / [DECISIONS.md](https://github.com/felmonon/agent-reliability-harness/blob/main/DECISIONS.md) | Design and ADRs | ## Compatibility -v0.1.x trace and policy files produce identical scores, verdicts, and finding -messages under this version (pinned by golden tests). New fields are additive. -Details: [COMPATIBILITY.md](COMPATIBILITY.md). +v0.1.x trace and policy files remain accepted and preserve verdicts and finding +messages. Two review-driven detection fixes are documented exceptions: nested safety +scanning changes one sample score from 70.0 to 70.83, and strict telemetry validation +rejects malformed values that v0.1 accepted. New report fields are additive. +Details: [COMPATIBILITY.md](https://github.com/felmonon/agent-reliability-harness/blob/main/COMPATIBILITY.md). ## Development @@ -187,7 +189,7 @@ python benchmarks/run.py # thresholds enforced ``` CI runs the suite on Linux/macOS/Windows across Python 3.11-3.13, plus -packaging checks. Contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). +packaging checks. Contributions welcome — see [CONTRIBUTING.md](https://github.com/felmonon/agent-reliability-harness/blob/main/CONTRIBUTING.md). ## Author diff --git a/pyproject.toml b/pyproject.toml index 688ebcb..829a04f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,14 +1,15 @@ [build-system] -requires = ["setuptools>=68.0"] +requires = ["setuptools>=77.0"] build-backend = "setuptools.build_meta" [project] name = "agent-reliability-harness" -version = "0.2.0" +version = "0.2.1" description = "Local-first policy and trajectory-regression harness for tool-using AI agents: deterministic trace validation, policy-as-code trajectory rules, baseline-vs-candidate regression gates, and CI-ready JSON/Markdown/JUnit/SARIF reports." readme = "README.md" requires-python = ">=3.11" -license = { file = "LICENSE" } +license = "MIT" +license-files = ["LICENSE"] authors = [{ name = "Felmon Fekadu" }] keywords = [ "agents", @@ -25,7 +26,6 @@ keywords = [ classifiers = [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", - "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", diff --git a/src/agent_reliability_harness/__init__.py b/src/agent_reliability_harness/__init__.py index 6873c42..c144e8f 100644 --- a/src/agent_reliability_harness/__init__.py +++ b/src/agent_reliability_harness/__init__.py @@ -32,7 +32,7 @@ from agent_reliability_harness.rules import RULES, Rule from agent_reliability_harness.validator import validate_trace -__version__ = "0.2.0" +__version__ = "0.2.1" __all__ = [ "ArgSpec",