Skip to content

Tighten v1 EnvConfig typing#1371

Merged
willccbb merged 1 commit into
mainfrom
codex/strict-envconfig-typing
May 14, 2026
Merged

Tighten v1 EnvConfig typing#1371
willccbb merged 1 commit into
mainfrom
codex/strict-envconfig-typing

Conversation

@willccbb
Copy link
Copy Markdown
Member

@willccbb willccbb commented May 14, 2026

Summary

  • tighten v1 EnvConfig to only expose taskset and harness
  • default nested vf.Config fields from their annotations, recursively
  • move v1 examples, init stubs, docs, and skills to taskset-first config
  • add Semgrep policy for SkipValidation and optional public loader configs

Validation

  • uv run pre-commit run --files ...
  • uv run --group policy semgrep --metrics=off --disable-version-check --config .semgrep/verifiers.yml --error --quiet
  • uv run --python 3.13 ty check verifiers
  • focused pytest slice for v1 config, examples, BFCL, MCP, RLM SWE, env server, and types
  • git diff --check

Note

Medium Risk
This changes core v1 configuration parsing/merging and loader contracts across many bundled environments, which could break existing callers that relied on optional/kwargs-based env args or None child sections.

Overview
Tightens the v1 loader/config contract around typed EnvConfig child sections. vf.EnvConfig now exposes only required taskset and harness config objects (no root args), defaults them to base TasksetConfig/HarnessConfig, rejects None, forbids extra root fields in EnvConfig subclasses, and adds automatic default factories for annotation-only nested vf.Config fields.

Updates v1 environments, docs, and templates to be taskset-first and config-driven. Public load_taskset/load_harness loaders now require a concrete config type (no | None defaults/kwargs mirroring), many example environments add concrete EnvConfig subclasses to bind custom child config types, and the prime env init template adds --with-harness to generate an explicit load_harness stub.

Adds policy enforcement and coverage. New Semgrep rules ban broad SkipValidation usage and optional public loader configs, verifiers/serve/types.py centralizes the remaining SkipValidation behind protocol-boundary type aliases with waivers, and tests are updated/added to validate the new EnvConfig behavior and updated environment loaders.

Reviewed by Cursor Bugbot for commit 1719184. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread environments/bfcl_v3/bfcl_v3.py
Comment thread environments/dspy_rlm/dspy_rlm.py
Comment thread environments/hello_group_reward_v1/hello_group_reward_v1.py Outdated
@willccbb willccbb force-pushed the codex/strict-envconfig-typing branch from bd77d34 to 5452b7e Compare May 14, 2026 11:21
@willccbb willccbb force-pushed the codex/strict-envconfig-typing branch from 5452b7e to 5e004c4 Compare May 14, 2026 11:42
Comment thread verifiers/v1/config.py
Comment thread docs/evaluation.md
Comment thread verifiers/v1/utils/config_utils.py
@willccbb willccbb force-pushed the codex/strict-envconfig-typing branch from 5e004c4 to 925b80f Compare May 14, 2026 12:00
Comment thread environments/langchain_deep_agents_wikispeedia/README.md Outdated
Comment thread verifiers/v1/config.py Outdated
@willccbb willccbb force-pushed the codex/strict-envconfig-typing branch from 925b80f to 8b242c6 Compare May 14, 2026 12:16
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 8b242c6. Configure here.

Comment thread environments/bfcl_v3/bfcl_v3.py Outdated
Comment thread verifiers/v1/utils/config_utils.py Outdated
@willccbb willccbb force-pushed the codex/strict-envconfig-typing branch from 8b242c6 to a5bcb59 Compare May 14, 2026 12:39
@willccbb willccbb marked this pull request as ready for review May 14, 2026 12:53
@willccbb willccbb requested review from mikasenghaas and xeophon May 14, 2026 17:27
@willccbb willccbb force-pushed the codex/strict-envconfig-typing branch from a5bcb59 to 1719184 Compare May 14, 2026 17:34
) -> GroupRewardTaskset:
config = GroupRewardTasksetConfig(config, num_examples=num_examples)

def load_taskset(config: GroupRewardTasksetConfig) -> GroupRewardTaskset:
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdym here? GRTasksetConfig is a TasksetConfig

@willccbb willccbb merged commit 3b031f5 into main May 14, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant