release: 0.6.7 - #968
Merged
Merged
Conversation
Version 0.6.7 (from 0.6.7.dev0), CITATION date 2026-08-09, and the 0.6.7 CHANGELOG section covering #949-#967. Also in this cut: - Point the wheel-shipped env0 pins at the org-owned ghcr.io/benchflow-ai/env0:0.2.0 base image (they named a personal Docker Hub image while their own comments and the docs declared ghcr authoritative) and drop the PR-archaeology comments from files that ship to PyPI. Validated on Daytona: 8/8 services ready, reward 1.0. - Document mean_reward in the summary.json layout, the per-task failure block and live-token footer in the CLI reference, and soften the external-agents cost figure (the underlying rate table is a placeholder).
xdotli
temporarily deployed
to
pypi-internal-preview
August 9, 2026 20:32 — with
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cuts 0.6.7 from current main. Everything in this release is backward-compatible; the BREAKING
environment→sandboxtask-config rename (#966) is deliberately not here — it opens 0.7 and merges after this tags.Contents — 19 PRs (#949–#967)
Highlights, all dogfood-driven (7 blind fresh-user rounds this cycle, each validating the previous fix or surfacing the next):
38 calls · 412.0k tok), phase labels so the cell is never blank, a ~45s heartbeat for non-TTY runs, and live token totals mid-run — including mid-prompt via the gateway's existing capture loop, which is what single-task evals actually need (fix(eval): console UX from fresh-user dogfood — progress heartbeat + clean teardown #951, feat(cli): live per-task activity in the eval dashboard; console polish #956, feat(cli): phase labels in the activity column; one-line failure reasons #957, feat(cli): live token usage in the dashboard footer; token-forward activity cell for single-tool agents #963, feat(cli): mid-prompt live token usage via the gateway's existing live capture #965).mean reward 0.30), one failure-reason line per failed task mined from in-memory metrics or bounded verifier artifacts (CTRF /reward.json/ stdout tail), multi-failure roll-ups, and a(details: …/verifier)pointer whenever artifacts exist (feat(cli): failure reasons from verifier artifacts (CTRF / test-stdout) #959, feat(cli): surface fractional rewards in console summaries #960, feat(cli): full failure count on per-task console lines; preserve param ids #962, feat(cli): env0-shaped failure breakdowns; details pointer keyed off on-disk artifacts #964).--environment-manifest env0@prodresolves on a barepip install— verified from an empty directory by an independent fresh-user run (feat(environment): built-in manifest registry shipped in the wheel #961).[task] versionacceptance, clawsbench task.md migration, env0 pin sync + restored outage perturbation, compact flow-style frontmatter arrays (fix(task): accept Harbor 1.3 [task] version field #953, chore(clawsbench): migrate to task.md + actionable no-manifest verifier error #952, fix(env0): sync drifted service pins + fail loud on zero startable services #954, docs: fix post-merge drift from #949–#954; restore env0@outage perturbation #955, feat(task): compact flow-style arrays in emitted task.md frontmatter #967).Release-specific changes in this PR
.dev0), CITATION date, CHANGELOG 0.6.7 section with per-PR entries.ghcr.io/benchflow-ai/env0:0.2.0: the wheel-shipped pins named a personal Docker Hub image while their own comments anddocs/environment-plane.mddeclared the ghcr image authoritative, andbase_imageis recorded as rollout provenance — so every env0 result carried the wrong base. Validated on Daytona before flipping: env0auth-least-privilege-summary, 8/8 services ready, reward 1.0 in 2.8 min. PR-archaeology comments stripped from the files that ship to PyPI.mean_rewarddocumented in the summary.json layout; the failure block and live-token footer documented in the CLI reference; the external-agents cost figure softened (the underlying rate table self-describes as a placeholder).Gates
ruff format/check, ty — clean. Full suite: 5368 passed (3 known host-specific failures: the rbenv-sensitive heartbeat test and two smoke-wiring credential tests that trip when real credentials exist on the host).
Post-merge steps for the maintainer: tag
v0.6.7on the merge commit and publish to PyPI per the release-manager process.