Reduce project scope declaration misses - #1990
Merged
Merged
Conversation
Aaronontheweb
enabled auto-merge (squash)
August 18, 2026 16:26
Collaborator
Author
|
Sanitized post-swap evidence supports this guidance change. In one frozen fresh-session workflow, all 20 approval prompts with complete command evidence were compound shell calls; 18 used inline directory changes, 15 targeted platform temporary storage, and 19 were primarily diagnostic or read operations. A later structured file edit correctly failed because the disposable checkout was outside configured file roots. No raw session identifiers, paths, repository names, commands, or timestamps are included here. This is agent scope/tool-selection debt, not evidence for widening policy authority. |
Aaronontheweb
force-pushed
the
fix/issue-1892-seeded-project-eval
branch
from
August 18, 2026 17:07
e8ae7ee to
719e4ac
Compare
This was referenced Aug 18, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1892.
This adds a seeded Git-project eval and strengthens generic project-scope guidance so agents declare the task-named path before probing it. If that declaration fails, the user-provided fallback is declared before project tools continue. The same contract is aligned across always-loaded guidance, subagents, the tool schema, and bundled project/operations skills.
It also aligns generic scratch guidance: disposable writable work outside a project uses
session_dir, not platform temporary storage, unless the task explicitly requires a platform temporary path. Shell guidance asks for one operation per call and permits a pipeline only when the requested result requires it.This changes guidance and eval coverage only. It does not change shell authority or add executable-specific parsing.
Final direct-provider evals on the exact local image:
These trials used the direct DeepSeek API. Spark2 was unavailable and is not part of the evidence.
Validation: