Skip to content

Fix Model Requirements & Fallbacks (PR #3)#4

Merged
heidi-dang merged 4 commits into
mainfrom
fix/model-requirements-pr3
Mar 6, 2026
Merged

Fix Model Requirements & Fallbacks (PR #3)#4
heidi-dang merged 4 commits into
mainfrom
fix/model-requirements-pr3

Conversation

@heidi-dang
Copy link
Copy Markdown
Owner

This PR addresses the hardcoded model requirements that restricted Hephaestus and other agents to GPT models, as requested in PR #3.

Key changes:

  • Removed hardcoded 'gpt-5.3-codex' from 'hephaestus' fallback chains.
  • Removed 'requiresModel: gpt-5.3-codex' from 'deep' and other categories.
  • Added regression test 'tests/hephaestus-model-resolution.test.ts' to verify that user-configured models are respected.
  • Refined CLI fallback generation logic.

… ports

- Revert src/hooks/runtime-enforcement/hook.ts to HEAD (runtime authority unchanged)
- Revert src/agents/runtime/state-ledger.ts to HEAD (runtime authority unchanged)
- Revert src/agents/runtime/tool-runner.ts to HEAD (runtime authority unchanged)
- Revert src/agents/oracle.ts to HEAD (Heidi baseline is already stronger)
- Revert src/agents/sisyphus.ts (flat file) to HEAD - Heidi's modular flow preserved
- Revert src/agents/builtin-agents.ts to HEAD - sisyphus-agent wiring preserved
- Hephaestus dir: prompt-only, imports hard-blocks/anti-patterns from Heidi's prompts module
- Sisyphus dir: prompt-only, imports hard-blocks/anti-patterns from Heidi's prompts module
- Add isGpt5_4Model/isGpt5_3CodexModel type guards to types.ts for Hephaestus dispatch
- Update doctor check: remove dynamic-agent-prompt-builder as forbidden (passive library)
- Update doctor check: loop guard deferred to separate runtime-only PR

Phase 0 changes (Atlas/Gemini/GPT verification wave) and Sisyphus/Hephaestus
prompt-layer capability improvements are preserved. No runtime changes in this PR.
…edger

- Removed dynamic-agent-prompt-builder imports from Sisyphus and Hephaestus
- Ported official wording/orchestration into src/agents/prompts/orchestration.ts
- Moved Agent/Skill/Tool/Category types into src/agents/types.ts
- Deleted dynamic-agent-prompt-builder and restored it to the doctor forbidden list
- Tightened runtime enforcement hook to scan current chat flow instead of global historical ledger
- Dropped generic phrase matching ('success') from enforcement checks
- Unused/shadow state-ledger from agents runtime deleted
- Unified state-ledger across agent and core runtime
@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the flexibility, intelligence, and reliability of the agent system. It removes previous hardcoded model restrictions, allowing agents like Hephaestus to operate with a wider range of models through dynamic prompt generation. The iterative development loop (Ralph Loop) has been made more robust with advanced verification and session management. Crucially, the underlying 'Truth Model' has been hardened to ensure that all agent actions are verifiable and accurately recorded, preventing agents from making unproven claims. These changes collectively lead to more adaptable and trustworthy autonomous agents.

Highlights

  • Model Flexibility: Removed hardcoded model requirements for the 'hephaestus' agent and the 'deep' category, allowing for broader model compatibility beyond GPT-specific models.
  • Dynamic Agent Prompting: Implemented dynamic prompt generation for Hephaestus and Sisyphus agents, tailoring their behavior based on the specific model being used (e.g., GPT-5.4, GPT-5.3 Codex, Gemini, or generic GPT).
  • Enhanced Ralph Loop: Significantly upgraded the Ralph Loop (iterative development loop) with features like ultrawork verification, a session reset strategy, and more robust completion detection mechanisms.
  • Truth Model & Runtime Enforcement: Strengthened the 'Truth Model' with a new state ledger, ensuring that tool executions are strictly verified, successful, and tied to the current session's completion flow, preventing false claims.
  • Stricter QA and Verification: Updated Atlas and Momus agents with more stringent QA requirements and a 'Final Verification Wave' logic to ensure higher quality and executability of plans.
  • XAI Usage Patch: Introduced a new hook to correct token usage display for xAI/Grok models by patching cache.read tokens into input tokens.
  • Refactored Prompt Building: Centralized agent prompt building logic into a new 'prompts' directory, improving modularity and maintainability.
  • Comprehensive Testing: Added extensive new tests for model resolution, Ralph Loop functionality, and the new truth model integration to ensure reliability and correctness.
Changelog
  • .official/master/oracle.ts
    • Added a new Oracle agent configuration, defining its role as a strategic technical advisor with specific use cases and avoidance scenarios.
  • .official/master/sisyphus.ts
    • Added a new Sisyphus agent configuration, defining its role as a powerful AI orchestrator with core competencies and operating modes.
  • .official/src/agents/hephaestus-agent.ts
    • Modified the maybeCreateHephaestusConfig function to remove hardcoded model requirements and adjust fallback logic for Hephaestus agent creation.
  • .official/src/agents/hephaestus/agent.test.ts
    • Added new tests for getHephaestusPromptSource, getHephaestusPrompt, and createHephaestusAgent to verify model-specific prompt generation and agent configuration.
  • .official/src/agents/hephaestus/agent.ts
    • Added new agent logic for Hephaestus, centralizing agent creation and dynamic prompt building based on the model type.
  • .official/src/agents/hephaestus/gpt-5-3-codex.ts
    • Added a new prompt builder specifically optimized for GPT-5.3 Codex models for the Hephaestus agent.
  • .official/src/agents/hephaestus/gpt-5-4.ts
    • Added a new prompt builder specifically optimized for GPT-5.4 models for the Hephaestus agent.
  • .official/src/agents/hephaestus/gpt.ts
    • Added a new generic GPT prompt builder for the Hephaestus agent, serving as a fallback for models without specific variants.
  • .official/src/agents/hephaestus/index.ts
    • Added an index file to export Hephaestus agent components, including creation functions and metadata.
  • .official/src/agents/oracle.ts
    • Modified the Oracle agent configuration to use ORACLE_DEFAULT_PROMPT for non-GPT models and ORACLE_GPT_PROMPT for GPT models, and updated tool restrictions.
  • .official/src/agents/sisyphus.ts
    • Modified the Sisyphus agent configuration to dynamically build prompts based on the model type (GPT-5.4, Gemini, or default) and integrate Gemini-specific overrides.
  • .official/src/agents/sisyphus/default.ts
    • Added a default Sisyphus prompt builder, including task management sections for both task and todo systems.
  • .official/src/agents/sisyphus/gemini.ts
    • Added Gemini-specific prompt overlays for the Sisyphus agent, addressing Gemini's tendencies to skip tool calls, avoid delegation, and claim completion without verification.
  • .official/src/agents/sisyphus/gpt-5-4.ts
    • Added a GPT-5.4 native Sisyphus prompt builder, designed with OpenAI's GPT-5.4 prompting guidance principles.
  • .official/src/agents/sisyphus/index.ts
    • Added an index file to export Sisyphus prompt builders and their model-specific variants.
  • .official/src/hooks/ralph-loop/AGENTS.md
    • Updated the generation date in the Ralph Loop documentation.
  • .official/src/hooks/ralph-loop/command-arguments.ts
    • Added new types and a function for parsing Ralph Loop command arguments, including strategy for loop behavior.
  • .official/src/hooks/ralph-loop/completion-handler.ts
    • Added a new handler for detected completion in Ralph Loop, incorporating ultrawork verification logic and toast notifications.
  • .official/src/hooks/ralph-loop/completion-promise-detector.test.ts
    • Added tests for completion detection in Ralph Loop, specifically verifying behavior with sinceMessageIndex and different message scenarios.
  • .official/src/hooks/ralph-loop/completion-promise-detector.ts
    • Modified completion detection logic to use sinceMessageIndex for more precise scanning and improved handling of transcript entries.
  • .official/src/hooks/ralph-loop/constants.ts
    • Added ULTRAWORK_VERIFICATION_PROMISE constant for ultrawork loop verification.
  • .official/src/hooks/ralph-loop/continuation-prompt-builder.ts
    • Modified continuation prompt building to support ultrawork verification and dynamically display max iterations as 'unbounded' when not set.
  • .official/src/hooks/ralph-loop/continuation-prompt-injector.ts
    • Added logic to inject continuation prompts, ensuring agent, model, and tools are inherited from the source session.
  • .official/src/hooks/ralph-loop/index.test.ts
    • Added extensive tests for Ralph Loop hook functionality, covering ultrawork mode, session management, completion detection, and race conditions.
  • .official/src/hooks/ralph-loop/index.ts
    • Updated exports for Ralph Loop components, including new storage functions and the main hook.
  • .official/src/hooks/ralph-loop/iteration-continuation.ts
    • Added logic for continuing Ralph Loop iterations, including handling the 'reset' strategy by creating a new session.
  • .official/src/hooks/ralph-loop/loop-session-recovery.ts
    • Added a new module for managing loop session recovery state, tracking and clearing recovery flags.
  • .official/src/hooks/ralph-loop/loop-state-controller.ts
    • Modified loop state controller to support ultrawork, verification pending states, and new session management functionalities.
  • .official/src/hooks/ralph-loop/message-storage-directory.ts
    • Added an export for getMessageDir from the shared utility.
  • .official/src/hooks/ralph-loop/ralph-loop-event-handler.ts
    • Modified the event handler for Ralph Loop to incorporate ultrawork, session existence checks, and new completion detection logic.
  • .official/src/hooks/ralph-loop/ralph-loop-hook.ts
    • Modified the Ralph Loop hook to initialize the state controller and event handler, and to handle message count at the start of a loop.
  • .official/src/hooks/ralph-loop/reset-strategy-race-condition.test.ts
    • Added tests for race conditions in Ralph Loop's reset strategy, ensuring proper handling of concurrent idle events.
  • .official/src/hooks/ralph-loop/session-event-handler.ts
    • Added new handlers for deleted and errored loop sessions, including logic for user aborts and recovery.
  • .official/src/hooks/ralph-loop/session-reset-strategy.ts
    • Added functions for creating new iteration sessions and selecting sessions in the TUI for the 'reset' strategy.
  • .official/src/hooks/ralph-loop/storage.ts
    • Modified storage functions for Ralph Loop state to handle new fields like ultrawork, verification_pending, initial_completion_promise, verification_attempt_id, verification_session_id, and strategy.
  • .official/src/hooks/ralph-loop/types.ts
    • Added new types for Ralph Loop state and options, including ultrawork, verification_pending, and strategy.
  • .official/src/hooks/ralph-loop/ulw-loop-verification.test.ts
    • Added tests for ultrawork loop verification, ensuring the correct transition to verification phase and completion.
  • .official/src/hooks/ralph-loop/with-timeout.ts
    • Added a utility function withTimeout to handle promises with specified timeouts.
  • .runtime/journal/execution.jsonl
    • Added new execution journal entries, reflecting various tool executions and their verification states.
  • bun.lock
    • Updated dependencies, including playwright and @heidi-dang/oh-my-opencode-* versions.
    • Removed the @opencode-ai/plugin/zod override.
  • docs/implementation_upstream_capability_merge.md
    • Added a new documentation file outlining the technical plan for upstream capability merge, marking Phase 0 and 1 as completed and detailing future phases.
  • src/agents/atlas/agent.ts
    • Updated imports for AvailableAgent, AvailableSkill, and AvailableCategory to use the new ../types and ../prompts modules.
  • src/agents/atlas/default.ts
    • Modified the Atlas default prompt to include 'Final Verification Wave' logic and updated todo tracking for implementation tasks and final wave approval.
  • src/agents/atlas/gemini.ts
    • Modified the Atlas Gemini prompt to include 'Final Verification Wave' logic and updated todo tracking for implementation tasks and final wave approval.
  • src/agents/atlas/gpt.ts
    • Modified the Atlas GPT prompt to include 'Final Verification Wave' logic and updated todo tracking for implementation tasks and final wave approval.
  • src/agents/atlas/prompt-section-builder.ts
    • Updated imports for AvailableAgent and AvailableSkill to use the new ../types module.
  • src/agents/builtin-agents.ts
    • Updated imports for AvailableCategory to use the new ../types module.
  • src/agents/builtin-agents/atlas-agent.ts
    • Updated imports for AvailableAgent and AvailableSkill to use the new ../types module.
  • src/agents/builtin-agents/available-skills.ts
    • Updated imports for AvailableSkill to use the new ../types module.
  • src/agents/builtin-agents/general-agents.ts
    • Updated imports for AvailableAgent to use the new ../types module.
  • src/agents/builtin-agents/hephaestus-agent.ts
    • Updated imports for AvailableAgent, AvailableCategory, and AvailableSkill to use the new ../types module.
  • src/agents/builtin-agents/sisyphus-agent.ts
    • Updated imports for AvailableAgent, AvailableCategory, and AvailableSkill to use the new ../types module.
  • src/agents/dynamic-agent-prompt-builder.test.ts
    • Updated imports for AvailableSkill, AvailableCategory, and AvailableAgent to use the new ../prompts module.
  • src/agents/hephaestus.ts
    • Updated imports for AvailableTool, AvailableSkill, AvailableCategory to use the new ../types and ../prompts modules, reflecting the refactoring of prompt building logic.
  • src/agents/index.ts
    • Updated exports for AvailableAgent, AvailableCategory, and AvailableSkill to be sourced from types.
  • src/agents/metis.ts
    • Updated the Metis prompt to include stricter QA/Acceptance Criteria Directives, emphasizing executable and concrete QA scenarios.
  • src/agents/momus.ts
    • Updated the Momus prompt to include a 'QA Scenario Executability' check and introduced a GPT-5.4 optimized prompt variant for improved review quality.
  • src/agents/prompts/index.ts
    • Added exports for anti-patterns, execution-rules, and orchestration modules, centralizing prompt section builders.
  • src/agents/prompts/orchestration.ts
    • Added a new module orchestration.ts containing functions for building various prompt sections, such as tool selection, explore, librarian, delegation, and Oracle usage, now using the new ../types module.
  • src/agents/types.ts
    • Added new interfaces AvailableAgent, AvailableTool, AvailableSkill, AvailableCategory to standardize agent-related types.
    • Added new model detection functions isGpt5_4Model and isGpt5_3CodexModel for more granular GPT model identification.
  • src/cli/model-fallback-requirements.ts
    • Removed requiresProvider from hephaestus and requiresModel from the deep category.
    • Added claude-opus-4-6 to the Hephaestus fallback chain.
  • src/config/schema/hooks.ts
    • Added new hook names: execution-journal, tool-contract, runtime-enforcement, plan-enforcement, semantic-loop-guard, and xai-usage-patch.
  • src/create-hooks.ts
    • Updated imports for AvailableSkill to use the new agents/types module.
  • src/create-tools.ts
    • Updated imports for AvailableCategory and AvailableSkill to use the new agents/types module.
  • src/hooks/category-skill-reminder/formatter.ts
    • Updated imports for AvailableSkill to use the new agents/types module.
  • src/hooks/category-skill-reminder/hook.ts
    • Updated imports for AvailableSkill to use the new agents/types module.
  • src/hooks/category-skill-reminder/index.test.ts
    • Updated imports for AvailableSkill to use the new agents/types module.
  • src/hooks/execution-journal/hook.ts
    • Modified to record success, verified, changedState, stdout, and sessionID in the ledger.
    • Removed heuristic-based bash command logging fallbacks.
  • src/hooks/index.ts
    • Added export for createXaiUsagePatchHook.
  • src/hooks/no-hephaestus-non-gpt/hook.ts
    • Removed the logic that previously warned or forced Hephaestus to GPT models, allowing explicit configuration of non-GPT models.
  • src/hooks/no-hephaestus-non-gpt/index.test.ts
    • Updated tests to reflect the removal of non-GPT model restrictions for Hephaestus, ensuring no warnings or agent switches occur.
  • src/hooks/runtime-enforcement/hook.ts
    • Modified to use ledger.startNewFlow() to isolate checks to the current completion flow.
    • Updated logic to verify tool execution within the current flow, rather than relying on historical ledger entries.
  • src/hooks/tool-contract/hook.ts
    • Modified to enforce strict boolean types for success, verified, and changedState in tool metadata.
    • Added a check to verify that claimed state changes have a matching successful and verified entry in the State Ledger for the current session.
  • src/hooks/xai-usage-patch/hook.test.ts
    • Added tests for the xai-usage-patch hook, verifying correct token patching for xAI providers.
  • src/hooks/xai-usage-patch/hook.ts
    • Added a new hook createXaiUsagePatchHook to correct xAI token usage display by adding cache.read to input tokens.
  • src/plugin/available-categories.ts
    • Updated imports for AvailableCategory to use the new agents/types module.
  • src/plugin/hooks/create-session-hooks.ts
    • Added createXaiUsagePatchHook and enabled the xaiUsagePatch hook.
  • src/plugin/hooks/create-skill-hooks.ts
    • Updated imports for AvailableSkill to use the new agents/types module.
  • src/plugin/hooks/create-tool-guard-hooks.ts
    • Added new tool guard hooks: executionJournal, toolContract, runtimeEnforcement, planEnforcement, and semanticLoopGuard.
  • src/plugin/messages-transform.ts
    • Integrated the runtimeEnforcement hook into the experimental.chat.messages.transform pipeline.
  • src/plugin/skill-context.ts
    • Updated imports for AvailableSkill to use the new agents/types module.
  • src/plugin/tool-execute-after.ts
    • Integrated executionJournal and toolContract hooks into the tool.execute.after event handler.
  • src/plugin/tool-execute-before.ts
    • Integrated planEnforcement and semanticLoopGuard hooks into the tool.execute.before event handler.
  • src/plugin/tool-registry.ts
    • Updated imports for AvailableCategory to use the new agents/types module.
    • Integrated DETERMINISTIC_TOOLS from runtime/tools/registry into the tool registry.
  • src/plugin/truth-model-integration.test.ts
    • Added new integration tests for authoritative truth and flow isolation, verifying ledger behavior and runtime enforcement.
  • src/runtime/state-ledger.ts
    • Modified LedgerEntry interface to include success, verified, changedState, stdout, and sessionID.
    • Added startNewFlow() to mark the beginning of a new completion flow.
    • Updated record() and has() methods to consider flow isolation based on lastFlowStartTime.
  • src/runtime/tools/complete-task.ts
    • Modified to filter ledger entries strictly by verified, success, changedState, and sessionID for the current flow.
    • Updated metadata to include success, verified, changedState, and sessionID.
  • src/runtime/tools/fs-safe.ts
    • Updated metadata for fs_safe tool to include success, verified, and changedState.
  • src/runtime/tools/git-safe.ts
    • Updated metadata for git_safe tool to include success, verified, and changedState.
  • src/runtime/tools/plan.ts
    • Updated metadata for submit_plan and mark_step_complete tools to include success, verified, and changedState.
  • src/runtime/tools/query-ledger.ts
    • Modified to filter ledger entries strictly by verified, success, and sessionID for the current flow.
    • Updated metadata to include success, verified, and changedState.
  • src/runtime/tools/verify.ts
    • Updated metadata for verify_action tool to include success, verified, and changedState.
  • src/runtime/truth-model.test.ts
    • Added new tests for truth model live wiring evidence, verifying correct filtering and recording of ledger entries.
  • src/shared/model-requirements.test.ts
    • Removed the test case specifically checking Hephaestus's requiresProvider.
  • src/shared/model-requirements.ts
    • Removed requiresProvider from hephaestus and requiresModel from the deep category.
    • Added claude-opus-4-6 to the Hephaestus fallback chain.
  • src/shared/model-resolver.test.ts
    • Added tests for Hephaestus model resolution, ensuring explicit user models are respected and fallback rules apply when unconfigured.
  • src/tools/delegate-task/constants.ts
    • Updated imports for AvailableCategory and AvailableSkill to use the new agents/types module.
  • src/tools/delegate-task/tools.ts
    • Updated imports for AvailableCategory and AvailableSkill to use the new agents/types module.
  • src/tools/delegate-task/types.ts
    • Updated imports for AvailableCategory and AvailableSkill to use the new agents/types module.
  • tools/checks/check_upstream_capability_merge.py
    • Added a new Python script to verify upstream merge capabilities, checking for required prompt content and absence of forbidden files/strings.
  • tools/doctor.py
    • Modified to run check_upstream_capability_merge.py as part of the overall doctor checks.
  • tools/doctor_runtime_reliability.py
    • Updated the list of required files, removing old runtime files and adding new hook files to reflect the refactored architecture.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

…ating ledger entries and implement flow isolation in runtime enforcement to validate state claims against the current execution flow.
@heidi-dang heidi-dang force-pushed the fix/model-requirements-pr3 branch from e63d6a3 to 34de622 Compare March 6, 2026 19:02
@heidi-dang heidi-dang merged commit 71387ef into main Mar 6, 2026
1 check passed
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a significant and impressive refactoring that achieves two major goals. First, it successfully decouples agents from hardcoded model requirements, particularly removing the GPT-only restriction for Hephaestus and other categories, which aligns perfectly with the PR's stated objective. The addition of regression tests for model resolution is a great touch.

Second, it introduces a substantial new reliability layer, which you've called the "Truth Model" and "Runtime Enforcement". This includes new safe tools (git_safe, fs_safe), a state ledger, and several hooks to enforce tool contracts and prevent agent hallucinations about state changes. The introduction of flow isolation in the state ledger is a particularly strong improvement for ensuring deterministic verification. While this was a large addition not mentioned in the PR description, the implementation appears robust and well-tested.

Overall, these changes represent a major step forward in the platform's reliability and flexibility. I've left a couple of minor comments below.

Note: Security Review did not run due to the size of the PR.

I am having trouble creating individual review comments. Click here to see my feedback.

.official/src/agents/hephaestus/gpt-5-3-codex.ts (525)

medium

There's a small typo in the agent description. It says 'GPT 5.2 Codex' in a file that is for 'gpt-5-3-codex'. This should be updated for consistency.

      "Autonomous Deep Worker - goal-oriented execution with GPT 5.3 Codex. Explores thoroughly before acting, uses explore/librarian agents for comprehensive context, completes tasks end-to-end. Inspired by AmpCode deep mode. (Hephaestus - OhMyOpenCode)",

.runtime/journal/execution.jsonl (1-17)

medium

This appears to be a runtime log file. Log files are typically not checked into source control as they can bloat the repository and cause unnecessary merge conflicts. Please consider adding this file path (.runtime/journal/execution.jsonl) to your .gitignore file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant