Skip to content

Add ChatGPT OAuth model backend and fix evolution runtime regressions#22

Open
kyzer1022 wants to merge 2 commits into
NousResearch:mainfrom
kyzer1022:fix/chatgpt-oauth-evolution-runtime
Open

Add ChatGPT OAuth model backend and fix evolution runtime regressions#22
kyzer1022 wants to merge 2 commits into
NousResearch:mainfrom
kyzer1022:fix/chatgpt-oauth-evolution-runtime

Conversation

@kyzer1022
Copy link
Copy Markdown

@kyzer1022 kyzer1022 commented Apr 14, 2026

Summary

  • add a ChatGPT OAuth-backed DSPy LM for chatgpt/* models so skill evolution can run without OpenAI API keys
  • rebuild ChatGPTOAuthLM instances on copy() so DSPy avoids deepcopying the OpenAI/httpx client stack during optimization
  • fix evolve_skill.py to prefer explicit dataset_path loading, validate the full skill text, and pass valset into the MIPRO fallback path so tiny datasets stop crashing
  • add regression coverage for OAuth token resolution/request building, LM copy behavior, dataset-path precedence, full-skill validation, and GEPA → MIPRO fallback wiring

Testing

  • .venv/bin/pytest -q
  • .venv/bin/python -m evolution.skills.evolve_skill --skill github-code-review --iterations 1 --dataset-path /home/kyzer/Documents/projects/hermes-agent-self-evolution/tmp/github-code-review-mini --optimizer-model chatgpt/gpt-5.4 --eval-model chatgpt/gpt-5.4 --hermes-repo /home/kyzer/.hermes/hermes-agent

Notes

  • Verified that a real evolution run now completes end-to-end with ChatGPT OAuth models.
  • The latest mini-dataset run completed successfully and saved artifacts under output/github-code-review/20260414_164858/.
  • The generated baseline/evolved skill artifacts were still byte-identical, which points to a separate optimizer/artifact-mutation issue outside the runtime fixes in this PR.

- rebuild ChatGPTOAuthLM clients during LM copies so DSPy avoids deepcopying the OpenAI/httpx stack
- pass valset into the MIPRO fallback path for tiny datasets
- validate full skill text during evolution and prefer explicit dataset_path loading
- add regression tests for OAuth copy, dataset loading, full-skill validation, and GEPA fallback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant