Skip to content

Bug: gemini-eval.mjs reports inaccurate, user profile not passed to endpoint #617

@DSnoNintendo

Description

@DSnoNintendo

Code of Conduct

Existing issues

  • I searched existing issues and this hasn't been reported yet

What happened?

A user who has customized profile.yml gets evaluations that silently ignore those customizations when running via Gemini

gemini-eval.mjs is documented as the free-tier alternative to the Claude/agent path, but it does not load all the context files the agent path is required to load. Specifically, it skips config/profile.yml and modes/_profile.md, so offers are evaluated without the user's archetypes, North Star, comp targets, location policy, narrative, or proof-point metrics. This skews report accuracy

This contradicts the explicit "Sources of Truth" contract in modes/_shared.md, which lists those files as always loaded.

Steps to reproduce

.

Expected behavior

No response

CLI tool

No response

OS and Node.js version

No response

Output of npm run doctor

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions