Skip to content

feat(commands): make @gittensory chat's frontier fallback configurable (#4595 follow-up) - #5015

Merged
JSONbored merged 1 commit into
mainfrom
feat/chat-qa-frontier-fallback
Jul 11, 2026
Merged

feat(commands): make @gittensory chat's frontier fallback configurable (#4595 follow-up)#5015
JSONbored merged 1 commit into
mainfrom
feat/chat-qa-frontier-fallback

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Follow-up to #4595 (@gittensory chat). That PR shipped chatQa as strictly Ollama-only, with no fallback path — this adds a companion opt-in flag for self-hosters who'd rather use their own frontier subscription/tokens for chat than run a local GPU.

  • New advisoryAiRouting.chatQaFrontierFallback: boolean (default false). When env.AI_ADVISORY is unconfigured and this is explicitly enabled, chat falls back to the shared frontier env.AI chain instead of declining.
  • Ollama stays strictly preferred: even with the flag on, env.AI_ADVISORY is always tried first — the frontier chain is only ever touched when Ollama is genuinely unavailable.
  • Global default + per-repo override, identical shape to every other advisoryAiRouting capability (yml-only, config-as-code, both normalizers + engine-package twin + focus-manifest merge + OpenAPI schema updated).
  • Usage/audit events now record which provider actually served a given answer (provider: "advisory" | "frontier"), so an operator who enables this can see at a glance whether they're spending frontier tokens.

Also verified and closed a real gap while implementing this: chatQa had never actually been activated on any production repo (the capability existed in code from #4595 but no .gittensory.yml set it), and no repo had commandRateLimitPolicy turned on — meaning the command would have had zero real rate-limit ceiling if enabled. Both are now fixed on the self-host deployment (this repo + awesome-claude): chatQa: true, chatQaFrontierFallback left unset (Ollama-only, matching this deployment's own preference), commandRateLimitPolicy: hold so the existing 5-per-24h AI_COST_BEARING_COMMANDS ceiling actually applies.

Test plan

  • New tests: frontier-fallback success path, Ollama-still-preferred-when-fallback-enabled, neither-provider-configured message, plus the untouched default-off behavior re-asserted
  • Capability-flag normalization (it.each field lists) + focus-manifest sparse-parse/merge tests, both app and engine-package copies
  • npm run typecheck, npm run test:coverage (unsharded, 100% on every changed line/branch), npm run test:ci, npm audit --audit-level=moderate all green
  • npm run ui:openapi regenerated and committed
  • npx tsx scripts/check-engine-parity.ts passes

#4595 follow-up)

Adds advisoryAiRouting.chatQaFrontierFallback (default false), letting a
self-hoster without a local GPU opt chat into the frontier chain when
env.AI_ADVISORY is unconfigured, instead of declining outright. Ollama stays
strictly preferred even when the flag is on — the frontier chain is only
ever touched as a last resort. Global default + per-repo override, same as
every other advisoryAiRouting capability.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui 362c203 Commit Preview URL

Branch Preview URL
Jul 11 2026, 08:58 AM

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.34%. Comparing base (a086033) to head (362c203).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5015   +/-   ##
=======================================
  Coverage   94.34%   94.34%           
=======================================
  Files         456      456           
  Lines       39188    39196    +8     
  Branches    14290    14297    +7     
=======================================
+ Hits        36970    36978    +8     
  Misses       1570     1570           
  Partials      648      648           
Flag Coverage Δ
shard-1 46.81% <0.00%> (-0.01%) ⬇️
shard-2 33.42% <15.38%> (+0.15%) ⬆️
shard-3 31.49% <0.00%> (+0.22%) ⬆️
shard-4 32.74% <84.61%> (-0.40%) ⬇️
shard-5 33.56% <0.00%> (-0.01%) ⬇️
shard-6 45.16% <15.38%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/gittensory-engine/src/focus-manifest.ts 99.17% <100.00%> (+<0.01%) ⬆️
...ry-engine/src/review/advisory-ai-routing-config.ts 100.00% <ø> (ø)
src/openapi/schemas.ts 100.00% <ø> (ø)
src/review/advisory-ai-routing-config.ts 100.00% <ø> (ø)
src/services/ai-chat-qa.ts 100.00% <100.00%> (ø)
src/signals/focus-manifest.ts 99.64% <100.00%> (+<0.01%) ⬆️
src/types.ts 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 11, 2026
@loopover-orb

loopover-orb Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-11 09:04:38 UTC

16 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · unstable

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This PR adds an opt-in `advisoryAiRouting.chatQaFrontierFallback` flag so `@​gittensory chat` can fall back to the shared frontier `env.AI` when `env.AI_ADVISORY` is unconfigured, while still preferring Ollama first. The core logic in `src/services/ai-chat-qa.ts` (`env.AI_ADVISORY ?? (frontierFallbackAllowed ? env.AI : undefined)`) correctly preserves Ollama-first behavior and only touches the frontier chain when explicitly opted in, and the flag is wired consistently through every layer this repo requires for a new `advisoryAiRouting` capability: type twins (`src/types.ts` and `packages/gittensory-engine/src/types/manifest-deps-types.ts`), both normalizer copies, the zod schema, OpenAPI spec, focus-manifest sparse-override parsing, and `resolveEffectiveSettings` merge — matching the identical pattern used for `chatQa` itself. Tests cover both branches (Ollama-preferred-even-when-fallback-enabled, frontier-used-when-Ollama-absent, neither-configured) plus the default-off regression case, so the new logic is real coverage, not fabricated.

Nits — 5 non-blocking
  • The PR description claims `.gittensory.yml` changes for this repo and awesome-claude (`chatQa: true`, `commandRateLimitPolicy: hold`) were made, but no such file diff is visible in the provided changeset — only the `.gittensory.yml.example`/`config/examples/gittensory.full.yml` docs are touched; confirm those live config changes actually shipped in this PR or a companion commit.
  • The description doesn't link an eligible open issue for this follow-up work, only referencing PR feat(commands): add @gittensory chat <question> — grounded LLM Q&A via local Ollama #4595 — worth a maintainer note per repo convention even though the change itself is narrow and well-scoped.
  • The `advisory-ai-routing-config.ts` normalizer is duplicated verbatim between `src/review/` and `packages/gittensory-engine/src/review/` (both edited identically here); pre-existing twin-file pattern, not introduced by this diff, but a DRY candidate.
  • Consider a short comment or CHANGELOG note pointing at wherever `chatQa`/`chatQaFrontierFallback` get enabled in this repo's actual `.gittensory.yml`, so the audit trail matches the description's claim.
  • If the twin normalizer files (`src/review/advisory-ai-routing-config.ts` vs `packages/gittensory-engine/src/review/advisory-ai-routing-config.ts`) are meant to converge, this would be a reasonable moment to extract a shared implementation given how mechanically identical the two diffs are.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 47 registered-repo PR(s), 39 merged, 428 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 47 PR(s), 428 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 47 PR(s), 428 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
Visual preview
Route Viewport Before (production) After (this PR's preview) Diff
/ desktop before / after /
/ mobile before / (mobile) after / (mobile)

Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 11, 2026
@JSONbored
JSONbored merged commit 9429030 into main Jul 11, 2026
20 checks passed
@JSONbored
JSONbored deleted the feat/chat-qa-frontier-fallback branch July 11, 2026 09:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant