Skip to content

[BUG] Raw fetch to GitHub Violates the Codebase's Own "No Raw Fetch" Hard Rule and Silently Degrades Sync + Rate-Budget Tracking #859

Description

@ionfwsrijan

Description

src/lib/github/app.ts:12 states the invariant: "Hard rule: no raw fetch to GitHub anywhere in the codebase." The Octokit install factory wraps every request in a hook that parses x-ratelimit-remaining / x-ratelimit-reset and persists them via updateRateBudget (app.ts:71-98), which feeds checkRateBudget used by all background jobs. The user-sync path never uses Octokit:

  • src/app/actions/github-sync.ts:49-74 builds headers manually, treats the install token as optional (catch → "proceed without auth — public PRs still visible", lines 54-61), then calls fetch(url, { headers }) (line 67) against the search API (author:<handle>, per_page=100, line 64). If the request fails it silently return [] (lines 72-74).
  • src/app/actions/github-sync-helpers.ts:50 (fetchMergedCount, search) and :90 (fetchContributionCalendar, GraphQL) also use raw fetch and throw on non-OK (lines 60, 102).
  • src/inngest/functions/github-stats-sync.ts:39-44 re-uses the same raw helpers in the background job.
  • src/app/actions/github-sync.ts:156-164 runs these three raw helpers in a Promise.all, so one search-API failure kills the entire sync.

Expected Behavior

All GitHub traffic goes through getInstallOctokit() so tokens, rate-budget, and secondary-rate-limit backoff are centralized, and a token failure fails closed.

Actual Behavior

  1. Search API quota is a separate, far tighter bucket (≈10/min unauthenticated, ≈30/min per token) than the 5,000/hr core budget that checkRateBudget tracks. When exhausted, fetchMergedCount throws and the whole Promise.all rejects — the action surfaces github_api_error and the PR backfill is skipped. No retry/backoff, no Retry-After handling.
  2. Unauthenticated fallback is both leaky and broken: anonymous search requests share the server egress IP's 10/min quota — concurrent syncs can 403 each other. The GraphQL call can't work unauthenticated at all.
  3. Rate budget goes stale: raw fetches never parse rate-limit headers, so gh:budget:install:{id} isn't updated by sync traffic, and issues-sweep/pr-backfill can over-consume believing the budget is fresh.
  4. Behavioral divergence: fetchAndBackfillPRs swallows errors while the other two helpers throw; github-stats-sync.ts doesn't call fetchAndBackfillPRs at all, so background sync never backfills PRs.

Affected Files

  • src/lib/github/app.ts (lines 12, 71-98) — Enforced invariant + rate-budget tracking
  • src/app/actions/github-sync.ts (lines 49-74, 156-164) — Raw fetch, optional token, silent failure
  • src/app/actions/github-sync-helpers.ts (lines 50, 60, 90, 102) — Raw search + GraphQL fetches
  • src/inngest/functions/github-stats-sync.ts (lines 39-44) — Background job reusing raw helpers

Proposed Fix

Route all three helpers through getInstallOctokit(installId) using Octokit's search.issues, graphql, and paginated list methods; fail closed when the token can't be minted; add secondary rate-limit handling honoring Retry-After; centralize backfill into one implementation shared by the action and the Inngest job.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions