Skip to content

Fix GEO optimizer project-site AI discovery false negatives - #53800

Merged
pelikhan merged 10 commits into
mainfrom
copilot/deep-report-fix-geo-optimizer
Aug 18, 2026
Merged

Fix GEO optimizer project-site AI discovery false negatives#53800
pelikhan merged 10 commits into
mainfrom
copilot/deep-report-fix-geo-optimizer

Conversation

Copilot AI commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

The GEO optimizer audit was reporting missing llms.txt and AI discovery files because scanner findings did not account for the /gh-aw/ GitHub Pages project-site base path. Those false negatives caused duplicate follow-up issues for files that are already live.

  • Project-site verification

    • Adds authoritative HTTP checks for project-scoped AI discovery URLs:
      https://github.github.com/gh-aw/llms.txt
      https://github.github.com/gh-aw/.well-known/ai.txt
      https://github.github.com/gh-aw/ai/summary.json
      https://github.github.com/gh-aw/ai/faq.json
      https://github.github.com/gh-aw/ai/service.json
    • Persists verification metadata and successful response bodies in the GEO optimizer artifact.
  • False-negative handling

    • Updates the audit instructions to treat project-site verification as authoritative for llms.txt and AI discovery findings.
    • Prevents missing-file recommendations from being selected when the project-site verification succeeds.
  • Workflow output

    • Adds a structured docs-ai-discovery-verification.json summary:
      {
        "base_url": "https://github.github.com/gh-aw",
        "summary": {
          "llms_txt_found": true,
          "ai_discovery_found_count": 4,
          "ai_discovery_total": 4,
          "ai_discovery_all_found": true
        }
      }
    • Regenerates the compiled workflow lock file.

Copilot AI and others added 9 commits August 18, 2026 19:03
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix false 0-scores reported by GEO Optimizer for live files Fix GEO optimizer project-site AI discovery false negatives Aug 18, 2026
Copilot AI requested a review from pelikhan August 18, 2026 19:24
@pelikhan
pelikhan marked this pull request as ready for review August 18, 2026 19:25
Copilot AI balanced review requested due to automatic review settings August 18, 2026 19:25
@pelikhan
pelikhan merged commit 360ae9f into main Aug 18, 2026
@pelikhan
pelikhan deleted the copilot/deep-report-fix-geo-optimizer branch August 18, 2026 19:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds project-scoped verification to prevent GEO audit false negatives for existing AI discovery files.

Changes:

  • Verifies and preserves AI discovery responses.
  • Updates audit guidance to prioritize verification results.
  • Regenerates the compiled workflow.
Show a summary per file
File Description
.github/workflows/daily-geo-optimizer.md Adds verification and audit instructions.
.github/workflows/daily-geo-optimizer.lock.yml Regenerates the compiled workflow.

Review details

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (2)

.github/workflows/daily-geo-optimizer.md:284

  • The aggregate ai_discovery_all_found gate only handles the all-or-nothing case. If three authoritative checks succeed and one fails, the scanner's blanket “no AI discovery signals” result can still be repeated for the three verified endpoints in the discussion. Evaluate each entry in checks so partial availability does not recreate the false negatives this change is intended to remove.
verification reports `summary.llms_txt_found: true`, do not report llms.txt as missing. When it
reports `summary.ai_discovery_all_found: true`, do not report AI discovery files as missing.

.github/workflows/daily-geo-optimizer.md:373

  • In a partial result, “found the file or files” can discard the entire recommendation after any named endpoint is found even when another named endpoint genuinely failed verification. Filter the recommendation per endpoint, and select the next recommendation only when every missing-file claim was disproved.
If the audit's highest-impact recommendation says documentation-site robots.txt, llms.txt, or AI
discovery files are missing, first compare it against the authoritative verification files above.
If the project-site verification found the file or files, treat that recommendation as a scanner
false negative and select the next actionable recommendation instead.
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment on lines +109 to +120
local curl_metadata
local http_status
local content_type
local found=false

rm -f "$body_path"
# runner-guard:ignore RGS-012 -- unauthenticated GET from the public documentation site; no secrets are sent.
curl_metadata="$(curl --silent --show-error --location --max-time 30 \
--output "$body_path" --write-out '%{http_code}\t%{content_type}' \
"$url" || true)"
IFS=$'\t' read -r http_status content_type <<< "$curl_metadata"
if [[ "$http_status" == "200" ]]; then
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.87.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[deep-report] GEO Optimizer scanner reports false 0-scores for llms.txt/AI-discovery files that are live and working

3 participants