Skip to content

docs: Redesign public docs with Starlight #63

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dcramer merged 3 commits into from
May 20, 2026
Merged

docs: Redesign public docs with Starlight #63

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e5bd6fa
ref: redesign docs
dcramer May 19, 2026
e616853
take 2
dcramer May 19, 2026
19c5649
docs: Redesign public docs with Starlight
dcramer May 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ dist

# Astro generated types and cache
**/.astro/
packages/docs/src/content/docs/api/

# vitepress build output
**/.vitepress/dist
Expand Down
14 changes: 14 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,20 @@ Monorepo for the explicit-run `vitest-evals` shape:
- `apps/demo-openai-agents`: end-to-end OpenAI Agents demo evals with
app-local refund tools

## Reading Guide

- Start with the public docs site for the guided setup path:
`https://vitest-evals.sentry.dev/docs`
- Use [packages/vitest-evals/README.md](packages/vitest-evals/README.md) for
the core package authoring model and examples.
- Use [docs/github-actions.md](docs/github-actions.md) when changing CI
reporting behavior.
- Use [docs/architecture.md](docs/architecture.md) and
[docs/development-guide.md](docs/development-guide.md) before changing package
boundaries or product shape.
- Follow [policies/docs-writing.md](policies/docs-writing.md) for docs copy,
hierarchy, and visual consistency.

## Workspace Layout

```text
Expand Down
131 changes: 94 additions & 37 deletions packages/docs/astro.config.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,52 +1,109 @@
import mdx from "@astrojs/mdx";
import { defineConfig, fontProviders } from "astro/config";
import rehypeAutolinkHeadings from "rehype-autolink-headings";
import rehypeSlug from "rehype-slug";
import starlight from "@astrojs/starlight";
import { defineConfig } from "astro/config";
import starlightTypeDoc, { typeDocSidebarGroup } from "starlight-typedoc";
import {
monochromeCodeTheme,
vitestEvalsStarlightTheme,
} from "./src/theme/vitest-evals-starlight-theme.mjs";

export default defineConfig({
site: "https://vitest-evals.sentry.dev",
integrations: [mdx()],
markdown: {
shikiConfig: {
theme: "vitesse-black",
},
rehypePlugins: [
rehypeSlug,
[
rehypeAutolinkHeadings,
{
behavior: "prepend",
properties: { className: ["heading-anchor"] },
content: { type: "text", value: "#" },
},
],
],
devToolbar: {
enabled: false,
},
experimental: {
fonts: [
{
name: "Geist Mono",
provider: fontProviders.local(),
cssVariable: "--font-geist-mono",
options: {
variants: [
integrations: [
starlight({
title: "vitest-evals",
description: "Harness-backed AI testing on top of Vitest.",
pagination: false,
components: {
ThemeSelect: "./src/components/StarlightThemeSelect.astro",
},
sidebar: [
{
label: "Documentation",
items: [
{ label: "Overview", link: "/docs" },
{
weight: 400,
style: "normal",
src: [
"./node_modules/geist/dist/fonts/geist-mono/GeistMono-Regular.woff2",
label: "Harnesses",
items: [
{ label: "Overview", link: "/docs/harnesses" },
{ label: "AI SDK", link: "/docs/harnesses/ai-sdk" },
{
label: "OpenAI Agents",
link: "/docs/harnesses/openai-agents",
},
{ label: "Pi", link: "/docs/harnesses/pi-ai" },
{
label: "Custom Harnesses",
link: "/docs/harnesses/custom",
},
],
},
{
weight: 600,
style: "normal",
src: [
"./node_modules/geist/dist/fonts/geist-mono/GeistMono-SemiBold.woff2",
label: "Judges",
items: [
{ label: "Overview", link: "/docs/judges" },
{ label: "ToolCallJudge", link: "/docs/judges/tool-call" },
{
label: "StructuredOutputJudge",
link: "/docs/judges/structured-output",
},
{ label: "Custom Judges", link: "/docs/judges/custom" },
],
},
{ label: "Tool Replay", link: "/docs/tool-replay" },
{ label: "GitHub Reporting", link: "/docs/github" },
],
},
},
],
{
label: "API Reference",
items: [{ label: "Overview", link: "/api" }, typeDocSidebarGroup],
},
],
social: [
{
icon: "github",
label: "GitHub",
href: "https://github.com/getsentry/vitest-evals",
},
],
plugins: [
vitestEvalsStarlightTheme(),
starlightTypeDoc({
entryPoints: ["../vitest-evals/src/index.ts"],
tsconfig: "../vitest-evals/tsconfig.json",
output: "api",
pagination: false,
sidebar: {
label: "Exports",
},
typeDoc: {
disableSources: true,
entryPointStrategy: "resolve",
intentionallyNotExported: [
"OutputField",
"JudgeAssertionArgs",
"JudgeAssertionHarness",
"JudgeAssertionInput",
"JudgeAssertionMetadata",
"JudgeAssertionOutput",
"JudgeAssertionParams",
"JudgeForReceived",
"HarnessInput",
"HarnessMetadataFor",
"HarnessOutput",
],
},
}),
],
}),
mdx(),
Copy link
Copy Markdown

@sentry sentry Bot May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The explicit mdx() integration in astro.config.mjs is redundant because starlight() already provides MDX support, which could cause build conflicts or warnings.
Severity: LOW

Suggested Fix

Remove the explicit mdx() call from the integrations array in astro.config.mjs. Rely on the built-in MDX support provided by the starlight integration, which is sufficient for the project's needs.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: packages/docs/astro.config.mjs#L102

Potential issue: The Astro configuration file `astro.config.mjs` includes both the
`starlight()` and `mdx()` integrations. The `starlight` integration handles MDX
processing internally, making the explicit inclusion of the `mdx()` integration
redundant. This configuration follows a pattern that can lead to build-time warnings or
conflicts related to initialization order, even if modern versions of Astro might handle
it silently. Since all MDX content is managed by Starlight's content loader, the
separate `mdx()` integration is unnecessary and should be removed to follow best
practices and avoid potential issues.

Did we get this right? 👍 / 👎 to inform future reviews.

],
markdown: {
shikiConfig: {
theme: monochromeCodeTheme,
},
},
});
17 changes: 8 additions & 9 deletions packages/docs/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,20 @@
"type": "module",
"version": "0.0.1",
"scripts": {
"api:generate": "typedoc --options typedoc.json",
"dev": "pnpm run api:generate && astro dev",
"build": "pnpm run api:generate && astro build",
"dev": "astro dev",
"build": "astro build",
"preview": "astro preview"
},
"dependencies": {
"@astrojs/mdx": "^4.0.0",
"@astrojs/mdx": "^5.0.6",
"@astrojs/starlight": "^0.39.2",
"@vercel/functions": "^2.0.0",
"astro": "^5.0.0",
"geist": "^1.5.1",
"rehype-autolink-headings": "^7.1.0",
"rehype-slug": "^6.0.0",
"astro": "^6.3.5",
"starlight-typedoc": "^0.22.0",
"vitest-evals": "workspace:*"
},
"devDependencies": {
"typedoc": "^0.28.19"
"typedoc": "^0.28.19",
"typedoc-plugin-markdown": "^4.6.0"
}
}
4 changes: 4 additions & 0 deletions packages/docs/public/favicon.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading