Halve the job sitemap chunk to restore the timeout margin

[strelov1]

Measured on prod after #1990 shipped, and noted at the time as a follow-up.

25k page, warm                ~2.5 s
25k page, deepest offset,
  host under load               8 s     ← route's fetch timeout is 10 s

A file that renders fine until the box is busy, and then 500s. That is the same shape of latent failure /sitemap.xml itself was in before #1990: it answered 200 from a stale nginx cache while every uncached request errored, so nothing looked wrong from outside.

10k puts a page at ~1-3 s. The cost is 127 sub-sitemaps instead of 51 — a sitemap index carries that for free, its own cap being 50,000 entries.

Cheap files beat a narrow deadline here because there is no partial credit: the crawler either gets a file or gets an error. Being slightly too slow does not deliver the URLs late, it loses 25,000 of them.

Both constants move together — the backend's jobSitemapChunk and the SPA's JOB_SITEMAP_CHUNK must match or the offsets stop lining up with the files they open.

[coderabbitai]

Warning

Review limit reached

@strelov1, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 19 minutes

Limit details: You’ve used all 2 included reviews currently available under your plan.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a31b7e2c-99ab-4cb4-80c8-dbb176cd78cd

📥 Commits

Reviewing files that changed from the base of the PR and between a58f9fc and d996c0d.

📒 Files selected for processing (2)

• internal/handler/sitemap.go
• web/src/lib/sitemap.ts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}