Problem
The aicollection tag in website/static/templates.json is reserved for templates listed in the Microsoft AI collection at https://azure.github.io/ai-app-templates/. Today we enforce that rule only through manual PR review.
This came up in #780, which proposes adding 36 templates all carrying the aicollection tag. Checking each entry by hand against the AI collection site is slow and error-prone, and at scale it blocks otherwise-clean PRs from getting merged.
Proposed approach
Add a check to the existing npm test suite in website/ that:
- Fetches the canonical list of templates from https://azure.github.io/ai-app-templates/ (ideally from a JSON manifest, not by scraping HTML)
- For each entry in
templates.json carrying the aicollection tag, asserts the source repo URL (or another stable identifier) appears in the canonical list
- Fails with a clear message that names the offending templates if any
aicollection-tagged entry is missing from the canonical list
Open questions
- Does the AI collection site expose a machine-readable manifest, or should we check in a local allow-list and refresh it on a cadence?
- How should we handle network failures in CI, soft-fail with a warning or skip the check entirely?
- Should this also cover other reserved tags like
msft, popular, and featured?
Acceptance criteria
Context
Raised in the review of #780.
Problem
The
aicollectiontag inwebsite/static/templates.jsonis reserved for templates listed in the Microsoft AI collection at https://azure.github.io/ai-app-templates/. Today we enforce that rule only through manual PR review.This came up in #780, which proposes adding 36 templates all carrying the
aicollectiontag. Checking each entry by hand against the AI collection site is slow and error-prone, and at scale it blocks otherwise-clean PRs from getting merged.Proposed approach
Add a check to the existing
npm testsuite inwebsite/that:templates.jsoncarrying theaicollectiontag, asserts the source repo URL (or another stable identifier) appears in the canonical listaicollection-tagged entry is missing from the canonical listOpen questions
msft,popular, andfeatured?Acceptance criteria
npm testfails when a template carriesaicollectionwithout a match in the canonical sourcetemplates.jsonpasses the new check, or any known violations are tracked in a follow-upContext
Raised in the review of #780.