Skip to content

refactor: improve azd sample selection guidance and agent creation workflow in foundry skill - #3056

Merged
anchenyi merged 15 commits into
mainfrom
anchenyi/improve-skill
Aug 12, 2026
Merged

refactor: improve azd sample selection guidance and agent creation workflow in foundry skill#3056
anchenyi merged 15 commits into
mainfrom
anchenyi/improve-skill

Conversation

@anchenyi

@anchenyi anchenyi commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Description

  • Added shared "azd sample selection guidance" for both new agent creation and existing agent development.
  • Use the complete azd ai agent sample list catalog instead of limiting selection to featured samples.
  • Ensure project-scoped azd commands run from the generated project directory.
  • Fix the nested project path issue in the quick-start-hosted workflow.
  • Skip asking users and directly submit evaluation-suite generation and include its status in the final summary.

Checklist

  • Tests pass locally (cd tests && npm test)
  • Title has one of the prefixes: fix:, feat:, feature:, chore:, misc:, test:, eval:
  • If modifying skill descriptions: verified routing correctness with integration tests (In tests/, npm run test:integration -- <skill> or npm run test:vally -- --skill <skill>)

Related Issues

@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor
Details# 🔍 Token Analysis Report

@github-copilot-for-azure/scripts@1.0.0 tokens
node --import tsx src/tokens/cli.ts compare --base origin/main --head HEAD --markdown

📊 Token Change Report

Comparing origin/mainHEAD

Summary

Metric Value
📈 Total Change +419 tokens (+2%)
Before 17,843 tokens
After 18,262 tokens
Files Changed 4

Changed Files

File Before After Change
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/quick-start-hosted.md 5,028 5,260 +232 (+5%)
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/create-hosted.md 7,514 7,661 +147 (+2%)
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/azd-guidance/azd-guidance.md 285 308 +23 (+8%)
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/deploy/deploy.md 5,016 5,033 +17 (0%)

@github-copilot-for-azure/scripts@1.0.0 tokens
node --import tsx src/tokens/cli.ts check --markdown

📊 Token Limit Check Report

Checked: 846 files
Exceeded: 116 files

⚠️ Files Exceeding Token Limits

File Tokens Limit Over By
.github/skills/analyze-skill-issues/SKILL.md 2109 500 +1609
.github/skills/analyze-test-run/SKILL.md 2471 500 +1971
.github/skills/file-test-bug/SKILL.md 628 500 +128
.github/skills/sensei/SKILL.md 861 500 +361
.github/skills/skill-authoring/SKILL.md 839 500 +339
.github/skills/vally-eval/SKILL.md 1737 500 +1237
plugins/azure-kusto-graph-skills/skills/azure-kusto-graph/SKILL.md 4810 500 +4310
plugins/azure-kusto-graph-skills/skills/azure-kusto-irql/SKILL.md 2625 500 +2125
plugins/azure-kusto-graph-skills/skills/azure-kusto-irql-graph/SKILL.md 4781 500 +4281
plugins/azure-kusto-graph-skills/skills/azure-kusto-irql-graph/references/DEPLOY_IRQL_FUNCTIONS.md 2854 2000 +854
plugins/azure-skills/changelog-2026-07-24.md 4226 2000 +2226
plugins/azure-skills/skills/airunway-aks-setup/SKILL.md 1025 500 +525
plugins/azure-skills/skills/appinsights-instrumentation/SKILL.md 937 500 +437
plugins/azure-skills/skills/azure-ai/SKILL.md 820 500 +320
plugins/azure-skills/skills/azure-aigateway/SKILL.md 1261 500 +761
plugins/azure-skills/skills/azure-aigateway/references/policies.md 2342 2000 +342
plugins/azure-skills/skills/azure-app-onboard/SKILL.md 1623 500 +1123
plugins/azure-skills/skills/azure-app-onboard/deploy/SKILL.md 1816 500 +1316
plugins/azure-skills/skills/azure-app-onboard/deploy/references/code-deployment-container-apps.md 2140 2000 +140
plugins/azure-skills/skills/azure-app-onboard/deploy/references/deploy-checklist-template.md 2564 2000 +564
plugins/azure-skills/skills/azure-app-onboard/prepare/SKILL.md 1566 500 +1066
plugins/azure-skills/skills/azure-app-onboard/prepare/references/pricing-guide-services.md 2142 2000 +142
plugins/azure-skills/skills/azure-app-onboard/prepare/references/service-mapping.md 2022 2000 +22
plugins/azure-skills/skills/azure-app-onboard/prepare/references/sku-quota-validation.md 2278 2000 +278
plugins/azure-skills/skills/azure-app-onboard/references/session-protocol.md 2026 2000 +26
plugins/azure-skills/skills/azure-app-onboard/scaffold/SKILL.md 3592 500 +3092
plugins/azure-skills/skills/azure-app-onboard/scaffold/references/bicep-container-apps.md 2354 2000 +354
plugins/azure-skills/skills/azure-app-onboard/scaffold/references/self-review-checklist.md 2269 2000 +269
plugins/azure-skills/skills/azure-app-onboard/scaffold/references/subagent-iac-gen.md 2406 2000 +406
plugins/azure-skills/skills/azure-app-onboard-prereq/SKILL.md 2517 500 +2017
plugins/azure-skills/skills/azure-app-onboard-prereq/references/dependency-compatibility.md 2239 2000 +239
plugins/azure-skills/skills/azure-cloud-migrate/SKILL.md 1085 500 +585
plugins/azure-skills/skills/azure-cloud-migrate/references/services/container-apps/cloudrun-deployment-guide.md 2029 2000 +29
plugins/azure-skills/skills/azure-cloud-migrate/references/services/container-apps/deployment-guide.md 2458 2000 +458
plugins/azure-skills/skills/azure-cloud-migrate/references/services/container-apps/fargate-deployment-guide.md 2587 2000 +587
plugins/azure-skills/skills/azure-cloud-migrate/references/services/container-apps/spring-deployment-guide.md 3871 2000 +1871
plugins/azure-skills/skills/azure-cloud-migrate/references/services/functions/lambda-to-functions.md 2600 2000 +600
plugins/azure-skills/skills/azure-cloud-migrate/references/services/functions/runtimes/javascript.md 2181 2000 +181
plugins/azure-skills/skills/azure-compliance/SKILL.md 1188 500 +688
plugins/azure-skills/skills/azure-compute/SKILL.md 657 500 +157
plugins/azure-skills/skills/azure-compute/workflows/essential-machine-management/references/emm-enable-flow.md 2344 2000 +344
plugins/azure-skills/skills/azure-deploy/SKILL.md 1645 500 +1145
plugins/azure-skills/skills/azure-deploy/references/pre-deploy-checklist.md 4692 2000 +2692
plugins/azure-skills/skills/azure-deploy/references/recipes/azd/errors.md 4004 2000 +2004
plugins/azure-skills/skills/azure-deploy/references/troubleshooting.md 2038 2000 +38
plugins/azure-skills/skills/azure-diagnostics/SKILL.md 1596 500 +1096
plugins/azure-skills/skills/azure-enterprise-infra-planner/SKILL.md 911 500 +411
plugins/azure-skills/skills/azure-enterprise-infra-planner/references/constraints/compute-apps.md 2022 2000 +22
plugins/azure-skills/skills/azure-kubernetes/SKILL.md 2723 500 +2223
plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-automatic-readiness/SKILL.md 3690 500 +3190
plugins/azure-skills/skills/azure-kusto/SKILL.md 2152 500 +1652
plugins/azure-skills/skills/azure-messaging/SKILL.md 821 500 +321
plugins/azure-skills/skills/azure-prepare/SKILL.md 3145 500 +2645
plugins/azure-skills/skills/azure-prepare/references/aspire.md 4617 2000 +2617
plugins/azure-skills/skills/azure-prepare/references/plan-template.md 2560 2000 +560
plugins/azure-skills/skills/azure-prepare/references/recipes/azd/aspire.md 2275 2000 +275
plugins/azure-skills/skills/azure-prepare/references/recipes/azd/terraform.md 3555 2000 +1555
plugins/azure-skills/skills/azure-prepare/references/research.md 2196 2000 +196
plugins/azure-skills/skills/azure-prepare/references/resources-limits-quotas.md 3322 2000 +1322
plugins/azure-skills/skills/azure-prepare/references/security.md 2147 2000 +147
plugins/azure-skills/skills/azure-prepare/references/services/functions/bicep.md 3043 2000 +1043
plugins/azure-skills/skills/azure-prepare/references/services/functions/templates/recipes/composition.md 2813 2000 +813
plugins/azure-skills/skills/azure-prepare/references/services/functions/terraform.md 3404 2000 +1404
plugins/azure-skills/skills/azure-quotas/SKILL.md 3006 500 +2506
plugins/azure-skills/skills/azure-quotas/references/commands.md 2644 2000 +644
plugins/azure-skills/skills/azure-reliability/SKILL.md 5922 500 +5422
plugins/azure-skills/skills/azure-reliability/references/configure-multi-region.md 4729 2000 +2729
plugins/azure-skills/skills/azure-reliability/references/services/app-service/reliability.md 2591 2000 +591
plugins/azure-skills/skills/azure-resource-lookup/SKILL.md 1367 500 +867
plugins/azure-skills/skills/azure-resource-visualizer/SKILL.md 2122 500 +1622
plugins/azure-skills/skills/azure-storage/SKILL.md 1228 500 +728
plugins/azure-skills/skills/azure-upgrade/SKILL.md 1542 500 +1042
plugins/azure-skills/skills/azure-upgrade/references/languages/java/INSTRUCTION.md 2893 2000 +893
plugins/azure-skills/skills/azure-upgrade/references/languages/java/package-specific/com.microsoft.azure.management.md 2428 2000 +428
plugins/azure-skills/skills/azure-upgrade/references/languages/java/templates/PLAN_TEMPLATE.md 2411 2000 +411
plugins/azure-skills/skills/azure-upgrade/references/languages/java/templates/PROGRESS_TEMPLATE.md 2315 2000 +315
plugins/azure-skills/skills/azure-upgrade/references/languages/java/templates/SUMMARY_TEMPLATE.md 2190 2000 +190
plugins/azure-skills/skills/azure-upgrade/references/services/functions/automation.md 3463 2000 +1463
plugins/azure-skills/skills/azure-upgrade/references/services/functions/consumption-to-flex.md 2773 2000 +773
plugins/azure-skills/skills/azure-validate/SKILL.md 897 500 +397
plugins/azure-skills/skills/entra-agent-id/SKILL.md 3994 500 +3494
plugins/azure-skills/skills/entra-app-registration/SKILL.md 2058 500 +1558
plugins/azure-skills/skills/entra-app-registration/references/api-permissions.md 2545 2000 +545
plugins/azure-skills/skills/entra-app-registration/references/cli-commands.md 2211 2000 +211
plugins/azure-skills/skills/entra-app-registration/references/console-app-example.md 2752 2000 +752
plugins/azure-skills/skills/entra-app-registration/references/oauth-flows.md 2375 2000 +375
plugins/azure-skills/skills/microsoft-foundry/SKILL.md 6501 500 +6001
plugins/azure-skills/skills/microsoft-foundry/finetuning/SKILL.md 1376 500 +876
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/azd-guidance/references/azd-ai-cli.md 2230 2000 +230
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/create-hosted.md 7661 2000 +5661
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/quick-start-hosted.md 5260 2000 +3260
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/references/foundry-tool-catalog.md 10922 2000 +8922
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/references/local-run.md 2443 2000 +443
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/references/use-toolbox-in-hosted-agent.md 2643 2000 +643
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/deploy/deploy.md 5033 2000 +3033
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md 2863 2000 +863
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/eval-datasets/references/generate-seed-dataset.md 2212 2000 +212
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md 4325 2000 +2325
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/invocations-ws/invocations-ws.md 2652 2000 +652
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/observe/observe.md 3856 2000 +1856
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/observe/references/continuous-eval.md 3855 2000 +1855
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md 2175 2000 +175
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/observe/references/evaluation-suite-generation.md 3134 2000 +1134
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/routine/routine.md 2032 2000 +32
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md 2701 2000 +701
plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/SKILL.md 1798 500 +1298
plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/capacity/SKILL.md 1739 500 +1239
plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/customize/SKILL.md 2236 500 +1736
plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/customize/references/customize-workflow.md 3336 2000 +1336
plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/preset/SKILL.md 1227 500 +727
plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/preset/references/preset-workflow.md 5539 2000 +3539
plugins/azure-skills/skills/microsoft-foundry/project/create/create-foundry-project.md 2287 2000 +287
plugins/azure-skills/skills/microsoft-foundry/quota/quota.md 2289 2000 +289
plugins/azure-skills/skills/microsoft-foundry/quota/references/capacity-planning.md 2081 2000 +81
plugins/azure-skills/skills/microsoft-foundry/references/agent-metadata-contract.md 2217 2000 +217
plugins/azure-skills/skills/python-appservice-deploy/SKILL.md 688 500 +188

Consider moving content to references/ subdirectories.


Automated token analysis. See skill authoring guidelines for best practices.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the microsoft-foundry hosted-agent creation guidance to improve sample selection (using the full azd ai agent sample list catalog), clarify where azd commands should be run, and streamline the hosted quickstart workflow (including eval-suite generation submission).

Changes:

  • Switch sample selection guidance from --featured-only to the full azd ai agent sample list catalog and centralize selection guidance in create-hosted.md.
  • Update hosted quickstart flow to remove the manual project directory creation step and adjust step numbering/references accordingly.
  • Add shared azd guidance to run project-scoped commands from the project folder.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/quick-start-hosted.md Updates the hosted quickstart flow, sample selection, and eval generation step sequencing.
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/create-hosted.md Adds shared sample selection guidance and updates references across new-agent and existing-agent workflows.
plugins/azure-skills/skills/microsoft-foundry/foundry-agent/azd-guidance/azd-guidance.md Adds a rule to run project-scoped azd commands inside the project folder.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/quick-start-hosted.md:300

  • The PR description says to include the eval-suite generation status in the final summary, but Step 14 currently only asks for “the eval suite generation submission” and doesn’t instruct including the queued/in_progress status (and IDs) shown in Step 13’s expected output. This makes the workflow guidance fall short of the stated intent.
Produce a concise summary covering: agent name/version/status/endpoints, a Playground link, the resources created, the eval suite generation submission, and the three follow-up commands below. Read `playground_url` directly from `azd ai agent show --output json`. If it is absent, construct the Playground URL from `azd env get-values`:

@anchenyi
anchenyi merged commit e34ca55 into main Aug 12, 2026
12 of 13 checks passed
@anchenyi
anchenyi deleted the anchenyi/improve-skill branch August 12, 2026 04:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants