Add pr-build-status skill for querying Azure DevOps builds by PureWeen · Pull Request #33325 · dotnet/maui

PureWeen · 2025-12-29T19:07:41Z

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description

Adds a new GitHub Copilot CLI skill for querying Azure DevOps build information for PRs.

What it does

This skill allows Copilot CLI to:

Get Azure DevOps build IDs associated with a GitHub PR
Query build status, stages, and failed jobs
Retrieve detailed UI test results from build logs

Scripts included

Get-PrBuildIds.ps1 - Get build IDs for a PR
Get-BuildInfo.ps1 - Get detailed build status
Get-TestResults.ps1 - Get test results from build logs

Security

All Azure DevOps API calls are to public endpoints (dnceng-public org)
No authentication tokens or secrets in the code
GitHub CLI (gh) uses the user's existing authentication

Example

can you give me specifics about why iOS UITests Mono is failing?

Copilot

Pull request overview

This PR adds a new GitHub Copilot CLI skill that enables querying Azure DevOps build information for pull requests. The skill provides scripts to retrieve build IDs, detailed build status, and test results from Azure DevOps pipelines.

Key changes:

Adds three PowerShell scripts for querying Azure DevOps build data via public APIs
Provides a SKILL.md documentation file describing usage and workflows
Implements test result parsing from build logs with error message extraction

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.

File	Description
.github/skills/pr-build-status/SKILL.md	Documents the skill's purpose, prerequisites, available scripts, and recommended workflow for querying PR build status
.github/skills/pr-build-status/Get-TestResults.ps1	Retrieves and parses UI test results from Azure DevOps build logs, including failed test details with error messages and stack traces
.github/skills/pr-build-status/Get-PrBuildIds.ps1	Extracts Azure DevOps build IDs from GitHub PR checks using the GitHub CLI
.github/skills/pr-build-status/Get-BuildInfo.ps1	Queries Azure DevOps build timeline API to retrieve build status, stages, and failed jobs

Copilot · 2025-12-31T15:02:31Z

+# Filter to Azure DevOps checks and extract build IDs
+$builds = $checks | Where-Object { $_.link -match "dev\.azure\.com" } | ForEach-Object {
+    $buildId = if ($_.link -match "buildId=(\d+)") { $matches[1] } else { $null }
+    $pipeline = ($_.name -split " ")[0]


The regex pattern extracts only the first part of the pipeline name before a space. This could lead to loss of information if pipeline names contain important details after the first word. Consider whether this truncation is intentional, or if the full pipeline name should be preserved for clarity.

Suggested change

$pipeline = ($_.name -split " ")[0]

$pipeline = $_.name.Trim()

kubaflo · 2026-01-01T23:48:41Z

@PureWeen can scripts be moved to the scripts folder?😅

.github/skills/
└── skill-name/
    ├── SKILL.md          # Required - skill definition
    ├── scripts/          # Optional - executable scripts (self-contained)
    ├── assets/           # Optional - templates/resources
    └── references/       # Optional - documentation

kubaflo · 2026-01-01T23:52:44Z

+---
+name: pr-build-status
+description: "Retrieve Azure DevOps build information for GitHub Pull Requests, including build IDs, stage status, and failed jobs."
+---


To keep it consistent with our skills:

metadata: author: dotnet-maui version: "1.0" compatibility: Requires GitHub CLI (gh) authenticated with access to dotnet/maui repository.

PureWeen · 2026-01-02T17:15:23Z

/azp run

azure-pipelines · 2026-01-02T17:15:30Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

- Add metadata (author, version, compatibility) to SKILL.md - Fix bash/pwsh documentation inconsistency - Move scripts to scripts/ subfolder for consistency - Fix PowerShell variable escaping (${BuildId} syntax) - Fix deduplication sort in Get-PrBuildIds.ps1 - Rename Get-TestResults.ps1 to Get-BuildErrors.ps1 - Rewrite script to detect both build errors and test failures - Add -ErrorsOnly and -TestsOnly switches - Search failed Tasks (not just Jobs) for MSBuild errors

Copy pasted from dotnet/maui#33325

* Add a SKILL.md for getting build results from AzDO Copy pasted from dotnet/maui#33325 * Fix Get-BuildErrors script

…939+ tests passing) (#18950) * Initial plan * Update package versions and central props for xUnit3 migration Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Remove UnitTestType and IsTestProject properties, update xunit.runner.json files Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Update xunit.runner.json files to xUnit3 schema (backward compatible) Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Add test infrastructure documentation to TESTGUIDE.md Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Add nuget.org source, update to xUnit3 packages, begin API migration Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Document xUnit3 API migration requirements and create detailed guide Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Investigate xUnit3 API availability - DataAttribute confirmed present Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Add CaptureTrace attribute and remove console capturing logic Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Add explicit xunit.v3 DLL references, investigate DataAttribute issue Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Document DataAttribute accessibility blocker with F# compiler Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Disable DataAttribute classes, document remaining xUnit3 API changes Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Disable XUNIT_EXTRAS and custom framework - FSharp.Test.Utilities builds successfully Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Update migration status - FSharp.Test.Utilities complete, document next steps Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Remove FsCheck from central Directory.Build.props Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Revert NuGet.config to original state - remove nuget.org source Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Update all test projects to OutputType=Exe for xUnit3 requirement Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Document F# compiler DataAttribute resolution blocker - 7 workaround attempts failed Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Fix DataAttribute resolution - use Xunit.v3.IDataAttribute interface Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Fix package versions and AllowMultiple attribute - test projects build successfully Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Fix FsCheck version and update test logger for xUnit3 - build succeeds Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Final update: xUnit3 migration complete with 95/101 tests passing Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Fix console output capture for FSI tests - remove CaptureTrace, auto-install TestConsole Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Update vsintegration test projects: remove UnitTestType, add OutputType=Exe Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Fix package reference: change XUnit to xunit.v3 in VisualFSharp.Salsa.fsproj Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Fix xUnit3 build issues: OutputType, ValueTask net472 compatibility, Program.fs Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Add Program.cs entry point for FSharp.Editor.IntegrationTests xUnit3 requirement Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Addressing PR comments Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Fix build errors: remove duplicate Program.cs, correct F# entry points, add net10 runtime Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Update xUnit3 migration docs to reflect verified complete status with 5,939 tests passing Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> * Fix CI failures: TestConsole.ProvideInput install() and remove Program.fs from LanguageServer.Tests Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> * Remove accidentally committed preprocessed.xml temp file Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> * Fix EndToEndBuildTests: add explicit xUnit3 package versions Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> * Fix EndToEndBuildTests: add xUnit3 version properties to Directory.Build.props Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Add timeout to flaky MailboxProcessor race condition tests to prevent test host crash Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> * Fix MailboxProcessor test crashes: restore TestConsole initialization that was lost when custom framework was disabled Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> * Fix XunitSetup.fs build error: remove TestFramework.log calls that reference undefined functions Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> * Change backgroundTask to async * Restore xunit test logger with LogFilePath in Build.ps1 Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> * Use .trx for test results * Escape semicolon in bash script * Fix LogFilePath->LogFileName * Fix logger argument formatting in build script * Temporary remove trx logger to check CI Removed trx logger from test command arguments. * Remove fake failing CI test case Removed a failing test meant to verify test suite failure. * Revert "Temporary remove trx logger to check CI" This reverts commit b0c9a3e. * Build arg list with an array This should avoid an issue with parsing the arg list later on, that is somehow triggered by two semicolons in the string * Update md files * Bring back IsTestProject * Fix test results path * Finish migration to MTP * Remove VSTest references * Remove duplicate entry * Change to xunit query syntax * Migrate tests except FSharp.Editor.IntegrationTests * Remove xunit.runner from FSharp.Editor * Remove xunit.runner from vsintegration * Make component tests 64-bit Before xunitv3 this was decided by the runner, now it depends on the dll. There are some tests in that suite that depend on the size of certain pointers. * Make test projects 64-bit After moving to xunit v3 they are executables now and they were defaulting to 32-bit, while before the 'bitness' was decided by the test runner. Many tests were running out of memory on 32-bit regime. * Prevent completion tests from hanging Scripting completion tests were hanging waiting on input * Remove batching from CI Temporarily remove batching. Want to see current status of migration on CI. Batching might need to be implemented in an entirely different way with xunit v3 anyway. * Fix forcing 64bit * Fix forcing 64bit v2 * Ignore warning These unit tests don't depend on types from fsc, they just need fsc.exe, platform differences shouldn't matter * Ensure legacy project tests are not parallel * Remove test utilities from eng/build.sh * Fix legacy project tests * Disable parallel execution of fsi tests * Add a SKILL.md for getting build results from AzDO Copy pasted from dotnet/maui#33325 * Fix Get-BuildErrors script * Remove VSTest references * Restore culture setting after test * Disable parallel execution * Add testconfig.json * Fix testconfig.json naming for MTP MTP requires the config file to be named [AssemblyName].testconfig.json in the output directory. Use MSBuild Link metadata to rename the file during copy to output directory. This should fix the 26 FSI stdin test failures on desktop (net472) that were failing in CI because MTP was not picking up the parallelization settings from the incorrectly named testconfig.json file. * Try synchronous write * Make even more synchronous * Add temporary logging * Fix temporary logging * One last try with logging * Log in exception message * Fix with ManualResetEvent sync * Undo failed experiments * Wait for async workloads * Log fsi stdin writing * Try readline- * More logging * Skip passing tests temporarily * Skip passing tests temporarily v3 * Revert skipping tests * Temporarily remove some tests * Remove VSTest IsTestProject * Remove more tests * Revert "Remove more tests" This reverts commit 8a0e230. * Remove 3 CI legs temporarily * Skip some tests * Temp trim CI * Alloc console for net472 fsi tests * Remove doc file with mostly hallucinations * Strip UTF BOM in net472 in tests * Update EndToEnd to MTP syntax * Fix encoding? * Disable implicit namespace * Rename test modules * Fix encoding! * Fix fsi option syntax * Bring back skipped tests * Undo unnecessary codepage setting * Undo readline- change * IsTestProject is back * Revert "IsTestProject is back" This reverts commit 0f66268. * Bring back projects to FSharp.sln * Check if AllocConsole is required * Remove unused open * Move to mtp-2, bump xunit packages * Bump hangdump package * Bump missed packages * Switch back to xml test results * Change report-xunit option passed * Use XunitXml logger * Tweak command line args * Rename result files, check CI * Try trx again * Don't use --report-xunit-filename for .sln files * Update filter syntax * Remove unused? package * Add serialization for CompilerHelper * Make Salsa back into a lib project * Remove redundant install call * Remove diagnostic logging from FSI stdin test infrastructure Remove all diagLog definitions and calls from scriptlib.fsx and TestFramework.fs. Remove fsi_stdin_diag.log file reading from single-test.fs and tests.fs. Remove the duplicated ensureConsole() from scriptlib.fsx (keeping the copy in TestFramework.fs). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add DataAttributeBase * Delete MTP_MIGRATION_PLAN.md * Delete XUNIT3_API_MIGRATION_GUIDE.md * Delete xunit3-fsi-stdin-debugging-summary.md --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> Co-authored-by: Tomas Grosup <Tomas.Grosup@gmail.com> Co-authored-by: abonie <20281641+abonie@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add pr-build-status skill for querying Azure DevOps builds

7784919

PureWeen marked this pull request as ready for review December 31, 2025 14:58

Copilot AI review requested due to automatic review settings December 31, 2025 14:58

PureWeen added this to the .NET 10.0 SR3 milestone Dec 31, 2025

PureWeen added this to MAUI SDK Ongoing Dec 31, 2025

github-project-automation Bot moved this to Todo in MAUI SDK Ongoing Dec 31, 2025

PureWeen assigned jfversluis and rmarinho Dec 31, 2025

Copilot started reviewing on behalf of PureWeen December 31, 2025 14:59 View session

Copilot AI reviewed Dec 31, 2025

View reviewed changes

kubaflo reviewed Jan 1, 2026

View reviewed changes

rmarinho approved these changes Jan 6, 2026

View reviewed changes

github-project-automation Bot moved this from Todo to Approved in MAUI SDK Ongoing Jan 6, 2026

rmarinho merged commit 1789d47 into main Jan 6, 2026
2 of 3 checks passed

rmarinho deleted the pr-build-status-skill branch January 6, 2026 10:39

github-project-automation Bot moved this from Approved to Done in MAUI SDK Ongoing Jan 6, 2026

abonie added a commit to abonie/fsharp that referenced this pull request Jan 20, 2026

Add a SKILL.md for getting build results from AzDO

238af91

Copy pasted from dotnet/maui#33325

abonie mentioned this pull request Jan 20, 2026

Add a SKILL.md for getting build results from AzDO dotnet/fsharp#19228

Merged

abonie added a commit to dotnet/fsharp that referenced this pull request Jan 21, 2026

Add a SKILL.md for getting build results from AzDO

eced836

Copy pasted from dotnet/maui#33325

T-Gro pushed a commit to dotnet/fsharp that referenced this pull request Jan 22, 2026

Add a SKILL.md for getting build results from AzDO (#19228)

def2b82

* Add a SKILL.md for getting build results from AzDO Copy pasted from dotnet/maui#33325 * Fix Get-BuildErrors script

kubaflo added the area-ai-agents Copilot CLI agents, agent skills, AI-assisted development label Jan 28, 2026

kubaflo added the copilot label Feb 6, 2026

github-actions Bot locked and limited conversation to collaborators Mar 9, 2026

Navigation Menu

Uh oh!

Add pr-build-status skill for querying Azure DevOps builds - #33325

Add pr-build-status skill for querying Azure DevOps builds#33325
rmarinho merged 2 commits into
mainfrom
pr-build-status-skill

PureWeen commented Dec 29, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kubaflo commented Jan 1, 2026

Uh oh!

kubaflo Jan 1, 2026 •

edited

Loading

Uh oh!

PureWeen commented Jan 2, 2026

Uh oh!

azure-pipelines Bot commented Jan 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	$pipeline = ($_.name -split " ")[0]
	$pipeline = $_.name.Trim()

Uh oh!

Conversation

PureWeen commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What it does

Scripts included

Security

Example

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kubaflo commented Jan 1, 2026

Uh oh!

kubaflo Jan 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PureWeen commented Jan 2, 2026

Uh oh!

azure-pipelines Bot commented Jan 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

PureWeen commented Dec 29, 2025 •

edited

Loading

kubaflo Jan 1, 2026 •

edited

Loading