The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2763485	dotnet-orleans	Test	Log
2763483	dotnet-performance	Send job to Helix (Windows)	Log
2763456	dotnet-runtime	Send job to Helix (Unix)	Log
1112094	dotnet/runtime	Send tests to Helix (Windows)	Log
1112093	dotnet/runtime	AOT compile CoreCLR tests	Log
2763398	dotnet-runtime	Run performance job script	Log
2763349	dotnet-runtime	Run performance job script	Log
1112022	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#118165
1112018	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#118241
1112015	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#118302
1111931	dotnet/runtime	Send tests to Helix (Unix)	Log
2763287	dotnet-performance	Send job to Helix (Windows)	Log
1111950	dotnet/roslyn	Test	Log	dotnet/roslyn#79751
1111940	dotnet/roslyn	Run Integration Tests	Log
1111938	dotnet/roslyn	Run Integration Tests	Log
1111939	dotnet/roslyn	Test	Log
2763146	dotnet-maui	Provision Android SDK - Common Packages	Log
2763118	dotnet-runtime	Send job to Helix (Windows)	Log
1111871	dotnet/roslyn	Run Integration Tests	Log
2763092	dotnet-runtime	Run performance job script	Log
1111846	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79562
1111635	dotnet/runtime	Send to Helix	Log
1111778	dotnet/roslyn	Run Integration Tests	Log
2763028	dotnet-performance	Send job to Helix (Windows)	Log
1111698	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118256
1111732	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79746
2762999	dotnet-runtime	Run performance job script	Log
1111687	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79742
2762982	dotnet-dotnet	Checkout dotnet-dotnet@non-prod-pipeline to s	Log
1111646	dotnet/roslyn	Run Integration Tests	Log
2762908	dotnet-dotnet	Build (centOSStreamContainer)	Log
2762894	dotnet-performance	Send job to Helix (Windows)	Log
1111371	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79737
2762717	dotnet-runtime	Send job to Helix (Windows)	Log
2762733	dotnet-dotnet-docker	🛡 Guardian: CodeQL - TypeScript and JavaScript	Log
1111220	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79736
1111072	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118156
1111000	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118060
1110967	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118256
2762377	dotnet-runtime	Send job to Helix (Windows)	Log
2762386	dotnet-dotnet-docker	🛡 Guardian: CodeQL - TypeScript and JavaScript	Log
2762362	dotnet-runtime	Send job to Helix (Windows)	Log
2762380	dotnet-dotnet-docker	🛡 Guardian: CodeQL - TypeScript and JavaScript	Log
2762359	dotnet-orleans	Test	Log
2762351	dotnet-orleans	Test	Log
2762336	dotnet-roslyn	Test	Log
1110663	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118235
1110786	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79578
2762292	dotnet-roslyn	Test	Log
2762285	dotnet-sdk	🟣 Run AoT Tests	Log
1110710	dotnet/runtime	Send to Helix	Log
1110732	dotnet/runtime	Send to Helix	Log	dotnet/runtime#117398
2762182	dotnet-dotnet-monitor	Build	Log
1110442	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118207
1110673	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118037
2762259	dotnet-sdk	🟣 Run AoT Tests	Log
2762239	dotnet-runtime	Send job to Helix (Unix)	Log
1110544	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118232
2762236	dotnet-roslyn	Test	Log
1110660	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118245
1110655	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118228
2762224	dotnet-roslyn	Test	Log
1110637	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118037
1110642	dotnet/runtime	Send to Helix	Log
1110631	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118180
2762187	dotnet-roslyn	Test	Log
1110602	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118246
2762183	dotnet-dotnet-monitor	Build	Log
2762167	dotnet-roslyn	Test	Log
2762132	dotnet-runtime	Send job to Helix (Unix)	Log
2762107	dotnet-orleans	Test	Log
1110573	dotnet/runtime	Send job to Helix (Unix)	Log
1110520	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#117788
1110523	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#118190
1110560	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#50000
1110533	dotnet/roslyn	Run Integration Tests	Log
2762118	dotnet-dotnet	Build (azurelinuxX64MuslCrossContainer)	Log
1110481	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#116157
2762102	dotnet-roslyn	Test	Log
1110532	dotnet/roslyn	Test	Log
1110491	dotnet/aspnetcore	Run build.cmd helix target	Log
1110489	dotnet/aspnetcore	Run build.cmd helix target	Log
1110510	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#50016
2762085	dotnet-roslyn	Test	Log
1110496	dotnet/roslyn	Test	Log
1110474	dotnet/aspnetcore	Run build.cmd helix target	Log
1110466	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118241
2762065	dotnet-sdk	🟣 Run AoT Tests	Log
1110460	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#50015
1110451	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#62903
1110452	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#62903
1110434	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118050
1110438	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#62213
1110439	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#62213
1110102	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118227
1110280	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63010
1110247	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63020
1110283	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63010
1110361	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118239
1110388	dotnet/sdk	Run AoT Tests	Log	dotnet/sdk#49843
Displaying 100 of 1419 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
16	184	1419

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez