[tests] Tweak the AppSize test a bit. #24306

rolfbjarne · 2025-11-20T14:04:11Z

Fail if a file was added to or removed from the app bundle.
Fail if the set of APIs that survived trimming changed.
Use Assert.Multiple to support multiple failing asserts.
Improve diagnostic output a bit.

* Fail if a file was added to or removed from the app bundle. * Fail if the set of APIs that survived trimming changed. * Use Assert.Multiple to support multiple failing asserts. * Improve diagnostic output a bit.

Copilot

Pull Request Overview

This PR enhances the AppSize test to be more strict and provide better diagnostics. The test now fails explicitly when files are added/removed from the app bundle or when APIs change in the preserved set, rather than only checking total size differences. The changes also improve test output using Assert.Multiple to report multiple failures at once.

Refactored test logic into separate AssertAppSize and AssertAssemblyReport methods
Added explicit failure assertions for file additions/removals and API changes
Improved diagnostic messages to guide users on how to update baselines

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
tests/dotnet/UnitTests/AppSizeTest.cs	Refactored test to use Assert.Multiple, split into separate assertion methods, added explicit failures for file and API changes, improved diagnostic messages
tests/dotnet/UnitTests/expected/iOS-MonoVM-preservedapis.txt	Updated baseline with newly preserved API `GetNSObject(System.IntPtr)`
tests/dotnet/UnitTests/expected/iOS-MonoVM-interpreter-preservedapis.txt	Updated baseline with newly preserved API `GetNSObject(System.IntPtr)`

Copilot · 2025-11-20T14:14:36Z

tests/dotnet/UnitTests/AppSizeTest.cs

-					return;
+			var withinTolerance = toleranceInBytes >= Math.Abs (appSizeDifference);
+			if (!withinTolerance) {
+				Console.WriteLine ($"App size difference is {FormatBytes (appSizeDifference)}, which is less than the tolerance ({toleranceInBytes})");


The console message has incorrect logic. When !withinTolerance is true, it means the difference is NOT within tolerance (i.e., it's greater than the tolerance), but the message says "which is less than the tolerance". This should say "which is greater than or equal to the tolerance" or similar.

Suggested change

Console.WriteLine ($"App size difference is {FormatBytes (appSizeDifference)}, which is less than the tolerance ({toleranceInBytes})");

Console.WriteLine ($"App size difference is {FormatBytes (appSizeDifference)}, which is greater than or equal to the tolerance ({toleranceInBytes})");

tests/dotnet/UnitTests/AppSizeTest.cs

…e-tweaks

vs-mobiletools-engineering-service2 · 2025-11-21T08:20:14Z

✅ [CI Build #530689f] Build passed (Build packages) ✅

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T08:34:03Z

✅ [PR Build #530689f] Build passed (Detect API changes) ✅

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T08:57:48Z

✅ [CI Build #530689f] Build passed (Build macOS tests) ✅

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T09:07:48Z

💻 [CI Build #530689f] Tests on macOS X64 - Mac Sonoma (14) passed 💻

✅ All tests on macOS X64 - Mac Sonoma (14) passed.

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T09:13:17Z

✅ API diff for current PR / commit

NET (empty diffs)

iOS: vsdrops gist ( No breaking changes )
tvOS: vsdrops gist ( No breaking changes )
MacCatalyst: vsdrops gist ( No breaking changes )
macOS: vsdrops gist ( No breaking changes )

✅ API diff vs stable

NET (empty diffs)

iOS: vsdrops gist ( No breaking changes )
tvOS: vsdrops gist ( No breaking changes )
MacCatalyst: vsdrops gist ( No breaking changes )
macOS: vsdrops gist ( No breaking changes )

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T09:17:51Z

💻 [CI Build #530689f] Tests on macOS M1 - Mac Monterey (12) passed 💻

✅ All tests on macOS M1 - Mac Monterey (12) passed.

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T09:20:50Z

💻 [CI Build #530689f] Tests on macOS arm64 - Mac Sequoia (15) passed 💻

✅ All tests on macOS arm64 - Mac Sequoia (15) passed.

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T09:21:36Z

💻 [CI Build #530689f] Tests on macOS arm64 - Mac Tahoe (26) passed 💻

✅ All tests on macOS arm64 - Mac Tahoe (26) passed.

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T09:21:44Z

💻 [CI Build #530689f] Tests on macOS M1 - Mac Ventura (13) passed 💻

✅ All tests on macOS M1 - Mac Ventura (13) passed.

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

vs-mobiletools-engineering-service2 · 2025-11-21T10:14:46Z

🚀 [CI Build #530689f] Test results 🚀

Test results

✅ All tests passed on VSTS: test results.

🎉 All 117 tests passed 🎉

Tests counts

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 5 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 4 tests passed. Html Report (VSDrops) Download
✅ linker: All 44 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 9 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 11 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 9 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 9 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ windows: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: 530689fbfa850d97db28c04e541d3f58050b3e0d [PR build]

[tests] Tweak the AppSize test a bit.

d949d23

* Fail if a file was added to or removed from the app bundle. * Fail if the set of APIs that survived trimming changed. * Use Assert.Multiple to support multiple failing asserts. * Improve diagnostic output a bit.

rolfbjarne enabled auto-merge (squash) November 20, 2025 14:04

rolfbjarne requested a review from Copilot November 20, 2025 14:04

Copilot started reviewing on behalf of rolfbjarne November 20, 2025 14:04 View session

Copilot finished reviewing on behalf of rolfbjarne November 20, 2025 14:06

dalexsoto approved these changes Nov 20, 2025

View reviewed changes

Copilot AI reviewed Nov 20, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

rolfbjarne added 2 commits November 21, 2025 08:49

Simplify and tweak

bec3da8

Merge remote-tracking branch 'origin/main' into dev/rolf/tests-appsiz…

530689f

…e-tweaks

rolfbjarne mentioned this pull request Nov 21, 2025

[tests] Update expected app size in size test. #24286

Closed

rolfbjarne merged commit db2ec8e into main Nov 21, 2025
45 checks passed

	Console.WriteLine ($"App size difference is {FormatBytes (appSizeDifference)}, which is less than the tolerance ({toleranceInBytes})");
	Console.WriteLine ($"App size difference is {FormatBytes (appSizeDifference)}, which is greater than or equal to the tolerance ({toleranceInBytes})");

[tests] Tweak the AppSize test a bit. #24306

[tests] Tweak the AppSize test a bit. #24306

Uh oh!

Conversation

rolfbjarne commented Nov 20, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Nov 21, 2025

✅ [CI Build #530689f] Build passed (Build packages) ✅

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

✅ [PR Build #530689f] Build passed (Detect API changes) ✅

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

✅ [CI Build #530689f] Build passed (Build macOS tests) ✅

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

💻 [CI Build #530689f] Tests on macOS X64 - Mac Sonoma (14) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

✅ API diff for current PR / commit

✅ API diff vs stable

ℹ️ Generator diff

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

💻 [CI Build #530689f] Tests on macOS M1 - Mac Monterey (12) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

💻 [CI Build #530689f] Tests on macOS arm64 - Mac Sequoia (15) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

💻 [CI Build #530689f] Tests on macOS arm64 - Mac Tahoe (26) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

💻 [CI Build #530689f] Tests on macOS M1 - Mac Ventura (13) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Nov 21, 2025

🚀 [CI Build #530689f] Test results 🚀

Test results

Tests counts

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants