feat: GitHub-hosted runners for benchmarks #29955

HowardBraham · 2025-01-29T02:03:23Z

Description

Major changes

benchmarks.yml - Does some important and new-to-us things like:
- ~~Self-hosted runner runs-on: gha-mm-scale-set-ubuntu-22.04-amd64-med~~
- ~~Running in the same kind of Docker container we use in CircleCI container: image: cimg/node:22.13-browsers~~
- ~~Unlike on CircleCI, does not work without installing Xvfb~~
codespaces.yml
- I originally added this workflow, and it was important to caching in the past, but with the new workflows in GHA, it's no longer needed, so I deleted it
test-circular-deps.yml
- We were hitting the workflow limit of 20, so I inlined this simple workflow, that did not need to be reusable

Prerequisites to merging this PR

✔️ Merge this benchmarks branch of github-tools: https://github.com/MetaMask/github-tools/blob/benchmarks/.github/actions/setup-environment/action.yml
✔️ Change setup-environment@benchmarks to setup-environment@1d657e262aea7e3f216754febb624831527d2565
~~Vastly increase the number of allocated self-hosted runners~~ (no longer needed because not using self-hosted anymore)

This is just Part 1 of a larger 4-part task to make the startup time quality gate, but I think it's the hardest part

Run on GitHub-hosted runners so that hopefully the numbers will be more stable (this PR)
Get this to measure the same measurements in the same way as we do for Sentry
Make metamaskbot post with the results of this instead of the current CircleCI source
Set up the quality gate

Related issues

Progresses: MetaMask/MetaMask-planning#3679

github-actions · 2025-01-29T02:03:35Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

metamaskbot · 2025-01-29T02:38:57Z

Builds ready [9c55124]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- content-script: 0
- offscreen: 0

Page Load Metrics (1730 ± 64 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	225	2038	1648	351	169
		domContentLoaded	1508	2026	1699	128	61
		load	1568	2041	1730	133	64
		domInteractive	23	97	42	25	12
		backgroundConnect	10	68	34	20	10
		firstReactRender	15	97	50	28	13
		getState	4	57	13	14	7
		initialActions	0	0	0	0	0
		loadScripts	1052	1569	1246	123	59
		setupStore	8	91	19	23	11
		uiStartup	1772	2595	2011	234	112

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

metamaskbot · 2025-01-29T03:35:56Z

Builds ready [9c55124]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- content-script: 0
- offscreen: 0

Page Load Metrics (1730 ± 64 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	225	2038	1648	351	169
		domContentLoaded	1508	2026	1699	128	61
		load	1568	2041	1730	133	64
		domInteractive	23	97	42	25	12
		backgroundConnect	10	68	34	20	10
		firstReactRender	15	97	50	28	13
		getState	4	57	13	14	7
		initialActions	0	0	0	0	0
		loadScripts	1052	1569	1246	123	59
		setupStore	8	91	19	23	11
		uiStartup	1772	2595	2011	234	112

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

.github/workflows/benchmarks.yml

.github/workflows/update-attributions.yml

metamaskbot · 2025-01-29T04:57:31Z

Builds ready [e5a4333]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- content-script: 0
- offscreen: 0

Page Load Metrics (2151 ± 76 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	227	2507	1986	600	288
		domContentLoaded	1760	2409	2105	165	79
		load	1842	2476	2151	158	76
		domInteractive	30	150	58	32	16
		backgroundConnect	15	94	51	28	13
		firstReactRender	18	90	47	22	10
		getState	9	90	35	26	12
		initialActions	0	1	0	0	0
		loadScripts	1289	1877	1578	152	73
		setupStore	10	66	22	18	9
		uiStartup	2110	3372	2513	279	134

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

metamaskbot · 2025-01-29T05:39:18Z

Builds ready [a6f5acb]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- content-script: 0
- offscreen: 0

Page Load Metrics (1708 ± 72 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	1554	2173	1715	143	69
		domContentLoaded	1509	2097	1676	133	64
		load	1514	2175	1708	151	72
		domInteractive	25	94	39	19	9
		backgroundConnect	10	226	40	47	23
		firstReactRender	16	95	47	26	12
		getState	5	89	20	25	12
		initialActions	0	1	0	0	0
		loadScripts	1075	1614	1220	116	56
		setupStore	7	61	14	13	6
		uiStartup	1722	2398	1979	202	97

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

Gudahtt · 2025-01-29T14:58:42Z

Could you split this into multiple PRs? A lot of these changes seem to be distinct from each other

metamaskbot · 2025-01-29T22:24:03Z

Builds ready [6266945]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- content-script: 0
- offscreen: 0

Page Load Metrics (1958 ± 284 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	289	3234	1555	702	337
		domContentLoaded	1386	3608	1945	585	281
		load	1394	3645	1958	592	284
		domInteractive	22	106	45	23	11
		backgroundConnect	8	68	19	15	7
		firstReactRender	15	84	49	23	11
		getState	4	54	15	14	7
		initialActions	0	1	0	0	0
		loadScripts	1008	2763	1440	483	232
		setupStore	7	61	24	20	9
		uiStartup	1532	4169	2280	685	329

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

itsyoboieltr · 2025-01-30T19:50:07Z

The migration to GitHub Actions is a necessary step, but before committing to self-hosted runners, we should first validate the stability of measurements across different environments.

We need to compare benchmark results between CircleCI, GitHub-hosted (ubuntu-latest), and self-hosted runners to determine if self-hosted runners indeed reduce fluctuations. I am not convinced that they actually would.
We need to ensure that the same type of build is measured across all environments to make results comparable. This PR is testing the webpack build, but CircleCI is testing browserify.
We should definitely avoid premature infrastructure changes (e.g., provisioning self-hosted runners) without confirming that they improve measurement stability. We need reproducible experiments, and most importantly, NUMBERS!

The suggested approach is to conduct benchmark runs on GitHub-hosted runners first (ubuntu-latest) and compare them against CircleCI and self-hosted runner results. If fluctuations remain high across all environments, this method may not be suitable for a quality gate. However, the migration to GitHub Actions remains beneficial regardless.

Next steps:

Run benchmarks on ubuntu-latest and self-hosted runners, compare results.
If stability issues persist, scrap the idea of using the results for a quality gate while still proceeding with the migration to GitHub Actions.

Would be great to see some comparison data in the PR before merging.

HowardBraham · 2025-01-30T22:12:10Z

As requested by @itsyoboieltr, here are benchmarks running in 4 different configurations:

browserify build on self-hosted runners
browserify build on ubuntu-latest
webpack build on self-hosted runners
webpack build on ubuntu-latest

Probably the most important stat here is uiStartup (btw our old system highlights load). Two additional things I see:

Standard deviation is pretty low across all configurations, which is nice
All configurations are approximately twice as fast as our CircleCI machine

I'm going to run this again to see if it varies over time, and also with some other options.

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Browserify-self-host	Home	firstPaint	158	1195	714	302	145
		domContentLoaded	821	1177	868	75	36
		load	824	1179	873	74	36
		domInteractive	14	27	20	2	1
		backgroundConnect	5	11	7	1	1
		firstReactRender	12	17	12	1	1
		getState	3	5	4	1	0
		initialActions	0	1	0	0	0
		loadScripts	618	809	655	42	20
		setupStore	6	9	7	1	0
		uiStartup	899	1292	971	81	39
Browserify-ubuntu-latest	Home	firstPaint	216	1061	869	299	143
		domContentLoaded	965	1144	1016	40	19
		load	973	1150	1022	40	19
		domInteractive	26	69	36	14	7
		backgroundConnect	8	12	10	1	1
		firstReactRender	18	37	21	4	2
		getState	5	11	8	2	1
		initialActions	0	1	0	0	0
		loadScripts	751	874	791	30	15
		setupStore	9	14	11	1	1
		uiStartup	1090	1371	1169	63	30
Webpack-self-host	Home	firstPaint	154	736	668	119	57
		domContentLoaded	666	718	685	15	7
		load	670	734	692	15	7
		domInteractive	22	44	25	5	2
		backgroundConnect	16	24	19	2	1
		firstReactRender	11	13	12	0	0
		getState	3	6	5	1	0
		initialActions	0	1	0	0	0
		loadScripts	656	710	676	15	7
		setupStore	4	7	5	1	0
		uiStartup	804	882	833	17	8
Webpack-ubuntu-latest	Home	firstPaint	170	896	520	318	153
		domContentLoaded	708	890	829	35	17
		load	714	898	836	36	17
		domInteractive	21	40	34	4	2
		backgroundConnect	22	29	25	2	1
		firstReactRender	13	15	13	1	0
		getState	3	8	6	1	1
		initialActions	0	0	0	0	0
		loadScripts	697	879	818	35	17
		setupStore	5	8	6	1	0
		uiStartup	883	1049	998	33	16

HowardBraham · 2025-01-30T22:59:40Z

Second run of this, numbers are pretty close to the first run

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Browserify-self-host	Home	firstPaint	157	905	776	200	96
		domContentLoaded	802	868	833	16	8
		load	807	875	839	17	8
		domInteractive	14	24	19	2	1
		backgroundConnect	5	8	7	1	0
		firstReactRender	11	13	12	1	0
		getState	3	5	4	1	0
		initialActions	0	1	0	0	0
		loadScripts	602	669	630	17	8
		setupStore	5	7	6	1	0
		uiStartup	893	986	938	23	11
Browserify-ubuntu-latest	Home	firstPaint	209	1107	863	322	155
		domContentLoaded	960	1082	1008	34	16
		load	963	1091	1014	35	17
		domInteractive	17	64	36	15	7
		backgroundConnect	7	16	10	2	1
		firstReactRender	18	37	22	4	2
		getState	5	12	7	2	1
		initialActions	0	1	0	0	0
		loadScripts	754	856	788	29	14
		setupStore	9	23	11	3	1
		uiStartup	1107	1265	1152	40	19
Webpack-self-host	Home	firstPaint	172	752	677	117	56
		domContentLoaded	655	745	695	21	10
		load	669	754	705	20	9
		domInteractive	22	70	28	10	5
		backgroundConnect	17	24	19	2	1
		firstReactRender	11	13	12	1	0
		getState	3	6	4	1	0
		initialActions	0	1	0	0	0
		loadScripts	647	737	687	21	10
		setupStore	4	7	5	1	1
		uiStartup	822	897	847	20	10
Webpack-ubuntu-latest	Home	firstPaint	169	878	458	307	148
		domContentLoaded	805	895	830	21	10
		load	812	904	837	21	10
		domInteractive	26	81	37	12	6
		backgroundConnect	21	27	24	2	1
		firstReactRender	12	14	13	0	0
		getState	4	10	6	1	1
		initialActions	0	1	0	0	0
		loadScripts	794	883	819	21	10
		setupStore	5	8	6	1	0
		uiStartup	963	1081	991	26	13

.github/workflows/benchmarks.yml

test/e2e/webdriver/chrome.js

metamaskbot · 2025-02-12T03:49:19Z

Builds ready [7c6c00b]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
- content-script: 0
- offscreen: 0

Page Load Metrics (1679 ± 75 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	1449	2021	1679	157	75
		domContentLoaded	1440	1971	1650	153	73
		load	1451	2020	1679	157	75
		domInteractive	24	76	33	14	7
		backgroundConnect	9	77	33	23	11
		firstReactRender	15	95	36	27	13
		getState	5	54	14	15	7
		initialActions	0	0	0	0	0
		loadScripts	1032	1480	1201	126	61
		setupStore	7	73	22	22	11
		uiStartup	1717	2499	1961	236	113

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

metamaskbot · 2025-02-14T21:14:25Z

Builds ready [c888fbd]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
- content-script: 0
- offscreen: 0

Page Load Metrics (1692 ± 67 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	226	2025	1620	345	166
		domContentLoaded	1473	2004	1668	134	65
		load	1479	2026	1692	140	67
		domInteractive	22	93	46	26	13
		backgroundConnect	10	72	26	21	10
		firstReactRender	14	72	32	23	11
		getState	5	57	21	21	10
		initialActions	0	1	0	0	0
		loadScripts	1046	1496	1216	114	55
		setupStore	8	63	16	16	8
		uiStartup	1662	2306	1959	152	73

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

metamaskbot · 2025-02-17T01:18:40Z

Builds ready [e1404b6]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
- content-script: 0
- offscreen: 0

Page Load Metrics (1617 ± 52 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	1447	1813	1617	112	54
		domContentLoaded	1403	1757	1589	108	52
		load	1447	1797	1617	108	52
		domInteractive	23	120	44	28	13
		backgroundConnect	9	71	31	20	9
		firstReactRender	14	67	30	21	10
		getState	4	57	19	19	9
		initialActions	0	1	0	0	0
		loadScripts	994	1319	1158	95	46
		setupStore	7	52	13	12	6
		uiStartup	1661	2059	1839	114	55

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

.github/workflows/benchmark-matrix.yml

.github/workflows/main.yml

.github/workflows/benchmark-matrix.yml

.github/workflows/main.yml

.github/workflows/wait-for-circleci-workflow-status.yml

…hash

…hash

removed "Will enable this as a required job in a later PR" multiline list new github-tools hash

.github/workflows/benchmark-matrix.yml

.github/workflows/main.yml

Co-authored-by: Norbert Elter <[email protected]>

itsyoboieltr

LGTM! Great work!

NicholasEllul

LGTM! Regarding the 90 day retention period, we can monitor to see the costs and what savings we may get by switching to a lower number later on.

metamaskbot · 2025-02-18T19:20:42Z

Builds ready [891d0a4]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
- content-script: 0
- offscreen: 0

Page Load Metrics (1682 ± 98 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	1499	2445	1679	204	98
		domContentLoaded	1491	2420	1652	205	98
		load	1499	2450	1682	205	98
		domInteractive	23	135	46	32	15
		backgroundConnect	8	71	34	22	10
		firstReactRender	14	81	35	24	12
		getState	4	58	16	17	8
		initialActions	0	1	0	0	0
		loadScripts	1017	1955	1196	194	93
		setupStore	7	54	12	10	5
		uiStartup	1705	2813	1924	260	125

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

HowardBraham added the team-extension-platform Extension Platform team label Jan 29, 2025

HowardBraham self-assigned this Jan 29, 2025

metamaskbot added the team-tiger Tiger team (for tech debt reduction + performance improvements) label Jan 29, 2025

HowardBraham removed the team-tiger Tiger team (for tech debt reduction + performance improvements) label Jan 29, 2025

HowardBraham force-pushed the self-hosted-benchmarks branch from 9c55124 to ce07be7 Compare January 29, 2025 04:08

HowardBraham commented Jan 29, 2025

View reviewed changes

.github/workflows/benchmarks.yml Outdated Show resolved Hide resolved

.github/workflows/update-attributions.yml Show resolved Hide resolved

HowardBraham requested review from itsyoboieltr, Gudahtt and NicholasEllul January 29, 2025 04:12

HowardBraham force-pushed the self-hosted-benchmarks branch from ce07be7 to e5a4333 Compare January 29, 2025 04:16

HowardBraham force-pushed the self-hosted-benchmarks branch from e5a4333 to a6f5acb Compare January 29, 2025 05:03

HowardBraham marked this pull request as ready for review January 29, 2025 05:13

HowardBraham added the DO-NOT-MERGE Pull requests that should not be merged label Jan 29, 2025

HowardBraham force-pushed the self-hosted-benchmarks branch from a6f5acb to c9f1e1e Compare January 29, 2025 17:59

HowardBraham changed the title ~~ci: self-hosted runners for benchmarks, with a new prep-deps workflow~~ ci: self-hosted runners for benchmarks Jan 29, 2025

HowardBraham mentioned this pull request Jan 29, 2025

ci: a new prep-deps workflow with caching #29979

Merged

HowardBraham force-pushed the self-hosted-benchmarks branch from c9f1e1e to 6266945 Compare January 29, 2025 21:50

HowardBraham mentioned this pull request Jan 30, 2025

ci: self-hosted runners for benchmarks (prereq) MetaMask/github-tools#38

Merged

NicholasEllul reviewed Feb 6, 2025

View reviewed changes

.github/workflows/benchmarks.yml Outdated Show resolved Hide resolved

.github/workflows/benchmarks.yml Outdated Show resolved Hide resolved

test/e2e/webdriver/chrome.js Outdated Show resolved Hide resolved

HowardBraham force-pushed the self-hosted-benchmarks branch from 6266945 to abcd5a1 Compare February 11, 2025 07:24

HowardBraham force-pushed the self-hosted-benchmarks branch from 7c6c00b to c888fbd Compare February 14, 2025 20:41

HowardBraham force-pushed the self-hosted-benchmarks branch from c888fbd to e1404b6 Compare February 15, 2025 05:04

itsyoboieltr reviewed Feb 18, 2025

View reviewed changes

HowardBraham added 6 commits February 18, 2025 12:52

ci: self-hosted runners for benchmarks

426411d

responding to reviews

f379e78

needs: run-benchmark-matrix

c15e0e2

ubuntu-22.04 and headless=new

ba55443

replaced all setup-environment@main and lint-workflows.yml@main with @…

8135785

…hash

env.COMMANDS as JSON

61028ec

removed "Will enable this as a required job in a later PR" multiline list new github-tools hash

HowardBraham force-pushed the self-hosted-benchmarks branch from e1404b6 to 61028ec Compare February 18, 2025 18:03

HowardBraham removed the DO-NOT-MERGE Pull requests that should not be merged label Feb 18, 2025

default retention-days

dfdcfc0

itsyoboieltr reviewed Feb 18, 2025

View reviewed changes

.github/workflows/benchmark-matrix.yml Outdated Show resolved Hide resolved

.github/workflows/main.yml Outdated Show resolved Hide resolved

.github/workflows/main.yml Outdated Show resolved Hide resolved

Apply suggestions from itsyoboieltr code review

2d7f1eb

Co-authored-by: Norbert Elter <[email protected]>

itsyoboieltr changed the title ~~ci: GitHub-hosted runners for benchmarks~~ feat: GitHub-hosted runners for benchmarks Feb 18, 2025

rename to run-benchmarks.yml

891d0a4

itsyoboieltr approved these changes Feb 18, 2025

View reviewed changes

HowardBraham requested a review from NicholasEllul February 18, 2025 18:44

NicholasEllul approved these changes Feb 18, 2025

View reviewed changes

HowardBraham enabled auto-merge February 18, 2025 19:15

HowardBraham added this pull request to the merge queue Feb 18, 2025

Merged via the queue into main with commit 2b39e3b Feb 18, 2025
79 checks passed

HowardBraham deleted the self-hosted-benchmarks branch February 18, 2025 19:51

github-actions bot locked and limited conversation to collaborators Feb 18, 2025

metamaskbot added the release-12.14.0 Issue or pull request that will be included in release 12.14.0 label Feb 18, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: GitHub-hosted runners for benchmarks #29955

feat: GitHub-hosted runners for benchmarks #29955

HowardBraham commented Jan 29, 2025 •

edited

Loading

github-actions bot commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

Gudahtt commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

itsyoboieltr commented Jan 30, 2025

HowardBraham commented Jan 30, 2025

HowardBraham commented Jan 30, 2025

metamaskbot commented Feb 12, 2025

metamaskbot commented Feb 14, 2025

metamaskbot commented Feb 17, 2025

itsyoboieltr left a comment

NicholasEllul left a comment

metamaskbot commented Feb 18, 2025

feat: GitHub-hosted runners for benchmarks #29955

feat: GitHub-hosted runners for benchmarks #29955

Conversation

HowardBraham commented Jan 29, 2025 • edited Loading

Description

Major changes

Prerequisites to merging this PR

This is just Part 1 of a larger 4-part task to make the startup time quality gate, but I think it's the hardest part

Related issues

github-actions bot commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

Gudahtt commented Jan 29, 2025

metamaskbot commented Jan 29, 2025

itsyoboieltr commented Jan 30, 2025

HowardBraham commented Jan 30, 2025

HowardBraham commented Jan 30, 2025

metamaskbot commented Feb 12, 2025

metamaskbot commented Feb 14, 2025

metamaskbot commented Feb 17, 2025

itsyoboieltr left a comment

Choose a reason for hiding this comment

NicholasEllul left a comment

Choose a reason for hiding this comment

metamaskbot commented Feb 18, 2025

HowardBraham commented Jan 29, 2025 •

edited

Loading