Skip to content

BES firmware: CI/CD builds + pinned toolchain #3350

Description

@aisraelov

Set up CI/CD for mentra-live-bes so anyone (and any agent) can build correct BES firmware without hand-assembling a toolchain — the same play we ran for mobile (monorepo CI) and cloud.

Why now

From Slack (Alex → Nicolo):

How can I build BES firmware? Whenever I do, my update_ota.bin just bricks the BES and I have to factory flash from the windows laptop. No compiler version is specified in the BES repo. One helpful thing would be for you to set up CI for BES builds; that way we can all develop BES without having the same toolchain set up (and would also be a forcing function for correct documentation, since there would be code backing it)

Nicolo: "I also have the same issue. I have to go through Liu to get BES OTA … I think it might be a mac problem … Can you create a ticket for the BES automation" — this is that ticket.

Findings from the 2026-07-04 #2780 session that this should encode:

  • No pinned toolchain anywhere. README says "brew install gcc-arm-embedded" (floating version). Shipped-binary fingerprints show at least two different build environments in the wild (fw-team builds with no toolchain markers; Nicolo's Jun 22 zip shows an arm-13-era ARM GNU toolchain). Local ARM GCC 10.3-2021.10 builds boot fine when factory-flashed.
  • The bricks are (at least partly) the OTA apply pipeline, not the compilers: byte-identical images that brick via DEBUG_BES_OTA/update_ota.bin boot fine when factory-flashed. All 3/3 failures were images ≥ ~1.98MB raw; all known-good OTA-delivered images are ≤ ~1.96MB. The historical "bad update_ota.bin" incidents match this pattern. (The OTA-pipeline bug itself may deserve a sibling ticket; CI can't fix it but can gate on image size until it's fixed.)
  • tools/build_compressed_ota.sh has a latent stale-chunk bug: it rms its x0?? split chunks only after packaging, so a rebuild that produces fewer chunks concatenates stale ones from an earlier failed run → corrupt update_ota.bin that passes its own CRC step. Fix is rm -f x0?? before split.

Scope

  1. GitHub Actions workflow: checkout → install pinned ARM toolchain (exact version + checksum) → ./build_pz.sh && ./build_ota.sh && ./build_pz.sh OTA → upload ota_copy.bin, best1502x_ibrt_bpone.bin, update_ota.bin as artifacts on every PR + main push.
  2. Pin the toolchain version in the README and in the workflow (single source of truth; CI goes red on drift).
  3. Sanity gates in CI: image builds, CRC patch verified, raw size < the OTA-safe threshold (warn/fail above ~1.9MB until the OTA apply bug is fixed), and an unpack-and-compare check that update_ota.bin decompresses byte-identical to the raw image.
  4. Stretch: release workflow producing the dated bes_fw_release_*.zip like build_release.sh.

Related: #2780 (session findings), fengyue120/mentra-live-bes#6 (throughput PR where the toolchain question is also raised).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions