Unable to perform an ota update with custom build micropython firmware

[synodriver]

Thank you for making this great library! I meet some problem when performing an ota update,

using
ota.update.from_file("http://192.168.0.88:8000/ESP32_GENERIC-OTA-git-16MB-resize=vfs.bin", reboot=True)
I got

Writing new micropython image to OTA partition 'ota_1'...
Device capacity: 384 x 4096 byte blocks.
Opening firmware file http://192.168.0.88:8000/ESP32_GENERIC-OTA-git-16MB-resize=vfs.bin...
BLOCK 376 + 1376 bytes
Verifying SHA of the written data...Passed.
SHA256=fea0dab2c5f1e5c799e243e6d16a7749bb14675673442e587a724b80393be838
OTA Partition 'ota_1' updated successfully.
Micropython will boot from 'ota_1' partition on next boot.
Remember to call ota.rollback.cancel() after successful reboot.
Rebooting in  1 seconds (ctrl-C to cancel)
ets Jul 29 2019 12:21:46

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4112
load:0x40078000,len:15084
load:0x40080400,len:4
load:0x40080404,len:3332
entry 0x400805ac
E (91) esp_image: Segment 0 0x3fff0030-0x3fff1040 invalid: overlaps bootloader data
E (91) boot: OTA app partition slot 1 is not bootable

Note that Segment 0 0x3fff0030-0x3fff1040 invalid: overlaps bootloader data, after reboot, the firmware did not update.
The firmware I'm using is a custom build micropython, the build script is make BOARD=ESP32_GENERIC BOARD_VARIANT=OTA, the board I'm
using is esp32-wroom-32e with 16Mb flash, so I'm using your another tool mp-image-tool-esp32 firmware.bin -f 16M --resize vfs=0 to create a suitable firmware, and the problem occured.

Here is the board info

ota.status.status()
Micropython v1.26.0-preview.1587.df421577a1 has booted from partition 'ota_0'.
Will boot from partition 'ota_1' on next reboot.
The next OTA partition for update is 'ota_1'.
The / filesystem is mounted from partition 'vfs'.
Partition table:
# Name       Type     SubType      Offset       Size (bytes)
  nvs        data     nvs          0x9000     0x4000     16,384
  otadata    data     ota          0xd000     0x2000      8,192
  phy_init   data     phy          0xf000     0x1000      4,096
  ota_0      app      ota_0       0x10000   0x180000  1,572,864
  ota_1      app      ota_1      0x190000   0x180000  1,572,864
  vfs        data     fat        0x310000   0xcf0000 13,565,952
OTA record: state=PENDING, seq=2, crc=1442199412, valid=False
OTA record is PENDING.
OTA record: state=ABORTED, seq=2, crc=1442199412, valid=False
OTA record is ABORTED.
Next boot is 'ota_0'.

Extra info

The firmware I build is able to flash using esptool, so I don't think it's a firmware problem.

[glenn20]

Thanks for reporting this issue. My initial suspicion is that a bootloader may have been installed onto your device which does not respect the 16MB flash size (that does produce problems like this one).

If that is the problem, you can fix it by patching the bootloader on your device by connecting to the device over serial port and (change u0 to the serial port to which your device is attached).

First, check the flash size setting in the bootloader:

mp-image-tool-esp32 u0

If you need to change it, use:

mp-image-tool-esp32 u0 -f 16M

It is easy to inadvertently overwrite the bootloader (as I have learnt) when you are working on devices by performing an esptool install, rather than an ota install.

If this is not the cause, let me know and I'll look further. If it is the cause, it is a good prompt for me to perform a check in the ota.update module. One wants to be as cautious as possible when performing ota updates.

[synodriver]

The mp-image-tool-esp32 COM4 just hangs unexpectly, but I know the old firmware running on that board is already a 16Mb flash firmware

MicroPython v1.26.0-preview.1587.df421577a1 on 2025-05-26; Generic ESP32 module with OTA with ESP32
Type "help()" for more information.
>>> import esp
>>> esp.flash_size() 
16777216
>>> esp.flash_size()  // 1024**2
16

it's compilerd the same way, modified using mp-image-tool-esp32 and flashed using esptool, so I don't think the old firmware caused this problem.

Besides, here is my custom micropython build (just merged 2 pending pr I want), and
here is the compiled bin file, hope that would help debugging the strange problem

[glenn20]

If I understand correctly, the firmware.bin file linked above is what you are trying to flash into the ota_1 partition. (If I have that wrong, let me know).

That firmware.bin is a full micropython image which includes its own bootloader, partition table and 'app' binary bundled together. To perform an OTA update, you need a pure micropython 'app binary'.

See firmware for ota updates for 3 ways to get such an app image.

In your case, you can:

• extract the 'app-binfrom yourfirmware.bin` using option 3, OR,
• use the micropython.bin file in your ports/esp32/build-ESP32_GENERIC directory.

PS. I actually have some work in progress to automatically extract the app image if provided with a full, combined firmware file - but I confess I do have too many WIPs.

[synodriver]

Thank you for your answer, I remember esphome has a ota check when update new firmware，so maybe we can do the same？