Update bootloaders with IDF release/v5.4 ones

[JurajSadel]

closes #853

[JurajSadel]

Some help with testing will be appreciated here 😅

[jessebraham]

Thanks, but I'm not sure it makes sense doing this now when we're not planning on releasing until mid-June. There may potentially be updates made in the meantime, this will probably be one of the last things we do.

[MabezDev]

I don't think there is any reason to hold off here, idf 5.4 is the latest, and idf 5.5's full release won't be till August at the earliest. I'm not aware of anything from 5.5 that we need bootloader wise, so I think using a bit of a battle tested 5.4 seems like a good plan here.

Regarding HIL, you'll need to rebuild the example binaries with the new https://github.com/esp-rs/esp-hal/tree/main/esp-bootloader-esp-idf crate, so that the app desc can be found by the bootloader.

[bjoernQ]

There is one catch: newer bootloaders need the app-descriptor which we support via the unreleased esp-bootloader-esp-idf - users will need to use that to have a app-descriptor

I guess updating to the latest bootloader which doesn't require the app-descriptor would be the best option for now

[MabezDev]

There is one catch: newer bootloaders need the app-descriptor which we support via the unreleased esp-bootloader-esp-idf - users will need to use that to have a app-descriptor

I guess updating to the latest bootloader which doesn't require the app-descriptor would be the best option for now

but all v5.x versions need the app image desc, and no 4.x boot loader is in "maintenance", though the maintenance is probably not an issue.

I think we should switch now, and start tackling this breakage. We'll have various tests/CI workflows etc that will need to be updated. From that we can find edge cases and troubleshooting steps ready for a migration guide entry in esp-hal later (doesn't strictly belong there, but gets the most eyeballs). If we wait, we only delay this process until the espflash release.

[bjoernQ]

but all v5.x versions need the app image desc

I think some older 5.x bootloaders don't strictly require it (at least w/o OTA support). Let me have a look

[bjoernQ]

Seems 5.3.1 is the latest which doesn't fail to boot with an absent app-descriptor

[bjoernQ]

In any case we should document which exact version the bootloaders are from (ideally including the exact commit) and what config options were used (or maybe we should also define them here - see https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/kconfig.html#bootloader-config)

[elipsitz]

This could also be a good opportunity to fix the issue with the esp32-s3 bootloader (#657)

[jessebraham]

but all v5.x versions need the app image desc

This is not correct. The published version of espflash is using the bootloader from v5.1 and does not require it. IIRC v5.2 also does not require it. The dependency on the application descriptor was added with either v5.3 or v5.4 (I don't recall which).

Anyway, HIL is failing, so can't be merged as-is anyway.

[bjoernQ]

"I (76) esp_image: MMU page size mismatch, configured: 0x8000, found: 0x10000"

This is not great given we take all the effort to generate new bootloaders

esp-hal (for now) uses 0x10000 fixed/hard-coded (applies to C2, C6, H2 IIRC)

[jessebraham]

"I (76) esp_image: MMU page size mismatch, configured: 0x8000, found: 0x10000"

This is not great given we take all the effort to generate new bootloaders

esp-hal (for now) uses 0x10000 fixed/hard-coded (applies to C2, C6, H2 IIRC)

Probably we should build the bootloaders with the same values being used by the HAL, in this case.

[JurajSadel]

"I (76) esp_image: MMU page size mismatch, configured: 0x8000, found: 0x10000"

This is not great given we take all the effort to generate new bootloaders
esp-hal (for now) uses 0x10000 fixed/hard-coded (applies to C2, C6, H2 IIRC)

Probably we should build the bootloaders with the same values being used by the HAL, in this case.

I rebuilt C2, C6, and H2 with 0x10000 so the warning should be gone

[MabezDev]

This is not correct. The published version of espflash is using the bootloader from v5.1 and does not require it. IIRC v5.2 also does not require it. The dependency on the application descriptor was added with either v5.3 or v5.4 (I don't recall which).

Thanks for clearing that up, I thought it was all 5.x versions 😅. We could only update to v5.2 here, but I think we should just rip this band aid off as this app desc thing will be a requirement going forward if we want to update the bootloaders again in the future.

[bjoernQ]

Sooner or later we want/have to break most users projects

I still think we should warn users when they get the pre-built bootloader (i.e. not supply their own).
We could even check the app-descriptor and tell them they need it. (the error they will get from the bootloader is a bit cryptic)

[jessebraham]

Are these default settings the same for all chips? I also don't think there's any value in including the commented-out lines here, it's just visual noise.

[jessebraham]

Thanks, LGTM!