[WEB Installer] Lolin S3 mini loads but does't boot wled

[marcobrianza]

Hello, just tested the web installer with release 0.14.2 on Lolin s3 mini
The installer does its thing but the board does not boot wled afterwards.
The serial loops with these messages

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x3 (RTC_SW_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x403cd98e
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x43c
load:0x403c9700,len:0xbec
load:0x403cc700,len:0x2a3c
entry 0x403c98d8

FYI tasmota web installer works correctly on this board

[lost-hope]

Does it work, when flashed from PlatformIO?

[marcobrianza]

Using PIO I noticed that there is no profile for ESP32-S3 at 4MB Flash but only 8M and above. (The S3 mini is 4MB) so the S3 mini does not work also in PIO.

I tried an S3 PRO (16MB) and from PIO it works, but not from the web installer. I tried 0.14.1 and 0.14.2 and get different kind of errors

[blazoncek]

@lost-hope I think we may need to rethink the way web installer is constructed.

WLED binaries themselves are not dependent on the flash size (though they may not fit on every partition) so there can only be limited number of them for particular chip (i.e. 2 for ESP32, one with PSRAM and one without, excluding Ethernet).
So instead of hiding flash size form a user it could be part of the install process. Partition map binaries AFAIK are the same for every chip, their flash offset may vary though.

Classic ESP32 needs 2 files as PSRAM version requires GPIO16 and GPIO17.
C3 only ever needs 1 build as there are no HW variants requiring otherwise.
Even S2 could use only one build as there are no differences if using PSRAM or not.
I am not sure about S3 as I do not have one.

We can also integrate audio into every (compatible) build.

@marcobrianza there will never be every possible board configuration in platformio.ini. If you need particular configuration use platformio_override.ini and extend existing environment.

[lost-hope]

@blazoncek already thought of that cause of the esp8266 partitions.
the flash size selection needs to be done manually, so by the user, cause there is no automated detection.

and that will introduce the issue with users selecting the wrong size.

Partition map binaries AFAIK are the same for every chip, their flash offset may vary though.

So we only need 5 partition files: 1M 2M 4M 8M 16M

i think i have an idea on how to handle it, but i am not completely sure if that works out.

Then we have 6 Binaries for the chips (ESP8266, ESP32,ESP32 PSRAM, ESP32-S2 EPS32-S3 ESP32-C3) to cover all of the possible devices right.
plus variants for Ethernet and Audioreactive.
What about the new Wrover binary in the newest release?

[blazoncek]

and that will introduce the issue with users selecting the wrong size.

The user has the ESP still physically attached to the computer. I find this to be unproblematic as the user can flash again and again until correct flash size is selected.

ESP8266 is a different beast as the flash size is embedded in the binary and esptool extracts it.

WROVER needs separate binary since otherwise WLED defaults to GPIO16 for LEDs. PSRAM models of ESP32 use GPIO16 and GPIO17 for PSRAM and using regular ESP32 causes crashes upon boot. If we change default GPIO for LEDs (could also be dynamic) then only one binary would be needed.

[blazoncek]

@marcobrianza THANK YOU.

[blazoncek]

I now have all compatible ESP variants with various flash sizes and was able to determine and verify how flashing works (using PIO & esptool).

As mentioned above ESP8266 is a special case since everything is included within firmware binary (including flash size, CPU clock, flash clock, etc) so each variant needs its own binary file (currently 6 in total: 80MHz 1MB, 80MHz 2MB, 80MHz 4MB, 160MHz 1MB, 160MHz 2MB and 160MHz 4MB, but the future may bring 4 additional files for 8MB and 16MB flash). There is a possibility to change CPU clock speed at runtime but current firmware can't do that. Yet.

ESP32 and its variants S2, S3 and C3 use bootloader, partition map and firmware files to successfully flash a device.

In my testing the bootloader was identical regardless of features used on a particular board, so one bootloader per ESP variant seems reasonable choice. Bootloader needs to be flashed at address 0x00000 for ESP32, ESP32-S3 and ESP32-C3 but at address 0x01000 for ESP32-S2.

Partition map binary depends on the amount of flash memory a particular ESP device has. So far I've seen ESP32 (and variant) devices with 2MB, 4MB, 8MB and 16MB of flash. 2MB is unusable (even though it could run WLED) and will not be supported by WLED. Partition map needs to be flashed at address 0x08000 for all ESP32 variants.

Firmware binaries depend on the features selected. For classic ESP32 we can have 3 main binaries: plain, with PSRAM and with Ethernet (each of them can/may have up to 6 usermods). Ethernet needs its binary because it needs (allocates) quite a few commonly used GPIO pins, otherwise it could be included in plain binary. PSRAM support needs its own binary for chip revisions < rev.3 which have hardware bug, the fix for hardware bug increases firmware binary by 100kB. For newer chips plain will (or should) correctly select and use PSRAM if available (from 0.15.0-b2 onward). 0.15.0-b2 (and newer) will also correctly detect ESP32-PICO variant which uses GPIO16 for flash.
S2 and S3 may only need a single binary as PSRAM support can be detected and used without workarounds. The firmware binary file is the same regardless of flash size but may differ for S3 depending on the PSRAM type used (QSPI, OSPI, QPI, OPI, etc). These chips do not support Ethernet. (I do not own a S2 or S3 device without PSRAM and all devices I own have QSPI PSRAM.)
C3 has no PSRAM or Ethernet support so it only needs one firmware binary.

As I have no knowledge of how web installer works I am asking anyone who knows that, if an update to web installer could be made (and PRed) to include above information. I know that @softhack007 has a lot of experience with S3 so he may add something for devices with OPI.

References on PSRAM for S3 (we use Arduino framework so some configuration may be hidden and not available):
https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/flash_psram_config.html
https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/external-ram.html

[softhack007]

I know that @softhack007 has a lot of experience with S3 so he may add something for devices with OPI.

@blazoncek @lost-hope I remember that espressif docs say that -S3 with qspi and opi modes require a different bootloader. I have some devices here (no psram, qspi, opi) so I'll test them and report back.

How do you want to handle "USB CDC" vs "serial UART usb" devices? Improv (setting wifi password after install) needs to communicate with WLED over Serial, so it's important that the right serial driver is active.

[softhack007]

ESP32 and its variants S2, S3 and C3 use bootloader, partition map and firmware files to successfully flash a device.

Additionally, you need boot_app0.bin at 0xe000.

https://github.com/espressif/arduino-esp32/blob/49b94644c693df22f0a15ffb9454e2d7c3e6c679/tools/platformio-build.py#L212

[blazoncek]

I am leaning towards only supporting 2 or maximum 3 S3 boards as otherwise we'll get into thousand devices mess soon.
I recommend:

• 8MB/16MB OPI/OSPI flash with 8MB OPI/OSPI PSRAM (something @softhack007 has)
• 4MB QSPI flash with 2MB/4MB QSPI PSRAM (S3-Zero or Lolin S3 mini and some custom devices I received)
• optionally 4MB/8MB flash no PSRAM
  All with CDC serial only (as it is most common and easy to implement).

Unfortunately that would mean users with unsupported devices will need to compile themselves. Those that are unable to do so should be instructed to get supported device instead or seek help on the forum or Discord.

[lost-hope]

How do you want to handle "USB CDC" vs "serial UART usb" devices? Improv (setting wifi password after install) needs to communicate with WLED over Serial, so it's important that the right serial driver is active.

to be honest the Improv wifi setting is nice but can also be seen as a addition. If it woks: Nice. But as the possible list of binaries is already pretty long, i would decide against adding CDC and non CDC binaries...

[blazoncek]

Additionally, you need boot_app0.bin at 0xe000.

https://github.com/espressif/arduino-esp32/blob/49b94644c693df22f0a15ffb9454e2d7c3e6c679/tools/platformio-build.py#L212

That's interesting. Where did you find that info?
When I flashed S2 (and S3) there was no such file present in .pio/build/env/ folder.

I also did a binary diff on bootloaders for all 3 variants of S3 build environment and they were identical on my system.

Flash will be erased from 0x00000000 to 0x00003fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x0000e000 to 0x0000ffff...
Flash will be erased from 0x00010000 to 0x0016dfff...
...
Wrote 14768 bytes (10149 compressed) at 0x00000000 in 0.2 seconds (effective 544.3 kbit/s)...
...
Wrote 3072 bytes (128 compressed) at 0x00008000 in 0.0 seconds (effective 541.7 kbit/s)...
...
Wrote 1431696 bytes (963320 compressed) at 0x00010000 in 10.8 seconds (effective 1061.2 kbit/s)...

The output of actual flashing of S3-Zero board.