firmware crashes on Waveshare "ESP32-S3-Touch-LCD-4.3"

[Rai-Sp]

Describe the bug
I tried to build lvgl-micropython firmware using https://github.com/lvgl-micropython/lvgl_micropython for a Waveshare "ESP32-S3-Touch-LCD-4.3" device.

With the currently newest commit d7f69f3 a simple test script (see below) crashes with "Guru Meditation Error: Core 0 panic'ed (StoreProhibited). Exception was unhandled." I also tried several commits in between, but none worked. Commenting out the line with "task_handler = task_handler.TaskHandler()" avoids the crash, but of course, the display does not show anything.

Expected behavior
When I use commit 561cf5a for building, the same test script works as expected. This should also be the case with the newest commit.

Build Command

python3 \
make.py \
esp32 \
clean \
BOARD=ESP32_GENERIC_S3 \
BOARD_VARIANT=SPIRAM_OCT \
--flash-size=8 \
DISPLAY=ili9341 \
DISPLAY=rgb_display \
INDEV=xpt2046 \
INDEV=gt911 \
CONFIG_FREERTOS_HZ=100 \
CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_240=y \
CONFIG_ESPTOOLPY_FLASHMODE_QIO=y \
CONFIG_ESPTOOLPY_FLASHFREQ_120M=y \
CONFIG_SPIRAM_MODE_OCT=y \
CONFIG_IDF_EXPERIMENTAL_FEATURES=y \
CONFIG_SPIRAM_SPEED_120M=y \
CONFIG_SPIRAM_FETCH_INSTRUCTIONS=y \
CONFIG_SPIRAM_RODATA=y \
CONFIG_ESP32S3_DATA_CACHE_LINE_64B=y \
CONFIG_COMPILER_OPTIMIZATION_PERF=y

Test-Script

#-----------------------------------------------------------------------------
# The following is based on:
# https://github.com/kwinter745321/ESP32LVGL/blob/main/Videos/video58/Flash/display_driver.py
#
# Device used: Waveshare "ESP32-S3-Touch-LCD-4.3"
#
# Firmware built using https://github.com/lvgl-micropython/lvgl_micropython
#
# Works as expected when built from commit 561cf5a3979db021b3394a1aaf7bb0e3550d4c97
# (based on micropython 1.25.0, LVGL 9.3.0 and esp-idf 5.4.0)
#
# Crashes when built from currently newest commit d7f69f3954df87352c9e42612544323a319441da
# with "Guru Meditation Error: Core  0 panic'ed (StoreProhibited). Exception was unhandled."
# (based on micropython 1.27.0, LVGL 9.4.0 and esp-idf 5.5.1)
# I also tried several commits in between, but none worked.
# Commenting out the line with "task_handler = task_handler.TaskHandler()"
# avoids the crash, but of course, the display does not show anything
#-----------------------------------------------------------------------------
# Setting up IO-Expander and resetting touch and display is missing in the above-mentioned:
import io_exp

io_exp.init()
io_exp.reset()
#-----------------------------------------------------------------------------
import lcd_bus
import rgb_display_framework as rgb_display
import lvgl as lv
import task_handler
import i2c
import gt911

rgb_bus = lcd_bus.RGBBus(
    hsync=46,
    vsync=3,
    de=5,
    pclk=7,
    data0=14, data1=38, data2=18, data3=17, data4=10,
    data5=39, data6=0, data7=45, data8=48, data9=47, data10=21,
    data11=1, data12=2, data13=42, data14=41, data15=40,
    freq=13_000_000,
    hsync_front_porch=8,
    hsync_back_porch=8,
    hsync_pulse_width=4,
    vsync_front_porch=16,
    vsync_back_porch=16,
    vsync_pulse_width=4,
    vsync_idle_low=True,
    pclk_active_low=True
)
print("RGBBus", rgb_bus)

_WIDTH, _HEIGHT = 800, 480
_BUF1 = rgb_bus.allocate_framebuffer(_WIDTH * _HEIGHT * 2, lcd_bus.MEMORY_SPIRAM)
_BUF2 = rgb_bus.allocate_framebuffer(_WIDTH * _HEIGHT * 2, lcd_bus.MEMORY_SPIRAM)

display = rgb_display.RGBDisplayDriver(
    data_bus=rgb_bus,
    display_width=_WIDTH,
    display_height=_HEIGHT,
    frame_buffer1=_BUF1,
    frame_buffer2=_BUF2,
    color_space=lv.COLOR_FORMAT.RGB565
)
print("display", display)
display.set_power(True)
display.init()

i2c_bus = i2c.I2C.Bus(host=0, scl=9, sda=8, freq=100_000)
touch_i2c = i2c.I2C.Device(i2c_bus, gt911.I2C_ADDR, gt911.BITS)
indev = gt911.GT911(touch_i2c, reset_pin=None, startup_rotation=0)
if indev.hw_size != (_WIDTH, _HEIGHT):
    fw_config = indev.firmware_config
    fw_config.width = _WIDTH
    fw_config.height = _HEIGHT
    fw_config.save()

task_handler = task_handler.TaskHandler()
#-----------------------------------------------------------------------------
scr = lv.screen_active()
scr.set_style_bg_color(lv.color_hex(0x000000), 0)

slider = lv.slider(scr)
slider.set_size(600, 25)
slider.center()

label = lv.label(scr)
label.set_text('HELLO LVGL_MICROPYTHON!')
label.align(lv.ALIGN.CENTER, 0, -100)
#-----------------------------------------------------------------------------
# The module io_exp.py looks like:
# #-----------------------------------------------------------------------------
# from i2c import I2C
# import ch422g
# import time
#
# class Vars:
#     tprst_pin  = None
#     lcdbl_pin  = None
#     lcdrst_pin = None
#     sd_cs_pin  = None
#     usel_pin   = None
#
# def init():
#     i2c_bus = I2C.Bus(host=0, scl=9, sda=8, freq=100_000, use_locks=False)
#
#     io_expander_device = I2C.Device(i2c_bus, dev_id=ch422g.I2C_ADDR,
#                                     reg_bits=ch422g.BITS)
#     ch422g.Pin.set_device(io_expander_device)
#
#     Vars.tprst_pin  = ch422g.Pin(ch422g.EXIO1, mode=ch422g.Pin.OUT, value=0)
#     Vars.lcdbl_pin  = ch422g.Pin(ch422g.EXIO2, mode=ch422g.Pin.OUT, value=0)
#     Vars.lcdrst_pin = ch422g.Pin(ch422g.EXIO3, mode=ch422g.Pin.OUT, value=0)
#     Vars.sd_cs_pin  = ch422g.Pin(ch422g.EXIO4, mode=ch422g.Pin.OUT, value=0)
#     Vars.usel_pin   = ch422g.Pin(ch422g.EXIO5, mode=ch422g.Pin.OUT, value=1)
#
# def reset():
#     time.sleep_ms(500)
#     Vars.tprst_pin.high()
#     time.sleep_ms(100)
#     Vars.tprst_pin.low()
#
#     Vars.lcdrst_pin.high()
#     time.sleep_ms(100)
#     Vars.lcdrst_pin.low()
# #-----------------------------------------------------------------------------

[kdschlosser]

when you get the error at the bottom you should see a section that says "backtrace" If you can decode that for me and post what the decoded information is that would be the only way I would be able to track down the problem.

Here is a website on how to decode the backtrace.

https://jsimoesblog.wordpress.com/2022/11/04/decoding-esp32-back-trace/

The files you are going to need to use will be located in ~.espressif and in lvgl_micropython/lib/micropython/ports/esp32/build_*

[kdschlosser]

I do have something I want to note.. You are using HUGE frame buffers. The RGB driver allocates 2 full sized frame buffers for writing to the display. The buffers you use for LVGL can be a lot smaller Typical size is 10th of the size you are using. I say this because with the frame buffers alone you are using up 3,840,000 bytes of ram. if you are using the XIP config setting that is going to use up another grip of memory. probably around 3,000,000 bytes or so. IDK how much memory your ESP has available but you are using up a lot of it right out of the gate. Not that this is causing the problem at hand I just thought I should mention it to ya.

[Rai-Sp]

Hi, thanks for your answer.

First your second note: I know, i am using more than 3 MB (out of 8 MB) PSRAM and already tried to reduce it to 1/10, but the problem persists.

Here is the full output containing the backtrace information:

RGBBus <RGBBus>
display <RGBDisplayDriver object at 3c346800>
Touch Product id: 911
Touch Firmware version: 0x1060
Touch Vendor id: 0x0
Touch resolution: width=800, height=480
>>>
A fatal error occurred. The crash dump printed below may be used to help
determine what caused it. If you are not already running the most recent
version of MicroPython, consider upgrading. New versions often fix bugs.

To learn more about how to debug and/or report this crash visit the wiki
page at: https://github.com/micropython/micropython/wiki/ESP32-debugging

LVGL MicroPython
IDF version : fcae3288-dirty
Machine     : Generic ESP32S3 module with Octal-SPIRAM with ESP32S3

Guru Meditation Error: Core  0 panic'ed (StoreProhibited). Exception was unhandled.

Core  0 register dump:
PC      : 0x40056f60  PS      : 0x00060b30  A0      : 0x82023a6c  A1      : 0x3fcbe140
A2      : 0x00000000  A3      : 0x3c350904  A4      : 0x000bb800  A5      : 0x00000000
A6      : 0x00000000  A7      : 0x00000000  A8      : 0x00000000  A9      : 0x3fcbe160
A10     : 0x00060f23  A11     : 0x00000000  A12     : 0x00060f20  A13     : 0x3fca7a28
A14     : 0x00000000  A15     : 0x0000cdcd  SAR     : 0x0000000c  EXCCAUSE: 0x0000001d
EXCVADDR: 0x00000000  LBEG    : 0x40056f5c  LEND    : 0x40056f72  LCOUNT  : 0x0000bb7f


Backtrace: 0x40056f5d:0x3fcbe140 0x42023a69:0x3fcbe150 0x420238e1:0x3fcbe1b0 0x420236dc:0x3fcbe200

I tried to decode the backtrace and got (unfortunately without line numbers):

0x40056f5d: ?? ??:0
0x42023a69: rotate0 at rgb_bus_rotation.c:?
0x420238e1: copy_pixels at rgb_bus_rotation.c:?
0x420236dc: rgb_bus_copy_task at ??:?

Next, I created a disassembly around 0x42023a69:

42023a61:	87b8      	l32i.n	a11, a7, 32
42023a63:	0927a2        	l32i	a10, a7, 36
42023a66:	7c0b81        	l32r	a8, 42002a94 <_stext+0x2a74> (400011f4 <memcpy>)
42023a69:	0008e0        	callx8	a8
42023a6c:	003b46        	j	42023b5d <rotate0+0x15d>
42023a6f:	c79800        	ee.stf.64.xp	f9, f12, a0, a8
42023a72:	a788      	l32i.n	a8, a7, 40

It's certainly the memcpy in rgb_bus_rotation.c causing the crash.

[kdschlosser]

if you reduce the buffer sizes this problem here still comes up yes?

Have you tried to use a different RGB display board at all? I ask this because I have the exact same board you are using currently and I have no issues with it running. I am wondering if you may have gotten one with defective memory...

[Rai-Sp]

As I wrote before, reduction of buffer sizes to 1/10 does not change anything.

I only have a second display of exactly the same type, no other type of esp32s3 display board.
I will try my second copy of the same Waveshare board soon.

For now, a little more information:

I have modified rgb_bus_rotation.c to find out, which occurence of memcpy using which parameters causes the crash.
Renamed memcpy to x_memcpy and added a first argument "n".
Inserted x_memcpy with printf (added vTaskDelay to allow output to be sent before the crash occurs).

diff --git a/ext_mod/lcd_bus/esp32_src/rgb_bus_rotation.c b/ext_mod/lcd_bus/esp32_src/rgb_bus_rotation.c
index efb8ae0..184cb76 100644
--- a/ext_mod/lcd_bus/esp32_src/rgb_bus_rotation.c
+++ b/ext_mod/lcd_bus/esp32_src/rgb_bus_rotation.c
@@ -28,6 +28,11 @@

     #define RGB_BIT_0 (1 << 0)

+    void *x_memcpy (int n, void *__restrict a, const void *__restrict b, size_t c) {
+        printf("*** memcpy: %d %p %p %u ***\n", n, a, b, c);
+        vTaskDelay(10);
+        return memcpy(a, b, c);
+    }

     void rgb_bus_event_init(rgb_bus_event_t *event)
     {
@@ -278,7 +283,7 @@
                 } else {
                     rgb_bus_event_clear(&self->swap_bufs);
                     rgb_bus_event_wait(&self->swap_bufs);
-                    memcpy(self->idle_fb, self->active_fb, self->width * self->height * bytes_per_pixel);
+                    x_memcpy(1, self->idle_fb, self->active_fb, self->width * self->height * bytes_per_pixel);
                 }
             }

@@ -356,7 +361,7 @@
     {
         dst += ((y_start * dst_width + x_start) * bytes_per_pixel);
         if(x_start == 0 && x_end == (dst_width - 1) && !rgb565_dither) {
-            memcpy(dst, src, dst_width * (y_end - y_start + 1) * bytes_per_pixel);
+            x_memcpy(2, dst, src, dst_width * (y_end - y_start + 1) * bytes_per_pixel);
         } else {
             uint32_t src_bytes_per_line = (x_end - x_start + 1) * bytes_per_pixel;
             uint32_t dst_bytes_per_line = dst_width * bytes_per_pixel;
@@ -372,7 +377,7 @@
                 }
             } else {
                 for(uint32_t y = y_start; y < y_end; y++) {
-                    memcpy(dst, src, src_bytes_per_line);
+                    x_memcpy(3, dst, src, src_bytes_per_line);
                     dst += dst_bytes_per_line;
                     src += src_bytes_per_line;
                 }

The output now looks like:

RGBBus <RGBBus>
display <RGBDisplayDriver object at 3c346800>
Touch Product id: 911
Touch Firmware version: 0x1060
Touch Vendor id: 0x0
Touch resolution: width=800, height=480
>>> *** memcpy: 2 0 0x3c350904 768000 ***

A fatal error occurred. The crash dump printed below may be used to help
determine what caused it. If you are not already running the most recent
version of MicroPython, consider upgrading. New versions often fix bugs.

It says "*** memcpy: 2 0 0x3c350904 768000 ***" which means: the second occurence of memcpy with the first pointer == 0 causes the crash.

[Rai-Sp]

I just tried the second copy of Waveshare board: The same problem.

Remember: firmware built from commit 561cf5a
(based on micropython 1.25.0, LVGL 9.3.0 and esp-idf 5.4.0) still works as expected.

When I build from commit d7f69f3
(based on micropython 1.27.0, LVGL 9.4.0 and esp-idf 5.5.1) the problem occurs.

I still guess, it's something related to micropython 1.25.0 vs. 1.27.0, LVGL 9.3.0 vs. 9.4.0 or esp-idf 5.4.0 vs. 5.5.1.

[Rai-Sp]

May be, I should mention, that firmware built from commit 561cf5a for the "cheap yellow display board" works without any problems.

I guess, the crash is related to the different display architectures.

[Rai-Sp]

Sorry, I meant firmware built from commit https://github.com/lvgl-micropython/lvgl_micropython/commit d7f69f3 for the "cheap yellow display board" works without any problems.