Crash under GPU load

[make-your-soft-better]

Wayfire version

0.10.1-746bc7e9 (Apr 10 2026) branch makepkg wlroots-0.19.3

GPU / Driver

$ glxinfo -B
name of display: :1
display: :1  screen: 0
direct rendering: Yes
Memory info (GL_NVX_gpu_memory_info):
    Dedicated video memory: 8188 MB
    Total available memory: 8188 MB
    Currently available dedicated video memory: 7381 MB
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: NVIDIA GeForce RTX 4060 Ti/PCIe/SSE2
OpenGL core profile version string: 4.6.0 NVIDIA 580.142
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.6.0 NVIDIA 580.142
OpenGL shading language version string: 4.60 NVIDIA
OpenGL context flags: (none)
OpenGL profile mask: (none)

OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 580.142
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

$ vulkaninfo --summary
Vulkan Instance Version: 1.4.341
Instance Extensions: count = 24
-------------------------------
VK_EXT_acquire_drm_display             : extension revision 1
VK_EXT_acquire_xlib_display            : extension revision 1
VK_EXT_debug_report                    : extension revision 10
VK_EXT_debug_utils                     : extension revision 2
VK_EXT_direct_mode_display             : extension revision 1
VK_EXT_display_surface_counter         : extension revision 1
VK_EXT_surface_maintenance1            : extension revision 1
VK_EXT_swapchain_colorspace            : extension revision 5
VK_KHR_device_group_creation           : extension revision 1
VK_KHR_display                         : extension revision 23
VK_KHR_external_fence_capabilities     : extension revision 1
VK_KHR_external_memory_capabilities    : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2         : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2       : extension revision 1
VK_KHR_portability_enumeration         : extension revision 1
VK_KHR_surface                         : extension revision 25
VK_KHR_surface_protected_capabilities  : extension revision 1
VK_KHR_wayland_surface                 : extension revision 6
VK_KHR_xcb_surface                     : extension revision 6
VK_KHR_xlib_surface                    : extension revision 6
VK_LUNARG_direct_driver_loading        : extension revision 1
VK_NV_display_stereo                   : extension revision 1

Instance Layers: count = 2
--------------------------
VK_LAYER_NV_optimus NVIDIA Optimus layer 1.4.312  version 1
VK_LAYER_NV_present NVIDIA GR2608 layer  1.4.312  version 1

Describe the bug

With a high GPU load, sometimes Wayfire suddenly ends the session. It's quite difficult to track the exact cause, but it always happens under high (and even medium) GPU load, for example: Minecraft with shaders, render in Blender.

Wayfire crashes precisely when the GPU is under load.
Wayfire crashes suddenly, then, after few seconds, SDDM shows.

This is very annoying because playing games turns into Russian roulette, and rendering projects at work turns into a nightmare.

I suspect it's something like a malloc error (returns NULL pointer and use it after that, although stable systems should handle the returned NULL pointers without crash and using NULL pointers) when there is not enough video memory or something like that, but I don't know for sure.

What to do?

Should I try to start rendering in Blender when Wayfire is running with debug? But which flags should I use? Something like this? WAYLAND_DEBUG=1 WAYFIRE_DEBUG=1 wayfire 2> wayfire.log

[ammen99]

I have recently given up on nvidia for this precise issue, the driver will crash randomly :( I am back to AMD iGPU for now.

[ammen99]

I mean that the bug is in the nvidia vulkan driver.

[ammen99]

Actually, are you running with vulkan? I assumed so because of your vulkaninfo, but maybe upload the full log file as you suggested.

[make-your-soft-better]

I noticed that Blender under OpenGL backend (and not under Vulkan) most likely does not crash under high load... It seems that this is indeed a problem with Vulkan... I also noticed that Luanti (MineTest, using Vulkan) lags horribly under Wayfire. It seems the same problem?

[ammen99]

better use gles2 with nvidia

[make-your-soft-better]

Alarm! Blender just crashed even with OpenGL Backend right now. This doesn't look like a simple driver error anymore... Looks more like this, something with video memory:

I suspect it's something like a malloc error (returns NULL pointer and use it after that, although stable systems should handle the returned NULL pointers without crash and using NULL pointers) when there is not enough video memory or something like that, but I don't know for sure.

[make-your-soft-better]

Also, I play Minecraft on the version what use OpenGL.
Now the Wayfire died again while I was trying to build something: first there were strong lags, and then a crash.
By the way, Blender has a similar problem - it strong lagging for 1 second before Wayfire crash.
And it doesn't look like an overflow of regular RAM, because Minecraft is very limited in this regard (I intentionally limited RAM for JavaVM).
Most likely, the problem is with overflowing video memory (graphics card memory)

[ammen99]

Yes, Wayfire doesn't handle that gracefully. If you want, we can work on fixing some of the common cases, but you'd need to compile Wayfire with address sanitizer, collect the crash logs and post the stacktrace here, so that I can see where we need to add more checks :) It will likely be an interative process - I'll fix one place, then you'll test, discover another crash, and so on. The reason for this is that I currently don't have access to my nvidia gpu, and on my iGPUs this problem does not happen - it uses system RAM up to the point where the entire system freezes.

[make-your-soft-better]

Now I'm going to talk nonsense that I took out of my head, but it may be useful in some way.
I don't want to offend anyone.

I'll fix one place, then you'll test, discover another crash, and so on

Or, perhaps, it is worth finding all the lines with the allocation of video memory, and replacing them with a custom function that will allocate video memory and check for allocation errors... I don't know what it will look like yet (I haven't read the code yet), but I just thought it would look something like this:

Was:

gpu_memory_pointer* pointer = mallocGPU(128); // Allocate 128 bytes in GPU
some_func(pointer); // Use allocated memory... **Or crash if pointer is NULL (can't allocate memory)**

Fixed:

// Function will return only allocated GPU memory
gpu_memory_pointer* mallocGPUSafe(size_t size) {
  gpu_memory_pointer* pointer = mallocGPUSafe(size);
  while (!pointer) { // If NULL (can't allocate video memory) then do some anti-garbage stuff and then try to allocate again
    anti_garbage_stuff();
    pointer = mallocGPUSafe(size);
  }
  return pointer;
}
...
gpu_memory_pointer* pointer = mallocGPUSafe(128); // Allocate 128 bytes in GPU safely
some_func(pointer); // Use allocated memory (It's always not NULL here)

Perhaps this approach will not fix some of the problems, and there will still be lags... But according to my assumptions, the loading program (Minecraft, Blender) will crash, but not the entire Wayfire with every another working program.

Again, I'm sorry if you already using something like this, I just sharing my thoughts.

[soreau]

A couple things come to mind when talking about problems when the gpu is under load. This can point to some hardware issue such as overheating, a graphics driver bug in the kernel or userspace drivers. You might check the kernel log (i.e. dmesg) after this happens. Another idea is to run the offending program using strace.