[ERROR] Memory overflow when increasing fourth-order force constant cutoff

[Lamed-git]

When training fourth-order force constants using extract_forceconstants, increasing the fourth-order cutoff radius (-rc4) from 4.5 to 5.0 causes a memory overflow, despite successful execution at lower cutoffs.
Working command (completes normally):
extract_forceconstants -rc2 15 -rc3 6 -rc4 4.5 --stride 100
Failing command (memory error):
extract_forceconstants -rc2 15 -rc3 6 -rc4 5 --stride 100
The process fails with an out-of-memory error:
**Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0 0x2af7d480f3ff in ???
#1 0x55e1a42e7124 in ???
#2 0x55e1a430076e in ???
#3 0x55e1a4038f93 in ???
#4 0x55e1a3edebb4 in ???
#5 0x55e1a3eeb9fa in ???
#6 0x2af7d47fb554 in ???
#7 0x55e1a3ed8b78 in ???
#8 0xffffffffffffffff in ???**
I have 768 GB of memory and am using TDEP version 25.03. I'd like to know if there are currently any workarounds or solutions to resolve the 'out-of-memory' issue when increasing the cutoff radius for fourth-order force constants?
Thank you for your insights!

[mjv500]

Hello, What is unclear about this error? The phase space for fourth order scattering is huge, and increases combinatorially with rc4. Alois has optimized things very strongly, but if you keep increasing the cutoff it will explode the memory. Increase allocated ram and/or number of mpi threads, and above all check with smaller rc4 to see how close you are to convergence. Btw: you provide no information about your system size, number of atoms, grids, cpu, architecture, RAM etc, so there is little we can say. M. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Professor Matthieu J Verstraete Fellow, American Physical Society <https://www.aps.org> Chair, Steering Committee, European Theoretical Spectroscopy Facility <https://www.etsf.eu> Alumnus Fellow, Young Academy of Europe <https://www.yacadeuro.org> <http://yacadeuro.org/> Institute for Theoretical Physics (ITP) Department of Physics Buys Ballot Gebouw/Building, Princetonplein 5, office University of Utrecht, 3584 CC Utrecht Phone: +31 30 253 5926 Secretariat: +31 30 253 5929 and ***@***.*** Group web page (Liège): http://www.nanomat.ulg.ac.be/ Nanomat lab, Q-Mat center, Université de Liège Département de Physique, Bat. B5a, 4/50 Allée du 6 août, 19 B-4000 Sart Tilman, Liège Belgium Phone : +32 4 366 90 17 European Theoretical Spectroscopy Facility <https://www.etsf.eu> (ETSF) Mail : ***@***.*** ***@***.*** ***@***.***

…

On Wed, Jun 18, 2025, 04:27 Lamed-git ***@***.***> wrote: *Lamed-git* created an issue (tdep-developers/tdep#147) <#147> When training fourth-order force constants using extract_forceconstants, increasing the fourth-order cutoff radius (-rc4) from 4.5 to 5.0 causes a memory overflow, despite successful execution at lower cutoffs. Working command (completes normally): extract_forceconstants -rc2 15 -rc3 6 -rc4 4.5 --stride 100 Failing command (memory error): — Reply to this email directly, view it on GitHub <#147>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWOT37FZPLZ2NV3WVUW6A33EDFBXAVCNFSM6AAAAAB7RYXG32VHI2DSMVQWIX3LMV43ASLTON2WKOZTGE2TKMRVGQ4TQOA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Lamed-git]

Thank you for the quick response. I understand that fourth-order phase space grows combinatorially with -rc4 and that memory requirements become prohibitive and will rigorously test the convergence of the fourth-order cutoff distance. When encountering out-of-memory issues, should I consider single-core execution to avoid this problem?

[ejmeitz]

How many configurations are you trying to fit the force constants to? My experience has been that much more than ~1000 you will run into memory issues. (I only have 96GB or RAM though) See #122. It's also possible your material does not have many symmetries or that the symmetry was not identified properly which would make force constants more expensive than they would otherwise be.

This error message does not necessarily indicate memory overflow I don't think? Does htop show you running out of memory?