Mirror of forum.nim-lang.org

7097 :: out of memory

• kobi (orginal) [2020-11-15T23:13:22+01:00] view original

  Another question: My program exits with an "out of memory" failure.

  I'm on linux, and I have ~ 8gb of ram on this computer. The program is parsing files in a directory, collecting all this info in a tree, and then generating from this tree. Now, the current folder is pretty big, almost 36,000 files.

  How can I know what the culprit is? is it the design? so many objects in memory, or is it just a silly mistake

  What are the usual tools you use to find where all this memory went to? Thanks!

---

• Araq (orginal) [2020-11-16T16:37:04+01:00] view original

  Inject echo getOccupiedMem() into strategic places in your program.

  Or maybe collect the information in a database.

  Or maybe by not collecting everything into a single tree.

  Usually there is a correlation between what you do and the amount of memory it requires, the more memory it takes, the more important it is and you don't want to live in this fragile world where you have a single process that loses all the work when it crashes. Some incremental approach can really work wonders here.

---

• snej (orginal) [2020-11-16T20:40:34+01:00] view original

  There are likely some Linux tools you can use to inspect a process's heap. I don't develop on Linux, but for example, on macOS I would turn to tools like heap and MallocStackLogging.

---

• tmsa04 (orginal) [2020-11-16T22:13:27+01:00] view original

  I've used Valgrind on C programs, which worked well on those. I think you just ran the app like valgrind --some-args appname after compiling it with debugging symbols, and it tells you if there are any memory leaks and what functions they originate in, so it's pretty simple to use. Never tried it with Nim! I think it's possible from what I have read. There's a stack ulimit set on most Linux distros, but that would probably give a stack overflow error not an out of memory error.

---