I've been trying to find a memory leak. The program is processing data retrieved over the net, uploading results to another site, and runs for days at a time with multiple copies running at once. There is no database. After a few days, the memory usage increases to a point then the program stops working. On cntrl-c it reports stuck at execCmEx() .. this may or may not be the source of the problem, only it is where it halts.
I have no experience with debugging tools so I added a function to log how much memory the program is using line by line. Discovered the memory jumps on strutils.find() .. it can be replicated to the same line each time, when done within my program, but when trying to create a test program that uses the same data and code section, it does not leak. So something else is involved. The memory leak is both consistent and inconsistent - consistent it will process about the same amount of data before crashing, but inconsistent in that the memory usage increases only rarely, like maybe by 50Mb every few hours in a single big jump, like a stair case. This suggests a problem in the data, but the data is very simple text and I checked it out when it jumped there is nothing unusual. Adding the memory debug statements caused it to move a little to a different nearby occurrence of strutils.find()
Two questions: is there anything in strutils.find() that might conceivably cause a memory leak? What method would you use to figure out where memory is leaking for something like this that is so intermittent and long-term (I have zero experience with debuggers so any practical pointers appreciated).
Same behaviour when compiled in debug mode, and with -d:release and -d:danger?
Can you try another GC, maybe boehm?
Try another GC, I know that there are (were) some leaks on small memory allocations that were not returned to the OS.
For full debugging, compile with --debugger:native -d:useMalloc and pass the program through Valgrind. That will replace all the GC calls by system malloc and Valgrind will be able to do its magic. You might want to compile in release mode but with --stacktraces:on for speed (or maybe stacktraces don't need to be on with debugger native, not sure)
Are you using async await in your program?
In any case, the best way I've found to diagnose memory leaks for long-running programs is by logging the memory usage of each object to something like prometheus. My prometheus package can do that automatically for you, but you'll need to set it up and have a web server running in your process for it to work properly. If you're not using async then this probably isn't the best approach.
You can alternatively just print out the same data that my package collects, see how I do this here: https://github.com/dom96/prometheus/blob/master/src/prometheus/collectors/gccollector.nim#L84. The dumpHeapInstances function gives you the raw data, but there is one that returns a human-readable string as well if you look around. You can just call it to get how much memory each type is using.