Windows
Nim version: 2.2.0
Following code is trying to demonstrate that any allocation inside threads leak memory with default memory allocator! I understand that i am supposed to use a pool of threads, but following code doesn't leak memory with -d:useMalloc. I am trying to ascertain if there is a bug in Nim default memory allocator or am i not understanding some basic concept!
I have written a lot of code with typed threads, i just was careful to allocate (shared) memory in the main thread before accessing that from threads. With enough work this approach has always scaled nicely , but sometime i may want to use default Nim data-structures from a thread, without writing a lot of code from scratch to get around to this limitation! Any help is appreciated.
when defined(windows):
import std/winlean
import os
proc test_leak(){.thread.}=
var tmp = newSeq[int](1_000_000)
let arr = allocShared0(1024 * 1024)
deallocShared(arr) # doesn't release !
# reset(tmp) # Doesn't work !
for i in 0..<100:
var thr:Thread[void]
createThread(thr, test_leak)
joinThread(thr) # wait for it to finish
when defined(windows):
# have to do this manually.. https://github.com/nim-lang/Nim/issues/23350
discard closeHandle(cast[Handle](thr.sys))
os.sleep(1000)
As far as i understand, default allocator is thread-safe !
If you have no leak with -d:useMalloc, it doesn't seem like you're actually leaking memory.
The allocator may keep released memory around to reuse it without going through the OS.
Well yes, what you describe here is a well known limitation of the current allocator. Use -d:useMalloc as a workaround or stop pretending that threads are cheap and use one of the available thread pools.
The limitation isn't that easy to fix ("dying thread needs to return memory back some other thread which might be busy?").