I wrote a little test case - it's not real code, but I'd really like to understand what goes wrong here.
state.nim - compile with nim c state
import tables
type
arr* = ref array[128 * 128, int64]
testState* = ref tuple[
mem1: seq[arr],
mem2: Table[string, arr]]
main.nim - compile with nim c main
import dynlib, state
type initProc = proc (s: testState) {.nimcall.}
let handle = loadLib("./lib.dll") # repalce with liblib.so on linux
if handle.isNil: quit("Couldn't load DLL")
let init = cast[initProc](handle.symAddr("initialize"))
let s = new testState
init(s)
lib.nim - compile with nim --app:lib c lib
import state, tables
proc initialize (s: testState) {.exportc, dynlib, cdecl.} =
s.mem1 = newSeq[arr]()
s.mem2 = initTable[string, arr]()
for i in 0..100:
echo i
s.mem2[$i] = new arr
s.mem1.add(new arr)
On my system this crashes after ~16 iterations due to illegal storage access with a traceback to gc.nim:growObj or tableimpl.nim:rawInsert.
This does not happen when I move the allocations to main.nim.
Is there a way to do stable heap allocations within procs I sym-linked from a DLL?
Any pointers are greatly appreciated.
Nim version is 0.17.0, compiled it on win10x64 using x86_64-w64-mingw32-gcc, and confirmed the crash happening on a linux machine as well (where it too crashed after 16 iterations).
I'm unable to find any docs mentioning NimMainModule, and just putting NimMainModule() at the beginning of the DLL source gives me an undeclared identifier error. Could you please elaborate on how to manually run NimMainModule?
Also, I take it that on a first glance, at least you didn't see any glaring flaws in the code above? Would you say it should run in theory?
From Nim Compiler user guide:
Nim supports the generation of DLLs. However, there must be only one instance of the GC per process/address
space. This instance is contained in nimrtl.dll. This means that every generated Nim DLL depends on
nimrtl.dll.
To generate the "nimrtl.dll" file, goto Nim/lib directory and use the command:
nim c -d:release nimrtl //if nimrtl.nim.cfg is available
or
nim c -d:release --app:lib -d:createNimRtl nimrtl
then compile your project with:
nim --app:lib -d:useNimRtl c lib
nim c -d:useNimRtl main
nimRtl not used if there is only single instance of GC, such as C app + Nim DLL, or Nim app + C dll. nimRtl used in situation like Nim app + Nim DLL, or C app + multiple Nim DLL
it is not necessary to call NimMain() in above scenario. call NimMain() only needed if C app linked statically against Nim code, or you generate your dll with --nomain
Oh, that explains a lot, thanks jangko.
Though by following your steps, I run into another error message..
lib\system\threads.nim(431, 8) Error: implementation of 'system.deallocOsPages()' expected
That doesn't sound like something I have control over. Any idea? Can someone reproduce this?
again from Compiler User Guide
Note: Currently the creation of nimrtl.dll with thread support
has never been tested and is unlikely to work!
avoid using -d:useNimRtl with --threads:on
Alright I have no idea why thread support has been enabled. Shouldn't it be disabled by default?
Only when explicitly specifying --threads:off, it compiles. No nim.cfg other than the default one used. Strange.
Well nevermind, it works now, the allocations cause no more crashes! Thanks a lot jangko.
Though I am actually very interested in using threads at some point in the future. Hopefully someone will be able to implement this because without it, for example Nim's support for hot loading (compiling changes without having to terminate & restart the process) is pretty restricted.