Hi all,
I'm working in the creation of a C-bindings nim library. The library is built with nim c ....
This library integrates correctly in a C example but I have issues to make it properly work on another environments. Concretely, I'm interested in NodeJS.
It is quite evident that there is a conflict between the two GC's. The issue I'm facing is that a certain nim ptr becomes invalid due to the NodeJS GC "touching" it (I also experienced a similar problem in Golang.)
How could I achieve a good integration of a C-nim based library into another GC languages such as NodeJs or Golang?
Nim version: Nim Compiler Version 1.6.11 [Linux: amd64]
For the sake of understanding the issue better, I created a simple integration example.
Nim library code:
import chronos
var a {.threadvar.} : seq[string]
proc allocate_obj() {.dynlib, exportc.} =
a = newSeq[string](10)
for i in 0..<10:
a[i] = "value"
proc do_sth() {.dynlib, exportc, async.} =
echo "Hello"
for i in 0..<10:
echo a[i]
asyncSpawn sleepAsync(chronos.milliseconds(0))
poll()
nim c --out:libwakusimple.so --app:lib --opt:size --noMain --header --verbosity:0 --hints:off -d:release libwakusimple.nim
In order to create the NodeJs module I use the N-API C API framework with the next code:
#include <assert.h>
#include <node_api.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include "libwakusimple.h"
// As a convenience, wrap N-API calls such that they cause Node.js to abort
// when they are unsuccessful.
#define NAPI_CALL(call) \
do { \
napi_status status = call; \
if (status != napi_ok) { \
napi_fatal_error(#call, NAPI_AUTO_LENGTH, "failed", NAPI_AUTO_LENGTH); \
} \
} while (0)
static napi_env my_env;
static napi_value InitAddon(napi_env env, napi_callback_info info) {
NimMain();
allocate_obj();
return NULL;
}
static napi_value Do(napi_env env, napi_callback_info info) {
do_sth();
return NULL;
}
#define DECLARE_NAPI_METHOD(name, func) \
{ name, 0, func, 0, 0, 0, napi_default, 0 }
static napi_value Init(napi_env env, napi_value exports) {
napi_property_descriptor init = DECLARE_NAPI_METHOD("init", InitAddon);
NAPI_CALL(napi_define_properties(env, exports, 1, &init));
napi_property_descriptor dosth = DECLARE_NAPI_METHOD("do_sth", Do);
NAPI_CALL(napi_define_properties(env, exports, 1, &dosth));
return exports;
}
NAPI_MODULE(NODE_GYP_MODULE_NAME, Init)
The NodeJs module is created with the next commands:
npm -i bindings <- to be called once
node-gyp configure <- to be called once
node-gyp build
Next, we have the waku_simple.js code that loads and uses this module:
console.log("Running")
var wakuMod = require('bindings')('waku');
wakuMod.init()
const run = async () => {
setInterval(() => {
wakuMod.do_sth()
}, 0)
}
run()
And I run the above code with the next command:
node waku_simple.jsWhere the previous run generates a core dump after a few seconds running, that looks like:
Core was generated by `node waku_simple.js'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f3f4fcd073a in rawAlloc__system_5123 (a=a@entry=0x67f2ad8, requestedSize=requestedSize@entry=120) at /home/ivansete/.choosenim/toolchains/nim-1.6.12/lib/system/alloc.nim:735
735 c.freeList = c.freeList.next
[Current thread is 1 (Thread 0x7f3f500717c0 (LWP 101111))]
(gdb) print c.freeList.next
Cannot access memory at address 0x65756c6176
I can bring more details if needed.
Thanks in advance!
Well you don't call NimMain at startup but you need to. And then there a gazillion things that can still go wrong, use at least --mm:arc or --mm:orc and a Nim version that is actually officially supported because "Nim Compiler Version 1.6.11" surely is not.
And sorry but a newcomer (that you seem to be) shouldn't attempt these stunts. Use Nim on its own first and write all of your project in it. Mixed language development is a productivity killer, esp bridging between runtimes with different memory management schemes. Nothing is ever type checked and you spend hours in valgrind to ensure it's correct.
when it comes to intermingling managed code in which multiple memory management schemes have no actual knowledge of each other, you need to use the language's built-in utilities to preclude object destruction. C#, for instance, has `unsafe` contexts and `unmanaged`.
on the nim-side of things, we can protect managed references in ORC/refc/etc. via `GC_ref/unref`. to make sure we are a good citizen and deallocate memory appropriately, in, e.g. node/NAPI, you might want to try using a cleanup hook callback. golang similarly affords a finalizer API. this really depends on the language environment you are meshing together with.
Thanks for the rapid response and the feedback! I will check everything carefully.
The code I shared is a bit messy due to the N-API layer but it actually calls NimMain.
wakuMod.init()
...
static napi_value InitAddon(napi_env env, napi_callback_info info) {
NimMain();
allocate_obj();
return NULL;
}
For building .node addons (for NodeJS and Bun). Be sure you have choosenim.
nimble install denim.
Check it on GitHub / Documentation
Hello world example:
import pkg/denim
init proc(module: Module) =
module.registerFn(2, "hello"):
# `Env` to use the current `napi_env`
# `args` to retrieve function arguments
echo args.len
if not Env.expect(args, "MyApp", ("name", napi_string), ("?age", napi_number)):
# Check for types. prefix function arg with ? to make it optional,
# if not matching, will throw a NAPI Error
return
# Use %* to convert Nim types to NAPI
return %* ("Hello " & args[0].getStr)
Now, use denim cli to build your .nim
denim build myapp.nim
In JavaScript
const app = require('myapp.node')
app.hello(true)
// error: Type mismatch parameter: `name`. Got `boolean`, expected `string`
app.hello()
// error: Type mismatch parameter: `name`. Got `undefined`, expected `string`
let str = app.hello("World!")
console.log(str)
On todo list:
Wanna help? :)