Hello everyone,
I would like to know if and how Nim could be used as a C replacement to create native dynamic system libraries which can be distributed and used like any library written in C. This is where all the supposed C replacements like Rust or Go fall flat. Only C++ so far seems suitable for that, but it's... well, C++.
Here is what I would like:
I have been able to accomplish the first two points. Here is the Nim source file:
# src/add5.nim
proc add5*(b: int): int {.exportc, dynlib.} =
b + 5I compiled it as nim c -d:release --noMain --header:a.h --app:lib -o:lib/a.so src/add5.nim, this gave me the dynamic library which I was able to load into a Python script:
from ctypes import cdll
a = cdll.LoadLibrary('./lib/a.so')
x = a.add5(4)
print(x)This works, but in order to make it properly distributable I need the header file. The --header:a.h option does create a header file in ~/.cache/nim/add5_r, but this file looks like an intermediate build artifact, not like something that can be distributed to #include in a C project.
/* Generated by Nim Compiler v1.6.0 */
#ifndef __a__
#define __a__
#define NIM_INTBITS 64
#define NIM_EmulateOverflowChecks
#include "nimbase.h"
#undef LANGUAGE_C
#undef MIPSEB
#undef MIPSEL
#undef PPC
#undef R3000
#undef R4000
#undef i386
#undef linux
#undef mips
#undef near
#undef far
#undef powerpc
#undef unix
N_LIB_PRIVATE N_NOCONV(void, signalHandler)(int sign);
N_LIB_PRIVATE N_NIMCALL(NI, getRefcount)(void* p);
N_LIB_IMPORT N_CDECL(NI, add5)(NI b);
N_LIB_IMPORT N_CDECL(void, NimMain)(void);
#endif /* __a__ */In particular, the included header file nimbase.h is stored under /usr/lib/nim/lib/nimbase.h, which is not a regular include path.
So my question is, is it possible to use Nim to create independent standalone dynamic C libraries? And if not, can this be added, or would such a feature be considered out of scope?
This is what we are trying to do with Genny: https://github.com/treeform/genny
You write some Nim code and it generates .dll/.so the .h file needed for C. And other languages like python, js, and ruby as well.
See my presentation about it at this time stamp: https://youtu.be/y6GQeWdmSb8?t=1180
See example Nim library in python: https://pypi.org/project/pixie-python/
For the sake of completeness and in case someone finds this in the future, here is how I compiled the library:
nim c --gc:orc -d:useMalloc --app:lib -o:lib/liba.so src/add5.nimHere is the C program to use the library:
#include <stdio.h>
int add5(int b);
int main(int argc, char *argv[]) {
int x = add5(9);
printf("%d\n", x);
return 0;
}I compiled and linked it as gcc test.c -Llib -la (-Llib adds the lib directory to the library search path, -la tells GCC to link with the liba file), then ran it as LD_LIBRARY_PATH=lib: ./a.out (set the LD_LIBRARY_PATH environment variable so the dynamic library can be found at runtime).
Old habits... yes --gc:orc and --mm:orc are the same thing.
Is there any downside to using -d:useMalloc?
Sometimes worse performance. Sometimes much better performance.
What about memory management, is there no need to include a garbage collector when using ORC?
There is no need for more than what orc builds for you. You can wrap the memory management like so:
type
Foo = object
...
proc copyFoo(a: var Foo; b: Foo) {.exportc, dynlib.} = a = b
proc destroyFoo(a: var Foo) {.exportc, dynlib.} = `=destroy`(a)
If you're careful and avoid top level statements in your DLL that need to run in NimMain then there is no need to call NimMain in the C code.
Is there any downside to using -d:useMalloc?
See this here #18612, should answer your questions. My tests match those presented by xflywind.
If you're careful and avoid top level statements in your DLL that need to run in NimMain then there is no need to call NimMain in the C code.
Are top-level statements all statements which are not just function/procedure-, type- or variable definitions? Basically anything that would not be allowed in the top level of a C file either.
See this here #18612, should answer your questions. My tests match those presented by xflywind.
I see, so performance can tank if I use dynamic dispatch, but that's considered a bug.
Nim int is 64 bit (in a 64 bit platfrom), the same as a C long.
So the C form should be:
long add5(long b);Or, if you like calling it with C int, the Nim form should be:
proc add5*(b: cint): cintBasically anything that would not be allowed in the top level of a C file either.
Correct.
This is not all though, after calling a Nim proc from C you need to check the errorflag and make sure an exception wasn't raised, (api missing, bug <#18215 https://github.com/nim-lang/Nim/issues/18215>_
Also in case there are globals you need to free them afterwards
This is not all though, after calling a Nim proc from C you need to check the errorflag and make sure an exception wasn't raised...
Not if your exported procs cannot raise. And turning an exception into an error code for the C API is really cheap and can be done easily in the Nim code (try ... except).