Thanks for your awesome articles.
It would be great to continue nim multi tasking articles (Multi-threading,Communication)
+1 to this
I'll be happy to see the Multi-threading article.
Hello! Thank you for your work.
Is it possible to separate generated bindings to different files based on header .h files? I am trying to use Futhark on ESP-IDF but the the generated .nim file/bindings from one header file also contains all of the dependencies from #include "..." included headers. Also is there an option to ignore certain header files?
It should only include the parts used by your program which Nim needs to be able to properly type everything. When you use path in your importc block everything from that path which is included will generate bindings, if you use sysPath the path will be searched but no bindings will be generated (as long as they aren't used elsewhere). There is currently no system to use path and ignore only specific files (apart from generating the data manually, the importc macro actually just creates a call to an implementation macro which takes arrays, so you can write your own importc implementation which also calls the underlying macro with a different DSL).
In theory it would be possible to not generate bindings for things which only gets pointed to by pointers. This would reduce the amount of stuff the binding creates, but also make the binding less complete. Why exactly do you want to achieve this? Nim has dead code elimination, so having too much stuff bound shouldn't cause any issues.
Intriguing.
Did you have to deal with a build system to get CoAP working?
@PMunch nice writeup! I look forward to reading more. Any plans to do write ups on dynlib's?
@adokitkat I'm planning at some point of setting up Nesper to use cimporter. It automates c2nim by taking some ideas from Futhark. Importantly, it uses the C compiler's built-in preprocessor to handle defines so it's able to parse almost all C code. There's an example project I'm using it to wrap ROS2 libraries. There's still tweaking required but it's sort of meant to be a middle between Futhark and c2nim where you still want to tweak the headers.
And I don't want to rewrite my C code to make it pass through c2nim, because that introduces human error risks.
Well the "editing" consists to 99% of "removing junk that won't result in Nim code anyway" so there is hardly any "error risks" involved.
I hope that's not all you think of tools like Futhark. As I mention in the article I spent a week trying to get c2nim to work (with 8 hour workdays).
In the same time period I wrapped most of wxWidgets which is C++, not C, with c2nim back in the day. Different people have different experiences. I'm biased but I've seen plenty of people using c2nim successfully, not just me.
I think tools like Futhark might be doing more harm than good for Nim's ecosystem. The same is true for c2nim. Python doesn't concern itself with C interop at the Python language level at all. Python's exception handling is not mapped to C++'s. The Python interop story with C/C++ is much worse than Nim's! Right? So why is Python's ecosystem much better, even when it comes to wrappers? There is Qt for Python, wxWidgets for Python, Tensorflow, etc. Sure, Python is more popular but any of these wrappers was much more work than it would have been for Nim. Or wasn't it?
The key lessons to take from Python (or Ruby for that matter) are these:
And boom. Python can remain a language without a gazillion of interop pragmas and every package adheres with Python's reference counting scheme and exception handling implementation, even if suboptimal. It doesn't need ever more pragmas in order to deal with e.g. C++'s shiny new coroutines and concepts features. Python doesn't care about C++'s coroutines. Instead you can write C++ coroutines code that call into the Python API for Python's generators.
Now, can you do the same for Nim? Write the wrappers in the host language and use a Nim C API/ABI in order to make it all work out. No, currently you cannot because Nim has no C API/ABI and its 4 different production GCs don't help, neither does the fact that Nim can produce both C or C++ code. Or its 3 different exception handling implementations.
I think it's high time we rethink the problem. Adding a C API is not that hard, but I've been completely blind to it for ideological reasons.
c2nim does that, yes. No idea why you are all so keen on using a C preprocessor to throw away lots of useful information.
@Araq haha spoken like a compiler developer: every token is precious! ;) More seriously the C Macros tend to just add crap on top of the C code like fake-namespacing or C compiler compatibility that's not needed in Nim.
As a general note almost every C macro I've manually converted was done better with proper Nim features after expanding the raw C macros. They're just not worth it, except in a few cases where you need to manually write them anyway to get the types.
To the point @Pmunch makes it's more about just getting to use the C project. I prefer spending time making a nice Nim api on top of the raw C one than trying to tweak the C bindings themselves to be nicer.
dynlib is definitely something I should do a writeup on at some point, but I think I want to get this fixed first: https://github.com/nim-lang/Nim/issues/21403
Oh, I didn't think of that sort of thing. Mostly I use one-time load dynlibs.
I think it's high time we rethink the problem. Adding a C API is not that hard, but I've been completely blind to it for ideological reasons.
What do you mean by this?
As a general note almost every C macro I've manually converted was done better with proper Nim features after expanding the raw C macros. They're just not worth it, except in a few cases where you need to manually write them anyway to get the types.
So why not use c2nim's #def then? I honestly don't get it, I always have a config.c2nim file where I collect all the switches, the name mangling rules and #def etc.
Well the "editing" consists to 99% of "removing junk that won't result in Nim code anyway" so there is hardly any "error risks" involved.
This is fairly naïve. Sure if you're proficient in C and Nim then maybe you'd have a fairly good grasp of what can be removed without any issues. But hiding behind layers of macros might be a calling convention attribute which suddenly means that your able to build it on your machine, but a different machine defaults to the wrong calling convention and now you have subtle bugs. Just an example of course, but I'm pretty confident in saying that a C compiler is better at reading C code than any of us.
In the same time period I wrapped most of wxWidgets which is C++, not C, with c2nim back in the day. Different people have different experiences. I'm biased but I've seen plenty of people using c2nim successfully, not just me.
I honestly have no idea how you managed to do this. Every time I've tried using c2nim I've struggled for ages and never gotten anywhere. Doesn't help that there are very little docs on how to use it I guess. And as I'm chipping away at one error after another I have no idea if it will start working after the next one or if I'm 10000 changes away from a working wrapper. With Futhark I've been able to use a complex library like Gtk with WebKit2 support (which none of the existing wrappers had) in a matter of minutes. And when I needed to use the JavaScriptCore library as well it was literally adding three lines of code and it just worked.
I think tools like Futhark might be doing more harm than good for Nim's ecosystem. The same is true for c2nim. [...]
This is an interesting point, and I don't necessarily disagree. Of course Nim has the speed of C and since it compiles to C it feels like a strange restriction to have to force every library to convert things to Nim types. It would also mean a substantial performance loss which isn't as apparent when you're working with Python. Besides, I write Nim code because I enjoy writing Nim code more than I enjoy writing C code. This is of course an argument with no technical merit, but as @elcritch points out I'd much rather spend my time building a Nim API on top of a C library in Nim than having to build it in C. Futhark focuses on simply wrapping a library as directly as possible, and then you as the library owner should build the Nim API abstractions on top of this. This is what I did with libCoap and I probably wouldn't have had any energy left to build the async API and automatic destructors if Futhark hadn't built a complete wrapper for me instantly before I even started looking at the API.
With enough contributors and man hours I'm sure it would be possible to hand-craft the most exquisite wrappers. But Futhark isn't the antithesis to this, but rather a tool to skip the painful step of just getting a C library to work in Nim.
I'd much rather spend my time building a Nim API on top of a C library in Nim than having to build it in C.
That means that you need tooling support for generating this C(++) wrapping code. Not a bad thing to have but regardless of whether you generate or hand-write the wrapper code it means Nim itself can be significantly simpler because we can avoid plenty of interop pragmas.
That means that you need tooling support for generating the C(++) wrapper code.
Not sure what you mean by this. Isn't this exactly what Futhark does?
There's a reason Futhark doesn't yet support C++, it's quite a bit more of a complex beast than C.