Hi
I have started learning nim and so far I am liking it a lot and thinking about porting a medium sized project to nim. I expect it to be around 50k lines of code without counting in dependencies that could double that.
I want to know how long does it take to compile such a project on a not beefy laptop. knowing from previous experience how I tend to test my project very so often every time I make small changes. I am afraid my development time would be many times higher than if done using python or php just because of compilation time.
I have read that nim transpiler to c re-transcribes the whole project every time I compile it and doesn't support incremental build. It can even be too taxing on the hard drive.
so my question is. should I expect higher re-compilation times than 2s ?
cloc deps/znak
537 text files.
516 unique files.
28 files ignored.
github.com/AlDanial/cloc v 2.08 T=0.35 s (1463.8 files/s, 457619.5 lines/s)
-----------------------------------------------------------------------------------
Language files blank comment code
-----------------------------------------------------------------------------------
C 178 7987 10536 65281
C/C++ Header 174 4749 10224 38900
Nim 66 1143 539 7674
C++ 17 495 276 3227
Markdown 19 887 0 2349
CMake 5 225 224 1469
JavaScript 12 20 42 942
make 13 137 33 750
m4 1 83 45 697
Bourne Shell 4 69 90 482
Gradle 3 21 12 482
Windows Resource File 5 15 0 190
Python 1 33 45 167
HTML 2 22 2 166
Java 1 30 3 123
Starlark 1 20 2 122
GLSL 4 12 0 83
DOS Batch 1 24 2 74
XML 1 0 0 21
Properties 3 3 26 19
JSON 5 0 0 16
-----------------------------------------------------------------------------------
SUM: 516 15975 22101 123234
-----------------------------------------------------------------------------------first run with empty nimcache:
nim r deps/znak/znak build web 27,89s user 4,00s system 631% cpu 5,050 total
nim r deps/znak/znak build web 1,66s user 0,15s system 103% cpu 1,753 totalTarget is emscripten wasm build.
Thanks for putting in the effort
1.6s recompilation time is tolerable
Do you notice an increase in time when you make changes to the nim code? or does it stay the same ?
I work on a decent-sized project with a few dozen modules, a bunch of compile-time activity, lots of Nimja templates, a bunch of DataFrame manipulations, and even linking some single-header C projects. Plus I statically link against musl.
Without being at my desk, I know a fresh debug comp takes 5-10 seconds, and because Nim is very good at isolating compilation changes (since each module becomes its own unique C source file), a recomp with changes takes only 1-5 sec.
I also do release builds with LTO, which on its own adds maybe 5 sec.
Not sure what a "medium" project is, but a project like Nimbus takes ~2-3 minutes to recompile on a pretty decent 24-core machine with ccache and a bunch of other things to make it faster - the initial compile is much longer:
/nimbus-eth1$ make nimbus
Building: build/nimbus
real 2m48.438sThe nim compiler itself is single-threaded so for about half of that time, only one core is being used. Our experience is that compile time grows exponentially with project size, also because of how generics and symbol lookups share a massive global namespace - while it's pretty fast on small projects, once you start using a few libraries that are actually written in Nim and not just wrappers, it grows very quickly.
Well - most of the time is spent doing 3 things: processing async/closures, running macros/compile-time computations, and indeed looking up symbols due to typeclass-related things, according to both -d:profileVM and intel VTune.
For example, in the VM just like in generated code, zeroing things is a major source of slowness - if you create an array or object in the VM, it first zeroes it once or twice and then gives it the actual value - this costs a lot of of processing and the VM is rarely smart about this - in runtime code, we've finally gotten a usable newSeqUninit and some elision can be done by the C compiler when lucky but it's still a significant issue - in the VM, these opportunities remain to be explored.
async is slow of course because its introspection logic runs in the VM but also because we have a construct like when typeof(asyncBody) is void: asyncBody else: result = asyncBody - this is the only way we've found to support all the 3 ways of returning things in Nim (result, return and "expression") - this means that the compiler has to examine asyncBody 3 times at least - hopefully this is cached (?), but it's still 3 times that a "block" of code needs to be touched potentially expanded.
We do a lot of DSL-based serialization - this is slow because of introspection cost - for example, one of the more reliable ways to lookup a field is to use fieldPairs(default(T)) and iterate over all fields one by one to find the index of it by name and the offset by comparing addresses - here, we hit the VM slowness of course because even though we don't actually use / need a T instance, the VM goes and creates/zeroes one - very slow - better introspection would allow us to build a more efficient serialization library that still offers the same features, but since this doesn't exist, we have to use fieldPairs to build a name->code Table first and do lookups in there (to avoid quadratic algos in the VM) - it's faster to do it this way, but of course building the Table in the VM isn't great.
Finally, of course we could refrain from using language features (what's the point of having them, then?) or write large parts in a different language - I prefer to think though that there exist opportunities to make the compilation process faster in many ways and a project like Nimbus that actually uses a bit more Nim code is a great test case and source of inspiration for potential improvements:
A few things I think you're conflating.
Yes: Nim's fundamental scaling problem is that everything gets recompiled single-threaded. IC, native packages, and better modularity would push the boundary. I don't disagree with any of that.
But Nimbus is not "pretty representative" of Nim usage. It's an outlier that stress-tests the compiler in ways almost nothing else does.
Concrete numbers from our IC nimcache for nimbus_beacon_node:
light_client_processor.nim — 589 lines → 890 MB precompiled IR
light_client.nim — 437 lines → 295 MB
forks.nim — ~2200 lines → 99 MB (and ~800 MB RAM to compile)
Whole nimcache for one binary: 3 GB
The 890 MB module is mostly empty static hash table slots — millions of (uintlit … 0u) nodes. That's not "compile-time expansion paying off at runtime." That's compile-time expansion with no runtime payoff, paid on every build.
IC already works for compiler bootstrapping (~200k LOC). On Nimbus we've hit bug after bug for weeks. That pattern — works on normal code, breaks on Nimbus — tells you this isn't representative; it's pathological.
The "don't use Nim features" strawman isn't the choice. The choice is: use static generics and monomorphization where the fork is known at compile time, and runtime case kind / proc pointers in hub code that touches all forks. The file documents both approaches; one was chosen for hub modules anyway.
Saying "even if we shave 5s off light_client_processor it'll still be slow" misses the point. The question isn't whether a 589-line module can compile in 2s instead of 5s. It's why compiling it produces 890 MB of IR in the first place.
Nim is fast for what it does. Nimbus asks it to do absurd things. Conflating the two and calling it "representative" is how you end up profiling VM zeroing for months instead of fixing architecture that doesn't buy proportional runtime wins.
The 890 MB module is mostly empty static hash table slots — millions of (uintlit … 0u) nodes. That's not "compile-time expansion paying off at runtime." That's compile-time expansion with no runtime payoff, paid on every build.
Well, not quite - as I wrote above, if you need a default(T) for whatever reason, Nim generates a lot of runtime code to zero that instance at the call site. It's not "mostly" zeroes, it's "all" zeroes and its done one byte at a time.
In this particular case, it's not a hash table but rather an array of hash values. A hash is an array of 32 bytes - it's really nothing special, as far as types go - in fact, I'd say arrays of bytes are pretty common in programming.
This really shows up as a major performance cost if done in the wrong place. Hence, we move this zeroing to compile-time so that we don't get this nested for loop of byte-by-byte zeroing at runtime.
Sometimes, we indeed do it a bit too much because it's not something we want to spend time on analyzing, ie we bias the decision towards avoiding this silly runtime cost / bad codegen.
Of course, if the AST has an explosive representation of an array of bytes with their default values, this ends up being costly for the compiler.
What C compilers do in this case is that they generate no AST and no zeroes at all - a static global is assumed to be zero at init time and takes up no space - neither in AST nor in the final binary, nor when you access it at runtime (BSS) - it only starts taking up memory when you write to it - a problem here I guess is that this approach doesn't carry over to Nim (yet).
This optimization that's been around for quite a while now on the C side and would alone bring the size of light_client_processor down to almost nothing - and the memory usage of the compiler and so on - we're after all agreeing here that it's not generics or any other fancy things, but just .. zeroes.
Nevertheless, with the nim compiler today, it's a tradeoff we're willing to make, ie if the choice is between a byte-by-byte zeroing at runtime or compile-time, we'll choose compile-time when possible.
It's why compiling it produces 890 MB of IR in the first place.
Well, this is the crux of the matter. Is using arrays of bytes an "avoid" kind of thing? Or can we do something about the AST so that it's smaller for this fairly common case - common, because nim after all inits everything to zero by default.
It's not really that we don't know how to write a more efficient zero comparison - we could of course spend developer cycles on doing raw pointer comparisons with vectorized loops and perfect alignment and so on - it's more that it's such a trivial construct that we expect it to eventually become efficient in the compiler and when it does, all nim users will benefit, not just those that stress the compiler.
The choice is: use static generics
But the example you mention doesn't even use that much generics - it's zeroes, not monomorphization - so I'm not sure why the generics matter. I would guess that something similar happens for forks.nim - it computes a hash of types with fields of these byte arrays, so there's bound to be a lot of zeroing involved.
If we were to look, how much of forks' AST would be zeroes, ie array[32, byte] by expanded to take up at least 33 AST byte-by-byte nodes just to represent its default value?
Of course, when we write code, we're conscious of the fact that using generics will create more code and more work for the compiler, and we regularly review things to take out the worst offenders. It's just that almost every time we do it, it's a discussion about removing zeroing in some shape or form, so this is an obvious thing to address on the compiler side.
Nimbus is not "pretty representative" of Nim usage.
No, but it's pretty representative of a typical project size in other languages, once you using a few libraries to bring in features to you application that you didn't build from scratch.
This is an important point: since light_client_processor compile-time/memory bloat is almost exclusively made up of zeroing things, it also doesn't change between compilations - this is the essence of the scalability comment: with first-class packages/libraries, we can avoid this cost being repeated for every compilation.
If anything, it's more like boost in its early days when people discovered templates in C++ were turing complete and started using them for compile-time execution
True, that's a bit more apt. Just similar patterns when you get lots of smart people + new toys. ;)
You might want to revisit that benchmark since what you benchmarked was an early alpha commit done to play with the API - it wasn't hard to beat since it wasn't done for performance
Fair enough. Makes me curious to check out. The other CBOR library was almost identical at 1.7x, so it's reasonable ballpark for a first pass. Still cutting out layers of abstractions is hard.
we do compile-time expansion so that we don't have to hand-optimize things - it keeps the volume of code that people have to write low and lets the compiler do the heavy lifting - in a project like Nimbus that does a lot of computation, this pays off at runtime, for the user (at the expense of compile time)
Well we seem to agree mostly. My point was partly that "normal" Nim users aren't doing that sort of optimizations. You're likely adding a 3-4x amplifier to the Nimbus code. Great to not hand roll that work but it's also doing a lot more than it seems from the LOC.
If you're writing a CRUD server or data processing you're not doing such things.
we could go down the path of "just don't use any nim features" - sure - we could also use golang and compile a project of the size of nimbus in <10s. we could use dynlibs and other runtime boundaries to bring compile times down - this is similar to "just don't use nim features" - ie it misses the point of using Nim to begin with
Not using those Nim features wouldn't make sense. Personally adding the missing typetraits/compiler instrinsics to help speed them up makes more sense.
However dynlibs are a key part of "scalability" for larger projects which is why Swift recently got an ABI because Apple needed it in order to scale Swift now they're using it in WebKit and core OS libs. Nimbus seems like it'd surely have a natural spot for a couple of dynlibs.
ie you couldn't write a firefox or Qt in Nim in its current state - you can write a Nimbus if you accept minutes of compilation time - the original question in this thread was about where this boundary lies and Nimbus is probably pretty representative
Totally agree that Nim is hitting a "scalability" limit and without either (or both) a stable ABI and first class dynlibs or incremental compilation, then larger project become overly expensive. It's probably part of why Nim hasn't landed in many larger projects.
Though I'd also say between the "clever" code, Nim's 2x leverage on LOC versus say C++, Nimbus is closer to a large project. It does show there's a limit though.
Though I'd also say that Nimbus still compiles faster than a C++/ROS2 project I worked on which was solidly in the "medium" project size and took 3-4 minutes to rebuild. Man that was a pain.
Compared to Python loading almost instantly though Nim takes longer. However I'd still say stronger types avoids so many Python issues (even with type annotations) that it makes up for the compilation speed any day.
However dynlibs are a key part of "scalability" for larger projects which is why Swift recently got an ABI because Apple needed it in order to scale Swift now they're using it in WebKit and core OS libs. Nimbus seems like it'd surely have a natural spot for a couple of dynlibs.
Makes me a bit curious if the new Itanium C++ naming could be used to make proper Nim dynlibs. Not as a stable ABI, but at least supporting overloads, generic procs, etc. Currently a library like Genny has to manually handle all of that and it's a pain.
This thread has devolved into subjects I can't yet comprehend!
to keep it short I understand there is IC in the works, so I would like to know if it is going to land in nim 2 , when should we expect it? and how fast (theoriticaly) is it going to speed up compile time?
Frankly I started diving into nim because I liked the readability of python. But it turned out to be harder to grasp. The manual is quite lacking and at many occasions assumes the reader is familiar with the concepts it doesn't elaborate on. I wish there was a book of the style "Python cookbook" for nim.
So far I am stuck choosing between nim and golang. the later having impressive compile times, small syntax and a huge ecosystem gives it a great advantage over nim. sticking with nim is now most importantly a matter of how fast I can get hold of it and how confidently I can solve problems even if they originate from the dependencies.
There is of course no right answer when picking a language to code with. Either your business or your own taste will help with the decision :-)
To me, Nim is a much friendlier language than Go: the initial setup is much easier, the syntax is far less cryptic, and the ideology is less religious (Go is all about minimalism as if minimalism is something good in its own right; Nim just wants to be the best language for the human in front of the keyboard).
The official tutorials are a great way to learn Nim. Also, the book Nim in Action helped me. I haven't read Mastering Nim but I have no reason to doubt that it's a great book. Anyway, there are options.
to keep it short I understand there is IC in the works, so I would like to know if it is going to land in nim 2 , when should we expect it? and how fast (theoriticaly) is it going to speed up compile time?
It is already in Nim devel, expect it in one month with the 2.4 release. But for a 50K LOC project Nim's build times are not an issue anyway.
I was working on a frontend library until it got to the point of ~30 second compiles.
I spent days trying to find anything that would speed this up, I tried everything.
Eventually, I just gave up on Nim and my current plan.... Araq says I'm doing something out of the ordinary, but I don't feel so at all. If my usage of macros resulted in this (and there isn't some outright bug taking up resources), than macros are fundamentally flawed in what they allow to work. Where was my build warning?