Nim doesn't rely on headers but has to (re-)declare all the referenced symbols in each .c or .cpp file
C++ modules (https://clang.llvm.org/docs/Modules.html) aim among other things at improving Compile-time scalability,
Compile-time scalability: The std.io module is only compiled once, and importing the module into a translation unit is a constant-time operation (independent of module system). Thus, the API of each software library is only parsed once, reducing the M x N compilation problem to an M + N problem.
Could Nim benefit from upcoming C++ modules to speedup compilation times? eg by using C++ import foo instead of re-declaring each needed symbol This would speedup compilation, eg in non-separate compilation model, but possibly also separate compilation model too (eg with compilation server)
In the last few months I haven't really kept up with the C++ world, so I hadn't heard of this new feature. I see it was in fact proposed way back in 2014? (According to Wikipedia)...
Anyway, for this to work, several things must be considered
As tewtzel59 said, Nim usually compiles to C (and I guess that's what most people who program in Nim use). Even if modules were to be added to C, it wouldn't happen before 2022[1], which is when the new standard will probably be published. And even then, you'd have to wait until the major compilers implement the feature.
You probably already know this, but if you want to speed up compile times you can use tcc, which compiles 20x faster (!!!) than gcc and clang. Simply add
--cc:tcc@timothee - @amalek beat me to the punch, but user-visible compilation time is very sensitive to backend compiler/gcc options. Consider two invocations compiling my cligen test suite (30 programs with somewhat complex macros running at compile-time):
cligen$ ./test.sh
** ./test.sh * Time: 10.43s (u) + 0.92s (s)=11.05s (102%)
cligen$ ./test.sh -d:r
** ./test.sh -d:r * Time: 57.54s (u) + 2.85s (s)=56.96s (106%)
The Linux perf record/perf report tell me that in the former case Nim is about 82% and 14% in the latter case. So, in both cases Nim is 9 seconds or about 300 milliseconds per program (probably 10% more with kernel/libc activity...). That's not very "real-time" (like re-compile every keystroke), but it's pretty good. tcc is much faster (only 3% of the time for that faster run), but I think there are enough open issues with the Nim compiler that correctness should be the focus more than compile-time performance.
@amalek @cblake > if you want to speed up compile times you can use tcc
unfortunately tcc doesn't work on OSX, see https://github.com/wheineman/nrpl/issues/16 besides, this would come at a cost of runtime performance (but would still be good for some use cases eg nrpl) What I'm suggesting would have 0 negative impact on runtime performance, but would likely speedup compilation (by how much is to be determined).
@twetzel59 > What about C? Nim often compiles to C, not C++
that could just be an option to compile to C++ with C++ modules enabled; if it generates faster compile times in this case, that's good enough use case
@ Will Nim modules map directly to C++ modules? right now Nim doesn't support circular dependencies, so the module mapping should work. In case Nim starts supporting these, fwd declarations are an option.
@Araq > C++'s module feature is not gonna help us, since we got signature hashing most time is spent in the Nim frontend
on a small project:
nim c --compileOnly -f tests/t04_sample.nim
1.5 sec
nim c --noLinking -f tests/t04_sample.nim
2.6 sec
nim c -f tests/t04_sample.nim
2.8 sec
so we have: compiling Nim files=1.5 sec, compiling C sources = 1.1 sec, linking = 0.2 sec 1.1 sec is a significant fraction (40%) of the total time (2.8 sec)
so improvements there (as I suggested via C++ modules) could be meaningful, independently of improving the compilation of Nim files (eg via --symbolFiles)
@amalek > As tewtzel59 said, Nim usually compiles to C (and I guess that's what most people who program in Nim use). Even if modules were to be added to C, it wouldn't happen before 2022[1]
C++ modules already work today, I just tried. Eg, see https://stackoverflow.com/questions/48596805/linking-c-modules-ts-using-clang/49322001#49322001 for a hello world example. Even if still considered experimental feature, it'd be worthwhile to use them in Nim now (via an option flag) if it can provide compile time speedups.
@clabke > I think there are enough open issues with the Nim compiler that correctness should be the focus more than compile-time performance
compile time speed is pretty important for productivity (and turns of some D users I've spoken to when they tried out Nim and compared similar programs in Nim vs D). Obviously, correctness is also very important.
How difficult would this be to implement? My guess is probably not much: how about (as controlled via a new flag --experimental-cppmodule:
clang++ -c foo.cpp -o foo.o (etc)
clang foo.o foo2.o (etc) -o main
by:
opt=--std=c++17 -fmodules-ts -fprebuilt-module-path=.
clang++ $opt --precompile foo.cppm -o foo.pcm (etc)
clang++ $opt -c foo.pcm -o foo.o (etc)
clang++ $opt foo.o foo2.o (etc) -o main
on a small project
Yes, and things change for big projects.