This is a question as a category, but it's actually a report of what I've done. (If there are any mistakes, I would be very happy if you could specify them.) There are a few stories on github/Nim issues and forums about using ccache, but this will be a report of how much it specifically improves, measured in my environment, so it's something that advanced Nim users(or People who still use C language frequently) know about after all these years. I know this is a new and familiar story to advanced Nim users, but I would like to introduce this as it has the potential to save a lot of people time and CPU energy.
Nim can make very good use of optimized peripheral tools such as gcc and clang.
This reduces build time when using ccache. https://ccache.dev/
The following is an introduction to ccache. (Some excerpts)
My annotations: Works with major environments and compilers supported by Nim.
My annotations: Avoids copying files, thus improving TBW indicators such as SSD. and of course, speed.
Another reason to use ccache is that the same cache is used for builds in different directories. If you have several versions or branches of a software stored in different directories, many of the object files in a build directory can probably be taken from the cache even if they were compiled for another version or branch. A third scenario is using ccache to speed up clean builds performed by servers or build farms that regularly check that the code is buildable.
My annotations: Due to the deeply thought-out nature of nim, the nim file is translated into one (or more?) c source code, but the same modules used in different projects are not shared (Perhaps simply spec'd for a complete and full build). For example, lib/system.nim is compiled each time. Of course nimcache is cached for a project, but it is not shared between different projects.(by default).
That R*** language is very slow to compile in that respect, and its deep dependence on llvm in the early stages (Recently, implementations have been created that also support gcc, but...still unstable & experimental) contrasts greatly with Nim's advantage when compared to Nim's compilation, which is very fast as is.
# To measure time, aloso a simple time command would be fine.
$ alias t="/usr/bin/time -f 'time: %E, memory: %MkB, exit: %x'"
# The example below is the source for the Nim compiler itself, but it works equally well for large projects.
# However, Even small sources can be expected to operate efficiently.
# ubuntu/debian
$ sudo apt-get install ccache
# Other *nix
#$ wget https://github.com/ccache/ccache/releases/download/v4.7.4/ccache-4.7.4-linux-x86_64.tar.xz
#$ tar xvf ccache-4.7.4-linux-x86_64.tar.xz
#$ cd ccache-4.7.4-linux-x86_64
#$ make install # /usr/local
$ ccache -V
ccache version 4.5.1 # Note that the latest is 2022-11-21: Version 4.7.4.
(..snip..)
## Example: Compile Nim compiler source. Need to source and cd Nim
$ t sh build_all.sh
(..snip..)
time: 1:37.76, memory: 637376kB, exit: 0
# koch build
$ t ./koch boot -d:release
time: 0:21.60, memory: 616468kB, exit: 0
## Under using ccache & nim
# clear nimcache's cache
$ rm -rf nimcache
# clear ccache's cache(Needed for the 2nd time)
#$ ccache -c
## using 'ccache gcc' or 'ccache clang' with nim
# If you install with apt on debian/ubuntu, a symbolic link to ccache such as gcc will be created in /lib/ccache, so you may prefer that path.
$ export PATH=/lib/ccache:$PATH
# Other *nix
# If you get binaries with wget, make install will be installed in /usr/local/bin by default, so symbolically link to it.
#$ mkdir -p ~/.local/bin
# Use gcc-11 or clang, but specify the binary version you are using depending on your environment(gcc-10).
#$ ln -s /usr/local/bin/ccache ~/.local/bin/gcc
#$ ln -s /usr/local/bin/ccache ~/.local/bin/clang
#$ export PATH=~/.local/bin:$PATH
# 1st
$ t sh build_all.sh
(..snip..)
time: 1:33.28, memory: 638376kB, exit: 0
# 2nd
$ t sh build_all.sh
(..snip..)
time: 0:41.29, memory: 638084kB, exit: 0
# koch differential Build(nim only)
$ t ./koch boot -d:release
time: 0:22.04, memory: 615804kB, exit: 0
The result of the difference build will not change, but the second full-build will complete more quickly compared to the first full-build. (However, the possibility remains that this may change depending on the environment.)
$ ccache -x # compression_level = 3, default compression_level: 0
(..snip..)
Total data: 6.8 MB (7.8 MB disk blocks)
Compressed data: 6.8 MB (27.6% of original size)
Original size: 24.6 MB
Compression ratio: 3.627 x (72.4% space savings)
Incompressible data: 0.0 kB
# View cache statistics
$ ccache --show-stats
Cacheable calls: 484 / 524 (92.37%)
Hits: 240 / 484 (49.59%)
Direct: 240 / 240 (100.0%)
Preprocessed: 0 / 240 ( 0.00%)
Misses: 244 / 484 (50.41%)
Uncacheable calls: 40 / 524 ( 7.63%)
Local storage:
Cache size (GB): 0.01 / 2.00 ( 0.39%)
Note: If you set export CC='ccache gcc' in the environment variable, this will cause inconvenience for C projects that are not Nim projects. According to the ccache documentation, some minor compilation options are not supported, so perhaps specifying unsupported options will produce identical object files(Compiler runs without cache). Very small projects and first-time builds are expected to be slightly slower. However, if you are working on multiple projects, the cache will be shared, so the total time across all projects, version upgrades, etc. will be reduced.
You can turn off ccache in the configuration file if something goes wrong or if you really want a complete rebuild.
$ ccache -o disable=true # Written to ccache.conf
If used on a shared build server, additional configuration is required, such as using umask=002. See the documentation for details.
https://ccache.dev/manual/4.7.4.html#_sharing_a_local_cache
To use Visual C++ in MS-Windows, see below.
https://github.com/ccache/ccache/wiki/MS-Visual-Studio
Regarding the coordination of such tools, please do not issue this as a Bug to the main developers of Nim, it is very difficult or almost completely impossible to support all of them due to the wide variety of tools in gcc, clang and C language. Instead, it would be much more beneficial to spend time improving core features of the language, such as new gc/mm algorithms, gc algorithms for a wide variety of situations and memory pre-allocation for predictable cases, and implementing memory arenas. Occasional articles appear by Mr.Araq, but thread-shared caches with as few locks as possible, zero-cost exception implementations, etc., are beneficial to all. (I am not sure I understand what you have posted yet...)
Disclaimer: I am neither a ccache developer nor an interested party of ccache. I am not responsible for any problems or other damages caused by the changes.
Sorry for the long phraseology and thin content of this post. I just hope it helps someone.
If you want to investigate these kind of things, I also ran ptrace in nim itself, in a few random compilations of nim and see it reading the stdlib files from disk, very very frequently, I have enough ram to fit the stdlib files on RAM, maybe something like tmpfs can be used automatically when free RAM >= size(stdlib) (???).
I do not know whats the tmpfs equivalent for Windows, if any. I do not know how all of this can be implemented to be really useful beyond experiments.
Just thinking out loud... ;P
This tip was really good! I did the test here on Windows 10, with gcc 12.2.0, Built by MinGW-W64 project.
koch boot -d:release on the first build took 00:01:38.10 (98.10s total). On the second build it only took 00:00:28.67 (28.67s total).
The initial nim.exe is from csources_v2 for both and I'm building the devel version.
Note: I deleted nimcache before.
Compared to without ccache?
I agree with auxym, the FS should cache anything that's needed.
To those of you who have conversely taken up my time with my unfamiliar posts, I'm sorry & thanks.
The ccache's cache is not shared by different projects, sorry for posting without checking carefully.
After some trial and error(ccache -o debug=true), I think the ccache manual needs to turn off the inclusion of the current directory in the hash in hash_dir(ccache -o hash_dir=false). It also seems to be "-I/cwd/path/to/headers" in the c compiler options so that nim can import c source files and header files, and I assume it is also necessary to avoid including this in the hash that is keyed to the ccache cache.(I don't know/find how to do that.)
Nevertheless, it will be faster when nimcache is cleared, for example. Sorry for being so defensive...
bung, juancarlospaco, auxym, rockcavera, jasonfi, thindil
Thank you for the detailed tracing instructions. If the environment has enough RAM as you say, frequently loaded files, etc. will be loaded into the OS cache and reused. As you pointed out, it is thought that frequently read source files, etc. will be cached in tmpfs, as well as object files that are not updated. However, I did not know how to get detailed statistics (how many hits or cache misses) from tmpfs.
Once again, we have summarized the results.
I really wish I could actually measure it in a tangible way like the Nim hackers do, instead of just predicting it, but I'm frustrated that I can't present it. Nim compiles so fast as it is that few people question what it is, but I think one of Nim's elegant features that does not appear in its characteristics or syntax is that it also runs efficiently with little power!
Please forgive me if I have strayed quite far from the topic of Nim.
I have found a way to share the cache with the project, but since very few people will benefit from this, I will only report on it.
After some trial and error(ccache -o debug=true), I think the ccache manual needs to turn off the inclusion of the current directory in the hash in hash_dir(ccache -o hash_dir=false). It also seems to be "-I/cwd/path/to/headers" in the c compiler options so that nim can import c source files and header files, and I assume it is also necessary to avoid including this in the hash that is keyed to the ccache cache.(I don't know/find how to do that.)
# Add the -c and --nimcache options when compiling the source.
$ cd ex1
$ nim --nimcache=./build -c c src/ex1.nim
# Set ccache not to do hash dir.
$ ccache -o hash_dir=false
# Set the parent directories of the two or more different project parents on which to base ccache at the first point. Normally only /home seems to be good, but it says not to set /
$ ccache -o base_dir=/home
$ cd build # Second point, ccache's base_dir setting changes absolute paths to relative paths, so cd to one or more inner project directories to do this
# Compile using gcc symbolically linked to ccache. The json file is created in the same directory, so execute the command for the array of the compile. Wildcards are not allowed. One file at a time.
$ gcc -c -I/home/xxx/.choosenim/toolchains/nim-1.6.10/lib -I/home/xxx/src/ex1/src -o /home/xxx/src/ex1/nimcache/@m..@s..@[email protected]@[email protected]@slib@[email protected] /home/xxx/src/ex1/nimcache/@m..@s..@[email protected]@[email protected]@slib@[email protected]
# Now all that's left is to link the objects
# Do the same for different projects as well.
$ cd ex2
$ nim --nimcache=./build -c c src/ex2.nim
# (...snip)
If you can use jq or something to extract the commands from the json and prepare a Makefile or something, you can automate it, but it would be faster to compile with nimble as usual than the time to do that, and it would take more time to set up than the time you can reduce by having ccache share it. Perhaps even if you build with nimble build, if you cd to the internal directory of the project with nimscript or something and start gcc, it will be cached. But I tried cd("src"), but it didn't work.
Sorry for the disturbance.
This sounds interesting, is it possible to tell the Nim compiler to use a copy of stdlib residing in /tmp for example? If so, how?
Nim C file generation is not deterministic between projects: because of dead code elimination and the random order in which the compiler generates code, the c files for each module will be different depending on how it's used, even if it is the same module - things like changing import order in one place is enough to change any C file in the whole project - this leads to poor cache performance in general and makes solutions like ccache and any other caching less efficient.