Mirror of forum.nim-lang.org

7337 :: How to make Nim compilation faster?

• drkameleon (orginal) [2021-01-05T16:43:10+01:00] view original

  Surprisingly, I'm having some issues with Nim compilation speed.

  Given that I use to make frequent changes to the project I'm working on (https://github.com/arturo-lang/arturo), that also means frequent re-builds.

  However, this seems too slow:

  
  Hint: 213449 lines; 144.014s; 624.43MiB peakmem; Dangerous Release build
  Run

  The above is on a 13-inch, 2018 MacBook Pro.

  These are my build options:

  
  c --opt:speed --colors:off -d:release -d:danger --panics:off --gc:orc --checks:off --overflowChecks:on -d:ssl --threads:on
  Run

  And here is my Nim version:

  
  Nim Compiler Version 1.4.2 [MacOSX: amd64]
  Compiled at 2020-12-29
  Copyright (c) 2006-2020 by Andreas Rumpf
  
  active boot switches: -d:release
  Run

  Any ideas/suggestions?

---

• Araq (orginal) [2021-01-05T18:11:16+01:00] view original

  Your code uses template and hardly any proc. Probably you are getting a combinatorial code size explosion. If true, you don't only have slow compile-times, you also have bad performance because of many icache misses.

---

• drkameleon (orginal) [2021-01-05T19:27:19+01:00] view original

  Thanks a lot for the input.

  It's true that I've rather overdone it with the templates.

  Is there any easy way to track/report where most template expansions take place? (I've obviously lost track of them all...)

  P.s. Regarding performance, I know what you mean. I guess I'll have to start benchmarking again - although I wanted to reach a stable version first... Still the performance doesn't look too bad.

---

• treeform (orginal) [2021-01-05T19:58:07+01:00] view original

  I made dumpincludes a utility that can at least tell you which modules are getting huge expansions:

  https://forum.nim-lang.org/t/7309

  This can be useful to let you know how bad the problem really is.

---

• cblake (orginal) [2021-01-05T21:26:14+01:00] view original

  Your Arturo does not look presently very GUI centric using Apple OS facilities/etc. So, one possibility might be for you to install a Linux virtual machine on your MacBook and then within that VM use TinyCC/tcc backend for Nim. For me, tcc is about 5x faster than gcc on an empty .nim file, and can be over 11x faster than gcc on larger code bases. (I do not think you can run tcc directly on the Mac since the native OSX executable format is this Mach-O from NeXT, and this format is unsupported by the inbuilt tcc assembly/linker, to the best of my knowledge.)

---

• SolitudeSF (orginal) [2021-01-06T08:11:22+01:00] view original

  programs that were working perfectly fine with gcc/clang would segfault with tcc on my machine.

---

• cblake (orginal) [2021-01-06T11:56:37+01:00] view original

  It's true that not everything works all the time. (I almost warned about that. The C++ backend also fails me from time to time.)

  It is still worth trying tcc on a program-by-program basis. It's my default compiler, and I rarely have any trouble. If you can track down the errors inducing your segfaults then you can maybe help Nim keep a faster backend working.

---

• Araq (orginal) [2021-01-06T12:12:57+01:00] view original

  For me the compiler is slow because the frontend keeps recompiling the world. The C compilation phase is both parallelized and cached and works really well, I have no need for "Tiny C". I do long for IC though...

---

• cblake (orginal) [2021-01-06T12:26:46+01:00] view original

  Not repeating work is often the biggest "easy" optimization. I almost mentioned IC as a near-term hope for @drkameleon, but it seems not yet ready to give speed-up numbers.

  Anyway, I think we all probably long for IC. :-) That said, your comment raises a question - would a large, single file .nim compile be sped up by IC or not? I just haven't followed the design well enough to know.

---

• cblake (orginal) [2021-01-06T12:40:33+01:00] view original

  @drkameleon - you did not mention your workflow about "small changes", but if it is helpful to first just test if something compiles before waiting on the full compile then it may also help to use Nim's -c flag (note the minus/dash) to only compile the Nim code to C. E.g., nim c -c foo.nim. All just ideas/suggestions as per your request.

---

• Araq (orginal) [2021-01-06T15:06:08+01:00] view original

  Anyway, I think we all probably long for IC. :-) That said, your comment raises a question - would a large, single file .nim compile be sped up by IC or not (EDIT: well, I guess its import web may not change and that alone is a huge boost)? I just haven't followed the design well enough to know the granularity.

  The granularity is the Nim module. A large, single file .nim compile itself won't see speedups. However, one of the biggest module I've seen is system.nim itself. It's rare that you change the stdlib when you recompile, so in practice every project will see speedups.

---