Mirror of forum.nim-lang.org

8776 :: a thought about faster iteration

• kobi (orginal) [2022-01-05T13:27:39+01:00] view original

  I have noticed (basically seeing all the dots when nim compiles) that the compiler currently enters (compiles) all imports before checking the current file for even syntax mistakes. In addition, why enter the first import, if the second one possibly doesn't exist. I think a few such heuristics can improve iteration speed. for example: first gather all imports, and see if the files exist, exit quickly with an error if fails. then do a syntax check of current file, maybe handle the: from x import y, if y doesn't exist. ...maybe other quick checks... afterwards read and compile everything.

  it could be that relatively simple changes like that can improve compilation speed for development iterations or for IDE error reporting. When IC finally lands it will take care of this, but for now it seems like a nice hack to get a speedup.

  I like to get quick feedback and Nim is quite good at this, so this is a suggestion for improvement. What do you think?

---

• SolitudeSF (orginal) [2022-01-05T13:35:58+01:00] view original

  a nice hack

  no such thing

  first gather all imports

  what if some imports come from constant strings declared in other modules? or even generated from macros?

---

• kobi (orginal) [2022-01-05T19:35:55+01:00] view original

  I realize those things that sound simple are usually much more complex. But isn't it possible there are some easy gains, low hanging fruits by tweaking the order of code validation?

---

• cantanima (orginal) [2022-01-05T22:33:50+01:00] view original

  I'm not familiar with Nim's compiler internals, so someone correct me if I'm wrong (and I'm sorry if so), but:

  Nim's import isn't like C, which has loads every file you #include. When Nim import's a module, it reads a symbol file. This is relatively quick.

  Nim can also include a file, much like C, but this is rarer.

  So, your suggested optimization might well give the Nim compiler a marginal improvement with import, but it would also add a fair amount of complexity, and the improvement would be marginal.

  There's a paper somewhere about something like this in the Oberon compiler, decades ago.

  PS If I'm wrong I'll be happy to know.

---

• ElegantBeef (orginal) [2022-01-06T00:41:36+01:00] view original

  This almost matches what the "lazy sem" idea that was proposed does. The idea being it gathers all exported symbols and only sems the body on call, which means instead of semantically checking an entire file, you only semantically check what's used speeding up compile time of large libraries where you only use a small foot print. The PR is here if interested to read about.

---

• Araq (orginal) [2022-01-07T09:05:29+01:00] view original

  Lazy sem'checking has been discussed for years before that PR. Unfortunately it's inherently incompatible with separate compilation and modularity. Consider a module A offering two concrete procs p1, p2. Module B only uses p1. So when the compile order is B -> A only p1 would be compiled. Now a module C is added that uses p2. So now p2 needs to be compiled but if we assume that we want to compile A only once and save the result on disk in an intermediate format that's impossible. So we need to recompile A even though it didn't change at all.

  Now there are ways to compensate for these problems, but the solution is hardly elegant nor particularly peformant -- and you pay for this complex logic even for programs that don't contain too much dead code.

---