Mirror of forum.nim-lang.org

3584 :: How is the compiler so clever, and how clever does it get?

• StasB (orginal) [2018-02-27T01:00:54+01:00] view original

  Hi! Suppose I have a module like this:

  # foo.nim
  const text = slurp("text.txt")
  
  static:
    echo text
  Run

  Let's say I import it from another module, named bar.nim. Every time I compile bar.nim, the static block gets executed, even if nothing has changed in either module. It's a little odd that the compiler bothers to parse the unaltered foo.nim, but it looks like it doesn't actually compile anything, because it doesn't modify the corresponding .c and .o files in the cache. If I change only text.txt, though, the compiler catches this and updates the cache. How does it know that the contents of the text file have changed? Is there some special handling related to slurp which compares the modification date on the module to that of the text file? If I remove the echo line from the static block, will the compiler keep reading the text file anyway, or does that happen lazily? What if I have some macro that generates code based on the contents of text.txt and I call it in foo.nim? Will it be ignored as long as these files don't change?

  [EDIT] Upon further experimentation, it looks like macros get invoked even when no files are modified. The cache doesn't get updated, so I guess the results of the macro invocation are simply discarded instead of being compiled? Is there any way to prevent this useless work from being done? Does this only happen for macros with side-effects? I tried testing this using a long computation in a macro, but the VM just bails out.

---

• Araq (orginal) [2018-02-28T09:42:52+01:00] view original

  I don't understand your questions. Macros get evaluated when required, there is no "useless" work.

---

• Stefan_Salewski (orginal) [2018-02-28T10:34:20+01:00] view original

  It's a little odd that the compiler bothers to parse the unaltered foo.nim, but it looks like it doesn't actually compile anything, because it doesn't modify the corresponding .c and .o >files in the cache.

  Araq, I think he wants that Nim source files are only compiled when changed, maybe when creation date has changed.

  Stas, I guess the Nim Compiler compiles Nim source text to temporary destination, maybe in Ram, and compares it to files in nimcache directory. And copies it only to mimcache directory when it is not identical. So C compiler can skip it when unchanged, as .o file is available. Just my personal guess.

---

• StasB (orginal) [2018-02-28T13:13:23+01:00] view original

  @Araq: As far as I can tell, when a macro has side-effects (e.g. echo), its invocations get re-evaluated, even if the macro itself and the modules containing the invocations remain unchanged. Is the compiler smart enough to only do that with macros that have side-effects? What happens if the macro breaks referential transparency in a trickier way (e.g. it generates code based on the contents of a file)?

  @Stefan_Salewski: I hope you're wrong, because that would defeat the purpose of having a cache in the first place.

---

• Araq (orginal) [2018-02-28T13:47:47+01:00] view original

  Well the "compilation cache" / "symbol files" / "incremental recompilation" feature is in the works and indeed it treats slurp like include and so will notice if the dependency changed or not. But this feature is not yet ready for anything. Regardless of it, macros are always re-evaluated since that needs to be done. There is no caching.

---

• StasB (orginal) [2018-02-28T13:58:30+01:00] view original

  Why does it need to be done for pure macros if the code hasn't changed?

---

• mashingan (orginal) [2018-02-28T18:54:43+01:00] view original

  I don't know how clever a compiler you want, but as far I understand, macro always evaluated during compile time.

  If not, you get the macro what do you usually works with C, just a simple textual substitution. And that's not what you want, really.

---

• StasB (orginal) [2018-02-28T20:31:39+01:00] view original

  If the macro hasn't changed and the module where the macro is used hasn't changed, then the result of compiling the module remains the same, so what reason is there to call the macro again, except for side-effects?

---

• Udiknedormin (orginal) [2018-02-28T22:36:45+01:00] view original

  @mashingan @StasB What if a macro uses, let's say, system date? Or connects to a database (you can't check whenever it changed until you do connect)? And still, even a noSideEffect macro is a code generator/transformer, not a textual substituter so there is no analogy to C.

---

• mashingan (orginal) [2018-03-01T05:51:48+01:00] view original

  @StasB, the macro wouldn't be evaluated if not used, evaluated if used. If not used, you won't get the transformed/generated code

  @Udiknedorm, yes, some operations just cannot be evaluated during compile time, but macro in Nim still evaluating what it can for compile-time operations.

---

• StasB (orginal) [2018-03-01T05:57:36+01:00] view original

  I wasn't making analogies to C, and I don't see what difference it makes in this case. If the macro has no side-effects, and the macro definition hasn't changed, and the module where the macro is used hasn't changed (both can be deduced from comparing modification dates with the cache), then the result of macro expansion will not have changed from the last time the module was compiled, and neither will the surrounding context, so there is no reason to call the macro except for side-effects. As for what happens when the macro breaks referential transparency in some fishy way, it'd be nice if you could flag it to be treated as if it was pure anyway, and then add whatever external data the expansion depends on as a dependency to the module defining the macro.

---