Mirror of forum.nim-lang.org

6535 :: Understanding Nim compiler

• spip (orginal) [2020-07-11T20:37:21+02:00] view original

  Is there a way to understand and trace Nim compiler? For instance, this piece of code does not compile, while if you replace template by proc, it does. Is it possible to trace the compiler parsing the code and understand the difference between both behaviours? One one would start looking at the compiler internals? Is the wiki page still the best way to jump into debugging the compiler?

---

• Yardanico (orginal) [2020-07-11T20:38:47+02:00] view original

  Well yeah, the wiki page is the best way to debug the compiler currently (as far as I know, you can also ask @Araq :P) :) You can also read https://nim-lang.org/docs/intern.html

---

• spip (orginal) [2020-07-11T21:59:05+02:00] view original

  When compilation fails, ./koch temp c /tmp/foo.nim displays the stack trace down to the compilation error message, so we can have a snapshot understanding of the parsing process. But when the compilation succeeds, there's no way to understand what were the compilation phases.

  For instance, is there a compiler option to dump the parsed AST before code generation?

---

• spip (orginal) [2020-07-12T23:16:13+02:00] view original

  Replying to myself and for future use: at least, there's an undocumented scan compiler command to dump the result of the lexer/scanner.

  Try nim scan foo.nim to see the tokens identified in the source file.

  There's also a parse command but it discards its result. So apart from checking that the source is syntactical valid, like command check does, it is of no real use. It can be easily enhanced to dump the source AST while rewriting the line by

  debug parseFile(conf.projectMainIdx, cache, conf)
  Run

  after including module astalgo in main.nim. The output being very verbose, this must be used for tiny sources!

---

• Araq (orginal) [2020-07-12T23:29:50+02:00] view original

  Well if you want to know why something works for procs and not for templates, look at the different handling of skTemplate inside sigmatch.nim and elsewhere. Looking at the AST is pointless, no matter how much you look at it and under which views and wether it's the AST for the proc or for the full module and whether it's the AST after parsing and before sem'checking or after sem'checking.

---

• spip (orginal) [2020-07-13T15:11:04+02:00] view original

  Thanks. For this particular example, I've started looking at how static generic expressions are handled by procs and templates into semtypes.nim.

  But the question wanted to be more general. How a new comer can understand how the Nim compiler works? What tools/options are available to have a global understanding of the compilation process? Are there ways to see the results of the different compilation phases? I've listed 2 undocumented options to see the scan and parser phases, but after the AST is built there are many phases when the AST is reworked before code generation. What are the important internal data structures?

  Is there a wish to build a compiler's internals documentation or is it a knowledge that lives in the compiler hackers' minds?

---

• demotomohiro (orginal) [2020-07-13T16:45:27+02:00] view original

  I wrote how to debug Nim program and Nim compiler with GDB long time ago. https://internet-of-tomohiro.netlify.app/nim/gdb.en.html

  It might contain outdated content.

---

• Araq (orginal) [2020-07-13T20:41:16+02:00] view original

  Is there a wish to build a compiler's internals documentation or is it a knowledge that lives in the compiler hackers' minds?

  Well we have the internal documentation and we're always accepting PRs but I generally assume that compiler contributors have read some book about compiler development, there are good ones available like https://people.inf.ethz.ch/wirth/CompilerConstruction/index.html

  What are the important internal data structures?

  It's ASTs all the way down and the compiler's AST is the same AST that is exposed in the macro system. :-)

---

• spip (orginal) [2020-07-17T01:55:46+02:00] view original

  Niklaus Wirth's books on compiler design and structured programming are very didactic. Wirth is a great author to explain how things work. And I just discovered that one of the books I read and used a long time ago is now available for free on the Web: Compiler design in C from Allen Holub. There was a curses GUI to go step by step and understand parsing. Would be nice if Nim compiler showed its internals in a similar way... ;-)

---