Mirror of forum.nim-lang.org

4917 :: Does Nim support compiling code at runtime?

• gct (orginal) [2019-06-07T18:32:03+02:00] view original

  Specifically I'd like to write a file calculator application for a numeric file format that I have. The idea is you'd be able to do something like:

  skcalc out file1 fft file2 fft ~* abs_sq

  (RPN notation). This would find the cross-spectral power between two files. I'd like to support much more complicated operations, and if I could compile my command line expression at runtime and execute that'd be a huge win. In particular I'd like to be able to support whatever SIMD subset the host processor gives me.

---

• mratsim (orginal) [2019-06-07T19:08:51+02:00] view original

  You can:

  • Execute shell commands at compile-time
  • read file at compile-time
  • do computation at compile-time (be aware that this is done in a VM that is much slower than compiled-code and not suitable for CPU-intensive computation)

  But what you want is basically a JIT.

  Here is an example JIT where I translate brainfuck programs to assembly:

  • https://github.com/mratsim/jitterland/blob/master/bfVM_v03_jit.nim

  Alternatively you can just use an interpreter.

  Some inspiration in C/C++

  • https://github.com/codeplea/tinyexpr
  • https://github.com/zserge/expr

  Also I'm currently implementing a compiler for deep learning with SIMD support (and GPU and parallelism later). For now I'm only supporting compile-time code generation but later I will also add runtime code generation, probably via LLVM JIT capabilities: https://github.com/mratsim/compute-graph-optim

---

• gct (orginal) [2019-06-07T19:14:56+02:00] view original

  It is effectively a JIT, unfortunately I haven't been able to find anything to fit my needs (currently working in C++). LLVM is just too big to haul around for my usage.

---

• shashlick (orginal) [2019-06-07T19:44:25+02:00] view original

  You could always use HCR.

  I did something similar in my text editor which recompiles its plugins which are Nim dlls when the code changes.

  In your case, you could generate a nim file for the expression and load it as a dll and execute it that way.

---

• gct (orginal) [2019-06-07T20:02:06+02:00] view original

  That's the best I've come up with, just watched Viktor Kirilov's talk at ACCU about the hot reloading. I'd have to generate NIM, which would then compile to C which makes me a little sad. I think at that point I might as well just generate and compile the C myself in my current C++ code...

---

• Araq (orginal) [2019-06-07T20:30:41+02:00] view original

  You can import the Nim compiler as a library into your code and either compile or interpret the Nim code. However, it seems like it doesn't help you if you're stuck with this "numeric file format" and cannot switch to use Nim's syntax instead.

---

• gct (orginal) [2019-06-07T20:38:34+02:00] view original

  I think that'd give me the same "generate nim, compile to C" flow as the hot reloading mentioned above, no?

---

• Araq (orginal) [2019-06-07T23:45:44+02:00] view original

  Pretty much, yes.

---

• cy (orginal) [2019-06-08T19:06:43+02:00] view original

  You could generate C from your numeric file format, then use the dynlib module in nim to load it during runtime. That is to say, replace C++ with nim, but don't bother with nim's format when generating code, since it's easier to generate C/ASM for your specific needs. I find "hard C" like that, for well defined, specialized transformations of numeric data, to be very unlikely to require any shenanigans that might result in stack smashing, undefined states, or race conditions. Using nim everywhere else where those shenanigans would be required in spades? Still a good idea.

  But I've never really done much with fourier transforms, so I dunno. Have you looked at the standardized fftw library at http://www.fftw.org ? They have a "fftw-wisdom" command for doing some sort of compilation/optimization beforehand.

---