Mirror of forum.nim-lang.org

1743 :: Calling Nim functions from NimScript

• jyelon (orginal) [2015-10-28T17:10:46+01:00] view original

  I was thinking about what would be necessary to allow NimScript to call any Nim function that is compiled into the binary that's running the NimScript. I imagine a world where NimScript can say "import zlib", and the interpreter will actually read and compile zlib.nim in an interpreted mode, but when it sees a proc definition, it short-circuits the compilation of the proc and instead fetches the compiled version of the proc out of the running binary. If zlib wasn't compiled into the binary, it would fail to load in the interpreter (because we cannot interpret importc functions), but if zlib was compiled in, it would work.

  As I see it, we'd need:

  • The compiler would have to generate a wrapper for each compiled-in proc. (Or, alternately, one wrapper for each distinct signature of compiled-in proc).
  • You'd need the compiler to generate a symbol table containing the identities and addresses of the compiled-in procs.
  • We want to generate wrappers so that we can convert an interpreted function into a compiled proc, so that you could store it in a compiled procvar. You would need one wrapper for every unique type of procvar in the compiled code.
  • You wouldn't need compiled-in templates or macros: those could be just plain old interpreted.
  • You would want to be able to get at global variables too.

  Am I crazy to even consider this?

---

• rob10c (orginal) [2015-10-28T17:25:22+01:00] view original

  as a noob i shouldn't even consider a reply to your post, though only to answer the last question... did you see any one about this ? actually for real, if i were you i would install nim in a different folder maybe test x86 while original folder is x64 setup.

  then with your knowledge which i don't even come close to, try convert an interpreted function into a compiled proc or what ever it is you think should be applied to original symbol table (yet again did not understand any of this) was i crazy to even answer this?

---

• andrea (orginal) [2015-10-28T18:03:04+01:00] view original

  If I understand correctly, one would need to do something like this, but preloading other modules in the setup as well.

  It would be great if one were to make this, for two reasons:

  • the possibility of having custom interpreters that allow existing C libraries (and one day, maybe, a REPL...)
  • the possibility of creating sandboxes that are allowed only whitelisted modules to run untrusted Nim code securely

---

• zielmicha (orginal) [2015-10-28T22:28:06+01:00] view original

  This is a good idea, but I can't see how generics could work in this scheme.

---

• andrea (orginal) [2015-10-29T10:02:48+01:00] view original

  Nimscript already has generics. Wrapping functions and pointing to their already compiled version only serves for C imports, which are not polymorphic

---

• jyelon (orginal) [2015-10-29T18:33:00+01:00] view original

  Actually, I wasn't just thinking of this being purely for C imports. I was also thinking of being able to use the compiled version of any proc, if that proc was already compiled into the binary.

  With generics, I haven't seen how the compiler compiles them, but I imagine it probably compiles each instance separately. I could imagine having the interpreter expand the generic into a generic instance, and then check if that instance is compiled into the binary.

  Big picture, I was imagining a development model where I can load a module into the interpreter, test it in a REPL, and when it's all ready, compile it. I used to do that in common lisp, it's a very efficient way to work.

---

• OderWat (orginal) [2015-10-29T19:23:32+01:00] view original

  I wonder if one could mark procs to not compile which then, when the executable is running will be executed as NimScript?

---

• jyelon (orginal) [2015-10-29T19:58:48+01:00] view original

  Or - hash the code of the proc into a fingerprint, and only use the compiled-in version if the fingerprints match.

---

• Araq (orginal) [2015-10-29T23:05:20+01:00] view original

  The VM used to support the FFI (via libffi) and I can dig out this code again, but it's really messy and I cannot see how to ever get it stable enough so that it can be relied upon. The stdlib often uses Nim as a better C (because it is ;-) ), but this also means it heavily relies on low level details. For example, when you call a void externalFoo(int* x) you might need to ensure that the changes to x are copied back into the VM's representation of x, it is not enough to allocate an int buffer under the hood that is filled with the required values.

---

• andrea (orginal) [2016-01-04T10:49:24+01:00] view original

  @jyelon Did you ever start experimenting in this direction?

---