Thanks, Artur
The best solution seems to be to use the compiler as a library. Creating the PNode structures isn't as easy as what macros.nim offers, but only a few helper procs like newTree are required to make it comparable. These helper procs are in compiler/ast.nim but maybe you need more of them. Then you can feed the trees to the semantic checker and then to the VM to run it. Or send them after sem'checking to the C code generator and compile it.
Using the compiler as a library is harder than Nim's macro system but since you understand opcNNewNimNode you definitely have what it takes. The reward is an unmatched amount of power over your compilation pipeline.
My library Nimscripter may actually finally serve a purpose?!
Using it I quickly threw together an example(albiet very very meh).
# This is your compiled Nim file
import nimscripter/nimscripted
template exportToNim(a: untyped) = a
exportCode:
import macros
var
toExport: seq[string]
pos = 0
proc genAst(): string {.exportToNim.} =
result = toExport[pos]
inc pos
proc hasMoreAst(): bool {.exportToNim.} = toExport.len > pos
macro exportAst(body: untyped) =
toExport.add treeRepr body
import nimscripter
let intr = loadScript("./toexport.nims", stdpath = "/home/jason/nimscripter/stdlib")
if intr.isSome:
while intr.get.invoke("hasMoreAstExported", "", bool):
let ast = intr.get.invoke("genAstExported", "", string)
echo ast
# This is your nimscript file that you want to generate your Nim code/ast
macro genForLoop() =
let code = quote do:
for i in 0..100:
echo i
toExport.add code.repr
macro genAddition() =
let code = quote do:
var a = 10 + 100
toExport.add code.repr
genForLoop()
genAddition()
exportAst:
for i in 0..100:
echo i
All that together has the exectuable output this pretty output which is reliant on the script, so can be changed between runs of course. Truthfully I am probably(almost certainly :P) off the mark of what you wanted, but this seemed close to atleast my understanding of what you were after.
for i`gensym23 in 0 .. 100:
echo i`gensym23
var a`gensym24 = 10 + 100
StmtList
ForStmt
Ident "i"
Infix
Ident ".."
IntLit 0
IntLit 100
StmtList
Command
Ident "echo"
Ident "i"
Hello Araq, hello ElegantBeef (Jason?),
many thanks for your fast and helpful answers, Nim has a truly great community :-)
I think that I can benefit from both suggestions. I could first use Nimscripter for faster prototyping with macros tools, and if at a later point more flexibility and performance is needed, I can switch to the PNode/compiler pipeline.
For the Nimscripter example, if I understand that correctly: the compiled nim code (1st file) manipulates at its runtime the script file "toexport.nims", compiles it (at "loadScript"?), and then executes parts of the "toexport.nims" (all at nim runtime). So effectively this moves the "compile phase of the nimscript (inclusive macro usage)" to the runtime of the nim code. I can then use the macro tools in the script, but I can change the script contents and so the generated ASTs without recompiling the 1st file (nim).
Still here some questions about the Nimscripter, in order to use it properly:
Thanks! Artur
I prefer using exportCode due to it being in the Nim code, being semantically checked on compile and ensured it's usable.
Yes you should be able to pass most data types to Nimscript using a similar method to the ones I've demonstrated, basically using properties to do it. Procs probably are not passible, simple due to how we have to use primitives to pass types(all my passing is done converting to strings, can checkout msmarshal to see how)
The entire point of Nimscripter is to remove the tedium of making Nim code/Nimscript code accessible either way. Making it so you can quickly and easily interact with the VM, I have 0 idea the performance implications of my code, as it was not much of a concern, the goal was to enable easy scripting Nim programs from Nimscript. For the most part it just provides an abstraction from the direct compiler calls, making it easier to write.