I'm tinkering with templates and macros to learn. I've had success with full ast-based macros but am trying to use some of the sugar for macros and some templates.
I've tried some variations of the below simple example without success. The compiler complains about undeclared x, y.
import macros
macro createProc(name: untyped, body: untyped): untyped =
quote do:
proc `name`(x, y: int): int =
`body`
template createProc(name, body: untyped): untyped =
proc name(x, y: int): int =
body
createProc add:
x+yI've been looking at manual template docs and ssalewski template docs.
I would appreciate a beatdown on what's wrong :)
You have encountered symbol injection & gensym.
By default, symbols of variable names in templates (and quote) are gensym'ed. This means that they receive a unique identifier only used inside the context of said template.
You can in fact inspect this behavior using repr:
import macros
macro createProc(name: untyped, body: untyped): untyped =
echo repr quote do:
proc `name`(x, y: int): int =
`body`
createProc add:
x + y
# Prints:
# proc add(x`gensym0, y`gensym0: int): int =
# x + y
What happens if you make a second one?
createProc add2:
x + y
# Prints:
# proc add2(x`gensym1, y`gensym1: int): int =
# x + y
Ok, so it incremented the number. This is not very relevant in procedure arguments, but it's useful for code like this:
macro oneMacro(): untyped =
quote do:
let x = "hello world"
echo x
macro anotherMacro(): untyped =
quote do:
let x = "dlrow olleh"
echo x
oneMacro
anotherMacro
# works, as it expands to something like:
# let x`gensym0 = "hello world"
# echo x`gensym0
# let x`gensym1 = "dlrow olleh"
# echo x`gensym1
Ok, so what if you want to make your symbols available outside the template/macro? You use {.inject.}.
macro createProc(name: untyped, body: untyped): untyped =
quote do:
proc `name`(x {.inject.}, y {.inject.}: int): int =
`body`
# Also works in templates btw.
#[
template createProc(name, body: untyped): untyped =
proc name(x {.inject.}, y {.inject.}: int): int =
body
]#
createProc add:
x+y
echo "1 + 2 = ", add(1, 2) # 3
If you check with repr, you will see that the symbol has been preserved as you declared it; no gensym anywhere.
Now let's check what happens with the other example.
macro oneMacro(): untyped =
quote do:
let x {.inject.} = "hello world"
echo x
macro anotherMacro(): untyped =
quote do:
let x {.inject.} = "dlrow olleh"
echo x
oneMacro
anotherMacro
# /tmp/x.nim(15, 1) template/generic instantiation of `anotherMacro` from here
# /tmp/x.nim(10, 9) Error: redefinition of 'x'; previous declaration here: /tmp/x.nim(5, 9)
# welp.
# same as if you wrote
# let x = "hello world"
# echo x
# let x = "dlrow olleh"
# echo x
Note however, that not everything is gensym'ed by default. See https://nim-lang.org/docs/manual.html#templates-hygiene-in-templates for a list of the default behavior for separate types of symbols. (In general, variables & types are gensym'ed, procedures and the like are inject'ed. This is why you can call add without an inject pragma.)
In general, variables & types are gensym'ed, procedures and the like are inject'ed. This is why you can call add without an inject pragma.
This is not the reason. The symbol is inject because name is a parameter of the template. But it would also be injected if it wasn't because its a proc.
But if name was used elsewhere, it would still be injected
What @nrk says is completely correct. But just for completeness sake I have some additional comments.
By default symbols are gensym'ed, meaning they get non-colliding names. But we can explicitly name the arguments like so:
macro createProc(name: untyped, body: untyped): untyped =
let
x = "x".ident
y = "y".ident
quote do:
proc `name`(`x`, `y`: int): int =
`body`
As you can see the x and y arguments are now bound to the new idents we create carrying the actual names of x and y. This might be useful if you want to generate the identifiers from code.
For templates you also have the option of marking the entire template dirty:
template createProc(name, body: untyped): untyped {.dirty.} =
proc name(x, y: int): int =
body
Why inject isn't the default for arguments is a bit trickier to spot. But it has to do with allowing the use of values from outside the scope to the inside. Consider this:
macro createProc(name: untyped, body: untyped): untyped =
let x = "x".ident
quote do:
proc `name`(`x`, y: int): int =
`body` + y
let
y = 100
createProc add:
x+y # x here is taken from the call, while y is taken from the let block above. The y from the call is available in the macro.
echo "1 + 2 = ", add(1, 2) # 103
Thanks all @PMunch , choltreppe, and @nrk!
@nrk I'd seen those docs, but I couldn't see why even the proc parameters would be gensym'd. Is there a way to actually see the expanded Nim code. Like your comment:
# works, as it expands to something like:
# let x`gensym0 = "hello world"
# echo x`gensym0
# let x`gensym1 = "dlrow olleh"
# echo x`gensym1could I actually see that without getting headed off by compiler errors? Kind of like how dumpAst works but without having to do it in the macro (which now somehow seem clearer than templates to me).
Thanks to all the help here I was able to apply some of this very simply to a toy parsing project with {.inject.} (which I have asked other question for in the forum)!
template step*(input: untyped): untyped =
(`input`).parse_partial(stream, index)
template generate(body: untyped): untyped =
block:
proc temp[T, R](stream{.inject.}: Stream[T], index{.inject.}: var int): Result[R] =
`body`
Parser(fn: temp, description: "")
let t: Parser[char, string] = generate:
let x = step test_string("he")
discard step "ll"
let s = step "o"
return x&s
echo t.parse("hello")Any idea why with the templates, I get this message about mismatched types:
type mismatch: got 'Parser' for 'block:
proc temp[T; R](stream: Stream[T]; index: var int): Result[R] =
let x =
(test_string("he")).parse_partial(stream, index)
discard
("ll").parse_partial(stream, index)
let s =
("o").parse_partial(stream, index)
return x & s
Parser(fn: temp, description: "")' but expected 'Parser[system.char, system.string]'I don't see any ambiguity in the typings with let t: Parser[char, string]. Without the type on t I get other errors about ambiguity not mis-match. I also tried just adding types to the template with
template generate(T, R: typedesc; body: untyped) and template generate(T, R: untyped; body: untyped)
which brought other problems where the errors suggested (in the first case) that the types were bound to the temp proc in some way (like type mismatch got temp.char expected system.char...)
Is there a way to actually see the expanded Nim code. Like your comment
Do like the code snippet above. Return the repr of the Nim's AST.
There is nothing in your code saying that T should be char and R should be a string.
Parser seems to be a generic type that is not specialized. Shouldn't that be:
1. Just to see if it works correctly:
template generate(body: untyped): untyped =
block:
proc temp[T, R](stream{.inject.}: Stream[T], index{.inject.}: var int): Result[R] =
`body`
Parser[char, string](fn: temp, description: "")
2. If we can do generic templates:
template generate[T, R](body: untyped): untyped =
block:
proc temp[U, V](stream{.inject.}: Stream[U], index{.inject.}: var int): Result[V] =
`body`
Parser[T, R](fn: temp, description: "")
let t: Parser[char, string] = generate[char, string]:
For repr though, it seems is has to be in a macro (I'd been using it before asking this question actually and I didn't even put together that the gensym I was seeing was actually modifying the identifiers!)
🤔 I suppose I can just make a utility macro for the repr in which case I guess my question is just "does a utility already exist that does this in global code like dumpAst?"
These posts: recent: https://forum.nim-lang.org/t/10494#70006 older: https://forum.nim-lang.org/t/9441#61985
might have been another.
I thing the generic template will work. I'll try that.
There is nothing in your code saying that T should be char and R should be a string.
sorry here is the definition of Parser
ParseFn*[T, R] = proc(stream: Stream[T]; index: var int): Result[R]
Parser*[T, R] = object
fn*: ParseFn[T, R]
description*: stringHere is my logic on how the type could have been inferred:
(forgive some adhoc notation) Parser[T, R](temp[T, R]) -> block[Parser[T, R]] -> t: Parser[char, string]
in my mind, the information is all there if the type checker can walk this chain in reverse.
will actually dump x + y, but at the same time will print at compile time the expansion of...
its definition is just
echo body.toStrLit
result = bodyand it takes typed and I can't avoid compiler errors while using it. What I was hoping for is something that will fully expand all the code so you can see what the code would look like with the macro expanded even if the expanded code is invalid which seems crucial to debugging especially for a beginner like me. That's why I referenced dumpAst[Gen] because it is a good teaching tool.
So, I tried these (macros all commented out):
# 1. shows everything but just as written e.g. not expanded
# macro echoMacro(b: untyped)=
# echo repr(b)
# 2. (produced same as 1)
# macro echoMacro(b: untyped)=
# echo b.toStrLit
# 3. raises the compiler errors without showing anything
# macro echoMacro(b: typed)=
# echo b.toStrLit
# 4. at this point I'm reaching because idk what I'm doing :)
# and this isn't valid according to the compiler
# macro echoMacro(b: untyped)=
# let expanded = expandMacros:
# echo expanded.toStrLit
echoMacro:
let t: Parser[char, string] = generate:
let x = step test_string("he")
discard step "ll"
let s = step "o"
return x&s
echo t.parse("hello")@dlesnoff I realized I had already run your experiment (1) and it had worked. Your #2 worked too!
Would you mind explaining the distinction between the generic parameters of something like a template/macro vs the actual template/macro arguments? My initial thoughts looking at generic parameters for such a thing is not for using them in the actual template but for using them to type arguments and return types so this solution never would have occurred to me.
For example, I had even tried these before your answer, but they didn't work:
template generate(T, R, body: untyped): untyped =
block:
proc temp(stream{.inject.}: Stream, index{.inject.}: var int): Result =
`body`
Parser[T, R](fn: temp, description: "")
template generate(T, R: typedesc; body: untyped): untyped =
...same body
let t = generate char, string:
...Would you mind explaining the distinction between the generic parameters of something like a template/macro vs the actual template/macro arguments?
I am not a Nim expert, I program in Nim just as an hobby. The following sections in the Nim manual can be of help to you:
I would use typedesc arguments if I want to treat type names as values – that is, if I want to perform operations on type names, like:
It is called implicit genericity, because you can rewrite typedesc parameters functions with generics:
proc p(a: typedesc)
# is roughly the same as:
proc p[T](a: typedesc[T])
I am not very sure why your second template definition doesn't compile. The first generate definition surely won't work. You haven't specialized Stream and Result, that are just like Seq[T]: they expect a type description between brackets.
Actually, I do not know where they are coming from. Not from std/streams nor from Github arnetheduck/nim-results. I tried to compile:
import macros, streams
import results
type
ParseFn*[T, R] = proc(stream: Stream[T]; index: var int): Result[R]
Parser*[T, R] = object
fn*: ParseFn[T, R]
description*: string
template step*(input: untyped): untyped =
(`input`).parse_partial(stream, index)
template generate[U, V](body: untyped): untyped =
block:
proc temp[T, R](stream{.inject.}: Stream[T], index{.inject.}: var int): Result[R] =
`body`
Parser[U, V](fn: temp, description: "")
let t: Parser[char, string] = generate[char, string]:
let x = step test_string("he")
discard step "ll"
let s = step "o"
return x&s
# echo t.parse("hello")