I confess that Nim macros defeated me. My expectations were that I would learn Nim macros as easily as I had learned Lisp macros, but no, Nim macros are tough. Then I started to create theories about my difficulty in learning Nim macros.
Nim macros are easy, but I am so used to Lisp macros that I cannot see the Nim solution. It is easy to test this theory: If I present the problem of learning Nim and Lisp to young programmers, they will learn Nim macros as easily as Lisp macros.
In fact, since I am a professor, it was easy to perform an experiment -- I have presented the challenge of learning Lisp macros and Nim macros to students of universities, where I have good contacts. The subjects of this experiment already knew Python, and Nim has a Python like syntax, therefore, I believed that the students would consider Nim much easier than Lisp.
I gave the students the choice of three books from which to learn Lisp, On Lisp, by Paul Graham, Let Over Lambda, by Doug Hoyte, and Land of Lisp, by Conrad Barski.
After two weeks, most of the students were creating Lisp macros for lazy evaluation, loops of different kinds, reverse polish notation, and so on. However, the students had more difficulty with Nim macros than myself.
Therefore, I dropped the theory that I was a Lisp chauvinist, and created another one...
Lisp has great books in Chinese, English, French and Russian that are distributed for free throughout the Internet. Many students told me that studying Lisp from a book written by Svyatoslav Sergeevich Lavrov was inspiring, since he is the very man, who was responsible for the calculation of the trajectories of so many spaceships at the dawn of astronautics, such as Sputnik 1, Vostok 1 with Yuri Gagarin, the Venera probes, Luna 3, etc.
Besides the great book by Lavrov, On Lisp and Let Over Lambda deal almost exclusively with macros. The environment that the students use to learn Lisp, I mean PTC, is very exciting too. The PTC CAD/CAM program seems like Jarvis, the butler from Iron Man, and you can use it to design screws, gear cogs and whole machines, if you are smart enough.
To test this second theory, I decided to write a book about Nim macros, and see whether the students could learn from my book. The problem is that I don't really know Nim macros that well, and my experience in programming is not as inspiring, as is the case of Lavrov, or come to think of it even Paul Graham. In fact, I was unable to write a single interesting macro in Nim. My students did not have any success in this endeavor either. Thus, I decided to write simple macros, and ask the Nim community for help in improving them.
I will start with loop macros. The most desirable goal would be to code something like the named-let that is coded on four lines of page 45 in the book Let Over Lambda, and works straight out of the box in all implementations of Scheme. However, I will start with a repeat macro. This is what I came up with:
import macros, strutils, os
macro theMagicWord(statments: untyped): untyped =
result = statments
for st in statments:
for node in st:
if node.kind == nnkStrLit:
node.strVal = node.strVal & ", Please."
macro rpt(ix: untyped, cnt: int, statements: untyped)=
quote do:
for `ix` in 1..`cnt`:
`statements`
rpt j, paramStr(1).parseInt :
theMagicWord:
echo j, "- Give me some bear"
echo "Now"
As you noted from the above code, I read Steve Kellock's article Nim Language Highlights and I liked it. In the book that I am writing, I intend to explain how the rpt macro works, and also how to reason about macro coding in Nim. However, I must confess that I still don't have any clear ideas of how to design a Nim macro. I am not even sure whether a multi-tool such as the Lisp loop is viable in Nim. To give an opinionated answer, I think that Lisp SEXPR is very flexible, a ball of mud, in the words of Joel Moses. I am afraid that I would not be able to deal with Nim trees as easily as I can shape, mold and cast Lisp SEXPRs.
Right now, I am accepting suggestions both on how to code a named-let and also on how to improve my rpt macro. For instance, I would like to write something like the following:
rpt paramStr(1).parseInt times -> j:
theMagicWord:
echo j, "- Give me some bear"
echo "Now"
I am gladly awaiting your suggestions, and macro examples. It is, from a didactic point of view, desirable that the examples be limited to 5 lines, as is the case of chapter 3 in Let over Lambda.
While I'm not sure what kind of features the times -> j syntax should allow (or if times and -> are fixed), the simplest implementation for the second usage I can come up with:
import macros, strutils, os
macro theMagicWord(statments: untyped): untyped =
result = statments
for st in statments:
for node in st:
if node.kind == nnkStrLit:
node.strVal = node.strVal & ", Please."
proc parseArgs(cmd: NimNode): (NimNode, NimNode) =
doAssert cmd.len == 2
expectKind(cmd[1], nnkInfix)
expectKind(cmd[1][0], nnkIdent)
expectKind(cmd[1][1], nnkIdent)
expectKind(cmd[1][2], nnkIdent)
doAssert cmd[1][0].strVal == "->"
doAssert cmd[1][1].strVal == "times"
result = (cmd[0], # leave cmd[0] as is, has to be valid integer expr
cmd[1][2]) # identifier to use for loop
macro rpt(cmd: untyped, stmts: untyped): untyped =
expectKind(cmd, nnkCommand)
expectKind(stmts, nnkStmtList)
let (toIdx, iterVar) = parseArgs(cmd)
result = quote do:
for `iterVar` in 1..`toIdx`:
`stmts`
echo result.repr
# old macro
#rpt j, paramStr(1).parseInt :
# theMagicWord:
# echo j, "- Give me some bear"
# echo "Now"
rpt paramStr(1).parseInt times -> j:
theMagicWord:
echo j, "- Give me some bear"
echo "Now"
So far, after reading some macros and nim manual, i can understand that we can print the Nimnode (I dont what is this exactly) with "treeRepr" But i also know that we can get the code and parameters like an array (arg[0], arg[1]. I really really want to learn this beautiful & fun filled stuff but there is not enough tutorials with proper comments.
Hi, Juan Carlos.
Your example was of great help, indeed. I wrote a small macro based on it. Please, take a look and let me know where it can be improved:
import macros
macro iter(i:untyped, c1:untyped,
c2:untyped, stm:untyped): untyped =
result = newNimNode(nnkStmtList) # Genera un result vacio.
var mi_bucle_for=
newNimNode(nnkForStmt) # Genera un bucle for vacio.
mi_bucle_for.add(i) # Usa indice en el bucle para la iteration.
var rango_para_iterar =
newNimNode(nnkInfix).add(ident("..")).add(c1,c2) # range.
let spc= newLit("- ")
var mi_echo = newCall(ident("write"),ident("stdout"), i, spc)
var stmList= newNimNode(nnkStmtList)
stmList.add(mi_echo)
for s in stm: stmList.add(s)
mi_bucle_for.add(rango_para_iterar) # Mete range para ite.
mi_bucle_for.add(stmList)
result.add(mi_bucle_for) # Mete el bucle for en el result.
iter(i, 0, 3):
echo "Hello, world."
echo "End"
I guess Nim has a method of creating an nnkStmtList already initialized with mi_echo, but I could not find it in the documentation. I suppose that I can also add the stm list to mi_bucle_for without using the for s in stm: stmList.add(s) statement.
I am waiting for your feedback. My opinion is that macro writing in Nim is much harder than in Lisp. The same macro in Lisp:
(defmacro itr(ixx c1xx c2xx &rest body)
`(loop for ,ixx from ,c1xx to ,c2xx do
(format t "~a- " ,ixx) ,@body))
(itr i 0 3 (format t "Hello, world~%")
(format t "End~%"))
Since writing macros in Nim is less intuitive than in Lisp, I believe that a Nim macro tutorial must cover all the NimNode constructors with small examples for everything. I hope to keep counting on your help for this task.
Hi, thanks for this macro. I just changed little bit and now, i can call it like this.
forLoop( i, 0, 2) :
echo "My loop statement"
Hi, Vindaar. I am retired right now. However, one of my PhD students, Marcus Tolentino, is a professor at Ryerson University, Toronto, Canada. He is teaching Data Structures to 260 students. I myself worked at Utah State University, Logan, Utah (but I am not Mormon nor believe that God lives in the planet Kolob). By the way, officially, the Data Structures course is in Python, but we decided to choose another language for the next semester. In any case, since Nim has a Python like syntax, now that I started to figure out how macros work in Nim, and write material on the subject, I will suggest my coworkers to give Nim another try. I think macros in Nim is great to learn Data Structures.
Our group is called Victor Della-Vos, after the founder of The Bauman Moscow Technical University. The members of the Della-Vos group teach in 8 different universities. By the way, these are not top universities by any means, but even students of second rate universities need to learn and get a job.
I became very impressed on how fast you solved the problem of inserting keywords in macros. Thank you very much about the example. It is exactly what I was looking for. I am sure the students will appreciate your macro.
Hi, kcvinu.
I agree with your post in every point. Therefore, I am writing two books, one on macros, and the other on why computer languages become obsolete, and how this phenomenon impacts on long range projects, like Cyc, ACL2, Emacs and Maxima:
http://www.cs.utexas.edu/users/moore/acl2/
http://maxima.sourceforge.net/
I hope to do for Nim macros what Doug Hoyte and Paul Graham did for Lisp macros. I am writing the books with the help of Dr. Vernon Sipple, since his English is better than mine. Books and article that my group produces are signed as Della-Vos, the founder of The Bauman State University. You know, French mathematicians sign their work as Nicolas Bourbaki, who was a French General, therefore we decided to sign our work as Della-Vos, a Russian Engineer. As soon as the book on Nim macros is ready, I will let you know, so you can give us feedback.
Is https://nim-lang.org/docs/tut3.html so bad?
But yeah, macros in Nim might as well be harder than macros in Lisp. For multiple reasons. Nim's syntax doesn't make compromises for homoiconicity as most Nim code is written outside of macros. ;-)
As soon as the book on Nim macros is ready,
A Nim macro book would be really great. I read https://nim-lang.org/docs/tut3.html and https://flenniken.net/blog/nim-macros/ but have still a lot problems with macros. And recently I had a discussion with an experienced C++ developer, who told me that he will not consider switching to Nim until really good learning resources for Nim are available, which mostly means macros, threads and parallel processing and async.
Because in this case, defmacro is used as a template in nim.
$ sbcl
This is SBCL 1.5.5, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (defmacro itr(ixx c1xx c2xx &rest body)
`(loop for ,ixx from ,c1xx to ,c2xx do
(format t "~a- " ,ixx) ,@body))
(itr i 0 3 (format t "Hello, world~%")
(format t "End~%"))ITR
*
0- Hello, world
End
1- Hello, world
End
2- Hello, world
End
3- Hello, world
End
NIL
* $
$ ed tm.nim
tm.nim: No such file or directory
a
template itr(ixx,c1xx,c2xx,body:untyped):untyped =
for ixx in c1xx..c2xx:
write stdout,ixx,"- "
body
itr i,0,3:
echo "Hello, world"
echo "End"
.
w
157
q
$ nim c -r tm.nim
Hint: used config file '/usr/home/jin/src/Nim_devel/config/nim.cfg' [Conf]
Hint: used config file '/usr/home/jin/src/Nim_devel/config/config.nims' [Conf]
Hint: system [Processing]
Hint: widestrs [Processing]
Hint: io [Processing]
Hint: tm [Processing]
CC: stdlib_io.nim
CC: stdlib_system.nim
CC: tm.nim
Hint: [Link]
Hint: operation successful (21814 lines compiled; 1.067 sec total; 24.379MiB peakmem; Debug Build) [SuccessX]
Hint: /usr/home/jin/tmp/tm [Exec]
0- Hello, world
End
1- Hello, world
End
2- Hello, world
End
3- Hello, world
End
Hi, akavel.
Just to let you know that I read your interesting and quite complete exposition on macros to the book our group is writing. By the way, I always considered match a necessity of modern discipline of programming.
newTree is the canonical way to use macros nowadays (if quote do is not enough).
Here are some examples of macros from my code:
[Medium] Declarative configuration of an emulator opcodes: https://github.com/mratsim/glyph/blob/8b278c5e/glyph/snes/opcodes.nim#L85-L127
Macro: https://github.com/mratsim/glyph/blob/8b278c5e/glyph/snes/private/macros_opcodes.nim
[Complex] Declarative configuration of an assembler opcode -> machine code mapping: https://github.com/numforge/laser/blob/e660eeeb/laser/photon_jit/x86_64/x86_64_ops.nim
Macro: https://github.com/numforge/laser/blob/master/laser/photon_jit/x86_64/x86_64_op_generator.nim
Hi, mratsin.
Thank you for the link to your work. By studying your first example, I was able to create a Lisp like AST, however I could not discover a way to convert it to a Nim AST. Could you take a look at my program, and suggest a way of converting it into a functional Nim macro? By the way, I changed symbol to symb, since the compiler issues a warning against using symbol.
import os, strutils, macros
type
SExprKind = enum
IntScalar, FloatScalar, St, Sym, consp
SExpr = ref object
case kind: SExprKind
of IntScalar: intVal: int
of FloatScalar: floatVal: float
of Sym: symb: string
of St: str: string
of consp: car, cdr: SExpr
template mI(a:int): SExpr=
SExpr(kind: IntScalar, intVal: a)
template sy(s: string): SExpr=
SExpr(kind: Sym, symb: `s`)
template mS(s:string): SExpr=
SExpr(kind: St, str: s)
template car(s:SExpr) : SExpr=
if s == nil: s
else: s.car
template cdr(s:SExpr) : Sexpr=
if s == nil: s
else: s.cdr
template cons(x:SExpr, y:SExpr) : SExpr=
SExpr(kind: consp, car: x, cdr: y)
proc `$`*(se: SExpr): string =
case se.kind
of IntScalar: result= $se.intVal
of FloatScalar: result = $se.floatVal
of ST: result = '"' & se.str & '"'
of Sym: result = se.symb
of consp:
result.add("(")
var r = se
if r != nil:
result.add($r.car)
r= r.cdr
while r != nil:
result.add(indent($r.car, 2))
r= r.cdr
result.add(")")
proc walkAST(e:SExpr): NimNode =
case e.kind:
of Sym: return newIdentNode e.symb
of IntScalar: return newLit e.intVal
else: return newLit "Erro"
let ii{.compileTime.} = "i".sy.walkAST
macro iter(jj: untyped, c1: untyped, c2: untyped, stm: untyped): untyped=
let
jx= jj.strVal
p= int(c1.intVal)
result= nnkStmtList.newTree(
nnkForStmt.newTree(jx.sy.walkAST,
infix(p.mI.walkAST, "..", c2), stm))
proc main() =
iter(j, 2, paramStr(1).parseInt):
echo j, cons("hi".sy, cons(cons("world".sy, nil), nil))
main()
Hi, martsim.
I tried to develop a Lisp like consp data structures for writing macros. I used your Domain Specific Language as a starting point. The experiment is far from a success, but I decided to share the results with you, maybe you can help me. Here is what I got:
import os, strutils, macros
type
SExprKind = enum
IntScalar, FloatScalar, St, Sym, consp
SExpr = ref object
case kind: SExprKind
of IntScalar: intVal: int
of FloatScalar: floatVal: float
of Sym: symb: string
of St: str: string
of consp: car, cdr: SExpr
template mI(a:int): SExpr=
SExpr(kind: IntScalar, intVal: a)
template sy(s: string): SExpr=
SExpr(kind: Sym, symb: `s`)
template mS(s:string): SExpr=
SExpr(kind: St, str: s)
template car(s:SExpr) : SExpr=
if s == nil: s
else: s.car
template cdr(s:SExpr) : Sexpr=
if s == nil: s
else: s.cdr
template cons(x:SExpr, y:SExpr) : SExpr=
SExpr(kind: consp, car: x, cdr: y)
proc `$`*(se: SExpr): string =
case se.kind
of IntScalar: result= $se.intVal
of FloatScalar: result = $se.floatVal
of ST: result = '"' & se.str & '"'
of Sym: result = se.symb
of consp:
result.add("(")
var r = se
if r != nil:
result.add($r.car)
r= r.cdr
while r != nil:
result.add(indent($r.car, 2))
r= r.cdr
result.add(")")
let plus{.compileTime.}= "+".sy
proc walkAST(e:SExpr): NimNode =
case e.kind:
of Sym: return newIdentNode e.symb
of IntScalar: return newLit e.intVal
of consp:
if car(e) == plus:
var callTree = nnkCall.newTree()
callTree.add newIdentNode"+"
callTree.add e.cdr.car.walkAST
callTree.add e.cdr.cdr.car.walkAST
return callTree
else: return newLit "Erro"
macro def(id:untyped, x:untyped, y:untyped): untyped=
let ix= x.strVal.sy
let iy= y.strVal.sy
let body= cons(plus, cons(ix, cons(iy, nil))).walkAST
quote do:
proc `id`(`x`: int, `y`: int): int=
`body`
def(sum, cx, cy)
proc main() =
echo cons("hi".sy, cons(cons("world".sy, nil), nil))
echo sum(paramStr(1).parseInt, 8)
main()
I will have a look.
Also I forgot this repo hydra which implements a constraint parser for constraint programming / integer linear programming:
Example of syntax parsed Parser It transforms a mathematical set notation into a sequence of constraint objects described here.
For example:
With
[T,N] -> { S[t,i] : 1<=t-i<T} will be transformed into 2 constraints
@[t - i - 1 >= 0, T - t + i - 1 >= 0]
and { S[i,j] : 1<=i<=2 and 1<=j<=3 } will be transformed into 4 constraints:
@[i - 1 >= 0, -i + 2 >= 0, j - 1 >= 0, -j + 3 >= 0]
Ultimately I have another conversion to matrix form:
| 1 0 -1 |
| -1 0 2 |
| 0 1 -1 |
| 0 -1 3 |