I've been reading about macros, mainly how they're used in LISP (such as https://stackoverflow.com/questions/267862/what-makes-lisp-macros-so-special , https://jameshunt.us/writings/why-lisp/ and Andy Gavin's explanation of GOOL), but the thing that I can't seem to find on any discussion is the why or when you should use a macro.
What I mean is, many times, the examples used are too simple, like "I want to define a macro for calculating the square of a number", followed by a lengthy explanation of how it works, sometimes comparing LISP macros to C macros, or showing how to make a macro that is the same as a specific function. Ok, but why should I bother with that if I can make a function/procedure that does the same thing? Even the example in the Nim by Example talks about "repetition", but the macro code only reduces a bit of what's needed to define a new class. In my untrained eyes, that doesn't look worth the effort.
I understand the "rewriting code at runtime", but, again, why or when should I actually care about that? What are some real use cases where a macro "works better" than a function? Are there any good resources (books, courses, posts) that explain this in depth?
In Nim, macros aren't really meant to replace functions. They operate on Nim's basic constructs (procedures, types, variables) so programmers can make things less (or more) complicated, depending on the usage. Nim is designed to be a minimal language that can be extended with macros.
A good example of this that is implemented in Nim's standard library is `async`. This module implements asynchronous IO, and is equivalent to the language feature of the same name in Javascript. The cool thing here is that in Nim this is just a library, and nothing had to be added to the compiler. (macro definition here)
Of course this great power is a double-edged sword, and can be misused like most other things. To avoid unneeded complexity, most Nim devs recommend trying procs first, then templates, and then macros to solve problems.
Personally I'm a hands-on person, so I dived straight into it and started making mistakes, learning as I went. There's a book called "Nim in Action" (link) people recommend though.
To add to @rb3's answer and resolve your confusion a bit: > I understand the "rewriting code at runtime"
Nim macros operate on compile-time - they rewrite the code by the logic you've written. The resulting program binary doesn't have anything related to macros.
I tend to use meta-programming quite a bit when I write code. The benefit of a macro over a simple procedure is that you can re-write code during compile time. This can be used to create a nicer interface for yourself, reducing errors and increasing readability. I wrote an article about that here: https://peterme.net/metaprogramming-and-read-and-maintainability-in-nim.html
It can also be used to turn high-abstraction code into something more low-level. Zero-functional is a nice example of a macro that turns one style of programming which would normally be expensive to do in Nim into something that's much cheaper. I also use this quite a bit in my custom keyboard firmware (along with the great type-system) both for increasing performance, but also the aforementioned read and maintainability. You can see my NimConf talk on the subject or my video series where I show the entire implementation and explain what I'm doing and why as I go along.
Macros and procs are two vastly different tools. A procedure is for writing code that runs(at runtime or compile time). A macro is for writing code that expands code at compile time. Macros enable introspection and automation of writing code. Due to this you can do many magical things with macros, say you have a serialization library and want to only serialized fields of an object that are tagged, this can be done with a macro and a pragma. Macros should not be used in place of procedures, they should be used since procedures cannot be.
The following is a small example that is hopefully easily understandable to showcase what macros bring that procedures cannot do. Unpacking tuples/arrays into proc calls for less redundant code.
import std/macros
macro `<-`(p: proc, args: tuple): untyped =
## Takes a proc and varargs, unpacking the varargs to the proc call if it's a tuple or array
result = newCall(p) # Makes result `p()`
echo args.treeRepr # Shows the input tuple
for arg in args:
let
typ = arg.getType # Gets arg typ
isSym = typ.kind == nnkSym
echo typ.treeRepr # Shows the type we got
if not isSym and typ[0].eqIdent("tuple"): # This is a tuple so unpack values from arg
for i, _ in typ[0..^2]:
result.add nnkBracketExpr.newTree(arg, newLit(i))
elif not isSym and typ[0].eqIdent("array"): # This is a tuple so unpack values from arg
for i in typ[1][1].intVal .. typ[1][2].intVal:
result.add nnkBracketExpr.newTree(arg, newLit(i))
else: # Otherwise we just dumbly add the value
result.add arg
proc doThing(a, b: int) = echo a, " ", b
proc doOtherThing(a, b: int, c, d: float) =
doThing <- (a, b)
echo c, " ", d
doThing <- (10, 20)
doThing <- ([10, 20],)
doThing <- ([10], (20,))
doOtherThing <- ([10, 20], (30d, 40d))
doOtherThing <- (10, 20, 30d, 40d)
This is the rule which works for me because I don't know anything about macro and template.
If a proc works, use a proc. Elif, make a problem-specific solution. Else drop the project and start with another language.
Recently published Zen of Nim has a nice section on meta programming features of Nim with various examples: https://nim-lang.org/blog/2021/11/15/zen-of-nim.html
Regarding the why it says:
“We need to be able to do locking, logging, lazy evaluation, a typesafe Writeln/Printf, a declarative UI description language, async and parallel programming! So instead of building these features into the language, let’s have a macro system.”
(and I have to say something more or else the formatting is broken; I imagine that a new release of the forum using 1.6 would solve also this thanks to the great work @amr has been doing to fix related bugs)
But in practice I think it's better to avoid using it and keep things simple.
But what if in order to "keep things simple" you need a macro...
But I know that I don't have any problem with modularity in other languages.
You want module extension. In fact, you have a module A with type X and a module B with a type Y and then you extend A with B that can be read as A[B] so you "plugged in" the B into the A. Now you can have access to both A[B].X and A[B].Y with the same object. You can find the feature everywhere, especially in languages that inherit from Java. This is a convenient feature and typically some runtime (virtual tables) is involved. The programmer is satisfied because of the convenience and thinks that it is "modular". However, If you want to do it efficiently, you have to recompile both A und B if you change one of them. So there is not much modularity here.
But I think about separate compilation. We can automatize this (see post above) or we can do it with module descriptions like C++'s upcoming module feature. (They are a bit late....).
So our focus is a bit different.
But I know that I don't have any problem with modularity in other languages.
So try some F#, ML, Ocaml, Haskell...
And again, "everything depends on everything else" is the opposite of "modularity", it cannot hurt to use the correct terms.
So, do you want to go for "a program is a single (albeit large) equation"
or do you want to go for orthogonalization (factoring out parts of the program and their equations will be evalutated seperately" ?
If we're going to talk about modularity, one could do worse than making modules instantiatable using type replacements. Take strutils presently it relies on system.string, so if someone makes their own string type it requires rewriting a fair bit of code, but if the module was written with generics in mind(using init/new that take parameters instead of newString) one could easily allow replacing all usages of string with MyString and instantiate a module with a given type. For a code example looking at a mathematical vector library:
# vectormath.nim
type Vec2 = (float32, float32)
proc `+`*(a, b: Vec2): Vec2 = Vec2.init(a[0] + b[0], a[1] + b[1])
proc `-`*(a, b: Vec2): Vec2 = Vec2.init(a[0] - b[0], a[1] - b[1])
proc `*`*(a, b: Vec2): Vec2 = Vec2.init(a[0] * b[0], a[1] * b[1])
proc `/`*(a, b: Vec2): Vec2 = Vec2.init(a[0] / b[0], a[1] / b[1])
# myModule.nim
type Vec2Arr = array[2, float32]
proc init(_: typedesc[Vec2Arr], x, y: float32): Vec2Arr = [x, y]
import vectormath[Vec2: Vec2Arr] # Replaces all `Vec2` instances(doesnt declare the type in this module) with `Vec2Arr`
# vectormath has a internal concept that this needs to match,
# in this case `init`, `[]`, and operators for the fields
assert [10f32, 10f32] + [-1f32, 3f32] == [9f32, 13f32] # We can use operators normally.
assert [10f32, 10f32] * [-1f32, 3f32] == [-10f32, 30f32] # We can use operators normally.
If we're going to talk about modularity, one could do worse than making modules instantiatable using type replacements.
This would allow for (re)instantiable module-wide type parameters. Unfortunately, Araq already dismissed it: "too ugly".
The topic has been discussed extensively in : https://forum.nim-lang.org/t/7925#50452 (some problems about Nim's overloadings could be identified as a side effect)
where modules now can be regarded as "traits" and "mixins" at the same time. As said, Araq is not interested. But now, a have a point. If fast (separate) compilation is a value in itself, then Nim should take it. IC is a pure technical thing instead, it will not hold for long time.
The key is: the compiler could derive itself the appropriate export qualifiers for orthogonal solutions.