After diving deep in the wonderful world of metaprogramming today, I'm happy to share the loopfusion package.
For now you need to install it through nimble install https://github.com/numforge/loop-fusion.
This will give you 2 macros forEach and forEachIndexed that will allow you to iterate and operate over any number of sequences of any same type.
Examples:
import loopfusion
let a = @[1, 2, 3]
let b = @[11, 12, 13]
let c = @[10, 10, 10]
forEach [x, y, z], [a, b, c]:
echo (x + y) * z
forEachIndexed j, [x, y, z], [a, b, c]:
echo "index: " & $j & ", " & $((x + y) * z)
120
140
160
index: 0, 120
index: 1, 140
index: 2, 160
Looks interesting, but why does the macro generate a for loop plus a zip iterator (which itself contains another for loop) instead of just a single for loop with injected symbols?
forEachIndexed j, [x, y, z], [a, b, c]:
body
# would expand to:
for j in 0 ..< commonLengthWithSanityCheck(a, b, c):
let x = a[j]
let y = b[j]
let z = c[j]
body
Note: The length of b in your example differs. I was wondering if this is not handled, but I think it is, so probably just a typo :).
Syntactically I would prefer something that aligns the loop variables with the containers, i.e.,
forEachIndexed j, x in xs, y in ys, z in zs:
...
If we take a simple seq this is equivalent to the following
iterator foo_item[T](s: seq[T]): T =
for i in 0 ..< s.len:
yield s[i]
let a = @[1, 2, 3, 4]
for val in foo_item(a):
echo val
The reason why I use an intermediate zip is for Arraymancer. This is a proof of concept before I generalize it to tensors.
On tensors, it would be wrapped in an OpenMP template that splits the work on multiple cores like here. I also really need an iterator because the "next" item is not straightforward if the tensor is not contiguous, see here.
Bonus, I added loopFusion:
import loopfusion
let a = @[1, 2, 3]
let b = @[11, 12, 13]
let c = @[10, 10, 10]
let d = @[5, 6, 7]
loopFusion(d,a,b,c):
let z = b + c
echo d + a * z
Syntactically I would prefer something that aligns the loop variables with the containers
Yes, this looks nice.
Btw, a similar syntax exists in Julia, but what it does is completely different - it is syntactic sugar for nested for-loops.
Indeed that's a much better syntax, I have absolutely no idea how to introduce it though.
For me, a step forward to a better syntax would be just introducing in instead of , between elements and containers:
forEach [x, y, z] in [a, b, c]:
Also, changing square brackets to parentheses looks more intuitive to me, as I expect zip to yield tuples:
forEach (x, y, z) in (a, b, c):
Done
import loopfusion
let a = @[1, 2, 3]
let b = @[11, 12, 13]
let c = @[10, 10, 10]
forEach x in a, y in b, z in c:
echo (x + y) * z
# 120
# 140
# 160
# i is the iteration index [0, 1, 2]
forEach i, x in a, y in b, z in c:
d.add (x + y) * z * i
echo d # @[0, 140, 320]
The tricky thing was the difference between typed and untyped macro. I can't do all in the same one.
I have removed the ForEachIndexed and added it to forEach. It was a workaround because somehow I use overloading previously.
I've updated the package, it now supports in-place mutation of the inputs and sequence of different subtypes as input.
Here is a snippet of the syntax.
import loopfusion
block: # Simple
let a = @[1, 2, 3]
let b = @[11, 12, 13]
let c = @[10, 10, 10]
forEach x in a, y in b, z in c:
echo (x + y) * z
# 120
# 140
# 160
block: # With index
let a = @[1, 2, 3]
let b = @[11, 12, 13]
let c = @[10, 10, 10]
var d: seq[int] = @[]
forEach i, x in a, y in b, z in c:
d.add i + x + y + z
doAssert d == @[22, 25, 28]
block: # With mutation
var a = @[1, 2, 3]
let b = @[11, 12, 13]
let c = @[10, 10, 10]
forEach x in var a, y in b, z in c:
x += y * z
doAssert a == @[111, 122, 133]
block: # With mutation, index and multiple statements
var a = @[1, 2, 3]
let b = @[11, 12, 13]
let c = @[10, 10, 10]
forEach i, x in var a, y in b, z in c:
let tmp = i * (y - z)
x += tmp
doAssert a == @[1, 4, 9]
block: # With iteration on seq of different types
let a = @[1, 2, 3]
let b = @[false, true, true]
forEach integer in a, boolean in b:
if boolean:
echo integer
Yes, I actually started with a macro that produces a variadic zip. It's still untouched in my code.
The reason is that the following does not compile, error is Error: iterator within for loop context expected
template zipTest(arguments: varargs[untyped]): untyped =
iterator zipZipZip(a: seq[int], b: seq[bool], c: seq[int], d: seq[float]): (int, bool, int, float) =
let size0 = a.len
for i in 0..<size0:
yield (a[i], b[i], c[i], d[i])
zipZipZip(arguments)
for a, b, c, d in zipTest(@[1,10,15], @[false, false, true], @[3,2,1], @[4.0,5.0,6.0]):
echo (a,b,c,d)
since we can't create an iterator within a for loop call
Bonus, I just added forEach expressions. It works if there is a return value at each step. I still have to figure out conditional returns.
Also we have to use parenthesis in expression mode.
import loopfusion
let a = @[1, 2, 3]
let b = @[4, 5, 6]
let c = forEach(x in a, y in b):
x + y
doAssert c == @[5, 7, 9]