According to the documentation, inline operators can be passed as parameters only to other inlining code facilities like templates.
I am trying to do exactly this, but I am not getting this to work. First, an inline iterators over pairs:
iterator zipped[X, Y](xs: seq[X], ys: seq[Y]): auto {. inline .} =
for i in 0 .. < xs.len:
yield (xs[i], ys[i])It can be used like this:
let q = @[1, 2, 3]
let r = @[4, 5, 6]
for a, b in zipped(q, r):
echo a
echo bNow, I would like to try to map an expression over those pairs. For instance, I would like to have the following
for t in map2(zipped(q, r), a + b):
echo texpand to
for i in 0 .. < q.len
let a = q[i]
let b = r[i]
echo a + bI thought this would be doable with unhygienic templates. Copying the implementation of foldl I tried
template map2(xs, op: untyped): auto =
iterator inner: auto {.inline.} =
for x, y in xs:
let a {.inject.} = x
let b {.inject.} = y
yield op
innerUnfortunately, when I try to use it, I get
Error: type mismatch: got (iterator (): int{.inline, locks: 0.})
but expected one of:
system.items(a: array[IX, T])
system.items(E: typedesc[enum])
system.items(s: Slice[items.T])
system.items(a: string)
system.items(a: openarray[T])
system.items(a: seq[T])
system.items(a: set[T])
system.items(a: cstring)Is there anything I am doing wrong?
Some unrelated comments, feel free to ignore them: I'm making small progress with sequtils, I'll try to speed things up, sorry for the delay. When I wrote an iterator library ("lazy", which doesn't compile now, I should take care), I had to use {.closure.} iterators all the time, it was the only way to make them work. If you are interested in benchmarks, I have a branch for them (the results are outdated, I'll try to update them). I believe that zipped should go for the minimum of the two sequences (like in python's itertools), so something like 0..<min(xs.len,ys.len). And please test all your templates twice in your unit tests, your inner iterator might behave as a proc (I can't test it right now), and proc are not {.gensym.}-ed (therefore you can have conflicts when calling it twice).
I'm very open to talk about these stuff, and I'll try to help you with this particular question.
Cheers, Peter
Yeah, zipped was just a small example, in a concrete case where I know the lengths are equal by construction. The reason I am going with inline iterators is that I am doing this inside a benchmark where very low latency is desired. I can achieve it with procedural code, but as soon as I operate with map over sequences, with all intermediate allocations, the time goes us by 10x (to be fair we are talking about ~15microseconds vs ~200microseconds).
I also tried to see what happens with lazy, but I had compilation issues. It would be nice if you updated it for devel, if you have time (but if I understand correctly, the plan is to eventually have this inside sequtils)
Just a quick answer for the benchmarks: please try to add {.inline.} at the end of the map implementation in system.nim (careful, there are many map functions there). This had a huge speed up in my benchmarks with gcc. Basically, gcc could inline my function.
The library lazy is super slow because the seqs are copied in toIter(). This is why I'm focusing on lambdalifting right now.
Well, I would rather not rely on modifying the system libraries directly, even if this worked for this particular benchmark.
I tried to look in lazy, but I am not sure where the copy is happening. It seems to me that toIter should capture a reference to the seq, so just a pointer is copied. Could you give me some more explanation?
I suggested to modify the system.nim because I have a pending PR which adds {.inline.} to map: https://github.com/nim-lang/Nim/pull/3372
You are right, it should be only a pointer copy. It is not that simple to do. This issue should give more background: https://github.com/nim-lang/Nim/issues/3377