There's no way to do the following, easily, is there? (Yes, I can always cast/unsafeAddr, etc.)
# Easy & efficient mutable iteration (no unnecessary copying)
proc iterate_over(a:var openarray[int]) =
for c in a.mitems:
echo cast[int](c.unsafeAddr)
break
var data = [1,3,5]
echo cast[int](data.addr)
iterate_over(data)
94580668782512
94580668782512
# Easy & inefficient immutable iteration (unnecessary copying)
proc iterate_over(a:openarray[int]) =
for c in a:
echo cast[int](c.unsafeAddr)
break
var data = [1,3,5]
echo cast[int](data.addr)
iterate_over(data)
93899394672560
140734195148608
proc iterate_over(a:openarray[int]) =
for c in cast[var array[3,int]](a.unsafeAddr).mitems:
echo cast[int](c.unsafeAddr)
break
var data = [1,3,5]
echo cast[int](data.addr)
iterate_over(data)
94865883576240
94865883576240
I strongly assume that
# Easy & inefficient immutable iteration (unnecessary copying)
proc iterate_over(a:openarray[int]) =
IS a efficient proc call in general case. Note that small value parameters may be copied, but large parameters may be accessed by reference. So it may depend on size of your array parameter. And efficiency is still improving, I heard about destructors, move semantics and more are coming soon. So maybe while investigating addresses of parameters may be interesting, doing real life benchmarks may give more correct performance information.
@Stefan_Salewski Good point about main vs global! I'd forgotten about that - unfortunately it doesn't change the result.
@mratsim Looking at the generated c code, it seems openarray iteration is indeed done with copy, even when an array of 1 billion elements (@Stefan_Salewski).
I'm not picking on nim here, rather I'm getting ready to teach a class on nim this summer at my university to some grad students and professors. I know some of the people who will be there and that they will ask about efficiency from a c++ perspective. I'll take your suggestion and try some actual data - will report back.
I've been following the work on move semantics - looking forward to it!
I've been following the work on move semantics - looking forward to it!
Strange how these things tend to align, just today I've been thinking about "cursor" detection which eliminates copies like yours. Anyway, this version is to be preferred for now, it's sad but at least it doesn't require unsafe code:
proc iterate_over(a:openarray[int]) =
for i in 0..<a.len:
echo a[i]
New test, still has full copy of items for iteration.
## performs full copy in iteration of immutable
proc iterate_over(a:openarray[int]) =
for c in a:
echo c
when isMainModule:
let data = newSeq[int](1000000000)
iterate_over(data)
N_LIB_PRIVATE N_NIMCALL(void, iterate_over_9cD7bqNN5ugFAFo1Y0BNo9cw)(NI* a, NI aLen_0) {
nimfr_("iterate_over", "trial.nim");
{
NI c;
NI i;
c = (NI)0;
// var i = 0
nimln_(2247, "system.nim");
i = ((NI) 0);
{
// while i < len(a):
nimln_(2248, "system.nim");
while (1) {
tyArray_nHXaesL0DJZHyVS07ARPRA T4_;
NI TM_dkUgyuXqTi3PT7xs72dvsA_3;
// while i < len(a):
// while i < len(a):
if (!(i < aLen_0)) goto LA3;
// yield a[i]
nimln_(2249, "system.nim");
if ((NU)(i) >= (NU)(aLen_0)) raiseIndexError();
c = a[i];
// echo c
nimln_(3, "trial.nim");
nimZeroMem((void*)T4_, sizeof(tyArray_nHXaesL0DJZHyVS07ARPRA));
// echo c
T4_[0] = nimIntToStr(c);
echoBinSafe(T4_, 1);
// inc(i)
nimln_(2250, "system.nim");
TM_dkUgyuXqTi3PT7xs72dvsA_3 = addInt(i, ((NI) 1));
i = (NI)(TM_dkUgyuXqTi3PT7xs72dvsA_3);
} LA3: ;
}
}
popFrame();
}
## Performs ref copy in iteration of mutable
proc iterate_over(a:var openarray[int]) =
for c in a.mitems:
echo c
when isMainModule:
var data = newSeq[int](1000000000)
iterate_over(data)
// <redacted>
c = (&a[i]);
// <redacted>
This gets me close, but I lack a way to say Hey nim, cast to a read-only reference
iterator refitems[T](a:openarray[T]):var T {.inline.} =
## iterates over each item of `a` so that you can ~~modify the yielded value~~ get a reference to the yielded value
var i = 0
while i < len(a):
yield cast[var T](a[i].unsafeAddr)
inc(i)