Mirror of forum.nim-lang.org

2971 :: [SOLVED IMPOSSIBLE] Closure iterators that yield mutable values

• mratsim (orginal) [2017-05-25T11:59:36+02:00] view original

  To avoid copies and index computation when mutating slices of sequence, I would need to use iterator that yields mutable values. This closure iterator will be passed to another zip iterator so I can iterate on 2 iterators at the same time and modify the values yielded by the first one.

  1. Small example of an inline iterator

  Please note that in my full use case, some values will be skipped or iteration may not be in order.

  var s = @[1, 2, 3, 4]
  
  ## inline iterator with mutable output
  for val in s.mitems:
      val *= 2
  
  echo s #@[2, 4, 6, 8]
  Run

  2. A closure iterator with immutable yield can be written like this

  proc values*[T](s: seq[T]): auto {.noSideEffect.}=
    return iterator(): T =
      for val in s:
        yield val
  
  let it = s.values
  
  for val in it():
    echo val
  Run

  3. Now this is not working (expression has no address)

  proc mvalues[T](s: var seq[T]): auto {.noSideEffect.}=
    return iterator(): var T =
      for val in s:
        yield val
  
  var mit = s.mvalues
  Run

  4. Now this is my target goal. (Reminder, "mvalues" and "values" iterator in the full case will have a non-trivial iteration logic)

  for old_val, val in zip(seq1.mvalues, seq2.values):
    old_val = val
  Run

  5. For reference this is the zip iterator that accepts 2 closure iterators

  iterator zip[T1, T2](inp1: iterator(): T1, inp2: iterator(): T2): (T1, T2) {.noSideEffect.} =
    ## Inline iteration on 2 closure iterators at the same time
    
    let it1 = inp1
    let it2 = inp2
    while true:
      let val1 = it1()
      let val2 = it2()
      if finished(it1) or finished(it2):
        break
      yield (val1, val2)
  Run

  Is there a way to make 4 works by modifying 3 and 5? I suspect it's not possible for a closure iterator to borrow a var seq.

---

• Parashurama (orginal) [2017-05-25T16:14:32+02:00] view original

  I think what you are missing is mitems. (from 'system.nim')

  proc mvalues[T](s: var seq[T]): auto {.noSideEffect.}=
    return iterator(): var T =
      for val in s.mitems:
        yield val
  
  var mit = s.mvalues
  Run

---

• mratsim (orginal) [2017-05-25T16:30:02+02:00] view original

  Indeed ! First mistake.

  Now I have illegal capture

  playground_mutiter.nim(9, 12) template/generic instantiation from here
  lib/system.nim(3590, 15) Error: illegal capture 's'
  Run

  var s = @[1, 2, 3, 4, 5]
  
  
  proc mvalues[T](s: var seq[T]): auto {.noSideEffect.}=
    return iterator(): var T =
      for val in s.mitems:
        yield val
  
  var mit = s.mvalues
  
  for i in mit():
    i *= 2
  Run

---

• mratsim (orginal) [2017-05-25T16:37:56+02:00] view original

  As suspected, it's not possible to yield mutable values with a closure iterator as it would violate memory safety, similarly to this github issue

  The only efficient alternative to my current way is if we could actually chain inline iterators: https://forum.nim-lang.org/t/2972.

  Edit: Current way example

---

• cdome (orginal) [2017-05-26T11:48:21+02:00] view original

  Hi mratsim,

  IMO, chaining inline iterators is much butter solution as the performance is as good as manually crafted loop. Closure iterators are slower, as they usually copy on capture and they need to maintain the state

---

• mratsim (orginal) [2017-05-26T15:56:33+02:00] view original

  @cdome I would LOVE to chain them but as you can see in this topic I was unable to make even enumerate(it) or zip(it1, it2) work at all :/

  I would switch immediately to inline iterator everywhere if someone can show me the way!

---