Hello, Nim beginner here.
I'm struggling with what's the best way of implementing access to the memory data of a particular sequence (only interested in byte type for now, not generic) for partial update.
var data_out = newSeq[byte](8)
shallow(data_out)
var qqq = data_out[6..7] #expecting qqq to be a slice using the same memory as data_out
# due to the shallow mark on the master variable
If I pass qqq to a procedure that changes the parameter [e.g. proc UpdateInput(qqq: var openarray[byte]): = ...] then qqq gets changed but data_out does not. So it seems qqq is but a decoupled copy.
So far I've had to resort to implementing my own wrapper "slice" to extract an addr and do the 'ugly' C casts (back and forth for get/set API). The wrapper changes the API I present to the upper layers, I would have liked to be able to pass a var openarray still.
Is there a more elegant way of updating windows/ranges/slices (whatever you want to call them) of a sequence/array?
Thank you, Lucian
Could you describe what you're trying to achieve overall? The shallow() function only affects the copying semantics for an entire sequence, not part of it.
Currently the only way to achieve what you want is to create a wrapper object that holds both the sequence and the bounds, like so:
type SeqView[T] = object
data = ref seq[T]
bounds: Slice[int]
Of course, if you have control over the procedures that you're calling, you can just pass in the sequence and bounds:
proc incrementRange[T](s: var seq[T], bounds: Slice[int]):
for i in bounds:
s[i] += 1
Overall I was trying to implement a "binary" style helper library for seq[uint8] to/from integers. Direct access to the desired index within seq would be useful and I was hoping some simple baked into laguage construct would give me that.
As you've mentioned, seems the only way for now is with a wrapper (mine was pointer and legth; your ref and slice is tidier). I will chage the API presented then to take an SeqView then.
Thanks again, Lucian
Varriount, I was trying to construct a working example from your proposal with the wrapper:
type SeqView[T] = object
data: ref seq[T]
bounds: Slice[int]
var
seq1 = @[1, 2, 3, 4, 5, 6]
sv = SeqView[int](data: seq1, bounds: 2..4)
seq2.nim(7, 29) Error: type mismatch: got (seq[int]) but expected 'ref seq[int]'
What am I doing wrong? Many thanks in advance.
try this
var seq1: ref seq[int]
new(seq1)
seq1[] = @[1, 2, 3, 4, 5, 6]
var sv = SeqView[int](data: seq1, bounds: 2..4)
I made this really quick:
type SeqView[T] = object
data: ref seq[T]
bounds: Slice[int]
proc box[T](x: T): ref T =
new(result); result[] = x
iterator items[T](sv: SeqView[T]): T =
for pos in sv.bounds:
yield sv.data[pos]
var
seq1 = @[1, 2, 3, 4, 5, 6]
sv = SeqView[int](data: box(seq1), bounds: 2..4)
for el in sv:
echo el
Please be aware that sv is a read only view as it holds a copy of the data. It cannot be used for updating the original sequence.
# update and print original data
for i in 2..4:
seq1[i] += 1
echo seq1
# test if true reference or copy
for el in sv:
echo el
@lucian Yep. But that can be changed.
Here an extended example:
type SeqView[T] = object
data: ref seq[T]
bounds: Slice[int]
proc box[T](x: T): ref T =
new(result); shallowCopy(result[], x) # beware :)
iterator items*[T](sv: SeqView[T]): T =
for pos in sv.bounds:
yield sv.data[pos]
iterator mitems*[T](sv: SeqView[T]): var T =
for pos in sv.bounds:
yield sv.data[pos]
var
seq1 = @[1, 2, 3, 4, 5, 6]
sv = SeqView[int](data: box(seq1), bounds: 2..4)
echo "a:"
for el in sv:
echo el
# inc using the mitems on the "slice"
for el in mitems(sv):
inc el
echo "\nb:"
for el in sv:
echo el
# update and print original data
for i in 2..4:
inc seq1[i]
echo "\nc:"
for el in seq1:
echo el
echo "\nd:"
# test if true reference or copy
for el in sv:
echo el
Or even nicer:
type SeqView[T] = object
data: ref seq[T]
bounds: Slice[int]
proc seqView*[T](x: seq[T], bounds: Slice): SeqView[T] =
var rf: ref seq[T]
new(rf)
shallowCopy(rf[], x) # beware :)
SeqView[T](data: rf, bounds: bounds)
iterator items*[T](sv: SeqView[T]): T =
for pos in sv.bounds:
yield sv.data[pos]
iterator mitems*[T](sv: SeqView[T]): var T =
for pos in sv.bounds:
yield sv.data[pos]
var
seq1 = @[1, 2, 3, 4, 5, 6]
sv = seqView(seq1, 2..4)
echo "a:"
for el in sv:
echo el
# inc using the mitems on the "slice"
for el in mitems(sv):
inc el
echo "\nb:"
for el in sv:
echo el
# update and print original data
for i in 2..4:
inc seq1[i]
echo "\nc:"
for el in seq1:
echo el
echo "\nd:"
# test if true reference or copy
for el in sv:
echo el
Here something I had stored for some time: memviews
It is not as safe as OderWat version as it don't keeps a pointer to the start of the data for the GC keep track of it, but it is a bit more general and means less bytes to pass around (2 words instead of 3). I also sketched a safer version, but no implementation of it yet.
I don't know if I should make a nimble library out of this, as I don't know if it is fundamentally broken, too unsafe, etc. But it works for my code (the converters used to work one year ago, but some update messed it up I guess).
I have nearly identical structure (to memviews) in reactor.nim:
https://github.com/zielmicha/reactor.nim/blob/master/reactor/datatypes/basic.nim
It may be worth adding structure like this to stdlib.