Hello,
There is something that I feel is missing in the Nim standard library, a good type structure for I/O streams. I am used to the Go language where streams are I think really well thought. In particular, there are multiple implementation of an input stream and an output stream allowing to perform zero-copy where possible by casting a type of stream to another type of stream, thanks to the duck-typing nature of the Go language.
I wanted to see if I could do something similar in Nim, and I need to declare stream types that are abstract enough to be implemented for files, network, encoding or decoding algorithms. Looking at the possibilities, I found the contepts feature in Nim in the experimental manual that I never used yet.
Here is how I would declare the stream types:
import std/options
import std/asyncfutures
type
Castable* = concept x
type T = auto # io.nim line 6
x.to(type T) is Option[T] # io.nim line 7
## Reader represents a readable stream. A function read must be available that
## takes an openArray, reads up to high(buffer) items in it and return the
## number of items read. At end of stream, none is returned to signal end of
## file
Reader*[T] = concept s of Castable
s.read(openArray[T]) is Option[int]
WriterTo*[T] = concept s of Castable
s.write_to(Writer[T]) is Option[int]
Writer*[T] = concept s of Castable
s.write(openArray[T], ref int)
ReaderFrom*[T] = concept s of Castable
s.read_from(Reader[T], ref int)
Closer* = concept s of Castable
s.close()
ReadWriter*[T] = concept s of Reader[T], Writer[T]
ReaderAsync*[T] = concept s of Castable
s.read_async(openArray[T]) is Future[Option[int]]
WriterToAsync*[T] = concept s of Castable
s.write_to_async(WriterAsync[T]) is Future[Option[int]]
WriterAsync*[T] = concept s of Castable
s.write_async(openArray[T], ref int) is Future[void]
ReaderFromAsync*[T] = concept s of Castable
s.read_from_async(ReaderAsync[T], ref int) is Future[void]
Now, trying to make use of this, I tried implementing the Castable concept, but did not succeed:
type Null* = ref object
var null: Null = new(Null)
proc id[T](t: typedesc[T], val: T): T = val
proc to*[T](self: Null, t: typedesc[T]): Option[T] =
when T is Castable:
result = some(T(self))
else:
result = none(T)
var nullCastable: Castable = id(Castable, null) {.explain.} # null.nim line 16
Here, I'm using the id method that returns the identity to get better error messages, and I got:
/home/mildred/Projets/nim-io/src/null.nim(16, 32) Error: type mismatch: got <typedesc[Castable], Null>
but expected one of:
proc id[T](t: typedesc[T]; val: T): T
first type mismatch at position: 2
required type for val: T
but expression 'null' is of type: Null
/home/mildred/Projets/nim-io/src/io.nim(7, 6) Castable: type mismatch: got <Alias, typedesc[T]>
but expected one of:
proc to[T](self: Null; t: typedesc[T]): Option[T]
first type mismatch at position: 2
required type for t: typedesc[T]
but expression 'typeof(T)' is of type: typedesc[T]
expression: to(x, typeof(T))
/home/mildred/Projets/nim-io/src/io.nim(7, 9) Castable: expression '' has no type (or is ambiguous)
/home/mildred/Projets/nim-io/src/io.nim(6, 5) Castable: concept predicate failed
expression: id(Castable, null)
What's really strange is the error:
required type for t: typedesc[T]
but expression 'typeof(T)' is of type: typedesc[T]
It seems I'm missing something here to understand the error.
While there are probably many issues in this code, one thing I can comment on is this: Concepts are non-concrete typeclasses as is, they don't have vtable semantics. Using a concept type for a function value is equivalent to using auto but with a type restriction.
proc id[T](t: typedesc[T], val: T): T = val
You call id here with id(Callable, null). The truth is that generic parameters are not meant to be non-concrete types, and here you are trying to force the generic parameter to be Callable. If you try id[Callable](Callable, null), you get a cryptic "expression cannot be called" error. If you remove the typedesc parameter (ie id[Callable](null)), you get:
Error: type mismatch: got <Null>
but expected one of:
proc (val: Castable){.noSideEffect, gcsafe, locks: 0.}
There is a (seemingly) completely undocumented, no-experimental-flag feature called "new style concepts" which are like this and should be easier to work with:
type
Castable* = concept
proc to(x: Self, T: type): T
They are still non-concrete types though, and you will need some kind of macro system that generates a proc table, or manually write proc table types for each type. I believe there are multiple libraries that already do this, a random one being traitor .
Sorry for the poorly written reply, I had to write this quickly
So in essence, you tell me that a variable type cannot be a concept, unless it's in a generic context (such as a procedure body). I understand better now.
So perhaps my idea might work (at the cost of rending the complete program using those types generic) but perhaps I'm not taking the right approach. I'll try to see what I can do with manual vtables, I don't ming writing them myself.
There may be others as well
Thanks for the suggestions, I think I'll go with runtime interfaces.
However, I need a specific feature with the interfaces : being able to cast from an interface object (say Reader) to another (WriterTo) to ensure limited copies. And instead of depending on a specific macro package, I'd prefer to have a manually implementable interface type if possible.
That's not concepts, but i could make something that works (without sugar):
import std/options
type
Iface*[T] = ref object
original*: ref RootObj
vtables*: seq[ref RootObj]
vtable*: T
None = ref object of RootObj
let noneVtable: None = new(None)
proc to*[S, T](self: S, t: typedesc[T]): Option[T] =
var res: T
var vt: seq[ref RootObj]
when compiles(vt = self.vtables()):
vt = self.vtables()
else:
vt = self.vtables
for i, v in vt.pairs():
if v of typeof(res.vtable):
return some(Iface[typeof(res.vtable)](
original: cast[ref RootObj](self),
vtables: vt,
vtable: cast[typeof(res.vtable)](v)))
result = none T
########################
type
ReaderVtable*[S, T] = ref object of RootObj
read*: proc(self: S, buffer: openArray[T]): Option[int]
WriterVtable*[S, T] = ref object of RootObj
write*: proc(self: S, buffer: openArray[T], num: ref int)
Reader*[T] = Iface[ReaderVtable[ref RootObj, T]]
Writer*[T] = Iface[WriterVtable[ref RootObj, T]]
# Should be auto generated from vtable type if possible
proc read*[T](self: Reader[T], buffer: openArray[T]): Option[int] =
self.vtable.read(self.original, buffer)
# Should be auto generated from vtable type if possible
proc write*[T](self: Writer[T], buffer: openArray[T], num: ref int = nil) =
self.vtable.write(self.original, buffer, num)
########################
type Null[T] = ref object of RootObj
proc read*[T](self: Null[T], buffer: openArray[T]): Option[int] =
result = some(buffer.len)
# Should be auto generated as this is just a type cast for self
proc readImplem[T](self: ref RootObj, buffer: openArray[T]): Option[int] =
cast[Null[T]](self).read(buffer)
proc write*[T](self: Null[T], buffer: openArray[T], num: ref int) =
echo "write"
discard
# Should be auto generated as this is just a type cast for self
proc writeImplem[T](self: ref RootObj, buffer: openArray[T], num: ref int) =
cast[Null[T]](self).write(buffer, num)
# Should be auto generated if possible although manual declaration makes it explicit
proc vtables*[T](self: Null[T]): seq[ref RootObj] =
let reader = ReaderVtable[ref RootObj, T](
read: readImplem
)
let writer = WriterVtable[ref RootObj, T](
write: writeImplem
)
return @[
cast[ref RootObj](reader),
cast[ref RootObj](writer)
]
var foo = new(Null[char])
var reader = foo.to(Reader[char]).get
echo reader.read(@['q'])
var writer = reader.to(Writer[char]).get
writer.write(@['q'], nil)
Posted here too: https://github.com/beef331/traitor/issues/1#issuecomment-1174446449