I am often in need of types which can carry a lot (or more complex) static compile-time information, à la:
type SomeType[A: static[...], B: static[...], C: static[...], ...] = object ...
I'm in general looking for some resources / documentation that goes more in depth about working with such types. Or even just if someone has some cool usage examples of types that depend on & use more complex or structured information than static[int].
Some problems I'm having:
And semi-related to the above, but on point with my ignorance, I'd also like resources for a deeper understanding of the rules of typedescs, especially functions that work on types and return other types. E.g. am I right in assuming that all typedesc[T] are distinct so long as T are distinct? In func xxx[T](ty: typedesc[T]), is there any semantic difference between using T and ty in constructing other types?
To illustrate my ignorance, it took me way too many tries to write something that returned the "innermost" type of some type of nested seqs or arrays (like seq[seq[array[_, T]]] -> T).
template lastType*(T: typedesc): auto = # must be a template?
# ^- originally intended as an internal function, but had to export it to make the (already exported) generic functions which used it work?
when T is (seq | array):
type I = typeof(block: (for x in default(T): x)) # elementType() from typetraits does not work? (very confusing error message, took me a long time to figure out)
lastType(I)
else:
T
But this is also illustrative of the kind of stuff I'm interested in and haven't yet mastered in Nim. For comparison, I find Zig's comptime functions à la fn Vec(T: type) type { return struct { ptr: [*]T, len: u64 } } to be pretty elegant, so I was experimenting along those lines myself:
template LiveChunk(t: typedesc, n: static[int]): auto =
# const n = n ?
type
Chunk = object
data: array[n, t] # just an arbitrary example
flagged: set[1 .. n]
Chunk
but then every LiveChunk(int, 5) invocation is a distinct type, tho the intention is obviously to get the same type for the same parameters. Is that possible? (Perhaps to implement as a library?)
Often I also get very confusing error messages (or even hit internal assertions in the compiler)
If you're crashing the compiler, you've found a bug! Search for it in the issue tracker on GitHub, see if someone's reported it already, and if so, add your test case to the comments. If not, add a new issue, either will be very appreciated
To work through this list a bit
type MyGeneric[A;B;C;D: static int, E: static float] = object
proc doThing(gen: MyGeneric) = discard # A generic type without parameters is a typeclass
3 is a compiler bug.
I think a more sensible approach to last type is to use a procedure as follows:
proc lastType[T](a: openarray[T]): auto =
when T is (seq or array):
default(T).lastType()
else:
default(T)
var
a: seq[seq[array[1, seq[int]]]]
b: array[10, seq[seq[seq[float]]]]
assert typeof(lastType(a)) is int
assert typeof(lastType(b)) is float
For your live chunk example
type LiveChunk[T; Size: static int] = object
data: array[Size, T]
active: set[0..Size]
template liveChunk(typ: typedesc, size: static int): untyped =
LiveChunk[typ, size] # I dont know why this is desired to be frank, or steve even.
Hey, thanks so much for reply.
proc doThing(gen: MyGeneric) = discard # A generic type without parameters is a typeclass
Yep, this I got; I think of MyGeneric = forall x, y: MyGeneric[x, y], but I meant if there's a way of getting MyGenericIntLike = forall x: MyGeneric[x, int]? Like a sub-typeclass where you say you don't care about certain parts of the type. (Or, for example I want to take arrays of exactly 2 elements or 0..1 index, but I don't care about what type is in the array. In such cases I just add dummy generic variables that I don't use, however it becomes rather messy/ugly when there's many such generic variables I don't care about.)
I think a more sensible approach to last type is to use a procedure as follows:
You're totally right, yours is way cleaner! (Although in my particular use case I plan to also cover a custom container which I don't think can be converted openArray[T] as the elements has a stride and aren't contiguous in memory, but I guess in that case one can just add overloads...)
For your live chunk example
I realize the example I gave was silly as there's no problem writing out the type definition. My venture into that kind of approach had to do with looking for alternative ways to express types that I was failing to express normally, like:
# type
# AdvType[I, J; T: static[array[I, int]], U: static[array[J, int]]] = object # want to get rid of I, J here
# stuff: array[len(T) + len(U), int]
# # ...
# First error I got:
# Error: type mismatch: got <T> but expected one of: ...
# Second error I got messing around with this:
# Error: internal error: invalid kind for lastOrd(tyGenericParam)
# etc
Thus I figured if I had some func AdvType(x, y: distinct array): typedesc which evaluated to the same type as long as parameters were equal (like in Zig). But it was more of a pipe-dream / wonder if such facility can be written in terms of macros. (Don't know how to add a top level type declaration from within in a macro though.)
You could make a type alias for the first part
type
MyComplexGeneric[X;Y;Z;W] = object
MyRelaxedGeneric[X;Y;Z;] = MyComplexGeneric[int, float, string, int]
The latter you could likely use a macro cache and cache a type for your special arrays.
You could make a type alias for the first part
Yes, you're right, tho again I admit my example was a bit stupid, as I put a concrete int there but actually is kind of what remains generic, I just wanted to "focus" on certain parameters. Like a typeclass-alias, rather than a type alias.
E.g. let's say I have Bar[S:static[int],T:Tar]s but I was wondering if there's a way I'm able to write stuff like Foo[2] or Foo[x] to represent the typeclass "all Bars where S=2 and T: Tar." (Tar might be another static needed to reify type.)
Maybe I'm not making much sense... This was originally just to get around having to specify proc foo[A: static[..], B: static[...], C: static[...]](xs: Fux[A,B,C,...]) when I don't really care about B or C in this proc, I just care about A, like proc foo[A: static[int]](xs: Fux[A, _, _, ...]).
It's not a huge deal, I was just finding it very noisy and repetitive in code.
For the latter you could use static[openarray[int]]
That's interesting! And exactly the kind of knowledge I feel I'm lacking. What's the logic here, how did you know this is the case, where do I find more information to understand it?
I didn't see the later part of your message! Did you add that in or did I just miss it.
And yeah, re: question 3 I couldn't convince the compiler to accept an expression there, I got: "Error, cannot generate code for: M" and wrote an omfg proc in frustration. This is, also, part of pushing the type system to the edges.
Yeah maybe related to the known bug ElegantBeef pointed out?
proc `[]=`(c:var ComplexBase,i:range[1..c.typeof.N], v:c.typeof.T) =
This is very interesting tho, and related to my question, how did you "know" typeof would work there, or that '.T' works to access the generic parameter(??). I've tried similar things, but that's often when I get super confusing error messages and have no idea what is possible.
Like some kind of solid spec info on what is possible semantically (regardless of bugs, but kind of what the language's intention is):
proc name[A: Foo](x: Bar,
# at this point in the declaration,
# what "is" x (and A), and to what extend can I use them to derive new types in other parameters / return type / etc. Is the intention that they're just as if there was a `type x = ...` (or `let x: typedesc[..] = ..` tho typedescs seem to be dirty black magic).
# if Bar is a typeclass, is the intention that x.P can be used to access concrete generic params? And the intention is that P is also fully reified, so inner parameters work, x.P.Q?
# if Bar is an alias for some static[T], is the intention that I should be able to use x just as I would a T at this point? (modulo bugs)
# similar questions about A, if the intention is that I can use it as if it's a fully reified type of Foo, etc.
That kind of stuff.
Indeed:
type A[T] = object
var a: A[int]
echo a.T
I am taking from this that learning Nim generics/static[x] is about exploring and investigating?
Here's one such exploration:
# Setup: I have some macro that takes a static[int]
import std/macros
macro foo*(d: static[int]): untyped = newLit(d)
# And a type that carries around an int
type Ix*[D: static[int]] = object
var z = Ix[123]()
# These are akin to what I intuitively thought would work, at first
func ideal1(ix: Ix): int = ix.D
func ideal2[D: static[int]](_: Ix[D]): int = D
# but neither do:
# echo foo(ideal1(z)) # fails: required type static[int] but ... is of type: int
# echo foo(ideal2(z)) # same
# ok, so let's fix the error:
func alt11*(ix: Ix): auto = ix.D
func alt12*(ix: Ix): static[int] = ix.D
# echo foo(alt11(z)) # fails
# echo foo(alt12(z)) # fails
# wild guess:
func alt13*(ix: Ix): ix.D = ix.D
echo foo(alt13(z)) # works?? why/how?
template alt2*(ix: Ix): int = ix.D
echo foo(alt2(z)) # templates work, as expected, even with purported int return type
func alt31*[T](_: T): T.D = T.D
func alt32*[T](_: T): auto = T.D
echo foo(alt31(z)) # works
# echo foo(alt32(z)) # fails - why is auto different?
func alt41[D: static[int]](_: Ix[D]): static[int] = D
func alt42[D: static[int]](_: Ix[D]): typeof(D) = D
# echo foo(alt41(z)) # fails
echo foo(alt42(z)) # works...
And it leaves me wondering: does auto hide info from return type? When is a static[int] not a static[int]?
If I want to export a function with return type int or static[int] (I feel this is best for documentation purposes rather than some "magic" typeclass.something) that users can use statically, is the only alternative the template one?
& another exploration:
type
FooType = enum
ftA, ftB
Foo[N: static[int]] = object
typ: FooType
# first guess: doesn't work!
#
# Test1[X: static[Foo]] = object
# when X.typ == ftA: # Error: undeclared field: 'typ' for type Bar.X
# prefix: string
# other: int
# view: array[X.N, int]
# works??
Test2[X] = object
when X.typ == ftA:
prefix: string
other: int
view: array[X.N, int]
# but then when actually trying to use it:
const fooA = Foo[0](typ: ftA)
var x: Test2[fooA]
# SIGSEGV: Illegal storage access. (Attempt to read from nil?)
So I'm guessing that's a compiler bug -- tho the question remains what's intended to work here.
Your exploration is really good, and there are in fact some things you can reason about. I don't want you to walk away thinking that the compiler is just completely fickle.
Your alt13 is actually a good illustration of it making sense.
Even though I've seen confusion on the issue tracker as to whether D contains a type or a value, it is a type variable, and it contains a type. That type is 123, which is a type that only has one possible value, a bit like void or nil, or range[123,123]
But, unlike range[123,123], D and the value of a variable of type D can be treated as the same thing, which is confusing, but convenient.
On the same lines, I find it easier to reason about ix.D as being sugar for ix.typeof.D, because in my mental model, type parameters belong to the type, not the instance.
Going back to your first idea
macro foo(ix:static int) = newLit(X)
func idea1(ix:Ix):int = ix.D
var z:Ix[123]
echo foo(idea1(z))
(This is if we think of static T as a higher order type, yes it sometimes makes more sense to think of it just as a T whose value is known at compile time, like proc regex(s:static string):Regex or something. But I would argue, if you can implicitly convert a static T to a value of type T, you can do the reverse if you're in a static context)
Aha, so how to get our int into a static context?
You could assign it to a const, maybe?
const tmp = idea1(z) # 😞
One way to express that we don't need any run-time information about z is:
const tmp = idea1(z.typeof.default) #😊
And indeed this is enough.
echo foo(idea1(z.typeof.default))
here I'm explicitly converting from the value, to type, and back to value, which I'm arguing is what's happening implicitly in alt13
To reiterate, having that mental model of static type parameters being types and not values is crucial.
Bearing that in mind, what's going on here:
Test2[X] = object
when X.typ == ftA:
prefix: string
other: int
view: array[X.N, int]
Here are two ways to make it work:
type
Foo = object
typ: FooType
N: int
Bar[N:static int] = object
typ: FooType
type
Test2[X: static Foo] = object
when X.typ == ftA:
prefix: string
other: int
view: array[X.typeof.default.N, int]
## yup, got that old 'couldn't instantiate X' message,
template Test3Impl[N:static int](b:Bar[N]):type =
type Impl = object
when b.typ == ftA:
prefex: string
other: int
virw: array[N,int]
Impl
type
Test3[X:static Bar] = Test3Impl(X)
here's that template pattern again it let's us reify Bar[N] along with it's typ field so we can actually use them
Too long already but I reread your post and realized I got a bit lost in the weeds.
What I believe you were actually asking for was a way to provide an API to get out these compile time values.
The actual answer is, they don't belong to the variable, they belong to the type. They are like static class variables in C++ or Java. So the API should be on the type, not the instance.
func idea0[T:Ix](t:typedesc[T]):int = T.N
echo foo(idea0(z.typeof))
But maybe you disagree, maybe the API needs to be on a runtime instance. And the result needs to be in a compile time context. In that case, you need to get that variable into a compile time context, and yes, the cleanest way to do that is with a template.
Templates are great, use templates. But also, do reconsider your idea that type.member is somehow hacky or magical.