I'm programming to a specification that uses numbers to identify specific data from binary data. Since I need to do this often (at least more than once) I'm hoping to implement a convenient helper function; something like this:
proc componentCodeToDatatype(componentCode: int): <some datatype return magic> =
case componentCode:
of 5120:
return int8
of 5121:
return uint8
of 5122:
return int16
of 5123:
return uint16
of 5125:
return uint32
of 5126:
return float
else:
// throw exception or something (should never happen)Or maybe there's a better mechanism for achieving this?
Yeah, bro, you can’t directly return a type and use it for variable declaration in Nim, because typedesc doesn’t work that way at runtime. But here’s a simple solution using a template, it works and keeps things clean:
proc componentCodeToDatatype(code: int): typedesc =
case code:
of 5120: int8
of 5121: uint8
of 5122: int16
of 5123: uint16
of 5125: uint32
of 5126: float32
else:
raise newException(ValueError, "Unknown component code: " & $code)
template declareVar(name: untyped, code: int, value: untyped) =
when code == 5120:
var name: int8 = value
elif code == 5121:
var name: uint8 = value
elif code == 5122:
var name: int16 = value
elif code == 5123:
var name: uint16 = value
elif code == 5125:
var name: uint32 = value
elif code == 5126:
var name: float32 = value
else:
{.error: "Unsupported component code".}
when isMainModule:
declareVar(myVal, 5121, 255)
echo myValThat 255 proves it's using uint8, since that's the max valid value. Works perfectly.
Looks like you're trying to work with OpenGL?
https://www.khronos.org/opengl/wiki/OpenGL_Type
The real question is do you really need to return typedesc? What's your usecase? Every procedure using Nim's typedesc becomes generic and will be instantiated for every type used in the code. Unless you're working with macros or already generic functions, you'll very likely will be better served with a custom enum. Then you can write a dispatcher for converting the enum to Nim types to use with macros/generic procs with a case statement.
import std/[enumutils, sequtils]
type
NimType* = enum
ntInt8 = 5120
ntUint8 = 5121
ntInt16 = 5122
ntUint16 = 5123
ntUint32 = 5125
ntFloat32 = 5126
proc toNimType*(componentCode: int): NimType {.warning[HoleEnumConv]:off.} =
const ValidCodes = NimType.mapIt(ord(it)).toSeq()
if componentCode in int16.low..int16.high and componentCode.int16 in ValidCodes:
NimType(componentCode.int16)
else:
raise newException(ValueError, "Unknown component code: " & $componentCode)
when isMainModule:
doAssert toNimType(5121) == ntUint8
doAssert ntFloat32.ord == 5126
doAssertRaises(ValueError): discard toNimType(867_53_09)Yes I am working with OpenGL. I am implementing the glTF Specification in Nim. Part of the specification is the Accessor object. An accessor outlines which binary data to reference and what that data means. So the accessor.componentType is an integer as defined above. The accessor.type specifies whether the componentType data is part of a larger data element. For example a componentType could be a simple SCALAR by itself or part of a VEC2, VEC3, VEC4, MAT2, MAT3 or MAT4 element. The ''accessor.count`` specifies how many elements make up this accessor data.
So, a simple (but not complete) example: This base64 data is provided: AAABAAIAAAAAAAAAAAAAAAAAAAAAAIA/AAAAAAAAAAAAAAAAAACAPwAAAAA=` This accessor defines how to get indices data which will be passed to your graphics library for rendering:
{
"bufferView" : 0,
"byteOffset" : 0,
"componentType" : 5123,
"count" : 3,
"type" : "SCALAR",
"max" : [ 2 ],
"min" : [ 0 ]
},So I'm hoping there's a way to achieve this data extraction via a single procedure. I was originally thinking to use generics:
proc extractFromData[T, U, V](data: seq[byte], count, offset: int): seq[T] =
# where T is the element eg: SCALAR, VEC2, VEC3, ...
# where U is the componentType eg: int8, uint8, ...
# where V is the datatype the graphics library expects. So the data provided might by VEC3:uint16 but the graphics library wants VEC3:uint32Your answer is going to take me a little while to digest. Maybe more context might help with a recommended approach.
@BeigeHornet Thanks for your response. Templates are a part of Nim I'm still trying to get my head around when they are best used. I like your approach. I started to implement it but it seems that Templates want to know datatypes prior to compilation. When I transposed your code to my project I received an error. I modified your sample code as follows to simulate the unknown of the datatype until runtime.
import std/random
randomize()
template declareVar(name: untyped, code: int) =
when code == 5120:
var name: int8
elif code == 5121:
var name: uint8
elif code == 5122:
var name: int16
elif code == 5123:
var name: uint16
elif code == 5125:
var name: uint32
elif code == 5126:
var name: float32
else:
{.error: "Unsupported component code".}
when isMainModule:
declareVar(myVal, 5120 + rand(6))
myVal = 123
echo myValI haven't yet (but will) investigate your typedesc approach. Even if it's not the final solution it will be handy to investigate for my own knowledge.
Which leads to the question here. If using generics how do I easily convert an int to a datatype.
Not reading deep into the details, here's my understanding of the problem at hand: Somewhere inside the extractFromData you'll be reading raw bytes from the underlying data and doing a cast operation to get it. The type you cast the data to is to be determined by the componentType field of the accessor object.
What I propose is to define a typed Nim accessor object first. Then, when you have the componentType field as an enum inside this object, you just use a case expression to cast the data. The most efficient way of doing the cast depends on how it's actually stored inside the byte buffer. I feel like the matrix type s are going to complicate things a bit, but for scalars you cast a single variable and for vectors you're casting the ptr UncheckedArray[type].
To sum up, I wouldn't complicate things and introduce some template/procedure returning a type. You just need to identify a single point in your logic where you data is converted from untyped raw bytes to Nim objects and use case there.
There's not much to digest in my previous comment. It boils down to the fact that you can use the hardcoded magic ints to use in a case statements, but it's better to encode all possible values in the enum. Then you'll have only one place where an error can happen (when reading the enum value), otherwise, you'll need to check for errors each time you use the int value.
PS: I have no experience working with OpenGL, but it's a performance critical area. You need to consider the context your specific code will be used to plan ahead. For example, when you use your accessor and convert the data to a Nim object, how long does the underlying data bytes live? If it's still available during the lifetime of your program and the read objects are going to be actively used and not just stored, it might be beneficial to avoid copying and return a typed view into the data and not a seq. It might be an OpenArray or a custom pointer-like object.
If you read files at runtime, you cannot change the type of variables based on the content of the file at runtime in statically typed programming languages. If you read files at compile time you probably can create a variables based on the file by using macros or templates.
If you need to read files at runtime and want to change the type of variables, there are workarounds: https://internet-of-tomohiro.pages.dev/nim/faq.en#type-how-to-store-different-types-in-seqqmark