https://nim-lang.org/docs/marshal.html
I just started playing with marshal module. Seems to be the easiest way to store objects in an human readable form to disk. My test is
import
marshal, streams
var s = newFileStream("somefile.txt", fmWrite)
type
R = object
x, y, w, h: int
O = object
name: string
r: seq[R]
var o: O
o.name = "test"
o.r.add(R(x: 1, y: 2, w: 2, h: 6))
o.r.add(R(x: 5, y: 7, w: 20, h: 8))
o.r.add(R(x: 9, y: 17, w: 28, h: 18))
store(s, o)
s.close()
Result is
$ cat somefile.txt
{"name": "test", "r": [{"x": 1, "y": 2, "w": 2, "h": 6}, {"x": 5, "y": 7, "w": 20, "h": 8}, {"x": 9, "y": 17, "w": 28, "h": 18}]}
So all is dumped to one single very long line? Then that is not really human readable. The other disadvantage of marshal is that it seems to work not with inheritance, it can only use compile time types, but that should be OK for my use case, as I try to avoid inheritance.
So I may have to use JSON or extern YAML module? I really like YAML, as it has good human readability and can be even edited with a text editor. And if I remember correctly its author was a bright one. But I think he has retired years ago, can not even remember his name.
So my question would be: What is the future of YAML module, will it at least still work with Nim 2.0 ?
JSON has a pretty proc that makes JSON pretty.
CI are dumb sometimes they fail for no reason or timeouts.
After a closer look at JSON and YAML it seems that the user interface for storing and loading objects is similar, so I can start with one and maybe later exchange it with the other :-)
YAML is really large, a small test program generates a 200k executable in release mode.
For module marshal we have
Restriction: For objects their type is not serialized. This means essentially that it does not work if the object has some other runtime type than its compiletime type.
Not a real problem for me, but does this restriction apply for JSON and YAML as well?
For JSON to() macro we have
Heterogeneous arrays are not supported.
I have never seen that term in Nim world before ???
For JSON to() macro we have
> Heterogeneous arrays are not supported.
I have never seen that term in Nim world before ???
That just refers to the fact that in JSON you can of course have a heterogeneous array, like:
let heterogeneous = %* [1, 2.5, "Hello"]
and these simply cannot be mapped to Nim types properly.
Marshaling is not about being human-readable though. The fact that it uses json is just because it was the easiest and it may change in the future.
Marshalling is just about dumping in-memory data so that it can be reloaded at a later time.
Think of it as Python pickle:
Well, fair point and a nice link. I mostly thought identical names for roughly identical things and worth mention.
Since this thread is about storing to disk, I should mention the too often neglected option of near-zero overhead native-binary-layout file formats that are "mmap & go" with no repeated parsing or even unnecessary paging. The file system can basically become your memory allocator. Ascii->binary and especially binary->Ascii is an expensive conversion for numbers (though any cost is always "compared to what").
Native binary does break "portability" to e.g. machines with different endianness/byte-order or non-IEEE floating point formats and whatnot. In the modern era, little endian and IEEE FP has won to such a degree that in practice this may matter little, if at all. In the old days, sharing a network volume between e.g., a big endian Sparc and a little endian Intel box was more common.
Anyway, Nim supports this all fine (as in, e.g., my https://github.com/c-blake/suggest), but A) that is far from pedagogical code, { but hey, at least it's a fully worked example }, and B) it can often be error prone work in general. Someone out in Nim land should maybe do a macro package someday to make it easier (unless they already have?).