We do have a module called "marshal", which takes an arbitrary object and generates a JSON schema, but there are various limitations, such as the inability to control how individual fields are serialized.
There are various ways in which the interface could be improved, including:
What are your thoughts on this?
By the way, why hasn't streams.read[T] been exported? Is it because it would be error-prone to handle objects that might be padded? Or maybe it was just overlooked. Either way, there are two approaches that come to mind:
I would say go ahead with porting something like thrift or protobuf if you want a binary serialization format. Advantage is that they're already widely used in other languages, should be easy to port, and would let Nim be used in more places.
We use Thrift at my work, so I may look into doing that.
perturbation2: Well, my use case has been the implementation of an existing protocol (the MongoDB protocol). Also, native implementations are always nice to have, and it's often feasible for documented protocols.
Araq: Yeah I know. I'll mess around with getType if I ever get to this.
mreiland: Ok I'll let you know if I have any concrete ideas and I don't have time to do everything myself
I really like this idea. Serialization is a very cool thing, and with Nim's powerful macros it would be awesome to see in action.
This makes me want to do some fiddling :)
I looked into adding a language to Thrift; Thrift is probably not a good choice for a Nim-specific marshal module since it's not a specific protocol, but instead an IDL with various protocols (one of which is binary), transports, and server types supported.
It would not be hard to write implementations in Nim that support these, but the main challenge (after reading through the Thrift documentation and whitepaper) would be adding Nim code generation to the Thrift compiler (written in C++). This shouldn't be too bad as its for a restricted subset (generating constants, enums, structs, etc.) and how to convert to/from Thrift types to Nim types. TBH this is a challenge since while I'm very good with C, I've not really used C++ for anything serious.
The lua code generation implementation is fairly easy to follow.
The Construct library has been ported from Python to Perl and Java: https://en.wikipedia.org/wiki/Construct_%28python_library%29
Could it be a good example?
Hm. Construct does well because it provides declarative typing to languages that don't normally have static types.
Nevertheless, I can see certain possibilities using construct-like design with Nim's macros.