Mirror of forum.nim-lang.org

5354 :: we need better support of serialized objects downcast in stdlib

• enthus1ast (orginal) [2019-10-14T12:54:57+02:00] view original

  The problem is that every serializer that i've tried (except json), cannot downcast a serialized message to its base type.

  json, the only serializer that does the job (but only on non ref types):

  import json
  type
    MsgBase* = object of RootObj
      version*: byte
      messageId*: string
      timestamp*: string
      senderName*: string
      senderPublicKey*: string
    MsgControlReq* = object of MsgBase
      command*: string
      param*: string
  
  var conreq = MsgControlReq()
  conreq.version = 1.byte
  conreq.messageId = "messageId"
  conreq.timestamp = "timestamp"
  conreq.senderName = "senderNameVal"
  conreq.senderPublicKey = "senderPublicKeyVal"
  conreq.command = "CMD" # <- first not in base
  
  var conreqj = (%* conreq)
  var base = conreqj.to(MsgBase)
  echo base
  Run

  works:

  
  (version: 1, messageId: "messageId", timestamp: "timestamp", senderName: "senderNameVal", senderPublicKey: "senderPublicKeyVal")
  Run

  not working marshal:

  import marshal
  type
    MsgBase* = object of RootObj
      version*: byte
      messageId*: string
      timestamp*: string
      senderName*: string
      senderPublicKey*: string
    MsgControlReq* = object of MsgBase
      command*: string
      param*: string
  
  var conreq = MsgControlReq()
  conreq.version = 1.byte
  conreq.messageId = "messageId"
  conreq.timestamp = "timestamp"
  conreq.senderName = "senderNameVal"
  conreq.senderPublicKey = "senderPublicKeyVal"
  conreq.command = "CMD" # <- first not in base
  
  var conreqj = $$conreq
  var base = to[MsgBase](conreqj)
  echo base
  Run

  error:

  
  /home/david/nimPepper/deleteme/mt.nim(22) mt
  /home/david/Nim/lib/pure/marshal.nim(333) to
  /home/david/Nim/lib/pure/marshal.nim(266) loadAny
  /home/david/Nim/lib/pure/marshal.nim(186) loadAny
  /home/david/Nim/lib/core/typeinfo.nim(370) []
  /home/david/Nim/lib/core/typeinfo.nim(372) []
  Error: unhandled exception: invalid field name: command [ValueError]
  Error: execution of an external program failed: '/home/david/nimPepper/deleteme/mt '
  Run

  not working msgpack4nim

  import msgpack4nim
  type
    MsgBase* = object of RootObj
      version*: byte
      messageId*: string
      timestamp*: string
      senderName*: string
      senderPublicKey*: string
    MsgControlReq* = object of MsgBase
      command*: string
      param*: string
  
  var conreq = MsgControlReq()
  conreq.version = 1.byte
  conreq.messageId = "messageId"
  conreq.timestamp = "timestamp"
  conreq.senderName = "senderNameVal"
  conreq.senderPublicKey = "senderPublicKeyVal"
  conreq.command = "CMD" # <- first not in base
  
  var conreqj = pack(conreq)
  
  var base: MsgBase
  unpack(conreqj, base)
  echo base
  Run

  error:

  
  /home/david/.nimble/pkgs/msgpack4nim-0.2.9/msgpack4nim.nim(1103) unpack
  /home/david/.nimble/pkgs/msgpack4nim-0.2.9/msgpack4nim.nim(1093) unpack
  /home/david/.nimble/pkgs/msgpack4nim-0.2.9/msgpack4nim.nim(987) unpack_type
  /home/david/.nimble/pkgs/msgpack4nim-0.2.9/msgpack4nim.nim(1093) unpack
  /home/david/.nimble/pkgs/msgpack4nim-0.2.9/msgpack4nim.nim(811) unpack_type
  /home/david/.nimble/pkgs/msgpack4nim-0.2.9/msgpack4nim.nim(245) unpack_imp_uint8
  Error: unhandled exception: uint8 [ObjectConversionError]
  Error: execution of an external program failed: '/home/david/nimPepper/deleteme/mpt '
  Run

  i think the reason is that the fields of the derived message are before the base type fields. This is at least what i've noticed, i don't know the technical reason.

  import msgpack4nim
  import msgpack2any
  type
    MsgBase* = object of RootObj
      version*: byte
      messageId*: string
      timestamp*: string
      senderName*: string
      senderPublicKey*: string
    MsgControlReq* = object of MsgBase
      command*: string
      param*: string
  
  var conreq = MsgControlReq()
  conreq.version = 1.byte
  conreq.messageId = "messageId"
  conreq.timestamp = "timestamp"
  conreq.senderName = "senderNameVal"
  conreq.senderPublicKey = "senderPublicKeyVal"
  conreq.command = "CMD" # <- first not in base
  
  var conreqj = pack(conreq)
  echo conreqj.toAny
  
  Run

  output:

  
  [ "CMD", "", 1, "messageId", "timestamp", "senderNameVal", "senderPublicKeyVal" ]
  Run

  you see that the derived object fields are on first position.

  PS: If you know a binary serializer that can do proper object downcasting please let me know! :)

---