Mirror of forum.nim-lang.org

3673 :: how to read/write object from/to binary file?

• luntik2012 (orginal) [2018-03-21T14:17:47+01:00] view original

  I can do something like this:

  type
    MyType* = object
      somefield*: array[10, int16]
  var t: MyType
  var f = newFileStream("/tmp/tmp.mytype", fmWrite)
  if not fff.isNil:
    f.write t
  Run

  but result file contains garbage, size of file is 20 bytes.

  So question is how to read/write object from/to file?

---

• luntik2012 (orginal) [2018-03-21T14:18:59+01:00] view original

  I do

  t.somefield[0] = 255
  t.somefield[1] = 255
  
  Run

  to check

---

• luntik2012 (orginal) [2018-03-21T14:34:06+01:00] view original

  Sorry, it works correctly. But how to read it back? File contains several instances of MyType, so I need to take subpart of file. I've tried this

  let data = readFile("/tmp/tmp.mytype")
  let ololo = cast[MyType](data[0 .. <sizeof(MyType)])
  Run

  Is it correct way?

---

• c0ntribut0r (orginal) [2018-03-21T14:45:19+01:00] view original

  Have a look at this:

  import streams
  type
    MyType* = object
      somefield*: array[10, int16]
  var t1, t2: MyType
  t1.somefield[0] = 1
  t2.somefield[0] = 2
  
  
  var f = newFileStream("/tmp/tmp.mytype", fmWrite)
  if not f.isNil:
    f.write t1
    f.write t2
  f.flush
  
  var r1, r2: MyType
  var f2 = newFileStream("/tmp/tmp.mytype", fmRead)
  discard f2.readData(r1.addr, r1.sizeof)
  discard f2.readData(r2.addr, r2.sizeof)
  echo r1.somefield
  echo r2.somefield
  Run

---

• luntik2012 (orginal) [2018-03-21T15:01:00+01:00] view original

  Thank you, it works. It strange that there is no possibility to make some cast from string of seq[byte] directly to object

---

• doofenstein (orginal) [2018-03-21T16:05:59+01:00] view original

  Because it's very dangerous.

  Simply having a compiler which puts structures in another layout or adding a field to the object is enough to render all your previously generated files useless.

  If you just want to save and load data, I would rather serialise it(https://nim-lang.org/docs/marshal.html ). If you want something more compact then e.g. JSON or YAML take a look at msgpack: https://nimble.directory/pkg/msgpack4nim (note: I haven't used msgpack with Nim, but I had good experiences with it with other programming languages)

---

• cblake (orginal) [2018-03-21T16:28:52+01:00] view original

  While doofenstein makes some good points especially about fragility with regard to adding new fields and old data files, there is some real efficiency charm to a "just memory map and go" pure binary native data approach. It's not always fun to be parsing data files again and again. So, I would also add another way to read the data. Note that you should probably use the {.packed.} pragma, note the use of ptr UncheckedArray which should really alert the reader to be careful, note the commented out inference of the number of objects in the file. My example code assumes you wrote out at least two objects from code like you had above (with a {.packed.} pragma). It will probably crash if the file is too short, etc. Just to give you a flavor.

  import memfiles
  type
    MyType* {.packed.} = object
      somefield*: array[10, int16]
    MyTypes* = ptr UncheckedArray[MyType]
  var f = memfiles.open("/tmp/tmp.mytype")
  # infer n object from file size (may need packed); Do own range checking!
  # let n = f.size / sizeof(MyType)
  let myobs = cast[MyTypes](f.mem)
  echo myobs[0].somefield[1]
  echo myobs[1].somefield[0]
  Run

---