Mirror of forum.nim-lang.org

2132 :: Read float values from binary file to float array or seq.

• genunix (orginal) [2016-03-18T09:16:56+01:00] view original

  Hello! I need to read a binary file of float values to array[0 .. N; float]. I know about stream.readFloat32/readFloat64, but these functions read one value at once and I need to read many values at once for perfomance (full buffer). I think I can use readData, but don't know how. Also, is there any way to read a lot of values at once in the sequence?

---

• Arrrrrrrrr (orginal) [2016-03-18T11:27:57+01:00] view original

  This is an example of readData:

  import streams, math
  
  var
    stream = newStringStream()
    floatsA, floatsB: array[32, float32]
  
  for f in mitems floatsA:
    f = random(1.0)
    write[float32](stream, f)
  
  setPosition(stream, 0)
  let lenInBytes = len(floatsB) * 4
  doAssert lenInBytes == readData(stream, addr(floatsB), lenInBytes) # <----
  doAssert floatsA == floatsB
  Run

---

• cblake (orginal) [2016-03-18T12:56:16+01:00] view original

  You might also find something like this useful. It memory maps the whole file.

  import memfiles
  var inp = memfiles.open("/dev/stdin")           # Map stdin on Unix
  for i in 0 ..< inp.size div 4:
    echo cast[ptr float32](cast[int](inp.mem) +% 4*i)[]
  Run

  You would need to change the two 4s to 8s and float32 to float64 (or just float) for 8 byte floats. You could wrap this in a type and define some operators to make access prettier.

---

• genunix (orginal) [2016-03-18T13:52:22+01:00] view original

  Thank You Arrrrrrrrr and cblake for great answers! I understand the Arrrrrrrrr's answer, but not fully realized the cblake's. Can You explain why You use '+%' operator and means of [] at the end of last line?

  Sorry for my bad english and thank You.

---

• cblake (orginal) [2016-03-18T14:02:02+01:00] view original

  +% here does the unsigned integer addition appropriate for memory addresses/pointers. Arithmetic isn't defined on pointer types themselves in Nim - you need the inner cast to allow +% to work. The outer cast and [] on the right should be more obvious -- they are to convert the integer back to a ptr float32 and then dereference it for the echo. ( [] is Nim's pointer dereference operator. )

---

• genunix (orginal) [2016-03-18T14:04:02+01:00] view original

  Thank You for great answer! I like this forum!

---

• cblake (orginal) [2016-03-18T14:19:07+01:00] view original

  Sure. You're welcome. For what it's worth, instead of cast[int] and +%, you could cast[uint], but then you also need to uint(i) and 4'u as well. The way I did it just seemed briefest.

  Also, this is a zero/minimal copy way to do the IO, but there are several warnings. You need a real, named file that is seekable (not pipes with "|" from the command-line) for memory mapping to work. If there is any file header to skip you'd need a bit more pointer arithmetic for that. Finally, the memory pointed to is only valid until you call inp.close(). If all that is ok with you, this is a pretty easy way to do bulk file input (for text as well as binary data).

---