Mirror of forum.nim-lang.org

1390 :: getFileSize and rawFileSize (sysio) have different upper limits

• xyz32 (orginal) [2015-07-02T02:12:18+02:00] view original

  Inside sysio the getFileSize proc returns an int64 but rawFileSize returns an int. Doesn't that mean that rawFileSize is limited to 4GB file size?

  Also readFile may read over the end of file, throwing an exception: readFile("/sys/devices/bone_capemgr.9/slots") will fail with:

  
  sysio.nim(187)           readAll
  sysio.nim(176)           readAllFile
  system.nim(2280)         raiseEIO
  Error: unhandled exception: error while reading from file [IOError]
  Run

  The reason is the fact that sysfs has a fix size to any config file (not attached to any IO) so the rawFileSize method returns more bytes than the file actually has.

  If I read line by line (or char by char) and handle eof separately I can read the file just fine. I have opened an issue: #3039

  Thank you.

---

• Varriount (orginal) [2015-07-02T09:03:46+02:00] view original

  Not necessarily - The 'int' data type is always the size of a native pointer in Nim. Even so, I'm a bit puzzled. Both those procedures use the C file functions ftell, which generally returns a 32-bit integer, yet the declaration for it is proc ftell(f: File): int , which seems incorrect.

---

• Araq (orginal) [2015-07-02T10:48:31+02:00] view original

  ftell returns a long which is Nim's int, at least it was considered close enough.

---

• jibal (orginal) [2015-07-02T13:08:43+02:00] view original

  Both those procedures use the C file functions ftell

  They should be using fgetpos, which handles large files.

---

• Varriount (orginal) [2015-07-02T20:22:20+02:00] view original

  @jibal Are you sure about that? fsetpos can only use values previously obtained by fgetpos, and the type returned by fsetpos is opaque, so how would you use it to efficiently get the size of a file? (Note, the value returned by fgetpos shouldn't be used as an integer) .

  From what I've read, the best way to get/set the position of the pointer in a file larger than 2/4GB is to use fseeko/ftello and define _FILE_OFFSET_BITS as 64.

  On a related note, I wonder how using Windows API file functions on Windows (such as GetFileSizeEx) would effect performance.

---

• jibal (orginal) [2015-07-02T23:05:48+02:00] view original

  Are you sure about that?

  No, sorry, I was confused.

  use fseeko/ftello and define _FILE_OFFSET_BITS as 64.

  Why not stat64?

  On a related note, I wonder how using Windows API file functions on Windows (such as GetFileSizeEx) would effect performance.

  One can always benchmark, but I would expect GetFileSizeEx to be cheapest.

---