Mirror of forum.nim-lang.org

6385 :: Stream IO errors with -d:release flag

• seves (orginal) [2020-05-28T06:52:52+02:00] view original

  The stream module seems strange to me. When I compile my avi demuxer with -d: release it throws IO errors, but not without: The same file is read in both release and debug mode.

  This is the part in which the error occurs:

  proc read_video_frame*(avi: Avi; vidbuf: ptr UncheckedArray[byte]; video_pos: int; isKeyframe: var bool): int  =
    var n: int
    if avi.video_index.isNil:
      throw Exception: "AVI_ERR_NO_IDX"
    if video_pos < 0 or video_pos >= avi.video_frames:
      return -1
    n = avi.video_index[video_pos].len
    isKeyframe = avi.video_index[video_pos].key == 0x00000010
    echo fmt" {video_pos} / {avi.video_frames} position"
    
    avi.stream.setPosition(avi.video_index[video_pos].pos)  ## <- this line produces the error in release mode
    if avi.stream.readData(vidbuf, n) != n:
      throw Exception: "AVI_ERR_READ"
    return n
  Run

  In release it reads some of frames but after a time it throws errors. Output with -d:release:

  
  ...
  ...
  159 / 424 position
  isKey: true readCount: 364446
  160 / 424 position
  isKey: true readCount: 363571
  161 / 424 position
  isKey: true readCount: 361489
  162 / 424 position
  io.nim(138)              raiseEIO
  Error: unhandled exception: cannot set file position [IOError]
  Run

  Maybe I'm missing something? Is my code buggy or is it a problem with stream module?

  Maybe someone can push me in the right direction

---

• Araq (orginal) [2020-05-28T10:40:13+02:00] view original

  It's likely that your code is buggy. Use setupForeignThreadGC() in callbacks that are called from C and run in a different thread.

---

• mratsim (orginal) [2020-05-28T11:01:23+02:00] view original

  @Araq, how do you know it's a callback?

  When you create a proc to be used in C (callback):

  • Don't use exceptions
  • Don't use string or sequences
  • Deactivate stacktraces
  • Use int32 or cint for int use uint or csize_t for size_t
  • tag {.cdecl, gcsafe.} your proc to detect at compile-time the issue

  Also one thing I've noticed is that Nim endOfFile() is quitting to early compared to c_eof from the system/ansi_c module (see my h264 encoder test case https://github.com/mratsim/trace-of-radiance/blob/99f7d85d/trace_of_radiance/io/h264.nim#L278)

  Example to pass a Nim callback to C code for MP4 muxing:

  https://github.com/mratsim/trace-of-radiance/blob/99f7d85d/trace_of_radiance/io/mp4.nim#L115-L126

  proc MP4E_open(
         sequential_mod_flag: cint,
         enable_fragmentation: cint,
         token: pointer,
         write_callback: proc(
           offset: int64,
           buffer: pointer,
           size: csize_t,
           token: pointer
         ): cint{.cdecl, gcsafe.}
       ): ptr MP4E_mux_t {.importc, header: "minimp4.h".}
    ## The C code
  
  
  {.push stackTrace: off.}
  proc writeToFile(
         offset: int64,
         buffer: pointer,
         size: csize_t,
         token: pointer
       ): cint {.cdecl, gcsafe.} =
    ## The callback
    let file = cast[File](token)
    file.setFilePos(offset)
    let bytesWritten = file.writeBuffer(buffer, size)
    return cint(bytesWritten.csize_t != size)
  {.pop.}
  Run

---

• seves (orginal) [2020-05-29T06:58:59+02:00] view original

  Thank you for your answers. I appreciate that very much. Its a bit funny but i don't use c in this context.

  The problem I have is that it didn't work with the flag -d: release. I have now tried something out and found out the following:

  When I create the stream from the previously opened file, these errors occur in the release build, but when I create a new stream from filepath, the program works fine

  broken:

  stream = newFileStream(FILE)
  ...
  stream.setPosition(blabla)
  Run

  working solutiuon:

  stream = newFileStream(filename)
  ...
  stream.setPosition(blabla)
  Run

  I'm not sure, maybe I still have a worm somewhere, but when I open the stream from the path it seems to work. It is still strange that it does not work with the stream from the FILE

---