Mirror of forum.nim-lang.org

7512 :: hotcodereloading and libfswatch

• mikebelanger (orginal) [2021-02-15T21:15:18+01:00] view original

  Hi guys,

  I've been really interested in hotcodereloading lately, and I'm trying to get that working with a file-watcher library: nim-fswatch.

  To test, I'm using a modified sdl2 example from the hotcodereloading documentation.:

  mymain.nim

  import logic
  import libfswatch
  import libfswatch/fswatch
  import osproc
  
  proc cb(event: fsw_cevent, event_num: cuint) =
    echo "saved"
    echo execProcess("nim c --hotcodereloading:on mymain.nim")
    update(true)
    echo "reloaded"
  
  # file watcher stuff
  proc main() =
    init()
    var monitor = newMonitor()
    monitor.addPath("logic.nim")
    monitor.setCallback(cb)
    
    while runGame:
      update(false)
      monitor.start()
    destroy()
  
  main()
  Run

  logic.nim:

  import sdl2
  
  #*** import the hotcodereloading stdlib module ***
  import hotcodereloading
  
  var runGame*: bool = true
  var window: WindowPtr
  var renderer: RendererPtr
  var evt = sdl2.defaultEvent
  
  proc init*() =
    discard sdl2.init(INIT_EVERYTHING)
    window = createWindow("testing", SDL_WINDOWPOS_UNDEFINED.cint, SDL_WINDOWPOS_UNDEFINED.cint, 640, 480, 0'u32)
    assert(window != nil, $sdl2.getError())
    renderer = createRenderer(window, -1, RENDERER_SOFTWARE)
    assert(renderer != nil, $sdl2.getError())
  
  proc destroy*() =
    destroyRenderer(renderer)
    destroyWindow(window)
  
  proc update*(reload: bool) =
    
    while pollEvent(evt):
      if evt.kind == QuitEvent:
        runGame = false
        break
    
    if reload:
      performCodeReload()
    
    discard renderer.setDrawColor(120, 150, 201, 1)
    discard renderer.clear()
    
    delay(16)
    renderer.present()
  Run

  Before running the above, in my project directory I executed:

  
  nim c --outdir:$PWD ~/.choosenim/toolchains/nim-#devel/lib/nimhcr.nim
  Run

  then

  
  nim c --outdir:$PWD ~/.choosenim/toolchains/nim-#devel/lib/nimrtl.nim
  Run

  I then compiled mymain:

  
  nim c --hotcodereloading:on mymain.nim
  Run

  I then ran

  ./mymain
  Run

  and started modifying the

  .setDrawColor(120, 150, 201, 1)
  Run

  numbers in logic.nim. I can get the color to change a few times, but then it freezes.

  Can anyone tell me why it's freezing? Both the hotcodereloading and libfswatch examples work on their own, which makes me think I'm doing something wrong with my project. Are there better ways of diagnosing this problem?

  Thanks in advance, I know my questions are probably very n00bish :)

---

• mikebelanger (orginal) [2021-02-17T04:05:24+01:00] view original

  update: the plot thickens. Sometimes, after a few saves, I get this:

  
  mymain(18971,0x7fff8a098380) malloc: *** error for object 0x7fc000c7bb50: pointer being freed was not allocated
  *** set a breakpoint in malloc_error_break to debug
  Run

  Some googling of the above indicates this is a C-compiler-level debugging idea. Sorry I'm woefully ignorant about debugging at the C-level. What would be the best way to debug knowing the above?

  I tried compiling with --debugger:on flag, but it doesn't seem to run the same way. I'm probably missing something basic about how debuggers work. But I can't see what.

  Any help is appreciated.

---

• timothee (orginal) [2021-02-17T10:10:25+01:00] view original

  please try to find a minimized reproducible example and file a bug :)

  (minimizing as much as possible)

---

• mikebelanger (orginal) [2021-02-18T01:22:36+01:00] view original

  I can definitely minimize. How about I remove SDL.

  mymain.nim

  import logic
  import libfswatch
  import libfswatch/fswatch
  import osproc
  
  var running = true
  
  proc cb(event: fsw_cevent, event_num: cuint) =
    echo "saved"
    echo execProcess("nim c --hotcodereloading:on mymain.nim")
    update(true)
  
  # file watcher stuff
  proc main() =
    var monitor = newMonitor()
    monitor.addPath("logic.nim")
    monitor.setCallback(cb)
    monitor.start()
    
    while running:
      update(false)
  
  main()
  Run

  logic.nim:

  import hotcodereloading, strformat, os
  var reloaded = 0
  
  proc update*(reload: bool) =
    if reload:
      performCodeReload()
      reloaded += 1
      echo fmt"reloaded {reloaded} times..."
  Run

  same deal as before:

  1. Compile with nim c --hotcodereloading:on mymain.nim
  2. Run ( on osx: ./mymain )
  3. Start changing logic.nim and save it. You should see the output of mymain print out an updated count of how many times it gets updated.
  4. Try making changes to the echo statement, and save it. That should print out the updated echo statement too.

  5. Try making changes, hitting save repeatedly. I usually make it to ~10 (min 5, max 30), then I get:

  
  Traceback (most recent call last)
  /Users/mikebelanger/Dev/Experiments/nim/hotcodereloading/mymain.nim(23) mymain
  /Users/mikebelanger/Dev/Experiments/nim/hotcodereloading/mymain.nim(18) main
  /Users/mikebelanger/.nimble/pkgs/libfswatch-0.1.0/libfswatch/fswatch.nim(25) start
  /Users/mikebelanger/Dev/Experiments/nim/hotcodereloading/mymain.nim(11) cb
  ((6)
  SIGSEGV: Illegal storage access. (Attempt to read from nil?)
  Run

  Now, is that bug with hotcodereloading? I'm not sure. I could very well be doing something wrong. That's why I'm posting here.

---

• shirleyquirk (orginal) [2021-02-18T02:21:03+01:00] view original

  on linux:

  i don't get a crash but it does occasionally stall with a inotify_rm_watch: Invalid argument

  but changing the file again (not even saving, i dunno what vim is doing) makes it work again.

  the inotify_rm_watch error is from inotify, the linux backend that libfswatch uses. circumstantial evidence that the problem might lie in fswatch?

  nim 1.4.2, libfswatch 1.15.0-1 installed with pakku on arch, a fresh libnimhcr, fresh libnimrtl

---

• mikebelanger (orginal) [2021-02-18T03:30:34+01:00] view original

  Interesting - thanks for testing this, @shirleyquirk!

  If the only diff between our setups is OS, and therefore the backend for libfswatch, its almost certainly libfswatch.

  I guess the next step is to try a different file-watching utility.

---

• mikebelanger (orginal) [2021-02-18T03:34:03+01:00] view original

  @shirleyquirk also your vim behavior is interesting.. is it doing some kind of autosave? That's even more of an automated code reload!

---

• mikebelanger (orginal) [2021-02-20T17:22:49+01:00] view original

  Alright, another stab at this. This time I tried using fsnotify instead. Same behavior. Most of the time it works, but I get the occasional segfault.

  mymain.nim

  import logic
  import fsnotify
  import osproc
  
  proc cb(event: seq[PathEvent]) =
    echo "saved"
    echo execProcess("nim c --hotcodereloading:on mymain.nim")
    update(true)
    echo "reloaded"
  
  # file watcher stuff
  proc main() =
    var watcher = initWatcher(1)
    watcher.register("logic.nim", cb, ms = 100)
    
    while true:
      poll(watcher, 2000)
      update(false)
  
  main()
  Run

  logic.nim:

  import hotcodereloading, strformat
  var reloaded = 0
  
  proc update*(reload: bool) =
    if reload:
      performCodeReload()
      reloaded += 1
      echo fmt"reloaded {reloaded} times......."
  Run

  I tried just commenting out the update function call in mymain, so its just mymain code - and it works fine. I never got it to crash that way.

  I'd be interested to hear from more non-OSX users.

---

• shirleyquirk (orginal) [2021-02-20T20:17:30+01:00] view original

  if i edit in neovim it reloads once then hangs.

  if i edit in nano it works fine.

  so i went back to the first version you posted and yep, works great in nano.

  thought it might be nimsuggest/nimcheck/nimlsp doing weird stuff but even with an empty vimrc, this second version hangs after the first reload. the first one works better with nimsuggest turned off, but still getting the inotify_rm_watch error.

  so, no idea what's going on, but worth trying different text editors and see if your crash goes away

---

• mikebelanger (orginal) [2021-02-21T20:20:45+01:00] view original

  Yeah I have no idea what's going on either.

  To be honest, I just want it to run reliably. The odd hangup is ok with me. I can definitely see a different editor causing the odd stall, especially if its invoking nimsuggest/nimcheck/nimlsp.

  Up until now, I was running this all with VSCode and its terminal emulator (emulator running fish).

  I just tried running it out of iTerm (a terminal emulator) with just nvim and bash. It saves properly the first dozen times or so, but then:

  
  Error: unhandled exception: /Users/mikebelanger/.nimble/pkgs/timerwheel-0.1.2/timerwheel/timerwheel.nim(126, 16) `false` Number is too large! [AssertionDefect]
  Run

  Looks like a dependency of fsnotify. Not sure that caused the segfaults before though.

---

• mikebelanger (orginal) [2021-02-27T23:04:22+01:00] view original

  I should add, I'm also getting

  : Illegal storage access. (Attempt to read from nil?)
  Run

  also in the above setup (running iTerm, bash, nvim).

  Maybe I should just use Linux.

---