Mirror of forum.nim-lang.org

9846 :: File logging in a thread using spawn (GC safe call expression)

• demetera (orginal) [2023-01-25T22:08:37+01:00] view original

  Dear All,

  i have a little 2 files project (main.nim as a runner and lib.nim with all global procedures I need for the project).

  Attempting to spawn (using threadpool) lib procedure (which periodically logging events using logging.newFileLogger) from main module. I'm getting this EM : spawn' takes a GC safe call expression

  I've tried to use {.gcsafe.} for that proc in lib module. EM : logProcess is not GC-safe as it accesses 'logger' which is a global using GC'ed memory

  So basically I would like to have logging to file in thread (cause I have to start and stop it at some point while main app is running).

  Any ideas, thoughts? Anything I'm doing wrong as a Nim newbie :)

  Thanks!

---

• Isofruit (orginal) [2023-01-26T00:48:41+01:00] view original

  I think a small code sample could help here, because I'm not sure where logger comes from. You can just log like this (handler creation was moved into a gcsafe proc to prove that its gcsafe):

  import std/[asyncdispatch, os, sugar, logging]
  
  proc setupLogger() {.gcsafe.} =
    addHandler(newConsoleLogger(levelThreshold = lvlDebug))
  
  proc a() {.gcsafe.} =
    debug "Starting Callbacks"
  
  setupLogger()
  a()
  Run

  No access to global state necessary. You'll need that per thread as each handler is per thread, but that all works.

---

• demetera (orginal) [2023-01-26T03:21:21+01:00] view original

  Thanks for the reply. I didn't add any handlers (as you did) in the meantime, but there is nothing in docs.

  Problematic proc spawned/triggered from the main, as case condition, when user enters "logon":

  of "logon":

  echo "Start logging" spawn logProc()

  logProc method looks like this in additional library:

  proc logProc*() =

  while true:

  let tried = chan.tryRecv()

  if tried.dataAvailable:

  echo tried.msg break

  sleep(5000) echo "Some event" logger.log(lvlInfo, "Logged")

  If I comment last line with logger.log proc, no error appear.

  Logger vars are defined as:

  \const LOGFILE : string = "fancon.log" \const fmtStr : string = "[$datetime]: " \let logger = logging.newFileLogger(LOGFILE, fmtStr=fmtStr)

---

• thindil (orginal) [2023-01-26T09:55:18+01:00] view original

  The call logger.log(lvlInfo, "Logged") isn't thread safe, because the logger, especially to a file, isn't thread safe. You have to use handlers to log to a file from different threads. You can't use logger directly in that way. Don't forget about notes: https://nim-lang.org/docs/logging.html#basic-usage-notes-when-using-multiple-threads

  Also, just a note from a grumpy guy. :) If you try to log to one file from different threads, you will have a beautiful race condition. Expect to have a mess in the log or crash, when two or more threads will try to write at the same moment. The better option is to have logging on a separated thread. It is because all file systems are designed as: "many can read, but only one can write at the same time".

---

• Cnerd (orginal) [2023-01-26T17:16:17+01:00] view original

  Or you can just use locks on your file logger and use {.cast[gcsafe].} to tell the compiler not to worry about gcsafety

---

• Cnerd (orginal) [2023-01-26T17:28:21+01:00] view original

  Like so

  import std / [locks, logging, exitprocs, threadpool]
  from std / sugar import `=>`
  
  var loggerLock : Lock
  let logger {.guard : loggerLock.} = newFileLogger("logfile.log.txt")
  
  initLock(loggerLock)
  addExitProc(() {.closure.} => (
      deinitLock(loggerLock)
  ))
  
  proc yourThreadedProc() {.gcsafe.} =
      
      withLock loggerLock:
          
          {.cast(gcsafe).}:
              
              logger.log(lvlInfo, "info logging")
  
  spawn yourThreadedProc()
  sync()
  Run

---

• demetera (orginal) [2023-01-26T22:24:17+01:00] view original

  Thank you very much for your suggestions. I messed up a bit with code formatting, and I can't edit my post. I will be more careful next time.

  It's getting too complex for little program, hence I ended up with using simple file writer (to record CPU temperature to the text file).

---