nimforum mirror - spawn/sync hangs -- any ideas?

cdunn2001 (orginal) [2019-08-01T17:59:07+02:00] view original

What could cause sync() to hang?

for x in stage1:
        spawn startStage1(x)
    sync()
    log("First sync() done.")

proc startStage1(args: Stage1) =
    log("StartProcess in Stage1: '", args.icmd, "'")
    var p = startProcess(command = args.icmd, options = {poEvalCommand,
            poStdErrToStdOut})
    var argsx = args
    argsx.sin = p.outputStream
    doStage1(argsx)
    log(" startStage1: p.close() ...")
    p.close()
    log(" startStage1: p.close() done")

I see everything except "First sync() done."

This is all after threadpool.setMaxPoolSize(1), and the size of the array is 1.

What could possibly prevent "sync()" from working. (Verified behavior with both 0.20.0 and 0.20.2.)

cdunn2001 (orginal) [2019-08-01T18:04:28+02:00] view original

@araq, if I don't call threadpool.setMaxPoolSize(1), this always works. I think there is a bug in threadpool resizing, which is a must-have for me.

Should I file a bug? I cannot provide a simple repro from my current codebase.

cdunn2001 (orginal) [2019-08-01T18:36:14+02:00] view original

Even in my current program (runs in a second, but builds a lot of code), this hangs only 1 in 10 tries. It's a "Heisenbug", an intermittent multi-threading error. And it's almost certainly in Nim because my code spawns only 1 task.

mratsim (orginal) [2019-08-01T19:50:36+02:00] view original

There might be a deadlock.

Can you compile with --debugger:native and attach gdb/lldb and do a backtrace if your programs hangs?

Also you can maybe compile with -d:useMalloc and use helGrind to try to find threading errors: http://valgrind.org/docs/manual/hg-manual.html

cdunn2001 (orginal) [2019-08-02T16:01:05+02:00] view original

Both the main thread (1) and the worker thread (3) are in "nanosleep()". (The rest are in pthread_cond_wait().)


(gdb) bt
#0  0x00007f172734be4d in nanosleep () from /lib64/libpthread.so.0
#1  0x000000000044655b in nossleep (milsecs=milsecs@entry=100) at /mnt/software/n/nim/0.20.2/Nim/lib/pure/os.nim:2791
#2  0x000000000046f4f2 in sync_WTnSSeVs19cfp5pRs6KBLXQ_2 ()
    at /mnt/software/n/nim/0.20.2/Nim/lib/pure/concurrency/threadpool.nim:596

(gdb) thread 3
[Switching to thread 3 (Thread 0x7f17231cd700 (LWP 32423))]

(gdb) bt
#0  0x00007f172734be4d in nanosleep () from /lib64/libpthread.so.0
#1  0x000000000044655b in nossleep (milsecs=milsecs@entry=100) at /mnt/software/n/nim/0.20.2/Nim/lib/pure/os.nim:2791
#2  0x000000000046dead in slave_1yqYR9cLeffanOjn0piJ2VQ (w=<optimized out>)
    at /mnt/software/n/nim/0.20.2/Nim/lib/pure/concurrency/threadpool.nim:346
#3  0x0000000000413144 in threadProcWrapDispatch_4bfnMli3YetX01hUbaU1ig_2 (
    thrd=0x79c040 <workers_bezHuru9a1bEti2XBmfodvA+704>) at /mnt/software/n/nim/0.20.2/Nim/lib/system/threads.nim:141
#4  0x000000000041360a in threadProcWrapStackFrame_4bfnMli3YetX01hUbaU1ig (
    thrd=thrd@entry=0x79c040 <workers_bezHuru9a1bEti2XBmfodvA+704>)
    at /mnt/software/n/nim/0.20.2/Nim/lib/system/threads.nim:156
#5  0x000000000041364d in threadProcWrapper_oTnP9cUoE9cVTUL7iHAoIIAA (closure=0x79c040 <workers_bezHuru9a1bEti2XBmfodvA+704>)
    at /mnt/software/n/nim/0.20.2/Nim/lib/system/threads.nim:165
#6  0x00007f1727344e25 in start_thread () from /lib64/libpthread.so.0
#7  0x00007f172707234d in clone () from /lib64/libc.so.6

338 proc slave(w: ptr Worker) {.thread.} =
339   isSlave = true
340   while true:
341     if w.shutdown:
342       w.shutdown = false
343       atomicDec currentPoolSize
344       while true:
345         if w.data != nil:  # <--
346           sleep(threadpoolWaitMs)

That was added recently, in https://github.com/nim-lang/Nim/issues/11275, to fix a threadpool seg-fault.

cdunn2001 (orginal) [2019-08-02T17:31:01+02:00] view original

Created Issue: https://github.com/nim-lang/Nim/issues/11875

Mirror of forum.nim-lang.org

5061 :: spawn/sync hangs -- any ideas?