Dear all,
when I compile my code (with sending value to the thread's Channel) using 1.6.12 compiler, I get the error message:
SIGSEGV: Illegal storage access. (Attempt to read from nil?)
This doesn't happen all the time - only if I send something repeatedly (of course I close/open the channel before using it) using a loop. Debugger says it occurs exactly when I trying to send a data to the channel. Due to complexity, I can't use debugger to dive deeper into the code.
At the meantime, using Nim 2.0 (1.9) compiler, there's no such an issue. Compiles and runs smoothly.
So my quick question is about whether I have to keep Nim 2.0 production, or rewrite the code using 1.6.12 and Locks to access global variables?
Thanks!
Hi, i am really intrested in this one. I guess i had similar issues with the channels before reading the docs carefully.
Channels have to be stored i a global or set up the complicated way... which i never have tried...
Are there any fundamental changes about this in 2.0? Would be a blast :)
Thank for your reply, Araq. In the main app file I have stopAll() function.
Simplified definition of this function in another module:
proc stopAll*() =
withLock isDownloadLock: # Guard for global isDownload bool var
if isDownload: # Used to check whether download is going on
echo "Stopping download"
commStopDown.send(true) # this line is crashing the app
sync()
This piece of code is included in the thread checking whether data received from the channel:
let commChanDownData = commStopDown.tryRecv()
if commChanDownData.dataAvailable:
commStopDown.close()
withLock isDownloadLock:
isDownload = false
echo "Stopped download"
break
Channel initialised in the same module as:
var commStopDown* : Channel[bool]
The code compiled and running perfectly in Nim 2.0 RC2 (1.9.3). Approach of the rookie, but I'm learning my beloved language slowly :)
You're welcome. Basically, it's enough to declare:
myChan*: Channel[bool]
myChan.open()
myChan.send(true)
let myChanData = myChan.tryRecv()
if myChanData.dataAvailable:
discard # do something when data received
But, as I've mentioned, in 1.6.12 it's not working as expected sometimes.
Hi @demetera, this seems to be the basic way of setup, which should work fine for 1.6.12
For me it does... even with this slightly broken example...
If you remove the comment from sleep(1) it fails for sure, with "Illegal storage access"
Is there any chance that you have some kind of race condition where a channel is accessed before it is open? Most likely it's not that obvious and i can imagine that there are slight differences in the nim versions that this timining occurs more often in 1.6. But that is wild guessing...
Can you try to break this example your way?
import os
#nim -r c -d:orc --threads:on test.nim
var queue: Channel[string]
proc reader() {.thread.} =
while true:
echo "recv"
let cData = queue.tryrecv()
if cData.dataAvailable:
echo cData.msg
sleep(3000)
proc writer() {.thread.} =
while true:
echo "send"
queue.send("hallo")
sleep(5000)
proc main()=
#open(queue,1) #this would be right
var reader_thread: Thread[void]
var writer_thread: Thread[void]
createThread[void](reader_thread, reader)
createThread[void](writer_thread, writer)
#
#sleep(1) # break it for sure
open(queue,1)
#
joinThread(reader_thread)
joinThread(writer_thread)
when isMainModule:
main()
By the way what i have tried and couldn't get to work was setting up the channels on the fly and pass a channel to one reader and one writer. Something like
for x in mySeq:
var queue: Channel[string]
open(queue)
var reader_thread: Thread[Channel[string]]
var writer_thread: Thread[Channel[string]]
createThread[Channel[string]](reader_thread, reader,queue)
createThread[Channel[string]](writer_thread, writer,queue)
I cant use threading/channels in production when tsan complains about them. Its a struggle already to advocate for using a language at work that's not c/c++, If I build this project on top of a buggy stdlib it'll be the end of Nim at my workplace. ( my job won't be more secure either lol)
Maybe isolated provides an invariant that tsan doesn't know about. I must admit I don't completely grok what class of bugs it protects against, certainly not enough for a code review.
mratsims various channels are pretty solidly tested, at least the mpsc version, though they're lockfree. @elcritch, you've done some channel wrangling too, yeah?
Let me explain, what was the problem.
I have a while loop, in which user enters commands to control the thread (start and stop it) with the channels. Application, compiled with 1.6.12 was crashing exactly, when I was stopping the thread during 2nd iteration. After I removed expression to close the channel (when , everything started to working in 1.6.12.
let commChanDownData = commStopDown.tryRecv()
if commChanDownData.dataAvailable:
# commStopDown.close() # Problem was here
withLock isDownloadLock:
isDownload = false
echo "Stopped download"
break
The main question, is it mandatory to close the channel and open it again (now I keep the channel opened until the app quit)? While Nim 2.0 tolerates reopening/closure of the channel on the go.
Sorry for the vague description, but I hope more or less it's clear what was happened
Works like a charm...
Nim Compiler Version 1.6.12 [MacOSX: arm64] Compiled at 2023-03-10
import os,locks
#nim -r c -d:orc --threads:on test.nim
var commStopDown: Channel[bool]
var isDownloadLock:Lock
var isDownload = false;
proc reader() {.thread.} =
while true:
let cData = commStopDown.tryrecv()
if cData.dataAvailable:
commStopDown.close()
withLock isDownloadLock:
isDownload = false
echo "Stopped download"
break
sleep(500)
echo "alive"
proc main()=
initLock(isDownloadLock)
var open = false
while true:
discard stdin.readLine()
open = not open
if open:
open(commStopDown)
var reader_thread: Thread[void]
createThread[void](reader_thread, reader)
isDownload = true
else:
commStopDown.send(true)
when isMainModule:
main()
Thanks for the testing. Not really. The app is relatively simple, but I had the mentioned issue. This is the demo code : https://codeberg.org/demetera/nimiptv_demo
Synchronous downloader for the IPTV streams
This seems to be the fixed version... How should we know what was wrong in the other one?
But two things... You use spawn, while the docs say
Note: Channels are designed for the Thread type. They are unstable when used with spawn
https://nim-lang.org/docs/channels_builtin.html
changed for 2.0 ?
And just to be sure you removed something like this when you removed the close?
of "d":
if not isDownload and not isFetch:
#openChannels() # This one?
spawn fetchStream(Url)
spawn downloadStream(Url)
Thanks for your analysis.
Yes, I saw in documentation, that channels are unstable with the spawn. Meanwhile, there were no issues so far, except mentioned in the initial post.
I've removed lines, which were intended to close the channels
let commChanDownData = commStopDown.tryRecv()
if commChanDownData.dataAvailable:
videoArray = @[]
withLock isDownloadLock:
isDownload = false
#commStopDown.close() # This one
echo "Stopped download"
break
let commChanStreamData = commStopStream.tryRecv()
if commChanStreamData.dataAvailable:
echo "Stopped fetch"
#commStopStream.close() # And this
withLock isFetchLock:
isFetch = false
break
from fetchStream.
And indeed I was opening the channels exactly as you've highlighted.