Hi, I am trying to figure out how to make Nim async modules for IO play nice with multithreading. I have started from the following simple example: I create a separate thread, which acts as a worker, and send requests to it via a channel. Requests are then completed in the worker thread. It goes like this:
import asynchttpserver, asyncdispatch, strtabs
var channel: Channel[Request]
open(channel)
proc work() {.thread.} =
proc loop() {.async.} =
while true:
let req = channel.recv()
await req.respond(Http200, "Hello World", {"Content-Type": "text/plain"}.newStringTable())
waitFor loop()
var worker: Thread[void]
createThread(worker, work)
proc handler(req: Request) {.async.} =
channel.send(req)
var server = newAsyncHttpServer()
waitFor server.serve(Port(5000), handler)
Of course, it fails. The exception is:
Traceback (most recent call last)
web2.nim(12) work
asyncdispatch.nim(1624) waitFor
asyncdispatch.nim(289) read
Error: unhandled exception: File descriptor not registered.
send's lead up to read of failed Future:
Traceback (most recent call last)
web2.nim(12) work
asyncdispatch.nim(1270) loop
asyncdispatch.nim(1257) cb
web2.nim(10) loopIter
asynchttpserver.nim(129) respond
asyncdispatch.nim(1270) send
asyncdispatch.nim(1257) cb
asyncnet.nim(283) sendIter
asyncdispatch.nim(1184) send
asyncdispatch.nim(1011) addWrite
loop's lead up to read of failed Future:
Traceback (most recent call last)
web2.nim(12) work
asyncdispatch.nim(1270) loop
asyncdispatch.nim(1262) cb
asyncdispatch.nim(260) callback=
asyncdispatch.nim(1257) cb
web2.nim(10) loopIter
asyncdispatch.nim(289) read [Exception]
It seems that this has to do with the fact that the file depscriptor is open in the main thread, and it is used in the worker thread, where it is "not registered" (whatever that means).
I think this has to do with the fact that these file descriptors are not thread-safe, and hence some mechanism prevents to use them in other threads.
Can anyone shed some light about how the whole mechanism works, and how much work would it be to make async sockets thread-safe (if that is even desirable, given that it might slow down the single-threaded case)?
Why are you talking about Go? Do not assume, please. I actually have never used Go, so I will not respond to this point.
I am well aware that Nim has thread-local heap, each one running its own GC (at least when using the default GC; other are available). The problem I am mentioning has nothing to do with garbage collection. In fact, I am using channels to send requests across threads, and those simply deep copy the data from one heap to another.
I think it would be very interesting be able to multiplex HTTP requests from an asyncHttpServer to various worker threads, in order to improve throughput.
andrea, I'm not sure how to get it to work past this, but I was able to get it to not crash at least :)
Your channel var needs to be a threadvar in order to be accessed from other threads (figured out by looking in the async modules).
Change your channel declaration to:
var channel {.threadvar.}: Channel[Request]
And you'll get a nice infinite loop. I'm not quite sure why, but hopefully this piece helps.
Hm, I got an error: too large thread local storage size requested
At least, it did not crash...
@jyapayne I think what is going on is the following.
Since you make the channel thread-local, the main thread and the worker thread see different instances of the channel. This means that the message sent from the main thread is never received in the worker thread (because it is listening on a different channel), and that is why you get an infinite loop.
If the channel is global, the request is correctly sent to the worker: in fact, the crash happens when the worker tries to complete the request (see the line asynchttpserver.nim(129) respond in the stacktrace).
The reason for this is that the file descriptors are stored inside a thread-local variable. So, when the worker thread tries to respond, it does not find the suitable file descriptor for the socket, and an exception arises here.
So, the question is: why make the file descriptors thread-local? I guess that the reason is that writing to them is not thread-safe.
I see two possible approaches:
I would like to try the second approach. (Actually it needs to be refined: in order to support response streaming, one also needs to be able to send response chunks).
But I am not sure how to make it work: the main thread is busy in the server loop, and I don't know where I can find a spot to also listen to the response channel
An example of the approach I am talking about:
import asynchttpserver, asyncdispatch, strtabs
type Response = object
req: Request
body: string
headers: StringTableRef
var
requests: Channel[Request]
responses: Channel[Response]
open(requests)
open(responses)
proc work() {.thread.} =
while true:
let req = requests.recv()
let resp = Response(
req: req,
body: "Hello World",
headers: {"Content-Type": "text/plain"}.newStringTable()
)
responses.send(resp)
var worker: Thread[void]
createThread(worker, work)
proc handler(req: Request) {.async.} =
requests.send(req)
let (ok, resp) = responses.tryRecv()
if ok:
await resp.req.respond(Http200, resp.body, resp.headers)
var server = newAsyncHttpServer()
waitFor server.serve(Port(5000), handler)
But this has the obvious drawback that the server only checks whether it needs to complete a request when serving the next one. Hence, requests are left waiting until some other request arrives later.
What I would need is the ability to intersperse two kind of events:
and listen for both in the main thread. Hopefully without adding so much overhead that going multithreaded slows down everything :-P