Hello, I am still looking for a way to produce multi-threaded async HTTP server. Current implementation of asynchttpserver relies on asyncdispatch/net, but effectively it is single threaded. Everything runs inside single (main) thread. The problem is that modern servers have multi-core processors, and if we really want to be able to produce scalable solutions using Nim, comparable to Java solutions, we need to either go to multi-process or multi-threaded architecture.
Multi-process architecture may be the easiest solution at first glance, but requires extra proxy service, proxying single public port to many processes local ports. This adds unnecessary layer of complication. So it would be better if dispatch to multiple cores can be done within the app itself.
IMHO the easiest solution would be to have client socket bound to a thread from the pool, each thread in the pool running its own event loop for its sockets. The binding could be done when accept(serverSocket) -> clientSocket, adding clientSocket to some thread event loop. Since clientSocket is (usually) integer FD, passing it from main listen thread to worker threads is very simple.
However I find current implementation of asyncdispatch somehow hard to understand (chain of callbacks and implicit iterators) and I need some help on how to marry it with threadpool :)
So far I managed to produce stupid MT server via:
import asyncnet, asyncdispatch, threadpool
proc processClient(client: AsyncSocket) {.async.} =
while true:
let line = await client.recvLine()
await client.send(line & "\c\L")
proc runClientOnNewThread(client: AsyncSocket) =
client.getFd.AsyncFD.register() #<- need to manually register FD in this
# thread dispatcher
asyncCheck processClient(client)
runForever() #<- need that for event loop in the new thread,
# PROBLEM 1: we never exit from this loop
proc serve() {.async.} =
var server = newAsyncSocket()
server.bindAddr(Port(1235))
server.listen()
while true:
let client = await server.accept()
client.getFd.AsyncFD.unregister()
spawn runClientOnNewThread(client)
asyncCheck serve()
runForever() #<- event loop in main thread
But it is wrong, since if effectively processes only one client on each spawned thread. Ideally it would be if there was no need for boilerplate like that and we got something like asyncSpawn, that would move future processing into some other (worker) thread, eg.:
import asyncnet, asyncdispatch, threadpool
proc processClient(client: AsyncSocket) {.async.} =
while true:
let line = await client.recvLine()
await client.send(line & "\c\L")
proc serve() {.async.} =
var server = newAsyncSocket()
server.bindAddr(Port(1235))
server.listen()
while true:
let client = await server.accept()
asyncSpawn processClient(client) #<- future (asyncsocket) gets processed
# in separate worker thread
asyncCheck serve()
runForever()
Any ideas how this can be effectively implemented and what are the caveats here?
Hey, i know this old thread and it is not recommended to use asyncdispatch with threads.
But here is one example: http://pastebin.com/0q27aNvN
It works fine if you compile it with -d:release and --gc:markandsweep on windows(I've only tested it on windows). Though pretty much same performance as singlethreaded, sometimes can be faster. I tested the server with ab: ab -n 15000 -c 50 -k http://127.0.0.1/