My question is how to make db_mysql module work asynchronously?
I know, that there are other mysql implementations, but this module is part of the standard library and is simple to use.
But, there Is any way to call any synchronous function asynchronously in Nim?
The only thing you can do is spawn a new thread and then use a rather inefficient busy loop to poll its completion. This is how I did stdin reading in Chapter 3 of Nim in Action, you can use it as an example: https://github.com/dom96/nim-in-action-code/blob/master/Chapter3/ChatApp/src/client.nim#L42
That said, I wouldn't recommend this. Hopefully soon we will have a better solution to this.
Thanks @dom96 for the reply.
I wrote this code to test your solution:
import asyncdispatch
import os
import threadpool
proc testSync(): bool =
var count = 0
while true:
sleep(1000)
echo "wakeup sync... ", $count
count += 1
if count == 10:
return true
return false
proc testAsync(): Future[bool] {.async.} =
var count = 0
while true:
await sleepAsync(1000)
echo "wakeup async... ", $count
count += 1
if count == 10:
return true
return false
proc test() {.async.} =
var messageFlowVar = spawn testSync()
while true:
if messageFlowVar.isReady():
echo "result from testSync"
break
sleep(1000)
echo "wait for the result..."
proc main() {.async.} =
while true:
asyncCheck testAsync() # function is never called
await test()
echo "continue..."
asyncCheck main()
I mean, did you check that testAsync definitely doesn't get called? Like did you add an echo at the beginning of it?
Anyway, I bet the problem you're seeing is because you're blocking the thread in test by calling sleep, that will block all of async. Never call sleep in an async proc, instead use sleepAsync.
This is a test to simulate the use of a synchronous function in an asynchronous environment!
What should I do to run the synchronous sleep function at the same time as the asynchronous sleep function without blocking?
The only thing you can do is spawn a new thread and then use a rather inefficient busy loop to poll its completion.
There aren’t any async-friendly ways to wait/notify? Could you open a pipe, create an asyncsocket on it, and then read a byte from it?
This is a test to simulate the use of a synchronous function in an asynchronous environment!
I feel like you're trying to say something by this but I am missing it. I know that this what you are doing, your testSync proc can use sleep since it's spawned in another thread, but you cannot use sleep inside the async test proc. You're blocking the main thread where the async loop runs in!
What should I do to run the synchronous sleep function at the same time as the asynchronous sleep function without blocking?
Run it in a different thread, which you are doing in testSync via spawn.
There aren’t any async-friendly ways to wait/notify? Could you open a pipe, create an asyncsocket on it, and then read a byte from it?
That's true, you could do this but as @xendi noted you'd need to run the event loop on each thread you're using. Not a bad idea to be fair, might be good to create a generic library that enables running synchronous code this way.
The usual way to build an async/non-blocking interface to synchronous/blocking system calls like read(hard disk) is a thread (or kid process or even unrelated (and likely multi-OS-threaded) "server" process) that does the invoking of the possibly blocking system calls. This "helper entity" then has a more async pipe or socket interface to other things that have a more explicit event loop/async interface. @snej and @dom96 and others have probably understood this, but I thought some explicitness and alternate keywords might make things more clear. A user-level busy wait should not be necessary { even though we now live in an age with CPUs with several dozen cores :-) }. Also worth mentioning explicitly - this helper can be profitably long-lived, started on demand at the first request or some such.
There is still, as @xendi & @dom96 say, a conceptual "event loop", but it is "implicit"/in kernel space waiting for IO interrupts and the like and much more CPU scheduler friendly than a user-level busy-wait. Indeed, how busy of a wait the kernel does likely depends on the OS and hardware in play and sometimes background activity. For example, under heavy load, polling a network device is faster than waiting for it to tell you data is ready, but under light load that wastes time. { And, of course, there is still the high level outermost async event loop, but I believe that was implicit in the question here. }
For even more explicitness/full elaboration, one might see the Usenix paper on the Flash webserver which has nice diagrams and such. The design is not Unix-centric, though the language/venue may be. { I do not think this paper is the first to use such a design, but it's a pretty readable / pedagogical paper... }
There is an extra memory copy or two in the usual simple impl of this. Those are usually cheap compared to "real IO".
After some investigation I found a way to make it work! But I don't know if it is the best solution, or if it is correct. I write a small program to illustrate the idea.
import asynchttpserver, asyncdispatch
import os
import threadpool
import random
import strutils
import times
proc mySyncProc(n: int): int =
var count = 0
while true:
let t = rand(1000..2000)
sleep(t)
count += 1
if count == n:
return count
return 0
### Is this correct? ###
proc testSync(): int =
let messageFlowVar = spawn mySyncProc(rand(10..20))
while not messageFlowVar.isReady(): ## what happen if never is ready???
asyncdispatch.poll() ## This prevents the blocking???
return ^messageFlowVar
proc main() =
var server = newAsyncHttpServer()
proc cb(req: Request) {.async.} =
let start = now()
let r = testSync()
await req.respond(Http200, "Hello World $1 Duration: $2" % [$r, $(now() - start)])
waitFor server.serve(Port(8080), cb)
main()