Hi everyone,
I have worked hardly in last months to introduce a full feature async I/O library in Nim. I think it is a good thing to share what I have realized with you, and that the project won't go very far if people are not interested in it.
It has still a long way to go, but there is quite a lot working now. If you want to see it, you can here : https://github.com/Alogani/NimGo. I hope you will find it a cool project. I'm still quite novice developer, so don't try to use it in production ;-)
Now I will sleep try to sleep a little ^^
Thanks for your attention
Thanks Araq, I appreciate the compliment. Being quite novice programmer, "looks nice" and "has good ideas" is quite a lot :-)
Threads are really nice for CPU bound tasks, but very limited for I/O bound tasks for multiple reasons :
Of course, it can be associated with a dispatcher you can surely do things, but sharing a dispatcher between multiple threads is complex and easily error prone.
NimGo on the other end (which is simply stackful coroutines with an event dispatcher very close to the one of std/ascyndispatch) :
The advantages of NimGo compared to thread are close to the advantages of std/asyncdispatch over threads. They just don't answer the same problematics. And like std/asyncdispatch, you can have one dispatcher by thread, allowing to efficiently handles a very high number of connexions.
For me NimGo should be compared instead with std/asyncdispatch with in fact only one big advantage : no function coloring (and everything that it implies). But at the cost of a higher memory usage.
It proposes itself as an alternative for std/asyncdispatch by proposing a different paradigm (but has no vocation of being a concurrent).
It is directed for the ones who wants async with no headache and aren't primarily focused on raw performance.
In fact, my big project is to propose a framework for creating semi-automated script that wraps the shell but with a higher level API, because I am really into system administration. By nature, it is async, because there can be complex streams manipulation, but not always async. And by goal, it must be concise.
So my NimGo project is just a step for me to make it possible.
Well your stackful coroutine stack is backed up by virtual memory much like a thread. You can have thousands of threads on Linux without any trouble and Linux's scheduler is O(1). However, it is complex to make an event loop threadsafe, that is true.
Please look at https://github.com/guzba/mummy and see what threads can accomplish. ;-)
Woaw, impressive. Even if it is not a fully feature I/O library, having an efficient multithreaded dispatcher is impressive. But for now, there doesn't exists any good asyncio library not relying on async/await. I have looked over projects using threads, but they are not focused on I/O. It is still the consensus that if you want async I/O, you should use something like std/asyncdispatch.
But I still think that concurrent programming has usage case different than parallel programming and are best suited for I/O and complex flow of execution. But yes, there are tradeoffs. And if there is a thread library that can handle efficiently tens of thousands of multiple I/O including non-regular files and with a simple API, I just say a big YES (otherwere that would throw to trash my last three months of works, but so be it, the journey is the destination and I learned a lot)
For precision, coroutines can optionally be backed by virtual memory (not the default in NimGo). If so, generally we assign them as much virtual memory as a thread. But contrary to Go which integrates its own scheduler, they are not automatically resized, so there is a risk of stackoverflow.
Looks nice and has good ideas. But what's the benefit of this over just using threads? Cooperative scheduling is bug-prone much like multi-threading is IMHO without its performance benefits.
Cooperative scheduling/fibers and threads are orthogonal. Threads are handled at the scheduler level.
The fastest multi-threading runtime I have seen actually uses fibers, similar to NimGo, see: https://github.com/chaoran/fibril And in particular the stack switching:
There are many reasons not to use fibers: see C++c oro author criticism - https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1364r0.pdf but performance is not one of the reasons.
Debugging and control-flow obfuscations (i.e. exception incompatibility), the need of assembly, and mmap-ing a memory region as executable (i.e. large attack surface) are more problematic.
Cooperative scheduling/fibers and threads are orthogonal.
Well you can have both in the same program but I have never seen a cooperative scheduling system to produce parallelism.
@mratsim thanks for the sharing. This is indeed a very informative paper you shared. I will link it on top of my library to warn of the risks of using stackful coroutines. I bet Go language have incorporated security mechanisms directly in their runtime, which must induce some overhead to control the avoid stackoverflow.
I bet the remaining options are either keeping the colorful stackless coroutines like async/await, or going fully into threads. But sadly, full async I/O library relying on threads are almost unexistent (and in any languages).
It's just such a joy to have a library like NimGo library where you can do asynchronous with no friction, and be able to just do something like that goStdin.readFile(canceller = sleepTask(500)) with not having to deal with Futures. But indeed, not advisable to use in production code like servers.
And again, concerning performance :
It takes only 244ms on my low end laptop :
import nimgo
import std/times
proc coroEntry() =
suspend()
let t0 = getTime()
for i in 0..100_000:
resume(newCoroutine(coroEntry))
echo "Time=", (getTime() - t0).inMilliseconds()
Where with threads, even when tweaking with ulimits, I'm capped at 9000 threads, and spawning only 9000 took 320ms.
And dealing with asynchronicity, having a thousands of pending/blocking tasks seems not to be a rare case.
A beautiful fiber that threads and yarn are produced from is Silk, Seta in Interlingua
SetaFile, SetaSoccet SilkFile, SilkSoccet
@ingo I am not fond of, but this is original and could be declined well :-D SilkIo, SilkFile, SilkProc, SilkStream, etc.
@ElegantBeef: I agree with you about name clash (it rarely happens, but when it does, it can be annoying). Some problems: (1) Async has already a strong connotation. Even if technically, NimGo do a form of async, it is not the same. (2) So the two API are not similar, asyncdispatch return futures, nimgo returns directly the value (3) I don't imagine the cumbersome if a library has both NimGo and asyncdispatch, to must do "nimgoroutines.AsyncFile" and "asyncdispatch.AsyncFile" at some places. Although I agree with you, it might not be really a problem in most cases and the user would certainly deal with it fine
If Go's implementation was the primary inspiration for this package, than to me, it's fun/nice to give them a nod. It's also a nice, shiny hook for people with Go experience to try Nim. Beyond that, I don't consider the history of the underlying feature all that important, but I don't have a dog in this fight. Also, marketing should be considered to a reasonable degree when naming things when possible, IMO.
Mainly, I just think one would want the name to be a little more specific than it is now.
@ITwrx I agree with you and your arguments are relevant. Please be sure I don't try to argue with you, only to exchange constructively as much as I can.
I also think -like you- that the underlying implementation is not so important. "NimGo" is not only Coroutines, it is a full IO library. With a different kind of async than javascript people are used to (and for me better ! But I am biased). Coroutines/Fibers/Light threads (no matter how one call them) is an implementation detail the user don't need to know.
I don't have a dog in this fight
I do ! I have more or less initiated this project months ago, have spend numerous hours. And I really would like to be proud of it, and that people find it useful. So I try to listen and take the best, but it is not always easy. Everyone is very opinionated in this forum (including me).
And to be fair my Coroutine implementation is very close to Lua's Coroutines (because I wrapped a library of a talented guy who created a compiled version of Lua), but I never used Coroutines in Lua before, neither Java virtual threads.
I took inspirations from everywhere, finally, it resembles Go when we don't look closely. But it does a lot of things drastically differently :