I'm still trying to decide what to use for networking. I've noticed there was Nim support for libuv, which I hadn't heard of before (thanks to not wanting to have anything to do with JS and node.js), but I'm not sure if it's used at all. The design of libuv sounds perfect for my use-case.
First, there is libuv from the Standard Library. Based on the comment in the header, which says it was build using this repo, which was "moved" 3 years ago, I have to assume that that module has not been updated for at least 3 years (unless it was, but no one updated the doc for it, and corrected the header). So I guess it's not actually used, otherwise someone would have updated it.
Then there is libuv from the "Unofficial packages", which was updated (created?) "2 months ago", but I can't see any hint about which version of libuv it was based on; it just says "Extracted from Nim standard library", which might mean it is also based on an (at least) 3 years old version of libuv.
When it comes to SSL, it seems libuv team is against integration, because they don't want to "take sides" (choose a specific SSL impl). A bit of research resulted in 2 possible solutions:
In a world where "everyone" gets hacked all the time, I can't relate to "networking libraries/frameworks" which treat "security" as an "optional feature".
Please use the rapper I made.
It supports APIs included in the latest version(libuv 1.18.1), and it can be used comfortably as API can be used like the original.
In order to use it, the latest libuv must be installed.(made when libuv 1.18.1)
Since this wrapper is still in beta, please send me a pull request to tell me if there is any problem.
this is toy project using my wrapper made by me.
However, it is faster than Rust's tokio-minihttp.
If you can do an asynchronous approach yourself, using the event loop of the Selectors module will make for even faster things.
Or, if you do not support Windows, libev will be slightly faster than Selectors.
Here is the wrapper for libev.
I have seen evt-tls now.
dont the code base to look so big, so I would like to write a wrapper or implement it pure.
@2vg Nice! :) I'm going to give it a try. I do need to support Windows; that is my primary target for the client (the server will be primarily Linux).
I won't need encryption until I release the first alpha version, and I'm a long way away. If you plan to have a look at SSL support, that would be great. :)
perhaps Araq is talking about SSL wrapping.
looking at devel, it seems that the wrapper of libuv has been delete.
@yglukhov Maybe, but if I understand asyncdispatch propertly, it's really not comparable to libuv. I haven't checked the implementation of asyncdispatch, but it looks to me as if, under the hood, it's still one-thread-per-blocking-IO-call. So If I was listening to 100 sockets, it would need 100 threads? If that is the case, then asyncdispatch simply isn't an option for me, as it won't scale well.
@2vg Well, if the libuv wrapper was deleted from devel, then Araq couldn't possibly have been implying that SSL support for libuv should belong to stdlib. We'll just have to wait for Araq to reply.
@mashingan I think what I'm currently working on (an 'actor' system) would be an 'alternative' to 'reactor'. So building on top of reactor makes little sense for me. Reactor says 'code looks very similar to a code written in a blocking style'. That's kind of the opposite of what I want (no use of "await" anywhere). Also, 'reactor' is the name I wanted to use (because my actors are "passive"; they mostly "react" to messages, therefore 'reactor' seems like the perfect name for it), and so far all other names I thought of aren't as good a match for what I'm working on.
@yglukhov Unless otherwise informed, I always read "async" as "runs in some other thread". And since the easiest way to do this is just to queue the tasks in a thread-pool, that is also what I assumed so far. Apparently, I was wrong. This changes everything; I'll have a second look at AsyncDispatch.
EDIT: The "second look" was rather short... "as long as you only care about non-Windows platforms" basically mean asyncnet (networking over AsyncDispatch) is useless for me. :(
I'll also note that I'm currently assuming my code will be CPU-bound, so I'm more worried about threads being created in the background, outside of my own code, than worried about "ultimate IO efficiency".
"Or, if you do not support Windows, libev will be slightly faster than Selectors. Here is the wrapper for libev."
@2vg, please prove your words with benchmarks, or stop saying non-sense like this.
@monster, please understand properly, i'm writing this with no offence.
After reading all topic, the only thing i understand, that you absolutely have no experience in asynchronous networking. Everything you are looking for - present in stdlib (maybe not properly documented), but you still able to use SSL and handle thousands of sockets in single thread.
@cheatfate No offense taken. :) If asyncdispatch is implemented as an "event loop", rather than a "thread-pool + task queue", and it supports SSL too, than it seems "good enough" for my use-case. Until now, I admit I only worked with synchronous IO (in Java). But what I want to do is very different from what I've done before, and async IO seems much better suited.
What I'd like to know is, what does the statement "as long as you only care about non-Windows platforms" from asyncnet really mean? Windows is going to be my primary target, so I do care about Windows, a lot (note, care != like). I would rather go with something that is part of stdlib, as it generally reduces the overall "friction", but not if that means I cannot use it on my primary target platform. If it only means "it's going to be a bit slower on Windows", then I can totally live with that for now, at least until I reach "beta", and need to work on performance.
What I'd like to know is, what does the statement "as long as you only care about non-Windows platforms" from asyncnet really mean?
@Monster, It means that the layers for implementing asyncnet which consist of selectors module, asyncdispatch, etc can be used interchangeably, perhaps you want to use kqueue instead of epoll for the selectors, you can do that.
The different for Windows that windows has no independent loop dispatcher and it's already available on win 32 api as IOCP (I/O Completion Port), hence when in Windows you can only use what's given on asyncdispatch.
I think libuv (or libev? I forget which) actually is a wrapper for calling those native api, on Windows using IOCP, on Linux using epoll, and it's same with asyncdispatch.
On Windows, you can take it different approach with coroutines, which the implementation on Windows using native API, Fiber. However using IOCP is certainly easier and maybe somehow more optimized instead of implementing our own.
@yglukhov, could you give some simple pointer the approach you took for implementing asyncSpawn ?
@mashingan I'll just "rephrase" to be sure I get it. On Linux (and maybe other OSs), there are several OS APIs to do non-blocking IO. On Windows, there is only one. asyncdispatch/asyncnet (as well as libuv) are wrappers on top of (some of) those. asyncdispatch/asyncnet essentially give you the same high functionality on all platforms. And the "as long as you only care about non-Windows platforms" comment only relates to "you should be able to work with any of these layers[epoll/kqueue] interchangeably". Beyond "Warning: The Peek socket flag is not supported on Windows." in asyncdispatch, I haven't seen an actual limitation of the API behaviour on Windows.
So, assuming I got the above correctly, since I don't care about which API implements the low-level non-blocking IO magic under the hood, I can just use asyncdispatch/asyncnet everywhere, and call it a day?
@Monster, yes, you're right.
Disclaimer: I don't have much experience with asynchronous programming, so hopefully others can confirm whether you're right or not, above is based on how I understand about asynchronous programming
@monster, Unix/Linux/BSD OSes and MacOS there compatible event-driven api and also you can set file descriptor to be non-blocking, so all blocking operations become non-blocking. Asyncdispatch using epoll on Linux, kqueue on Free/Open/Dragonfly/NetBSD and MacOS, poll on Solaris.
Windows has much more ways to making perform asynchronous operations. Asyncdispatch using OVERLAPPED operations and IOCP ports.
All low-level procedures are inside of asyncdispatch.nim, all high-level procedures are inside of asyncnet.nim.
Windows implementation is compatible with Unix implementation but not 100%. If you start to use addRead, addWrite, current Windows implementation of addRead and addWrite is tricky and i think its pretty heavy, but it working. But i'm still not advise you to use it. This problem if because of Windows IOCP ports nature.
So, if you are using only accept/connect/send/recv/sendTo/recvFrom you will have no problems.