Wondering what's the current state of Channels (Go-like) or Actor (Erlang-like) parallelism in Nim?
I know there are Async and Weave. But I don't like Async approach, and Weave seems to be nice solution, but it's more like specialised high throughput solution for numerical calculations.
There's a channels library, but seems like not much discussion about it, is it used and developed? Are there other Go / Erlang like libraries?
Channels, Actors are more models of computation than parallelism. I'd say models of computation is a theoretical way of describing your programs.
So to give you more info, here are examples models of computation
Go uses CSP (Communicating Sequential Processes) which I like to tag as "named channels + anonymous functions". Erlang uses Actors which I tag as "named behaviors + anonymous channels".
It is possible to implement them in Nim but a lot of ground work was and still is necessary to make them efficient.
Technical roadblocks:
Note that in this model:
Ergonomic roadblock:
Besides that, the current research in the Nim community is around Continuation-Passing Style, to make continuations a core primitive of the language. Continuations reify the "rest of the program" (aka transform it into a concrete object) which would allow event loops, reactors, executors pass around what to do after an event occur. The most interesting thing is that it would make everything in Nim play nice with each other including:
Continuations are able to express any control flow, including exceptions and are so fundamental that functional language compilers use it as their intermediate representation of a program control flow. Having them as first-class would significantly help Nim async and parallel programming in particular making them play well together.
What's your reasoning for disliking the Async approach? Is it specifically Nim's implementation of it or do you dislike async I/O in general?
If you're looking into performant web servers then take a look at httpbeast which combines Nim's threads and async. It doesn't support Windows though.
What's your reasoning for disliking the Async approach? Is it specifically Nim's implementation of it or do you dislike async I/O in general?
I dislike Async in general. Used it for many years in JavaScript UI and Node.JS Web Servers.
It works more or less well in JS, because it's the only thing that's available in JS. I find it surprising that other languages trying to adopt async approach, when there are better ways.
Problems with Async (I may miss some details how async works in Nim, it's mostly my experience in Node.JS):
1 It adds lots of meaningless noise and complications to the code. Let's consider example, getting list of post titles from the database.
let titles = db.posts.all().map((post) => post.title)
I don't care about the minor details of how this piece of code going to be executed. I assume the computer is smart enough to execute it in a reasonable efficient manner.
But with async approach computer acts as if it's really dumb, and requires from develoepr to explain every minor detail and every line, how to execute the most trivial piece of the code. I need to put await to explain computer how to do the simplest things.
let titles = (await db.posts.all()).map((post) => post.title)
It adds another dimensions to function. Now it's not just a mapping of Arguments -> Result. But it has also a flag if it's async or not.
What's worse, it's not easy to combine sync functions with async.
As a consequence, higher order functions like map won't work with async. You can't do await inside of map like posts.map((post) => await post.title()) and similar things.
3 No execution context.
You can't easily intercept exceptions with try/catch you need to carefully propagate async errors up if you want to intercept and handle it.
And you can't have an isolated context associated with say web server request. You can't localize error to that context only, you can't terminate and kill that context, and it's not isolated problems in one request may leak and affect other requests.
Funny thing, in Node.JS unhandled exception would crash the whole server (there are ways around that). That's wrong. You should have tools to isolate errors, so failure in some part of the system would not crash the whole system. Async approach won't help much with that. (Compare to Erlang where you can safely crash/kill/restart not just individual components, but whole trees of such cmponents).
4 Even with all that complexities, async is still not good enough.
It solves only one problem, how to deal with IO. It doesn't solve how to efficiently utilize multiple CPU cores. So, on top of all that complexities with async you also need to add even more complexities to intergate it with multicore.
5 Async Streams
In theory, sounds great, easy and efficient streaming. In practice - try properly connect couple of streams in Node.JS with proper error handling and resource clean up. Works great for example like req.send(file) not so well when you need to build robust servers working reliably without crashes, surprising errors and leaks.
6 Async Errors
Maybe not the case in Nim. In Node.JS in some cases the stack trace is lost in Async error, and you have fun time trying to understand what's going on and where that error is coming from.
I agree, but it's mostly a matter of how you design your code.
When you have a code with just one happy path, then - sure, async is very uncomfortable. However, when you have an (essentially) random list of events being fired at you and having to fire back, e.g. when implementing a low-level GUI, or a "chatty" protocol such as IRC or git's line protocol, then it is very natural and (for me) even more comfortable than sync code, especially if you have two event sources -- e.g. a socket and a GUI; Socket may close which means you need to update GUI; GUI may want socket shutdown even though socket is still blocking on an earlier send. In such a scenario, the async version is simpler in just about every respect, and often much more responsive to the user (I haven't used Outlook recently, but at least as of 5 years ago, the IMAP/SMTP code was sync, and it interacted horribly with the GUI if you had timeouts from the server).
The bottom line, I think is, that it's not just about the specific job, but the entire context; You can't even say "email is better done sync" or "better done async" - if you wrote a batch program to download mail, sync is always simpler. If you write a GUI program to read mail, doing it both sync AND responsive is much more complicated than async.
Personally, unless it's a batch program I prefer async. I subscribe to the Alan Cox view that most programs are state machines; and state machines are often easier to implement in an async way, especially when you have multiple running at the same time.
when you have an (essentially) random list of events being fired at you and having to fire back, e.g. when implementing a low-level GUI, or a "chatty" protocol such as IRC
That's the exact case the Erlang with Actor Model was created for :). As a reliable and simple way to handle unreliable streams of events / messages for the phone network. Async way is not the only and not the best way to create event based systems.
Async way is not the only way to create event based systems.
Of course it isn't. But it works well in Erlang thanks to many other aspects ("crash-only" error handling style, supervision trees, supercheap threads, message passing and matching). Some of these are less useful for e.g. GUI -- at least, I haven't seem them applied in a way that is both convenient and provides a reasonable experience.
I've dabbled in Erlang, do not claim to be an expert, and would love to be proven wrong -- and when I am proved wrong about this, there's still all of these other features which are more than just "library level" that make Erlang what it is - they go through the entire runtime, ecosystem, and culture. Adopting only the actor model without them is unlikely to deliver the same robustness and ease you get in Erlang.
2 more problems with async in Nim that don't exist in Node.JS
8 Event Loop could be easily accidentally blocked by calling non-async IO function. And it would block the whole server, the whole server could be made slow and unresponsive by just one accidental non-async IO call. This problem doesn't exist in Node.JS because there are no non-async IO functions (they exists, but are special and practically never used).
9 API fragmentation. IO libraries (database, requests, API-wrappers, cache, etc.) used with async server needs to be written in async way. Or provide both sync and async functions, like db.query and db.queryAsync. No such problem in Node.JS as there're no sync IO libraries, all libraries related to IO are async.