I've been a user of the Parallel gem (https://github.com/grosser/parallel) for a long, long time and I've always loved its simplicity and efficiency. (You may have a look into the README and I guess it shouldn't take you more than 3 minutes to figure out how this is to be used).
What is the closest to this in Nim?
(I've tried the parallel command, I've also tried Weave, but with absolute honestly I've never managed to get any of the two working as expected - or at least in any obvious way as above)
Any ideas? (I'm thinking of ways I could potentially convert the aforementioned library for Nim, so any input is more than welcome)
template each(iter, v, body): untyped =
for v in iter:
block:
let v = v
spawn (proc() = body)()
sync()
[1, 2, 3].each num:
echo numThis also works:
macro parallelEach(loop: ForLoopStmt): untyped =
let
v = loop[0]
iter = loop[1][1]
cmd = loop[2]
result = quote do:
for `v` in `iter`:
block:
let `v` = `v`
spawn (proc() = `cmd`)()
sync()
for i in parallelEach(1 .. 3):
echo iIt may or may not be deprecated/well supported, but @drkameleon's list should also include
iterator system.`||`
for i in `||`(1..10): echo i
Also, very often process-level parallelism is practical/sufficient, safer, and neglected. As a syntactically noisier solution, you can do that with cligen/procpool with two fully worked out examples for parallel string search as fast/faster than threaded ripgrep and parallel file selection by magic-based file type. procpool is presently Unix-only, but Windows has pipes, it should be easy to add support for that, and I'm happy to shepherd a PR if anyone with a good test set up is interested (ideally, VisualC + gcc on Windows). PRs for lower syntax noise are also ok. :)
Personnaly I use these two library :
https://github.com/status-im/nim-taskpools https://github.com/yglukhov/asyncthreadpool
Obligatory, you likely don't need multithreading but async for IO-bound tasks like networking.
Not exactly the same, but perhaps this is useful to you: experimental parallel:
https://nim-lang.org/docs/manual_experimental.html#parallel-amp-spawn-parallel-statement
and set the max threads: https://nim-lang.org/docs/threadpool.html#setMaxPoolSize%2Crange%5B%5D
it takes care of synchronization issues itself. It's not async I/O, but thread parallelism. It will still speedup execution time, and wrt blown memory usage, I think it works with modern computers. if it's for simple tools I think it fits well.
If you want a general purpose task manager, I think a friendly version doesn't yet exist in Nim