The library has been running 24/7 for the last couple months on our Ethereum fleet in test networks and some in production.
Example usage: https://github.com/status-im/nim-taskpools/blob/26e3b1e/examples/e01_simple_tasks.nim
import ../taskpools/taskpools
block: # Async without result
proc display_int(x: int) =
stdout.write(x)
stdout.write(" - SUCCESS\n")
proc main() =
echo "\nSanity check 1: Printing 123456 654321 in parallel"
var tp = Taskpool.new(numThreads = 4)
tp.spawn display_int(123456)
tp.spawn display_int(654321)
tp.shutdown()
main()
block: # Async/Await
var tp: Taskpool
proc async_fib(n: int): int =
if n < 2:
return n
let x = tp.spawn async_fib(n-1)
let y = async_fib(n-2)
result = sync(x) + y
proc main2() =
echo "\nSanity check 2: fib(20)"
tp = Taskpool.new()
let f = async_fib(20)
tp.shutdown()
doAssert f == 6765
main2()
The primitives are:
The main logic is really short, just under 500 lines of code https://github.com/status-im/nim-taskpools/blob/26e3b1e/taskpools/taskpools.nim. The threadpool is work-stealing based, lock-free, except for the logic to put threads to sleep (the lockfree alternative "eventcount" is quite error-prone and would need formal verification to ensure no deadlocks are possible and run contrary to the "easily auditable/maintainable goal).
Compared to Weave, here are the main differences:
Note, despite all those perceived shortcomings, taskpools should be a high performing threadpool even compared to all other languages, and especially compared to the one included in the standard library.
While it relies on the 1.6 std/tasks, it has a compatibility shim to support Nim 1.2 and Nim 1.4 as well.
Awesome project, thanks for sharing!
Nitpicking: The readability of
let x = tp.spawn async_fib(n-1)
let y = async_fib(n-2)
is really bad though. Would following NEP1 improve it?
let x = tp.spawn asyncFib(n-1)
let y = asyncFib(n-2)
Yes, unsurprisingly. ;-)
@Araq, sure you can use whatever style. If readability is a problem you can also change fonts ;).
@elcritch, it works with plain-old-data (no need for arc) and any GC that is not thread-local (i.e. memory can be reclaimed from any thread), so malloc/Boehm/ARC/ORC would work.