lol, first step's first right?
Yah, I noticed the channel doesn't use Isolated[T] yet... but you can still call isolate to test your data, then pass the non-isolated data through the current channel impl. It kinda seems to work that way.
I could probably write an example channel impl as I'm testing some data passing with threading. Would it be just changing the API to require Isolated[T] or would there be other design changes? Unfortunately, I won't have time to make a nice PR, but I could post a gist for testing purposes.
That would work but I wanted to adopt one of mratsim's channel implementations instead because ours is slow...
Make sense... looks like the current impl just copies the entire message queue. It works, but it would be nice to have a more efficient version. I'll leave it for someone more experienced for now. :-)
Beyond my "lock-less" channels (no lock but not lock-free in the sense of the academic comprehension), there is also a lock-based fast channel that comes from the Michael & Scott paper https://www.cs.rochester.edu/u/scott/papers/1996_PODC_queues.pdf
https://github.com/mratsim/weave/blob/v0.1.0/unused/channels/channels_legacy.nim
It is only slightly slower than lock-free even when a high number of allocations is demanded because the channels are recycled, only issue for long-running application is that the memory of recycled channels is never freed.
I'm not too sure how people want to organize stuff but I was thinking of just like we have a "SciNim" umbrella for science, publishing threading related library under a Weaver's Guild organization: https://github.com/weavers-guild
For now I plan to:
And then more speculative task consuming things: explore/research multithreaded async and/or latency optimized multithreading for IO needs (instead of throughput optimized like Weave).
It is only slightly slower than lock-free even when a high number of allocations is demanded because the channels are recycled, only issue for long-running application is that the memory of recycled channels is never freed.
Not sure precisely what you mean with allocations here, but malloc appears to be wrapped with a lock when threading is enabled. Not sure if that'd invalidate the methods performance. Back pressure would be handy!
Out of curiosity, does anyone know how Go-lang implements their channels?
That said, I did look into Go scheduler which was mostly written by Dmitry Vyukov who is well known for his very high performance channels in his personal website and that I used as reference to derive Weave's channels (the big difference is my channels support batching):
Side-note: Dmitry is also behind the Relacy Race Detector and LLVM ThreadSanitizer, another race detector. He is also behind Tensorflow multithreading scheduler (for Deep Learning), though Tensorflow doesn't use channels.