I've used nim to create a single threaded library. Now separate instances of that library need to be run on many cores. A coordinator will fans changes out to all of them. Ideally this all would be 0-copy. I see that Channel can be used to move memory from one thread to another but I cannot seem to figure out how I can:- allocate on coordinator thread
- fan out to all worker/reader threads
- when worker threads complete, de-allocate on coordinator
Note that the shared data is an object graph and may contain ref object types.
The main thing I tried was creating a ring buffer on the coordinator. Items stay in the buffer until all workers are done then the coordinator removes the item from the buffer. However, it seems that if any types in the graph rooted at item are of ref object then the worker threads that are reading will concurrently bump ref counts and thus break the ref counts.