Hello Nim world,
I'm doing mostly IO related processing in various threads and I want them to be able to communicate in an asynchronous way.
An example scenario would be:
In the optimal scenario both threads would be asynchronous and just pass messages and along and process the corresponding (evented) IO. In between we also have to deal with disconnects, which should be no problem with exceptions.
Now the thing is - channel handling isn't asynchronous. I can do a blocking send and recv, which I don't want to.
I can simulate asynchronous with trySend and tryRecv, but this somehow doesn't feel right.
If I do it this way, I have the problem that sleepAsync can introduce a latency up to 500ms, which introduces too much response delay for the main threads' clients. But I have to do some sleeping, otherwise - if I understand this right - the IO events aren't processed. So I have to use a loop with trySend / tryRecv until there are messages to process with blocking send and recv, and in between doing cpuRelax to get the pending IO events processed and prevent the CPU from doing a total busy loop.
Are these assumptions correct? Am I missing something elementary here?
Thanks in advance,
SID
I'm currently writing a spec for asynchronous processing of blockchain events using Nim channels. This might be interesting for you: https://github.com/mratsim/blocksmith/blob/master/quarantine.md
Zoom is on the quarantine service which accepts blocks from the network and enqueue verifications in other workers, drop them if invalid or enqueue them in a DB if valid.
Some explanation
type
Quarantine* = ptr object
## Quarantine service
inNetworkBlocks: ptr Channel[QuarantinedBlock] # In from network and when calling "resolve" on the quarantineDB
inNetworkAttestations: ptr Channel[QuarantinedAttestation] # In from network and when calling "resolve" on the quarantineDB
quarantineDB: quarantineDB
slashingDetector: SlashingDetectionAndProtection
rewinder: Rewinder
outSlashableBlocks: ptr Channel[QuarantinedBlock]
outClearedBlocks: ptr Channel[ClearedBlock]
outClearedAttestations: ptr Channel[ClearedAttestation]
shutdown: Atomic[bool]
# Internal result channels
areBlocksCleared: seq[tuple[blck: QuarantinedBlock, chan: ptr Channel[bool], free: bool]]
areAttestationsCleared: seq[tuple[att: QuarantinedAttestation, chan: ptr Channel[bool], free: bool]]
logFile: string
logLevel: LogLevel
Other workers/services have the addresses of your input channels, they enqueue asynchronously tasks in them for you to process.
You run an event loop that dispatch the incoming tasks either to a worker or process them in the current context. When you dispatch to another worker, you need to send a result channel to receive results. Once you emptied your incoming queue, you loop on the result channels with tryRecv until either you received everything or only pending/incomplete tasks are left. And you restart.
Some platforms support nanosecond sleep. For IO operations asyncdispatch gives you abstractions to wait on those and properly sleep and be wakeup as soon as they are ready. You can do exponential backoff starting from 1ms up to 16ms for example to limit the latency loss.
More complex backoff strategies like log-log-iterated backoff are possible as well, see research on backoff strategies in Weave, https://github.com/mratsim/weave/blob/943d04ae/weave/cross_thread_com/event_notifiers_and_backoff.md
Note that Weave core building blocks are channels (I have 3 kinds https://github.com/mratsim/weave/tree/943d04ae/weave/cross_thread_com) and Future/Flowvar == Channels in Weave (though my own custom implementation), i.e. you can build an asynchronous application with just createThread and channels.
Yes, we have been after marrying the parallelism and concurrency primitives in Nim for a while now. You can see some efforts for this being made in https://github.com/nim-lang/Nim/pull/11724, but it didn't get to a point where it could be merged. The ideal is to be able to await a recv on a channel and a FlowVar, but right now you indeed need to busy loop waiting for the data which is enormously inefficient.
As far as your use case though, if it is a HTTP server then you may wish to consider following the model that httpbeast takes: an async event loop per thread. That way you can get Linux/BSD to send each thread requests evenly and keep all of them busy.