Mirror of forum.nim-lang.org

10723 :: GAsyncQueue vs system/channels vs. threading/channels

• Isofruit (orginal) [2023-12-03T00:44:01+01:00] view original

  Heyho everybody.

  This Thread sent me down somewhat of a rabbit hole that I'm still trying to climb out of about Multithreading within an application. Specifically, setting a GTK-application up with a server-client architecture in cases where you have a rather heavy program or other kinds of heavier computation that you don't want to block the GTK main thread.

  There are mechanisms to push one-off tasks into separate threads, but for tasks that require the thread to stay alive the entire time and being pinged with commands occasionally that mechanism isn't suitable.

  system/channel

  So I did as suggested and implemented something like that for owlkettle using nim's system/channels (see my later posts in the prior thread or this SO question ). From what I understand, nim's system/channels take care of memory for you and deep-copy the message from Thread A to Thread B, thus guaranteeing memory safety.

  GAsyncQueue

  Chatting a bit on the GTK matrix server about it lead me down the rabbit hole of their thread-safe mechanism for inter-thread communication: GAsyncQueue. That one appears to work based on everything being a pointer and moving ownership of pointer1 from Thread A to Thread B. I haven't implemented this in nim yet, but that seems a bit more difficult to do as I'd need to convert a given ref-type into a pointer, disable destructors for said ref-data to avoid use-after-frees, only to cast them from pointers to "propper" types in Thread B and free them manually after the fact.

  threading/channel

  Chatting a bit more on our discord then lead me down the rabbit hole of system/channel's possible (drop-in ?) replacement: threading/channel. That one... actually seems like basically system/channel but while trying to do less copying when possible and more moving ownership around (?). The module warns about being experimental, which makes me slightly hesitant to propose adding it to a library that more than just a few others may depend on.

  Question

  Given that I am not that well versed in multi-threaded programming and in many respects this is my first time doing such a deep-dive into things:

  Does anyone have any thoughts on what might be the most sensible to use?

  Any aspects I am overlooking or where I have the wrong idea?

  With my current knowledge, this is my understanding:

  • Generally, GAsyncQueue has the benefit of zero-copying and a stable API. But the drawback of likely being a mild pain to properly implement to play nice with ARC/ORC. I'd imagine particularly ensuring messages don't get free'd on thread 1 and do get free'd on thread 2 would be meh.
  • The system/channel approach has the benefit that it currently exists and basically just works flawlessly. But the drawback that it does a lot of copying (which I don't really think has that much of an impact unless a ton of messaging is happening).
  • The threading/channel implementation has the benefit (I think?) of being a drop-in replacement for system/channel while simultaneously opening up the option to also transferring pointer ownership instead of deep-copying them. Which kinda seems like the best of both worlds, but with the drawback that it seems like a risky decision in this scenario.
  • I'm aware Malebolgia exists but my brain is mildly toast after today and I think it doesn't focus on having mechanisms for inter-thread communication (?)

---

• Araq (orginal) [2023-12-03T07:53:33+01:00] view original

  I would use threading/channel.

---