Araq's recent post "A cost model for Nim" states that "Memory can be shared effectively between threads without copying in Nim version 2".
This is related to a problem I have been struggling with for a long time.
Basically, the problem is this:
My question is: how can this be done in Nim (or Nim 2), without violating the underlying C memory model?
I can proved more technical details about approaches I have tried and what the problems are I run into, but maybe there just is a very simple and obvious solution I do not know about.
Untested but what about:
type
X = ref object
a: string
var g: X
proc foreign {.thread.} =
var local = X(a: "abc")
g = move local
spawn foreign()
# g now has an X
Is that safe? How can we tell? What are the rules with multithreading and ARC? For us who didn't implement it we're just left guessing about what we can and can't do here.. This example "works", but I have no idea if it actually works (added in some echos to make sure the string is actually moved and not copied):
import threadpool
type
X = ref object
a: string
var g: X
proc foreign {.thread.} =
var local = X(a: "abc")
echo cast[int](local.a.addr)
{.cast(gcsafe).}:
assert g == nil
g = move local
spawn foreign()
sync()
# g now has an X
echo cast[int](g.a.addr)
echo g.repr
When I'm just curious about the implementation details, I read the C code. And as I can tell, it's safe after moving, as local is then set to nil (, which is the side effect of moving.) But the operation of moving can be unsafe, as it's unknown if move is an atomic operation. You may need a Lock to prevent the main thread from accessing g during the move.
var g: X
proc foreign {.thread.} =
var local = X(a: "abc")
{.cast(gcsafe).}:
g = move local
# local is now `nil`
# local is about to be destoryed, but it's now `nil`. No operation.
# g destroyed.
Is there a way to make sure (by inspection or enforced by the compiler) that the moved ref RC == 1, other then wrapping the ref in an isolated[]? When the ref counter of the moved object > 1, both threads will have a reference to the same object, leading to disaster.
isolated[] is not practical to work with, as the object needs to be wrapped in an isolated[] at construction time, which makes it cumbersome to work with the encapsulated value.
Here is my minimal example: https://play.nim-lang.org/#ix=4g1k
(This does not actually run in the playground because it needs devel)
This example creates two threads, a ref object is created in one thread and sent to the other over a channel. (For now, ignore the fact there is no locking or memory barrier in place protecting the object itself).
This example works fine. The output looks like this:
send a thing, @t=140031723291264 @t.val=140031728230496 t=42
recv a thing, @t=140031725388432 @t.val=140031728230496 t=42
This shows that the ref object itself is copied by the channel, as the sender and receiver have their t on a different address. The object the ref points to is properly moved though, as it has the same address (@t.val)
This works because the created object t in txProc() has only one reference, so channel.send() will properly move the object because RC == 1 and send explicitly sinks its argument.
Uncommenting line 13 will cause problems though: there will be more then one reference to the same object, so after the send(), both threads will have a reference to the same object. Because the RC itself is not locked or atomic, this will cause UB or worse.
So you can use unsafeIsolate to do what we want?
No; unsafeIsolate will happily do what you ask it, but it will not check or enforce the actual RC count. It creates an isolated[T] for you which might or might not be isolated. I do not really see any use cases in which this would be useful.
Clyybber implemented something slightly better some time ago to only isolate the ref only if its RC == 1, or throw an exception otherwise. This is slightly better, but there is still the problem that the object might have other references which are not isolated.
I do not really see any use cases in which this would be useful.
There are plenty of cases where the human being can see/verify/test that the subgraph is isolated but the compiler cannot. Every tool that can detect races at runtime helps you here too.
We can try to ensure this at compile-time but there is no experience yet that suggests this would be worth while.
I wrote some simple and naive code to check if a tree is isolated at runtime, see https://github.com/zevv/nimsafesend.
isIsolated[]() recursively traverses the passed (ref) objects and finds out if the passed ref is isolated, and thus safe to move to a different thread.
In main.nim I create a simple isolated tree of only 2 objects and send this over between two threads. This works fine for arc, but unfortunately orc does not agree with this and throws me a segfault. Valgrind tells me:
==126265== Invalid read of size 16
==126265== at 0x10F3CB: unregisterCycle__system_2993 (orc.nim:147)
==126265== by 0x10F3CB: rememberCycle__system_3352 (orc.nim:469)
==126265== by 0x1134BB: nimDecRefIsLastCyclicStatic (orc.nim:497)
==126265== by 0x1134BB: eqdestroy___main_219 (main.nim:16)
==126265== by 0x1134BB: eqdestroy___main_206 (main.nim:16)
==126265== by 0x1134BB: eqdestroy___main_206 (main.nim:16)
==126265== by 0x1136E7: rxProc__main_23 (main.nim:16)
==126265== by 0x111CE6: threadProcWrapDispatch__stdZthreads_105 (threadimpl.nim:71)
==126265== by 0x10A6C0: threadProcWrapper__stdZthreads_81 (threadimpl.nim:106)
==126265== by 0x4906849: start_thread (pthread_create.c:442)
==126265== by 0x498952F: clone (clone.S:100)
==126265== Address 0xfffffffffffffff0 is not stack'd, malloc'd or (recently) free'd
Should I re-root all refs of my tree after moving them?
Exactly but there is no API for this yet. You can try GC_runOrc. :-)