I just noticed a burst of GitHub commits on "destructors", but I found nothing in the docs. Is there some change coming?
How Nim handles destructors (and GC and memory regions, and ...) could be an interesting selling point. C++ and Rust shows that people are interested in no-GC approaches (even D is bending a bit and will become more no-GC friendly) so why not Nim too?
Looking forward to the blog, and I hope whatever you come up with will not be wrong. Destructors have been a topic for a while. I'm also very interested in seeing where memory regions in Nim go, as per the "Future directions" part of the manual.
@mratsim, will this Twitch stream presentation make it to YouTube or something similar at some point?
It's not a presentation, it's Araq live coding the feature, probably destructors given the number of votes.
It's also him who'll do whatever he wants with the video so pick his mind :p.
Ok, blog post is here: https://nim-lang.org/araq/destructors.html
I will copy the "spec worthy" parts that have been implemented already into a wiki page.
Thanks @Araq! I'll watch the livestream later.
Is the doc on regions now obsolete? It would seem that destructors now obviate much of the need for regions.
Is the doc on regions now obsolete? It would seem that destructors now obviate much of the need for regions.
It's a bit early to say but I think so, yes.
Is all of that coming, or is it "just an idea" you want feedback on?
Destructors, assignment operators, the move optimization are coming behind a --newruntime switch and expected to be useful within days/weeks for you to tinker with, how to introduce even more moves ("sink parameters?") is unclear. The other outlined features have no ETA.
The really hard part is replacing the existing runtime with one with a different performance profile ("yay, deterministic freeing, yay more efficient multi threading possibilities, ugh, overall slower?!") and is likely stuff for Nim v2.
But yeah, feedback is always appreciated.
It's a bit early to say but I think so, yes.
I think so too, though there is the ability of regions (used to call them "arenas" a long time ago) to free many allocated objects at once, even if they aren't in the same container. I don't think destructors give you that. I also remember using region APIs that would have setMark and freeToMark and stuff like that, but I don't think those kind of APIs lasted, mostly just the /allocateFromeRegion/freeAllInRegion ones.
Good stuff, I hope to see it in Nim 1.0 really soon, like in a few weeks ;-).
@bpr Not in the core (or std, I guess, but's more realistic). But I never said I was talking about the core.
@rayman22201 I was referring to a GC library for Rust. ;) I can see @mratsim already mentioned one example (there are more, if my memory serves me well). There are two main reasons for it to be useful:
@Araq
I eventually want to have a simpler, more consistent language that does not provide every feature under the sun.
You never cease to amaze me. xD But I'm glad you came to this conclusion. ;)
I don't really have experience with GCs but Nim's GC seems nice. I'm quite surprised you'd like to replace it with refcounting with a cycle collector, which, as far as I know, isn't particularly elegant, smart or efficient. :-/
@Udiknedorm: The biggest problem with the current GC is that it doesn't support shared memory (though, I don't know why we can't do something like Go's channels).
To me, these changes seem like an effort to reasonably support programs that don't want to rely on the GC.
@Varriount It doesn't? :-o I was pretty sure there was something called shared heap... Well, whatever. I think GC is much better for functional languages, RAI seems better for stateful ones.
Oh, by the way --- I didn't really try it, but I guess idiomatic Nim programs could bring problmes on some microcontrollers. ;) I once wrote a custom language for a really simple microcontroller. I actually struggled to fit communication, parsing, bytecode, user code, VM with a GC. I'm not sure if I would succeed if not for how trivial the GC was. Most of the language was stack-based, I needed a GC to manage cleaning unreachable bytecode. And in general, I don't think it's a really great idea to do dynamic allocations on small microcontrollers.
Many microcontroller only have a stack so you just can't use heap ;).
Also @Varriount, @Udiknedormin today you can do:
type
SharedArray[T] = object
dataRef: ref[ptr T]
data*: ptr UncheckedArray[T]
len*: int
proc deallocSharedArray[T](dataRef: ref[ptr T]) =
if not dataRef[].isNil:
deallocShared(dataRef[])
dataRef[] = nil
proc newSharedArray[T](size: int): SharedArray[T] =
# Note the finalizer to call the GC when needed
new(result.dataRef, deallocSharedArray)
# Note the shared memory allocation
let address = cast[ByteAddress](allocShared0(sizeof(T) * size))
result.dataRef[] = cast[ptr T](address)
result.data = cast[ptr UncheckedArray[T]](address)
result.len = size
proc `[]`[T](a: SharedArray[T], idx: int): T =
assert idx >= 0 and idx < a.len # bounds-checking
a.data[idx]
proc `[]`[T](a: var SharedArray[T], idx: int): var T =
assert idx >= 0 and idx < a.len # bounds-checking
a.data[idx]
proc `[]=`[T](a: var SharedArray[T], idx: int, v: T) =
assert idx >= 0 and idx < a.len # bounds-checking
a.data[idx] = v
var a = newSharedArray[int](5)
a[3] = 10
echo a[0] # 0
echo a[3] # 10
@mtrasim I've already said it's a bad idea (even on many architectures on which it's possible). ;) And even if you don't really have a heap, it doesn't mean you couldn't use dynamic memory if a language supports custom allocators. You can provide a memory pool on a stack, that's what I actually did in the VM I mentioned.
allocShared, just like I said. :) But can I make sure about why you use both dataRef and data fields? data is a public pointer to the data so I guess you'd like to have raw access to the fixed location (what a constant pointer in C would do). But then, as far as I know, you can reassign data field to another location? I guess an inlined getter would be better. The same for len as you can definitely reassign it, breaking all the assertions (in general case you'll end up desynchronizing copies).
Also: a deep copy will make a copy of the ref object, i.e. a ptr. So the _actual data won't be copied and so you end up with two semantically different objects which point to the same data. Both can possibly try to deallocate it then. Not a really good idea, as not only this object can't be properly deepCopied but does it quite wrong. You'd need to provide a separate type (not just ptr T) instead so you could overload deepCopy. As you can see, it's not as trivial as your example.
Also: I don't think you'd like to reimplement seq and all the other standard containers for them to use shared heap? ;) Especially considering it being non-trivial, as I proved to you?
dataRef to have the deallocation of the data managed by Nim GC.
data for normal use through foo[10] (thanks to UncheckedArray). Yes data is public because in my use case this is only used internally, and getter/setter would just add boilerplate for no gain. Same thing for len.
Regarding copy, just think of it as an object with reference semantics, any ref object will have the same issue.
My example was just to show that you can have the GC manage shared memory if you manually allocate it and use a finalizer for deallocation.
Well, you said "today you can do" as a general advice so I assumed you try to show us "how simple it is to do it by hand", not "how easy it is to make sth only for internal usage".
About this object being a ref one --- no, please read my comment again. Long story short: ref object can be deepCopied, your example cannot.
Besides: no, that's not the same. Why? Because GC finalizes a pointer. So it will finalize it when it comes to the conclusion it should free the memory for a pointer. It cares not about how huge memory block on shared heap it corresponds to nor think whenever shared heap should be cleaned at all. So you could have 90% of the shared heap filled with unused arrays and GC saying "Why free anythin? I've still got plenty of space on my thread's heap!" because pointers themselves are very small. That's the same I've told you about CUDA memory. Simply put: you don't have a GC-ed shared heap. You have an illusion of GC-ed shared heap.
Right it seems like there are a bunch of getOccupiedMem in the gc file so that it can get the memory managed, including a collection threshold.
In my case the shared mem array is only used to hold temporaries for computation (main long-lived data is held in a seq) so I can trigger a manual GC_collect after computation is done, but for non-temporary usage I could get out-of-mem even though enough space can be collected.
@mratsim Well, if you call GC_collect manually then it's much worse than manual allocation and deallocation from memory pool, I guess.
Could you explain further what getOccupiedMem changes in the case we were talking about (ref to ptr to memory external to thread heap)?