Hi I'm profiling my application and want to squeeze out as much performance as possible.
If I know most of the values created in my application will be of some type, is it possible to create a custom allocator that uses a reusable pool, when a value's refcount becomes 0, we put it back to the pool instead of letting GC kick in in order to reduce memory allocation. My goal is to see whether this can improve performance if memory allocation and GC are big part of execution time.
Please advise whether this is something worth trying, or if there are better ways for this. Thank you in advance!
Pseudo code
type
Value = ref object
var pool: seq[Value] = @[]
proc get(): Value =
if pool.len > 0:
return pool.pop()
else:
return Value()
proc free(v: Value) =
if v.refcount == 1: # refcount is not currently accessible
pool.add(v)
proc test() =
var v = get()
free(v)
proc test2() =
var v2 = get()
free(v2)
# There will be only one Value object created in below calls
test()
echo "pool size: ", pool.len
test2()
echo "pool size: ", pool.len
test()
echo "pool size: ", pool.len
test2()
echo "pool size: ", pool.len
Well ok but if you think that a refcount makes your program faster then that's what --gc:arc does.
Anyhow, there are plenty of ways to use a custom allocator, most start by turning your Value = ref object into Value = ptr object.
Actual code
This is a very simple bounded object pool you can draw inspiration from https://github.com/mratsim/weave/blob/v0.4.0/weave/memory/persistacks.nim#L14-L124. It does not support multiple owners and so no refcounting scheme is needed.
This allows multiple owners via refcounting: https://github.com/mratsim/weave/blob/2e528bd2a6a04306fd029f04a0af87fa201e5d89/weave/cross_thread_com/flow_events.nim#L176-L201
This allows dynamic allocations of pools (though most of the code is related to dealing with thread-safety): https://github.com/mratsim/weave/blob/ce8cac849b1c21b92a8b41cf52dc51c8b959a053/weave/memory/memory_pools.nim#L232-L259
@mratsim2d Between the below two approaches, are there noticable difference in performance when accessing the fields, e.g. event.union.single.field vs event.field? I don't know a lot about how compilers work. I guess they may compile both expressions to similar read/write operations. Am I correct?
type
EventKind = enum
Single
Iteration
EventUnion {.union.} = object
single: SingleEvent
iter: EventIter
EventPtr = ptr object
kind: EventKind
union: EventUnion
type
EventKind = enum
Single
Iteration
EventPtr = ptr object
kind: EventKind
case kind:
of Single:
# SingleEvent's fields go here
of Iteration:
# EventIter's fields go here
There will be absolutely no difference in terms of low-level representation or access.
However will add checks in the second case to ensure that when accessing fields you are in the correct kind. Those checks will likely be lifted or done once per context once we have the DrNim proof engine (https://nim-lang.org/docs/drnim.html)
In the first case you have to do the work yourself or you'll have bugs that may be hard to track down (i.e. similar to casting bugs).
I recommend using the first version as even if the checks are not elided, the hardware branch prediction will likely make them free if done repeatedly.
The reason I used the second one in my code is because I had to. Nim unions require some copy on initialization but when compiling to C++, Nim uses C++ atomics and they cannot be copied:
So unless you also need atomic refcounting, I suggest you use the Nim variants instead building your own via the {.union.} pragma like I did.
The issue is I can't assign "kind", got error: "Error: unhandled exception: assignment to discriminant changes object branch;".
result = cast[Value](alloc(sizeof(Value)))
result.kind = kind
first off, careful there, you allocated sizeof(pointer) when you meant to allocate enough for a GeneValue[]
Setting the discriminant will require unsafe hacks. If you're sure you won't use GC:arc you can use typeinfo to do it for you but it requires lots of allocations and a string comparison so it's not exactly an optimization But here it is anyway:
import typeinfo
type
GeneKind = enum GeneNilKind,GeneIntKind
GeneValue* {.acyclic.} = object
case kind*: GeneKind
of GeneNilKind:
discard
of GeneIntKind:
intVal:int
Hack = ptr object
kind:GeneKind
let x = cast[ptr GeneValue](alloc(sizeof(GeneValue)))
echo x[]
var kind = GeneIntKind
x[].toAny["kind"]=kind.toAny
echo x[]
let x = cast[ptr GeneValue](alloc(sizeof(GeneValue)))
echo x[]
#var kind = GeneIntKind
#x[].toAny["kind"]=kind.toAny
cast[Hack](x).kind=GeneIntKind
echo x[]
To change an object variant's kind, produce a new object variant. Like so:
obj = Obj(k: newKind, v: obj.v, x: obj.x)