Hello everybody ,
Just started to try some nim. So it's my second day...
Besides of some hussle with linking native libraries the experience is amazing.
I have a rest api with mummy, which should persist to leveldb. For the keys I use ulid with some prefix…
When I write it like this
proc writeToDB(prefix, msg: string) =
var uid = fmt“{prefix}-{ulid()}"
withLock(lmdb_lock):
db.put(uid,msg)
I get an error after some requests
SIGSEGV: Illegal storage access. (Attempt to read from nil?)
or it hangs(unable to quit with CTRL-C)
on the other hand if I put the uid inside the lock it works like a charm with 100000 Requests
What I should mention is that the ulid is seems to be considered not gcsafe because there is a global variable used
var rng = initSystemRandom()
so my call inside the multithreaded handler is
{.gcsafe.}:
writeToDB(x,z)
Hint: gc: orc; threads: on; opt: none (DEBUG BUILD, -d:release generates faster code)
Can somebody help me to understand what happens here? I guess I have to rewrite the ulid, maybe an object that holds the random generator would be better…
But why does the lock matter?
As I can remember, accessing to a global variable from one of the threads, is considered unsafe (non-multithreaded safe) because another thread can modify it at the same time.
By enforcing every thread to hold something (the "lock," something that can only be held by excatly one thread at the same time) when accessing the global, we ensure that only one per thread does it (so that they don't interfere each other.)
In your example, the act of calling ulid() is consider unsafe, so threads should hold something when doing that.
Thank you very much... Now i am feeling a bit dump... I have the lock for the db but the compiler surely does not know that :) It protects the ulid global as well...
But does this mean that this global inside the ulid package gets allocated everytime i call the ulid function and deallocated when the thread ends? Or when the package goes out of scope from the view of the thread?
I guess i am a bit misseducated from go where globals get initialzied on startup and will never get freed.
Whether it's pushed onto the stack or allocated on the heap has nothing to do with the fact that you cannot access the variable at the same time with two threads, unless they both don't modify it, or there'll be an error since it could be accessed while not fullly changed. If the DB lock also protects ulid, you should call it as
withLock(lmdb_lock):
{.gcsafe.}:
uid = fmt“{prefix}-{ulid()}"
db.put(uid,msg)
Thanks again for the explanation.
I am not entirely convinced, because i assume that there are no writes to that global...
But there is an other library used inside of that ulid.... so i have to dig deeper...
And in the end i am sure that you are right!
in the meanwhile i have found that if i use the std/oids and pass the db to the function it works without locks and gcsafe blocks :)
proc writeToDB(prefix, msg: string,db:LevelDb) =
var uid = fmt"{prefix}-{$genOid()}"
db.put(uid,msg)
and
proc createHandler(script:string,db:LevelDb):proc(request: Request){.gcsafe.}=
return proc(request: Request) =
...
writeToDB(x,z,db)
...
Thanks again!