Hello,
Since I upgraded my code to 1.4.0 and updates some dependencies, I have issues with my code. I have a warning that it is not gc-safe. Investigating (by adding the gcsafe annotation), I found out that a library I'm using is not gc-sage. Here is the code:
https://github.com/ba0f3/scram.nim/blob/master/scram/client.nim#L36
let
SERVER_FIRST_MESSAGE = peg"'r='{[^,]*}',s='{[^,]*}',i='{\d+}$"
SERVER_FINAL_MESSAGE = peg"'v='{[^,]*}$"
proc prepareFirstMessage*(s: ScramClient, username: string): string {.raises: [ScramError]} =
if username.len == 0:
raise newException(ScramError, "username cannot be nil or empty")
var username = username.replace("=", "=3D").replace(",", "=2C")
s.clientFirstMessageBare = "n="
s.clientFirstMessageBare.add(username)
s.clientFirstMessageBare.add(",r=")
s.clientFirstMessageBare.add(s.clientNonce)
s.state = FIRST_PREPARED
GS2_HEADER & s.clientFirstMessageBare
The error message:
~/.nimble/pkgs/scram-0.1.11/scram/client.nim(46, 6) Warning: 'prepareFinalMessage' is not GC-safe as it accesses 'SERVER_FIRST_MESSAGE' which is a global using GC'ed memory [GcUnsafe2]
I transformed the global let by a const, which I suppose would help, but it's still not GC-Safe. What would be the recommended solution? This pattern is often used, compute a constant globally and use it in a procedure.
Replying to myself. gcsafe is needed because otherwise a global variable could be collected by multiple threads.
Is there a way to mark an object as immutable and impossible to collect so it could be shared between threads ?
Looks like the code is missing a closing . for the pragma on line 36.
proc prepareFirstMessage*(s: ScramClient, username: string): string {.raises: [ScramError].} =
vs
proc prepareFirstMessage*(s: ScramClient, username: string): string {.raises: [ScramError]} =
When cloning the repo in question, changing the let to const, and building with nim version 1.4.0 and 1.5.1 (devel), I get the following. No errors.
test 1 passed test 2
The linked code solves GC safety when compiled with threads. The problem is that my procedure requires gcsafe even when not compiled with threads.
The code:
when compileOption("threads"):
var
SERVER_FIRST_MESSAGE_VAL: ptr Peg
SERVER_FINAL_MESSAGE_VAL: ptr Peg
template SERVER_FIRST_MESSAGE: Peg =
if SERVER_FIRST_MESSAGE_VAL.isNil:
SERVER_FIRST_MESSAGE_VAL = cast[ptr Peg](allocShared0(sizeof(Peg)))
SERVER_FIRST_MESSAGE_VAL[] = peg"'r='{[^,]*}',s='{[^,]*}',i='{\d+}$"
SERVER_FIRST_MESSAGE_VAL[]
template SERVER_FINAL_MESSAGE: Peg =
if SERVER_FINAL_MESSAGE_VAL.isNil:
SERVER_FINAL_MESSAGE_VAL = cast[ptr Peg](allocShared0(sizeof(Peg)))
SERVER_FINAL_MESSAGE_VAL[] = peg"'v='{[^,]*}$"
SERVER_FINAL_MESSAGE_VAL[]
else:
let
SERVER_FIRST_MESSAGE = peg"'r='{[^,]*}',s='{[^,]*}',i='{\d+}$"
SERVER_FINAL_MESSAGE = peg"'v='{[^,]*}$"
The code is handling GC safety well by using a thread local variable when compiled with threads. Why is gcsafe warning emitted when not compiled with threads ?
Ah. OK.
This may help https://github.com/UNIcodeX/PrologueTemplate/blob/872e6740305f96b9838149d9fc6efff1e02287cf/src/main.nim#L5-L7
Also lines 21-23 and line 26
My understanding is that strings are sequences of bytes, allocated on the heap, and therefore are cleaned up by GC (?more frequently?).
You could also do a deepCopy :
import prologue
import nwt
## To just set up threadvar and initialize per thread.
var
globalTemplates = newNwt("templates/*.html")
templates {.threadvar.} : Nwt
let settings = newSettings(
address="0.0.0.0",
debug=false,
port=Port(8000),
)
var app = newApp(
settings=settings
)
proc initTemplates() =
{.cast(gcsafe).}:
if templates.isNil:
deepCopy(templates, globalTemplates)
proc index(ctx: Context) {.async, gcsafe.} =
initTemplates()
resp templates.renderTemplate("index.html")
app.addRoute("/", index)
app.run()
BUT... doing that requires another compiler flag => --tlsEmulation:off
I see two main things there:
I've been reading through this as an utter newbie to the language (picked it up literally yesterday), trying to extract the "intended way" in which one is to set global read-only constants (some of which are to be set on application startup), but I don't think I quite got this. What I'm talking of is immutable stuff, like the path to an sqlite database file, or the secretKey on a webserver (my usecase is a web application using prologue, but similar files full of constants could apply to anywhere else as well).
From what I gather so far, the way I should be doing "constants"-files from which I can import in a gc-safe way is using " {.cast(gcsafe).}:"
#my_configs.nim
let my_constant* : string = read_in_config_file_and_extract_my_constant("path/to/settings/file")
#my_func.nim
import my_configs
proc do_something(someVar: string): string =
{.cast(gcsafe).}:
let my_constant_copy = deepCopy(my_constant)
result = do_a_thing(someVar, my_constant_copy)
Is this the way one is intended to do these things to get thread-safe constants? (I guess an alternative would be to have getter procs in my_config.nim that return a deepCopy of the corresponding variable).