I've read pretty much everything on the subject but still keeping moving things around to make it work without really knowing what I'm doing, or why it's not working in the first place...
Here's the code:
var server = newAsyncHttpServer()
proc handler(req: Request) {.async,gcsafe.} =
if verbose:
stdout.write fgMagenta & "<< [" & req.protocol[0] & "] " & req.hostname & ": " & fgWhite & ($(req.reqMethod)).replace("Http").toUpperAscii() & " " & req.url.path
if req.url.query!="":
stdout.write "?" & req.url.query
stdout.write "\n"
stdout.flushFile()
var status = 200
var headers = newHttpHeaders()
var body: string
var routeFound = ""
for k in routes.d.keys:
let route = req.url.path.match(nre.re(k & "$"))
if not route.isNone:
var args: ValueArray = @[]
let captures = route.get.captures.toTable
for group,capture in captures:
args.add(newString(group))
if req.reqMethod==HttpPost:
for d in decodeData(req.body):
args.add(newString(d[0]))
for d in (toSeq(decodeData(req.body))).reversed:
stack.push(newString(d[1]))
for capture in (toSeq(pairs(captures))).reversed:
stack.push(newString(capture[1]))
try:
discard execBlock(routes.d[k], execInParent=true, args=args)
except:
let e = getCurrentException()
echo "Something went wrong." & e.msg
body = stack.pop().s
routeFound = k
break
if routeFound=="":
let subpath = joinPath(env.currentPath(),req.url.path)
if fileExists(subpath):
body = readFile(subpath)
else:
status = 404
body = "page not found!"
if verbose:
echo fgGreen & ">> [" & $(status) & "] " & routeFound & fgWhite
await req.respond(status.HttpCode, body, headers)
try:
if verbose:
echo ":: Starting server on port " & $(port) & "...\n"
waitFor server.serve(port = port.Port, callback = handler, address = "")
except:
let e = getCurrentException()
echo "Something went wrong." & e.msg
server.close()
And here's what I'm getting:
..... template/generic instantiation of `async` from here
..... /.choosenim/toolchains/nim-1.4.2/lib/pure/asyncmacro.nim(218, 31) Error: 'handlerIter' is not GC-safe as it calls 'execBlock'
Here are my compilations flags (I've tried without --threads:on too - no idea what effect is, really...)
--warning[UnusedImport]:off --colors:off -d:release -d:danger --panics:off --gc:orc --checks:off --overflowChecks:on -d:ssl --threads:on --passC:-O3 --cincludes:extras --nimcache:.cache --embedsrc:on --path:src --opt:speed
I've also played with {.threadvar.}, but that simply seems too random as well...
Any concrete ideas?
The code snippet you provided doesn't compile, and I'm not really sure what your expecting this to do. Why are you trying to use threads? Asynchronous code runs inside of an event loop in a single thread.
If you get your code sample compiling, and let me know what you expect the sample code to do, I'd be glad to try and help further.
The main issue is that I'm not 100% what each of all these things does. I've added and removed {.gcsafe.}, but still not sure what it enforces.
@carterza Your example does work. I have no problem creating a simply server. The problem is when the implementation is not so simple anymore and I dare to call or access anything outside the handler.
@enthus1ast I have played between arc and orc several times. I don't think it makes any difference. Re: {.gcsafe.} for execBlock, I've tried that too... but then it just complains for different parts of the code, called or accessed by execBlock...
P.S. In the end, I managed to get it to compile by setting --threadAnalysis:off, but still I cannot find what this does for sure. I guess I'll have to experiment...
Okay but it's impossible to know what your code is doing and why you are doing the things in your code that you're doing, so the simple web server is a naïve assumption about what you're trying to achieve?
What does execBlock do? Why are you using threads or trying to make this code gcsafe in the first place?
You've supplied an incomplete code snippet which doesn't compile, so it's a guessing game until you provide us with more information.
The code is just too complicated and long - that's why I can give too much information in a couple of lines or create a minimal example.
Here's the asyncHttpServer in use: https://github.com/arturo-lang/arturo/blob/master/src/library/Net.nim#L82
And here's the implementation of execBlock: https://github.com/arturo-lang/arturo/blob/master/src/vm/exec.nim#L107
Thank you for the links, they definitely help provide context.
I searched for the use of thread throughout your codebase and I don't see any, so you're not using threads. You don't need to compile with --threads:on unless you plan on using threads. You also shouldn't need any flags regarding thread analysis.
Maybe I'm missing something or not understanding why these need to be gcsafe but to my knowledge, you only need to worry about this when using threads.
I guess you're right. (With so many changes since the beginning, I guess - and hope - it's just a leftover that's not needed anymore...)
Let me try eliminate --threads:on, etc and I'll let you know!
Thanks! :)
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/library/Net.nim(82, 9) template/generic instantiation of `builtin` from here
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/library/Net.nim(117, 42) template/generic instantiation of `async` from here
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/vm/exec.nim(479, 34) Warning: 'doExec' is not GC-safe as it performs an indirect call here [GcUnsafe2]
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/library/Net.nim(82, 9) template/generic instantiation of `builtin` from here
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/library/Net.nim(117, 42) template/generic instantiation of `async` from here
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/vm/exec.nim(107, 6) Warning: 'execBlock' is not GC-safe as it calls 'doExec' [GcUnsafe2]
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/library/Net.nim(82, 9) template/generic instantiation of `builtin` from here
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/library/Net.nim(117, 42) template/generic instantiation of `async` from here
/Users/drkameleon/.choosenim/toolchains/nim-1.4.2/lib/pure/asyncmacro.nim(218, 31) Warning: 'handlerIter' is not GC-safe as it calls 'execBlock' [GcUnsafe2]
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/library/Net.nim(82, 9) template/generic instantiation of `builtin` from here
/Users/drkameleon/Documents/Code/OpenSource/arturo-lang/arturo/src/library/Net.nim(186, 67) Error: handler is not GC safe
In a few words, a complete mess. I have no idea what is going on and why, and that's never a good sign lol
Maybe I'm missing something or not understanding why these need to be gcsafe but to my knowledge, you only need to worry about this when using threads.
And no, I have no idea why the have to be "GC safe" either... If the whole thing is just an out-of-place warning, that has no effect whatsoever, then I'd just silence it.
Indeed, you only need to care about gc safety if you've got threads enabled. You may still get warnings/errors about it even if you haven't turned on threads, just because the Nim compiler wants you to make sure your library/app works when compiled with --threads:on.
The --threadAnalysis:off flag is the escape hatch here. Nowadays as far as I know (I don't know if this was a proposal, or has been implemented already) you can also wrap the call that Nim is complaining about with {.gcsafe.}::
{.gcsafe.}:
execBlock(...)
GC Safety itself concerns itself with global variables, because the default GC in Nim cannot handle shared memory, the compiler needs to make sure you're not accessing data that's been allocated by thread #1 from thread #2. This isn't a problem with the new orc memory management scheme, so you can try using it, but it's quite new so you may run into bugs (if I recall correctly orc still doesn't disable this old thread analysis so it will incorrectly warn you).
You can also do the correct thing and mark all global vars with the {.threadvar.} pragma, which just means "this variable needs to be initialised in each thread". Without threads enabled you can just treat it as a normal global var. With threads on you'll need to make sure it is initialised properly in each thread (simple way to do this is via an accessor proc that will initialise it implicitly)
I hope that helps a little bit, it is sort of a stream of what popped into my mind and not very well organised. I personally either disable thread analysis or use {.threadvar.}.
Very very helpful and thorough explanation!
Thanks a lot! :)
but then it just complains for different parts of the code, called or accessed by execBlock...
yes this is the reason why i would enable {.gcsafe.} for this proc. Then you can further analyze what is touching a global variable. Then either fix it (what @dom96 written), or annotate this too, to get to the root cause.