Sure, serialization/deserialization takes some time, but disk write/read is VERYYYY SLOW.
Redis is an in memory no sql key/value database, I think it really fits your needs very well.
The previous answers do not answer my question. How to share a table between processes? With python I can share a dictionary between processes using a Manager. Is there any way to do the same with Nim?
#!/usr/bin/env python
import multiprocessing
manager = multiprocessing.Manager()
shared_dict = manager.dict()
def worker1(d):
d["a"] = 1
def worker2(d):
d["b"] = 2
process1 = multiprocessing.Process(
target = worker1, args=[shared_dict])
process2 = multiprocessing.Process(
target=worker2, args=[shared_dict])
process1.start()
process2.start()
process1.join()
process2.join()
print(shared_dict)
# output {'a': 1, 'b': 2}
You can also store binary blobs in redis, so if you really wanna avoid serialization, there is no need for serialization.
However, maybe a memory mapped file could work from multiple processes (never have done this though.). Also, the implementation of the Python manager module could be a good read to get an idea how they do it.
The Nim equivalent is sharedtable: https://nim-lang.org/docs/sharedtables.html
Just make sure you use non-GC backed data values, i.e., avoid refs, strings, seqs.
@mrhdias as @enthus1ast suggested, look at the python implementation - you can do the same thing with Nim.
I'm not sure Nim has any built-in mechanism for doing what you've described. Sharedtable is not what you're looking for.
I did an experiment with the asynchttpserver and instead of pre-fork several processes I try the threads and it apparently works. Is this the right way to do this?
# nim c -r --threads:on test.nim
import asynchttpserver, asyncdispatch
import strutils
const port = 8080
proc launchServer(c: int) {.async.} =
var server = newAsyncHttpServer(reuseAddr = true, reusePort = true)
proc cb(req: Request) {.async.} =
echo (req.reqMethod, req.url, req.headers)
let headers = {"Content-type": "text/plain; charset=utf-8"}
await req.respond(Http200, "Hello World: $1" % $c, headers.newHttpHeaders())
echo "test this with: curl localhost:" & $port & "/"
server.listen(Port(port))
while true:
if server.shouldAcceptRequest():
await server.acceptRequest(cb)
else:
await sleepAsync(500)
proc threadFunc(c: int) {.thread.} =
echo "thread: ", c
waitFor launchServer(c)
proc main() =
var thr: array[0..4, Thread[int]]
for i in 0 .. high(thr):
createThread(thr[i], threadFunc, i)
joinThreads(thr)
main()
here's writing to a memfile, for what its worth:
import memfiles, os
let fn = "/tmp/test.mmap"
if fileExists(fn): removeFile(fn)
var mm = open(fn, mode = fmWrite, newFileSize = 20)
let data = "Hello".cstring
copyMem(mm.mem, cast[pointer](data), data.len+1)
#mm.flush()
mm.close()
var mm_full = memfiles.open(fn, mode = fmRead)
echo "size:", mm_full.size
echo cast[cstring](mm_full.mem)
mm_full.close()
A "daemon" is a process to most people with many traits. None of the traits are ever "cannot be multi-threaded" in my experience. So, if you first become a demon (with your own code or those packages) and then do your network stuff you are "a demon".
Maybe as @shirleyquirk mentioned, the one trait you want is "background/detached". It's not easy to guess what you want. So, you should use more precise language for better help.