This is using Nim 0.17.2 on Linux.
The code below works fine when run serially (as shown). When I try to run it in parallel (uncomment the 3 lines, and comment the other) the compiler throws and error about the garbage collector. I specifically compile it without using garbage collection - --gc:none to let me (hopefully) do want I want to do.
I have a nextp[r_row x c_column] array and I want to load nextp on a column by column basis, as the serial code does, but the compiler won't let me do it in parallel.
How do I get the compiler to follow orders and do what I want it to do?
All memory is thread safe, and each thread only works on memory solely associated with any individual prime column. Please don't tell my I need to transpose the array so the columns becomes rows, because the compiler is only able to work that way (another section of code does parallelize row functions).
Here's the code.
proc column_load(prime, j, k, r: int) =
for ri in residues: # for each prime|residue pair
let prod = r * ri # compute res cross-product
let row = posn[prod mod modpg] * pcnt # compute restrack address
# compute|store resgroup val of 1st prime mult for prime|residue pair
nextp[row + j] = uint(k*(prime + ri) + (prod-2) div modpg)
# Initialize the [rescnt x pcnt] 'nextp' table with resgroup values of the
# 1st prime multiples for each prime r1..sqrt(N) for each PG residue track.
proc nextp_init() =
# load 'nextp' with 1st prime multiples regroups vals on each residue track
#parallel:
for j, prime in primes: # for each prime r1..sqrt(N)
let k = (prime-2) div modpg # find the resgroup it's in
let r = (prime-2) mod modpg + 2 # and its residue value
#spawn column_load(prime, j, k, r)
column_load(prime, j, k, r)
#sync()
Here's the compiler directive:
$ nim c --cc:gcc --d:release --threads:on --gc:none ssozp11x1e5bparnew64.nimHere's the compiler output.
➜ nim nim c --cc:gcc --d:release --threads:on --gc:none ssozp11x1e5bparnew64.nim
Hint: used config file '/home/jzakiya/nim-0.17.2/config/nim.cfg' [Conf]
Hint: system [Processing]
Hint: ssozp11x1e5bparnew64 [Processing]
Hint: math [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: algorithm [Processing]
Hint: typetraits [Processing]
Hint: times [Processing]
Hint: os [Processing]
Hint: posix [Processing]
Hint: ospaths [Processing]
Hint: osproc [Processing]
Hint: strtabs [Processing]
Hint: hashes [Processing]
Hint: streams [Processing]
Hint: cpuinfo [Processing]
Hint: linux [Processing]
Hint: threadpool [Processing]
Hint: cpuload [Processing]
Hint: locks [Processing]
generating parameters for P11
lib/system.nim(709, 9) Warning: 'newSeq(result, len)' uses GC'ed memory [GcMem]
lib/system.nim(709, 9) Warning: 'newSeq(result, len)' uses GC'ed memory [GcMem]
lib/system.nim(2702, 6) Warning: 'new(e94742)' uses GC'ed memory [GcMem]
lib/pure/strutils.nim(945, 65) Warning: '&("invalid unsigned integer: ", s)' uses GC'ed memory [GcMem]
lib/system/sysio.nim(138, 16) Warning: 'setLen(string(line), chckRange(sp, 0, 9223372036854775807))' uses GC'ed memory [GcMem]
lib/system/sysio.nim(150, 20) Warning: 'setLen(string(line), chckRange(last - 1, 0, 9223372036854775807))' uses GC'ed memory [GcMem]
lib/system/sysio.nim(158, 18) Warning: 'setLen(string(line), chckRange(last, 0, 9223372036854775807))' uses GC'ed memory [GcMem]
lib/system/sysio.nim(166, 16) Warning: 'setLen(string(line), chckRange(pos + sp, 0, 9223372036854775807))' uses GC'ed memory [GcMem]
lib/system.nim(2722, 10) Warning: 'new(e)' uses GC'ed memory [GcMem]
lib/system.nim(709, 9) Warning: 'newSeq(result, len)' uses GC'ed memory [GcMem]
lib/system/sysstr.nim(275, 9) Warning: 'setLen(result, chckRange(base + 32, 0, 9223372036854775807))' uses GC'ed memory [GcMem]
lib/system/sysstr.nim(287, 9) Warning: 'setLen(result, chckRange(base + i, 0, 9223372036854775807))' uses GC'ed memory [GcMem]
ssozp11x1e5bparnew64.nim(207, 39) Warning: 'KB' uses GC'ed memory [GcMem]
ssozp11x1e5bparnew64.nim(213, 31) Warning: 'maxpcs' uses GC'ed memory [GcMem]
lib/system/repr.nim(39, 14) Warning: '$ buf' uses GC'ed memory [GcMem]
lib/system.nim(709, 9) Warning: 'newSeq(result, len)' uses GC'ed memory [GcMem]
ssozp11x1e5bparnew64.nim(120, 13) Warning: '[]' uses GC'ed memory [GcMem]
ssozp11x1e5bparnew64.nim(125, 31) Warning: 'add(primes, modk + res[r])' uses GC'ed memory [GcMem]
lib/system.nim(709, 9) Warning: 'newSeq(result, len)' uses GC'ed memory [GcMem]
ssozp11x1e5bparnew64.nim(144, 24) Error: 'spawn' takes a GC safe call expression
➜ nim
Yes, you are right -- it is possible to write Nim code that does not compile or does not work as you expected.
But for most of us Nim seems to work well most of the time. When you do not like Nim, why are you not just using another one, there are many interesting languages available.
For your example code: Of course the Nim developers are very bright, but I guess even for smart people it may be difficult to guess what the problem with your program is, when you post only a small section of it. And note, many people and Nim manual told us that we do not need sync() when we use parallel statement. Why do you refuse to learn that? And no, I don't think that is the problem in your code.
Generally I have the feeling, that your attempts to calculate primes in parallel may be misguided, because you do not care for data cache that much. Most other people coding prime sieves for fun are using OpenMP and care much for data cache layout. And that makes much sense, as accessing data in cache is often 30 times faster than random RAM access.
I have just done a very short minimal test of parallel/spawn myself, with an interesting compiler output when using a seq instead of a fixed size array. But I think I did something wrong, I have never used parallel statement in the last two years, and have not read the manual again. https://github.com/nim-lang/Nim/issues/6814
Well you access a global with GC'ed memory. You can use something like this to make the compiler shut up
var globalSeqHere: seq[int] = @[]
proc foo() {.gcsafe.} =
{.gcsafe.}:
for i in 0..<N:
echo globalSeqHere[i]
This code works as desired in parallel. Thanks for the instructions.
proc column_load(prime, j, k, r: int) {.gcsafe.} =
{.gcsafe.}:
for ri in residues: # for each prime|residue pair
let prod = r * ri # compute res cross-product
let row = posn[prod mod modpg] * pcnt # compute restrack address
# compute|store for each row the resgroup vals for prime in column j
nextp[row + j] = uint(k*(prime + ri) + (prod-2) div modpg)
# Initialize the [rescnt x pcnt] 'nextp' table with resgroup values of the
# 1st prime multiples for each prime r1..sqrt(N) for each PG residue track.
proc nextp_init() =
# load 'nextp' with 1st prime multiples regroups vals on each residue track
parallel: # do in parallel
for j, prime in primes: # for each prime r1..sqrt(N)
let k = (prime-2) div modpg # find the resgroup it's in
let r = (prime-2) mod modpg + 2 # and its residue value
spawn column_load(prime, j, k, r) # load column resgroups for prime
sync()
I guess it was surprising that even when I turned off compiling with garbage collection (--gc:none) it still applied gc rules to this structure. Well, I learned a little bit more about Nim.