Mirror of forum.nim-lang.org

2289 :: Problem with 'nim -c' in parallel

• cdunn2001 (orginal) [2016-05-30T21:12:34+02:00] view original

  When a build some nim programs sequentially, there is no problem. But when I run make -j, I get errors like this:

  
  make: *** [nvill] Error 1
  CC: stdlib_sequtils
  Error: execution of an external compiler program 'clang -c  -w -O3  -I/Users/cdunn2001/repo/gh/Nim/lib -o /Users/cdunn2001/repo/localhost/nim-parse/nimcache/stdlib_system.o /Users/cdunn2001/repo/localhost/nim-parse/nimcache/stdlib_system.c' failed with exit code: 256
  
  /Users/cdunn2001/repo/localhost/nim-parse/nimcache/stdlib_system.c:392:16: error: redefinition of 'TMP26' as different kind of symbol
  STRING_LITERAL(TMP26, "no exception to reraise", 23);
                 ^
  /Users/cdunn2001/repo/localhost/nim-parse/nimcache/stdlib_system.c:212:33: note: previous definition is here
  typedef N_CLOSURE_PTR(NIM_BOOL, TMP26) (Exception* e);
                                  ^
  /Users/cdunn2001/repo/localhost/nim-parse/nimcache/stdlib_system.c:634:3: error: use of undeclared identifier 'LOChunk29682'
                  LOChunk29682));
                  ^
  Run

  Obviously, the nimcache is not locked. If I use nim -d:release --parallelBuild:1 c, then I get linkage errors:

  
  Undefined symbols for architecture x86_64:
    "_NTI106028", referenced from:
        _nsuSplitChar in stdlib_strutils.o
    "_readline_15513", referenced from:
        _euant0Init000 in euant0.o
  ld: symbol(s) not found for architecture x86_64
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  Error: execution of an external program failed: 'clang   -o /Users/cdunn2001/repo/localhost/nim-parse/euant0  /Users/cdunn2001/repo/localhost/nim-parse/nimcache/stdlib_parseutils.o /Users/cdunn2001/repo/localhost/nim-parse/nimcache/stdlib_strutils.o /Users/cdunn2001/repo/localhost/nim-parse/nimcache/stdlib_system.o /Users/cdunn2001/repo/localhost/nim-parse/nimcache/euant0.o  -ldl'
  Run

  How do I build in parallel? Is this a feature worth adding to nimble?

  • https://github.com/cdunn2001/nim-parse/tree/parallel

---

• OderWat (orginal) [2016-05-30T21:45:38+02:00] view original

  Maybe you can use nim c --nimcache:nimcache_$RANDOM foo.nim in this case?

  EDIT: I made a PR with what works for me https://github.com/cdunn2001/nim-parse/pull/1/files

---

• cdunn2001 (orginal) [2016-05-30T23:25:28+02:00] view original

  I can see how that would work. Not a bad idea. But that's just rebuilding everything it depends on. Overall, it goes from:

  
  real	0m6.295s
  user	0m5.600s
  sys	0m0.596s
  Run

  to

  
  real	0m3.058s
  user	0m10.182s
  sys	0m0.996s
  Run

  Less realtime, but more CPU time and disk I/O (which matters on a network). A real parallel build would sure be nice.

---

• OderWat (orginal) [2016-05-30T23:43:18+02:00] view original

  You could rewrite "nim.helper" to use the project name for the cache name such that this stays constant. I am to lazy for that right now :)

  EDIT: I guess thats even possible without the helper, directly in the makefile!

---