nimforum mirror - watch out with OpenSSL and Threads

wiffel (orginal) [2015-03-08T11:52:06+01:00] view original

I've been playing around with the multiple parallel/concurrency mechanisms in Nim. The program I'm writing to test this uses OpenSSL to access services on the internet.

Maybe everybody already knows, but I didn't and did loose a lot of time debugging the combination of using --threads:on and OpenSSL. This is just a post to save some time for people that hit the same problem.

I've tried different ways to use OpenSLL: directly importing it using dynlib, using the different openssl libraries and wrappers, implicitly by using https using httpclient. All of them resulted in some strange errors, most of the time during garbage collection.

After lots of debugging, I did figure out that OpenSSL was causing this. OpenSSL is not thread-safe by default. You need to add some locking support for OpenSSL to run correctly in a multi-threaded program. Annoyingly it does compile and run without any error messages if you don't do that, but it will result in severe random errors if you do so.

I did add the following code to add the locking support OpenSSL expects and that solved most of the (see below) run-time errors.


import locks

proc SSL_library_init(): cint
     {.cdecl, dynlib: "libssl.so", importc.}

proc CRYPTO_num_locks(): int
     {.cdecl, dynlib: "libssl.so", importc.}

proc CRYPTO_set_locking_callback(funptr : pointer)
     {.cdecl, dynlib: "libssl.so", importc.}

discard SSL_library_init()
var LibSslLocks : seq[TLock]
LibSslLocks.newSeq(CRYPTO_num_locks())

proc openSslLockingCallback(mode : cint, lock_num : cint,
                            file : cstring, line : cint) {.cdecl .} =
  if 1 == (mode and 1):
    acquire(LibSslLocks[lock_num])
  else:
    release(LibSslLocks[lock_num])

CRYPTO_set_locking_callback(openSslLockingCallback)

A second problem ...

I also use the SHA1 function from OpenSLL in my program. When using --threads:on, that resulted in some strange behavior. It did not crash, but it - sometimes - gave wrong results. Then I noticed a little line in the documentation of OpenSSL:


SHA1() computes the SHA-1 message digest ...
   ... Note: setting md to NULL is not thread safe.

In summary: Using the return value of SHA1() (with parameter md set to the default NULL value) is OK in a non-threaded program. It is not OK in a threaded program. It will compile and run without error messages, but will just give wrong results. The following code (adapted from rosettacode.org) works fine:


import strutils

proc SHA1(d: cstring, n: culong, md: cstring): cstring
         {.cdecl, dynlib: "libssl.so", importc.}

const SHA1Len = 20

proc sha1(s: string): string  =
  result = newStringOfCap(1 + 2 * SHA1Len)
  let md = newString(SHA1Len)
  discard SHA1(s.cstring, s.len.culong, md.cstring)
  for i in 0 .. < SHA1Len:
    result.add(md[i].BiggestInt.toHex(2).toLower)

I hope this can save somebody some time.

Mirror of forum.nim-lang.org

999 :: watch out with OpenSSL and Threads