Hi, first message/thread in this forum so please bare with me :)
I've recently come across an issue with Nim that I'm unable to debug/understand and hoping I can get some insight. So, I'm trying to statically link the OpenSSL 1.1.1 library as well as glibc(2.27) to my Linux Nim application. I'm also trying to run this as a daemon, so I must run all code under a fork() in a child process.
This is the nim compiler command I'm using on an ssltest.nim file.
nim c --os:linux --cpu:amd64 -d:ssl --passL:-static --dynlibOverride:crypto --dynlibOverride:ssl --passL:/usr/lib/x86_64-linux-gnu/libssl.a --passL:/usr/lib/x86_64-linux-gnu/libcrypto.a --threads:on ssltest.nim
Issue:
When calling newContext(verifyMode = CVerifyNone) in the child process, the application hangs. No error or exception is thrown. I've tried to mimic the exact procedure of calls in a C file with a static link to OpenSSL and glibc, yet I don't get this behavior. It successfully creates a new context.
ssltest.nim:
import posix, net
let pid: Pid = posix.fork()
if(pid < 0 or pid > 0):
quit()
when(defined(ssl)):
var socket = newSocket(buffered = false)
var clientContext = newContext(verifyMode = CVerifyNone)
echo "Successfully created context"
Here is something similar using C:
C code gcc compile for ssltest.c
gcc -static -L:/usr/lib/x86_64-linux-gnu/libcrypto.a -L:/usr/lib/x86_64-linux-gnu/libssl.a -o ssltest_c ssltest.c -lssl -lcrypto
#include <stdio.h>
#include <stdlib.h>
#include <openssl/bio.h>
#include <openssl/err.h>
#include <openssl/ssl.h>
#include <openssl/crypto.h>
#include <unistd.h>
int main()
{
int pid;
pid = fork();
if(pid > 0 || pid < 0)
{
return 0;
}
const SSL_METHOD* method = TLSv1_2_client_method();
if(method == NULL)
{
printf("method is null, this failed\n");
}
SSL_CTX* ctx = SSL_CTX_new(method);
if(ctx == NULL)
{
printf("CTX is null, this failed\n");
}
printf("Successfully created context.\n");
}
I thought the order of operations in my nim compiler command was my issue, but whenever I moved them around and used --compileOnly --genScript the same gcc commands were always present.
I also tried some older OpenSSL functions that were not wrapped in Nim, like OPENSSL_fork_prepare() using importc:, or even an explicit call to OpenSSL init functions, but always received the same results.
Unsure at this point what I could try next.
Yes, you're right it would. This was just a quick and dirty example to showcase the fork() issue with I'm having with Nim. Wasn't completely meant to be a daemon. Here's an updated nim block from the OP with waitpid() as the issue still persists.
import posix, posix_utils, net
var pid: Pid = posix.fork()
var status: cint
if(pid == 0):
when(defined(ssl)):
var socket = newSocket(buffered = false)
var clientContext = newContext(verifyMode = CVerifyNone)
echo "Successfully created context"
discard waitpid(pid, status, 0)
You're statically linking OpenSSL and glibc as well with that exact nim compiler command?
A few colleagues and I have had issues running a fork() followed by SSL calls when statically linking OpenSSL. Without the fork, everything runs fine.
My goal is to have a single statically linked Nim compiled Linux Binary detach itself from the terminal so that it runs in the background as an orphan much like this basic daemon design in C: https://www.thegeekstuff.com/2012/02/c-daemon-process/. I cannot use vfork() with a parent's exit as the child shares the memory address space and thus the process stays open in the foreground until the child's exit.
I experimented with poDaemon, but it doesn't give the results I expected. I'm not actually sure what it does if starting a Nim Linux application from the terminal doesn't start a new process with startProcess() in another terminal as is without it. I tried startProcess() with the executable itself and with the poDaemon option, but a parent process still needs to exit in order to give the user the terminal control back. Trying to exit the parent process this way resulted in defunct process in the process list which I wouldn't get with a fork() call, despite its issues.
Regardless, my issue is with calling OpenSSL functions following a fork() call with a Nim compiled Linux binary linking with OpenSSL and glibc.
You've already done this, but create a static OpenSSL:
$ git clone git://git.openssl.org/openssl.git
$ cd openssl
$ git checkout OpenSSL_1_1_1j
$ ./config -static && make -j4
Then add zig and this 'zigmusl' script to your PATH:
#!/bin/sh
exec zig cc --target=native-linux-musl "$@"
Then, with your example as forkssl.nim:
$ CC=zigmusl nim -d:ssl --dynlibOverride:ssl --dynlibOverride:crypto --passL:./libssl.a --passL:./libcrypto.a --cc:env c sslfork.nim
$ ldd sslfork
not a dynamic executable
$ ./sslfork
Successfully created context
$
... and, your exact test works with just --passL:-static, with some warnings:
$ nim -d:ssl --dynlibOverride:ssl --dynlibOverride:crypto --passL:./libssl.a --passL:./libcrypto.a --passL:-static c sslfork.nim
/usr/bin/ld: ./libcrypto.a(dso_dlfcn.o): in function `dlfcn_globallookup':
dso_dlfcn.c:(.text+0x11): warning: Using 'dlopen' in statically linked applications requi
res at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: ./libcrypto.a(b_addr.o): in function `BIO_lookup':
b_addr.c:(.text+0xb47): warning: Using 'getaddrinfo' in statically linked applications re
quires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: ./libcrypto.a(b_sock.o): in function `BIO_gethostbyname':
b_sock.c:(.text+0x2a1): warning: Using 'gethostbyname' in statically linked applications
requires at runtime the shared libraries from the glibc version used for linking
$ ldd sslfork
not a dynamic executable
$ ./sslfork
Successfully created context
It's probably also bad to mix glibc OpenSSL with otherwise musl objects.