Hello all, today I wanted to share Mummy, my new multithreaded HTTP + WebSocket server written entirely in Nim.
I started writing Mummy because I wanted an alternative to async. There are very good historical reasons for async (refc + threadlocal heaps making multithreaded stuff extra challenging), however ORC / ARC and Nim 2.0 coming very soon have really opened the door to another option: threads.
Mummy takes what I think a pretty modern approach to threads and IO. The general model for Mummy goes like this: multiplex socket IO on one thread and dispatch ready-to-go requests to a pool of worker threads. This keeps worker threads far away from client sockets while still enabling the pleasant inline blocking boring Nim code that makes me a very happy programmer.
I see enormous value in simple blocking code and would be willing to make some performance sacrifice to have it. It turns out, though, that I don't have to.
Mummy is able to outperform AsyncHttpServer, as well as Node and Go. Mummy even outperforms HttpBeast, though this may be due to a bug.
You can confirm the results yourself by looking at the benchmarks, the code behind them, and running them yourself.
Mummy basically offers the maximum performance potential while also enabling you to write the simplest code. This is a dream combo.
Please check out the Mummy README if you want to learn more.
How would I go about using global objects with Mummy? I assume I will have to use atomics/locks manually myself?
And since there's a maxThreads which is set to 2x of CPU count by default, does that mean that if requests take a long time to do, all other requests will just stall and have to wait for the worker threads to finish the current requests?
Today it does not, so those using Mummy would want to consider having something like Nginx or whatever do the SSL stuff and reverse proxy to your Mummy server. Or put Cloudflare infront of the server.
I've personally found self-hosting + Cloudflare Tunnels to be an interesting option too, very old+new. Similar things can be done with a public VPS or whatever, lots of ways to skin this cat.
Doesn't seem so, but usually those services are served behind a Nginx/Apache/Caddy instance or Cloudflare (or some other cloud) that handle all the details.
Yes, I know. So let me rephrase my question. How much work would it be to implement SSL support?
Great work, although this is obviously more of a proof of concept and not yet ready for serious work.
Besides SSL, there doesn't appear to be any handling/parsing of URL parameters. The handling of such parsing is also critical to performance. Jester creates a very useful Request object, but that probably doesn't explain the full gap in performance.
The await bug is also interesting, not sure if any of the core devs have ideas about that.
I will not claim Mummy is DoS-proof by any means, that'd be nuts without extensive evidence to back it up, but I have been mindful in my choices to not leave obvious weak points.
I did audit asynchttpserver fwiw and I will happily audit Mummy too. I'm not an expert on these things but I have some experience. Please tell me once you consider it ready for me to review.
I'll have to look through your epoll setup! I did a of work with them and I'm curious how you set them up.
What are you using to send data between the IO thread and workers? Just the threading channels or something else?
I'll have to look through your epoll setup!
I ended up having success with Nim's std/selectors package so the main socket IO loop uses that. I am happy this low-level library was available.
What are you using to send data between the IO thread and workers?
There's only 2 memory-thread transition points: requests from IO thread -> worker threads and responses from worker threads -> IO thread. This is also true for WebSocket messages.
For incoming requests I'm manually managing the memory since I want the Request object to be many-thread-safe to enable even further use of threading (fanning out in an incoming request onto a few more worker threads to do some stuff in parallel before responding). That's only something I have running locally, not shown in the repo right now.
As for responses from workers -> IO thread, that's an ownership transfer move into a queue + Atomic-as-lock. Same idea as a channel but I had very simple needs so I just kept it simple.
This is the step in the right direction (I think it's called the Reactor Pattern). It covers most of cases needed from the server.
Good job! @boia01 and other implementations are good job too :).
Thanks for trying out Mummy. I have a theory on the issue you're facing.
By default Mummy only listens on "localhost" meaning the port is not externally accessible. This is an important default for security reasons, however for a public web server this is a no-go. To address this you'll want to use something like server.serve(Port(80), "0.0.0.0") as shown here: https://github.com/treeform/nimdocs/blob/master/src/nimdocs.nim#L157
Then when you have Cloudlfare proxy requests, it will hit IP_ADDRESS:80 for example and work since "0.0.0.0" is publicly available.