I've written a Nim library around libmdbx, a super-fast key-value store. I call it NimDBX. It's still early in development, but it's passed a handful of unit tests so I think it's ready to show off :)
NimDBX just maps arbitrary keys to values, in multiple namespaces, storing them in a local file. The file is memory-mapped, which makes reads ridiculously fast, with zero copying and zero system calls: the library literally just locates the data in the mapped address space and hands you a pointer to it.
Even though it's low-level, it provides full ACID semantics: Atomic commits, Consistent views of data, Isolation from other commits, and Durable changes. Multiple threads and processes can open the same database file without trouble. The database is not corrupted by program crashes, kernel panics, nor power failures.
I haven't yet gotten Nimble to build lmdbx for you, so you'll have to run make by hand the first time (details are in the readme.) I will fix that soon, and submit the package to the Nimble directory once I'm a bit more confident in its stability.
libmdbx, by Leonid Yuriev, is pretty amazing. It's of the same family as Berkeley DB, but it's just ridiculously fast thanks to memory-mapping. (If you've heard of LMDB, libmdbx is an improved fork of that; LMDB doesn't seem to be in active development anymore.)
Also using Nimterop to generate a Nim wrapper of the C header, instead of my hand-written one.
Er, please keep the hand-written one... :-( IMO Nimterop is for when you don't have the time to hand-craft a wrapper.
For some reason this forum didn't notify me of your reply.
That's great! I hope you'll consider documenting install and usage with us poor souls coming from high level languages, frameworks and ORMs in mind. :)
I, for one, like the Arch wiki style: short code examples that show all the necessary operations with the deeper explanation/source code being optional reading. As opposed to the FreeBSD/Gentoo style where you have to read through the deeper/more exhaustive explanation, and try to glean the needed info.
For instance, the nimdbx readme seems to have replaced the concise install info with a link to Nimterop, and if i go there i get a bunch of lower level info i don't really care about right now, presumably with the needed info embedded in unrelated context. It's not just that new users have to read through more stuff, but that it can make it harder to understand and/or get started.
Just some feedback in case it's useful.
Thanks
Well, Nimterop has the advantage of being automatic: if some functions are added in the header, it will be easy to keep it up to date
And it has the disadvantage that you cannot ship the Nim code as it's OS and CPU specific. So what if functions were added to the header, who says that you need them.
the nimdbx readme seems to have replaced the concise install info with a link to Nimterop
The install actually got much easier — you should now be able to just check out or download the repo, then type “nimble test” and it builds and self-tests.
Nimterop is just behind-the-scenes infrastructure that takes care of building the C code and translating its header to Nim. You don’t need to know it’s there.
nimdbx is in the Nimble directory now, so if you’ve got keys you want to value, you can install it easily now.
Although the Nimble page says its install is failing:
/tmp/nimble_21298/githubcom_snejnimdbx/nim.cfg(2, 3) Error: 'none', 'boehm' or 'refc' expected, but 'orc' found
That’s a weird error, since the config file explicitly gives Nim 1.4 as a dependency.
It’s also got some new features: the Cursor class supports duplicate-key Collections, and there’s a Collatable class that lets you have compound (primary/secondary) keys. I’m working on support for indexing now, I.e. making one Collection be an index of another.
This is absolutely amazing stuff. I was using lmdb and it was ok, but the Nim layer on NimDBX is just beautiful.
Proper database as easy to use as the filesystem... finally!