Mirror of forum.nim-lang.org

14032 :: RocheDB v0.2.3: a technical preview of a new-concept NoSQL database written in Nim

• puffball1567 (orginal) [2026-07-07T15:01:38+02:00] view original

  Hi Nim community,

  I have released RocheDB v0.2.3, a technical preview of a new-concept NoSQL database written in Nim.

  Repository: https://github.com/puffball1567/rochedb

  Documentation: https://puffball1567.github.io/rochedb/

  Install:

  nimble install rochedb

  roche --help

  RocheDB is ring-oriented. Documents and vectors are stored in logical placement units called rings, such as:

  docs/japan/support

  training/math/algebra

  tenant/acme/orders

  A ring is not just a directory-like label. RocheDB stores ring metadata and uses it during retrieval. Ring-scoped reads touch only the selected ring or ring subtree, and the API reports statistics such as scanned, skippedVectors, ringsTouched, and candidateReduction. The goal is to make the application's natural data structure part of the query plan, not just part of the name.

  The naming and some of the mental model are inspired by astronomy.

  The idea is that the placement of data should help reads focus on a smaller, more relevant working set. Instead of treating the database only as a key/value lookup layer, RocheDB tries to make the application's natural data structure part of the retrieval model.

  Some current benchmark results from the repository:

  • Working-set benchmark: 100 rings / 10k documents reduced scanned records per query from 10,000 to 100.
  • Memory-pressure benchmark: 100 rings / 100k documents reduced estimated candidate memory per query from 93.079 MiB to 0.931 MiB.
  • Synthetic RAG benchmark: recall stayed at 1.000 while scanned/query dropped from 8,000 to 1,000, and estimated tokens/query dropped from 3,960 to 657.8.
  • Generated AI/RAG JSONL case study: recall stayed at 1.000 while scanned/query dropped from 400 to 40, and estimated tokens/query dropped from 615.2 to 231.6.

  Current v0.2.x features include:

  • Nim native embedded API
  • memory-only and WAL-backed persistent stores
  • hierarchical rings
  • JSON document put/get/query/list/count
  • GraphQL-like JSON projection
  • atlas output for galaxy/ring descriptions
  • vector retrieval with exact backend and optional FAISS bridge
  • roche CLI with a small interactive shell
  • static cluster prototype
  • username/password auth plus secret-key gate
  • backup / restore / encrypted backup
  • dump / JSONL import
  • universe sync prototype for eventual cross-store synchronization
  • Docker Compose demos
  • C ABI and early driver foundations for several languages

  This is still a technical preview / research OSS release. It is not production-ready, and I am not presenting it as a Redis/PostgreSQL/MongoDB replacement yet.

  Known gaps include TLS, dynamic cluster membership, coordinator redundancy, and larger real-world benchmarks.

  I would appreciate feedback from the Nim community on:

  • the Nim API design
  • Nimble install / packaging
  • CLI usability
  • portability on Linux/macOS/Windows
  • the C ABI and wire protocol boundary
  • benchmark or failure-test ideas

  The project is Apache-2.0 licensed.

  Thanks.

---

• puffball1567 (orginal) [2026-07-08T16:24:06+02:00] view original

  Additional local and Docker benchmark numbers

  I also added reproducible benchmark helper scripts for PostgreSQL and Redis comparisons.

  The PostgreSQL comparison now has both local-local and Docker-Docker runs.

  RocheDB vs PostgreSQL, local-local

  • RocheDB 3-node TCP read, persistence enabled: 45.9 us/op
  • RocheDB 3-node TCP write, persistence enabled: 47.7 us/op
  • PostgreSQL 14.23 primary-key SELECT: 67 us/op
  • PostgreSQL 14.23 single-row write with synchronous_commit=off: 79 us/op

  • PostgreSQL 14.23 single-row write with synchronous_commit=on: 1998 us/op

    RocheDB vs PostgreSQL, Docker-Docker

  • RocheDB 3-container TCP read: 53.5 us/op
  • RocheDB 3-container TCP write: 56.4 us/op
  • PostgreSQL 14.23 Docker primary-key SELECT: 92 us/op
  • PostgreSQL 14.23 Docker single-row write with synchronous_commit=off: 130 us/op
  • PostgreSQL 14.23 Docker single-row write with synchronous_commit=on: 1134 us/op

  RocheDB vs Redis, local-local

  • RocheDB TCP GET: 43.14 us/op
  • RocheDB TCP BGET: 1.45 us/op
  • Redis TCP GET: 40.64 us/op
  • Redis pipeline GET: 3.53 us/op

  RocheDB vs Redis, Docker-Docker

  • RocheDB TCP GET: 53.18 us/op
  • RocheDB TCP BGET: 1.55 us/op
  • Redis TCP GET: 51.58 us/op
  • Redis pipeline GET: 2.13 us/op

  The benchmark helpers are now in the repository:

  • examples/postgres_bench.sh
  • examples/postgres_docker_bench.sh
  • examples/redis_local_bench.sh
  • examples/redis_docker_bench.sh

  These are local benchmark results, not universal performance claims. But they are useful because the comparison is no longer only "RocheDB on my machine"; there are now local-local and Docker-Docker reproduction paths for the main PostgreSQL and Redis comparisons.

  The short interpretation is:

  • RocheDB was faster than PostgreSQL primary-key read and synchronous_commit=off single-row write in both local-local and Docker-Docker runs.
  • Redis single GET was still slightly faster than RocheDB single TCP GET.
  • RocheDB BGET was faster than Redis pipeline GET in both local-local and Docker-Docker runs.

  This is separate from RocheDB’s main working-set reduction idea. The point of these latency numbers is just to show that the reduced working set is not being bought with an obviously slow read path.

---

• Araq (orginal) [2026-07-08T17:04:26+02:00] view original

  Use nim devel and try to get it to run with the new --strings:sso switch. Might speed up things further, though it is unlikely.

---

• puffball1567 (orginal) [2026-07-08T17:50:17+02:00] view original

  Thanks for the suggestion. I tried a quick experimental run with Nim devel and --strings:sso.

  Environment:

  • Same local machine
  • payload=100B
  • n=1000
  • local Redis at 127.0.0.1:6379
  • one local roched
  • RocheDB persistence disabled for this Redis comparison

  Stable Nim 2.2.10, normal -d:release:

  • RocheDB TCP GET: 43.21 us/op
  • RocheDB TCP BGET: 1.42 us/op
  • Redis TCP GET: 40.62 us/op
  • Redis pipeline GET: 3.59 us/op

  Nim devel 2.3.1, normal -d:release:

  • RocheDB TCP GET: 43.42 us/op
  • RocheDB TCP BGET: 1.46 us/op
  • Redis TCP GET: 39.96 us/op
  • Redis pipeline GET: 3.89 us/op

  Nim devel 2.3.1, -d:release --strings:sso:

  • RocheDB TCP GET: 44.58 us/op
  • RocheDB TCP BGET: 1.94 us/op
  • Redis TCP GET: 40.39 us/op
  • Redis pipeline GET: 4.46 us/op

  In this small local run, Nim devel itself was roughly neutral, and --strings:sso did not improve this RocheDB benchmark. This benchmark may simply not be a good fit for showing benefits from the new string mode.

---

• timbuktu (orginal) [2026-07-08T18:13:52+02:00] view original

  I really truly wonder if these libraries will one day prove their weight, or just always remain a load of LLM generated slop.

  IMO farrrr too many projects are being churned out here and then forgotten.

  This is just my impression when I see projects created on GitHub like literally every day.

  I could be wrong though, perhaps there are just a lot of people in the world.

  I’m not really sure.

  Also the language you used here is not nearly technical enough… I’m confused on the entire thing.

---