Hi all may I know are there any good ML frameworks / bindings in Nim ?
I am myself building a bindings for mlpack and it is just near completion I will release them soon,
I wanted to know did anyone know any great ml frameworks in Nim? Nim should be made as language for ML and AI inference also therefore I am asking,
I have plan to make additional bindings for other ml frameworks and inference also but I want to know the current situation.
Note I know about SciNim : https://github.com/orgs/SciNim/repositories
But it's project are too advanced currently not directly suitable for beginners entry into AI ML in Nim,
They are great but lack examples to use more specifically some project should be made using them but they are not good to be used directly for beginners.
https://github.com/mratsim/Arraymancer
Maybe. I was able to make a simple DQN a few years ago with it
https://github.com/Niminem/DeepLearningNim
I was a beginner then too, just following along YouTube tutorials but applying arraymancer instead of numpy
The Deep learning Nim is is a good start for making in pure nim but it is too basic,
By beginners I don't mean I am a beginner but I want to design frameworks/ binding which even anyone can use, I already use and do ML in python but I want such ecosystem in nim too,
Currently I think to bring cpp libs in Nim, Arraymancer is like Rust ndarray I earlier tried to make numpy alternative in rust but the project is moving very slowly and need dedicated time, which is the reason I avoid a single project from basic.
Creating anything from basic will cost a lot of time, therefore my focus should is also on cpp target that Nim support, I think Nim cpp has much great future equally as c if it is used correctlly, just fixing the concept on system and C will not help in every domain nor can we wait for maintaining and appearance of stable c api for many useful libs like mlpack,
I don't know the performance of Arraymancer is comparable to rust ndarray and numrs(https://github.com/rjaguiluz/numrs/tree/main/bindings/numrs-c) but I will try to use it (but I think it will not influence me similar to how rust ndarray failed to influence)
So far Nim cpp is much better and faster than haxe hxcpp if haxe is mostly useful due to that then we should value Nim cpp also.
Nim is easiest compiled language I have seen ever similar to golang therefore it inspire me as a replacement for python in ML
Nim has good potential for ML. But having good potential is not enough. A lot of dedicated effort required to make language not just good potential but actually good for ML.
Like having standard algos, plots etc work out of the box.
And features like interactivity, I doubt many people going to use lang for day to day work like exploratory data analysis, learning or experimenting without shift + enter executing current line in VS Code, etc.
Those features are not minor, they are must have basics.
It is not possible to make many things like xgboost lightgbm and other completely in Nim for now it need dedicated time and team. But it can be utilised so easily by bindings and will not need python bridge.
And if we continue that soon Nim ml ecosystem can be as comparable as Rust, even many people doubted Rust but slowly over period Rust as we can see is improving it's ML ecosystem, infact few of best vector storage for RAG are in Rust.
If we continue to watch will only lag behind in it.
I think You should see this : https://github.com/vaaaaanquish/Awesome-Rust-MachineLearning
Even zig coders are trying to enter the ML domains (we know how zig is (so much changing stdlib) still they are trying) like : zml https://github.com/zml/zml
And it is not the problem of Nim it is our non serious nature and laziness.
I've been trying to keep up with my limited time to maintain https://github.com/SciNim/flambeau. I've recentlt pushed more change and updated torch version. Flambeau is Nim bindings to C++ lib-torch, trying to make a higher level API on top of it for ergonomic usage.
I'm also maintaining https://github.com/SciNim/nimjl/tree/master/nimjl which are bindings to the Julia language so you can embed the Julia interpreter into a Nim binary and get benefit of both language. Julia has a great scientific computing ecosystem !
Your project are actually very well and You did the harder work already, but currently I can't understand how exactly I can use them and if there will be examples or documentation of using them I will appreciate a lot.
If I once understand how exactly to use SciNim correctly I will personally make many examples and contribution to SciNim project.
I would also appreciate if You can can make example of python workflow and side by side counter example in SciNim.
Also it will be useful if You can provide Colab example of how exactly we can make or train model using SciNim because it provide a good environment for testing.
A question might arise why and how ? because without colab there is no big advantage of frameworks if they can't utilise it.
it is much easy and I hope You know better :
!curl https://nim-lang.org/choosenim/init.sh -sSf | sh -s -- -y
os.environ['PATH'] = f"{os.environ['HOME']}/.nimble/bin:{os.environ['PATH']}"
%%bash
cat << 'EOF' > example_scinim.nim
EOF
And finnally running that :
%%bash
nim cpp -d:release
-- all other SiNim specification example_scinim.nim
./example_scinim
And finally I would like to say You have made something very rare and useful and Do at Your own pace no hurry.
And it is not the problem of Nim it is our non serious nature and laziness.
I don't see the need of guilt-tripping people who 99% of the time are coding in Nim as a hobby.
Otherwise that reads as entitlement, you're asking a community of a couple hundreds people to compete with Facebook+Google+Microsoft+Baidu+Alibaba+Apache Foundation+Red Hat+Open AI+Intel+Nvidia and that's only the high-profile companies.
For the last couple months I had an itch to write a LLM inference engine. Given the discussion, I refined my plan with concrete steps to reach a minimal general-purpose inference engine.
Here it is: https://github.com/mratsim/tattletale/issues/1
Tattletale will be an LLM inference engine targeting decent state-of-the-art quantization, good tool-calling support and decent concurrency support at the very least. Another goal is ease of deployment and embeddability in frontends.
Further development might bring CPU+GPU hybrid inference and vision support.
Unfortunately none of the current main inference engines meet all my goals.
Framework Key Strengths Pain Points & Limitations vLLM • Latest model compatibility<br>• Concurrency<br>• PagedAttention<br>• Fast prompt processing<br>• Tensor parallelism<br>• Sampling overrides<br>• Some mixed-precision support (int4/fp4, int8/fp8, bf16)<br>• Latency-hiding via async-scheduling • Devops: Dependency hell + hours of compilation (transformers, flashinfer, flashattention, etc.).<br>• Rigid Quantization: Using over 2-year-old algos (GPTQ/AWQ) with inflexible sizes.<br>• GPU-Only: No true hybrid CPU+GPU support.<br>• Limited KV-Cache Quantization: FP8-only, no hadamard transforms for KV-cache<br>• Very slow startup (slow safetensors load, cuda graphs + flashinfer JIT autotuning) SGLang • Concurrency<br>• RadixAttention<br>• Hierarchical KV cache<br>• Speculative decoding<br>• Fast startup<br>• Latency-hiding via async-scheduling • Consumer Hardware Unfriendly: Default triton kernels use >101kB shared memory (breaks on RTX 5090/RTX Pro 6000).<br>• Unflexible configs: No sampling overrides; no mixed precision AWQ+FP8.<br>•**Rigid Quantization:** Using over 2-year-old algos (GPTQ/AWQ) with inflexible sizes.<br>• Limited CPU+GPU hybrid inference: via KTransformers, requiring AMX CPUs (recent Intel Xeon Platinum), and different weights for CPUs and GPUs<br>• Limited KV-Cache Quantization: FP8-only, no hadamard transforms for KV-cache KTransformers (optionally+SGLang) • Hybrid CPU+GPU handling (MoE offloading) • Hardware Constraints: Kernels for AMX CPUs and some WIP for AVX512 ExLlamaV3 / TabbyAPI • State-of-the-art EXL3 quantization (Hadamard transform + Trellis + Lattices codebook)<br>• State-of-the-art KV-cache quantization • Tool Calling: No support for recent models (Qwen, GLM, MiniMax) using XML tool calls. Existing models need a modified tabby-specific chat template<br>• Reasoning: Thinking/reasoning patterns are not handled server-side.<br>• GPU-Only: No hybrid inference support.<br>• Alternative attention support: No support for MLA (DeepSeek, Kimi, GLM-4.7-Flash) or attention sinks (GPT-OSS, MiMo-V2-Flash) Llama.cpp • Single binary deployment<br>• Excellent ROCm, Vulkan, Metal support<br>• Hybrid CPU+GPU • No Concurrency Focus: Context is statically split when using --parallel<br>• Basic Context: Lacks advanced memory management (PagedAttention, RadixAttention) needed for high-throughput multi-user scenarios.<br>• No tensor parallelism: Matrix multiplication scales with O(n³), tensor parallelism over 2 GPUs will reduce size by 2 (token generation is memory-bandwidth-limited for a single query) and compute needed by 2³ = 8 (context processing is compute-limited). Making it an incredible performance improvement
Now in terms of actual implementation I'm strongly considering a Nim backend + Rust frontend for several reasons:
Anyway, so that people realize the scope of the MVP, those are the tasks, assuming pure Nim, I can trivialize 2-3-4-5-13-17 with Rust, and I might have missed some tasks:
LLMs are much better at Rust than Nim
I don't see how, the good models effectively always produce valid, working Nim code. But for you there is always something anyway, so whatever.
I think one of the main reasons LLMs are better at Rust than Nim is besides the much higher number of codebases they can use for training, the dev tools through cargo are easier to use due to standardization. For example MiniMax-M2.1 (currently my go to, shameless plug: https://huggingface.co/mratsim/MiniMax-M2.1-FP8-INT4-AWQ) knows how to use nim check but can go much further for testing in Rust with cargo build, cargo check, cargo test, cargo test -- --ignored --nocapture, cargo test --test mytestfile mytestproc, meaning I can let them loop with a task with less manual intervention.
Out of the box perhaps, add a simple AGENTS.md / CLAUDE.md overcome that.
Mine are < 30 lines. Basically use atlas and setup a config.nims with some tasks so you can run nim test or nim build. It'll make your life 100x simpler.
I made Lotty, Metalx, and Neonim all without telling GPT5/Codex how to run Nim by copying the AGENTS.md around. I just ask it to solve problems and walk away.
Otherwise yeah, I've seen GPT5/Codex go into spirals trying to get Nimble to work and figure out how to debug nimble test.
# Repository Guidelines
## Project Structure & Modules
- `src/`: Core library modules for library.
- `tests/`: Unit tests using Nim's `unittest` plus a `tests/config.nims` that enables ARC/threads and debug flags and sets the project source path.
- Root files: `lotty.nimble` (package manifest), `README.md` (usage), `CHANGES.md` (history).
## Build, Test, and Development
- Install deps (atlas workspace): `atlas install` (ensure `atlas` is installed and configured for your environment). *Never* use Nimble - it's horrible. *Always* use Atlas and it's `deps/` folder and `nim.cfg` file to see paths.
- Run all tests: `nim test` (uses the `test` task in `config.nims` to compile and run every `tests/*.nim`). Create a `test` task in `config.nims` using `listFiles("tests")` if ones missing.
- Run a single test locally:
- `nim c -r tests/ttransfer.nim`
## Coding Style & Naming
- Indentation: 2 spaces; no tabs. `procs` must be declared before use.
- Nim style: Types in `PascalCase`, procs/vars in `camelCase`, modules in `lowercase` or concise `lowerCamel` (e.g., `threadAsyncs.nim`).
- Formatting: run `nph src/*.nim` and format any touched test files.
## Testing Guidelines
- Framework: `unittest` with descriptive `suite` and `test` names.
- Location: add new tests under `tests/`, mirroring module names (e.g., `tslots.nim` for `slots.nim`).I don't said that vibe coding can't make inference to run tiny llama running a single small model is easy task and even 2-3 years ago it can be run in a single c file with just 700 lines of code.
So for making something like karpathy llama2.c which was already made 3 years ago isn't a great task for which llm should be praised infact llm might have directly copied the algorithms and inference logic from project like this. And also llama2.c can run models with 7b size. While tinyllama is just 1.1b
https://github.com/karpathy/llama2.c/blob/master/run.c
Also I never meant to make something like llama2.c I asked You like llama.cpp which was claimed.
If the task was just to make something like llama2.c I would have said it directly nor did the creator of llama2.c claimed that llama.cpp is a junk but instead he thanked that llama.cpp has inspired his code.
Making an inference for tiny llama wasn't the task and I know how easy it is to make something like that, therefore I gave You a time of six months,
And well You think I can't understand junk or I judge code by loc in it absolutely not but I think You judge me and even senior AI expert like Georgi Gerganov that they put junk in their code and people just give them ~94k stars for junk.
Overall since arguments will not lead to any conclusion and as You are my senior I respect Your opinion.
So for making something like karpathy llama2.c which was already made 3 years ago
Dude llama2.c wasn't done in 2 hours 3 years ago... You completely miss the point! The point is once again: Thanks to AI a language's ecosystem becomes more and more of a non-issue.
" Thanks to AI a language's ecosystem becomes more and more of a non-issue, you can literally type "give me a neural net implementation in Nim that can read and execute llama.cpp's model files" and have it work in an afternoon. If not right now then next week..."
If above comes to fruition no one will care about Nim because they will be able to get what they need with a prompt. LOL.
Stay to the topic this thread is not about usefulness of AI in coding task including Nim if someone want to discuss that make a new thread.
Already the topic got deviated by introduction of alien topic, hope that the final conclusion is : Yes AI is really helpful for coding task but it has it's own limits including premium model tokens, context limits (large repos) and hallucinations.
Since the Famous Hello Algo was not available in Nim or I was not able to find hello algo implementation in Nim therefore I thought to make it.
https://github.com/krahets/hello-algo
As the Python code of Hello Algo was most updated among all I utilised it for reference also as I am a Python dev and earlier learnt hello algo in Python only therefore it was easier.
I know it is not a big deal but I thought it will help me to practice and also help utilise great work of hello algo for Nim users therefore, also I want them in Nim.
The hello algo comments are in Chinese so I translated them from it using Google and llm, and I also reviewed them with help of my chinese friend I hope they are correct. Although the code is tested and I compared the results of Python vs Nim but still some human errors can be there.