I'd like to publish a Rust frontend to a Nim library that works as if native when use from cargo build and cargo run.
This means at one point, the build.rs script should call the Nim compiler.
I'm wondering if there is an example of that, or even better a package like cc which abstract away calling the C compiler in Rust.
From https://docs.rs/cc/latest/cc/:
The purpose of this crate is to provide the utility functions necessary to compile C code into a static archive which is then linked into a Rust crate. Configuration is available through the Build struct.
This crate will automatically detect situations such as cross compilation or other environment variables set by Cargo and will build code appropriately.
The crate is not limited to C code, it can accept any source code that can be passed to a C or C++ compiler. As such, assembly files with extensions .s (gcc/clang) and .asm (MSVC) can also be compiled.
(buggy RST quoting workaround)
To my knowledge there is no such package, but this build.rs should get you started:
use std::env;
use std::fs;
use std::path::Path;
use std::process::Command;
fn main() {
let main_file = "src/nimlib.nim";
let name = Path::new(main_file).file_stem().unwrap().to_str().unwrap();
let out_dir = env::var("OUT_DIR").unwrap();
let nimcache = format!("{}/nimcache", out_dir);
let mut nimcmd = Command::new("nim");
nimcmd.args(&[
"c",
"--app:staticlib",
&format!("--outdir:{}", out_dir),
&format!("--nimcache:{}", nimcache),
"--genDeps",
]);
if env::var("PROFILE").unwrap() == "release" {
nimcmd.arg("-d:release");
}
let target = env::var("TARGET").unwrap();
if target.contains("msvc") {
nimcmd.arg("--cc:vcc");
}
nimcmd.arg(main_file).status().unwrap();
println!("cargo:rustc-link-search=native={}", out_dir);
println!("cargo:rustc-link-lib=static={}", name);
let deps_file = format!("{}/{}.deps", nimcache, name);
for file in fs::read_to_string(deps_file).unwrap().lines() {
println!("cargo:rerun-if-changed={}", file);
}
}
Note: the path handling in this script is a bit primitive (this may not work on windows…)
Then it could be just something like
Command::new(nimble).arg("bindings").status().unwrap();
println!("cargo:rustc-link-search=native={}", build_dir);
println!("cargo:rustc-link-lib=static=constantine_bls12_381");
// …
// Maybe, to avoid full recompilation
println!("cargo:rerun-if-changed={}", some_file);
In case you are not aware of it: you can use bindgen in the build.rs with your generated headers.
Reporting back,
Infiltration successful, we can now accelerate slow Rust with fast Nim ;)
For debugging Cargo gobbles up the output of any command not prefixed by "cargo:" unless you use cargo build -vv but that is very verbose.
Congratulation @mratsim.
It would be great, if you have the time and will, to explain your choice of Nim over Rust for this package. I'm curious to hear the reasoning of creating a Nim-based Rust package instead of using Rust directly.
Buckle up and let grandpa tell you a story.
Constantine started in July 2018 as "hardy". We, at Status, identified the need of elliptic curves in Nim and so cryptographically hardened bigints.
The goal was to implement or port:
All of those were implemented in Python in https://github.com/ethereum/py_ecc
After spending a couple days on it https://github.com/mratsim/constantine/commit/12cc5dc, it was obvious that it was way to over my head and to resource intensive to actually implement it from scratch in Nim.
So, instead, we:
Then early 2020, I switched to a personal project, it was only 39 commit at the time, without even multiplication implemented. https://github.com/mratsim/constantine/tree/5dc9792
And I kept it as Nim because, I'd rather write Nim in my spare time.
Now on the unique selling points of Nim compared to Rust for cryptography:
- This has made Rust a non-starter in 2018:
- https://github.com/rust-lang/rfcs/issues/1038
- https://internals.rust-lang.org/t/pre-rfc-generic-integers-uint-n-and-int-n/7641
- https://github.com/rust-lang/rfcs/issues/1038
- https://rust-lang.github.io/rfcs/2000-const-generics.html
- Rust doesn't support static if (when in Nim), I use that pervasively in the library.
- You cannot parse and encode some hex compile-time bigints as well which makes things look like magic when reading code.
Having a generic configuration for Nim to generate types and compile-time properties is just so ergonomic and clean: https://github.com/mratsim/constantine/blob/0a17002/constantine/math/config/curves_declaration.nim#L245-L263
curve BN254_Snarks: # Zero-Knowledge proofs curve (SNARKS, STARKS, Ethereum) bitwidth: 254 modulus: "0x30644e72e131a029b85045b68181585d97816a916871ca8d3c208c16d87cfd47" family: BarretoNaehrig # G1 Equation: Y^2 = X^3 + 3 # G2 Equation: Y^2 = X^3 + 3/(9+𝑖) order: "0x30644e72e131a029b85045b68181585d2833e84879b9709143e1f593f0000001" orderBitwidth: 254 cofactor: 1 eq_form: ShortWeierstrass coef_a: 0 coef_b: 3 nonresidue_fp: -1 # -1 is not a square in 𝔽p nonresidue_fp2: (9, 1) # 9+𝑖 9+𝑖 is not a square or cube in 𝔽p² embedding_degree: 12 sexticTwist: D_Twist curve BLS12_381: bitwidth: 381 modulus: "0x1a0111ea397fe69a4b1ba7b6434bacd764774b84f38512bf6730d2a0f6b0f6241eabfffeb153ffffb9feffffffffaaab" family: BarretoLynnScott # u: -(2^63 + 2^62 + 2^60 + 2^57 + 2^48 + 2^16) # G1 Equation: y² = x³ + 4 # G2 Equation: y² = x³ + 4 (1+i) order: "0x73eda753299d7d483339d80809a1d80553bda402fffe5bfeffffffff00000001" orderBitwidth: 255 cofactor: "0x396c8c005555e1568c00aaab0000aaab" eq_form: ShortWeierstrass coef_a: 0 coef_b: 4 nonresidue_fp: -1 # -1 is not a square in 𝔽p nonresidue_fp2: (1, 1) # 1+𝑖 1+𝑖 is not a square or cube in 𝔽p² embedding_degree: 12 sexticTwist: M_Twist
- I use it to implement spawn/sync in my threadpool and a compile-time assembler to properly use MULX/ADCX/ADOX instruction for bigint multiplication (70% faster than GCC generated code)
- The compile-time assembler usage is actually quite maintainable and debuggable as I have access to for-loops, control flow like if/then/else and even functions. Writing assembly is made quite easy this way (see https://github.com/mratsim/constantine/blob/0a17002/constantine/math/arithmetic/assembly/limbs_asm_mul_x86_adx_bmi2.nim )
I was also hoping for DrNim to bring some more formal verification benefits as well. I was also excited by the concepts potential, though they did make me like 2x slower than necessary at one point (https://github.com/nim-lang/Nim/issues/16897). And I'm still waiting for view types to be fully fledged.
To conclude, for cryptography Nim is at an unique crossroad of safety (type system, destructors, borrow checker), escape hatches (assembly), DSLs to safeguard those escape hatches or wrap low-level in high-level (threadpools), syntax (overloading and custom operators), integer & enum generics support and compile-time evaluation of anything support to make me extremely productive.