Hi guys,
in the last few days I came across a project called MetaCall
https://github.com/metacall/core
It aims to allow different programming languages to interoperate.
I am not sure of the technical details, but this is similar to efforts such as nimpy, nimjl and genny.
Implementing a Nim metaCall "Port" would allow using other languages from Nim so it's quite useful productivity wise. (but it does run their VM -- so could be slow depending on the language). It's not a transpiler or a compatibility layer mimicking a certain std. It is embedding them as far as I understand.
Implementing the Nim metaCall "Loader" would allow something similar to genny - exporting Nim modules to allow other languages to use them.
I haven't delved further, but I imagine it needs time, effort and (low-level?) expertise to help implement it.
What do you guys think? This approach seems to bring many benefits.
I wrote a small wrapper about the MetaCall C API: https://github.com/metacall/core/commit/f449792d0e09f4536b50d964102812163036a0ba
The test is invoking a NodeJS function from Nim. Here's the GitHub issue if you want to follow the thread: https://github.com/metacall/core/issues/217
graal and metacall aren't related. graal is a compiler-interpreter platform where it keeps the AST around to do speculative JIT compilation and metacall is basically a fancy RPC framework. it's basically using some shenanigans to generate wrappers so you call some function signature and it turns around and makes it a QUIC/http3 request. "loaders" are little processes that launch ex. a python stub and have it run a python script to register handlers for those.
i read through their paper last night. metacall is an RPC framework.
I think maybe the paper is wrongly explained. MetaCall is not RPC (or not limited to), it is an inter-language FFI library. Loaders are literally JITs, VMs or compilers embedded into the library as plugins. Here's a conference about it (focused on NodeJS): https://www.youtube.com/watch?v=1AYYSX9bOMk
The idea of the library is to provide a standard interface for interoperability no matter the languages that are used in between. The main difference between GraalVM is that they reimplement the languages on top of their own Virutal Machine so all languages run on the same infrastructure, like LLVM but for more general purpose (i.e common garbage collection, AOT...). MetaCall just embeds the JITs by allowing them to be written through plugins (loaders) as you said, the support for a language is based on the actual runtimes (V8, CPython, CRuby, NodeJS, DotNet Core) and not by ourselves as in GraalVM.
I wrote a small wrapper about the MetaCall C API for Nim: https://github.com/metacall/core/commit/f449792d0e09f4536b50d964102812163036a0ba
The test is invoking a NodeJS function from Nim. Here's the GitHub (closed) issue if you want to follow the thread: https://github.com/metacall/core/issues/217
I am open to improve the interface and make it template based and type safe abstraction. I hope this can bring a kickstart for Nim community in order to use libraries from other languages, like Python, NodeJS, Ruby, C#, Java, C, Wasm...
The paper mentions using introspection and macros to define the interface; so you're using an implied IDL instead of a declared one (ex. gRPC, Thrift, ZeroC Ice.) Calls are being resolved by other runtimes (called loaders here, but in CORBA it was portable object adapters) and I'm guessing there is some intelligence to determine when a call is handed over to the loader vs serialized on the wire to http/3 and launched somewhere else.
It basically looks like you've reinvented OpenBinder albeit without mounting objects to paths and the IDLs are implied rather than stated up front.
if I had to hazard a guess (i haven't read the source) it's not too far off something like doing a fork/exec and speaking thrift to a child process to handle calls to specific functions in the interface.
we might be missing one another on what the definition of RPC means.