I've also been writing a tutorial-like series of posts on every step of the process, although it's a bit behind the code. I hope you enjoy it!
Thanks! This is intentionally not going to be a Unix-like OS, so no POSIX :)
I haven't thought about the API yet, there's much to be done before I get to that stage. But I have some rough ideas:
All of this is obviously just ideas at this point. But it's what I'm aiming for.
A soon as you type your channels you run into composability problems as you cannot easily wait for either Channel[T1] or Channel[T2].
Rather than avoiding signals I would embrace them as a general purpose interrupt mechanism. And an interrupt can be scheduled on a different thread. And then I would base the entire API on top of it. This way everything is async in a natural fashion. The context switches become more visible in the code. No channels, sinks or sources required.
For example:
proc requestReadOp() =
var buf: SomeBuffer
req(url = "somefile.txt", op = fmRead, mode = Blocking, env = addr(buf), oncomplete = proc (env: pointer, src: pointer, size: int) {.cdecl.} =
copyMem env, src, size
)
This way the OS only need to provide the callback based version. Well ... I suppose it's close enough to a traditional event loop.
And an interrupt can be scheduled on a different thread
This sounds very close to scheduler activations, a formalized way for an upcall mechanism from the kernel to userspace. I need to give this more thought, but it is an interesting idea.
A soon as you type your channels you run into composability problems as you cannot easily wait for either Channel[T1] or Channel[T2].
I'm sure I'm going to run into all sorts of problems :D The idea is to start from user space with typical use cases (e.g. implement a simple grep) and start fleshing out the API and kernel support based on what would take to make it work. I'm sure the first iteration will be crappy, but it's useful for learning.
Capability-based Security
woot!
This looks fantastic! one for https://github.com/dckc/awesome-ocap
I just dropped by to see what's up in nim-land lately, cuz I'm thinking about single-executable deployment but not go or rust. Finding this is fun!!!
Here's hoping for time to dig in more...
Such an interesting project with awesome documentation.
Keep up the great work!
Offtopic:
As a guy who absolutely knows nothing about OS / kernel design, there is something that bothers me a lot: Why do I always need to close things? Why can't the kernel or the PL do that for me? I mean, at some point it just goes out of scope. Why do I need to manually close things? The things are files, network connections, channels and even mounting a filesystem.
The Nim PL can do that for you with the destructors feature. However, most of the standard library has yet to be ported to Nim v2 to do that.
The OS cannot do it for you as the OS does not know the type layouts at runtime, for the OS the running process is mostly pages of random bits.
Not much right now :)
One thing that is unorthodox about it is that it's a single address space operating system, i.e. all tasks share the same 64-bit address space. Why? Because I'd like to be able to pass pointers between tasks to shared heap memory (protected by synchronization of course). This should be more efficient than e.g. unix pipes which copies data between processes, and also less fussy than unix shared memory which is not the default and requires special setup for shared memory regions (it also doesn't synchronize access to shared memory).
It's still early days, and I'm doing it for fun, so we'll see where it goes :)
FWIW i always thought the singularity os research project from Microsoft was really cool, fibers, unikernel and a single address space
see: https://en.wikipedia.org/wiki/Singularity_(operating_system)
It sounds similar to what you have in mind.
Indeed, I am familiar with Singularity OS (I read some of the papers). Singularity relies on the language to produce verifiable binaries. Unfortunately Nim doesn’t have this kind of verifiable software fault isolation. The Singularity team created Sing# specifically for this purpose, and it would be a huge undertaking to build something similar or patch Nim to support this model.
In Fusion the protection is achieved through page table mapping permissions, and hopefully at some point I'm going to start adding capability-based security as well.
But yes, I like two aspects of Singularity:
There was a time when operating systems was all the rage, i thought these times have passed. These esoteric and alternative OSes have an exciting facet to them. Anyway, Genode is relatively modern and has capabilities-based security if you want to see how it was done.
sing# may have went too far, unless you're talking about malicious intent between programs. i think when windows 95 was crashing, it was related to the low level hard-to-get-right languages like c and c++.
... and it would be a huge undertaking to build something similar or patch Nim to support this model.
Correct but it's worse than that: Do you want your OS to only be able to run Nim code? ;-)
Webassembly is worth looking into as they have effectively solved the "runs in trusted environment" problems. In fact, arguably, it's a nice solution for how to add memory safety to C or any comparable language.
Correct but it's worse than that: Do you want your OS to only be able to run Nim code? ;-)
Of course not :) But I think the main target of this verifiable code would be code that needs to live in kernel space, e.g. drivers and various kernel components that may be developed independently (think eBPF). User mode tasks of course cannot be trusted, and must be isolated using hardware-level memory protection.