Nim is a language that has intrigued me for years. I tried and dabbled in Nim. Struggled with various things because of the difference and never pushed through the barrier because of my mindset. I really like Nim the language, a Python-like syntax (but fixed a lot of Python's ugliness). But I could never quite understand how Nim could be a language for everything without a proper amount of interactivity.
The static compile cycle with no interactivity, no REPL. It was hard. But recently I began thinking about this style of static, compiled development vs interactive REPL style of development available in Python, Pharo and others in a new way.
An app that I am developing in Pharo (Smalltalk) is server side and I do not know of any implicit interactivity that I need for it. It is primarily start up, run continuously, stop. It produces some output that I can view.
As I looked at the apps I am developing. I have data acquisition, generation, and transformation. That most of my data exploration was not during any of those phases. In general any interactivity I would have with my live app would be explicit interactivity and not ad hoc implicit interactivity.
In this case I would be explicitly writing an UI either CLI, GUI or Web app for explicit interaction. You do not need a dynamic, interactive language and environment, or REPL for that. And even in those types of languages, you would still have to explicitly write your CLI, GUI or Web app.
For those areas that I really want dynamic, interactive, exploratory environment or REPL. I could explicitly write into my app to export any data which the app acquired, generated or transformed that I would like to explore. Many apps already do this when they log events and other data they export for external use. Then in whatever tool I thought best, Pharo, Python, SQL, ..., I could do all the ad hoc exploration and interactivity I needed. With the insight I gain, I iterate my compiled app and its explicit UI.
I realize that I didn't need implicit, ad hoc access to the internal state of the app as much as I had been used to. And that I could learn to do things differently. While there is not anything wrong with that kind of access that a dynamic, interactive environment or REPL can provide. It frequently has its costs. The apps are frequently larger is disk space, harder and often difficult to deploy, not memory efficient, slower, ...
It can be done differently, and with that different, can come other benefits. Statically compiled app which can be faster, smaller, easier to deploy, memory efficient.
The nice thing Nim brings to this new way of thinking (mindset), is that I can still do this in a language with a nice syntax, and nice features. I can take advantage of the essence of Nim. Efficient, Expressive and Elegant.
Nim is a nice middle ground between the dynamic languages like Python and Pharo, and C/C++, Rust, and others.
I just wanted to put these thoughts out there for anybody who struggled like me to hopefully acquire a mindset when using Nim to overcome those struggles. I know for many here, these are already understood and known facts. But I know that someone crossing a divide, static/dynamic, oo/procedural/declaritive/functional, might have a hard time seeing the benefits of the other side. I have seen people struggle who are very familiar with static, but can't see why anyone would want dynamic.
And yes, I do know of the inim, nimscript, nimscripter and the other means of helping bring interactivity into Nim. I will explore those later, after I learn (Master) Nim proper.
If you have anything to contribute to help me and others cross that divide and learn a new way of thinking. A way of thinking which helps us with the Zen of Nim. There should be one and only one programming language for everything. That language is Nim. Please do so.
And now to help me cross this barrier with my new mindset, is my new hard cover copy of Mastering Nim which was just delivered.
Interactive programming when writing Nim's macros might be amazing.
But in general REPL-driven development throws away valuable test cases as it's so easy to change everything. Still a Nim REPL would be nice. Esp when not done in a terminal but within a real editor.
IMHO a REPL is table stakes in some of the domains in which nim is competing with other languages that provide it. REPLs have their faults, but a lot of people, particularly engineers that need to program to get their job done but aren't computer scientists, are used to these REPL-based workflows (which have a lot of advantages in some areas) and expect to be able to use them, otherwise they are just not interested (I'm thinking of people who currently use Matlab or python for data analysis or simulation, for example).
Beyond that, a proper nim REPL would be awesome in multiple contexts:
If you interactively change something and then delete the code that changed it, the change will remain – but there's no longer any trace of where it came from.
That tricky problem can be solved with
proc after_delete(code) =
File.append 'if_you_wonder_where_it_came_from_look_here.log', code
REPL-driven development throws away valuable test cases
Never heard about that problem, nether in Ruby nor in JS/TS :)
I actually dislike the interactive model of languages like Pharo or Python+Jupyter because it introduces a kind of hysteresis. If you interactively change something and then delete the code that changed it, the change will remain – but there's no longer any trace of where it came from.
That what's annoys me the most when I use notebooks. However, if one uses Julia (my current language of choice for exploratory programming), Pluto offers a very nice solution.
Have a look at the first ~5 minutes of this video, what Fons (Pluto's main developer) achieves with just a few lines of code written interactively is impressive: <https://youtu.be/Rg3r3gG4nQo&t=147>
Im trying to achieve that experience in UE: https://twitter.com/_jmgomez_/status/1704534758263239105
You can change the "state" of the world from a dll or the vm, it doesnt matter. It persists :)
At any instant, the program state is completely described by the code you see.
I think that is generally not true in Pluto. Citing from https://github.com/fonsp/Pluto.jl/issues/316:
# Example (each row in a separate Pluto cell):
a = [4, 3, 2, 1]
a # prints [4, 3, 2, 1]
sort!(a) # prints [1, 2, 3, 4], but does not update previous cell
And I believe that if instead of sort! you used something like append_zero! (which does in-place modification of the given vector and appends number 0 to it) then a's value would depend on the number of evaluations of append_zero!(a).
For instance, considerer the Twitter example:
proc onEditorTick(self: ATickEditorPtr, deltaSeconds: float32) {.ueborrow.} =
let rotator = self.otherActor.getActorRotation() + FRotator(yaw: -2)
discard self.otherActor.setActorRotation(rotator, false)
The implementation to getActorRotation is:
proc getActorRotation*(self : AActorPtr): FRotator =
var call = UECall(self: 0, value: RuntimeField(kind: FieldKind(0),
intVal: 0), kind: UECallKind(0), fn: UEFunc(name: "GetActorRotation",
className: "AActor"))
call.value = ().toRuntimeField()
call.self = cast[int](self)
let returnVal {.used, inject.} = uCall(call)
when FRotator is ptr:
if returnVal.get.intVal == 0:
return nil
else:
return cast[FRotator](returnVal.get.intVal)
else:
return returnVal.get.runtimeFieldTo(FRotator)
Which then goes into https://github.com/jmgomez/NimForUE/blob/84f986f8e1a5531d679aa00bf4c7bdb50f0ef0fc/src/nimforue/vm/uecall.nim#L245 and figure how to call the native function.
The ueborrow pragma, what it does is that it replaces the implementation of an existing function, meaning that when called the function inside the VM is the one that will be called instead of the real implementation. It's a bit more convoluted but happens in here for those interested: https://github.com/jmgomez/NimForUE/blob/84f986f8e1a5531d679aa00bf4c7bdb50f0ef0fc/src/nimforue/vm/nimvm.nim#L181
If it was trivial to do and/or I knew exactly how to do it, I would have written one by now, heh (I have multiple toy REPLs, but they all have certain problems).
For an example of a REPL of a compiled language - C++ - you can check out cling:
It effectively uses the LLVM JIT compilation facilities to JIT compile your snippets of code. So this is one possible avenue, but we need a JIT compiler for general Nim code for that. This is why I got interested in helping with the ORCv2 JIT for nlvm (last year? this year? sorry @arnetheduck for not continuing, but I've been too busy :( ). arnetheduck has implemented it in nlvm by now though, so at least for nlvm it might be quite doable to build something like cling now. Maybe he can comment on what is missing for that / how much work it would be. In a similar vein one might consider using libgccjit to JIT compile Nim code (as to avoid the heavy LLVM dep).
Another approach would be using hot code reloading. After the feature was added I tried to do some experiments with it here:
https://github.com/Vindaar/brokenREPL
but unfortunately HCR was way too buggy and under documented (so I'm not sure if part of the issues I was seeing was me using it wrongly). I guess one could try to fix it up. A large chunk of the work has already been done after all, but understanding the details of the current implementation might be a time sink in itself.
A further approach that is essentially very similar to HCR, but more manual, is to compile pieces of code into small shared libraries that link to each other. I have some code lying around that does that. While it 'works', you still have to pay the time to compile the small snippets to a shared library. So every command also comes with a decent delay of 200-400ms, which is quite annoying. For this approach I haven't really thought much / come up with a decent way to handle state though (which generally is a bit more tricky in a statically typed host language; if the REPL itself is to save the state { instead of saving the state as "code" in a shared lib } I guess you need to use some serialization to store arbitrary data and deserialize in the calling procedures of the shared libs? Not sure honestly).
If incremental compilation was a thing already, the time to compile such small snippets of code should go down to pretty much zero though. Something that I assume touches on IC would be to use the Nim compiler as a library. Then keep the module graph around so you don't have to do the sem pass of the system library and the modules you import again and again. Essentially only the new code from the REPL would go through the sem pass and finally you would generate the C code + call gcc/clang to produce a shared library. What I've tried in those directions runs into all sorts of trouble due to producing duplicate C code etc. I assume this is just because the Nim compiler was not meant for this (until IC is here?) - or it might be me not knowing how to use it as a library correctly. In this case though most code snippets would also compile in << 100ms.
Anyhow, for the time being I don't have the time to invest into any of these approaches (but tbh I would love to!). Still, I do think if someone could work on this full time for a while (what is a while? I don't know, seems like 2-3 months should be plenty, but what do I know) it should be quite possible to build something nice.
I believe it's not expected because in Pluto order of cells should not matter. They write:
Cells can even be placed in arbitrary order - intelligent syntax analysis figures out the dependencies between them and takes care of execution.
in Readme. But reality is different.
Or you can create notebook with cells
a = [1] # Output [1].
a[1] += 1; # No output.
a # Output [7] (after evaluating the second cell 6 times)
a # Output [8] (evaluate the second cell one more time and then evaluate this cell)
here is a picture.