This question of making Nim more functional seems to come up from time to time, so I thought it might be useful to summarize "the state of Nim as to being a functional language", at least as I see it...
History
Nim's background story would seem to in no way include any back trace to what have been called "functional languages" of the past such as Lisp/Scheme/etc. The reason for Nim's existance would seem to be to build a better C/C++, which it does quite well to the point where I avoid using any "imperative" language other than Nim other than to learn enough about them so as to compare features and abilities. As a better C/C++, it shares some common goals with quite a few other newish languages such as Zig, the V language, and so on all the way up to Rust, but having learned something about all of these, I reject them in favor of Nim because none of them offer the features I want and they either reject automatic memory management altogether or use a very complex system of memory management that makes them onerous to work with (Rust, I'm looking at you, and you can't even break cycles automatically).
The Tendencies of Modern Languages to be "Safe"
Modern imperative languages have adopted some functional programming precepts in order to enable writing safer code, so most new languages have some way of expressing that a variable should not be mutated (Nim's let; other language's const, etc.), some control or indication of what functions are or are not side effect free (Nim's strict func), support for anonymous functions (lambda's), support for closure functions (which may be anonymous - which Nim has), some form of support for Algebraic Date Types (Nim's variadic objects) and the means to pattern match based on these types (which Nim can do in a limited way). In order to be more "safe" Nim has in the works a version of Haskell's Type Classes and Rust's Trait's called Concept's and a version of Rust's control of mutation called View's that are intended to be easier to use, but they are "opt-in" features and will have to be for the foreseeable future ax they are experimental and no libraries use them
Functional Things that Imperative Languages Lack
However, some features that functional programmers would expect to be able to use such as full type inference and currying/partial application of function arguments (very difficult to implement in current Nim due to function overloading), the ability to eliminate imperative loops by using tail call recursive functions (difficult but not impossible in the current Nim as Nim mostly compiles directly to C or C++ which do not guarantee tail call safe code, so Nim would have to emit code that forces C/C++ to be tail call safe). Also, as long as a language such as Nim accepts mutable (var) function arguments and can return var/lent values, it is not side effect free and it is no use pretending that it is. Another thing that many imperative languages lack, especially those that restrict control of variable bindings in the interest of "safety" is recursive references to variable definitions, even when such recursive definitions can be shown to be safe; in Nim and many other imperative languages, we can only do this (when necessitated by the algorithm) by forward defining a mutable variable and then assigning it with the recursively derived final definition but that breaks being "purely" functional. I have read some proposal from some years ago to make Nim more functional (can't find it now), but it degraded into just using a package of templates and macros that would prevent the use of var arguments, provide better versions of pattern matching (of which there are template/macro packages that do this), and so on, and this proposal died on the vine and hasn't gone anywhere, or at least that I am aware.
Assumptions about Functional Programming that are Incorrect
There are two things that are commonly assumed about functional languages that aren't necessarily true, as follows:
Attitudes Against Functional Programming in Nim
The general attitude seems to be as @Araq's "if we want functional programming, we know where to find Haskell", along with the above two incorrect assumptions about functional languages in general, and the common acceptance that if one wants to write purely functionally in Nim, one can do it by discipline and by using template/macro packages that would help enforce those disciplines. However, I don't know that is correct in that, in order to be efficient, functional paradigms need to be optimized with some compiler support.
Final Observations
It is my feeling that a lot of the recent work in imperative languages regarding "safety" as in Rust's lifetimes and mutation control/Nim's views, wouldn't be necessary or would be so much simpler if the languages were purely functional in the first place, as when everything appears to be immutable, we don't have to control mutation other that for some carefully sequenced mutation wrapper code that can all be elided away when compiled (for instance, IO/ST monad control of mutation).
It is true that all system code can't be written in a purely functional language, as even the Haskell compiler source includes quite a bit of C code for the "primitive" operations, but it is my feeling that many of the types of errors that plague Nim currentlly could have been avoided if the "outer" code had been more functional.
It is still true that Haskell has its fair share of issues in the code base, but part of this is that Haskell is mostly an experimental academic language for investigating extremely advanced functional concepts; I don't propose that a new functional layer be like that at all as to its complexity (I have spent about 10 years off and on learning Haskell and still regard myself at only an intermediate level) but would rather use a purely functional language like Elm as a model with about three or four extensions for things it can't currently do, but with C/C++ output as in Nim's. In Elm, there is no talk of "Functor's", "Applicative's", and "Monad's" although it uses all of those, just not named that way. For instance, in Elm a monad is a container that supports a very specific definition of the "andThen" (which is actaully a "bind") function with examples such as the Task type which is actually something like Haskell's IO monad except that it also has some elements of the Either monad in that it can indicate success/failure. Functional langauges don't have to be complicated in that Elm is so simple that it can be learned in a few days to a week by an experienced programmer, and is used to teach school children of as little as age about ten.
Elm only has about 20 keywords, a few more than that operators, and all of those can be learned gradually due to its nature. It lacks some desirable features such as templates/macros that could be of use for advanced purposes, but there is likely a clean way way to implement those; in addition it also should have some form of Type Classes/Families/Traits for more extensive type use as the current system has some deficiencies that will restrict its progress as a language.
The above is the kind of purely functional language I would like to see as a front end to Nim used as a build chain to generate C/C++...
There must be others out there that feel as I do?
There would appear to be two approaches to be able to use Nim as a code generator but enable writing more functional code, as follows:
Thanks for taking the time to write this message! :-)
I agree with your opinion that some FP languages make it seem that FP is necessarily complicated, whereas that's not the case. (This is like concluding from Java experience that "statically-typed languages are verbose".)
While I appreciate the "FP-like functionality" that has gone into Nim (for example let, immutable function arguments and strict funcs), I have the impression that one feature always seems lacking in programming languages that weren't designed as FP from the start: persistent data structures . Although such data structures could be added to Nim, most of the standard library would still use seq s and mutable hashes, so persistent data structures would always be "second-class citizens."
I've spent by far most of my "programming life" with imperative languages. I learned a bit of Haskell years ago, but didn't stick because of the perceived complexity, but it got me intrigued by FP.
About one and a half year ago I switched to Racket (a Lisp/Scheme variant) for most of my free time programming, and my impression is that it's actually pretty easy to use. I guess the perception that imperative programming is easier is probably mostly due to familiarity.
And yes, some algorithms can be expressed more "naturally" in an imperative style, but then other algorithms can be expressed more "naturally" in an FP style. ;-) Also, many problems where you would immediately think about an imperative solution can be expressed in a functional style just fine. I had this experience several times. :-)
I write Haskell full time and I don't see a reason to make Nim more FP, it's an imperative language with good metaprogramming features and it's best to approach it that way. Persistent data structures are nice but tend to fork a collections library because now you need two versions of all the CR(U)D operations with completely different perf characteristics.
Scala is an example of a language that is weighed down by massive complexity ( and cultural fragmentation ) by trying to support it all, luckily it has become successful so people are paid full time just to deal with that complexity but I think having to spend your funding on accidental complexity is pretty dissatisfying. New languages like Nim can easily avoid this by accepting what they are not.
I _do think it would be good for Nim to make it easy to call Nim functions from other runtimes via a C ABI, orc/arc already get it a lot of the way there. There are some efforts in this space like Genny but something compiler supported and mature that can generate the intermediate C and headers would be a game changer vs. other languages. Now you can write FP using Haskell/OCaml or whatever and FFI out to Nim for hotspot optimization or simply as a glue to a C++ API which by itself is a killer use case.
Build a new language that uses Nim as a back end, using the functional code from a language such as Elm to emit Nim code that would be converted to c/C++ by Nim.
As macros are part of Nim language definition, so is Nim AST definition. If it were possible to plug into Nim compiler to provide an AST created with a functional language, you could have best of both worlds. A functional syntax after writing a parser for a new FP language, and efficiency of Nim binary generation (and of course stdlib and environment).
I don't think that "FP is slow" I think "FP is inconvenient", for example program this in a pure FP language with tail recursion (!) for the for statement and see if you like the result better:
proc traverseDir(dir: string) =
for kind, name in walkDir(dir):
if kind == Dir: traverseDir(dir / name)
else: echo "found a file: ", dir / name
Or try to do this in Haskell (and please understand why it is written in this style):
proc toString(n: Node; result: var string) =
# ^ why do you think FP is obsessed with lists? Because it sucks at `var string`.
result.add "("
for child in n:
result.add " "
toString child, result
result.add ")"
proc `$`(n: Node): string = result = ""; toString(n, result)
And I don't like persistent data structures fwiw; for the most part they assume that I'm interested in the history of a data-structure, but most of the time I'm not.
And yes, I'm aware that modern compilers translate imperative code into SSA which is functional programming, proving once and for all that FP is easier to reason about. Except that SSA conventionally doesn't touch arrays or hash tables. To a good first approximation SSA only deals with local variables (the stack), everything that is on the heap remains in the "imperative" style. That might mean that either imperative really is too hard for a machine to analyse or that a mutable heap is too useful to give up or to downplay its importance like FP does. Or it might mean that a mutable heap allows for a massive code size reduction which is why you cannot turn it into an FP'ish representation easily.
But anyway, mutability isn't bad, shared mutability is, restrict the mutability to a single owner and you get the same bug reductions as FP offers while at the same time you get a system that is much easier to use or at least models the machine much more effectively. Computers are not so much about "computation", they are also IO and communication devices.
Although such data structures could be added to Nim, most of the standard library would still use seq s and mutable hashes, so persistent data structures would always be "second-class citizens."
In my view another case for encouraging usage of concepts, or having the ability to import types/procedures/modules replacing types explicitly.
Sorry, but it feels like you started this thread with an idea like "FP is better and only myths and misconceptions keep it from becoming more mainstream" whereas I think that FP itself is a big misconception -- it ignores the reality of what computers are used for and what you need to do when you "program".
For me programming was mostly solved in the 60ies with Algol and the likes. It was a "general purpose" language which means "imperative" programming and most inventions that came afterwards were nice, very welcome additions (objects, mutability control, sum types, pattern matching, exceptions, macros...) but not essential. Throw away the "imperative programming" and focus on OOP or on FP or on "declarative programming" or on "actors" and the result gets much worse as you removed the essence. That doesn't mean that OOP or FP or declarative programming are bad, it means they are not as important.
Some believe "FP is better and only myths and misconceptions keep it from becoming more mainstream" whereas I think that FP itself is a big misconception -- it ignores the reality of what computers are used for and what you need to do when you "program".
How come it's exactly on my mind ?
Haven't followed the rest of the discussion but just read this and wanted to chime in. It's such a broad and odd statement.
Yeah, well, read the full discussion then. I used the word in the context that I tried to setup. They are all "pretty bad" because they neglect the "essence" of programming. And what that means I've already explained earlier. And if you disagree, fine, but it's not a "broad and odd statement" to be concerned about.
I'd be interested in talking more about this! Here's some thoughts...
I only propose forking Elm, re-enabling a couple of features that are currently turned off, eventually having some form of type classes as an often requested feature (and one the BDFL has been considering for about seven years without rejecting it but also without coming up with an implementation), and converting the AST to Nim instead of JavaScript. The amount of code to change to do with converting the AST is only about 2500 lines, which I think I could handle in a few months.
This is an interesting idea. I've spent a some time trying to recreate the benefits of Elm in other languages, but it always seems like the increase in complexity and the loss of guarantees makes it kind of pointless. I think fundamentally it does two things nicely, concise ADTs and a helpful compiler.
The algebraic data types are very concise to define and unpack with the pattern matching syntax, especially when its a union of mixed types, such as:
type alias Thing =
{ name : String
, kind: String
}
type alias Object2D =
{ width : Int
, height : Int
}
type alias Object3D
{ width: Int
, height : Int
, depth : Int
}
type Selected = NamedThing Thing | SizedObject Object2D | SizedObject Object3DThe second nice feature is that the compiler is very helpful due to the sound type system and the lack of type classes and metaprogramming, so the error messages are always relevant. Selected in the case above becomes a sort of state machine guaranteed by the compiler.
I only propose forking Elm, re-enabling a couple of features that are currently turned off, eventually having some form of type classes as an often requested feature
In Nim, there's definitely some friction in understanding error messages due to the flexibility and complexity of the language. I think the usefulness of error messages and the flexibility/power of the language are always at odds with each other, which is one of the reasons Evan has deliberately kept it simple.
I would like to make the language self hosting rather than having the compiler written in Haskell as it currently is
I think having a language that's not self-hosted can be a feature, especially when the hosted language has different ergonomics than the host language. Like the Python/C relationship has enabled its scientific computing infrastructure to evolve since there's a cleanish boundary between what's "safe" and convenient, versus what's unsafe and fast. It also ensured that the stewards of the language preserved their expertise in the faster host language, though that whole ecosystem is pretty hairy under the hood.
I think FP is really about consistent guarantees. Elm can ensure all of its libraries are behaving in a way that's comprehensible to the compiler for the purpose of showing you cases you've forgotten to handle. A pure function guarantees that it's not going to unexpectedly change your state, but this guarantee isn't much if it's opt-in for other libraries.
I do wonder what something like Elm with nice escape hatches to something like Nim would look like...
There have been many good points here regarding the essence of the topic. Thanks to everyone for contributing. It was a joy to read this discussion.
I wanted to extend one specific part of the topic: what makes life easier in the long run.
I'm sure some of you already know the following video, but I still want to share it, because I think it's a great video on exactly the aforementioned aspect.
It touches the topics of dynamic typing, polymorphism, etc. and compares ways of doing something one way (functional) to doing it the other way (OOP, etc.).
@Akito:
I'm sure some of you already know the following video, but I still want to share it, because I think it's a great video on exactly the aforementioned aspect.
Thanks for reminding us of this valuable video by the BDFL of the Closure programming language. The points made in the video are mostly relevant to the new language that Elm could become.
There will also likely be available Haskell's do notation "sugaring" as applied only to andThen's in the interests of ease of writing monadic chains, and a form of code generation macro definitions that will be more than C's preprocessor #define's but less than Nim's AST macro's which open up a way to abuse in redefining the language.
What you wrote there -- it's pretty much all wrong.
@Araq:
> and a form of code generation macro definitions that will be more than C's preprocessor #define's but less than Nim's AST macro's which open up a way to abuse in redefining the language.
What you wrote there -- it's pretty much all wrong.
I may be wrong, but would be interested in knowing how I am wrong in my thoughts about "macro's", which I am proposing won't really be "macro's" at all but more a form of compile time templating of code generation. The thought about "macro's being subject to abuse as they offer a way to change the language" isn't completely mine but is common to the authors of many languages that don't have them such as Zig, Odin, and the V language that are all kind of "better" Go languages.
It is also along the lines of the thinking of Rich Hickey in his talk as per the video link in @Akito's post above: value simplicity above complexity; although his language, Clojure, does have a form of macro definition and expansion, but which is based on "macro forms" and is perhaps more comparable to what I have proposed as a sort of "super templating" capability. Although powerful AST modifying macros are very powerful in the hands of an able user such as @shirleyquirk, they are also harder to read, understand, and thus to maintain, than general Nim code. Yes, in the Nim documentation it is recommended that one attempt to first use neither templates nor macros, then attempt to use only templates, and finally to resort to macros only if there seems to be no other way, but in practice adepts such as @shirleyquirk may find it easier to just jump straight to the use of macros before fully exploring the other options. This is true not only for Nim but for many other languages that have an easily accessible macro system: Julia and most LISP derived languages come to mind; in these languages (especially Julia) macros seem to be used as a first chosen "go-to" solution, even for basic language features...
In not providing this "super" AST-generating capability unless it becomes clear that it would be a valuable option when the new language is fully implemented, I merely am trying to avoid the complexity of AST-generating macros...
proc traverseDir(dir: string) =
for kind, name in walkDir(dir):
if kind == Dir: traverseDir(dir / name)
else: echo "found a file: ", dir / name
compiles into sort of
traverseDir dir =
forM_ (walkDir dir) $ \case
(Dir, name) -> traverseDir (dir / name)
(_, name) -> print ("found a file: " .. (name / entry))
-- or a bit more verbose:
traverseDir dir = mapM_ go (walkDir dir)
where
go (Dir, name) -> traverseDir (dir / name)
go (_, name) -> print ("found a file: " .. (name / entry))
And string builders are replaced with difference lists, employed by Haskell stdlib's equivalent of $: https://stackoverflow.com/questions/9197913/what-is-the-shows-trick-in-haskell
FWIW one of the appealing things about Nim for me is that it doesn't blindly do things in a functional style "just because".
It would be kinda cool to have a high-quality library of functional data structures (and maybe some referentially unstable data structures that take advantage of view types).