As I understand it, Nim's development has always been done with tooling in mind.
This has some wonderful advantages: uniform function call syntax, awesome IDE support built-in, unobtrusive syntax, convenient import system, etc. All of these would not be possible without proper tooling in the first place and it is a delight to code in Nim in vim with all these.
But at the same time, I am a bit worried the lack of namespaces (or any other strong form of encapsulation, by default). Let me insist on that last point: Nim can be as encapsulated as I would wish it to be, but the majority of the code out there just import Foo.
How is that a problem? In two specific cases:
That said, I am not for explicit import by default either: I think we may find some way to keep the convenience of the current system while improving on the two points above. What do you think?
Edit: I am well aware that similar issues has been discussed already with strong opinions about what the "solution" should be. This invariably ends by Araq stating that there is no problem since an IDE is just the right tool to code. While I agree with that, I could also see in practice that it is possible to ease our understanding of a code base through smart syntax or even tooling (go fmt comes to mind): in practice today, it's not hard to wrap one's head around almost any non-trivial Python code and figure out pretty quickly what we are actually looking for. Julia (in which I have more experience) and Nim really don't convey the same feeling. I find it harder (but not overwhelmingly hard!) to focus only on some subpart of a program: everything seems much more interwoven but that impression may vanish in Nim though (but dedinitely not in Julia).
To sum up, I am not saying that there is a problem here, I am just asking to think about ways to improve on encapsulation and readability for systems without IDE (github and the like most notably) "for free" (i.e. without making development in a proper environment a worse experience).
This invariably ends by Araq stating that there is no problem since an IDE is just the right tool to code.
Well to encourage further discussions: If a decent proposal arrives, I am willing to accept and even implement changes.
Thank you for taking the time to write this up. Unfortunately I don't have any suggestions on what we can do to improve the current situation.
The solution? Prefixing or more-explicit-but-far-less-obvious naming. This is exactly what's happening in Julia (which shares a lot of concepts with Nim) and the result is not nice: package writers change the names because package users cannot be bothered by the compiler saying they must use full name resolution for that specific function OR import it as something else themselves.
That's interesting. Could you give a bit more details about how this works in Julia, perhaps with some code examples?
Well, first things first, in Julia you have explicit modules. Which means that the vast majority of the files start with the module keyword and end with an end keyword (NB: blocks in Julia are ended by end).
When one wants to include some external module, one used to have to include the file containing the module (a.k.a. know where to look) and then import the module itself. Nowadays if module Foo is contained in the file Foo.jl, the file inclusion is done automatically (but the module boilerplate is here to stay).
Anyway, the official way to include a module now is by making use of either using or import. The first one, using Foo is Julia's equivalent to Nim's import Foo while Julia's import Foo is the equivalent of Nim's from Foo import nil.
Said otherwise, if module Foo is imported by import Foo in Julia, one can access any symbols from Foo using its full name (e.g. Foo.bar()) whereas using Foo imports the exportable symbols from the module in the current namespace. In Nim, you do that by marking any symbol with * while in Julia, there is an explicit export keyword. Note that there is no concept of privacy in Julia: exported symbols define what is imported in the current namespace by using; in particular, the other symbols are accessible if prefixed by the module name.
In practice, using Foo is almost always used, which can be easily justified by the fact that most tutorials and the quasi-totality of the examples in the manual use it exclusively. For Julia demographics, this is not a big problem as most "technical computing" people tend to write short length/lived scripts anyway so maintainance and ease of code reading by others is not really a concern.
As Julia is built on a JIT, most of its analysis is done at runtime. It used to warn the user about ambiguous definitions at import time but now it does only when one tries to actually use an ambiguous symbol. This has the extra advantage (for a JIT-ed language) that the complete function signature can be used instead of its name to determine whether or not this is a problem (just as Nim does).
In addition to the JIT, note that Julia actually allows to pass default values to functions which makes things even more complicated even when the signatures do not match to begin with:
function f(x::Int, y::Int=1)
x.+y
end
function f(x::Int)
2.*x
end
What should f(2) give?
From my perspective, it seems like Julia and Nim have quite a lot in common: they are definitely different but they made similar choices in their import system which is why I find it relevant to compare the two.
Or even if they do, such warnings are considered bugs by developers and are therefore "fixed" by resorting to different methods (look at these pages if you are not convinced)... prefixing, opening bug reports on other developer's package, renaming or even not exporting:
Let's just not export them, then people can call Lazy.cycle etc. if they need it.
Prefixing for instance is quite common, e.g. (and I can assure you that this is only a minute sample, look for "export"-ed function names):
Is really wavread that much better than wav.read? For a Nimean (or whatever), the answer is most certainly "no" as wavread, wavRead and wav_read (to cite a few) are already considered identical in Nim. So, if one is ready to write wavread or wav_read, why not wav.read? Namespace prefixing has also another advantage: if all the functions are defined as wave_file_codec_* (wave_file_codec_read, wave_file_codec_write, ...), one would have to write a custom macro to rename them to something sane whereas a WaveFileCodec module can always be trivially renamed wav (wav.read, wav.write, ...).
But all that said putting namespaces everywhere, especially in small or "well-designed" programs, brings very little value with regard to the potential increase in verbosity.
And unfortunately I have no straightforward solution either.
This isn't the first time I've seen the module prefix debate.
I don't really see what the issue is myself: you can force prefixes with the "import ... nil" (IIRC) if you want. What really needs changing?
When you write code, you feel necessary not to use "common names" for your procedures: if I have a module dedicated to reading one kind of files, I would avoid using read/open/parse/etc. because the probability that read(string) is already imported somewhere is pretty high.
Hmm I don't seem to have this problem. Perhaps it's because I'm using VSCode (does that count as an IDE?)
I mean, one of Nim's strengths IMO is the ease of overloading. If I had a module dedicated to reading one type of file I'd probably have a type that I'd pass to it, aka:
proc open*(myFileObj: MyFileType, filename: string) =
or whatnot. Then I'd just call myFileType.open(filename).
But equally if you wanted to use myFileModule.open(filename) then why not? That's fine too.
The last thing I personally would want is to have to prefix modules all over the place, but you can always specify in the import if you want to force this, anyway.
When you read code, you always need a full-fledged IDE. I am not arguing that Nim should be conveniently writable without an IDE... certainly not! However, I would like to be able to quickly be able to get an idea of what is coming from where when looking at a package on github instead of using a full-sized computer with a properly configured IDE each time I dare reading code.
One time, I broke my nimsuggest (think it was when it moved back to main nim instead of being a nimble package or something?) anyway, that really sucked because I lost my precious control-click to go to definition, and parameter suggestions. But then, if you want your code to be specific without any of that, can't you just use module prefixes anyway?
I don't know, maybe I misunderstand but can't really see what the problems are? Are you thinking it'd be better if module prefixes are mandatory? Because then you can use the import nil or whatever it is, surely?
EDIT Can we provide aliases for module imports currently? Perhaps that would help being more specific without having to write long module names everywhere. Or, maybe it'd just lead to a.theproc b.anotherproc everywhere.
May I ask why you plan to make self implicit? Is this because your eyes are tired of seeing them or because your fingers are tired of writing them? (or both of them?)
Neither. Because relative addressing (param.foo) should not be less convenient than absolute addressing (global variable foo).
It also helps refactoring. The Nim compiler contains many global variables that are migrated to become fields of some context parameter, so that the Nim compiler can be used better as a library.
@coffeepot: Please don't make this a prefix vs. no-prefix debate. I may not have been clear but this is really not the purpose.
The problems were stated from the start:
About things I stated above, I was merely realising that namespace prefixes may not always be more verbose and actually have some advantages. Actually, up until an hour ago, prefixing the names of a procedure (like wav_read) seemed a good idea to avoid the problem until I "said" it out loud. But I am well aware that forcing people to use explicit namespaces just won't work, especially with Nim!
I think Nim just might be the first major post-paper programming language, one that embraces the fact that programmers no longer deal with code the way they did in the past. Making your code readable in notepad / print-out / cave-wall format has tradeoffs that are no longer worth it. Pretty soon even every tile in your bathroom will be a touchscreen tablet!
What I think we need, for starters, is a good interactive Nim code reading environment that takes Nim code and renders it not just with ideal syntax highlighting but with tooltips, code navigation features, etc. Mouseover / touch / fingerover / gesture at / whatever an identifier, and you see its whole genealogy; select a module, and see not only its contents but what other modules use it; select a type, and you see everything that deals with it; etc. This can be done as an in-browser JS app, but it would be even better if done on top of something like nimx.
IDEs are complicated because there are many of them and they have different extensibility opportunities, but the same code viewer can work for anybody.
Perhaps the next step after that would be an interface for writing / modifying code in a structured way, one piece at a time.
Hi, I just had an idea (that may be terrible :/) that could help slightly refocusing the thread.
I completely embrace the idea that in this day and age, there are trade-offs that are no longer worth it. But once again, there is Github and the rest... No grep there...
So instead of forcing the developer to put hints (e.g. explicit module names) for future readers to understand the code structure more easily, why not having the compiler do that for you? So my suggestion (which again, might be a very very bad one and it is fine to say so) would be to have compiler comments (in the source code).
These comments could be autogenerated at each compilation/commit and introduced by a different symbol than comments provided by the developer. The use of a different symbol brings nice properties: the compiler doesn't have to guess what was generated by itself, it knows it; the user is warned that changing such comments is useless; there is no overlap between what the developer wrote and what the compiler added and more importantly, the very first thing a Nim-aware IDE could do is to strip them (as such comments would provide inferior, but otherwise useful hints).
Such comments could be used for instance to emulate explicit imports but only as needed:
import Foo
discard add_two_and_display(5)
could be translated to something like this (taking "!@ .. @!" to declare compiler comment):
import Foo
!@ add_two_and_display(int): int -> Foo/bar/baz.nim @!
discard add_two_and_display(5)
As the compiler knows about all things, such compiler comment should obviously not list every symbol exported by the module, but just the ones actually used. Since it is generated at each compilation, the list can be kept up to date. And obviously this kind of things:
import Foo
discard Foo.add_two_and_display(5)
should not generate any compiler comments since the hint is already there, provided directly by the developer.
Likewise, such concept could be extended to other areas in the program (e.g. state what is returned from a procedure if there is no explicit return statement or the result of executed code at compile time like in when statements)... or not... (again, I don't see everything: the idea could be very bad).
The few negative points about this that I can see are:
The last point is obviously the main problem with the approach. But again, modern IDEs and text editors may behave better than what I think.
For plain text viewing, perhaps a cmd line tool that uses nimsuggest to edit sources and automatically prefix the module to everything for you (or strip them) would be useful. Could have rules that weight whether to add prefixes based on conditions like if it's third party (eg in the nimble pkgs folder) or only add module prefixes if there are other items with the same name in a single module. This could then be added to your build process, if explicit declarations are desired.
Anyway, just brainstormin' :)
As long as this mechanism is widely used in the community, I like the idea. The concept of rules (that may possibly be tweaked) is also great as it would allow people to choose what they want displayed as opposed to what others forced onto them. In that sense such a tool could become even more powerful than go-fmt!
For this to work reliably, one may need to work at the AST level which has the extra advantage of standardising what's written on disk (like what go does with go-fmt). From the outside, the code would seem pretty standardized which is always nice, especially in larger teams. But at the same time, once in your IDE, the code would look like what you prefer (unlike what go-fmt currently does).
May I enquire which problems that might be?
Editors, backup systems, dependency checkers, and version control systems are sensitive to timestamps. Changing one file could result in changing the compiler comments in every file of the project, resulting in a lot of unneeded and unwanted activity. And suppressing changing the timestamps can result in inconsistencies.