Hi everyone, this is my first day of learning Nim and I love it so far!
For my first program I wrote an even simpler version of the "Nim is simple" example from the nim-lang.org main page:
var count = 0
for line in stdin.lines:
count += 1
echo(count)Coming from a Python background I wanted to compare the two languages syntax-wise and performance-wise, so I also implemented this simple program in Python and ran it. To my surprise, the Python version was much. much faster!
Python: 8.8 secs
Pypy: 2.5 secs
Nim: 1 minute 13 secs
Both programs output the same number of lines count. I have run everything several times and in different order, but the result is the same every time.
I compile the Nim program with
> nim c -d:release count.nim
I run it with
> time cat ~/data/tweets/* | ./count
It is 13 data files, each about 100 MB.
I am on a MacBook Pro running OS X 10.10.3 Yosemite. I installed Nim using Homebrew:
>brew info nimrod nimrod: stable 0.10.2 (bottled), HEAD http://nim-lang.org/ /usr/local/Cellar/nimrod/0.10.2 (312 files, 8.4M) * Poured from bottle From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/nimrod.rb >nim --version Nim Compiler Version 0.10.2 (2014-12-29) [MacOSX: amd64] Copyright (c) 2006-2014 by Andreas Rumpf active boot switches: -d:release
I also tried downloading Nim from the download page and building it myself, but with the same result.
What am I doing wrong?
I ran into just the same issue. Previous thread about this: http://forum.nim-lang.org/t/503
The main problem is the way Nim handles newlines. It has compatibility with normal and Windows newlines. Otherwise it could be sped up a lot. Also, the compiler doesn't like top level statements, you should put the for loop into a proc.
This PR should have made it faster: https://github.com/Araq/Nim/pull/2329
Unfortunately it doesn't work on Windows yet. For other platforms you can try the devel compiler branch (or wait for the 0.10.4 release).
Thank you!
I downloaded the devel branch and the run time went from 1 min 13 secs to 4.9 secs! I also put the code in a proc and the time went down to 4.3 secs.
Also, the compiler doesn't like top level statements, you should put the for loop into a proc.
Can you elaborate on that? Is that still the case today?
Top-level variables are globals and so not optimized by the compiler and also never released if they do allocate. They are fine for quick scripting or quick prototyping.
Similarly, top level statement are not optimized.
So it's more of a performance issue than correctness issue. That said, reading from stdin involves IO which is an order of magnitude slower than the rest of the code anyway.
Why is that top-level code is not optimized?
Simply put, we just don't have anyone volunteering their time into making that happen. The walkaround is really simple (wrap the code in a proc), so the issue is low-priority and development time is spent elsewhere.
I do not know how the compiler works except for the fact that it generates C code (correct? is there an overview of the compiler somewhere?).
Other than this slightly outdated document, there aren't much about the compiler innards unfortunately. But here is a simplified view into the compilation process:
That is what I was wondering too. Elixir, for example, does that, it wraps everything into a function built by hand using low level code. I explain it in this post https://medium.com/@fxn/how-does-elixir-compile-execute-code-c1b36c9ec8cf. (I do not know if the example translates to Nim, though.)
However, as @leorize said, even if that was possible, someone would need to write it.
Why can the compiler not wrap top-level code into main automatically?
It could do that but currently it doesn't. Also, with arc/orc the performance impact of global variables diminished.
Globals are, well, global. Most nim top level modules could also be imported by a client module. In that scenario, their (exported) globals would need to be available to client code, which they would not be were they auto-wrapped in a proc scope. Then also, any *-exported names which would ordinarily be fine at global scope need to be de-exported or the error about exporting not at top-level needs to be suppressed.
So, at a minimum, the compiler would have to use the knowledge of starting from myTopLevel.nim, not just blindly proc main() = stuff; main(). However, the user can probably just blindly do that because the user knows more about overall code usage context, like that some module is "only ever a program top-level one".
Globals are also subject to the experimental code reordering which I doubt works in inner scopes. There may be several other side aspects as well. So, while I understand the impulse to have a compiler flag nim c --make-my-code-faster, it's sometimes wiser to offload to users, and I suspect this is one of those times.
I have rarely seen wrapping all the logic in a proc make that much of a difference in performance anyway, as @mratsim guesses at here. It's just a trivial thing "to try" as @leorize said. You should really always measure both ways before concluding there is a performance issue, though.
it's sometimes wiser to offload to users, and I suspect this is one of those times.
Agreed.
The last section of top level statements of the main .nim file is turned into a proc. A section is a list of statements that is not disrupted by proc, func, ... declarations.
It would be pretty easy to implement. But it's totally not clear that it's worth it.
It would be pretty easy to implement. But it's not clear that it's worth it.
Given the amount of times I've seen this same conversation unfold in this forum it might be worth it :-)
@Araq: We could come up with a simple rule like: The last section of top level statements of the main .nim file is turned into a proc. A section is a list of statements that is not disrupted by proc, func, ... declarations. It would be pretty easy to implement. But it's not clear that it's worth it.
Please, do NOT do this! Nim is already a big language. Special treatment of such a dubious corner-case would be bad. Global variables, unless they are compile-time constants, come with their own set of problems and better be avoided in well designed code. Besides, as you already mentioned --gc:arc or '--gc:orc' (soon to be default) alleviates the problem.
Global variables, unless they are compile-time constants, come with their own set of problems and better be avoided in well designed code
Sure, but turning a bunch of these into local variables seems pretty tame to me.
@Araq: Sure, but turning a bunch of these into local variables seems pretty tame to me.
This is exactly my point. Nim should not turn globals into locals or anything else. Nim's treatment of globals is good enough as is, no further actions required. Might be gently advise users to minimize their use of globals altogether. As far as scripts are concerned, it is not really that hard to manually write and call a main function if needed :-)