I have successfully solved the challenge from http://rosettacode.org/wiki/Compiler/lexical_analyzer.
Currently this challenge is missing an example in Nim, I'm willing to suggest my solution there. I would like to ask you to review my code and possibly give me some advice on how to write better code. I'm new to Nim and free to suggestions.
The code is ~200 lines so please take a deep breath:
https://github.com/greenfork/rosetta-compiler/blob/master/lexical_analyzer.nim
View readme if needed.
Great work!
Personally I think you should build one using lexbase instead of re as re seems to be too big just for this (the C version is written in plain C without re).
Actually if I could find sometime I'd try to build a lexbase version as it seemed fun :)
lexbase is totally the case! In the challenge it is also suggested to use one variant as a "raw" version and one with a lexing support from the language. I also wrote a version which reads one character at a time and it became very dirty very fast as well as it was hard to debug it, so the 2nd, current version is with re :)
Actually it's overwhelming how much support Nim has for parsing grammar, there's lexbase, re, pegs, strscans.
I've spent sometime making the lexbase version: https://gist.github.com/alaviss/83a48f0344d55efbebcca3bce35e4157
This beast has ~310 LoC, so it's just ~100 LoC more than your re version :)
After writing mine, I noted a few points in your version that could be changed:
P/s: If you are looking to submit your version to Rosetta Code, please submit mine as well, I'm rather lazy creating yet another account :P
Sadly I cannot maintain it much, but shameless plug ahead: https://github.com/Araq/lexim
Lexim compiles regular expressions into no-overhead Nim matching code and can be used with lexbase.
I finally got my hands on it and now it's there http://rosettacode.org/wiki/Compiler/lexical_analyzer#Nim.
@leorize I would also like to say that I really appreciate your corrections, they were very useful, thank you.