I thought I'd like more eyes on this.
https://github.com/nim-lang/Nim/pull/25499
Current std/parseopt is nice, but it's idiosyncratic and requires getting used to. Some things are hard or impossible with it, though.
Instead of writing another replacement, that will be criticized to oblivion, I just added two other parser modes and a way to configure a custom parser behaviour, all totally transparent, hidden under compile-time switches. So nothing should break and if you need that sense of comfort and familiarity in the CLI parser of your program behaves, now it might be possible!
Hope the discussion doesn't devolve to arguing about personal preferences and standard compliance, though in the case of the new Gnu mode, I'd like to get it to be as close to people's expectations as possible.
So, if you think that:
you might find this PR to your liking. But if you don't, still you might.
-c val as k:c, v:val
+1
In my opinion, the current course of action is very unfortunate; even when shortNoVal is defined, parseopt, in my experience, does not behave as intended. At least, it doesn't behave as I would like. There have already been extensive discussions about the implementation with regard to POSIX, with arguments for and against. Perhaps the various alternative suggestions and discussions also indicate that parseopt does not behave as expected for many users.
The proposed interpretation would, in my opinion, represent a significant improvement without requiring external sources.
I assume --config file would work as well: k: config, v: file
This should work already and will work with every of the three modes, as long as longNoVal is not empty and config is not in it. See lines 266:269 in the test file.
Thank you for the feedback!
According to the discussion at https://github.com/nim-lang/Nim/pull/25499, this pull request will likely not be merged.
Wouldn't it be an option to add another POSIX-compliant CLI parser to the standard library? https://github.com/c-blake/cligen/blob/master/cligen/parseopt3.nim would be one candiate (using another name to avoid the impression that it is superior of parseopt)
I haven't tested parseopt3 yet, because I like to avoid external dependencies, but according to the documentation, it seems promising.
This goes against your more or less uninstructive attempt, but we have xmlparser, parsexml, and xmltree in the std lib. Why not add another CLI parsing lib? Where to draw the line?
That keeps existing parseopt users happy and satisfy users which think that parseopt simply sucks.
My usual comment here will be that if there are opinions, just put it in a nimble package until it's established and everyone agrees on a single mode?
ie the nim standard library is not really the place for "experimental" modes and opinions - these will ossify with only bad options for changing them, with the bad ones causing maintenance overhead and debates without ever being removed and without providing utility.
If something is documented from the start as "experimental", you really want the ability to make releases and improvements of it outside of the regular Nim release cycle and ideally apply it to older nim versions as well.
A much better flow is to establish the modes as something that users want and use (via a package) then extract the actually used improvements and add these to stdlib, if still desired.
As I mentioned just about 8 years ago in this PR thread https://github.com/nim-lang/Nim/pull/7297#issuecomment-371180659 (and perhaps related cross-ref material), I am happy to donate cligen/parseopt3 to become a new std/parseoptN or whatever. That was actually my original idea writing it a little over 10 years ago - std/parseopt* were just not quite flexible enough / had default behaviors my muscle memory at the CLI disagreed with.
I'm actually in the middle of adding/testing a new optional strictness feature (inspired by my last comment on the old PR, actually) with some documentation for how command-people can have a low ceremony/effort blend of both worlds. As far as I can tell, this idea of set an environment variable (with export foo=bar or os.putEnv or however) early/before strict syntax checking (not so unlike Perl's use strict declaration) is novel in the command-line framework field (or was when cligen added it back in version 0.9.46), but talking to security people about shell scripts may also be of very high value in the wider world (if all those CL frameworks aren't too frozen in). I mostly did that env-var keyed stuff to support dynamic color theming because I use day & night themes simultaneously, but figured people disagree on way more than just that.
Back in 2018, later on the "how to grow stdlib" RFC which landed on "fusion", and still today, I think the way human organizations have always scaled complexity is through delegation to trusted lieutenants, and this would work better for the Nim stdlib which has barely grown in 10 years. Doubtless trust is tricky { subsystem-oriented, more like Linux, seems better than some one guy does whole stdlib } and there may be alternative approaches.
E.g., there could be other reasons like simple obscurity, but at least since 2014 on Gentoo, getting a library has meant multiple package managers - both your host OS which you must know and some 3rd party babel/nimble thing. That is both bothersome and fragile. The Status stew approach and "a la carte microlibs to the maximum" also seem inferior to just an ever more expansive stdlib, but of course it cannot include everything. The sales pitch of Linux distros is "let us be your trusted agents", but maybe Nim doesn't make it easy. So, one alternative approach might be for Nim to add a simple "default search path" addition of somewhere standard to install 3rd party libs like /usr/lib/nim/site/ (which could just be site/ under the repo, maybe with an empty ".keep" file in it). This would be closest to the emacs/python idea with a standard "site-packages/" and while it might be ugly/suck in some way, because it is so close conceptually it should be easy to adapt to any of the Linux distros (which may be said to have "a C-shaped hole" or maybe python-shaped in this case). Can even brew on OSX install 3rd party Nim libs? Windows may be a bad model here since installing hierarchies of open source packages seems uncommon outside of Cygwin/WSL*/etc. environments.
Like it or not, new devs very often judge PLang's by their stdlibs / the level of effort it takes to "get going". I'm really unsure how international vs. Just American this metaphor (and so its understanding) is. "Batteries included" always meant to refer to "getting some present as a child but not being able to try it out 'for real' because you don't yet have the batteries". So, nimble/pip/venvs/all that are really not a great answer. They say, "this is where to buy batteries" rather than just including them. Any "2nd package manager" is already an undesirable friction for some user who "just wants to try out c-blake's lc" or whatever program depending on a lib ecosubsystem. Rust may have succeeded in spite of cargo, not because of it. It's not easy to attribute these things, but Perl/Python/Ruby/OCaml/Haskell/etc. are a bunch of whole ecosystems that have better integrated with "primary package managers" at the ecosystem level and that Nim has not seems a real problem.
Anyway, I don't meant to derail Zoom's thread with unresolved Big Questions, but unless Nim provides a bunch of nimble2(ebuild|flake|deb|rpm|...) translators for .nimble/ files, a site/ seems the easiest road to integrate with extant alien package managers. Seems a near trivial patch even compared to this parseopt thing. Maybe a delegation along a different dimension (e.g. Gentoo ambassador, Debian ambassador, etc.) could work. Maybe this last bit should have just been a new thread.
Yes, these are all great points and it's time to add things to the stdlib. But even if we all agreed on "don't put X into the stdlib", the following fact remains to be true:
We already have `std/parseopt` we might as well improve it as long as we don't break code.
Avoid single letter options except for -h and -v as good switches always end up in some script where the longer version is much more readable. Commonly used switches also tend to get auto-completion.
This really does fit in well with the overall tension I mentioned in "my last comment" link. Copy-pasting from CL history to a file generally involves other careful thought. Forcing expansion of short options to long variants is another kind of expansion. At the same time, hot & heavy at the shell prompt, I would hate being forced to tab complete/expand everything if I can remember the 2-4 single letter options I use a lot (but I realize that I have a better memory for those kinds of things than others).
So, that actually constitutes another idea for enhanced strictness for "scripts/saved-in-a-file-runners" above & beyond requiring a key-value delimiter -- no short options at all (not even -h & -v which barely apply then anyway) with a fast fail in the CL-parse as soon as you test your "script" (however it is represented, shell, Nim, etc.) the first time. I haven't done that one yet as its less logically isolated, but I did just push the cannot have options after first non-option argument -- another oft mentioned POSIX shell footgun (with argument generation from filenames aka "globbing"). People like abbreviations/even sloppy thinking when they are first attacking problems, but are usually fine layering in qualifications as it becomes systematized. In this case, it's pretty easy to activate a strict mode for a whole batch of commands.
Anyway, it is great to hear about stdlib expansion! And I agree we might as well improve std/parseopt. One thing cligen/parseopt3 has not in std/parseopt is suggestions for misspelled idents which is, I think, a popular feature. { No getting around a symbol table there, though! Lol. But layering can matter. :-) } stdlib-expansion-wise, if you get some kind of concept refinement working for new style concepts, I think the way adix/oats is layered so you can just back a table with a MemFile is pretty nice and just needs a Robin Hood hashing option.
My usual comment here will be that if there are opinions, just put it in a nimble package until it's established and everyone agrees on a single mode?
Two main issues: user expectations and portability.
Doing thing in idiosyncratic ways is great when it's a great design. However, great design is often not obvious and you often have to convince people. Great or not, each time it's a bump on the road. I think we all agree that stdlib should minimize the number of bumps, not just from the ergonomics pov, but from people retention pov too.
ie the nim standard library is not really the place for "experimental" modes and opinions - these will ossify with only bad options for changing them, with the bad ones causing maintenance overhead and debates without ever being removed and without providing utility.
They will ossify as long as people use them and don't complain, or no one uses them and they don't get in the way. If they complain, it's usually a sign for a change. This is especially true for things marked experimental. If you use them, don't be too upset when they change due to people disliking it.
A much better flow is to establish the modes as something that users want and use (via a package) then extract the actually used improvements and add these to stdlib, if still desired.
Practically no one uses Nim packages in development besides authors and their friends and employees. The numbers are just not there to make any significant conclusions regarding package design from the returning signals.
That's the point, at this moment we can only gather more or less reliable data from the reaction on the stdlib, and in regards to parseopt it's not that supportive.
When you add the problem of Nim-based software distribution, it follows stdlib should grow, at least in the experimental part. Fusion is probably not the right approach, though.
Look at the Rust docs, experimental features are everywhere in the stdlib right beside stable ones, clearly marked as such, including tracking issues, version of introduction and, as I vaguely recall, earlier you could even see planned stabilization versions. They are just more lucky being able to test them on more people quicker. You don't even need to go to an unlinked github-pages site for docs of the development version, it's right there so you know what to expect and can whet your appetite and get interested in participating in development/feature design.
these are all great points and it's time to add things to the stdlib.
The issue with the std lib isn't that it lacks stuff - it's that the stuff that is in there is unmaintained, slightly broken, and not using language features past 2016.
Ie destructors were developed but you still have to manually close files - this is the main issue of the std lib: it has lots of people that want to add things one-off but nobody that wants to maintain them later.
Closing files can return a Failed indicator and a destructor cannot as easily.
close (on posix) closes the file even if there are failures to be reported - when looking at semantics for closing things or releasing resources of any kind, this is a common way of addressing the error-at-close problem, ie by completing the "resource release" part even if there's an error to report.
Given the above point, you don't need a global flag or anything like that when closing things in destructors: you simply offer a separate "close" function that returns an (advisory) error for callers that care about such things.
This reasoning applies to a lot of things - ie sockets, when you close them, hang around in the operating system through their final TCP FIN/ACK phases but the OS will discard any further errors that could have been reported if you had closed the socket in a "regular" way (via shutdown / read).
it's that the stuff that is in there is unmaintained, slightly broken
You're using this general fact in this thread, implying the PR in question adds to the problem (maybe unintentionally). This is a fallacy of division. There's far more loc of tests there than new logic (same for docs). That would help maintenance of parseopt even without the changes. The changes themselves are rather timid.
and not using language features past 2016
That's an apt observation, but it doesn't make much sense to use them in a disorganized way. The core of the stdlib should use them so the outer layers can follow. I'd gladly use concepts everywhere, for example, but I don't want to design them each time, we need a blessed "move everything to this set of concepts where applicable".
Some things are tied to the compiler a bit too much for my taste, complicating maintenance and trying stuff. That's probably why the string.setLenUninit PR got stuck in the end.
But you all know about it better than me.
Woo-hoo! The PR got merged
Module docs were refreshed a bit, but the relevant part is here: https://nim-lang.github.io/Nim/parseopt.html#parser-modes
Please, use the new functionality and report bugs or just your general experience with them.
Thanks everyone!
BTW, it doesn't really require a plug from me, but just to draw your attention. For those feeling a bit limited by std/parseopt, there's always cblake's cligen, which got some interesting updates recently, like this or this one. It's awesome.