On 500 packages evaluated as of today:
One can see that the average quality of projects is low. Usually, a small effort is required to improve the score: add a user-oriented documentation, structure the files of the project... This small effort is particularly valuable to attract Nim wana be users or for package discovery. Sometimes, quality is better than quantity.
I've adapted Nimble to filter packages based on that package maturity level (https://github.com/nim-lang/nimble/pull/691). This is interesting for developers looking for good packages as they can use this extra information in Nimble searches.
But a problem happens in maintaining this type of information. For the moment, the evaluation has been done based on three criteria that I evaluated. As packages evolve, this rating will evolve too. Updating these criteria can be done by the package maintainers, following the evaluation methodology, but it can generate an avalanche of pull requests on the packages repository. It would be better if the package maturity was evaluated automatically based on objective criteria...
I plan 2-3 weeks to complete the evaluation of the remaining packages. I'll get better stats at the end of the job. Do you think it's worth it? Do you have better ideas on how to measure packages quality/maturity?
I already made this automatically, been some years trying to get this merged: https://github.com/nim-lang/packages/pull/916#issue-227422899
But manual one-by-one package checking is also nice to have, I dunno if that can scale tho...
I plan 2-3 weeks to complete the evaluation of the remaining packages. I'll get better stats at the end of the job. Do you think it's worth it? Do you have better ideas on how to measure packages quality/maturity?
Yes, it is definitely worth it and the idea to somehow formalize these ideas in Nimble is also a good one! However, we don't usually move Nimble packages into the stdlib -- different development speeds and github issue trackers are useful things.
Instead the idea of different "Nim distributions" came up, we would have a list of "important / curated" Nimble packages and bundle these with Nim. And include it in the official documentation.
Btw are you aware of https://github.com/VPashkov/awesome-nim ?
What bothers me is that my manual evaluation is really subjective and I only skim over the code, the documentation and the issues. I'm unable to test if the package compiles with the latest version of Nim. I can't evaluate if it's easy to use, with a high-level Nim API or only a low-level binding. I can't evaluate the examples or the tests coverage. Etc.
Sometimes, I see very nice projects where the authors put a lot of energy to build a large system, but they did not comment correctly the code and there is no documentation and I can't understand (neither would be a potential user) how to use it or even what it does. Or a good project where the author neglected writing documentation, for instance if English is not his native language, and for that case the whole project score is low. Probably that these projects should attract more attention from external developers to get help, but this does not translate in the maturity score that is given and this information (package with good potential) is lost.
@araq Regarding the standard library, it was only a side comment. This should only occur when the package is stable and totally mature and covers a fundamental aspect missing from the standard library. Also in case of replacing an existing library, it must be totally compatible and bring new valuable features.
But Nim Distributions are worth having so that Nim can be declared Batteries included.
Awesome Nim is a bit short on packages and has not been updated recently. When I've completed my evaluation, I'll try to update it with new packages.
@andrea This is a nuance I would like to capture too but I can't without testing the package. I've used the *Dead* category to classify packages that are abandoned. But a package that hasn't changed for 4 years could be abandoned too or just doing what it is supposed to. I can add a column in the sheet or another category (for instance Mature) to capture this notion. But I need input from packages authors or users to set this information.
@federico3 I went multiple times on https://nimble.directory but until tonight, where I persevered to understand where was the package documentation you were referring in the About page, I never understood that the Nim Package DIrectory was a continuous integration server... That's really nice! Am I alone not having seen that you can access a package page without having to go to Github and browse the package documentation? In the front page, the link to package pages should be visually more attractive than the Project Web site button.
By the way, it seems there is a bug in the documentation generation not using the latest version of a package. I displayed the page of npeg (https://nimble.directory/pkg/npeg) and checked the reason why the documentation was failing (https://nimble.directory/ci/badges/npeg/nimdevel/doc_build_output.html). The doc builder is using npeg version 0.10.0 though version 0.13.0 is available... And I was unable to have the Search symbol return results. Is it working?
For sure, the Nim Package Directory can check much more extensively the quality of packages than what I'm doing, and it can do it automatically. On the other hand, Nimble is a light package manager and users would benefit from having better insight about the packages it manages. The best of both worlds would be that Nim Package Directory would server as a package registry and would generate regularly the packages.json files that is used by Nimble, adding automatically extra metadata like latest version, link to the hosted documentation, missing dependencies, etc. That way, Nimble can stay simple and light but use rich information.
@dom96, reading the discussions in https://github.com/nim-lang/packages/issues, do you think that this scenario would be worth exploring?
@juancarlospaco, perhaps the changes that you put in your Nimble pull request would be better into the Nim Package Directory?