I think we need Nimble website as a subdomain at nim-lang.org, making it more official and easier to find (nimble.nim-lang.org, or pkgs, packages?), but we need an eco-efficient way to build the pages for the packages, something without a backend, just static files.
I really like crates.io and pub.dev
For long-time storage of the packages, https://wasabi.com/pricing seems to be the most affordable option (no fees for egrees or API requests), but it has a minimum storage duration of 90 days, which in is not a problem in this case (because this should be like an archive of all open source packages). Check here https://wasabi.com/pricing/rcs-faq
For the static website we can use GitHub pages.
About 15GB with full git histories if I recall correctly. Maybe 5gb for pkg tar.xz
This server costs me €5/mo: https://www2.elcritch.net/atlas-packages/
Linux mirrors (what @enthus1ast suggested) solve this by mirroring cryptographic signatures in addition to the actual software, and the user/client program is expected to verify the signature before doing anything with the downloaded software.
Having a similar system in Nimble so that it can automatically verify the signatures of package creators would be pretty cool but it's also quite complex and I don't think anyone wants to deal with the headache of figuring out this system.
Linux mirrors introduce a lot of complexity, but they are quite robust and reliable if done right.
That's right! Just discovered this: https://github.com/LunaStev/mirror-docker/tree/master. Excluding docker, is pretty simple: auto rsync + nginx (+ limit_rate and limit_rate_after modules) should do the do
So nimble/atlas should use gpg to sign packages on publish, then verify before downloading. Solid.
From the README:
### Background Fetcher
The fetcher:
1. Fetches the official [nimble packages.json](https://github.com/nim-lang/packages) periodically
2. Validates packages by compiling them in Docker
3. Ingests all GitHub-hosted packages
4. Runs on a configurable interval (default: 1 hour)
5. Is resilient to individual package failures---
- Validates packages by compiling them in Docker
A lot of packages can't even successfully nimble dump, let alone compile (though I don't really know what the compilation unit for validation is in the context of a library)
- Ingests all GitHub-hosted packages
Not all packages are on github:
8 https://bitbucket.org
27 https://codeberg.org
1 https://git.adyxax.org
1 https://git.dog
1 https://git.envs.net
2550 https://github.com
2 https://gitlab.3dicc.com
33 https://gitlab.com
2 https://git.skylarhill.me
30 https://git.sr.ht
2 https://git.syndicate-lang.org
1 https://git.termer.net
1 https://njoseph.me
1 https://notabug.org
1 https://pf4sh.eu
2 https://www.github.comAs @daylinmorgan said, I don't know why this is dependent on GitHub. You're depending on GitHub to handle fetching repositories and for fetching releases but you can do both with just git.
AFAIK, Nimble uses git tags to retrieve release information. Any git tag that looks like a version number (fx. "v0.3.3" or "0.1.2") is a release. As for fetching repositories, you can just use git clone like Nimble does.
Also, let's not forget that Nimble also supports cloning Mercurial packages, and it might be worth adding support for that to archive the (few) Mercurial packages that are available.
Support for mercurial would be necessary so long as nimble supports it.
Worth noting though that in nim-lang/packages there are 5 packages from two users and none of them actually exist anymore...
❯ jq -r '.[] | select(.method == "hg") | "\((.name + " ")[:10]) \(.url)"' packages.json
linagl https://bitbucket.org/BitPuffin/linagl
vorbis https://bitbucket.org/BitPuffin/nim-vorbis
portaudio https://bitbucket.org/BitPuffin/nim-portaudio
scrypt.nim https://bitbucket.org/BitPuffin/scrypt.nim
gemf https://bitbucket.org/abudden/gemf.nim