Mirror of forum.nim-lang.org

9778 :: Downloading files with missing protocol in redirect url

• yister (orginal) [2022-12-29T20:53:48+01:00] view original

  I'm trying to download a file ( https://micro.mamba.pm/api/micromamba/linux-64/latest ) using httpclient, and it looks like the download is redirected twice, the first one being:

  
  //api.anaconda.org/download/conda-forge/micromamba/1.1.0/linux-64/micromamba-1.1.0-0.tar.bz2
  Run

  I've never come across a Location: ` header like this, with a missing `http/https prefix, but wget seems to be able to download it, so I assume it's not that uncommon. Anyway, trying to download this with the nim httpclient I'm getting an error:

  
  Error: unhandled exception: No uri scheme supplied. [ValueError]
  Run

  Is there any way to use httpclient to download from a url like this, or should I use libcurl or maybe puppy (not sure if that can download binary files). I guess I could try to resolve the redirects myself somehow, but before I do that, I wanted to see if anyone else has encountered something similar...

---

• Araq (orginal) [2022-12-29T23:30:42+01:00] view original

  Patch httpclient please and create a PR.

---

• yister (orginal) [2022-12-30T11:40:52+01:00] view original

  Thanks for confirming that its an issue with httpclient. Not sure if I feel confident enough in my nim-foo to contribute something like this (plus tests, etc), but I'll have a look at the docs whats required to contribute to nim, and at the httpclient code.

  For now, I'm using puppy for the download which seems to work, and has the additional benefit of not requiring ssl which I think is pretty neat, but it'd be nice to have finer level control like httpclient offers.

---

• PMunch (orginal) [2022-12-30T13:10:46+01:00] view original

  Pretty sure URLs without a scheme is technically invalid. But I guess wget either just tries a couple common ones, or just tries http/s. I guess for httpclient it could make sense to assume a scheme of http/s.

---

• yister (orginal) [2022-12-30T14:51:10+01:00] view original

  Yeah, probably. I guess it would make most sense to just use whatever the url in the chain before the redirect used.

---

• djazz (orginal) [2022-12-30T14:57:24+01:00] view original

  I think it should use the scheme of the current/previous request. In webdev when specifying resources, like an image on a webpage, you can use // to substitute the current scheme. Curl seems to do the same.

---