Hello! I'm wondering if anybody can help me with the Proxy on httpclient. I'm using the following code to set a local proxy and get a session id.
import httpclient, json, strutils
var client = newHttpClient(proxy=newProxy("http://127.0.0.1:8888"))
var session = parseJson(client.getContent("https://yummy.xyz.com/get-session"))["session"].str
echo session
At some point during the getContent call, it attempts to connect to my proxy, but it seems to have dropped the server name (yummy.xyz.com). Here are the raw headers it sends to my proxy:
CONNECT /get-session HTTP/1.1
Host: /get-session
Connection: Keep-Alive
user-agent: Nim httpclient/0.17.0
When I run my program, it gives me the following error
Error: unhandled exception: The proxy server rejected a CONNECT request, so a secure connection could not be established. [HttpRequestError]
My proxy says it rejected the request because it rejected the connection because the DNS lookup failed (error 502).
Am I doing something wrong here?
I ported my code to python and it works. I did make a few discoveries, though.
First, here's my python code:
#!/usr/bin/env python3
from functools import wraps
import requests
if __name__ == "__main__":
proxies = {
'https': 'https://127.0.0.1:8888',
}
print(requests.get("https://yummy.xyz.com/get-session", proxies=proxies, verify=False).json()['session'])
and here's the resulting raw packet that gets sent to my proxy
GET /get-session HTTP/1.1
Host: yummy.xyz.com
Accept: */*
Accept-Encoding: gzip, deflate, compress
User-Agent: python-requests/2.2.1 CPython/3.4.3 Linux/4.4.0-43-Microsoft
I thought maybe the verify=false is the part I was missing in my nim code, so I changed it to this:
import httpclient, json, strutils, net
var client = newHttpClient(proxy=newProxy("http://127.0.0.1:8888"), sslContext=newContext(verifyMode=CVerifyNone))
var session = parseJson(client.getContent("https://yummy.xyz.com/get-session"))["session"].str
echo session
But that didn't change anything. verifyMode defaults to CVerifyNone in the first place. Then I noticed my proxy url was http instead of https. When I change it to https and run it again, the program hangs on the getContent call. Eventually, it times out and I get this error:
error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol [SslError]
Ahh, looking at the headers that httpclient sends again it seems like it's buggy. The Host header is completely wrong. Please report this on GitHub: https://github.com/nim-lang/Nim/issues
In case you're interested in trying to fix it, here is the relevant code in the stdlib: https://github.com/nim-lang/Nim/blob/devel/lib/pure/httpclient.nim#L1094 (please create a PR if you do manage to fix it :) )