I was writing an HTTP proxy during last weeks. I need to address the origin server with a custom IP address, but I should keep the "Host" header.
I googled about "HttpWebRequest set server IP", "WebRequest modify Host", etc. I read through several posts, and realized:
- When HttpWebRequest is created, the "Host" header is automatically set to the "host:port" portion of the Uri.
- Attempt to modify HttpWebRequest.Headers["Host"] will throw an exception.
- The server IP address of the HTTP Request is determined by the host portion of the Uri, and is automatically resolved against the DNS server.
- When HttpWebRequest.Proxy is set to use a proxy, the server IP address is determined by resolving the host of Proxy property. I can set the origin server as the proxy, this is called "Proxy hack".
- When using a proxy, the HTTP Request-Line looks like "GET http://host/ HTTP/1.1" rather than "GET / HTTP/1.1". Every HTTP/1.1 compliant origin server should accept this, but firewalls may think it's a "proxy request" and block it, and some non-compliant servers does not accept it.
My purpose is: make HttpWebRequest send its request to a custom IP address, but leave the Request-Line as "GET / HTTP/1.1".
After hours of Reflector, I found the solution:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://blogs.msdn.com");//url and Host header FieldInfo field_ServicePoint_ProxyServicePoint = (typeof(ServicePoint)) .GetField("m_ProxyServicePoint", BindingFlags.NonPublic | BindingFlags.Instance); req.Proxy = new WebProxy("www.google.com:80");//server IP and port field_ServicePoint_ProxyServicePoint.SetValue(req.ServicePoint, false); HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
When the above code is executed, I can see the following request in Wireshark:
GET / HTTP/1.1 Host: blogs.msdn.com Connection: Keep-Alive
And, the request is sent to 188.8.131.52:80, a www.google.com server.
BEHIND THE SCENES
ServicePoint class provides connection management for HTTP connections.
A ServicePoint instance is created for each "host:port" combination, and reused across several requests.
ServicePoint.m_ProxyServicePoint determines whether this ServicePoint connects to a proxy.
m_ProxyServicePoint is set to false, HttpWebRequest don't think it's a proxy, so the Request-Line becomes "GET / HTTP/1.1".
m_ProxyServicePoint is a private property of ServicePoint, so I have to use Reflection to change it.
HttpWebRequest.Host in .Net Framework 4.0
2010-05-31 update: As of .Net Framework 4.0, you can use HttpWebRequest.Host property to set Host header independent from the request URI.
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://www.google.com");//url path and actual server req.Host = "blogs.msdn.com");//Host header HttpWebResponse resp = (HttpWebResponse)req.GetResponse();