How to specify server IP in HttpWebRequest

sunny boy, 2009-01-01

I was writing an HTTP proxy during last weeks. I need to address the origin server with a custom IP address, but I should keep the "Host" header.

I googled about "HttpWebRequest set server IP", "WebRequest modify Host", etc. I read through several posts, and realized:

  • When HttpWebRequest is created, the "Host" header is automatically set to the "host:port" portion of the Uri.
  • Attempt to modify HttpWebRequest.Headers["Host"] will throw an exception.
  • The server IP address of the HTTP Request is determined by the host portion of the Uri, and is automatically resolved against the DNS server.
  • When HttpWebRequest.Proxy is set to use a proxy, the server IP address is determined by resolving the host of Proxy property. I can set the origin server as the proxy, this is called "Proxy hack".
  • When using a proxy, the HTTP Request-Line looks like "GET http://host/ HTTP/1.1" rather than "GET / HTTP/1.1". Every HTTP/1.1 compliant origin server should accept this, but firewalls may think it's a "proxy request" and block it, and some non-compliant servers does not accept it.

SOLUTION

My purpose is: make HttpWebRequest send its request to a custom IP address, but leave the Request-Line as "GET / HTTP/1.1".

After hours of Reflector, I found the solution:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://blogs.msdn.com");//url and Host header
FieldInfo field_ServicePoint_ProxyServicePoint = (typeof(ServicePoint))
    .GetField("m_ProxyServicePoint", BindingFlags.NonPublic | BindingFlags.Instance);
req.Proxy = new WebProxy("www.google.com:80");//server IP and port
field_ServicePoint_ProxyServicePoint.SetValue(req.ServicePoint, false);
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

When the above code is executed, I can see the following request in Wireshark:

GET / HTTP/1.1
Host: blogs.msdn.com
Connection: Keep-Alive

And, the request is sent to 64.233.189.99:80, a www.google.com server.

DOWNLOAD: C# code & Wireshark capture

BEHIND THE SCENES

ServicePoint class provides connection management for HTTP connections. A ServicePoint instance is created for each "host:port" combination, and reused across several requests. ServicePoint.m_ProxyServicePoint determines whether this ServicePoint connects to a proxy. When m_ProxyServicePoint is set to false, HttpWebRequest don't think it's a proxy, so the Request-Line becomes "GET / HTTP/1.1".

However, m_ProxyServicePoint is a private property of ServicePoint, so I have to use Reflection to change it.

HttpWebRequest.Host in .Net Framework 4.0

2010-05-31 update: As of .Net Framework 4.0, you can use HttpWebRequest.Host property to set Host header independent from the request URI.

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://www.google.com");//url path and actual server
req.Host = "blogs.msdn.com");//Host header
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

Author: sunny boy

Note: to help international users, this article is written in English.

©2006-2010 yoursunny.com