I recently tried to do a web request in .NET C#. So I looked it up in the MSDN and wrote down some lines of code and was happy that it worked straightaway (ok, it’s not very complex anyhow).
It looked something like this:
string responseContent;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
using (StreamReader sr = new StreamReader(responseStream))
responseContent = sr.ReadToEnd();
return responseContent;
But this code was very slow and it took several seconds to get the response.
One reason for that is that if you don’t use a proxy or just don’t set the proxy property of the WebRequest, it tries to auto-detect and configure a proxy what causes a significant delay.
So just adding
request.Proxy = null;
before the GetResponse() call will fix that problem.
Remark/Update: To be more correct, you should set ”request.Proxy = GlobalProxySelection.GetEmptyWebProxy();” instead of just null. (MSDN: HttpWebRequest.Proxy property)
Another point is that you should always close the WebResponse after processing its contents. That might also speed up your requests. I prefer to use “using” statements to do that. It will get a bit nested sometimes but this way you just can’t forget it.
So here’s the final code for a simple “GET” request:
public string MakeWebRequest(string url)
{
try
{
string responseContent;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
request.Proxy = GlobalProxySelection.GetEmptyWebProxy(); // null;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (StreamReader sr = new StreamReader(responseStream))
responseContent = sr.ReadToEnd();
}
}
return responseContent;
}
catch (Exception ex)
{
// Error handling
return ex.Message;
}
}
Extending the code for a “POST” request:
To make a “POST” request we have to add the following:
public string MakeWebRequest(string url, string postData)
{
try
{
string responseContent;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
// Set Method to "POST"
request.Method = "POST";
// Convert "POST" data to a byte array
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
// Set the content type of the WebRequest
request.ContentType = "application/x-www-form-urlencoded";
// Set the content length
request.ContentLength = byteArray.Length;
// Get the request stream
using (Stream requestStream = request.GetRequestStream())
{
// Write the "POST" data to the stream
requestStream.Write(byteArray, 0, byteArray.Length);
}
// Don't forget to set the proxy
request.Proxy = GlobalProxySelection.GetEmptyWebProxy(); // null;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (StreamReader sr = new StreamReader(responseStream))
responseContent = sr.ReadToEnd();
}
}
return responseContent;
}
catch (Exception ex)
{
// Error handling
return ex.Message;
}
}
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = “GET”;
request.Proxy = null;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Stream responseStream = response.GetResponseStream();
using (StreamReader sr = new StreamReader(responseStream))
{
string responseContent = sr.ReadToEnd();
return responseContent;
}
}
}
catch (Exception ex)
{
return ex.Message;
}
}
May 27, 2010 at 3:53 pm
Hi Steffen,
Where does it gets the proxy list from? ie settings?
I actually tried to set proxy=null but there wasnt much difference. Does setting proxy=null means avoid lookup for proxy?
Thanks
Vincent
May 27, 2010 at 8:09 pm
Hi Vincent,
if you don’t specify a proxy, the app/web/machine.config will be searched for the “system.net/defaultProxy” property. If this property is not defined it uses the IE proxy settings, that’s right. Though I’m not really sure if these operations are the main reason for the large delay. On my machine it takes up to 10-15sec longer than with a nulled out proxy property but I’ve also head of people who didn’t have that much problems with that.
But if you set the proxy to a value different than the default one, the lookup will be skipped.
As a side note, if you don’t use a proxy you should set the “request.Proxy = GlobalProxySelection.GetEmptyWebProxy();” instead of just null. I will correct that in the Blog post.
Here are some links to the MSDN pages that may be interesting. You should have a look at the remarks sections:
HttpWebRequest.Proxy Property
GlobalProxySelection
system.net/defaultProxy
Regards,
Steffen
September 23, 2010 at 4:41 am
You Sir, are an absolute lifesaver.
I was fiddling with WebRequest myself and was puzzled for at least an hour. Scanned network activity to see if it might be some odd connection problem.
What I don’t understand though, is why Microsoft did this. Wouldn’t it be more logical to pick the proxy config from a config file or the IE settings? This should literally only take seconds in my opinion. I guess MS’s ambition to make things smart bit them in the back this time…
Thanks a lot anyway!
September 24, 2010 at 5:33 pm
Hi Michael,
it actually does search the .config files and finally falls back to the IE settings, as I stated above. And the IE seems to be the bottleneck.
A few days ago I wanted to set up a WebDAV drive and it was incredibly slow on Windows 7 while on XP everything worked just fine. And the solution I found was to open IE and head to Tools->Options->Connections->LAN Settings and uncheck “Automatically detect settings”.
And this little checkbox also affects the WebRequest. So if you uncheck it you will also notice that the delay is gone.
January 31, 2011 at 12:24 am
Kudos
April 5, 2011 at 2:02 am
“It will get a bit nested sometimes…”
You can combine using statements like this:
using (foo)
using (bar) {
// do stuff
}
April 5, 2011 at 5:11 pm
Hi Grant,
Thanks for the tip. I didn’t know that. This definately makes it more readable.
December 22, 2011 at 4:59 pm
Does it happned on all external URL or just internal URL contain localhost?
We also got a delay when use localhost in URL when we change the URL to the external URL the delay gone.
Any how you can change the setting in the web.config in order to remove the proxy:
January 17, 2012 at 2:35 am
request.Proxy=null ; worked for me.
also you can add the following to youw config file:
January 25, 2012 at 3:45 pm
Hi,
I’ve saw this workaround somewhere else and I don’t recall where.
I don’t actually really know and didn’t have time to research yet and tell why or how it works.
But it worked for me, made my connections quite very faster when I’ve add this parameter at my class’ constructor before instantiating my HTTPWebRequest object:
System.Net.ServicePointManager.Expect100Continue = false;
Please, could you try it and tell us?
Best wishes, mate!
Felipe Ranieri
January 29, 2012 at 7:20 pm
Hi Felipe,
the Expect100Continue property should only affect POST requests. I think normally it should be left at the default value (true), so if the server requires authentication the client just sends the request headers and after it is authenticated it continues to send the POST body. Otherwise it sends the POST data twice, which would increase the request time.
I tried it out anyway and inserted the line but it had no effect on the delay on my machine.
March 14, 2012 at 3:22 pm
I think you should set the proxy before you send the request, not after like you have in the POST request example.