How to read a remote web page with ASP.NET 2.0

Two classes in the System.Net namespace make it very easy to obtain the html of a remote web page. These are the HttpWebRequest and HttpWebResponse. Here's a quick demo.

The static method below makes an http request for a web page, and the resulting string of html within the http response is captured and returned to the caller.


static string GetHtmlPage(string strURL)

  String strResult;
  WebResponse objResponse;
  WebRequest objRequest = HttpWebRequest.Create(strURL);
  objResponse = objRequest.GetResponse();
  using (StreamReader sr = new StreamReader(objResponse.GetResponseStream()))
    strResult = sr.ReadToEnd();
  return strResult;
Shared Function GetHtmlPage(ByVal strURL As String) As String

  Dim strResult As String
  Dim objResponse As WebResponse
  Dim objRequest As WebRequest = HttpWebRequest.Create(strURL)
  objResponse = objRequest.GetResponse()
  Using sr As New StreamReader(objResponse.GetResponseStream())
    strResult = sr.ReadToEnd()
  End Using
  Return strResult
End Function

And to call the method:

string TheUrl = "";
string response = GetHtmlPage(TheUrl);
Dim TheUrl As String = ""
Dim response As String = GetHtmlPage(TheUrl)

The HttpWebRequest object can be configured to supply header information. Most of the default values are null. For example, the User-Agent value is null, but can be set to anything you want. Cookies can also be "enabled" for the object through the CookiesContainer property, if required to maintain sessions.

Date Posted:
Last Updated:
Posted by:
Total Views to date: 106311


- titus

How to get the content with out tag,images and unwanted information from this return result.Kindly help me

- Mike


You need to use Regualr Expressions. Google "Regular Expressions Strip HTML" for examples.

- Michael

It seems there is a loss in translation with the reader.
The original file that is read has : 7S£ §I‚xsß*ُ£
The information read
from the file only reads : 7S�� �I�xs�*ُ�

Any ideas? or are the characters just not allowed

- Michael

Hello again. Thanks all for the help.
I just converted to binary.

- kumar

Hi mike
Is there any technique through which we can read a single line or value from a webpage like reading a specific number from a webpage

Pawan Kumar

- Dipesh

Simple & Awesme... tq...

- mahendra

thanx a lot

- Tina

When I try the C# version, I just get a blank response. It's as if it's not loading the URL at all. No errors, just nothing returned. Help?

Recent Comments

adam 05/10/2015 14:35
In response to Integrating Web API with ASP.NET Razor Web Pages
Can you re-open this web api project in webmatrix, once you've added web api? Basically I'm looking...

nish 24/09/2015 18:48
In response to Managing Checkboxes And Radios In ASP.NET Razor Web Pages
Very Interresting stuff! it really helped me to send an int value by checking a checkbox!...

Uğur Dinç 24/09/2015 16:45
In response to Scheduled Tasks In ASP.NET With Quartz.Net
Simplest and best explanation on Quartz.NET. Thank you!...

woo 24/09/2015 15:34
In response to Implementing Google's EU End User Consent Policy
Is there any way for the banner to appear only to EU visitors? I am referring to the jQuery code...

Justin 24/09/2015 11:10
In response to Using ASP.NET Identity with Razor Web Pages
Hi Mike, Very helpful article again, thanks. One query that I'm trying to work out is how you to...

hb 23/09/2015 23:12
In response to WebMatrix Opens Wrong Version Of Visual Studio
Mike - I got this working when I went decided to look at the community edition of VS2015 and tried I...

Muneer 22/09/2015 14:55
In response to Scheduled Tasks In ASP.NET With Quartz.Net
I have an error with these two is not recognizing the commands. am i missing any library? I am web...

David 22/09/2015 13:57
In response to iTextSharp - Working with Fonts
Mike your articles about itextsharp are excellent, really helpful and well written. In the future...

Peter 22/09/2015 06:39
In response to Accessing Your Model's Data from a Controller
Thanks.Got it. ...

sreedhar kandukuri 21/09/2015 14:05
In response to Integrating Web API with ASP.NET Razor Web Pages
Nice Overview...