How to read a remote web page with ASP.NET 2.0

4.3 (27 votes)

Two classes in the System.Net namespace make it very easy to obtain the html of a remote web page. These are the HttpWebRequest and HttpWebResponse. Here's a quick demo.

The static method below makes an http request for a web page, and the resulting string of html within the http response is captured and returned to the caller.


static string GetHtmlPage(string strURL)

  String strResult;
  WebResponse objResponse;
  WebRequest objRequest = HttpWebRequest.Create(strURL);
  objResponse = objRequest.GetResponse();
  using (StreamReader sr = new StreamReader(objResponse.GetResponseStream()))
    strResult = sr.ReadToEnd();
  return strResult;
Shared Function GetHtmlPage(ByVal strURL As String) As String

  Dim strResult As String
  Dim objResponse As WebResponse
  Dim objRequest As WebRequest = HttpWebRequest.Create(strURL)
  objResponse = objRequest.GetResponse()
  Using sr As New StreamReader(objResponse.GetResponseStream())
    strResult = sr.ReadToEnd()
  End Using
  Return strResult
End Function

And to call the method:

string TheUrl = "";
string response = GetHtmlPage(TheUrl);
Dim TheUrl As String = ""
Dim response As String = GetHtmlPage(TheUrl)

The HttpWebRequest object can be configured to supply header information. Most of the default values are null. For example, the User-Agent value is null, but can be set to anything you want. Cookies can also be "enabled" for the object through the CookiesContainer property, if required to maintain sessions.

Date Posted:
Last Updated:
Posted by:
Total Views to date: 108018


- titus

How to get the content with out tag,images and unwanted information from this return result.Kindly help me

- Mike


You need to use Regualr Expressions. Google "Regular Expressions Strip HTML" for examples.

- Michael

It seems there is a loss in translation with the reader.
The original file that is read has : 7S£ §I‚xsß*ُ£
The information read
from the file only reads : 7S�� �I�xs�*ُ�

Any ideas? or are the characters just not allowed

- Michael

Hello again. Thanks all for the help.
I just converted to binary.

- kumar

Hi mike
Is there any technique through which we can read a single line or value from a webpage like reading a specific number from a webpage

Pawan Kumar

- Dipesh

Simple & Awesme... tq...

- mahendra

thanx a lot

- Tina

When I try the C# version, I just get a blank response. It's as if it's not loading the URL at all. No errors, just nothing returned. Help?

Recent Comments

Phil 29/11/2015 18:48
In response to iTextSharp - Working with images
Hi Mike, great article as ever. Any chance you can do an example of adding a swf to a pdf with I it...

makhlas 27/11/2015 17:05
In response to How To Send Email In ASP.NET MVC
hello, thanks for sharing these tutos , but when i apply the send button, i'm gettin this error...

raphael 27/11/2015 13:59
In response to ASP.NET 5: Managing Client-side Dependencies with NPM, Bower and Gulp
i fixed it. i'm not sure *what* it is that VS does, but you get the same result if you do a "npm on...

Raphael 27/11/2015 13:02
In response to ASP.NET 5: Managing Client-side Dependencies with NPM, Bower and Gulp
There seems to be a major issue with the dependency manager in VS2015. I haven't yet figured out but...

Joaquín Bresan 26/11/2015 17:58
In response to Getting the identity of the most recently added record
Great contribution friend. Thank you so much for sharing....

JR 26/11/2015 02:49
In response to ASP.NET MVC 5 with EF 6 - Working With Files
Hello, Where is the location of the database? Thanks...

sumalatha 24/11/2015 22:09
In response to ASP.NET 5 Middleware, Or Where Has My HttpModule Gone?
Great article. I have gone through hundreds of articles, finally this gives the comprehensive of...

TimPat10 24/11/2015 17:35
In response to A Better Way To Export Gridviews To Excel
Great article. I've noticed that date fields have to be re-formatted when the spreadsheet is pulled...

Rajasekar 24/11/2015 12:27
In response to Import Data From Excel to Access with ASP.NET
While use this code i'm facing on error: "Unrecognized Database format C:\mydabase.accdb" can any...

Parmod 24/11/2015 07:28
In response to ASP.NET 5 Project Basics
For a new learner (Fresher) in ASP.NET there is a issue Fresher have to learn two types of , old...