How do I get the HTML content of a web page (Screen Scraping) in Visual Basic.net?

In a Visual basic .NET application you will sometimes need to get information from a web page. The first step to retrieving information from a web page is to get the HTML of the web page into a string that you can then process in your code.

This code involves two namespaces:

SYSTEM.NET includes classes used to access the web.

SYSTEM.IO includes the classes needed to read the response stream into a string.

This code works by creating an Http Web Request like a browser would when you type in a web address. The request is sent to the web server, and a web response is returned. The response object provides a response stream which contains the HTML of the page. A stream reader is used to flow the response into a string, where you can easily access it.

Dim URL as string = "http://www.somewebsite.com"
Dim HRq As Net.HttpWebRequest = Net.WebRequest.Create(url)
Dim HRsp As Net.HttpWebResponse = HRq.GetResponse()
Dim sr As IO.StreamReader = New StreamReader(HRsp.GetResponseStream())
Dim HTM as string = sr.ReadToEnd()

When the code finishes, the string named HTM will contain the entire HTML code for the web page that you specified in the URL.

You can then process the HTML using string manipulation functions to get the specific data that you need from the web page.

Study and Research resources:

Microsoft Certification Info:

See the list of .Net articles

TOGGIT IN SEARCH OF CERTIFICATION

All information on this site is copyright ToggIT.com unless otherwise specified. Material submitted or posted this site may also contain materials that are copyrights of individual contributors. It is illegal to copy, publish, reproduce, or distribute any materials from this site with the express permission of the owner of this material.

The material on this web site is not sponsored by, endorsed by or affiliated with Microsoft, Inc, CompTIA, or Cisco Systems, Inc. Microsoft, Inc.® ,Windows XP, Windows 2000, Windows 2000 Server, Windows 98, Windows NT, Visual Basic, Visual C++, Visual FoxPro, SQL server and Microsoft logos are trademarks or registered trademarks of Microsoft, Inc. in the United States and certain other countries. A+, Server+, and Network+ are all registered trademarks of CompTIA. CCNA, CCNE, and the Cisco logo are all registered trademarks of Cisco Systems. All other trademarks are trademarks of their respective owners.