I require some help / sample code (ASP.Net 2.0 / vb.net preferred) that can help me analyse a web page. The sort of breakdown required:

1.       All displayable text in a string.

2.       Word Count

3.       Each word listed together with the number of times the word appears.

4.       All URL’s listed showing the link text and the URL.

5.       List of Image URL’s.

Sounds like a job for regular expressions, but not too good with those.

Any help, advice most appreciated

This tool might give you some ideas:

http://www.nikhilk.net/WebDevHelperHTTPTracingUpdates.aspx

You will need to use your custom code for other tasks like word count etc which is not tough.

HTH,

Vivek

I am looking in particular for source code (asp.net 2.0 / vb) examples.

1.       Coding examples of a web crawler or spider.

2.       Examples of how to extract link text and url’s

3.       Examples of how to extract all visible text

Try these links:

http://www.codeproject.com/aspnet/Spideroo.asp

http://www.codeproject.com/useritems/ZetaWebSpider.asp

http://www.webconcerns.co.uk/aspnet/search/default.asp

http://www.learnasp.com/freebook/learn/usercontrols-scraper.aspx

HTH,

Vivek