Last post Feb 16, 2009 07:39 AM by arunkumar2003
Feb 15, 2009 09:02 AM|buran|LINK
Dear ASP.NET Developers,
I want to web scrape some values from a web site. I had managed to do this by the code snippet given below, but the web site now uses AJAX functionality to display the values in a regular interval.
string requestUrl = "http://www.buranburanburan.com/buranburan/buran.aspx";
Uri address = new Uri(requestUrl);
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(address);
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
StreamReader stream = new StreamReader(response.GetResponseStream());
string source = stream.ReadToEnd();
source = source.Substring(source.IndexOf("USDTRL", 0), 40);
source = source.Substring(34, 6);
The web site now uses AJAX rather than directly outputting the values:
var dt = new Date();
httpRequest.open( "GET", "./somewebpage.aspx?callajax=true&dt="+dt.getTime(), true);
How can I get the values now?
Thanks in advance,
Feb 15, 2009 09:38 AM|arunkumar2003|LINK
Please refer this for more info.
You have to implement asynchronous web client. It should be like XMLHTTPRequest in Client side.
Feb 16, 2009 06:37 AM|buran|LINK
Thank you Arunkumar, but could you please explain this a little more?
Feb 16, 2009 07:39 AM|arunkumar2003|LINK
a method to handle the data when the server is responsed.
All the methods needed are already implemented and available in the below link.
Let me know if you have any issues.
MARK AS ANSWER IF IT HELPS YOU