How to Read the HTML of a remote Web Page : The Official Microsoft ASP.NET Forums

Reply

cms9651

Member

175 Points

573 Posts

How to Read the HTML of a remote Web Page

Dec 11, 2012 01:34 PM|LINK

Hello guys!

I need your help.
This is code of my net page (C#).

My problem is this error:

A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed
because connected host has failed to respond

How can i do it?

Any help?
Thank you.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Net;
using System.IO;

public partial class CallRemota_Default : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        WebRequest req = WebRequest.Create("http://www.asp.net");
        WebResponse res = req.GetResponse();
        StreamReader sr = new StreamReader(res.GetResponseStream());
        string html = sr.ReadToEnd();
    }
}

Moved by mbanavige on Dec 12, 2012 11:42 AM

Reply

johnyM456

Contributor

2177 Points

347 Posts

Re: How to Read the HTML of a remote Web Page

Dec 11, 2012 02:29 PM|LINK

public static string ScreenScrape(string Url)
{
	System.Net.WebClient objWebClient = new System.Net.WebClient();
	UTF8Encoding objUTF8 = new UTF8Encoding();
	string result = objUTF8.GetString(objWebClient.DownloadData(Url));
	return result;
}

Reply

cms9651

Member

175 Points

573 Posts

Re: How to Read the HTML of a remote Web Page

Dec 11, 2012 02:55 PM|LINK

thank you, i don't have error but the blank page in the browser....

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Net;
using System.IO;

public partial class CallRemota_Default : System.Web.UI.Page
{

    string Url = "http://www.asp.net";

    public static string ScreenScrape(string Url)
    {
        System.Net.WebClient objWebClient = new System.Net.WebClient();
        System.Text.UTF8Encoding objUTF8 = new System.Text.UTF8Encoding();
        string result = objUTF8.GetString(objWebClient.DownloadData(Url));
        return result;
    }
}

Edited by cms9651 on Dec 11, 2012 02:57 PM

Reply

johnyM456

Contributor

2177 Points

347 Posts

Re: How to Read the HTML of a remote Web Page

Dec 11, 2012 03:15 PM|LINK

the only thing you've did at this point is load a complete webpage content as a string in memory.

If you want to display this entire string in your page you can add something like this in you page-markup:

But if that's what you want why don't you use an iframe?

Reply

cms9651

Member

175 Points

573 Posts

Re: How to Read the HTML of a remote Web Page

Dec 11, 2012 03:23 PM|LINK

thank you, but I have the first error.

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="Default.aspx.cs" Inherits="CallRemota_Default" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
     <% = ScreenScrape("http://www.asp.net") %>
    </div>
    </form>
</body>
</html>

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Net;
using System.IO;

public partial class CallRemota_Default : System.Web.UI.Page
{

    public static string ScreenScrape(string Url)
    {
        System.Net.WebClient objWebClient = new System.Net.WebClient();
        System.Text.UTF8Encoding objUTF8 = new System.Text.UTF8Encoding();
        string result = objUTF8.GetString(objWebClient.DownloadData(Url));
        return result;
    }
}

johnyM456

But if that's what you want why don't you use an iframe?

I need check if the remote web page does not go in timeout....

Reply

johnyM456

Contributor

2177 Points

347 Posts

Re: How to Read the HTML of a remote Web Page

Dec 11, 2012 05:24 PM|LINK

Here's a sample of how you can do it using another aproach. It includes possible timeout problemens. It's in VB, but should be easy to translate...

Reply

cms9651

Member

175 Points

573 Posts

Re: How to Read the HTML of a remote Web Page

Dec 12, 2012 07:19 AM|LINK

johnyM456

Here's a sample of how you can do it using another aproach. It includes possible timeout problemens. It's in VB, but should be easy to translate...

thanks but where a sample ?

Edited by cms9651 on Dec 12, 2012 08:44 AM

Reply

johnyM456

Contributor

2177 Points

347 Posts

Re: How to Read the HTML of a remote Web Page

Dec 12, 2012 09:35 AM|LINK

Sorry, forgot to paste the link: http://bhaskarrajukonduru-dotnet.blogspot.be/2010/06/net-screen-scraping.html

Reply

cms9651

Member

175 Points

573 Posts

Re: How to Read the HTML of a remote Web Page

Dec 12, 2012 11:49 AM|LINK

johnyM456

Sorry, forgot to paste the link: http://bhaskarrajukonduru-dotnet.blogspot.be/2010/06/net-screen-scraping.html

thank you for your time and hints.

I have this error:

A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed
because connected host has failed to respond

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Net;
using System.IO;

public partial class CallRemota_Default : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        GetResponse();
    }

    public string GetResponse()
    {
        string url = "http://www.meteowebcam.it/export_reg.php?reg=Toscana&day=0";
        int timeout = 80000;
        HttpWebRequest webRequest = WebRequest.Create(url) as HttpWebRequest;

        if (timeout != 0)
        {
            webRequest.Timeout = timeout;//milliseconds
        }

        StreamReader responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
        string strResponse = responseReader.ReadToEnd();
        responseReader.Close();
        return strResponse;
    }
    
}

Reply

johnyM456

Contributor

2177 Points

347 Posts

Re: How to Read the HTML of a remote Web Page

Dec 12, 2012 04:56 PM|LINK

I've tested the code and it works fine...
A string like below is produced:

<a href="http://www.meteowebcam.it/meteo/Toscana.html" title="Previsioni meteo Toscana"><img src="../../export/toscana_1355288400_0.jpg" alt="Previsioni meteo Toscana" /></a>

Keep in mind that using this method won't be enough to display the image in your webpage because it has an url relative to the meteowebcam-site.
The reason why you're getting a connectionerror is a mysterie. Are you sure your testing it on a machine with a live internetconnection?

How to Read the HTML of a remote Web Page RSS

21 replies

cms9651

How to Read the HTML of a remote Web Page

johnyM456

Re: How to Read the HTML of a remote Web Page

cms9651

Re: How to Read the HTML of a remote Web Page

johnyM456

Re: How to Read the HTML of a remote Web Page

cms9651

Re: How to Read the HTML of a remote Web Page

johnyM456

johnyM456

Re: How to Read the HTML of a remote Web Page

cms9651

Re: How to Read the HTML of a remote Web Page

johnyM456

johnyM456

Re: How to Read the HTML of a remote Web Page

cms9651

Re: How to Read the HTML of a remote Web Page

johnyM456

johnyM456

Re: How to Read the HTML of a remote Web Page

This site is managed for Microsoft by Neudesic, LLC. | © 2013 Microsoft. All rights reserved.

Follow Us On:

Microsoft