ASP.NET抓取網(wǎng)頁(yè)內(nèi)容的實(shí)現(xiàn)方法

    這篇文章主要介紹了ASP.NET抓取網(wǎng)頁(yè)內(nèi)容的實(shí)現(xiàn)方法,涉及使用HttpWebRequest及WebResponse抓取網(wǎng)頁(yè)內(nèi)容的技巧,需要的朋友可以參考下
    代碼如下:
    代碼如下:
    本文實(shí)例講述了ASP.NET抓取網(wǎng)頁(yè)內(nèi)容的實(shí)現(xiàn)方法。分享給大家供大家參考。具體實(shí)現(xiàn)方法如下：
    一、ASP.NET 使用HttpWebRequest抓取網(wǎng)頁(yè)內(nèi)容
    代碼如下:
    /// <summary>方法一：比較推薦
    /// 用HttpWebRequest取得網(wǎng)頁(yè)源碼
    /// 對(duì)于帶BOM的網(wǎng)頁(yè)很有效，不管是什么編碼都能正確識(shí)別
    /// </summary>
    /// <param name="url">網(wǎng)頁(yè)地址" </param>
    /// <returns>返回網(wǎng)頁(yè)源文件</returns>
    public static string GetHtmlSource2(string url)
    {
    //處理內(nèi)容
    string html = "";
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.Accept = "*/*"; //接受任意文件
    request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322)"; //
    request.AllowAutoRedirect = true;//是否允許302
    //request.CookieContainer = new CookieContainer();//cookie容器，
    request.Referer = url; //當(dāng)前頁(yè)面的引用
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    Stream stream = response.GetResponseStream();
    StreamReader reader = new StreamReader(stream, Encoding.Default);
    html = reader.ReadToEnd();
    stream.Close();
    return html;
    }
    代碼如下:
    二、ASP.NET 使用 WebResponse 抓取網(wǎng)頁(yè)內(nèi)容
    代碼如下:
    public static string GetHttpData2(string Url)
    {
    string sException = null;
    string sRslt = null;
    WebResponse oWebRps = null;
    WebRequest oWebRqst = WebRequest.Create(Url);
    oWebRqst.Timeout = 50000;
    try
    {
    oWebRps = oWebRqst.GetResponse();
    }
    catch (WebException e)
    {
    sException = e.Message.ToString();
    }
    catch (Exception e)
    {
    sException = e.ToString();
    }
    finally
    {
    if (oWebRps != null)
    {
    StreamReader oStreamRd = new StreamReader(oWebRps.GetResponseStream(), Encoding.GetEncoding("utf-8"));
    sRslt = oStreamRd.ReadToEnd();
    oStreamRd.Close();
    oWebRps.Close();
    }
    }
    return sRslt;
    }
    代碼如下:
    希望本文所述對(duì)大家的C#程序設(shè)計(jì)有所幫助。