Example Analysis of Sharp Crawler configuring Agent ip
In this issue, the editor will bring you an example analysis of the Sharp crawler configuration agent ip. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.
/ / destination page string targetUrl = "http://httpbin.org/ip"; / / proxy server string proxyHost =" http://t.16yun.cn"; string proxyPort = "6447"; / / proxy tunnel authentication information string proxyUser = "16IXNGXO"; string proxyPass = "366338" / / set proxy server WebProxy proxy = new WebProxy (string.Format ("{0}: {1}", proxyHost, proxyPort), true); ServicePointManager.Expect100Continue = false; var request = WebRequest.Create (targetUrl) as HttpWebRequest; request.AllowAutoRedirect = true; request.KeepAlive = true; request.Method = "GET"; request.Proxy = proxy; / / request.Proxy.Credentials = CredentialCache.DefaultCredentials Request.Proxy.Credentials = new System.Net.NetworkCredential (proxyUser, proxyPass); / / set Proxy Tunnel / / Random ran=new Random (); / / int tunnel = ran.Next (1pm 10000); / / request.Headers.Add ("Proxy-Tunnel", String.valueOf (tunnel)); / / request.Timeout = 20000; / / request.ServicePoint.ConnectionLimit = 512; / / request.UserAgent = "Mozilla/5.0 (Windows NT 6.3; Win64) X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.82 Safari/537.36 "; / request.Headers.Add (" Cache-Control "," max-age=0 "); / / request.Headers.Add (" DNT "," 1 "); / / String encoded = System.Convert.ToBase64String (System.Text.Encoding.GetEncoding (" ISO-8859-1 ")) .GetBytes (proxyUser +": "+ proxyPass)) / / request.Headers.Add ("Proxy-Authorization", "Basic" + encoded); using (var response = request.GetResponse () as HttpWebResponse) using (var sr = new StreamReader (response.GetResponseStream (), Encoding.UTF8)) {string htmlStr = sr.ReadToEnd ();} the above is an example of configuring proxy ip for the Sharp crawler program shared by you. If you happen to have similar doubts, please refer to the above analysis for understanding. If you want to know more about it, you are welcome to follow the industry information channel.
