How does Python crawl the information of new real estate all over the world?

This article mainly introduces Python how to climb the new real estate information, the article is very detailed, has a certain reference value, interested friends must read it!

Preface

Import requestsfrom lxml import etreeimport refrom bs4 import BeautifulSoupimport openpyxlimport csvdef get_price (): headers = {'User-Agent':' Mozilla/5.0 (Windows NT 10.0; Win64; x64) Rv:73.0) Gecko/20100101 Firefox/73.0', 'Refrer':' https://ganzhou.newhouse.fang.com/house/s/b9{}/',} for i in range (1,10): url = "https://ganzhou.newhouse.fang.com/house/s/b9{}/".format(str(i)) response = requests.get (url) Headers=headers) if response.status_code = = 200: xml = etree.HTML (response.content.decode ('gbk')) name = xml.xpath (' / / div [@ class= "nl_con clearfix"] / / div [@ class= "nlc_details"] / a [@ data-yd= "] / / text ()') for index in range (len (name)): name [index] = name [index] .strip () address = xml.xpath ('/ / div [@ class= "nl_con clearfix"] / / div [@ class= "nlc_details"] / / div [@ class= "address"] / a/text ()') for index in range (len (address)): address [index] = addre ss.strip () price = xml.xpath ('/ / / div [@ class= "nl_con clearfix"] / / div [@ class= "nlc_details"] / / div [@ class= "nhouse_price"] / span/text () 'for index in range (len (price)): price [index] = price.strip () with open (' Ganzhou house price .csv' 'w') as f: writer = csv.writer (f) writer.writerows (zip (name, price, address)) f.close () if _ _ name__ = ='_ _ main__': get_price () these are all the contents of the article "how Python crawls new buildings all over the world" Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!