Shulou(Shulou.com)06/02 Report--

Today, I would like to talk to you about the data of Python climbing Weibo in Taihang Mountain, which may not be well understood by many people. in order to let you know more, the editor has summarized the following for you. I hope you can get something according to this article.

One of the biggest functions of the crawler is to integrate data, to get more comprehensive information, and to really do a good job in big data's analysis. In this era of data speaking, the impact is decisive. (be careful not to infringe)

♦ thought flow

1. Use chrome browser to get your own cookie.

2. Get the Weibo User_id of the user you want to crawl

3. Fill in the two items obtained into weibo.py, replace YOUR_USER_ID and # YOUR_COOKIE in the code, and run the code.

♦ complete code

Import requestsimport reimport pandas as pdimport time as tmimport random#-id= "2304132803301701" timedata = [] for p in range (1Power3): page= str (p) url = "https://m.weibo.cn/api/container/getIndex?containerid=" + id +" _ _ WEIBO_SECOND_PROFILE_WEIBO&luicode=10000011&lfid= "+ id +" & page_type=03&page= "+ page data = requests .get (url) data_text = data.text data_num = re.findall (r'\ "mid\"\:\ "(\ d {16})\"' Data_text) num = len (data_num) for i in range (0 Num): url_detail = "https://m.weibo.cn/detail/" + data_ Numi] html = requests.get (url_detail) time = re.search (r'\" created_at\ "\:\ s\" (\ w\ s\ w\ d\:\ d\ d)\ s\ +\ d {4}\ s\ d {4}\ d Html.text) timedata.append (time.group (1)) tm.sleep (random.uniform (1p4)) # Anti-crawling interval print ("collecting Weibo data from% d page% d"% (pcent I)) name = ["time"] data_save = pd.DataFrame (columns=name, data=timedata) data_save.to_csv ('. / data.csv')

Use wordcloud and other software to generate word clouds, which will display the font size of keywords according to the frequency and weight of the information.

After reading the above, do you have any further understanding of the data of Python crawling Weibo in Taihang Mountain? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.