How does Python crawl embarrassing encyclopedia short videos in batch?

This article mainly explains "how Python crawls embarrassing encyclopedia short videos". Interested friends might as well take a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "Python how to batch climb embarrassing encyclopedia short video" bar!

This goal

Climb an embarrassing encyclopedia video

Address

Https://www.qiushibaike.com/video/

Environment

Python3.6

Pycharm

Analyze the web page to get the real video address

1. Click the right mouse button to check, and you can see that there is a video address on the web page.

2. Right-click to view the source code and search the video address to see if the source code of the web page also has a video address.

Embarrassing encyclopedia is a static web page, which is relatively simple to crawl website videos.

Crawler code

Import tool

Import requestsimport reimport time

Request website

Url= 'https://www.qiushibaike.com/video/page/{}/'.format(page)headers = {' User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'} response = requests.get (url=url, headers=headers)

Analyze the website

For li in lis: video_url = 'https:' + li title = li.split (' /') [- 1] download (video_url, title)

Save data

Def download (url, title): path = 'save data address' + title response = requests.get (url=url) with open (path, mode='wb') as f: f.write (response.content)

Run the code, and the result is as follows

At this point, I believe you have a deeper understanding of "Python how to crawl embarrassing encyclopedia short videos". You might as well do it in practice! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!