How to format request headers in Python crawler

This article mainly introduces "how to format the request header in the Python crawler". In the daily operation, I believe that many people have doubts about how to format the request header in the Python crawler. The editor consulted all kinds of materials and sorted out a simple and useful method of operation. I hope it will be helpful to answer the doubt of "how to realize the request header formatting in the Python crawler". Next, please follow the editor to study!

1. When in use, copy and paste the Request Headers of the package crawled by the developer tool into the headerStr in the code.

2. After running the code, the Headers string will be printed after the format, which can be used directly in the code.

Example

Import re headerStr = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9 Accept-Encoding: gzip, deflate Accept-Language: zh-CN,zh Qroom0.9 Connection: keep-alive Cookie: * Host: paper.people.com.cn Referer: http://paper.people.com.cn/rmrb/html/2021-10/08/nbs.D110000renmrb_01.htm Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64 X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36''ret = "" for i in headerStr: if I ='\ n crawler: I = ",\ n'" ret + = I ret = re.sub (":", "':", ret) print (ret3:-3]) this ends the study of "how to format request headers in Python crawlers" I hope I can solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!