The method of using XPath to crawl novels in python
Most people do not understand the knowledge points of this article "how to use XPath to climb a novel in python", so the editor summarizes the following content, detailed content, clear steps, and has a certain reference value. I hope you can get something after reading this article. Let's take a look at this "python method of using XPath to climb a novel".
Description
Xpath is a language that searches for information in XML documents.
Process
1. Get the title and author test in the browser.
2. Install the xpath plug-in in the browser:
3. Find book-mid-info in html:
4. Find the name and author of the novel we want.
Namely / / div [@ class='book-mid-info'] / h5/a/txt ()
Example
# author: # Development time: 2021 + 4-pound-8-pound-0008-8:24-import requestsfrom lxml import etreeurl= "https://www.qidian.com/rank/yuepiao"headers={'user-agent':'Mozilla/5.0 (Windows NT 10.0 WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3861.400 QQBrowser/10.7.4313.400'} # send request resp=requests.get (url,headers) e=etree.HTML (resp.text) # Type conversion Transform str into class' lxml.etree._ELementprint (type (e)) names=e.xpath ('/ / div [@ class= "book-mid-info"] / h5/a/text ()') authors=e.xpath ('/ / p [@ class= "author"] / a [1] / text ()') print (names) print (authors) # name and author correspondence for name,authors in zip (names,authors): print (name, ":" Authors) these are the contents of this article on "how to use XPath to crawl novels in python" I believe we all have a certain understanding. I hope the content shared by the editor will be helpful to you. If you want to know more about the relevant knowledge, please pay attention to the industry information channel.
