Shulou(Shulou.com)06/01 Report--

This article mainly shows you how Python crawls MM photos. The content is simple and easy to understand. It is clearly organized. I hope it can help you solve your doubts. Let Xiaobian lead you to study and learn how Python crawls MM photos.

1. Format of URL

The URL we're using here is http://mm.taobao.com/json/request_top_list.htm? page=1, the base address is in front of the question mark, and the parameter page is the number of pages. You can change the address at will. Click open, you will find some Taobao MM profile, and with hyperlinks to personal details page.

We need to grab this page's avatar address, MM name, MM age, MM residence, and MM's personal details page address.

2. Capture brief information

I believe that after several actual battles, we are very familiar with the address of the grab and extract page. There is no difficulty here. We first grab the MM details page address of this page, name, age, etc. The information is printed out directly. The code is as follows:

import urllibimport urllib2import reclass Spider: def __init__(self): self.siteURL = 'http://mm.taobao.com/json/request_top_list.htm' def getPage(self,pageIndex): url = self.siteURL + "? page=" + str(pageIndex) print url request = urllib2.Request(url) response = urllib2.urlopen(request) return response.read().decode('gbk') def getContents(self,pageIndex): page = self.getPage(pageIndex) pattern = re.compile('

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.