Python's docker crawler Technology-fiddler Analysis app crawling (12)

Original articles, welcome to reprint. Reprint please indicate: reproduced from IT Story Association, thank you!

Original link address: "docker practice" python's docker crawler technology-fiddler analysis app crawling (12)

Previously said that the use of Android simulator and grab package tools, before is actually in the knife sharpening, as the saying goes good knife sharpening does not miss the firewood, the next step is to grab the data.

Get ready to start fiddler

Native ip 192.168.1.122

Night God Simulator setup

Bridging

Wifi Settin

The ip queried by the windows system just now

App installation (XX cuisine)

Find something easier to grab and get started. Search the Night God Simulator market: Douguo, download the app in the picture.

Grab the target

Click on the category in the picture

Analyze the data of app

For a normal app, the requested connection is in the form of a domain name and is related to the name of your app.

For example: bean and fruit delicacies, app request connections are also dougou.

Enter fiddler, ctrl+f, and type dougou

Found a request, found that the return is Unicode encoding, need to do some decoding

Copy the returned result and encode it to json.cn for viewing

Classification discovery is consistent with app, and any dynamic development that app wants to do is obtained through requests.

View details

A rule was found in the drop-down category of beef. It turned out to be 15 items per page. Haha, there is a rule in url.

PS: this time monitor and discover the rules of the app request through fiddler. Next time start writing python scripts to crawl data.