How to use meta parameter in python crawler

Editor to share with you how to use the meta parameter in python crawler. I hope you will get something after reading this article. Let's discuss it together.

Description

1. Meta can transfer data in different parsing functions, and the corresponding value of meta parameter must be a dictionary.

2. When meta is generated by Request, it is passed in and taken out from the Response object.

3. There is a fixed key proxy in meta dictionary.

Represents the proxy ip.

Example

Def parse (self, response):... Yield scrapy.Request (detail_url, callback=self.parse_detail, meta= {'item': item})... Def parse_detail (self,response): # get the previously passed item item = response.meta ['item'] after reading this article, I believe you have some understanding of "how to use the meta parameter in python crawler". If you want to know more about it, welcome to follow the industry information channel, thank you for reading!