Python's method of using generators

This article mainly introduces the relevant knowledge of the method of python using generator, the content is detailed and easy to understand, the operation is simple and fast, and has a certain reference value, I believe you will have something to gain after reading this article on the method of using generator for python, let's take a look at it.

Use generators whenever possible

When dealing with a large amount of data on a single machine, the generator is often useful because it is read in small pieces one by one to maximize memory savings, such as using yield when crawling web pages

Import requestsimport redef get_pages (link): pages_to_visit = [] pages_to_visit.append (link) pattern = re.compile ('https?') While pages_to_visit: current_page = pages_to_visit.pop (0) page = requests.get (current_page) for url in re.findall ('