Python – Understanding Generators

Many functions in Python use or generate in-memory lists for iteration.  This is fine for relatively small list, but for large lists it becomes a problem.

Lazy Lists

A lazy list is when the next item is supplied when needed, and not before.  If we stop processing before reaching the end of the list then those unused items will never be generated or use up resources. It is also possible to run an endless list. Lazy lists are implemented by supplying an iterator to the list, rather than the whole list itself.

Simple Example

The function simp returns a generator , calling next() return the next value

We can use the generator like a list object:

Instead of building a big list, just generate the next value:

For example, return a list:

And with generator:

Note: I didn’t use a for loop because in python 2 range() function returns a list and in python 3 returns a generator (use xrange() in python 2 to get a generator)

Endless sequence 

Sometimes we receive data from a stream and want to handle it like a sequence , with generator its easy:

You can replace it with a function that receives a message from a socket and return it etc.

Co-routines via Enhanced Generators

You can use the send() generator method to sends a value back to the generator function, which can be picked-up as the return value from yield().

When the send() method is not used, yield returns None.


As you can see, using send we change the next value. The inner loop is finished, num gets the value we sent and continue

List Comprehensions as Generators

An alternative syntax to yield, we can use list comprehension , use parentheses instead of squared brackets. You can’t send value to the generator



Implementing a Custom Iterable Class

You can write a new class that behaves like an iterator. For example if your class holds a data structure and we want to provide a way to iterate over its internal structure without knowing how it implemented.


We use a simple list inside the class but we can change it to any other data structure without worry as long as we reimplement the inner class functions




Leave a Reply

Your email address will not be published. Required fields are marked *