# Python yield Keyword and Generator Functions Explained

> Understand the Python yield keyword and generator functions through iterables, iterators, and lazy evaluation, with examples comparing return and yield behavior

**URL:** https://sentry.io/answers/python-yield-keyword/

---

## The Problem

What does the `yield` keyword in Python do? And how do the generator functions work?

## The Solution

In order to understand what `yield` does, we must first understand how generators work. And to understand generators, we must understand what iterables and iterators are.

For example, consider a staircase. A staircase has steps, and the way you climb the staircase is by climbing the steps one by one until you reach the top.

An alternative way of climbing the stairs is to take several steps at a time. You may take two, three, or even four steps at a time, depending on the length of your legs. The length of your legs here is the limitation. If the staircase is longer than your legs, you are forced to climb the staircase in chunks.

In programming, the limitation is your computer's memory.

Let's say we have a list:

```python
my_list = [1, 2, 3, 4, 5, 6]
```

It is a small list and most computers' memories can store it without any problem. But if we are working with significant amounts of data, reading a large file for example, your computer may not be able to store the data in its memory.

Generator functions are a solution to overcome this limitation. By generating values on the fly, generators allow you to handle large datasets with minimal consumption of memory and processing cycles.

Just like going over a staircase in chunks when the staircase is longer than your legs.

Now that we have an intuitive understanding, let's learn more.

### Iterables

In Python, all sequence types (lists, tuples, and strings) and some container objects (dictionaries, sets, and file objects) are like staircases. You can visit each element one by one. These are called iterables. Iterables contain a countable number of values that are stored in memory.

### Iterators

An iterator is an object that implements the iterator protocol, which consists of the methods `__iter__()` and `__next__()`.

When you use a for loop to iterate over an iterable like a list, the iterable object is passed to the [built-in `iter()` function](https://docs.python.org/3/library/functions.html?highlight=iter#iter) (different from `__iter__()), which returns an iterator object. Although the iterable object contains the items, the iterator object keeps track of which item is next to be used in a loop.

### Generator Functions

Generator functions are a special kind of function that return a [lazy iterator](https://en.wikipedia.org/wiki/Lazy_evaluation) called a generator iterator. These are objects that you can loop over like a list. However, unlike lists, lazy iterators do not store their contents in memory.

A generator iterator generates and returns a value on each call of its `__next__()` method. When the object runs out of values to return, it raises a `StopIteration` exception.

### The `yield` Keyword

The `yield` keyword in Python controls the flow of a generator function. This is similar to a return statement used for returning values in Python. However, there is a difference.

When you call a function that has a `yield` statement, as soon as a `yield` is encountered, the execution of the function halts and returns a generator iterator object instead of simply returning a value. **The state of the function, which includes variable bindings, the instruction pointer, the internal stack, and a few other things, is saved.**

In other words, the `yield` keyword will convert an expression that is specified along with it to a generator iterator, and return it to the caller.

If you want to get the values stored inside the generator object, you need to iterate over it. You can iterate over it using for loops or special [functions like `next()`](https://docs.python.org/3/library/functions.html?highlight=iter#next).

### Example

Here's a basic example that explains the difference between using a regular function with `return` and a generator function with `yield`:

```powershell

>>> def plain_old_func():
...     my_list = [1, 2, 3]
...     for i in my_list:
...             return i * 2
...
>>> plain_old_func()
2
>>> next(plain_old_func)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'function' object is not an iterator
```

Notice the function execution stopped at the first `return`.

```powershell

>>> def fancy_generator():
...     my_list = [1, 2, 3]
...     for i in my_list:
...             yield i * 2
...
>>> mygen = fancy_generator()
>>> print(mygen)
<generator object fancy_generator at 0x00000257344D9A10>
>>> next(mygen)
2
>>> next(mygen)
4
>>> next(mygen)
6
>>> next(mygen)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
```

Notice the function execution was paused and could be resumed again by iterating over the generator iterator.

You could also use a for loop to iterate over the generator iterator all at once:

```
>>> for x in fancy_generator():
...   print(x)
>>> 2
>>> 4
>>> 6
```

### Further Reading

- <https://docs.python.org/3/reference/simple_stmts.html#yield>
- <https://docs.python.org/3/reference/expressions.html#yieldexpr>
- <https://docs.python.org/3/glossary.html#term-generator>
- <https://realpython.com/introduction-to-python-generators/>

---

*Source: [sentry.io/answers/python-yield-keyword/](https://sentry.io/answers/python-yield-keyword/)*