Generators and the yield keyword

Everywhere I look, generator tutorials are doing a bad job of teaching the material. The reason for this is that generators are taught right from the iteration point of view (for/in).

The problem with this is it doesn’t help the beginner. A better example of generators is to look at them as a result of a sequence of events (step-by-step) instead of in a loop.

This is primarily for Python users (optimally with iPython).

A simple generator

Here is a simple generator function:

def gentastic():
    print("The generator hath begun!")
    yield "All hail the generator"
    print("The generator hath returned!")
    yield "All hail the generator"
    print("The generator is risen!")
    yield "All hail the generator"
    print("The generator hath now ended!")
    yield "It is right to give him thanks and praise"

Is this the most efficient generator? No. Does it love you? Of course it does.

Yield

When you use the yield keyword in a function it returns a generator object (it doesn’t just return that string) and immediately exits the scope it’s in. It yields control to the function that called it with a little gift sometimes. In our case we’re yielding with strings like “All hail the generator!” Those strings get wrapped into the generator object.

To get a better idea type the following:

awesome = gentastic()

Then type awesome again and hit return. You should see something like this:

<generator object gentastic at 0x100462370>

Boom. Generator object. Thanks yield.

Next

A generator object has many methods but the most important one is next.

Let’s manually call next on the previous awesome object:

print(awesome.next())

Here is the output:

The generator hath begun!
All hail the generator

I’m using the print function for a very specific reason. I want to see the string value that the generator object is carrying right away. I could also assign it like q = awesome.next() and it will give me that string (“All hail the generator!”) as the value of q.

Now let’s call the next method again:

print(awesome.next())

and I get this:

The generator hath returned!
All hail the generator

Did you see what happened? Isn’t that fantastic! When we called next() it brought us back into the function at a different starting point (after the previous yield) each time. That’s why yield isn’t a return statement, it remembers! It knows what you did last summer.

Calling awesome.next() again returns:

The generator is risen!
All hail the generator

The grand finale

Now, this is the typical example that follows tutorials on generators:

for a in gentastic():
    print(a)

More on this sequence and using iterables when I’m not feeling incredibly lazy and want to follow-up.

Update: In Python 3 the correct way to call next is not object.next() but rather next(object). Thanks to takluyver.