Generators

• Writing a class-based iterator requires __iter__() and __next__(), plus manual state management and StopIteration handling.
• Generator functions let you express the same logic in plain Python functions, using yield to produce values one at a time.
• Any function with yield becomes a generator: calling it returns a generator object (an iterator) without running its body immediately.

def count_up_to(limit):
    """Generates numbers from 1 up to (and including) the limit.

    Args:
        limit (int): The upper limit for counting.

    Returns:
        generator(int): The generator to lazily count up to limit.
    """
    print("Generator function started...")
    n = 1

    while n <= limit:
        print(f"Yielding {n}")
        yield n
        print(f"Resumed after yielding {n}.")
        n += 1

    print("Generator function finished.")

count_gen = count_up_to(3)
print(f"Returned object: {count_gen} of type {type(count_gen)}")

print("First call to next outside of for loop.")
next(count_gen)

print("Remaining output from for loop.")
for number in count_gen:
    print(number)

Generator Functions & the yield Keyword

• A function becomes a generator by including yield; no other boilerplate is needed.
• Calling a generator function returns an object that implements __iter__() and __next__().
• The code inside runs only when iteration begins (e.g., in a for loop or via next()).

def filter_evens(data):
    """Yield only the even items from the input sequence.

    Args:
        data (iterable(int or float)): The data to iterate through and filter.

    Returns:
        generator(int or float): A generator object that yields the even items.
    """
    print("filter_evens: starting")

    for item in data:
        if item % 2 == 0:
            print(f"filter_evens: yielding {item}")
            yield item

    print("filter_evens: finished")

evens_from_range = filter_evens(range(6))

print(f"Generator object created: {evens_from_range}")

for num in evens_from_range:
    print(f"Received even: {num}")

evens_from_list = filter_evens([0, 1, 2, 3, 4, 5])

print(f"Generator object created: {evens_from_list}")

for num in evens_from_list:
    print(f"Received even: {num}")

How yield Works: Pause and Resume

• On each next() (or loop iteration), execution runs until it hits yield, returns the value, then pauses with all local state intact.
• The next next() call resumes immediately after the yield, preserving variables and the instruction pointer.
• When the function ends (no more yield), a StopIteration is raised automatically.

def demo_three_yields():
    """Demonstrate how having multiple yield statements work."""
    print("Generator started")
    yield 1
    print("Generator resumed after yielding 1.")
    yield 2
    print("Generator resumed after yielding 2.")
    yield 3
    print("Generator finished.")

demo_gen = demo_three_yields()

print(next(demo_gen))
print(next(demo_gen))
print(next(demo_gen))
# print(next(demo_gen)) # Uncommenting will raise a StopIteration Exception because there are no more yields

Generator State

• Generators keep their local variables alive between yields, making explicit state objects unnecessary.
• This persistent state allows infinite or long-running sequences without full data storage.

count_gen = count_up_to(5)

print("First call to next outside of for loop.")
print(next(count_gen))

print("Second call to next outside of for loop - now the value yielded is 2.")
print(next(count_gen))

print("Remaining output from for loop - prints from 3 onwards.")
for number in count_gen:
    print(number)

count_gen = count_up_to(5)

# Since generators have state, using the same generator object in nested loops can lead to issues.
# The inner for loop will complete the iteration, and the outer for loop will have a sinle pass.
for num in count_gen:
    for num2 in count_gen:
        print(f" - {num}:{num2}")

# The solution to this is to use distinct generator objects.
for num in count_up_to(5):
    for num2 in count_up_to(5):
        print(f" - {num}:{num2}")

Exhaustion

• Once a generator’s code path completes (falls off the end or hits return), further next() calls immediately raise StopIteration.
• A for loop over an exhausted generator does nothing on subsequent passes—you must call the function again for a fresh iterator.

count_gen = count_up_to(2)

print(next(count_gen))
print(next(count_gen))

try:
    print(next(count_gen)) # Will raise StopIteration exception
except StopIteration:
    print("Generator finished")

# Nothing will happen because the generator is already exhausted
for number in count_gen:
    print(number)