What Does the 'yield' Keyword Do in Python?

The yield keyword in Python transforms a regular function into a generator function, which produces a sequence of values one at a time instead of returning them all at once. When a function contains yield, it becomes a generator that can be iterated over, pausing execution at each yield statement and preserving its local state for the next iteration. This makes generators particularly efficient for handling large datasets or infinite sequences, as they only compute and provide values when needed rather than storing everything in memory.

Contents

• What is the yield keyword?
• How yield transforms functions into generators
• Understanding the flow of execution
• Analyzing your specific code example
• Advanced yield features
• Practical applications and benefits

What is the yield keyword?

The yield keyword in Python is special because it marks a function as a generator function. Unlike regular functions that use return to send back a value and terminate, generator functions use yield to produce a value and temporarily suspend their execution. The key difference is that generator functions maintain their state between calls, allowing them to resume exactly where they left off.

When Python encounters yield, it does the following:

1. Pauses function execution at that point
2. Returns the value specified after yield
3. Preserves all local variables and the execution context
4. Waits for the next iteration to continue from where it stopped

This behavior makes generators incredibly memory-efficient, as they don’t need to store all values in memory at once. Instead, they produce values lazily, one at a time, only when requested.

How yield transforms functions into generators

Any function containing yield automatically becomes a generator function. When you call a generator function, it doesn’t execute the function body immediately. Instead, it returns a generator object that can be iterated over.

Here’s the fundamental difference:

# Regular function
def regular_function():
    return [1, 2, 3]
    # Returns complete list all at once

# Generator function  
def generator_function():
    yield 1
    yield 2
    yield 3
    # Returns generator object, produces values one at a time

According to Real Python’s documentation, “Generator functions look and act just like regular functions, but with one defining characteristic. Generator functions use the Python yield keyword instead of return.”

The transformation process works like this:

1. Function definition with yield → becomes generator function
2. Calling generator function → returns generator object (not executed yet)
3. Iterating over generator → executes function until first yield
4. Each subsequent iteration → resumes from last yield statement

Understanding the flow of execution

The execution flow of a generator function is quite different from a regular function. Let’s trace through a simple example:

def simple_generator():
    print("Generator starting...")
    yield "first value"
    print("After first yield")
    yield "second value"
    print("After second yield")
    yield "third value"
    print("Generator ending")

# Usage
gen = simple_generator()  # Generator object created, function not executed
print(f"Generator object: {gen}")

# First iteration
value1 = next(gen)  # Executes until first yield
print(f"Got: {value1}")

# Second iteration  
value2 = next(gen)  # Resumes after first yield
print(f"Got: {value2}")

# Third iteration
value3 = next(gen)  # Resumes after second yield
print(f"Got: {value3}")

# Fourth iteration would raise StopIteration

This output would be:

Generator object: <generator object simple_generator at 0x7f8c1a1b3e50>
Generator starting...
Got: first value
After first yield
Got: second value
After second yield
Got: third value
Generator ending

As Mozilla Developer Network explains, generators are special iterators that allow values to be produced lazily, one at a time, instead of returning them all at once. This lazy evaluation is what makes them so powerful for memory management.

Analyzing your specific code example

Now let’s break down your code example to answer your specific questions:

def _get_child_candidates(self, distance, min_dist, max_dist):
    if self._leftchild and distance - max_dist < self._median:
        yield self._leftchild
    if self._rightchild and distance + max_dist >= self._median:
        yield self._rightchild  

And its caller:

result, candidates = [], [self]
while candidates:
    node = candidates.pop()
    distance = node._get_dist(obj)
    if distance <= max_dist and distance >= min_dist:
        result.extend(node._values)
    candidates.extend(node._get_child_candidates(distance, min_dist, max_dist))
return result

1. What happens when the method _get_child_candidates is called?

When _get_child_candidates is called, it doesn’t execute immediately. Instead, it returns a generator object that will produce child nodes when iterated. The function body only executes when the generator is actually consumed.

2. Does it return a list, a single element, or something else?

It returns neither a list nor a single element. It returns a generator object that can produce values one at a time. When you call candidates.extend(), Python automatically iterates over this generator, consuming all the yielded values and adding them to the candidates list.

3. How many times is it called, and when do subsequent calls stop?

The method is called once per node during the traversal. Each call produces all qualifying child nodes for that specific node. The calling loop continues until the candidates list is empty, meaning the traversal is complete.

4. What is the relationship between yield and Python generators?

The yield keyword is what creates generators in Python. As guru99 explains, “The yield keyword converts the expression given into a generator function that gives back a generator object.” Your example demonstrates a perfect use case for generators in tree traversal, where you want to explore child nodes lazily without building complete lists upfront.

Advanced yield features

Beyond basic yielding, Python offers several advanced features for working with generators:

yield from

Python 3.3 introduced yield from, which allows you to delegate to a sub-generator:

def nested_generator():
    yield "start"
    yield from inner_generator()  # Delegates to inner generator
    yield "end"

def inner_generator():
    yield "inner1"
    yield "inner2"

As mentioned in the Stack Overflow answer, “Use Python’s yield from expression to connect a nested generator to your caller.”

Sending values to generators

Generators can also receive values through the send() method:

def receiver():
    while True:
        value = yield
        print(f"Received: {value}")

gen = receiver()
next(gen)  # Prime the generator
gen.send("hello")  # Send value to generator

Generator expressions

Similar to list comprehensions, but with parentheses instead of brackets:

# List comprehension (creates full list in memory)
squares_list = [x**2 for x in range(10)]

# Generator expression (lazy evaluation)
squares_gen = (x**2 for x in range(10))  # Parentheses, not brackets

Practical applications and benefits

Generators with yield are particularly useful in several scenarios:

Memory efficiency

For large datasets, generators save significant memory because they only hold one value in memory at a time.

Infinite sequences

Generators can represent infinite sequences that would be impossible to store:

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

# Can use in for loops with break conditions
fib = fibonacci()
for _ in range(10):
    print(next(fib))

Pipeline processing

Generators enable efficient data processing pipelines:

def read_lines(file_path):
    with open(file_path) as file:
        for line in file:
            yield line.strip()

def filter_lines(lines, min_length):
    for line in lines:
        if len(line) >= min_length:
            yield line

def process_data(file_path):
    lines = read_lines(file_path)
    filtered = filter_lines(lines, 10)
    return list(filtered)  # Consume generator when needed

Tree and graph traversal

As your example demonstrates, generators are excellent for tree traversal algorithms, allowing elegant and memory-efficient implementations of BFS, DFS, and other traversal methods.

Conclusion

The yield keyword fundamentally changes how functions work in Python, transforming them into generators that can produce values lazily. Key takeaways include:

1. Generator creation: Any function with yield becomes a generator function that returns a generator object when called
2. Lazy evaluation: Generators produce values one at a time, only when needed, making them memory-efficient
3. State preservation: Generators maintain their execution state between calls, resuming exactly where they left off
4. Infinite sequences: Generators can represent infinite sequences that would be impossible to store in memory
5. Pipeline processing: Generators enable efficient data processing pipelines with minimal memory overhead

In your tree traversal example, the _get_child_candidates method uses yield to create a generator that produces child nodes on demand, allowing the main algorithm to explore the tree structure efficiently without building complete node lists upfront. This pattern is particularly powerful for large data structures where memory efficiency is crucial.

To master generators, practice creating simple generator functions and experiment with different iteration patterns. Start with basic sequences and gradually move to more complex use cases like tree traversal, data processing pipelines, and infinite sequences.

Sources

1. How to Use Generators and yield in Python – Real Python
2. yield Keyword - Python - GeeksforGeeks
3. Python yield Keyword: What Is It and How to Use It? | DataCamp
4. Python Yield: How It Works and Why It’s Useful | Simplilearn
5. Python yield Keyword | W3Schools
6. Python Generators (With Examples) | Programiz
7. Yield in Python Tutorial: Generator & Yield vs Return Example | Guru99
8. Python generators and the yield keyword | Python Central
9. Python : Yield Keyword & Generators explained with examples - thisPointer
10. Python inorder traversal with generator(yield , yield from) | LeetCode