Accessing Return Values of Patched Methods in Python Mock

When working with Python’s Mock library, accessing return values of patched methods requires understanding how to properly implement spy functionality without triggering infinite recursion. The key is to store a reference to the original method before patching it, then use this reference in your side_effect function to capture and log the return value while avoiding recursive calls.

Contents

• Understanding Python Mock Library and Patching
• Accessing Return Values of Patched Methods
• Implementing Spy Functionality Without Recursion
• Advanced Mock Techniques and Best Practices
• Common Pitfalls and Solutions
• Sources
• Conclusion

Understanding Python Mock Library and Patching

Python’s unittest.mock library provides powerful tools for creating mock objects and replacing parts of your system during testing. At its core, the mock library allows you to replace objects with Mock instances that track calls, arguments, and return values. This is particularly useful when you want to isolate the code you’re testing from its dependencies.

When you patch a method using patch(), you’re temporarily replacing the original method with a Mock object. This Mock object can be configured to return specific values, raise exceptions, or have its behavior defined by a function passed to the side_effect parameter. The challenge arises when you want to access the return value of the original method while still using the mock for testing purposes—this is where the spy pattern comes in.

A spy, in the context of mocking, is a special type of mock that records information about calls made to it (like arguments and return values) while still delegating to the original implementation. This is different from a stub, which simply returns predefined values without calling the original method.

Let’s look at a basic example of patching a method:

from unittest.mock import patch

# Original class with a method we want to spy on
class Calculator:
 def add(self, a, b):
 return a + b

# Test function
def test_calculator():
 with patch.object(Calculator, 'add') as mock_add:
 calculator = Calculator()
 result = calculator.add(2, 3)
 
 # The mock returns a default value of None
 print(f"Mock call: {mock_add.call_args}")
 print(f"Result: {result}")

test_calculator()

In this example, the add method is replaced with a mock, but we lose the original functionality. To maintain the original behavior while still spying on calls, we need a more sophisticated approach.

Accessing Return Values of Patched Methods

To access return values of patched methods, you need to understand how Python’s Mock library handles method calls and return values. When you patch a method, the mock object intercepts all calls to that method. By default, mocks return None, but you can configure them to return specific values or use a function to determine the return value.

The key to accessing return values lies in the side_effect attribute. As mentioned in the Python documentation, side_effect can be a function that is called instead of the mock. This function receives the same arguments as the mock and can return a value that the mock will then return to the caller.

Here’s how you can use side_effect to access return values:

from unittest.mock import patch

class Calculator:
 def add(self, a, b):
 print(f"Original add called with {a} and {b}")
 return a + b

def test_with_side_effect():
 with patch.object(Calculator, 'add') as mock_add:
 # Define a function that will be called instead of the mock
 def side_effect(a, b):
 print(f"Side effect called with {a} and {b}")
 # Call the original method and return its result
 return mock_add._mock_call(a, b)
 
 mock_add.side_effect = side_effect
 
 calculator = Calculator()
 result = calculator.add(2, 3)
 
 print(f"Final result: {result}")
 print(f"Mock calls: {mock_add.call_args_list}")

test_with_side_effect()

However, this approach has a critical flaw: it leads to infinite recursion. When you call mock_add._mock_call(a, b), you’re actually calling the mock, which triggers the side_effect again, creating an endless loop.

The correct approach, as explained in the Real Python tutorial, is to store a reference to the original method before patching it. This allows you to call the original method directly from your side_effect function without triggering the mock.

Implementing Spy Functionality Without Recursion

To implement a spy that logs return values without causing infinite recursion, you need to follow these steps:

1. Store a reference to the original method before patching it
2. Create a side_effect function that calls this original method
3. Capture and log the return value
4. Return the result from the original method

Here’s a complete implementation:

from unittest.mock import patch
import logging

# Set up logging to see our spy output
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

class Calculator:
 def add(self, a, b):
 return a + b
 
 def multiply(self, a, b):
 return a * b

def test_spy_pattern():
 # Step 1: Get a reference to the original method
 original_add = Calculator.add
 
 with patch.object(Calculator, 'add') as mock_add:
 # Step 2: Create a side_effect function
 def spy_side_effect(self, a, b):
 # Call the original method
 result = original_add(self, a, b)
 
 # Step 3: Log the return value
 logger.info(f"Spy: add({a}, {b}) returned {result}")
 
 # Step 4: Return the result
 return result
 
 mock_add.side_effect = spy_side_effect
 
 # Now use the calculator
 calculator = Calculator()
 
 # These calls will go through our spy
 result1 = calculator.add(2, 3)
 result2 = calculator.add(5, 7)
 
 # The mock still tracks all calls
 print(f"Mock call count: {mock_add.call_count}")
 print(f"Mock calls: {mock_add.call_args_list}")
 print(f"Results: {result1}, {result2}")

test_spy_pattern()

This approach works because we’re calling the original method directly through our stored reference, bypassing the mock entirely. This prevents infinite recursion while still allowing us to log the return values.

For more complex scenarios, you can use the wraps parameter, which is designed specifically for creating spies. The wraps parameter takes another object (like the original method) and forwards calls to it. Here’s how you can use it:

from unittest.mock import patch

class Calculator:
 def add(self, a, b):
 return a + b

def test_wraps_spy():
 with patch.object(Calculator, 'add', wraps=Calculator.add) as mock_add:
 calculator = Calculator()
 
 # These calls will go through the spy
 result1 = calculator.add(2, 3)
 result2 = calculator.add(5, 7)
 
 # The mock tracks calls and forwards them to the original method
 print(f"Mock call count: {mock_add.call_count}")
 print(f"Mock calls: {mock_add.call_args_list}")
 print(f"Results: {result1}, {result2}")
 
 # You can also access the return value from the mock
 last_call = mock_add.call_args_list[-1]
 print(f"Last call arguments: {last_call}")
 print(f"Last call result: {mock_add.return_value}")

test_wraps_spy()

The wraps parameter is particularly useful because it automatically handles the delegation to the original method while still providing all the mocking functionality. According to the GitHub source code, this is the recommended approach for creating spies in Python’s Mock library.

Advanced Mock Techniques and Best Practices

Once you’ve mastered the basic spy pattern, you can explore more advanced techniques for working with Python’s Mock library. These techniques will help you write more sophisticated tests and handle complex mocking scenarios.

1. Chained Method Calls

When dealing with chained method calls, you need to ensure each mock in the chain is properly configured. Here’s an example:

from unittest.mock import patch, MagicMock

class DataProcessor:
 def get_data(self):
 return {"items": [1, 2, 3]}
 
 def process_data(self, data):
 return [x * 2 for x in data["items"]]

def test_chained_calls():
 # Create a spy for get_data
 with patch.object(DataProcessor, 'get_data', wraps=DataProcessor.get_data) as mock_get_data:
 with patch.object(DataProcessor, 'process_data', wraps=DataProcessor.process_data) as mock_process_data:
 processor = DataProcessor()
 
 # This call will trigger both spies
 result = processor.process_data(processor.get_data())
 
 print(f"get_data called: {mock_get_data.called}")
 print(f"process_data called: {mock_process_data.called}")
 print(f"Final result: {result}")

test_chained_calls()

2. Conditional Spies

Sometimes you only want to spy on certain calls. You can implement conditional spies by checking the arguments in your side_effect function:

from unittest.mock import patch

class Calculator:
 def add(self, a, b):
 return a + b

def test_conditional_spy():
 original_add = Calculator.add
 
 with patch.object(Calculator, 'add') as mock_add:
 def conditional_side_effect(self, a, b):
 # Only log calls where the first argument is greater than 5
 if a > 5:
 print(f"Spy: add({a}, {b}) would return {a + b}")
 
 # Always call the original method
 return original_add(self, a, b)
 
 mock_add.side_effect = conditional_side_effect
 
 calculator = Calculator()
 
 # This call will be logged
 result1 = calculator.add(6, 3)
 
 # This call won't be logged
 result2 = calculator.add(2, 3)
 
 print(f"Results: {result1}, {result2}")

test_conditional_spy()

3. Return Value Manipulation

You can also use spies to modify return values before they’re returned to the caller:

from unittest.mock import patch

class Calculator:
 def add(self, a, b):
 return a + b

def test_return_value_manipulation():
 original_add = Calculator.add
 
 with patch.object(Calculator, 'add') as mock_add:
 def manipulative_side_effect(self, a, b):
 # Call the original method
 result = original_add(self, a, b)
 
 # Modify the result before returning
 return result + 1
 
 mock_add.side_effect = manipulative_side_effect
 
 calculator = Calculator()
 result = calculator.add(2, 3)
 
 # The result will be 7 instead of 6
 print(f"Modified result: {result}")

test_return_value_manipulation()

4. Async Method Spying

Python’s Mock library also supports async methods. Here’s how you can spy on async methods:

import asyncio
from unittest.mock import patch

class AsyncService:
 async def fetch_data(self):
 await asyncio.sleep(0.1) # Simulate async operation
 return {"data": "example"}

async def test_async_spy():
 original_fetch = AsyncService.fetch_data
 
 with patch.object(AsyncService, 'fetch_data', wraps=AsyncService.fetch_data) as mock_fetch:
 service = AsyncService()
 
 # Call the async method
 result = await service.fetch_data()
 
 print(f"Async method called: {mock_fetch.called}")
 print(f"Result: {result}")

# Run the async test
asyncio.run(test_async_spy())

5. Context Managers as Spies

You can also spy on context managers by patching the __enter__ and __exit__ methods:

from unittest.mock import patch

class DatabaseConnection:
 def __enter__(self):
 print("Connection opened")
 return self
 
 def __exit__(self, exc_type, exc_val, exc_tb):
 print("Connection closed")
 
 def query(self, sql):
 return f"Result for: {sql}"

def test_context_manager_spy():
 original_enter = DatabaseConnection.__enter__
 original_exit = DatabaseConnection.__exit__
 original_query = DatabaseConnection.query
 
 with patch.object(DatabaseConnection, '__enter__', wraps=original_enter) as mock_enter:
 with patch.object(DatabaseConnection, '__exit__', wraps=original_exit) as mock_exit:
 with patch.object(DatabaseConnection, 'query', wraps=original_query) as mock_query:
 with DatabaseConnection() as conn:
 result = conn.query("SELECT * FROM users")
 
 print(f"__enter__ called: {mock_enter.called}")
 print(f"__exit__ called: {mock_exit.called}")
 print(f"query called: {mock_query.called}")
 print(f"Query result: {result}")

test_context_manager_spy()

These advanced techniques demonstrate the flexibility of Python’s Mock library and how you can implement sophisticated spying patterns for your testing needs.

Common Pitfalls and Solutions

When working with Python’s Mock library and implementing spy functionality, you’ll likely encounter several common pitfalls. Understanding these issues and their solutions will help you write more robust tests and avoid frustrating debugging sessions.

1. Infinite Recursion

Problem: As mentioned earlier, the most common issue when implementing spies is infinite recursion. This happens when your side_effect function calls the mock instead of the original method.

Solution: Always store a reference to the original method before patching it, and call this reference directly from your side_effect function. Never call the mock from within its own side_effect.

# Wrong - causes infinite recursion
def bad_side_effect(self, a, b):
 return mock_add(a, b) # This calls the mock again!

# Right - stores reference to original method
original_add = Calculator.add

def good_side_effect(self, a, b):
 return original_add(self, a, b) # Calls the original method

2. Incorrect Argument Handling

Problem: When patching instance methods, you need to handle the self argument correctly. Many developers forget that instance methods receive the instance as the first argument.

Solution: Make sure your side_effect function accepts the correct number of arguments, including self for instance methods.

# Wrong - missing self parameter
def bad_side_effect(a, b): # Missing self!
 return original_add(self, a, b)

# Right - includes self parameter
def good_side_effect(self, a, b):
 return original_add(self, a, b)

3. Not Restoring Original Methods

Problem: If you don’t properly restore the original methods after patching, you can have unintended side effects on other tests or the application itself.

Solution: Always use the patch context manager or ensure you call stop() on your patch objects. The context manager is preferred as it automatically handles cleanup.

# Wrong - manual patch management that might not clean up
mock_patch = patch.object(Calculator, 'add')
mock_patch.start()
# ... do testing ...
# Might forget to call mock_patch.stop()

# Right - using context manager for automatic cleanup
with patch.object(Calculator, 'add') as mock_add:
 # ... do testing ...
 # Original method is automatically restored when exiting the block

4. Mocking Non-existent Methods

Problem: Trying to patch methods that don’t exist can lead to confusing errors or silent failures.

Solution: Always verify that the method you’re trying to patch actually exists. Use hasattr() or similar checks before patching.

# Wrong - might fail if method doesn't exist
with patch.object(SomeClass, 'nonexistent_method') as mock_method:
 # This will raise an AttributeError if the method doesn't exist

# Right - check if method exists first
if hasattr(SomeClass, 'method_to_patch'):
 with patch.object(SomeClass, 'method_to_patch') as mock_method:
 # Safe to patch

5. Overusing Mocks

Problem: It’s easy to fall into the trap of mocking everything, which can lead to tests that don’t actually verify the behavior of your code.

Solution: Follow the principle of “mock only what you need to control.” Focus on testing the behavior of your code rather than implementation details. If you find yourself mocking too many things, consider refactoring your code to be more testable.

# Wrong - mocking too many implementation details
def test_over_mocked():
 with patch('module.submodule.ClassA') as mock_a:
 with patch('module.submodule.ClassB') as mock_b:
 with patch('module.submodule.function_c') as mock_c:
 # Test that's probably testing implementation rather than behavior
 mock_a.return_value.method_x.return_value = "expected"
 mock_b.return_value.method_y.return_value = "expected"
 result = some_function()
 assert result == "expected"

# Right - focused on behavior
def test_behavior():
 # Mock only what's necessary to isolate the code under test
 with patch('module.external_dependency') as mock_dependency:
 mock_dependency.return_value.get_data.return_value = {"key": "value"}
 
 # Test the actual behavior
 result = some_function()
 assert result["processed_key"] == "processed_value"

6. Not Considering Thread Safety

Problem: If you’re using mocks in a multithreaded environment, you might encounter issues with shared state between threads.

Solution: Be careful when using mocks that maintain state (like call counts) in multithreaded tests. Consider using thread-local storage or synchronizing access to shared mock state.

import threading

def test_thread_safety():
 shared_mock = MagicMock()
 results = []
 
 def worker():
 result = shared_mock(1, 2)
 results.append(result)
 
 threads = []
 for _ in range(5):
 t = threading.Thread(target=worker)
 threads.append(t)
 t.start()
 
 for t in threads:
 t.join()
 
 print(f"Total calls: {shared_mock.call_count}")
 print(f"Results: {results}")

7. Ignoring Mock Documentation

Problem: Python’s Mock library is quite powerful, but many developers only use the basic features and miss out on advanced functionality.

Solution: Take the time to read the official documentation and explore all the features available. The wraps parameter, for example, is specifically designed for creating spies and can save you a lot of work.

By being aware of these common pitfalls and their solutions, you can write more effective tests using Python’s Mock library and avoid many of the frustrations that come with advanced mocking techniques.

Sources

1. Python documentation — Comprehensive reference for Python’s unittest.mock library: https://docs.python.org/3/library/unittest.mock.html
2. Real Python — In-depth tutorial on Python’s mock library with practical examples: https://realpython.com/python-mock-library/
3. GitHub Source Code — Implementation details of unittest.mock in Python’s source code: https://github.com/python/cpython/blob/main/Lib/unittest/mock.py

Conclusion

Accessing return values of patched methods in Python’s Mock library is essential for implementing effective spy functionality. By understanding how to properly store references to original methods and use the side_effect attribute or wraps parameter, you can create spies that log return values without causing infinite recursion. This approach allows you to observe the behavior of your code while still maintaining the original functionality, making it a powerful tool for testing and debugging.

The key takeaways are: always store a reference to the original method before patching it, use this reference in your side_effect function to avoid recursion, and consider using the wraps parameter for a cleaner implementation. By following these practices and being aware of common pitfalls, you can leverage Python’s Mock library to write more comprehensive and reliable tests for your applications.