What is the reasoning behind Python’s design decision to bind default arguments at function definition time rather than at function execution time, and how does this relate to the concept of “Least Astonishment” in programming language design?
In Python, the following function demonstrates unexpected behavior for beginners:
def foo(a=[]):
a.append(5)
return a
When called multiple times without arguments, instead of always returning [5], it produces:
>>> foo()
[5]
>>> foo()
[5, 5]
>>> foo()
[5, 5, 5]
This behavior seems counterintuitive because:
- Novices expect the default argument to be re-evaluated each time the function is called
- This can lead to bugs when mutable objects are used as default arguments
- The behavior differs from what might be expected based on other programming language paradigms
Consider this extended example:
def a():
print("a executed")
return []
def b(x=a()):
x.append(5)
print(x)
This demonstrates that the default argument is evaluated when the function is defined, not when it’s called.
The explanation suggests that binding defaults at definition time creates a more consistent model where everything in the function definition line is evaluated at the same time. An alternative approach would create a “hybrid” model where part of the binding happens at definition and part at invocation.
Python binds default arguments at function definition time rather than execution time to maintain consistency in the function definition model, where all default values are evaluated as part of the function creation process. This design decision, while technically consistent, often violates the principle of least astonishment because it leads to unexpected behavior with mutable default arguments that persist across function calls.
Contents
- What Happens Under the Hood
- The Rationale Behind Design Time Binding
- Definition Time vs Execution Time Models
- Least Astonishment Principle in Python
- PEP 671: The Solution for Late-Bound Defaults
- Best Practices and Workarounds
- Comparative Analysis with Other Languages
What Happens Under the Hood
When Python processes a function definition, it creates a function object and evaluates all default arguments at that moment. This means that the default values are computed once during function definition and stored as part of the function’s __defaults__ attribute. As the Reddit discussion explains, “default argument value binding occurs at compilation and not during execution.”
def foo(a=[]):
a.append(5)
return a
# The default value [] is created here, at function definition
print(foo.__defaults__) # ([],) - the same list object is used every time
This behavior creates a reference to the same object for all subsequent function calls. When you modify that object, the changes persist because you’re always working with the same reference.
The technical implementation shows that default arguments are treated as part of the function’s signature metadata, not as part of the function’s executable code. This is why they’re evaluated once, when the function is defined.
The Rationale Behind Design Time Binding
The core reason for binding defaults at definition time is consistency in the function definition model. In Python’s design philosophy, everything on the function definition line should be evaluated at the same time - during function creation. This creates a clear boundary between what happens at definition time versus what happens at execution time.
According to the Stack Overflow discussion, this approach avoids creating a “hybrid” model where part of the binding happens at definition and part at invocation. A hybrid model would be more complex to implement and understand.
The Python documentation explains that “due to Python’s aforementioned behavior concerning evaluating default arguments to functions,” the system maintains consistency by evaluating all defaults as part of the function definition process. This consistency simplifies the language’s semantics and implementation.
Additionally, this behavior aligns with how Python handles other aspects of function definitions. The function signature, including defaults, is treated as immutable metadata about the function, rather than executable code that runs each time the function is called.
Definition Time vs Execution Time Models
The distinction between definition time and execution time binding represents two different philosophical approaches to function default arguments:
Definition Time Binding (Python’s approach):
- All default values are computed once when the function is defined
- Default values become part of the function’s immutable signature
- The function object contains references to the default values
- This model is consistent but can be surprising with mutable objects
Execution Time Binding (alternative approach):
- Default values would be computed each time the function is called
- This would match the intuitive expectation of most beginners
- However, it would create a hybrid execution model
- It might have performance implications
As the Reddit comment notes, “There is no such thing as ‘definition time’, every time is runtime” - but Python’s implementation treats function definition as a special case where certain operations happen during compilation rather than pure runtime execution.
The Digital Cat blog explains that “while plain values are hardcoded, thus needing no evaluation except that made at compilation time, function calls are expected to be executed at run time.” This highlights the philosophical distinction between static and dynamic evaluation.
Least Astonishment Principle in Python
The principle of least astonishment (or least surprise) suggests that programming language behavior should be intuitive and predictable to users. In the context of Python’s default arguments, this principle is actually violated by the current implementation.
As noted in the Stack Overflow discussion, “For even more evidence that this a design flaw, if you Google ‘Python gotchas’, this design is mentioned as a gotcha, usually the first gotcha in the list.” This indicates that the behavior violates most programmers’ intuitive expectations.
The W3Docs resource states that “the principle of ‘least astonishment’ suggests that the behavior of a function should be as predictable and unsurprising as possible. In the context of default arguments in Python, this principle suggests that the value of a default argument should not change between function calls.”
However, proponents of Python’s design argue that the behavior is actually consistent once understood, and that the surprise factor comes from misunderstanding how Python works. The SourceBae article explains that “despite its initial surprises, it follows naturally from Python’s design philosophy where default arguments are evaluated at definition time.”
This creates a tension between technical consistency and intuitive behavior - Python’s design prioritizes technical consistency over what might be more intuitive for beginners.
PEP 671: The Solution for Late-Bound Defaults
Recognizing the tension between consistency and least astonishment, Python introduced PEP 671 - “Syntax for late-bound function argument defaults.” As the PEP documentation states, “Function parameters can have default values which are calculated during function definition and saved. This proposal introduces a new form of argument default, defined by an expression to be evaluated at function call time.”
PEP 671 proposes a syntax like:
def foo(a=[]): # Current behavior - evaluated at definition time
pass
def foo(a=()): # PEP 671 proposal - evaluated at call time
pass
This would allow developers to choose between definition-time and execution-time binding for their default arguments, solving the least astonishment problem while maintaining backward compatibility.
The PEP acknowledges that “the current behavior is surprising to many programmers” and that “late-bound defaults would provide a way to express the common pattern of ‘use this expression to compute the default value at call time.’”
However, PEP 671 is still a proposal and hasn’t been implemented in Python as of 2024. The discussion around it highlights the ongoing debate about how to balance consistency with intuition in language design.
Best Practices and Workarounds
Given Python’s current behavior, developers have adopted several best practices to avoid the mutable default argument gotcha:
- Use None as a sentinel value:
def foo(a=None):
if a is None:
a = []
a.append(5)
return a
- Use immutable defaults:
def foo(a=()):
a = list(a) # Convert to mutable if needed
a.append(5)
return a
The GeeksforGeeks article recommends that “Whenever a mutable type is provided as the default argument, assign it to be None value in the function head.”
The Hitchhiker’s Guide to Python suggests “creating a closure that binds immediately to its arguments by using a default arg” as a workaround.
These patterns have become so widespread that they’re considered standard Python practice, effectively working around the design limitation.
Comparative Analysis with Other Languages
Python’s behavior differs from many other programming languages:
- JavaScript: Evaluates default arguments at call time
- Ruby: Evaluates default arguments at call time
- C++: Evaluates default arguments at call time
- Java: Doesn’t have default arguments (method overloading instead)
As noted in the Stack Overflow discussion, “in contrast, if you Google ‘JavaScript gotchas’, the behaviour of default arguments in JavaScript is not mentioned as a gotcha even once.”
This suggests that Python’s approach is relatively unique among popular programming languages, which generally evaluate defaults at call time to match intuitive expectations.
However, Python’s approach has advantages too:
- It’s more efficient (defaults don’t need to be recomputed)
- It’s more consistent with the function-as-object model
- It avoids potential race conditions in multi-threaded code
The trade-off between efficiency and intuition continues to be a topic of discussion in the Python community.
Sources
- Why are default arguments evaluated at definition time? - Stack Overflow
- PEP 671 – Syntax for late-bound function argument defaults
- Least Astonishment and the Mutable Default Argument - Stack Overflow
- Python mutable default argument: Why? - Software Engineering Stack Exchange
- Least Astonishment and the Mutable Default Argument in Python - GeeksforGeeks
- Common Gotchas — The Hitchhiker’s Guide to Python
- Today I re-learned: Python function default arguments are retained between executions - Reddit
- Least astonishment in python: the mutable default argument - Reddit
- The Drawback of Python’s Changeable Default Arguments - Medium
- Default arguments in Python - GeeksforGeeks
Conclusion
Python’s design decision to bind default arguments at function definition time stems from a philosophical commitment to consistency in the function definition model, where all default values are evaluated as part of the function creation process. While this approach creates technical consistency and efficiency, it often violates the principle of least astonishment because it leads to unexpected behavior with mutable objects that persist across function calls.
The key takeaways are:
-
Technical consistency vs. intuitive behavior: Python prioritizes technical consistency, where everything in the function definition line is evaluated once, over what might be more intuitive behavior for beginners.
-
Mutable vs. immutable defaults: The issue primarily affects mutable default arguments, which get shared between function calls, while immutable defaults work as expected.
-
PEP 671 as a potential solution: The proposed late-bound defaults syntax would allow developers to choose between definition-time and execution-time binding, addressing the least astonishment issue.
-
Established workarounds: The Python community has developed robust patterns (like using None as a sentinel) that effectively work around the limitation.
-
Unique among languages: Python’s approach differs from most other popular languages, which generally evaluate defaults at call time.
For developers, the key is understanding that Python’s behavior is intentional and consistent once you grasp the underlying model. The real challenge is balancing this technical consistency with the need for intuitive, beginner-friendly behavior - a challenge that continues to shape Python’s evolution.