Return function

"Reverse output" capability of higher-order functions

We talked about high-order functions before, which can be used to abstract general logic by passing functions as parameters. But there is a more interesting direction for higher-order functions: returning functions as results.

This design of "encapsulating logic first, then passing in data, and delaying triggering" can help us implement delayed calculation, dynamically generate similar functions, and even encapsulate state variables-this leads to the classic and practical closure concept in Python.


Look at "delayed triggering" from a summation scenario

Let’s look at the most intuitive example of delayed calculation first: we need to collect a bunch of numbers at once, but we don’t want to calculate the sum immediately, but wait until we actually need to use it.

Normal version: Calculate immediately

def calc_sum(*args):
    total = 0
    for n in args:
        total += n
    return total

print(calc_sum(1, 3, 5, 7, 9))  # 输出 25

This method is simple and direct, but the result will be calculated immediately every time it is called, lacking the flexibility of "waiting until the right time to calculate".

Advanced version: return the function and call it again

We can separate the "accumulation logic" and "trigger timing":

def lazy_sum(*args):
    def sum_logic():
        total = 0
        for n in args:
            total += n
        return total
    return sum_logic   # 返回函数对象,不执行

The calling method becomes two steps:

  1. Pass in parameters and get an "exclusive calculation function"
  2. Call it explicitly when needed to actually perform the calculation.
worker = lazy_sum(1, 3, 5, 7, 9)
print(worker)   # <function lazy_sum.<locals>.sum_logic at 0x...>
print(worker()) # 25

This model is like "write the recipe on a note, put away the ingredient list, and wait until the meal is ready to cook according to the recipe" - the logic and parameters are sealed.


What exactly is closure?

abovelazy_suminsidesum_logic, which is a typical closure:

**Closure refers to an internal function that references the variables or parameters of the external function. Even if the external function has completed execution, these referenced variables will still "live" in the internal function and will not be recycled. **

In other words, closure can "remember" the external environment when it was born.

Two important features

1. Each time an external function is called, a new closure instance will be created.

w1 = lazy_sum(1, 3, 5)
w2 = lazy_sum(1, 3, 5)

print(w1 == w2)     # False(不同内存地址)
print(w1() == w2()) # True(计算结果相同)

Although created with the same parameters, the function objects returned each time are independent.

2. Closure remembers the reference of the variable, not the current value

This feature is both a source of flexibility and a source of common pitfalls, which we will focus on next.


Classic trap of closures: circular variable reference

If the closure directly references a loop variable, the code written is often inconsistent with intuition.

Example of trapping

Suppose we want to generate 3 functions that return

def create_funcs():
    funcs = []
    for i in range(3):
        def f():
            return i * i
        funcs.append(f)
    return funcs

f0, f1, f2 = create_funcs()
print(f0(), f1(), f2())  # 输出 4 4 4,而不是 0 1 4

Why are they all 4?

Because Python variables are lazy bound:

  • After the loop ends, the external variableihas become2(the value of the last loop)
  • closurefWhat is stored internally isiquote, not a copy of that moment
  • when we actually callf0()only wheniThe current value of , so we get all2²=4

Two classic solutions to traps

The core idea is: **Let the closure "remember the variable value at that time" when it is created, instead of always saving the reference. **

Option 1: Add a layer of auxiliary functions

Each time it loops, use an auxiliary function to receive the currentiThe value of is used as a parameter, and the closure refers to a copy of the parameter of the auxiliary function:

def create_funcs_safe():
    funcs = []
    for i in range(3):
        def make_f(x):          # x 是 i 的副本
            def inner():
                return x * x
            return inner
        funcs.append(make_f(i)) # 立即调用,固定 x
    return funcs

f0, f1, f2 = create_funcs_safe()
print(f0(), f1(), f2())  # 输出 0 1 4 ✔️

The default parameters of Python functions are already bound when they are defined. This feature can be used to write very concisely:

def create_funcs_lambda():
    return [lambda x=i: x * x for i in range(3)]

f0, f1, f2 = create_funcs_lambda()
print(f0(), f1(), f2())  # 输出 0 1 4 ✔️

⚠️ Note: It is recommended to use immutable types for default parameters (such asintstrtuple). If you use a variable type (such aslistdict), a new default parameter sharing trap will be generated.


Modify external variables:nonlocalstatement

The previous closure only reads external variables. If we want to modify external immutable variables (intstretc.), Python will treat assignment as defining local variables by default, thus reporting an error.

Error demonstration

def bad_counter():
    count = 0
    def increment():
        count += 1   # ❌ Python 认为这是局部变量,报错
        return count
    return increment

c = bad_counter()
# c() → UnboundLocalError

Solution: Addnonlocal

usenonlocalClearly tell Python: "This variable is not local or global, it is a variable in the outer function."

def good_counter():
    count = 0
    def increment():
        nonlocal count
        count += 1
        return count
    return increment

c1 = good_counter()
print(c1(), c1(), c1())  # 输出 1 2 3

c2 = good_counter()
print(c2(), c2())        # 输出 1 2(完全独立)

📌 Supplement: If the external variable is a variable type (such aslistdict), you can directly calllist.append()dict['key'] = valueto modify the content withoutnonlocal. But if you want to reassign the entire variable (likelist = []), you still neednonlocal


Modern Python simplified writing: walrus operator:=(Python 3.8+)

For this simple counter scenario, the walrus operator introduced in Python 3.8 can make the code more compact:

def walrus_counter():
    count = 0
    return lambda: (count := count + 1)

c = walrus_counter()
print(c(), c(), c())  # 输出 1 2 3 ✔️

:=Assignment can be done within an expression and the new value returned, making it ideal for writing "one-line closures".


Best practices for closures

  1. Loop variables should be used with caution and direct references When encountering a closure in a loop, give priority to using default parameters to bind the current value, or adding a layer of auxiliary functions.

  2. Modifying external immutable variables must be declarednonlocal
    Mutable types can modify internal elements, but reassigning the entire variable also requiresnonlocal

  3. Avoid holding large object references in closures for a long time Closures will keep the referenced objects in memory, which may lead to memory leaks, especially in long-running scenarios.

  4. Prioritize using closures to implement lightweight "function factories" For example, generating calculation functions for different tax rates and filtering functions for different thresholds is more portable than defining classes.


Summarize

Returning functions is one of the important applications of higher-order functions, and closures are the soul of this usage:

  • It can remember the external environment at the time of creation to achieve flexible delayed calculations
  • It can encapsulate independent state variables and make lightweight "stateful" tools
  • It is also the underlying implementation basis of Python’s decorators (we’ll talk about decorators next time!)

Master closures and you can write more elegant and flexible Python functional code.