python closure
back

Your free e-book!

See when it is not worth using Scrum.
"Why Scrum Doesn't Work" Download

Behind the scenes of Python closure function

In this article, we will learn what python closure is, how to use it and what it can be used for. Also, we will explain some extra terms like nonlocal and free variables, what a scope for variables is and what nested and first-class functions are.

First, let’s look at the definition from Wikipedia:

(…) closure, (…) is a technique for implementing lexically scoped name binding in a language with first-class functions. Operationally, a closure is a record storing a function together with an environment. The environment is a mapping associating each free variable of the function (variables that are used locally but defined in an enclosing scope) with the value or reference to which the name was bound when the closure was created. Unlike a plain function, a closure allows the function to access those captured variables through the closure’s copies of their values or references, even when the function is invoked outside their scope.

It might be a bit difficult to understand for now so let’s explain some terms and look at the examples.

Nested functions

Function that is defined inside another function is called nested function. For example:

def outer(variable):
    var = variable

    def nested():
        print(var)

    nested()

outer(1) # 1

Variable scope

In Python, the variable can be accessed from inside the scope that it was created. In the above example, we can call the nested function only inside the outer function. The same with var variable, we can’t call it outside outer function.

First-class functions

In Python, functions are first class functions. It means that functions can be treated as objects. So, for example, it can be passed as an argument of another function, can be assigned or returned.

Closure example compared to a class-based example

Let’s say we want to get the maximum number from the collection to which we can add elements.

Class based example

class Maximum():
    def __init__(self):
        self.collection = []

    def __call__(self, new_value):
        self.collection.append(new_value)
        return max(self.collection)

maximum = Maximum()
print(maximum(1)) # 1
print(maximum(3)) # 3
print(maximum(2)) # 3

Closure example

def make_maximum():
    collection = []

    def maximum(new_value):
        collection.append(new_value)
        return max(collection)

    return maximum

maximum = make_maximum()
print(maximum(1)) # 1
print(maximum(3)) # 3
print(maximum(2)) # 3

As we can see here both examples are very similar. It is obvious where class Maximum keeps collection. It is an attribute of its instance. But where closure keeps collection? While we call maximum, we are long after the execution of make_maximum. Note that collection is a local variable of make_maximum, we initialize it thereby = []. We can even extend this example by removing make_maximum and it will still work.

maximum = make_maximum()
del make_maximum
print(maximum(1)) # 1
print(maximum(3)) # 3
print(maximum(2)) # 3

So how can we access collection? Well, it is a free variable. It means that it is not bound to its local scope. Closure keeps binding to a free variable that exists when the function is defined and it can be used later even when make_maximum no longer exists.

Nonlocal variable

Our example code can be easily refactored. Every time that we add a new variable to collection, we get the max value of it. max function is O(n) time complexity because it always goes through all elements to get it. We can store max value of the previous state and then just compare it with the new variable and return a higher number. It can look like this:

def make_maximum():
    highest = 0

    def maximum(new_value):
        highest = new_value if new_value > highest else highest
        return highest

    return maximum

maximum = make_maximum()
print(maximum(1))
print(maximum(3))
print(maximum(2))

We replace collection with highest and it should work fine. Well, not so quickly, it throws us an error.

UnboundLocalError: local variable 'highest' referenced before assignment

The problem here is that previously we had a mutable type and we just append to it. Now it is an immutable type. So highest = new_value or highest = highest actually makes a new variable with local scope. It is not a free variable anymore.

To fix this we need to use a nonlocal keyword. It makes a variable a free variable and allows us to change the immutable values stored in closure. A correct example should look like this:

def make_maximum():
    highest = 0

    def maximum(new_value):
        nonlocal highest
        highest = new_value if new_value > highest else highest
        return highest

    return maximum

maximum = make_maximum()
print(maximum(1)) # 1
print(maximum(3)) # 3
print(maximum(2)) # 3

Requirements for closure

There are three requirements to make closure:

  • We need to have a nested function
  • We need to have access to a free variable (it is a collection variable in our example). 
  • We need to return a nested function.

When closures can be used

We can find some cases when closures might be a better choice

  • When we want to reduce the use of global variables. It is some kind of data hiding.
  • When we have some simple functions to implement. With more complex tasks it is better to go with classes and OOP.

Let me provide simple example here

elements = list(range(20))

def stepper_of(n):
    def every(l):
        return l[::n]
    return every

every_second = stepper_of(2)
every_third = stepper_of(3)
print(every_second(elements))              # [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
print(every_third(elements))               # [0, 3, 6, 9, 12, 15, 18]
print(every_third(every_second(elements))) # [0, 6, 12, 18]

__closure__ attribute

What if someone asked us to get highest number without calling it? There is a way to get values encolsed by closure. We need to use __closure__ attribute. It returns tuple of objects. For us:

print(maximum.__closure__) # (<cell at 0x7fe3fdb9b5b8: int object at 0x55f6a5f0b040>,)

and to get it we need to add cell_contents to get its value

print(maximum.__closure__[0].cell_contents) # 3


cto - Chris Gibas

Free 30-minute consultation with our CTO

Chris Gibas - our CTO will be happy to discuss your project! Let's talk!

More blog posts
How UX team empowers software development with design systems

Dawid Podsiedlik

How UX team empowers software development with design systems

How can implementing design systems in the work of graphic designers and programmers be helpful? This is the best way to efficiently deliver products to customers in accordance with their needs. In this article you will find some of the most important rules to follow!

Guide on how to deliver 100% sprint

Probably you have read about different tools for project managers that allow you to track, measure, and deliver the project on time. However, using the right tool or project approach is just half of the successful work. In this article, we will share with you what helps Project Managers in Idego Group to delight our clients and deliver 100% sprint […]

Guide on how to deliver 100% sprint

Oleksandra Bilokrys

What are the types of cloud and which will be the best for your company?

Idego

Substantive support - Oleksandra Bilokrys

What are the types of cloud and which will be the best for your company?

Working in a cloud has many advantages. You can easily collaborate on various tasks and access your data and application anytime from anywhere. Those are just a few benefits of leveraging cloud technology for your business. In our article, we explain what types of cloud computing are available on the market and how they differ. If you have never used […]

Machine learning in finance: how does machine learning transform financial operations?

Machine learning is a subset of artificial intelligence widely applied in financial applications and used to improve cost-effectiveness and overall efficiency of financial services. Implementing new technologies can help you gain a competitive advantage and reduce the costs of running the company. Learn how you can apply machine learning in your business tools and applications. Powerful financial analytics, improved cybersecurity […]

Machine learning in finance: how does machine learning transform financial operations?

Idego

Substantive support - Oleksandra Bilokrys

Get a free estimation

Need a successful project?