---
title: "Python 101: Context Managers"
date: 2020-01-06
description: "Understanding Python context managers: what they are, why they matter for resource cleanup, and how to implement them."
tags: [python]
---

## Introduction

In this post I wanted to discuss a relatively simple, but important topic: Context Managers. I want to cover the what, the why and the how.

## Summary

- Content Managers abstract away 'clean-up' logic.
- Define a class with `__enter__`/`__exit__` methods.
- The `__enter__` method is similar to a `try` block.
- The `__exit__` method is similar to a `finally` block.
- Reduce boilerplate with [`@contextlib.contextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.contextmanager).

## What is a Context Manager?

Officially...

> [!CITE]
> A context manager is an object that defines the runtime context to be established when executing a with statement. The context manager handles the entry into, and the exit from, the desired runtime context for the execution of the block of code. Context managers are normally invoked using the `with` statement, but can also be used by directly invoking their methods. -- [Python Docs](https://docs.python.org/3/reference/datamodel.html#context-managers)

In simpler terms it means: a Context Manager can ensure code that requires 'clean-up' logic to be executed, is done so in a more idiomatic/Pythonic way.

## Why use a Context Manager?

The classic example given is when opening lots of file in Python:

```
files = []

for _ in range(100000):
    f = open('foo.txt', 'w')
    files.append(f)
    f.close()
```

Notice each file object's `.close()` method is called to ensure the file descriptor is released. If we _didn't_ do that, then your Operating System would exhaust its allowed limit of open file descriptors.

To make this code more Pythonic and cleaner, we can utilize Context Managers.

## How to use a Context Manager?

There are two ways to utilize a Context Manager...

1. `with`
1. `@contextlib.contextmanager`

### `with`

We can use the `with` statement to define a similar block of code to our earlier 'open multiple files' example.

The `with` statement expects a 'Context Manager' to be provided, and there are already a few built-in Python objects designed as Context Managers; such as the `open` function we saw used in our above example code.

> [!EXAMPLE]
> another example is [threading.Lock](https://docs.python.org/3/library/threading.html#threading.Lock).

Here is what the code might look like when using `with`:

```
files = []

for _ in range(100000):
    with open('foo.txt', 'w') as f:
        files.append(f)
```

Notice how we didn't have to explicitly call `.close()` on each file object generated by `open`. That's because `open` works as a Context Manager and knows how to clean-up after itself when called via the `with` statement.

We'll show you how to implement your own Context Manager in the following section: [How to implement a Context Manager?](#how-to-implement-a-context-manager).

### `@contextlib.contextmanager`

Python provides a decorator function `@contextlib.contextmanager` which is actually a callable class (i.e. it defines `__call__` magic method) that enables custom context managers (e.g. your own code you want to act as a context manager) to use simpler code than the traditional 'class-based' implementation we previously mentioned.

This means if you have custom objects that need to implement clean-up logic (similar to how `open` does), then you can decorate your own function so it _behaves_ like a Context Manager, while your function itself simply uses a `yield` statement, like so:

```
from contextlib import contextmanager

files = []

@contextmanager
def open_file(path, mode): 
    file = open(path, mode)
    yield file
    file.close()

for _ in range(100000):
    with open_file('foo.txt', 'w') as f:
        files.append(f)
```

In the above example code we've effectively recreated the `open` Context Manager just to demonstrate the principle.

## How to implement a Context Manager?

Now we've already seen how to implement a Context Manager using the `@contextlib.contextmanager` decorator (see previous sub-section), but how do we implement a class-based version of a Context Manager?

That requires us to define a class which implements `__enter__` and `__exit__` methods. Below is an example, again replicating the `open` function to keep things simple:

```
files = []

class Open():
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode

    def __enter__(self):
        self.open_file = open(self.filename, self.mode)
        return self.open_file

    def __exit__(self, *args):
        self.open_file.close()

for _ in range(100000):
    with Open('foo.txt', 'w') as f:
        files.append(f)
```

> [!INFO]
> for more information, see [Context Manager Types](https://docs.python.org/3/library/stdtypes.html#typecontextmanager).

## When to use one or the other?

One thing I noticed recently was that the `contextmanager` variation wouldn't execute an 'exit' if an exception was raised during execution of the code, while the more verbose 'class-based' implementation _would_. See the following code for an example...

```
from contextlib import contextmanager

@contextmanager
def foo():
    print("enter!")
    yield "foobar"
    print("exit!")

try:
    with foo() as f:
        raise Exception("unexpected")
        print(f"f was: {f}")
except Exception as e:
    print(f"whoops: {e}")
```

The output is not what I expected:

```
enter!
whoops: unexpected
```

Notice how there is no `exit!` printed.

Now compare this to a 'class-based' example...

```
class Foo():
    def __enter__(self):
        print("enter!")

    def __exit__(self, *args):
        print("exit!", args)

try:
    with Foo() as f:
        raise Exception("unexpected")
        print(f"f was: {f}")
except Exception as e:
    print(f"whoops: {e}")
```

The output is as expected:

```
enter!
exit! (<class 'Exception'>, Exception('unexpected'), <traceback object at 0x108882d00>)
whoops: unexpected
```

i.e. we see _both_ an enter and exit message.

We can get the `contextmanager` to behave as we might have expected it to (e.g. the same as the 'class-based' implementation) by ensuring the function that calls `yield` is wrapped in a `try/finally` block, like so:

```
from contextlib import contextmanager

@contextmanager
def foo():
    print("enter!")
    try:
        yield "foobar"
    finally:
        print("exit!")

try:
    with foo() as f:
        raise Exception("unexpected")
        print(f"f was: {f}")
except Exception as e:
    print(f"whoops: {e}")
```

The output of this is now what we might expect...

```
enter!
exit!
whoops: unexpected
```

When choosing between the two options `contextmanager` and 'class-based' implementation, it might be worth keeping this caveat in mind.

## Multiple Context Managers in a single With statement

One interesting aspect of the `with` statement is that you can execute multiple context managers as part of its block control. Meaning when the `with` block completes, then all context managers will be cleaned up.

```
from contextlib import contextmanager

@contextmanager
def foo():
    print("enter!")
    try:
        yield "foobar"
    finally:
        print("exit!")


with foo() as f1, foo() as f2, foo() as f3:
    print(f"f1 was: {f1}")
    print(f"f2 was: {f2}")
    print(f"f3 was: {f3}")
```

Alternatively you can utilize `contextlib.ExitStack`:

```
from contextlib import contextmanager, ExitStack

@contextmanager
def foo():
    print("enter!")
    try:
        yield "foobar"
    finally:
        print("exit!")

with ExitStack() as stack:
    managers = [stack.enter_context(foo()) for cm in range(3)]
    print(managers)  # ['foobar', 'foobar', 'foobar']
```