We need to talk about Protocols in Python

Introduction

In recent years, Python has grown significantly as a programming language. In order to meet expectations of modern web developers and data engineers, it has introduced necessary solutions and advanced implementations to become even more flexible than ever before. But little did we know that Python still keeps many of its precious gems out of sight. In this article I would like to promote one of them, a healthy alternative to Abstract Base Classes (ABC) and inheritance – Protocols, which is not as new a concept as one would imagine.

Abstract Base Class (ABC) example

Python implements class inheritance in its own way. The dedicated module abc allows creating abstract classes and declaring abstract methods to organize structure for our data models within the project. To show its capabilities, in the following example, we have created a simple service to organize a “Home Zoo”. The structure of the files is the following:

In animal.py we keep our simple Animal parent class with two abstract methods. We inherit from it in Dog and Cat classes kept in the animals.py file.

Implemented child classes are the following:

Both child classes inherit directly from the Animal parent class, implementing both of its methods. Next, we have a feeding.py file where a simple function feeding_time() takes an Animal instance as a parameter and runs the feed() method declared in a class.

Finally, inside service.py we have created our service class for a home zoo:

HomeZooService keeps the register of Animal abstract class instances as well as allows adding new animals. Any class that inherits from the parent Animal class and implements all of its methods might be used and stored within our service. To present how it works, let us take a look at the main.py file.

Objects provided to HomeZooService are the child classes of the Animal parent class. The result of running main() function is following:

As one could imagine, everything works properly since overridden methods in child classes were executed. The solution proposed in this example seems elegant, yet simple and straightforward. Creation of parent and child classes structure looks perfectly fine. 

The advantage of defining abstract classes is that we create a blueprint of a common interface, in this case the interface is a set of methods. Therefore, all child classes must implement mentioned functions. This allows us to take advantage of the polymorphism mechanism and program other parts of code to an interface not an implementation. Thanks to that we are able to store and use any child class (e.g. Dog), while defining our input as an instance of the parent class (in our case: Animal).

There are a few problems, though, that someone curious may point out. Firstly, removing any of the overridden methods in child classes will instantaneously cause trouble. Besides IDE warnings, we will not be able to instantiate any new object:

This creates some issues as our child classes are much dependent on the parent class. On one hand, this is just what we wanted in the first place – straightforward inheritance, on the other hand, we strengthened coupling in our code, making classes, objects and methods more dependent on each other. Python interpreter has created a relation between parent and child classes based on inheritance stated in the definition of Dog and it sees that it does not implement all necessary methods at the moment of object instantiating, thus executing our code will throw a following error:

Furthermore, Animal class import is required in most of the files, including children classes. It is not a terrible mistake, but it strengthens the dependency between different parts in the code. 

Last and probably most critical issue is the fact that each part of the code requires a specified interface as an input. For instance, the function feeding_time() only uses the feed() method of the Animal class but right now it has to accept the whole object as a parameter. Our example is fairly simple, but imagine that the parent class implements more than 10 methods – most of the functionalities provided in the input interface (the Animal object) would basically become redundant in the scope of this function. We could potentially separate Animal into smaller classes but it would unnecessarily complicate our code design. Also, multiple inheritance is rarely a good idea, especially in such simple applications. 

Luckily, there is a way to solve these issues as well as organize and simplify our code, reduce coupling and loose dependencies in existing files. The solution is called “Protocols”.

Protocol introduction

Protocols are not a new concept, even in Python. Introduced in version 3.8 as an alternative to Abstract Base Classes, they rely on structural typing. In this mechanism, Python interpreter checks the structure of the object (its methods, attributes etc.) to determine whether it can be used in a specific function/class and the types match. This strategy, based on comparing objects and checking their types, has a specific name in dynamically typed languages (such as Python): duck typing. It follows a very simple rule:

| If it walks like a duck and it quacks like a duck, then it must be a duck.

It means that if two objects have the same methods and attributes, Python will treat them as the same type. For comparison, ABCs use nominal typing, where object relationship is defined by inheritance stated deliberately in our class definition (e.g. class B(A)). 

Protocols are, therefore, interfaces which help define what is expected as an input of our methods. Hopefully, by using them we might be able to avoid problems mentioned before and improve our code design.

So, let us make a few changes to the Animal parent class. This time we will be using Python’s inbuilt Protocol class.

The code has not changed all that much. We remove the abc package link and still keep our methods, but this time we write ... instead of the implementation which is a standardized Protocols method notation. Now, let us apply changes to the Dog and Cat classes.

We removed all links to the parent class here – we do not need them anymore. Thanks to the use of Protocol in Animal, we reduced import dependency in new classes. It also means that any changes to them will not instantaneously raise warnings or errors. This is due to the fact that neither Dog nor Cat is directly linked to the Animal class anymore, loosening the coupling as well as making them much more independent and flexible. We will show how this would affect our application in a moment.

For the other files, we do not need to change any of the code we had before, external function and service class will still require Animal instance as an input. The execution method and its result will also not change at all.

The real difference emerges when we remove one of the methods in Dog or Cat classes, same as we did before, and run the code. The result is presented below:

This time our code did not crash on instantiating an object but on executing a not implemented method of the Animal Protocol. It shows a very important conclusion for the use of Protocols – their main focus is. Thanks to the duck typing mechanism, we are able to instantiate the object without an error, while being focused on using an instance.

This example has shown that Protocols provide us with a defined interface to other parts of the program that may require it in some way. The solution is not only simple and elegant but also beneficial as we were able to easily reduce coupling between different parts of code. But the real advantage of using them will be discovered in a few extra steps.

Further improvement

There is room for further improvement in our code. One of the problems that was raised before was not adjusting the input interface for a specific function. This issue might be solved by providing dedicated Protocols as interfaces.

Let us show an example of the feeding_time() function. The truth is that all this function needs as an input is an object that implements the feed() method, the rest is irrelevant. Here lies the true beauty of using Protocols, as we may prepare a specified interface just for this function. Let us create new Protocol class:

Now, let us use it in feeding_time() function:

Although the change is not spectacular, it gives a very interesting result: the application execution is the same. It also successfully limited the interface use and definition in the application. What we have just implemented suits very nicely into the “Interface Segregation” rule, known in SOLID principles, stating that no part of the application should depend on methods it does not require. All in all, Protocol usage will result in elegant, well organized code.

Obviously, there are situations where Protocol implementation might not be our first choice, especially when the operation of our service focuses on instance creation. The flexibility that Protocols provide is based on Python’s dynamic typing, meaning that types are checked in the runtime. Therefore, we lose straightforward information from the inheritance that our subclass does not implement all necessary methods. This might be crucial when instantiating the objects plays a very important role in our application.

Also, no inheritance means no class hierarchy. When we see a subclass inheriting from its parent, it is much easier to determine the structure and purpose of its use, making it a good source of design documentation. With Protocols, we do not have such easy and clear options since any subclass is basically a stand-alone entity, identified as an appropriate input only at a runtime (although modern IDEs provide tools to statically check the types before code execution).

The “Home Zoo” example obviously could be extended, providing new functions or services that will all require a specific input. Right now it seems intuitive that each one of them might have its own dedicated Protocol, the only requirement we would need are the classes that implement methods defined within the interface.

Summary

To summarize, Protocols seem very similar to Abstract Base Classes, they have different uses in terms of application design, though. They handle dependency problems differently – while ABCs, depending on class hierarchy, focus on instance creation, Protocols are dedicated for instance usage thanks to the duck typing mechanism. The capability to split interfaces in order to fit a specific part of the application cannot be overestimated and even though it is achievable using multiple inheritance, with Protocols it seems much more natural and elegant. Last but not least, most modern Python IDEs support use of Protocols by checking the implementation of the required functions, allowing easy and organized application designing.

Pros:

• Protocols are an intuitive alternative to ABCs.
• They reduce coupling and dependencies in code.
• They create an adjustable interface for each functionality.

Cons:

• No class hierarchy requires more careful usage and less explicit code documentation.
• When the main goal is instance creation – ABCs are still a better and safer choice.