Unpacking what’s in Python 3.5

Python 3.5 – the latest major release of my favorite programming language comes with a few handy additions – one of which is a small new convenience feature I wanted to write about today. Among definitely more complex enhancements like async/await, type hinting support, Windows C-runtime and installer changes, it might be easily overlooked. Still, I think python is often perceived as a very succinct and readable language – and the inclusion of those unpacking generalizations make it even more so, and deserve a short article.

What’s new

The new feature – described in detail in PEP 0448 is really simple. You’re probably familiar with functions taking variable number of either positional of keyword arguments:

def positional(*args):  
    print(args)
def keywords(**kwargs):  
    # tip for the uninitiated - kwargs stands for keyword arguments.
    print(kwargs)
def any_arg_combination(*args, **kwargs):  
    print(args)
    print(kwargs)

It’s all cool if you call those functions while providing arguments explicitly:

positional(1, 2, 3)  
keywords(a=1, b=2, c=3)  
any_arg_combination(1, 2, 3, a=1, b=2, c=3)  

will output (notice that positional arguments are tuples, which are immutable, whereas keywords are mutable dicts):

(1, 2, 3)
{'b': 2, 'a': 1, 'c': 3}
(1, 2, 3)
{'b': 2, 'a': 1, 'c': 3}

But suppose that we are not operating on explicit arguments, but we’ve already received a tuple or dictionary, and want to pass it forward with some modification – this is not an exceptionally rare problem – it can occur for example when writing function decorators. In case of positional arguments we can, just stick it in front:

args = (1, 2, 3)  
positional(0, *args)  

If the order is important though, we’re out of luck, prior to python 3.5, trying to call:

positions(*args, 4)  
>>> SyntaxError: only named arguments may follow *expression

To make it work as desired, we would have to turn the received args tuple into a mutable object (possibly list), append to it, and then pass it forward, or use itertools.chain – both solutions adding boilerplate code. In python 3.5 though, this will just work as expected:

positions(*args, 4)  
(1, 2, 3, 4)

It is also possible to have arbitrary number of unpacking operations in a function call:

positional(*args, *args)  
(1, 2, 3, 1, 2, 3)
positional(*args, 4, 5, *args, 6)  
(1, 2, 3, 4, 5, 1, 2, 3, 6)

This also works in assignments:

al1 = [1, 2, 3]  
al2 = [4, 5]  
a = *al1, *al2  
a  
(1, 2, 3, 4, 5)

In 3.4 we’ll get:

SyntaxError: can use starred expression only as assignment target  

The dictionary unpacking changes are in similar fashion. Previously, if you wanted to combine two dictionaries, you had to either call the update method, or use collections.ChainMap. Now it just works, and corner cases seem to follow the principle of least surprise:

ad1 = {1:11, 2:22, 3:33}  
keywords(**ad1)  
TypeError: keywords() keywords must be strings

ad1 = {'1':11, '2':22, '3':33}  
keywords(**ad1)  
{'1': 11, '3': 33, '2': 22}

ad2 = {'1':11, '4':22, '5':33}  
keywords(**ad1, **ad2)  
TypeError: keywords() got multiple values for keyword argument '1'

ad2 = {'4':22, '5':33}  
keywords(**ad1, **ad2)  
{'1': 11, '5': 33, '2': 22, '3': 33, '4': 22}

One limitation that is still valid, is that you cannot mix the order of iterable and keyword arguments:

any_arg_combination(*al1, **ad1, *al2, **ad2)  
SyntaxError: iterable argument unpacking follows keyword argument unpacking  

This does not seem like a needed feature though.

Conclusion

There you have it. The addition is really small, but I find it convenient, and I am under the impression that the small features like this combined together are actually a crucial part of the overall Python programming experience. Not enough attention is being given to them (e.g: I only learned about Python 3 keyword only arguments syntax a week ago, while it was around since python 3.0) – hence, this article. Hope you enjoyed the read.