In place operator overloading

Martin McBride, 2019-01-09
Tags in-place operator
Categories magic methods
In section Python language

Following on from the previous article on operator overloading, we will now take a quick look in place operators.

In place operators

In place operators take the form +=:

x = 3
x += 2
print(x)     # 5

There are equivalent operators for all the numerical operators: -=, *=, etc

In place operators for mutable objects

When we are working with immutable objects such as numbers, strings or tuples, we can treat the following statements as being equivalent:

x = x + 2
x += 2

But when we are using mutable objects, there is a difference. Here is the first case:

a = [10, 20, 30]
b = a
a = a + [40]
print(a, b)       # [10, 20, 30, 40] [10, 20, 30]

We create a list [10, 20, 30] and assign it to a. We then assign the a to b. So a and b reference the same list object.

In the third line, the expression a + [40] creates a new list object with the value [10, 20, 30, 40]. This is assigned to a.

In this case, a references the new list, but b still references the old, unchanged list, so a and b have different values.

Now look at this:

a = [10, 20, 30]
b = a
a += [40]
print(a, b)       # [10, 20, 30, 40] [10, 20, 30, 40]

This start out the same, but the line a += [40] does something different. It appends [40] to the end of the existing list in a.

So now, a still references the original list, whose value has changed in place (ie without creating a new list). b of course also still references the original list, so both variables now contain the same value - the updated list.

In place operators and efficiency

Now suppose a contained a huge list and we wanted to append a value to it.

In the first case above, we would create a brand new copy of the list, with the extra value added on.

In the second case we would simply append the value to the existing list. In place operators can be much more efficient.

In this example, of course, we could also use the list append method to do the same thing, but in other cases where you are susing large data objects, consider using in place operators to avoid memory copying.

In place operations with the Matrix object

Now consider our Matrix class with the __add__ method to implement +, and the __str__ method so print works correctly:

class Matrix:

    def __init__(self, a, b, c, d): = [a, b, c, d]

    def __str__(self):
        return '[{}, {}][{}, {}]'.format([0],

    def __add__(self, other):
        if isinstance(other, Matrix):
            return Matrix([0] +[0],
                [1] +[1],
                [2] +[2],
                [3] +[3])
            return NotImplemented

What happens if we try to use +=?

a = Matrix(10, 20, 30, 40)
b = a
a += Matrix(1, 2, 3, 4)
print(a, b)                 # [11, 22][33, 44] [10, 20][30, 40]

Well, the good thing is we don't get an error! In fact += works as if by magic.

Unfortunately, it isn't quite correct. This is isn't surprising really, because we haven't actually told Python how we expect += to work for a Matrix. So Python has used the existing + operator and used it to simulate the += operator by effectively doing this:

a = a + Matrix(1, 2, 3, 4)

As we saw with the list example, this isn't ideally what we want.

Implementing +=

To properly implement += we need to provide another in place operator to Matrix, called __iadd__ (short for in place add):

    def __iadd__(self, other):
        if isinstance(other, Matrix):
  [0] +=[0]
  [1] +=[1]
  [2] +=[2]
  [3] +=[3]
            return self
            return NotImplemented

This is similar to __add__, but instead of creating a new Matrix, it updates the values of the object itself. Notice that it also returns self (referring to the object itself) at the end.

When we do this, our += works as expected.

There are in place variants of all the numerical operators (__isub__, __imul__ etc).

If you found this article useful, you might be interested in the book Functional Programming in Python, or other books, by the same author.


Tag cloud

2d arrays abstract data type alignment and array arrays bezier curve built-in function close closure colour comparison operator comprehension context conversion data types design pattern device space dictionary duck typing efficiency encryption enumerate filter font font style for loop function function composition function plot functools generator gif gradient greyscale higher order function html image processing imagesurface immutable object index inner function input installing iter iterator itertools lambda function len linspace list list comprehension logical operator lru_cache mandelbrot map monad mutability named parameter numeric python numpy object open operator optional parameter or partial application path positional parameter print pure function radial gradient range recipes recursion reduce rgb rotation scaling sequence slice slicing sound spirograph str stream string subpath symmetric encryption template text text metrics transform translation transparency tuple unpacking user space vectorisation webserver website while loop zip

Copyright (c) Axlesoft Ltd 2020