In place operator overloading


Martin McBride, 2019-01-09
Tags in-place operator
Categories magic methods

Following on from the previous article on operator overloading, we will now take a quick look in place operators.

In place operators

In place operators take the form +=:

x = 3
x += 2
print(x)     # 5

There are equivalent operators for all the numerical operators: -=, *=, etc

In place operators for mutable objects

When we are working with immutable objects such as numbers, strings or tuples, we can treat the following statements as being equivalent:

x = x + 2
x += 2

But when we are using mutable objects, there is a difference. Here is the first case:

a = [10, 20, 30]
b = a
a = a + [40]
print(a, b)       # [10, 20, 30, 40] [10, 20, 30]

We create a list [10, 20, 30] and assign it to a. We then assign the a to b. So a and b reference the same list object.

In the third line, the expression a + [40] creates a new list object with the value [10, 20, 30, 40]. This is assigned to a.

In this case, a references the new list, but b still references the old, unchanged list, so a and b have different values.

Now look at this:

a = [10, 20, 30]
b = a
a += [40]
print(a, b)       # [10, 20, 30, 40] [10, 20, 30, 40]

This start out the same, but the line a += [40] does something different. It appends [40] to the end of the existing list in a.

So now, a still references the original list, whose value has changed in place (ie without creating a new list). b of course also still references the original list, so both variables now contain the same value - the updated list.

In place operators and efficiency

Now suppose a contained a huge list and we wanted to append a value to it.

In the first case above, we would create a brand new copy of the list, with the extra value added on.

In the second case we would simply append the value to the existing list. In place operators can be much more efficient.

In this example, of course, we could also use the list append method to do the same thing, but in other cases where you are susing large data objects, consider using in place operators to avoid memory copying.

In place operations with the Matrix object

Now consider our Matrix class with the __add__ method to implement +, and the __str__ method so print works correctly:

class Matrix:

    def __init__(self, a, b, c, d):
        self.data = [a, b, c, d]

    def __str__(self):
        return '[{}, {}][{}, {}]'.format(self.data[0],
                                         self.data[1],
                                         self.data[2],
                                         self.data[3])

    def __add__(self, other):
        if isinstance(other, Matrix):
            return Matrix(self.data[0] + other.data[0],
                          self.data[1] + other.data[1],
                          self.data[2] + other.data[2],
                          self.data[3] + other.data[3])
        else:
            return NotImplemented

What happens if we try to use +=?

a = Matrix(10, 20, 30, 40)
b = a
a += Matrix(1, 2, 3, 4)
print(a, b)                 # [11, 22][33, 44] [10, 20][30, 40]

Well, the good thing is we don't get an error! In fact += works as if by magic.

Unfortunately, it isn't quite correct. This is isn't surprising really, because we haven't actually told Python how we expect += to work for a Matrix. So Python has used the existing + operator and used it to simulate the += operator by effectively doing this:

a = a + Matrix(1, 2, 3, 4)

As we saw with the list example, this isn't ideally what we want.

Implementing +=

To properly implement += we need to provide another in place operator to Matrix, called __iadd__ (short for in place add):

    def __iadd__(self, other):
        if isinstance(other, Matrix):
            self.data[0] += other.data[0]
            self.data[1] += other.data[1]
            self.data[2] += other.data[2]
            self.data[3] += other.data[3]
            return self
        else:
            return NotImplemented

This is similar to __add__, but instead of creating a new Matrix, it updates the values of the object itself. Notice that it also returns self (referring to the object itself) at the end.

When we do this, our += works as expected.

There are in place variants of all the numerical operators (__isub__, __imul__ etc).

If you found this article useful, you might be interested in the book Functional Programming in Python or other books by the same author.

Prev

Popular tags

2d arrays abstract data type alignment and animation arc array arrays behavioural pattern bezier curve built-in function callable object chain circle classes close closure cmyk colour combinations comparison operator comprehension context context manager conversion count creational pattern data types design pattern device space dictionary drawing duck typing efficiency ellipse else encryption enumerate fill filter font font style for loop function function composition function plot functools game development generativepy tutorial generator geometry gif gradient greyscale higher order function hsl html image image processing imagesurface immutable object index inner function input installing iter iterable iterator itertools l system lambda function len line linear gradient linspace list list comprehension logical operator lru_cache magic method mandelbrot mandelbrot set map monad mutability named parameter numeric python numpy object open operator optional parameter or partial application path pattern permutations polygon positional parameter print pure function python standard library radial gradient range recipes rectangle recursion reduce repeat rgb rotation scaling sector segment sequence setup shape singleton slice slicing sound spirograph sprite square str stream string stroke structural pattern subpath symmetric encryption template text text metrics tinkerbell fractal transform translation transparency triangle tuple turtle unpacking user space vectorisation webserver website while loop zip