Broadcasting in in numpy


Martin McBride, 2021-02-27
Tags arrays broadcasting vectorisation
Categories numpy

When we combine two arrays using vectorisation or a ufunc, it will often be the case that both arrays are identical shape. For example, they might both be 2-dimensional array of shape 4 by 5.

However, you might sometimes want to combine arrays of different shapes. For example, you might want to take a vector (1-dimensional array) and add it to each row of a matrix (2-dimensional array). NumPy allows you to do this, and other things, using broadcasting.

We will look at some common cases, and then the general rule.

Broadcasting from 1 to 2 dimensions

We will look at the example suggested above, with the following two arrays:

m = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9],
              [10, 11, 12]])
v = np.array([10, 20, 30])

r = m + v

m has shape (4, 3), v has shape (3,)

You might think that m + v would fail, but in fact NumPy automatically broadcasts v across 4 rows, to make an effective array that is the same shape as m. The broadcast v becomes:

[[10, 20, 30],
 [10, 20, 30],
 [10, 20, 30],
 [10, 20, 30]]

NumPy doesn't actually create this array, it simulates it by looping multiple times over the array v, as we will see later.

Where is the result of m + v:

r = [[11 22 33]
     [14 25 36]
     [17 28 39]
     [20 31 42]]

Broadcasting 1 to 3 dimensions

Now consider an array of shape (3, 2, 3):

m = np.array([[[1, 2, 3],
               [4, 5, 6]],

              [[7, 8, 9],
               [10, 11, 12]],

              [[13, 14, 15],
               [16, 17, 18]]])
v = np.array([10, 20, 30])

r = m + v

This time the value v is broadcast over every row of every sheet, giving an effective array of:

[[[10 20 30]
  [10 20 30]]

 [[10 20 30]
  [10 20 30]]

 [[10 20 30]
  [10 20 30]]]

The result is:

r = [[[11 22 33]
      [14 25 36]]

     [[17 28 39]
      [20 31 42]]

     [[23 34 45]
      [26 37 48]]]

Broadcasting 2 to 3 dimensions

Now consider an array of shape (2, 3) being broadcast to the previous array (3, 2, 3):

m = np.array([[[1, 2, 3],
               [4, 5, 6]],

              [[7, 8, 9],
               [10, 11, 12]],

              [[13, 14, 15],
               [16, 17, 18]]])
v = np.array([[10, 20, 30],
              [40, 50, 60]])

r = m + v

In this case, the 2 by 3 matrix is broadcast to each of the 3 sheets, like this:

[[[10 20 30]
  [40 50 60]]

 [[10 20 30]
  [40 50 60]]

 [[10 20 30]
  [40 50 60]]]

Giving the result:

r = [[[11 22 33]
      [44 55 66]]

     [[17 28 39]
      [50 61 72]]

     [[23 34 45]
      [56 67 78]]]

Broadcasting rules

We have seen several examples of broadcasting, now we will formalise the general rules. If two arrays have different shapes, NumPy will attempt to make them compatible by following two steps.

Step 1 if the arrays have different rank (ie number of dimensions), they are equalised by adding dimensions of length 1 to the start of the shape tuple.

Step 2 for each dimension, if the two values are equal then the shapes already match in that dimension so nothing needs to be done. If the two values are not equal but one of the values is 1, then the array with the dimension of 1 will be replicated along that axis to match the other array.

If, for any dimension, the two arrays do not match but neither has the value 1, then the arrays are not compatible and broadcasting will not occur.

Here are some examples.

An 2-dimensional array of shape (4, 3) and a 1-dimensional array of shape (3,), see the example above:

  • Ones are added to the start of the shape of the second array to make its shape (1, 3).
  • The array is replicated 4 times along axis 0 to make its shape (4, 3).

An 3-dimensional array of shape (3, 2, 3) and a 1-dimensional array of shape (3,), see the example above:

  • Ones are added to the start of the shape of the second array to make its shape (1, 1, 3).
  • The array is replicated 3 times along axis 0, and 2 times along axis to make its shape (3, 2, 3).

Finally a failure example, a 2-dimensional array of shape (2, 6) and a 1-dimensional array of shape (3,), see the example above:

  • Ones are added to the start of the shape of the second array to make its shape (1, 3).
  • The second axis doesn't match - it is 6 for the first array, but 3 for the second array. NumPy thows a ValueError.

Broadcasting a column vector

We can also broadcast a column vector across m, like this:

m = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9],
              [10, 11, 12]])
c = np.array([[10],
              [20],
              [30],
              [40]])

r = m + c

This time, c has a shape (4, 1) - it has 4 rows and 1 column. m has a shape (4, 3). The two arrays are already both 2-dimensional, but c must be broadcast along axis 1 to make them match. This gives:

[[10 10 10]
 [20 20 20]
 [30 30 30]
 [40 40 40]]

This is compatible with m, so they can be added together.

Broadcasting a row vector and a column vector

We can also broadcast a row vector and a column vector across m, like this:

v = np.array([1, 2, 3])
c = np.array([[10],
              [20],
              [30]])
r = v + c

This is quite an interesting case:

  • c has a shape (3, 1).
  • v has a shape (3,), which will be extended to (1, 3) to match the dimensions of c.
  • To match these two shapes, c is broadcast along axis 1, and v is broadcast along axis 0.
  • The result is a 3 by 3 matrix!

Here is the result:

r = [[11 12 13]
     [21 22 23]
     [31 32 33]]

Broadcasting scalars

We have already seen code like this:

a = np.array([1, 2, 3, 4])
b = a + 2

But how does this actually work?

One way to think about it is the scaler value 2 is a bit like a 1-dimensional array of length 1. This array can be broadcast to any shape, simply by replicating it in every required dimension.

Visit the PythonInformer Discussion Forum for numeric Python.

If you found this article useful, you might be interested in the book NumPy Recipes or other books by the same author.

Prev

Popular tags

2d arrays abstract data type alignment and animation arc array arrays bezier curve built-in function callable object circle classes close closure cmyk colour comparison operator comprehension context context manager conversion creational pattern data types design pattern device space dictionary drawing duck typing efficiency else encryption enumerate fill filter font font style for loop function function composition function plot functools game development generativepy tutorial generator geometry gif gradient greyscale higher order function hsl html image image processing imagesurface immutable object index inner function input installing iter iterable iterator itertools l system lambda function len line linspace list list comprehension logical operator lru_cache magic method mandelbrot mandelbrot set map monad mutability named parameter numeric python numpy object open operator optional parameter or partial application path polygon positional parameter print pure function pycairo radial gradient range recipes rectangle recursion reduce rgb rotation scaling sector segment sequence singleton slice slicing sound spirograph sprite square str stream string stroke subpath symmetric encryption template text text metrics tinkerbell fractal transform translation transparency tuple turtle unpacking user space vectorisation webserver website while loop zip