Broadcasting in in numpy

When we combine two arrays using vectorisation or a ufunc, it will often be the case that both arrays are identical shape. For example, they might both be 2-dimensional array of shape 4 by 5.

However, you might sometimes want to combine arrays of different shapes. For example, you might want to take a vector (1-dimensional array) and add it to each row of a matrix (2-dimensional array). NumPy allows you to do this, and other things, using broadcasting.

We will look at some common cases, and then the general rule.

Broadcasting from 1 to 2 dimensions

We will look at the example suggested above, with the following two arrays:

m = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9],
              [10, 11, 12]])
v = np.array([10, 20, 30])

r = m + v

m has shape (4, 3), v has shape (3,)

You might think that m + v would fail, but in fact NumPy automatically broadcasts v across 4 rows, to make an effective array that is the same shape as m. The broadcast v becomes:

[[10, 20, 30],
 [10, 20, 30],
 [10, 20, 30],
 [10, 20, 30]]

NumPy doesn't actually create this array, it simulates it by looping multiple times over the array v, as we will see later.

Where is the result of m + v:

r = [[11 22 33]
     [14 25 36]
     [17 28 39]
     [20 31 42]]

Broadcasting 1 to 3 dimensions

Now consider an array of shape (3, 2, 3):

m = np.array([[[1, 2, 3],
               [4, 5, 6]],

              [[7, 8, 9],
               [10, 11, 12]],

              [[13, 14, 15],
               [16, 17, 18]]])
v = np.array([10, 20, 30])

r = m + v

This time the value v is broadcast over every row of every sheet, giving an effective array of:

[[[10 20 30]
  [10 20 30]]

 [[10 20 30]
  [10 20 30]]

 [[10 20 30]
  [10 20 30]]]

The result is:

r = [[[11 22 33]
      [14 25 36]]

     [[17 28 39]
      [20 31 42]]

     [[23 34 45]
      [26 37 48]]]

Broadcasting 2 to 3 dimensions

Now consider an array of shape (2, 3) being broadcast to the previous array (3, 2, 3):

m = np.array([[[1, 2, 3],
               [4, 5, 6]],

              [[7, 8, 9],
               [10, 11, 12]],

              [[13, 14, 15],
               [16, 17, 18]]])
v = np.array([[10, 20, 30],
              [40, 50, 60]])

r = m + v

In this case, the 2 by 3 matrix is broadcast to each of the 3 sheets, like this:

[[[10 20 30]
  [40 50 60]]

 [[10 20 30]
  [40 50 60]]

 [[10 20 30]
  [40 50 60]]]

Giving the result:

r = [[[11 22 33]
      [44 55 66]]

     [[17 28 39]
      [50 61 72]]

     [[23 34 45]
      [56 67 78]]]

Broadcasting rules

We have seen several examples of broadcasting, now we will formalise the general rules. If two arrays have different shapes, NumPy will attempt to make them compatible by following two steps.

Step 1 if the arrays have different rank (ie number of dimensions), they are equalised by adding dimensions of length 1 to the start of the shape tuple.

Step 2 for each dimension, if the two values are equal then the shapes already match in that dimension so nothing needs to be done. If the two values are not equal but one of the values is 1, then the array with the dimension of 1 will be replicated along that axis to match the other array.

If, for any dimension, the two arrays do not match but neither has the value 1, then the arrays are not compatible and broadcasting will not occur.

Here are some examples.

An 2-dimensional array of shape (4, 3) and a 1-dimensional array of shape (3,), see the example above:

• Ones are added to the start of the shape of the second array to make its shape (1, 3).
• The array is replicated 4 times along axis 0 to make its shape (4, 3).

An 3-dimensional array of shape (3, 2, 3) and a 1-dimensional array of shape (3,), see the example above:

• Ones are added to the start of the shape of the second array to make its shape (1, 1, 3).
• The array is replicated 3 times along axis 0, and 2 times along axis to make its shape (3, 2, 3).

Finally a failure example, a 2-dimensional array of shape (2, 6) and a 1-dimensional array of shape (3,), see the example above:

• Ones are added to the start of the shape of the second array to make its shape (1, 3).
• The second axis doesn't match - it is 6 for the first array, but 3 for the second array. NumPy thows a ValueError.

Broadcasting a column vector

We can also broadcast a column vector across m, like this:

m = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9],
              [10, 11, 12]])
c = np.array([[10],
              [20],
              [30],
              [40]])

r = m + c

This time, c has a shape (4, 1) - it has 4 rows and 1 column. m has a shape (4, 3). The two arrays are already both 2-dimensional, but c must be broadcast along axis 1 to make them match. This gives:

[[10 10 10]
 [20 20 20]
 [30 30 30]
 [40 40 40]]

This is compatible with m, so they can be added together.

Broadcasting a row vector and a column vector

We can also broadcast a row vector and a column vector across m, like this:

v = np.array([1, 2, 3])
c = np.array([[10],
              [20],
              [30]])
r = v + c

This is quite an interesting case:

• c has a shape (3, 1).
• v has a shape (3,), which will be extended to (1, 3) to match the dimensions of c.
• To match these two shapes, c is broadcast along axis 1, and v is broadcast along axis 0.
• The result is a 3 by 3 matrix!

Here is the result:

r = [[11 12 13]
     [21 22 23]
     [31 32 33]]

Broadcasting scalars

We have already seen code like this:

a = np.array([1, 2, 3, 4])
b = a + 2

But how does this actually work?

One way to think about it is the scaler value 2 is a bit like a 1-dimensional array of length 1. This array can be broadcast to any shape, simply by replicating it in every required dimension.

See also

• Universal functions in in numpy
• Reducing and accumulating numpy arrays
• Advanced vectorisation in numpy