Transforming iterables

Higher order functions are functions that act on other functions.

Two important built-in Python higher order functions - map and filter.

map

map applies a function to each element in an iterable, and returns the results as a new iterable.

Here is an example:

import math

k = [1, 25, 81]
e = map(math.sqrt, k)

print(list(e))         #[1.0, 5.0, 9.0]

In this case the map function applies the sqrt function to each of the elements in the list [1, 25, 81]. The result is a sequence containing the values 1.0, 5.0 and 9.0 (the square roots of the numbers in the original list). Note that map creates an iterator - we must convert it to a list before we can print it.

The code above is roughly equivalent to this loop:

import math

k = [1, 25, 81]
e = []
for x in k:
    e.append(math.sqrt(x))

print(e)         #[1.0, 5.0, 9.0]

The main benefit of using map is that it makes the intent clear. Although the loop in the second example is very simple, you still need to read it to understand what it is doing, because a loop could be doing various different things.

It also makes the code shorter and simpler, so it is less likely to contain bugs.

Finally, map uses lazy evaluation, so that it never needs to store all the elements of the output in memory. If k was a very long list, it could be useful to be able to process it one item at a time so save memory.

filter

filter accepts a predicate and an iterable. A predicate is any function that accepts a single parameter and return a boolean result.

filter applies the predicate to each item in the input iterable, and returns a new iterable that contains only those items for which the predicate returns True.

For example:

def gt_0(x):
    return x > 0

k = [1, -2, 3, 2, 0, -1]
f = filter(gt_0, k)

Here our predicate accepts only value that are greater than 0, so f contains the sequence:

1, 3, 2

For more examples see Looping over selected items.

Using lambdas

In the filter example above, we declared a function gt_0 that only gets used is the filter call. We could avoid creating a function by using a lambda function. This allows a simple function to be defined where it is used, without needing to name it (it is an anonymous function). We can define a lambda version of gt_0 like this:

lambda x : x > 0

Or code would now look like:

k = [1, -2, 3, 2, 0, -1]
f = filter(lambda x : x > 0, k)

See also

• Introduction to Functional Programming
• Functions
• Pure functions
• Lambda functions
• Iterators
• Iterators vs iterables
• Built-in functions on iterables
• Map/reduce example
• Generators
• Functional design patterns
• Recursion and the lru_cache in Python
• Partial application
• Closures
• Monads
• Failure monad
• List monad
• Maybe monad