Function objects and lambdas

One of the first things you learn to do in Python is to call functions, for example, you might have used print and input functions to create a simple "What is your name" type program. You will also have learned how to create your own functions using the def operator.

There is one more important thing to know about functions in Python - a function is an object, just like an integer, a string or a list. This means, for example:

• You can store a function in a variable.
• You can store a collection of functions in a list or dictionary.
• You can pass a function into another function as a parameter.
• A function can return another function as its return value.

Functions are first-class objects in Python - this simply means that function objects are just like any other objects, with no special restrictions on their use. This feature allows Python to support the functional programming paradigm.

This tutorial doesn't cover functional programming in depth, it mainly looks at some practical uses of function objects. See the functional programming articles for more details.

We will also meet lambda functions aka anonymous functions.

All functions are objects

This code defines a simple function and then prints its type

def square(x):
    return x*x

print(type(square))

The result is

<class 'function'>

This tells us that the object square is of type function. Notice that we take the type of square, the variable that holds the object, rather than square() (that would simply call the function).

Since square holds an object, we can assign it to another variable:

sq = square

Now sq and square both hold the same function object.

Applying a function

So now that we have a variable sq that holds a function object, how do we call the function? This is often called applying the function. Well in Python it is very easy. You just use the normal function calling syntax:

x = sq(2)

The name sq is the variable that holds the function pointer. When we use the () operator, it applies the function to the value passed in (ie it calls the function with the value 2).

We sometimes say that sq is an alias of square, but all this means is that the two variables both refer to the same function object.

Passing a function as a parameter

In the next example, we will create a function that takes a list of names and prints them out. The names are stored as tuples containing the first name and last name. The function print_names prints the names out, one per line.

However, we would like to be able to control how the names are formatted. So we make our print_names function accept a formatter function object. The formatter function accepts a name tuple and returns a string.

As an example, the format_space function joins the first and last names, with a space between them:

names = [ ('John', 'Smith'),
          ('Anne', 'Jones'),
          ('Robert', 'Davis')]

def print_names(names, formatter):
    for name in names:
        print(formatter(name))

def format_space(name):
    return ' '.join(name)

print_names(names, format_space)

As you might expect, this creates the following output:

John Smith
Anne Jones
Robert Davis

Now suppose we wanted to print a list of the names in a different format, say surname plus initial. We can do this without changing the print_names function, simply by writing a new formatter function and passing that into the call:

def format_initial(name):
    return f"{name[1]}, {name[0][0]}"

print_names(names, format_initial)

This prints the second part of the name, plus a comma, plus the first character of the first name, giving this output:

Smith, J
Jones, A
Davis, R

Finally, what if we wanted to print out the email addresses - formed from the lowercase names, separated by a dot, followed by the domain name? Like this:

def format_email(name):
        return f"{name[0]}.{name[1]}@example.com"

print_names(names, format_email)

This gives:

john.smith@example.com
anne.jones@example.com
robert.davis@example.com

As you can see, using a function object allows us to change the behaviour of the print_names function without changing its code. It is a very powerful technique.

Higher order functions

A higher-order function is a function that operates on functions - it either accepts function objects as parameters, returns a function object, or both.

The map() function

One example of a higher-order function is the built-in map function. This takes a function object and applies it to a sequence of inputs. Here is an example using a square function:

def square(x):
    return x*x

numbers = [1, 3, 5, 7]

result = map(square, numbers)
print(list(result))

The map function applies the square function to each element in numbers, and returns the result. Note that the result is a map object, which is a lazy operator. You can iterate over a map object, for example in a for loop. But if you want to print the result out as a list of values, you must first convert the lazy sequence into an actual list (using the list function). The result is a list of the squares of each input value:

[1, 9, 25, 49]

Functions that return functions

Python functions have another powerful feature. A higher-order function can transform one function into another, related function. For example:

def twice(f):
    def g(x):
        return f(f(x))
    return g

The function twice accepts a parameter f, which must take a single parameter.

Inside the function twice, we define a nested function g that calculates f of f of x - that is, it applies f twice. The function g is called an inner function.

But here is the clever bit. twice returns the function g, which is a brand new function that apples f twice, whatever f might be!

So if we have an increment function that adds 1 to x, we can create an inc2 function that adds 1 to x and then adds 1 to the result (that is, it adds 2 to x). This code prints 7:

def increment(x):
    return x + 1

inc2 = twice(increment)
print(inc2(5))

Similarly, if we have a square function, we can create an sq2 function that squares x and then squares the result (that is, it raises x to the power 4). This prints 81:

def square(x):
    return x*x

sq2 = twice(square)
print(sq2(3))

Notice that the inc2 function remembers that the function passed into twice was the increment function. And also the sq2 function remembers that the function passed into twice was the square function. A function that retains the values of its local variables and make sthem available to an inner function after the main function has returned is called a closure.

Lambda functions

In the map example above, we have defined a function square simply so that we can use it in the call to the map function:

def square(x):
    return x*x

numbers = [1, 3, 5, 7]

result = map(square, numbers)
print(list(result))

Since we only use the square function in one place, it seems a bit pointless to have to declare it and give it a name. It would be a lot simpler just to be able to specify the function where we use it, in the map call. That is what lambda functions are for.

Despite sounding quite exotic, a lambda function is simply a neat bit of syntax you can use to create small simple functions in line, without the hassle of creating a full function definition, and without needing to bother giving it a name (it is an anonymous function).

Here is how it is used:

numbers = [1, 3, 5, 7]

result = map(lambda x: x*x, numbers)
print(list(result))

This is the bit of code that defines the function:

lambda x: x*x

It creates a function object that takes a parameter x and returns the value x squared. The function object isn't given a name because it is used in place, passed directly into the map call.

In the case of the twice function, lambdas can really simplify things. Here is the original code:

def twice(f):
    def g(x):
        return f(f(x))
    return g

We can replace the inner function g with a lambda (the name g isn't used outside the function anyway), giving us this:

def twice(f):
    return lambda x: f(f(x))

Not only is this shorter, but it is also a clearer expression of what is going on. twice returns a function that has the effect of applying f twice.

See also

• List comprehensions
• Objects and variables
• Objects and identity
• Immutable objects
• Global variables
• Data types
• Lists vs tuples
• Sequences
• Named tuples
• Operators
• Short circuit evaluation
• Walrus Operator
• For loops
• For loop using range vs iterables
• Changing the loop order
• Using enumerate in a for loop
• Using zip in a for loop
• Looping over multiple items (old article)
• Looping over selected items
• Functions
• Declaring functions
• Calling functions
• Function decorators
• With statements
• Exception handling
• String functions
• Built-in functions
• Optimisation
• Optimisation good practice
• Low level code optimisation
• Structural optimisation