itertools module

By Martin McBride, 2022-08-16
Tags: itertools python standard library
Categories: python standard library


The itertools module provides a set of functions that create iterators.

Some of these functions work in similar ways to the built-in functions on iterables, in particular the transforming and reducing functions. You may want to read that article if you are not familiar with map, filter, zip and sum. The itertools library generally extends the capabilities of the built-in functions.

This article covers all the functions in the itertools module, with examples. The functions are divided into groups that cover similar functionality:

  • General iterators - mainly variants on the built-in functions
    • Map-like iterators - starmap.
    • Filter-like iterators - filterfalse, dropwhile, takewhile, compress, and isslice.
    • Zip-like iterators - chain, chain.from_iterable, and zip_longest.
    • Splitting iterators - tee and groupby.
    • Accumulating iterators - accumulate.
  • Infinite iterators - count, cycle, and repeat.
  • Combinatoric iterators - product, permutations, combinations and combinations_with_replacement.

All these functions are contained in the itertools module, so you always import this module to use the functions. We haven't shown this in all the examples, to avoid repetition, but you should include the import at the top of every example if you want to run them.

Using iterators

The itertools library, as its name suggests, works with iterables and iterators. See the iterators vs iterables article if you are not familiar with these.

Generally, the inputs to itertools functions are iterable objects. This means that they can accept lists, tuples, and strings, because those objects behave as iterables. They can also accept iterator objects because every iterator also behaves as an iterable. This means that you can chain itertools functions in various ways.

itertools functions always return an iterator. An iterator is a lazy object that only calculates values when it needs to. Here is an example:

import itertools

r = itertools.repeat(5, 3)

print(r)

for i in r:
    print(i)

The repeat function generates a stream of 3 elements, each with value 5. However, it doesn't generate those values immediately. Instead, it creates a lazy iterator.

When we call print(r), we see the following:

repeat(5, 3)

This is because r holds a repeat object rather than a sequence of values. The repeat object is a type of iterator.

We can loop over an iterator using a for loop, in the same way that we might loop over a list. The iterator r will return the value 5, three times, and then the loop will terminate. This means that the loop will print 5, three times, then exit.

Converting an iterator to a list

Another useful thing we can do with an iterator is to convert it to a list:

r = itertools.repeat(5, 3)

k = list(r)
print(k)

The first this to know is that when we previously looped over the iterator r we effectively "used up" the values in it. So the iterator has no values left. In this code, we must first call repeat again, to create a brand new iterator.

When we pass this to the list function a new list is created. The list function gets all the values from the iterator (internally it does something like the for loop above) and uses the values to create a new list. We can then print the list just like any other list.

Once again, this code uses up the iterator r, but of course the values in the list can be accessed as often as you wish. It is just a normal list.

See also

If you found this article useful, you might be interested in the book NumPy Recipes or other books by the same author.

Join the PythonInformer Newsletter

Sign up using this form to receive an email when new content is added:

Popular tags

2d arrays abstract data type alignment and angle animation arc array arrays bar chart bar style behavioural pattern bezier curve built-in function callable object chain circle classes clipping close closure cmyk colour combinations comparison operator comprehension context context manager conversion count creational pattern data science data types decorator design pattern device space dictionary drawing duck typing efficiency ellipse else encryption enumerate fill filter font font style for loop formula function function composition function plot functools game development generativepy tutorial generator geometry gif global variable gradient greyscale higher order function hsl html image image processing imagesurface immutable object in operator index inner function input installing iter iterable iterator itertools join l system lambda function latex len lerp line line plot line style linear gradient linspace list list comprehension logical operator lru_cache magic method mandelbrot mandelbrot set map marker style matplotlib monad mutability named parameter numeric python numpy object open operator optimisation optional parameter or pandas partial application path pattern permutations pie chart pil pillow polygon pong positional parameter print product programming paradigms programming techniques pure function python standard library radial gradient range recipes rectangle recursion reduce regular polygon repeat rgb rotation roundrect scaling scatter plot scipy sector segment sequence setup shape singleton slice slicing sound spirograph sprite square str stream string stroke structural pattern subpath symmetric encryption template tex text text metrics tinkerbell fractal transform translation transparency triangle truthy value tuple turtle unpacking user space vectorisation webserver website while loop zip zip_longest