F2.1 · LESSON 1 OF 6

Comprehensions and transformations

Organize a small program into modules and classes while handling files, failures and iteration.

THE IDEA

Building a list from another one

A very common job is: take a collection, keep some of its items, change them, and collect the results. With a loop you write four lines every time: make an empty list, loop, test, append. A comprehension says the same thing in one expression. It reads almost like the sentence "the square of x, for every x in nums, if x is greater than 0".

Same job, two shapes

nums = [-1, 2, 3]

result = []                     # loop version
for x in nums:
    if x > 0:
        result.append(x * x)

result = [x * x for x in nums if x > 0]     # comprehension: [4, 9]

The parts always come in this order: [ expression for item in source if condition ]. Python walks the source one item at a time. The if decides whether the item takes part, and the expression decides what goes into the new collection. The original is never changed: you get a new list.

Watch items go through the pipeline

[x * x for x in nums if x > 0]

keep items greater than0

For each item in nums

-1

-4

Result so far

[]

0 of 5 items processed.

The same job as a loop

result = []for x in nums: if x > 0: result.append(x * x)

The if at the end is a filter: items that fail it are dropped and produce nothing.

Press Process next item. Green items pass the filter and produce a value, dashed ones are dropped. Move the slider to change the filter and compare with the loop on the right.

Curly braces work the same way: {w: len(w) for w in words} builds a dict, and {x % 3 for x in nums} builds a set.

Two different ifs

This is the detail that trips almost everyone. An if after the source is a filter: it removes items, so the output can be shorter. An if ... else in front of the for is a conditional expression: it picks a value for each item, so the output is exactly as long as the input. Try the third example above to see all four items survive.

Predict first. What is [v if v > 0 else 0 for v in [-3, 5, -1]]?

Predict first. How many items are in [w for w in ["a", "bb", "ccc"] if len(w) > 1]?

When to stop using one

Use a comprehension to build a collection from one source with a simple expression and at most one filter.

Don't

Cram several nested conditions into one line just because it fits.

Chapter context

As a program grows, the difficult part becomes keeping state and responsibilities understandable. Use simple functions for transformations, classes for related state and behavior, and modules to give those responsibilities names.

TRY IT YOURSELF

Exercise 1: square the positives

Write positive_squares(values). It returns a **new list** holding the square of every value that is greater than 0, in the original order.
Write it as one list comprehension: [expression for item in values if condition]. A loop with append gives the right numbers but this exercise checks for the comprehension.
Remember that 0 is not greater than 0.

def positive_squares(values):
    # TODO: one list comprehension, like [item for item in values if ...]
    pass

print(positive_squares([-1, 2, 3]))   # should print [4, 9]

Ctrl/⌘ + Enter runs it

Exercise 2: keep the length, or filter

clip_negatives(values) keeps **every** item but replaces each negative number with 0. The result has the same length as the input. This needs the conditional expression a if condition else b in front of the for.
word_lengths(words, min_len) returns a dict that maps each word to its length, only for words with at least min_len letters. Use a dict comprehension {key: value for ... if ...}.
Notice the difference: an if at the end removes items, an if ... else at the start changes them.

def clip_negatives(values):
    # TODO: same length as values; negatives become 0
    pass

def word_lengths(words, min_len):
    # TODO: {word: len(word)} for words with len(word) >= min_len
    pass

print(clip_negatives([-1, 2, -3]))        # [0, 2, 0]
print(word_lengths(["a", "bee", "cats"], 3))   # {'bee': 3, 'cats': 4}

Ctrl/⌘ + Enter runs it

Both exercises must be solved to finish this lesson.

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Square the positive values in [-1, 2, 3].

AThe result is [-1, 4, 9], because the condition is applied only to the squared values.BThe result is [1, 4, 9], because the filter only affects the order, not which items are kept.CThe result is [4, 9].

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

Reason about state ownership before adding abstraction. For a transformation f, repeated calls should have independent results unless shared state is explicitly intended. A lazy iterator retains only the state required for its next step; it does not imply that downstream consumers also use constant memory.

Worked example

An inventory starts with 5 items. reserve(2) leaves 3. A second independent inventory still contains 5. If both objects instead refer to the same list, mutating one changes what both observe. Draw object references as arrows to make this distinction explicit.

Reference code

The following example uses the Python standard library and does not call a remote service.

class Inventory:
    def __init__(self, count):
        self.count = count
    def reserve(self, n):
        if n > self.count:
            raise ValueError("insufficient stock")
        self.count -= n
first, second = Inventory(5), Inventory(5)
first.reserve(2)
assert (first.count, second.count) == (3, 5)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Type annotations are not validation. Catching Exception and continuing can hide corrupt state. Lazy iteration does not make a later list(...) memory-free.