Foundations: Python, Git and toolsLoading progress…
F4.3 · LESSON 3 OF 6

Environments

Navigate a project and describe how its interpreter, dependencies and configuration are selected.

THE IDEA

Which python am I running?

When you type python, the shell does not know where that program is. It walks the directories listed in the PATH variable from left to right and runs the first match. Everything about virtual environments follows from that one rule.

  1. /usr/local/binno python here
  2. /usr/binno python here
  3. /binno python here

$ which python → nothing: “command not found”

Switch activation on and off, and pick a command. The first directory containing it wins; later ones are never searched.

What a virtual environment is

A virtual environment is a folder with its own Python interpreter links and its own site-packages directory, where installed packages live. Create one with python3 -m venv .venv. Packages installed into it stay in it, so two projects can need different versions without fighting.

source .venv/bin/activate only changes your current shell: it puts .venv/bin at the front of PATH, sets VIRTUAL_ENV and adds a prefix to your prompt. deactivate undoes it. Activation is a convenience, not a requirement: .venv/bin/python script.py uses the environment directly.

The classic trap: installed, but cannot import

pip installs into the environment that its own interpreter belongs to. If the pip you ran and the python you import with belong to different environments, pip reports success and the import still fails.

Environment A .venv
site-packages: requests ✓
Environment B .venv-old
site-packages: (empty)

import works: the package sits in the interpreter that is importing it.

This is why which python and which pip should point at the same environment, and why python -m pip is safer: it uses the pip of the interpreter you name.

Pick which environment pip installs into and which one Python runs from.
Do

Check which python and which pip point into the same folder, or run python -m pip install ... so the pip belongs to that exact interpreter.

Don't

Assume the pip in your terminal belongs to the python your editor, notebook or script uses.

Chapter context

A terminal runs commands in a current working directory. An environment makes the program's interpreter, dependencies and configuration explicit so a successful run can be repeated elsewhere.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

A reproducible run can be described as (source revision, interpreter, dependency resolution, input data, configuration). Equal source code alone is insufficient. For pipelines, compare the records emitted at each stage rather than assuming every command preserves row counts.

Worked example

A project processes 12 lines, filters to 5 matching lines, then counts them. The final count is 5, not 12. In a path puzzle, joining /work/project with data and train.csv identifies the input without changing the real filesystem.

Reference code

The following example uses the Python standard library and does not call a remote service.

from pathlib import PurePosixPath
path = PurePosixPath('/work/project') / 'data' / 'train.csv'
assert str(path) == '/work/project/data/train.csv'
lines = ['keep' if i < 5 else 'drop' for i in range(12)]
assert sum(line == 'keep' for line in lines) == 5

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A virtual environment is not a security sandbox. A lockfile does not preserve external input datasets. Ignoring a secret after committing it does not undo exposure.