Foundations: Python, Git and toolsLoading progress…

Paths and directories

Navigate a project and describe how its interpreter, dependencies and configuration are selected.

Every command runs somewhere

A terminal is always standing in one directory, called the current working directory. Every relative path you type is measured from that spot, so the same command can succeed in one folder and fail in another. Many "file not found" errors are not about the file at all. They are about where you are standing.

$ pwd
/work/project
$ python load.py        # opens "data/train.csv"
loaded 4 rows

$ cd /work
$ python load.py
FileNotFoundError: data/train.csv     # there is no /work/data/train.csv

Absolute and relative paths

An absolute path starts with / and means the same thing from anywhere. A relative path has no leading slash and is resolved against the current directory. Three short names do most of the work:

  • . is the current directory.
  • .. is the parent directory, one level up.
  • ~ is your home directory, for example /home/learner.
  1. (start at /work/project)
    /workproject
  2. data
    /workprojectdata
  3. train.csv
    /workprojectdatatrain.csv

Absolute path: /work/project/data/train.csv

A path starting with / ignores the current directory. Anything else starts from it. Try ../../etc, ./data, or /data/train.csv.

Type any path and watch it resolve one piece at a time. A .. removes the last folder.

Check yourself first

Predict first. You are in /work/project. Which absolute path does data/train.csv point to?

Predict first. You are in /work/project/data. Where does ../../etc lead?

Chapter context

A terminal runs commands in a current working directory. An environment makes the program's interpreter, dependencies and configuration explicit so a successful run can be repeated elsewhere.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

A reproducible run can be described as (source revision, interpreter, dependency resolution, input data, configuration). Equal source code alone is insufficient. For pipelines, compare the records emitted at each stage rather than assuming every command preserves row counts.

Worked example

A project processes 12 lines, filters to 5 matching lines, then counts them. The final count is 5, not 12. In a path puzzle, joining /work/project with data and train.csv identifies the input without changing the real filesystem.

Reference code

The following example uses the Python standard library and does not call a remote service.

from pathlib import PurePosixPath
path = PurePosixPath('/work/project') / 'data' / 'train.csv'
assert str(path) == '/work/project/data/train.csv'
lines = ['keep' if i < 5 else 'drop' for i in range(12)]
assert sum(line == 'keep' for line in lines) == 5

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A virtual environment is not a security sandbox. A lockfile does not preserve external input datasets. Ignoring a secret after committing it does not undo exposure.