Configuration and secrets
Navigate a project and describe how its interpreter, dependencies and configuration are selected.
THE IDEA
Configuration lives outside the code
The same program often runs with different settings: a test key on your laptop, a real one in production. If a setting is written into the source, changing it means editing code, and secrets end up in version control. A common convention is to read configuration from environment variables.
An environment variable belongs to a process and is inherited by the processes it starts, but only if it is exported. A plain assignment is private to your shell.
API_KEY=abc123 $ echo $API_KEY abc123
$ env | grep API_KEY (no output: it never saw the variable)
The .env pattern
.envholds the real values for this machine. It is never committed..gitignorelists.envso git does not pick it up by accident..env.exampleis committed and lists the variable names with no values, so a teammate knows what to set.
A .env file is a convention that many tools understand, not a feature of the shell. In a shell, source .env loads its NAME=value lines as shell variables. To make them visible to programs you start, export them: export NAME, or set -a before sourcing and set +a after.
Ignoring a file does not erase history
.gitignore only affects files git is not already tracking. If a secret was committed once, it stays in the history after you ignore or delete the file. Try the sequence yourself and watch the history.
Start by creating the .env file that holds your key.
no commits yet
No.
Nothing leaked.
Try step 3 before step 2 and compare. Once a secret is in a commit, only rotating it makes the leak harmless.
Predict first. You committed .env, then added it to .gitignore. Is the old copy gone?
Chapter context
A terminal runs commands in a current working directory. An environment makes the program's interpreter, dependencies and configuration explicit so a successful run can be repeated elsewhere.
TRY IT YOURSELF
The key below is a fake demo value. Never paste a real key into any tutorial, terminal log or chat.
- Find every hard-coded copy of the key with grep
- Put the secret in a file named .env as API_KEY=...
- Rewrite app.py so it reads the key from os.environ and no longer contains it
- Tell git to ignore .env by adding it to .gitignore
- Create .env.example that lists the variable name with no secret value
- Load .env into your shell and export the variable so programs you start can see it
- Confirm a child program sees it: env | grep API_KEY should print the line
PATH search order
- /usr/local/bin
- /usr/bin
- /bin
Exported variables
Exported variables only. Your private shell variables are not listed here.
Stuck, or something not working? You can come back to it any time.
CHECK YOUR UNDERSTANDING
Does placing a key in .gitignore remove an earlier committed copy?
Chapter reference: math, worked example & code
This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.
Math and notation
A reproducible run can be described as (source revision, interpreter, dependency resolution, input data, configuration). Equal source code alone is insufficient. For pipelines, compare the records emitted at each stage rather than assuming every command preserves row counts.
Worked example
A project processes 12 lines, filters to 5 matching lines, then counts them. The final count is 5, not 12. In a path puzzle, joining /work/project with data and train.csv identifies the input without changing the real filesystem.
Reference code
The following example uses the Python standard library and does not call a remote service.
from pathlib import PurePosixPath
path = PurePosixPath('/work/project') / 'data' / 'train.csv'
assert str(path) == '/work/project/data/train.csv'
lines = ['keep' if i < 5 else 'drop' for i in range(12)]
assert sum(line == 'keep' for line in lines) == 5
Python examples are provided for study and local execution; this page does not execute Python.
Common mistakes
A virtual environment is not a security sandbox. A lockfile does not preserve external input datasets. Ignoring a secret after committing it does not undo exposure.