D8.2 · LESSON 2 OF 6

Transfer learning

Compare learned embeddings, transfer learning, fine-tuning and contrastive training objectives.

THE IDEA

Reuse what was already learned

Training a good encoder takes lots of data and compute. Transfer learning reuses an encoder trained elsewhere and learns only what is new. The cheapest version freezes the encoder and trains a fresh output head on top: a baseline you should always run, because if it already works, deeper adaptation may not be worth it.

freeze the encoder (head-only training)

part	parameters	state	after one update
pretrained encoder	100	frozen	unchanged
new output head	10	trainable	changed
trainable total	10 of 110

head weight w = 0.5, x = 2, y = 3, lr = 0.1 w ← w − lr·2(w·x − y)·x = 0.5 − 0.1·2·(1 − 3)·2 = 1.3

Tick or untick the freeze. The update shown is the head-only step from the exercise.

Predict first. Does a frozen encoder update during head-only training?

NumPy

F = np.tanh(X @ W)               # frozen encoder: W is never assigned to again
w = w - lr * (2/len(F)) * F.T @ (F @ w - y)   # only the head moves

Chapter context

A representation is useful when its structure supports a downstream task. Learning a representation changes which examples appear similar, but those similarities inherit assumptions from training data and objectives.

TRY IT YOURSELF

Part 1. Do it hands-on

Count trainable parameters and do one head update by hand. In Python, build a head-only probe and prove the encoder never changes.

freeze the encoder (head-only training)

part	parameters	state	after one update
pretrained encoder	100	frozen	unchanged
new output head	10	trainable	changed
trainable total	10 of 110

head weight w = 0.5, x = 2, y = 3, lr = 0.1 w ← w − lr·2(w·x − y)·x = 0.5 − 0.1·2·(1 − 3)·2 = 1.3

trainable parameters (frozen encoder) new head weight frozen encoder weight afterwards (was 0.7)

To do: An encoder has 100 parameters, a new output head 10. Head-only training with a frozen encoder: how many parameters are trainable?
To do: The head weight is w = 0.5, input x = 2, target y = 3, learning rate 0.1, loss (w·x − y)². One update: w ← w − lr·2(w·x − y)·x. Type the new w
To do: During that same head-only step, the frozen encoder weight was 0.7. Type its value afterwards

Part 2. Do it in code

features(X, W): the frozen encoder, np.tanh(X @ W).
train_head(F, y, lr, steps): fit a linear head w (no bias, start at zeros) on the features F with squared error. Each step: w = w − lr · (2/n) · F.T @ (F @ w − y). Return w.
probe(X, y, W, lr, steps): head-only training on top of the frozen encoder. Compute features once, train the head, and return (w, W_after) where W_after must be exactly the encoder weights you started with (never modified).
count_trainable(groups, frozen): groups is a dict name -> number of parameters; frozen is a set of names. Return the number of trainable parameters.

import numpy as np

def features(X, W):
    pass

def train_head(F, y, lr, steps):
    pass

def probe(X, y, W, lr, steps):
    pass

def count_trainable(groups, frozen):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Does a frozen encoder update during head-only training?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

One contrastive loss for an anchor a and positive p is

L=-\log\frac{\exp(s(a,p)/\tau)}{\sum_{j\in\mathcal C}\exp(s(a,j)/\tau)}.

The candidate set includes the designated positive; other candidate definitions depend on the method. Temperature tau is positive and controls concentration.

Worked example

For a candidate set with positive logit ln 2 and one alternative logit 0, exponentiated weights are 2 and 1. The positive probability is 2/3 and loss is −ln(2/3)≈0.405465. This demonstrates the normalization; actual contrastive batches usually contain many more candidates.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import log, exp, isclose
logits=[log(2),0]
prob=exp(logits[0])/sum(exp(v) for v in logits)
assert isclose(prob,2/3)
assert isclose(-log(prob),.40546510810816444)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Embedding proximity is task-dependent. Frozen parameters can coexist with a trainable head. Parameter-efficient tuning still requires evaluation for forgetting and task failures.