D8.3 · LESSON 3 OF 6

Fine-tuning

Compare learned embeddings, transfer learning, fine-tuning and contrastive training objectives.

THE IDEA

Adapting everything, and what it can cost

Fine-tuning continues training the pretrained parameters on the new task. It usually fits the new task better than a frozen encoder, but the parameters move away from where the old skills lived, and those old skills can degrade. This is often called forgetting.

One weight w. Task A (the pretrained skill) is best at w = 1. Task B (the new task) is best at w = 3. Fine-tuning minimises task B only.

fine-tuning steps3

steps	w	task B loss	task A loss
0	1	4	0
1	1.4	2.56	0.16
2	1.72	1.638	0.518
3	1.976	1.049	0.953
4	2.181	0.671	1.394
5	2.345	0.429	1.808
6	2.476	0.275	2.178
7	2.581	0.176	2.498
8	2.664	0.113	2.77
9	2.732	0.072	2.998
10	2.785	0.046	3.187
11	2.828	0.03	3.342
12	2.863	0.019	3.469

w ← w − 0.1·2·(w − 3) at 3 steps: B loss 1.049 (want ≤ 1.1), A loss 0.953 (want ≤ 1) ✓ balanced

One weight. The pretrained value is best for task A; fine-tuning drags it toward task B. Slide the step count.

Predict first. Is more fine-tuning always a quality improvement?

Chapter context

A representation is useful when its structure supports a downstream task. Learning a representation changes which examples appear similar, but those similarities inherit assumptions from training data and objectives.

TRY IT YOURSELF

Part 1. Do it hands-on

Take fine-tuning steps by hand and find the stopping point that balances both tasks. Then code the path, a stop picker and a forgetting measure.

One weight w. Task A (the pretrained skill) is best at w = 1. Task B (the new task) is best at w = 3. Fine-tuning minimises task B only.

fine-tuning steps0

steps	w	task B loss	task A loss
0	1	4	0
1	1.4	2.56	0.16
2	1.72	1.638	0.518
3	1.976	1.049	0.953
4	2.181	0.671	1.394
5	2.345	0.429	1.808
6	2.476	0.275	2.178
7	2.581	0.176	2.498
8	2.664	0.113	2.77
9	2.732	0.072	2.998
10	2.785	0.046	3.187
11	2.828	0.03	3.342
12	2.863	0.019	3.469

w ← w − 0.1·2·(w − 3) at 0 steps: B loss 4 (want ≤ 1.1), A loss 0 (want ≤ 1)

w after one step task A loss after 5 steps (3 decimals)

To do: A model pretrained for task A sits at w = 1. Task B wants w = 3 (loss (w − 3)²). One gradient step with learning rate 0.1: type the new w
To do: After 5 steps w = 2.34464. Task A's loss (w − 1)² has risen from 0. Type it (3 decimals)
To do: Choose how many fine-tuning steps to run so that task B's loss is at most 1.1 AND task A's loss stays at most 1

Part 2. Do it in code

finetune(w0, target, lr, steps): gradient descent on the loss (w − target)². Return the list of w values, starting with w0, length steps + 1.
loss(w, target): (w − target)².
pick_stop(path, target_b, target_a, max_b, max_a): the first index n of the path where loss(path[n], target_b) <= max_b AND loss(path[n], target_a) <= max_a, or None if no index qualifies.
forgetting(path, target_a): the list of task A losses along the path minus the starting task A loss (how much was forgotten).

def finetune(w0, target, lr, steps):
    pass

def loss(w, target):
    pass

def pick_stop(path, target_b, target_a, max_b, max_a):
    pass

def forgetting(path, target_a):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Is more fine-tuning always a quality improvement?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

One contrastive loss for an anchor a and positive p is

L=-\log\frac{\exp(s(a,p)/\tau)}{\sum_{j\in\mathcal C}\exp(s(a,j)/\tau)}.

The candidate set includes the designated positive; other candidate definitions depend on the method. Temperature tau is positive and controls concentration.

Worked example

For a candidate set with positive logit ln 2 and one alternative logit 0, exponentiated weights are 2 and 1. The positive probability is 2/3 and loss is −ln(2/3)≈0.405465. This demonstrates the normalization; actual contrastive batches usually contain many more candidates.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import log, exp, isclose
logits=[log(2),0]
prob=exp(logits[0])/sum(exp(v) for v in logits)
assert isclose(prob,2/3)
assert isclose(-log(prob),.40546510810816444)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Embedding proximity is task-dependent. Frozen parameters can coexist with a trainable head. Parameter-efficient tuning still requires evaluation for forgetting and task failures.