D6.2 · LESSON 2 OF 6

Unrolling and backpropagation

Trace a recurrent state and explain memory loss, gating, sequence-to-sequence models and attention.

THE IDEA

Unrolling shows the repeated use

To train a recurrent network, unroll it: draw one copy of the cell per time step, all sharing the same weights. The result is an ordinary deep network, and backpropagation works on it as before. Two rules follow.

A gradient travelling back along the states is multiplied by the local sensitivity of each step.
A weight used at every step collects a gradient contribution from every step, and these are added.

For the one-number recurrence with loss equal to the last state h_T, the gradient reaching the input at step k is exactly a^(T−k): the weight multiplied in once per step in between.

weight a0.8steps T8

input	dloss/dx_k = a^(T−k)
x_1	0.2097
x_2	0.2621
x_3	0.3277
x_4	0.4096
x_5	0.512
x_6	0.64
x_7	0.8
x_8	1

The loss is the last state. The gradient reaching x_1 passes through T−1 copies of the same weight, so its size is a^(T−1). The gradient on the shared weight itself is the sum of one term per use.

Move a and T. Early inputs get tiny gradients when a is below 1.

Predict first. Why can an early input have a tiny gradient?

Chapter context

A sequence model processes ordered observations. A recurrent state summarizes previous inputs, creating both a mechanism for memory and a long differentiation path whose sensitivity can decay or grow.

TRY IT YOURSELF

Part 1. Do it hands-on

Compute two gradients by hand, then tune the weight to make an early input vanish or stay alive. The Python part implements backpropagation through time and checks it against finite differences.

weight a0.8steps T8

input	dloss/dx_k = a^(T−k)
x_1	0.2097
x_2	0.2621
x_3	0.3277
x_4	0.4096
x_5	0.512
x_6	0.64
x_7	0.8
x_8	1

dloss/dx_1 (T = 5, a = 0.5)

dloss/da (inputs [1,0,0,0], a = 0.5)

a (make it vanish)0.5a (keep it alive)0.5

T = 10, first-input gradient a^9: a = 0.5 → 1.95e-3 a = 0.5 → 1.95e-3

To do: Linear RNN h_t = a·h_(t−1) + x_t, loss = h_5, a = 0.5. What is dloss/dx_1 (the very first input)? Type the number
To do: Inputs [1, 0, 0, 0], a = 0.5, loss = h_4. What is dloss/da? Type the number
To do: With 10 steps, choose a so the gradient to the FIRST input has vanished (below 0.001)
To do: Now choose a so that, over the same 10 steps, the first input still has an influence between 0.5 and 2

Part 2. Do it in code

bptt_linear(a, xs): the linear recurrence h_t = a*h_(t-1) + x_t with h_0 = 0 and loss = h_T (the last state). Return (h, dx, da): the list of states, the list dloss/dx_t for every input and dloss/da. Compute them with the backward recurrence: g = 1 at the last step, going backwards dx_t = g, da += g * h_(t-1), then g = g * a.
numeric_grad(f, x, eps=1e-6): central finite difference (f(x+eps) - f(x-eps)) / (2*eps) of a scalar function.
gradient_error(a, xs): the largest absolute difference between your analytic dx and finite differences of the loss with respect to each x_t, and between da and the finite difference in a. Use numeric_grad and bptt_linear.

def bptt_linear(a, xs):
    pass

def numeric_grad(f, x, eps=1e-6):
    pass

def gradient_error(a, xs):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Why can an early input have a tiny gradient?

AThe gradient reaches it through many steps, each multiplying in a local sensitivity, and a product of many factors below 1 is tiny.BEarly inputs are ignored by the loss function.CGradients are only ever computed for the last input.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

A simple recurrence is $h_t=\phi(W_hh_{t-1}+W_xx_t+b)$ . In a scalar linear example $h_t=a h_{t-1}+x_t$ , sensitivity to the initial state is

\frac{\partial h_T}{\partial h_0}=a^T.

This illustrates gradient decay or growth; nonlinear multidimensional RNNs have Jacobian products instead.

Worked example

Set a=0.5 and h0=0. Input [1,0,0] produces states [1,0.5,0.25]. The initial impulse is still present but attenuated. Over ten subsequent multiplicative transitions, its factor is 0.5^10=0.0009765625.

Reference code

The following example uses the Python standard library and does not call a remote service.

state, states = 0.,[]
for x in [1,0,0]:
    state=.5*state+x
    states.append(state)
assert states == [1,.5,.25]
assert .5**10 == .0009765625

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A recurrent model shares weights through time. Longer hidden vectors do not automatically prevent gradient decay. Training history and generated inference history can differ.