ML10.5 · LESSON 5 OF 6

Time series and recommendation

Distinguish sequential decision-making, temporal prediction, recommendation and causal estimation.

THE IDEA

Time series: the future is not shuffled

Data ordered in time breaks the usual train/test habit. If you shuffle rows, a test point sits between training points from the same week, so the model only has to interpolate a gap. In use it must predict a future it has never seen. A fair evaluation trains on the past and tests on the future.

Always compare against simple baselines. A seasonal naive forecast repeats the value from one period ago. A moving average predicts the mean of the last few values. If a complicated model cannot beat these on a time-ordered split, it has not learned anything useful.

Predict first. You evaluate the same forecaster once with a shuffled split and once with a time-ordered split. Which gives the smaller error, and why?

trainingtestpredictionunused

training months30

train points: 30 · test points: 12 (months 30 to 41, all after the training months)

MAE (time-ordered): 20

same model, shuffled split: 15.75

Choose the split and the forecaster. Watch the error change and where the training and test regions lie on the series.

Recommendation: sparse, biased feedback

A recommender ranks items a person has not seen, using ratings that are mostly missing. With no history for a user, the only sensible fallback is popularity (the cold start case). Once a few ratings exist, items can be ranked by similarity to what the person liked.

Feedback is biased by what was shown. An item with no click was not necessarily rejected: it may never have been displayed. Treating every missing entry as a dislike teaches the model to repeat whatever it already showed.

user	Cosmic Heist	Neon Chase	Iron Orbit	Quiet Harbor	Tiny Garden	Soft Rain	Fresh Release
Asha	5	4	5	1	·	2	·
Ben	4	5	·	2	1	·	·
Chen	·	5	4	1	2	1	·
Dana	1	·	2	5	4	5	·
Eli	2	1	·	4	5	·	·
You

Cold start: nothing known about you, so everyone gets the popularity list

Neon Chase: 3.75 (popular)

Iron Orbit: 3.67 (popular)

Cosmic Heist: 3 (popular)

Tiny Garden: 3 (popular)

Soft Rain: 2.67 (popular)

Quiet Harbor: 2.6 (popular)

Fresh Release: no ratings yet, cannot be recommended

Why this score?

You have not rated anything, so there is no taste to compare with. A real system falls back to popularity, location, or asking you a few questions.

Rate a few items and watch the recommendations switch from popularity to your own taste.

Chapter context

These topics change the structure of the learning problem. Their first lesson is to identify what a prediction or action is allowed to depend on and how feedback is produced. Each section is a bridge to a larger specialist subject.

TRY IT YOURSELF

Part 1. Evaluate honestly

Compare a shuffled split with a time-ordered one, then pick the best forecaster for the test year.

trainingtestpredictionunused

training months30

train points: 30 · test points: 12 (months 30 to 41, all after the training months)

MAE (time-ordered): 20

same model, shuffled split: 15.75

To do: Evaluate the nearest-point model with a shuffled split and see a suspiciously small error (MAE under 10)
To do: Evaluate the same model with a time-ordered split: the honest MAE must be at least twice as large
To do: With the time-ordered split, choose the forecaster with the smallest honest MAE for the test year (train on at least 24 months)

Part 2. Shape a recommender

Your ratings are the only data. Change them to move items to the top of the list.

user	Cosmic Heist	Neon Chase	Iron Orbit	Quiet Harbor	Tiny Garden	Soft Rain	Fresh Release
Asha	5	4	5	1	·	2	·
Ben	4	5	·	2	1	·	·
Chen	·	5	4	1	2	1	·
Dana	1	·	2	5	4	5	·
Eli	2	1	·	4	5	·	·
You

Cold start: nothing known about you, so everyone gets the popularity list

Neon Chase: 3.75 (popular)

Iron Orbit: 3.67 (popular)

Cosmic Heist: 3 (popular)

Tiny Garden: 3 (popular)

Soft Rain: 2.67 (popular)

Quiet Harbor: 2.6 (popular)

Fresh Release: no ratings yet, cannot be recommended

Why this score?

You have not rated anything, so there is no taste to compare with. A real system falls back to popularity, location, or asking you a few questions.

To do: Rate at least two items so the list switches from popularity (cold start) to predictions made from your own ratings
To do: Rate items so that Tiny Garden is your top recommendation (and you have not rated it)
To do: Now rewrite your taste so that Iron Orbit is the top recommendation instead

Part 3. Do it in code

Write time_split(y, test_size): return (train, test) where test is the last test_size values and train everything before, in order. Raise ValueError if test_size is negative or larger than len(y).
Write seasonal_naive(train, horizon, period): forecast step i as train[len(train) - period + (i % period)]. Raise ValueError when period is not positive or train is shorter than one period.
Write moving_average_forecast(train, horizon, k): a flat forecast equal to the mean of the last k training values (use all of them if there are fewer than k). Raise ValueError for k below 1 or an empty train.
Write mae(actual, predicted): the mean absolute error; 0.0 for two empty lists; ValueError if the lengths differ.

def time_split(y, test_size):
    pass

def seasonal_naive(train, horizon, period):
    pass

def moving_average_forecast(train, horizon, k):
    pass

def mae(actual, predicted):
    pass

y = [10, 12, 14, 11, 13, 15]
print(time_split(y, 2))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Does an unclicked item always mean the user disliked it?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

G_t=\sum_{k\ge0}\gamma^k r_{t+k+1},\quad Q(s,a)\leftarrow Q(s,a)+\alpha[r+\gamma\max_{a'}Q(s',a')-Q(s,a)].

For a terminal transition the continuation term is zero. A policy-gradient estimator uses a term proportional to $\nabla\log\pi_\theta(a\mid s)$ times a return or advantage estimate.

Worked example

For a nonterminal transition, current Q=2, reward=1, discount=0.9 and best next Q=4 give target 1+0.9×4=4.6. With alpha=0.5, updated Q is 2+0.5×(4.6−2)=3.3. Reusing the same target for a terminal transition would be incorrect.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import isclose
q, reward, discount, next_q, rate = 2, 1, .9, 4, .5
target = reward + discount*next_q
assert isclose(target, 4.6)
assert isclose(q+rate*(target-q), 3.3)
terminal_target = reward
assert q+rate*(terminal_target-q) == 1.5

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Prediction is not intervention. Randomly splitting overlapping time windows can leak future information. An environment reward is a modeling choice and may omit important real-world consequences.