D5.2 · LESSON 2 OF 6

Stride and padding

Calculate a convolutional output and explain filters, stride, padding, pooling and residual connections.

THE IDEA

Stride and padding decide the output size

Stride is how far the window moves each step. Stride 1 visits every position, stride 2 skips every other one and roughly halves the output. Padding adds zeros around the border so that edge pixels can sit in the centre of a window and so that the output does not shrink.

The output length for input size n, kernel k, padding p and stride s is

out = floor((n + 2p − k) / s) + 1

input length n7kernel k3padding p1stride s2

012345670

■■■······window 1

··■■■····window 2

····■■■··window 3

······■■■window 4

out = floor((n + 2p − k) / s) + 1 = floor((7 + 2 − 3) / 2) + 1 = 4

Each row marks one window position. Zeros at the edges are padding.

Predict first. A length-five input, a length-three kernel, stride one and no padding. How many outputs?

The worked example: n = 7, k = 3, p = 1, s = 2 gives floor((7 + 2 − 3) / 2) + 1 = 4 outputs.

Chapter context

A convolutional layer reuses a local filter across positions. Weight sharing encodes a structural assumption about repeated local patterns while greatly reducing parameters compared with a fully connected map over all pixels.

TRY IT YOURSELF

Part 1. Do it hands-on

The calculator on top is free to explore. Below it, set the numbers each task asks for, then implement the size formula and a full strided, padded 2-D convolution in Python.

input length n7kernel k3padding p1stride s2

012345670

■■■······window 1

··■■■····window 2

····■■■··window 3

······■■■window 4

out = floor((n + 2p − k) / s) + 1 = floor((7 + 2 − 3) / 2) + 1 = 4

Length-5 input, filter 3, stride 1, no padding: outputs

n=7: padding0n=7: stride1n=8, k=5: padding0n=16, k=3, s=2: padding0

n=7, k=3: 5 outputs n=8, k=5, s=1: 4 outputs n=16, k=3, s=2: 7 outputs

To do: A length-5 input, a length-3 filter, stride 1, no padding: how many outputs? Type the number
To do: Input length 7, filter 3: choose padding and stride so that there are exactly 4 outputs
To do: Input length 8, filter 5, stride 1: choose the padding that keeps the output length at 8 ("same" padding)
To do: Input length 16, filter 3, stride 2: choose the padding that halves it to exactly 8

Part 2. Do it in code

conv_out_size(n, k, p=0, s=1, d=1): floor((n + 2p - d(k-1) - 1) / s) + 1. Raise ValueError if the result would be less than 1, or if k, s or d is below 1 or p is negative.
conv2d(img, ker, stride=1, padding=0): 2-D cross-correlation of a 2-D image with a 2-D kernel. Pad the image with zeros on all sides first, then move the window by `stride`. Return an array whose shape follows conv_out_size in each direction.

import numpy as np

def conv_out_size(n, k, p=0, s=1, d=1):
    pass

def conv2d(img, ker, stride=1, padding=0):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

For a length-five input, length-three filter and stride one without padding, how many outputs?

AThree: floor((5 − 3) / 1) + 1 = 3, because without padding each layer loses k − 1 positions.BTwo: the kernel covers three positions, so only two windows remain.CFive: the output has the same length as the input.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

For one spatial dimension, output size is

n_{out}=\left\lfloor\frac{n+2p-d(k-1)-1}{s}\right\rfloor+1.

Here n is input size, k kernel size, p padding, d dilation and s stride. Cross-correlation is $y_t=\sum_iw_i x_{t+i}$ . A residual block has form $h_{out}=h+F(h)$ when shapes agree.

Worked example

For input [1,2,3,4] and kernel [1,−1], stride-one valid cross-correlation returns [−1,−1,−1]. Separately, n=7,k=3,p=1,s=2,d=1 gives floor((7+2−2−1)/2)+1=4 output positions.

Reference code

The following example uses the Python standard library and does not call a remote service.

x,w = [1,2,3,4],[1,-1]
y = [sum(x[t+i]*w[i] for i in range(len(w))) for t in range(len(x)-len(w)+1)]
assert y == [-1,-1,-1]
n,k,p,s,d = 7,3,1,2,1
assert (n+2*p-d*(k-1)-1)//s+1 == 4

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Padding does not reveal missing real pixels. Kernel flipping conventions change calculations. A residual shortcut needs a defined shape-compatible path.