Image Processing Fundamentals

InIn 1921, a system called the Bartlane cable picture transmission service began sending newspaper photographs across the Atlantic by submarine telegraph cable, coded at the sending end into five intensity levels and reconstructed at the receiving end into a printable image — cutting the time to move a picture from London to New York from more than a week by ship to under three hours¹. It's a reasonable candidate for the first working piece of digital image processing, decades before a general-purpose computer existed to do the arithmetic by any other means: a photograph was already being represented as a small number of discrete levels and transmitted as data, which is the same basic move every technique on this page still makes.

A digital image is, underneath any amount of visual meaning a human reads into it, a grid of numbers: a 2D array for a greyscale image, where each cell (pixel) holds an intensity value, or three stacked arrays — red, green, blue — for a colour image. RGB is an additive representation, matched to how a screen actually produces colour (mixing red, green and blue light), but it isn't always the most convenient one to compute with: RGB tangles brightness together with colour, so a shadow falling across a red object changes all three of its R, G and B values at once. HSV (hue, saturation, value) re-parameterises the same colour as an angle around a colour wheel (hue), how far from grey it is (saturation), and how bright it is (value) — separating "what colour is this" from "how brightly lit is it," which is why a segmentation step (covered on the next page) that thresholds on hue rather than raw RGB tends to be far more robust to shadows and uneven lighting. Every technique below can be read as one question asked in different ways: given a grid of numbers, what arithmetic on it produces a grid that's more useful for some downstream purpose¹?

Spatial-Domain Enhancement

SpatialSpatial domain" just means working directly on pixel values, as opposed to transforming the image into some other representation first (more on that below). The simplest spatial operation, a histogram of pixel intensities, already does useful work: an image whose histogram is bunched into a narrow range looks flat and low-contrast, and histogram equalisation — remapping intensities so the histogram spreads out to use the full available range — is a cheap, purely arithmetic way to make a dim or washed-out image easier to read, with no knowledge of what the image actually depicts.

Convolution-based filtering goes further: slide a small matrix (a kernel) over the image, and at each position, replace the centre pixel with a weighted sum of itself and its neighbours, weighted by the kernel's values. A averaging (box blur) kernel and a sharpening kernel are opposites built from the same mechanism:

# 3x3 averaging kernel: smooths (blurs) the image
blur_kernel = [
    [1/9, 1/9, 1/9],
    [1/9, 1/9, 1/9],
    [1/9, 1/9, 1/9],
]

# 3x3 sharpening kernel: exaggerates local contrast
sharpen_kernel = [
    [ 0, -1,  0],
    [-1,  5, -1],
    [ 0, -1,  0],
]

def convolve_pixel(image, x, y, kernel):
    total = 0
    for ky in range(-1, 2):
        for kx in range(-1, 2):
            total += image[y + ky][x + kx] * kernel[ky + 1][kx + 1]
    return total

Run the blur kernel over a small 3×3 neighbourhood with values [[10,10,10],[10,90,10],[10,10,10]] (a bright pixel surrounded by dark ones) and the centre pixel becomes (10*8 + 90)/9 ≈ 18.9 — the bright outlier gets pulled toward its neighbours' average, which is exactly what a blur does to noise. The sharpen kernel does the reverse: it subtracts a fraction of each neighbour from an amplified centre value, pushing the centre away from the local average and making edges more pronounced.

Why a Frequency Domain Exists

EveryEvery image can be described as a sum of 2D sine waves of different spatial frequencies and orientations — the same idea as decomposing an audio signal into pitches, just in two spatial dimensions instead of one temporal one. The Fourier transform converts an image from "value at each pixel position" into "how much of each spatial frequency is present," and some operations that are awkward in the spatial domain become simple arithmetic in the frequency domain: removing regular, repeating noise (a striped scan artefact, say) means finding the specific frequency spike it produces and zeroing it out — a single, targeted edit — rather than trying to design a spatial kernel that removes exactly that pattern and nothing else. The trade-off is that a frequency-domain edit affects the whole image at once (frequency is a global property of the picture), where a spatial kernel's effect is local and easy to reason about pixel by pixel — each domain is the natural home for a different kind of operation, not a strictly better or worse one.

Morphological Operations

MorphologyMorphology treats a binary (black/white) image as a set of shapes and asks purely geometric questions about it, using a small structuring element — typically a 3×3 or plus-shaped stencil — as the unit of comparison.

Operation	Effect
Erosion	A foreground pixel survives only if every neighbour under the structuring element is also foreground — shrinks shapes, removes small specks
Dilation	A background pixel becomes foreground if any neighbour under the structuring element is foreground — grows shapes, fills small holes
Opening	Erosion then dilation — removes small specks without significantly changing the size of larger shapes
Closing	Dilation then erosion — fills small holes and gaps without significantly changing the size of larger shapes

Take a tiny 5×5 binary image (1 = foreground) with a single isolated noise pixel sitting apart from a larger 3×3 block:

0 0 0 0 0
0 1 1 1 0
0 1 1 1 0
0 1 1 1 0
0 0 0 1 0   <- isolated noise pixel, bottom row

Eroding with a plus-shaped structuring element removes any foreground pixel that doesn't have foreground neighbours on all four sides — the isolated pixel at the bottom, having no foreground neighbours at all, disappears immediately, while the solid 3×3 block shrinks to a single surviving centre pixel. Opening (erode then dilate) restores the block close to its original size while the noise pixel, having been erased in the erosion step, never comes back — exactly the "remove small specks without disturbing large shapes" behaviour the operation is named for.

Where This Leads

EverythingEverything above treats "what should this pixel become?" as a question answered by fixed, hand-designed arithmetic — a kernel or a structuring element chosen because someone understood the geometry of the problem. The next step, covered in Computer Vision and Object Recognition, is asking a harder question — not "what should this pixel become?" but "what object is this?" — which is where fixed filters stop being enough and feature extraction, and eventually learned models, take over.

References