AI Glossary: Convolutional Neural Network
Convolutional Neural Networks (CNNs) are the deep learning architecture that revolutionized computer vision. They exploit the spatial structure of images using shared filters that slide across the input, detecting hierarchical patterns from edges to objects.
What is a Convolutional Neural Network (CNN)?
A Convolutional Neural Network (CNN) is a deep learning architecture specifically designed for processing grid-structured data such as images, audio signals, and sequential data. CNNs are built around the idea of translation equivariance and parameter sharing—these properties make them uniquely suited for computer vision tasks and far more parameter-efficient than dense (fully connected) networks.
The breakthrough came in 2012 when AlexNet (a deep CNN) won the ImageNet image classification competition by a huge margin, beating all traditional computer vision approaches. Today, CNNs are the foundation of almost all computer vision applications—from face recognition to medical imaging to autonomous driving.
Core Idea
Use small learned filters that slide across the image to detect local patterns like edges, textures, and shapes. The same filter can detect the same pattern anywhere in the image.
Key Advantages
Parameter-efficient, exploits spatial locality, provides translation invariance, automatically learns hierarchical features.
Main Use Cases
Image classification, object detection, semantic segmentation, face recognition, video analysis, medical imaging.
Translation Invariance vs Equivariance
Translation invariance means: if you move an object in the image, the CNN still detects it. Translation equivariance means: if you move the object in the input, the detection moves in the output feature map. CNNs are equivariant by construction, which is exactly what you want for spatial data. Pooling gives some translation invariance on top of that.
How Convolution Works
Convolution is the core operation in CNNs. Here's a step-by-step explanation:
- Define the filter (kernel) — A small matrix of learnable weights, typically 3×3 or 5×5. Each filter detects one specific pattern.
- Slide the filter across the image — The filter moves step-by-step (the step size is called the stride) over the entire width and height of the input.
- Compute the dot product — At each position, compute the dot product between the filter weights and the local image patch.
- Produce a feature map — The output is a 2D map where high values indicate that the filter's pattern was detected at that location.
The convolution operation: a filter slides across the image, producing a feature map that indicates where the pattern was found.
Multiple filters produce multiple feature maps—each filter specializes in detecting a different pattern. The first layer might detect edges, the second layer detects shapes composed of edges, the third layer detects objects composed of shapes, and so on.
Key CNN Components
Convolutional Layer
The core layer that performs convolution. Defined by filter size (k×k), number of filters (output channels), stride (step size when sliding), and padding (adding pixels around the border to preserve spatial dimensions).
Activation Function (ReLU)
Introduces nonlinearity, allowing the network to learn complex patterns. ReLU is standard: f(x) = max(0, x). It's simple, fast, and avoids the vanishing gradient problem that plagued older activation functions like sigmoid.
Pooling Layer
Downsamples the feature map to reduce spatial dimensions and computational cost. The most common is max pooling, which takes the maximum value in each region. This makes the network more robust to small translations of the object.
Batch Normalization
Normalizes layer activations to stabilize training, allow higher learning rates, and reduce sensitivity to initialization. Almost universal in modern CNNs.
Dropout
Regularization technique that randomly deactivates a fraction of neurons during training to prevent overfitting. Less commonly used today with batch norm, but still useful for small datasets.
Residual Connections
Adds the input of a block directly to its output (y = F(x) + x). This solves the vanishing gradient problem in very deep networks, enabling architectures with hundreds of layers. Introduced in ResNet, now ubiquitous.
Hierarchical Feature Learning
The beauty of CNNs is that they automatically learn hierarchical features. Each layer builds on the previous layer's outputs:
- Layer 1 (low-level) — Detects simple patterns: edges, lines, corners, gradients.
- Layer 2 (mid-level) — Detects shapes: circles, rectangles, textures, parts.
- Layer 3+ (high-level) — Detects complex structures: objects, faces, whole scenes.
This hierarchy emerges automatically from data—you don't need to hand-engineer the features like in traditional computer vision. The network learns what features are important for the task.
| Layer Depth | Feature Type | Semantic Level |
|---|---|---|
| Shallow (1-3 layers) | Edges, gradients, corners | Pixel-level |
| Medium (4-10 layers) | Shapes, textures, parts | Local region |
| Deep (10+ layers) | Objects, categories, scenes | Whole image |
Popular CNN Architectures
- LeNet-5 — LeCun et al. (1998). The original CNN, designed for handwritten digit recognition. Small and simple, but established the pattern.
- AlexNet — Krizhevsky et al. (2012). The breakthrough that won ImageNet 2012 by a huge margin, launching the deep learning revolution.
- VGG — Simonyan & Zisserman (2014). Very deep (16-19 layers), homogeneous architecture with all 3×3 convolutions. Simple and effective, still widely used for feature extraction.
- Inception/GoogLeNet — Szegedy et al. (2014). Introduced inception modules with multiple parallel filter sizes, efficiently increasing depth and width.
- ResNet — He et al. (2015). Introduced residual connections, enabling very deep networks (50-152 layers). Still the most widely used baseline.
- EfficientNet — Tan & Le (2019). Scales depth, width, and resolution uniformly for better accuracy/efficiency trade-off.
- ConvNeXt — Liu et al. (2022). Modernizes the CNN design, matching transformer performance on many vision tasks.
Frequently Asked Questions
Same padding adds pixels around the input border so that the output feature map has the same spatial dimensions as the input. This preserves resolution through many layers. Valid padding means no padding—the output is smaller than the input, because the filter can only go where it fits entirely inside the input. Modern architectures often use same padding to maintain spatial size.
Stride is the number of pixels the filter slides at each step. A stride of 1 means the filter moves one pixel at a time, giving dense output. A stride of 2 means it moves two pixels each time, which halves the spatial dimensions and reduces computation. Stride greater than 1 is used for downsampling, replacing pooling in many modern architectures.
Yes. CNNs are widely used for 1D tasks like time series forecasting, audio processing, and text classification. 1D CNNs are often faster and simpler than RNNs for processing sequences, especially when patterns are local. They're also used for 3D medical imaging (3D CNNs) and video processing (3D convolutions over space-time).
CNNs use parameter sharing—the same filter is applied across the entire image. So a single 3×3 filter with 9 weights can detect its pattern anywhere in the image. In a dense network, every input pixel would need its own weight for every output neuron, leading to millions more parameters. Parameter sharing makes CNNs much smaller and less prone to overfitting.
Yes. CNNs are still widely used and are often preferred for deployment because they're faster, more efficient, and easier to optimize than transformers. Many modern hybrid architectures combine CNNs with attention—CNNs extract local features, attention models long-range interactions. CNNs remain the workhorse of computer vision, especially for resource-constrained applications like mobile devices.
Continue your AI learning journey
Next, explore Recurrent Neural Networks (RNNs)—the original architecture for sequential data.
Read Next: RNN →