What is underfitting in machine learning?

Underfitting occurs when a machine learning model is too simple to capture the underlying patterns in the training data. The model fails to learn the relationships between inputs and outputs, resulting in poor performance on both training and test data. It's like trying to fit a straight line through data that follows a curved pattern—the model simply doesn't have enough capacity to represent the true relationship.

How do you detect underfitting?

Underfitting is detected when the model performs poorly on both training and validation data. Key signs include: (1) High training error (the model can't even fit the data it was trained on); (2) Training and validation errors are similar but both are high; (3) Learning curves that plateau at a high error level; (4) The model makes systematic errors—it consistently gets certain types of examples wrong because it can't capture the necessary patterns.

What causes underfitting?

Common causes include: (1) Model is too simple—too few parameters, layers, or capacity to capture the complexity of the data; (2) Insufficient training—not enough epochs or iterations; (3) Too much regularization—excessive L1/L2 penalties or dropout rates that constrain the model too much; (4) Poor feature selection—using features that don't contain enough information to solve the task; (5) Wrong model architecture—using a model type that's fundamentally unsuitable for the data.

How do you fix underfitting?

To fix underfitting: (1) Increase model complexity—add more layers, neurons, or parameters; (2) Train for more epochs; (3) Reduce regularization strength—lower L1/L2 penalties or dropout rates; (4) Add more relevant features or engineer better features; (5) Use a more powerful model architecture; (6) Reduce learning rate if the model is oscillating; (7) Ensure the data is properly preprocessed and normalized.

Is underfitting worse than overfitting?

Neither is strictly 'worse'—they're both failures of model fitting that need to be addressed. Overfitting is often considered more insidious because it gives misleadingly good training performance, making you think the model works when it doesn't. Underfitting is usually more obvious because both training and validation performance are poor. In practice, underfitting is often easier to diagnose and fix than overfitting.

How does underfitting relate to the bias-variance tradeoff?

Underfitting corresponds to high bias in the bias-variance tradeoff. A high-bias model makes overly simplistic assumptions about the data, leading to systematic errors that persist even with more training data. The model's predictions are consistently off-target (high bias) but don't vary much with different training sets (low variance). The fix is to increase model complexity to reduce bias, while carefully monitoring for the onset of overfitting.

📉Model Training

AI Glossary: Underfitting

Underfitting occurs when a machine learning model is too simple to capture the underlying patterns in the data. It's the opposite of overfitting—and often the first problem beginners encounter when building models.

📅Published: Jan 2024

⏱️12 min read

📊Level: Beginner-Intermediate

Table of Contents

What is Underfitting?
How to Detect Underfitting
Common Causes
How to Fix Underfitting
Underfitting vs Overfitting
Frequently Asked Questions

What is Underfitting?

Underfitting occurs when a machine learning model is too simple to capture the underlying patterns and relationships in the training data. The model fails to learn the structure of the data, resulting in poor performance on both the training data and any new data it encounters. It's like trying to describe a complex curve with a straight line—the line simply can't capture the shape of the data.

Underfitting is the opposite of overfitting. While an overfit model is too complex and memorizes noise, an underfit model is too simple and misses the signal entirely. Both result in poor generalization, but for different reasons.

Core Problem

The model lacks the capacity or flexibility to learn the true relationships between inputs and outputs.

Key Symptom

Poor performance on both training and validation data—the model can't even fit the data it was trained on.

Mathematical Cause

High bias: the model's assumptions are too restrictive to represent the true data-generating function.

Why Underfitting Matters

Underfitting is often the starting point for beginners. When you train your first model and it performs poorly, you're likely experiencing underfitting. The good news is that underfitting is usually easier to diagnose and fix than overfitting. The bad news is that the fixes (more complexity, more training) can push you into overfitting territory if you're not careful.

How to Detect Underfitting

Underfitting is relatively straightforward to detect because it affects both training and validation performance:

Underfitting is characterized by both training and validation loss plateauing at a high value. The model simply can't learn the data.

Key signs of underfitting:

High training error — The model performs poorly even on the data it was trained on
Similar training and validation error — Both are high, indicating the model hasn't learned much
Plateauing learning curves — Both curves flatten out quickly at a high error level
Systematic errors — The model consistently makes the same kinds of mistakes because it can't capture the necessary patterns

Common Causes of Underfitting

1. Model Too Simple

The most common cause. Using a linear model for non-linear data, a shallow neural network for complex patterns, or a model with too few parameters. The model simply doesn't have the capacity to represent the true relationship.

2. Insufficient Training

Not training for enough epochs, or using too low a learning rate so the model converges slowly. The model hasn't had enough time to learn the patterns.

3. Excessive Regularization

Too much L1/L2 regularization, too high dropout rates, or too aggressive early stopping. These are designed to prevent overfitting, but overdoing them prevents the model from learning anything.

4. Poor Feature Engineering

Using features that don't contain enough information to solve the problem. The model can't learn what isn't in the data.

5. Wrong Model Architecture

Using a model type that's fundamentally unsuitable for the data. For example, using a feedforward network for sequential data when an RNN or Transformer would be more appropriate.

How to Fix Underfitting

Increase Model Complexity

Add more layers to your neural network
Increase the number of neurons per layer
Use a more powerful model architecture (e.g., switch from linear regression to a neural network, or from a simple CNN to ResNet)
Add more features or use feature crosses to capture interactions

Train Longer

Increase the number of training epochs
Increase the learning rate if the model is converging too slowly
Use learning rate scheduling to help the model converge

Reduce Regularization

Lower the L1/L2 regularization strength
Reduce dropout rates
Disable or loosen early stopping criteria

Improve Data Quality

Add more relevant features
Engineer better features that capture the underlying patterns
Ensure data is properly normalized and preprocessed
Remove irrelevant features that add noise

Practical Approach

Start with the simplest model and gradually increase complexity. The goal is to find the point where training error drops to an acceptable level without the validation error starting to increase. This iterative approach—increase complexity, check for overfitting, then adjust—is the core workflow of machine learning model development.

Underfitting vs Overfitting: Quick Reference

Indicator	Underfitting	Good Fit	Overfitting
Training Error	High	Low	Very Low
Validation Error	High	Low	High
Bias	High	Low	Low
Variance	Low	Low	High
Fix	More complexity	—	More data/regularization

Frequently Asked Questions

Can a model underfit and overfit at the same time?

Not in the classical sense, but a model can have both issues across different parts of the data. For example, a model might underfit the overall trend (high bias) while overfitting to specific outliers (high variance). This is sometimes called "localized overfitting." The solution is typically to use a more flexible model with appropriate regularization.

How do I know if my model is underfitting or just needs more training?

If the training loss is still decreasing steadily (not plateauing), your model may just need more training time. If the training loss has plateaued at a high value, your model is underfitting and needs more capacity—not more training. A good diagnostic: try training for significantly more epochs. If the loss doesn't improve, you need a more complex model.

Is underfitting more common in deep learning or traditional ML?

Underfitting is more common in traditional machine learning (linear models, decision trees with limited depth) because these models have inherently limited capacity. In deep learning, overfitting is more common because neural networks have enormous capacity. However, underfitting can still occur in deep learning if the network is too shallow, the learning rate is too low, or the regularization is too strong.

How does data preprocessing affect underfitting?

Poor data preprocessing can directly cause underfitting. If data isn't properly normalized or scaled, the model may struggle to learn meaningful patterns. Missing values, unencoded categorical variables, or raw text without tokenization can all prevent the model from learning. Always ensure your data is properly cleaned, normalized, and encoded before training.

What's the relationship between underfitting and model capacity?

Model capacity is the model's ability to fit a wide variety of functions. Higher capacity = more flexibility. Underfitting occurs when model capacity is too low for the complexity of the data. The universal approximation theorem tells us that a sufficiently large neural network can approximate any function, but "sufficiently large" depends on the function's complexity. If your network is too small, it will underfit.

Continue your AI learning journey

Next, explore Gradient Descent—the fundamental optimization algorithm that makes machine learning possible.