How does few-shot learning differ from zero-shot and supervised learning?

Zero-shot learning uses zero examples of the target class (relying on semantic descriptions). Few-shot learning uses 1-10 examples per class. Full supervised learning uses hundreds to millions of examples. Few-shot sits in the middle—it requires some data but dramatically less than traditional supervised learning.

What is N-way K-shot classification?

N-way K-shot classification is the standard evaluation framework for few-shot learning. N is the number of classes to distinguish between, and K is the number of examples per class. The most common benchmarks are 5-way 1-shot (distinguish 5 classes with 1 example each) and 5-way 5-shot (5 classes with 5 examples each).

What is in-context learning in LLMs?

In-context learning (ICL) is the few-shot learning paradigm used by large language models. Instead of updating model weights through training, you provide a few examples directly in the prompt. The LLM uses these examples to infer the task pattern and apply it to new inputs—all without any gradient updates. This is a form of learning through attention rather than through weight updates.

What are the main approaches to few-shot learning?

Main approaches include: (1) Meta-learning (learning to learn) where the model is trained across many few-shot tasks to develop a general ability to adapt quickly; (2) Metric learning methods like Prototypical Networks and Matching Networks that learn an embedding space where similar classes cluster together; (3) Data augmentation and hallucination techniques that generate additional training examples; and (4) In-context learning in LLMs that uses examples in the prompt.

Why is few-shot learning important?

Few-shot learning is important because in many real-world scenarios, collecting large amounts of labeled data is expensive, time-consuming, or impossible. Examples include rare medical conditions, endangered species identification, niche industrial applications, and personalized AI systems. Few-shot learning makes AI practical in data-scarce domains.

📚Learning Paradigm

AI Glossary: Few-shot Learning

Q: What is few-shot learning?

Few-shot learning is a machine learning paradigm where a model learns to perform a new task using only a small number of training examples (typically 1-10 per class). The goal is to build models that can generalize from limited data, similar to how humans can learn new concepts from just a few examples.

Few-shot learning is the ability of AI models to learn new tasks from only a handful of examples—typically 1 to 10 per class. It bridges the gap between zero-shot generalization and data-hungry supervised learning.

📅Published: Jan 2024

⏱️13 min read

📊Level: Intermediate

Table of Contents

What is Few-shot Learning?
N-way K-shot Classification
Meta-learning: Learning to Learn
Metric-based Methods
In-context Learning in LLMs
Applications and Use Cases
Frequently Asked Questions

What is Few-shot Learning?

Few-shot learning (FSL) is a machine learning paradigm where a model is trained to recognize new concepts or perform new tasks using only a small number of labeled examples. While traditional deep learning models often require thousands or millions of labeled examples per class, few-shot learning models aim to achieve good performance with as few as 1-10 examples per class.

The motivation behind few-shot learning is deeply human: we don't need to see thousands of examples of a new object to recognize it. Show a child one picture of a giraffe and they can recognize giraffes in the future. Few-shot learning tries to bring this same efficiency to AI systems.

Core Challenge

How to generalize from extremely limited data without overfitting to the few available examples.

Human Inspiration

Humans can learn new concepts from one or a few examples by leveraging prior knowledge and experience.

Key Insight

The solution lies in leveraging prior knowledge from related tasks to make the most of limited new data.

Why Few-shot Learning Matters

In many real-world scenarios, collecting large labeled datasets is impractical:

Medical imaging — Rare diseases have few documented cases
Endangered species — Limited photographic evidence exists
Industrial defects — Rare manufacturing defects are hard to capture
Personalization — Each user's preferences are unique with limited data
Rapid adaptation — New product categories emerge faster than data can be collected

Key Distinction

Few-shot learning is not about training a model with small data from scratch. It's about leveraging prior knowledge (from pre-training or meta-training) so that only a few examples are needed to adapt to a new task. The "few shots" are used for adaptation, not for the entire learning process.

N-way K-shot Classification

The standard framework for evaluating few-shot learning is N-way K-shot classification. In each episode:

Randomly sample N classes from the dataset
For each class, provide K labeled examples (the support set)
Ask the model to classify new query examples from those same N classes

The most common benchmarks are 5-way 1-shot (distinguish 5 classes with 1 example each) and 5-way 5-shot (5 classes with 5 examples each).

This episodic training setup is crucial: it mimics the test-time scenario during training, teaching the model to quickly adapt to new tasks. The model sees thousands of different 5-way tasks during training, each with different classes, so it learns a general "how to adapt" ability rather than memorizing specific classes.

An example of a 5-way 1-shot episode: the model sees 1 example of each of 5 classes, then must classify new query images.

Meta-learning: Learning to Learn

Meta-learning, or "learning to learn," is the dominant paradigm for few-shot learning. Instead of learning to solve a specific task, the model learns to quickly adapt to any new task.

How Meta-learning Works

Meta-learning operates on two levels:

Inner loop (task-level) — The model adapts to a specific task using the few available examples. This is fast adaptation.
Outer loop (meta-level) — The model's learning algorithm or initialization is updated based on how well it adapted across many tasks. This is slower, meta-level learning.

MAML (Model-Agnostic Meta-Learning)

MAML is one of the most influential meta-learning algorithms. Its key insight: learn an initialization of model parameters that can be quickly fine-tuned to any new task with just a few gradient steps. The model is trained so that after one or a few gradient updates on a task's support set, it performs well on that task's query set.

MAML in a Nutshell

1. Sample a task (e.g., 5-way 1-shot classification). 2. Take one gradient step on the support set. 3. Evaluate on the query set. 4. Update the original parameters (not the task-adapted ones) to minimize the query set loss. This teaches the model to have an initialization that's "one gradient step away" from good performance on any task.

Metric-based Methods

Metric-based approaches learn an embedding space where similar examples cluster together, making classification a simple nearest-neighbor lookup.

Prototypical Networks

For each class, compute a prototype—the mean of the support example embeddings. To classify a query, find the nearest prototype in the embedding space. This is simple, elegant, and surprisingly effective.

Matching Networks

Matching networks use attention over the support set to compute a weighted nearest-neighbor classifier. The query is compared to every support example via an attention mechanism, and the class label is computed as a weighted combination of support labels.

Relation Networks

Instead of using a fixed distance metric (like Euclidean or cosine), Relation Networks learn a deep neural network to compute the "relation score" between a query and each support example. This learnable metric can capture more complex similarity patterns.

Prototypical Networks

Compute class prototypes as the mean of support embeddings. Classify by nearest prototype. Simple, fast, and effective for many tasks.

Matching Networks

Use attention over the entire support set. The query attends to support examples, and the attention-weighted average of labels is the prediction.

Relation Networks

Learn a neural similarity function instead of using a fixed distance metric. Can capture more complex, non-linear relationships between examples.

In-context Learning in LLMs

In the era of large language models, few-shot learning has taken on a new form: in-context learning (ICL). Instead of updating model weights through gradient descent, you simply provide a few examples in the prompt. The LLM uses these examples to infer the task pattern and apply it to the new input.

For example, to perform few-shot sentiment analysis:

Review: "This product is amazing!" → Positive
Review: "Terrible experience, would not recommend." → Negative
Review: "It's okay, nothing special." → Neutral
Review: "Absolutely love it, best purchase ever!" →

The model sees the three examples and then correctly predicts "Positive" for the final review—all without any weight updates. The "learning" happens through the attention mechanism during the forward pass.

Why ICL Works

Pattern matching — Transformers can identify patterns in the input sequence and continue them. The few-shot examples form a pattern that the model follows.
Implicit Bayesian inference — ICL can be viewed as the model performing implicit Bayesian inference over latent task concepts, using the examples to narrow down which task it should perform.
Gradient descent equivalence — Some research suggests that transformer attention can implement forms of gradient descent internally, effectively "learning" from the examples during the forward pass.

Applications and Use Cases

Domain	Application	Why Few-shot is Needed
Healthcare	Rare disease diagnosis	Limited patient cases for rare conditions
Robotics	Learning new manipulation tasks	Each new object requires quick adaptation
NLP	Custom text classification	Users need personalized classifiers with few examples
Computer Vision	Fine-grained species recognition	Rare species have few photographic records
Security	Face verification	Only a few photos per person for authentication
Recommendation	Cold-start recommendations	New users/items have minimal interaction history

Frequently Asked Questions

What is the difference between few-shot and one-shot learning?

One-shot learning is a special case of few-shot learning where K=1—the model gets exactly one example per class. Few-shot learning is the broader term that includes one-shot (K=1) and any small K (typically 2-10). One-shot is the hardest setting because the model has minimal information to work with.

How does few-shot learning avoid overfitting?

Few-shot learning avoids overfitting through several mechanisms: (1) Meta-learning across many tasks teaches the model to extract generalizable features; (2) Metric-based methods use embedding spaces where generalization is natural; (3) The episodic training setup prevents memorization of specific classes; (4) In-context learning doesn't update weights at all, so there's no risk of overfitting to the few examples.

What is the relationship between few-shot learning and transfer learning?

Transfer learning is a broader concept: using knowledge from one task to help with another. Few-shot learning can be seen as an extreme form of transfer learning where the target task has very few examples. However, few-shot learning often uses meta-learning rather than standard fine-tuning, as standard fine-tuning would overfit with only a few examples.

How many examples are needed for "few-shot" learning?

There's no strict cutoff, but in practice, few-shot learning typically refers to K=1 to K=10 examples per class. The most common benchmarks use K=1 (one-shot) and K=5 (five-shot). Beyond K=20-30, it's generally considered traditional supervised learning with a small dataset rather than few-shot learning.

Can few-shot learning work without pre-training?

In practice, no. Few-shot learning almost always relies on some form of prior knowledge, whether from pre-training on large datasets, meta-training on related tasks, or the broad knowledge encoded in large language models. Without this prior knowledge, it's impossible to generalize from just a few examples. The "few shots" are for adaptation, not for the entire learning process.

Continue your AI learning journey

Next, explore Chain of Thought—a prompting technique that dramatically improves reasoning in AI models.