What is Zero-Shot Learning?

Try talking to an LLM about something it’s never encountered. Can it figure it out? This is often viewed as the true test of intelligence. When a model uses inference and generalizations to learn without training data, this is called zero-shot learning.

Traditionally, AI models need giant datasets with labeled examples. Zero-shot learning expects a model to learn on the fly without training data. Zero-shot learning isn’t a replacement for standard training. Zero-shot is used to take pre-trained models to the next level. You can throw an AI into something it’s never seen and the AI will still perform well.

Follow along and learn the ins and outs of zero-shot learning.

Where is Zero-Shot Learning Used?

Have you ever needed someone to look at your work from a different perspective? This is where zero-shot comes in. With zero-shot learning, an AI model takes input, processes it, and gives you an opinion with zero training whatsoever. This yields promising results in all types of industry. When you ask AI to process the unknown and you get results, this is zero-shot learning in action.

• Healthcare: When diagnosing rare or unseen conditions, models use zero-shot to diagnose rare and never-before-seen medical conditions. In these situations, the data’s scarce or even non-existent.
• Pharmaceuticals: Models can analyze previously unseen data to predict the efficacy of compounds that don’t even exist yet.
• Natural Language Processing: Large Language Models (LLMs) are talking to people non-stop all day, everyday. When new slang emerges, or someone talks about their individual problems, models utilize zero-shot to make inferences and generalizations that regular humans would.
• Computer Vision and Robotics: It’s virtually impossible to train a model on every image they might encounter in the real world. Models recognize new images and figure out what to do with them. A self driving car stops at an intersection it’s never seen. A Roomba sees your furniture and avoids it.
• Entertainment and Creative Industries: Zero-shot allows models to spin up unique game characters. DALL-E and similar models generate unique pieces of art that no one has ever seen before.

Zero-shot learning is used all over the world already. The more AI adoption we see, the more zero-shot will continue to grow.

Zero-Shot vs. Other Paradigms

Have you ever worked a job with terrible management and no real training? If so, you’ve used zero-shot learning. Zero-shot learning is part of a larger paradigm called “n-shot” learning. N represents the number of labeled examples. Zero-shot learning implies zero previous training. Traditional machine learning uses giant datasets of labeled inputs.

• One-shot Learning: A model is trained on data with only one labeled sample per class.
• Few-shot Learning: The model gets trained on a small amount of labeled examples.
• Traditional Machine Learning: With traditional learning, a model is trained on enormous datasets with labeled examples. This is the opposite of zero-shot.
• Zero-shot Learning: The model sees things its never seen or been taught before. It’s just thrown into the mix and expected to figure things out and learn.

Zero-shot learning is comparable to on the fly, real world learning. Your boss throws you into the mix and just expects you to figure it out.

Conventional Zero-Shot Learning (ZSL)

Looking for a fountain of useless information to answer a single practical question? An LLM can do this for you. LLMs are classic examples of conventional ZSL. These models are pre-trained on more data than you or I can imagine. Think all of Wikipedia, whatever social media the company deems appropriate, thousands of books– and much, much more.

When you formally train an AI, it’s given a number of classes. If we wish to train an AI on horses, we can give it pictures and books about horses. When we do this, we create a class: “Horse”. The model then comes up with internal rules and generalizations for how it handles information related to its horse class.

Once a model has received adequate pre-training it can receive new data and create its own classes. If we give our horse trained model a picture of a Zebra, it can infer that a horse with stripes is a Zebra. Even though it hasn’t been trained on Zebras it is smart enough to create a new internal Zebra class and start making rules on how to handle the Zebra.

Because of the large pre-training requirements, ZSL comes at a pretty steep cost. Our model might understand Zebras, but we trained it on half the world’s to get there! Due to pre-training, ZSL is not very efficient. Next time you ask ChatGPT something pointless, think about what the machine had to go through just to answer your simple question.

Generalized Zero-Shot Learning (GZSL)

GZSL takes the concepts from ZSL and strips them down to be more efficient. With GZSL, we use chaos to simplify the learning process. Generalized zero-shot learning mixes multiple unknowns into the training process. The model then uses generalizations to create internal classes and rules from these unknowns.

Instead of pre-training our model on horses, why don’t we give it a single picture containing horses and a zebra? We can feed it a little text too: “The picture I’m giving you contains several horses and a zebra. A zebra is a horse with stripes.”

The model can use this brief description and single image to create both a horse class and a zebra class.

• Horse Class: The model will create a horse class and store data from the non striped horses in the picture.
• Zebra Class: It will create a zebra class using only our brief description and the striped horse from the image.

This drastically reduces the size of our training data. We now trained our model to recognize multiple horses and a zebra from a single image with some text. If our average picture is roughly 4kb, training on four horses would give us a minimum dataset of 16kb. When we add some chaos and include all the animals in one picture, our training dataset is only 4kb. With GZSL we provide leaner, high quality data for a quicker training process and a smaller model.

How Zero-Shot Learning Works

Let’s dissect the brain of our hypothetical LLM to see what’s really going on. We know that a model takes input data. Then, it creates new rules and classes on its own. Let’s get a better understanding of how it does this.

Labels

Pre-training is kind of like school. The model learns the basics of how to process information and “think.” When pre-training is finished, the model has learned all sorts of labeled classes and rules from us. During this stage, we provide the model with classes and labels. By the time it’s graduated, it knows how to learn. We don’t need to keep spoon feeding it the way we did early on.

Our model doesn’t wait for us to provide labels. Remember our horse and zebra example from earlier? The model creates the classes and labels them without our help. This saves us precious time on training while allowing the model to practice some level of autonomy.

Transfer Learning

Models make inferences. When our horse-trained model learns the zebra, it will transfer many (if not all) of the existing rules from the horse class to its new zebra class. Learning is transferred from one part of the model to the other.

Imagine you train a model to scrape hotel data from Google (you can learn to do this manually here). Then, you teach it to scrape Booking.com (you can learn how to scrape it manually here). When it scrapes Booking.com, it will use its knowledge of Google’s hotels to help it scrape the new ones from Booking.com.

Reasoning

At the heart of zero-shot learning is the ability to reason. When you were thrown into that terrible job with no training or experience, how did you survive? You likely figured it out using reasoning and common-sense. Imagine we give our AI toddler a “See and Say” dataset. We’d set up a class and rules for each class. Think: “Cow says moo!”. We’d create a cow class and write the rule that it says “moo.”

Once our AI has grown up, we don’t need to do this. Our model sees a picture of a chicken with bad captions like “cluck” or “feathers.” Using these simple hints, the pre-trained model figures out that this is a chicken. Then, it creates a chicken class with rules like “cluck” and “feathers.” When it reasons, our model uses common sense and street smarts to solve real-world problems (no matter how farm related they might be).

Pre-Trained Foundation Models

Our model actually starts out pretty similar to a newborn baby. It’s completely helpless and can’t do anything for itself. Pre-training is how our model grows up to think for itself. Before it can learn using zero-shot, the model needs to “learn how to learn.”

All humans do this when growing up. First, we learn how to feed. Then, we learn to eat solid foods and how to sit up. Around a year old, we learn to walk and talk. Instead of learning how to walk and talk and use the potty, AI models begin by learning basic things like math and language processing. Then, they learn how to ingest data.

Once a model knows how to process data, we then feed it all the data we can find. Then, we feed it more data! Eventually, it learns how to access its own internal classes. Once the model can read and write classes, it will begin make generalizations which evolve into reasoning over time. With effective pre-training, models can then use zero-shot to learn independently.

Zero-Shot Learning Methods

From the outside, zero-shot learning looks like magic. But like any magic trick, this is all an illusion. AI models rely on a very particular set of skills. Raw data gets taken and converted into real answers that we can read or listen to. Let’s see what goes on before the rabbit gets pulled out of the hat.

Attributes

Our model deciphers different animals using traits, or attributes. Attributes are just as simple as they sound. When our model looks at a picture with a variety of animals, it uses their traits to figure out what’s what.

• Horse: Neigh, 4 legs, hooves.
• Chicken: Cluck, 2 legs, wings.
• Cow: Moo, 4 legs, hooves.

Attributes allow the machine to make educated guesses– just the way a human would.

Embeddings

Machines don’t see data the way you and I do. Instead, they hold numerical lists of data called matrices. Let’s pretend we want to reperesent our horse, chicken and cow attributes as numbers.

Each row from this table can be represented as a list.

• Horse: [Neigh, 4, Hooves]
• Chickens: [Cluck, 2, Wings]
• Cow: [Moo, 4, Hooves]

However, the lists above aren’t yet machine readable. Machines excel when understanding numbers. For sound, we’ll encode 1,2 and 3 to represent “neigh”, “cluck” and “moo.” Since we only have two features to worry about (hooves and wings), 1 will represent hooves and 2 will represent wings.

Here is how our model might see this information.

• Horse: [1, 4, 1]
• Chicken: [2, 2, 2]
• Cow: [3, 4, 1]

By embedding our data using numbers, AI models can efficiently process it to discover relationships and rules. This is the foundation of its generalization and reasoning abilities. Learn more about embeddings in ML.

Generative

Models invent new classes out of thin air. Generative methods allow the model to draw a conclusion by seeing relationships in the embedded attributes. When our model identifies the zebra without training, this is generative. The model saw that it was a striped horse. It then generated the conclusion that a striped horse is a zebra.

If you’re scraping hotel data but you don’t have a rating, an AI model could generate one based on the information provided. AI models use their imagination to generate new data. The model might decide that if a room has a big bed and a hot tub, it’s 5 stars. This is incredibly powerful, but can also lead to hallucinations.

When using generative methods, it’s important to be careful. It’s great if a model can assign hotel ratings. If you ask your model, “What’s the last thing Confucius wrote in 2025?” Confucius has been dead for thousands of years, however AI models will rarely tell you “I don’t know”. There’s a possibility you’ll get a response like the one below.

The output above is actually more Taoist than Confucian. Modern AIs have pretty strong safeguards against hallucination. I actually had to give ChatGPT permission to hallucinate it! If you ever want to experiment with a model’s imagination, tell it to go “completely unhinged” and watch it descend into total madness.

Contrastive Learning

How does an AI tell the difference between a cat and a dog without training? The answer lies in contrastive learning. Below, we break dog and cat into attributes like we did with other animals earlier.

• Dog: Woof, 4 legs, paws
• Cat: Meow, 4 legs, paws

The animals above are almost identical but not quite. These animals make contrasting sounds. The dog says “woof” while the cat says “meow.” The model converts this data to numbers. Then, it quickly finds the difference in the two animals. Using zero shot, AI models quickly filter through their embeddings for contrasting information.

Prompt Engineering

Prompt engineering is the art of talking to the AI. When you know what to say, you can get the model to generate the exact output you want. In a previous article about web scraping with Claude I use the following prompt.

"""Hello, please parse this chunk of the HTML page and convert it to JSON.  Make sure to strip newlines, remove escape characters, and whitespace:  {response.text}"""

The prompt is clear and the model knows exactly what I want it to do. It spits back a list of quotes from the page. Here’s just a snippet of it.

"quotes": [
    {
      "text": "The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.",
      "author": "Albert Einstein",
      "tags": ["change", "deep-thoughts", "thinking", "world"]
    },
    {
      "text": "It is our choices, Harry, that show what we truly are, far more than our abilities.",
      "author": "J.K. Rowling",
      "tags": ["abilities", "choices"]
    },

If I hadn’t specified the data format, it would likely give me all the output in plaintext format. Plaintext is fine for human readability, but if you’re writing a program, JSON is far better to work with. The model gets me what I want because I wrote the prompt to spit out exactly what I want. Prompt engineering reigns in the generative output to be factual and properly formatted.

Challenges and Limitations of Zero-Shot Learning

Zero-shot learning comes at a price. As we touched on earlier, zero-shot leaves room for hallucinations. AI models don’t like to say “I don’t know” or admit when they’re wrong.

To safeguard against hallucination, we rely heavily on pre-training. Training data is expensive and often messy. If you’re harvesting the data yourself, you’ll need to create an ETL Pipeline. ETL stands for “Extract, Transfer, Load.” At scale, ETL is no walk in the park. You need to scrape terabyte upon terabyte of relevant data. Next, you need to clean and format it (transfer). Finally, you load it into the model. Learn more about pitfalls in AI.

Here at Bright Data, we offer clean, pre-made datasets. These can take your pre-training to the next level and save you hours (even days) of extracting, cleaning and formatting. Take a look at our structured datasets.

Conclusion

Zero-shot learning is revolutionizing AI by enabling models to process new information without prior training. As AI adoption grows, this technique will become even more essential across industries.

Ready to power your AI with high-quality data? Start your free trial with Bright Data and access top-tier datasets today!