AI Daddy › Prompting & Context
Few-Shot and In-Context Learning (ICL) · Prompting & Context
In-Context Learning (ICL) is the ability of an LLM to learn a new task simply by seeing examples in the prompt, without any weight updates. Maximizing ICL…
Few-Shot and In-Context Learning (ICL)
In-Context Learning (ICL) is the ability of an LLM to learn a new task simply by seeing examples in the prompt, without any weight updates. Maximizing ICL efficiency is a key lever for prompt stability.
Table of Contents
The Anatomy of a Few-Shot Example
A high-quality example consists of three parts:
- Input: A realistic sample of potential user data.
- Reasoning (Optional): A short explanation of why the output is what it is.
- Output: The "Gold Standar" result.
User: "The weather is okay, but the flight was late."
Reasoning: The user is neutral about the weather but negative about the service.
Sentiment: Mixed
How many examples?
| Model Size | Sweet Spot | Scaling Behavior |
|---|
| Small (8B) | 5 - 10 | Gains continue until ~20 examples. |
| Medium (70B) | 3 - 5 | Plateaus early; more examples increase latency. |
| Frontier (405B) | 1 - 2 | Highly capable; "Instruction Following" usually suffices. |
Rule of thumb: If you need more than 20 examples to get a stable output, your task is likely too complex for the model, or you should consider Fine-tuning.
Dynamic Example Selection
In production RAG or Classification, don't use the same static examples for every user.
The Dynamic Pattern:
- User provides a query.
- Search a "Vector DB of Gold Examples" for the 3 most semantically similar cases.
- Inject those 3 specific cases into the prompt.
Result: Drastically higher accuracy because the model sees "local" patterns relevant to the current user.
The Importance of Labelling Nuance
Frontier models are sensitive to Distribution Bias in examples.
- If you provide 5 "Positive" examples and 1 "Negative," the model will bias toward "Positive."
- Fix: Always use Label Balancing. Ensure your few-shot examples roughly mirror the expected output distribution or are perfectly balanced (1:1).
Advanced ICL: Analogy and "Few-Shot CoT"
Analogy Prompting: Instead of saying "Do X," provide an analogy.
"Translate this code like a translator would move a poem from French to English—preserving the soul (logic) but changing the syntax."
Few-Shot CoT: Providing 2 examples where the reasoning is explicit. This "primes" the model's attention to focus on logic rather than just mimicking the output string.
Interview Questions
Q: Why not just provide all 50 examples we have in the prompt?
Strong answer:
There are three primary reasons:
- Context Window Latency: Every example adds tokens, increasing the "Prefill" time and the cost per request.
- Attention Dilution: Even with 128k context, models can "lose" specific constraints if buried under too much irrelevant data (the "lost-in-the-middle" effect).
- Overfitting: Providing too many narrow examples can cause the model to mimic the format of the examples too strictly, losing its general capability to handle edge cases outside that set.
Q: What is "Label Bias" in In-Context Learning?
Strong answer:
Label bias occurs when the model predicts a specific label more frequently simply because it appeared more often in the few-shot examples or because it appeared at the end of the list. The standard mitigations are:
- Shuffling the order of examples for different requests.
- Ensuring an equal number of positive/negative/neutral samples.
- Using "Permutation Testing" during prompt development to ensure the model responds to the content, not the order.
References
- Brown et al. "Language Models are Few-Shot Learners" (2020)
- Min et al. "Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?" (2022)
Next: Chain-of-Thought