Small vs. Large AI Models Trade-Offs & Use Cases Explained

Metadata

• Author: IBM Technology
• Full Title: Small vs. Large AI Models: Trade-Offs & Use Cases Explained
• Type: #video
• Tags: #ai
• URL: https://www.youtube.com/watch?v=0Wwn5IEqFcg

Highlights

• And in broad strokes, extra parameters buy extra capability. Larger models have more room to memorize more facts and support more languages and carry out more intricate chains of reasoning. (View Highlight)
• They demand exponentially more compute, energy, and memory, both to train them in the first place and then to run them in production. (View Highlight)
• We measure progress in language model capability with benchmarks. One of the most enduring benchmarks is the MMLU, which stands for massive multitask language understanding. Now the MMLU contains more than 15,000 multiple choice questions across all sorts of domains and sub-subjects, like math, history, law, and medicine. Anyone taking the test needs to combine both factual recall with problem solving across many fields. (View Highlight)
• Now, if you took the MMLU and you were just guessing at random, you would score around 25% on the test. But if you weren’t guessing at random, if you’re just kind of a regular Joe, just a regular human, and you took the test, you might score somewhere around 35%. It’s a pretty hard test. (View Highlight)
• Well, a domain expert would score far higher, something like around 90% on questions that are within their specialty. So that’s humans. (View Highlight)
• Well, when GPT-3 came out in 2020, This is a 175 billion parameter model.
  It posted a score on the MMLU view of 44%. (View Highlight)
• What about today’s models? Well, if we take a look at today’s frontier models, kind of the best models we have, they can score in the high 80s, maybe 88% on the test. (View Highlight)
• and then in March of 2024, Qwen 1.5 MOE became the first model with fewer than 3 billion active parameters to clear 60%. In other words, month by month, we are learning to squeeze Competent generalist behavior is being distilled into smaller and smaller footprints. (View Highlight)
• One of the first really comes down to broad spectrum code generation. A small model can master a handful of programming languages. But a frontier model has room for dozens of ecosystems and can reason across multi-file projects, unfamiliar APIs, and weird edge cases. (View Highlight)
• Another good example is when you have document-heavy work that you need to process. We might need to ingest a very large contract, a medical guideline, and a technical standard. A large model’s longer context window means it can keep more of the source text in mind, reducing hallucinations and improving citation quality. (View Highlight)
• The same scale advantage appears in high-fidelity multilingual translation as well, where we're going from one language to another. The extra parameters that the network carves out create richer subspaces for each language, capturing idioms and nuance that smaller models might gloss over. (View Highlight)
• Also, when it just comes down to everyday summarization, that's another sweet spot. In a news summarization study, Mistral 7B instruct achieved ROGUE and BERT score metrics that were statistically indistinguishable from a much larger model, GPT 3.5 turbo. And that's despite the model running 30 times cheaper and faster. (View Highlight)
• And another good use case comes down to enterprise chatbots. So with these, a business can fine-tune a seven or a 13 billion parameter model on its own manuals, and it can reach near expert accuracy. And IBM found that the granite 13B family matched the performance of models that were five times larger on typical enterprise Q&A tasks. (View Highlight)