This is Essays on AI. I'm Tabrez Syed. Actually, no. I'm an AI voice reading his essays. But
you probably guessed. This one is called, When the Map Runs Out.
The opening section draws from Timothy B. Lee's excellent explanation of reinforcement
learning at his newsletter, Understanding AI. In 2009, a graduate student named Stefan
Ross at Carnegie Mellon University was trying to solve a problem that seemed straightforward
– teach a computer to play Super Tux Kart, an open-source racing game similar to Mario
Kart. His approach was elegantly simple. Ross would play the game himself, while his software
captured screenshots and recorded which buttons he pressed. Then he'd train a neural network
to predict which buttons Ross would push in any given situation. If the network could
learn to mimic Ross' gameplay perfectly, it should be able to play the game itself.
Just push the same buttons Ross would have pushed in the same situations.
The initial results looked promising. The AI-controlled car would zip around the track,
taking turns at reasonable speeds, staying roughly in the center of the road. For a few
seconds, it looked like Ross had cracked the code. Then something would go wrong. The car
would drift slightly to the left, or take a turn a fraction too wide, or hit a small
bump that pushed it off its ideal line. These weren't catastrophic errors, just tiny deviations
from the perfect gameplay Ross had demonstrated. But tiny deviations became bigger ones. The
car would drift further off course, then make increasingly erratic corrections. Within moments,
what had started as a small mistake would cascade into complete chaos. The animated
vehicle would careen off the track entirely, tumbling into the virtual abyss while Ross
watched in frustration. The problem wasn't that the neural network was stupid. It had learned
Ross' driving patterns remarkably well. The issue was more subtle and more fundamental. Ross was a
pretty good SuperTuxCart player, which meant his car spent most of its time near the center of the
road, driving at reasonable speeds, taking smooth turns. The neural network had learned to drive
perfectly, as long as everything went perfectly. But it had almost no training data showing what
to do when things went wrong. When the car drifted off course, it found itself in situations
that weren't well represented in its training data. And in those unfamiliar situations,
it made increasingly poor decisions. Ross had discovered something that would become crucial
to understanding artificial intelligence. Systems trained on past experience work beautifully
until they encounter situations that weren't in their training data. Then they don't just fail,
they fail catastrophically, with small errors compounding into complete breakdowns.
Ross, who now works on self-driving cars at Waymo, formerly Google's self-driving car project,
had identified a fundamental challenge that would prove relevant far beyond video games.
This wasn't just a quirk of Ross' particular approach. It's how most AI systems learn,
by studying massive amounts of data and finding patterns. Language models learn by processing
billions of text examples. Image recognition systems train on millions of labeled photos.
The fundamental approach is the same. Show the system enough examples, and it will learn to
recognize and respond to similar situations. But Ross' SuperTuxKart car revealed the Achilles'
heel of this approach. What happens when you encounter something that wasn't in the training
data? Ross' problem wasn't unique to video games. The same challenge was playing out in the real
world, where the stakes were considerably higher than a virtual car tumbling into a digital abyss.
Google had started its self-driving car project in 2009, the same year Ross was wrestling with
his racing game. In typical Google fashion, they took a Google approach to the training
data problem. They decided to collect massive amounts of it. For nearly a decade, from 2009
to 2018, Waymo vehicles drove millions of miles with human safety drivers behind the wheel,
meticulously recording every scenario, every edge case, every moment when human judgment was
required. The strategy worked. By the time Waymo launched its first commercial service in December
2018, their cars had logged over 10 million miles on public roads. The vehicles that had once
struggled with basic scenarios, like Ross' SuperTuxKart model drifting off course, more data had largely
solved the SuperTuxKart problem. Where Ross' model failed after encountering a single unfamiliar
situation, Waymo's cars could handle thousands of edge cases they'd seen during their extensive
training period. The compounding errors that plagued early AI systems became increasingly rare
as the training datasets grew more comprehensive. This approach represents how we've tackled most AI
challenges. Identify the gaps in training data, then systematically fill them. Can't recognize cats
in photos? Train on more cat images. Struggling with medical diagnoses? Feed the system more patient
records. Having trouble with language translation? Process more bilingual text pairs. Today, we've
taken this data collection strategy even further. AI systems can now manufacture their own training
data through synthetic data generation. Need more examples of rare medical conditions? AI can
generate realistic patient scenarios. Want to train a model on edge cases that rarely occur in real
driving? AI can create thousands of synthetic scenarios with unusual weather conditions,
unexpected obstacles, or complex traffic patterns. This ability to generate synthetic data means we
can fill gaps in training datasets more systematically than ever before. It's a
powerful strategy, and it works remarkably well for a specific type of problem. Statisticians
call this epistemic uncertainty. Uncertainty that exists because we don't know something, but we
could learn it if we gathered more information. Ross' SuperTuxKart model suffered from epistemic
uncertainty. It didn't know how to recover from mistakes because it had never seen examples of
recovery. Waymo's early struggles were also largely epistemic. Their cars couldn't handle construction
zones because they hadn't seen enough construction zones. But epistemic uncertainty is solvable. You
can collect more data, train on more examples, and gradually fill in the gaps. And here's where AI
systems will eventually surpass human intelligence. They can process vastly more data than any human
could ever experience. While a human driver might encounter a few dozen construction zones in their
lifetime, an AI system can learn from millions of construction zone scenarios captured by thousands
of vehicles. The epistemic uncertainty gap between humans and AI is temporary. Humans currently have
an advantage because we've been training on visual and spatial data since birth, accumulating decades
of experience navigating the world. But AI systems are rapidly catching up, and they'll eventually
have access to far more comprehensive datasets than any individual human could process. For
epistemic uncertainty, the solution is clear. More data, better algorithms, and time. Here's where the
story takes a turn. Our predictions of the future are reflections from our rearview mirror. Even with
over 20 million miles of real-world driving data, Waymo's systems still encounter situations that
weren't in their training. Not because of gaps in the dataset, but because the future draws from a
vast theoretical space of unrealized possibilities. Consider the microbursts that hit Austin in 2025.
A sudden downdraft brought down trees and scattered debris across roads in patterns no
training dataset had ever captured. Or the early days of the COVID-19 pandemic, when governments
implemented lockdown policies based on historical precedent from vastly different eras. In this
theoretical space, tree limbs can land on roads, locusts can swarm highways, microbursts can
scatter debris in patterns no algorithm has ever seen. These aren't just rare events, they're
genuinely unprecedented combinations that emerge from the complex interactions of countless variables.
These events represent aleatory uncertainty, from the Latin word for dice, the inherent randomness
and unpredictability in the world itself. Even our synthetic data generation can only create
variations of what we can imagine from past experience. We extrapolate from history, but the
future contains genuinely novel combinations that exist in theoretical possibility but have never
manifested in any dataset. You could collect data for eons, train on every scenario from the past,
and still encounter situations that have never existed before. This is because there is no data
about the future, not because we haven't collected enough data about the past, but because the future
contains genuinely novel combinations of factors that have never existed before. Here's what makes
this distinction important. Both human intelligence and artificial intelligence are sophisticated
pattern matching systems. We excel at recognizing situations we've encountered before and applying
lessons from past experience. But when faced with genuinely unprecedented events, both types of
intelligence hit the same fundamental wall. Consider how humans handled the COVID-19 pandemic.
We had data from previous pandemics, the 1918 flu outbreak, various smaller disease outbreaks,
historical accounts of quarantine measures. But the world of 2020 was fundamentally different
from 1918. Our interconnected global economy, modern healthcare systems, digital communication
networks, and social structures created a context that had never existed before. The result? Human
decision makers implemented policies that were essentially educated guesses. Lockdown strategies
varied wildly between countries and regions. Some measures worked, others didn't, and many had
unintended consequences that no amount of historical analysis could have predicted.
We were pattern matching from incomplete and often irrelevant historical data, just like an AI system
trying to navigate an unprecedented scenario. The same dynamic plays out in financial markets
during genuine crises, in political systems facing novel challenges, and in any domain where the
future presents combinations of factors that have never occurred before. Humans don't have some
magical ability to handle aleatory uncertainty better than AI systems. We're just as dependent
on pattern matching from past experience. This distinction matters because it challenges a
common assumption about human versus artificial intelligence. We often think of humans as
fundamentally superior to AI systems, possessing some ineffable quality that machines can never
replicate. But when we separate epistemic uncertainty from aleatory uncertainty, a
different picture emerges. For epistemic uncertainty, situations where more data and
better pattern recognition can solve the problem, AI systems will likely surpass human performance.
They can process more information, learn from more examples, and update their models more
systematically than biological intelligence. For aleatory uncertainty, genuinely unprecedented
events that emerge from novel combinations of factors, both human and artificial intelligence
face the same fundamental limitation. Neither can predict what has never happened before.
Neither can prepare for combinations of factors that have never existed.
But humans have developed something valuable over millennia of facing the unknown—mental
models and frameworks for navigating uncertainty itself. We've learned to reason by analogy,
to break down complex problems into manageable parts, to consider multiple scenarios simultaneously.
These aren't perfect solutions, but they're evolved responses to the fundamental challenge
of operating without complete information. AI systems will likely learn these same mental
models from the vast repository of human experience they train on. They'll absorb our
approaches to uncertainty, our problem-solving frameworks, our ways of thinking through
unprecedented situations. But they'll also inherit our biases, our blind spots, and our systematic
errors. After all, they're learning from our exhaust, both the wisdom and the mistakes embedded
in human decision-making. The future will always contain surprises that no amount of historical
data can anticipate. But this raises a deeper question about the nature of intelligence itself.
Is human decision-making reducible to a set of mental models and rules?
Or is there more to gut feeling than the total sum of carbon-based neurons buzzing in our heads?
Thanks for listening. The full essay, with links to Timothy B. Lee's explainer
and Stéphane Ross's paper, is on the episode page at mandolivia.com.
If you'd like more essays and audio, subscribe wherever you're listening.
Essays on AI is a Mandalivia production.