For half a century, computing  power grew exponentially.

In the 1960s, most computers  were the size of a refrigerator, and they took a long time  to do pretty much anything. But by the 2010s, hundreds of millions of people   were walking around with personal  computers right in their pockets, and near infinite information  available at the swipe of a fingertip.

The change was unbelievably fast, it  was like nothing history had ever seen. And yet, it’s possible that  it won’t even hold a candle   to what’s happening now: Artificial Intelligence. Hi, I’m Kousha Navidar, and this  is Crash

Course: Futures of AI. [THEME MUSIC] Back in the ‘60s, that exponential  growth in computing power seemed   virtually impossible to most people. But one guy saw it coming: an  engineer named Gordon Moore. In 1965, he predicted that every two years, we’d find a way to get twice as many transistors  onto a computer chip as the previous one – making future computers capable of things  Moore and his colleagues couldn’t even imagine. And he turned out to be totally right, and his  prediction came to be known as Moore's Law.

And it held true for just about fifty years. Today, progress has shifted  from transistors and chips   to the new frontier of Artificial Intelligence. Artificial intelligence, or AI, is  really just a type of computer system– only, it’s a really advanced computer system  that tries to emulate intelligent behavior.

AI is kind of an umbrella term that includes some   pretty far out-there stuff  that doesn’t even exist yet, but also much more basic programs,   like the facial recognition software  that unlocks your smartphone... as long as you don’t grow out your beard… or shave your head… or ditch your glasses thanks  to the wonders of Lasik…. ask me how I know. AI is sometimes defined by its ability to  do stuff kind of like what humans can do,   like learning from errors, or making predictions  based on prior knowledge, or drawing hands. But it can also do stuff humans can’t, like  remembering every single thing it’s ever heard,   processing millions of points of data in seconds,   and making hyperrealistic fake videos of  Tom Cruise covering Dave Matthews Band.

But this capability is a pretty new thing,  and it’s taken decades to get to this point. Back in the 1950s, computers couldn’t even help  you with your homework or play a game of chess. At least, not until 1957.

That is the year IBM created  the Bernstein Chess Program. The Bernstein Chess Program  was an example of narrow AI. It’s a kind of artificial intelligence  that only does one thing– in this case,   play a very good game of chess.

Basically, it used algorithms  that mimicked human strategies – evaluating the board, picking out plausible moves,   and simulating them to figure out  which one would work the best. Eventually it got good enough to beat  an “inexperienced human opponent.” But it would have to be a  pretty patient human opponent – the thing took, like, eight  minutes to make a single move. Although, when I used to play chess with my dad,  sometimes we’d take, like, 20 minutes to make a   single move, and that makes the Bernstein Chess  Program seem like a certifiable speed racer.

We were thinking, like, really hard. In the decades to come, though, computers got  faster, and the chess bots got better and better. But they were still no match  for good human players.

In the ‘70s a Russian program named  Kaissa became the world champion of   computer chess engines, but it still  lost to human chess Master David Levy. In the ‘80s, Grandmaster Garry Kasparov, the  highest-ranked chess player in the world,   played against 32 bots at once and beat them all. Then came the 90s, and Deep Blue.

Building on their success with their  earlier chess bot, IBM created Deep Blue. It could evaluate 200 million  chess positions per second. And at that rate, Deep Blue could burn through   all those different simulations  to pick the best possible one.

And in 1997, Deep Blue beat Kasparov in a six game   match – it was the first time a machine  had beaten a human chess world champion. But Deep Blue, like all those other early chess  bots, was still an example of symbolic AI, it’s a kind of AI that relies  on the logic coded directly into   its hardware and makes moves by  hardcoded search and evaluation– meaning it was never actually  learning or improving,   it was just following the same human  instructions it was programmed with. And while that can be good, it wasn’t great.

But what if there was a system that operated a   little less like an instruction manual  and a little more like a human brain? In 2020, a chessbot called  Stockfish got something new – an efficiently updatable  neural network, or NNUE – and it totally changed the game. Well, the game was still chess  but– you know what I mean… Neural networks are a special  kind of computer architecture   that works kind of like the human brain, processing information with layers of  connected nodes that pass information   between them along weighted connection pathways.

Unlike symbolic AI, bots that run on any type of  neural networks can change based on experience. Like, if during practice a chess  move leads to a check mate in three,   that pathway will stay and the AI will  probably repeat that pattern in the future. But if it costs the network its queen, that  AI’s probably gonna revise its strategy.

That process is called deep learning,  and it depends on three things: One- the amount of data, or information,  available for AIs to learn from; Two- an algorithm to instruct  its learning process; and three, what your friend who’s, like, really   into AI calls compute – which includes  processing power, memory, and storage. And because Stockfish is an  efficiently updatable neural network,   it’s got even more space for that deep  learning than a regular neural network. While other AIs would have to reevaluate  the whole chess board every single move,   Stockfish can update incrementally,  keeping up with every move it plays.

And that frees up a lot of processing space,   and lets Stockfish think even  harder about its next move. Which turns out to be really helpful. Since 2020 it’s won basically every  computer chess championship there is.

Although it hasn’t played me yet. And not  to brag, but I have an account on Chess.com. Since the deep learning revolution of  the 2010s, neural networks have been   the foundation for basically all our AI up  until right now as I’m filming this episode.

And that includes the rise of general purpose AI. Unlike narrow AI, these systems can take  on lots of different tasks to achieve lots   of different goals, like writing books,  generating images, summarizing articles,   recognizing faces, driving cars,  and making Will Smith eat Spaghetti. General purpose AI isn’t only  able to engage in deep learning,   it’s able to do so really really fast.

That’s thanks to a tool called a transformer,  which lets certain kinds of AI process entire   sequences of data at once, rather than  reading word by word or number by number. That means general purpose AI is getting  really good at this stuff really quickly. So quickly that some people are  starting to wonder, how can we keep up?

Enter AI benchmarks and scaling laws. In AI, benchmarks are standardized tests that let  us measure how well different AI models perform. For narrow AI systems, benchmarks  can be pretty straightforward.

Like, chess engines have a lot of  the same benchmarks as human chess   players – getting certain FIDE ratings,  or beating other players or engines. But for more generalized AI, benchmarks have to   evaluate performance across  lots of different domains. Like, large language models, which are neural  networks that use transformers to process   language, are supposed to be able to do lots  of different things, so they might be tested   on whether they can answer academic questions,  and accurately summarize things they’ve read,   and write a ten minute romantic comedy  in the style of When Harry Met Sally.

EEE! AI agents like self-driving cars might have  to prove they can drive safely in the rain,   and not hit a pedestrian on the street. When AI models can consistently perform well  on a benchmark, it’s considered saturated.

But these days, benchmark saturation  is happening kind of freaky fast. In   a world where AI can identify faces, compose  music, drive cars, and take PhD level exams,   the question is less “what can AI  do?” and more “what can’t it do?” And it’s something we want to figure out before   we’re staring down the barrel of the  AI-pocalypse. The question is how.

Thankfully the things we already know about  AI, like past benchmarks, can help with that. If you look at how different AI systems perform on   different benchmarks, you  can see certain patterns. And one leaps out right away.

In general, AI neural networks that  are fed more data perform better. The more books a Large Language Model reads,  the better it’ll be at writing its own. The more videos of Tom Cruise  a video-generator watches,   the more lifelike its fake Tom Cruise will look.

He’s beautiful! Particularly if that’s paired with more  compute and other scaling increases. If we squint, we can actually put some  numbers on those patterns and come up   with formulas that describe exactly how  an AI’s performance seems to change with   different amounts of data, compute, or both.

We call those formulas scaling laws, and so far,   they help us predict what future AI with even  more data and more compute might be able to do. Full disclosure: the formulas themselves  kind of look like nonsense for everyone   but the most saturated AI nerd, but they  all mean the same thing – bigger is better. In the world of neural networks and deep learning,  being able to find patterns and learn from huge   quantities of data is gonna work better than  waiting for a human expert to code them in.

And the more compute these machines have,  the faster and the more powerful they’ll be. Now, of course, there are caveats. China’s DeepSeek is able to  compete with models like ChatGPT,   despite having a fraction of the compute.

Plus throwing all that compute at pretraining  isn’t showing the returns it once was. But still, scaling laws explain  why some of the highest-performing   generalized AI models belong to huge  companies like Microsoft and Google,   since they’ve got access to tons of  data and tons of processing power. And following that pattern into the future,  scaling laws can help us predict how capable   AI might become, gaining the power to transform  our economy, or hand us over to authoritarian   dictators, or make all our likenesses  sing covers of the Dave Matthews Band.

CRASH (course)... INTO ME! Yeah baby.

I’m old. Phew! Are you feeling those  tingles of excitement yet??

Imagine how AI feels! Just  kidding. It can’t feel.

Probably. Just like Moore’s Law foretold  unimaginably small and fast computers,   scaling laws see a big future for AI. As algorithms improve and data and compute keep  increasing, AI could deep-learn its way to some   pretty advanced stuff, like finding new treatments  for cancer or achieving superintelligence.

Moore’s Law held up for a good 50 years,   bringing us into this new era of  computing before slowing down. And there’s no guarantee that scaling  laws will last even that long,   thanks to potential bottlenecks down the road. But any day now, AI could turn some  new corner, and scaling laws might   not be enough to tell if it’s still harmlessly  marching pawns around, or coming for us all.

Next time we’ll look closer at what  AI could really do in our society,   other than lock us out of our phones if  (and when) emo bangs come back in style. Crash Course Futures of AI was produced in  partnership with the Future of Life Institute. This episode was filmed at  our studio in Indianapolis,   Indiana and was made with the  help of all these nice people.

If you want to help keep Crash  Course free for everyone,   forever, you can join our community on Patreon.