The Greatest Threat Humans Just Can’t Stomach with Yuval Noah Harari

The Rest Is Politics

Rory Stewart interviews historian Yuval Noah Harari to discuss the existential threats posed by the emergence of 'alien' intelligence. They

Key takeaways

• Yuval Noah Harability argues that unlike the Industrial Revolution, AI is an 'agent' rather than a 'tool,' meaning it can make decisions and invent new ideas independently.
• The Aztec Empire's failure to defeat Cortés was rooted in an inability to comprehend alien technology and biological threats, a parallel Harari draws to modern AI.

Main topics

• Existential Risk
• Artificial Intelligence

Notable quotes

AI is not artificial in this sense. It's really an alien intelligence.

Conclusion

The episode serves as a warning that political leaders must move beyon

Transcript preview

Speaker 1 (0:08) Welcome back to Thinking the Unthinkable with me, Rory Stewart. Every week on this podcast, we talk about things that absolutely fascinate me. But they're often the stories of the week. An election in Germany, an earthquake, a flood in Nepal, something that's happening in the British Parliament, new prime minister. But there are other things that we don't talk about as much, which may be far, far more important and are much less visible. the things that could actually completely upend our societies, things that could destroy us. And in this series, I'm going to try to go into the heart of some of those things. Some of them are things we used to talk about, but we don't talk about very much anymore. Nuclear war. Some of them are things that we talk about, but we barely understand yet. AI. And some of it Speaker 2 (0:57) are Speaker 1 (0:58) things that we hear about. terrorism, ISIS, totalitarianism, but which we don't really understand the texture of, where I'm going to be sitting down with people who've actually lived on the ground in Mosul under ISIS, in Membij, northeast Syria, as ISIS is coming up, talking about human nature, about totalitarianism, about extremism. And in all of this, I hope we're getting to the real core of what really, in the end, matters about politics, the really biggest things. The things that our minds struggle to predict, that our political systems, our newspapers struggle to really talk about, that voters aren't incentivized to raise. Things which are unthinkable. And the question we'll be raising is, how do you think about the unthinkable? Speaker 1 (1:52) This episode is brought to you by NordVPN. And I must thank NordVPN because Speaker 4 (1:57) on my Grand European tour with Fiona, I was away from the Hawthorns for West Bromwich against Burnley, but Speaker 1 (2:05) I managed to watch it. There you go. I actually had a NordVPN thank you. I was in the new Starbucks here in Creef. And of course, the thing flashed up saying this is an unsecured public network. And I was able to go to my VPN and it was a little reminder. of something we often forget when we're in all those cafes and things, which is actually genuinely people can do all kinds of weird cyber attacks and spying if you don't have a virtual private network when you're sitting in that Starbucks. Speaker 4 (2:31) Well, I think you're not really sort of proud of me that I managed to do Speaker 1 (2:34) this, or should I say Fiona managed to do this? I'm incredibly proud of you. That's a tech transformation. So to get the best discount off your NordVPN plan, go to nordvpn.com slash restlesspolitics. That's... nordvpn.com slash restispolitics. And the link is in the episode description. Speaker 1 (2:55) My guest today is Yuval Noah Harari, one of the most famous public intellectuals alive today, the author of Sapiens. Today, we're going to be asking perhaps the most fundamental question in existential risk. What happens if with AI, we are building an alien intelligence that turns out to basically create a new form of empire, an empire in which AI is the emperor and we are the vassals? What's that going to mean for our economy? What's that going to mean for our politics? Ultimately, what's it going to mean for our minds and our sense of ourselves? And how can we learn from past civilizations on what happens when this incredible rupture occurs? Yuval Noah Harari. Speaker 1 (3:39) Can we sort of start with the general idea of this thinking the unthinkable? I don't know whether we want to call them existential risks, but what this category of things are and whether you recognize it as a category? You Speaker 2 (3:49) know, the purpose of politics, I think, is all about is making choices, is defining challenges and problems that face society, giving the public two, three, five options. And then we have a conversation about it and we need to choose what is the right policy. This is when politics functions well. But for this to happen, the politicians need to focus on the most important problems that face society. And every now and then something happens in history which is just outside. the normal political discourse, that we don't think about it. We don't even have words to describe what it is, but it could be the most important next thing. And then the job of politicians is to think the unthinkable. You know, the range of examples from human history is quite wide. I mean, if you're in the Aztec empire and in the early 16th century, and you just heard the reports, about this very strange thing that happened on the coastline. These huge hills appeared over the ocean and out of them came strange men or ghosts or demons, we are not sure what they are, and they have these huge creatures that can run and they have sticks that can launch lightning and thunder and this is the outside. the normal course of Aztec politics, but this is the most important thing you now need to confront. And the Aztecs did a very bad job confronting that. Speaker 1 (5:30) They Speaker 2 (5:30) couldn't Speaker 1 (5:31) really understand what the hell is this and what should we do about it. So it's a lovely place to start. So obviously, Yuval, you're describing the arrivals of the conquistador, the Spanish invaders into Latin America. And the astonishing thing there is that it's a few hundred people arriving. into an empire of many millions. And despite the fact that they have horses and simple muskets… That couldn't fit anything. I mean, it was just for show. In practice, it would have been… very, very easy for a million people to over- Yes, I mean, Speaker 2 (6:01) the tragedy in a way of the Aztec Empire, they could have squashed Cortes very, very easily. But they didn't understand what they were facing. They made a lot of mistakes. And these few hundred people, they destroyed the Aztec Empire. And of course, with the help of the microbes they brought. Annoyingly, they brought all these old world diseases. They decimated the entire population of Central America and beyond. According to some estimates, about up to 90 % of the people died. entire cultures were wiped out and they never recovered from it. There are, of course, people today in America who are the descendants of the Aztecs and the other people of the empire, but they never recovered from it. And really, as you said, it was just a few hundred people. They could have squashed them easily, but they just didn't understand Speaker 1 (6:55) what they were facing. So I think this is an interesting route through. as a medieval historian, you then become very interested in these moments of discontinuity. So this is not exactly like the question of why William the Conqueror won at the Battle of Hastings against King Harold, which is a more ordinary engagement, which would have been comprehensible to an Anglo-Saxon or a Norman. Somebody won a battle, they lost a battle. And you then begin to take an interest in these incredible historical moments like that, the arrival of Cortes in Latin America, and the fact that what's hitting you is not just an alien people, it's also an alien technology and something even more unexpected, an alien pandemic. Now fast forward, in your later writing, you begin to focus on things like nanotechnology, robotics. artificial intelligence, pandemics as well. What begins to draw you into those sort of issues and categories away from medieval warfare? I Speaker 2 (7:51) think it's not away from medieval warfare or away from the Middle Ages. I mean, the Middle Ages are more relevant than ever because modernity is coming to an end. And then the question, what is there other than modernity, becomes very relevant. And the obvious place to go with is the Middle Ages. But of course, we are not going back. to the Middle Ages. We are just exiting modernity into something else, which we don't know what it is. But some of the challenges we are now facing remind me of what the Aztec empires faced unsuccessfully back then. And when we talk, and we'll talk a lot about AI, so AI is this alien invasion. I mean, for me, AI is the acronym is not artificial intelligence. There is nothing artificial anymore. about AI. Artificial brings to mind something that we create and is under our control. But AI is not artificial in this sense. It's really an alien intelligence. It's not alien coming from outer space, of course, but it's totally alien to our way of thinking and understanding the world. It's an inorganic intelligence. We have never in history met anything like it. And the job of politicians today is to do a bit better than Montezuma and understand what is happening, take it seriously, explain it to the public and Speaker 1 (9:17) present choices. What do we do? To make the case for Montezuma, which is an unfashionable thing to do. The problem that Montezuma faces is all the structures of his government. and all his historical experience and all his technology and tactics, which have made him the head of the Aztec Empire, it's no small thing. I mean, he's obviously a very nimble, able politician in general, are suddenly confronting something in those boats which has never been seen before, which are alien. And so, part of, I think, what this series, I hope, is going to be about is how we think about the alien. Now, we'll have a chance in our second conversation to get deeper. into the question of the alien intelligence that's AI. But just before we focus on that, I want to circle around some of the other things that might seem a bit alien or difficult for us to understand. Let's maybe have a look at pandemics. That's something that you've thought about and written about. And of course, it's something we went through in a smaller way with COVID. Speaker 3 (10:16) And Speaker 1 (10:17) again, where I felt I was in government coming through the end of 2019 and very involved in this debate going to the beginning of 2020 was an interesting example of ways governments can and cannot respond to something which is really strange. Something comes out of a viral lab, it seems, in Wuhan or in a fish market in Wuhan, however it gets there, and it ends up with the British government spending something close to 400 billion pounds trying to deal with this. In fact, most of the debt problem that we have in Britain, most of our financial problem today is