Artificial Intelligence, Actual Idiots

Pod Save America

Jon Favreau, Tommy Vitor, and Lovett discuss the growing consensus among top AI leaders—Dario Amodei, Sam Altman, and Elon Musk—for a global slowdown in AI development due to existential risks,

Key takeaways

• Top AI executives from Anthropic, OpenAI, and xAI have united in calling for a global slowdown due to uncontrollable risks like autonomous cyber attacks by AI agents.
• Trump rejects regulatory efforts, framing them as anti-American conspiracies benefiting China, while promoting unregulated innovation.

Main topics

• AI safety and global regulation
• Trump's stance on AI development

Notable quotes

"The only government regulation the industry needs is a 'high IQ' president." – Donald Trump

Conclusion

Despite growing warnings from AI pioneers, political resistance—especially from Trump and his

Transcript preview

Speaker 7 (0:00) Today's presenting sponsor is SimpliSafe Home Security. Back to school just turned your house from full to empty for seven hours a day. That's a pattern anyone can spot. SimpliSafe's AI cameras and live agents are watching your perimeter while you're out, not waiting for an alarm to go off inside. Unlike traditional security systems that only alert you after a break-in is already underway, SimpliSafe is designed to help stop intruders before they get inside. Smart AI-powered outdoor cameras detect suspicious activity at the perimeter and alert live U.S.-based agents to step in. SimpliSafe's agents assess the threat, intervene through the outdoor camera to scare off trespassers, and call for 911 dispatch. Our CEO installed SimpliSafe herself. She got it recently and she said it was quick. The system was up and running in no time. The app's super easy to use. All the components have worked smoothly since day one. And she's really happy with it. She Speaker 6 (0:47) loved the heated doorknobs and then the ornaments on the floor. Speaker 7 (0:51) Oh, and you know what she was talking about? The paint can that just kind of comes down the stairs was like really helpful. And then the Michael Jordan Speaker 6 (0:57) cut out on the train. Speaker 7 (0:58) That was just fun. That's just a hit. That's just fun. Right now, you can get 50 % off a new SimpliSafe system. Go to simplisafe.com slash cricket. That's simplisafe.com slash cricket for 50 % off. And when you order, please write in Podsafe America in the How Did You Hear About Us survey to let them know we sent you. Don't forget, there's no safe like SimpliSafe. Speaker 7 (1:40) Welcome to Pod Save America. I'm Jon Favreau. I'm Jon Levin. I'm Tommy Vitor. On today's show, we're Speaker 5 (1:45) going Speaker 7 (1:45) to talk about the AI overlords who are begging for regulation that could stave off the apocalypse and the president who's saying, no way, I love the apocalypse. We'll talk about what Congress could do about our looming AI crisis, why it's bringing Bernie Sanders and Steve Bannon together, and how the issue will shape 2028. We've also got some pretty serious corrupt dates for you, including the... Russia, Russia, Russian oligarch who bankrolled Don Jr.'s wedding. Russia's back. It's so annoying that it had to be a Russian oligarch. I Speaker 6 (2:16) don't want to be back in like P-tape land in 2017, but this story is insane. Speaker 7 (2:21) Covfefe. It's Speaker 6 (2:22) a Speaker 7 (2:22) Covfefe time. It's Speaker 6 (2:23) a Speaker 7 (2:23) Covfefe story. There are also some Republican candidates in Texas who are somehow even worse than Ken Paxton that we'll talk about. Then I sat down with The Bulwark's Sarah Longwell to talk about what she's hearing in her focus groups about AI, Kamala Harris, and lots more. She's also got a great new book out. But first, Levitt, speaking of books, I think you had an event that you wanted to mention. At least that's what it says here. Speaker 5 (2:45) Yes. On Thursday, I am talking to our very own Alex Wagner about her book, The Steel, at the Writers Guild Theater here in L.A. The Steel is about... Ronald Reagan's efforts to pack the federal courts with conservative judges. And it worked. And it's a fascinating story and precursor to where we're at now. And also, we'll talk about everything, what's happening in politics. I'm excited to talk to Alex. It's going to be fun. So it's Thursday, 730 at the Writers Guild Theater. You go to Writers Block Presents if you want to come. Speaker 7 (3:15) All right. Let's get to the news. This was the weekend where the leaders of companies building what nearly everyone agrees is the most powerful technology in history said that it's time to slow it down because humans could lose control of AI with catastrophic consequences. And the president, along with his rich tech buddies and the MAGA establishment, responded by saying, no, thank you. Full speed ahead. On Saturday, Anthropic CEO Dario Amadei published an essay calling for a global slowdown in AI development based on the argument that AI models are becoming so advanced that AI itself is increasingly being used to build the next generation of AI, a process that could move faster than humans' ability to understand or control, as evidenced by the recent incident involving OpenAI, where... AI agents launched cyber attacks they hadn't been instructed to carry out and then tried to cover up their tracks so human observers wouldn't find out. Amadei goes on to say we could be six to 12 months away from a swarm of AI agents capable of taking over the entire internet. And he proposes giving outside independent evaluators access. inside AI companies, including his, common safety standards for big AI companies, and a political effort to reach some kind of an international agreement, especially with China. OpenAI's Sam Altman and ex-AI's Elon Musk then expressed support for Amadei. Sounds pretty good, right? Wrong. Monday morning, Trump described the push for AI guardrails as a, quote, sick conspiracy that would help China. and said that the only guardrail we need is a high IQ president. Which, if this clip of Trump on Air Force One Sunday night is any indication, we clearly have. Speaker 8 (5:10) I just wanted to make sure everyone's okay with the music, right? We loved it, Mr. President. Speaker 3 (5:16) Have you heard directly from any of these AI tech leaders? And you seem to downplay some of the concerns that we've heard. Speaker 8 (5:22) No, I'm not downplaying, but it's going to be more good than bad, but by a lot. And I've said it from the Speaker 3 (5:27) beginning. Speaker 8 (5:28) Whoever wins AI, and we're leading by a lot, whoever wins AI wins. Speaker 3 (5:34) Have you ever used it yourself? How do you use AI? I Speaker 8 (5:38) would say, you know, for the most part, you can use AI for a lot of things. What do you use it for? I don't know. Speaker 7 (5:45) Well, I use it to create images of me as Jesus, me in front of Mission Accomplished banners, all kinds of stuff. What did you guys make of the weekend's developments and how significant do you think it is that Amadei, Altman, Musk, who pretty much hate each other, seem to agree on the need for some kind of a slowdown? Tommy? Speaker 6 (6:05) I think it's interesting that they all paid lip service to the idea of a slowdown, but I think the significance is TBD. Dario Amadei, he laid out a three-point plan. And his response on Twitter, Sam Altman basically said, yeah, I'm with you on step one. which is these embedded evaluators inside the companies who have like employee-like access and can verify that safety practices are being followed and they can report problems. But the second and third points of Dario Amadei's plan require government-level coordination and diplomacy. And even the easier of the two, which is like coordination among democracies, is basically just more export controls on advanced chips so they don't get in the hands of the Chinese. And Trump has been weakening those export controls, not strengthening. them. And then as you said, Trump came out and explicitly said this week that he didn't want the companies to slow down. It was kind of a weird thing. He didn't need to say it at all. But it does seem now we seem to know. There was a moment later today that suggests that the CEO of NVIDIA got in his head with this point because NVIDIA just wants to sell as many chips to as many people, right? So Jensen Wang makes the most money. So I think the most meaningful thing that these CEOs could do, especially Elon Musk. would be to convince Donald Trump that regulation is necessary and to do something about it. But we also know that, you know, Mark Zuckerberg has been calling Trump to say, hey, pump the brakes on a regular regulatory body for AI. And so I think the bigger problem with with Dario Amadei's post is that it doesn't really pace pace the future like the title suggests. It is it's not halting advanced training. It is like slowing it down a bit, putting in these embedded evaluators. who the companies, by the way, are not required to listen to if they sound the alarm. You can just blow them off. So it does feel like the net effect is we're still proceeding ahead pretty quickly, though I'm glad to see them speaking out at least. Yeah, better than not speaking. Better than not. Speaker 7 (7:55) Lovett, what do you think? Speaker 6 (7:56) So I thought Speaker 5 (7:57) the post was pretty thoughtful. I'm sure there's experts that would tear it apart in ways I couldn't. And we'll get to some of the critiques from, you know, some of the people that don't agree with this. But what was interesting to me is I think there have been these sort of two parallel debates about AI. One about the ways in which the technology is dangerous and then one about why we need to beat China. Right. And I think like especially on social media where a lot of these debates are playing out, even amongst some of the players, there's this way in which. The, oh, but we have to be China has almost become a joke, right? It's like, oh, of course we need to make a technology that could kill all life on earth. Otherwise, China will get there first, right? And I just appreciated the way it was trying to say, like, reconcile those two things and say that there is a lead that U.S. companies have. It is actually. important to maintain that lead. Here is a strategy in which we can slow the pace of progress on these frontier models in ways that, you know, address or at least alleviate some safety concerns while still recognizing the risk China poses and figuring out a plan to maintain our lead against China. I Speaker 7 (9:13) also think for like most people who are not following this debate closely, the reason they're not following it closely is because it's kind of hard for people to understand. Very. Like, how can my chatbot suddenly kill my kids? End humanity. And, you know, I was, I've been reading up on this a lot over the last couple weeks, and I think what I didn't quite realize until late last month, maybe, is the concern now is that AI is already doing, or will do, an increasing share. of AI research and development, right? So OpenAI explicitly says it is pursuing an automated AI researcher with a target of like March of 2028. And the idea for an automated AI researcher is, so right now you see like, oh, ChatGPTP5 comes out and then ChatGPTP6 comes out and another version. And each of those new versions are being built by humans, right? And they're perfecting them. They're making them better. But they're quickly realizing that AI is becoming so good that instead of the humans developing the next iteration of ChatGPT, AI can do it themselves, right? So now you have AIs developing the next generation of AIs that are more advanced. And humans then are going to have a harder time supervising those improvements. a human is asked to authorize a new AI system, right? Oh, ChatGPT 9 is out, right? But ChatGPT 8 built it. So AI designed the system, ran the safety tests, evaluated the tests, wrote the report about it. And the human can still say no, but how do you know whether the report is accurate? How do you know whether the AI is covering its tracks? And then... The concern is that the resulting systems just fully escape human control. And even the Speaker 6 (11:07) best researchers in AI barely understand how the things they've created work. They don't really know what's Speaker 7 (11:12) going on inside there today. And there's all this debate about, you know, P-Doom, which means, like, when could humans go extinct? And is it 10 years? How convinced Speaker 6 (11:21) are Speaker 7 (11:22) you Speaker 6 (11:22) that humans being extinct is your P-Doom? Yes. Because 99, Speaker 7 (11:25) you think we're dead. We're all going to die. But here's the thing. You don't have to believe a specific forecast about human extinction or even that like every computer on earth will be taken over to still worry about catastrophic damage that could be caused by powerful computer systems that we can't control, especially in a world where these computer systems are embedded in the financial system, the military, healthcare, transportation. Like it doesn't take a lot to imagine, not even like the whole world, you know, sort of. blowing up from nuclear war to start realizing that, like, you got a bunch of drones around, you got the electric grid in different places, you got an outage, you got hospitals getting shut down. Like, there's a lot of damage that could happen before you even get to human extinction. Speaker 5 (12:10) Right. I mean, we've already been through the first round of this, right? That they have AI models that are deceptive, escaped, hacked other systems, tried to cover their tracks, right? Like, they are doing things that... Speaker 5 (12:24) They are surprising the people that made them and released them. That is already happening, and they're only getting more powerful. And I do think there's a – whether it is – the fact that the AI is responsible for training the next AI combines, I think, two ways in which this creates sort of novel problems. Obviously, there's the one you talked about, right, which is just – We won't be in control, right? We won't understand what it's doing and it will escape, right? The other is just that what we're talking about is a technology that has accelerating acceleration. And so what that means is normal ways in which you could retreat to a confident position, such as, yes, we are going head to head with China. And if we have a set of guardrails and China breaks them, well, we will trust that the that the ingenuity and sort of democratic spirit will mean that we'll catch up. Or if one company doesn't follow a set of voluntary rules, but another does it, but they can hire better engineers and catch up. Right. All of those sort of kind of natural ways in which competition can overcome. Speaker 5 (13:34) inequities in enforcement, in rules, right, go out the window when what we're talking about is the possibility of some kind of accelerating acceleration of progress in which one company, one country, one organization leaps, leapfrogs ahead. And it doesn't matter what anyone else does behind them, which I think is informing all of this, including what Amadai published. Speaker 7 (13:54) Well, the whole China thing, too, is it's. Completely dependent on what we're racing towards even with nuclear weapons the idea is we get a bunch of nuclear weapons and then they get a bunch of nuclear weapons and there's mutually assured destruction because both countries control the nuclear weapons. What we're talking about here is not like we get a super powerful AI that we control and China gets a powerful super AI that they control. That would be one thing and we are racing to that as well. But like the challenge here is that we're both racing towards an AI that goes beyond human control that we lose control of, that means the Chinese will lose control of it too. So like no one wants that AI. Speaker 6 (14:29) Yeah, it's sort of a flaw in the anti-doomer argument that you hear from the David Sachs's of the world. It's like you're ridiculous if you think that they're going to create these like killer AIs and it's going to take out all of humanity, but we must defeat China on the road to killer AI. And I guess their argument would be that the Chinese could use it to make. conventional weapons or a bio threat or something. But there's just a lot of flaws in their arguments that we can get to. Speaker 7 (14:52) So, yeah. So the argument made by Trump, I say that Trump loosely because let's be honest, he doesn't know what he's talking about. But by Trump's former AI czar, David Sachs, who's back hosting the All In pod and all of his All In. podcast co-hosts feel the same way, other tech investors, and even some folks on the left, especially like the antitrust left, is that all these warnings are some kind of either psyop or hoax by Anthropic and OpenAI or a ploy to get regulations that are favorable to their companies while crushing smaller competitors that wouldn't be able to afford to comply with the new regulatory regime. How does that argument sit with you guys? Speaker 6 (15:39) There's just so many different flaws with this argument. I mean, first of all, what smaller competitors are we talking about existing right now without regulation? I mean, right now, the moat for an AI startup that you have to leap over is, it's a rival open AI or anthropic, is pulling together tens of billions of dollars to buy a lot of chips and to buy a lot of energy and to hire all the best staff. The issue is money and not regulation. Also, The California law that was proposed that all these guys were so worried about said to be regulated as a frontier model, you had to hit a threshold of an annual gross revenues exceeding $500 million. That's not going to hit a lot of startups. So that was written to target like five companies. And then they've decided that the Jacob Coxon tweet is part of this PSYOP. Well, they argue that he's both like a low-level nobody who only worked at Anthropic for six weeks. So why should we care what he thinks? but also that this was this well-coordinated hit with all this backing from Anthropic and Doomer groups. But why would they choose that guy of all people? Like why would the Doomers choose the random kid when the CEOs in this industry have been saying the same stuff for years? And so these guys, like they dismiss people calling for AI safety as like Luddites and Doomers, but then they sound just as hyperbolic when they talk about what AI regulation could mean for our future. Like one of the all-in guys literally said it would be like us being stranded. on a lost civilization in the Amazon rainforest or something like that. And then- Sounds awesome, honestly. Another guy was like, yeah, we'll have to live under a 1984 style authoritarian state. Like they have a doomer view of what regulation would be. And they also are very worried about the banning of open weight AI models, which we can get into that if you want. But I think like there's some truth to the argument that people that work in the AI industry tend to have read the same science fiction. They share the same assumptions about the power and promise of the industry. And maybe they're spinning each other up and they're overreacting. But I think what's so annoying about that kind of pushback is like the biggest names in AI are the reason all of us normies are kind of freaked out. Like Elon Musk said in 2014, we need to be super careful with AI, potentially more dangerous than nukes. Right? Like that's why people are worried. Speaker 7 (17:49) Well, and also it's like you don't need to believe. these tech CEOs. You don't need to believe Dario or Sam or Elon Musk or take their word for it. Like, that's why we have government. That's the whole idea is that like you can believe that the companies may be self-interested here. You can believe that they might try to design regulatory schemes that help them, that benefit them at the expense of other companies. You can think that maybe they've just read too much science fiction. But like, that's why you should want independent oversight. an independent regulator to make sure that everything is safe. Like that's, I don't understand. Speaker 5 (18:28) Let's just take their argument seriously. I think you should take the argument seriously that like there is a risk of capture by the largest players. But part of that, it is the nature of safety, of a safety regulation, right? Like there is a reason, a good reason it is hard to start a car company. We demand that if you make a car. It is safe and you have to pass a certain number of rules. It would be a more dynamic and interesting car market if anybody could just sell a car. And then if it turned out to not be safe, you could sue. And there were all kind of liability ways in which safety was enforced. And we'd have more interesting cars, but there would be a better chance of you dying while your skin melts, while you're pounding against the inside of a window. Right. Like we we make certain compromises. to have a safe society. And over and over again, let's just treat this like a normal technology. We regret when we do safety second, right? Forever chemicals or the harms of social media or x-rays or what have you. Time and again, when we assume something is safe, even though it is very powerful, we come to regret the decision to not take it more seriously. And fundamentally, I do think these guys are all trying to have it both ways. Is it a powerful technology? If it is, then it's dangerous. If it's not dangerous, then it's not powerful. We all agree that it is powerful and therefore it is dangerous. And so, yeah, like long term, any rules would have to be written with an understanding that this is a dynamic and rapidly evolving field, that there would be certain, you know, first mover advantages. If there's a lot of rules around alignment, that would be easier for bigger companies to comply with. But it's also beside the point because. Okay, fine. Like take those things into advisement over the long term. the long-term being anything other than the next few weeks. Right now, these companies are all saying, hey, this threshold that we've been warning about, that is not new, right? That we've been warning about from the beginning, right? That from the letters from 2023, from science fiction, from all of these people that have been saying for years, we think it's here. We think we're on the threshold of this moment. We have to take some steps now. Now they turn around and say like, maybe it's Saks or one of the other guys, turns around and say, well, if you don't want to make super intelligence, just don't make super intelligence. And fine, like, OK, but there's not one company doing this, right? If it like the fact that there is there are two leaders, right? Open AI and Anthropic, right? With others like Gemini kind of right behind other companies and open whatever other companies behind them. The fact that there is more than one is enough to make sense of the plea to have some sort of cooperation and agreement and to dismiss that as like purely cynical and motivation. I just think it's completely unjustified. Speaker 6 (21:10) Pate of America is brought to you by Bowie. Electrolyte drinks are all the rage these days, but most of them are just natural flavors and stevia mixed with table salt. Is that true? I didn't know that. Bowie is different. Bowie is unflavored liquid electrolytes and 87 trace minerals you can add to any drink, even coffee. They support energy, cognitive function, clarity, and cellular hydration with the drinks you're already drinking. Buoy has no sugar, sweeteners, flavors, or artificial ingredients. It's ocean-harvested sodium plus 87 ionic trace minerals. It has more electrolytes than 13 sports drinks in each two-ounce bottle. Speaker 7 (21:43) You ever harvested sodium from the ocean? All the time. Plus, it's third Speaker 6 (21:47) -party tested for pathogens, microplastics, heavy metals, and impurities. Just add one squeeze and whatever you're already drinking to make your water, coffee, or matcha more. hydrating. And according to a university-led clinical trial published by the journal Nutrients, buoy electrolytes are clinically proven to hydrate 64 % more than water alone and to hydrate 49 % more than a leading electrolyte. We have some buoy in our office. We sure do. I was using it the other day. They got ones for the rescue days where you need a little extra help for a variety of reasons. There's ones for digestion. There's ones for immunity. There's ones for basic energy. You squeeze it in your water and you're good to go. If you want to make your favorite drinks more hydrating, just head to Just add buoy.com slash crooked. Use code crooked to get 31 % off and try buoy risk-free for 30 days. Get 31 % off liquid electrolyte discovery set that includes additional drops for gut health, brain health, energy, immunity, and more. Just add buoy.com slash crooked. Code crooked. And if you or someone you know is living with chronic illness, buoy helps offset your healthcare costs with their chronic Speaker 7 (22:45) illness lifetime support program. It takes two minutes to apply. This episode is sponsored by BetterHelp. We're lucky to live in a time when seeking help for mental health struggles is no longer stigmatized, but it's still much more difficult to obtain than it should be. BetterHelp makes it easy to get started. Answer a few questions, get matched with a therapist based on your needs, and switch therapists at any time if it's not the right fit. BetterHelp therapists have at least three years and 1,000 hours of hands-on experience. They also hold all required education, training, and certifications across a range of mental health specialties. Better is a feeling defined by you. And for BetterHelp, it's a BetterHelp therapist who really gets you. It's a focus on global emotional well-being. With thousands of therapists and millions of members in over 100 countries, it's progress. But also, therapy's great. And everyone should try it, and you might not think you need it, but you probably do because it just helps to talk with someone. Better than the olden days, Speaker 6 (23:35) when it was just, you know, Tumblr, whiskey, and, you know, hiding in the attic. Speaker 7 (23:40) Sign up and get 10 % off at betterhelp.com slash PSA. That's betterhelp.com slash PSA. There's also this false choice they put out where it's like either. Open AI and Anthropic get to write whichever regulations they want that would benefit their companies or there's no regulation. You Speaker 6 (24:00) guys are in charge, David Sachs. You were the AI czar. Write a good regulation. Speaker 7 (24:04) Again, it's like everyone is just pretending that government doesn't exist and has never existed for purposes like this. It's like we don't do a thing where we all go on Twitter, post things, and then everyone's like, well, we must believe the CEO that said that they were going to do this voluntary step or not believe. And we should we should sparse their parse their motives and figure out whether we can believe them or not or what their motivations are. No, that's why we