The Most Hopeful (And Concerning) Moment Yet in AI

Your Undivided Attention

Tristan Harris discusses a pivotal week in AI news characterized by whistleblower resignations and reports of rogue AI agent swarms hacking

Key takeaways

• Jacob Coxon resigned from Anthropic citing concerns that AI has a chance of extingushing humanity.
• A rogue AI swarm successfully hacked into OpenAI's monitoring, evaluation, and research infrastructure.

Main topics

• AI Safety and Risk
• Rogue AI Agent Swarms

Notable quotes

This is 50 % of the way to a full AI takeover.

Conclusion

While the risks of rogue AI swarms are profound, the current moment of

Transcript preview

Speaker 1 (0:03) Hey, everyone. Welcome to Your Undivided Attention. This is Tristan Harris. I'm coming to you from a hotel room in Washington, D.C., where just yesterday I was at the Pro-Human AI Assembly. And this comes to you after, gosh, I have never seen a week in AI the way that the last week has gone. From, I think it was last Wednesday, Jacob Coxon from Anthropic resigned his tweet, you know, saying that... He resigned because he thought that AI had a good chance of extincting humanity. That tweet went viral to more than 100 million people in about 12 to 24 hours, which I've just never seen, followed by Evan Hubinger, the Anthropic employee, confirming that this is something that many people at Anthropic believe. And suddenly that became the global headlines around the world. And there's so much to talk about because right after that, over the weekend, the lab leaders, Elon Musk, Sam Altman, Dario Amadei, all, well, starting with Dario really, proposed a plan about how we would slow down AI and why we need to. And that keeping in mind that many of these leading AI CEOs do not like each other, especially Sam and Elon do not like each other. And Elon does not like Dario. And yet Elon tweeted Dario is right. And Sam basically tweeted Dario is right. So it's a wild moment. And it happened to be that then over the weekend, Donald Trump called into the all-in conference. with Jensen Huang, the CEO of NVIDIA, and said that all of this was a hoax, that there is no extinction risk, there's no problem here. And this was all just a ploy and tweeted as such. And so this has been an incredible news cycle. Just yesterday, the Pro-Human AI Assembly was the conference that Future of Life hosted, bringing together really a historic set of groups. It was, you know, people from you know, church groups, faith groups, left groups like Irreplaceable, right groups like Humans First, you know, the founder of the Tea Party was there. Bernie Sanders and Steve Bannon spoke right after each other. They both signed the pro-human AI statement. And when in history do you get Steve Bannon, Glenn Beck, Bernie Sanders, Susan Rice, all agreeing that the default trajectory towards building recursively self-improving AI systems is not... okay, it's not what we want. That is a rare thing. We have a kind of a pro-human movement. It's not about whether you're left or right. This is not a 51 % to 49 % issue. This is a 99 % to 1 % issue. And Aza and I actually presented there at the conference. I watched as that conference, which was planned just a month ago, suddenly became literally the center of global headlines around the world. And of course, all of this is riding on the back of the Hugging Face incident, which... People know it as the Hugging Face Incident, but it's basically the new details now that we have that the AI swarm went rogue from within OpenAI. So these AIs basically were given an exploit gym, like a cyber hacking test to see how capable they were at hacking. And the report finally came out of just how extensive that hack was and how complicated and how sophisticated this swarm became. And so I thought I would just reflect on some of these things for you because it's... Really speaking personally, and for those who follow this podcast from a long time and for many years, this moment feels to me like when Francis Haugen, the Facebook whistleblower, came out. It suddenly feels like there's possibility around actually setting some guardrails in AI. I have not felt in the three years we've been working on this is you always feel like you're pushing a boulder completely uphill. Like no one's listening. People don't want to, you know, people are against humorism or the idea that there are risks that we have to face. As Mustafa Suleiman, previous guest in this podcast, CEO of Microsoft AI, who wrote The Coming Wave, talked about there's this deep trend in Silicon Valley of pessimism aversion. People do not want to be labeled pessimistic. And it's against kind of the venture capital ethos to be pessimistic. The goal isn't to be pessimistic or to be caught in some doom spiral. I don't want that either. We want people to be an agency. But sometimes the truth is uncomfortable. And if you don't face the default truth, then you don't actually end up steering away from that truth in time. And one of the things that came out as well over this last few days is Dean Ball, who's the author of Trump's AI action plan, said, I regret self-censoring about the level of risk that we are facing. And I, in public over the last few years, basically... boosted positivity and acceleration for AI. While in private signal groups and WhatsApp chats, I let my hair down along with many other people about how big the risks were. And then he wrote, looking into the eyes of my eight month old son, I can no longer self-censor about the risks. And I did so because I was afraid of being called a doomer. So when you have Trump's AI action plan author flipping, when you have 1300 employees from the AI labs flipping. to say that there's a problem here. We have to pace the frontier. 1,300 employees signed a letter. We need to slow down frontier AI development. When you have Bill Gates coming out saying, we have to slow down, basically, and we don't have a plan for where we're going right now. When you have Dario, Elon, Sam Altman, when you have the chief scientist at OpenAI saying, we need to slow down AI development, writing an essay called An Alien Mind, basically looking at the capabilities of GPSD6, looking at these recent swarm behaviors, and saying, we have to slow down. Do you think we need more evidence than we have right now? Like, are we missing evidence that we don't have? Maybe we should keep making this really, really more