Siliconsciousness: Why Is Trump Ok With AI Armageddon?

The DSR Network

This episode of The DSR Network's 'Siliconsciousness' explores growing concerns among top AI researchers about existential risks posed by artificial

Key takeaways

• AI researchers' warnings about existential risk (e.g., 10% P-Doom probability) are not new but have gained renewed attention amid market volatility and public protests.
• The current political climate—marked by distrust in institutions, anti-science sentiment, and dismantling of scientific integrity policies—amplifies fear and undermines effective regulation.

Main topics

• AI existential risk and P-Doom estimates
• Political inaction on AI regulation under the Trump administration

Notable quotes

"We've just ended a summer in the United States in our hemisphere in which, you know, it's the highest August on record. So we've had climate threats sort of bearing down on us for a very long time that we're not doing anything about."

"You know, there is this sort of general sense that like catastrophic things could happen and we've got to get ready for things we could imagine and things that are totally imaginable."

Conclusion

Alondra Nelson emphasizes that while existential AI risks are real, they are compounded by deeper

Transcript preview

Speaker 2 (0:01) Welcome to Scylla Consciousness, the DSR Network podcast focusing on the artificial intelligence revolution, politics and policy. Speaker 4 (0:19) Hello and welcome to DSR's Scylla Consciousness. I'm David Rothkopf, your host. And this week, as every week, we're going to talk about some big issues pertaining to tech and to AI. And we are very fortunate to have back with us one of our favorite and wisest guests, Alondra Nelson. She is the Harold F. Linder Professor of Social Science and Director of the Science, Technology, and Social Values Lab at the Institute for Advanced Study in... Princeton, New Jersey. Between 2021 and 2023, she held the position of Deputy Assistant to President Joe Biden and Acting Director and Principal Deputy Director for Science and Society at the White House Office of Science and Technology Policy. So not only are we happy to see Alondra because she's our friend, but there's so much to talk about Alondra. Speaker 1 (1:13) So much, David. Great to see you. Speaker 4 (1:15) Great to see you. Well, you know, the reason I originally reached out is that I saw you had co-authored an article in Science magazine about the consequences of the war on science, which seemed to me to be a pretty big deal because science. But let's get to that in a second, okay? Because in the past few days, we've had some really weird stuff going on. And it ties to conversations we've had in the past. At some point last week, I guess triggered by a statement by the leadership at Anthropic, but was quickly followed by other statements. And actually, it was triggered by a resignation there that was then triggered. Yes, Speaker 1 (2:01) Jacob Coxon from Anthropic, yes. Right, Speaker 4 (2:03) right. And so, you know, here we got this kind of strange situation. where the leading experts in a key area said, you know, there's about a 10 % chance that this technology will destroy the world. And of course, when I saw that, I thought, gee, that's interesting. Because if all the leading cosmologists in the world got together and said, there's a 10 % chance this asteroid will kill us all, the world would get pretty freaked out. And I thought, well, the people aren't freaking out. But then they actually started to freak out. And I will get to the nature of the freakout subsequently, and then I will get to the president's reaction to the response to the freakout. But let's just start with the freakout. How did you respond to all that? Speaker 1 (2:51) I mean, I think, you know, we'd heard this before. I mean, part of what Jacob Coxon was articulating was what's called a P-Doom, like what's your P-Doom score? And so sometimes it's 5 % for him. It's 10 % among some people who describe what they imagine the future to be in the frame of P-Doom. And so it wasn't a new caution or a new warning. And I think, you know, sometimes. You know, there are others who had resigned, you know, over the last year. There are others who have resigned and wrote letters who had, you know, from the AI labs and, you know, voiced their concern and worry. I mean, you know, the introduction of Chow GPT in the fall of 2022 is followed in a couple of months by a big letter by lots of leaders asking for a pause. And so in some ways, it felt a little bit like Groundhog's Day. But I think different for this moment is that we have a context of people being really worried about a kind of bubble in the markets. And we saw overnight there was a sell-off in the markets and the AI stocks. We have people have, you know, ordinary Americans have spent the last six months picketing in front of data centers or potential data centers. So there's lots of that concern. And then the polling around AI. anything from healthcare to effects on our young people to people's future work prospects is just growing. The negative sentiment is just growing. And so that message about, you know, Skynet, AI is going to kill us, 10 % PDoom, that drum had been beating for a very long time. But I think now you have this sort of perfect window of opportunity where there is... wide discontent. It's completely cross-partisan. And I think people are, from all walks of life, increasingly fed up. And I think also not understanding how one of the world's most powerful nations is seemingly unable to do anything about these very powerful technologies. Speaker 4 (4:51) And there's one other thing that adds to that context, which is a couple of weeks prior to that, we had the so-called Jailbreak is not the right term, but we did have some incidents where AI sort of crawled out of the lab and started going after other people. And people were like, uh-oh, this is that story. And so that too affected it. But let me ask you, because you're a very sensible person and you're immersed in this world and you don't seem like your hair's on fire right at the moment. How do you feel about that PDoom score? How do you feel about... the level of alarm, and is it amplified and distorted by the things that you just talked about? People don't like AI companies. People don't like AI company leaders. People don't like data centers. People think that this bubble could blow up. And so you take all those things in and it sort of gets conflated with this and people go, oh yeah, maybe I really should be afraid. Speaker 1 (5:55) Yeah, I mean... Listen, I mean, David used to work in government, used to work on the national security side. I mean, listen, I think part of the job of working in the White House and working in national security is that like someone's trying to kill us, right? Whatever. I mean, like, so, you know, there is this sort of general sense that like catastrophic things could happen and we've got to get ready for things we could imagine and things that are totally imaginable. And so, you know, part of me feels like. You know, of course, there are like terrible things that could happen that are catastrophic. One could imagine, you know, you know, a state actor, non-state actor sort of using these tools. There was some reporting that, you know, that Iran potentially had been trying to use these tools, according to Claude. Again, we have no no verification of this. We just have to take their word for it. to sort of create maybe chemical weapons or something. I don't know. I can't remember what exactly the bad thing was they were trying to do using Claude. So that's just a truism of life. Like I think so part of me is not hair on fire because like that's how it is. We've just ended a summer in the United States in our hemisphere in which, you know, it's the highest August on record. So we've had. you know, climate threats sort of bearing down on us for a very long time that we're not doing, haven't been doing anything about. And, you know, I bet if you asked some leading climate scientists and gave them the frame of a P-Doom, they would probably be around 10 % as well with regards to climate. So, you know, I would add that initial context. And then I would say why this feels like Groundhog's Day is that over the last three years, every time