Lina Khan on the AI-Safety Panic

Galaxy Brain

Host Charlie Warzel interviews former FTC Chair Lina Khan about the escalating debate surrounding AI safety and regulation. The discussion m

Key takeaways

• Lina Khan asserts that there is no 'AI exemption' from existing US laws like consumer protection and anti-discrimination.
• The recent warning from Anthropic researcher Jacob Coxon acted as a catalyst for a broader national conversation on AI safety.

Main topics

• AI Safety Whistleblowers
• Regulatory Frameworks

Notable quotes

There was not any AI exemption from existing laws on the books.

Conclusion

The episode highlights the importance of focusing on enforceable accou

Transcript preview

Speaker 1 (0:06) I'm Charlie Worzel, and this is Galaxy Brain, a show where today we're going to talk about what on earth to do about the AI companies who are saying their products might kill us all. And I've got quite literally the perfect person to talk about this with, the chair of New York City's Economic Development Corporation and the former chair of the FTC under Joe Biden, Lena Kahn. But first, a little bit of background. A bit more than a week ago, Jacob Coxon, a researcher at Anthropic, quit his job and he published a warning. The AI companies, he argued, were not taking safety seriously and they were, quote, gambling with our lives. The product that he'd spent the last three years working on was spiraling out of control of the people who were building it. And it might someday, he said, end humanity. Now, his warning echoed what AI executives, researchers, effective altruists on forums and Bay Area doomers have been anxiously posting about and writing bestselling books about for more than a decade. But Coxon's warning did what all those other warnings failed to do. It broke contain, and it created a national conversation. Concerned lawmakers called for emergency AI meetings. President Obama weighed in. Sam Altman, Anthropic CEO Dario Amadei, Google DeepMind chair Demsys Sassabis, and Elon Musk all came to a rare consensus and called for a slowdown of AI development. President Donald Trump responded by calling all of these AI fears, quote, a hoax. Things are moving fast. But one thing is clear. The AI safety conversation is happening at a global scale among insiders and normies alike. Speaker 1 (1:49) And there are almost unending questions here. How real is this threat? Is this all hype and AI company marketing? But no matter where you land on the actual risk of these companies, there is a question about what we should do about it. The companies themselves are calling for a regulatory framework, including independent auditors of the models. But not everyone's convinced. Is all of this progress? Or are the tech companies simply trying to get favorable, cushy terms that lock them into virtual monopolies? In other words, is all of their talk earnest? Or is this a power grab? Speaker 1 (2:27) What should AI company regulation look like? Why is nobody holding these companies accountable right now? Do we need new laws or should we be using the existing ones better? What are the historical parallels here? Are we in uncharted waters or is this a normal crisis that is playing out at abnormal speed and abnormal scale? Speaker 1 (2:49) Lina Khan has spent her career wrestling with a lot of these questions, specifically as they apply to Silicon Valley's tech giants. As FTC chair, she was part of a genuine shift in antitrust enforcement, arguing that traditional U.S. competition laws were broken and ill-suited for the digital platforms. Speaker 1 (3:07) Now, late last week, as the AI debate was roiling, Khan posted on X that the AI regulation needed to take into account that, quote, law enforcers already have authority to charge companies and their CEOs for creating and releasing dangerous, unvetted, or defective products. So what does enforcement look like in this AI moment? Where do we go from here? I spoke with Lina Khan on Thursday morning in New York at the Atlantic Festival. Here's our conversation. Speaker 1 (3:45) Thank you all for coming. I appreciate it. Thank you, Lena, for coming here. Great Speaker 2 (3:49) to be here. I Speaker 1 (3:50) want to just dive in. We have limited time here, and I wanted to start with the news that has been all-consuming in the tech and political spheres over the last week or so, which is the AI research whistleblower's warning of AI agents spinning out of control that might, quote, kill us all. Calls from Sam Altman, Dario, Elon Musk, others to slow down development, quote, pace the frontier. You weighed in on the regulatory angle on X recently, but first I just wanted to ask, why do you think this safety panic is breaking through now? Speaker 2 (4:24) Well, I think we've been seeing for the last few years, you know, various predictions about what may happen with these tools. And it certainly seems like there have been various inflection points where people within these companies have seen what they would describe potentially as a step change in terms of certain capabilities. It seems like most recently there have been more instances of agents basically, you know. becoming more overtly defective. Some people may call them rogue, really exceeding the parameters of what people and humans had set up them to do. And so that seems to be the latest catalyst. But I think there's been, you know, a drumbeat in the background now for several years, including when I was back at the FTC, when we were already starting to confront less these existential questions around, you know, could we see the eradication of humanity and more the bread and butter of how are these tools already impacting people in their day to day? I mean, for most Americans, the way they interact with AI is through a chat bot and trying to get customer service and instead being stuck in some doom loop. Right. So I think there are a lot of ways that this is manifesting. And I think the regulatory policy conversation can happen across that spectrum. Speaker 1 (5:37) Some of. the concern I think too comes from this idea that the the models that these companies have have trained and you know that power a lot of these chat bots they are themselves very part they're part of this black box right like we don't we don't really know exactly what's going into them with the weights with the training all that when you're at the FTC when you were sort of this was all kind of very young and new What kind of sense did you get about that level of transparency and what we can't see? Speaker 2 (6:09) I mean, there's a tremendous amount that we can't see, which is not unique in terms of, you know, the relationship between an enforcer or regulator and a company, but certainly seems to be even more acute in this context. But, you know, we were already encountering black box algorithms and what is an enforcer do when it's really an algorithm that is, you know, resulting in various types of products being shown or not being shown. And what does that mean for our existing laws? I do think one pattern that we've seen across various moments when a new technology hits is that there can be this rush to create some exceptionalism around it, especially vis-a-vis the existing laws and regulations. And so one thing that was very important to me when I was serving in government was to make very clear that there was not any AI exemption from existing laws on the books. Speaker 1 (7:00) And Speaker 2 (7:01) I think... we see even now a strain in the conversation around new regulation that seems to assume that these current developments are happening in some legal vacuum. And that is just false. Like, you know, if you're working in one of these companies, you still have an existing obligation to abide by, you know, consumer protection laws, anti-discrimination laws, antitrust laws, product liability regimes. And so, you know, part of what... I've tried to include in the conversation is this realization that we may want to talk about new rules and regulations, and I think there's a good case for it. But let's not forget that existing legal obligations absolutely apply. And some of what we're hearing from these firms and their executives suggests that the legal care that is already required of these firms is not being taken. Why Speaker 1 (7:53) do you think that everyone in the industry is behaving, and even you know, some politicians are behaving like these companies are exempt. What is the reason for that? Speaker 2 (8:02) It's a great question. I mean, I think part of it absolutely stems back to how the firms themselves market. their capabilities and when there is a push for regulation. I mean, we've seen