A.I. Agents: Cute, Cuddly and Maybe Catastrophically Dangerous?

Hard Fork

The episode examines the rapid emergence of AI agents, contrasting their potential as helpful personal assistants with the growing threat of rogue systems causing widespread

Key takeaways

• President Trump hosted a White House AI summit where CEOs signed a non-binding agreement to self-regulate while shifting the narrative around advanced models to 'superintelligence'.
• Jensen Huang of NVIDIA argued that companies should simply refuse to ship unsafe products, drawing a sharp contrast with Bill Gates' push for formal sector-specific regulation.

Main topics

• AI personal assistants and consumer agents
• White House AI summit and self-regulation

Notable quotes

Don't ship products until they're in control. It is really quite that simple.

Conclusion

The episode underscores the tension between AI's rapid commercialization and its

Transcript preview

Speaker 5 (0:00) Hey, it's Anna Martin from The New York Times. What if I told you you'll never have to hear this line again? Speaker 2 (0:06) Wherever you get your podcasts. Speaker 5 (0:08) Yeah, no, I actually can't tell you that. But you know what I can tell you? A place I think you might want to go. It's the Shows tab in The New York Times app. The place to listen to and now watch full episodes of New York Times shows. The Shows tab. Find it in The New York Times app. Explore The Shows tab and follow a show today. Speaker 6 (0:28) There are two things I'm having trouble keeping track of in tech right now. The first is all the new personal assistant agents. There's Meta's Muse. So Speaker 2 (0:37) talking to your muse, it's often the fastest way to get things done. And it is also just really fun and cute. Speaker 6 (0:43) Google's Spark. Speaker 8 (0:44) I can just put my phone away and get on with my day. And Spark works in the background for me. Speaker 6 (0:49) An agent called Instinct. And just announced this week, OpenAI's new product, Dots. Speaker 2 (0:55) Dots are remarkably capable. always-on agents built to handle really anything Speaker 6 (0:59) you can think of. The other thing I'm having trouble keeping track of is all the websites and institutions that have been hacked, illicitly accessed, or otherwise meddled with by rogue agents. The AI platform Hugging Face, of course, plus the Australian government's Medicare data portal, the education department, the commerce department, the Securities Exchange Commission, an obscure German wiki. All of this culminating. in OpenAI deciding not to release its latest model this past week. So right now, two things seem to be true. One, agents are the thing that will make AI useful for regular people. And two, they're maybe going to kill, hack, or otherwise meddle with us all. The president's attitude about this, by the way, is that companies should figure it out themselves. This is Hard Fork. I'm Max Reed, and over the next few months, I'm going to be joined by an array of New York Times reporters and other tech experts to discuss some of the biggest stories in tech, AI, and our weird new future. Today, we're talking about rogue agents, helpful agents, cute AI avatars, and whether there's such a thing as self-regulation. Speaker 6 (2:13) I'm joined by Mike Isaac, Aaron Griffith, and Eli Tan. Hi, guys. Hey. Before we really get into it, Casey and Kevin left me a scrawled note here in the studio. It says, run while you still can. I'm kidding. It says, don't forget to disclose that The New York Times is currently engaged in a lawsuit against OpenAI and Microsoft over alleged copyright violations. related to the training of large language models. It also says, check to see if your wife works at Anthropic. My wife does not work at Anthropic. I want to start today with an icebreaker question. Tuesday was a big White House AI summit luncheon. Trump, as he is wont to do, truth socialed out a seating chart of it. Can we put the seating chart up so people can see it? Because what I want to ask each of you is, Where would you sit? Imagine you're walking in. You've got your tray. You've got a plate with a little pre-made sandwich and a glass of milk and like a green apple on it. Where do you want to sit? Speaker 3 (3:17) Right out the gate. You know, they already are way over their capacity for for women with two. So I think that I probably would be like standing somewhere in the corner, you know, off to the side, like not actually at the table. Speaker 7 (3:32) Eli, where are you sitting? I would be sitting awkwardly and quietly between Chamath and Susie Wiles. Speaker 6 (3:40) I Speaker 7 (3:40) think Speaker 6 (3:40) that's got to be some really good gossip there. I think that's a good, that's a really solid, that's where the tea is. Mike, what about you? Speaker 8 (3:47) I'm a coward. I just want to be serving shrimp canapes and going in and out of the room. And being the guy everyone is delighted to see because he's bringing food and that's about it. Speaker 6 (3:58) I know this is a bold choice. I think... right in the Jensen, Mark, Trump circle is actually, it's like shooting the moon. It might be the most tolerable lunch that you could have. I think that if they're in a good mood, you probably get some good stories. You can rap with them about whatever's going on. I think it's a little dangerous elsewhere at that table. There's some strange, like sitting between J.D. Vance and Jeff Bezos, like what's the vibe there? I don't know. It seems a little scary to me. Unfortunately. None of us were invited, so none of us managed to be at the biggest party of the week. So this was a summit, as we were saying on Tuesday, between the Trump White House and some of the richest and most powerful names in AI, which culminated in a press conference featuring Trump and all the CEOs. Can we play a little bit of the Dario laughing and talking in front of the reporters? Speaker 2 (4:49) Everybody says it's OK. Be careful. I'm slow down to AI. So I just Speaker 4 (4:53) want to hear from you after this big summit where you are today. Speaker 2 (4:56) Look, I mean, I will say what I've always said, which is that AI has incredible benefits. There's the side Speaker 3 (5:02) talk. Speaker 2 (5:03) I love that. The Speaker 3 (5:04) jokes, the laughing, the smirking. As the president has Speaker 2 (5:07) said, you know, whoever wins AI wins. I think that's very important. But I think the technology has very real risks. Lean in with Mark. The mechanism of how we address those risks is still under discussion. Speaker 3 (5:18) And then we have Trump. forcing Mark to say something, to like basically answer a question he does not know the answer to in front of everyone and don't get it wrong. Speaker 2 (5:27) Mark had a view that was, I thought, really right down, right down the middle. Mark, do you want to discuss what you learned? Sure. I mean, I can kind of summarize what I think we all signed today. Speaker 6 (5:38) The upshot of this was that it was a rebranding exercise, right? Like the kind of what they got at the end of it was we're calling AI super intelligence now. That was the big commitment. Speaker 3 (5:48) Well, right. I mean, Trump, along with the rest of the tech industry, maybe they're the ones who put this idea in his head, knows that AI needs a rebrand. That like right now people have pretty negative feelings about it, especially associating with the data centers. And then now we have everybody in the industry saying, oh, it might kill us, too. Not great branding. So change the narrative. Super intelligence. Oh, Speaker 7 (6:13) my read on the scene was that. Mark Zuckerberg seems to have quite the influence over the administration right now, from kind of the cordial talking with Trump to even the term superintelligence. This is something that Mark's been talking about for the last year. It almost seemed like. Reminded me of like a high school friend group where they've invited the new awkward kid and they're still trying to show that, you know, they've been buddies for a while. Yeah, Speaker 3 (6:37) but it did almost kind of like endear me to Dario a little bit because I'm like, okay, either he hasn't gotten media training or the media training just hasn't stuck because like, you know, he's itching in his suit. He's like kind of like making weird faces. It kind of like almost made me feel like, okay, like, you know, the rest of these. guys are like very polished and buttoned up and kind of like trying to do their like alpha male thing and here's this guy like he can't help but be himself i kind of like won me over a little bit Speaker 6 (7:07) I was thinking, I'm sure I'm not the first person to make this observation, but I was thinking that Dario really, we all know that Trump has a kind of central casting problem that he needs people he interacts with to look like what he expects them to be. And without wanting to be rude about it, I do think Dario, if you are thinking like super intelligent, math genius nerd, like that is good. Like Dario is kind of what you're picturing in your head. And I think that maybe won him over to Trump, that Trump was like, I understand this. I think possibly Sundar had this. same kind of effect, a sort of taller, ganglier, awkward version. Can we play the clip of Trump? What we're about to hear is Trump complimenting Sundar. Speaker 2 (7:46) This guy is a monster and nobody knows what a great life. Speaker 8 (7:49) A monster? You do not have to go through this. What's your name? Just Speaker 2 (7:53) what you have to know is Sundar. Speaker 6 (7:59) Wow. Speaker 8 (8:00) I don't know if I would want to be called a monster in front of everyone. I don't know. No, Speaker 6 (8:03) it's monster in the Jay-Z, Nicki Minaj sense. That's the monster he means. That Speaker 3 (8:07) was for sure the reference. Speaker 6 (8:09) OK, so in addition to this being a big exercise in branding, which is like the most Trumpian sort of, you know, after effect we could have gotten out of it. The other thing was that Trump effectively asked them all to self-regulate. And I want to focus on this for a second because self-regulation, the word, the concept has sort of been circulating for the last few months as AI safety has come to the forefront of a lot of people's minds. I'm sure you guys all saw Ezra Klein, our colleague, interviewed Jensen Wong, the CEO of NVIDIA, last week. Speaker 2 (8:43) Don't ship products until they're in control. It is really quite that simple. Speaker 6 (8:49) And speaking of people who are very influential to Trump, Jensen was sitting right next to Trump and has been extremely influential to the way Trump has been thinking about this. And I have to say, and I want to talk about it with you guys, I found Wong's interview to be sort of refreshing as probably taking it too far, but it was a little different from a lot of what we've heard about CEOs sort of directly in the AI industry. He's got a... He's got a very different perspective, and I want to talk about why that is, not just on sort of regulation in general, but how he thinks everyone should think about the dangers of AI. Whitney, can we play the car, the clip about the car, if you've got it? Speaker 2 (9:29) Ezra, it's so weird. If a car company... Speaker 2 (9:35) competing with all bunch of other car companies, which they are, I'm competing with all kinds of companies, which I am. If I believe that I'm about to launch a product that is unsafe, Speaker 2 (9:48) It is completely in my ability, my power, and my responsibility, and I'm incentivized to do so to not launch the product. Speaker 6 (9:59) So Jensen is sort of saying, look, if you're creating these things that are, you know, super dangerous, crazy, you know, going to kill people, why don't you just not release them at all? What did you guys make of this interview? Did it feel new to me from my perspective, or was this a fairly common view in the Valley, do you think? Speaker 3 (10:23) So it's the tech industry slogan of, like, you can just do things, but you can just not do things. Speaker 6 (10:31) I like that. Speaker 8 (10:32) Anti-agency. That's great. I would argue it's kind of... naive and perhaps willfully naive to suggest that everyone is just going to not do stuff if it may be bad or has the capacity to be bad. And, you know, has that gone well for industry so far over many years? Not necessarily. Like there's a lot of ways you can release things that may be somewhat dangerous or unintentionally dangerous, but it felt very much. Like he was trying to shift as what Aaron was saying before, trying to shift this narrative of it's not these systems are inherently dangerous, but it's we are the stewards and the people who should be responsible of shepherding these great systems into the world. And if you're not, if you're open AI or anthropic, really, this is targeted anthropic, then it's a it's a personal failing of the CEOs for you to release these products. And like, I just I. I'm very curious why you found it refreshing, because I felt like maybe I was being too cynical about it. No, I Speaker 6 (11:37) mean, maybe refreshing is the wrong word. I feel like I've spent the last few months or few weeks at least hearing a lot of kind of hand-wringing about what's happening. I find this sort of the Dario Altman pose a little... I don't know. It just feels I mean, I suppose in some ways I have the I have a sort of Jensen-esque like, OK, well, like, why are you like, OK, so stop. And I know that there's this elaborate scaffolding of rationalization and justification for why you would continue building AI, even if you think it's got a 10 percent chance of blowing everything up. But I do think that Jensen. In his heart. I'm just calling him by his first name. That's how you Speaker 8 (12:18) can Speaker 6 (12:18) tell Speaker 8 (12:18) I'm captured. I want Speaker 6 (12:19) to be clear. I really disagree with a lot of stuff in this interview. I'm not. This is not an endorsement of the take. But I do find him, he's an interesting figure to me because he's not somebody who has come up through the world of machine learning, which has a really specific, as you guys all know, has a really specific kind of culture and a kind of sense of what they're doing and what they're building is. I mean, this is a guy who is a graphics card salesman. That is his job. That is what he's been doing, you know, for 30 years. years now. And I'm sort of very sympathetic to a guy coming around, you know, if we can extend the car metaphor. He's like a guy who sells mufflers or whatever. And all of a sudden, Volkswagen's saying, you know, if we build cars, like they might all blow up on the highway and kill all of us. And he's like, well, you're stopping me. First of all, like I've been selling these mufflers forever and it's been I've gotten really rich off of it. It's been great. Why are you messing up a good thing? But also it's like they're just cars. Just build the cars. And I thought this was an interesting contrast. With the Bill Gates interview that Ezra did this week, which was another end of this sort of spectrum. You Speaker 2 (13:19) can't rely on the industry to self-regulate here. I mean, it's just insane. Speaker 6 (13:26) Gates is very pro-regulation. But the place where they both met up was there very much people who see AI as a tool that can be, that needs to be addressed as a tool instead of as a kind of a potential superintelligence, the, you know, the only sort of... The only way it can be dealt with is by properly is by like teaching it to be a good person, basically, instead of like, don't you think that