OpenAI Says “AGI” Is Here — What Does That Actually Mean?

Prof G Markets

Ed Elson hosts a discussion on OpenAI's Astra model and the claim that Artificial General Intelligence (AGI) has arrived, featuring Gary Mar

Key takeaways

• OpenAI's Astra model represents a significant technical advancement but does not meet the true definition of AGI as historically understood.
• The term 'AGI' has been diluted for marketing purposes, with companies using it to describe systems that perform well on benchmarks but fail in real-world applications.

Main topics

• OpenAI's Astra model and claims of AGI arrival
• The evolving definition and misuse of the term 'AGI'

Notable quotes

"If this were really AGI, well... capitalism and wages remain intact, which would imply one of two things. Either the industry's predictions about AGI were wildly wrong, or this isn't really AGI."

Conclusion

While OpenAI's Astra model marks a milestone in AI development, the claim that AGI has arrived is premature and

Transcript preview

Speaker 5 (0:01) I'm Ina Garten. On my new podcast, Happy Hour with Ina, I'm inviting really interesting guests to join me for a drink and a fun conversation at my kitchen table in New York City. I'll be getting personal with chefs, actors, comedians, musicians, and writers I admire. So grab a snack, pull up a chair, and join us. Subscribe to Happy Hour with Ina on YouTube or wherever you get your podcasts. New episodes will be available every Wednesday starting September 16th. Speaker 5 (0:38) Megan Rapinoe here. This week on Why Are You Like This, I am talking with Roxane Gay. That's right, Roxane Gay. Roxane is a fearless writer and cultural critic whose impact goes way beyond her New York Times bestselling books. We dig into all the big moments of her life. And of course, she shares if she's more mommy, daddy, or baby. Check out the latest episode of Why Are You Like This wherever you get your podcasts and on YouTube. Speaker 1 (1:05) What is a life worth? Calculating dollars isn't the tough part. The tough part is the emotion and the stories you hear about life's unfairness. I'm Preet Bharara, and this week, Ken Feinberg, one of the nation's foremost mediators and special master of the federal 9-11 Victim Compensation Fund, joins me to discuss his reflections on the 25th anniversary of 9-11. The episode is out now. Search and follow Stay Tuned with Preet wherever you get your podcasts. Speaker 3 (1:43) If money is evil, then that building is hell. The Speaker 1 (1:47) show goes on! Speaker 7 (1:53) Welcome to Profiteer Markets. I'm Ed Elson. It is September 8th. Let's check in on yesterday's market vitals. U.S. markets were closed for Labor Day, but stock futures fell as the U.S. and Iran escalated their attacks. Brent crude climbed towards $98 per barrel. Copper hit an all-time high on the prospect that Trump will expand his tariffs on the metal. And finally, the Japanese yen rose to its highest level since February as the dollar fell. Okay, what else is happening? OpenAI's newest and most powerful model is officially here, GPT-6 Astra. Released on Friday, Astra was trained on more than 100,000 NVIDIA GPUs at the Stargate data center in Speaker 2 (2:40) Texas. That makes it the largest training run in OpenAI's history, and it costs two and a half times the cost of OpenAI's previous model. Astra represents a large advance in capabilities Speaker 7 (2:52) for OpenAI, and across the AI industry, leaders have been celebrating. In response to the model, NVIDIA CEO Jensen Huang said that Artificial General Intelligence, or AGI, has, quote, arrived. Meanwhile, OpenAI's president, Greg Brockman, said, quote, welcome to the AGI era, describing Astra as a generational leap. Now, if Astra is indeed AGI, then that would be a very big deal, as many AI researchers have specifically cited AGI as the milestone that might change society forever. Sam Altman, for example, once said that AGI could, quote, break capitalism. Meanwhile, Stuart Russell, a prominent AI scientist, has said that when we achieve AGI, human wages will, quote, go to zero. Well, AGI has supposedly arrived, but... capitalism and wages remain intact, which would imply one of two things. Either the industry's predictions about AGI were wildly wrong, or this isn't really AGI. Here to give us his view, we are speaking with Gary Marcus, author, scientist, entrepreneur, and emeritus professor of psychology and neuroscience at NYU. Gary, thank you so much for joining us on the show. I'd like to just start with your reactions to the model itself, Astra. What do you make of it? Can you describe some of its capabilities? And then we will get into whether or not it is artificial general Speaker 3 (4:20) intelligence, AGI. I mean, in terms of the model itself, it's a bit better than the models before. What we have now is an era where some people call it benchmark maxing or bench maxing, where these systems are often it would appear trained on lots of benchmarks they do really well on the benchmarks and then they don't typically do as well in the real world so every time we see the phenomena the same phenomenon model comes out everybody's super excited about it and then after a few days people are like yeah but i tried it and it doesn't really work on this it doesn't really work on that it deleted by files the code it writes is for um the code is right is for is something i saw about astra yesterday on twitter You know, some people will like it, some people won't. But every model is hyped as if it's this quantum leap forward. And it never turns out to be true. You know, each new model is better than the last one. There's no doubt about that. Astra is better than Fable, its competitor on a bunch of measures. In real-world practice, it's better on some and not others, you know, based on people's different accounts. It used to be when a new model came out, I would personally test a lot of things about it. But now the scope of what people want to do with them is so vast, no one person can do that. And you sort of have to see what the world's experience with it is. And what the world's experience has been with all prior models is they don't live up to expectations. We've talked about this before. You know, the return on investment isn't there. Certainly wages haven't dropped to zero. Instead, we have what I call the Klarna effect, which is Klarna said they were going to get rid of a bunch of humans, and then they quietly walked that back and started rehiring humans. We've now seen that dozens of times. We may see a version of the Klarna effect with Astro, where a bunch of people say we're going to lay off a bunch of people and then quietly rehire them. So going back to your opening question, right, if this were really AGI, well... Let's put an asterisk around that. What do you mean by AGI? We'll come back to that in a second. Then, yeah, I guess you would expect wages to drop to zero. I mean, assuming that the AI itself were cheap enough and we could argue about that. The thing is that there's what I would call the AGI bait and switch, right? There's a way that the term was originally initially defined, which was basically that you could do any cognitive work that a human could do. And that would indeed... Assuming the price was right, it'd be pretty valuable. I mean, even if it cost just as much as humans, maybe it would still be valuable because you could make them work all the time with less requirements and so forth. So debate and switch, and then people will panic about that in various ways, like we're going to lose all our jobs or they're going to take over humanity or whatever. All of these are premised on, let's call it, a generous version of AGI. They really could do everything that people do. And then there's a less generous version of AGI. Like the least generous version is something like it can do most of the things that the drunk guy at the bar can do when he's human. So let's just call that AGI, right? So you have a sort of lower bound and an upper bound. Maybe that's being a little bit crude about it, but there's the kind of like, it's verging on superintelligence. Oh my God, there's nothing I as an individual human being can do anymore that AI can't. And there's the version where I can do a bunch of the things that I can do, but I can't really do all of them. And it does a few things better than me, but I do a few better. And I can't really trust it, but I just want it for marketing purposes, call it AGI. That's what we have now, right? Is the marketing version of AGI. There's a historical literature. And you can think about these things in different ways. There's a historical literature about what AGI meant when people like Ben Goertzel and Peter Voss first coined the term, and Jane Legg, who is a co-founder of DeepMind. And they were talking about the strong version that I was talking about, that it could do anything. And then you have... Different people have tried to weaken it in different ways. People accuse me of goalpost shifting. I'm always like, which goalpost did I shift? And they always leave the conversation that I can't actually document the claims, even though I write all the time. But I can document how this goalpost has shifted, right? Because I can go and show you where people talked about originally, or I can give you my own goalpost. For example, I said, it should be able to watch a movie and understand what's going on. Nobody's ever showed me that like Astra can watch a movie and tell me what's going on as well as a high school student. Right. That was my, you know, one of my targets examples in 2014. It's 12 years later. We can't do that. I haven't shifted the goalposts. So, you know, one way to think about it is historically. Has it met those historical things? No. Another way to think about it is how has the history changed? Well, after an original period. all the way up until 2023, when people pretty much agreed that it meant like doing whatever people could do. In 2024, about when I see it starting to happen, people started talking about it economically. So I know Vinod Khosla was interested in this. He was, I guess, on the board of OpenAI or at least he was an investor in OpenAI. And I think he pushed for definition, which is like can do 80 % of the work that humans can do. We haven't met that one either, right? It can do maybe a bunch of tasks that people can do, but doing a whole job is harder. Eric Brynjolfsson has always made the distinction between doing tasks and doing jobs. There are lots of tasks that current AI can do. I don't doubt that. And maybe some of them even reliably. You know, certainly it's not really a task, but it can write, you know, rhymes or whatever, make a song about such and such. That's not really somebody's job, right? do that, but it's not a job, right? A job typically involves a lot of things. A different way to think about this is, like, what is it doing for you? You know, if you have a term, that term does something for you, let's say, scientifically, right? So if you've got a term like mass, that's in the context of a theory, and now you can, you know, make some calculations relative to