How AI bosses 'lost control' and are now begging for regulation

The Andrew Neil Report

Andrew Neil and Professor Steve Keen discuss the growing alarm among AI leaders regarding the loss of control over large language models. Th

Key takeaways

• AI leaders like Sam Altman and Dario Amodei are calling for regulation because they believe control over technology is already being lost.
• Professor Keen argues that LLMs lack conceptual understanding, performing pattern recognition rather than true intelligence, which can lead to catastrophic errors in databases.

Main topics

• AI Regulation
• Large Language Models

Notable quotes

They have already lost control.

Conclusion

The episode concludes that without technical competence in governance

Transcript preview

Speaker 1 (0:00) AI bosses keep telling us they want their own technology to slow down. Why? This is the Andrew Newell Report. Speaker 2 (0:08) They can be incredibly destructive as well, and now they're beginning to realize just how destructive they can be. Speaker 1 (0:14) Have they already lost control? Speaker 2 (0:15) They have already lost control. Whether they can get control back is another huge problem. Who Speaker 1 (0:21) are they directing this at? I mean... Aren't they the gatekeepers themselves? I mean, if they want to slow things down, why don't they slow things down? Speaker 2 (0:30) This is what the AI, the tech bros are starting to realize. It's a bit like Frankenstein's monster. They've released the monster into the wild and they don't quite know what the monster is going to do next. And now they're getting worried about it. Speaker 1 (0:49) Welcome to the show. If you're just joining us for the first time, make sure you follow us in your podcast app so you never miss an episode and subscribe to the Andrew Newell Report on YouTube. Now, if you've kept up to date with any of the UN General Assembly in New York this week, you may have seen speeches from world leaders, of course, Donald Trump, Andy Burnham, Benjamin Netanyahu. But among all these political leaders, we heard from... two of the most important men on the planet, and they're not politicians, OpenAI boss Sam Altman and Anthropic CEO Dario Amodi. Now, they and others have been calling earlier this month for the development of artificial intelligence to slow down after claims it poses a risk to humanity. All this in the week that the two AI superpowers, that would be America and China, I've met in Washington with Mr. Trump hosting President Xi for a state dinner. So what's going on? Lots to make sense of here. It's a difficult field and a complicated one and a fast moving one. So I'm delighted to be able to welcome to the show, economist, honorary professor at University College London, an expert on AI and China, Professor Steve Keen. Professor, welcome back to the Andrew Neil Report. Speaker 2 (2:14) Thank you, Andrew. It's been enjoyable last time. I'm looking forward to today's conversation as well. Speaker 1 (2:20) Professor, something has happened to lead to this shift in tone from these AI leaders, but I don't know what it is. I've been thinking about the options. I mean, are they genuinely concerned about the pace of AI development as an event or events? Has something happened behind the scenes that we don't know about? We know there's not enough power in the U.S. grid system to support AI development, so are they kind of positioning themselves for a slowdown anyway? Does it suit them? Or are they easing investor fears about the mounting debt that they face, that development won't be so fast, the debt mountain may not be so big? I'm not quite sure which. Which do you think it is? Or maybe you've got another one I haven't mentioned. Speaker 2 (3:06) Yeah. The basic answer to all those questions, Andrew, is yes. But to actually give detail on what's actually going on, you need to have understanding of what the underlying technology behind so-called AI is. And they call them large language models, which is more realistic. And what they actually are is actually mathematical operations, which take in a set of inputs. They feed those inputs through a... array of numbers and then another array of numbers and a third array of numbers. So there's enormous amounts of numerical processing going on. And what they're basically doing is pattern recognition. And there are all sorts of consequences that flow from that. So if you, the original... thing out of which they developed called neural networks. And neural networks attempted to emulate what occurs in the human brain, which is where we have one set of neurons which will fire and cause another set of neurons to fire, another set of, et cetera. Others will cause one neurofarm will cause another one to suppress, et cetera, et cetera. And this was taken as an anagram of what the human mind actually does. And the idea was we've created a silicon version of the intelligence in our own brains. Speaker 2 (4:26) The understanding we have of the brain is still relatively limited. We still don't really know what intelligence itself is. So to call this artificial intelligence is itself a stretch. But what it means is you have a research assistant who can go out and scour the web and pull in all the information from the web. And therefore, what you get fundamentally is a distillation of what... Speaker 2 (4:51) not Keynes, John Kenneth Galbraith once called conventional wisdom. Now, that can be enlightening or it can be a mistake. And secondly, these systems, they do not have a residual of intelligence. So if you learn something, you'll know the French language, for example, then later on you've still got the knowledge of what French is and you can distinguish it. from English. These things learn a pattern, but they don't actually build the type of conceptual understanding humans have of a particular issue. So you can have an AI have a brilliant conversation with you, but then effectively learn nothing. They can't then apply that in a situation different to your original conversation. And one of the great dangers of what's happening now, and I've seen a few quite dramatic examples reported to me. by correspondents largely on Twitter, is that if they can strike a problem in interpreting, for example, a database of customer records at a service station management company, and they find that they can't reconcile one set of records with another set of records, they can make the decision, let's just delete the database. Speaker 1 (6:03) And I Speaker 2 (6:04) literally have seen someone to whom that has actually happened. So you entrust them with what looks like... something that an intelligent person should do, but have pattern recognition they don't have intelligence. And this can be a dramatic damage to the systems you entrust them to. And I think this is what the AI, the tech bros are starting to realize. It's a bit like Frankenstein's monster. They've released the monster into the wild and they don't quite know what the monster is going to do next. And now they're getting worried about it. Speaker 1 (6:34) Did they not see this coming, Professor? Speaker 2 (6:37) Largely not. I mean, this is a common factor. For example, I was talking with friends who were involved in the original design of the internet in the very early days of that being designed by, it came out of, not out of DARPA, out of a predecessor to the Advanced Research Projects organization that the American government has. And this is Tim Berners-Lee who put it together. And it was designed to avoid a nuclear war. So the idea was you wanted to have a distributed way of sending data. So if the bomb took out half the country, there'd be computers all over the other half that you can send a signal through. Now, they called it packet switching. And the packets give a destination. But because the people who put it together were all friends, they didn't include the origin package. Okay. Now, what that means is all the security hassles we have these days came out of the fact that this was designed by friends who weren't thinking about the long-term consequences. And now we have all the scamming. Is this a real person? Is this a bug? All this sort of stuff comes out of an initial simple mistake of not including the source as well as the... destination of each of those packets. So a similar thing is happening here. They're massively enthusiastic. It can do brilliant work. My designer of my Revell software uses AI to find out why a program that Ravel relied upon suddenly doesn't work anymore. And then what fix do we need to make? That's sort of houseclean. Speaker 2 (8:11) housekeeping type programming is extremely useful, but they can be incredibly destructive as well. And now they're beginning to realize just how destructive they can be when you give them authority over systems on which we all rely. For example, I'd hate to have what happened to this service station management company happen to a banking system. Speaker 1 (8:30) Yeah, no, I can understand that. And we've now got Altman and Modi. They're both talking in the different ways about the need for regulation to slow down development. I see that even Elon Musk chipped in earlier this month as well on the same theme. I'm certainly puzzled, though, that who are they directing this at? Aren't they the gatekeepers themselves? I mean, if they want to slow things down, why don't they slow things down? Speaker 2 (9:01) Well, partially because once they've set these things up, they can then self-develop. The basic, the way the data is stored in these things is huge square arrays of numbers called matrices. And they talk about having billions and up to trillions of what they call parameters. They're basically numbers in an array. And then the program itself will try to recognize something. It makes a mistake. And this is what's called, the original technology was called back propagation. So you get it to guess something. It makes a mistake. You then go back and change the weights and slowly it converges to the correct answer. That's how these things were built. So the parameters themselves are always changing and the systems themselves are changing. And once you've set the process in train, you can't stop it. And this is the problem. Have they already lost control? They have already lost control. Speaker 1 (9:57) And Speaker 2 (9:58) whether they can get control back is another huge problem. And one thing you would have heard a lot about is that they talk about AI hallucinations. So you'd ask for research on a particular academic topic, and the AI would come back with a whole sort of nicely formatted references, 80 % of which it made up itself. because it's just, again, it's recognising a pattern. Now, those hallucinations themselves are a sign that