1580: The AI Lie Detector: How to Know When Your AI is Lying to You w/ Vanja Todorovic

Wantrepreneur to Entrepreneur | Start and Grow Your Own Business

Vanja Todorovic shares his journey from a high-stakes career in mortgage compliance at Wells Fargo to founding MelMat, an AI research platfo

Key takeaways

• Vanja Todor::Todorovic's background in financial regulation at Wells Fargo instilled a necessity for absolute factual accuracy.
• AI models are programmed to provide answers even when they lack the data, leading to well-disguised 'hallucinations'.

Main topics

• AI Hallucinations
• Multi-engine AI Research

Notable quotes

It's really not [intelligence]. Further down at the base of it, it's actually, ones and zeros.

Conclusion

The episode serves as a cautionary tale for AI users, highlighting the

Transcript preview

Speaker 2 (0:00) Hey, what is up? Welcome to this episode of the Wantrepreneur to Entrepreneur podcast. As always, I'm your host, Brian LoFermento, and there is one giant question that I don't think societally we figured out yet in the AI era, and that is... How do we know if and when AI is telling us the truth? Well, that is why I'm so excited about today's episode because I'm convinced that we have the forefront innovator in this space answering that very important question. I'm so excited for all of us to learn from him. Let me introduce you to Vanya Todorovic. Vanya is the founder of Melmat, which is a patent-pending multi-engine AI research platform. He spent... 25 years in financial services, including as a VP of mortgage compliance at Wells Fargo, where being wrong on a fact literally had regulatory consequences. He holds a master's in executive leadership from Drake University, completed MIT's AI and machine learning professional program in John Hopkins, AI and healthcare. And Mel Matt is part of Tampa's Embark Collective. So Vanya is local to me, which gets me excited. And alongside Mel Matt, he runs several ventures at once. He's the founding architect of Amplify, a volleyball performance program, co-authoring a course for parents of competitive youth athletes with a sports psychologist and consulting across financial services. So when we talk about balancing all the things that we have to do as entrepreneurs this guy is the real deal and on top of that he's one of my long lost Balkan cousins because he's also from that region of the world he speaks English, German, Serbian, Croatian and Bosnian this is someone who's very brilliant and very talented so I'm excited about it let's dive straight into my interview with Vanya Todorovic. Vanya I am Speaker 1 (1:48) so very excited that you're here with us today first things first welcome to the show. Thank you very much for having me. Glad to be here. Heck Speaker 2 (1:55) yes. I always wonder if all other cultures, you know, everyone's our cousin when we're from the Bahamas regions. But Vanya, I feel like instant connection with you. I'm so excited to dive into your story. Let's kick things off. Take us beyond the bio. Who's Vanya? How'd you start doing these cool things? Oh, Vanya. Speaker 1 (2:12) Who is Vanya? Vanya is an immigrant who came to the United States with big dreams. And some of these dreams are coming to fruition now. So I appreciate the chance and I appreciate you having me on the show here. I came to the United States in 99, having lived in Germany before that and former Yugoslavia before that. So I have a kind of a wealth of inputs as it pertains to my upbringing. Speaker 1 (2:39) probably my accent resonates as well. You know, I sound like, sometimes I sound Eastern European, sometimes I sound English and sometimes very German. So, but having had a chance to come to the United States and then realizing some of these dreams, I got the opportunity to work in AI and I kind of surprised myself with the ability to build. and what I'm able to accomplish. So at my late age here, at late stage in life, I was able to kind of figure out what, you know, the constraints I placed on myself throughout my life were self-imposed and that I'm able to so much more. So that's how Melmet came to fruition, basically. Yeah. Yeah. Speaker 2 (3:25) I love that overview. I'm going to put you totally on the spot to also introduce listeners to Melman because I think it is brilliant what you've done there because as you heard me tease at the very top of today's episode, I don't think... societally, we figured out how to make sense of what AI is telling us. A lot of people, we kind of have two sides of the coin. Some people either say, AI told me this, so this is gospel. On the flip side, I'm going to call out my parents' generation. My parents are just like, what does AI know? We have all this life experience. So how do we even begin to make sense of what's accurate and what's not? Speaker 1 (3:59) That is a really good question. And the societal issue in itself presents a barrier, but it also presents an opportunity, right? Like, how do you really figure out what an answer is a wrong one? A lot of people tend to kind of equate AI to intelligence because, you know, artificial intelligence in the name itself kind of indicates that, but it's really not. Further down at the base of it, it's actually, ones and zeros. And people tend to forget that, right? And it's that programming that really allows AI to kind of bring you these answers that keep you glued to the specific AI program that you're currently using. And no AI engine itself, a large language model is there also called, will tell you that, hey, I don't know the answer. because that's not how they're programmed. They want to give you an answer. And the problem that you people really run into is that if they're researching or if they're working with AI to figure out an area or topic that they're not really familiar with, the answers that they're going to receive back are going to sound and look like they're the right ones. And majority of the time, they will be the right ones. But if you're financially liable or if your reputation is on the line, You can't really 100 % rely on them because if they don't have the answer, they will make one up. And the way that they make one up is going to be really, really, really, really good, basically. So it's going to sound, it's going to look like, it's going to smell like, it's going to taste like the right answer. But, you know, it's not going to be one. And I run into issues where I was able to see and feel and kind of go through these. made up hallucinations and you will be surprised if you if you if you ever encountered them or if you ever uh had an opportunity to to to see one you you yeah you'll be surprised uh they're very really really well uh disguised let's say that way Speaker 2 (6:04) Yeah. And Vanya, I'm going to make the argument here early on in our conversation that not only is this the right solution and the right approach and the right technology and the right all of those, but I'm going to put you front and center and call you out as the right person to be bringing this in for two reasons in particular that I'm so excited for you to share with our listeners about part of your backstory. The first of which is that you come from a very successful career path where you couldn't afford to be wrong. There were real consequences for being wrong from a factual perspective. And then secondly, I love your startup story about asking what to charge. Please share with listeners both of those influences in the way that you've started Melma, because I think that they're equally important. And especially that second one for listeners, we've probably all experienced it from AI. Speaker 1 (6:50) Yes, sir. So the first one, the financial aspect of it, and I worked in a big corporation bank for 25 years. Speaker 1 (7:01) Part of that job consisted of going to court, actually defending decisions that were made back in time. So not only the decisions and what kind of stood behind it, but also trying to figure out the state of mind of the people who were in at that time. And having defended some of these decisions and having been in the front lines and having been part of a reputational and financial... Speaker 1 (7:30) consequences that they carried with it kind of given me a different and unique perspective of how do I approach issues and problems and how do I really approach really the solutions that come along with it. The consequences that kind of follow are not good ones when you do make a decision and the decision is based on wrong facts because you cannot really stand behind it, especially in the future. And if you're if you're doing your research, if you're doing anything that kind of involves a result, you want to have all the facts that are currently available to you right now. So the second aspect of MelMath and how I really came up with the product itself is that having left