Jev: 8 real use cases for the fastest, cheapest model I’ve ever used | John Lindquist

How I AI

The episode explores the capabilities of Jev, a new high-speed, low-cost decision model from TypeSafe AI. Hosts Claire Veaux and John Lindqu

Key takeaways

• Jev acts as a decision engine that outputs structured, type-safe data instead of unstructured text.
• The model is exceptionally cost-effective, with the hosts noting processing 5GB of JSON for roughly 40 cents.

Main topics

• Jev decision engine architecture
• Real-time voice to-do applications

Notable quotes

Jev is a decision engine, not a chatbot.

Conclusion

Jev represents a shift toward using specialized, low-latency models to

Transcript preview

Speaker 1 (0:00) In some ways, managers are really at risk. This concept of like org design and role design, you can put those skills to use when crafting your agents, which is why I am currently running, no joke, 40 Groffbots right now. Speaker 2 (0:13) Whenever you have constrained inputs that are interacting with apps, it's, Jev is a good thing to reach for. This one is a favorite of mine because it represents... Like a workplace where you have multiple agents that could all have their own tasks and avoiding collisions so that they never end up on the same space. At each step, it's going to be able to control these different agents and make sure that they get to their specific task as quickly as possible and that they never collide or interfere with the other agents. This one's really fun to me because it's a presentation coach. I click the microphone, I'd start talking. I'd have a list of bullet points in either a podcast or something that I want to make sure I cover or an interview or anything. It Speaker 1 (0:51) listens through everything I say Speaker 2 (0:52) and then starts checking the boxes to make sure that I covered everything. Speaker 1 (0:56) This model, which is basically free and incredibly fast, allows you to do discovery over data in a way that feels like it's opening up my opportunities and allowing me to look at things that I thought weren't high ROI before. Speaker 1 (1:15) Welcome back to How I AI. I'm Claire Veaux, product leader and AI obsessive here on a mission to help you build better with these new tools. Today, I have repeat guest John Lindquist, who has done one of our most popular episodes to date on the podcast. And here he is joining us on Jev Week at How I AI to talk about our favorite new model and all the amazing product facing use cases that he's discovered testing Jev from TypeSafe AI. Let's get to it. John, welcome back to How I AI. I have to say this is Jev Week on How I AI. And the reason it's Jev Week is a lot of models have come out in the past seven days. You and I do a lot of AI things. And yet, when we're DMing back and forth, we're talking about one thing and one thing only, which is Jev. You know, folks have maybe listened to my episode earlier this week about why I'm hyped on Jev. I would love to hear why you're so excited. You know, I'm looking up here. You have 23 demos we could probably look at. So you are going deep Jev as well. What about this model, this framework, this way of thinking about building has gotten you excited? Speaker 2 (2:31) Fast and free are both amazing. If you've ever been tired. Speaker 2 (2:37) sending a basic request to an LLM and waiting around for a bit just to have it call a function. This is now essentially instant. And I think of all the time I've spent on this and the hundreds of demos I made, I think I've spent 73 cents on it. And I love the one quote someone said, I think I'm going to pass this inheritance of Javon to my children once I die. Speaker 1 (3:00) Yeah, I like the one where it was like... TypeSafe is really taking off. They have to have made double digits in revenue by now. Speaker 2 (3:08) Yes. Speaker 1 (3:09) So yes, fast and free. I also have had that experience where it ran over like, I don't know, 20,000 records. And it was like, I have charged you 0.4 cents. Speaker 2 (3:21) Right. And it exposes those scenarios where I thought, like, I had five gigabytes of JSON. I'm like, I had never passed this into an LLM. I just, like, I don't have that budget, on and on. I'm like, let's see what happens. And it was 40 cents, you know. And I wasted it on a really stupid thing, like trying to organize this or whatever. Speaker 2 (3:44) But five gigabytes of JSON is a lot of text. And it went through and did the whole thing in probably a couple of minutes. Speaker 1 (3:51) That's been my experience, too, which is I'm looking at corpuses of data that I would never just dump into an LLM, both from a cost perspective and honestly from a time perspective. It's like this is not worth my time to even see if something's here. But then this model, which is basically free. and incredibly fast allows you to do discovery over data in a way that feels like it's opening up my opportunities and allowing me to like look at things that I thought weren't high ROI before. Speaker 2 (4:27) Yeah, absolutely. Same. It's just opening doors that weren't closed before. So Speaker 1 (4:32) just a quick reminder for folks, Jev is not like the LLMs that you're used to and love. It does not output text. It outputs basically like decisions and scores and yes, no probabilities. And so you kind of put information in and then you get out very structured, type safe, very structured, limited set of options. Speaker 1 (4:54) the ways you can use this are almost unlimited. And so John's going to walk us through some of his favorite demos of how he's using it. Hopefully we'll get a combination of inspiration on where a real-time, fast, cheap model can build cool things and a little insight of like, what else is around this? Because what I found is I see a lot of these like shiny demos on the timeline. And then you're like, you're hiding your... you know, your embeddings and you're hiding like how this is actually working. And so I think if you can lift the veil for us on how some of this works and where it's Jev alone, where it's Jev plus, that would be awesome. Speaker 2 (5:29) Yeah, I would just to add to your definition of Jev, of it being type safe. I think of LLMs being unstructured to unstructured, where you put text in, you get text out. And this one is similar. You put in unstructured sentences and data. But then you get structured data out. And the power there is it's humans talking to machines. And they're the scenarios where you have functions and APIs and everything set up. But I want to communicate with them using unstructured data. Speaker 2 (6:03) We're so used to chatbots, and it's just not a chatbot anymore. You need services, you need functions, you need things you want to throw data at first, but you can talk to it however you want, which is incredible. Speaker 1 (6:14) Yep. Well, let's talk about talking to it. Speaker 2 (6:16) All right, yeah. So let's go ahead and try this out right away. So traditional to-do app, we have a list of to-dos. And the fascinating thing here, which I'll try right away, is I'll just say something like... Book dentist appointment. Remove. Buy oat milk. Complete. Review pull request. Low priority. Speaker 2 (6:45) Now the speed there was all done through Jev. It does a few passes of looking through the text where it ensures that it lines up. If there's any like dictation mistakes and you say it doesn't get oat milk quite right, it'll go through and see, does this quite match up with something or do I need to do a different, do I need to categorize this into something that exists like a confidence score? Like it's all a lot of confidence scores there. It matches it with the thing in the list, and then it matches it with the function that it's going to call, right, based on the way that the operation that you want to perform on top of that data. So it's multiple layers of... analyze the dictation itself to see if that's like a valid dictation, then analyze if it matches up with the task, and then analyze if it matches up with the operation you want to perform on the task. And it can do that all live. And it even, while you're dictating, it classifies if you've spoken to a point where it can take an action. So as the words come in, it's like, is now a time where I can, where I have enough data to make a function call, where I can reliably... keep on talking because you notice that entire time I was talking I never paused I never hit enter it was able to classify is this a moment where I have enough information to take action So it's just showing all these pieces of Jev fitting together to give you this real time, like give me kind of streaming data in, like unstructured streaming data in, and I'll start putting it in all the places it needs to go. I Speaker 1 (8:17) have a question for folks that just have no idea how to actually put this into place and want more of the tactics, which is, are those like sequential Jev calls? Are you like, yes, no, it's time to analyze. Yes, no, it's... This is a good transcript. Score, here's the top task that relates to it. And score, this is the top tool call that relates to it. You know, is this like, because people are, I'm sure, trying to put in their mental model, like a tool call where you get all that structured data back. Like, you know, how have you set this up so people can kind of understand how they chain these JEP tools together? Speaker 2 (8:55) My mental model is, remember back when we used to write code by hand. There was no conditions. If else, switch statements. All these things that were the moment of classification. And if you just think of a line-by-line program where things execute, you think of where do I need those conditions? And so as you're breaking the problem apart, whatever the thing is that you're building, just think of all the conditions, the if else, the moments where. It either needs to be put into this bucket or that bucket. And so as I was going through this, I built a lot of it backwards because I started with what I was building, marking it to do as either done or removing it or adding a new one and such. And I'm like, oh, wait, now I want the dictation to be able to select it. And so I just kind of built it backwards. And it was so easy with Opus 5.5 recently just to kind of wing it and like... take it different directions and go through scenarios and see the cases that were exposed. Because I had that final idea of, I just want to, if you've ever used a Siri or a Google Assistant or any of those, and you tried to manage like add to my to-do list and they're all terrible. And like, how can I build, like, does Jeff enable this to actually be good? And I think it does. Like it's one of the most common things like managing. You could integrate your calendar into this. You could integrate. Because they're all behind function calls, right? Or MCPs or however you want to build it out. You could integrate anything with this and just have this top level dictation live inference running. And it's something anyone could build cheap and free. Like it's amazing. Amazing. Okay. Speaker 1 (10:38) So you and I both did live voice apps. Mine was much more emo, which was just talking to the screen and getting a color and an inspirational quote. Yours is much more practical, which may be. reflects our different personalities. Let's see what other use cases you have. And what I like about what you're showing us is I talked a lot about like internal data analysis and clustering and all this stuff. You have such great inspiration for how you can use this inside products or to do things that were previously really tedious. So I'd love to see another one of your favorite demos. Yeah, Speaker 2 (11:09) let's dive in. So this one kind of breaks it down into its smallest