From AGENTS.md to Enterprise Deployment

Practical AI

The episode explores the transition of AI agents from local prototypes to production-ready enterprise environments. Guest Nick Kuhn explains

Key takeaways

• Enterprise environments require handling strict compliance (PCI, HIPAA) and often operate in air-gapped or limited internet scenarios.
• The 'agents.md' concept allows for human-readable instructions that can be deployed via build packs similar to traditional Java apps.

Main topics

• Enterprise AI deployment
• Platform Engineering for Agents

Notable quotes

The enterprise... is a whole different ballgame where you could potentially do something that has dramatic consequences.

Conclusion

Deploying agents successfully requires leveraging established platform

Transcript preview

Speaker 1 (0:01) Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content and AI insights. You can learn more at practicalai.fm. Now, on to the show. Speaker 3 (0:41) Welcome to another episode of the Practical AI podcast. This is Daniel Whitenack. I am CEO at PredictionGuard and I'm joined as always by my co-host, Chris Benson, who is a principal AI and autonomy research engineer. How are you doing, Chris? Hey, doing great today. How's it going? It's going great. And I know this is going to be a great conversation for a few different reasons today. One of those being our guest is one of the speakers at the upcoming Midwest AI Summit, which is going to be amazing. There's going to be a bunch of amazing speakers there. I'm going to do a bit of emceeing, see if I don't mess that up. Would encourage our listeners to check that out. October 15th in Indianapolis. You can get 20 % off with Practical AI 20. So check that out. So amazing speaker, going to be at the Midwest AI Summit. Also a fellow podcaster, which is great. One of the hosts of Cloud Foundry Weekly and a tech marketing whiz. at Broadcom Tanzu VMware. So welcome, Nick. Great to have you. Speaker 2 (1:50) Thanks for having me. Great to be on the show. Yeah. Excited. Also excited to see all in person in Indianapolis. Speaker 3 (1:57) Yeah, it's going to be fun. We're representing the good representation of the Silicon Prairie here. I like it. So yeah, Nick. I know you're going to be talking. One of the things I thought was cool about your talk at the Midwest AI Summit is you bring in this concept of taking agents.md and shipping that to a production environment where you run an actual agent in an, quote, enterprise environment, which seems to be really fascinating. We recently had on... Someone from the Agentic AI Foundation who is kind of stewarding the model context protocol and agents.md and those sorts of things. So very relevant kind of carry on from that conversation. But I'm wondering as we set that up, what exactly does it mean to be operating in an enterprise environment? So the customers that you're working with, your kind of day to day. What makes an enterprise environment an enterprise environment? What are some of the concerns or characteristics of that type of environment? And I know you're always experimenting. You do a lot of home lab stuff as well. So always have a whole range of experience in working in different sorts of environments. So yeah, if you could just help us understand from a general perspective, what does that mean when you say? kind of enterprise. Speaker 2 (3:30) Right. So, yeah, if we think of the enterprise, we think about, you know, a few things. One, let's just assume you're going to have not internet, limited or no internet access. And, you know, I've been at VMartainzio for about five years. And before that, I was at one of these large enterprises for about 14 years. So, when we had vendors come in, it would always be like... they would always assume that we could just go to the internet. And it was always like, well, try again, because that's not going to work here. So that's kind of always like the biggest hurdle where you're going to be regulated and or, you know, you're not going to, what you can do at home is not going to be what you can do at work, so to speak. So you're going to be highly regulated. You're going to have to deal with, you know, potential like, you know, PCI or SOX compliance or HIPAA or... FIPS or, you know, and I could probably go on and on about all the different compliance tiers that you'd have to deal with. So it's just a whole different ballgame where, you know, you could potentially do something that has dramatic consequences. And you have a, you know, highly controlled, highly regulated environment where even some of the customers I work with are like, you know, they actually have the real air gap where we're carrying in things, you know, physically into the data center. because there is no internet access type of thing. So it's a wild world, and it frustrates a lot of, I think, new folks to the enterprise space. But if you've been in a while, you know how to deal with it and how to handle it type of thing. Speaker 3 (4:59) And I guess people have been dealing with these types of environments for quite a while, right? Because there have been enterprises for quite a while. Maybe just give us a sense of like, leading up to AI agents, and we'll talk about agents here in a second, but leading up to AI agents, like what were some of the ways that you could, I guess, ease, you know, put some ointment on those pains of working in that sort of environment? How would you get an application into that sort of environment or have it be enterprise ready, quote unquote? And then we can shift and talk about, you know, what changes with. with agents? Speaker 2 (5:42) Yeah. So, I mean, I think, so, you know, I currently work for VMware Tainzoo and we run a, you know, a platform or a platform as a service, you know, geared for private cloud, whether that be on your, like on your own bare metal in your data center, or you could deploy it on a private cloud and your own like VPC on a hyperscaler, that type of thing. But we think about that. you know, we want to in the scope of an enterprise, right, you have all these regulations, controls, etc, you want to make that path to production, kind of the least resistant and easiest path. And if you make that path to production, like all the checkboxes and everything good to go, that's going to be a highly adopted path. So the whole concept of, you know, a Tainzoo platform, and it's actually based on an open source project called Cloud Foundry, which is I think I mean, it's, you know, predates Kubernetes and Docker. So, you know, 2011-ish is when it kind of started originally out of VMware and spun out and then became part of an open source project. So it's been around and battle tested and seen a lot of things where you have that concept of like, I just want to take my code and send it to the platform. And the platform is going to know like the best practices for it's going to build a container for the developer. So the developer doesn't have to worry about like, you know, securing the container or anything like that. That platform will take care of that. It will handle ingress, certs, certificates, load balancers, all of those things just handle health, monitoring, potentially even kind of like sandboxing the apps from each other. Something that maybe some of these labs may learn from, so that the apps can't escape type of thing. Speaker 2 (7:18) you know, if the app needs services like a database or messaging, middleware, any type of thing, even maybe a large language model, it can what we call bind or it kind of connects to a service on the fly. And that's all kind of handled underneath the covers. So it's just a few commands like you push, bind, scale your app with just a few simple constructs. And that's how you go to production. And what we see with enterprises, once that path, like once they take this like... this is a Tainzoo platform. It's been certified through all these different standards. It's just kind of like an unlock for these enterprises because it's so easy and so simple to use and they don't have to, you know, they don't have to recertify everything. It's just like this framework that they can use to get apps into production. And then the biggest thing is that you want these, the enterprises or the businesses, they don't want their developers, you know, hand crafting a new platform or a new way to deploy apps for every sub team and have like 100 different snowflakes, right? They want the same repeatable pattern across the board so that they can audit it and secure it and have their developers actually spend time writing the business logic versus like fiddling with infrastructure. So that's kind of like the whole premise behind, you know, the whole platform as a service construct and, you know, it's Speaker 4 (8:31) a Speaker 2 (8:31) platform. And as you can imagine, go ahead. Yeah. Speaker 4 (8:34) No, no, go ahead. Finish up and then I'll follow up. All good. Speaker 2 (8:37) Okay. As you could imagine, those principles when we're talking about apps could probably play nicely into this agent. world that we're living in. Speaker 4 (8:45) So it sounds like you're drawing a lot of commonality between kind of traditional app development and all of the structure that we're all used to, that we've been doing forever. And kind of now we're into the Syngentic world and maybe people have been thinking about that in a different way, but it sounds like you've really kind of said, and correct me if I'm wrong, I'm going to... going to throw something out there. Like it's kind of the same thing in the sense of there are differences, but agents and apps should sort of be treated the same way. Is that, am I getting that correctly in terms of how you're structuring that and that maybe people need to look at what they've been doing for years and make that work for agents in the same way? Is that fair? Speaker 2 (9:33) Yeah, I think it's a pretty fair assessment. And we're starting to see that more and more just within our customer base. And, you know, even like if you think about like apps back in the day, like someone would write some app on their laptop and like run it on their laptop. But it's like, we can't just run it. You know, like you got to get it off your laptop somewhere. And the same thing with agents. You're like, I got this. I've got all these agents on my laptop. If I shut my lid, everything stops. Like, I don't really want that to happen. Right. I want them to run 24-7, you know, 365 and do all the work I give them. So, yeah, I think that's kind of precisely what we're going with. Speaker 3 (10:04) So I guess then it begs the question, why are people kind of throwing out what maybe their intuition, like in certain ways people are saying, oh, we have agents now we need to do something totally different, which is maybe driven by some things that make an agent not an app. not just a sort of simple application, but also it doesn't necessarily mean, you know, obviously we have to throw out everything we've learned about platform engineering and how to deploy things. So from your mindset, where does the kind of agent equals just another app? Where might that fall apart? Or what are the kind of unique characteristics that you're dealing with on the agentic side? on the deployment pathway that you might not have had to consider before or at least are considering in a new way. Speaker 2 (11:00) Well, I mean, I think some of the biggest differences, right, where like you talk about Cloud Foundry and Teensy platform, it was kind of based on this premise of a 12-factor application where like storage and state are a little, you know, decoupled cleanly from the actual apps. You could like, you know, I can scale up to a thousand instances and be fine and you handle session state and things. The traditional harnesses or agents, were kind of built originally just be like, I've got file system access and I can just write a bunch of MD files and that's like my memory. Everything's great. And you're like, well, you start to get into a cloud world that things spin up and down and are ephemeral. You're going to want to save those MD files somewhere. They're going to go off into the ether. So that's probably the, I think the biggest challenge when we start talking about how do we take the app deployment methodology, right? And then push that into like an agent running on a... kind of a cloud platform per se. It's probably the biggest challenge. And then just kind of getting the fact that there's some valid use cases for like your laptop or whatever, but then also as we're starting to see more and more, like enterprise was, enterprises want agents to be on demand and like usable kind of in a, like say in a CICD pipeline or like on, you know, within part of their, you know, e-commerce suite or just normal applications. And you want those to be kind of really close to the apps, right? Because if you're... If you have them way off, even just from a pure networking perspective, if you have the agent or the LLM or whatever that's like 30 hops