Robot-Use Agents: Why General-Purpose Models May Win in Robotics
Y Combinator Startup Podcast
The episode explores the transition into the era of 'robot-use agents,' where general-purpose large language models are applied to robotic c
Key takeaways
- The RT2 paper established Vision Language Action (VLA) models by using pre-trained language models to output end-effector poses.
- Waddle Labs focuses on building a harness and collecting data to train LLMs specifically for robotic control tasks.
Main topics
- Robot-use agents
- Vision Language Action (VLA) models
Notable quotes
We may be entering the era of robot use agents.
Conclusion
The episode demonstrates how the scaling laws observed in LLMs are dir
Transcript preview
Speaker 2 (0:00) One of the big surprises the last few years has been the generalizability of coding agents across different domains. And now, frontier researchers are showing that this includes controlling robots. This led MIT professor Philip Isola to suggest in a recent viral essay that we may be entering the era of robot use agents, where general purpose models could make different robots more capable. So today, Francois and I invited the founders of Waddle Labs and RoboCurve, two groups of startups that are working at the very frontier of making robots more capable with LLMs. Speaker 2 (0:35) Maybe you guys want to briefly introduce yourselves and just say a little bit about what each of your companies focuses on. I'm Hanmei. I'm from Waddle Labs together with Vincent. Speaker 4 (0:42) We work on building LLMs that control robots. And we do this by doing two things. Building a harness that allows the LLMs to do this very effectively. Collecting data and then using that data to train better LLMs. Speaker 6 (0:54) Hi, I'm Jay, co-founder of RoboCurve. We are an evals company for physical AI. We measure everything, any robot, any model. including LLMs and also classical approaches such as VLAs, Vision Language Action Models, and World Action Models. And we evaluate all kinds of embodiments, like hands, grippers, arms, humanoids, quadrupeds, all kinds of stuff. Speaker 2 (1:20) So in the last few weeks, videos from both of you guys went quite viral on Twitter, and I think inspired. I think both of them are actually quoted in that Philip Isola essay talking about robot eustachians. The videos showed things like LLMs being able to unscrew caps, being able to communicate between multiple robots, being able to do tasks like uncapping a pen, for example. And so I thought maybe what would be fun is for Francois and I to dig in with you guys on some of the research that led to this moment to begin. And then maybe we can also show some demonstrations of what this can actually do. To start, why don't we talk about some of the early research around transfer between language models and robot control. So I want to talk about... Some of the early work on pre-training language models for decision making, and then also the RT2 paper, which established the VLA. So maybe Jay, do you want to tell us a little bit about some of this early work involving being able to use language models to do any kind of robot control? Speaker 6 (2:11) One of the earliest successful approaches of using AI on robots is the RT2 paper, where they use a pre-trained language model on web, text, and images, and use that to control. robots. And the interesting thing about that is that it's a fine-tuned version of a language model. And instead of outputting English, for example, they just output what we call an Nth effector pose, which is the coordinates that you can then translate into joint commands that can control the robots. In some cases, it's in some sense, it's very similar to what we see now with large language models, just that instead of fine tuning, The models are good enough to just do it out of the box. And Speaker 3 (2:58) this is actually quite similar to like the mapping that I have in my head is kind of like the COT moment. And so like when we did GSM 8K and you said Susie had $8, she spent five. How much does she have now? It had to output like four hash marks and then the answer and then EOS. There was no ability for it to chain of thought. And so then we allowed it to actually have like, okay, let me go. 8 minus 5 is 3, and so like, hash, hash, hash, hash, 3. And so you allowed it to do this kind of thinking before it actually gave an action or an answer. And similarly now, VLAs were basically like, RT2 is basically like, it has to output an action. I can't allocate more compute for a more complex task. And then now I have this code chain of thought thing that I can do. And I can say, OK, even if you were using the example where you're not actually outputting the code, you're just giving it to Astra and let it think, think, think, think, think, and then output an action. Similarly now, like, and then with code, it's a more Kolmogorov complexity or lower Kolmogorov complexity code length to just use code and really compact code to just like, here's Speaker 6 (4:06) the Speaker 3 (4:06) code, just go. Speaker 6 (4:07) I totally agree with you on that. I feel like at the end of the day, it's a lot about the bitter lesson, right? If you give the agents or you give the AI model more autonomy and you, if you unshackle it a bit more. and give it more resources. It can actually do a lot of the things that we fine tune it to do. Speaker 5 (4:25) I'm just going to hop in here as well. I think the really interesting thing about the Bitsa lesson here is that, like VLAs by design, architecturally, they're built on top of language models as well, right? They can potentially reason, they can write code. So perhaps the Bitsa lesson here isn't necessarily what architecture you build necessarily, but it's what data is most useful, right? Like people have been sort of nagging at this. VLA data bottleneck for years now and we're seeing very very slow progress and so the lesson here is maybe we have a modality of data that we know works very well and this idea of transferring across different modalities which Ham and I are super excited about like how do you take something that's traditionally out of distribution like robotics and make it something that's in distribution like maybe the way of harnessing the beta lesson is saying let's pick a data for which we know this modality is pretty blessed there's a lot of data these language what these models understand it well and then use this as a way of unlocking a lot of other domains as well. Speaker 6 (5:21) Yeah. And to add to that, the reason why RT2 is such a good model is because it is using a language model that taps into the modality of all the data that a language model is trained on. So all those web images and web text actually improves the VLA compared to just training a specific robotics foundation model without any pre-training. Speaker 2 (5:45) So you're benefiting in the case of RT2, these early VLA approaches from pre-training. I guess, what's the limitation in the, I guess, action-taking fine-tuning approach that you can now get around if you can directly write code? Like you're referring to like a difference in data there. What exactly is that difference in data you're referring to? There's a few things that models have got much better Speaker 5 (6:07) at since RD2. Like one, I mean, they're much better tool use, for example. So, you know, one kind of data is now these models can, well, I mean, they write code much better. So now they can write complex policies as code. They're also much better using tools to explore the kinds of environments they're in, like what arms they have access to. And a lot of this comes Speaker 4 (6:27) from in-context learning. Like the key difference between like these VLAs and we're like GPT-6 or LLM isn't necessarily the architecture, but kind of the approach you take towards training. We want to be bitter lesson pill, right? We want to benefit from all kinds of data. We want to pour in computer, computer use data into our robot models. You want to put encoding data into our robot models, but when you do that, you just end up getting what's what we think of as like these general purpose LLM agents. And then, so to me, it's like, okay, why not just build a really good foundational LLM and use that to control robots rather than training like a model that's specifically for robotics and that's more dependent on just robot data. On Speaker 2 (7:05) that note, maybe one of the things that I think, um, if we could rewind the clock a couple of years may have been a good sign that we were trending in a good direction to this is research on code as policies, right? Do you guys want to talk a little bit about when the research community refers to code as policies, what exactly that means? And I think. It's worth putting this in the context of when this paper came out, which was coding agents just starting to work. I think this was still in the era in which people were like putting comments and auto filling, you know, Python code blocks, which, you know, now we think of as