Accelerate AI Workflows with Nemotron

Reshaping Workflows with Dell Pro Precision and NVIDIA RTX PRO GPUs

This episode explores the advancements in agentic AI through NVIDIA's Nemotron 3.5 Lightning and the NeMo Switchyard routing library. Host Lo

Key takeaways

Nemotron 3.5 Lightning is a 30B parameter MoE model that delivers up to 4x faster throughput compared to Gemma or Qwen in its class.
The 'system of models' strategy uses large models for complex reasoning and smaller models for efficient, specialized tasks like summarization.
CrowdStrike achieved 80% cost savings by using the Lightning model for specific cybersecurity tasks instead of larger Super models.
NeMo Switchyard serves as an open library that allows developers to set routing policies based on task requirements and hardware availability.
Open-weight models are critical for allowing researchers, startups, and enterprises to customize AI for domain-specific use cases.

Main topics

Nemotron 3.5 Lightning
Agentic AI systems
NeMo Switchyard routing

Notable quotes

When you think about an agentic AI, it really is built off a system of models.

Conclusion

The integration of efficient open-weight models with intelligent routi

Transcript preview

Speaker 1 (0:04) Welcome to Reshaping Workflows with Dell Pro Precision and NVIDIA, where innovation meets real-world impact in high-performance computing. Speaker 3 (0:20) Welcome back to another exciting episode of Reshaping Workflows with Dell Pro Precision and NVIDIA RTX Pro GPUs. I'm your host, Logan Lawler. At the time of the recording, we've got some interesting announcements and things that have happened around NVIDIA and Nemotron, and we have no one better here to talk about it than Chitton. So Chitton, tell everyone, well, one, welcome to the show, but two, tell everyone just a little bit about yourself, what your role is in NVIDIA, and then we'll jump right in. Speaker 4 (0:49) Really excited to be here. I'm a product marketing manager at NVIDIA working on Numotron models. Actually, Numotron is a family of open models as well as open data sets and open techniques. And so we recently launched Numotron 3.5 Lightning, so I would love to talk all about it, but I want to make sure that it obviously aligns with the Dell product offering that you have and how the Numotron models work really well on that platform. Speaker 3 (1:18) All right. So, Nemotron, you kind of took away my first question. So, it is a family of models. So, we're definitely talking about Lightning. But let's talk about kind of current state. You know, you have the ultra super nano. Like, tell us a little bit, high level, you know, where those run compute-wise. Are they data center? Are they cloud? Which ones can kind of run locally? Your version is the NVIDIA DGX Spark. Ours is the Dell Pro Max with GV10. and kind of the use case for those models, and then we'll get into Lightning. Speaker 4 (1:48) We have a bunch of different open model families, depending on the domain. Nemotron family is specific to agentic AI. And like you said, we have a nano, super, and I'll try the three T-shirt sizes, if you may. When you think about an agentic AI, it really is built off a system of models. You've got small, you've got large language models that are really good at... doing frontier class reasoning, advanced reasoning. They're really good at orchestration. You also need smaller models that are extremely efficient, extremely fast, and they deliver highest accuracy, but for very specific specialized tasks. And then the reason why you don't want to use just one model is because, you know, for something as simple as summarization, reaching out to a frontier class model is just wasting tokens. We end up using a lot more tokens than we really can afford to in many cases. And so with the three model sizes that the Nemotron family offers, it is to optimize the efficiency of a genetic AI. Speaker 3 (2:53) Makes total sense. So, you know, you made an important point, and I want to call it out, is there are frontier models, right? Which, think of those as the Anthropix. My favorite, personally, is Cloud Opus. And then there is, you know, kind of open weight model families that are kind of open source, which can be kind of US, they can be overseas, they come from a variety of places. So I know Jensen, as well as Michael Dell, kind of put out a letter or to the market talking about the importance of open weight models. So maybe tell me, because it's a great question, is that, you know, NVIDIA with the stick that you swing, you could have absolutely made Nemotron, all of them, a paid model. Why is it important for them to be open-weight models? Speaker 4 (3:36) NVIDIA truly believes in open source. We have been publishing, we've published hundreds of live open source libraries, right? And when it comes to AI, really it is unlocking a lot of opportunities across the board. So what open models give is that with open models, researchers have the ability to learn and iterate. Startups have the foundation to build on top of. And then you've got enterprises and they get the flexibility to adapt AI to their workloads for their domain-specific use cases and then increase efficiency and accuracy for their tasks. And so that's why open models is extremely important. And like you said, NVIDIA as well as Dell has signed the open models and American AI leadership letter for that very reason. We believe that it is not just about, like I said earlier, system of models. It's not just about frontier models and specialized models. You also need these frontier models that have all the knowledge that is available in the world, but you also need these open models that every single developer can customize for their use cases and get the most accuracy and efficiency out of that agentic system. Speaker 3 (4:48) Agreed. And, you know, you're