Your Agent Can Now Train Models — Merve Noyan, Hugging Face
Summary
Open-source models now match closed-source performance—with three million models available on Hugging Face Hub, developers can build agentic AI systems locally with full privacy control. Vision LMs are becoming day-zero releases, making it trivial to run sophisticated agents with just a few lines of code.
Key Takeaways
- Open models like GLM 5.1 are now outperforming closed alternatives on benchmarks like SWE-bench Pro and AIME. Use Hugging Face's benchmark datasets feature to compare model performance before deployment.
- Vision Language Models (VLMs) can function as computer-use agents—they understand screenshots and know where to click. Expect all major LLM releases to ship with vision capabilities day zero going forward.
- Deploy models to edge devices and browsers without data leaving your infrastructure. This guarantees end-user privacy and protects against cloud performance degradation or security breaches.
- Agents can now train models directly via skills—just tell your agent 'train Q1 3.5 on this dataset' and it executes the full training pipeline. This enables zero-friction fine-tuning workflows.
- Use Hugging Face inference providers for routing queries to optimal providers (fastest/cheapest). The tool-use column lets you select open-source models specifically for agentic workflows.
Topics
- Vision Language Models (VLMs) as Agents
- Open-Source Model Benchmarking
- Edge Deployment & Privacy
- Agent-Driven Model Training
- Hugging Face Hub Agentic Ecosystem
Transcript Excerpt
Hello everyone and welcome to this talk in open agent uh ecosystem and uh I would like to call it having an AI engineer at your fingertips. Um I'm Marvia and I work in the open source team of hugging face. How many of you are hugging using hugging face on daily basis? Oh, let's change that. This is not okay. Um but first let's talk a bit about open source and what it is. So when it comes to machine learning, open source is absolutely differential. Basically you have the open weight models um that go in with non-commercial licenses. we call them open weight and then we have open source models that have uh commercially available licenses such as this one from deepseek it's called MIT license or Apache 2.0 And then there is like even more open models that have the code open. If you have like …