Andrej Karpathy on one of AI's weirdest flaws: the car wash problem
Summary
Andrej Karpathy on the "jagged frontier" and why the same model that finds zero-day vulnerabilities cannot figure out basic common sense. From his conversation with Stephanie Zhan at Sequoia Capital'
Related topics
Transcript Excerpt
So, I think the reason I wrote about verifiability is I'm trying to understand why these things are so jagged. And some of it has to do with how the labs train the models, but I think some of it also has to do with the focus of the labs and what they happen to put into the data distribution. >> [music] >> Because some things basically are significantly more valuable in economy and end up trading more environments because the labs wanted to work in those settings. So, I think code is a good example of that. There's probably lots of verifiable environments that you could think about that happen not to make it into the mix because they're just not that useful to have the capability around. The favorite example for a while was that how many layers are in strawberry? And the models would famous…
More from Sequoia Capital
- Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI
- Databricks’ Ali Ghodsi Never Wanted to Be CEO. Now He’s Among the Best
- Rich Sutton: synthetic data is "just a big mistake"
- Parallel’s Parag Agrawal: Building a New Web for AI Agents
- Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again