Andrej Karpathy on one of AI's weirdest flaws: the car wash problem
Summary
Andrej Karpathy on the "jagged frontier" and why the same model that finds zero-day vulnerabilities cannot figure out basic common sense. From his conversation with Stephanie Zhan at Sequoia Capital'
Transcript Excerpt
So, I think the reason I wrote about verifiability is I'm trying to understand why these things are so jagged. And some of it has to do with how the labs train the models, but I think some of it also has to do with the focus of the labs and what they happen to put into the data distribution. >> [music] >> Because some things basically are significantly more valuable in economy and end up trading more environments because the labs wanted to work in those settings. So, I think code is a good example of that. There's probably lots of verifiable environments that you could think about that happen not to make it into the mix because they're just not that useful to have the capability around. The favorite example for a while was that how many layers are in strawberry? And the models would famous…