Andrej Karpathy: What if neural nets became the computer itself?
Summary
Andrej Karpathy imagines a future where neural networks are the host process and CPUs are just co-processors. A complete inversion of today. From his conversation with Stephanie Zhan at Sequoia Capit
Transcript Excerpt
What will look completely obvious in hindsight that is still mostly unbuilt today? So, you could imagine completely neural computers in a certain sense. I kind of feel like in the early days of computing actually, people were a little bit confused as to whether computers would look like calculators or computers would look like neural nets. And in '50s and '60s it was not really obvious which way would go. And of course we went down the calculator path and ended up building classical computing. And then neural nets are currently running virtualized on existing computers. But you could imagine, I think that a lot of this will flip and that the neural net becomes kind of like the host process and [music] the CPUs become kind of like the co-processor. So, we saw the diagram of, you know, of ne…