A New Approach To AI Models

By Y Combinator

Categories: VC, Startup

Summary

Cartisia is building state-space models as an alternative to transformers, arguing that compression and multimodal reasoning are fundamental to human-like intelligence. The company believes transformers' retrieval-oriented design limits long-context reasoning and abstraction, while SSMs offer a better path for truly intelligent AI systems.

Key Takeaways

1. Transformers are fundamentally retrieval machines that keep all context in raw form, while state-space models compress information into abstractions—representing a tradeoff between factual recall and abstract reasoning capabilities.
2. The next frontier in AI isn't incremental improvements on existing architectures, but fundamentally new designs that enable multimodal learning, long-context reasoning, and human-like intelligence over extended time scales.
3. Compression is a fundamental primitive for intelligence—humans consolidate multiple representations (audio, video, text) and use them interactively, something transformers struggle with due to architectural limitations.
4. Modern hybrid models combining transformers and SSMs are emerging (like Qwen) because neither extreme is optimal, suggesting the winning architecture will thoughtfully combine strengths of both approaches.
5. Founded by Stanford PhD researchers in 2022, Cartisia exemplifies how deep domain expertise in architecture research can lead to commercializable products solving real developer problems with voice applications.

Topics

• AI Architecture Research
• State-Space Models vs Transformers
• Multimodal AI Systems
• Model Compression and Abstraction
• Founding Research-Driven Companies

Transcript Excerpt

[music] I'm Ankit from YC. I'm here at Nurups with our YC and Arc Prize afterparty. I'm here with Karan Gaul, the CEO of Cartisia. And we're really excited to have you here. >> Great to be here. Thanks so much for having me. >> Why don't you start by telling us a little bit about what Cartisia is? Yeah. So, um, Cartisia is a two-year-old company. We are, um, formerly, um, researchers from Stanford where we did our PhDs. Um, and, uh, we worked on architecture research there. Uh, decided that it w...