How Anthropic builds products like Claude Code before the AI models are ready | Dianne Penn

By Lenny's Podcast

Categories: Product, Startup, VC

Summary

Anthropic's head of product reveals how to build AI products before models are ready: ship products alongside model improvements in tight feedback loops, treat 'evals as the new PRDs,' and optimize for token efficiency like you optimize pixels. This flywheel of product-model co-development helped Claude Code accelerate Opus 3.5 adoption exponentially.

Key Takeaways

  1. Use 'evals as the new PRDs'—make model evaluations the primary product requirements document. This shifts product thinking from feature specs to capability benchmarks, letting you define what 'good' looks like before building UX around it.
  2. Build the product experience in parallel with model development, not after. Claude Code wouldn't have driven Opus 3.5 adoption without tight co-development; the product unlocked the model's potential and vice versa—a true flywheel.
  3. Optimize token efficiency as obsessively as pixel efficiency. Spending $100K/year in tokens today means living like a 2028 user—use this constraint to drive better ideas and features, not just to cut costs.
  4. Anticipate future models when building today. Ask: 'What changes when Claude 8 arrives?' This future-proofing mindset prevents building features tied to current model limitations and keeps architecture flexible for capability leaps.
  5. Prioritize user feedback loops over traditional roadmaps. For AI products, real user signal on what's possible beats speculative feature lists—driving value means identifying the right user feedback at each stage.

Related topics

Transcript Excerpt

In 2023 when I started, nobody said anthropic and claude and coding in the same sentence. >> I want to go back to the beginning of anthropic. I remember dealing, man, these guys have no chance. OpenAI is so far ahead. >> At the time, I saw people were starting to use these models not just for code autocomplete, but actually writing long form code and [music] sat an opportunity for us to train Opus 3 to be better at. That was the inflection. [music] I always think about Opus 45 a year later during winter break when everyone was home able to code. >> What was magical about Opus 45 is we also now not just had a model but a vehicle a great product experience like cloud code. Opus 45 wouldn't have had that moment without a product like cloud code and cloud code wouldn't have had that type of ad…

More from Lenny's Podcast