Your Old Prompts Are Making Claude Worse - Lance Martin, Anthropic

By ai.engineer

Categories: AI, Tools

Summary

Your Claude prompts are likely costing you 10-20% more than necessary. Lance Martin from Anthropic reveals that older prompt patterns like 'think step-by-step' and verification rituals now hurt frontier models' performance and token efficiency—simply running a prompt audit tool can simultaneously reduce costs and improve accuracy.

Key Takeaways

  1. Remove legacy prompt patterns like 'think step-by-step,' verification rituals, and 'be maximally thorough' boosters from Claude prompts. Frontier models naturally think and verify without explicit instructions, so these patterns increase token consumption and can reduce performance.
  2. Run 'checkup prompt audit' or 'Claude API prompt audit' commands to automatically detect and remove anti-patterns. Lance's evaluation showed this single fix improved performance while reducing costs on Opus 3.5 migrations.
  3. Frontier models get confused by contradictory rules in prompts, unlike earlier models that would simply ignore one rule. Audit and resolve conflicting instructions to prevent performance degradation.
  4. Implement prompt caching for multi-turn applications. Cached tokens cost roughly 1/10th the price of base tokens and process faster, making it critical for agent-based workflows with repeated context across turns.
  5. Prompts accumulate technical debt like code does. Migrate old patterns systematically when upgrading models rather than assuming existing prompts remain optimal with new frontier models.

Related topics

Transcript Excerpt

We can kick it off. Thank you all for coming. I'm Lance. I'm member of technical staff at Anthropic. And I actually want to share some very simple things with you today. A few simple ways to get the most out of frontier models. So we typically think of cost and quality as a trade-off. So in order to get, you know, better performance, you have to spend more money. And in order to kind of reduce cost, you have to scratch sacrifice performance. And so you can kind of think about it on any given evaluation, you have models that kind of create this Pareto frontier. And you can imagine this case, Fable is kind of up on the far right of the curve, and then you have Opus, you have Sonnet, then you have Haiku. Now what I want to share is actually that many times you're not actually on this frontier…

More from ai.engineer