How to Build Your Own Data Center & Why Every Startup Should Do It

By 20VC

Categories: VC, Startup

Summary

Speechify CEO reveals why spending tens of millions on owned GPUs beats cloud rental by 1.5x annually—and shares the counterintuitive economics: buying a $30K H100 costs less than renting one for a year while enabling massive-scale model training competitors can't access.

Key Takeaways

  1. Owning GPUs vs. renting creates a 1.5x cost advantage: H100 rental costs $35K-$50K/year on AWS/Azure but only $30K to purchase, with 3-10 year lifespan making ownership the clear winner for training workloads.
  2. Remove friction in engineer behavior by building internal infrastructure: engineers became parsimonious with rented GPUs but unleashed creativity with owned hardware—the 'hoop in your house' framework eliminates cost hesitation.
  3. Data co-location with GPU clusters is impossible on cloud platforms: large-scale training requires gigantic memory cards next to GPU clusters accessing shared data simultaneously—a capability cloud providers cannot provide.
  4. Separate training from inference economics: use newest GPUs (A100s, H100s) for training where speed creates competitive advantage, but reuse older K80s for inference where 100ms delivery times work with lower-spec hardware.
  5. Open-source model inference on owned hardware costs a fraction of closed APIs: running open-source models internally costs 'a fraction of a fraction of a cent per token' vs. paying per-token to Anthropic or OpenAI.

Related topics

Transcript Excerpt

You said 11 raps kind of leaprogged you. >> Is that on you? >> Yeah, 100% it's on me. It's the biggest strategic mistake I made in the history of Speedify. >> How do you reflect on that? >> So today is a real freaking discussion. Cliff Whitesman, founder and CEO at Speechify, one of the fastest growing text to speech startups in the world on the show. >> The best way to lose is not to be in the race. Be in the race. You don't want to be a fat manager who is like a general sitting in the back saying, "Take that hill." You want to be the warrior who runs up with their sword and engages the enemy first. >> Ready to go. >> Cliff, it is so good to have you back in the studio, dude. I I was looking forward to this one cuz when I was writing it up, it's a very different thread of conversation to …

More from 20VC