How S3 is built

By Pragmatic Engineer

Categories: Product, Startup

Summary

AWS S3 manages over 500 trillion objects across 38 regions, processing a quadrillion requests annually using tens of millions of hard drives. What started as an eventual consistency model in 2006 has evolved into a massively scalable, strongly consistent cloud storage platform.

Key Takeaways

1. S3 manages 500 trillion objects across 120 availability zones and 38 global regions
2. Original S3 design optimized for durability and availability with eventual consistency
3. Entire S3 hard drive infrastructure could stack from Earth to International Space Station
4. Individual enterprise customers now manage exabytes of data in 'data oceans'

Topics

• Cloud Storage Architecture
• AWS Infrastructure Scaling
• Distributed Systems Design
• Enterprise Data Management

Transcript Excerpt

AWS S3 is the world's largest cloud storage service, but just how big is it and how is it engineered to be as reliable as it is at such a massive scale? Milon is the VP of data and analytics at AWS and has been running S3 for 13 years. Today we discuss the sheer scale of S3 in the data stored and the number of servers it runs on. How seemingly overnight AWS went from an eventually consistent data store to a strongly consistent one and the massive injury and complexity behind this move. What is c...