Search a title or topic

Over 20 million podcasts, powered by 

Player FM logo
Artwork

Content provided by Corey Quinn. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Corey Quinn or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.
Player FM - Podcast App
Go offline with the Player FM app!

S3 and the Evolution of Storage with Andy Warfield

36:39
 
Share
 

Manage episode 464931099 series 2937944
Content provided by Corey Quinn. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Corey Quinn or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.

Andy Warfield joins Corey in this episode to discuss the evolution of storage technology at Amazon. This includes the evolution of S3 from archival storage to supporting modern AI and analytics. As Vice President and Distinguished Engineer at AWS, Andy is able to explain performance-enhancing innovations like S3 Tables and Common Runtime (CRT). On the other hand, challenges like compaction and namespace structuring are discussed. Reflecting on his journey from working on the Xen hypervisor to AWS, Andy shares insights into scaling S3, including buckets spanning millions of hard disks.

Show Highlights

(0:00) Intro

(1:09) The Duckbill Group sponsor read

(1:43) Andy’s background

(3:38) How AWS envisioned services being used vs. what customers actually do with them

(6:54) The frustration of legacy applications not keeping up with the times

(10:14) Why S3 is so accurate

(15:29) S3 as a role model for how a service should be run

(18:04) The Duckbill Group sponsor read

(18:46) Why AWS made Iceberg into a native offering

(23:50) Why S3 Tables is slightly more expensive

(28:23) How Andy handled the transition from Zen to Nitro

(32:22) What Andy is currently excited about

About Andy Warfield

Andrew Warfield is a VP / Distinguished Engineer at Amazon. As a senior technical leader at one of the world's largest technology companies, he plays a crucial role in shaping Amazon's engineering strategies and initiatives.

Links

Sponsor

The Duckbill Group: duckbillgroup.com

  continue reading

643 episodes

Artwork
iconShare
 
Manage episode 464931099 series 2937944
Content provided by Corey Quinn. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Corey Quinn or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.

Andy Warfield joins Corey in this episode to discuss the evolution of storage technology at Amazon. This includes the evolution of S3 from archival storage to supporting modern AI and analytics. As Vice President and Distinguished Engineer at AWS, Andy is able to explain performance-enhancing innovations like S3 Tables and Common Runtime (CRT). On the other hand, challenges like compaction and namespace structuring are discussed. Reflecting on his journey from working on the Xen hypervisor to AWS, Andy shares insights into scaling S3, including buckets spanning millions of hard disks.

Show Highlights

(0:00) Intro

(1:09) The Duckbill Group sponsor read

(1:43) Andy’s background

(3:38) How AWS envisioned services being used vs. what customers actually do with them

(6:54) The frustration of legacy applications not keeping up with the times

(10:14) Why S3 is so accurate

(15:29) S3 as a role model for how a service should be run

(18:04) The Duckbill Group sponsor read

(18:46) Why AWS made Iceberg into a native offering

(23:50) Why S3 Tables is slightly more expensive

(28:23) How Andy handled the transition from Zen to Nitro

(32:22) What Andy is currently excited about

About Andy Warfield

Andrew Warfield is a VP / Distinguished Engineer at Amazon. As a senior technical leader at one of the world's largest technology companies, he plays a crucial role in shaping Amazon's engineering strategies and initiatives.

Links

Sponsor

The Duckbill Group: duckbillgroup.com

  continue reading

643 episodes

All episodes

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Listen to this show while you explore
Play