It’s the storage system for Dropbox and internal use-cases.

Magic Pocket is Dropbox's exabyte scale blob storage system. I'm currently leading the team that manages Magic Pocket. What is the focus of your work these days? We discuss these challenges in more detail for others looking to make this transition and what these trade-offs look like. Our system is much cheaper than operating in the cloud, but it operates with a high bar.

How we are able to operate such a system, while being extremely cost efficient.

Key architecture lessons, which had the most impact on Magic Pocket.

This includes key services, databases, how multi-region replication works, repairs, and a discussion on the storage devices.

Provide an overview of the architecture of Magic Pocket.In this talk we will deep dive into the architecture of Magic Pocket, some early key design patterns that we still live by to this day, and the challenges of operating such a system at this scale in order to be cost efficient and support many critical requirements. Given the blast radius of single device failures, it is critical that our erasure codes and traffic are all built with this in mind. Each storage device contains 100+ drives and is multiple petabytes in size. We are constantly innovating in this space and work closely with hard drive vendors to adopt the latest drive technology ( ). This system is able to facilitate new drive technology, handle millions of queries per second, and automatically identify and repair hundreds of hardware failures per day. It is a horizontally scalable exabyte-scale blob storage system which operates out of multiple regions, is able to maintain 99.99% availability and has extremely high durability guarantees, while being more cost efficient than operating in the cloud. Magic Pocket is used to store all of Dropbox’s data.

Find ups dropbox