ELI5: RocksDB - Fast Data Storage

Intro

If you are looking for an open source storage engine library with a key/value interface, look no further. Open sourced in 2013, RocksDB continues to be widely used in the community and at Facebook to manage persistent data and server queries. Broadly, RocksDB is known to be particularly suited for flash drives, as well as pure memory, hard disks or remote storage.

To learn more, read on below or watch the embedded video, where we’ll explain it in a way that is easy to understand (or, as it’s commonly known online, ELI5).

What is the project?

RocksDB was originally forked from LevelDB by the Facebook Database Engineering team. Since then, the project has grown to be heavily optimized for Flash with extremely low latencies. Some of the main features of RocksDB include the ability to develop on processors with many cores, flexibility in storing small to medium size key/values on fast storage and optimally working with application servers storing terabytes of data.

From an architectural perspective, the keys and values in RocksDB are arbitrary byte streams. With a RocksDB backend, you have the ability to quickly set and query column families, update, access, and iterate through values in the database, perform multi-operational transactions, and incorporate multi-threaded compactions.

If you are considering using RocksDB, you should also ensure that your primary design optimizes for performance, production support and backwards compatibility. RocksDB servers should be configurable to support high read-workloads and update-workloads, support debugging and deployment environments and function effectively regardless of releases of newer or older versions of the library.

How is the library used by FB?

RocksDB is used extensively for storing persistent data on SSD at Facebook, and is also used by various services that serve online queries on hard drives.

Through RocksDB, we are able to create various data compression algorithms tools for production support and debugging in a way that is adaptive and automatic.

What resources does it have?

To learn more about RocksDB, visit its website, which contains documentation and a more in-depth overview of how it works. The RocksDB GitHub repo contains guides and examples on how to start using the library for your own applications.

If you have any further questions about RocksDB, let us know on our YouTube channel, or by tweeting at us. We always want to hear from you and hope you will find this open source project and the new ELI5 series useful.

About the ELI5 series

In a series of short videos (~1 min in length), one of our Developer Advocates on the Facebook Open Source team explains a Facebook open source project in a way that is easy to understand and use.

We will write an accompanying blog post (like the one you're reading right now) for each of these videos, which you can find on our YouTube channel.

To learn more about Facebook Open Source, visit our open source site, subscribe to our YouTube channel, or follow us on Twitter and Facebook.