Back to News for Developers

The future of Buck

July 1, 2021ByChris Hopman

When Buck was unveiled at Facebook Mobile DevCon in 2013, it was a fast Android build system. Since then, it's expanded to support building applications written in 15+ languages and targeting platforms from mobile phones to servers to smart devices to VR headsets and more.

Since that time, many users have found Buck to be a very powerful, fast build system that vastly improves their developer experience. We continue to innovate in approaches to scaling builds, like our work with Android app compilations. The community has shared many stories on how they’ve accelerated their own development by using Buck, for example at Droidcon NYC 2016 and Devoxx Belgium 2017.

Over time, though, Buck has not kept up with this growth in features and flexibility with the necessary architectural changes and improvements to manage this complexity. As we develop new features, both inside and outside Facebook, we’ve found that this has led to significant implementation complexity and it is increasingly challenging to make changes to the core and the language rules. While we started an initiative nearly 4 years ago to incrementally make some of these cross-cutting architectural improvements to Buck, those changes have been extremely difficult.

Shifting focus to our next-gen build system

Since early 2020, we have been experimenting with what it would look like if we were to redesign the build system from scratch. Taking the lessons we have learned over the last 7 years, how could we write a build system that would continue to scale and be extensible for the next 10 or 20 years?

There is plenty that we think Buck got just right, and the next generation will be keeping many of the core concepts. For many languages and workflows, the new system will be completely backward compatible. Build files will continue to be written in the Starlark language (using the Rust starlark interpreter we’ve previously written about). The rules (cxx_library, etc) are the same and have mostly the same behavior. The high-level concepts like rules, targets, actions, all carry over.

As for the big changes, our next-gen build system is built on a single incremental computation framework. This provides fine-grained tracking of computation dependencies and very high performance for incremental computations. It is heavily inspired by research like Adapton and implementations like Salsa, Skip and the Shake build system.

The limited extensibility of Buck v1 has been a recurring source of pain for our community. With this new build system, we've focused on extensibility as a primary goal and every build rule is written outside of the core build language as a user-defined rule.

While considering this re-write, we took the opportunity to experiment and validate Rust as the tech stack for the build system. Facebook is increasingly investing in the Rust programming language and many projects have been using it with great success. We’ve found the language to be a great fit for many reasons, including:

  • Rust’s async/await syntax makes writing asynchronous code really smooth and Rust helps get the complex concurrency details correct. Each migration of Buck’s Java computations to a concurrent computation was a months-long, difficult transition and there are still significant single-threaded bottlenecks there.
  • Rust has a lot of high-level language features that make development easier and more enjoyable. These are things like enums, pattern matching, traits, procedural macros and all the other features that contribute to Rust developers in general loving it so much.
  • Rust gives greater control over memory allocations. Garbage collecting languages (even with generational collection) have challenges dealing with incremental computations like Buck performs.
  • Rust is high performance. We’ve seen significant speedups from the translation of things into Rust.

Sharing the next-gen build system with the community

We are committed to sharing this new work and project with the community. The process to prepare that has not yet been completed, but we have released part of it with the Rust Starlark library, and we are planning to release other components as reusable libraries like our core incremental computation engine and our "Super Console" textui framework, soon.

So, over the next several months you may see less progress on Buck, but rest assured we are continuing to work on providing the best build system to the community. We recognize that an important part of the sharing process will be in defining a smooth transition for users of Buck and ensuring that the community can move forward with us. We are planning to have this publicly available by January 2022 and will have more details about transitioning for current Buck users then.

What does this mean for Buck users?

For now, not much changes from the current state. Early last year we announced that active development of Buck was moving off of the stable main branch. We will continue to support and maintain the stable branch, and the plan is for that to continue at least one year after the public release of the new system. For most users, the transition should be seamless and we’ll provide support and tooling for more complex cases.

Some may have noticed that even on Buck's dev branch, development has been slowing and that will continue to slow as we focus more on the new build system. As before, use of the dev branch in production is discouraged and will have less support than the stable branch.

We anticipate having more to discuss about this next-gen build system later this year. You can always follow our work at Facebook’s Open Source blog and our github repositories.

To learn more about Facebook Open Source, visit our open source site, subscribe to our YouTube channel, or follow us on Twitter and Facebook.