The Rust Starlark library

We’re excited to announce our open-source Rust library for working with the Starlark programming language, including a parser, evaluator, linter and debugger.

Starlark as a configuration language

Starlark is a programming language designed for configuring applications. It sits between a full programming language (like Python) and a configuration language (like JSON or YAML), providing expressivity and abstraction, but with simplicity and determinism. As an example of where Starlark is used, consider the Buck build system. A simple Buck build file, written in Starlark, looks like:

            android_library(
              name = 'activity',
              srcs = glob(['*.java']),
              visibility = [ 'PUBLIC' ],
            )
          

It’s not too hard to imagine what that might look like in JSON. But once you have 20 rules, you might want to abstract common details, like making visibility implicit, or adding some standard additional arguments. You might want to take some boolean arguments, or read configuration options. While formats like YAML/HOCON add some limited abstraction, sometimes you need to turn to something more powerful. Starlark offers functions and modules alongside a rich expression language.

But why not turn all the way to a programming language like Python? For a build system, there are three properties we want that Python doesn’t provide:

• We want to evaluate all imports in parallel, so we get answers quicker.
• We only want to reevaluate changed imports, so less needs recomputing.
• We want determinism, so that all users are guaranteed to see the same configuration every time, enabling caching and predictability.

As an example of a potential problem with Python, consider a module that when evaluated increments a global variable - that would break all the properties above. The elegant solution of Starlark is that after a module is evaluated, its values are frozen, never to be mutated again. If you create a global variable, once the module that defines it finishes executing, it will be frozen - and thus immutable and no longer problematic. This freezing trick, combined with a few simplifications compared to regular Python, gives us Starlark.

Starlark in Rust

Google uses Starlark in their Bazel build system, and wrote a specification for the language. Google engineers also wrote three implementations, in Go, Java and Rust. Facebook is starting to use more Rust, and configuration is always a big part of any project, so naturally the Rust Starlark library was very attractive. Therefore, thanks to the beauty of open source, we were able to fork Google’s Starlark Rust library and improve it, releasing our fork of the library. In cooperation with Google and the Starlark Rust authors, Facebook are now the maintainers of the Starlark Rust library - allowing Rust authors to easily embed Starlark into their projects.

For the Starlark Rust library, starting from the open-source base, we’ve been focusing on four dimensions:

• Usability, providing convenience functions like procedural macros to allow defining Rust code which can be called by Starlark. We’ve also been thoroughly documenting and testing the library, to help new users pick up the ropes.
• Compatibility, aiming to adhere to the Starlark standard where feasible, and working with the upstream specification where that’s not possible (e.g. Rust’s strings don’t allow invalid UTF8, but Starlark did). We’ve improved the compatibility significantly, and are looking to add support for new features like floats and bytestrings in the near future.
• Performance, to keep the cost of configuration low. We’ve invested in a faster interpreter, a faster lexer, garbage collected values and lots of performance optimisations on critical operations like string manipulation and dictionary creation.
• Features, with the hope of making Starlark a delightful experience for users. We’ve added an IDE integration mode (using LSP), a debugger (using DAP) and a linter. We’re still experimenting with new features, like types in Starlark.

This project was only possible thanks to the open source availability of the Starlark specification and implementations. In particular we owe a debt of gratitude to Damien Martin-Guillerez, who wrote the initial implementation. We’re happy to share our Starlark implementation with the Rust community, and welcome feedback, users and contributors.

About Developer Infrastructure

Facebook’s Developer Infrastructure team, (DevInfra) owns most of the coding life cycle, from the time code leaves an engineer’s mind until it reaches the people who use our apps. Our mission is to increase developer efficiency so that we can continue to ship awesome products quickly. We lead the industry by building innovative developer tools and automation infrastructure that are reliable and fast, ensuring that every second of engineering time is spent on the things that matter.

Interested in working in Infrastructure at Facebook? Check out our job postings on our Infrastructure career page here.

To learn more about Facebook Open Source, visit our open source site, subscribe to our YouTube channel, or follow us on Twitter and Facebook.