Meet the Rustaceans: Hasnain Lakhani

This article was written in collaboration with Hasnain Lakhani, a software engineer at Facebook.

For today’s interview, we have Hasnain Lakhani who is a software engineer on the product security team at Facebook. Specifically, Hasnain’s team works on dynamic analysis through building large-scale fuzzing infrastructure and tooling. While working on this team, Hasnain has been using Rust as one of the main languages for development. Most of his work these days centers around finding security and reliability bugs in C/C++/Rust/Go code used at Facebook. Let’s hear from him about how his experience with Rust has been and learn more about his work.

In what capacity have you been using Rust?

I, along with a few other engineers on my team, maintain a fuzzing scheduler service. This service is the 'brain' of our broader system: it's responsible for dispatching work to a large fleet of machines, and ensuring all of our fuzzing specific business logic works correctly. We recently migrated this service from Python to Rust. I started designing the service, and worked with my teammates to refine the design, write the code, and deploy it. Nowadays, I spend about 30% of my time working in Rust, with the rest of it spent on our C++/Python/Hack codebases and wishing those were also in Rust!

Why did your team at Facebook choose to use Rust over other languages?

Our service was due for a rewrite, as the old system was over 2 years old and was not extensible to the new use cases we needed to build. Once we agreed a rewrite was needed, the first question we faced was language choice. We ruled out Python quickly, as our existing Python service was facing significant scaling challenges despite large investments optimizing it. That left C++ or Rust as our options. Given that not everyone in the team was familiar with the intricacies of C++, there was worry that we would be stuck debugging tricky memory corruption or race condition bugs in a multi-threaded service; instead of spending our time building value for our users.

I had already been using Rust for some personal projects and some side projects at work, and wanted to use more of it in a professional capacity. After discussing the memory safety benefits, the performance benefits, and the correctness guarantees when writing multi-threaded async code; we felt Rust would allow us to more quickly ship bug-free code. The rest of the team had also been curious to learn Rust for a while, and this opportunity was too good to pass up!

What are some of the projects that you’ve worked on at Facebook that use Rust?

• My team's scheduler service.
• CLI tools to run large scale fuzzing experiments on the scheduler service.
• An internal tool for analyzing networking configurations for security risks.
• I've contributed a few fixes here and there to common Rust libraries and tools at Facebook; including writing Rust clients for a few services that were needed by my team.

How do you feel about Rust's growth trajectory at Facebook?

I've been really passionate about Rust for the last few years and the trajectory is looking greater than ever. A few years ago, when we were writing the initial scheduler service, we actually decided *against* Rust because there wasn't enough integration with Facebook internal libraries and tools. Now that a lot of the foundations have been built, it is really easy to stand up a service in Rust and focus on your business logic instead of having to integrate with missing libraries. And, on the off chance that you still have to add integrations, tools like cxx make it easy to get it done within a few hours! When people ask me about language choices for new code, I now tend to bias towards Rust by default.

What value does Facebook add to Rust?

Facebook contributes a lot of libraries and tools to make writing Rust easier.

I think cxx is a game-changer: One of the main blockers I saw for Rust adoption was a desire to work with a large mass of existing code. With cxx, integrating Rust and C++ code takes at most a few hours even for complex libraries, and the result is ergonomic in both languages, and safe to use. I think this will be really vital as people look to incrementally migrate code over to Rust, and it's a great way to migrate security critical pieces of code to Rust within a larger codebase. I have used cxx in every possible direction (Rust calling C++, C++ calling Rust, and a case where C++ calls Rust which calls back into C++). It's all been great!

Facebook also maintains a library of crates which are generally useful, and used across all our projects. We use a lot of these in my team's production service, most notably the sql crate for all our storage needs.

How do you think Rust is growing as a language in 2021?

A few years ago, I felt like the odd one out in my team/peer group of engineering friends when I was learning Rust, and resources felt scarce. Nowadays, I hear an increasing number of stories and blog-posts from companies using Rust. More excitingly, when I talk to people about Rust, they often have heard about the language and why it's great, and are eager to learn more about it!

Some people who have used Rust have come to really like it. Why do you think that is and what is your favorite feature about Rust?

When I started learning Rust, I was excited by the promise of writing code which was more likely to be free of security issues like memory corruption bugs, while being performant.

I did not expect that I would become an advocate for Rust because of just how much more productive I am when writing Rust! When I was talking with a teammate, I heard how the tooling and libraries made it very easy to get started and get quick feedback (the error messages from rustc are best in class). Language features like the borrow checker, and pattern-matching with enums reduced the mental burden when authoring and reviewing code. This matched my experience as well: the compiler is like a free engineer available for pair-programming at all times, making sure I don't add bugs!

Rust has opened me to a new way of thinking when it comes to designing code - being able to find a bug, and then change the code so that similar bugs become errors at compile time is mind blowing—as discussed in depth here and here.

Where can people learn more about Rust and how can they start contributing to it?

When I started learning Rust, I spent a lot of time going over the canonical Rust language book, and the exercises within it, to learn by example. I still reach for the Rust Cookbook when I need to do common tasks in Rust, and it's really handy. I would recommend trying to port an existing project over to Rust and attempting to write it in idiomatic Rust to really learn the language.

I subscribe to the This Week in Rust newsletter to keep my knowledge up to date.

We would like to thank Hasnain for taking the time to do this interview. It was very interesting to learn how Rust is being used as a primary language for building fuzzing infrastructure and tooling, and how we are learning from all the things we build here and contributing back to the Rust community. We hope you found this interview useful and it gave you some insight into how and where Rust is being used at Facebook. Follow Hasnain on Twitter at @mhlakhani.

Check out our previous blogs from the Meet the Rustaceans series:

Meet the Rustaceans: Chris Konstad
Meet the Rustaceans: Pedro Rittner
Meet the Rustaceans: Neil Mitchell
Meet the Rustaceans: Daniel Xu
Meet the Rustaceans: Eric Kuecks
Meet the Rustaceans: Gus Wynn
Meet the Rustaceans: Brendon Daugherty

About the Meet the Rustaceans series

Rust has consistently been ranked as the “most loved” language for the last 5 years and we at Facebook believe that Rust is an outstanding language that shines in critical issues such as memory safety, performance, and reliability, and is being used widely over a large range of projects here. We joined the Rust Foundation to help contribute towards the improvement and growth of Rust, which not only strengthens our commitment towards the Rust language but also towards a sustainable development of open source technologies and developer communities across the world.

This blog is a part of our Meet the Rustaceans series, where we invite the engineers and developers who use Rust on a regular basis to share their experiences and tell us about the amazing products that they are building using Rust here at Facebook. Look out for more interview blogs where we meet with many more engineers and hear their thoughts on this topic.

To learn more about Facebook Open Source, visit our open source site, subscribe to our YouTube channel, or follow us on Twitter and Facebook.

Interested in working in Infrastructure at Facebook? Check out our job postings on our Infrastructure career page here.