Joining the Linux Community: The Easy Way

The Linux-kernel project is open source and is available to developers with all skill levels to contribute, including those with limited coding skills. As a result, there is never a shortage of developers wishing to contribute to Linux kernel development. But, there is no denying that it takes some effort to get established as a contributor in the Linux community. It is also good to assume that success in personal, academic and proprietary projects does not always guarantee success in the Linux-kernel community. In fact, many years of experience indicates that not all of the people attempting to join the Linux-kernel community succeed.

In this blog post, Paul McKenney, a long-term Linux kernel subsystem maintainer, gives insider information about how the Linux community works and the steps that aspiring contributors can take to become successful in the community. We hope this information will help you join and work in the community with minimum frustration. Here “join the Linux-kernel community” means being a developer who gets patches accepted on a routine basis. There are many other roles; however, much of the material in this document still applies to those roles.

What is the Linux-Kernel Community?

Let’s start with what the Linux-kernel community is not:

• Not a strict hierarchy. Linus almost never asks anyone to contribute to Kernel development. Instead, he usually limits himself to rejecting submissions and making suggestions.
• Not a group of managers.
• Not waiting for non-members of the Linux community to tell it what to do.
• Not necessarily a group that will invest large amounts of time in educating aspiring contributors.
• Not a group that gives priority to what aspiring contributors want.
• Not a group that knows about aspiring contributors' backgrounds and skills.

Instead, the Linux-kernel community is a group that has come together (more or less, anyway) to develop software to solve specific problems, and with substantial success. If you think of the community as a standards committee that produces source code rather than English prose, you won’t go too far wrong. In particular, working with the community is as much about working with people as it is about working with code.

The community is neither fungible nor homogeneous. For a given issue of sufficient severity, the relevant community is composed of those individuals who will drop what they are doing to address that issue. The lack of homogeneity can be irritating at times, but on the other hand, it means that if you had a terrible experience with one subsystem, you could try another.

Let’s look in more detail at what the Linux-kernel community is not.

Not a Strict Hierarchy

Or to the extent that the Linux-kernel community is a hierarchy, it is a very flat one. Most maintainers send their patches directly to Linus Torvalds.

But flat or deep, maintainership has its responsibilities. For example, suppose that a developer sends a subtly flawed RCU patch, and both rcutorture and the maintainer fail to see the flaw, and thus pull that patch into the subsystem tree and eventually send it along to Linus.

When the bug makes its presence known, which bugs eventually do, who takes the heat from Linus?

The answer is that the subsystem maintainer does, in their capacity as maintainer of the Linux-kernel RCU.

This means that each maintainer has full responsibility for quality control for their subsystem, which differs from the common practice of placing quality control responsibilities on a separate group. This difference is not at all subtle. As a developer, you will have far less power over a maintainer than you would over a separate quality control group. This situation can come as a bit of a shock to developers accustomed to quality control groups.

Not a Group of Managers

Linux-kernel maintainers are, for the most part, not managers at all. Their focus is instead on the technology with emphasis on getting things right, especially within their subsystems. So if some patch has broken a maintainer’s subsystem, you should expect that maintainer to focus on fixing the problem (most likely by fixing that patch or ejecting it completely) rather than paying attention to the submitter’s emotional state.

Style and Culture

Maintainers hail from across much of the world and therefore have diverse styles and cultures. Some maintainers are more flexible than others, but contributors should expect to adapt to their maintainer’s style. If the maintainer for a subsystem is flexible, so much the better, but contributors cannot count on this.

Waiting For a Contributor to Tell it What to Do

Linus has gone on record stating that his Linux-kernel goals were achieved in the early 1990s. While he does occasionally make suggestions, his main role is maintaining the full kernel. This means that he is accepting patches to help the Linux kernel achieve the goals of others. Or rejecting patches that interfere with users, though he works hard to get his maintainers to say “no” on his behalf.

So one could argue that Linus is, in fact, waiting for contributors to tell him what to do, at least in response to any patches that they convince maintainers to accept.

But most of the maintainers are quite busy and are not waiting for tasks from contributors. In fact, it can sometimes take considerable time, effort and creativity to convince them to accept the patches that contributors put so much work into. By the same token, maintainers are not waiting for a contributor’s manager to tell them what to do.

What to Expect

We hope that developers get a smooth entry into the Linux kernel community. However, this is one of those things that is much easier if developers start by assuming that it is going to be hard. Therefore, developers should avoid assuming that the Linux kernel community will appreciate their skills, invest in their education or give priority to knowing their background. And sometimes, your background might not be relevant or useful to your contributions to the Linux community.

The most important thing is to build sufficient trust that the maintainer will accept patches. This requires the two “C’s”: (1) Character and (2) Competence. This process can be reasonably fast and easy if developers do it right so that maintainers trust more with passing time.

Tips for Success in the Linux Community

Now that you have an overview of how the Linux community works, here are some tips to succeed in the Linux kernel community.

Leverage Your Related Experience

Suppose that you have worked for at least six months as a systems administrator, as a systemd developer, on a large application that interacts closely with the kernel, or have perused Linux kernel for fun or profit. In that case, this experience can be beneficial to you. So much so that if you lack such experience, you might be well-advised to acquire it before taking on serious Linux kernel work.

However, it is possible to be successful in the Linux kernel even from a standing start, especially if you take the following sections to heart.

Do Your Homework

The Linux kernel conducts its business on public mailing lists along with the occasional conference, most of which retain video. This means that you can easily review their discussions to determine what sorts of help they might be interested in, especially how they have reacted to earlier proposals similar to yours. (After all, if you didn’t have a proposal in mind, you probably would not be interested in contributing to the Linux kernel, right?)

So do your homework, which includes reading the relevant portions of the documentation, the source code, scanning the email archives for relevant discussions, and reviewing relevant conference presentations and Linux Weekly News articles. If you are not sure whether a few emails are relevant or not, it is probably best to read them. On the other hand, if you find yourself reading hundreds of irrelevant emails, you should think more carefully about what search terms you are using.

In the case of the Linux kernel, you should search lore.kernel.org and lwn.net for keywords that are related to your proposal. There are quite a few additional mailing lists related to the subsystems of the Linux kernel as well (Google for the keywords or see the MAINTAINERS file to determine what list to use). Read the discussions and learn why the community did not like the proposal. After all, if they did like it, they would likely have already implemented it, right? Use this information to refine and reformulate your proposal.

Before submitting your first patch, you owe it to yourself to carefully read and re-read the following Linux-kernel documentation files:

• Documentation/process/submitting-patches.rst
• Documentation/process/submitting-drivers.rst (if you are working with drivers)
• Documentation/process/submit-checklist.rst

Whether you like these rules or not, you need to follow them. In addition, the mailing lists are a great source of information on per-subsystem coding-style guidelines, preferred techniques and algorithms, the overall goals of the project and the personalities of key project members. Taking all of these things into account will help you produce an acceptable contribution. For example, if the group prefers hash tables to trees, you should of course make sure that your contributions avoid trees in favor of hash tables. Alternatively, if you absolutely must use trees, you will know that you need to justify your choice. And you must justify your choice within the confines of their viewpoints, not only within your own. This might mean sending them a first submission that uses a hash table so that you have something to compare your tree to.

Give Them the Benefit of the Doubt

All too often, when we see a maintainer rejecting our proposal (or something like it), it is common for us to assume that it is the maintainer's mistake. This is unproductive behavior, to say the least. In fact, if they have been working on their project for some time, it is safe to assume that they understand it much better than we do. And even if we are an expert in that specific area, the upstream developers have a much better understanding of the project as a whole, and therefore understand implications of the changes that we may not have even considered. Furthermore, the Linux kernel supports a staggering variety of hardware, configurations and workloads. Therefore, there are very likely constraints that aspiring contributors are unaware of, as fancifully illustrated by the parable of the six blind Tuxes and the elephant. Therefore, you should expect to justify your proposed changes to the maintainers.

Be a Valuable Member

Although it is natural to expect a new group of people to appreciate an aspiring contributor’s experience, skills and worth, such expectations are usually wildly unrealistic. After all, how would the members of the free/open-source software (FOSS) project know anything about aspiring contributors? Do they have a page on Wikipedia? Have they published extensively in their area of expertise? Does the material in the publications have a long history of being successfully used in practice?

Even if an aspiring contributor has a page on Wikipedia or a long list of publications that are frequently used in practice, this is no guarantee that the members of the Linux kernel community are aware of that.

Aspiring contributors will therefore need to show that they are valuable to the Kernel community. One excellent way of doing this is to solve a problem that the Kernel community is facing. Please note that this must be a problem that they realize they face and are highly motivated to solve. In other words, you need to solve a problem they know they have rather than a problem that you think that they have. The most straightforward way to do this is to provide a fix to a bug that they already recognize as a bug.

Conclusion

Please note that when contributing to the Linux kernel, even an expert in Linux technology cannot ignore the concerns of the Linux kernel maintainers. The approval process helps you improve the quality and robustness of your proposed design through feedback, and the maintainers benefit from learning about your technology.

The next challenge is getting your fix accepted. This can work a bit differently in the Linux kernel than elsewhere. We will cover that in another blog post. Stay tuned.

Resources

An early case study (non-paywalled) from Paul McKenney explains his journey of introducing RCU technology (non-paywalled) into the Linux Kernel.

To learn more about Meta Open Source, visit our open source site, subscribe to our YouTube channel, or follow us on Twitter and Facebook.