[MAINTAINERS SUMMIT] The Rust experiment

[Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>]

Hi all,

In a few months it will be a year since the Rust experiment in
mainline began. The first major users are getting upstreamed and more
companies are publicly supporting the effort [1]. However, when will
we consider the experiment successful (or not)? And how can willing
maintainers make the most out of the experiment?

The first question is important for companies and maintainers to start
investing in Rust. It is also tied to how we can resolve some
chicken-and-egg situations in subsystems. For instance, a few
maintainers are open to the idea of experimenting with Rust, but they
may want to start simple with a driver they are familiar with. But
such a driver would violate the "no duplicate drivers" rule.

Similarly, external people have expressed an interest in writing Rust
drivers, but given the required abstractions are not there, they may
decide to wait. But if nobody writes a first use case, the
abstractions cannot be merged without breaking the "no code without an
expected in-tree user" rule.

A potential solution to these deadlocks is the "Rust reference
drivers" proposal. The idea here is to relax the rules temporarily
(meanwhile the Rust experiment goes on) for willing maintainers that
opt into it, so that they can have, in-tree, an equivalent
implementation of a real, paradigmatic C driver of their subsystem.
This would allow them:

  1) to bootstrap abstractions for new drivers, i.e. not the
     "duplicate"/"rewritten" one, but future new drivers that would use
     those abstractions; while avoiding breaking the "no dead code" rule.

  2) to serve as a reference for existing C maintainers on how such
     drivers would look like in Rust, as "live" documentation , e.g. like
     how LWN featured a 1:1 comparison between a C and Rust driver. And it
     has to be buildable at all times.

  3) to use all the in-tree kernel infrastructure and to prepare their
     subsystem for Rust over time, e.g. setting up tests and CI.

  4) to learn over time, especially for subsystems that have several
     maintainers where not everybody may have time for it at a given
     moment. Reading Rust patches from time to time for APIs one is
     familiar with can help a lot.

  5) and, most importantly, to evaluate if the effort is worth it for
     their subsystem: how much can we write in safe code? How many issues
     has the reference driver had over time vs. the C one? Did Rust help
     prevent some? How hard is it to maintain the Rust side? Do we have
     enough resources in our subsystem? etc.

A "Rust reference driver" does not necessarily need to be considered a
real driver, e.g. it could be behind `EXPERT`, be tagged
`(EXPERIMENTAL)`, staging...

Some of this could be done out-of-tree, of course, and in fact we have
been considering maintaining `rust-*` branches for particular
subsystems to alleviate this situation for those users that want
particular abstractions and to coordinate everyone in a single place,
similar to how we started the Rust effort initially with our old
pre-merge `rust` branch.

However, the Rust experiment could be much more productive if willing
maintainers would be allowed to upstream and develop their
abstractions in-tree, just like the overall Rust support has been. In
other words, we started small with the Rust merge a year ago, adding
small features over time, and now it is the time for subsystems to go
ahead and let them do the same.

To be clear, this is something that maintainers would opt-into, i.e.
they know they are investing in something that they may not pay off
and may need to be removed. But that is the point of the Rust
experiment, after all.

We are trying to find a compromise that can work for everybody
(subsystem/driver maintainers wanting to start with things they know
well, companies wanting to use more Rust, external people looking for
"example code" and "what has been done so far", etc.). Last year I
asked for a bit of flexibility from everyone when required to make
progress: we do not want that people take this as an excuse to break
the rules or to give the impression that "Rust is going to be
special", but this may be one of those cases it could pay off.

Thanks!

Cheers,
Miguel

[1] https://rust-for-linux.com/industry-and-academia-support