[TECH TOPIC] Rust for Linux

[TECH TOPIC] Rust for Linux

[Miguel Ojeda]

The Rust for Linux project is adding support for the Rust language to
the Linux kernel. This talk describes the work done so far and also
serves as an introduction for other kernel developers interested in
using Rust in the kernel.

It covers:

- A quick introduction of the Rust language within the context of the kernel.
- How Rust support works in the kernel: overall infrastructure,
compilation model, the standard library (`core` and `alloc`), etc.
- How Documentation for Rust code looks like.
- Testing Rust code (unit tests and self tests).
- Overview of tooling (e.g. compiler as a library, custom linters, Miri, etc.).
- Explanation of coding guidelines (e.g. automatic formatting) and
policies we follow (e.g. the `SAFETY` comments).
- How kernel driver code looks like in Rust.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Sat, Jun 26, 2021 at 12:09 AM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:

> The Rust for Linux project is adding support for the Rust language to
> the Linux kernel. This talk describes the work done so far and also
> serves as an introduction for other kernel developers interested in
> using Rust in the kernel.

I think this is a good tech topic.

I think one thing that need discussing around this is implementation
language fragmentation and brainshare: kernel developers are all
assumed to grasp C, assembly, Make, KConfig, bash and some
YAML, perl and python here and there. I bet there is some sed
and awk in there and possibly more.

So, admitting how lazily we sometimes learn languages, which is
by copying, pasting and tweaking, then when that doesn't work
actually looking at the language spec (or StackOverflow) how hard
is that going to be with Rust? We get the hang of it quickly, yes?

I also repeat my challenge from the mailing list, which is that
while I understand that leaf drivers is a convenient place to start
doing Rust code, the question is if that is where Rust as a language
will pay off. My suggestion is to investigate git logs for the kind of
problems that Rust solves best and then introduce Rust where
those problems are. I believe that this would make be the
Perfect(TM) selling point for the language in the strict
hypothetico-deductive sense:

Hypothesis: Rust solves kernel problems A,B,C.

Collect occurrence data about problems A,B,C from git logs.

Identify code sites often exposing problems A,B,C.

Rewrite those in Rust.

Demonstrate how the Rust code now elegantly avoid
ever seeing problems A,B,C again.

From this it is very easy for the kernel community to deduce that
introducing Rust as an implementation language is worth it.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Leon Romanovsky]

On Tue, Jul 06, 2021 at 01:51:11AM +0200, Linus Walleij wrote:
> On Sat, Jun 26, 2021 at 12:09 AM Miguel Ojeda
> <miguel.ojeda.sandonis@gmail.com> wrote:
> 
> > The Rust for Linux project is adding support for the Rust language to
> > the Linux kernel. This talk describes the work done so far and also
> > serves as an introduction for other kernel developers interested in
> > using Rust in the kernel.
> 
> I think this is a good tech topic.

<...>

> Demonstrate how the Rust code now elegantly avoid
> ever seeing problems A,B,C again.
> 
> From this it is very easy for the kernel community to deduce that
> introducing Rust as an implementation language is worth it.

I don't know about advantages of Rust, but from what I see there are
some disadvantages that no one talks about them.

1. The addition of another language to the mix will hurt a lot our
ability to refactor code for cross-subsystem changes.

Just as an example, there are many changes in DMA/SG areas that are
applicable for many drivers. Even simple grep->replace pattern will
be harder and longer if it is needed for the drivers that are written
in different languages.

2. Testing matrix will be duplicated, both in compilation tests and in
verification coverage. Even now, there are some kernel subsystems that 
so much in-love with CONFIG_* options that their combination is slightly
tested or not tested at all. So imagine, we will need to recompile
everything with Rust too and execute same coverage tests again.

And yes, proper testing costs a lot of money, which IMHO better to
invest in improving coverage/fixing bugs/tooling instead of adding
new language to the pack.

Thanks

> 
> Yours,
> Linus Walleij
> 

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Tue, Jul 6, 2021 at 6:30 AM Leon Romanovsky <leon@kernel.org> wrote:

> 1. The addition of another language to the mix will hurt a lot our
> ability to refactor code for cross-subsystem changes.
>
> Just as an example, there are many changes in DMA/SG areas that are
> applicable for many drivers. Even simple grep->replace pattern will
> be harder and longer if it is needed for the drivers that are written
> in different languages.

This is a fair point.

> 2. Testing matrix will be duplicated, both in compilation tests and in
> verification coverage. Even now, there are some kernel subsystems that
> so much in-love with CONFIG_* options that their combination is slightly
> tested or not tested at all. So imagine, we will need to recompile
> everything with Rust too and execute same coverage tests again.

AFAIU the idea is to replace an entire piece of code in C with the
same piece in Rust, it's not like we are going to develop a second
kernel in Rust inside the kernel in C. So both/and not either/or.
I.e. you have to compile some pieces of the kernel written in
Rust to even get to a working kernel.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Linus,

On Tue, Jul 6, 2021 at 11:56 AM Linus Walleij <linus.walleij@linaro.org> wrote:
> I.e. you have to compile some pieces of the kernel written in
> Rust to even get to a working kernel.

So this would break all architectures not yet supported by Rust?

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Tue, Jul 6, 2021 at 12:16 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Tue, Jul 6, 2021 at 11:56 AM Linus Walleij <linus.walleij@linaro.org> wrote:
> > I.e. you have to compile some pieces of the kernel written in
> > Rust to even get to a working kernel.
>
> So this would break all architectures not yet supported by Rust?

I brought this up for discussion with Miguel et al and it turns out
the project to develop a GCC front-end for Rust is now well under
way and there is already a frankencompiler using LLVM first
and then feeding that into GCC IIUC.

I.e. support for m68k is on the way.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 8:00 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> I brought this up for discussion with Miguel et al and it turns out
> the project to develop a GCC front-end for Rust is now well under
> way and there is already a frankencompiler using LLVM first
> and then feeding that into GCC IIUC.

By the latter, I think you are referring to `rustc_codegen_gcc` -- in
that case, a quick note: it does not involve LLVM. `rustc` has its own
set of IRs which then gets translated into LLVM IR (in the most common
backend), but in the case of `rustc_codegen_gcc` it gets translated
into something libgccjit can consume. There is also a third backend in
the works, called cranelift, for different use cases.

So `rustc_codegen_gcc` is basically a very quick way of supporting the
entirety of the Rust language (exactly as `rustc` supports it) but
having a GCC codegen path.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Tue, Jul 6, 2021 at 8:36 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
> On Tue, Jul 6, 2021 at 8:00 PM Linus Walleij <linus.walleij@linaro.org> wrote:
> >
> > I brought this up for discussion with Miguel et al and it turns out
> > the project to develop a GCC front-end for Rust is now well under
> > way and there is already a frankencompiler using LLVM first
> > and then feeding that into GCC IIUC.
>
> By the latter, I think you are referring to `rustc_codegen_gcc` -- in
> that case, a quick note: it does not involve LLVM. `rustc` has its own
> set of IRs which then gets translated into LLVM IR (in the most common
> backend), but in the case of `rustc_codegen_gcc` it gets translated
> into something libgccjit can consume. There is also a third backend in
> the works, called cranelift, for different use cases.
>
> So `rustc_codegen_gcc` is basically a very quick way of supporting the
> entirety of the Rust language (exactly as `rustc` supports it) but
> having a GCC codegen path.

Yeah that was the thing I read about in The Register through my
newsfeed.

Excellent hack! It shows that this can be done, but perhaps partly
due to licensing a Rust front-end needs to be written for GCC
from scratch as well.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 9:12 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> Yeah that was the thing I read about in The Register through my
> newsfeed.

Ah, yes, I have read the article now -- it has a few mistakes,
including that `rustc_codegen_gcc` explanation, so it is
understandable where the confusion came from.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Arnd Bergmann]

On Tue, Jul 6, 2021 at 8:36 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Tue, Jul 6, 2021 at 8:00 PM Linus Walleij <linus.walleij@linaro.org> wrote:
> >
> > I brought this up for discussion with Miguel et al and it turns out
> > the project to develop a GCC front-end for Rust is now well under
> > way and there is already a frankencompiler using LLVM first
> > and then feeding that into GCC IIUC.
>
> By the latter, I think you are referring to `rustc_codegen_gcc` -- in
> that case, a quick note: it does not involve LLVM. `rustc` has its own
> set of IRs which then gets translated into LLVM IR (in the most common
> backend), but in the case of `rustc_codegen_gcc` it gets translated
> into something libgccjit can consume. There is also a third backend in
> the works, called cranelift, for different use cases.
>
> So `rustc_codegen_gcc` is basically a very quick way of supporting the
> entirety of the Rust language (exactly as `rustc` supports it) but
> having a GCC codegen path.

I suppose using this for building kernels is actually easier than doing
an architecture port for user space, because it does not have to
wrap the syscall interface, right?

I've had a quick look at what it would take to integrate this into the prebuilt
toolchains on kernel.org to give developers the chance to try their rust
code across architectures. I've managed to cross-build the patched
gcc sources for rustc_codegen_gcc, but haven't actually figured out how
to build rustc_codegen_gcc itself.

Before I look at this further, can you clarify what the version dependencies
are? I.e. if I want to build this for five gcc versions, 27 target
architectures,
and seven versions of rustc as shipped by the most common distros, can
I do this with a single rustc_codegen_gcc binary, or do I need to build
5*27*7=945 versions of the binary, or something inbetween?

       Arnd

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 4:32 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> I suppose using this for building kernels is actually easier than doing
> an architecture port for user space, because it does not have to
> wrap the syscall interface, right?

What do you mean by architecture port? Do you mean native vs. cross-compiling?

> I've had a quick look at what it would take to integrate this into the prebuilt
> toolchains on kernel.org to give developers the chance to try their rust
> code across architectures. I've managed to cross-build the patched
> gcc sources for rustc_codegen_gcc, but haven't actually figured out how
> to build rustc_codegen_gcc itself.

I discussed with Konstantin a while back about having the Rust
toolchain in kernel.org too -- so I am glad you are looking into this.

On `rustc_codegen_gcc`, it is early days -- I think all these projects
just got accelerated a lot thanks to the possibility of Rust being in
the kernel. But you can find information on this Compiler Explorer
issue because they are also trying to set it up [1]. The author looks
quite open to help us, so you may want to contact him too.

[1] https://github.com/compiler-explorer/compiler-explorer/issues/2683

> Before I look at this further, can you clarify what the version dependencies
> are? I.e. if I want to build this for five gcc versions, 27 target
> architectures,
> and seven versions of rustc as shipped by the most common distros, can
> I do this with a single rustc_codegen_gcc binary, or do I need to build
> 5*27*7=945 versions of the binary, or something inbetween?

I have not yet used `rustc_codegen_gcc` since it is not ready for our
purposes. But:

  - Only a single version of `rustc` is supported per kernel release
(until we stop using unstable features), so that at the very least
would cut down things by 7.

  - I would imagine the patched GCC of `rustc_codegen_gcc` only
supports the latest GCC release, do we really want to try to use it in
all GCC versions supported by the kernel? Of course, it would be great
if that means we can support Rust everywhere, do not get me wrong!

  - I don't know how stable the `libgccjit` interface is -- if it is,
then we only need a single one, as you say. I would ask the author.

  - We could try to get upstream GCC to accept the patches needed to
make `rustc_codegen_gcc` work, though I am not sure how well they
would be received, since they are working on their GCC Rust frontend
too.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Andrew Lunn]

>   - Only a single version of `rustc` is supported per kernel release
> (until we stop using unstable features), so that at the very least
> would cut down things by 7.

How does that work with -stable? Are you saying that rust patches
which are backported to an earlier kernel also need to be backported
to an earlier rustc?

   Andrew

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 5:50 PM Andrew Lunn <andrew@lunn.ch> wrote:
>
> How does that work with -stable? Are you saying that rust patches
> which are backported to an earlier kernel also need to be backported
> to an earlier rustc?

Yes, but changes are only needed if they rely on an unstable feature
and only if that feature changed. If that happens, we should know
about it when we migrate to a newer version if something breaks.

Thus this mainly applies to code in `rust/kernel` and `rust/alloc`,
because we statically enforce that modules only use a fixed set of
unstable features that we really need. Thus modules will not be able
to introduce new unstable features themselves.

Please note that we use unstable features either out of necessity or
because they lead to substantially better code, and only if we expect
them to be stabilized. In other words, we are conscious about it and
trying to strike the right balance here.

Moreover, this problem will go away progressively as features get
stabilized. That, of course, may take a long time. This is why we are
also trying to get support from companies to help fund people to
develop `rustc` if Rust gets into the kernel. Meanwhile, there is
already several people involved with `rustc` that know what we need
and will try to help us as much as possible (and have done so
already!).

Finally, please note that we explicitly say Rust support is
experimental, thus until we mark it as "stable enough", we could
simply avoid promising backports if -stable prefers that. One of the
reasons we say it is experimental is precisely because we need
unstable features from the compiler.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Arnd Bergmann]

On Wed, Jul 7, 2021 at 5:28 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Wed, Jul 7, 2021 at 4:32 PM Arnd Bergmann <arnd@arndb.de> wrote:
> >
> > I suppose using this for building kernels is actually easier than doing
> > an architecture port for user space, because it does not have to
> > wrap the syscall interface, right?
>
> What do you mean by architecture port? Do you mean native vs. cross-compiling?

No, I meant just which targets are enabled at all, regardless of whether
we are cross-building.

https://doc.rust-lang.org/nightly/rustc/platform-support.html lists only
arm64, arm32, hexagon, mips, powerpc, riscv, sparc64, s390x,
x86-32 and x86-64, in a number of variations (plus a few others
that don't run Linux). I assume these all have user space support,
while the other architectures that have llvm support (arc, csky, m68k,
sparc32) would work in principle for building the kernel but are lacking
a standard library port.

> > I've had a quick look at what it would take to integrate this into the prebuilt
> > toolchains on kernel.org to give developers the chance to try their rust
> > code across architectures. I've managed to cross-build the patched
> > gcc sources for rustc_codegen_gcc, but haven't actually figured out how
> > to build rustc_codegen_gcc itself.
>
> I discussed with Konstantin a while back about having the Rust
> toolchain in kernel.org too -- so I am glad you are looking into this.
>
> On `rustc_codegen_gcc`, it is early days -- I think all these projects
> just got accelerated a lot thanks to the possibility of Rust being in
> the kernel. But you can find information on this Compiler Explorer
> issue because they are also trying to set it up [1]. The author looks
> quite open to help us, so you may want to contact him too.
>
> [1] https://github.com/compiler-explorer/compiler-explorer/issues/2683

Ok

> > Before I look at this further, can you clarify what the version dependencies
> > are? I.e. if I want to build this for five gcc versions, 27 target
> > architectures,
> > and seven versions of rustc as shipped by the most common distros, can
> > I do this with a single rustc_codegen_gcc binary, or do I need to build
> > 5*27*7=945 versions of the binary, or something inbetween?
>
> I have not yet used `rustc_codegen_gcc` since it is not ready for our
> purposes. But:
>
>   - Only a single version of `rustc` is supported per kernel release
> (until we stop using unstable features), so that at the very least
> would cut down things by 7.

Ah, so I assume this is not going to be the version from my distro then.

>   - I would imagine the patched GCC of `rustc_codegen_gcc` only
> supports the latest GCC release, do we really want to try to use it in
> all GCC versions supported by the kernel? Of course, it would be great
> if that means we can support Rust everywhere, do not get me wrong!

Well, this was my question: if the libgccjit interface is stable enough,
it should be possible to port the patches to every version we still
support in the kernel. gcc-4.9.4 does not have libgccjit, but the later
ones do.

The patches are only for an unreleased gcc-12 tree but they
don't look too invasive, so maybe they could eventually get
backported to stable compilers (not upstreamed, but at least
used for my builds).

I don't really care about the outdated fix releases (9.3, 8.4,
7.3, ...) but I try to make sure the latest build of each major
gcc version works.

>   - I don't know how stable the `libgccjit` interface is -- if it is,
> then we only need a single one, as you say. I would ask the author.

According to
https://gcc.gnu.org/onlinedocs/jit/topics/compatibility.html
the ABI should be stable across all gcc versions, so if we
can get it to work with an old gcc release it would work with
all newer ones as well, but obviously anything built against the
patched library won't work against a newer version of the library
if that doesn't also have those patches.

My guess is that the ABI is also stable across target architectures,
but I could not confirm that. If this is true, then building against
the oldest supported libgccjit would make it work with any later
(patched) libgccjit, but we'd need to pass the correct libgccjit.so
file that matches the gcc version used for building the kernel
and the target architecture.

>   - We could try to get upstream GCC to accept the patches needed to
> make `rustc_codegen_gcc` work, though I am not sure how well they
> would be received, since they are working on their GCC Rust frontend
> too.

This would clearly only work for gcc-12 or later, otherwise it
does seem plausible. As far as I can tell, the patches just add features
that may be generally useful, not just for rust.

     Arnd

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 6:55 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> No, I meant just which targets are enabled at all, regardless of whether
> we are cross-building.
>
> https://doc.rust-lang.org/nightly/rustc/platform-support.html lists only
> arm64, arm32, hexagon, mips, powerpc, riscv, sparc64, s390x,
> x86-32 and x86-64, in a number of variations (plus a few others
> that don't run Linux). I assume these all have user space support,
> while the other architectures that have llvm support (arc, csky, m68k,
> sparc32) would work in principle for building the kernel but are lacking
> a standard library port.

Ah, I see what you mean. Yes, you are right, building "for user
space", i.e. including building `std` etc. is more work (because `std`
includes things that rely on an operating system being present; e.g.
printing, sockets, etc.), while supporting `core` and `alloc` is
easier (and it is the only thing we need for the kernel).

> Ah, so I assume this is not going to be the version from my distro then.

Yeah, in the beginning, I don't think they will always match. A couple
ways around that:

  - Distros could distribute the Rust toolchain they used to build the
kernel as a separate package; for the kernel only. This is what I
would recommend.

  - Distros could simply make the kernel compile with a different
version even if it is not supported by us -- i.e. it is possible, it
is just that we cannot guarantee it can be done *if* an unstable
feature happens to change between versions. This has a higher
probability of happening until we drop `alloc` from the kernel tree,
because `alloc` is part of the standard library and currently uses
some compiler magic.

Later on, when we manage to compile the kernel without unstable
features, then we only need to state a minimum version, like for
GCC/Clang.

> Well, this was my question: if the libgccjit interface is stable enough,
> it should be possible to port the patches to every version we still
> support in the kernel. gcc-4.9.4 does not have libgccjit, but the later
> ones do.
>
> The patches are only for an unreleased gcc-12 tree but they
> don't look too invasive, so maybe they could eventually get
> backported to stable compilers (not upstreamed, but at least
> used for my builds).
>
> I don't really care about the outdated fix releases (9.3, 8.4,
> 7.3, ...) but I try to make sure the latest build of each major
> gcc version works.

I see -- let's contact offline the author about this. I will Cc you
(or if you write the email, please Cc me).

> According to
> https://gcc.gnu.org/onlinedocs/jit/topics/compatibility.html
> the ABI should be stable across all gcc versions, so if we
> can get it to work with an old gcc release it would work with
> all newer ones as well, but obviously anything built against the
> patched library won't work against a newer version of the library
> if that doesn't also have those patches.
>
> My guess is that the ABI is also stable across target architectures,
> but I could not confirm that. If this is true, then building against
> the oldest supported libgccjit would make it work with any later
> (patched) libgccjit, but we'd need to pass the correct libgccjit.so
> file that matches the gcc version used for building the kernel
> and the target architecture.

Agreed, we could try that.

> This would clearly only work for gcc-12 or later, otherwise it
> does seem plausible. As far as I can tell, the patches just add features
> that may be generally useful, not just for rust.

What I meant is trying to convince GCC to have the patches in the
older releases -- but yeah, it is unlikely (they only accept bugfixes,
right?).

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Leon Romanovsky]

On Tue, Jul 06, 2021 at 11:55:47AM +0200, Linus Walleij wrote:
> On Tue, Jul 6, 2021 at 6:30 AM Leon Romanovsky <leon@kernel.org> wrote:
> 
> > 1. The addition of another language to the mix will hurt a lot our
> > ability to refactor code for cross-subsystem changes.
> >
> > Just as an example, there are many changes in DMA/SG areas that are
> > applicable for many drivers. Even simple grep->replace pattern will
> > be harder and longer if it is needed for the drivers that are written
> > in different languages.
> 
> This is a fair point.
> 
> > 2. Testing matrix will be duplicated, both in compilation tests and in
> > verification coverage. Even now, there are some kernel subsystems that
> > so much in-love with CONFIG_* options that their combination is slightly
> > tested or not tested at all. So imagine, we will need to recompile
> > everything with Rust too and execute same coverage tests again.
> 
> AFAIU the idea is to replace an entire piece of code in C with the
> same piece in Rust, it's not like we are going to develop a second
> kernel in Rust inside the kernel in C. So both/and not either/or.
> I.e. you have to compile some pieces of the kernel written in
> Rust to even get to a working kernel.

Yeah, it can work.

Thanks

> 
> Yours,
> Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 11:55 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> AFAIU the idea is to replace an entire piece of code in C with the
> same piece in Rust, it's not like we are going to develop a second
> kernel in Rust inside the kernel in C. So both/and not either/or.
> I.e. you have to compile some pieces of the kernel written in
> Rust to even get to a working kernel.

Let me clarify this, since it is important: no, we are not replacing C
code In fact, the Rust side is based on the C one.

But that does not mean we are not rewriting a second kernel either --
for instance, we have red-black trees "abstracted" in the Rust side by
reusing C's API.

In other words, what we are doing is "abstract" the C APIs into Rust
APIs that can ensure as many invariants as possible statically, using
Rust facilities for that. Thus Rust is one more consumer of the C
APIs.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 4:30 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> But that does not mean we are not rewriting a second kernel either --

s/we are not/we are/

i.e. we are *not* rewriting a second kernel, but abstracting the C APIs.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Sasha Levin]

On Tue, Jul 06, 2021 at 04:30:06PM +0200, Miguel Ojeda wrote:
>On Tue, Jul 6, 2021 at 11:55 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>>
>> AFAIU the idea is to replace an entire piece of code in C with the
>> same piece in Rust, it's not like we are going to develop a second
>> kernel in Rust inside the kernel in C. So both/and not either/or.
>> I.e. you have to compile some pieces of the kernel written in
>> Rust to even get to a working kernel.
>
>Let me clarify this, since it is important: no, we are not replacing C
>code In fact, the Rust side is based on the C one.
>
>But that does not mean we are not rewriting a second kernel either --
>for instance, we have red-black trees "abstracted" in the Rust side by
>reusing C's API.
>
>In other words, what we are doing is "abstract" the C APIs into Rust
>APIs that can ensure as many invariants as possible statically, using
>Rust facilities for that. Thus Rust is one more consumer of the C
>APIs.

Does this mean that anyone who wishes to modify these C APIs must also
know Rust to fix up those abstractions too?

-- 
Thanks,
Sasha

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 5:03 PM Sasha Levin <sashal@kernel.org> wrote:
>
> Does this mean that anyone who wishes to modify these C APIs must also
> know Rust to fix up those abstractions too?

Please see my answer to James and Leon, i.e. if we have abstractions
for a particular subsystem, it should mean somebody is happy to write,
use and maintain them.

That means that, yes, for subsystems that have Rust abstractions, if
you want to touch the C API, you also need to do so for the Rust
abstractions. But for any heavy refactor, I would expect maintainers
being the ones doing it, or at least helping to do so if somebody else
wants to change something in the C side and does not know how to
update the Rust side.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

On Tue, Jul 06, 2021 at 05:33:42PM +0200, Miguel Ojeda wrote:
> On Tue, Jul 6, 2021 at 5:03 PM Sasha Levin <sashal@kernel.org> wrote:
> >
> > Does this mean that anyone who wishes to modify these C APIs must also
> > know Rust to fix up those abstractions too?
> 
> Please see my answer to James and Leon, i.e. if we have abstractions
> for a particular subsystem, it should mean somebody is happy to write,
> use and maintain them.

There are lots of core APIs that are used by most drivers and that are
not subsystem-specific, so those will need to be considered too.

Additionally, even if there's a subsystem maintainer willing to maintain
a Rust abstraction, it also means that someone doing tree-wide or
subsystem-wide refactoring will need to pull the maintainer(s) in and
make it a team project. I really don't see how that can scale, tree-wide
changes are already very painful.

> That means that, yes, for subsystems that have Rust abstractions, if
> you want to touch the C API, you also need to do so for the Rust
> abstractions. But for any heavy refactor, I would expect maintainers
> being the ones doing it, or at least helping to do so if somebody else
> wants to change something in the C side and does not know how to
> update the Rust side.

I'm afraid that doesn't really match how development is done today :-)
Lots of subsystem-wide refactoring is done by developers who are not
subsystem maintainers.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Mike Rapoport]

On Tue, Jul 06, 2021 at 06:42:18PM +0300, Laurent Pinchart wrote:
> On Tue, Jul 06, 2021 at 05:33:42PM +0200, Miguel Ojeda wrote:
> > On Tue, Jul 6, 2021 at 5:03 PM Sasha Levin <sashal@kernel.org> wrote:
> > >
> > > Does this mean that anyone who wishes to modify these C APIs must also
> > > know Rust to fix up those abstractions too?
> > 
> > Please see my answer to James and Leon, i.e. if we have abstractions
> > for a particular subsystem, it should mean somebody is happy to write,
> > use and maintain them.
> 
> There are lots of core APIs that are used by most drivers and that are
> not subsystem-specific, so those will need to be considered too.
> 
> Additionally, even if there's a subsystem maintainer willing to maintain
> a Rust abstraction, it also means that someone doing tree-wide or
> subsystem-wide refactoring will need to pull the maintainer(s) in and
> make it a team project. I really don't see how that can scale, tree-wide
> changes are already very painful.

It seems to me that refactoring of core APIs would be impossible without
Rust knowledge. If the change touches more that one subsystem, coordinating
between the developer doing the refactoring and the people who maintain the
Rust abstractions is not going to be easy.
 
> > That means that, yes, for subsystems that have Rust abstractions, if
> > you want to touch the C API, you also need to do so for the Rust
> > abstractions. But for any heavy refactor, I would expect maintainers
> > being the ones doing it, or at least helping to do so if somebody else
> > wants to change something in the C side and does not know how to
> > update the Rust side.
> 
> I'm afraid that doesn't really match how development is done today :-)
> Lots of subsystem-wide refactoring is done by developers who are not
> subsystem maintainers.

Agree, many times people doing large refactoring are not subsystem
maintainers. And maintainers not necessary have time and/or desire to help.

That said, anybody doing tree-wide changes, even as relatively simple as
splitting several functions into a dedicated header, would need to learn
Rust.

-- 
Sincerely yours,
Mike.

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 5:43 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> There are lots of core APIs that are used by most drivers and that are
> not subsystem-specific, so those will need to be considered too.

Yes, but those are maintained in the "common"/shared support, e.g.
data structures, sync-related facilities, etc.; and hopefully will be
much better understood by everyone.

> Additionally, even if there's a subsystem maintainer willing to maintain
> a Rust abstraction, it also means that someone doing tree-wide or
> subsystem-wide refactoring will need to pull the maintainer(s) in and
> make it a team project. I really don't see how that can scale, tree-wide
> changes are already very painful.
>
> I'm afraid that doesn't really match how development is done today :-)
> Lots of subsystem-wide refactoring is done by developers who are not
> subsystem maintainers.

What I was saying is that the changes need to go through the
maintainers, which then would know Rust and the abstractions they
maintain.

Of course, somebody wishing to do tree-wide refactoring needs to know
at least a bit of Rust to at least know how to refactor and submit the
code for the maintainers -- but that is a given.

After all, if we are going to have Rust as a second language in the
kernel, we should try to have as many people on board as possible, at
least to some degree, within some reasonable time frame.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Miguel,

On Tue, Jul 06, 2021 at 08:29:21PM +0200, Miguel Ojeda wrote:
> On Tue, Jul 6, 2021 at 5:43 PM Laurent Pinchart wrote:
> >
> > There are lots of core APIs that are used by most drivers and that are
> > not subsystem-specific, so those will need to be considered too.
> 
> Yes, but those are maintained in the "common"/shared support, e.g.
> data structures, sync-related facilities, etc.; and hopefully will be
> much better understood by everyone.
> 
> > Additionally, even if there's a subsystem maintainer willing to maintain
> > a Rust abstraction, it also means that someone doing tree-wide or
> > subsystem-wide refactoring will need to pull the maintainer(s) in and
> > make it a team project. I really don't see how that can scale, tree-wide
> > changes are already very painful.
> >
> > I'm afraid that doesn't really match how development is done today :-)
> > Lots of subsystem-wide refactoring is done by developers who are not
> > subsystem maintainers.
> 
> What I was saying is that the changes need to go through the
> maintainers, which then would know Rust and the abstractions they
> maintain.
> 
> Of course, somebody wishing to do tree-wide refactoring needs to know
> at least a bit of Rust to at least know how to refactor and submit the
> code for the maintainers -- but that is a given.
> 
> After all, if we are going to have Rust as a second language in the
> kernel, we should try to have as many people on board as possible, at
> least to some degree, within some reasonable time frame.

I agree with you here, it's a honest way to look at it: adopting Rust as
a second language in the kernel isn't just a technical decision with
limited impact, but also a process decision that will create a
requirement for most kernel developers to learn Rust. Whether that
should and will happen is what we're debating, but regardless of the
outcome, it's important to phrase the question correctly, with a broad
view of the implications.

(That being said, there are also lots of technical challenges of course)

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Steven Rostedt]

On Tue, 6 Jul 2021 21:38:09 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> I agree with you here, it's a honest way to look at it: adopting Rust as
> a second language in the kernel isn't just a technical decision with
> limited impact, but also a process decision that will create a
> requirement for most kernel developers to learn Rust. Whether that
> should and will happen is what we're debating, but regardless of the
> outcome, it's important to phrase the question correctly, with a broad
> view of the implications.

I for one, welcome our new Rust overlords!

-- Steve

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 8:38 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> I agree with you here, it's a honest way to look at it: adopting Rust as
> a second language in the kernel isn't just a technical decision with
> limited impact, but also a process decision that will create a
> requirement for most kernel developers to learn Rust. Whether that
> should and will happen is what we're debating, but regardless of the
> outcome, it's important to phrase the question correctly, with a broad
> view of the implications.

Exactly -- very well put. We will definitely get the most advantages
from Rust if we all treat it as an actual second language.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Sasha Levin]

On Tue, Jul 06, 2021 at 05:33:42PM +0200, Miguel Ojeda wrote:
>On Tue, Jul 6, 2021 at 5:03 PM Sasha Levin <sashal@kernel.org> wrote:
>>
>> Does this mean that anyone who wishes to modify these C APIs must also
>> know Rust to fix up those abstractions too?
>
>Please see my answer to James and Leon, i.e. if we have abstractions
>for a particular subsystem, it should mean somebody is happy to write,
>use and maintain them.

I strongly disagree. If we have abstractions for a particular subsystem
all it means that someone at some point did that work, nothing more than
that.

That someone might have moved on, working elsewhere, retired, etc, and
those abstractions are living orphaned in the kernel.

>That means that, yes, for subsystems that have Rust abstractions, if
>you want to touch the C API, you also need to do so for the Rust
>abstractions. But for any heavy refactor, I would expect maintainers
>being the ones doing it, or at least helping to do so if somebody else
>wants to change something in the C side and does not know how to
>update the Rust side.

That's quite an ask. If the driver work goes in and I want to touch the
driver API, I need to get Greg KH to write code for me?

-- 
Thanks,
Sasha

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 10:47 PM Sasha Levin <sashal@kernel.org> wrote:
>
> I strongly disagree. If we have abstractions for a particular subsystem
> all it means that someone at some point did that work, nothing more than
> that.
>
> That someone might have moved on, working elsewhere, retired, etc, and
> those abstractions are living orphaned in the kernel.

That problem is orthogonal to Rust/C. I already acknowledged that if
you want to refactor Rust code, you will need some bare minimum Rust
knowledge -- there is no way around that, for any language.

And indeed, a second language introduces downsides like this -- this
is also acknowledged in the original RFC.

What I am saying is that, in the beginning (when most kernel
developers are not accustomed to Rust), a solution would be that
subsystem maintainers could step up and help those doing a refactor
that touches their Rust code. We also discussed a bit of this in the
original LKML discussion back in April; and we also offered ourselves
(the Rust support folks) to help as much as we can if anybody is
having issues on the Rust side.

But yes, if Rust usage grows in the kernel, then eventually it would
be much better if most kernel developers have a handle on Rust.

> That's quite an ask. If the driver work goes in and I want to touch the
> driver API, I need to get Greg KH to write code for me?

I have not said that, and that obviously that does not scale. I am
just stating what I think could help everyone during the transition
period, assuming we share a common goal and accept Rust into the
kernel.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Leon Romanovsky]

On Tue, Jul 06, 2021 at 11:50:51PM +0200, Miguel Ojeda wrote:
> On Tue, Jul 6, 2021 at 10:47 PM Sasha Levin <sashal@kernel.org> wrote:

<...>

> > That's quite an ask. If the driver work goes in and I want to touch the
> > driver API, I need to get Greg KH to write code for me?
> 
> I have not said that, and that obviously that does not scale. I am
> just stating what I think could help everyone during the transition
> period, assuming we share a common goal and accept Rust into the
> kernel.

I suggest to go one step further and loose expectation/requirement of
"you break, you fix" rule for the Rust code.

Any developer who does refactoring outside of the Rust-enabled subsystem
can simply leave that subsystem broken and relevant maintainers will fix
it later if they want.

We did it with staging in the fun days when Lustre was in the tree.
https://lore.kernel.org/linux-rdma/20150813055450.GA19344@infradead.org/
https://lore.kernel.org/linux-rdma/20150807141929.GA12442@infradead.org/

Thanks

> 
> Cheers,
> Miguel

Re: [TECH TOPIC] Rust for Linux

[Alexandre Belloni]

On 06/07/2021 23:50:51+0200, Miguel Ojeda wrote:
> On Tue, Jul 6, 2021 at 10:47 PM Sasha Levin <sashal@kernel.org> wrote:
> >
> > I strongly disagree. If we have abstractions for a particular subsystem
> > all it means that someone at some point did that work, nothing more than
> > that.
> >
> > That someone might have moved on, working elsewhere, retired, etc, and
> > those abstractions are living orphaned in the kernel.
> 
> That problem is orthogonal to Rust/C. I already acknowledged that if
> you want to refactor Rust code, you will need some bare minimum Rust
> knowledge -- there is no way around that, for any language.
> 
> And indeed, a second language introduces downsides like this -- this
> is also acknowledged in the original RFC.
> 
> What I am saying is that, in the beginning (when most kernel
> developers are not accustomed to Rust), a solution would be that
> subsystem maintainers could step up and help those doing a refactor
> that touches their Rust code. We also discussed a bit of this in the
> original LKML discussion back in April; and we also offered ourselves
> (the Rust support folks) to help as much as we can if anybody is
> having issues on the Rust side.

So that means that if a subsystem maintainer doesn't know Rust, Rust
code should not be merge in that subsystem at all.

-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 3:39 PM Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> So that means that if a subsystem maintainer doesn't know Rust, Rust
> code should not be merge in that subsystem at all.

I think that could be a sensible approach in the beginning, until more
people get a handle on Rust. But I defer to Greg/Linus et. al. on
particular policies.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Tue, Jul 6, 2021 at 4:30 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:

> In other words, what we are doing is "abstract" the C APIs into Rust
> APIs that can ensure as many invariants as possible statically, using
> Rust facilities for that. Thus Rust is one more consumer of the C
> APIs.

This reasoning assumes that Rust is in the "leafs" of the kernel, if
such a thing even exists, and device drivers have been mentioned.

If we want to *write* a subsystem in Rust then of course it will
go the other way: Rust need to expose APIs to C. And I assume
the grand vision is that after that Rust will eat Linux, one piece
at a time. If it proves better than C, that is.

The leaf nodes imagined as a starting point is some device
driver subsystem where individual drivers for that subsystem
can be written or rewritten in Rust, correct?

And I asked very pointed questions as to whether device drivers
is really a good place to start.

If the whole rationale with doing device drivers first is strategic,
not necessarily bringing any benefits to that device driver
subsystem but rather serving as a testing ground and guinea
pig, then that strategy needs to be explicit and understood
by everyone. So while we understand that Rust as
all these $UPSIDES doing device drivers first is purely
strategic, correct? I think the ability to back out the whole
thing if it wasn't working out was mentioned too?

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 8:27 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> This reasoning assumes that Rust is in the "leafs" of the kernel, if
> such a thing even exists, and device drivers have been mentioned.
>
> If we want to *write* a subsystem in Rust then of course it will
> go the other way: Rust need to expose APIs to C. And I assume
> the grand vision is that after that Rust will eat Linux, one piece
> at a time. If it proves better than C, that is.

In general, I would avoid exposing Rust subsystems to C.

It is possible, of course, and it gives you the advantages of Rust in
the *implementation* of the subsystem. However, by having to expose a
C API, you would be losing most of the advantages of the richer type
system, plus the guarantees that Rust bring as a language for the
consumers of the subsystem.

In any case, I agree it is a good idea to make things as smooth as
possible for interop in both directions. In that regard, for instance,
I presented a `[[safe]]` attribute in the C committee, and while it
did not make it, I hope to get things like that into Clang and GCC.

> The leaf nodes imagined as a starting point is some device
> driver subsystem where individual drivers for that subsystem
> can be written or rewritten in Rust, correct?
>
> And I asked very pointed questions as to whether device drivers
> is really a good place to start.
>
> If the whole rationale with doing device drivers first is strategic,
> not necessarily bringing any benefits to that device driver
> subsystem but rather serving as a testing ground and guinea
> pig, then that strategy needs to be explicit and understood
> by everyone. So while we understand that Rust as
> all these $UPSIDES doing device drivers first is purely
> strategic, correct? I think the ability to back out the whole
> thing if it wasn't working out was mentioned too?

So let me try to clarify the whole "leaf modules" thing.

It is possible to rewrite code in Rust and to also write Rust code and
expose it to C. It is also possible to write non-leaf modules in Rust.

However, the key is that the moment you go back from Rust to C, you
would be losing some of the advantages of Rust, such as the rich type
system, and the ability to write safe code (since C does not have it).

So, the current design is having Rust as a consumer of C APIs,
transform those into Rust APIs, and then Rust modules consume those
APIs. That is:

  C APIs --> Rust abstractions --> Rust modules

Now, nothing prevents you to write a Rust subsystem from scratch --
and in fact, that would be ideal. But if you want to make that
subsystem available back to C, then you are losing part of the appeal
(for the C modules only, of course).

That is why we say it is best for "leaf modules", in the sense that it
is Rust code that uses Rust APIs only. Whether those APIs come from a
new Rust-only subsystem or from some abstractions based on C APIs is
not very important.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Johannes Berg]

On Tue, 2021-07-06 at 16:30 +0200, Miguel Ojeda wrote:
> 
> But that does not mean we are not rewriting a second kernel either --
> for instance, we have red-black trees "abstracted" in the Rust side by
> reusing C's API.
> 
> In other words, what we are doing is "abstract" the C APIs into Rust
> APIs that can ensure as many invariants as possible statically, using
> Rust facilities for that. Thus Rust is one more consumer of the C
> APIs.

I couldn't really find a good place too hang this question, so it's here
now ;-)

Mostly from what I've seen you've been talking about rust being the
'leaves' in the graph, in the sense that it consumes C APIs exposed by
the kernel elsewhere, etc. In drivers using things, for example.

How about the other way around? What if we'd want to slowly replace
(parts of) a subsystem with rust code, but leave drivers? Or let's say
write some data structures in rust because of the stated benefits, but
let C consumers exist?

johannes

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 9:13 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> I couldn't really find a good place too hang this question, so it's here
> now ;-)
>
> Mostly from what I've seen you've been talking about rust being the
> 'leaves' in the graph, in the sense that it consumes C APIs exposed by
> the kernel elsewhere, etc. In drivers using things, for example.

Almost! Ideally, Rust modules should consume Rust APIs, not C APIs.
Please see the other responses on the "leaf" modules I just sent and
what the "Rust abstractions" are all about.

> How about the other way around? What if we'd want to slowly replace
> (parts of) a subsystem with rust code, but leave drivers? Or let's say
> write some data structures in rust because of the stated benefits, but
> let C consumers exist?

This is definitely possible too, and I would love if that happens.
However, note two points:

  - It would only net you advantages inside your implementation and
for Rust modules (both are of course important), but not on the C
drivers, of course, because you cannot expose Rust features back to C
(such as the ability to describe what is safe and what is not, types
that guarantee certain invariants, etc.).

  - Currently there is no GCC codegen path for the Rust side -- so
your subsystem would not work in all architectures until
`rustc_codegen_gcc` or GCC Rust are mature enough. The former may be
ready sooner than we expect. The latter will take longer -- I was told
about 2 years for kernel compilation purposes.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[James Bottomley]

On Tue, 2021-07-06 at 07:30 +0300, Leon Romanovsky wrote:
> On Tue, Jul 06, 2021 at 01:51:11AM +0200, Linus Walleij wrote:
> > On Sat, Jun 26, 2021 at 12:09 AM Miguel Ojeda
> > <miguel.ojeda.sandonis@gmail.com> wrote:
> > 
> > > The Rust for Linux project is adding support for the Rust
> > > language to the Linux kernel. This talk describes the work done
> > > so far and also serves as an introduction for other kernel
> > > developers interested in using Rust in the kernel.
> > 
> > I think this is a good tech topic.
> 
> <...>
> 
> > Demonstrate how the Rust code now elegantly avoid
> > ever seeing problems A,B,C again.
> > 
> > From this it is very easy for the kernel community to deduce that
> > introducing Rust as an implementation language is worth it.
> 
> I don't know about advantages of Rust, but from what I see there are
> some disadvantages that no one talks about them.

The main advantage is supposed to be "memory safety":

https://en.wikipedia.org/wiki/Memory_safety

But that's tempered by the fact that all non-rust code is deemed unsafe
and it's hard to write drivers without calling into the core, so
there's a lot of unsafe stuff going on even if you write your driver in
rust.

The other thing that makes comparison with C hard is the fact that
compilers and fuzzers are pretty good at detecting memory problems in
the existing code, so it's unclear what memory safety ab initio
actually buys for the kernel.

> 1. The addition of another language to the mix will hurt a lot our
> ability to refactor code for cross-subsystem changes.
> 
> Just as an example, there are many changes in DMA/SG areas that are
> applicable for many drivers. Even simple grep->replace pattern will
> be harder and longer if it is needed for the drivers that are written
> in different languages.
> 
> 2. Testing matrix will be duplicated, both in compilation tests and
> in verification coverage. Even now, there are some kernel subsystems
> that so much in-love with CONFIG_* options that their combination is
> slightly tested or not tested at all. So imagine, we will need to
> recompile everything with Rust too and execute same coverage tests
> again.

3. Less review: The lack of kernel programmers understanding rust
hampers reviewing.  Since most of our CVE type problems are usually
programming mistakes nowadays, the lack of review could contribute to
an increase in programming fault type bugs which aren't forbidden by
the safer memory model.

James

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 12:20 PM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> The main advantage is supposed to be "memory safety":
>
> https://en.wikipedia.org/wiki/Memory_safety

Yes, although please note it covers other things, like data races.

In general, Rust prevents the kind of thing that is UB in C, which
means it also covers you from the usual surprises that optimizers may
give you.

> But that's tempered by the fact that all non-rust code is deemed unsafe
> and it's hard to write drivers without calling into the core, so
> there's a lot of unsafe stuff going on even if you write your driver in
> rust.

Yes, but bugs on the C side are still bugs, and they need to be fixed
regardless of Rust.

That is to say: the fact that the C side has bugs does not mean Rust
is pointless in the kernel. The value is to write drivers with a
safe-as-possible API. By doing that, we avoid extra bugs (but, of
course, bugs on the unsafe Rust code or in the C side remain can still
happen).

> The other thing that makes comparison with C hard is the fact that
> compilers and fuzzers are pretty good at detecting memory problems in
> the existing code, so it's unclear what memory safety ab initio
> actually buys for the kernel.

Compilers definitely do not detect all memory safety issues -- not
even close. They cannot anyway, in the general case. Not even in C++
with `std::unique_ptr`, `std::vector`, etc. Rust can do so because it
places extra restrictions in the modeling capabilities (in the safe
subset only).

Runtime detection of UB in C is, of course, possible, but the idea is
to have static guarantees vs. runtime-checked ones. There is also
runtime detection of UB in Rust for unsafe code with tooling like
Miri. plus all the language-independent tooling, of course.

> 3. Less review: The lack of kernel programmers understanding rust
> hampers reviewing.  Since most of our CVE type problems are usually
> programming mistakes nowadays, the lack of review could contribute to
> an increase in programming fault type bugs which aren't forbidden by
> the safer memory model.

Yes, this is a fair point. Initially, our plan is to work with
subsystem maintainers that do want to start providing a Rust API for
their subsystem. Then they can maintain drivers using such API.

We definitely do not want to have drivers written for a particular
subsystem if nobody is able to review (or even write) the Rust
abstractions for that particular subsystem.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Sasha Levin]

On Tue, Jul 06, 2021 at 04:55:56PM +0200, Miguel Ojeda wrote:
>On Tue, Jul 6, 2021 at 12:20 PM James Bottomley
>> 3. Less review: The lack of kernel programmers understanding rust
>> hampers reviewing.  Since most of our CVE type problems are usually
>> programming mistakes nowadays, the lack of review could contribute to
>> an increase in programming fault type bugs which aren't forbidden by
>> the safer memory model.
>
>Yes, this is a fair point. Initially, our plan is to work with
>subsystem maintainers that do want to start providing a Rust API for
>their subsystem. Then they can maintain drivers using such API.
>
>We definitely do not want to have drivers written for a particular
>subsystem if nobody is able to review (or even write) the Rust
>abstractions for that particular subsystem.

How do you see this working on the -stable side of things? It's hard
enough to get review for C code, the amount of people who would do a
second review in the context of a -stable release would be non-existant.

-- 
Thanks,
Sasha

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 5:02 PM Sasha Levin <sashal@kernel.org> wrote:
>
> How do you see this working on the -stable side of things? It's hard
> enough to get review for C code, the amount of people who would do a
> second review in the context of a -stable release would be non-existant.

Hopefully, by the time Rust code is fixed through the -stable channel,
more people will have more knowledge on Rust.

In the worst case, we could have either the maintainers of the
abstractions review the -stable patches if they are complex enough
(but then why would they be in -stable? i.e. the same applies for any
complex C change, no?) or people with Rust knowledge looking at
-stable patches.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Marco Elver]

On Tue, Jul 06, 2021 at 04:55PM +0200, Miguel Ojeda wrote:
> On Tue, Jul 6, 2021 at 12:20 PM James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> >
> > The main advantage is supposed to be "memory safety":
> >
> > https://en.wikipedia.org/wiki/Memory_safety
[...]
> > The other thing that makes comparison with C hard is the fact that
> > compilers and fuzzers are pretty good at detecting memory problems in
> > the existing code, so it's unclear what memory safety ab initio
> > actually buys for the kernel.
> 
> Compilers definitely do not detect all memory safety issues -- not
> even close. They cannot anyway, in the general case. Not even in C++
> with `std::unique_ptr`, `std::vector`, etc. Rust can do so because it
> places extra restrictions in the modeling capabilities (in the safe
> subset only).

I think the main point was about the combination of sanitizers paired
with fuzzers like syzkaller.

> Runtime detection of UB in C is, of course, possible, but the idea is
> to have static guarantees vs. runtime-checked ones. There is also
> runtime detection of UB in Rust for unsafe code with tooling like
> Miri. plus all the language-independent tooling, of course.

I sincerely hope that not too much trust will be put into Rust-only
dynamic analysis via something like Miri (for the unsafe parts). For the
kernel, first and foremost, the Rust integration will require proper
integration with existing sanitizers (with `rustc -Zsanitizer=`??):
KASAN, KCSAN (possibly KMSAN which is still out-of-tree).

We have years of experience with kernel dynamic analysis, and discover
over and over that bugs are missed due to uninstrumented code paths
(including inline asm and such), and put in a lot of effort to
instrument as much as possible.

It is very likely that if the Rust portion is analyzed alone, be it
statically or dynamically, that there will remain undiscovered bugs due
to improper abstractions between C and Rust. While I fully see that
Rust's static guarantees are strong for safe code, I'm pragmatic and
just do not believe those building the safe abstractions from unsafe
code will not make mistakes nor will those abstractions shield from
changed behaviour on the C side that directly affects safety of the Rust
abstraction.

Not only will Rust integration with K*SANs be required to catch early
bugs in the abstractions, but also be necessary to catch e.g.
use-after-frees in Rust code where C code freed the memory erroneously,
or data races between Rust and C code.

But it will ultimately also prove that the main proposition of Rust in
the kernel holds: less bugs in the Rust parts over time.

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Fri, Jul 9, 2021 at 12:02 PM Marco Elver <elver@google.com> wrote:
>
> I think the main point was about the combination of sanitizers paired
> with fuzzers like syzkaller.

Yes, and my reply was that compile-time detection of bugs is way, way
better than runtime detection + relying on being lucky enough to hit
the bug via testing and/or fuzzing.

> I sincerely hope that not too much trust will be put into Rust-only
> dynamic analysis via something like Miri (for the unsafe parts). For the
> (...)

I never claimed that we should blindly trust unsafe code written in
Rust, nor that we should only perform Rust-only dynamic analysis.
Quite the opposite: I mentioned there is already tooling around it
precisely because we need as much of it as possible.

Put another way: the topic is "what Rust buys us", not "what unsafe
Rust buys us". That is, the goal is writing abstractions in a way that
we maximize the amount of safe Rust code for modules etc. But,
obviously, the Rust abstractions that deal with unsafe code (e.g.
calling C) need to be as carefully reviewed and analyzed as C code is
-- nobody claimed otherwise.

> It is very likely that if the Rust portion is analyzed alone, be it
> statically or dynamically, that there will remain undiscovered bugs due
> to improper abstractions between C and Rust. While I fully see that

Definitely, but there are some things that are amenable to be analyzed
on their own. For instance, for data structures written in pure Rust,
Miri is a powerful tool we should be using right away.

> Rust's static guarantees are strong for safe code, I'm pragmatic and

Note that even within the unsafe subset there are some benefits over
C, such as the borrow checker (it is still enabled in unsafe code),
pattern matching and type system in general (e.g. like the
`Option`/`Result` example a few emails above about being unable to
mistakenly use its contents if there are none; that you can create
types that enforce invariants, that it is stronger overall than
C's...), etc.

It also has some downsides, though: the rules one needs to abide by in
unsafe code are different than C's (e.g. due to new concepts like
references), and thus one should be careful about conflating
assumptions.

Side-note: our coding guidelines enforce that every `unsafe` block
must be documented with a proof of why such block is sound. Same for
invariants that a type holds. Moreover, we require all public APIs to
be documented, etc. This is a way of saying that, even for unsafe
code, we are trying to be stricter than the C side.

> just do not believe those building the safe abstractions from unsafe
> code will not make mistakes nor will those abstractions shield from
> changed behaviour on the C side that directly affects safety of the Rust
> abstraction.

Again, please note we never claimed anything like that.

It is the opposite: we want to use Rust precisely because we want to
be able to statically enforce as much as we can -- that is, maximizing
the amount of safe code.

> Not only will Rust integration with K*SANs be required to catch early
> bugs in the abstractions, but also be necessary to catch e.g.
> use-after-frees in Rust code where C code freed the memory erroneously,
> or data races between Rust and C code.

It might be possible to eliminate some classes of bugs if we avoid
mixing C and Rust too much.

For instance, if we have a model where we allow the Rust side to
manage ownership of some objects (instead of being forced to always go
through the C APIs), then we may be able to statically guarantee more
things and further maximize the amount of safe code, in particular in
modules -- which is why I raised the question earlier.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Tue, Jul 6, 2021 at 12:20 PM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:

> 3. Less review: The lack of kernel programmers understanding rust
> hampers reviewing.  Since most of our CVE type problems are usually
> programming mistakes nowadays, the lack of review could contribute to
> an increase in programming fault type bugs which aren't forbidden by
> the safer memory model.

The rationale not stated explicitly by the Rust enthusiasts is based
on the assumption that the language has a broad uptake and that
the crowd of developers embracing Rust will increase over time,
and especially that new young, fresh developers will know Rust
better than C.

If over time this becomes (by assumption) an increasing crowd of
Rust-interested developers who think C is primitive and last years
news and do not want to work with it, then the assumption is true.

This has limited evidence but is I think based on StackOverflow
surveys of "most loved language" and such. Overall the people
promoting Rust believe in this and they also apparently feel a strong
sense of belonging wrt the language.

I think it is a fair reason. Any push into any new technology needs to
be driven by belief and enthusiasm, i.e. emotions. That's fine. Such
was Unix and C, and such was Linux, too. Not everything is rational.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 6:30 AM Leon Romanovsky <leon@kernel.org> wrote:
>
> 1. The addition of another language to the mix will hurt a lot our
> ability to refactor code for cross-subsystem changes.
>
> Just as an example, there are many changes in DMA/SG areas that are
> applicable for many drivers. Even simple grep->replace pattern will
> be harder and longer if it is needed for the drivers that are written
> in different languages.

In the design we chose, Rust drivers do not call C code directly --
they go through the abstractions a given subsystem may provide. Thus
only the abstractions would need to be updated, like any other C
consumer of the APIs.

> 2. Testing matrix will be duplicated, both in compilation tests and in
> verification coverage. Even now, there are some kernel subsystems that
> so much in-love with CONFIG_* options that their combination is slightly
> tested or not tested at all. So imagine, we will need to recompile
> everything with Rust too and execute same coverage tests again.

I am not sure I understand this point.

On its own, `CONFIG_RUST` only changes 1 single place currently (the
addition of a `vsprintf` format specifier).

Moreover, the goal is that Rust is enabled by default for
architectures that support it. So there would be no need for
duplication at all.

> And yes, proper testing costs a lot of money, which IMHO better to
> invest in improving coverage/fixing bugs/tooling instead of adding
> new language to the pack.

The original RFC includes an overview of advantages and disadvantages of Rust:

  https://lore.kernel.org/lkml/20210414184604.23473-1-ojeda@kernel.org/

Rust brings things to the table that less-than-100%-coverage and
tooling cannot. For instance, UBSAN catches UB, but the safe subset of
Rust forbids UB statically (assuming unsafe code is sound).

Trying to clarify this sort of thing is why I included the "quick
introduction of the Rust language" point in the abstract of the tech
topic -- I think it is worth having a common ground to discuss things
and getting everybody on the same page.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

On Tue, Jul 06, 2021 at 04:24:07PM +0200, Miguel Ojeda wrote:
> On Tue, Jul 6, 2021 at 6:30 AM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > 1. The addition of another language to the mix will hurt a lot our
> > ability to refactor code for cross-subsystem changes.
> >
> > Just as an example, there are many changes in DMA/SG areas that are
> > applicable for many drivers. Even simple grep->replace pattern will
> > be harder and longer if it is needed for the drivers that are written
> > in different languages.
> 
> In the design we chose, Rust drivers do not call C code directly --
> they go through the abstractions a given subsystem may provide. Thus
> only the abstractions would need to be updated, like any other C
> consumer of the APIs.

I think the point is that if a C API that has a Rust abstraction built
on top changes, then the Rust API exposed by abstraction may have to
change, so all Rust code that use it would need to be updated too. I
dread to think how we could do large refactoring if we can't use
Coccinelle for Rust code.

> > 2. Testing matrix will be duplicated, both in compilation tests and in
> > verification coverage. Even now, there are some kernel subsystems that
> > so much in-love with CONFIG_* options that their combination is slightly
> > tested or not tested at all. So imagine, we will need to recompile
> > everything with Rust too and execute same coverage tests again.
> 
> I am not sure I understand this point.
> 
> On its own, `CONFIG_RUST` only changes 1 single place currently (the
> addition of a `vsprintf` format specifier).
> 
> Moreover, the goal is that Rust is enabled by default for
> architectures that support it. So there would be no need for
> duplication at all.
> 
> > And yes, proper testing costs a lot of money, which IMHO better to
> > invest in improving coverage/fixing bugs/tooling instead of adding
> > new language to the pack.
> 
> The original RFC includes an overview of advantages and disadvantages of Rust:
> 
>   https://lore.kernel.org/lkml/20210414184604.23473-1-ojeda@kernel.org/
> 
> Rust brings things to the table that less-than-100%-coverage and
> tooling cannot. For instance, UBSAN catches UB, but the safe subset of
> Rust forbids UB statically (assuming unsafe code is sound).
> 
> Trying to clarify this sort of thing is why I included the "quick
> introduction of the Rust language" point in the abstract of the tech
> topic -- I think it is worth having a common ground to discuss things
> and getting everybody on the same page.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Leon Romanovsky]

On Tue, Jul 06, 2021 at 04:24:07PM +0200, Miguel Ojeda wrote:
> On Tue, Jul 6, 2021 at 6:30 AM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > 1. The addition of another language to the mix will hurt a lot our
> > ability to refactor code for cross-subsystem changes.
> >
> > Just as an example, there are many changes in DMA/SG areas that are
> > applicable for many drivers. Even simple grep->replace pattern will
> > be harder and longer if it is needed for the drivers that are written
> > in different languages.
> 
> In the design we chose, Rust drivers do not call C code directly --
> they go through the abstractions a given subsystem may provide. Thus
> only the abstractions would need to be updated, like any other C
> consumer of the APIs.

In such case, I see no easy way to do refactoring. Many (at least for me)
refactoring changes are not simple sed commands, but more comprehensive
deep dive into the code which requires to see the actual API usage.

Even for the simple cases, where someone changes return value type, he/she
will need to update the logic in the driver. How will abstractions help us here?

IMHO, even worse, people will be less willing to change in-kernel APIs
because of such abstractions.

> 
> > 2. Testing matrix will be duplicated, both in compilation tests and in
> > verification coverage. Even now, there are some kernel subsystems that
> > so much in-love with CONFIG_* options that their combination is slightly
> > tested or not tested at all. So imagine, we will need to recompile
> > everything with Rust too and execute same coverage tests again.
> 
> I am not sure I understand this point.
> 
> On its own, `CONFIG_RUST` only changes 1 single place currently (the
> addition of a `vsprintf` format specifier).
> 
> Moreover, the goal is that Rust is enabled by default for
> architectures that support it. So there would be no need for
> duplication at all.

My point is that we have a customers who run upstream/stable/distro kernels
on old systems which we need to support. The default CONFIG_RUST=y in .config
is nice, but most users won't get it or enable it.

So we will need CONFIG_RUST=n to make sure that code for the customers
work and we will need CONFIG_RUST=y to make sure that at least
compilation passes.

> 
> > And yes, proper testing costs a lot of money, which IMHO better to
> > invest in improving coverage/fixing bugs/tooling instead of adding
> > new language to the pack.
> 
> The original RFC includes an overview of advantages and disadvantages of Rust:
> 
>   https://lore.kernel.org/lkml/20210414184604.23473-1-ojeda@kernel.org/
> 
> Rust brings things to the table that less-than-100%-coverage and
> tooling cannot. For instance, UBSAN catches UB, but the safe subset of
> Rust forbids UB statically (assuming unsafe code is sound).
> 
> Trying to clarify this sort of thing is why I included the "quick
> introduction of the Rust language" point in the abstract of the tech
> topic -- I think it is worth having a common ground to discuss things
> and getting everybody on the same page.

Thanks for the summary, I just disagree with the punch line:
 "the advantages of using Rust outweighs the cost".

Thanks

> 
> Cheers,
> Miguel

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 4:56 PM Leon Romanovsky <leon@kernel.org> wrote:
>
> In such case, I see no easy way to do refactoring. Many (at least for me)
> refactoring changes are not simple sed commands, but more comprehensive
> deep dive into the code which requires to see the actual API usage.

(Also answering to Laurent's)

Yes, of course, that is why we should avoid having Rust abstraction a
subsystem that have maintainers that actually want to have Rust
drivers, i.e. see my reply to James:

"""
Initially, our plan is to work with
subsystem maintainers that do want to start providing a Rust API for
their subsystem. Then they can maintain drivers using such API.

We definitely do not want to have drivers written for a particular
subsystem if nobody is able to review (or even write) the Rust
abstractions for that particular subsystem.
"""

> Even for the simple cases, where someone changes return value type, he/she
> will need to update the logic in the driver. How will abstractions help us here?
>
> IMHO, even worse, people will be less willing to change in-kernel APIs
> because of such abstractions.

The abstractions help because you would only need to change things
there, vs. in all the Rust drivers.

Of course, if it is such a heavy change that new concepts themselves
appear/disappear, then that does not help -- but that is why we should
avoid abstracting what nobody will maintain (see the previous point
for that).

> My point is that we have a customers who run upstream/stable/distro kernels
> on old systems which we need to support. The default CONFIG_RUST=y in .config
> is nice, but most users won't get it or enable it.
>
> So we will need CONFIG_RUST=n to make sure that code for the customers
> work and we will need CONFIG_RUST=y to make sure that at least
> compilation passes.

Ah, I think I see where the confusion comes from. The idea is that if
we are to have Rust as an actual second language, then `CONFIG_RUST`
is enabled by default. Or, putting it another way, `CONFIG_RUST` will
go away at some point.

IIRC this is what Linus wanted -- for the moment the option is there
because we thought it would give a bit of freedom to folks in case
they consider it too experimental for their purposes, plus it allowed
us to allow Rust-enabled kernels built with GCC (which are a hack).

But we can definitely take it out already if kernel folks prefer to
reduce the matrix already (in which case, we will just remove
GCC-built kernels support or put *that* under a different flag).

> Thanks for the summary, I just disagree with the punch line:
>  "the advantages of using Rust outweighs the cost".

That's fair, but you said you didn't know what the advantages are at
the top of your message, so you cannot really tell if they outweigh or
not the cost ;P

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Leon Romanovsky]

On Tue, Jul 06, 2021 at 05:29:06PM +0200, Miguel Ojeda wrote:
> On Tue, Jul 6, 2021 at 4:56 PM Leon Romanovsky <leon@kernel.org> wrote:

<...>

> > Thanks for the summary, I just disagree with the punch line:
> >  "the advantages of using Rust outweighs the cost".
> 
> That's fair, but you said you didn't know what the advantages are at
> the top of your message, so you cannot really tell if they outweigh or
> not the cost ;P

There are different levels of not knowing :). The more accurate sentence
will be: "I'm aware of general message from Rust believers, but I don't know
about advantages of Rust for the kernel and what REAL problems it solves, which
are not possible or extremely hard to solve with C."

Thanks

> 
> Cheers,
> Miguel

Re: [TECH TOPIC] Rust for Linux

[Roland Dreier]

On Mon, Jul 5, 2021 at 4:51 PM Linus Walleij <linus.walleij@linaro.org> wrote:

> I also repeat my challenge from the mailing list, which is that
> while I understand that leaf drivers is a convenient place to start
> doing Rust code, the question is if that is where Rust as a language
> will pay off. My suggestion is to investigate git logs for the kind of
> problems that Rust solves best and then introduce Rust where
> those problems are. I believe that this would make be the
> Perfect(TM) selling point for the language in the strict
> hypothetico-deductive sense:

One area I see where Rust could make a big improvement for drivers is
in using RAII for error paths.  Drivers often have a lot of code like

    if (something())
        return err;

    if (something_else())
        goto undo_something;

    if (a_third_thing())
        goto undo_two_things;

and so on, which is difficult to get right in the first place and even
harder to keep correct in the face of changes.

That leads to a lot of bugs in error paths, which maybe affect no one
until they become exploitable security bugs, and also means that
maintainers get a lot of patches like
https://lore.kernel.org/linux-rdma/20200523030457.16160-1-wu000273@umn.edu/
that need careful review.

"devres" / devm_xxx was an attempt to deal with this in C, but it only
solves some cases of this and has not received a lot of adoption (we
can argue about the reasons).  Better language-level support for
ownership / lifetime tied to scope would probably lead to driver code
that is better in several dimensions.  Even in C++, the RAII idiom is
quite nice and I often miss it in kernel code.

 - R.

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier <roland@kernel.org> wrote:

> One area I see where Rust could make a big improvement for drivers is
> in using RAII for error paths.  Drivers often have a lot of code like
>
>     if (something())
>         return err;
>
>     if (something_else())
>         goto undo_something;
>
>     if (a_third_thing())
>         goto undo_two_things;
>
> and so on, which is difficult to get right in the first place and even
> harder to keep correct in the face of changes.

Yes.

> "devres" / devm_xxx was an attempt to deal with this in C, but it only
> solves some cases of this and has not received a lot of adoption (we
> can argue about the reasons).

Really? From my point of view that is adopted all over the map.
I add new users all the time and use it as much as I can when
writing new drivers.

$ git grep devm_  | wc -l
26112

Dmitry in the input subsystem even insist to use it for e.g. powering
down and disabling regulators on remove(), like recently in
drivers/input/touchscreen/cy8ctma140.c

/* Called from the registered devm action */
static void cy8ctma140_power_off_action(void *d)
{
        struct cy8ctma140 *ts = d;

        cy8ctma140_power_down(ts);
}
(...)
error = devm_add_action_or_reset(dev, cy8ctma140_power_off_action, ts);
if (error) {
        dev_err(dev, "failed to install power off handler\n");
        return error;
}

I think it's a formidable success, people just need to learn to do it more.

But if an easier path to learn better behaviours is to shuffle the whole
chessboard and replace it with drivers written in Rust, I don't know?
Maybe?

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:
> 
> > One area I see where Rust could make a big improvement for drivers is
> > in using RAII for error paths.  Drivers often have a lot of code like
> >
> >     if (something())
> >         return err;
> >
> >     if (something_else())
> >         goto undo_something;
> >
> >     if (a_third_thing())
> >         goto undo_two_things;
> >
> > and so on, which is difficult to get right in the first place and even
> > harder to keep correct in the face of changes.
> 
> Yes.
> 
> > "devres" / devm_xxx was an attempt to deal with this in C, but it only
> > solves some cases of this and has not received a lot of adoption (we
> > can argue about the reasons).
> 
> Really? From my point of view that is adopted all over the map.
> I add new users all the time and use it as much as I can when
> writing new drivers.
> 
> $ git grep devm_  | wc -l
> 26112
> 
> Dmitry in the input subsystem even insist to use it for e.g. powering
> down and disabling regulators on remove(), like recently in
> drivers/input/touchscreen/cy8ctma140.c
> 
> /* Called from the registered devm action */
> static void cy8ctma140_power_off_action(void *d)
> {
>         struct cy8ctma140 *ts = d;
> 
>         cy8ctma140_power_down(ts);
> }
> (...)
> error = devm_add_action_or_reset(dev, cy8ctma140_power_off_action, ts);
> if (error) {
>         dev_err(dev, "failed to install power off handler\n");
>         return error;
> }
> 
> I think it's a formidable success, people just need to learn to do it more.

devres is interesting and has good use cases, but the devm_k*alloc() are
a nightmare. They're used through drivers without any consideration of
life time management of the allocated memory, to allocate data
structures that can be accessed in userspace-controlled code paths.
Open a device node, keep it open, unplug the device, close the device
node, and in many cases you'll find that the kernel can crash :-( When
the first line of the probe function of a driver that exposes a chardev
if a devm_kzalloc(), it's usually a sign that something is wrong.

I wonder if we could also benefit from the gcc cleanup attribute
(https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#index-cleanup-variable-attribute).
I don't know if it's supported by the other compilers we care about
though.

> But if an easier path to learn better behaviours is to shuffle the whole
> chessboard and replace it with drivers written in Rust, I don't know?
> Maybe?

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Julia Lawall]

On Wed, 7 Jul 2021, Laurent Pinchart wrote:

> On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:
> >
> > > One area I see where Rust could make a big improvement for drivers is
> > > in using RAII for error paths.  Drivers often have a lot of code like
> > >
> > >     if (something())
> > >         return err;
> > >
> > >     if (something_else())
> > >         goto undo_something;
> > >
> > >     if (a_third_thing())
> > >         goto undo_two_things;
> > >
> > > and so on, which is difficult to get right in the first place and even
> > > harder to keep correct in the face of changes.
> >
> > Yes.
> >
> > > "devres" / devm_xxx was an attempt to deal with this in C, but it only
> > > solves some cases of this and has not received a lot of adoption (we
> > > can argue about the reasons).
> >
> > Really? From my point of view that is adopted all over the map.
> > I add new users all the time and use it as much as I can when
> > writing new drivers.
> >
> > $ git grep devm_  | wc -l
> > 26112
> >
> > Dmitry in the input subsystem even insist to use it for e.g. powering
> > down and disabling regulators on remove(), like recently in
> > drivers/input/touchscreen/cy8ctma140.c
> >
> > /* Called from the registered devm action */
> > static void cy8ctma140_power_off_action(void *d)
> > {
> >         struct cy8ctma140 *ts = d;
> >
> >         cy8ctma140_power_down(ts);
> > }
> > (...)
> > error = devm_add_action_or_reset(dev, cy8ctma140_power_off_action, ts);
> > if (error) {
> >         dev_err(dev, "failed to install power off handler\n");
> >         return error;
> > }
> >
> > I think it's a formidable success, people just need to learn to do it more.
>
> devres is interesting and has good use cases, but the devm_k*alloc() are
> a nightmare. They're used through drivers without any consideration of
> life time management of the allocated memory, to allocate data
> structures that can be accessed in userspace-controlled code paths.
> Open a device node, keep it open, unplug the device, close the device
> node, and in many cases you'll find that the kernel can crash :-( When
> the first line of the probe function of a driver that exposes a chardev
> if a devm_kzalloc(), it's usually a sign that something is wrong.

Where should the free have been?  Will Rust help in this case?  Will it
result in a memory leak?

julia

>
> I wonder if we could also benefit from the gcc cleanup attribute
> (https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#index-cleanup-variable-attribute).
> I don't know if it's supported by the other compilers we care about
> though.
>
> > But if an easier path to learn better behaviours is to shuffle the whole
> > chessboard and replace it with drivers written in Rust, I don't know?
> > Maybe?
>
> --
> Regards,
>
> Laurent Pinchart
>
>

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Wed, Jul 07, 2021 at 09:27:32AM +0200, Julia Lawall wrote:
> 
> 
> On Wed, 7 Jul 2021, Laurent Pinchart wrote:
> 
> > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:
> > >
> > > > One area I see where Rust could make a big improvement for drivers is
> > > > in using RAII for error paths.  Drivers often have a lot of code like
> > > >
> > > >     if (something())
> > > >         return err;
> > > >
> > > >     if (something_else())
> > > >         goto undo_something;
> > > >
> > > >     if (a_third_thing())
> > > >         goto undo_two_things;
> > > >
> > > > and so on, which is difficult to get right in the first place and even
> > > > harder to keep correct in the face of changes.
> > >
> > > Yes.
> > >
> > > > "devres" / devm_xxx was an attempt to deal with this in C, but it only
> > > > solves some cases of this and has not received a lot of adoption (we
> > > > can argue about the reasons).
> > >
> > > Really? From my point of view that is adopted all over the map.
> > > I add new users all the time and use it as much as I can when
> > > writing new drivers.
> > >
> > > $ git grep devm_  | wc -l
> > > 26112
> > >
> > > Dmitry in the input subsystem even insist to use it for e.g. powering
> > > down and disabling regulators on remove(), like recently in
> > > drivers/input/touchscreen/cy8ctma140.c
> > >
> > > /* Called from the registered devm action */
> > > static void cy8ctma140_power_off_action(void *d)
> > > {
> > >         struct cy8ctma140 *ts = d;
> > >
> > >         cy8ctma140_power_down(ts);
> > > }
> > > (...)
> > > error = devm_add_action_or_reset(dev, cy8ctma140_power_off_action, ts);
> > > if (error) {
> > >         dev_err(dev, "failed to install power off handler\n");
> > >         return error;
> > > }
> > >
> > > I think it's a formidable success, people just need to learn to do it more.
> >
> > devres is interesting and has good use cases, but the devm_k*alloc() are
> > a nightmare. They're used through drivers without any consideration of
> > life time management of the allocated memory, to allocate data
> > structures that can be accessed in userspace-controlled code paths.
> > Open a device node, keep it open, unplug the device, close the device
> > node, and in many cases you'll find that the kernel can crash :-( When
> > the first line of the probe function of a driver that exposes a chardev
> > if a devm_kzalloc(), it's usually a sign that something is wrong.
> 
> Where should the free have been?  Will Rust help in this case?  Will it
> result in a memory leak?

This is going to be the "interesting" part of the rust work, where it
has to figure out how to map the reference counted objects that we
currently have in the driver model across to rust-controlled objects and
keep everything in sync properly.

For the normal code, the fact that the memory was assigned to one
specific object (the struct device) but yet referenced from another
object (the cdev).  devm_* users like this do not seem to realize there
are two separate object lifecycles happening here as the interactions
are subtle at times.

I am looking forward to how the rust implementation is going to handle
all of this as I have no idea.

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[James Bottomley]

On Wed, 2021-07-07 at 09:45 +0200, Greg KH wrote:
> On Wed, Jul 07, 2021 at 09:27:32AM +0200, Julia Lawall wrote:
[...]
> > Where should the free have been?  Will Rust help in this
> > case?  Will it result in a memory leak?
> 
> This is going to be the "interesting" part of the rust work, where it
> has to figure out how to map the reference counted objects that we
> currently have in the driver model across to rust-controlled objects
> and keep everything in sync properly.

Rust has a reference counted pointer: Rc<T>.  However the problem is
the rust memory model is via strict accounting, so once you start using
refcounts, which need shared accounting, you can then get memory leaks
and it's unclear if rust actually buys you anything over C for this use
case.

James

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 9:52 AM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> Rust has a reference counted pointer: Rc<T>.  However the problem is
> the rust memory model is via strict accounting, so once you start using
> refcounts, which need shared accounting, you can then get memory leaks
> and it's unclear if rust actually buys you anything over C for this use
> case.

Even if you are using `Rc<T>`, Rust buys you a *lot*. For starters, it
still prevents UB and data races -- e.g. it prevents mistakenly
sharing the `Rc` with other threads.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[James Bottomley]

On Wed, 2021-07-07 at 15:49 +0200, Miguel Ojeda wrote:
> On Wed, Jul 7, 2021 at 9:52 AM James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> > Rust has a reference counted pointer: Rc<T>.  However the problem
> > is the rust memory model is via strict accounting, so once you
> > start using refcounts, which need shared accounting, you can then
> > get memory leaks and it's unclear if rust actually buys you
> > anything over C for this use case.
> 
> Even if you are using `Rc<T>`, Rust buys you a *lot*. For starters,
> it still prevents UB

UB is Undefined Behaviour, right?  C plus the memory model should mean
we don't have that in the kernel, although that's enforced by
inspection and checking rather than the compiler.

>  and data races -- e.g. it prevents mistakenly sharing the `Rc` with
> other threads.

I'm not sure I get the point here: all the kernel refcount models are
explicitly multi-threaded, because we usually expect multiple CPU
threads to be using objects simultaneously ... it's why the base
implementation, kref, uses atomics.  That's not to say we don't have
single threaded use cases ... it's just that's not what's commonly
used.

That does beg another question which I couldn't find the answer to in
the base docs: the Rust Rc<T> is saturation protected, isn't it? or
would the rust implementation inherit that from the kernel?

James

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 4:08 PM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> UB is Undefined Behaviour, right?  C plus the memory model should mean
> we don't have that in the kernel, although that's enforced by
> inspection and checking rather than the compiler.

Yes, it is Undefined Behavior. I am sure you know, but just in case:
it covers things like the usual out-of-bounds accesses, using objects
after their lifetime has ended, data races, etc., plus quite a few
other things.

There are 200+ instances of UB listed in the Annex J.2 list of the C
standard [1]. SEI CERT also has a nice list [2].

On its own, UB is not bad. Nowadays, some instances of UB are used by
optimizers to give us the performance we all love.

However, the problem comes when we, as humans, mistakenly introduce
UB. Even someone that is very knowledgeable about C introduces UB from
time to time. We all do. The reason is simple: reasoning about UB is
typically non-local, e.g. we may be doing a change in some function
that changes a pointer, and that change on its own may be fine, but we
did not account for some invariant in another function that uses the
pointer.

Other times UB appears because the optimizer had different assumptions
than the person writing the code / project / environment, e.g. the
`-fno-delete-null-pointer-checks` issue we had in the kernel a decade
ago, where the optimizer removes a branch because you already accessed
a pointer.

In summary, relying on inspection to avoid UB is a losing battle. That
is why many tools appeared throughout the decades to attack different
parts: Valgrind (Memcheck, Helgrind, DRD...), ASAN, UBSAN, Miri, etc.

Rust, like C, has UB. The difference is that Rust has a safe subset
where UB is statically guaranteed to not occur (as long as the unsafe
code is sound). This is key: you "only" need to make sure the unsafe
parts are not breaking the rules, and if that is true, Rust guarantees
the safe code does not break them either.

This means the work of inspection / checking / running UBSAN and
friends, etc. can be focused on the unsafe bits.

[1] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2596.pdf
[2] https://wiki.sei.cmu.edu/confluence/display/c/CC.+Undefined+Behavior.

> I'm not sure I get the point here: all the kernel refcount models are
> explicitly multi-threaded, because we usually expect multiple CPU
> threads to be using objects simultaneously ... it's why the base
> implementation, kref, uses atomics.  That's not to say we don't have
> single threaded use cases ... it's just that's not what's commonly
> used.

Yes, `Rc` is a type meant for single-threaded reference-counting
scenarios. For multi-threaded scenarios, Rust provides `Arc` instead.
It still guarantees no UB and no data races.

This is actually a good example of another benefit. `Rc` is there
because there is a performance advantage when you do not need to share
it among threads (you do not need to use atomics) -- and Rust checks
you do not do so. In C, you can still have an `Rc`, but you would not
have a compile-time guarantee that it is not shared. So in C you may
decide to avoid your single-threaded `Rc` and use `Arc` "just in case"
-- and, in fact, this is the only one C++ has (`std::shared_ptr`), and
even then, it does not statically prevent UB.

> That does beg another question which I couldn't find the answer to in
> the base docs: the Rust Rc<T> is saturation protected, isn't it? or
> would the rust implementation inherit that from the kernel?

We can do both: we can introduce new types with the precise semantics
we need in pure Rust, but we can also "abstract" C facilities.

For instance, we have a `Ref` type that is similar to `Arc` but reuses
the `refcount_t` from C and introduces saturation instead of aborting
[3]

Another example are the red-black trees that abstract the C implementation [4].

[3] https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/sync/arc.rs
[4] https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/rbtree.rs

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Wed, Jul 07, 2021 at 05:15:01PM +0200, Miguel Ojeda wrote:
> For instance, we have a `Ref` type that is similar to `Arc` but reuses
> the `refcount_t` from C and introduces saturation instead of aborting
> [3]
> 
> [3] https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/sync/arc.rs

This is interesting code in that I think you are missing the part where
you still need a lock on the object to prevent one thread from grabbing
a reference while another one is dropping the last reference.  Or am I
missing something?

The code here:

   fn drop(&mut self) {
        // SAFETY: By the type invariant, there is necessarily a reference to the object. We cannot
        // touch `refcount` after it's decremented to a non-zero value because another thread/CPU
        // may concurrently decrement it to zero and free it. It is ok to have a raw pointer to
        // freed/invalid memory as long as it is never dereferenced.
        let refcount = unsafe { self.ptr.as_ref() }.refcount.get();

        // INVARIANT: If the refcount reaches zero, there are no other instances of `Ref`, and
        // this instance is being dropped, so the broken invariant is not observable.
        // SAFETY: Also by the type invariant, we are allowed to decrement the refcount.
        let is_zero = unsafe { rust_helper_refcount_dec_and_test(refcount) };
        if is_zero {
            // The count reached zero, we must free the memory.
            //
            // SAFETY: The pointer was initialised from the result of `Box::leak`.
            unsafe { Box::from_raw(self.ptr.as_ptr()) };
        }
   }

Has a lot of interesting comments, and maybe just because I know nothing
about Rust, but why on the first line of the comment is there always
guaranteed a reference to the object at this point in time?  And yes
it's ok to have a pointer to memory that is not dereferenced, but is
that what is happening here?

I feel you are trying to duplicate the logic of a "struct kref" here,
and that requires a lock to work properly.  Where is the lock here?

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Wed, Jul 07, 2021 at 05:44:41PM +0200, Greg KH wrote:
> On Wed, Jul 07, 2021 at 05:15:01PM +0200, Miguel Ojeda wrote:
> > For instance, we have a `Ref` type that is similar to `Arc` but reuses
> > the `refcount_t` from C and introduces saturation instead of aborting
> > [3]
> > 
> > [3] https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/sync/arc.rs
> 
> This is interesting code in that I think you are missing the part where
> you still need a lock on the object to prevent one thread from grabbing
> a reference while another one is dropping the last reference.  Or am I
> missing something?

You are missing something :)

> The code here:
> 
>    fn drop(&mut self) {
>         // SAFETY: By the type invariant, there is necessarily a reference to the object. We cannot
>         // touch `refcount` after it's decremented to a non-zero value because another thread/CPU
>         // may concurrently decrement it to zero and free it. It is ok to have a raw pointer to
>         // freed/invalid memory as long as it is never dereferenced.
>         let refcount = unsafe { self.ptr.as_ref() }.refcount.get();
> 
>         // INVARIANT: If the refcount reaches zero, there are no other instances of `Ref`, and
>         // this instance is being dropped, so the broken invariant is not observable.
>         // SAFETY: Also by the type invariant, we are allowed to decrement the refcount.
>         let is_zero = unsafe { rust_helper_refcount_dec_and_test(refcount) };
>         if is_zero {
>             // The count reached zero, we must free the memory.
>             //
>             // SAFETY: The pointer was initialised from the result of `Box::leak`.
>             unsafe { Box::from_raw(self.ptr.as_ptr()) };
>         }
>    }
> 
> Has a lot of interesting comments, and maybe just because I know nothing
> about Rust, but why on the first line of the comment is there always
> guaranteed a reference to the object at this point in time?

It's an invariant of the `Ref<T>` type: if a `Ref<T>` exists, there necessarily
is a non-zero reference count. You cannot have a `Ref<T>` with a zero refcount.

`drop` is called when a `Ref<T>` is about to be destroyed. Since it is about to
be destroyed, it still exists, therefore the ref-count is necessarily non-zero.

> And yes
> it's ok to have a pointer to memory that is not dereferenced, but is
> that what is happening here?

`refcount` is a raw pointer. When it is declared and initialised, it points to
valid memory. The comment is saying that we should be careful with the case
where another thread ends up freeing the object (after this thread has
decremented its share of the count); and that we're not violating any Rust
aliasing rules by having this raw pointer (as long as we don't dereference it).
 
> I feel you are trying to duplicate the logic of a "struct kref" here,

`struct kref` would give us the ability to specify a `release` function when
calling `refcount_dec_and_test`, but we don't need this round-trip to C code
because we know at compile-time what the `release` function is: it's the `drop`
implementation for the wrapped type (`T` in `Ref<T>`).

> and that requires a lock to work properly.  Where is the lock here?

We don't need a lock. Once the refcount reaches zero, we know that nothing else
has pointers to the memory block; the lifetime rules guarantee that if there is
a reference to a `Ref<T>`, then it cannot outlive the `Ref<T>` itself. To
produce a new `Ref<T>` from an existing one, `clone` is called, which increments
the refcount.

For cases when you want to hold on to something without incrementing its
refcount, the common pattern in Rust is to use "weak" pointers. `Ref<T>` doesn't
support them because we haven't needed them yet and they have extra cost.
(`Arc<T>` supports them though.)

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Wed, Jul 07, 2021 at 06:01:41PM +0100, Wedson Almeida Filho wrote:
> On Wed, Jul 07, 2021 at 05:44:41PM +0200, Greg KH wrote:
> > On Wed, Jul 07, 2021 at 05:15:01PM +0200, Miguel Ojeda wrote:
> > > For instance, we have a `Ref` type that is similar to `Arc` but reuses
> > > the `refcount_t` from C and introduces saturation instead of aborting
> > > [3]
> > > 
> > > [3] https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/sync/arc.rs
> > 
> > This is interesting code in that I think you are missing the part where
> > you still need a lock on the object to prevent one thread from grabbing
> > a reference while another one is dropping the last reference.  Or am I
> > missing something?
> 
> You are missing something :)
> 
> > The code here:
> > 
> >    fn drop(&mut self) {
> >         // SAFETY: By the type invariant, there is necessarily a reference to the object. We cannot
> >         // touch `refcount` after it's decremented to a non-zero value because another thread/CPU
> >         // may concurrently decrement it to zero and free it. It is ok to have a raw pointer to
> >         // freed/invalid memory as long as it is never dereferenced.
> >         let refcount = unsafe { self.ptr.as_ref() }.refcount.get();
> > 
> >         // INVARIANT: If the refcount reaches zero, there are no other instances of `Ref`, and
> >         // this instance is being dropped, so the broken invariant is not observable.
> >         // SAFETY: Also by the type invariant, we are allowed to decrement the refcount.
> >         let is_zero = unsafe { rust_helper_refcount_dec_and_test(refcount) };
> >         if is_zero {
> >             // The count reached zero, we must free the memory.
> >             //
> >             // SAFETY: The pointer was initialised from the result of `Box::leak`.
> >             unsafe { Box::from_raw(self.ptr.as_ptr()) };
> >         }
> >    }
> > 
> > Has a lot of interesting comments, and maybe just because I know nothing
> > about Rust, but why on the first line of the comment is there always
> > guaranteed a reference to the object at this point in time?
> 
> It's an invariant of the `Ref<T>` type: if a `Ref<T>` exists, there necessarily
> is a non-zero reference count. You cannot have a `Ref<T>` with a zero refcount.

What enforces that?  Where is the lock on the "back end" for `Ref<T>`
that one CPU from grabbing a reference at the same time the "last"
reference is dropped on a different CPU?

Does Rust provide "architecture-specific" locks like this somehow that
are "built in"?  If so, what happens when we need to fix those locks?
Does that get fixed in the compiler, not the kernel code?

> `drop` is called when a `Ref<T>` is about to be destroyed. Since it is about to
> be destroyed, it still exists, therefore the ref-count is necessarily non-zero.

"about to", yes, but what keeps someone else from grabbing it?

> > And yes
> > it's ok to have a pointer to memory that is not dereferenced, but is
> > that what is happening here?
> 
> `refcount` is a raw pointer. When it is declared and initialised, it points to
> valid memory. The comment is saying that we should be careful with the case
> where another thread ends up freeing the object (after this thread has
> decremented its share of the count); and that we're not violating any Rust
> aliasing rules by having this raw pointer (as long as we don't dereference it).
>  
> > I feel you are trying to duplicate the logic of a "struct kref" here,
> 
> `struct kref` would give us the ability to specify a `release` function when
> calling `refcount_dec_and_test`, but we don't need this round-trip to C code
> because we know at compile-time what the `release` function is: it's the `drop`
> implementation for the wrapped type (`T` in `Ref<T>`).

That's not what I meant by bringing up a kref.  I was trying to ask
where the "real" lock here is.  It has to be somewhere...

> > and that requires a lock to work properly.  Where is the lock here?
> 
> We don't need a lock. Once the refcount reaches zero, we know that nothing else
> has pointers to the memory block; the lifetime rules guarantee that if there is
> a reference to a `Ref<T>`, then it cannot outlive the `Ref<T>` itself.

Even with multiple CPUs?  What enforces these lifetime rules?

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Wed, Jul 07, 2021 at 07:20:44PM +0200, Greg KH wrote:
> On Wed, Jul 07, 2021 at 06:01:41PM +0100, Wedson Almeida Filho wrote:
> > On Wed, Jul 07, 2021 at 05:44:41PM +0200, Greg KH wrote:
> > > On Wed, Jul 07, 2021 at 05:15:01PM +0200, Miguel Ojeda wrote:
> > > > For instance, we have a `Ref` type that is similar to `Arc` but reuses
> > > > the `refcount_t` from C and introduces saturation instead of aborting
> > > > [3]
> > > > 
> > > > [3] https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/sync/arc.rs
> > > 
> > > This is interesting code in that I think you are missing the part where
> > > you still need a lock on the object to prevent one thread from grabbing
> > > a reference while another one is dropping the last reference.  Or am I
> > > missing something?
> > 
> > You are missing something :)
> > 
> > > The code here:
> > > 
> > >    fn drop(&mut self) {
> > >         // SAFETY: By the type invariant, there is necessarily a reference to the object. We cannot
> > >         // touch `refcount` after it's decremented to a non-zero value because another thread/CPU
> > >         // may concurrently decrement it to zero and free it. It is ok to have a raw pointer to
> > >         // freed/invalid memory as long as it is never dereferenced.
> > >         let refcount = unsafe { self.ptr.as_ref() }.refcount.get();
> > > 
> > >         // INVARIANT: If the refcount reaches zero, there are no other instances of `Ref`, and
> > >         // this instance is being dropped, so the broken invariant is not observable.
> > >         // SAFETY: Also by the type invariant, we are allowed to decrement the refcount.
> > >         let is_zero = unsafe { rust_helper_refcount_dec_and_test(refcount) };
> > >         if is_zero {
> > >             // The count reached zero, we must free the memory.
> > >             //
> > >             // SAFETY: The pointer was initialised from the result of `Box::leak`.
> > >             unsafe { Box::from_raw(self.ptr.as_ptr()) };
> > >         }
> > >    }
> > > 
> > > Has a lot of interesting comments, and maybe just because I know nothing
> > > about Rust, but why on the first line of the comment is there always
> > > guaranteed a reference to the object at this point in time?
> > 
> > It's an invariant of the `Ref<T>` type: if a `Ref<T>` exists, there necessarily
> > is a non-zero reference count. You cannot have a `Ref<T>` with a zero refcount.
> 
> What enforces that?  Where is the lock on the "back end" for `Ref<T>`
> that one CPU from grabbing a reference at the same time the "last"
> reference is dropped on a different CPU?

There is no lock. I think you might be conflating kobject concepts with general
reference counting.

The enforcement is done at compile time by the Rust aliasing and lifetime rules:
the owner of a piece of memory has exclusive access to it, except when it is
borrowed. When it is borrowed, the borrow *must not* outlive the memory being
borrowed.

Unsafe code may break these rules, but if it exposes a safe interface (which
`Ref` does), then this interface must obey these rules.

Here's a simple example. Suppose we have a struct like:

struct X {
    a: u64,
    b: u64,
}

And we want to create a reference-counted instance of it, we would write:

  let ptr = Ref::new(X { a: 10, b: 20 });

(Note that we don't need to embed anything in the struct, we just wrap it with
the `Ref` type. In this case, the type of `ptr` is `Ref<X>` which is a
ref-counted instance of `X`.)

At this point we can agree that there are no other pointers to this. So if we
`ptr` went out of scope, the refcount would drop to zero and the memory would be
freed.

Now suppose I want to call some function that takes a reference to `X` (a
const pointer to `X` in C parlance), say:

fn testfunc(ptr_ref: &X) {
  ...
}

This reference has a lifetime associated with it. The compiler won't allow
implementations where using `ptr_ref` would outlive the original `ptr`, for
example if it escapes `testfunc` (for example, to another thread) but doesn't
"come back" before the end of the function (for example, if `testfunc` "joined"
the thread). Here's a trivial example with scopes to demonstrate the sort of
compiler error we'd get:

fn main() {
    let ptr_ref;
    {
        let ptr = Ref::new(X { a: 10, b: 20 });
        ptr_ref = &ptr;
    }
    println!("{}", ptr_ref.a);
}

Compiling this results in the following error:

error[E0597]: `ptr` does not live long enough
  --> src/main.rs:12:19
   |
12 |         ptr_ref = &ptr;
   |                   ^^^^ borrowed value does not live long enough
13 |     }
   |     - `ptr` dropped here while still borrowed
14 |     println!("{}", ptr_ref.a);
   |                    ------- borrow later used here

Following these rules, the compiler *guarantees* that if a thread or CPU somehow
has access to a reference to a `Ref<T>`, then it *must* be backed by a real
`Ref<T>` somewhere: a borrowed value must never outlive what it's borrowing. So
incrementing the refcount is necessarily from n to n+1 where n > 0 because the
existing reference guarantees that n > 0.

There are real cases when you can't guarantee that lifetimes line up as required
by the compiler to guarantee safety. In such cases, you can "clone" ptr (which
increments the refcount, again from n to n+1, where n > 0), then you end up with
your own reference to the underlying `X`, for example:

fn main() {
    let ptr_clone;
    {
        let ptr = Ref::new(X { a: 10, b: 20 });
        ptr_clone = ptr.clone();
    }
    println!("{}", ptr_clone.a);
}

(Note that the reference owned by `ptr` has been destroyed by the time
`ptr_clone.a` is used in `println`, but `ptr_clone` has its own reference due to
the clone call.)

The ideas above apply equally well if instead of thinking in terms of scope, you
think in terms of threads/CPUs. If a thread needs a refcounted object to
potentially outlive the borrow keeping it alive, then it needs to increment
the refcount: if you can't prove the lifetime rules, then you must clone the
reference.

Given that by construction the refcount starts at 1, there is no path to go from
0 to 1. Ever.

Where would a lock be needed in the examples above?

> Does Rust provide "architecture-specific" locks like this somehow that
> are "built in"?  If so, what happens when we need to fix those locks?
> Does that get fixed in the compiler, not the kernel code?

There are no magic locks implemented by the compiler.

> > `drop` is called when a `Ref<T>` is about to be destroyed. Since it is about to
> > be destroyed, it still exists, therefore the ref-count is necessarily non-zero.
> 
> "about to", yes, but what keeps someone else from grabbing it?

See my comments above. I'm happy to discuss any details that may not be clear.

> > > And yes
> > > it's ok to have a pointer to memory that is not dereferenced, but is
> > > that what is happening here?
> > 
> > `refcount` is a raw pointer. When it is declared and initialised, it points to
> > valid memory. The comment is saying that we should be careful with the case
> > where another thread ends up freeing the object (after this thread has
> > decremented its share of the count); and that we're not violating any Rust
> > aliasing rules by having this raw pointer (as long as we don't dereference it).
> >  
> > > I feel you are trying to duplicate the logic of a "struct kref" here,
> > 
> > `struct kref` would give us the ability to specify a `release` function when
> > calling `refcount_dec_and_test`, but we don't need this round-trip to C code
> > because we know at compile-time what the `release` function is: it's the `drop`
> > implementation for the wrapped type (`T` in `Ref<T>`).
> 
> That's not what I meant by bringing up a kref.  I was trying to ask
> where the "real" lock here is.  It has to be somewhere...
> 
> > > and that requires a lock to work properly.  Where is the lock here?
> > 
> > We don't need a lock. Once the refcount reaches zero, we know that nothing else
> > has pointers to the memory block; the lifetime rules guarantee that if there is
> > a reference to a `Ref<T>`, then it cannot outlive the `Ref<T>` itself.
> 
> Even with multiple CPUs?  What enforces these lifetime rules?

The compiler does, at compile-time. Each lifetime usage is a constraint that
must be satisfied by the compiler; once all constraints are gathered for a given
function, the compiler tries to solve them, if it can find a solution, then the
code is accepted; if it can't find a solution, the code is rejected. Note that
this means that some correct code may be rejected the compiler by design: it is
conservative in that it only accepts what it can prove is correct.

Re: [TECH TOPIC] Rust for Linux

[Jan Kara]

On Wed 07-07-21 20:19:19, Wedson Almeida Filho wrote:
<snip good explanations how refs work>

> There are real cases when you can't guarantee that lifetimes line up as required
> by the compiler to guarantee safety. In such cases, you can "clone" ptr (which
> increments the refcount, again from n to n+1, where n > 0), then you end up with
> your own reference to the underlying `X`, for example:
> 
> fn main() {
>     let ptr_clone;
>     {
>         let ptr = Ref::new(X { a: 10, b: 20 });
>         ptr_clone = ptr.clone();
>     }
>     println!("{}", ptr_clone.a);
> }
> 
> (Note that the reference owned by `ptr` has been destroyed by the time
> `ptr_clone.a` is used in `println`, but `ptr_clone` has its own reference due to
> the clone call.)
> 
> The ideas above apply equally well if instead of thinking in terms of scope, you
> think in terms of threads/CPUs. If a thread needs a refcounted object to
> potentially outlive the borrow keeping it alive, then it needs to increment
> the refcount: if you can't prove the lifetime rules, then you must clone the
> reference.
> 
> Given that by construction the refcount starts at 1, there is no path to go from
> 0 to 1. Ever.
> 
> Where would a lock be needed in the examples above?

So I think Greg speaks about a situation where you have multiple threads
and the refcounted object can be looked up through some structure all the
threads see. And the problem is that the shared data structure cannot hold
ref to the object it points to because you want to detect the situation
where the data structure is the only place pointing to the object and
reclaim the object in that case. Currently I don't see how to model this
idiom with Rust refs.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Wed, Jul 07, 2021 at 10:38:27PM +0200, Jan Kara wrote:
> On Wed 07-07-21 20:19:19, Wedson Almeida Filho wrote:
> > Where would a lock be needed in the examples above?
> 
> So I think Greg speaks about a situation where you have multiple threads
> and the refcounted object can be looked up through some structure all the
> threads see. And the problem is that the shared data structure cannot hold
> ref to the object it points to because you want to detect the situation
> where the data structure is the only place pointing to the object and
> reclaim the object in that case. Currently I don't see how to model this
> idiom with Rust refs.

The normal idiom in Rust for this is "weak" pointers. With it, each
reference-counted object has two counts: strong and weak refs. Objects are
"destroyed" when the strong count goes to zero and "freed" when the weak count
goes to zero.

Weak references need to upgraded to strong references before the underlying
objects can be accessed; upgrading may fail if the strong count has gone to
zero. It is, naturally, implemented as an increment that avoids going from 0 to
1. It is safe to try to do it because the memory is kept alive while there are
weak references.

For the case you mention, the list would be based on weak references. If the
object's destructor also removes the object from the list, both counts will go
to zero and the object will be freed as well. (If it fails to do so, the
*memory* will linger allocated until someone removes the object from the list,
but all attempts to upgrade the weak reference to a strong one will fail.)

The obvious cost is that we need an extra 32-bit number per reference-counted
allocation. But if we have specialized cases, like the underlying object always
being in some data structure until the ref count goes to zero, then we can build
a zero-cost abstraction for such a scenario.

We can also build specialised zero-cost abstractions for the case when we want
to avoid the 1 -> 0 transition unless we're holding some lock to prevent others
observing the object-with-zero-ref. For this I'd have to spend more time to see
if we can do safely (i.e., with compile-time guarantees that the object was
actually removed from the data structure).

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Thu, Jul 08, 2021 at 12:09:16AM +0100, Wedson Almeida Filho wrote:
> On Wed, Jul 07, 2021 at 10:38:27PM +0200, Jan Kara wrote:
> > On Wed 07-07-21 20:19:19, Wedson Almeida Filho wrote:
> > > Where would a lock be needed in the examples above?
> > 
> > So I think Greg speaks about a situation where you have multiple threads
> > and the refcounted object can be looked up through some structure all the
> > threads see. And the problem is that the shared data structure cannot hold
> > ref to the object it points to because you want to detect the situation
> > where the data structure is the only place pointing to the object and
> > reclaim the object in that case. Currently I don't see how to model this
> > idiom with Rust refs.
> 
> The normal idiom in Rust for this is "weak" pointers. With it, each
> reference-counted object has two counts: strong and weak refs. Objects are
> "destroyed" when the strong count goes to zero and "freed" when the weak count
> goes to zero.
> 
> Weak references need to upgraded to strong references before the underlying
> objects can be accessed; upgrading may fail if the strong count has gone to
> zero. It is, naturally, implemented as an increment that avoids going from 0 to
> 1. It is safe to try to do it because the memory is kept alive while there are
> weak references.
> 
> For the case you mention, the list would be based on weak references. If the
> object's destructor also removes the object from the list, both counts will go
> to zero and the object will be freed as well. (If it fails to do so, the
> *memory* will linger allocated until someone removes the object from the list,
> but all attempts to upgrade the weak reference to a strong one will fail.)
> 
> The obvious cost is that we need an extra 32-bit number per reference-counted
> allocation. But if we have specialized cases, like the underlying object always
> being in some data structure until the ref count goes to zero, then we can build
> a zero-cost abstraction for such a scenario.
> 
> We can also build specialised zero-cost abstractions for the case when we want
> to avoid the 1 -> 0 transition unless we're holding some lock to prevent others
> observing the object-with-zero-ref. For this I'd have to spend more time to see
> if we can do safely (i.e., with compile-time guarantees that the object was
> actually removed from the data structure).

Thanks for the detailed explainations, it seems rust can "get away" with
some things with regards to reference counts that the kernel can not.
Userspace has it easy, but note that now that rust is not in userspace,
dealing with multiple cpus/threads is going to be interesting for the
language.

So, along those lines, how are you going to tie rust's reference count
logic in with the kernel's reference count logic?  How are you going to
handle dentries, inodes, kobjects, devices and the like?  That's the
real question that I don't seem to see anyone even starting to answer
just yet.

And that's the reason some of us are asking to see a "real" driver, as
those have to deal with these kernel-controlled-reference-counted
objects properly (as well as hardware control).  Seeing how that is
going to work in this language is going to be the real sign of if this
is even going to be possible or not.

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Thu, Jul 08, 2021 at 08:11:10AM +0200, Greg KH wrote:
> Thanks for the detailed explainations, it seems rust can "get away" with
> some things with regards to reference counts that the kernel can not.
> Userspace has it easy, but note that now that rust is not in userspace,
> dealing with multiple cpus/threads is going to be interesting for the
> language.
>
> So, along those lines, how are you going to tie rust's reference count
> logic in with the kernel's reference count logic?  How are you going to
> handle dentries, inodes, kobjects, devices and the like?  That's the
> real question that I don't seem to see anyone even starting to answer
> just yet.

None of what I described before is specific to userspace, but I understand your
need to see something more concrete.

I'll describe what we've done for `task_struct` and how lifetimes, aliasing
rules, sync & send traits help us achieve zero cost (when compared to C) but
also gives us safety. The intention here is to show that similar things can (and
will) be done to other kernel reference-counted objects when we get to them.

So we begin with `current`. It gives us access to the current task without
incurring any increments or decrements of the refcount; in Rust, we'd do the
following:

  let current = Task::current();

We can use it for however long we'd like as long as it's in the same task. But
how do we restrict that? Rust has this `Send` trait that tells it that a type
can be used by another thread/CPU; `current`'s type doesn't isn't `Send`, so
attempting to send it another thread fails. For example, if we tried this:

  send_to_thread(current);

We'd get the following compiler error:

    |
195 | fn send_to_thread<T: Send>(t: T) {
    |                      ---- required by this bound in `send_to_thread`
...
201 |     send_to_thread(current);
    |     ^^^^^^^^^^^^^^ `*mut ()` cannot be sent between threads safely
    |
    = help: within `TaskRef<'_>`, the trait `Send` is not implemented for `*mut ()`
    = note: required because it appears within the type `(&(), *mut ())`
    = note: required because it appears within the type `PhantomData<(&(), *mut ())>`
note: required because it appears within the type `TaskRef<'_>`

One option we do have is to send a "reference" to `current` to another thread.
This works and is zero-cost. It is safe because the lifetime of the reference is
tied to that of `current`. (And `current`'s type is `Sync`, which means that a
reference to it is safely shareable with another thread/CPU.)

So calling:

  send_to_thread(&current);

Works fine. But its implementation must convince the compiler that by the time
it returns the sharing with another thread is over. Otherwise it would be a
violation of lifetime requirements (a borrow cannot outlive the borrowed value)
and compilation would fail.

Now, similarly to my example in another email, if you really want the task to
outlive `current`, then you can call `clone`. This results in `get_task_struct`
being called, so now we can send it another thread that can hold on to it and
this is all safe. For example:

    let current = Task::current();
    let task = current.clone();
    send_to_thread(task);

Now, the other task *owns* the reference, so we're not supposed to use `task` at
all (suppose for a moment that it's some arbitrary task, not current). In C,
given that there is no ownership discipline enforced by the compiler, one could
easily make the mistake of using `task` (which is unsafe because the other
thread/CPU may have decremented its refcount and freed it by now). In Rust, an
attempt to use `task` would fail; for example:

    let current = Task::current();
    let task = Task::current().clone();
    send_to_thread(task);
    pr_info!("Pid is {}", task.pid());

Would result in the following compilation error:

error[E0382]: borrow of moved value: `task`
   --> rust/kernel/task.rs:203:34
    |
201 |     let task = Task::current().clone();
    |         ---- move occurs because `task` has type `Task`, which does not implement the `Copy` trait
202 |     send_to_thread(task);
    |                    ---- value moved here
203 |     pr_info!("Pid is {}", task.pid());
    |                           ^^^^ value borrowed here after move

(Note that `task` is inaccessible despite still being in scope.)

Another common mistake is to leak references by forgetting to call
`put_task_struct`. Rust helps prevent this by automatically calling it when
needed, including error paths. Suppose we have a function that allocates some
struct that includes a task field, for example:

struct X {
    task: Task,
    a: u64,
    b: u64,
}

fn alloc_X(task: Task) -> Result<Box<X>> {
    Box::try_new(X {
        task: task,
        a: 10,
        b: 20
    })
}

Here, if the function fails (e.g., the allocation in `try_new` fails), the task
refcount is automatically decremented when the function returns. If it succeeds,
ownership is transferred to the new instance of X, and the refcount will be
decremented automatically when this instance of X is eventually freed.

Another example that shows lifetimes clearly is that of `group_leader`. Given a
task, its group leader can be accessed with zero-cost but this access is
subjected to lifetime requirements. For example:

    let task = get_some_task();
    let leader = task.group_leader();
    pr_info!("Pid is {}", leader.pid());

Works with zero cost (i.e., no extra inc/ref of the group leader). But the
following:

    let leader;
    {
        let task = get_some_task();
        leader = task.group_leader();
    }
    pr_info!("Pid is {}", leader.pid());

Fails with the following error:

error[E0597]: `task` does not live long enough
   --> rust/kernel/task.rs:209:18
    |
209 |         leader = task.group_leader();
    |                  ^^^^ borrowed value does not live long enough
210 |     }
    |     - `task` dropped here while still borrowed
211 |     pr_info!("Pid is {}", leader.pid());
    |                           ------ borrow later used here

Because task's refcount is decremented at the end of the scope.

So, you see, I understand that you want to see refcounts in action on the
objects you care about. But I don't think the claim that no one has even tried
to answer the general refcount question is accurate. I hope it is clear that we
have thought about some of this. You could also check out what we've done for
file references and even file-descriptor creation with transaction semantics for
how it all fits beautifully (and safely). (I won't get into them here because
this email is already too long.)

I'm happy to go into more details for any of the examples above if anything
isn't clear.

> And that's the reason some of us are asking to see a "real" driver, as
> those have to deal with these kernel-controlled-reference-counted
> objects properly (as well as hardware control).  Seeing how that is
> going to work in this language is going to be the real sign of if this
> is even going to be possible or not.

Here's what I'd like to avoid: spending time on something that you all still
think is not typical enough. Would you be able to point us to a driver that
would showcase the interactions you'd like to see so that (once we have it in
Rust) we can have a discussion about the merits of the language?

Hopefully something with interesting interactions with the kernel, but not
with overly complex hardware, that is, something that doesn't require us to read
a 400-page specification to implement.

Thanks,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Thu, Jul 08, 2021 at 02:36:40PM +0100, Wedson Almeida Filho wrote:
> On Thu, Jul 08, 2021 at 08:11:10AM +0200, Greg KH wrote:
> > Thanks for the detailed explainations, it seems rust can "get away" with
> > some things with regards to reference counts that the kernel can not.
> > Userspace has it easy, but note that now that rust is not in userspace,
> > dealing with multiple cpus/threads is going to be interesting for the
> > language.
> >
> > So, along those lines, how are you going to tie rust's reference count
> > logic in with the kernel's reference count logic?  How are you going to
> > handle dentries, inodes, kobjects, devices and the like?  That's the
> > real question that I don't seem to see anyone even starting to answer
> > just yet.
> 
> None of what I described before is specific to userspace, but I understand your
> need to see something more concrete.
> 
> I'll describe what we've done for `task_struct` and how lifetimes, aliasing
> rules, sync & send traits help us achieve zero cost (when compared to C) but
> also gives us safety. The intention here is to show that similar things can (and
> will) be done to other kernel reference-counted objects when we get to them.

<snip loads of good stuff here, thanks for helping describe all of this>


> > And that's the reason some of us are asking to see a "real" driver, as
> > those have to deal with these kernel-controlled-reference-counted
> > objects properly (as well as hardware control).  Seeing how that is
> > going to work in this language is going to be the real sign of if this
> > is even going to be possible or not.
> 
> Here's what I'd like to avoid: spending time on something that you all still
> think is not typical enough. Would you be able to point us to a driver that
> would showcase the interactions you'd like to see so that (once we have it in
> Rust) we can have a discussion about the merits of the language?
> 
> Hopefully something with interesting interactions with the kernel, but not
> with overly complex hardware, that is, something that doesn't require us to read
> a 400-page specification to implement.

Examples were provided in the first series that was submitted a few
months ago.  What was wrong with them?  And really, there is no need to
dig through a huge spec, try porting an existing C driver will be much
easier.

But if you didn't like the previous examples (nvme block driver, i2c
driver, gpio driver), how about looking at the drivers used by your
current desktop and picking something that you use today that actually
talks to hardware?

We just want to see something "real", so far nothing posted has touched
hardware, and nothing has had to interact with an existing hardware bus
such that it will be automatically loaded and bind to the hardware
present in the system.

Without showing that Rust can actually work in Linux for a real use
case (and binder is not a real use case outside of Android), and how it
will interact with all of the existing kernel objects and lifetimes we
have today, it's going to be a very hard sell.

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Andy Lutomirski]

> On Jul 8, 2021, at 11:51 AM, Greg KH <greg@kroah.com> wrote:
> 
> On Thu, Jul 08, 2021 at 02:36:40PM +0100, Wedson Almeida Filho wrote:
>>> On Thu, Jul 08, 2021 at 08:11:10AM +0200, Greg KH wrote:
>>> Thanks for the detailed explainations, it seems rust can "get away" with
>>> some things with regards to reference counts that the kernel can not.
>>> Userspace has it easy, but note that now that rust is not in userspace,
>>> dealing with multiple cpus/threads is going to be interesting for the
>>> language.
>>> 
>>> So, along those lines, how are you going to tie rust's reference count
>>> logic in with the kernel's reference count logic?  How are you going to
>>> handle dentries, inodes, kobjects, devices and the like?  That's the
>>> real question that I don't seem to see anyone even starting to answer
>>> just yet.
>> 
>> None of what I described before is specific to userspace, but I understand your
>> need to see something more concrete.
>> 
>> I'll describe what we've done for `task_struct` and how lifetimes, aliasing
>> rules, sync & send traits help us achieve zero cost (when compared to C) but
>> also gives us safety. The intention here is to show that similar things can (and
>> will) be done to other kernel reference-counted objects when we get to them.
> 
> <snip loads of good stuff here, thanks for helping describe all of this>
> 
> 
>>> And that's the reason some of us are asking to see a "real" driver, as
>>> those have to deal with these kernel-controlled-reference-counted
>>> objects properly (as well as hardware control).  Seeing how that is
>>> going to work in this language is going to be the real sign of if this
>>> is even going to be possible or not.
>> 
>> Here's what I'd like to avoid: spending time on something that you all still
>> think is not typical enough. Would you be able to point us to a driver that
>> would showcase the interactions you'd like to see so that (once we have it in
>> Rust) we can have a discussion about the merits of the language?
>> 
>> Hopefully something with interesting interactions with the kernel, but not
>> with overly complex hardware, that is, something that doesn't require us to read
>> a 400-page specification to implement.
> 
> Examples were provided in the first series that was submitted a few
> months ago.  What was wrong with them?  And really, there is no need to
> dig through a huge spec, try porting an existing C driver will be much
> easier.
> 
> But if you didn't like the previous examples (nvme block driver, i2c
> driver, gpio driver), how about looking at the drivers used by your
> current desktop and picking something that you use today that actually
> talks to hardware?

I would suggest a virtio device.  The core virtio guest code is horribly unsafe, and various stakeholders keep trying to make it slightly less unsafe. A Rust implementation of the entire virtio ring (or at least the modern bits) and of some concrete device (virtio-blk?) would be quite nice.

I think that, at least for an initial implementation, erroring out in the !use_dma case would be fine. The !use_dma virtio variant is, in my opinion, a historical error and does not really deserve to survive going forward. The powerpc people may beg to differ.  Any port of the !use_dma variant to Rust (or anything that safely manages device memory) will quickly notice that the resulting use of device memory is nonsense and ought not to compile without unsafe annotations in various places. And those unsafe annotations may well deserve comments like /* yes, this is indeed wrong. */.

Fortunately, at least on non-powerpc, you can easily boot a system in the sane use_dma mode. Virtme will do this for you with minimal effort.

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Andy,

On Thu, Jul 8, 2021 at 9:31 PM Andy Lutomirski <luto@amacapital.net> wrote:
> > On Jul 8, 2021, at 11:51 AM, Greg KH <greg@kroah.com> wrote:
> > On Thu, Jul 08, 2021 at 02:36:40PM +0100, Wedson Almeida Filho wrote:
> >>> On Thu, Jul 08, 2021 at 08:11:10AM +0200, Greg KH wrote:
> >>> Thanks for the detailed explainations, it seems rust can "get away" with
> >>> some things with regards to reference counts that the kernel can not.
> >>> Userspace has it easy, but note that now that rust is not in userspace,
> >>> dealing with multiple cpus/threads is going to be interesting for the
> >>> language.
> >>>
> >>> So, along those lines, how are you going to tie rust's reference count
> >>> logic in with the kernel's reference count logic?  How are you going to
> >>> handle dentries, inodes, kobjects, devices and the like?  That's the
> >>> real question that I don't seem to see anyone even starting to answer
> >>> just yet.
> >>
> >> None of what I described before is specific to userspace, but I understand your
> >> need to see something more concrete.
> >>
> >> I'll describe what we've done for `task_struct` and how lifetimes, aliasing
> >> rules, sync & send traits help us achieve zero cost (when compared to C) but
> >> also gives us safety. The intention here is to show that similar things can (and
> >> will) be done to other kernel reference-counted objects when we get to them.
> >
> > <snip loads of good stuff here, thanks for helping describe all of this>
> >
> >
> >>> And that's the reason some of us are asking to see a "real" driver, as
> >>> those have to deal with these kernel-controlled-reference-counted
> >>> objects properly (as well as hardware control).  Seeing how that is
> >>> going to work in this language is going to be the real sign of if this
> >>> is even going to be possible or not.
> >>
> >> Here's what I'd like to avoid: spending time on something that you all still
> >> think is not typical enough. Would you be able to point us to a driver that
> >> would showcase the interactions you'd like to see so that (once we have it in
> >> Rust) we can have a discussion about the merits of the language?
> >>
> >> Hopefully something with interesting interactions with the kernel, but not
> >> with overly complex hardware, that is, something that doesn't require us to read
> >> a 400-page specification to implement.
> >
> > Examples were provided in the first series that was submitted a few
> > months ago.  What was wrong with them?  And really, there is no need to
> > dig through a huge spec, try porting an existing C driver will be much
> > easier.
> >
> > But if you didn't like the previous examples (nvme block driver, i2c
> > driver, gpio driver), how about looking at the drivers used by your
> > current desktop and picking something that you use today that actually
> > talks to hardware?
>
> I would suggest a virtio device.  The core virtio guest code is horribly unsafe, and various stakeholders keep trying to make it slightly less unsafe. A Rust implementation of the entire virtio ring (or at least the modern bits) and of some concrete device (virtio-blk?) would be quite nice.

Another deviation from real hardware drivers? ;-)
How many kernel maintainers have experience with writing a virtio driver?

What is wrong with something driving real hardware?

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Andy Lutomirski]

> On Jul 8, 2021, at 12:35 PM, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Hi Andy,
>
> On Thu, Jul 8, 2021 at 9:31 PM Andy Lutomirski <luto@amacapital.net> wrote:
>>>> On Jul 8, 2021, at 11:51 AM, Greg KH <greg@kroah.com> wrote:
>>> On Thu, Jul 08, 2021 at 02:36:40PM +0100, Wedson Almeida Filho wrote:
>>>>> On Thu, Jul 08, 2021 at 08:11:10AM +0200, Greg KH wrote:
>>>>> Thanks for the detailed explainations, it seems rust can "get away" with
>>>>> some things with regards to reference counts that the kernel can not.
>>>>> Userspace has it easy, but note that now that rust is not in userspace,
>>>>> dealing with multiple cpus/threads is going to be interesting for the
>>>>> language.
>>>>>
>>>>> So, along those lines, how are you going to tie rust's reference count
>>>>> logic in with the kernel's reference count logic?  How are you going to
>>>>> handle dentries, inodes, kobjects, devices and the like?  That's the
>>>>> real question that I don't seem to see anyone even starting to answer
>>>>> just yet.
>>>>
>>>> None of what I described before is specific to userspace, but I understand your
>>>> need to see something more concrete.
>>>>
>>>> I'll describe what we've done for `task_struct` and how lifetimes, aliasing
>>>> rules, sync & send traits help us achieve zero cost (when compared to C) but
>>>> also gives us safety. The intention here is to show that similar things can (and
>>>> will) be done to other kernel reference-counted objects when we get to them.
>>>
>>> <snip loads of good stuff here, thanks for helping describe all of this>
>>>
>>>
>>>>> And that's the reason some of us are asking to see a "real" driver, as
>>>>> those have to deal with these kernel-controlled-reference-counted
>>>>> objects properly (as well as hardware control).  Seeing how that is
>>>>> going to work in this language is going to be the real sign of if this
>>>>> is even going to be possible or not.
>>>>
>>>> Here's what I'd like to avoid: spending time on something that you all still
>>>> think is not typical enough. Would you be able to point us to a driver that
>>>> would showcase the interactions you'd like to see so that (once we have it in
>>>> Rust) we can have a discussion about the merits of the language?
>>>>
>>>> Hopefully something with interesting interactions with the kernel, but not
>>>> with overly complex hardware, that is, something that doesn't require us to read
>>>> a 400-page specification to implement.
>>>
>>> Examples were provided in the first series that was submitted a few
>>> months ago.  What was wrong with them?  And really, there is no need to
>>> dig through a huge spec, try porting an existing C driver will be much
>>> easier.
>>>
>>> But if you didn't like the previous examples (nvme block driver, i2c
>>> driver, gpio driver), how about looking at the drivers used by your
>>> current desktop and picking something that you use today that actually
>>> talks to hardware?
>>
>> I would suggest a virtio device.  The core virtio guest code is horribly unsafe, and various stakeholders keep trying to make it slightly less unsafe. A Rust implementation of the entire virtio ring (or at least the modern bits) and of some concrete device (virtio-blk?) would be quite nice.
>
> Another deviation from real hardware drivers? ;-)
> How many kernel maintainers have experience with writing a virtio driver?
>

Me?  Definitely more than zero :)

> What is wrong with something driving real hardware?

Nothing! But with virt hardware, anyone can test it.

That being said, the virtio case has real use cases from the secure
virt folks.  A Linux guest in a VM host that has untrusted virtio host
devices (TDX, SEV, various Xen or userspace host device driver
implementations, etc) wants to avoid being compromised by a
misbehaving virtio device.

--Andy


>
> Gr{oetje,eeting}s,
>
>                        Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
>                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Thu, Jul 8, 2021 at 8:51 PM Greg KH <greg@kroah.com> wrote:

> But if you didn't like the previous examples (nvme block driver, i2c
> driver, gpio driver), how about looking at the drivers used by your
> current desktop and picking something that you use today that actually
> talks to hardware?

With my GPIO maintainer hat on I'd say a GPIO driver would be quite
interesting to look at. We are two GPIO maintainers and Bartosz is
doing the heavy lifting for the moment so I'm connecting Bartosz to this
discussion. (Now he has to read through the whole backlog,
sorry Bart!)

This is not to say I promise we will merge it or so, but I just generically
like new approaches to old problems so I like this whole thing
overall, despite being critical to some details.

I am also trying to learn Rust. Baby steps.

I also Cced Viresh Kumar who I think is thinking about Rust these
days, using it for one end of a virtio pipe (IIUC) and Andy also brought
up virtio, but now on the other side, in the kernel.

My concern is still whether this really brings Rust closer to the actual
problems we have and that Rust can solve. If the problem is really
about real world security issues, I would ask Kees where the actual
attack surface of the kernel is. He knows.

It kind of matters. EternalBlue (WannaCry) was a horrific Windows
exploit in that it shows us pretty well what kind of cyberweapons the
intelligence agencies of the world have been constructing and
stockpiling, and probably also used. We need to put countermeasures
where such exploits are likely to hit, yesterday. Intuitively I would
say any in-kernel network daemons, anything complex that
responds directly to network traffic, is a good thing to fix. I do not
know why people are so hung up on device drivers. I would
look at something like fs/nfsd or what else is there that act like
this? ksmbd is out-of-tree. Maybe contribute ksmbd in the form
of Rust and show how nicely this implementation of SMB avoid
all the dangers exposed in WannaCry for Windows? That kind of
stuff would build real trust.
KSMBD in C is there:
https://github.com/namjaejeon/ksmbd

If the problem is security in the sense of being secure from random
crashes and general instability, drivers is an as good place to start as
any.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Thu, Jul 8, 2021 at 9:49 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> With my GPIO maintainer hat on I'd say a GPIO driver would be quite
> interesting to look at. We are two GPIO maintainers and Bartosz is
> doing the heavy lifting for the moment so I'm connecting Bartosz to this
> discussion. (Now he has to read through the whole backlog,
> sorry Bart!)
>
> This is not to say I promise we will merge it or so, but I just generically
> like new approaches to old problems so I like this whole thing
> overall, despite being critical to some details.
>
> I am also trying to learn Rust. Baby steps.

Thanks a lot for this offer Linus.

Do you have a particular one in mind? Ideally, it would be one that
has QEMU support or a test suite of some kind, or at least one that
you can easily test for us etc.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Thu, Jul 8, 2021 at 10:34 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
> On Thu, Jul 8, 2021 at 9:49 PM Linus Walleij <linus.walleij@linaro.org> wrote:
> >
> > With my GPIO maintainer hat on I'd say a GPIO driver would be quite
> > interesting to look at. We are two GPIO maintainers and Bartosz is
> > doing the heavy lifting for the moment so I'm connecting Bartosz to this
> > discussion. (Now he has to read through the whole backlog,
> > sorry Bart!)
> >
> > This is not to say I promise we will merge it or so, but I just generically
> > like new approaches to old problems so I like this whole thing
> > overall, despite being critical to some details.
> >
> > I am also trying to learn Rust. Baby steps.
>
> Thanks a lot for this offer Linus.
>
> Do you have a particular one in mind? Ideally, it would be one that
> has QEMU support or a test suite of some kind, or at least one that
> you can easily test for us etc.

I don't use QEMU for GPIO development, we are so close to the
real hardware that it's often not appropriate. We have a testing
module but that is not a real world driver and would not meet the
expectations set here of creating real hardware drivers.

I have seen that QEMU has a piece of code for the Arm PrimeCell
PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
Note that this hardware apart from being used in all Arm reference
designs is used on ARMv4T systems that are not supported by
LLVM but only GCC, which might complicate things.

I am a bit oldschool in that I think real hardware is awesome to test
on. GPIO drivers exist in many shapes and sizes, some are directly
memory-mapped to hardware registers, some are I2C or SPI.
Most Raspberry Pis and Beagle Boards have them, albeit the
on-chip GPIOs are often also pin controllers which complicates
things. Expanders on I2C and SPI will be simpler. Maybe look
for an I2C or SPI expander that has no existing kernel support and
implement it in Rust?

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Linus,

On Fri, Jul 9, 2021 at 12:13 AM Linus Walleij <linus.walleij@linaro.org> wrote:
> On Thu, Jul 8, 2021 at 10:34 PM Miguel Ojeda
> <miguel.ojeda.sandonis@gmail.com> wrote:
> > On Thu, Jul 8, 2021 at 9:49 PM Linus Walleij <linus.walleij@linaro.org> wrote:
> > > With my GPIO maintainer hat on I'd say a GPIO driver would be quite
> > > interesting to look at. We are two GPIO maintainers and Bartosz is
> > > doing the heavy lifting for the moment so I'm connecting Bartosz to this
> > > discussion. (Now he has to read through the whole backlog,
> > > sorry Bart!)
> > >
> > > This is not to say I promise we will merge it or so, but I just generically
> > > like new approaches to old problems so I like this whole thing
> > > overall, despite being critical to some details.
> > >
> > > I am also trying to learn Rust. Baby steps.
> >
> > Thanks a lot for this offer Linus.
> >
> > Do you have a particular one in mind? Ideally, it would be one that
> > has QEMU support or a test suite of some kind, or at least one that
> > you can easily test for us etc.
>
> I don't use QEMU for GPIO development, we are so close to the
> real hardware that it's often not appropriate. We have a testing
> module but that is not a real world driver and would not meet the
> expectations set here of creating real hardware drivers.
>
> I have seen that QEMU has a piece of code for the Arm PrimeCell
> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> Note that this hardware apart from being used in all Arm reference
> designs is used on ARMv4T systems that are not supported by
> LLVM but only GCC, which might complicate things.

Still, the PL061 emulation may be incomplete.  It also doesn't do that
much yet, i.e. you cannot use it to blink a virtual LED.

> I am a bit oldschool in that I think real hardware is awesome to test
> on. GPIO drivers exist in many shapes and sizes, some are directly
> memory-mapped to hardware registers, some are I2C or SPI.
> Most Raspberry Pis and Beagle Boards have them, albeit the
> on-chip GPIOs are often also pin controllers which complicates
> things. Expanders on I2C and SPI will be simpler. Maybe look
> for an I2C or SPI expander that has no existing kernel support and
> implement it in Rust?

That still requires an I2C or SPI bus, which may not be that easy
to find for people who don't have embedded boards on their desk...
Do you know of any unsupported expanders that connect to USB?
Something like an MCP2210 dev board[1], but that one is mainly
intended for SPI, although some pins can be used as GPIOs.  There
is no upstream driver yet, but there is an out-of-tree driver and a
FOSDEM presentation.

[1] E.g. https://www.digikey.be/product-detail/en/microchip-technology/ADM00419/ADM00419-ND/3046570

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> I have seen that QEMU has a piece of code for the Arm PrimeCell
> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> Note that this hardware apart from being used in all Arm reference
> designs is used on ARMv4T systems that are not supported by
> LLVM but only GCC, which might complicate things.

Here is a working PL061 driver in Rust (converted form the C one):
https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs

(I tested it on QEMU through the sysfs interface and also gpio-keys as QEMU
uses one of the PL061 pins as the power button.)

I have a long list of ways in which Rust affords us extra guarantees but in the
interest of brevity I will try to describe how Rust helps us address the two (or
more) lifetime issues Greg mentioned the other day.

Rust allows us to build abstractions that guarantee safety. Here are the ones I
used/built for this:

1. State created on `probe` is ref-counted.
2. Hardware resources (device mem and irq in this case) are "revocable".
3. On `remove`, we automatically revoke access to hardware resources, then free
them.

What this gives us:
1. With ref-counted objects Rust allows us to avoid dangling pointers. No more
UAF because memory was freed when the device was removed. (C can also do this,
of course, but the compiler doesn't help us if/when we forget to
increment/decrement the ref count.)
2. Given that references to device state may outlive the device, revocable hw
resources allows us to prevent the use of these resources after the device is
gone. Rust ensures that such access is only allowed before resources are
revoked. (In C we can also do something similar, but the compiler won't enforce
this invariant for us, i.e., we can make mistakes where we forget to check if
something was revoked, or forget to hold locks keeping resources alive, etc.)
3. After revoking access, we need to ensure that existing concurrent users
finish before we can free resources. In this implementation, we use RCU so that
resource users need to hold an RCU read lock and we ensure that they've also
completed their use before freeing the resources (synchronize_rcu between
revoking & freeing). Locking/unlocking happens automatically.

This, naturally, doesn't solve any problems with the existing C code. However, I
think it addresses things on the Rust side. For example, suppose that in
addition to registering with gpio, we also wanted to expose the device as a
miscdev (I use this as an example because we have miscdevs in Rust). The
refcounted device state can be stored in the miscdev registration, and each
opened file can also have a reference to it (device state). We don't control
when the latter gets released, but it's ok for them to hold on to state because
they won't be able to use hw resources after the device is removed; once all
file descriptors are closed, the refcount goes to zero and the memory is freed.

Any thoughts on this?

(A quick disclaimer: I'm sure there are scenarios that don't fit exactly with
this, but the intent ATM is not to cover all scenarios, it's just to show a
working example of what Rust enables. Eventually we want to generalise these
ideas in cooperation with maintainers, who know about all scenarios and subtle
issues.)

Cheers,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

[-- Attachment #1: Type: text/plain, Size: 791 bytes --]

On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > I have seen that QEMU has a piece of code for the Arm PrimeCell
> > PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > Note that this hardware apart from being used in all Arm reference
> > designs is used on ARMv4T systems that are not supported by
> > LLVM but only GCC, which might complicate things.
> 
> Here is a working PL061 driver in Rust (converted form the C one):
> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs

I'm also attaching an html rending of the C and Rust versions side by side where
I try to line the definitions up to make it easier to contrast the two
implementations.

[-- Attachment #2: pl061.html --]
[-- Type: text/html, Size: 127606 bytes --]

Re: [TECH TOPIC] Rust for Linux

[Arnd Bergmann]

On Mon, Jul 19, 2021 at 3:15 PM Wedson Almeida Filho
<wedsonaf@google.com> wrote:
> On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > Note that this hardware apart from being used in all Arm reference
> > > designs is used on ARMv4T systems that are not supported by
> > > LLVM but only GCC, which might complicate things.
> >
> > Here is a working PL061 driver in Rust (converted form the C one):
> > https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
>
> I'm also attaching an html rending of the C and Rust versions side by side where
> I try to line the definitions up to make it easier to contrast the two
> implementations.

Thanks a lot, this looks extremely helpful!

I have a couple of questions to understand some of the differences to the
C version, and what the reasons are for those:

- All the dev_dbg() seem to be replaced with pr_debug!(). Does that mean
  we lose the information about the device instance in the output, or is
  that magically added back?

- There is a lock of type IrqDisableSpinLock, which sounds like it would
  correspond to a spinlock_t that is always locked with spin_lock_irqsave(),
  but the original driver uses raw_spin_lock_irqsave(). Does that mean it
  won't work on CONFIG_PREEMPT_RT until both types are supported?

- What's with all the CamelCase? Is that enforced by the language, or
  can we use the same identifiers here that we have in the C version?

- What is the mechanism behind power::Operations that replaces the
   #ifdef CONFIG_PM of the C version? Will the rust compiler just drop
   the dead code when CONFIG_PM is disabled?

      Arnd

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Mon, Jul 19, 2021 at 4:02 PM Arnd Bergmann <arnd@arndb.de> wrote:

> - What's with all the CamelCase? Is that enforced by the language, or
>   can we use the same identifiers here that we have in the C version?

To me it looks like that is used to distinguish classes,
factory methods and such, it actually seems pretty intuitive
for the perception to me, but maybe I am already Rust-tainted :/

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Arnd Bergmann]

On Mon, Jul 19, 2021 at 4:13 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> On Mon, Jul 19, 2021 at 4:02 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> > - What's with all the CamelCase? Is that enforced by the language, or
> >   can we use the same identifiers here that we have in the C version?
>
> To me it looks like that is used to distinguish classes,
> factory methods and such, it actually seems pretty intuitive
> for the perception to me, but maybe I am already Rust-tainted :/
>
> Yours,
> Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Arnd Bergmann]

On Mon, Jul 19, 2021 at 4:13 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> On Mon, Jul 19, 2021 at 4:02 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> > - What's with all the CamelCase? Is that enforced by the language, or
> >   can we use the same identifiers here that we have in the C version?
>
> To me it looks like that is used to distinguish classes,
> factory methods and such, it actually seems pretty intuitive
> for the perception to me, but maybe I am already Rust-tainted :/

I'm worried that this makes things harder to grep for, and to document,
when the same structure is used in both Rust and C.

       Arnd

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Mon, Jul 19, 2021 at 11:33 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> I'm worried that this makes things harder to grep for, and to document,
> when the same structure is used in both Rust and C.

For the former, by grepping you should find the Rust abstractions that
use the C structures, because binding generation does not change the
names or their case. Of course, the Rust abstractions on top of that
is a different matter, but those do not exist in C anyway. So, at
least for the use case of searching Rust abstractions that use a given
C structure or API, it should be fine.

For the latter, we started linking the C code and/or docs from the
Rust ones. We do it manually now, but ideally we should be able to
reference C docs (and/or code) just by writing the C name, something
like:

    /// Similar to a C [`spinlock_t`].

This is a feature that already works for Rust definitions and we use
it all the time, but not for C ones (obviously). My current plan is to
have `rustdoc` support a "external references map" that is added to
the internal map of Rust "intra-doc" links that it already has. With
that, we could have Sphinx generate the map of references pointing to
its own docs.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Johannes Berg]

On Tue, 2021-07-20 at 03:46 +0200, Miguel Ojeda wrote:
> On Mon, Jul 19, 2021 at 11:33 PM Arnd Bergmann <arnd@arndb.de> wrote:
> > 
> > I'm worried that this makes things harder to grep for, and to document,
> > when the same structure is used in both Rust and C.
> 
> For the former, by grepping you should find the Rust abstractions that
> use the C structures, because binding generation does not change the
> names or their case. Of course, the Rust abstractions on top of that
> is a different matter, but those do not exist in C anyway. So, at
> least for the use case of searching Rust abstractions that use a given
> C structure or API, it should be fine.

Keep in mind though that there are at least two different use cases for
grep:

1) I need to change the API here, need to find all the users - I agree
   this is covered here.

2) I need to change the semantics (e.g. locking) and for this I need to
   find all the users and analyse their use (e.g. locking-wise), which
   now becomes an exercise in digging through the abstractions and doing
   a kind of "indirect" grep for the next level.
   Not that this doesn't exist today (e.g. some netlink APIs I changed
   in the past have some abstractions in some subsystems), but it does
   make it more complex.


johannes

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

On Mon, Jul 19, 2021 at 4:03 PM Arnd Bergmann <arnd@arndb.de> wrote:
> On Mon, Jul 19, 2021 at 3:15 PM Wedson Almeida Filho
> <wedsonaf@google.com> wrote:
> > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > Note that this hardware apart from being used in all Arm reference
> > > > designs is used on ARMv4T systems that are not supported by
> > > > LLVM but only GCC, which might complicate things.
> > >
> > > Here is a working PL061 driver in Rust (converted form the C one):
> > > https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> >
> > I'm also attaching an html rending of the C and Rust versions side by side where
> > I try to line the definitions up to make it easier to contrast the two
> > implementations.
>
> Thanks a lot, this looks extremely helpful!

+1

For a moment I thought I found an off-by-one bug:

                for offset in 0..PL061_GPIO_NR {
                    if pending & bit(offset) != 0 {
                        router.deliver(offset.into());
                    }
                }

Turns out "a..b" in Rust does mean "range from a to b-1".
That's gonna be hard to (un)learn...

https://doc.rust-lang.org/reference/expressions/range-expr.html

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Mon, Jul 19, 2021 at 4:43 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Turns out "a..b" in Rust does mean "range from a to b-1".
> That's gonna be hard to (un)learn...

It may help to think about it as the usual `for` loop in C, typically
written with `<` and the "size" (instead of `<=` and the "size - 1"),
i.e.:

    for i in 0..N
    for (i = 0; i < N; ++i)

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Steven Rostedt]

On Mon, 19 Jul 2021 20:24:51 +0200
Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote:

> On Mon, Jul 19, 2021 at 4:43 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> >
> > Turns out "a..b" in Rust does mean "range from a to b-1".
> > That's gonna be hard to (un)learn...  
> 
> It may help to think about it as the usual `for` loop in C, typically
> written with `<` and the "size" (instead of `<=` and the "size - 1"),
> i.e.:
> 
>     for i in 0..N
>     for (i = 0; i < N; ++i)
> 

Or think of it as the python range() function, but not as a git log, where

	git log sha1..sha2

is a list of commits from sha1 + 1 through to sha2 inclusive :-p

-- Steve

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Mon, Jul 19, 2021 at 4:02 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> - All the dev_dbg() seem to be replaced with pr_debug!(). Does that mean
>   we lose the information about the device instance in the output, or is
>   that magically added back?

We do not have the `dev_*`-style printing facilities yet, but we will.

> - What's with all the CamelCase? Is that enforced by the language, or
>   can we use the same identifiers here that we have in the C version?

It is not a hard error, but we recommend following the standard Rust
style (and we enforce it, currently). As Linus said, it is a way to
easily differentiate what is what, instead of using a prefix or
suffix:
  - Types (and similar) are in `CamelCase`.
  - Functions (and similar) are in `snake_case`.
  - Statics/consts are in `SCREAMING_CASE`.

See https://rust-lang.github.io/api-guidelines/naming.html

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Mon, Jul 19, 2021 at 04:02:38PM +0200, Arnd Bergmann wrote:
> On Mon, Jul 19, 2021 at 3:15 PM Wedson Almeida Filho
> <wedsonaf@google.com> wrote:
> > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > Note that this hardware apart from being used in all Arm reference
> > > > designs is used on ARMv4T systems that are not supported by
> > > > LLVM but only GCC, which might complicate things.
> > >
> > > Here is a working PL061 driver in Rust (converted form the C one):
> > > https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> >
> > I'm also attaching an html rending of the C and Rust versions side by side where
> > I try to line the definitions up to make it easier to contrast the two
> > implementations.
> 
> Thanks a lot, this looks extremely helpful!

Thanks for taking the time to check it out.

> I have a couple of questions to understand some of the differences to the
> C version, and what the reasons are for those:
> 
> - All the dev_dbg() seem to be replaced with pr_debug!(). Does that mean
>   we lose the information about the device instance in the output, or is
>   that magically added back?

At the moment we lose information about the device, but simply because we
haven't implemented dev_ variants yet, there is no fundamental reason for this
though.

> - There is a lock of type IrqDisableSpinLock, which sounds like it would
>   correspond to a spinlock_t that is always locked with spin_lock_irqsave(),
>   but the original driver uses raw_spin_lock_irqsave(). Does that mean it
>   won't work on CONFIG_PREEMPT_RT until both types are supported?

Yes, it uses spinlock_t. But again, just because we don't have an implementation
that uses raw_spinlock_t yet.

> - What's with all the CamelCase? Is that enforced by the language, or
>   can we use the same identifiers here that we have in the C version?

I see the Miguel and Linus have already chimed in :)

> - What is the mechanism behind power::Operations that replaces the
>    #ifdef CONFIG_PM of the C version? Will the rust compiler just drop
>    the dead code when CONFIG_PM is disabled?

It's not there yet, but the idea is to wrap the place where `drv.pm` is
initialised in the amba code with `if cfg!(CONFIG_PM)` -- if CONFIG_PM is
disabled, all references to PM code are gone and they are removed by the
compiler. This way we move configuration-dependent code out of drivers and into
core libraries.

Cheers,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Arnd Bergmann]

On Mon, Jul 19, 2021 at 7:32 PM Wedson Almeida Filho
<wedsonaf@google.com> wrote:
> On Mon, Jul 19, 2021 at 04:02:38PM +0200, Arnd Bergmann wrote:
>
> At the moment we lose information about the device, but simply because we
> haven't implemented dev_ variants yet, there is no fundamental reason for this
> though.
>
...
>
> Yes, it uses spinlock_t. But again, just because we don't have an implementation
> that uses raw_spinlock_t yet.
>
...
> It's not there yet, but the idea is to wrap the place where `drv.pm` is
> initialised in the amba code with `if cfg!(CONFIG_PM)` -- if CONFIG_PM is
> disabled, all references to PM code are gone and they are removed by the
> compiler. This way we move configuration-dependent code out of drivers and into
> core libraries.

Ok, that all sounds good to me. I'll reply separately on the coding style.

       Arnd

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Mon, Jul 19, 2021 at 4:02 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> - What is the mechanism behind power::Operations that replaces the
>    #ifdef CONFIG_PM of the C version? Will the rust compiler just drop
>    the dead code when CONFIG_PM is disabled?

We support conditional compilation with the kernel configuration, e.g.:

    #[cfg(CONFIG_X)]      // `CONFIG_X` is enabled (`y` or `m`)
    #[cfg(CONFIG_X="y")]  // `CONFIG_X` is enabled as a built-in (`y`)
    #[cfg(CONFIG_X="m")]  // `CONFIG_X` is enabled as a module   (`m`)
    #[cfg(not(CONFIG_X))] // `CONFIG_X` is disabled

One can use it, like in C, to fully remove any item as a user, to make
an implementation a noop (so that callers do not need to care), etc.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Vegard Nossum]

On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
>> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
>>> I have seen that QEMU has a piece of code for the Arm PrimeCell
>>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
>>> Note that this hardware apart from being used in all Arm reference
>>> designs is used on ARMv4T systems that are not supported by
>>> LLVM but only GCC, which might complicate things.
>>
>> Here is a working PL061 driver in Rust (converted form the C one):
>> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> 
> I'm also attaching an html rending of the C and Rust versions side by side where
> I try to line the definitions up to make it easier to contrast the two
> implementations.
> 

This is really cool :-) As a Rust noob, I have a few questions:

1. I'm curious about some of the writeb() vs. try_writeb() calls:

fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
Result {
        let woffset = bit(offset + 2).into();
        let _guard = data.lock();
        let pl061 = data.resources().ok_or(Error::ENXIO)?;
        pl061.base.try_writeb((value as u8) << offset, woffset)?;
        let mut gpiodir = pl061.base.readb(GPIODIR);
        gpiodir |= bit(offset);
        pl061.base.writeb(gpiodir, GPIODIR);

        // gpio value is set again, because pl061 doesn't allow to set
value of a gpio pin before
        // configuring it in OUT mode.
        pl061.base.try_writeb((value as u8) << offset, woffset)?;
        Ok(())
    }

Here you have try_writeb() (and error return) where there was just a
writeb() without any error handling in the C version. Is this what
Miguel was answering a bit down the thread where the address is computed
((value as u8) << offset) so it _needs_ to use the try_() version?

If offset can be anything but a "correct" value here, should there be a
check for that somewhere else and then the computed value can be
subsequently treated as safe (i.e. there's a second try_writeb() in the
function that now presumably does the runtime check a second time,
redundantly)?

2. In many places you have the C code:

struct pl061 *pl061 = dev_get_drvdata(dev);

with the equivalent Rust code as:

let pl061 = data.resources().ok_or(Error::ENXIO)?;

Why doesn't the C code need to check for errors here? Or put
differently, why can the Rust version fail?

3. In probe() you have:

data.resources().ok_or(Error::ENXIO)?.base.writeb(0, GPIOIE); // disable
irqs
data.registrations()
    .ok_or(Error::ENXIO)?
    .gpio_chip
    .register_with_irq(PL061_GPIO_NR, None, dev, data.clone(), irq)?;

So here, if .register_with_irq() or any of the other ?-marked calls
fail, then the function returns and the local variable "data" and all
its members are freed/deallocated using destructors like in C++ RAII?

Thanks,


Vegard

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Mon, Jul 19, 2021 at 6:02 PM Vegard Nossum <vegard.nossum@oracle.com> wrote:
>
> Here you have try_writeb() (and error return) where there was just a
> writeb() without any error handling in the C version. Is this what
> Miguel was answering a bit down the thread where the address is computed
> ((value as u8) << offset) so it _needs_ to use the try_() version?

Not exactly: `((value as u8) << offset)` is the value -- the address
is still controlled by the `try_()` version.

However, the offset given is not a compile-time constant here, thus
the runtime-check is used. Or we could provide an `unsafe fn` which
means the caller would be responsible to guarantee the precondition.
But see next point.

(Actually, from a quick look, there are 3 or so `try_*()` that could
be using the non-`try_*()` version)

> If offset can be anything but a "correct" value here, should there be a
> check for that somewhere else and then the computed value can be
> subsequently treated as safe (i.e. there's a second try_writeb() in the
> function that now presumably does the runtime check a second time,
> redundantly)?

Indeed, that is a pattern that we use all the time to make things
safe, i.e. we create types that hold invariants.

Here, we could definitely keep the information that a given `offset`
is safe and then reuse that knowledge statically.

Ideally, Rust would allow using the loop variable as a compile-time
value if the range is statically known (without unrolling the loop
manually and without a macro).

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Mon, Jul 19, 2021 at 7:45 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> However, the offset given is not a compile-time constant here, thus
> the runtime-check is used. Or we could provide an `unsafe fn` which
> means the caller would be responsible to guarantee the precondition.
> But see next point.

To clarify, by offset here I mean the actual offset passed to the
function, i.e. the second parameter `woffset` (notice the `w`), not
`offset` in the value parameter:

    pl061.base.try_writeb((value as u8) << offset, woffset)?;

> (Actually, from a quick look, there are 3 or so `try_*()` that could
> be using the non-`try_*()` version)

e.g.

    let _ = pl061.base.try_writeb(gpioie, GPIOIE);

could be:

    pl061.base.writeb(gpioie, GPIOIE);

because `GPIOIE` is known at compile-time (vs. the ones with `woffset`).

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> 
> On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> >>> Note that this hardware apart from being used in all Arm reference
> >>> designs is used on ARMv4T systems that are not supported by
> >>> LLVM but only GCC, which might complicate things.
> >>
> >> Here is a working PL061 driver in Rust (converted form the C one):
> >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > 
> > I'm also attaching an html rending of the C and Rust versions side by side where
> > I try to line the definitions up to make it easier to contrast the two
> > implementations.
> > 
> 
> This is really cool :-) As a Rust noob, I have a few questions:
> 
> 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> 
> fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> Result {
>         let woffset = bit(offset + 2).into();
>         let _guard = data.lock();
>         let pl061 = data.resources().ok_or(Error::ENXIO)?;
>         pl061.base.try_writeb((value as u8) << offset, woffset)?;
>         let mut gpiodir = pl061.base.readb(GPIODIR);
>         gpiodir |= bit(offset);
>         pl061.base.writeb(gpiodir, GPIODIR);
> 
>         // gpio value is set again, because pl061 doesn't allow to set
> value of a gpio pin before
>         // configuring it in OUT mode.
>         pl061.base.try_writeb((value as u8) << offset, woffset)?;
>         Ok(())
>     }
> 
> Here you have try_writeb() (and error return) where there was just a
> writeb() without any error handling in the C version. Is this what
> Miguel was answering a bit down the thread where the address is computed
> ((value as u8) << offset) so it _needs_ to use the try_() version?

The `writeb` variant only works when we know at compile-time that the offset is
within bounds (the compiler will reject the code otherwise). When the value is
computed at runtime we use a `try` version that checks before performing the
write. We need this to guarantee memory safety.

> If offset can be anything but a "correct" value here, should there be a
> check for that somewhere else and then the computed value can be
> subsequently treated as safe (i.e. there's a second try_writeb() in the
> function that now presumably does the runtime check a second time,
> redundantly)?

Oh, that's a neat idea. We can certainly implement something like this:

let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;

Then woffset would be passed to writeb variants that are guaranteed to succeed.
(Rust helps us ensure that woffset cannot change without checks, which would be
harder to do in C.)

> 2. In many places you have the C code:
> 
> struct pl061 *pl061 = dev_get_drvdata(dev);
> 
> with the equivalent Rust code as:
> 
> let pl061 = data.resources().ok_or(Error::ENXIO)?;
> 
> Why doesn't the C code need to check for errors here? Or put
> differently, why can the Rust version fail?

There are two aspecs worth noting here:
1. In C there is cast from void * to struct pl061 * without really knowing if
the stored pointer is of the right type. For example, if I simply change the
struct type to say `struct mutex` in the code above, it will still compile,
though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
directly to drivers, and using type-specialised functions to set/get drvdata, so
it *knows* that the type is right. So in this sense Rust is better because it
offers type guarantees without additional runtime cost. (In Rust, if you change
the type of the function to say `&Mutex`, it won't compile.

2. The extra check we have here is because of a feature that the C code doesn't
have: revocable resources. If we didn't want to have this, we could do say
`data.base.writeb(...)` directly, but then we could have situations where `base`
is used after the device was removed. By having these checks we guarantee that
anyone can hold a reference to device state, but they can no longer use hw
resources after the device is removed.

> 3. In probe() you have:
> 
> data.resources().ok_or(Error::ENXIO)?.base.writeb(0, GPIOIE); // disable
> irqs
> data.registrations()
>     .ok_or(Error::ENXIO)?
>     .gpio_chip
>     .register_with_irq(PL061_GPIO_NR, None, dev, data.clone(), irq)?;
> 
> So here, if .register_with_irq() or any of the other ?-marked calls
> fail, then the function returns and the local variable "data" and all
> its members are freed/deallocated using destructors like in C++ RAII?

Yes, everything is freed a la C++ RAII destructors on failure. Additionally, the
same mechanism is used on success when the refcount on `data` goes to zero. No
need for chained goto-based exit paths nor devm-like bookkeeping.

Cheers,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > >>> Note that this hardware apart from being used in all Arm reference
> > >>> designs is used on ARMv4T systems that are not supported by
> > >>> LLVM but only GCC, which might complicate things.
> > >>
> > >> Here is a working PL061 driver in Rust (converted form the C one):
> > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > 
> > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > I try to line the definitions up to make it easier to contrast the two
> > > implementations.
> > 
> > This is really cool :-) As a Rust noob, I have a few questions:
> > 
> > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > 
> > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > Result {
> >         let woffset = bit(offset + 2).into();
> >         let _guard = data.lock();
> >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> >         let mut gpiodir = pl061.base.readb(GPIODIR);
> >         gpiodir |= bit(offset);
> >         pl061.base.writeb(gpiodir, GPIODIR);
> > 
> >         // gpio value is set again, because pl061 doesn't allow to set
> > value of a gpio pin before
> >         // configuring it in OUT mode.
> >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> >         Ok(())
> >     }
> > 
> > Here you have try_writeb() (and error return) where there was just a
> > writeb() without any error handling in the C version. Is this what
> > Miguel was answering a bit down the thread where the address is computed
> > ((value as u8) << offset) so it _needs_ to use the try_() version?
> 
> The `writeb` variant only works when we know at compile-time that the offset is
> within bounds (the compiler will reject the code otherwise). When the value is
> computed at runtime we use a `try` version that checks before performing the
> write. We need this to guarantee memory safety.
> 
> > If offset can be anything but a "correct" value here, should there be a
> > check for that somewhere else and then the computed value can be
> > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > function that now presumably does the runtime check a second time,
> > redundantly)?
> 
> Oh, that's a neat idea. We can certainly implement something like this:
> 
> let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> 
> Then woffset would be passed to writeb variants that are guaranteed to succeed.
> (Rust helps us ensure that woffset cannot change without checks, which would be
> harder to do in C.)
> 
> > 2. In many places you have the C code:
> > 
> > struct pl061 *pl061 = dev_get_drvdata(dev);
> > 
> > with the equivalent Rust code as:
> > 
> > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > 
> > Why doesn't the C code need to check for errors here? Or put
> > differently, why can the Rust version fail?
> 
> There are two aspecs worth noting here:
> 1. In C there is cast from void * to struct pl061 * without really knowing if
> the stored pointer is of the right type. For example, if I simply change the
> struct type to say `struct mutex` in the code above, it will still compile,
> though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> directly to drivers, and using type-specialised functions to set/get drvdata, so
> it *knows* that the type is right. So in this sense Rust is better because it
> offers type guarantees without additional runtime cost. (In Rust, if you change
> the type of the function to say `&Mutex`, it won't compile.
> 
> 2. The extra check we have here is because of a feature that the C code doesn't
> have: revocable resources. If we didn't want to have this, we could do say
> `data.base.writeb(...)` directly, but then we could have situations where `base`
> is used after the device was removed. By having these checks we guarantee that
> anyone can hold a reference to device state, but they can no longer use hw
> resources after the device is removed.

If the driver reached a code path with an I/O write after .remove()
returns, the game is likely over already. It would be more interesting
to see how we could prevent that from happening in the first place.
Checking individual I/O writes at runtime will not only add additional
CPU costs, but will also produce code paths that are not well tested. It
feels that we're inventing a problem just to be able to showcase the
solution :-)

> > 3. In probe() you have:
> > 
> > data.resources().ok_or(Error::ENXIO)?.base.writeb(0, GPIOIE); // disable irqs
> > data.registrations()
> >     .ok_or(Error::ENXIO)?
> >     .gpio_chip
> >     .register_with_irq(PL061_GPIO_NR, None, dev, data.clone(), irq)?;
> > 
> > So here, if .register_with_irq() or any of the other ?-marked calls
> > fail, then the function returns and the local variable "data" and all
> > its members are freed/deallocated using destructors like in C++ RAII?
> 
> Yes, everything is freed a la C++ RAII destructors on failure. Additionally, the
> same mechanism is used on success when the refcount on `data` goes to zero. No
> need for chained goto-based exit paths nor devm-like bookkeeping.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Mon, Jul 19, 2021 at 10:37:52PM +0300, Laurent Pinchart wrote:
> On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> > On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > >>> Note that this hardware apart from being used in all Arm reference
> > > >>> designs is used on ARMv4T systems that are not supported by
> > > >>> LLVM but only GCC, which might complicate things.
> > > >>
> > > >> Here is a working PL061 driver in Rust (converted form the C one):
> > > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > > 
> > > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > > I try to line the definitions up to make it easier to contrast the two
> > > > implementations.
> > > 
> > > This is really cool :-) As a Rust noob, I have a few questions:
> > > 
> > > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > > 
> > > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > > Result {
> > >         let woffset = bit(offset + 2).into();
> > >         let _guard = data.lock();
> > >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > >         let mut gpiodir = pl061.base.readb(GPIODIR);
> > >         gpiodir |= bit(offset);
> > >         pl061.base.writeb(gpiodir, GPIODIR);
> > > 
> > >         // gpio value is set again, because pl061 doesn't allow to set
> > > value of a gpio pin before
> > >         // configuring it in OUT mode.
> > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > >         Ok(())
> > >     }
> > > 
> > > Here you have try_writeb() (and error return) where there was just a
> > > writeb() without any error handling in the C version. Is this what
> > > Miguel was answering a bit down the thread where the address is computed
> > > ((value as u8) << offset) so it _needs_ to use the try_() version?
> > 
> > The `writeb` variant only works when we know at compile-time that the offset is
> > within bounds (the compiler will reject the code otherwise). When the value is
> > computed at runtime we use a `try` version that checks before performing the
> > write. We need this to guarantee memory safety.
> > 
> > > If offset can be anything but a "correct" value here, should there be a
> > > check for that somewhere else and then the computed value can be
> > > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > > function that now presumably does the runtime check a second time,
> > > redundantly)?
> > 
> > Oh, that's a neat idea. We can certainly implement something like this:
> > 
> > let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> > 
> > Then woffset would be passed to writeb variants that are guaranteed to succeed.
> > (Rust helps us ensure that woffset cannot change without checks, which would be
> > harder to do in C.)
> > 
> > > 2. In many places you have the C code:
> > > 
> > > struct pl061 *pl061 = dev_get_drvdata(dev);
> > > 
> > > with the equivalent Rust code as:
> > > 
> > > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > 
> > > Why doesn't the C code need to check for errors here? Or put
> > > differently, why can the Rust version fail?
> > 
> > There are two aspecs worth noting here:
> > 1. In C there is cast from void * to struct pl061 * without really knowing if
> > the stored pointer is of the right type. For example, if I simply change the
> > struct type to say `struct mutex` in the code above, it will still compile,
> > though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> > directly to drivers, and using type-specialised functions to set/get drvdata, so
> > it *knows* that the type is right. So in this sense Rust is better because it
> > offers type guarantees without additional runtime cost. (In Rust, if you change
> > the type of the function to say `&Mutex`, it won't compile.
> > 
> > 2. The extra check we have here is because of a feature that the C code doesn't
> > have: revocable resources. If we didn't want to have this, we could do say
> > `data.base.writeb(...)` directly, but then we could have situations where `base`
> > is used after the device was removed. By having these checks we guarantee that
> > anyone can hold a reference to device state, but they can no longer use hw
> > resources after the device is removed.
> 
> If the driver reached a code path with an I/O write after .remove()
> returns, the game is likely over already. It would be more interesting
> to see how we could prevent that from happening in the first place.
> Checking individual I/O writes at runtime will not only add additional
> CPU costs, but will also produce code paths that are not well tested.

You may be conflating checking offsets in individual writes/reads with accessing
hw resources. Note that these are different things.

> It
> feels that we're inventing a problem just to be able to showcase the
> solution :-)

Thanks for taking a look. I beg to differ though, as this solves (on the Rust
side) a problem you described the other day on this very thread. The solution is
different from what you propose though :)

- The internal data structures of drivers are refcounted. Drivers then share
  this internal representation with other subsystems (e.g., cdev).
- On `remove`, the registrations with other subsystems are removed (so no
  additional sharing of internal data should happen), but existing calls and
  references to internal data structures continue to exist. This part is
  important: we don't "revoke" the references, but we do revoke the hw resources
  part of the internal state.
- Attempts to access hardware resources freed during `remove` *must* be
  prevented, that's where the calls to `resources()` are relevant -- if a
  subsystem calls into the driver with one of the references it held on to, they
  won't be able to access the (already released) hw resources.

We have this problem specifically in gpio: as Linus explained, they created an
indirection via a pointer which is checked in most entry points, but there is no
synchronisation that guarantees that the pointer will remain valid during a
call, and nothing forces uses of the pointer to be checked (so as Linus points
out, they may need more checks).

For Rust drivers, if the registration with other subsystems were done by doing
references to driver data in Rust, this extra "protection" (that has race
conditions that, timed correctly, lead to use-after-free vulnerabilities) would
be obviated; all would be handled safely on the Rust side (e.g., all accesses
must be checked, there is no way to get to resources without a check, and use of
the resources is guarded by a guard that uses RCU read-side lock).

Do you still think we don't have a problem?

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Wedson,

On Mon, Jul 19, 2021 at 10:09:46PM +0100, Wedson Almeida Filho wrote:
> On Mon, Jul 19, 2021 at 10:37:52PM +0300, Laurent Pinchart wrote:
> > On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> > > On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > > > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > >>> Note that this hardware apart from being used in all Arm reference
> > > > >>> designs is used on ARMv4T systems that are not supported by
> > > > >>> LLVM but only GCC, which might complicate things.
> > > > >>
> > > > >> Here is a working PL061 driver in Rust (converted form the C one):
> > > > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > > > 
> > > > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > > > I try to line the definitions up to make it easier to contrast the two
> > > > > implementations.
> > > > 
> > > > This is really cool :-) As a Rust noob, I have a few questions:
> > > > 
> > > > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > > > 
> > > > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > > > Result {
> > > >         let woffset = bit(offset + 2).into();
> > > >         let _guard = data.lock();
> > > >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > >         let mut gpiodir = pl061.base.readb(GPIODIR);
> > > >         gpiodir |= bit(offset);
> > > >         pl061.base.writeb(gpiodir, GPIODIR);
> > > > 
> > > >         // gpio value is set again, because pl061 doesn't allow to set
> > > > value of a gpio pin before
> > > >         // configuring it in OUT mode.
> > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > >         Ok(())
> > > >     }
> > > > 
> > > > Here you have try_writeb() (and error return) where there was just a
> > > > writeb() without any error handling in the C version. Is this what
> > > > Miguel was answering a bit down the thread where the address is computed
> > > > ((value as u8) << offset) so it _needs_ to use the try_() version?
> > > 
> > > The `writeb` variant only works when we know at compile-time that the offset is
> > > within bounds (the compiler will reject the code otherwise). When the value is
> > > computed at runtime we use a `try` version that checks before performing the
> > > write. We need this to guarantee memory safety.
> > > 
> > > > If offset can be anything but a "correct" value here, should there be a
> > > > check for that somewhere else and then the computed value can be
> > > > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > > > function that now presumably does the runtime check a second time,
> > > > redundantly)?
> > > 
> > > Oh, that's a neat idea. We can certainly implement something like this:
> > > 
> > > let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> > > 
> > > Then woffset would be passed to writeb variants that are guaranteed to succeed.
> > > (Rust helps us ensure that woffset cannot change without checks, which would be
> > > harder to do in C.)
> > > 
> > > > 2. In many places you have the C code:
> > > > 
> > > > struct pl061 *pl061 = dev_get_drvdata(dev);
> > > > 
> > > > with the equivalent Rust code as:
> > > > 
> > > > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > 
> > > > Why doesn't the C code need to check for errors here? Or put
> > > > differently, why can the Rust version fail?
> > > 
> > > There are two aspecs worth noting here:
> > > 1. In C there is cast from void * to struct pl061 * without really knowing if
> > > the stored pointer is of the right type. For example, if I simply change the
> > > struct type to say `struct mutex` in the code above, it will still compile,
> > > though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> > > directly to drivers, and using type-specialised functions to set/get drvdata, so
> > > it *knows* that the type is right. So in this sense Rust is better because it
> > > offers type guarantees without additional runtime cost. (In Rust, if you change
> > > the type of the function to say `&Mutex`, it won't compile.
> > > 
> > > 2. The extra check we have here is because of a feature that the C code doesn't
> > > have: revocable resources. If we didn't want to have this, we could do say
> > > `data.base.writeb(...)` directly, but then we could have situations where `base`
> > > is used after the device was removed. By having these checks we guarantee that
> > > anyone can hold a reference to device state, but they can no longer use hw
> > > resources after the device is removed.
> > 
> > If the driver reached a code path with an I/O write after .remove()
> > returns, the game is likely over already. It would be more interesting
> > to see how we could prevent that from happening in the first place.
> > Checking individual I/O writes at runtime will not only add additional
> > CPU costs, but will also produce code paths that are not well tested.
> 
> You may be conflating checking offsets in individual writes/reads with accessing
> hw resources. Note that these are different things.

Yes, it's the data.resources().ok_or() that I was talking about, not the
I/O writes, sorry.

> > It
> > feels that we're inventing a problem just to be able to showcase the
> > solution :-)
> 
> Thanks for taking a look. I beg to differ though, as this solves (on the Rust
> side) a problem you described the other day on this very thread. The solution is
> different from what you propose though :)
> 
> - The internal data structures of drivers are refcounted. Drivers then share
>   this internal representation with other subsystems (e.g., cdev).

Refcounting the driver-specific structure is good, that matches what I
proposed (it's of course implemented differently in C and rust, but
that's expected).

> - On `remove`, the registrations with other subsystems are removed (so no
>   additional sharing of internal data should happen), but existing calls and
>   references to internal data structures continue to exist. This part is
>   important: we don't "revoke" the references, but we do revoke the hw resources
>   part of the internal state.

No issue here either. The handling of the internal data structure (the
"non-revoke" part to be precise) matches my proposal too I believe.
Revoking the I/O memory is of course rust-specific.

> - Attempts to access hardware resources freed during `remove` *must* be
>   prevented, that's where the calls to `resources()` are relevant -- if a
>   subsystem calls into the driver with one of the references it held on to, they
>   won't be able to access the (already released) hw resources.

That's where our opinions differ. Yes, those accesses must be prevented,
but I don't think the right way to do so is to check if the I/O memory
resource is still valid. We should instead prevent reaching driver code
paths that make those I/O accesses, by waiting for all calls in progress
to return, and preventing new calls from being made. This is a more
generic solution in the sense that it doesn't prevent accessing I/O
memory only, but avoids any operation that is not supposed to take
place. My reasoning is that drivers will be written with the assumption
that, for instance, nobody will try to set the GPIO direction once
.remove() returns. Even if the direction_output() function correctly
checks if the I/O memory is available and returns an error if it isn't,
it may also contain other logic that will not work correctly after
.remove() as the developer will not have considered that case. This
uncorrect logic may or may not lead to bugs, and some categories of bugs
may be prevented by rust (such as accessing I/O memory after .remove()),
but I don't think that's relevant. The subsystem, with minimal help from
the driver's implementation of the .remove() function if necessary,
should prevent operations from being called when they shouldn't, and
especially when the driver's author will not expect them to be called.
That way we'll address whole classes of issues in one go. And if we do
so, checking if I/O memory access has been revoked isn't required
anymore, as we guarantee if isn't.

True, this won't prevent I/O memory from being accessed after .remove()
in other contexts, for instance in a timer handler that the driver would
have registered and forgotten to cancel in .remove(). And maybe the I/O
memory revoking mechanism runtime overhead may be a reasonable price to
pay for avoiding this, I don't know. I however believe that regardless
of whether I/O memory is revoked or not, implementing a mechanism in the
subsytem to avoid erroneous conditions from happening in the first place
is where we'll get the largest benefit with a (hopefully) reasonable
effort.

> We have this problem specifically in gpio: as Linus explained, they created an
> indirection via a pointer which is checked in most entry points, but there is no
> synchronisation that guarantees that the pointer will remain valid during a
> call, and nothing forces uses of the pointer to be checked (so as Linus points
> out, they may need more checks).
> 
> For Rust drivers, if the registration with other subsystems were done by doing
> references to driver data in Rust, this extra "protection" (that has race
> conditions that, timed correctly, lead to use-after-free vulnerabilities) would
> be obviated; all would be handled safely on the Rust side (e.g., all accesses
> must be checked, there is no way to get to resources without a check, and use of
> the resources is guarded by a guard that uses RCU read-side lock).
> 
> Do you still think we don't have a problem?

We do have a problem, we just try to address it in different ways. And
of course mine is better, and I don't expect you to agree with this
statement right away ;-) Jokes aside, this has little to do with C vs.
rust in this case though, it's about how to model APIs between drivers
and subsystems.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Andy Lutomirski]

On Tue, Jul 20, 2021 at 4:54 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Wedson,
>
> On Mon, Jul 19, 2021 at 10:09:46PM +0100, Wedson Almeida Filho wrote:
> > On Mon, Jul 19, 2021 at 10:37:52PM +0300, Laurent Pinchart wrote:
> > > On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> > > > On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > > > > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > > > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > > > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > > >>> Note that this hardware apart from being used in all Arm reference
> > > > > >>> designs is used on ARMv4T systems that are not supported by
> > > > > >>> LLVM but only GCC, which might complicate things.
> > > > > >>
> > > > > >> Here is a working PL061 driver in Rust (converted form the C one):
> > > > > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > > > >
> > > > > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > > > > I try to line the definitions up to make it easier to contrast the two
> > > > > > implementations.
> > > > >
> > > > > This is really cool :-) As a Rust noob, I have a few questions:
> > > > >
> > > > > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > > > >
> > > > > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > > > > Result {
> > > > >         let woffset = bit(offset + 2).into();
> > > > >         let _guard = data.lock();
> > > > >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > >         let mut gpiodir = pl061.base.readb(GPIODIR);
> > > > >         gpiodir |= bit(offset);
> > > > >         pl061.base.writeb(gpiodir, GPIODIR);
> > > > >
> > > > >         // gpio value is set again, because pl061 doesn't allow to set
> > > > > value of a gpio pin before
> > > > >         // configuring it in OUT mode.
> > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > >         Ok(())
> > > > >     }
> > > > >
> > > > > Here you have try_writeb() (and error return) where there was just a
> > > > > writeb() without any error handling in the C version. Is this what
> > > > > Miguel was answering a bit down the thread where the address is computed
> > > > > ((value as u8) << offset) so it _needs_ to use the try_() version?
> > > >
> > > > The `writeb` variant only works when we know at compile-time that the offset is
> > > > within bounds (the compiler will reject the code otherwise). When the value is
> > > > computed at runtime we use a `try` version that checks before performing the
> > > > write. We need this to guarantee memory safety.
> > > >
> > > > > If offset can be anything but a "correct" value here, should there be a
> > > > > check for that somewhere else and then the computed value can be
> > > > > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > > > > function that now presumably does the runtime check a second time,
> > > > > redundantly)?
> > > >
> > > > Oh, that's a neat idea. We can certainly implement something like this:
> > > >
> > > > let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> > > >
> > > > Then woffset would be passed to writeb variants that are guaranteed to succeed.
> > > > (Rust helps us ensure that woffset cannot change without checks, which would be
> > > > harder to do in C.)
> > > >
> > > > > 2. In many places you have the C code:
> > > > >
> > > > > struct pl061 *pl061 = dev_get_drvdata(dev);
> > > > >
> > > > > with the equivalent Rust code as:
> > > > >
> > > > > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > >
> > > > > Why doesn't the C code need to check for errors here? Or put
> > > > > differently, why can the Rust version fail?
> > > >
> > > > There are two aspecs worth noting here:
> > > > 1. In C there is cast from void * to struct pl061 * without really knowing if
> > > > the stored pointer is of the right type. For example, if I simply change the
> > > > struct type to say `struct mutex` in the code above, it will still compile,
> > > > though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> > > > directly to drivers, and using type-specialised functions to set/get drvdata, so
> > > > it *knows* that the type is right. So in this sense Rust is better because it
> > > > offers type guarantees without additional runtime cost. (In Rust, if you change
> > > > the type of the function to say `&Mutex`, it won't compile.
> > > >
> > > > 2. The extra check we have here is because of a feature that the C code doesn't
> > > > have: revocable resources. If we didn't want to have this, we could do say
> > > > `data.base.writeb(...)` directly, but then we could have situations where `base`
> > > > is used after the device was removed. By having these checks we guarantee that
> > > > anyone can hold a reference to device state, but they can no longer use hw
> > > > resources after the device is removed.
> > >
> > > If the driver reached a code path with an I/O write after .remove()
> > > returns, the game is likely over already. It would be more interesting
> > > to see how we could prevent that from happening in the first place.
> > > Checking individual I/O writes at runtime will not only add additional
> > > CPU costs, but will also produce code paths that are not well tested.
> >
> > You may be conflating checking offsets in individual writes/reads with accessing
> > hw resources. Note that these are different things.
>
> Yes, it's the data.resources().ok_or() that I was talking about, not the
> I/O writes, sorry.
>
> > > It
> > > feels that we're inventing a problem just to be able to showcase the
> > > solution :-)
> >
> > Thanks for taking a look. I beg to differ though, as this solves (on the Rust
> > side) a problem you described the other day on this very thread. The solution is
> > different from what you propose though :)
> >
> > - The internal data structures of drivers are refcounted. Drivers then share
> >   this internal representation with other subsystems (e.g., cdev).
>
> Refcounting the driver-specific structure is good, that matches what I
> proposed (it's of course implemented differently in C and rust, but
> that's expected).
>
> > - On `remove`, the registrations with other subsystems are removed (so no
> >   additional sharing of internal data should happen), but existing calls and
> >   references to internal data structures continue to exist. This part is
> >   important: we don't "revoke" the references, but we do revoke the hw resources
> >   part of the internal state.
>
> No issue here either. The handling of the internal data structure (the
> "non-revoke" part to be precise) matches my proposal too I believe.
> Revoking the I/O memory is of course rust-specific.
>
> > - Attempts to access hardware resources freed during `remove` *must* be
> >   prevented, that's where the calls to `resources()` are relevant -- if a
> >   subsystem calls into the driver with one of the references it held on to, they
> >   won't be able to access the (already released) hw resources.
>
> That's where our opinions differ. Yes, those accesses must be prevented,
> but I don't think the right way to do so is to check if the I/O memory
> resource is still valid. We should instead prevent reaching driver code
> paths that make those I/O accesses, by waiting for all calls in progress
> to return, and preventing new calls from being made. This is a more
> generic solution in the sense that it doesn't prevent accessing I/O
> memory only, but avoids any operation that is not supposed to take
> place. My reasoning is that drivers will be written with the assumption
> that, for instance, nobody will try to set the GPIO direction once
> .remove() returns. Even if the direction_output() function correctly
> checks if the I/O memory is available and returns an error if it isn't,
> it may also contain other logic that will not work correctly after
> .remove() as the developer will not have considered that case. This
> uncorrect logic may or may not lead to bugs, and some categories of bugs
> may be prevented by rust (such as accessing I/O memory after .remove()),
> but I don't think that's relevant. The subsystem, with minimal help from
> the driver's implementation of the .remove() function if necessary,
> should prevent operations from being called when they shouldn't, and
> especially when the driver's author will not expect them to be called.
> That way we'll address whole classes of issues in one go. And if we do
> so, checking if I/O memory access has been revoked isn't required
> anymore, as we guarantee if isn't.
>
> True, this won't prevent I/O memory from being accessed after .remove()
> in other contexts, for instance in a timer handler that the driver would
> have registered and forgotten to cancel in .remove(). And maybe the I/O
> memory revoking mechanism runtime overhead may be a reasonable price to
> pay for avoiding this, I don't know. I however believe that regardless
> of whether I/O memory is revoked or not, implementing a mechanism in the
> subsytem to avoid erroneous conditions from happening in the first place
> is where we'll get the largest benefit with a (hopefully) reasonable
> effort.

Preventing functions from being called, when those functions are
scattered all over the place (sysfs, etc) may be hard.  Preventing
access to a resource seems much more tractable.  That being said,
preventing access to the I/O resource in particular seems a bit odd to
me.  It's really the whole device that's gone after it has been
removed.  So maybe the whole device (for some reasonable definition of
"whole device") could get revoked?

--Andy

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Andy,

On Tue, Jul 20, 2021 at 06:33:38PM -0700, Andy Lutomirski wrote:
> On Tue, Jul 20, 2021 at 4:54 PM Laurent Pinchart wrote:
> > On Mon, Jul 19, 2021 at 10:09:46PM +0100, Wedson Almeida Filho wrote:
> > > On Mon, Jul 19, 2021 at 10:37:52PM +0300, Laurent Pinchart wrote:
> > > > On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> > > > > On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > > > > > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > > > > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > > > > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > > > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > > > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > > > >>> Note that this hardware apart from being used in all Arm reference
> > > > > > >>> designs is used on ARMv4T systems that are not supported by
> > > > > > >>> LLVM but only GCC, which might complicate things.
> > > > > > >>
> > > > > > >> Here is a working PL061 driver in Rust (converted form the C one):
> > > > > > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > > > > >
> > > > > > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > > > > > I try to line the definitions up to make it easier to contrast the two
> > > > > > > implementations.
> > > > > >
> > > > > > This is really cool :-) As a Rust noob, I have a few questions:
> > > > > >
> > > > > > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > > > > >
> > > > > > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > > > > > Result {
> > > > > >         let woffset = bit(offset + 2).into();
> > > > > >         let _guard = data.lock();
> > > > > >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > > >         let mut gpiodir = pl061.base.readb(GPIODIR);
> > > > > >         gpiodir |= bit(offset);
> > > > > >         pl061.base.writeb(gpiodir, GPIODIR);
> > > > > >
> > > > > >         // gpio value is set again, because pl061 doesn't allow to set
> > > > > > value of a gpio pin before
> > > > > >         // configuring it in OUT mode.
> > > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > > >         Ok(())
> > > > > >     }
> > > > > >
> > > > > > Here you have try_writeb() (and error return) where there was just a
> > > > > > writeb() without any error handling in the C version. Is this what
> > > > > > Miguel was answering a bit down the thread where the address is computed
> > > > > > ((value as u8) << offset) so it _needs_ to use the try_() version?
> > > > >
> > > > > The `writeb` variant only works when we know at compile-time that the offset is
> > > > > within bounds (the compiler will reject the code otherwise). When the value is
> > > > > computed at runtime we use a `try` version that checks before performing the
> > > > > write. We need this to guarantee memory safety.
> > > > >
> > > > > > If offset can be anything but a "correct" value here, should there be a
> > > > > > check for that somewhere else and then the computed value can be
> > > > > > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > > > > > function that now presumably does the runtime check a second time,
> > > > > > redundantly)?
> > > > >
> > > > > Oh, that's a neat idea. We can certainly implement something like this:
> > > > >
> > > > > let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> > > > >
> > > > > Then woffset would be passed to writeb variants that are guaranteed to succeed.
> > > > > (Rust helps us ensure that woffset cannot change without checks, which would be
> > > > > harder to do in C.)
> > > > >
> > > > > > 2. In many places you have the C code:
> > > > > >
> > > > > > struct pl061 *pl061 = dev_get_drvdata(dev);
> > > > > >
> > > > > > with the equivalent Rust code as:
> > > > > >
> > > > > > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > >
> > > > > > Why doesn't the C code need to check for errors here? Or put
> > > > > > differently, why can the Rust version fail?
> > > > >
> > > > > There are two aspecs worth noting here:
> > > > > 1. In C there is cast from void * to struct pl061 * without really knowing if
> > > > > the stored pointer is of the right type. For example, if I simply change the
> > > > > struct type to say `struct mutex` in the code above, it will still compile,
> > > > > though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> > > > > directly to drivers, and using type-specialised functions to set/get drvdata, so
> > > > > it *knows* that the type is right. So in this sense Rust is better because it
> > > > > offers type guarantees without additional runtime cost. (In Rust, if you change
> > > > > the type of the function to say `&Mutex`, it won't compile.
> > > > >
> > > > > 2. The extra check we have here is because of a feature that the C code doesn't
> > > > > have: revocable resources. If we didn't want to have this, we could do say
> > > > > `data.base.writeb(...)` directly, but then we could have situations where `base`
> > > > > is used after the device was removed. By having these checks we guarantee that
> > > > > anyone can hold a reference to device state, but they can no longer use hw
> > > > > resources after the device is removed.
> > > >
> > > > If the driver reached a code path with an I/O write after .remove()
> > > > returns, the game is likely over already. It would be more interesting
> > > > to see how we could prevent that from happening in the first place.
> > > > Checking individual I/O writes at runtime will not only add additional
> > > > CPU costs, but will also produce code paths that are not well tested.
> > >
> > > You may be conflating checking offsets in individual writes/reads with accessing
> > > hw resources. Note that these are different things.
> >
> > Yes, it's the data.resources().ok_or() that I was talking about, not the
> > I/O writes, sorry.
> >
> > > > It
> > > > feels that we're inventing a problem just to be able to showcase the
> > > > solution :-)
> > >
> > > Thanks for taking a look. I beg to differ though, as this solves (on the Rust
> > > side) a problem you described the other day on this very thread. The solution is
> > > different from what you propose though :)
> > >
> > > - The internal data structures of drivers are refcounted. Drivers then share
> > >   this internal representation with other subsystems (e.g., cdev).
> >
> > Refcounting the driver-specific structure is good, that matches what I
> > proposed (it's of course implemented differently in C and rust, but
> > that's expected).
> >
> > > - On `remove`, the registrations with other subsystems are removed (so no
> > >   additional sharing of internal data should happen), but existing calls and
> > >   references to internal data structures continue to exist. This part is
> > >   important: we don't "revoke" the references, but we do revoke the hw resources
> > >   part of the internal state.
> >
> > No issue here either. The handling of the internal data structure (the
> > "non-revoke" part to be precise) matches my proposal too I believe.
> > Revoking the I/O memory is of course rust-specific.
> >
> > > - Attempts to access hardware resources freed during `remove` *must* be
> > >   prevented, that's where the calls to `resources()` are relevant -- if a
> > >   subsystem calls into the driver with one of the references it held on to, they
> > >   won't be able to access the (already released) hw resources.
> >
> > That's where our opinions differ. Yes, those accesses must be prevented,
> > but I don't think the right way to do so is to check if the I/O memory
> > resource is still valid. We should instead prevent reaching driver code
> > paths that make those I/O accesses, by waiting for all calls in progress
> > to return, and preventing new calls from being made. This is a more
> > generic solution in the sense that it doesn't prevent accessing I/O
> > memory only, but avoids any operation that is not supposed to take
> > place. My reasoning is that drivers will be written with the assumption
> > that, for instance, nobody will try to set the GPIO direction once
> > .remove() returns. Even if the direction_output() function correctly
> > checks if the I/O memory is available and returns an error if it isn't,
> > it may also contain other logic that will not work correctly after
> > .remove() as the developer will not have considered that case. This
> > uncorrect logic may or may not lead to bugs, and some categories of bugs
> > may be prevented by rust (such as accessing I/O memory after .remove()),
> > but I don't think that's relevant. The subsystem, with minimal help from
> > the driver's implementation of the .remove() function if necessary,
> > should prevent operations from being called when they shouldn't, and
> > especially when the driver's author will not expect them to be called.
> > That way we'll address whole classes of issues in one go. And if we do
> > so, checking if I/O memory access has been revoked isn't required
> > anymore, as we guarantee if isn't.
> >
> > True, this won't prevent I/O memory from being accessed after .remove()
> > in other contexts, for instance in a timer handler that the driver would
> > have registered and forgotten to cancel in .remove(). And maybe the I/O
> > memory revoking mechanism runtime overhead may be a reasonable price to
> > pay for avoiding this, I don't know. I however believe that regardless
> > of whether I/O memory is revoked or not, implementing a mechanism in the
> > subsytem to avoid erroneous conditions from happening in the first place
> > is where we'll get the largest benefit with a (hopefully) reasonable
> > effort.
> 
> Preventing functions from being called, when those functions are
> scattered all over the place (sysfs, etc) may be hard.  Preventing
> access to a resource seems much more tractable.  That being said,
> preventing access to the I/O resource in particular seems a bit odd to
> me.  It's really the whole device that's gone after it has been
> removed.  So maybe the whole device (for some reasonable definition of
> "whole device") could get revoked?

I agree that we should view this from a resource point of view, or
perhaps an object point of view, instead of looking at individual
functions. A GPIO driver creates GPIO controllers, which are objects
that expose an API to consumers. In most cases, it's the whole API that
shouldn't be called after .remove(), not individual functions (there are
a few exceptions, for instead the .release() file operation for objects
exposed through userspace as a cdev may need to reach the object). What
I'd like to see is the subsystem blocking this, instead of having
individual drivers tasked with correctly handling API calls from
consumers after .remove(). The correctness of the latter may be less
difficult to achieve and even guarantee with rust, but that's solving a
problem that shouldn't exist in the first place.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Wed, Jul 21, 2021 at 3:42 AM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> [Wedson]
> > Preventing functions from being called, when those functions are
> > scattered all over the place (sysfs, etc) may be hard.  Preventing
> > access to a resource seems much more tractable.  That being said,
> > preventing access to the I/O resource in particular seems a bit odd to
> > me.  It's really the whole device that's gone after it has been
> > removed.  So maybe the whole device (for some reasonable definition of
> > "whole device") could get revoked?
>
> I agree that we should view this from a resource point of view, or
> perhaps an object point of view, instead of looking at individual
> functions. A GPIO driver creates GPIO controllers, which are objects
> that expose an API to consumers. In most cases, it's the whole API that
> shouldn't be called after .remove(), not individual functions (there are
> a few exceptions, for instead the .release() file operation for objects
> exposed through userspace as a cdev may need to reach the object). What
> I'd like to see is the subsystem blocking this, instead of having
> individual drivers tasked with correctly handling API calls from
> consumers after .remove(). The correctness of the latter may be less
> difficult to achieve and even guarantee with rust, but that's solving a
> problem that shouldn't exist in the first place.

We have something like this for GPIO but it is coded by a notoriously
crappy programmer (me) in C, so there are some bugs in it pointed
out by Miguel.
https://lore.kernel.org/ksummit/CACRpkdasOaNgBAZVx5qpKJdU7h41jHDG2jWi2+pi9a1JBh7RTQ@mail.gmail.com/

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Wed, Jul 21, 2021 at 03:54:33PM +0200, Linus Walleij wrote:
> On Wed, Jul 21, 2021 at 3:42 AM Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> > [Wedson]
> > > Preventing functions from being called, when those functions are
> > > scattered all over the place (sysfs, etc) may be hard.  Preventing
> > > access to a resource seems much more tractable.  That being said,
> > > preventing access to the I/O resource in particular seems a bit odd to
> > > me.  It's really the whole device that's gone after it has been
> > > removed.  So maybe the whole device (for some reasonable definition of
> > > "whole device") could get revoked?
> >
> > I agree that we should view this from a resource point of view, or
> > perhaps an object point of view, instead of looking at individual
> > functions. A GPIO driver creates GPIO controllers, which are objects
> > that expose an API to consumers. In most cases, it's the whole API that
> > shouldn't be called after .remove(), not individual functions (there are
> > a few exceptions, for instead the .release() file operation for objects
> > exposed through userspace as a cdev may need to reach the object). What
> > I'd like to see is the subsystem blocking this, instead of having
> > individual drivers tasked with correctly handling API calls from
> > consumers after .remove(). The correctness of the latter may be less
> > difficult to achieve and even guarantee with rust, but that's solving a
> > problem that shouldn't exist in the first place.
> 
> We have something like this for GPIO but it is coded by a notoriously
> crappy programmer (me) in C, so there are some bugs in it pointed
> out by Miguel.
> https://lore.kernel.org/ksummit/CACRpkdasOaNgBAZVx5qpKJdU7h41jHDG2jWi2+pi9a1JBh7RTQ@mail.gmail.com/

Would you mind sharing details of the bugs Miguel found?

Thanks,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Wed, Jul 21, 2021 at 4:13 PM Wedson Almeida Filho
<wedsonaf@google.com> wrote:
> On Wed, Jul 21, 2021 at 03:54:33PM +0200, Linus Walleij wrote:

> > We have something like this for GPIO but it is coded by a notoriously
> > crappy programmer (me) in C, so there are some bugs in it pointed
> > out by Miguel.
> > https://lore.kernel.org/ksummit/CACRpkdasOaNgBAZVx5qpKJdU7h41jHDG2jWi2+pi9a1JBh7RTQ@mail.gmail.com/
>
> Would you mind sharing details of the bugs Miguel found?

Nah, just me being confused. They were found by you :)
https://lore.kernel.org/ksummit/CACRpkdb1W=M5EJkGbSS4QxObU-Gd5yZ1qE439k_D4K=jevgcrQ@mail.gmail.com/

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Wed, Jul 21, 2021 at 04:19:09PM +0200, Linus Walleij wrote:
> On Wed, Jul 21, 2021 at 4:13 PM Wedson Almeida Filho
> <wedsonaf@google.com> wrote:
> > On Wed, Jul 21, 2021 at 03:54:33PM +0200, Linus Walleij wrote:
> 
> > > We have something like this for GPIO but it is coded by a notoriously
> > > crappy programmer (me) in C, so there are some bugs in it pointed
> > > out by Miguel.
> > > https://lore.kernel.org/ksummit/CACRpkdasOaNgBAZVx5qpKJdU7h41jHDG2jWi2+pi9a1JBh7RTQ@mail.gmail.com/
> >
> > Would you mind sharing details of the bugs Miguel found?
> 
> Nah, just me being confused. They were found by you :)
> https://lore.kernel.org/ksummit/CACRpkdb1W=M5EJkGbSS4QxObU-Gd5yZ1qE439k_D4K=jevgcrQ@mail.gmail.com/

I had two more questions from reading the code that you may be able to answer:
1. Drivers like the one for pl061 call `irq_set_irq_wake` from their
   `irq_set_wake` callbacks. These are "counted", which is great because it
   allows us to just call the same function for each gpio/interrupt. However,
   when a gpio irq chip is removed, what ensures that these increments are
   accordingly decremented?
2. `irq_domain_remove` requires that all mappings have been disposed of before
   being called. I see that `gpiochip_irqchip_remove` calls
   `irq_dispose_mapping` for each pin before calling `irq_domain_remove`. But
   what prevents *new* mappings from being added back before `irq_domain_remove`
   is called?

Thanks,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Thu, Jul 22, 2021 at 1:34 PM Wedson Almeida Filho
<wedsonaf@google.com> wrote:

> I had two more questions from reading the code that you may be able to answer:
> 1. Drivers like the one for pl061 call `irq_set_irq_wake` from their
>    `irq_set_wake` callbacks. These are "counted", which is great because it
>    allows us to just call the same function for each gpio/interrupt. However,
>    when a gpio irq chip is removed, what ensures that these increments are
>    accordingly decremented?

Really a question for the irqchip people.

I think you will just see many shortcomings of this type since
the irqchip has very little state in general.
IIUC the framework is designed to be be very lightweight.

The following is based on my relatively superficial understanding
of irqchips.

I don't really get what you mean with "counted" here?

It's just consumers setting a flag that that IRQ should be able to wake
up the system.

In PL061 that percolates up to the parent, the cascaded IRQ.
The parent doesn't "count" the number of wake flags either, not
any driver I know of, the framework only provides these callbacks,
it has no state.

The only practical effect is setting a bit in the hardware (if available)
that the IRQ can wake up the system so the hardware bit *is* the
state. The driver may record it in its state container to ascertain not
to shut off clocks when sleeping for example but in practice it's often
not very thorough.

The IRQ wake status of the parent before we probe is unknown and
there is no way to retrieve it. It can very well be a power-on default,
so by definition undefined.

It could be dangerous to attempt to set it back to disabled when we
leave the since the IRQ may be shared and the irqchip doesn't count
so that will yield a first come-first serve bug.

In practice there are a few calls to this function and the same bit
will be set in hardware a few times from different callers and that
is fine because it is all that is needed most times. Then the whole
state goes away with the hardware it is in when the system goes
down.

I think this is partly by design to keep the state in the hardware.
If the hardware changes state because of external factors, then
the state reflects reality through reading the registers etc.
This is maybe a questionable design principle.

> 2. `irq_domain_remove` requires that all mappings have been disposed of before
>    being called. I see that `gpiochip_irqchip_remove` calls
>    `irq_dispose_mapping` for each pin before calling `irq_domain_remove`. But
>    what prevents *new* mappings from being added back before `irq_domain_remove`
>    is called?

Again mostly a question for the irqchip people I guess but nothing I
think since the irqdomain does not have any synchronization
primitives or any counting.

We have made it mostly work fine in practice for many years
because in practice there is often just one way in which the
IRQ tree can come up and down and once up it doesn't change
much.

It could be more elegant I guess. Especially since it could very well
have state as the domain is just a piece of software.

If you start to think that irqchip could use some more developers
and maintainers you are probably right.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Tue, Jul 20, 2021 at 06:33:38PM -0700, Andy Lutomirski wrote:
> On Tue, Jul 20, 2021 at 4:54 PM Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> >
> > Hi Wedson,
> >
> > On Mon, Jul 19, 2021 at 10:09:46PM +0100, Wedson Almeida Filho wrote:
> > > On Mon, Jul 19, 2021 at 10:37:52PM +0300, Laurent Pinchart wrote:
> > > > On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> > > > > On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > > > > > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > > > > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > > > > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > > > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > > > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > > > >>> Note that this hardware apart from being used in all Arm reference
> > > > > > >>> designs is used on ARMv4T systems that are not supported by
> > > > > > >>> LLVM but only GCC, which might complicate things.
> > > > > > >>
> > > > > > >> Here is a working PL061 driver in Rust (converted form the C one):
> > > > > > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > > > > >
> > > > > > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > > > > > I try to line the definitions up to make it easier to contrast the two
> > > > > > > implementations.
> > > > > >
> > > > > > This is really cool :-) As a Rust noob, I have a few questions:
> > > > > >
> > > > > > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > > > > >
> > > > > > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > > > > > Result {
> > > > > >         let woffset = bit(offset + 2).into();
> > > > > >         let _guard = data.lock();
> > > > > >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > > >         let mut gpiodir = pl061.base.readb(GPIODIR);
> > > > > >         gpiodir |= bit(offset);
> > > > > >         pl061.base.writeb(gpiodir, GPIODIR);
> > > > > >
> > > > > >         // gpio value is set again, because pl061 doesn't allow to set
> > > > > > value of a gpio pin before
> > > > > >         // configuring it in OUT mode.
> > > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > > >         Ok(())
> > > > > >     }
> > > > > >
> > > > > > Here you have try_writeb() (and error return) where there was just a
> > > > > > writeb() without any error handling in the C version. Is this what
> > > > > > Miguel was answering a bit down the thread where the address is computed
> > > > > > ((value as u8) << offset) so it _needs_ to use the try_() version?
> > > > >
> > > > > The `writeb` variant only works when we know at compile-time that the offset is
> > > > > within bounds (the compiler will reject the code otherwise). When the value is
> > > > > computed at runtime we use a `try` version that checks before performing the
> > > > > write. We need this to guarantee memory safety.
> > > > >
> > > > > > If offset can be anything but a "correct" value here, should there be a
> > > > > > check for that somewhere else and then the computed value can be
> > > > > > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > > > > > function that now presumably does the runtime check a second time,
> > > > > > redundantly)?
> > > > >
> > > > > Oh, that's a neat idea. We can certainly implement something like this:
> > > > >
> > > > > let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> > > > >
> > > > > Then woffset would be passed to writeb variants that are guaranteed to succeed.
> > > > > (Rust helps us ensure that woffset cannot change without checks, which would be
> > > > > harder to do in C.)
> > > > >
> > > > > > 2. In many places you have the C code:
> > > > > >
> > > > > > struct pl061 *pl061 = dev_get_drvdata(dev);
> > > > > >
> > > > > > with the equivalent Rust code as:
> > > > > >
> > > > > > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > >
> > > > > > Why doesn't the C code need to check for errors here? Or put
> > > > > > differently, why can the Rust version fail?
> > > > >
> > > > > There are two aspecs worth noting here:
> > > > > 1. In C there is cast from void * to struct pl061 * without really knowing if
> > > > > the stored pointer is of the right type. For example, if I simply change the
> > > > > struct type to say `struct mutex` in the code above, it will still compile,
> > > > > though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> > > > > directly to drivers, and using type-specialised functions to set/get drvdata, so
> > > > > it *knows* that the type is right. So in this sense Rust is better because it
> > > > > offers type guarantees without additional runtime cost. (In Rust, if you change
> > > > > the type of the function to say `&Mutex`, it won't compile.
> > > > >
> > > > > 2. The extra check we have here is because of a feature that the C code doesn't
> > > > > have: revocable resources. If we didn't want to have this, we could do say
> > > > > `data.base.writeb(...)` directly, but then we could have situations where `base`
> > > > > is used after the device was removed. By having these checks we guarantee that
> > > > > anyone can hold a reference to device state, but they can no longer use hw
> > > > > resources after the device is removed.
> > > >
> > > > If the driver reached a code path with an I/O write after .remove()
> > > > returns, the game is likely over already. It would be more interesting
> > > > to see how we could prevent that from happening in the first place.
> > > > Checking individual I/O writes at runtime will not only add additional
> > > > CPU costs, but will also produce code paths that are not well tested.
> > >
> > > You may be conflating checking offsets in individual writes/reads with accessing
> > > hw resources. Note that these are different things.
> >
> > Yes, it's the data.resources().ok_or() that I was talking about, not the
> > I/O writes, sorry.
> >
> > > > It
> > > > feels that we're inventing a problem just to be able to showcase the
> > > > solution :-)
> > >
> > > Thanks for taking a look. I beg to differ though, as this solves (on the Rust
> > > side) a problem you described the other day on this very thread. The solution is
> > > different from what you propose though :)
> > >
> > > - The internal data structures of drivers are refcounted. Drivers then share
> > >   this internal representation with other subsystems (e.g., cdev).
> >
> > Refcounting the driver-specific structure is good, that matches what I
> > proposed (it's of course implemented differently in C and rust, but
> > that's expected).
> >
> > > - On `remove`, the registrations with other subsystems are removed (so no
> > >   additional sharing of internal data should happen), but existing calls and
> > >   references to internal data structures continue to exist. This part is
> > >   important: we don't "revoke" the references, but we do revoke the hw resources
> > >   part of the internal state.
> >
> > No issue here either. The handling of the internal data structure (the
> > "non-revoke" part to be precise) matches my proposal too I believe.
> > Revoking the I/O memory is of course rust-specific.
> >
> > > - Attempts to access hardware resources freed during `remove` *must* be
> > >   prevented, that's where the calls to `resources()` are relevant -- if a
> > >   subsystem calls into the driver with one of the references it held on to, they
> > >   won't be able to access the (already released) hw resources.
> >
> > That's where our opinions differ. Yes, those accesses must be prevented,
> > but I don't think the right way to do so is to check if the I/O memory
> > resource is still valid. We should instead prevent reaching driver code
> > paths that make those I/O accesses, by waiting for all calls in progress
> > to return, and preventing new calls from being made. This is a more
> > generic solution in the sense that it doesn't prevent accessing I/O
> > memory only, but avoids any operation that is not supposed to take
> > place. My reasoning is that drivers will be written with the assumption
> > that, for instance, nobody will try to set the GPIO direction once
> > .remove() returns. Even if the direction_output() function correctly
> > checks if the I/O memory is available and returns an error if it isn't,
> > it may also contain other logic that will not work correctly after
> > .remove() as the developer will not have considered that case. This
> > uncorrect logic may or may not lead to bugs, and some categories of bugs
> > may be prevented by rust (such as accessing I/O memory after .remove()),
> > but I don't think that's relevant. The subsystem, with minimal help from
> > the driver's implementation of the .remove() function if necessary,
> > should prevent operations from being called when they shouldn't, and
> > especially when the driver's author will not expect them to be called.
> > That way we'll address whole classes of issues in one go. And if we do
> > so, checking if I/O memory access has been revoked isn't required
> > anymore, as we guarantee if isn't.
> >
> > True, this won't prevent I/O memory from being accessed after .remove()
> > in other contexts, for instance in a timer handler that the driver would
> > have registered and forgotten to cancel in .remove(). And maybe the I/O
> > memory revoking mechanism runtime overhead may be a reasonable price to
> > pay for avoiding this, I don't know. I however believe that regardless
> > of whether I/O memory is revoked or not, implementing a mechanism in the
> > subsytem to avoid erroneous conditions from happening in the first place
> > is where we'll get the largest benefit with a (hopefully) reasonable
> > effort.
> 
> Preventing functions from being called, when those functions are
> scattered all over the place (sysfs, etc) may be hard.  Preventing
> access to a resource seems much more tractable.  That being said,
> preventing access to the I/O resource in particular seems a bit odd to
> me.  It's really the whole device that's gone after it has been
> removed.  So maybe the whole device (for some reasonable definition of
> "whole device") could get revoked?

(Some details that I omitted from the original email for brevity.)

I actually split device state into three categories:
1. Registrations: state needed by other subsystems (e.g., `cdev`, `gpio_chip`)
2. Resources: part of the device state that is revoked when .remove() completes.
3. General: state available to anyone who holds a reference to the state.

The above is expressed in the following line on the driver:
type DeviceData = device::Data<PL061Registrations, PL061Resources, IrqDisableSpinLock<PL061Data>>;

Developers are free to put whatever they want in each of the categories. What
happens automatically when a device is about to be removed is:
1. .remove() callback is called, if one exists.
2. Registrations are dropped (e.g., subsystems are called to unregister previous
successful registrations)
3. Resources are revoked then dropped (e.g., io mem is unmapped).

[This happens automatically in Rust, even if a driver doesn't implement
.remove()]

The reason I split registration & resources is because I wanted to guarantee
that the latter are only revoked after registrations are gone. (Because we may
need the resources while unregistration is in progress.)

The reason there is a split between a revocable and non-revocable part is that
we could have things like links of intrusive data structures that code holding
on to references may need to have access to even after .remove().

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Wedson,

On Wed, Jul 21, 2021 at 05:39:31AM +0100, Wedson Almeida Filho wrote:
> On Tue, Jul 20, 2021 at 06:33:38PM -0700, Andy Lutomirski wrote:
> > On Tue, Jul 20, 2021 at 4:54 PM Laurent Pinchart wrote:
> > > On Mon, Jul 19, 2021 at 10:09:46PM +0100, Wedson Almeida Filho wrote:
> > > > On Mon, Jul 19, 2021 at 10:37:52PM +0300, Laurent Pinchart wrote:
> > > > > On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> > > > > > On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > > > > > > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > > > > > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > > > > > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > > > > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > > > > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > > > > >>> Note that this hardware apart from being used in all Arm reference
> > > > > > > >>> designs is used on ARMv4T systems that are not supported by
> > > > > > > >>> LLVM but only GCC, which might complicate things.
> > > > > > > >>
> > > > > > > >> Here is a working PL061 driver in Rust (converted form the C one):
> > > > > > > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > > > > > >
> > > > > > > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > > > > > > I try to line the definitions up to make it easier to contrast the two
> > > > > > > > implementations.
> > > > > > >
> > > > > > > This is really cool :-) As a Rust noob, I have a few questions:
> > > > > > >
> > > > > > > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > > > > > >
> > > > > > > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > > > > > > Result {
> > > > > > >         let woffset = bit(offset + 2).into();
> > > > > > >         let _guard = data.lock();
> > > > > > >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > > > >         let mut gpiodir = pl061.base.readb(GPIODIR);
> > > > > > >         gpiodir |= bit(offset);
> > > > > > >         pl061.base.writeb(gpiodir, GPIODIR);
> > > > > > >
> > > > > > >         // gpio value is set again, because pl061 doesn't allow to set
> > > > > > > value of a gpio pin before
> > > > > > >         // configuring it in OUT mode.
> > > > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > > > >         Ok(())
> > > > > > >     }
> > > > > > >
> > > > > > > Here you have try_writeb() (and error return) where there was just a
> > > > > > > writeb() without any error handling in the C version. Is this what
> > > > > > > Miguel was answering a bit down the thread where the address is computed
> > > > > > > ((value as u8) << offset) so it _needs_ to use the try_() version?
> > > > > >
> > > > > > The `writeb` variant only works when we know at compile-time that the offset is
> > > > > > within bounds (the compiler will reject the code otherwise). When the value is
> > > > > > computed at runtime we use a `try` version that checks before performing the
> > > > > > write. We need this to guarantee memory safety.
> > > > > >
> > > > > > > If offset can be anything but a "correct" value here, should there be a
> > > > > > > check for that somewhere else and then the computed value can be
> > > > > > > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > > > > > > function that now presumably does the runtime check a second time,
> > > > > > > redundantly)?
> > > > > >
> > > > > > Oh, that's a neat idea. We can certainly implement something like this:
> > > > > >
> > > > > > let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> > > > > >
> > > > > > Then woffset would be passed to writeb variants that are guaranteed to succeed.
> > > > > > (Rust helps us ensure that woffset cannot change without checks, which would be
> > > > > > harder to do in C.)
> > > > > >
> > > > > > > 2. In many places you have the C code:
> > > > > > >
> > > > > > > struct pl061 *pl061 = dev_get_drvdata(dev);
> > > > > > >
> > > > > > > with the equivalent Rust code as:
> > > > > > >
> > > > > > > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > > >
> > > > > > > Why doesn't the C code need to check for errors here? Or put
> > > > > > > differently, why can the Rust version fail?
> > > > > >
> > > > > > There are two aspecs worth noting here:
> > > > > > 1. In C there is cast from void * to struct pl061 * without really knowing if
> > > > > > the stored pointer is of the right type. For example, if I simply change the
> > > > > > struct type to say `struct mutex` in the code above, it will still compile,
> > > > > > though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> > > > > > directly to drivers, and using type-specialised functions to set/get drvdata, so
> > > > > > it *knows* that the type is right. So in this sense Rust is better because it
> > > > > > offers type guarantees without additional runtime cost. (In Rust, if you change
> > > > > > the type of the function to say `&Mutex`, it won't compile.
> > > > > >
> > > > > > 2. The extra check we have here is because of a feature that the C code doesn't
> > > > > > have: revocable resources. If we didn't want to have this, we could do say
> > > > > > `data.base.writeb(...)` directly, but then we could have situations where `base`
> > > > > > is used after the device was removed. By having these checks we guarantee that
> > > > > > anyone can hold a reference to device state, but they can no longer use hw
> > > > > > resources after the device is removed.
> > > > >
> > > > > If the driver reached a code path with an I/O write after .remove()
> > > > > returns, the game is likely over already. It would be more interesting
> > > > > to see how we could prevent that from happening in the first place.
> > > > > Checking individual I/O writes at runtime will not only add additional
> > > > > CPU costs, but will also produce code paths that are not well tested.
> > > >
> > > > You may be conflating checking offsets in individual writes/reads with accessing
> > > > hw resources. Note that these are different things.
> > >
> > > Yes, it's the data.resources().ok_or() that I was talking about, not the
> > > I/O writes, sorry.
> > >
> > > > > It
> > > > > feels that we're inventing a problem just to be able to showcase the
> > > > > solution :-)
> > > >
> > > > Thanks for taking a look. I beg to differ though, as this solves (on the Rust
> > > > side) a problem you described the other day on this very thread. The solution is
> > > > different from what you propose though :)
> > > >
> > > > - The internal data structures of drivers are refcounted. Drivers then share
> > > >   this internal representation with other subsystems (e.g., cdev).
> > >
> > > Refcounting the driver-specific structure is good, that matches what I
> > > proposed (it's of course implemented differently in C and rust, but
> > > that's expected).
> > >
> > > > - On `remove`, the registrations with other subsystems are removed (so no
> > > >   additional sharing of internal data should happen), but existing calls and
> > > >   references to internal data structures continue to exist. This part is
> > > >   important: we don't "revoke" the references, but we do revoke the hw resources
> > > >   part of the internal state.
> > >
> > > No issue here either. The handling of the internal data structure (the
> > > "non-revoke" part to be precise) matches my proposal too I believe.
> > > Revoking the I/O memory is of course rust-specific.
> > >
> > > > - Attempts to access hardware resources freed during `remove` *must* be
> > > >   prevented, that's where the calls to `resources()` are relevant -- if a
> > > >   subsystem calls into the driver with one of the references it held on to, they
> > > >   won't be able to access the (already released) hw resources.
> > >
> > > That's where our opinions differ. Yes, those accesses must be prevented,
> > > but I don't think the right way to do so is to check if the I/O memory
> > > resource is still valid. We should instead prevent reaching driver code
> > > paths that make those I/O accesses, by waiting for all calls in progress
> > > to return, and preventing new calls from being made. This is a more
> > > generic solution in the sense that it doesn't prevent accessing I/O
> > > memory only, but avoids any operation that is not supposed to take
> > > place. My reasoning is that drivers will be written with the assumption
> > > that, for instance, nobody will try to set the GPIO direction once
> > > .remove() returns. Even if the direction_output() function correctly
> > > checks if the I/O memory is available and returns an error if it isn't,
> > > it may also contain other logic that will not work correctly after
> > > .remove() as the developer will not have considered that case. This
> > > uncorrect logic may or may not lead to bugs, and some categories of bugs
> > > may be prevented by rust (such as accessing I/O memory after .remove()),
> > > but I don't think that's relevant. The subsystem, with minimal help from
> > > the driver's implementation of the .remove() function if necessary,
> > > should prevent operations from being called when they shouldn't, and
> > > especially when the driver's author will not expect them to be called.
> > > That way we'll address whole classes of issues in one go. And if we do
> > > so, checking if I/O memory access has been revoked isn't required
> > > anymore, as we guarantee if isn't.
> > >
> > > True, this won't prevent I/O memory from being accessed after .remove()
> > > in other contexts, for instance in a timer handler that the driver would
> > > have registered and forgotten to cancel in .remove(). And maybe the I/O
> > > memory revoking mechanism runtime overhead may be a reasonable price to
> > > pay for avoiding this, I don't know. I however believe that regardless
> > > of whether I/O memory is revoked or not, implementing a mechanism in the
> > > subsytem to avoid erroneous conditions from happening in the first place
> > > is where we'll get the largest benefit with a (hopefully) reasonable
> > > effort.
> > 
> > Preventing functions from being called, when those functions are
> > scattered all over the place (sysfs, etc) may be hard.  Preventing
> > access to a resource seems much more tractable.  That being said,
> > preventing access to the I/O resource in particular seems a bit odd to
> > me.  It's really the whole device that's gone after it has been
> > removed.  So maybe the whole device (for some reasonable definition of
> > "whole device") could get revoked?
> 
> (Some details that I omitted from the original email for brevity.)
> 
> I actually split device state into three categories:
> 1. Registrations: state needed by other subsystems (e.g., `cdev`, `gpio_chip`)
> 2. Resources: part of the device state that is revoked when .remove() completes.
> 3. General: state available to anyone who holds a reference to the state.
> 
> The above is expressed in the following line on the driver:
> type DeviceData = device::Data<PL061Registrations, PL061Resources, IrqDisableSpinLock<PL061Data>>;
> 
> Developers are free to put whatever they want in each of the categories. What
> happens automatically when a device is about to be removed is:
> 1. .remove() callback is called, if one exists.
> 2. Registrations are dropped (e.g., subsystems are called to unregister previous
> successful registrations)

In practice the order in which the unregistrations and other cleanups
are performed will matter in most cases, so I'd be very cautious about
automating this step. Drivers often need to register multiple resources
(I was looking for a different word here, to match your list above, but
I think the distinction between registrations and resources isn't
semantically correct, it's about what resources a driver provides and
registers and what resources it acquires and consumes).

> 3. Resources are revoked then dropped (e.g., io mem is unmapped).

I don't like the word "revoke" here. Revoking a resource sounds like
something that is done externally, where the resource disappears, and
its access thus needs to be revoked. Calling a driver releasing a
resource at remove time (for instance the equivalent of the clk_put()
call that matches a clk_get() in the probe function - I know that in
practice drivers use devm_clk_get(), this is just to provide an example)
"revoking" seems a bit misleading. The clock object doesn't disappear,
it only gets released by the driver.

> [This happens automatically in Rust, even if a driver doesn't implement
> .remove()]
> 
> The reason I split registration & resources is because I wanted to guarantee
> that the latter are only revoked after registrations are gone. (Because we may
> need the resources while unregistration is in progress.)

This is correct.

> The reason there is a split between a revocable and non-revocable part is that
> we could have things like links of intrusive data structures that code holding
> on to references may need to have access to even after .remove().

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

Hi Laurent,

On Wed, Jul 21, 2021 at 02:54:24AM +0300, Laurent Pinchart wrote:
> Hi Wedson,
> On Mon, Jul 19, 2021 at 10:09:46PM +0100, Wedson Almeida Filho wrote:
> > On Mon, Jul 19, 2021 at 10:37:52PM +0300, Laurent Pinchart wrote:
> > > On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> > > > On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > > > > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > > > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > > > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > > >>> Note that this hardware apart from being used in all Arm reference
> > > > > >>> designs is used on ARMv4T systems that are not supported by
> > > > > >>> LLVM but only GCC, which might complicate things.
> > > > > >>
> > > > > >> Here is a working PL061 driver in Rust (converted form the C one):
> > > > > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > > > > 
> > > > > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > > > > I try to line the definitions up to make it easier to contrast the two
> > > > > > implementations.
> > > > > 
> > > > > This is really cool :-) As a Rust noob, I have a few questions:
> > > > > 
> > > > > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > > > > 
> > > > > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > > > > Result {
> > > > >         let woffset = bit(offset + 2).into();
> > > > >         let _guard = data.lock();
> > > > >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > >         let mut gpiodir = pl061.base.readb(GPIODIR);
> > > > >         gpiodir |= bit(offset);
> > > > >         pl061.base.writeb(gpiodir, GPIODIR);
> > > > > 
> > > > >         // gpio value is set again, because pl061 doesn't allow to set
> > > > > value of a gpio pin before
> > > > >         // configuring it in OUT mode.
> > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > >         Ok(())
> > > > >     }
> > > > > 
> > > > > Here you have try_writeb() (and error return) where there was just a
> > > > > writeb() without any error handling in the C version. Is this what
> > > > > Miguel was answering a bit down the thread where the address is computed
> > > > > ((value as u8) << offset) so it _needs_ to use the try_() version?
> > > > 
> > > > The `writeb` variant only works when we know at compile-time that the offset is
> > > > within bounds (the compiler will reject the code otherwise). When the value is
> > > > computed at runtime we use a `try` version that checks before performing the
> > > > write. We need this to guarantee memory safety.
> > > > 
> > > > > If offset can be anything but a "correct" value here, should there be a
> > > > > check for that somewhere else and then the computed value can be
> > > > > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > > > > function that now presumably does the runtime check a second time,
> > > > > redundantly)?
> > > > 
> > > > Oh, that's a neat idea. We can certainly implement something like this:
> > > > 
> > > > let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> > > > 
> > > > Then woffset would be passed to writeb variants that are guaranteed to succeed.
> > > > (Rust helps us ensure that woffset cannot change without checks, which would be
> > > > harder to do in C.)
> > > > 
> > > > > 2. In many places you have the C code:
> > > > > 
> > > > > struct pl061 *pl061 = dev_get_drvdata(dev);
> > > > > 
> > > > > with the equivalent Rust code as:
> > > > > 
> > > > > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > 
> > > > > Why doesn't the C code need to check for errors here? Or put
> > > > > differently, why can the Rust version fail?
> > > > 
> > > > There are two aspecs worth noting here:
> > > > 1. In C there is cast from void * to struct pl061 * without really knowing if
> > > > the stored pointer is of the right type. For example, if I simply change the
> > > > struct type to say `struct mutex` in the code above, it will still compile,
> > > > though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> > > > directly to drivers, and using type-specialised functions to set/get drvdata, so
> > > > it *knows* that the type is right. So in this sense Rust is better because it
> > > > offers type guarantees without additional runtime cost. (In Rust, if you change
> > > > the type of the function to say `&Mutex`, it won't compile.
> > > > 
> > > > 2. The extra check we have here is because of a feature that the C code doesn't
> > > > have: revocable resources. If we didn't want to have this, we could do say
> > > > `data.base.writeb(...)` directly, but then we could have situations where `base`
> > > > is used after the device was removed. By having these checks we guarantee that
> > > > anyone can hold a reference to device state, but they can no longer use hw
> > > > resources after the device is removed.
> > > 
> > > If the driver reached a code path with an I/O write after .remove()
> > > returns, the game is likely over already. It would be more interesting
> > > to see how we could prevent that from happening in the first place.
> > > Checking individual I/O writes at runtime will not only add additional
> > > CPU costs, but will also produce code paths that are not well tested.
> > 
> > You may be conflating checking offsets in individual writes/reads with accessing
> > hw resources. Note that these are different things.
> 
> Yes, it's the data.resources().ok_or() that I was talking about, not the
> I/O writes, sorry.
> 
> > > It
> > > feels that we're inventing a problem just to be able to showcase the
> > > solution :-)
> > 
> > Thanks for taking a look. I beg to differ though, as this solves (on the Rust
> > side) a problem you described the other day on this very thread. The solution is
> > different from what you propose though :)
> > 
> > - The internal data structures of drivers are refcounted. Drivers then share
> >   this internal representation with other subsystems (e.g., cdev).
> 
> Refcounting the driver-specific structure is good, that matches what I
> proposed (it's of course implemented differently in C and rust, but
> that's expected).

The refcounting business is indeed different at the moment because it's easier
to implement it this way, but it doesn't have to be. (The `Ref` struct we use in
Rust is actually based on the kernel's `refcount_t`.)

> > - On `remove`, the registrations with other subsystems are removed (so no
> >   additional sharing of internal data should happen), but existing calls and
> >   references to internal data structures continue to exist. This part is
> >   important: we don't "revoke" the references, but we do revoke the hw resources
> >   part of the internal state.
> 
> No issue here either. The handling of the internal data structure (the
> "non-revoke" part to be precise) matches my proposal too I believe.
> Revoking the I/O memory is of course rust-specific.
> 
> > - Attempts to access hardware resources freed during `remove` *must* be
> >   prevented, that's where the calls to `resources()` are relevant -- if a
> >   subsystem calls into the driver with one of the references it held on to, they
> >   won't be able to access the (already released) hw resources.
> 
> That's where our opinions differ. Yes, those accesses must be prevented,
> but I don't think the right way to do so is to check if the I/O memory
> resource is still valid. We should instead prevent reaching driver code
> paths that make those I/O accesses, by waiting for all calls in progress
> to return, and preventing new calls from being made. This is a more
> generic solution in the sense that it doesn't prevent accessing I/O
> memory only, but avoids any operation that is not supposed to take
> place.

I like the idea of blocking functions.

What lead me to a resource-based approach was the following: we have to block
.remove() until ongoing calls complete; how do we do that? Let's take the cdev
example, if we take your approach, we may have to wait arbitrarily long for
say read() to complete because drivers can sleep and not implement cancellation
properly. What happens then? .remove() is stuck.

With a resource-approach, .remove() needs to wait on RCU only, so there are no
arbitrarily long waits. Bugs in drivers where they take too long in their calls
won't affect .remove().

> My reasoning is that drivers will be written with the assumption
> that, for instance, nobody will try to set the GPIO direction once
> .remove() returns. Even if the direction_output() function correctly
> checks if the I/O memory is available and returns an error if it isn't,
> it may also contain other logic that will not work correctly after
> .remove() as the developer will not have considered that case.

I agree the extra error paths are a disadvantage of what I implemented.

> This
> uncorrect logic may or may not lead to bugs, and some categories of bugs
> may be prevented by rust (such as accessing I/O memory after .remove()),
> but I don't think that's relevant. The subsystem, with minimal help from
> the driver's implementation of the .remove() function if necessary,
> should prevent operations from being called when they shouldn't, and
> especially when the driver's author will not expect them to be called.
> That way we'll address whole classes of issues in one go. And if we do
> so, checking if I/O memory access has been revoked isn't required
> anymore, as we guarantee if isn't.
> 
> True, this won't prevent I/O memory from being accessed after .remove()
> in other contexts, for instance in a timer handler that the driver would
> have registered and forgotten to cancel in .remove(). And maybe the I/O
> memory revoking mechanism runtime overhead may be a reasonable price to
> pay for avoiding this, I don't know. I however believe that regardless
> of whether I/O memory is revoked or not, implementing a mechanism in the
> subsytem to avoid erroneous conditions from happening in the first place
> is where we'll get the largest benefit with a (hopefully) reasonable
> effort.
> 
> > We have this problem specifically in gpio: as Linus explained, they created an
> > indirection via a pointer which is checked in most entry points, but there is no
> > synchronisation that guarantees that the pointer will remain valid during a
> > call, and nothing forces uses of the pointer to be checked (so as Linus points
> > out, they may need more checks).
> > 
> > For Rust drivers, if the registration with other subsystems were done by doing
> > references to driver data in Rust, this extra "protection" (that has race
> > conditions that, timed correctly, lead to use-after-free vulnerabilities) would
> > be obviated; all would be handled safely on the Rust side (e.g., all accesses
> > must be checked, there is no way to get to resources without a check, and use of
> > the resources is guarded by a guard that uses RCU read-side lock).
> > 
> > Do you still think we don't have a problem?
> 
> We do have a problem, we just try to address it in different ways. And
> of course mine is better, and I don't expect you to agree with this
> statement right away ;-) Jokes aside, this has little to do with C vs.
> rust in this case though, it's about how to model APIs between drivers
> and subsystems.

I'm glad we agree we have a problem, that's the first step :)

I also agree that the problem exists regardless of the language. I do think that
the C side of the solution relies more on developer discipline (explicitly
checking for certain conditions, remembering to inc/dec counts, etc.) whereas
the compiler helps enforce such disciplines on the Rust side. IOW, it's just a
tool to catch possible violations at compile time.

Cheers,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Wedson,

On Wed, Jul 21, 2021 at 05:23:43AM +0100, Wedson Almeida Filho wrote:
> On Wed, Jul 21, 2021 at 02:54:24AM +0300, Laurent Pinchart wrote:
> > On Mon, Jul 19, 2021 at 10:09:46PM +0100, Wedson Almeida Filho wrote:
> > > On Mon, Jul 19, 2021 at 10:37:52PM +0300, Laurent Pinchart wrote:
> > > > On Mon, Jul 19, 2021 at 07:06:36PM +0100, Wedson Almeida Filho wrote:
> > > > > On Mon, Jul 19, 2021 at 06:02:06PM +0200, Vegard Nossum wrote:
> > > > > > On 7/19/21 3:15 PM, Wedson Almeida Filho wrote:
> > > > > > > On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > > > > > >> On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > > > > >>> I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > > > > >>> PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > > > > >>> Note that this hardware apart from being used in all Arm reference
> > > > > > >>> designs is used on ARMv4T systems that are not supported by
> > > > > > >>> LLVM but only GCC, which might complicate things.
> > > > > > >>
> > > > > > >> Here is a working PL061 driver in Rust (converted form the C one):
> > > > > > >> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> > > > > > > 
> > > > > > > I'm also attaching an html rending of the C and Rust versions side by side where
> > > > > > > I try to line the definitions up to make it easier to contrast the two
> > > > > > > implementations.
> > > > > > 
> > > > > > This is really cool :-) As a Rust noob, I have a few questions:
> > > > > > 
> > > > > > 1. I'm curious about some of the writeb() vs. try_writeb() calls:
> > > > > > 
> > > > > > fn direction_output(data: &Ref<DeviceData>, offset: u32, value: bool) ->
> > > > > > Result {
> > > > > >         let woffset = bit(offset + 2).into();
> > > > > >         let _guard = data.lock();
> > > > > >         let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > > >         let mut gpiodir = pl061.base.readb(GPIODIR);
> > > > > >         gpiodir |= bit(offset);
> > > > > >         pl061.base.writeb(gpiodir, GPIODIR);
> > > > > > 
> > > > > >         // gpio value is set again, because pl061 doesn't allow to set
> > > > > > value of a gpio pin before
> > > > > >         // configuring it in OUT mode.
> > > > > >         pl061.base.try_writeb((value as u8) << offset, woffset)?;
> > > > > >         Ok(())
> > > > > >     }
> > > > > > 
> > > > > > Here you have try_writeb() (and error return) where there was just a
> > > > > > writeb() without any error handling in the C version. Is this what
> > > > > > Miguel was answering a bit down the thread where the address is computed
> > > > > > ((value as u8) << offset) so it _needs_ to use the try_() version?
> > > > > 
> > > > > The `writeb` variant only works when we know at compile-time that the offset is
> > > > > within bounds (the compiler will reject the code otherwise). When the value is
> > > > > computed at runtime we use a `try` version that checks before performing the
> > > > > write. We need this to guarantee memory safety.
> > > > > 
> > > > > > If offset can be anything but a "correct" value here, should there be a
> > > > > > check for that somewhere else and then the computed value can be
> > > > > > subsequently treated as safe (i.e. there's a second try_writeb() in the
> > > > > > function that now presumably does the runtime check a second time,
> > > > > > redundantly)?
> > > > > 
> > > > > Oh, that's a neat idea. We can certainly implement something like this:
> > > > > 
> > > > > let woffset = pl061.base.vet_offsetb(bit(offset + 2))?;
> > > > > 
> > > > > Then woffset would be passed to writeb variants that are guaranteed to succeed.
> > > > > (Rust helps us ensure that woffset cannot change without checks, which would be
> > > > > harder to do in C.)
> > > > > 
> > > > > > 2. In many places you have the C code:
> > > > > > 
> > > > > > struct pl061 *pl061 = dev_get_drvdata(dev);
> > > > > > 
> > > > > > with the equivalent Rust code as:
> > > > > > 
> > > > > > let pl061 = data.resources().ok_or(Error::ENXIO)?;
> > > > > > 
> > > > > > Why doesn't the C code need to check for errors here? Or put
> > > > > > differently, why can the Rust version fail?
> > > > > 
> > > > > There are two aspecs worth noting here:
> > > > > 1. In C there is cast from void * to struct pl061 * without really knowing if
> > > > > the stored pointer is of the right type. For example, if I simply change the
> > > > > struct type to say `struct mutex` in the code above, it will still compile,
> > > > > though it will be clearly wrong. In Rust we prevent this by not exposing drvdata
> > > > > directly to drivers, and using type-specialised functions to set/get drvdata, so
> > > > > it *knows* that the type is right. So in this sense Rust is better because it
> > > > > offers type guarantees without additional runtime cost. (In Rust, if you change
> > > > > the type of the function to say `&Mutex`, it won't compile.
> > > > > 
> > > > > 2. The extra check we have here is because of a feature that the C code doesn't
> > > > > have: revocable resources. If we didn't want to have this, we could do say
> > > > > `data.base.writeb(...)` directly, but then we could have situations where `base`
> > > > > is used after the device was removed. By having these checks we guarantee that
> > > > > anyone can hold a reference to device state, but they can no longer use hw
> > > > > resources after the device is removed.
> > > > 
> > > > If the driver reached a code path with an I/O write after .remove()
> > > > returns, the game is likely over already. It would be more interesting
> > > > to see how we could prevent that from happening in the first place.
> > > > Checking individual I/O writes at runtime will not only add additional
> > > > CPU costs, but will also produce code paths that are not well tested.
> > > 
> > > You may be conflating checking offsets in individual writes/reads with accessing
> > > hw resources. Note that these are different things.
> > 
> > Yes, it's the data.resources().ok_or() that I was talking about, not the
> > I/O writes, sorry.
> > 
> > > > It
> > > > feels that we're inventing a problem just to be able to showcase the
> > > > solution :-)
> > > 
> > > Thanks for taking a look. I beg to differ though, as this solves (on the Rust
> > > side) a problem you described the other day on this very thread. The solution is
> > > different from what you propose though :)
> > > 
> > > - The internal data structures of drivers are refcounted. Drivers then share
> > >   this internal representation with other subsystems (e.g., cdev).
> > 
> > Refcounting the driver-specific structure is good, that matches what I
> > proposed (it's of course implemented differently in C and rust, but
> > that's expected).
> 
> The refcounting business is indeed different at the moment because it's easier
> to implement it this way, but it doesn't have to be. (The `Ref` struct we use in
> Rust is actually based on the kernel's `refcount_t`.)
> 
> > > - On `remove`, the registrations with other subsystems are removed (so no
> > >   additional sharing of internal data should happen), but existing calls and
> > >   references to internal data structures continue to exist. This part is
> > >   important: we don't "revoke" the references, but we do revoke the hw resources
> > >   part of the internal state.
> > 
> > No issue here either. The handling of the internal data structure (the
> > "non-revoke" part to be precise) matches my proposal too I believe.
> > Revoking the I/O memory is of course rust-specific.
> > 
> > > - Attempts to access hardware resources freed during `remove` *must* be
> > >   prevented, that's where the calls to `resources()` are relevant -- if a
> > >   subsystem calls into the driver with one of the references it held on to, they
> > >   won't be able to access the (already released) hw resources.
> > 
> > That's where our opinions differ. Yes, those accesses must be prevented,
> > but I don't think the right way to do so is to check if the I/O memory
> > resource is still valid. We should instead prevent reaching driver code
> > paths that make those I/O accesses, by waiting for all calls in progress
> > to return, and preventing new calls from being made. This is a more
> > generic solution in the sense that it doesn't prevent accessing I/O
> > memory only, but avoids any operation that is not supposed to take
> > place.
> 
> I like the idea of blocking functions.
> 
> What lead me to a resource-based approach was the following: we have to block
> .remove() until ongoing calls complete; how do we do that? Let's take the cdev
> example, if we take your approach, we may have to wait arbitrarily long for
> say read() to complete because drivers can sleep and not implement cancellation
> properly. What happens then? .remove() is stuck.

Correct.

> With a resource-approach, .remove() needs to wait on RCU only, so there are no
> arbitrarily long waits. Bugs in drivers where they take too long in their calls
> won't affect .remove().

If an operation is still in progress and you complete .remove(), all
bets are off regarding what can happen next. This is a bug in the
driver, it needs to ensure that all operations will complete in a
reasonable amount of time. That reasonable amount of time may be long,
maybe the hardware is performing DMA and can't be stopped until it
completes, and maybe the DMA will take a few seconds to complete. That
would arguably be a badly designed piece of hardware, but it can happen,
and we need to ensure that the DMA completes before we shut down the
device in that case.

> > My reasoning is that drivers will be written with the assumption
> > that, for instance, nobody will try to set the GPIO direction once
> > .remove() returns. Even if the direction_output() function correctly
> > checks if the I/O memory is available and returns an error if it isn't,
> > it may also contain other logic that will not work correctly after
> > .remove() as the developer will not have considered that case.
> 
> I agree the extra error paths are a disadvantage of what I implemented.
> 
> > This
> > uncorrect logic may or may not lead to bugs, and some categories of bugs
> > may be prevented by rust (such as accessing I/O memory after .remove()),
> > but I don't think that's relevant. The subsystem, with minimal help from
> > the driver's implementation of the .remove() function if necessary,
> > should prevent operations from being called when they shouldn't, and
> > especially when the driver's author will not expect them to be called.
> > That way we'll address whole classes of issues in one go. And if we do
> > so, checking if I/O memory access has been revoked isn't required
> > anymore, as we guarantee if isn't.
> > 
> > True, this won't prevent I/O memory from being accessed after .remove()
> > in other contexts, for instance in a timer handler that the driver would
> > have registered and forgotten to cancel in .remove(). And maybe the I/O
> > memory revoking mechanism runtime overhead may be a reasonable price to
> > pay for avoiding this, I don't know. I however believe that regardless
> > of whether I/O memory is revoked or not, implementing a mechanism in the
> > subsytem to avoid erroneous conditions from happening in the first place
> > is where we'll get the largest benefit with a (hopefully) reasonable
> > effort.
> > 
> > > We have this problem specifically in gpio: as Linus explained, they created an
> > > indirection via a pointer which is checked in most entry points, but there is no
> > > synchronisation that guarantees that the pointer will remain valid during a
> > > call, and nothing forces uses of the pointer to be checked (so as Linus points
> > > out, they may need more checks).
> > > 
> > > For Rust drivers, if the registration with other subsystems were done by doing
> > > references to driver data in Rust, this extra "protection" (that has race
> > > conditions that, timed correctly, lead to use-after-free vulnerabilities) would
> > > be obviated; all would be handled safely on the Rust side (e.g., all accesses
> > > must be checked, there is no way to get to resources without a check, and use of
> > > the resources is guarded by a guard that uses RCU read-side lock).
> > > 
> > > Do you still think we don't have a problem?
> > 
> > We do have a problem, we just try to address it in different ways. And
> > of course mine is better, and I don't expect you to agree with this
> > statement right away ;-) Jokes aside, this has little to do with C vs.
> > rust in this case though, it's about how to model APIs between drivers
> > and subsystems.
> 
> I'm glad we agree we have a problem, that's the first step :)
> 
> I also agree that the problem exists regardless of the language. I do think that
> the C side of the solution relies more on developer discipline (explicitly
> checking for certain conditions, remembering to inc/dec counts, etc.) whereas
> the compiler helps enforce such disciplines on the Rust side. IOW, it's just a
> tool to catch possible violations at compile time.

Sure, rust clearly brings more compile-time checks, nobody can claim
otherwise :-) C will always rely more on the developer getting it right.
I however believe that we can lower the chance of developers getting it
wrong by improving the APIs between subsystems and drivers, regardless
of the language. That's applicable to rust too, as the compiler won't
catch all possible errors (a programming language where a developer
wouldn't be able to get anything wrong would be an interesting concept
:-)), and good API design will always help.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Alexandre Belloni]

Hello,

On 19/07/2021 14:15:57+0100, Wedson Almeida Filho wrote:
> On Mon, Jul 19, 2021 at 01:24:49PM +0100, Wedson Almeida Filho wrote:
> > On Fri, Jul 09, 2021 at 12:13:25AM +0200, Linus Walleij wrote:
> > > I have seen that QEMU has a piece of code for the Arm PrimeCell
> > > PL061 GPIO block which corresponds to drivers/gpio/gpio-pl061.c
> > > Note that this hardware apart from being used in all Arm reference
> > > designs is used on ARMv4T systems that are not supported by
> > > LLVM but only GCC, which might complicate things.
> > 
> > Here is a working PL061 driver in Rust (converted form the C one):
> > https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> 
> I'm also attaching an html rending of the C and Rust versions side by side where
> I try to line the definitions up to make it easier to contrast the two
> implementations.

I'd love to have a side by side disassembly of the generated object
files (ideally intermixed with the source).


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

Hi Alexandre,

On Tue, Jul 20, 2021 at 12:57 AM Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> I'd love to have a side by side disassembly of the generated object
> files (ideally intermixed with the source).

It is hard to do such a match because of different reasons, such as:

  - The extra work that the Rust version does (the revocable resources feature).

  - The Rust version not being able to inline the opaque helpers that
we use to avoid rewriting C macros (without cross-language LTO
support).

  - The Rust version inlining generics from the `kernel` crate (which
happens even without LTO, because they are generics).

  - The C version not being able to inline from other translation
units (without LTO).

In any case, to give you an example, I took `get_direction()` which is
fairly simple and under `-O2` with overflow checks disabled in Rust I
got:

    00000000000001b8 <pl061_get_direction>:
        1b8:    a9be7bfd    stp    x29, x30, [sp, #-32]!
        1bc:    f9000bf3    str    x19, [sp, #16]
        1c0:    910003fd    mov    x29, sp
        1c4:    2a0103f3    mov    w19, w1
        1c8:    94000000    bl     0 <gpiochip_get_data>
        1cc:    f9400408    ldr    x8, [x0, #8]
        1d0:    91100108    add    x8, x8, #0x400
        1d4:    39400108    ldrb   w8, [x8]
        1d8:    d50331bf    dmb    oshld
        1dc:    92401d08    and    x8, x8, #0xff
        1e0:    ca080109    eor    x9, x8, x8
        1e4:    b5000009    cbnz   x9, 1e4 <pl061_get_direction+0x2c>
        1e8:    9ad32508    lsr    x8, x8, x19
        1ec:    f9400bf3    ldr    x19, [sp, #16]
        1f0:    f240011f    tst    x8, #0x1
        1f4:    1a9f17e0    cset   w0, eq // eq = none
        1f8:    a8c27bfd    ldp    x29, x30, [sp], #32
        1fc:    d65f03c0    ret

    00000000000009d8 <...::get_direction>:
        9d8:    a9bd7bfd    stp     x29, x30, [sp, #-48]!
        9dc:    f9000bf5    str     x21, [sp, #16]
        9e0:    a9024ff4    stp     x20, x19, [sp, #32]
        9e4:    910003fd    mov     x29, sp
        9e8:    f9400014    ldr     x20, [x0]
        9ec:    2a0103f3    mov     w19, w1
        9f0:    94000000    bl      0 <rust_helper_rcu_read_lock>
        9f4:    39402288    ldrb    w8, [x20, #8]
        9f8:    72001d1f    tst     w8, #0xff
        9fc:    54000180    b.eq    a2c <...::get_direction+0x54> // b.none
        a00:    f9419288    ldr     x8, [x20, #800]
        a04:    91100100    add     x0, x8, #0x400
        a08:    94000000    bl      0 <rust_helper_readb>
        a0c:    92400a68    and     x8, x19, #0x7
        a10:    1ac82408    lsr     w8, w0, w8
        a14:    7200011f    tst     w8, #0x1
        a18:    1a9f17e8    cset    w8, eq // eq = none
        a1c:    aa1f03f4    mov     x20, xzr
        a20:    aa1f03f5    mov     x21, xzr
        a24:    d378dd13    lsl     x19, x8, #8
        a28:    14000005    b       a3c <...::get_direction+0x64>
        a2c:    d2dfff54    mov     x20, #0xfffa00000000 // #281449206906880
        a30:    aa1f03f3    mov     x19, xzr
        a34:    f2fffff4    movk    x20, #0xffff, lsl #48
        a38:    52800035    mov     w21, #0x1 // #1
        a3c:    94000000    bl      0 <rust_helper_rcu_read_unlock>
        a40:    aa140268    orr     x8, x19, x20
        a44:    aa150100    orr     x0, x8, x21
        a48:    a9424ff4    ldp     x20, x19, [sp, #32]
        a4c:    f9400bf5    ldr     x21, [sp, #16]
        a50:    a8c37bfd    ldp     x29, x30, [sp], #48
        a54:    d65f03c0    ret

To be a bit more fair to the Rust version by not having the revocable
part, I wrote a `test` function in both C and Rust with only the read
that `get_direction()` does:

    u8 test(struct pl061 *pl061, unsigned offset) {
            return readb(pl061->base + GPIODIR) & BIT(offset);
    }

    fn test(pl061: &PL061Resources, offset: u32) -> u8 {
        pl061.base.readb(GPIODIR) & bit(offset)
    }

This, of course, yields a much closer result:

    0000000000000000 <test>:
          0:    f9400408     ldr     x8, [x0, #8]
          4:    91100108     add     x8, x8, #0x400
          8:    39400108     ldrb    w8, [x8]
          c:    d50331bf     dmb     oshld
         10:    92401d09     and     x9, x8, #0xff
         14:    ca090129     eor     x9, x9, x9
         18:    b5000009     cbnz    x9, 18 <test+0x18>
         1c:    52800029     mov     w9, #0x1                    // #1
         20:    9ac12129     lsl     x9, x9, x1
         24:    0a090100     and     w0, w8, w9
         28:    d65f03c0     ret

    00000000000009d0 <gpio_pl061_rust::test>:
        9d0:    a9be7bfd     stp     x29, x30, [sp, #-32]!
        9d4:    f9000bf3     str     x19, [sp, #16]
        9d8:    910003fd     mov     x29, sp
        9dc:    f9400008     ldr     x8, [x0]
        9e0:    2a0103f3     mov     w19, w1
        9e4:    91100100     add     x0, x8, #0x400
        9e8:    94000000     bl      0 <rust_helper_readb>
        9ec:    92400a68     and     x8, x19, #0x7
        9f0:    f9400bf3     ldr     x19, [sp, #16]
        9f4:    52800029     mov     w9, #0x1                    // #1
        9f8:    1ac82128     lsl     w8, w9, w8
        9fc:    0a080000     and     w0, w0, w8
        a00:    a8c27bfd     ldp     x29, x30, [sp], #32
        a04:    d65f03c0     ret

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Alexandre Belloni]

Hi,

On 20/07/2021 09:15:44+0200, Miguel Ojeda wrote:
> Hi Alexandre,
> 
> On Tue, Jul 20, 2021 at 12:57 AM Alexandre Belloni
> <alexandre.belloni@bootlin.com> wrote:
> >
> > I'd love to have a side by side disassembly of the generated object
> > files (ideally intermixed with the source).
> 
> It is hard to do such a match because of different reasons, such as:
> 
>   - The extra work that the Rust version does (the revocable resources feature).
> 

Well, the point is exactly to have a look at that extra work.

>   - The Rust version not being able to inline the opaque helpers that
> we use to avoid rewriting C macros (without cross-language LTO
> support).
> 
>   - The Rust version inlining generics from the `kernel` crate (which
> happens even without LTO, because they are generics).
> 
>   - The C version not being able to inline from other translation
> units (without LTO).
> 
> In any case, to give you an example, I took `get_direction()` which is
> fairly simple and under `-O2` with overflow checks disabled in Rust I
> got:
> 

I was under the impression that you would compile the kernel with
overflow checks enabled, why would you disable them here?

>     00000000000001b8 <pl061_get_direction>:
>         1b8:    a9be7bfd    stp    x29, x30, [sp, #-32]!
>         1bc:    f9000bf3    str    x19, [sp, #16]
>         1c0:    910003fd    mov    x29, sp
>         1c4:    2a0103f3    mov    w19, w1
>         1c8:    94000000    bl     0 <gpiochip_get_data>
>         1cc:    f9400408    ldr    x8, [x0, #8]
>         1d0:    91100108    add    x8, x8, #0x400
>         1d4:    39400108    ldrb   w8, [x8]
>         1d8:    d50331bf    dmb    oshld
>         1dc:    92401d08    and    x8, x8, #0xff
>         1e0:    ca080109    eor    x9, x8, x8
>         1e4:    b5000009    cbnz   x9, 1e4 <pl061_get_direction+0x2c>
>         1e8:    9ad32508    lsr    x8, x8, x19
>         1ec:    f9400bf3    ldr    x19, [sp, #16]
>         1f0:    f240011f    tst    x8, #0x1
>         1f4:    1a9f17e0    cset   w0, eq // eq = none
>         1f8:    a8c27bfd    ldp    x29, x30, [sp], #32
>         1fc:    d65f03c0    ret
> 
>     00000000000009d8 <...::get_direction>:
>         9d8:    a9bd7bfd    stp     x29, x30, [sp, #-48]!
>         9dc:    f9000bf5    str     x21, [sp, #16]
>         9e0:    a9024ff4    stp     x20, x19, [sp, #32]
>         9e4:    910003fd    mov     x29, sp
>         9e8:    f9400014    ldr     x20, [x0]
>         9ec:    2a0103f3    mov     w19, w1
>         9f0:    94000000    bl      0 <rust_helper_rcu_read_lock>
>         9f4:    39402288    ldrb    w8, [x20, #8]
>         9f8:    72001d1f    tst     w8, #0xff
>         9fc:    54000180    b.eq    a2c <...::get_direction+0x54> // b.none
>         a00:    f9419288    ldr     x8, [x20, #800]
>         a04:    91100100    add     x0, x8, #0x400
>         a08:    94000000    bl      0 <rust_helper_readb>
>         a0c:    92400a68    and     x8, x19, #0x7
>         a10:    1ac82408    lsr     w8, w0, w8
>         a14:    7200011f    tst     w8, #0x1
>         a18:    1a9f17e8    cset    w8, eq // eq = none
>         a1c:    aa1f03f4    mov     x20, xzr
>         a20:    aa1f03f5    mov     x21, xzr
>         a24:    d378dd13    lsl     x19, x8, #8
>         a28:    14000005    b       a3c <...::get_direction+0x64>
>         a2c:    d2dfff54    mov     x20, #0xfffa00000000 // #281449206906880
>         a30:    aa1f03f3    mov     x19, xzr
>         a34:    f2fffff4    movk    x20, #0xffff, lsl #48
>         a38:    52800035    mov     w21, #0x1 // #1
>         a3c:    94000000    bl      0 <rust_helper_rcu_read_unlock>
>         a40:    aa140268    orr     x8, x19, x20
>         a44:    aa150100    orr     x0, x8, x21
>         a48:    a9424ff4    ldp     x20, x19, [sp, #32]
>         a4c:    f9400bf5    ldr     x21, [sp, #16]
>         a50:    a8c37bfd    ldp     x29, x30, [sp], #48
>         a54:    d65f03c0    ret
> 
> To be a bit more fair to the Rust version by not having the revocable
> part, I wrote a `test` function in both C and Rust with only the read
> that `get_direction()` does:

But do we care about very simple test cases? The pl061 driver is already
simple and this would be an example of what to expect for most of the
simple drivers that would get converted.

> 
>     u8 test(struct pl061 *pl061, unsigned offset) {
>             return readb(pl061->base + GPIODIR) & BIT(offset);
>     }
> 
>     fn test(pl061: &PL061Resources, offset: u32) -> u8 {
>         pl061.base.readb(GPIODIR) & bit(offset)
>     }
> 
> This, of course, yields a much closer result:
> 
>     0000000000000000 <test>:
>           0:    f9400408     ldr     x8, [x0, #8]
>           4:    91100108     add     x8, x8, #0x400
>           8:    39400108     ldrb    w8, [x8]
>           c:    d50331bf     dmb     oshld
>          10:    92401d09     and     x9, x8, #0xff
>          14:    ca090129     eor     x9, x9, x9
>          18:    b5000009     cbnz    x9, 18 <test+0x18>
>          1c:    52800029     mov     w9, #0x1                    // #1
>          20:    9ac12129     lsl     x9, x9, x1
>          24:    0a090100     and     w0, w8, w9
>          28:    d65f03c0     ret
> 
>     00000000000009d0 <gpio_pl061_rust::test>:
>         9d0:    a9be7bfd     stp     x29, x30, [sp, #-32]!
>         9d4:    f9000bf3     str     x19, [sp, #16]
>         9d8:    910003fd     mov     x29, sp
>         9dc:    f9400008     ldr     x8, [x0]
>         9e0:    2a0103f3     mov     w19, w1
>         9e4:    91100100     add     x0, x8, #0x400
>         9e8:    94000000     bl      0 <rust_helper_readb>

This is a function call, I would not call that closer to the C
version. Can we see rust_helper_readb?

>         9ec:    92400a68     and     x8, x19, #0x7
>         9f0:    f9400bf3     ldr     x19, [sp, #16]
>         9f4:    52800029     mov     w9, #0x1                    // #1
>         9f8:    1ac82128     lsl     w8, w9, w8
>         9fc:    0a080000     and     w0, w0, w8
>         a00:    a8c27bfd     ldp     x29, x30, [sp], #32
>         a04:    d65f03c0     ret

-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 20, 2021 at 11:39 AM Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> Well, the point is exactly to have a look at that extra work.

That is fine, but you did not say so :) I interpreted you wanted to
compare the codegen quality, rather than take a look at the extra
features.

> I was under the impression that you would compile the kernel with
> overflow checks enabled, why would you disable them here?

Please see the previous point: it is not that useful to compare object
files that do different things (if checking codegen quality), thus I
removed some of the differences, including the overflow checks.

The extra work can be done in both C and Rust (or in neither), so it
is not related to the language choice.

> But do we care about very simple test cases? The pl061 driver is already
> simple and this would be an example of what to expect for most of the
> simple drivers that would get converted.

Ditto.

> This is a function call, I would not call that closer to the C

I explained this in the beginning of the message: `bindgen` cannot
convert C macros, thus we use a helper function, and since
cross-language LTO is not yet supported, LLVM cannot inline the call.

Of course, we could do inline assembly in the Rust side too, but I
would say it is best to work on cross-language LTO and see if that is
good enough before duplicating code.

We could also do it here for demonstration purposes, if you want.

> version. Can we see rust_helper_readb?

Of course:

    00000000000004d8 <rust_helper_readb>:
        4d8:    39400000    ldrb    w0, [x0]
        4dc:    d50331bf    dmb     oshld
        4e0:    92401c08    and     x8, x0, #0xff
        4e4:    ca080108    eor     x8, x8, x8
        4e8:    b5000008    cbnz    x8, 4e8 <rust_helper_readb+0x10>
        4ec:    d65f03c0    ret

As you can see, it is just the inline assembly generated by the arm64
arch macros.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Mon, Jul 19, 2021 at 2:24 PM Wedson Almeida Filho
<wedsonaf@google.com> wrote:

> Here is a working PL061 driver in Rust (converted form the C one):
> https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs

My first impression is that this looks very good compared to other Rust
code I have seen, and it looks very intuitive and maintainable.

use kernel::{
    amba, bit, declare_id_table, device, gpio,
    io_mem::IoMem,
    irq::{self, IrqData, LockedIrqData},
    power,
    prelude::*,
    sync::{IrqDisableSpinLock, Ref},
};

I think people want to see the code inside these classes as well,
can we browse it? Especially the gpio class in my case,
because i want to know what happens when you do
impl gpio::Chip for PL061Device { ... } I suppose it is just
a C wrapper to the struct gpio_chip?

I've got questions but they are mostly due to my lack of understanding
of Rust idioms I think. For example:

    fn unmask(data: &Ref<DeviceData>, irq_data: &IrqData) {
        let mask = bit(irq_data.hwirq() % u64::from(PL061_GPIO_NR));
        let _guard = data.lock();
        if let Some(pl061) = data.resources() {
            let gpioie = pl061.base.readb(GPIOIE) | mask;
            let _ = pl061.base.try_writeb(gpioie, GPIOIE);
        }
    }

I suppose _guard will be unlocked automatically when we leave
the function? It's a bit like with Make, it is necessary to know
what is implicit.

Then the "_" variable is some kind of idiom for a throw-away
variable I guess. Can it be named "foo" or is the "_" just specially
magic?

It looks like stuff we can even maintain superficially without
being super versed in Rust. Simple things can be altered
without knowing exactly how the language works.

> (I tested iton QEMU through the sysfs interface and also gpio-keys as QEMU
> uses one of the PL061 pins as the power button.)

Fair enough.

> 1. State created on `probe` is ref-counted.
> 2. Hardware resources (device mem and irq in this case) are "revocable".
> 3. On `remove`, we automatically revoke access to hardware resources, then free
> them.

This is neat. Just generally speaking I like the looks of this
from a driver subsystem point of view. Unless the code in the
Kernel::classes is horrible that is.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Mon, Jul 19, 2021 at 03:53:31PM +0200, Linus Walleij wrote:
> On Mon, Jul 19, 2021 at 2:24 PM Wedson Almeida Filho
> <wedsonaf@google.com> wrote:
> 
> > Here is a working PL061 driver in Rust (converted form the C one):
> > https://raw.githubusercontent.com/wedsonaf/linux/pl061/drivers/gpio/gpio_pl061_rust.rs
> 
> My first impression is that this looks very good compared to other Rust
> code I have seen, and it looks very intuitive and maintainable.

Thanks for taking to time to check it out.

> use kernel::{
>     amba, bit, declare_id_table, device, gpio,
>     io_mem::IoMem,
>     irq::{self, IrqData, LockedIrqData},
>     power,
>     prelude::*,
>     sync::{IrqDisableSpinLock, Ref},
> };
> 
> I think people want to see the code inside these classes as well,
> can we browse it? 

The whole tree is available here: https://github.com/wedsonaf/linux/tree/pl061
-- I should caution you that the two most recent commits are not merged into the
rust tree yet because they're WIP. (I'll clean them up and eventually merge
after the feedback from you all.)

> Especially the gpio class in my case,
> because i want to know what happens when you do
> impl gpio::Chip for PL061Device { ... } I suppose it is just
> a C wrapper to the struct gpio_chip?

`impl gpio::Chip for PL061Device` is just saying that the functions within that
block conform to the ones in `gpio::Chip`. The C wrapper glue is really in
`gpio::Registration::register_with_irq`, called in `probe`: it calls the C
functions and registers "trampolines" that C code can call that routes those to
type-safe Rust code.

> I've got questions but they are mostly due to my lack of understanding
> of Rust idioms I think. For example:
> 
>     fn unmask(data: &Ref<DeviceData>, irq_data: &IrqData) {
>         let mask = bit(irq_data.hwirq() % u64::from(PL061_GPIO_NR));
>         let _guard = data.lock();
>         if let Some(pl061) = data.resources() {
>             let gpioie = pl061.base.readb(GPIOIE) | mask;
>             let _ = pl061.base.try_writeb(gpioie, GPIOIE);
>         }
>     }
> 
> I suppose _guard will be unlocked automatically when we leave
> the function? It's a bit like with Make, it is necessary to know
> what is implicit.

Correct, `guard` will unlock the spinlock automatically when it goes out of
scope. Also note that `pl061` is also a guard, it will unlock RCU read side when
it goes out of scope.

> Then the "_" variable is some kind of idiom for a throw-away
> variable I guess. Can it be named "foo" or is the "_" just specially
> magic?

You can choose any name, however the compiler will issue a warning for an
unused variable. Prefixing it with an underscore silences the warning.

Ordinarily this isn't a problem because you'd access some variable explicitly 
protected by the lock, which isn't the case here.

> It looks like stuff we can even maintain superficially without
> being super versed in Rust. Simple things can be altered
> without knowing exactly how the language works.
> 
> > (I tested iton QEMU through the sysfs interface and also gpio-keys as QEMU
> > uses one of the PL061 pins as the power button.)
> 
> Fair enough.
> 
> > 1. State created on `probe` is ref-counted.
> > 2. Hardware resources (device mem and irq in this case) are "revocable".
> > 3. On `remove`, we automatically revoke access to hardware resources, then free
> > them.
> 
> This is neat. Just generally speaking I like the looks of this
> from a driver subsystem point of view. Unless the code in the
> Kernel::classes is horrible that is.

Once you've had a chance to browse through the code, let me know if there are
concerns.

Cheers,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Mon, Jul 19, 2021 at 4:42 PM Wedson Almeida Filho
<wedsonaf@google.com> wrote:

> > I think people want to see the code inside these classes as well,
> > can we browse it?
>
> The whole tree is available here: https://github.com/wedsonaf/linux/tree/pl061
> -- I should caution you that the two most recent commits are not merged into the
> rust tree yet because they're WIP. (I'll clean them up and eventually merge
> after the feedback from you all.)

I found that the gpio_chip C wrappers lives here:
https://github.com/wedsonaf/linux/blob/pl061/rust/kernel/gpio.rs

This file is very interesting!

I think it's not a good idea to keep these wrappers in their own
rust directory, they need to be distributed out into the kernel
everywhere they are used. We have made this mistake before
with OF (device tree) that started exactly like this in
drivers/of/*, and now I have part of the OF GPIO handling
and tests inside files in that directory as a consequence.

The wrappers are a hard read, I notice a whole lot of unsafe
keywords here, Chip is missing .set_config() which is understandable
since PL061 isn't using it, the New call to create an instance
contains a lot of magic stuff I don't understand like
state: AtomicU8::new(0) and such but I think that's all right
it probably only needs a trained eye.

This binding only concerns the gpio_chip abstraction though,
which means that the struct gpio_device and the creation and
handling of the GPIO character device, all of the ioctl()
and passed in/out data from userspace is left in C in gpiolib.c

I think this is the current ambition of the Rust effort for now,
which is fine, one can not be judged for doing what one
attempts to do.

But we need to acknowledge that by leaving the userspace
facing code in C the attack surface for buffer overruns
etc is left in C and it buys us pretty much zero additional
security. And with security I do not mean the kernel
protecting itself from itself, but from things coming from
userspace.

It's not that I think GPIOs are an especially interesting
attack vector, but if this pattern persists then it becomes
a problem, because Rust isn't yet protecting us (GPIO)
against userspace, which is what it needs to do in the end.

What we get now is the ability to write drivers in Rust,
but no protection against a hostile userspace. Not really.

To that end the core of the GPIO library would probably
have to be rewritten in Rust and used on *all* platforms
in order to buy us any improved security. I understand that
this is not currently the plan.

This is what I have said before as well: Rust needs to go
where the attack surface of the kernel is to buy us any
real security. But getting there will require rewriting entire
subsystems in Rust AFAICT.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Tue, Jul 20, 2021 at 12:16:32AM +0200, Linus Walleij wrote:
> On Mon, Jul 19, 2021 at 4:42 PM Wedson Almeida Filho
> <wedsonaf@google.com> wrote:
> 
> > > I think people want to see the code inside these classes as well,
> > > can we browse it?
> >
> > The whole tree is available here: https://github.com/wedsonaf/linux/tree/pl061
> > -- I should caution you that the two most recent commits are not merged into the
> > rust tree yet because they're WIP. (I'll clean them up and eventually merge
> > after the feedback from you all.)
> 
> I found that the gpio_chip C wrappers lives here:
> https://github.com/wedsonaf/linux/blob/pl061/rust/kernel/gpio.rs
> 
> This file is very interesting!
> 
> I think it's not a good idea to keep these wrappers in their own
> rust directory, they need to be distributed out into the kernel
> everywhere they are used. We have made this mistake before
> with OF (device tree) that started exactly like this in
> drivers/of/*, and now I have part of the OF GPIO handling
> and tests inside files in that directory as a consequence.

That's great feedback. Our plan was to have the core parts in `rust/kernel`, but
once maintainers got involved, we would place things where it made more sense.
Since we have no maintainers involved in development yet, everything is in rust/
for now.

> The wrappers are a hard read, I notice a whole lot of unsafe
> keywords here, Chip is missing .set_config() which is understandable
> since PL061 isn't using it,

Right, I only implemented what was needed for the PL061 driver. As I said
originally, this isn't intended to be a general solution just yet.

> the New call to create an instance
> contains a lot of magic stuff I don't understand like
> state: AtomicU8::new(0) and such but I think that's all right
> it probably only needs a trained eye.

That code is interacting with C, both being called by C and calling into C, so
it necessarily has a lot of unsafe blocks. The idea though is to limit the
unsafety to these wrappers and have them carefully reviewed; users will
[ideally] need no unsafe code.

> This binding only concerns the gpio_chip abstraction though,
> which means that the struct gpio_device and the creation and
> handling of the GPIO character device, all of the ioctl()
> and passed in/out data from userspace is left in C in gpiolib.c

Right, we don't want to reinvent things already in C while trying to demonstrate
a driver. The wrappers I wrote are needed for us to have a safe Rust API that
drivers can call -- if they weren't necessary I wouldn't have written them
either. At least not to begin with.

> I think this is the current ambition of the Rust effort for now,
> which is fine, one can not be judged for doing what one
> attempts to do.
> 
> But we need to acknowledge that by leaving the userspace
> facing code in C the attack surface for buffer overruns
> etc is left in C and it buys us pretty much zero additional
> security. And with security I do not mean the kernel
> protecting itself from itself, but from things coming from
> userspace.
>
> It's not that I think GPIOs are an especially interesting
> attack vector, but if this pattern persists then it becomes
> a problem, because Rust isn't yet protecting us (GPIO)
> against userspace, which is what it needs to do in the end.

It's not protecting all the way yet, but it is offering more guarantees than C:
for example, it guarantees that driver code won't touch memory that it doesn't
own. Suppose there's a bug in gpiolib that allows an attacker to control the
value of `offset` in get/set operations, a driver written in C will take it and
read/write to memory it doesn't own; a Rust driver won't.

Suppose a driver developer forgets to call chained_irq_enter/chained_irq_exit in
their IRQ handler. An attacker could time things and trigger interrupts and DoS
the system, but this can't happen in Rust.

Suppose a driver developer copies some code from pl061_irq_type and pastes it
somewhere where irq_desc is not locked, but calls irq_set_handler_locked anyway.
An attacker could potentially exploit the resulting races. This can't happen in
Rust.

There are other bugs that Rust prevents by construction that improves the
overall quality of the drivers, with the end result being a reduced attack
surface. Less opportunities for attackers to chain vulnerabilities to exploit
the kernel -- remember that exploits are usually built as chain of
vulnerabilities to elevate from simple innocuous-looking misbehaviours to
something really bad like arbitrary execution.

> What we get now is the ability to write drivers in Rust,
> but no protection against a hostile userspace. Not really.

I do agree that having Rust on the interface with userspace would likely offer
the best protection. That's what I was working on with binder -- but you (as a
community) wanted to see a driver for "real" hardware. So here it is :)

> To that end the core of the GPIO library would probably
> have to be rewritten in Rust and used on *all* platforms
> in order to buy us any improved security. I understand that
> this is not currently the plan.

It's not the short-term plan, but if maintainers are interested in converting
GPIO to Rust, I would be happy to help now that I've learned a bit about it.
It would, for one, fix the race conditions you have when devices are removed.

> This is what I have said before as well: Rust needs to go
> where the attack surface of the kernel is to buy us any
> real security. But getting there will require rewriting entire
> subsystems in Rust AFAICT.

I'm not sure I agree with this. Incremental improvements are a good thing.

Cheers,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Andrew Lunn]

On Tue, Jul 20, 2021 at 02:20:34AM +0100, Wedson Almeida Filho wrote:
> On Tue, Jul 20, 2021 at 12:16:32AM +0200, Linus Walleij wrote:
> > On Mon, Jul 19, 2021 at 4:42 PM Wedson Almeida Filho
> > <wedsonaf@google.com> wrote:
> > 
> > > > I think people want to see the code inside these classes as well,
> > > > can we browse it?
> > >
> > > The whole tree is available here: https://github.com/wedsonaf/linux/tree/pl061
> > > -- I should caution you that the two most recent commits are not merged into the
> > > rust tree yet because they're WIP. (I'll clean them up and eventually merge
> > > after the feedback from you all.)
> > 
> > I found that the gpio_chip C wrappers lives here:
> > https://github.com/wedsonaf/linux/blob/pl061/rust/kernel/gpio.rs
> > 
> > This file is very interesting!
> > 
> > I think it's not a good idea to keep these wrappers in their own
> > rust directory, they need to be distributed out into the kernel
> > everywhere they are used. We have made this mistake before
> > with OF (device tree) that started exactly like this in
> > drivers/of/*, and now I have part of the OF GPIO handling
> > and tests inside files in that directory as a consequence.
> 
> That's great feedback. Our plan was to have the core parts in `rust/kernel`, but
> once maintainers got involved, we would place things where it made more sense.
> Since we have no maintainers involved in development yet, everything is in rust/
> for now.

Part of the issue here is -stable and back porting fixes. If files
move around within the tree, it makes this back porting harder. git
cherry-pick is a lot less likely to work, it needs more manual
intervention. I expect it will make the stable teams job much easier
if these files are in the right place from day 1 and never move.

   Andrew

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 20, 2021 at 3:21 PM Andrew Lunn <andrew@lunn.ch> wrote:
>
> Part of the issue here is -stable and back porting fixes. If files
> move around within the tree, it makes this back porting harder. git
> cherry-pick is a lot less likely to work, it needs more manual
> intervention. I expect it will make the stable teams job much easier
> if these files are in the right place from day 1 and never move.

We can definitely move things already if we agree on a particular
approach -- we definitely do not want to make anybody's job harder.

An alternative could be stating that Rust support will not get
backports for e.g. the first two or three releases to give it a bit of
time to mature in-tree; while making it easier to get maintainers
involved and in general everything rolling.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Andrew Lunn]

On Tue, Jul 20, 2021 at 03:38:24PM +0200, Miguel Ojeda wrote:
> On Tue, Jul 20, 2021 at 3:21 PM Andrew Lunn <andrew@lunn.ch> wrote:
> >
> > Part of the issue here is -stable and back porting fixes. If files
> > move around within the tree, it makes this back porting harder. git
> > cherry-pick is a lot less likely to work, it needs more manual
> > intervention. I expect it will make the stable teams job much easier
> > if these files are in the right place from day 1 and never move.
> 
> We can definitely move things already if we agree on a particular
> approach -- we definitely do not want to make anybody's job harder.
> 
> An alternative could be stating that Rust support will not get
> backports for e.g. the first two or three releases to give it a bit of
> time to mature in-tree; while making it easier to get maintainers
> involved and in general everything rolling.

Let see what the stable team say about that. But it is well known that
it is hard to separate security fixes out from other bug fixes. So by
not back porting, you are potentially leaving open security issues. I
guess these will be in the unsafe code, and that seems to be mostly in
the files we are talking about.

    Andrew

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Tue, Jul 20, 2021 at 03:21:32PM +0200, Andrew Lunn wrote:
> On Tue, Jul 20, 2021 at 02:20:34AM +0100, Wedson Almeida Filho wrote:
> > On Tue, Jul 20, 2021 at 12:16:32AM +0200, Linus Walleij wrote:
> > > On Mon, Jul 19, 2021 at 4:42 PM Wedson Almeida Filho
> > > <wedsonaf@google.com> wrote:
> > > 
> > > > > I think people want to see the code inside these classes as well,
> > > > > can we browse it?
> > > >
> > > > The whole tree is available here: https://github.com/wedsonaf/linux/tree/pl061
> > > > -- I should caution you that the two most recent commits are not merged into the
> > > > rust tree yet because they're WIP. (I'll clean them up and eventually merge
> > > > after the feedback from you all.)
> > > 
> > > I found that the gpio_chip C wrappers lives here:
> > > https://github.com/wedsonaf/linux/blob/pl061/rust/kernel/gpio.rs
> > > 
> > > This file is very interesting!
> > > 
> > > I think it's not a good idea to keep these wrappers in their own
> > > rust directory, they need to be distributed out into the kernel
> > > everywhere they are used. We have made this mistake before
> > > with OF (device tree) that started exactly like this in
> > > drivers/of/*, and now I have part of the OF GPIO handling
> > > and tests inside files in that directory as a consequence.
> > 
> > That's great feedback. Our plan was to have the core parts in `rust/kernel`, but
> > once maintainers got involved, we would place things where it made more sense.
> > Since we have no maintainers involved in development yet, everything is in rust/
> > for now.
> 
> Part of the issue here is -stable and back porting fixes. If files
> move around within the tree, it makes this back porting harder. git
> cherry-pick is a lot less likely to work, it needs more manual
> intervention. I expect it will make the stable teams job much easier
> if these files are in the right place from day 1 and never move.

Files move around every release, it's not a big deal for stable
maintainers.  Just do not do it for no good reason.

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 20, 2021 at 12:16 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> I think it's not a good idea to keep these wrappers in their own
> rust directory, they need to be distributed out into the kernel
> everywhere they are used. We have made this mistake before
> with OF (device tree) that started exactly like this in
> drivers/of/*, and now I have part of the OF GPIO handling
> and tests inside files in that directory as a consequence.

The `rust/kernel/*` folder currently contains all the "Rust
abstractions" that are shared by all the kernel, i.e. in the rest of
the kernel tree we have kernel modules that are users of the
abstractions.

The main reasons behind this approach are that it is the simplest one
and that it makes a clear distinction between abstractions and user
code. In fact, code inside `rust/` has some capabilities that we do
not allow for non-`rust/` code. For instance, `rust/*` code can use
the C bindings and has unrestricted access to Rust unstable features
(though we still carefully pick the ones we use, of course). This is
not the case for "normal" Rust kernel modules elsewhere.

Having said that, this is not set in stone or a hard requirement -- we
can definitely split things and move them elsewhere, e.g. having a
`gpio` crate in `drivers/gpio/`, for instance. We could still follow
special rules for "subsystem objects".

In any case, please note that the compilation model is different in
Rust than in C. In Rust, the translation unit is not each `.rs` file,
but the "crate" (i.e. a set of `.rs` files that are found by the
compiler given a `.rs` entry point file), there are no header files,
etc. Thus some differences apply. For instance, currently having
everything in the same "crate" means the compiler can see everything
even without LTO.

> The wrappers are a hard read, I notice a whole lot of unsafe

The Rust abstractions are indeed harder to read, specially now that
they aren't documented -- Wedson did a great job to have all the
pieces ready quickly to show a working GPIO driver; but later on the
files will be documented, with `SAFETY`/`INVARIANT` comments too (that
we require), etc. as you can see in other files.

In addition, a lot of complexity comes due to using the C APIs. As we
discussed in the driver model discussion the other week, if we all
decide to give the Rust side more freedom to reimplement things,
including entire subsystems without being constrained by the C API,
then we can definitely reduce the complexity (plus reduce the number
of `unsafe` blocks etc.).

Finally, as we get to write more abstractions/subsystems/wrappers, we
will all learn the code patterns that tend to appear, how to factorize
them, how to clean things up, etc.

> keywords here, Chip is missing .set_config() which is understandable
> since PL061 isn't using it, the New call to create an instance
> contains a lot of magic stuff I don't understand like
> state: AtomicU8::new(0) and such but I think that's all right
> it probably only needs a trained eye.

Yeah, that `{ state: ..., ... }` syntax is just initializing a
`struct`, similar to the `{ .state = ..., ... }` designated
initializers in C.

Thus if you have some `struct MyFoo` and you are writing a function
that initializes it, then you can think of that `new()` as something
like:

    struct MyFoo myfoo_new() {
      return (struct MyFoo) { .a = 42, .b = 43 };
    }

vs.

    impl myfoo {
        pub fn new() -> Self {
            Self { a: 42, b: 43 }
        }
    }

Similarly, `AtomicU8::new(0)` is just a call to a normal function --
you can think of it as `atomicu8_new(0)`, i.e. it is just namespaced
inside a type, but it is a normal / free / plain function (i.e. not a
member function / method).

> (...)
>
> To that end the core of the GPIO library would probably
> have to be rewritten in Rust and used on *all* platforms
> in order to buy us any improved security. I understand that

I do not agree: a bug in a driver can cause havok of all kinds in a
system, security-wise or otherwise. One does not need to rewrite the
entire kernel in Rust to start to see benefits.

Moreover, even if Rust did not buy us any security, writing Rust code
would still have all the other advantages as listed in the RFC.

> this is not currently the plan.

Let me clarify that it is not against the plan either.

In other words, we are following the approach of wrapping C APIs
because we think it is important to show that Rust can actually
cooperate with the C side, plus that one can actually write safe
abstractions for existing C code without leaking a lot of `unsafe`
into drivers.

But things can definitely be written from scratch too -- and that
gives extra advantages, definitely (and not just in security).

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1711 bytes --]

On Tue, Jul 20, 2021 at 03:21:45AM +0200, Miguel Ojeda wrote:
> On Tue, Jul 20, 2021 at 12:16 AM Linus Walleij <linus.walleij@linaro.org> wrote:

> > I think it's not a good idea to keep these wrappers in their own
> > rust directory, they need to be distributed out into the kernel
> > everywhere they are used. We have made this mistake before
> > with OF (device tree) that started exactly like this in
> > drivers/of/*, and now I have part of the OF GPIO handling
> > and tests inside files in that directory as a consequence.

> The `rust/kernel/*` folder currently contains all the "Rust
> abstractions" that are shared by all the kernel, i.e. in the rest of
> the kernel tree we have kernel modules that are users of the
> abstractions.

> The main reasons behind this approach are that it is the simplest one
> and that it makes a clear distinction between abstractions and user
> code. In fact, code inside `rust/` has some capabilities that we do
> not allow for non-`rust/` code. For instance, `rust/*` code can use
> the C bindings and has unrestricted access to Rust unstable features
> (though we still carefully pick the ones we use, of course). This is
> not the case for "normal" Rust kernel modules elsewhere.

> Having said that, this is not set in stone or a hard requirement -- we
> can definitely split things and move them elsewhere, e.g. having a
> `gpio` crate in `drivers/gpio/`, for instance. We could still follow
> special rules for "subsystem objects".

It does feel like it'd be good to work out some system for ensuring that
subsystem maintainers have sight of any Rust work that's going on for
their subsystems rather than having that happen off in a corner without
visibility.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [TECH TOPIC] Rust for Linux

[Linus Walleij]

On Tue, Jul 20, 2021 at 3:21 AM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
> On Tue, Jul 20, 2021 at 12:16 AM Linus Walleij <linus.walleij@linaro.org> wrote:

> > To that end the core of the GPIO library would probably
> > have to be rewritten in Rust and used on *all* platforms
> > in order to buy us any improved security. I understand that
>
> I do not agree: a bug in a driver can cause havok of all kinds in a
> system, security-wise or otherwise. One does not need to rewrite the
> entire kernel in Rust to start to see benefits.

I understand and I see that Wedson also came up with some ideas
on how Rust can actually protect against some DoS type attacks.

But what attackers (who by the way are organized, well-funded and
pretty evil people) are creating and what is our biggest headache is
remote root exploits that open up systems to random code execution,
and that is why we have to take Rust to the enemy lines IMO, because
that is where it will deliver shock and awe.

The problem is not a few saboteurs in our factories in our homeland
but on the frontline with the enemy if you excuse the war metaphor.
I want to see Rust weaponized against the people who attack our
kernel.

Yours,
Linus Walleij

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Mon, Jul 19, 2021 at 3:53 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
>     fn unmask(data: &Ref<DeviceData>, irq_data: &IrqData) {
>         let mask = bit(irq_data.hwirq() % u64::from(PL061_GPIO_NR));
>         let _guard = data.lock();
>         if let Some(pl061) = data.resources() {
>             let gpioie = pl061.base.readb(GPIOIE) | mask;
>             let _ = pl061.base.try_writeb(gpioie, GPIOIE);
>         }
>     }

Wedson will answer thoroughly your other questions, but a quick note
on handling `{read,write}{r,w,l,q}` (since you asked in the past and
you put here the example :)

`pl061.base.readb(GPIOIE)` like above and similar calls are safe --
the function knows the address is correct because it controls the
computation of the address, rather than leaving users to compute the
address and then call a generic `{read,write}{r,w,l,q}` e.g. via
`readb(foo->base + GPIOIE)`. This is key.

`regmap` is able to check things similarly, but it does it at runtime
-- here the check happens at compile-time.

In fact, we can take this further. For instance, take a look at this
function in another GPIO driver that uses `regmap`:

    static int ts4900_gpio_get_direction(struct gpio_chip *chip,
unsigned int offset)
    {
        struct ts4900_gpio_priv *priv = gpiochip_get_data(chip);
        unsigned int reg;

        regmap_read(priv->regmap, offset, &reg);

        if (reg & TS4900_GPIO_OE)
            return GPIO_LINE_DIRECTION_OUT;

        return GPIO_LINE_DIRECTION_IN;
    }

The entirety of this function and similar ones can be automatically
generated, including the masking and conversion into another type,
things like shifting, unit conversion, bit banging, etc. so that you
could call:

    ts4900.map.get_direction()

Of course, this can be automated in independent tools/scripts that
e.g. generate C code from a hardware description. However, we could
also do the codegen in Rust itself, and provide a custom syntax to
describe the register map easily.

For instance, the driver developer could write something like this
right in the Rust source code:

    memory_map!(
        0x0042  simple_value  2  value(u16)
        0x0044  some_flags    2  flags(..., FLAG1, FLAG2)
        0x0046  custom_type   1  value(MyCustomType)
        0x0047  across_bits   2  [
                                   0..2  other_flags  flags(FOO, BAR),
                                   2..4  _            unused,
                                   4..31 temp         value(Temperature),
                                 ]
    );

And then simply call:

    driver.map.read_temp()

and it would automatically perform the shifting, masking, unit
conversions, etc. to get the raw sensor data into a value in proper
Kelvin.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Andrew Lunn]

> For instance, the driver developer could write something like this
> right in the Rust source code:
> 
>     memory_map!(
>         0x0042  simple_value  2  value(u16)
>         0x0044  some_flags    2  flags(..., FLAG1, FLAG2)
>         0x0046  custom_type   1  value(MyCustomType)
>         0x0047  across_bits   2  [
>                                    0..2  other_flags  flags(FOO, BAR),
>                                    2..4  _            unused,
>                                    4..31 temp         value(Temperature),
>                                  ]
>     );
> 
> And then simply call:
> 
>     driver.map.read_temp()
> 
> and it would automatically perform the shifting, masking, unit
> conversions, etc. to get the raw sensor data into a value in proper
> Kelvin.

Hi Miguel

Can you express endianness here as well? There are often cases where
the hardware is always big endian independent of the CPU
endianness. Some of the readl/writel macros handle this, adding a swap
when the two don't match.

     Andrew

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Mon, Jul 19, 2021 at 5:15 PM Andrew Lunn <andrew@lunn.ch> wrote:
>
> Can you express endianness here as well? There are often cases where
> the hardware is always big endian independent of the CPU
> endianness. Some of the readl/writel macros handle this, adding a swap
> when the two don't match.

Yes, of course!

Querying the endianness can be done as usual, similar to the C preprocessor:

    pub fn f(n: i32) -> i32 {
        #[cfg(target_endian = "little")]
        return n + 42;

        #[cfg(target_endian = "big")]
        return n;
    }

As well as with the `cfg!` macro, more ergonomic for small things like this:

    pub fn f(n: i32) -> i32 {
        if cfg!(target_endian = "little") {
            n + 42
        } else {
            n
        }
    }

As well in macros (like the `memory_map!` shown above).

If you just need the most common case, i.e. converting from/to a given
endianness, the primitive types come with functions for that, e.g. the
following would include a `bswap` in x86:

    pub fn f(n: i32) -> i32 {
        i32::from_be(n)
    }

See `{from,to}_{be,le}` in https://doc.rust-lang.org/std/primitive.i32.html

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Viresh Kumar]

On 08-07-21, 21:49, Linus Walleij wrote:
> I also Cced Viresh Kumar who I think is thinking about Rust these
> days, using it for one end of a virtio pipe (IIUC) and Andy also brought
> up virtio, but now on the other side, in the kernel.

Thanks Linus.

Just so everyone is aware of our work with Rust, we (at Linaro's
Project Stratos [1] initiative) are looking to write Hypervisor
agnostic low-level virtio based vhost-user backends, which can be used
to process low-level (like I2C, GPIO, RPMB, etc) requests from guests
running on VMs to the host having access to the real hardware.

We are still working on getting the virtio GPIO spec merged, but the
I2C stuff has progressed well. Here is the virtio backend I have
written for I2C in Rust:

https://github.com/vireshk/vhost-device

It is under review [2] currently to get merged in rust-vmm project.

-- 
viresh

[1] https://linaro.atlassian.net/wiki/spaces/STR/overview
[2] https://github.com/rust-vmm/vhost-device/pull/1

Re: [TECH TOPIC] Rust for Linux

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 898 bytes --]

On Thu, Jul 08, 2021 at 09:49:15PM +0200, Linus Walleij wrote:
> On Thu, Jul 8, 2021 at 8:51 PM Greg KH <greg@kroah.com> wrote:

> > But if you didn't like the previous examples (nvme block driver, i2c
> > driver, gpio driver), how about looking at the drivers used by your
> > current desktop and picking something that you use today that actually
> > talks to hardware?

> With my GPIO maintainer hat on I'd say a GPIO driver would be quite
> interesting to look at. We are two GPIO maintainers and Bartosz is
> doing the heavy lifting for the moment so I'm connecting Bartosz to this
> discussion. (Now he has to read through the whole backlog,
> sorry Bart!)

SPI might also be interesting here and I think there's some SPI
controllers emulated in qemu, though no idea how well or specific
instructions for any boards.  There's a bit more concurrency and so on
stuff going on in the framework.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Fri, Jul 9, 2021 at 7:07 PM Mark Brown <broonie@kernel.org> wrote:
>
> SPI might also be interesting here and I think there's some SPI
> controllers emulated in qemu, though no idea how well or specific
> instructions for any boards.  There's a bit more concurrency and so on
> stuff going on in the framework.

[Cc'ing Arthur, Esteban and Martin since they have been working on SPI]

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Jonathan Cameron]

On 9 July 2021 18:43:09 BST, Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote:
>On Fri, Jul 9, 2021 at 7:07 PM Mark Brown <broonie@kernel.org> wrote:
>>
>> SPI might also be interesting here and I think there's some SPI
>> controllers emulated in qemu, though no idea how well or specific
>> instructions for any boards.  There's a bit more concurrency and so
>on
>> stuff going on in the framework.
>
>[Cc'ing Arthur, Esteban and Martin since they have been working on SPI]

I use qemu with both spi and gpio for testing IIO drivers..  Not bothered upstreaming
the IIO device emulation as generally throw away quality code to ensure I don't
break things when doing nasty driver reactors.  I have a request that I need to
respond to though to share one particular setup used for the ad7280a.  Should get
 to that this weekend. 

https://lore.kernel.org/linux-iio/YNIfkaRZtWIXPbAj@marsc.168.1.7/

Some devices are too much a pain to deal with in real hardware and the qemu route
gives very fast development cycles even if you plan to do final tests on real devices. 

Jonathan

>
>Cheers,
>Miguel

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [TECH TOPIC] Rust for Linux

[Kees Cook]

On Thu, Jul 08, 2021 at 09:49:15PM +0200, Linus Walleij wrote:
> My concern is still whether this really brings Rust closer to the actual
> problems we have and that Rust can solve. If the problem is really
> about real world security issues, I would ask Kees where the actual
> attack surface of the kernel is. He knows.

Oh yes indeed! Two big classes of flaws the kernel suffers from due
to being written in C are buffer overflows and use after free. Both
of these are strongly mitigated by Rust's borrow checking. The other
class I've been concerned with is arithmetic overflow (which usually
precedes a buffer overflow), and that's why I've been pushing for the
kernel's Rust to operate in Rust's "trap on overflow" mode by default,
where the exceptions that expect a wrap-around use a distinct API.

> It kind of matters. EternalBlue (WannaCry) was a horrific Windows
> exploit in that it shows us pretty well what kind of cyberweapons the
> intelligence agencies of the world have been constructing and
> stockpiling, and probably also used. We need to put countermeasures
> where such exploits are likely to hit, yesterday. Intuitively I would
> say any in-kernel network daemons, anything complex that
> responds directly to network traffic, is a good thing to fix. I do not

FWIW, wireless drivers have been a persistent source of buffer overflows.

I've been working on some changes to our memcpy() use in the kernel to
help address this, but it's not quite finished yet. Very soon, I hope!
But even these changes are just dealing with the "easy" cases where
the code is luckily enough arranged in a way that the compiler can do
the bounds checking. (The same goes for CONFIG_UBSAN_BOUNDS.) Rust just
makes the whole class of problem go away because of its borrow semantics.

-Kees

-- 
Kees Cook

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

    On Thu, Jul 8, 2021 at 8:11 AM Greg KH <greg@kroah.com> wrote:
>
> Thanks for the detailed explainations, it seems rust can "get away" with
> some things with regards to reference counts that the kernel can not.
> Userspace has it easy, but note that now that rust is not in userspace,
> dealing with multiple cpus/threads is going to be interesting for the
> language.

Regarding parallelism, userspace deals with the same problems, so I do
not see fundamental issues coming up there. The language was designed
with parallelism in mind and is a definite improvement over both C and
C++ in that regard.

> So, along those lines, how are you going to tie rust's reference count
> logic in with the kernel's reference count logic?  How are you going to

Let me clarify that Rust does not have "reference count logic" in the
sense of some compiler magic (nor a runtime) that somehow "knows" that
there is a reference count going on.

The confusion may come from similarly worded concepts: the
lifetimes/borrowing rules are orthogonal to reference-counting
semantics.

For instance, when Wedson was explaining the lifetime associated to a
reference when e.g. passed into a function:

    fn f(x: &X) {
        ...
    }

That `&X` (a "Rust reference") is unrelated to the "reference" in
"reference-counting semantics".

In fact, Rust's lifetime/reference/borrowing rules apply to all code.
For instance:

    let mut i = 42;

    let r1 = &mut i;
    let r2 = &mut i;

    f(r1, r2)

This code has two "Rust references", but there are no
"reference-counted semantics" at all here, nor any refcount involved.
Yet Rust prevents this code from compiling because you are attempting
to create two mutable "Rust references" that alias the same object.

> handle dentries, inodes, kobjects, devices and the like?  That's the
> real question that I don't seem to see anyone even starting to answer
> just yet.

Those cases can be handled, one way or another. We will try to have
something more concrete as soon as possible.

Having said that, if people is discussing how to improve the C model
anyway (i.e. the "cdev/devm_* issues" thread), it could be nice --
longer-term -- if we could also take the chance to discuss how to come
up with a model that suites Rust too. This could lead to simpler code,
more ergonomic abstractions and/or less `unsafe` code.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Thu, Jul 08, 2021 at 03:55:31PM +0200, Miguel Ojeda wrote:
>     On Thu, Jul 8, 2021 at 8:11 AM Greg KH <greg@kroah.com> wrote:
> >
> > Thanks for the detailed explainations, it seems rust can "get away" with
> > some things with regards to reference counts that the kernel can not.
> > Userspace has it easy, but note that now that rust is not in userspace,
> > dealing with multiple cpus/threads is going to be interesting for the
> > language.
> 
> Regarding parallelism, userspace deals with the same problems, so I do
> not see fundamental issues coming up there. The language was designed
> with parallelism in mind and is a definite improvement over both C and
> C++ in that regard.

Ok, great, then how would you handle the kref issue where you need an
external lock to properly handle the reference counting logic in Rust?
Why is C so "bad" here that we require a lock but Rust would not?

> > So, along those lines, how are you going to tie rust's reference count
> > logic in with the kernel's reference count logic?  How are you going to
> 
> Let me clarify that Rust does not have "reference count logic" in the
> sense of some compiler magic (nor a runtime) that somehow "knows" that
> there is a reference count going on.
> 
> The confusion may come from similarly worded concepts: the
> lifetimes/borrowing rules are orthogonal to reference-counting
> semantics.
> 
> For instance, when Wedson was explaining the lifetime associated to a
> reference when e.g. passed into a function:
> 
>     fn f(x: &X) {
>         ...
>     }
> 
> That `&X` (a "Rust reference") is unrelated to the "reference" in
> "reference-counting semantics".
> 
> In fact, Rust's lifetime/reference/borrowing rules apply to all code.
> For instance:
> 
>     let mut i = 42;
> 
>     let r1 = &mut i;
>     let r2 = &mut i;
> 
>     f(r1, r2)
> 
> This code has two "Rust references", but there are no
> "reference-counted semantics" at all here, nor any refcount involved.
> Yet Rust prevents this code from compiling because you are attempting
> to create two mutable "Rust references" that alias the same object.
> 
> > handle dentries, inodes, kobjects, devices and the like?  That's the
> > real question that I don't seem to see anyone even starting to answer
> > just yet.
> 
> Those cases can be handled, one way or another. We will try to have
> something more concrete as soon as possible.
> 
> Having said that, if people is discussing how to improve the C model
> anyway (i.e. the "cdev/devm_* issues" thread), it could be nice --
> longer-term -- if we could also take the chance to discuss how to come
> up with a model that suites Rust too. This could lead to simpler code,
> more ergonomic abstractions and/or less `unsafe` code.

Ok, what "model" would be better?  We need a "base object" that has
solid lifecycle rules.  Right now we do have that, but it's just not the
easiest to use for complex objects where we have multiple objects with
different lifecycles that all interact together.  The v4l example is the
best, but the input layer also has this type of issue.

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Thu, Jul 08, 2021 at 04:58:43PM +0200, Greg KH wrote:

> Ok, what "model" would be better?  We need a "base object" that has
> solid lifecycle rules.  Right now we do have that, but it's just not the
> easiest to use for complex objects where we have multiple objects with
> different lifecycles that all interact together.  The v4l example is the
> best, but the input layer also has this type of issue.

Audio too, the hardware is very similar v4l here with a lot of
mix'n'match of separate devices to make one user visible thing which is
itself exposed via multiple device nodes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [TECH TOPIC] Rust for Linux

[Andy Lutomirski]

On Thu, Jul 8, 2021 at 7:58 AM Greg KH <greg@kroah.com> wrote:
>
> On Thu, Jul 08, 2021 at 03:55:31PM +0200, Miguel Ojeda wrote:
> >     On Thu, Jul 8, 2021 at 8:11 AM Greg KH <greg@kroah.com> wrote:
> > >
> > > Thanks for the detailed explainations, it seems rust can "get away" with
> > > some things with regards to reference counts that the kernel can not.
> > > Userspace has it easy, but note that now that rust is not in userspace,
> > > dealing with multiple cpus/threads is going to be interesting for the
> > > language.
> >
> > Regarding parallelism, userspace deals with the same problems, so I do
> > not see fundamental issues coming up there. The language was designed
> > with parallelism in mind and is a definite improvement over both C and
> > C++ in that regard.
>
> Ok, great, then how would you handle the kref issue where you need an
> external lock to properly handle the reference counting logic in Rust?
> Why is C so "bad" here that we require a lock but Rust would not?

Can you point at a specific example in the kernel tree?  The
lock-and-then-put model is, at the very least, unusual, and the kref
docs talk about it like it's common and self-explanatory as to when
it's needed.  I have personally never encountered a need for this, and
I'd like to know exactly what type of use case you're thinking of.

> Ok, what "model" would be better?  We need a "base object" that has
> solid lifecycle rules.  Right now we do have that, but it's just not the
> easiest to use for complex objects where we have multiple objects with
> different lifecycles that all interact together.  The v4l example is the
> best, but the input layer also has this type of issue.
>

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Thu, Jul 08, 2021 at 09:38:01AM -0700, Andy Lutomirski wrote:
> On Thu, Jul 8, 2021 at 7:58 AM Greg KH <greg@kroah.com> wrote:
> >
> > On Thu, Jul 08, 2021 at 03:55:31PM +0200, Miguel Ojeda wrote:
> > >     On Thu, Jul 8, 2021 at 8:11 AM Greg KH <greg@kroah.com> wrote:
> > > >
> > > > Thanks for the detailed explainations, it seems rust can "get away" with
> > > > some things with regards to reference counts that the kernel can not.
> > > > Userspace has it easy, but note that now that rust is not in userspace,
> > > > dealing with multiple cpus/threads is going to be interesting for the
> > > > language.
> > >
> > > Regarding parallelism, userspace deals with the same problems, so I do
> > > not see fundamental issues coming up there. The language was designed
> > > with parallelism in mind and is a definite improvement over both C and
> > > C++ in that regard.
> >
> > Ok, great, then how would you handle the kref issue where you need an
> > external lock to properly handle the reference counting logic in Rust?
> > Why is C so "bad" here that we require a lock but Rust would not?
> 
> Can you point at a specific example in the kernel tree?

Easy ones to see are any callers of kref_put_mutex() and
kref_put_lock().  There's also the klist.h usage that removes the need
for using those calls as there is a different lock involved, and other
"bigger" locks that the driver core holds when dealing with kobjects so
it "knows" it can just call kref_put().

> The
> lock-and-then-put model is, at the very least, unusual, and the kref
> docs talk about it like it's common and self-explanatory as to when
> it's needed.  I have personally never encountered a need for this, and
> I'd like to know exactly what type of use case you're thinking of.

Look at the above examples for details, I can't remember them all at the
moment.  Last time I looked into it, it took me and two graduate
students with a whiteboard an hour to verify if we needed the lock or
not.  I was wrong and it turns out we did :)

It mostly is needed when you need to return a pointer to a reference
counted object from some sort of lookup as it was stored in a list or
something.  Which for kernel objects, is quite common.

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Thu, Jul 8, 2021 at 4:58 PM Greg KH <greg@kroah.com> wrote:
>
> Ok, great, then how would you handle the kref issue where you need an
> external lock to properly handle the reference counting logic in Rust?
> Why is C so "bad" here that we require a lock but Rust would not?

That's the thing: neither C nor Rust need a lock to handle the
refcount itself, i.e. `kref` does not contain a lock, neither `Ref<T>`
does.

A lock is needed when you actually interact with the contents that are
refcounted (see my previous message about `Ref<T>`) -- and this
applies the same way to Rust and C.

What Rust is buying is, though, and this is the critical point, is
that we cannot make a mistake forgetting that we need to have some
form of lock to concurrently mutate things nor forgetting we need to
actually lock it.

Thus, in a way, you could think that the rules in `Doc/kref.txt` would
be now enforced statically -- that is the difference.

> Ok, what "model" would be better?  We need a "base object" that has
> solid lifecycle rules.  Right now we do have that, but it's just not the
> easiest to use for complex objects where we have multiple objects with
> different lifecycles that all interact together.  The v4l example is the
> best, but the input layer also has this type of issue.

The model can be very similar conceptually, but if we can take the
freedom to manage the objects in the Rust side instead of forcing
ourselves to go through C APIs (e.g. `devm_*`, dealing with
self-referential structs, etc.), then things can be way simpler.

Which is why I am asking: so far we have tried hard to reuse the C
APIs and types, but if we can relax that constraint and come up with
our own model (where the Rust side keeps ownership of objects), then
we will likely end up with something better (for the Rust side).

Basically, the idea is doing things in a way that is amenable to
Rust's strengths, so that modules written in Rust are easier to write
and so that we minimize even further the need for `unsafe`.

To be extra clear: it is not that Rust cannot use/express/wrap those C
patterns -- the key is that some of those C patterns cannot be
statically proven to be correct by the Rust compiler (and thus require
trusting ourselves to get them right, like in C).

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Thu, Jul 08, 2021 at 08:00:15PM +0200, Miguel Ojeda wrote:
> On Thu, Jul 8, 2021 at 4:58 PM Greg KH <greg@kroah.com> wrote:
> >
> > Ok, great, then how would you handle the kref issue where you need an
> > external lock to properly handle the reference counting logic in Rust?
> > Why is C so "bad" here that we require a lock but Rust would not?
> 
> That's the thing: neither C nor Rust need a lock to handle the
> refcount itself, i.e. `kref` does not contain a lock, neither `Ref<T>`
> does.

kref does not contain the lock, but you need to use a lock lots of time
when dealing with them.  See my response to Andy for why.

> A lock is needed when you actually interact with the contents that are
> refcounted (see my previous message about `Ref<T>`) -- and this
> applies the same way to Rust and C.
> 
> What Rust is buying is, though, and this is the critical point, is
> that we cannot make a mistake forgetting that we need to have some
> form of lock to concurrently mutate things nor forgetting we need to
> actually lock it.
> 
> Thus, in a way, you could think that the rules in `Doc/kref.txt` would
> be now enforced statically -- that is the difference.

Ok, fair enough, thanks.

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Thu, Jul 8, 2021 at 8:44 PM Greg KH <greg@kroah.com> wrote:
>
> kref does not contain the lock, but you need to use a lock lots of time
> when dealing with them.  See my response to Andy for why.

A lock may be needed in certain scenarios, but that is because
something else is involved -- i.e. to maintain the refcount itself we
don't need a lock.

Let me give an inefficient, unrealistic counter-example: we could use
an atomic with sequential consistency for the refcount.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Wedson,

On Thu, Jul 8, 2021 at 1:09 AM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> On Wed, Jul 07, 2021 at 10:38:27PM +0200, Jan Kara wrote:
> > On Wed 07-07-21 20:19:19, Wedson Almeida Filho wrote:
> > > Where would a lock be needed in the examples above?
> >
> > So I think Greg speaks about a situation where you have multiple threads
> > and the refcounted object can be looked up through some structure all the
> > threads see. And the problem is that the shared data structure cannot hold
> > ref to the object it points to because you want to detect the situation
> > where the data structure is the only place pointing to the object and
> > reclaim the object in that case. Currently I don't see how to model this
> > idiom with Rust refs.
>
> The normal idiom in Rust for this is "weak" pointers. With it, each
> reference-counted object has two counts: strong and weak refs. Objects are
> "destroyed" when the strong count goes to zero and "freed" when the weak count
> goes to zero.
>
> Weak references need to upgraded to strong references before the underlying
> objects can be accessed; upgrading may fail if the strong count has gone to
> zero. It is, naturally, implemented as an increment that avoids going from 0 to
> 1. It is safe to try to do it because the memory is kept alive while there are
> weak references.

What does "may fail" imply?

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Thu, Jul 08, 2021 at 09:20:25AM +0200, Geert Uytterhoeven wrote:
> > Weak references need to upgraded to strong references before the underlying
> > objects can be accessed; upgrading may fail if the strong count has gone to
> > zero. It is, naturally, implemented as an increment that avoids going from 0 to
> > 1. It is safe to try to do it because the memory is kept alive while there are
> > weak references.
> 
> What does "may fail" imply?

Upgrading is essentially calling `refcount_inc_not_zero` on the strong count.
It succeeds when the count is already non-zero, it fails when the count is zero.

So "may fail" here means "your attempt to upgrade came too late, the object is
gone". (The memory is still around so that attempts to upgrade don't cause UAF,
but the object is gone.)

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Wedson,

On Thu, Jul 8, 2021 at 3:42 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> On Thu, Jul 08, 2021 at 09:20:25AM +0200, Geert Uytterhoeven wrote:
> > > Weak references need to upgraded to strong references before the underlying
> > > objects can be accessed; upgrading may fail if the strong count has gone to
> > > zero. It is, naturally, implemented as an increment that avoids going from 0 to
> > > 1. It is safe to try to do it because the memory is kept alive while there are
> > > weak references.
> >
> > What does "may fail" imply?
>
> Upgrading is essentially calling `refcount_inc_not_zero` on the strong count.
> It succeeds when the count is already non-zero, it fails when the count is zero.
>
> So "may fail" here means "your attempt to upgrade came too late, the object is
> gone". (The memory is still around so that attempts to upgrade don't cause UAF,
> but the object is gone.)

So what happens if this fails?

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Thu, Jul 08, 2021 at 03:43:28PM +0200, Geert Uytterhoeven wrote:
> Hi Wedson,

Hey Geert,

> On Thu, Jul 8, 2021 at 3:42 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> > On Thu, Jul 08, 2021 at 09:20:25AM +0200, Geert Uytterhoeven wrote:
> > > > Weak references need to upgraded to strong references before the underlying
> > > > objects can be accessed; upgrading may fail if the strong count has gone to
> > > > zero. It is, naturally, implemented as an increment that avoids going from 0 to
> > > > 1. It is safe to try to do it because the memory is kept alive while there are
> > > > weak references.
> > >
> > > What does "may fail" imply?
> >
> > Upgrading is essentially calling `refcount_inc_not_zero` on the strong count.
> > It succeeds when the count is already non-zero, it fails when the count is zero.
> >
> > So "may fail" here means "your attempt to upgrade came too late, the object is
> > gone". (The memory is still around so that attempts to upgrade don't cause UAF,
> > but the object is gone.)
> 
> So what happens if this fails?

You move on the next element in your data structure. This one doesn't really
exist anymore; once you release it lock, the cleanup code will likely come and
remove it.

This is a common pattern, see for example (or search for uses of
`refcount_inc_not_zero` and `kref_get_unless_zero`):
https://elixir.bootlin.com/linux/latest/source/net/core/skbuff.c#L4738

The difference between the above and Rust is that in C, one can mistakenly use
the "object" even if the refcount is zero. Rust would prohibit it.

Cheers,
-Wedson

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Wedson,

On Thu, Jul 8, 2021 at 3:54 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> On Thu, Jul 08, 2021 at 03:43:28PM +0200, Geert Uytterhoeven wrote:
> > On Thu, Jul 8, 2021 at 3:42 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> > > On Thu, Jul 08, 2021 at 09:20:25AM +0200, Geert Uytterhoeven wrote:
> > > > > Weak references need to upgraded to strong references before the underlying
> > > > > objects can be accessed; upgrading may fail if the strong count has gone to
> > > > > zero. It is, naturally, implemented as an increment that avoids going from 0 to
> > > > > 1. It is safe to try to do it because the memory is kept alive while there are
> > > > > weak references.
> > > >
> > > > What does "may fail" imply?
> > >
> > > Upgrading is essentially calling `refcount_inc_not_zero` on the strong count.
> > > It succeeds when the count is already non-zero, it fails when the count is zero.
> > >
> > > So "may fail" here means "your attempt to upgrade came too late, the object is
> > > gone". (The memory is still around so that attempts to upgrade don't cause UAF,
> > > but the object is gone.)
> >
> > So what happens if this fails?
>
> You move on the next element in your data structure. This one doesn't really
> exist anymore; once you release it lock, the cleanup code will likely come and
> remove it.

I'm confused. Which next element?
What happens if I have a weak reference to an object that cannot be
upgraded to a strong reference, and I try to access the object?
E.g. read from or write to a member of the object?

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Wedson Almeida Filho]

On Thu, Jul 08, 2021 at 04:16:54PM +0200, Geert Uytterhoeven wrote:
> Hi Wedson,
> 
> On Thu, Jul 8, 2021 at 3:54 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> > On Thu, Jul 08, 2021 at 03:43:28PM +0200, Geert Uytterhoeven wrote:
> > > On Thu, Jul 8, 2021 at 3:42 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> > > > On Thu, Jul 08, 2021 at 09:20:25AM +0200, Geert Uytterhoeven wrote:
> > > > > > Weak references need to upgraded to strong references before the underlying
> > > > > > objects can be accessed; upgrading may fail if the strong count has gone to
> > > > > > zero. It is, naturally, implemented as an increment that avoids going from 0 to
> > > > > > 1. It is safe to try to do it because the memory is kept alive while there are
> > > > > > weak references.
> > > > >
> > > > > What does "may fail" imply?
> > > >
> > > > Upgrading is essentially calling `refcount_inc_not_zero` on the strong count.
> > > > It succeeds when the count is already non-zero, it fails when the count is zero.
> > > >
> > > > So "may fail" here means "your attempt to upgrade came too late, the object is
> > > > gone". (The memory is still around so that attempts to upgrade don't cause UAF,
> > > > but the object is gone.)
> > >
> > > So what happens if this fails?
> >
> > You move on the next element in your data structure. This one doesn't really
> > exist anymore; once you release it lock, the cleanup code will likely come and
> > remove it.
> 
> I'm confused. Which next element?

If you have a list of weak references, like in the original example, I'm
referring to the next element of the list. If this is just a field of some other
struct, then you're out of luck, you have to act as if the object didn't exist.

> What happens if I have a weak reference to an object that cannot be
> upgraded to a strong reference, and I try to access the object?
> E.g. read from or write to a member of the object?

You can't by construction. The returned type would be `Option<Ref<T>>`, so to
access the fields of the returned object, you need something like:

    if let Some(obj) = weak.upgrade() {
        // `obj` is accessible here, you can access the fields.
    }

That is, if `upgrade` returns `None`, then you don't have a way to access the
fields of your struct. This is similar in concept to locks, where the fields of
a locked struct are only accessible when you acquire the lock.

Re: [TECH TOPIC] Rust for Linux

[Jerome Glisse]

On Thu, Jul 8, 2021 at 7:24 AM Wedson Almeida Filho <wedsonaf@google.com> wrote:
>
> On Thu, Jul 08, 2021 at 04:16:54PM +0200, Geert Uytterhoeven wrote:
> > Hi Wedson,
> >
> > On Thu, Jul 8, 2021 at 3:54 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> > > On Thu, Jul 08, 2021 at 03:43:28PM +0200, Geert Uytterhoeven wrote:
> > > > On Thu, Jul 8, 2021 at 3:42 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:
> > > > > On Thu, Jul 08, 2021 at 09:20:25AM +0200, Geert Uytterhoeven wrote:
> > > > > > > Weak references need to upgraded to strong references before the underlying
> > > > > > > objects can be accessed; upgrading may fail if the strong count has gone to
> > > > > > > zero. It is, naturally, implemented as an increment that avoids going from 0 to
> > > > > > > 1. It is safe to try to do it because the memory is kept alive while there are
> > > > > > > weak references.
> > > > > >
> > > > > > What does "may fail" imply?
> > > > >
> > > > > Upgrading is essentially calling `refcount_inc_not_zero` on the strong count.
> > > > > It succeeds when the count is already non-zero, it fails when the count is zero.
> > > > >
> > > > > So "may fail" here means "your attempt to upgrade came too late, the object is
> > > > > gone". (The memory is still around so that attempts to upgrade don't cause UAF,
> > > > > but the object is gone.)
> > > >
> > > > So what happens if this fails?
> > >
> > > You move on the next element in your data structure. This one doesn't really
> > > exist anymore; once you release it lock, the cleanup code will likely come and
> > > remove it.
> >
> > I'm confused. Which next element?
>
> If you have a list of weak references, like in the original example, I'm
> referring to the next element of the list. If this is just a field of some other
> struct, then you're out of luck, you have to act as if the object didn't exist.
>
> > What happens if I have a weak reference to an object that cannot be
> > upgraded to a strong reference, and I try to access the object?
> > E.g. read from or write to a member of the object?
>
> You can't by construction. The returned type would be `Option<Ref<T>>`, so to
> access the fields of the returned object, you need something like:
>
>     if let Some(obj) = weak.upgrade() {
>         // `obj` is accessible here, you can access the fields.
>     }
>
> That is, if `upgrade` returns `None`, then you don't have a way to access the
> fields of your struct. This is similar in concept to locks, where the fields of
> a locked struct are only accessible when you acquire the lock.

So if we have a double linked list and one item is a zombie (for lack
of better word)
then we are stuck ie we can no longer go to the element after the
zombie until the zombie
has been released and removed from the weak list ? Which I assume can only
happen if the zombie element can get a strong reference on the element we are
stuck on (as we are stuck on the one before the zombie in the list).
It seems like a
deadlock to me. If the list is not embedded inside the structure then
we will have
poor cache performance and this is likely to degrade performances.

The double linked list is one of the most used data patterns in the
kernel and I fail
to see how it would look in rust. Especially case like rcu list where you
traverse the list and jump over zombie entries without thinking twice about it.

Cheers,
Jerome

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Thu, Jul 8, 2021 at 3:43 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> So what happens if this fails?

Just in case: are you asking out of concern for "Rust panics" etc.?
The upgrade call does not need to panic, they can be fallible; e.g.
the `Weak` types in the Rust standard library return an `Option`.

Now, what the caller does if the upgrading fails depends on who the
caller is, as usual.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Miguel,

On Thu, Jul 8, 2021 at 4:04 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
> On Thu, Jul 8, 2021 at 3:43 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > So what happens if this fails?
>
> Just in case: are you asking out of concern for "Rust panics" etc.?

Exactly.

> The upgrade call does not need to panic, they can be fallible; e.g.
> the `Weak` types in the Rust standard library return an `Option`.
>
> Now, what the caller does if the upgrading fails depends on who the
> caller is, as usual.

Let's assume the caller access a member of the object regardless.
What happens?

Thanks!

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Thu, Jul 8, 2021 at 4:18 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Exactly.

I had a feeling... :)

> Let's assume the caller access a member of the object regardless.
> What happens?

That is the key: they cannot access it to begin with because the
returned `Option` is empty, so there is not even an object you can
"access", thus asking "what happens?" does not have a meaning.

In code:

    pub fn f(weak: Weak<i32>) {
        let strong = weak.upgrade();

        if let Some(content) = strong {
            println!("{}", content);
        }

        // There is no `content` here, so you cannot do anything with it
    }

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Miguel,

On Thu, Jul 8, 2021 at 4:28 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
> On Thu, Jul 8, 2021 at 4:18 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > Let's assume the caller access a member of the object regardless.
> > What happens?
>
> That is the key: they cannot access it to begin with because the
> returned `Option` is empty, so there is not even an object you can
> "access", thus asking "what happens?" does not have a meaning.
>
> In code:
>
>     pub fn f(weak: Weak<i32>) {
>         let strong = weak.upgrade();
>
>         if let Some(content) = strong {
>             println!("{}", content);
>         }
>
>         // There is no `content` here, so you cannot do anything with it

What if I would ignore that? I.e.

    let Some(content) = strong;
    println!("{}", content);

?

Please forgive my silly questions, I'm a Rust newbie ;-)

>     }

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Thu, Jul 8, 2021 at 4:33 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> What if I would ignore that? I.e.
>
>     let Some(content) = strong;
>     println!("{}", content);
>
> ?

It does not compile :-)

You can play with it here if you want

    https://godbolt.org/z/MeqznshPx

> Please forgive my silly questions, I'm a Rust newbie ;-)

No problem at all! It is perfectly understandable to ask these
questions -- from a C perspective, it is indeed quite shocking that
this "cannot happen".

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Miguel,

On Thu, Jul 8, 2021 at 4:36 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
> On Thu, Jul 8, 2021 at 4:33 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > What if I would ignore that? I.e.
> >
> >     let Some(content) = strong;
> >     println!("{}", content);
> >
> > ?
>
> It does not compile :-)

Oh, so it's like __must_check and -Werrror on steroids ;-)

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Andy Lutomirski]

> On Jul 8, 2021, at 7:34 AM, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Hi Miguel,
>
>> On Thu, Jul 8, 2021 at 4:28 PM Miguel Ojeda
>> <miguel.ojeda.sandonis@gmail.com> wrote:
>>> On Thu, Jul 8, 2021 at 4:18 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>>> Let's assume the caller access a member of the object regardless.
>>> What happens?
>>
>> That is the key: they cannot access it to begin with because the
>> returned `Option` is empty, so there is not even an object you can
>> "access", thus asking "what happens?" does not have a meaning.
>>
>> In code:
>>
>>    pub fn f(weak: Weak<i32>) {
>>        let strong = weak.upgrade();
>>
>>        if let Some(content) = strong {
>>            println!("{}", content);
>>        }
>>
>>        // There is no `content` here, so you cannot do anything with it
>
> What if I would ignore that? I.e.
>
>    let Some(content) = strong;
>    println!("{}", content);
>
> ?

That won’t compile. Pattern matching like this is fundamentally a
conditional operation. In Rust, or in pretty much any language with
proper “sum types” if you try to “destructure” something (the value
‘strong’ here) and there’s a possibility that the pattern doesn’t
match, then the compiler will require that you have some sort of
appropriate condition.

Personally, for some types of programming, I find sum types by
themselves to be an even bigger win than Rust-style safety.  C has
unions, which are painful to use. Python has various hacks that are
unpleasant. C++ likes to pretend that std::variant is decent, but it’s
really quite moderate.  Rust, Haskell, etc all have very nice sum
types.

As the most basic example, C often uses a null pointer to mean
“nothing here”, but you’re on your own to remember not to reference a
null pointer. In Rust a reference can’t be null, and Option can be
used to mean “maybe something here, maybe not”.  You can’t even
attempt to dereference the pointer without checking if it’s present.

For a more interesting example, a pointer to a page table entry could
be arranged to be a union of:

Empty: nothing there, but still has several bits of usable data
NextTable: a pointer to the next level of page tables
Page: a pointer to the physical address of the data along with access
rights and such

And one could manipulate page tables with no fear of forgetting about
the existence of huge pages — if you try to write code that
misinterprets a huge page (Page) as a pointer to the next table down
(NextLevel), the language won’t let you.

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 7:20 PM Greg KH <greg@kroah.com> wrote:
>
> That's not what I meant by bringing up a kref.  I was trying to ask
> where the "real" lock here is.  It has to be somewhere...

Given this sentence, I think I understand where the confusion comes
from. The key is that `Ref` (`Arc`) is not adding any thread-safety to
the `T` it wraps -- it is only concerned about providing
reference-counting semantics.

For that it only needs to keep a thread-safe refcount for a given `T`.
But the `T` might be (or not!) thread-safe. Since the refcount is an
atomic, that is enough for implementing `Ref<T>`, no locks are needed.

Now, if you need mutable access to the content of an `Ref<T>` that is
shared by several threads at the same time, you need something extra
-- that is where you would need a lock. For instance, in normal Rust
code you may see a `Arc<Mutex<T>>` being used.

The big thing is that if you get any of this wrong, you get a
compile-time error. For instance, if you create a `Ref<T>` out of a
`T` that is not marked as safe to be sent across threads, you get an
error when you try to send the `Ref`. Or if you create a `Ref<T>` and
try to call a method that mutates the `T` across threads, you will
also get a compile-time error. Thus you cannot forget to have a lock
if it is needed.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

On Wed, Jul 07, 2021 at 10:58:44PM +0200, Miguel Ojeda wrote:
> On Wed, Jul 7, 2021 at 7:20 PM Greg KH <greg@kroah.com> wrote:
> >
> > That's not what I meant by bringing up a kref.  I was trying to ask
> > where the "real" lock here is.  It has to be somewhere...
> 
> Given this sentence, I think I understand where the confusion comes
> from. The key is that `Ref` (`Arc`) is not adding any thread-safety to
> the `T` it wraps -- it is only concerned about providing
> reference-counting semantics.
> 
> For that it only needs to keep a thread-safe refcount for a given `T`.
> But the `T` might be (or not!) thread-safe. Since the refcount is an
> atomic, that is enough for implementing `Ref<T>`, no locks are needed.
> 
> Now, if you need mutable access to the content of an `Ref<T>` that is
> shared by several threads at the same time, you need something extra
> -- that is where you would need a lock. For instance, in normal Rust
> code you may see a `Arc<Mutex<T>>` being used.
> 
> The big thing is that if you get any of this wrong, you get a
> compile-time error. For instance, if you create a `Ref<T>` out of a
> `T` that is not marked as safe to be sent across threads, you get an
> error when you try to send the `Ref`. Or if you create a `Ref<T>` and
> try to call a method that mutates the `T` across threads, you will
> also get a compile-time error. Thus you cannot forget to have a lock
> if it is needed.

Out of curiosity: we have in the kernel objects shared between multiple
threads that require locking, but in some contexts taking the lock isn't
required. I'm thinking in particular about initialization time when you
create the object, before it is made visible to other users, or
destruction time, when you nobody else can have a reference to the
object anymore. Avoiding lock operations in those cases can be an
optimization. Cn rust handle that ?

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 11:47 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Out of curiosity: we have in the kernel objects shared between multiple
> threads that require locking, but in some contexts taking the lock isn't
> required. I'm thinking in particular about initialization time when you
> create the object, before it is made visible to other users, or
> destruction time, when you nobody else can have a reference to the
> object anymore. Avoiding lock operations in those cases can be an
> optimization. Cn rust handle that ?

Yes, you could provide e.g. an `unsafe fn` for users to access the
underlying data if the users know that such a thing is sound (by other
means that the compiler cannot verify).

To use it, users will, of course, need an `unsafe` block to make the
call compile; and a `SAFETY` comment on top of it explaining why it is
actually sound to make such a call (we require these).

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 9:27 AM Julia Lawall <julia.lawall@inria.fr> wrote:
>
> Where should the free have been?  Will Rust help in this case?  Will it
> result in a memory leak?

If the unsafe code is sound (which it should be -- otherwise we
consider it a bug), then we cannot have UB in Rust, so no crashes due
to memory corruption etc.

However, note that memory leaks are *not* UB -- you can definitely
leak memory in Rust.

Now, whether our abstractions should be thin and e.g. just call devres
primitives or instead write something else from scratch, is an open
question. The latter may lead to easier/faster code in the Rust side,
since it would give us more freedom. But, of course, we understand if
people prefer us to reuse as much C APIs as possible, like e.g. we did
with the red-black trees vs. using `alloc`'s `BTreeMap`.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier <roland@kernel.org> wrote:

> > "devres" / devm_xxx was an attempt to deal with this in C, but it only
> > solves some cases of this and has not received a lot of adoption (we
> > can argue about the reasons).

> Really? From my point of view that is adopted all over the map.
> I add new users all the time and use it as much as I can when
> writing new drivers.

Yes, it's *super* widely used in most of the kernel.  Perhaps there's
some subsystems that reject it for some reason.

> I think it's a formidable success, people just need to learn to do it more.

There *are* issues with people adopting it too enthusiastically - as
well as the memory lifetime issues that Laurent mentioned it's easy for
it to cause problems with interrupt handlers that are left live longer
than they should be and try to use things that were already deallocated.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [TECH TOPIC] Rust for Linux

[Julia Lawall]

On Wed, 7 Jul 2021, Mark Brown wrote:

> On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier <roland@kernel.org> wrote:
>
> > > "devres" / devm_xxx was an attempt to deal with this in C, but it only
> > > solves some cases of this and has not received a lot of adoption (we
> > > can argue about the reasons).
>
> > Really? From my point of view that is adopted all over the map.
> > I add new users all the time and use it as much as I can when
> > writing new drivers.
>
> Yes, it's *super* widely used in most of the kernel.  Perhaps there's
> some subsystems that reject it for some reason.
>
> > I think it's a formidable success, people just need to learn to do it more.
>
> There *are* issues with people adopting it too enthusiastically - as
> well as the memory lifetime issues that Laurent mentioned it's easy for
> it to cause problems with interrupt handlers that are left live longer
> than they should be and try to use things that were already deallocated.

I was also wondering what would be done with Rust in the case of this
issue.

julia

Re: [TECH TOPIC] Rust for Linux

[James Bottomley]

On Wed, 2021-07-07 at 12:56 +0200, Julia Lawall wrote:
> 
> On Wed, 7 Jul 2021, Mark Brown wrote:
> 
> > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier <roland@kernel.org>
> > > wrote:
> > > > "devres" / devm_xxx was an attempt to deal with this in C, but
> > > > it only solves some cases of this and has not received a lot of
> > > > adoption (we can argue about the reasons).
> > >  
> > > Really? From my point of view that is adopted all over the map.
> > > I add new users all the time and use it as much as I can when
> > > writing new drivers.
> > 
> > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > there's some subsystems that reject it for some reason.
> > 
> > > I think it's a formidable success, people just need to learn to
> > > do it more.
> > 
> > There *are* issues with people adopting it too enthusiastically -
> > as well as the memory lifetime issues that Laurent mentioned it's
> > easy for it to cause problems with interrupt handlers that are left
> > live longer than they should be and try to use things that were
> > already deallocated.
> 
> I was also wondering what would be done with Rust in the case of this
> issue.

Pretty much nothing.  Rust has a tracking system for exclusive memory
allocations, which is where it gets its memory safety guarantees. 
However, once you move the reference counting the guarantees become
much weaker.  The best it can do for us is the same as the current
kasan: detect use after free at runtime but not at compile time.

To have compile time tracking of reference counts, we'd have to encode
the lifetime rules semantically somehow and have a labelling mechanism
to encode the expectations in the consumer routines ... even if you put
a PhD student on it, I bet what we'd get back would be too complex to
be usable.

James

Re: [TECH TOPIC] Rust for Linux

[James Bottomley]

[-- Attachment #1: Type: text/plain, Size: 1535 bytes --]

On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:
> On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier <roland@kernel.org>
> > wrote:
> > > "devres" / devm_xxx was an attempt to deal with this in C, but it
> > > only solves some cases of this and has not received a lot of
> > > adoption (we can argue about the reasons).
> >  
> > Really? From my point of view that is adopted all over the map.
> > I add new users all the time and use it as much as I can when
> > writing new drivers.
> 
> Yes, it's *super* widely used in most of the kernel.  Perhaps there's
> some subsystems that reject it for some reason.
> 
> > I think it's a formidable success, people just need to learn to do
> > it more.
> 
> There *are* issues with people adopting it too enthusiastically - as
> well as the memory lifetime issues that Laurent mentioned it's easy
> for it to cause problems with interrupt handlers that are left live
> longer than they should be and try to use things that were already
> deallocated.

Fixing this should not be beyond the wit of humankind, though.  If we
delayed deallocation to module release, that would fix the interrupt
issue, wouldn't it?  Perhaps all devres memory for devices should live
until then anyway and thus be automatically deallocated instead of
needing an explicit free ... the problem with that being compiled in
devices currently optimize away the module refcounting, but that should
be fixable.

James


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:
> On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:
> > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier <roland@kernel.org>
> > > wrote:
> > > > "devres" / devm_xxx was an attempt to deal with this in C, but it
> > > > only solves some cases of this and has not received a lot of
> > > > adoption (we can argue about the reasons).
> > >  
> > > Really? From my point of view that is adopted all over the map.
> > > I add new users all the time and use it as much as I can when
> > > writing new drivers.
> > 
> > Yes, it's *super* widely used in most of the kernel.  Perhaps there's
> > some subsystems that reject it for some reason.
> > 
> > > I think it's a formidable success, people just need to learn to do
> > > it more.
> > 
> > There *are* issues with people adopting it too enthusiastically - as
> > well as the memory lifetime issues that Laurent mentioned it's easy
> > for it to cause problems with interrupt handlers that are left live
> > longer than they should be and try to use things that were already
> > deallocated.
> 
> Fixing this should not be beyond the wit of humankind, though.  If we
> delayed deallocation to module release, that would fix the interrupt
> issue, wouldn't it?  Perhaps all devres memory for devices should live
> until then anyway and thus be automatically deallocated instead of
> needing an explicit free ... the problem with that being compiled in
> devices currently optimize away the module refcounting, but that should
> be fixable.

module code lifespans are different than device structure lifespans,
it's when people get them confused, as here, that we have problems.

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[James Bottomley]

On Wed, 2021-07-07 at 14:20 +0200, Greg KH wrote:
> On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:
> > On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:
> > > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier <
> > > > roland@kernel.org>
> > > > wrote:
> > > > > "devres" / devm_xxx was an attempt to deal with this in C,
> > > > > but it only solves some cases of this and has not received a
> > > > > lot of adoption (we can argue about the reasons).
> > > >  
> > > > Really? From my point of view that is adopted all over the map.
> > > > I add new users all the time and use it as much as I can when
> > > > writing new drivers.
> > > 
> > > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > > there's some subsystems that reject it for some reason.
> > > 
> > > > I think it's a formidable success, people just need to learn to
> > > > do it more.
> > > 
> > > There *are* issues with people adopting it too enthusiastically -
> > > as well as the memory lifetime issues that Laurent mentioned it's
> > > easy for it to cause problems with interrupt handlers that are
> > > left live longer than they should be and try to use things that
> > > were already deallocated.
> > 
> > Fixing this should not be beyond the wit of humankind, though.  If
> > we delayed deallocation to module release, that would fix the
> > interrupt issue, wouldn't it?  Perhaps all devres memory for
> > devices should live until then anyway and thus be automatically
> > deallocated instead of needing an explicit free ... the problem
> > with that being compiled in devices currently optimize away the
> > module refcounting, but that should be fixable.
> 
> module code lifespans are different than device structure lifespans,
> it's when people get them confused, as here, that we have problems.

I'm not claiming they are.  What I am claiming is that module lifetime
must always encompass the device lifetimes.  Therefore, you can never
be incorrect by using a module lifetime for anything attached to a
device, just inefficient for using memory longer than potentially
needed.  However, in a lot of use cases, the device is created on
module init and destroyed on module exit, so the inefficiency is barely
noticeable.  The question I'm asking is shouldn't we optimize for this?
so let people allocate devm memory safe in the knowledge it will be
freed on module release?  It will certainly fix a lot of the use after
free reports and we can still keep devm_release around for cases where
devices really do appear and disappear multiple times over the life of
the module.

James

Re: [TECH TOPIC] Rust for Linux

[Greg KH]

On Wed, Jul 07, 2021 at 01:38:44PM +0100, James Bottomley wrote:
> On Wed, 2021-07-07 at 14:20 +0200, Greg KH wrote:
> > On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:
> > > On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:
> > > > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier <
> > > > > roland@kernel.org>
> > > > > wrote:
> > > > > > "devres" / devm_xxx was an attempt to deal with this in C,
> > > > > > but it only solves some cases of this and has not received a
> > > > > > lot of adoption (we can argue about the reasons).
> > > > >  
> > > > > Really? From my point of view that is adopted all over the map.
> > > > > I add new users all the time and use it as much as I can when
> > > > > writing new drivers.
> > > > 
> > > > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > > > there's some subsystems that reject it for some reason.
> > > > 
> > > > > I think it's a formidable success, people just need to learn to
> > > > > do it more.
> > > > 
> > > > There *are* issues with people adopting it too enthusiastically -
> > > > as well as the memory lifetime issues that Laurent mentioned it's
> > > > easy for it to cause problems with interrupt handlers that are
> > > > left live longer than they should be and try to use things that
> > > > were already deallocated.
> > > 
> > > Fixing this should not be beyond the wit of humankind, though.  If
> > > we delayed deallocation to module release, that would fix the
> > > interrupt issue, wouldn't it?  Perhaps all devres memory for
> > > devices should live until then anyway and thus be automatically
> > > deallocated instead of needing an explicit free ... the problem
> > > with that being compiled in devices currently optimize away the
> > > module refcounting, but that should be fixable.
> > 
> > module code lifespans are different than device structure lifespans,
> > it's when people get them confused, as here, that we have problems.
> 
> I'm not claiming they are.  What I am claiming is that module lifetime
> must always encompass the device lifetimes.  Therefore, you can never
> be incorrect by using a module lifetime for anything attached to a
> device, just inefficient for using memory longer than potentially
> needed.  However, in a lot of use cases, the device is created on
> module init and destroyed on module exit, so the inefficiency is barely
> noticeable.

In almost no use case is the device created on module init of a driver.
devices are created by busses, look at the USB code as an example.  The
usb bus creates the devices and then individual modules bind to that
device as needed.  If the device is removed from the system, wonderful,
the device is unbound, but the module is still loaded.  So if you really
wanted to, with your change, just do a insert/remove of a USB device a
few zillion times and then memory is all gone :(

> The question I'm asking is shouldn't we optimize for this?

No.

> so let people allocate devm memory safe in the knowledge it will be
> freed on module release?

No, see above.  Modules are never removed in a normal system.  devices
are.

And the drm developers have done great work in unwinding some of these
types of mistakes in their drivers, let's not go backwards please.

thanks,

greg k-h

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

On Wed, Jul 07, 2021 at 02:45:04PM +0200, Greg KH wrote:
> On Wed, Jul 07, 2021 at 01:38:44PM +0100, James Bottomley wrote:
> > On Wed, 2021-07-07 at 14:20 +0200, Greg KH wrote:
> > > On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:
> > > > On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:
> > > > > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > > > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:
> > > > > > > "devres" / devm_xxx was an attempt to deal with this in C,
> > > > > > > but it only solves some cases of this and has not received a
> > > > > > > lot of adoption (we can argue about the reasons).
> > > > > >  
> > > > > > Really? From my point of view that is adopted all over the map.
> > > > > > I add new users all the time and use it as much as I can when
> > > > > > writing new drivers.
> > > > > 
> > > > > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > > > > there's some subsystems that reject it for some reason.
> > > > > 
> > > > > > I think it's a formidable success, people just need to learn to
> > > > > > do it more.
> > > > > 
> > > > > There *are* issues with people adopting it too enthusiastically -
> > > > > as well as the memory lifetime issues that Laurent mentioned it's
> > > > > easy for it to cause problems with interrupt handlers that are
> > > > > left live longer than they should be and try to use things that
> > > > > were already deallocated.

(CC'ing Daniel Vetter as the author of the DRM devres-based resource
management)

I've given this lots of thoughts lately, in the context of V4L2, but it
should be roughly the same for character devices in general. Here's what
I think should be done for drivers that expose character devices.

- Drivers must stop allocating their internal data structure (the one
  they set as device data with dev_set_drvdata()) with devm_kzalloc().
  The data structure must instead be allocated with a plain kzalloc()
  and reference-counted.

  Most drivers will register a single character device using a
  subsystem-specific API (e.g. video_register_device() in V4L2). The
  subsystem needs to provide a .release() callback, called when the
  last reference to the character device is released. Drivers must
  implement this, and can simply free their internal data structure at
  this point.

  For drivers that register multiple character devices, or in general
  expose multiple interfaces to userspace or other parts of the kernel,
  the internal data structure must be properly reference-counted, with a
  reference released in each .release() callback. There may be ways to
  simplify this.

  This can be seen as going back to the pre-devm_kzalloc() era, but it's
  only about undoing a mistake that was done way too often (to be fair,
  many drivers used to just kfree() the data in the driver's .remove()
  operation, so it wasn't always a regression, only enshrining a
  preexisting bad practice).

  This is only part of the puzzle though. There's a remove/use race that
  still needs to be solved.

- In .remove(), drivers must inform the character device that new access
  from userspace are not allowed anymore. This would set a flag in
  struct cdev that would result in all new calls from userspace through
  file operations to be rejected (with -ENXIO for instance). This should
  be wrapped in subsystem-specific functions (e.g.
  video_device_disconnect() wrapping cdev_disconnect()). From now on, no
  new calls are possible, but existing calls may be in progress.

- Drivers then need to cancel all pending I/O and wait for completion.
  I/O (either direct memory-mapped I/O or through bus APIs, such as I2C
  or USB transactions) are not allowed anymore after .remove() returns.
  This will have the side effect of waking up userspace contexts that
  are waiting for I/O completion (through a blocking file I/O operation
  for instance). Existing calls from userspace will start completing.

  This is also a good place to start shutting down the device, for
  instance disabling interrupts.

- The next step is for drivers to wait until all calls from userspace
  complete. This should be done with a call count in struct cdev that is
  updated upon entry and exit of calls from userspace, and a wait queue
  to wait for that count to go to 0. This should be wrapped in
  subsystem-specific APIs. As the flag that indicates device removal is
  set, no new calls are allowed so the counter can only decrease, and as
  all pending I/O have terminated or have been cancelled, no pending
  calls should be blocked.

- Finally, drivers should unregister the character device, through the
  appropriate subsystem API.

  At this point, memory mappings and file handles referencing the device
  may still exist, but no file operation is in progress. The device is
  quiescent.

  Care needs to be taken in drivers and subsystems to not start any I/O
  operation when handling the file .release() operation or the
  destruction of memory mappings. Overall I don't expect much issues,
  but I'm sure some drivers do strange things in those code paths.

- When the least memory mapping is gone and the last file handle is
  closed, the subsystem will call the driver's .release() callback. At
  this point, the driver will perform the operations listed in the first
  item of this list.


The challenge here will be to make this as easy as possible for drivers,
to avoid risk of drivers getting it wrong. The DRM/KMS subsystem has
created a devres-based system to handle this, with the devres associated
with the drm_device (the abstraction of the cdevs exposed by DRM to
userspace), *not* the physical device. It has a drawback though, it
assumes that a DRM driver will only ever want to register a drm_device
and nothing else, and hardcodes that assumption in the way it releases
resources. That's fine for most DRM drivers, but if a driver was to
register a drm_device and something else (such as a V4L2 video_device,
an input device, ...), the DRM subsystem will get in the way.

I have two questions:

- Does the above make sense ?
- Assuming it does, how do we get from the current mess to a situation
  where writing a driver will be a pleasure, not a punishment ? :-)

> > > > Fixing this should not be beyond the wit of humankind, though.  If
> > > > we delayed deallocation to module release, that would fix the
> > > > interrupt issue, wouldn't it?  Perhaps all devres memory for
> > > > devices should live until then anyway and thus be automatically
> > > > deallocated instead of needing an explicit free ... the problem
> > > > with that being compiled in devices currently optimize away the
> > > > module refcounting, but that should be fixable.
> > > 
> > > module code lifespans are different than device structure lifespans,
> > > it's when people get them confused, as here, that we have problems.
> > 
> > I'm not claiming they are.  What I am claiming is that module lifetime
> > must always encompass the device lifetimes.  Therefore, you can never
> > be incorrect by using a module lifetime for anything attached to a
> > device, just inefficient for using memory longer than potentially
> > needed.  However, in a lot of use cases, the device is created on
> > module init and destroyed on module exit, so the inefficiency is barely
> > noticeable.
> 
> In almost no use case is the device created on module init of a driver.
> devices are created by busses, look at the USB code as an example.  The
> usb bus creates the devices and then individual modules bind to that
> device as needed.  If the device is removed from the system, wonderful,
> the device is unbound, but the module is still loaded.  So if you really
> wanted to, with your change, just do a insert/remove of a USB device a
> few zillion times and then memory is all gone :(
> 
> > The question I'm asking is shouldn't we optimize for this?
> 
> No.
> 
> > so let people allocate devm memory safe in the knowledge it will be
> > freed on module release?
> 
> No, see above.  Modules are never removed in a normal system.  devices
> are.
> 
> And the drm developers have done great work in unwinding some of these
> types of mistakes in their drivers, let's not go backwards please.

-- 
Regards,

Laurent Pinchart

cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Greg KH]

On Wed, Jul 07, 2021 at 08:17:08PM +0300, Laurent Pinchart wrote:
> On Wed, Jul 07, 2021 at 02:45:04PM +0200, Greg KH wrote:
> > On Wed, Jul 07, 2021 at 01:38:44PM +0100, James Bottomley wrote:
> > > On Wed, 2021-07-07 at 14:20 +0200, Greg KH wrote:
> > > > On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:
> > > > > On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:
> > > > > > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > > > > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:
> > > > > > > > "devres" / devm_xxx was an attempt to deal with this in C,
> > > > > > > > but it only solves some cases of this and has not received a
> > > > > > > > lot of adoption (we can argue about the reasons).
> > > > > > >  
> > > > > > > Really? From my point of view that is adopted all over the map.
> > > > > > > I add new users all the time and use it as much as I can when
> > > > > > > writing new drivers.
> > > > > > 
> > > > > > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > > > > > there's some subsystems that reject it for some reason.
> > > > > > 
> > > > > > > I think it's a formidable success, people just need to learn to
> > > > > > > do it more.
> > > > > > 
> > > > > > There *are* issues with people adopting it too enthusiastically -
> > > > > > as well as the memory lifetime issues that Laurent mentioned it's
> > > > > > easy for it to cause problems with interrupt handlers that are
> > > > > > left live longer than they should be and try to use things that
> > > > > > were already deallocated.
> 
> (CC'ing Daniel Vetter as the author of the DRM devres-based resource
> management)
> 
> I've given this lots of thoughts lately, in the context of V4L2, but it
> should be roughly the same for character devices in general. Here's what
> I think should be done for drivers that expose character devices.
> 
> - Drivers must stop allocating their internal data structure (the one
>   they set as device data with dev_set_drvdata()) with devm_kzalloc().
>   The data structure must instead be allocated with a plain kzalloc()
>   and reference-counted.
> 
>   Most drivers will register a single character device using a
>   subsystem-specific API (e.g. video_register_device() in V4L2). The
>   subsystem needs to provide a .release() callback, called when the
>   last reference to the character device is released. Drivers must
>   implement this, and can simply free their internal data structure at
>   this point.
> 
>   For drivers that register multiple character devices, or in general
>   expose multiple interfaces to userspace or other parts of the kernel,
>   the internal data structure must be properly reference-counted, with a
>   reference released in each .release() callback. There may be ways to
>   simplify this.
> 
>   This can be seen as going back to the pre-devm_kzalloc() era, but it's
>   only about undoing a mistake that was done way too often (to be fair,
>   many drivers used to just kfree() the data in the driver's .remove()
>   operation, so it wasn't always a regression, only enshrining a
>   preexisting bad practice).
> 
>   This is only part of the puzzle though. There's a remove/use race that
>   still needs to be solved.
> 
> - In .remove(), drivers must inform the character device that new access
>   from userspace are not allowed anymore. This would set a flag in
>   struct cdev that would result in all new calls from userspace through
>   file operations to be rejected (with -ENXIO for instance). This should
>   be wrapped in subsystem-specific functions (e.g.
>   video_device_disconnect() wrapping cdev_disconnect()). From now on, no
>   new calls are possible, but existing calls may be in progress.
> 
> - Drivers then need to cancel all pending I/O and wait for completion.
>   I/O (either direct memory-mapped I/O or through bus APIs, such as I2C
>   or USB transactions) are not allowed anymore after .remove() returns.
>   This will have the side effect of waking up userspace contexts that
>   are waiting for I/O completion (through a blocking file I/O operation
>   for instance). Existing calls from userspace will start completing.
> 
>   This is also a good place to start shutting down the device, for
>   instance disabling interrupts.
> 
> - The next step is for drivers to wait until all calls from userspace
>   complete. This should be done with a call count in struct cdev that is
>   updated upon entry and exit of calls from userspace, and a wait queue
>   to wait for that count to go to 0. This should be wrapped in
>   subsystem-specific APIs. As the flag that indicates device removal is
>   set, no new calls are allowed so the counter can only decrease, and as
>   all pending I/O have terminated or have been cancelled, no pending
>   calls should be blocked.
> 
> - Finally, drivers should unregister the character device, through the
>   appropriate subsystem API.
> 
>   At this point, memory mappings and file handles referencing the device
>   may still exist, but no file operation is in progress. The device is
>   quiescent.
> 
>   Care needs to be taken in drivers and subsystems to not start any I/O
>   operation when handling the file .release() operation or the
>   destruction of memory mappings. Overall I don't expect much issues,
>   but I'm sure some drivers do strange things in those code paths.
> 
> - When the least memory mapping is gone and the last file handle is
>   closed, the subsystem will call the driver's .release() callback. At
>   this point, the driver will perform the operations listed in the first
>   item of this list.
> 
> 
> The challenge here will be to make this as easy as possible for drivers,
> to avoid risk of drivers getting it wrong. The DRM/KMS subsystem has
> created a devres-based system to handle this, with the devres associated
> with the drm_device (the abstraction of the cdevs exposed by DRM to
> userspace), *not* the physical device. It has a drawback though, it
> assumes that a DRM driver will only ever want to register a drm_device
> and nothing else, and hardcodes that assumption in the way it releases
> resources. That's fine for most DRM drivers, but if a driver was to
> register a drm_device and something else (such as a V4L2 video_device,
> an input device, ...), the DRM subsystem will get in the way.
> 
> I have two questions:
> 
> - Does the above make sense ?

Yes, totally, thank you for taking the time to write this all out.  It's
been in the back of my mind for over a decade that we need to work on
these issues, but have not had any time to sit down and write it all
down like you have.

> - Assuming it does, how do we get from the current mess to a situation
>   where writing a driver will be a pleasure, not a punishment ? :-)

That's the real question.  Thanks to a lot of cleanups that Christoph
has recently done to the lower-level cdev code, the lower levels are now
in a shape where we can work on them better.  I'm going to try to carve
out some time in the next few months to start to work on these things.
I think that once we get the ideas of what needs to be done, and a
working core change, I can unleash some interns on doing tree-wide
cleanups/changes to help bring everything into alignment.

I'm going to save this email off for reference for later.  But of
course, if others want to start to attack this earlier, all the better :)

thanks,

greg k-h



> 
> > > > > Fixing this should not be beyond the wit of humankind, though.  If
> > > > > we delayed deallocation to module release, that would fix the
> > > > > interrupt issue, wouldn't it?  Perhaps all devres memory for
> > > > > devices should live until then anyway and thus be automatically
> > > > > deallocated instead of needing an explicit free ... the problem
> > > > > with that being compiled in devices currently optimize away the
> > > > > module refcounting, but that should be fixable.
> > > > 
> > > > module code lifespans are different than device structure lifespans,
> > > > it's when people get them confused, as here, that we have problems.
> > > 
> > > I'm not claiming they are.  What I am claiming is that module lifetime
> > > must always encompass the device lifetimes.  Therefore, you can never
> > > be incorrect by using a module lifetime for anything attached to a
> > > device, just inefficient for using memory longer than potentially
> > > needed.  However, in a lot of use cases, the device is created on
> > > module init and destroyed on module exit, so the inefficiency is barely
> > > noticeable.
> > 
> > In almost no use case is the device created on module init of a driver.
> > devices are created by busses, look at the USB code as an example.  The
> > usb bus creates the devices and then individual modules bind to that
> > device as needed.  If the device is removed from the system, wonderful,
> > the device is unbound, but the module is still loaded.  So if you really
> > wanted to, with your change, just do a insert/remove of a USB device a
> > few zillion times and then memory is all gone :(
> > 
> > > The question I'm asking is shouldn't we optimize for this?
> > 
> > No.
> > 
> > > so let people allocate devm memory safe in the knowledge it will be
> > > freed on module release?
> > 
> > No, see above.  Modules are never removed in a normal system.  devices
> > are.
> > 
> > And the drm developers have done great work in unwinding some of these
> > types of mistakes in their drivers, let's not go backwards please.
> 
> -- 
> Regards,
> 
> Laurent Pinchart

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Laurent Pinchart]

Hi Greg,

On Thu, Jul 08, 2021 at 08:49:39AM +0200, Greg KH wrote:
> On Wed, Jul 07, 2021 at 08:17:08PM +0300, Laurent Pinchart wrote:
> > On Wed, Jul 07, 2021 at 02:45:04PM +0200, Greg KH wrote:
> > > On Wed, Jul 07, 2021 at 01:38:44PM +0100, James Bottomley wrote:
> > > > On Wed, 2021-07-07 at 14:20 +0200, Greg KH wrote:
> > > > > On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:
> > > > > > On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:
> > > > > > > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > > > > > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:
> > > > > > > > > "devres" / devm_xxx was an attempt to deal with this in C,
> > > > > > > > > but it only solves some cases of this and has not received a
> > > > > > > > > lot of adoption (we can argue about the reasons).
> > > > > > > >  
> > > > > > > > Really? From my point of view that is adopted all over the map.
> > > > > > > > I add new users all the time and use it as much as I can when
> > > > > > > > writing new drivers.
> > > > > > > 
> > > > > > > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > > > > > > there's some subsystems that reject it for some reason.
> > > > > > > 
> > > > > > > > I think it's a formidable success, people just need to learn to
> > > > > > > > do it more.
> > > > > > > 
> > > > > > > There *are* issues with people adopting it too enthusiastically -
> > > > > > > as well as the memory lifetime issues that Laurent mentioned it's
> > > > > > > easy for it to cause problems with interrupt handlers that are
> > > > > > > left live longer than they should be and try to use things that
> > > > > > > were already deallocated.
> > 
> > (CC'ing Daniel Vetter as the author of the DRM devres-based resource
> > management)
> > 
> > I've given this lots of thoughts lately, in the context of V4L2, but it
> > should be roughly the same for character devices in general. Here's what
> > I think should be done for drivers that expose character devices.
> > 
> > - Drivers must stop allocating their internal data structure (the one
> >   they set as device data with dev_set_drvdata()) with devm_kzalloc().
> >   The data structure must instead be allocated with a plain kzalloc()
> >   and reference-counted.
> > 
> >   Most drivers will register a single character device using a
> >   subsystem-specific API (e.g. video_register_device() in V4L2). The
> >   subsystem needs to provide a .release() callback, called when the
> >   last reference to the character device is released. Drivers must
> >   implement this, and can simply free their internal data structure at
> >   this point.
> > 
> >   For drivers that register multiple character devices, or in general
> >   expose multiple interfaces to userspace or other parts of the kernel,
> >   the internal data structure must be properly reference-counted, with a
> >   reference released in each .release() callback. There may be ways to
> >   simplify this.
> > 
> >   This can be seen as going back to the pre-devm_kzalloc() era, but it's
> >   only about undoing a mistake that was done way too often (to be fair,
> >   many drivers used to just kfree() the data in the driver's .remove()
> >   operation, so it wasn't always a regression, only enshrining a
> >   preexisting bad practice).
> > 
> >   This is only part of the puzzle though. There's a remove/use race that
> >   still needs to be solved.
> > 
> > - In .remove(), drivers must inform the character device that new access
> >   from userspace are not allowed anymore. This would set a flag in
> >   struct cdev that would result in all new calls from userspace through
> >   file operations to be rejected (with -ENXIO for instance). This should
> >   be wrapped in subsystem-specific functions (e.g.
> >   video_device_disconnect() wrapping cdev_disconnect()). From now on, no
> >   new calls are possible, but existing calls may be in progress.
> > 
> > - Drivers then need to cancel all pending I/O and wait for completion.
> >   I/O (either direct memory-mapped I/O or through bus APIs, such as I2C
> >   or USB transactions) are not allowed anymore after .remove() returns.
> >   This will have the side effect of waking up userspace contexts that
> >   are waiting for I/O completion (through a blocking file I/O operation
> >   for instance). Existing calls from userspace will start completing.
> > 
> >   This is also a good place to start shutting down the device, for
> >   instance disabling interrupts.
> > 
> > - The next step is for drivers to wait until all calls from userspace
> >   complete. This should be done with a call count in struct cdev that is
> >   updated upon entry and exit of calls from userspace, and a wait queue
> >   to wait for that count to go to 0. This should be wrapped in
> >   subsystem-specific APIs. As the flag that indicates device removal is
> >   set, no new calls are allowed so the counter can only decrease, and as
> >   all pending I/O have terminated or have been cancelled, no pending
> >   calls should be blocked.
> > 
> > - Finally, drivers should unregister the character device, through the
> >   appropriate subsystem API.
> > 
> >   At this point, memory mappings and file handles referencing the device
> >   may still exist, but no file operation is in progress. The device is
> >   quiescent.
> > 
> >   Care needs to be taken in drivers and subsystems to not start any I/O
> >   operation when handling the file .release() operation or the
> >   destruction of memory mappings. Overall I don't expect much issues,
> >   but I'm sure some drivers do strange things in those code paths.
> > 
> > - When the least memory mapping is gone and the last file handle is
> >   closed, the subsystem will call the driver's .release() callback. At
> >   this point, the driver will perform the operations listed in the first
> >   item of this list.
> > 
> > 
> > The challenge here will be to make this as easy as possible for drivers,
> > to avoid risk of drivers getting it wrong. The DRM/KMS subsystem has
> > created a devres-based system to handle this, with the devres associated
> > with the drm_device (the abstraction of the cdevs exposed by DRM to
> > userspace), *not* the physical device. It has a drawback though, it
> > assumes that a DRM driver will only ever want to register a drm_device
> > and nothing else, and hardcodes that assumption in the way it releases
> > resources. That's fine for most DRM drivers, but if a driver was to
> > register a drm_device and something else (such as a V4L2 video_device,
> > an input device, ...), the DRM subsystem will get in the way.
> > 
> > I have two questions:
> > 
> > - Does the above make sense ?
> 
> Yes, totally, thank you for taking the time to write this all out.  It's
> been in the back of my mind for over a decade that we need to work on
> these issues, but have not had any time to sit down and write it all
> down like you have.
> 
> > - Assuming it does, how do we get from the current mess to a situation
> >   where writing a driver will be a pleasure, not a punishment ? :-)
> 
> That's the real question.  Thanks to a lot of cleanups that Christoph
> has recently done to the lower-level cdev code, the lower levels are now
> in a shape where we can work on them better.  I'm going to try to carve
> out some time in the next few months to start to work on these things.
> I think that once we get the ideas of what needs to be done, and a
> working core change, I can unleash some interns on doing tree-wide
> cleanups/changes to help bring everything into alignment.
> 
> I'm going to save this email off for reference for later.  But of
> course, if others want to start to attack this earlier, all the better :)

My too long todo list contains an entry to address this in the V4L2
subsystem to start with (see
https://lore.kernel.org/linux-media/20200816003315.GA13826@roeck-us.net/).
It may be a good prototype to then extend it to cdev. If I get a chance
to work on it, I'll CC you.

> > > > > > Fixing this should not be beyond the wit of humankind, though.  If
> > > > > > we delayed deallocation to module release, that would fix the
> > > > > > interrupt issue, wouldn't it?  Perhaps all devres memory for
> > > > > > devices should live until then anyway and thus be automatically
> > > > > > deallocated instead of needing an explicit free ... the problem
> > > > > > with that being compiled in devices currently optimize away the
> > > > > > module refcounting, but that should be fixable.
> > > > > 
> > > > > module code lifespans are different than device structure lifespans,
> > > > > it's when people get them confused, as here, that we have problems.
> > > > 
> > > > I'm not claiming they are.  What I am claiming is that module lifetime
> > > > must always encompass the device lifetimes.  Therefore, you can never
> > > > be incorrect by using a module lifetime for anything attached to a
> > > > device, just inefficient for using memory longer than potentially
> > > > needed.  However, in a lot of use cases, the device is created on
> > > > module init and destroyed on module exit, so the inefficiency is barely
> > > > noticeable.
> > > 
> > > In almost no use case is the device created on module init of a driver.
> > > devices are created by busses, look at the USB code as an example.  The
> > > usb bus creates the devices and then individual modules bind to that
> > > device as needed.  If the device is removed from the system, wonderful,
> > > the device is unbound, but the module is still loaded.  So if you really
> > > wanted to, with your change, just do a insert/remove of a USB device a
> > > few zillion times and then memory is all gone :(
> > > 
> > > > The question I'm asking is shouldn't we optimize for this?
> > > 
> > > No.
> > > 
> > > > so let people allocate devm memory safe in the knowledge it will be
> > > > freed on module release?
> > > 
> > > No, see above.  Modules are never removed in a normal system.  devices
> > > are.
> > > 
> > > And the drm developers have done great work in unwinding some of these
> > > types of mistakes in their drivers, let's not go backwards please.

-- 
Regards,

Laurent Pinchart

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Linus Walleij]

Thanks Laurent, Greg, this is important.

Replying here to get this in the right thread.

> On Wed, Jul 07, 2021 at 08:17:08PM +0300, Laurent Pinchart wrote:

> >  Here's what
> > I think should be done for drivers that expose character devices.
> >
> > - Drivers must stop allocating their internal data structure (the one
> >   they set as device data with dev_set_drvdata()) with devm_kzalloc().
> >   The data structure must instead be allocated with a plain kzalloc()
> >   and reference-counted.
> >
> >   Most drivers will register a single character device using a
> >   subsystem-specific API (e.g. video_register_device() in V4L2). The
> >   subsystem needs to provide a .release() callback, called when the
> >   last reference to the character device is released. Drivers must
> >   implement this, and can simply free their internal data structure at
> >   this point.

What we did in GPIO to get around the whole issue is to split
our device in two abstractions. The device that spawns a GPIO
driver can be a GPIO on any bus (platform, PCI, SPI, I2C, ...)
then that populates and registers one (or more) gpio_chips.
That is a struct that deal with handling the hardware but does
not contain any struct device.

When registering gpio_chip we create a struct gpio_device which
contains a struct device and a struct cdev (exposed to userspace)
and all in-kernel consumer handles (called struct gpio_desc). This
is reference counted on the struct device and uses
->release() to eventually clean itself up.

The crucial part is what happens when a device with GPIOs
disappears, if e.g. the USB device is unplugged or the driver
is rmmod:ed by force from the command line. We then unregister
the struct gpio_chip and drop all devm_*  resources taken by the
driver (referencing the struct dev in the USB device or so) so these
go away, but the struct gpio_device stays around
until the last reference from userspace is dropped.

In order to not crash calls from the character device the device is
"numbed", so any calls will just return "OK" but nothing happens.
We then hope userspace will be so nice to terminate once it realizes
that it is no longer needed, closing the chardev and releasing the
resources held.

This works for us but I admit it is a bit kludgy. I guess it is not a
generally useful practice, we just had to come up with something.

Yours,
Linus Walleij

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Dan Williams]

On Thu, Jul 8, 2021 at 4:07 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> Thanks Laurent, Greg, this is important.
>
> Replying here to get this in the right thread.
>
> > On Wed, Jul 07, 2021 at 08:17:08PM +0300, Laurent Pinchart wrote:
>
> > >  Here's what
> > > I think should be done for drivers that expose character devices.
> > >
> > > - Drivers must stop allocating their internal data structure (the one
> > >   they set as device data with dev_set_drvdata()) with devm_kzalloc().
> > >   The data structure must instead be allocated with a plain kzalloc()
> > >   and reference-counted.
> > >
> > >   Most drivers will register a single character device using a
> > >   subsystem-specific API (e.g. video_register_device() in V4L2). The
> > >   subsystem needs to provide a .release() callback, called when the
> > >   last reference to the character device is released. Drivers must
> > >   implement this, and can simply free their internal data structure at
> > >   this point.
>
> What we did in GPIO to get around the whole issue is to split
> our device in two abstractions. The device that spawns a GPIO
> driver can be a GPIO on any bus (platform, PCI, SPI, I2C, ...)
> then that populates and registers one (or more) gpio_chips.
> That is a struct that deal with handling the hardware but does
> not contain any struct device.
>
> When registering gpio_chip we create a struct gpio_device which
> contains a struct device and a struct cdev (exposed to userspace)
> and all in-kernel consumer handles (called struct gpio_desc). This
> is reference counted on the struct device and uses
> ->release() to eventually clean itself up.
>
> The crucial part is what happens when a device with GPIOs
> disappears, if e.g. the USB device is unplugged or the driver
> is rmmod:ed by force from the command line. We then unregister
> the struct gpio_chip and drop all devm_*  resources taken by the
> driver (referencing the struct dev in the USB device or so) so these
> go away, but the struct gpio_device stays around
> until the last reference from userspace is dropped.
>
> In order to not crash calls from the character device the device is
> "numbed", so any calls will just return "OK" but nothing happens.
> We then hope userspace will be so nice to terminate once it realizes
> that it is no longer needed, closing the chardev and releasing the
> resources held.
>
> This works for us but I admit it is a bit kludgy. I guess it is not a
> generally useful practice, we just had to come up with something.

We do something similar in the CXL driver. The driver data is
allocated with devm. When ->remove() is triggered the driver data is
disconnected from the cdev (because it's about to be freed by devres)
and all future ioctls attempts fail until userspace finally gives up
and closes the device-file to release the cdev.

I clumsily attempted to solve this problem in a generic way here:

https://lore.kernel.org/r/CAPcyv4hEpdh_aGcs_73w5KmYWdvR29KB2M2-NNXsaXwxf35Hwg@mail.gmail.com

...Christoph rightly pointed out that this is something debugfs
already handles with its file_operations proxy implementation. I'm
thinking that concept can be extended for drivers to deploy their
ioctl implementation behind a proxy that handles the ioctl shutdown
synchronization in the core and not have every driver open code it.
See fs/debugfs/file.c::FULL_PROXY_FUNC().

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Wedson Almeida Filho]

On Fri, Jul 09, 2021 at 01:06:22AM +0200, Linus Walleij wrote:
> The crucial part is what happens when a device with GPIOs
> disappears, if e.g. the USB device is unplugged or the driver
> is rmmod:ed by force from the command line. We then unregister
> the struct gpio_chip and drop all devm_*  resources taken by the
> driver (referencing the struct dev in the USB device or so) so these
> go away, but the struct gpio_device stays around
> until the last reference from userspace is dropped.
> 
> In order to not crash calls from the character device the device is
> "numbed", so any calls will just return "OK" but nothing happens.
> We then hope userspace will be so nice to terminate once it realizes
> that it is no longer needed, closing the chardev and releasing the
> resources held.

In preparation for writing the abstractions to implement a gpio driver in Rust,
I was reading through some of the code you describe above.

Unless I'm missing something (very much possible!), this "numbing" seems to not
be synchronised, that is, there are still race windows when userspace may cause
UAFs in the kernel.

For example, gpiochip_remove sets gdev->chip to NULL; gpio_ioctl returns -ENODEV
if gdev->chip is NULL, which I believe is an instance of what you describe
above. However, what ensures that it remains non-null? I see that in functions
called by gpio_ioctl (e.g., lineinfo_get), gdev->chip is used as if it were
guaranteed to be valid.

Is my reading correct or is there some synchronisation that I'm missing?

Thanks,
-Wedson

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Linus Walleij]

On Fri, Jul 9, 2021 at 6:54 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:

> In preparation for writing the abstractions to implement a gpio driver in Rust,
> I was reading through some of the code you describe above.

Nice, bonus review :)

> Unless I'm missing something (very much possible!), this "numbing" seems to not
> be synchronised, that is, there are still race windows when userspace may cause
> UAFs in the kernel.

That's possible.

> For example, gpiochip_remove sets gdev->chip to NULL; gpio_ioctl returns -ENODEV
> if gdev->chip is NULL, which I believe is an instance of what you describe
> above.

Yes.

> However, what ensures that it remains non-null?
(...)
> I see that in functions
> called by gpio_ioctl (e.g., lineinfo_get), gdev->chip is used as if it were
> guaranteed to be valid.
(...)
> Is my reading correct or is there some synchronisation that I'm missing?

No there are definately possible synchronization bugs there.

We probably need a few more
if (!gdev->chip) return -ENODEV;
in some of these callbacks for example.
There are probably also more narrow possible sync bugs.

They are a bit hard to reproduce in practice because people do not
unplug their GPIO devices so much, the one case that is used a bit
would be USB-based GPIO expanders which happens on e.g.
serial dongles (FTDI with additional GPIO is the most common).
These are used in practice for controlling lab boards and stuff
but when people unplug them it is usually part of tearing down an
entire setup so the circumstances are a bit chaotic and subtle
bugs are not noticed.

Yours,
Linus Walleij

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Linus Walleij]

On Fri, Jul 9, 2021 at 6:54 PM Wedson Almeida Filho <wedsonaf@google.com> wrote:

> In preparation for writing the abstractions to implement a gpio driver in Rust,
> I was reading through some of the code you describe above.

Nice, bonus review :)

> Unless I'm missing something (very much possible!), this "numbing" seems to not
> be synchronised, that is, there are still race windows when userspace may cause
> UAFs in the kernel.

That's possible.

> For example, gpiochip_remove sets gdev->chip to NULL; gpio_ioctl returns -ENODEV
> if gdev->chip is NULL, which I believe is an instance of what you describe
> above.

Yes.

> However, what ensures that it remains non-null?
(...)
> I see that in functions
> called by gpio_ioctl (e.g., lineinfo_get), gdev->chip is used as if it were
> guaranteed to be valid.
(...)
> Is my reading correct or is there some synchronisation that I'm missing?

No there are definately possible synchronization bugs there.

We probably need a few more
if (!gdev->chip) return -ENODEV;
in some of these callbacks for example.
There are probably also more narrow possible sync bugs.

They are a bit hard to reproduce in practice because people do not
unplug their GPIO devices so much, the one case that is used a bit
would be USB-based GPIO expanders which happens on e.g.
serial dongles (FTDI with additional GPIO is the most common).
These are used in practice for controlling lab boards and stuff
but when people unplug them it is usually part of tearing down an
entire setup so the circumstances are a bit chaotic and subtle
bugs are not noticed.

Yours,
Linus Walleij

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Linus Walleij]

On Wed, Jul 14, 2021 at 12:35 AM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:

> To me described scenario sounds rather like an object lifetime possible issue.
> In any case, shouldn’t VFS guarantee by a reference counting that
> gpiochip_remove() wouldn’t be called while file descriptor is in use?
> Or am I looking from the wrong end here?

What happens is that the GPIO device disappears (such as unplugging
a USB GPIO expander) while a multithreaded userspace is hammering
exotic ioctl() commands to the same device like crazy.

Under these circumstances (which should be rare, but you know,
developers) it could happen that an ioctl() sneak in before the
gpio_chip pointer is NULL if I read the code right.

Yours,
Linus Walleij

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Linus Walleij]

On Wed, Jul 14, 2021 at 12:35 AM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:

> To me described scenario sounds rather like an object lifetime possible issue.
> In any case, shouldn’t VFS guarantee by a reference counting that
> gpiochip_remove() wouldn’t be called while file descriptor is in use?
> Or am I looking from the wrong end here?

What happens is that the GPIO device disappears (such as unplugging
a USB GPIO expander) while a multithreaded userspace is hammering
exotic ioctl() commands to the same device like crazy.

Under these circumstances (which should be rare, but you know,
developers) it could happen that an ioctl() sneak in before the
gpio_chip pointer is NULL if I read the code right.

Yours,
Linus Walleij

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Andy Shevchenko]

On Wed, Jul 21, 2021 at 4:46 PM Linus Walleij <linus.walleij@linaro.org> wrote:
> On Wed, Jul 14, 2021 at 12:35 AM Andy Shevchenko
> <andy.shevchenko@gmail.com> wrote:
>
> > To me described scenario sounds rather like an object lifetime possible issue.
> > In any case, shouldn’t VFS guarantee by a reference counting that
> > gpiochip_remove() wouldn’t be called while file descriptor is in use?
> > Or am I looking from the wrong end here?
>
> What happens is that the GPIO device disappears (such as unplugging
> a USB GPIO expander) while a multithreaded userspace is hammering
> exotic ioctl() commands to the same device like crazy.
>
> Under these circumstances (which should be rare, but you know,
> developers) it could happen that an ioctl() sneak in before the
> gpio_chip pointer is NULL if I read the code right.

So, gpio_chip is NULL but gpiodev is not NULL, correct?
If so, it means that the above mentioned scenario applies to the
latter one and I understand the checks.

-- 
With Best Regards,
Andy Shevchenko

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Andy Shevchenko]

On Wed, Jul 21, 2021 at 4:46 PM Linus Walleij <linus.walleij@linaro.org> wrote:
> On Wed, Jul 14, 2021 at 12:35 AM Andy Shevchenko
> <andy.shevchenko@gmail.com> wrote:
>
> > To me described scenario sounds rather like an object lifetime possible issue.
> > In any case, shouldn’t VFS guarantee by a reference counting that
> > gpiochip_remove() wouldn’t be called while file descriptor is in use?
> > Or am I looking from the wrong end here?
>
> What happens is that the GPIO device disappears (such as unplugging
> a USB GPIO expander) while a multithreaded userspace is hammering
> exotic ioctl() commands to the same device like crazy.
>
> Under these circumstances (which should be rare, but you know,
> developers) it could happen that an ioctl() sneak in before the
> gpio_chip pointer is NULL if I read the code right.

So, gpio_chip is NULL but gpiodev is not NULL, correct?
If so, it means that the above mentioned scenario applies to the
latter one and I understand the checks.

-- 
With Best Regards,
Andy Shevchenko

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Dan Carpenter]

The other thing which is quite tricky to get right is kobj releases.  I
have a bunch of these static checker warning but they're annoying
because they're not something which really affects real life so I can't
be bothered to fix or report them.

Also some of them are just unfixable, for example
__video_register_device().

drivers/media/v4l2-core/v4l2-dev.c
  1031          /* Part 4: register the device with sysfs */
  1032          vdev->dev.class = &video_class;
  1033          vdev->dev.devt = MKDEV(VIDEO_MAJOR, vdev->minor);
  1034          vdev->dev.parent = vdev->dev_parent;
  1035          dev_set_name(&vdev->dev, "%s%d", name_base, vdev->num);
  1036          ret = device_register(&vdev->dev);

If device_register() fails then it's illegal to manually clean up.

  1037          if (ret < 0) {
  1038                  pr_err("%s: device_register failed\n", __func__);
  1039                  goto cleanup;
                        ^^^^^^^^^^^^^
But this does, but it turns out it's fine.

  1040          }
  1041          /* Register the release callback that will be called when the last
  1042             reference to the device goes away. */
  1043          vdev->dev.release = v4l2_device_release;

The ->release() function is not set until here so it's just going to
trigger the debug message in kobject_cleanup() about a missing
->release() function.  So no problems.

  1044  
  1045          if (nr != -1 && nr != vdev->num && warn_if_nr_in_use)
  1046                  pr_warn("%s: requested %s%d, got %s\n", __func__,
  1047                          name_base, nr, video_device_node_name(vdev));
  1048  
  1049          /* Increase v4l2_device refcount */
  1050          v4l2_device_get(vdev->v4l2_dev);
  1051  
  1052          /* Part 5: Register the entity. */
  1053          ret = video_register_media_controller(vdev);
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
But then there is no way to clean up from this is
video_register_media_controller() fails.  If we call put_device() it
would lead to use after frees in the callers.  We just have to ignore
the error.

It's a minor thing, but it's so frustrating.

  1054  
  1055          /* Part 6: Activate this minor. The char device can now be used. */
  1056          set_bit(V4L2_FL_REGISTERED, &vdev->flags);
  1057  
  1058          return 0;

regards,
dan carpenter

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Jason Gunthorpe]

On Sat, Jul 10, 2021 at 10:09:10AM +0300, Dan Carpenter wrote:
> The other thing which is quite tricky to get right is kobj releases.  I
> have a bunch of these static checker warning but they're annoying
> because they're not something which really affects real life so I can't
> be bothered to fix or report them.

I'm not a security researcher, but I often wonder if there are ways
root could exploit these kinds of error unwind bugs during module
loading to exploit the kernel? That is a relevant concern in the
higher integrity modes.

>   1045          if (nr != -1 && nr != vdev->num && warn_if_nr_in_use)
>   1046                  pr_warn("%s: requested %s%d, got %s\n", __func__,
>   1047                          name_base, nr, video_device_node_name(vdev));
>   1048  
>   1049          /* Increase v4l2_device refcount */
>   1050          v4l2_device_get(vdev->v4l2_dev);
>   1051  
>   1052          /* Part 5: Register the entity. */
>   1053          ret = video_register_media_controller(vdev);
>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> But then there is no way to clean up from this is
> video_register_media_controller() fails.  If we call put_device() it
> would lead to use after frees in the callers.  We just have to ignore
> the error.

Eg if we ignore these errors and keep going is the kernel object in a
bad state that userspace can exploit?

The solution here is the same as the other cases, use device_add() not
device_register(), put a device_initialize() immediately after the
allocation of vdev and audit/fix the release so it can work on
partially initialized objects.

The idea of a video_register_device() is just not right in a subsystem
so complicated the 'init & register' pattern is a shortcut to save a
few lines in simple drivers. It should never be baked into a subsystem
like this and it certainly shouldn't work the opposide of how
device_register() works: "if video_register_device fails, the
release() callback of &struct video_device structure is *not* called"

Which is why the release function assignment is out of order, and it
is forced to print a warning in a certain error flow :\

> It's a minor thing, but it's so frustrating.

Yes :(

And as others mentioned devm has its own set of bug classes - I've
seen issues with the sequencing of the devm unloads.. Especially when
work queues get involved.

Jason

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Daniel Vetter]

On Thu, Jul 8, 2021 at 8:49 AM Greg KH <greg@kroah.com> wrote:
> On Wed, Jul 07, 2021 at 08:17:08PM +0300, Laurent Pinchart wrote:
> > On Wed, Jul 07, 2021 at 02:45:04PM +0200, Greg KH wrote:
> > > On Wed, Jul 07, 2021 at 01:38:44PM +0100, James Bottomley wrote:
> > > > On Wed, 2021-07-07 at 14:20 +0200, Greg KH wrote:
> > > > > On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:
> > > > > > On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:
> > > > > > > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:
> > > > > > > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:
> > > > > > > > > "devres" / devm_xxx was an attempt to deal with this in C,
> > > > > > > > > but it only solves some cases of this and has not received a
> > > > > > > > > lot of adoption (we can argue about the reasons).
> > > > > > > >
> > > > > > > > Really? From my point of view that is adopted all over the map.
> > > > > > > > I add new users all the time and use it as much as I can when
> > > > > > > > writing new drivers.
> > > > > > >
> > > > > > > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > > > > > > there's some subsystems that reject it for some reason.
> > > > > > >
> > > > > > > > I think it's a formidable success, people just need to learn to
> > > > > > > > do it more.
> > > > > > >
> > > > > > > There *are* issues with people adopting it too enthusiastically -
> > > > > > > as well as the memory lifetime issues that Laurent mentioned it's
> > > > > > > easy for it to cause problems with interrupt handlers that are
> > > > > > > left live longer than they should be and try to use things that
> > > > > > > were already deallocated.
> >
> > (CC'ing Daniel Vetter as the author of the DRM devres-based resource
> > management)
> >
> > I've given this lots of thoughts lately, in the context of V4L2, but it
> > should be roughly the same for character devices in general. Here's what
> > I think should be done for drivers that expose character devices.
> >
> > - Drivers must stop allocating their internal data structure (the one
> >   they set as device data with dev_set_drvdata()) with devm_kzalloc().
> >   The data structure must instead be allocated with a plain kzalloc()
> >   and reference-counted.
> >
> >   Most drivers will register a single character device using a
> >   subsystem-specific API (e.g. video_register_device() in V4L2). The
> >   subsystem needs to provide a .release() callback, called when the
> >   last reference to the character device is released. Drivers must
> >   implement this, and can simply free their internal data structure at
> >   this point.
> >
> >   For drivers that register multiple character devices, or in general
> >   expose multiple interfaces to userspace or other parts of the kernel,
> >   the internal data structure must be properly reference-counted, with a
> >   reference released in each .release() callback. There may be ways to
> >   simplify this.
> >
> >   This can be seen as going back to the pre-devm_kzalloc() era, but it's
> >   only about undoing a mistake that was done way too often (to be fair,
> >   many drivers used to just kfree() the data in the driver's .remove()
> >   operation, so it wasn't always a regression, only enshrining a
> >   preexisting bad practice).
> >
> >   This is only part of the puzzle though. There's a remove/use race that
> >   still needs to be solved.
> >
> > - In .remove(), drivers must inform the character device that new access
> >   from userspace are not allowed anymore. This would set a flag in
> >   struct cdev that would result in all new calls from userspace through
> >   file operations to be rejected (with -ENXIO for instance). This should
> >   be wrapped in subsystem-specific functions (e.g.
> >   video_device_disconnect() wrapping cdev_disconnect()). From now on, no
> >   new calls are possible, but existing calls may be in progress.
> >
> > - Drivers then need to cancel all pending I/O and wait for completion.
> >   I/O (either direct memory-mapped I/O or through bus APIs, such as I2C
> >   or USB transactions) are not allowed anymore after .remove() returns.
> >   This will have the side effect of waking up userspace contexts that
> >   are waiting for I/O completion (through a blocking file I/O operation
> >   for instance). Existing calls from userspace will start completing.
> >
> >   This is also a good place to start shutting down the device, for
> >   instance disabling interrupts.
> >
> > - The next step is for drivers to wait until all calls from userspace
> >   complete. This should be done with a call count in struct cdev that is
> >   updated upon entry and exit of calls from userspace, and a wait queue
> >   to wait for that count to go to 0. This should be wrapped in
> >   subsystem-specific APIs. As the flag that indicates device removal is
> >   set, no new calls are allowed so the counter can only decrease, and as
> >   all pending I/O have terminated or have been cancelled, no pending
> >   calls should be blocked.
> >
> > - Finally, drivers should unregister the character device, through the
> >   appropriate subsystem API.
> >
> >   At this point, memory mappings and file handles referencing the device
> >   may still exist, but no file operation is in progress. The device is
> >   quiescent.
> >
> >   Care needs to be taken in drivers and subsystems to not start any I/O
> >   operation when handling the file .release() operation or the
> >   destruction of memory mappings. Overall I don't expect much issues,
> >   but I'm sure some drivers do strange things in those code paths.
> >
> > - When the least memory mapping is gone and the last file handle is
> >   closed, the subsystem will call the driver's .release() callback. At
> >   this point, the driver will perform the operations listed in the first
> >   item of this list.
> >
> >
> > The challenge here will be to make this as easy as possible for drivers,
> > to avoid risk of drivers getting it wrong. The DRM/KMS subsystem has
> > created a devres-based system to handle this, with the devres associated
> > with the drm_device (the abstraction of the cdevs exposed by DRM to
> > userspace), *not* the physical device. It has a drawback though, it
> > assumes that a DRM driver will only ever want to register a drm_device
> > and nothing else, and hardcodes that assumption in the way it releases
> > resources. That's fine for most DRM drivers, but if a driver was to
> > register a drm_device and something else (such as a V4L2 video_device,
> > an input device, ...), the DRM subsystem will get in the way.
> >
> > I have two questions:
> >
> > - Does the above make sense ?
>
> Yes, totally, thank you for taking the time to write this all out.  It's
> been in the back of my mind for over a decade that we need to work on
> these issues, but have not had any time to sit down and write it all
> down like you have.
>
> > - Assuming it does, how do we get from the current mess to a situation
> >   where writing a driver will be a pleasure, not a punishment ? :-)
>
> That's the real question.  Thanks to a lot of cleanups that Christoph
> has recently done to the lower-level cdev code, the lower levels are now
> in a shape where we can work on them better.  I'm going to try to carve
> out some time in the next few months to start to work on these things.
> I think that once we get the ideas of what needs to be done, and a
> working core change, I can unleash some interns on doing tree-wide
> cleanups/changes to help bring everything into alignment.
>
> I'm going to save this email off for reference for later.  But of
> course, if others want to start to attack this earlier, all the better :)

Since we're dropping notes, a few of my thoughts:

- Personally I think an uapi resource cleanup system needs a different
namespace, so that you don't mix these up. Hence drmm_ vs devres_ for
drm, and maybe the generic version could be called uapires or ures or
whatever. It needs to stick out.

- I'm wondering whether we could enlist checkers (maybe a runtime one)
to scream anytime we do an unprotected dereference from ures memory to
devres memory. This would help in subsystem where this problem is
solved by trying to decouple the uapi side from the device side (like
gpio and other subsystem with simpler interfaces). We have undefined
behaviour and data race checkers already, this should be doable. But I
have no idea how :-)

- Core infrastructure like cdev_disconnect sounds really good.
Especially if someone figures out how to punch out mmap without races
in a generic way.

- Many drivers are perfectly ok with their ures on the single cdev
they expose, but like Laurent points out, there's plenty where you
need to group them. We need some way to merge ures groups together so
a driver can say "only release ures for _any_ of these cdev if _all_
of them are released". I'm not sure how to do that, but can also be
done as an additional later on. Maybe an explicit struct ures_group
that complex drivers (or complex subsystems like drm/v4l) can
optionally pass to subordinate uapi interfaces like cdev could make
this work.

- Another good reason for a commont struct to manage uapi resources is
to that we don't need a per-subsystem dupe for everything. Once your
driver is more than a few lines you need more than just drmm_kzalloc,
you might have you own slab, other allocater, a few workqueues,
whatever else there is ....

- Automagic cleanup is great for small drivers, but is hitting a bit a
scaling wall for big drivers. The trouble there is the unwind code to
quiescent the driver with all it's kthread, work queues, workers and
everything else. That needs to happen in a very careful order (if you
flush the work before you disable the interrupt that schedules it,
it's no good) and is an absolute pain to validate. There's a few
reasons why we have barriers here:
* often the same or similar quiescent code is needed for
suspend/resume, so you're not gaining anything if the module unload is
automatic
* unwind as you create/init it not always the right thing to do, or at
least not obviously so, e.g. in i915 we have on interrupt handler, but
it's hiearchical internally, and we arm new subsystems irq sources as
we initialize these parts of the driver
* as soon as you hit something where there's not yet a devres/uapi
wrapper available it gets annoying and giving up is easier :-)

- the real cool stuff starts to happen if you combine devres with
ures, e.g. see devm_drm_dev_alloc(). With that you can achieve drivers
with no ->remove callback that actually get all the lifetime right,
unfortunately we're not there yet, we're missing a
devm_drm_dev_register (to call drm_dev_unplug() outmatically on
remove) and a devm_drm_atomic_reset (which calls drm_atomic_shutdown
on remove) so that for simplest drivers the ->remove hook becomes
empty (e.g. hx8357d_remove()). But you really need the ures side fully
rolled out first or things just go very wrong on hotunplug :-)

Anyway I'm very much interested in this topic and seeing some kind of
solution lifted to core code.

Cheers, Daniel

> thanks,
>
> greg k-h
>
>
>
> >
> > > > > > Fixing this should not be beyond the wit of humankind, though.  If
> > > > > > we delayed deallocation to module release, that would fix the
> > > > > > interrupt issue, wouldn't it?  Perhaps all devres memory for
> > > > > > devices should live until then anyway and thus be automatically
> > > > > > deallocated instead of needing an explicit free ... the problem
> > > > > > with that being compiled in devices currently optimize away the
> > > > > > module refcounting, but that should be fixable.
> > > > >
> > > > > module code lifespans are different than device structure lifespans,
> > > > > it's when people get them confused, as here, that we have problems.
> > > >
> > > > I'm not claiming they are.  What I am claiming is that module lifetime
> > > > must always encompass the device lifetimes.  Therefore, you can never
> > > > be incorrect by using a module lifetime for anything attached to a
> > > > device, just inefficient for using memory longer than potentially
> > > > needed.  However, in a lot of use cases, the device is created on
> > > > module init and destroyed on module exit, so the inefficiency is barely
> > > > noticeable.
> > >
> > > In almost no use case is the device created on module init of a driver.
> > > devices are created by busses, look at the USB code as an example.  The
> > > usb bus creates the devices and then individual modules bind to that
> > > device as needed.  If the device is removed from the system, wonderful,
> > > the device is unbound, but the module is still loaded.  So if you really
> > > wanted to, with your change, just do a insert/remove of a USB device a
> > > few zillion times and then memory is all gone :(
> > >
> > > > The question I'm asking is shouldn't we optimize for this?
> > >
> > > No.
> > >
> > > > so let people allocate devm memory safe in the knowledge it will be
> > > > freed on module release?
> > >
> > > No, see above.  Modules are never removed in a normal system.  devices
> > > are.
> > >
> > > And the drm developers have done great work in unwinding some of these
> > > types of mistakes in their drivers, let's not go backwards please.
> >
> > --
> > Regards,
> >
> > Laurent Pinchart



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Dan Carpenter]

On Thu, Jul 15, 2021 at 11:54:22AM +0200, Daniel Vetter wrote:
> Since we're dropping notes, a few of my thoughts:
> 
> - Personally I think an uapi resource cleanup system needs a different
> namespace, so that you don't mix these up. Hence drmm_ vs devres_ for
> drm, and maybe the generic version could be called uapires or ures or
> whatever. It needs to stick out.
> 
> - I'm wondering whether we could enlist checkers (maybe a runtime one)
> to scream anytime we do an unprotected dereference from ures memory to
> devres memory. This would help in subsystem where this problem is
> solved by trying to decouple the uapi side from the device side (like
> gpio and other subsystem with simpler interfaces). We have undefined
> behaviour and data race checkers already, this should be doable. But I
> have no idea how :-)

So you want a warning with code like:

	p->foo = bar;

Where "p" is devres memory and "bar" is ures?  There are a couple
approaches you could take and I would advise to implement both.

The first approach is to just check directly for if p is devres and bar
is ures.  The problem is that sometimes you won't know if p is devres so
some bugs will be missed.

The second approach is to say something like if we find one type "p"
that is devres, then let's assume they all are.  Same for ures.  Then
based on our assumptions about types, print a warning if they don't
match.  Then go through the warnings and make a list of types which lead
to false positives and add it to smatch_data/kernel.ignore_devres_types.
This is a bit hand wavey but that's basically the approach.

A third approach would be to do something with manual annotations.  You
could probably make Sparse work for something like that.

regards,
dan carpenter

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Daniel Vetter]

On Wed, Jul 21, 2021 at 11:09 AM Dan Carpenter <dan.carpenter@oracle.com> wrote:
> On Thu, Jul 15, 2021 at 11:54:22AM +0200, Daniel Vetter wrote:
> > Since we're dropping notes, a few of my thoughts:
> >
> > - Personally I think an uapi resource cleanup system needs a different
> > namespace, so that you don't mix these up. Hence drmm_ vs devres_ for
> > drm, and maybe the generic version could be called uapires or ures or
> > whatever. It needs to stick out.
> >
> > - I'm wondering whether we could enlist checkers (maybe a runtime one)
> > to scream anytime we do an unprotected dereference from ures memory to
> > devres memory. This would help in subsystem where this problem is
> > solved by trying to decouple the uapi side from the device side (like
> > gpio and other subsystem with simpler interfaces). We have undefined
> > behaviour and data race checkers already, this should be doable. But I
> > have no idea how :-)
>
> So you want a warning with code like:
>
>         p->foo = bar;
>
> Where "p" is devres memory and "bar" is ures?  There are a couple
> approaches you could take and I would advise to implement both.

Pointing from devres to ures is totally normal, otherwise how can you
unregister your interface (drm_device for gpus), which is ures, if you
don't have a pointer that you can access from your hotunplug code,
which leaves in the devres world. Or when you're called in any other
hook from the device side of things (callbacks, interrupts, whatever
really). The usual approach is to clear these out on hotunplug, which
doesn't need locks because you're guaranteed that nothing from the
devres world can use them anymore since the device is gone. But ofc
you can screw this up easily.

Also, you need to have pointers from ures to devres world too, because
without those it's pretty hard to access your actual hw and make it do
things, which is the entire point of exposing a uapi interface (which
is in the ures world). The trouble is this access isn't protected
against the devres side disappearing in a hotunplug with a lock or
barrier or similar.

E.g. in drm we have this drm_dev_enter/exit stuff, and within that it
should be save to deref from ures to devres, but outside it's not.

> The first approach is to just check directly for if p is devres and bar
> is ures.  The problem is that sometimes you won't know if p is devres so
> some bugs will be missed.
>
> The second approach is to say something like if we find one type "p"
> that is devres, then let's assume they all are.  Same for ures.  Then
> based on our assumptions about types, print a warning if they don't
> match.  Then go through the warnings and make a list of types which lead
> to false positives and add it to smatch_data/kernel.ignore_devres_types.
> This is a bit hand wavey but that's basically the approach.
>
> A third approach would be to do something with manual annotations.  You
> could probably make Sparse work for something like that.

Yeah I think a classifier like that makes sense, but I'm not sure we
can get at the information about how it's protected statically.
-Daniel

>
> regards,
> dan carpenter
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: cdev/devm_* issues (was Re: [TECH TOPIC] Rust for Linux)

[Dan Carpenter]

To be honest, I don't really understand the details here at all.

I feel like if you give me a few samples of buggy code and the warnings
you want printed then I can probably write it.

regards,
dan carpenter

Re: [TECH TOPIC] Rust for Linux

[Mauro Carvalho Chehab]

Em Wed, 7 Jul 2021 20:17:08 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu:

> On Wed, Jul 07, 2021 at 02:45:04PM +0200, Greg KH wrote:
> > On Wed, Jul 07, 2021 at 01:38:44PM +0100, James Bottomley wrote:  
> > > On Wed, 2021-07-07 at 14:20 +0200, Greg KH wrote:  
> > > > On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:  
> > > > > On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:  
> > > > > > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:  
> > > > > > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:  
> > > > > > > > "devres" / devm_xxx was an attempt to deal with this in C,
> > > > > > > > but it only solves some cases of this and has not received a
> > > > > > > > lot of adoption (we can argue about the reasons).  
> > > > > > >  
> > > > > > > Really? From my point of view that is adopted all over the map.
> > > > > > > I add new users all the time and use it as much as I can when
> > > > > > > writing new drivers.  
> > > > > > 
> > > > > > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > > > > > there's some subsystems that reject it for some reason.
> > > > > >   
> > > > > > > I think it's a formidable success, people just need to learn to
> > > > > > > do it more.  
> > > > > > 
> > > > > > There *are* issues with people adopting it too enthusiastically -
> > > > > > as well as the memory lifetime issues that Laurent mentioned it's
> > > > > > easy for it to cause problems with interrupt handlers that are
> > > > > > left live longer than they should be and try to use things that
> > > > > > were already deallocated.  
> 
> (CC'ing Daniel Vetter as the author of the DRM devres-based resource
> management)
> 
> I've given this lots of thoughts lately, in the context of V4L2, but it
> should be roughly the same for character devices in general. Here's what
> I think should be done for drivers that expose character devices.
> 
> - Drivers must stop allocating their internal data structure (the one
>   they set as device data with dev_set_drvdata()) with devm_kzalloc().
>   The data structure must instead be allocated with a plain kzalloc()
>   and reference-counted.
> 
>   Most drivers will register a single character device using a
>   subsystem-specific API (e.g. video_register_device() in V4L2). The
>   subsystem needs to provide a .release() callback, called when the
>   last reference to the character device is released. Drivers must
>   implement this, and can simply free their internal data structure at
>   this point.
> 
>   For drivers that register multiple character devices, or in general
>   expose multiple interfaces to userspace or other parts of the kernel,
>   the internal data structure must be properly reference-counted, with a
>   reference released in each .release() callback. There may be ways to
>   simplify this.

Good point. Yeah, indeed some work seems to be required on that area.

Yet, V4L2 is somewhat different here, as most (if not all) devices 
expose multiple cdevs. 

Also, in the case of V4L2 USB and PCI devices, their "dev->parent" is 
usually a PCI bridge, or an USB device with multiple functions on it,
as those hardware contain both audio and video and sometimes input
(either buttons or remote controllers), typically using different 
drivers. So, when the hardware is hot-unplugged or unbind, several 
drivers and multiple cdevs will be released. Ensuring that those will
happen at the right time can be a challenge, specially if there are
pending syscalls and/or threads by the time the device is unbound.

The DRM subsystem likely fits on the same case, as drivers also 
usually create multiple cdevs, and there are DMABUF objects shared
between different struct devices.

So, yeah, using devm_* for V4L2 and DRM can indeed bring troubles.
I can't see a solution there currently but to avoid using devm*,
handling it using an approach similar to the one you described.

-

I'm not so sure if using devm_* is a problem on several cases, though.
I mean, when the hardware is not hot-pluggable, the data lifetime
is a lot simpler.

So, for instance, a regulator driver probably can use devm_* without
any issues, as it doesn't seem to make much sense to unbind a regulator
once the device was probed. On drivers like that, not using devm_*
would just make the probing part of the driver more complex, without
bringing any real benefit.

Thanks,
Mauro

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

On Thu, Jul 08, 2021 at 11:08:52AM +0200, Mauro Carvalho Chehab wrote:
> Em Wed, 7 Jul 2021 20:17:08 +0300 Laurent Pinchart escreveu:
> > On Wed, Jul 07, 2021 at 02:45:04PM +0200, Greg KH wrote:
> > > On Wed, Jul 07, 2021 at 01:38:44PM +0100, James Bottomley wrote:  
> > > > On Wed, 2021-07-07 at 14:20 +0200, Greg KH wrote:  
> > > > > On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:  
> > > > > > On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:  
> > > > > > > On Tue, Jul 06, 2021 at 10:36:14PM +0200, Linus Walleij wrote:  
> > > > > > > > On Tue, Jul 6, 2021 at 10:00 PM Roland Dreier wrote:  
> > > > > > > > > "devres" / devm_xxx was an attempt to deal with this in C,
> > > > > > > > > but it only solves some cases of this and has not received a
> > > > > > > > > lot of adoption (we can argue about the reasons).  
> > > > > > > >  
> > > > > > > > Really? From my point of view that is adopted all over the map.
> > > > > > > > I add new users all the time and use it as much as I can when
> > > > > > > > writing new drivers.  
> > > > > > > 
> > > > > > > Yes, it's *super* widely used in most of the kernel.  Perhaps
> > > > > > > there's some subsystems that reject it for some reason.
> > > > > > >   
> > > > > > > > I think it's a formidable success, people just need to learn to
> > > > > > > > do it more.  
> > > > > > > 
> > > > > > > There *are* issues with people adopting it too enthusiastically -
> > > > > > > as well as the memory lifetime issues that Laurent mentioned it's
> > > > > > > easy for it to cause problems with interrupt handlers that are
> > > > > > > left live longer than they should be and try to use things that
> > > > > > > were already deallocated.  
> > 
> > (CC'ing Daniel Vetter as the author of the DRM devres-based resource
> > management)
> > 
> > I've given this lots of thoughts lately, in the context of V4L2, but it
> > should be roughly the same for character devices in general. Here's what
> > I think should be done for drivers that expose character devices.
> > 
> > - Drivers must stop allocating their internal data structure (the one
> >   they set as device data with dev_set_drvdata()) with devm_kzalloc().
> >   The data structure must instead be allocated with a plain kzalloc()
> >   and reference-counted.
> > 
> >   Most drivers will register a single character device using a
> >   subsystem-specific API (e.g. video_register_device() in V4L2). The
> >   subsystem needs to provide a .release() callback, called when the
> >   last reference to the character device is released. Drivers must
> >   implement this, and can simply free their internal data structure at
> >   this point.
> > 
> >   For drivers that register multiple character devices, or in general
> >   expose multiple interfaces to userspace or other parts of the kernel,
> >   the internal data structure must be properly reference-counted, with a
> >   reference released in each .release() callback. There may be ways to
> >   simplify this.
> 
> Good point. Yeah, indeed some work seems to be required on that area.
> 
> Yet, V4L2 is somewhat different here, as most (if not all) devices 
> expose multiple cdevs. 
>
> Also, in the case of V4L2 USB and PCI devices, their "dev->parent" is 
> usually a PCI bridge, or an USB device with multiple functions on it,
> as those hardware contain both audio and video and sometimes input
> (either buttons or remote controllers), typically using different 
> drivers. So, when the hardware is hot-unplugged or unbind, several 
> drivers and multiple cdevs will be released. Ensuring that those will
> happen at the right time can be a challenge, specially if there are
> pending syscalls and/or threads by the time the device is unbound.
> 
> The DRM subsystem likely fits on the same case, as drivers also 
> usually create multiple cdevs, and there are DMABUF objects shared
> between different struct devices.

The main difference is that DRM/KMS handles this inside the subsystem
core, instead of leaving it up to the drivers as in V4L2. We have to
live with historical mistakes, but I wish they could retire.

> So, yeah, using devm_* for V4L2 and DRM can indeed bring troubles.
> I can't see a solution there currently but to avoid using devm*,
> handling it using an approach similar to the one you described.

I want to emphasize that it's not all devm_* calls that are problematic.
The ioremap family of devm functions are perfectly fine, as memory
mapped I/O must not be accessed after a driver's .remove() function
returns.

> -
> 
> I'm not so sure if using devm_* is a problem on several cases, though.
> I mean, when the hardware is not hot-pluggable, the data lifetime
> is a lot simpler.

You can unbind a device from its driver through sysfs, and that brings
the exact same issues as hardware unplug.

> So, for instance, a regulator driver probably can use devm_* without
> any issues, as it doesn't seem to make much sense to unbind a regulator
> once the device was probed. On drivers like that, not using devm_*
> would just make the probing part of the driver more complex, without
> bringing any real benefit.

Don't forget that regulators, GPIO controllers and other such core
resources can be part of unpluggable devices.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Andy Lutomirski]

On Sat, Jul 10, 2021 at 9:43 AM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> On Thu, Jul 08, 2021 at 11:08:52AM +0200, Mauro Carvalho Chehab wrote:
> > Em Wed, 7 Jul 2021 20:17:08 +0300 Laurent Pinchart escreveu:
>
> > So, for instance, a regulator driver probably can use devm_* without
> > any issues, as it doesn't seem to make much sense to unbind a regulator
> > once the device was probed. On drivers like that, not using devm_*
> > would just make the probing part of the driver more complex, without
> > bringing any real benefit.
>
> Don't forget that regulators, GPIO controllers and other such core
> resources can be part of unpluggable devices.

It sounds like this type of use might be a good fit for a simple
precise garbage collector.  This could be arranged in C in a
reasonably safe way (with the assistance of sparse, perhaps).  Every
struct that could contain a GC pointer to a device would have a
special attribute, and the whole graph of devices could be walked as
needed to collect garbage.

Alternatively, weak references might be a good fit for this,
especially if hotplug is involved.  If a device gets hot-unplugged
without warning, every subsequent attempt to follow a weak reference
to that device could be made to fail.

--Andy

Re: [TECH TOPIC] Rust for Linux

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

On Wed, Jul 07, 2021 at 12:34:31PM +0100, James Bottomley wrote:
> On Wed, 2021-07-07 at 11:50 +0100, Mark Brown wrote:

> > There *are* issues with people adopting it too enthusiastically - as
> > well as the memory lifetime issues that Laurent mentioned it's easy
> > for it to cause problems with interrupt handlers that are left live
> > longer than they should be and try to use things that were already
> > deallocated.

> Fixing this should not be beyond the wit of humankind, though.  If we
> delayed deallocation to module release, that would fix the interrupt
> issue, wouldn't it?  Perhaps all devres memory for devices should live

No, that's half the problem - as Greg says there's the issue of module
vs device lifespans, and you sometimes also have to take care with the
ordering of your allocations and unwindings within device startup and
teardown so you don't end up trying to do something like deliver an
interrupt to the subsystem core after having told the subsystem the
device was gone, or trying to process a final subsystem call that needs
interrupts after you've freed interrupts.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [TECH TOPIC] Rust for Linux

[Bart Van Assche]

On 7/6/21 1:00 PM, Roland Dreier wrote:
> One area I see where Rust could make a big improvement for drivers is
> in using RAII for error paths.  Drivers often have a lot of code like
> 
>     if (something())
>         return err;
> 
>     if (something_else())
>         goto undo_something;
> 
>     if (a_third_thing())
>         goto undo_two_things;
> 
> and so on, which is difficult to get right in the first place and even
> harder to keep correct in the face of changes.

Although cumbersome, it is possible to implement RAII by using the gcc
extensions to the C language. An example:

#include <stdio.h>
#include <stdlib.h>

#define CONCAT2(a, b) a##b
#define CONCAT(a, b) CONCAT2(a, b)
#define DECL_WITH_DTOR_NAME(declaration, initializer, cleanup_func_name,\
                            arglist, cleanup_func_body)                 \
    void cleanup_func_name(arglist) { cleanup_func_body; }              \
    declaration __attribute__((cleanup(cleanup_func_name))) = initializer
#define DECL_WITH_DTOR(declaration, initializer, arglist, cleanup_func_body) \
    DECL_WITH_DTOR_NAME(declaration, initializer,                       \
                        CONCAT(cleanup_func, __COUNTER__), arglist,     \
                        cleanup_func_body)

int main(int argc, char** argv)
{
    DECL_WITH_DTOR(void *p, NULL, void **p,
                   printf("freeing p = %p\n", *p); free(*p););
    DECL_WITH_DTOR(void *q, NULL, void **q,
                   printf("freeing q = %p\n", *q); free(*q););
    p = malloc(7);
    q = malloc(3);
    printf("p = %p; q = %p\n", p, q);
    return 0;
}

The output of the above code:

p = 0xd952a0; q = 0xd952c0
freeing q = 0xd952c0
freeing p = 0xd952a0

Bart.

Re: [TECH TOPIC] Rust for Linux

[Stephen Hemminger]

On Tue, 6 Jul 2021 14:45:50 -0700
Bart Van Assche <bvanassche@acm.org> wrote:

> On 7/6/21 1:00 PM, Roland Dreier wrote:
> > One area I see where Rust could make a big improvement for drivers is
> > in using RAII for error paths.  Drivers often have a lot of code like
> > 
> >     if (something())
> >         return err;
> > 
> >     if (something_else())
> >         goto undo_something;
> > 
> >     if (a_third_thing())
> >         goto undo_two_things;
> > 
> > and so on, which is difficult to get right in the first place and even
> > harder to keep correct in the face of changes.  
> 
> Although cumbersome, it is possible to implement RAII by using the gcc
> extensions to the C language. An example:
> 
> #include <stdio.h>
> #include <stdlib.h>
> 
> #define CONCAT2(a, b) a##b
> #define CONCAT(a, b) CONCAT2(a, b)
> #define DECL_WITH_DTOR_NAME(declaration, initializer, cleanup_func_name,\
>                             arglist, cleanup_func_body)                 \
>     void cleanup_func_name(arglist) { cleanup_func_body; }              \
>     declaration __attribute__((cleanup(cleanup_func_name))) = initializer
> #define DECL_WITH_DTOR(declaration, initializer, arglist, cleanup_func_body) \
>     DECL_WITH_DTOR_NAME(declaration, initializer,                       \
>                         CONCAT(cleanup_func, __COUNTER__), arglist,     \
>                         cleanup_func_body)
> 
> int main(int argc, char** argv)
> {
>     DECL_WITH_DTOR(void *p, NULL, void **p,
>                    printf("freeing p = %p\n", *p); free(*p););
>     DECL_WITH_DTOR(void *q, NULL, void **q,
>                    printf("freeing q = %p\n", *q); free(*q););
>     p = malloc(7);
>     q = malloc(3);
>     printf("p = %p; q = %p\n", p, q);
>     return 0;
> }
> 
> The output of the above code:
> 
> p = 0xd952a0; q = 0xd952c0
> freeing q = 0xd952c0
> freeing p = 0xd952a0
> 

Why not use the GCC cleanup function attribute, it is a lot cleaner than
having ugly error prone macros.

#define _cleanup_free __atttribute__((__cleanup__(free)))

int main(int argc, char **argv)
{
          _cleanup_free_ void *p;
          _cleanup_free_  void *q;

Re: [TECH TOPIC] Rust for Linux

[Bart Van Assche]

On 7/6/21 4:08 PM, Stephen Hemminger wrote:
> Why not use the GCC cleanup function attribute, it is a lot cleaner than
> having ugly error prone macros.
> 
> #define _cleanup_free __atttribute__((__cleanup__(free)))
> 
> int main(int argc, char **argv)
> {
>           _cleanup_free_ void *p;
>           _cleanup_free_  void *q;

Huh? My code uses the gcc cleanup attribute.

The above code will trigger a crash because gcc passes a pointer to
local variable to the cleanup function (free) instead of the value of
the local variable.

As a sidenote, I'm surprised that C++ is not supported for Linux kernel
code since C++ supports multiple mechanisms that are useful in kernel
code, e.g. RAII, lambda functions, range-based for loops, std::span<>
and std::string_view<>. Lambda functions combined with std::function<>
allow to implement type-safe callbacks. Implementing type-safe callbacks
in C without macro trickery is not possible.

Bart.

Re: [TECH TOPIC] Rust for Linux

[Linus Torvalds]

On Tue, Jul 6, 2021 at 7:41 PM Bart Van Assche <bvanassche@acm.org> wrote:
>
> As a sidenote, I'm surprised that C++ is not supported for Linux kernel
> code since C++ supports multiple mechanisms [..]

You'd have to get rid of some of the complete garbage from C++ for it
to be usable.

One of the trivial ones is "new" - not only is it a horribly stupid
namespace violation, but it depends on exception handling that isn't
viable for the kernel, so it's a namespace violation that has no
upsides, only downsides.

Could we fix it with some kind of "-Dnew=New" trickery? Yes, but
considering all the other issues, it's just not worth the pain. C++ is
simply not a good language. It doesn't fix any of the fundamental
issues in C (ie no actual safety), and instead it introduces a lot of
new problems due to bad designs.

            Linus

Re: [TECH TOPIC] Rust for Linux

[Bart Van Assche]

On 7/7/21 11:57 AM, Linus Torvalds wrote:
> On Tue, Jul 6, 2021 at 7:41 PM Bart Van Assche <bvanassche@acm.org> wrote:
>> As a sidenote, I'm surprised that C++ is not supported for Linux kernel
>> code since C++ supports multiple mechanisms [..]
> 
> You'd have to get rid of some of the complete garbage from C++ for it
> to be usable.
> 
> One of the trivial ones is "new" - not only is it a horribly stupid
> namespace violation, but it depends on exception handling that isn't
> viable for the kernel, so it's a namespace violation that has no
> upsides, only downsides.
> 
> Could we fix it with some kind of "-Dnew=New" trickery? Yes, but
> considering all the other issues, it's just not worth the pain. C++ is
> simply not a good language. It doesn't fix any of the fundamental
> issues in C (ie no actual safety), and instead it introduces a lot of
> new problems due to bad designs.

Hi Linus,

Thank you for having shared your opinion. You may want to know that
every C++ project I have worked on so far enabled at least the following
compiler flags: -fno-exceptions and -fno-rtti.

What the C++ operator new does if not enough memory is available depends
on the implementation of that operator. We could e.g. modify the
behavior of operator new as follows:
- Add -fno-builtin-new to the compiler flags.
- Define a custom version of operator new.

An example (user space code):

include <stdlib.h>
#include <stdio.h>

void *operator new(size_t size)
{
  printf("%s\n", __func__);
  return malloc(size);
}

void operator delete(void *p)
{
  printf("%s\n", __func__);
  free(p);
}

void operator delete(void *p, size_t size)
{
  printf("%s\n", __func__);
  free(p);
}

void *operator new[](size_t size)
{
  printf("%s\n", __func__);
  return malloc(size);
}

void operator delete[](void *p)
{
  printf("%s\n", __func__);
  free(p);
}

void operator delete[](void *p, size_t size)
{
  printf("%s\n", __func__);
  free(p);
}

int main(int, char **)
{
  int *intp = new int;
  long *arrayp = new long[37];
  delete[] arrayp;
  delete intp;
  return 0;
}

The output of the above code:

operator new
operator new []
operator delete []
operator delete

Bart.

Re: [TECH TOPIC] Rust for Linux

[Linus Torvalds]

On Wed, Jul 7, 2021 at 1:32 PM Bart Van Assche <bvanassche@acm.org> wrote:
>
> Thank you for having shared your opinion. You may want to know that
> every C++ project I have worked on so far enabled at least the following
> compiler flags: -fno-exceptions and -fno-rtti.
>
> What the C++ operator new does if not enough memory is available depends
> on the implementation of that operator.

The point is, C++ really has some fundamental problems. Yes, you can
work around them, but it doesn't change the fact that it doesn't
actually *fix* any of the issues that make C problematic.

For example, do you go as far as to disallow classes because member
functions are horrible garbage? Maybe newer versions of C++ fixed it,
but it used to be the case that you couldn't sanely even split a
member function into multiple functions to make it easier to read,
because every single helper function that worked on that class then
had to be declared in the class definition.

Which makes simple things like just re-organizing code to be legible a
huge pain.

At the same time, C++ offer no real new type or runtime safety, and
makes the problem space just bigger. It forces you to use _more_
casts, which then just make for more problems when it turns out the
casts were incorrect and hid the real problem.

So no. We're not switching to a new language that causes pain and
offers no actual upsides.

At least the argument is that Rust _fixes_ some of the C safety
issues. C++ would not.

            Linus

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Linus,

On Wed, Jul 07, 2021 at 01:39:22PM -0700, Linus Torvalds wrote:
> On Wed, Jul 7, 2021 at 1:32 PM Bart Van Assche wrote:
> >
> > Thank you for having shared your opinion. You may want to know that
> > every C++ project I have worked on so far enabled at least the following
> > compiler flags: -fno-exceptions and -fno-rtti.
> >
> > What the C++ operator new does if not enough memory is available depends
> > on the implementation of that operator.
> 
> The point is, C++ really has some fundamental problems. Yes, you can
> work around them, but it doesn't change the fact that it doesn't
> actually *fix* any of the issues that make C problematic.
> 
> For example, do you go as far as to disallow classes because member
> functions are horrible garbage? Maybe newer versions of C++ fixed it,
> but it used to be the case that you couldn't sanely even split a
> member function into multiple functions to make it easier to read,
> because every single helper function that worked on that class then
> had to be declared in the class definition.

That's still the case today.

> Which makes simple things like just re-organizing code to be legible a
> huge pain.
> 
> At the same time, C++ offer no real new type or runtime safety, and
> makes the problem space just bigger. It forces you to use _more_
> casts, which then just make for more problems when it turns out the
> casts were incorrect and hid the real problem.

I beg to differ on that one. There are features of C++ that would be
very helpful for kernel development. Among them is native support for
destructors, which allow implementing RAII idioms. Unique pointers would
also be very helpful to explicitly expose object ownership rules (shared
pointers are a different story though, it's very easy to use them
incorrectly and infect the whole code base as a result). Templates are
another feature that is widely criticized (and often for good reasons),
but when seen as a type-safe implementation of macros, they can bring
increased safety to the code base.

C++ has upsides and fixes real issues. It also causes pain, and it's not
the only language to provide the above features, so I wouldn't call for
its usage in the kernel. I just wish we had objects with destructors in
plain C.

> So no. We're not switching to a new language that causes pain and
> offers no actual upsides.
> 
> At least the argument is that Rust _fixes_ some of the C safety
> issues. C++ would not.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Geert Uytterhoeven]

Hi Linus,

On Wed, Jul 7, 2021 at 10:39 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> At the same time, C++ offer no real new type or runtime safety, and
> makes the problem space just bigger. It forces you to use _more_
> casts, which then just make for more problems when it turns out the
> casts were incorrect and hid the real problem.

At least the C++ casts are easier to search for.  The C cast syntax
is very grep-unfriendly.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

On Wed, Jul 07, 2021 at 01:32:36PM -0700, Bart Van Assche wrote:
> On 7/7/21 11:57 AM, Linus Torvalds wrote:
> > On Tue, Jul 6, 2021 at 7:41 PM Bart Van Assche <bvanassche@acm.org> wrote:
> >> As a sidenote, I'm surprised that C++ is not supported for Linux kernel
> >> code since C++ supports multiple mechanisms [..]
> > 
> > You'd have to get rid of some of the complete garbage from C++ for it
> > to be usable.
> > 
> > One of the trivial ones is "new" - not only is it a horribly stupid
> > namespace violation, but it depends on exception handling that isn't
> > viable for the kernel, so it's a namespace violation that has no
> > upsides, only downsides.
> > 
> > Could we fix it with some kind of "-Dnew=New" trickery? Yes, but
> > considering all the other issues, it's just not worth the pain. C++ is
> > simply not a good language. It doesn't fix any of the fundamental
> > issues in C (ie no actual safety), and instead it introduces a lot of
> > new problems due to bad designs.
> 
> Hi Linus,
> 
> Thank you for having shared your opinion. You may want to know that
> every C++ project I have worked on so far enabled at least the following
> compiler flags: -fno-exceptions and -fno-rtti.
> 
> What the C++ operator new does if not enough memory is available depends
> on the implementation of that operator. We could e.g. modify the
> behavior of operator new as follows:

Or we could forbid usage of new() and require different memory
allocation primitives. The kernel doesn't use malloc() either :-)

> - Add -fno-builtin-new to the compiler flags.
> - Define a custom version of operator new.
> 
> An example (user space code):
> 
> include <stdlib.h>
> #include <stdio.h>
> 
> void *operator new(size_t size)
> {
>   printf("%s\n", __func__);
>   return malloc(size);
> }
> 
> void operator delete(void *p)
> {
>   printf("%s\n", __func__);
>   free(p);
> }
> 
> void operator delete(void *p, size_t size)
> {
>   printf("%s\n", __func__);
>   free(p);
> }
> 
> void *operator new[](size_t size)
> {
>   printf("%s\n", __func__);
>   return malloc(size);
> }
> 
> void operator delete[](void *p)
> {
>   printf("%s\n", __func__);
>   free(p);
> }
> 
> void operator delete[](void *p, size_t size)
> {
>   printf("%s\n", __func__);
>   free(p);
> }
> 
> int main(int, char **)
> {
>   int *intp = new int;
>   long *arrayp = new long[37];
>   delete[] arrayp;
>   delete intp;
>   return 0;
> }
> 
> The output of the above code:
> 
> operator new
> operator new []
> operator delete []
> operator delete

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 10:32 PM Bart Van Assche <bvanassche@acm.org> wrote:
>
> Thank you for having shared your opinion. You may want to know that
> every C++ project I have worked on so far enabled at least the following
> compiler flags: -fno-exceptions and -fno-rtti.
>
> What the C++ operator new does if not enough memory is available depends
> on the implementation of that operator. We could e.g. modify the
> behavior of operator new as follows:
> - Add -fno-builtin-new to the compiler flags.
> - Define a custom version of operator new.

The issue is that, even if people liked C++ a lot, there is little
point in using C++ once Rust is an option.

Even if you discuss "modern C++" (i.e. post-C++11, and even
post-C++17), there is really no comparison.

For instance, you mentioned `std::span` from the very latest C++20
standard; let's build one:

    std::span<int> f() {
        int a[] = { foo() };
        std::span<int> s(a);
        return s;
    }

Now anybody that accesses the returned `std::span` has just introduced
UB. From a quick test, neither Clang nor GCC warn about it. Even if
they end up detecting such a simple case, it is impossible to do so in
the general case.

Yes, it is a stupid mistake, we should not do that, and the usual
arguments. But the point is UB is still as easy as has always been to
introduce in both C and C++. In Rust, that mistake does not happen.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Miguel,

On Thu, Jul 08, 2021 at 12:11:16AM +0200, Miguel Ojeda wrote:
> On Wed, Jul 7, 2021 at 10:32 PM Bart Van Assche wrote:
> >
> > Thank you for having shared your opinion. You may want to know that
> > every C++ project I have worked on so far enabled at least the following
> > compiler flags: -fno-exceptions and -fno-rtti.
> >
> > What the C++ operator new does if not enough memory is available depends
> > on the implementation of that operator. We could e.g. modify the
> > behavior of operator new as follows:
> > - Add -fno-builtin-new to the compiler flags.
> > - Define a custom version of operator new.
> 
> The issue is that, even if people liked C++ a lot, there is little
> point in using C++ once Rust is an option.
> 
> Even if you discuss "modern C++" (i.e. post-C++11, and even
> post-C++17), there is really no comparison.
> 
> For instance, you mentioned `std::span` from the very latest C++20
> standard; let's build one:
> 
>     std::span<int> f() {
>         int a[] = { foo() };
>         std::span<int> s(a);
>         return s;
>     }
> 
> Now anybody that accesses the returned `std::span` has just introduced
> UB. From a quick test, neither Clang nor GCC warn about it. Even if
> they end up detecting such a simple case, it is impossible to do so in
> the general case.
> 
> Yes, it is a stupid mistake, we should not do that, and the usual
> arguments. But the point is UB is still as easy as has always been to
> introduce in both C and C++. In Rust, that mistake does not happen.

You're comparing apples and pears though. A C++ function that is meant
to transfer unique ownership of an object to the caller should return a
std::unique_ptr<> on a container that stores the data. We're getting
off-topic though, this mail thread isn't about comparing C++ and rust
:-)

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Thu, Jul 8, 2021 at 12:44 AM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> You're comparing apples and pears though. A C++ function that is meant
> to transfer unique ownership of an object to the caller should return a
> std::unique_ptr<> on a container that stores the data. We're getting

Nope, I am not comparing apples and pears. I just showed you a trivial
way to make UB in C++20 with one of the types someone else mentioned.

You mention `std::unique_ptr` now. Same deal:

    std::unique_ptr<int> f() {
        return {}; // returns a `nullptr`
    }

You will now reply: "oh, but you are *not* supposed to use it like
that!". But the point is: it is not about the simple examples, it is
about the complex cases where objects are alive for a long time,
passed across chains of calls, manipulated in several places, across
different threads, etc., etc. so that reasoning is non-local.

Don't get me wrong, `std::unique_ptr` is nice, and I have used it many
times in my career with good results. Still, it is far from what Rust
offers.

Another extremely typical example:

    std::vector<int> v;
    ...
    int * p = v.data(); // looks OK
    ...
    v.push_back(42); // looks OK
    ...
    f(p); // oh, wait...

> off-topic though, this mail thread isn't about comparing C++ and rust
> :-)

Well, if people bring up C++ as an alternative to Rust, they are
implying Rust does not offer much more than C++. Which is false,
misleading, and directly counters the Rust support proposal, thus I
feel the need to answer.

Again, don't get me wrong: while one can definitely see Rust as a
"cleaned up" C/C++ (it is, in a way); the key is that it *also* offers
major advantages using a few new research ideas that no other system
language had (even SPARK had to catch up [1]). It is not just a
sweeter syntax or a few fancy features here and there as many seem to
imply in many fora.

[1] https://blog.adacore.com/using-pointers-in-spark

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Miguel,

On Thu, Jul 08, 2021 at 01:21:55AM +0200, Miguel Ojeda wrote:
> On Thu, Jul 8, 2021 at 12:44 AM Laurent Pinchart wrote:
> >
> > You're comparing apples and pears though. A C++ function that is meant
> > to transfer unique ownership of an object to the caller should return a
> > std::unique_ptr<> on a container that stores the data. We're getting
> 
> Nope, I am not comparing apples and pears. I just showed you a trivial
> way to make UB in C++20 with one of the types someone else mentioned.
> 
> You mention `std::unique_ptr` now. Same deal:
> 
>     std::unique_ptr<int> f() {
>         return {}; // returns a `nullptr`
>     }
> 
> You will now reply: "oh, but you are *not* supposed to use it like
> that!". But the point is: it is not about the simple examples, it is
> about the complex cases where objects are alive for a long time,
> passed across chains of calls, manipulated in several places, across
> different threads, etc., etc. so that reasoning is non-local.
> 
> Don't get me wrong, `std::unique_ptr` is nice, and I have used it many
> times in my career with good results. Still, it is far from what Rust
> offers.
> 
> Another extremely typical example:
> 
>     std::vector<int> v;
>     ...
>     int * p = v.data(); // looks OK
>     ...
>     v.push_back(42); // looks OK
>     ...
>     f(p); // oh, wait...

I don't think anyone ever claimed that C++ offers the same kind of
compile-type checks that rust does, so there's no disagreement there.

> > off-topic though, this mail thread isn't about comparing C++ and rust
> > :-)
> 
> Well, if people bring up C++ as an alternative to Rust, they are
> implying Rust does not offer much more than C++. Which is false,
> misleading, and directly counters the Rust support proposal, thus I
> feel the need to answer.

The discussion has drifted from rust in the kernel to features that C is
missing and that make our life painful when it shouldn't be. Some of
those features are fairly basic (such as features that would allow
implementing RAII idioms with a syntax that wouldn't make all developers
want to jump through the window), and available in multiple languages,
including rust and C++. To that extent, C++ could be an alternative to
rust *if* the goal was limited to bringing those features in (I recall a
computer science teacher explaining to his class that they would program
in C+, which he defined as C++ without classes, just to use << instead
of printf...).

This being said, I don't think C++ would be a particularly good
alternative even for that limited goal, as there could be more drawbacks
than advantages. Furthermore, if we're considering supporting a second
language in the kernel, it would likely be best to pick a language to
would bring us as many benefits as possible. Rust is a good candidate,
even if I'm not convinced at this point that the gains outweight the
costs (especially when it comes to the disturbance to the development
flow, see the discussion in this mail thread about subsystem-wide or
kernel-wide changes). Time (and discussions) will tell.

> Again, don't get me wrong: while one can definitely see Rust as a
> "cleaned up" C/C++ (it is, in a way); the key is that it *also* offers
> major advantages using a few new research ideas that no other system
> language had (even SPARK had to catch up [1]). It is not just a
> sweeter syntax or a few fancy features here and there as many seem to
> imply in many fora.

Speaking of sweeter syntax, in the "if only I had a time machine"
category, I wish rust would have picked a syntax closer to C when
departing from it wasn't strictly necessary :-(

> [1] https://blog.adacore.com/using-pointers-in-spark

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Thu, Jul 8, 2021 at 1:41 AM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> I don't think anyone ever claimed that C++ offers the same kind of
> compile-type checks that rust does, so there's no disagreement there.

My apologies if I have misinterpreted your claims (or those of Bart).

The thing is, claims such as "C++ is as safe as Rust, you just need to
use modern C++ properly!" are still way too common online, and many
developers are unfamiliar with Rust, thus I feel we need to be crystal
clear (in a thread about Rust support) that it is a strict improvement
over C++, and not a small one at that.

In any case, please note that I am not a Rust "fan" or "believer"
(somebody mentioned that word) -- if the C committee opens up to add
these kind of things, I will gladly work on that as my free time
permits. But Rust works, and it works today, and the language brings
many other things too in a fairly well-designed package (though it is
not perfect either).

> The discussion has drifted from rust in the kernel to features that C is
> missing and that make our life painful when it shouldn't be. Some of
> those features are fairly basic (such as features that would allow
> implementing RAII idioms with a syntax that wouldn't make all developers
> want to jump through the window), and available in multiple languages,

The C committee is looking into adding `defer`, so you may actually
get RAII in C ;)

They are also looking into adding lambdas (with a similar syntax to C++'s).

> This being said, I don't think C++ would be a particularly good
> alternative even for that limited goal, as there could be more drawbacks
> than advantages. Furthermore, if we're considering supporting a second
> language in the kernel, it would likely be best to pick a language to
> would bring us as many benefits as possible. Rust is a good candidate,
> even if I'm not convinced at this point that the gains outweight the
> costs (especially when it comes to the disturbance to the development
> flow, see the discussion in this mail thread about subsystem-wide or
> kernel-wide changes). Time (and discussions) will tell.

This is fair, thank you.

> Speaking of sweeter syntax, in the "if only I had a time machine"
> category, I wish rust would have picked a syntax closer to C when
> departing from it wasn't strictly necessary :-(

Interesting -- in which regard? i.e. it is actually quite close, their
designers were clearly going for something relatively familiar to
C/C++ programmers (at least in its current state, not the very early
design iterations of Rust).

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Laurent Pinchart]

Hi Miguel,

On Thu, Jul 08, 2021 at 02:27:47AM +0200, Miguel Ojeda wrote:
> On Thu, Jul 8, 2021 at 1:41 AM Laurent Pinchart wrote:
> >
> > I don't think anyone ever claimed that C++ offers the same kind of
> > compile-type checks that rust does, so there's no disagreement there.
> 
> My apologies if I have misinterpreted your claims (or those of Bart).
> 
> The thing is, claims such as "C++ is as safe as Rust, you just need to
> use modern C++ properly!" are still way too common online, and many
> developers are unfamiliar with Rust, thus I feel we need to be crystal
> clear (in a thread about Rust support) that it is a strict improvement
> over C++, and not a small one at that.
> 
> In any case, please note that I am not a Rust "fan" or "believer"
> (somebody mentioned that word) -- if the C committee opens up to add
> these kind of things, I will gladly work on that as my free time
> permits. But Rust works, and it works today, and the language brings
> many other things too in a fairly well-designed package (though it is
> not perfect either).
> 
> > The discussion has drifted from rust in the kernel to features that C is
> > missing and that make our life painful when it shouldn't be. Some of
> > those features are fairly basic (such as features that would allow
> > implementing RAII idioms with a syntax that wouldn't make all developers
> > want to jump through the window), and available in multiple languages,
> 
> The C committee is looking into adding `defer`, so you may actually
> get RAII in C ;)

That would be nice.

> They are also looking into adding lambdas (with a similar syntax to C++'s).
> 
> > This being said, I don't think C++ would be a particularly good
> > alternative even for that limited goal, as there could be more drawbacks
> > than advantages. Furthermore, if we're considering supporting a second
> > language in the kernel, it would likely be best to pick a language to
> > would bring us as many benefits as possible. Rust is a good candidate,
> > even if I'm not convinced at this point that the gains outweight the
> > costs (especially when it comes to the disturbance to the development
> > flow, see the discussion in this mail thread about subsystem-wide or
> > kernel-wide changes). Time (and discussions) will tell.
> 
> This is fair, thank you.
> 
> > Speaking of sweeter syntax, in the "if only I had a time machine"
> > category, I wish rust would have picked a syntax closer to C when
> > departing from it wasn't strictly necessary :-(
> 
> Interesting -- in which regard? i.e. it is actually quite close, their
> designers were clearly going for something relatively familiar to
> C/C++ programmers (at least in its current state, not the very early
> design iterations of Rust).

One particular point that comes to my mind is function parameters,
declared as "variable_name: type" instead of "type variable_name", or
the need to prefix functions with "fn". It's not a huge deal, but if
small things like that could have been kept closer to C, it would have
been nice.

-- 
Regards,

Laurent Pinchart

Re: [TECH TOPIC] Rust for Linux

[Alexey Dobriyan]

On Wed, Jul 07, 2021 at 11:57:19AM -0700, Linus Torvalds wrote:
> On Tue, Jul 6, 2021 at 7:41 PM Bart Van Assche <bvanassche@acm.org> wrote:
> >
> > As a sidenote, I'm surprised that C++ is not supported for Linux kernel
> > code since C++ supports multiple mechanisms [..]
> 
> You'd have to get rid of some of the complete garbage from C++ for it
> to be usable.
> 
> One of the trivial ones is "new" - not only is it a horribly stupid
> namespace violation, but it depends on exception handling that isn't
> viable for the kernel, so it's a namespace violation that has no
> upsides, only downsides.

"new" is such a non-problem.

"Allocating" new simply doesn't work by default:

	undefined reference to `operator new(unsigned long)'

Class level operator new is deal with by "-fcheck-new" so that constructors
don't run if allocation fails, then the pointer is checked as usual.

Placement "new" can be disabled completely if needed.

> Could we fix it with some kind of "-Dnew=New" trickery?

No, developers will type "new_" and that's basically it.

Re: [TECH TOPIC] Rust for Linux

[Bart Van Assche]

On 6/25/21 3:09 PM, Miguel Ojeda wrote:
> The Rust for Linux project is adding support for the Rust language to
> the Linux kernel. This talk describes the work done so far and also
> serves as an introduction for other kernel developers interested in
> using Rust in the kernel.
> 
> It covers:
> 
> - A quick introduction of the Rust language within the context of the kernel.
> - How Rust support works in the kernel: overall infrastructure,
> compilation model, the standard library (`core` and `alloc`), etc.
> - How Documentation for Rust code looks like.
> - Testing Rust code (unit tests and self tests).
> - Overview of tooling (e.g. compiler as a library, custom linters, Miri, etc.).
> - Explanation of coding guidelines (e.g. automatic formatting) and
> policies we follow (e.g. the `SAFETY` comments).
> - How kernel driver code looks like in Rust.

Will the introduction of Rust drivers require that every maintainer of
every subsystem used by Rust drivers maintains two versions of each
header file that is used by Rust drivers - one version of each header
file in C and another version of the same header file in Rust?

Thanks,

Bart.

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Tue, Jul 6, 2021 at 9:05 PM Bart Van Assche <bvanassche@acm.org> wrote:
>
> Will the introduction of Rust drivers require that every maintainer of
> every subsystem used by Rust drivers maintains two versions of each
> header file that is used by Rust drivers - one version of each header
> file in C and another version of the same header file in Rust?

No, C bindings are automatically generated as needed.

What a maintainer would need to maintain, if they so wish, are Rust
"abstractions" of those C bindings. These are simply the Rust APIs
that are then used by other Rust modules to use (the "leaves"), and it
is what allows to maximize the amount of safe code in those Rust
modules.

Cheers,
Miguel

Re: [TECH TOPIC] Rust for Linux

[Steven Rostedt]

On Sat, 26 Jun 2021 00:09:00 +0200
Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote:

> The Rust for Linux project is adding support for the Rust language to
> the Linux kernel. This talk describes the work done so far and also
> serves as an introduction for other kernel developers interested in
> using Rust in the kernel.
> 
> It covers:
> 
> - A quick introduction of the Rust language within the context of the kernel.
> - How Rust support works in the kernel: overall infrastructure,
> compilation model, the standard library (`core` and `alloc`), etc.
> - How Documentation for Rust code looks like.
> - Testing Rust code (unit tests and self tests).
> - Overview of tooling (e.g. compiler as a library, custom linters, Miri, etc.).
> - Explanation of coding guidelines (e.g. automatic formatting) and
> policies we follow (e.g. the `SAFETY` comments).
> - How kernel driver code looks like in Rust.

[ Starting at the OP of the thread as this is a different topic ]

BTW, does rust support "fentry/mcount" code? If not, that means ftrace
(and all the hooks we have inside the kernel) are not going to be able
to use it? That's one big "notrace" across all drivers that implement
Rust?

-- Steve

Re: [TECH TOPIC] Rust for Linux

[Miguel Ojeda]

On Wed, Jul 7, 2021 at 5:48 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> BTW, does rust support "fentry/mcount" code? If not, that means ftrace
> (and all the hooks we have inside the kernel) are not going to be able
> to use it? That's one big "notrace" across all drivers that implement
> Rust?

From a quick look, there seems to be support for `mcount` [1], and an
open issue on `fentry` [2]. Rust is based on LLVM, so anything that we
already do through Clang should be doable in `rustc` too.

I have asked our in-house Rust experts in case they now something more
(thanks Alex for the `fentry` link!).

[1] https://github.com/rust-lang/rust/pull/57220
[2] https://github.com/rust-lang/rust/issues/55092

Cheers,
Miguel