[LSF/MM TOPIC] Rust

[LSF/MM TOPIC] Rust

[Kent Overstreet]

There's been ongoing work on safe Rust interfaces to the VFS, as well as
whole new filesystems in Rust (tarfs, puzzlefs), and talk of adding Rust
code for existing filesystems (bcachefs, any day now, I swear).

Maybe it's time to get together and see what sort of
trolling^H^H^H^Hserious design discussions we can come up with?

Possible subtopics:
 - Braces, brackets and arrow signs: does Rust have enough of them, too
   many, or both?

 - Obeying the borrow checker: C programmers are widely renouned for
   creative idiosyncrasies in pointer based data structures; structure
   design; will we be able to fully express our creativity within Rust?
   Or are there VFS abstractions and lifetime rules that will be
   difficult to translate?

 - Perhaps there are annoying VFS rules that in the past required hand
   tattoos to remember, but we'll now be able to teach the compiler to
   remember for us?

   (idmapping comes to mind as one area where we've recently been able
   to increase safety through effective use of the type system - this is
   about way more than just use-after-free bugs)

 - Moving objects around in memory: Rust rather dislikes being told
   objects can't be moved around (a consequence of algebraic data
   types), and this has been a source of no small amount of frustration
   on the Rust side of things. Could we on the C side perhaps find ways
   to relax their constraints? (object pinning, versus e.g. perhaps
   making it possible to move a list head around)

 - The use of outside library code: Historically, C code was either
   written for userspace or the kernel, and not both. But that's not
   particularly true in Rust land (and getting to be less true even in C
   land); should we consider some sort of structure or (cough) package
   management? Is it time to move beyond ye olde cut-and-paste?

 - What's the development and debugging experience been like so far?
   Who's got stories to share?

The Rust-for-Linux people have been great - let's see if we can get them
to join us in Utah this year, I think they'll have things to teach us.

Cheers,
Kent

Re: [LSF/MM TOPIC] Rust

[Matthew Wilcox]

On Mon, Jan 22, 2024 at 11:23:51PM -0500, Kent Overstreet wrote:
> There's been ongoing work on safe Rust interfaces to the VFS, as well as
> whole new filesystems in Rust (tarfs, puzzlefs), and talk of adding Rust
> code for existing filesystems (bcachefs, any day now, I swear).

I really want this to happen.  It's taken 50 years, but we finally have
a programming language that can replace C for writing kernels.

> Possible subtopics:
>  - Braces, brackets and arrow signs: does Rust have enough of them, too
>    many, or both?

Um.  Maybe we should limit ourselves to productive discussions
that will affect the MM/filesystems/storage/bpf subsystems?
While it's entertaining to read about Rust-as-it-might-have-been
https://graydon2.dreamwidth.org/307291.html I quickly get lost in trying
to learn the last three decades of language design to understand all
the points being made.

>  - Obeying the borrow checker: C programmers are widely renouned for
>    creative idiosyncrasies in pointer based data structures; structure
>    design; will we be able to fully express our creativity within Rust?
>    Or are there VFS abstractions and lifetime rules that will be
>    difficult to translate?
> 
>  - Perhaps there are annoying VFS rules that in the past required hand
>    tattoos to remember, but we'll now be able to teach the compiler to
>    remember for us?
> 
>    (idmapping comes to mind as one area where we've recently been able
>    to increase safety through effective use of the type system - this is
>    about way more than just use-after-free bugs)
> 
>  - Moving objects around in memory: Rust rather dislikes being told
>    objects can't be moved around (a consequence of algebraic data
>    types), and this has been a source of no small amount of frustration
>    on the Rust side of things. Could we on the C side perhaps find ways
>    to relax their constraints? (object pinning, versus e.g. perhaps
>    making it possible to move a list head around)

Death To List Heads!  They're the perfect data structure for a 1995
era CPU.  They leave 90% of your CPUs performance on the table if you
bought your CPU in the last five years.  If list heads make rust sad,
then that's just one more reason to abolish them.

>  - The use of outside library code: Historically, C code was either
>    written for userspace or the kernel, and not both. But that's not
>    particularly true in Rust land (and getting to be less true even in C
>    land); should we consider some sort of structure or (cough) package
>    management? Is it time to move beyond ye olde cut-and-paste?

Rust has a package manager.  I don't think we need kCargo.  I'm not
deep enough in the weeds on this to make sensible suggestions, but if
a package (eg a crypto suite or compression library) doesn't depend on
anything ridiculous then what's the harm in just pulling it in?

> The Rust-for-Linux people have been great - let's see if we can get them
> to join us in Utah this year, I think they'll have things to teach us.

It's definitely a two-way conversation.  There's too much for any one
person to know, so teaching each other what "we" need "them" to know
is a great thing.  I hear some people are considering pitching sessions
like "What filesystem people get wrong about MM", "The pagecache doesn't
work the way you think it does" and "Why directio and the pagecache are
mortal enemies".  And I'm in favour of those kinds of sessions (as long
as I don't end up leading five sessions again ...).

Re: [LSF/MM TOPIC] Rust

[Boqun Feng]

[...]
> >  - The use of outside library code: Historically, C code was either
> >    written for userspace or the kernel, and not both. But that's not
> >    particularly true in Rust land (and getting to be less true even in C
> >    land); should we consider some sort of structure or (cough) package
> >    management? Is it time to move beyond ye olde cut-and-paste?
> 
> Rust has a package manager.  I don't think we need kCargo.  I'm not
> deep enough in the weeds on this to make sensible suggestions, but if
> a package (eg a crypto suite or compression library) doesn't depend on
> anything ridiculous then what's the harm in just pulling it in?
> 

If we are talking about using a external library in kernel, then one of
the concerns is aduitting/reviewing external dependencies I think.

However I just want to point another way that userspace and kernel can
share the code: we can put the Rust code in kernel, and pulish it as
a crate (https://crates.io/) so that userspace can use. This would be
ideal for things like on-disk layout for filesystems for example, where
we maintain the data structure in kernel source code, and userspace can
use the same code directly. Probably this is not what Kent asked for
though ;-)

Regards,
Boqun

Re: [LSF/MM TOPIC] Rust

[David Howells]

Matthew Wilcox <willy@infradead.org> wrote:

> I really want this to happen.  It's taken 50 years, but we finally have
> a programming language that can replace C for writing kernels.

I really don't want this to happen.  Whilst I have sympathy with the idea that
C can be replaced with something better - Rust isn't it.  The syntax is awful.
It's like they looked at perl and thought they could beat it at inventing
weird and obfuscated bits of operator syntax.  Can't they replace the syntax
with something a lot more C-like[*]?

But quite apart from that, mass-converting the kernel to Rust is pretty much
inevitably going introduce a whole bunch of new bugs.

David

[*] That said, we do rather torture the C-preprocessor more than we should
have to if the C language was more flexible.  Some of that could be alleviated
by moving to C++ and using some of the extra features available there.  That
would be an easier path than rusting the kernel.

Re: [LSF/MM TOPIC] Rust

[Boqun Feng]

On Tue, Jan 23, 2024 at 10:58:09PM +0000, David Howells wrote:
> Matthew Wilcox <willy@infradead.org> wrote:
> 
> > I really want this to happen.  It's taken 50 years, but we finally have
> > a programming language that can replace C for writing kernels.
> 

(I'm not sure Matthew wants to rewrite the existing kernel piece in
Rust, my read is more like he feels Rust can be used for new or
experimental stuffs)

> I really don't want this to happen.  Whilst I have sympathy with the idea that
> C can be replaced with something better - Rust isn't it.  The syntax is awful.
> It's like they looked at perl and thought they could beat it at inventing
> weird and obfuscated bits of operator syntax.  Can't they replace the syntax
> with something a lot more C-like[*]?
> 

Isn't the feeling on the syntax (like, hate or can live with) really
based on personal experience? I'd rather not use this as an argument,
since I can find syntax haters for every language ;-)

> But quite apart from that, mass-converting the kernel to Rust is pretty much
> inevitably going introduce a whole bunch of new bugs.
> 

Desite whether this is what gets proposed here, I do really want to
agree with you, but I'm not able to tell whether this is an educational
prediction or unnecessary worry, since I could say the same thing for
every patchset that adds new features ;-)

To me, it doesn't matter which language wins the "best C replacement for
kernel programming" award, the lessons we learn from Rust-for-Linux will
likely apply for any other "high-level" language. Hope that we can all
agree on that it's all OK that people want to try out new stuffs and see
if they *actually* work. Because then we can discuss on something
concrete and objective.

Regards,
Boqun

> David
> 
> [*] That said, we do rather torture the C-preprocessor more than we should
> have to if the C language was more flexible.  Some of that could be alleviated
> by moving to C++ and using some of the extra features available there.  That
> would be an easier path than rusting the kernel.
> 
> 

Re: [LSF/MM TOPIC] Rust

[James Bottomley]

On Tue, 2024-01-23 at 19:09 +0000, Matthew Wilcox wrote:
> >   - The use of outside library code: Historically, C code was
> > either written for userspace or the kernel, and not both. But
> > that's not particularly true in Rust land (and getting to be less
> > true even in C land); should we consider some sort of structure or
> > (cough) package management? Is it time to move beyond ye olde cut-
> > and-paste?
> 
> Rust has a package manager.  I don't think we need kCargo.  I'm not
> deep enough in the weeds on this to make sensible suggestions, but if
> a package (eg a crypto suite or compression library) doesn't depend
> on anything ridiculous then what's the harm in just pulling it in?

The problem with this is that it leads to combinatoric explosions and
multiple copies of everything[1].  For crypto in particular the last
thing you want to do is pull some random encryption routine off the
internet, particularly if the kernel already supplies it because it's
usually not properly optimized for your CPU and it makes it a nightmare
to deduce the security properties of the system.

However, there's nothing wrong with a vetted approach to this: keep a
list of stuff rust needs, make sure it's properly plumbed in to the
kernel routines (which likely necessitates package changes) and keep it
somewhere everyone can use.

James

[1] just to support this point, I maintain a build of element-desktop
that relies on node (which uses the same versioned package management
style rust does).  It pulls in 2115 packages of which 417 are version
duplicates (same package but different version numbers).

Re: [LSF/MM TOPIC] Rust

[Matthew Wilcox]

On Wed, Jan 24, 2024 at 09:26:34AM -0500, James Bottomley wrote:
> On Tue, 2024-01-23 at 19:09 +0000, Matthew Wilcox wrote:
> > >   - The use of outside library code: Historically, C code was
> > > either written for userspace or the kernel, and not both. But
> > > that's not particularly true in Rust land (and getting to be less
> > > true even in C land); should we consider some sort of structure or
> > > (cough) package management? Is it time to move beyond ye olde cut-
> > > and-paste?
> > 
> > Rust has a package manager.  I don't think we need kCargo.  I'm not
> > deep enough in the weeds on this to make sensible suggestions, but if
> > a package (eg a crypto suite or compression library) doesn't depend
> > on anything ridiculous then what's the harm in just pulling it in?
> 
> The problem with this is that it leads to combinatoric explosions and
> multiple copies of everything[1].

OK, but why do we care?  We still have buffer_heads in the kernel (v1.0
of the block layer abstraction) while also have bios, iomap and numerous
NIH in various filesystems.  I don't even know if it's going to be
quantitatively worse.

> For crypto in particular the last
> thing you want to do is pull some random encryption routine off the
> internet, particularly if the kernel already supplies it because it's
> usually not properly optimized for your CPU and it makes it a nightmare
> to deduce the security properties of the system.

That seems like a strawman.  Why is it _so_ much worse to have your kernel
compromised than your web browser, your email client, or your corporate
authentication provider?  Why would we allow code in that pulls in random
shit from the internet instead of the vetted stuff on crates.io?

> However, there's nothing wrong with a vetted approach to this: keep a
> list of stuff rust needs, make sure it's properly plumbed in to the
> kernel routines (which likely necessitates package changes) and keep it
> somewhere everyone can use.

... like crates.io.  Why are we better at this than they are?

> [1] just to support this point, I maintain a build of element-desktop
> that relies on node (which uses the same versioned package management
> style rust does).  It pulls in 2115 packages of which 417 are version
> duplicates (same package but different version numbers).

I'd suggest that node.js has a very different approach from crates.io.
I don't see there being a rust left-pad.

Re: [LSF/MM TOPIC] Rust

[James Bottomley]

On Wed, 2024-01-24 at 15:43 +0000, Matthew Wilcox wrote:
> On Wed, Jan 24, 2024 at 09:26:34AM -0500, James Bottomley wrote:
> > On Tue, 2024-01-23 at 19:09 +0000, Matthew Wilcox wrote:
> > > >   - The use of outside library code: Historically, C code was
> > > > either written for userspace or the kernel, and not both. But
> > > > that's not particularly true in Rust land (and getting to be
> > > > less true even in C land); should we consider some sort of
> > > > structure or (cough) package management? Is it time to move
> > > > beyond ye olde cut- and-paste?
> > > 
> > > Rust has a package manager.  I don't think we need kCargo.  I'm
> > > not deep enough in the weeds on this to make sensible
> > > suggestions, but if a package (eg a crypto suite or compression
> > > library) doesn't depend on anything ridiculous then what's the
> > > harm in just pulling it in?
> > 
> > The problem with this is that it leads to combinatoric explosions
> > and multiple copies of everything[1].
> 
> OK, but why do we care?  We still have buffer_heads in the kernel
> (v1.0 of the block layer abstraction) while also have bios, iomap and
> numerous NIH in various filesystems.  I don't even know if it's going
> to be quantitatively worse.

Multiple copies lead to kernel bloat and problems for embedded systems
as well as security problems.

> > For crypto in particular the last thing you want to do is pull some
> > random encryption routine off the internet, particularly if the
> > kernel already supplies it because it's usually not properly
> > optimized for your CPU and it makes it a nightmare to deduce the
> > security properties of the system.
> 
> That seems like a strawman.  Why is it _so_ much worse to have your
> kernel compromised than your web browser, your email client, or your
> corporate authentication provider?

If I follow that argument to the logical conclusion you're saying
security of our crypto functions doesn't matter that much because
others also get it wrong?  I'd say that might be slightly controversial
...

To illustrate the problem with cryptography in rust: just because it's
rust safe doesn't mean its correct or bug free.  Crypto functions are
the most difficult to get right (algorithmically, regardless of memory
safety).  Look at this Medium report on the top ten bugs in blockchain:

https://medium.com/rektoff/top-10-vulnerabilities-in-substrate-based-blockchains-using-rust-d454279521ff

Number 1 is a rust crypto vulnerability due to insecure randomness in a
random number generating function (note it was rust safe code just not
properly checked for algorithmic issues by a cryptographer).

The reason for using the kernel functions is that they are vetted by
cryptographers and crafted for our environment.

>   Why would we allow code in that pulls in random shit from the
> internet instead of the vetted stuff on crates.io?

The pallet problem in the blockchain bug came from crates.io.

> > However, there's nothing wrong with a vetted approach to this: keep
> > a list of stuff rust needs, make sure it's properly plumbed in to
> > the kernel routines (which likely necessitates package changes) and
> > keep it somewhere everyone can use.
> 
> ... like crates.io.  Why are we better at this than they are?

The volume is way smaller so scrutiny can be way greater and they have
to be crafted for our environment anyway.

> > [1] just to support this point, I maintain a build of element-
> > desktop that relies on node (which uses the same versioned package
> > management style rust does).  It pulls in 2115 packages of which
> > 417 are version duplicates (same package but different version
> > numbers).
> 
> I'd suggest that node.js has a very different approach from
> crates.io.
> I don't see there being a rust left-pad.

Really?  crates.io currently has 135,010 packages which can all be
uploaded and changed instantly by their respective owners.  Security
vetting is mostly supposed to be done by the uploaders (it can't be
done by the repo since there are so many packages) ... is this starting
to sound familiar?  because it's the same security policy all the web
package repositories have.  Sure they've got safeties in place for
left-pad essential package removals problems, but they could still get
a log4j issue.

James

Re: [LSF/MM TOPIC] Rust

[Kent Overstreet]

On Wed, Jan 24, 2024 at 11:04:14AM -0500, James Bottomley wrote:
> On Wed, 2024-01-24 at 15:43 +0000, Matthew Wilcox wrote:
> > On Wed, Jan 24, 2024 at 09:26:34AM -0500, James Bottomley wrote:
> > > On Tue, 2024-01-23 at 19:09 +0000, Matthew Wilcox wrote:
> > > > >   - The use of outside library code: Historically, C code was
> > > > > either written for userspace or the kernel, and not both. But
> > > > > that's not particularly true in Rust land (and getting to be
> > > > > less true even in C land); should we consider some sort of
> > > > > structure or (cough) package management? Is it time to move
> > > > > beyond ye olde cut- and-paste?
> > > > 
> > > > Rust has a package manager.  I don't think we need kCargo.  I'm
> > > > not deep enough in the weeds on this to make sensible
> > > > suggestions, but if a package (eg a crypto suite or compression
> > > > library) doesn't depend on anything ridiculous then what's the
> > > > harm in just pulling it in?
> > > 
> > > The problem with this is that it leads to combinatoric explosions
> > > and multiple copies of everything[1].
> > 
> > OK, but why do we care?  We still have buffer_heads in the kernel
> > (v1.0 of the block layer abstraction) while also have bios, iomap and
> > numerous NIH in various filesystems.  I don't even know if it's going
> > to be quantitatively worse.
> 
> Multiple copies lead to kernel bloat and problems for embedded systems
> as well as security problems.

Yes, but it's a problem we already have, and sure introducing a package
manager might lead to lead to more duplication and bloat if we don't
bother to care about this issue.

But introducing a package manager also means we'll have standard tooling
for checking for duplicated dependencies, so if we make sure to use the
tooling and make it part of the review process we should be fine.

It wouldn't be hard to make checkpatch check for this on changes to
Cargo.lock.

> > > For crypto in particular the last thing you want to do is pull some
> > > random encryption routine off the internet, particularly if the
> > > kernel already supplies it because it's usually not properly
> > > optimized for your CPU and it makes it a nightmare to deduce the
> > > security properties of the system.
> > 
> > That seems like a strawman.  Why is it _so_ much worse to have your
> > kernel compromised than your web browser, your email client, or your
> > corporate authentication provider?
> 
> If I follow that argument to the logical conclusion you're saying
> security of our crypto functions doesn't matter that much because
> others also get it wrong?  I'd say that might be slightly controversial

I think the argument is more just "why exactly is the kernel special
here?". 

> To illustrate the problem with cryptography in rust: just because it's
> rust safe doesn't mean its correct or bug free.  Crypto functions are
> the most difficult to get right (algorithmically, regardless of memory
> safety).  Look at this Medium report on the top ten bugs in blockchain:
> 
> https://medium.com/rektoff/top-10-vulnerabilities-in-substrate-based-blockchains-using-rust-d454279521ff
> 
> Number 1 is a rust crypto vulnerability due to insecure randomness in a
> random number generating function (note it was rust safe code just not
> properly checked for algorithmic issues by a cryptographer).
> 
> The reason for using the kernel functions is that they are vetted by
> cryptographers and crafted for our environment.

Are you arguing that typical kernel code is more secure than typical
Rust code?

> >   Why would we allow code in that pulls in random shit from the
> > internet instead of the vetted stuff on crates.io?
> 
> The pallet problem in the blockchain bug came from crates.io.
> 
> > > However, there's nothing wrong with a vetted approach to this: keep
> > > a list of stuff rust needs, make sure it's properly plumbed in to
> > > the kernel routines (which likely necessitates package changes) and
> > > keep it somewhere everyone can use.
> > 
> > ... like crates.io.  Why are we better at this than they are?
> 
> The volume is way smaller so scrutiny can be way greater and they have
> to be crafted for our environment anyway.

No, and being special snowflakes isn't helpful.

They don't in general have to be crafted for our environment, and we're
slowly working to reduce the differences between the kernel environment
and userspace (gfp flags).

We can and should have our own review process when pulling in new
dependencies, but we shouldn't otherwise be making it difficult to use
crates.io dependencies just for the sake of it.

> Really?  crates.io currently has 135,010 packages which can all be
> uploaded and changed instantly by their respective owners.  Security
> vetting is mostly supposed to be done by the uploaders (it can't be
> done by the repo since there are so many packages) ... is this starting
> to sound familiar?  because it's the same security policy all the web
> package repositories have.  Sure they've got safeties in place for
> left-pad essential package removals problems, but they could still get
> a log4j issue.

Comitting the cargo lockfile pins your dependencies to an exact git
revision, and then updating to new versions of dependencies requires a
new commit in our repository. We have the means to do our own review
here.

Re: Re: [LSF/MM TOPIC] Rust

[Morten Linderud]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

On Wed, Jan 24, 2024 at 01:50:55PM -0500, Kent Overstreet wrote:
>
> We can and should have our own review process when pulling in new
> dependencies, but we shouldn't otherwise be making it difficult to use
> crates.io dependencies just for the sake of it.

One aspect I find overlooked is that downstream distros largely devendor Rust
dependencies from packages. Fedora and Debian comes to mind. Which means that
when Rust in the kernel is something that is turned on by downstream they would
need to deal with these vendored dependencies.

Is the intent here that `cargo` is pulling down dependencies from cargo.io or is
it vendored as part of the source-tree but managed by cargo?

Alternatively, is the dependencies included in the tarball?

The kernel being self-contained is also quite a nice property, and I'm not sure
if breaking this property because of cargo is a good enough reason? If you don't
vendor it as part of the source-tree then the kernel build will require an
network connection to build.

There is also the security handling aspect of depending on cargo.io as any
security issues in these dependencies would need to trigger kernel releases to
ensure they are patched.

I don't have any solutions, but I think there are a issues that needs to be
dealt with when considering pulling down external dependencies.

-- 
Morten Linderud
PGP: 9C02FF419FECBE16

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [LSF/MM TOPIC] Rust

[James Bottomley]

On Wed, 2024-01-24 at 13:50 -0500, Kent Overstreet wrote:
> > To illustrate the problem with cryptography in rust: just because
> > it's rust safe doesn't mean its correct or bug free.  Crypto
> > functions are the most difficult to get right (algorithmically,
> > regardless of memory safety).  Look at this Medium report on the
> > top ten bugs in blockchain:
> > 
> > https://medium.com/rektoff/top-10-vulnerabilities-in-substrate-based-blockchains-using-rust-d454279521ff
> > 
> > Number 1 is a rust crypto vulnerability due to insecure randomness
> > in a random number generating function (note it was rust safe code
> > just not properly checked for algorithmic issues by a
> > cryptographer).
> > 
> > The reason for using the kernel functions is that they are vetted
> > by cryptographers and crafted for our environment.
> 
> Are you arguing that typical kernel code is more secure than typical
> Rust code?

For crypto code?  Absolutely, that's what the example above showed. 
It's pretty much impossible to use an insecure rng in the kernel if you
plug into one of our existing APIs.  That's obviously not necessarily
true if you pull a random one from crates.io.

James

Re: [LSF/MM TOPIC] Rust

[Kent Overstreet]

On Wed, Jan 24, 2024 at 02:43:21PM -0500, James Bottomley wrote:
> On Wed, 2024-01-24 at 13:50 -0500, Kent Overstreet wrote:
> > > To illustrate the problem with cryptography in rust: just because
> > > it's rust safe doesn't mean its correct or bug free.  Crypto
> > > functions are the most difficult to get right (algorithmically,
> > > regardless of memory safety).  Look at this Medium report on the
> > > top ten bugs in blockchain:
> > > 
> > > https://medium.com/rektoff/top-10-vulnerabilities-in-substrate-based-blockchains-using-rust-d454279521ff
> > > 
> > > Number 1 is a rust crypto vulnerability due to insecure randomness
> > > in a random number generating function (note it was rust safe code
> > > just not properly checked for algorithmic issues by a
> > > cryptographer).
> > > 
> > > The reason for using the kernel functions is that they are vetted
> > > by cryptographers and crafted for our environment.
> > 
> > Are you arguing that typical kernel code is more secure than typical
> > Rust code?
> 
> For crypto code?  Absolutely, that's what the example above showed. 
> It's pretty much impossible to use an insecure rng in the kernel if you
> plug into one of our existing APIs.  That's obviously not necessarily
> true if you pull a random one from crates.io.
> 
> James

I can just as easily use prandom.h instead of random.h in the kernel;
this just comes down to Rust not being able to save you from arbitrary
logic errors. But all the data we have so far from CVEs and bug reports
shows that Rust code is _dramatically_ more secure than any C code, even
kernel code.

Re: [LSF/MM TOPIC] Rust

[Kent Overstreet]

On Tue, Jan 23, 2024 at 10:58:09PM +0000, David Howells wrote:
> Matthew Wilcox <willy@infradead.org> wrote:
> 
> > I really want this to happen.  It's taken 50 years, but we finally have
> > a programming language that can replace C for writing kernels.
> 
> I really don't want this to happen.  Whilst I have sympathy with the idea that
> C can be replaced with something better - Rust isn't it.  The syntax is awful.
> It's like they looked at perl and thought they could beat it at inventing
> weird and obfuscated bits of operator syntax.  Can't they replace the syntax
> with something a lot more C-like[*]?

I've heard you say this before, and I'm still curious what it is you
dislike about Rust syntax... because to my eyes, Rust syntax /is/
already quite C-like and the differences are all obvious fixes and
cleanups for things that have annoyed me for years.

Fixing the statement/expression distinction is really nice, so we no
longer have to use the ridiculous ({}) construct, and that also means
the ternary operator is no longer needed because if and match are now
normal expressions that can have values.

Changing how functions and variables are defined is also a nice fix -
the type of the variable or return value no longer being the first token
means Rust can be parsed in one pass; that's always been a painful part
of the C grammar.

Gegarding macros, C macros are pretty terrible in that they can
completely violate the syntax and know nothing about the syntax tree;
Rust macros operate on token trees and are much more regular (they're
also hygenic!).

Syntax wise, the only annoying thing for me has been getting used to
borrow checker syntax, but I'm more than willing to put up with that for
what it gets us...

Re: [LSF/MM TOPIC] Rust

[James Bottomley]

On Wed, 2024-01-24 at 14:57 -0500, Kent Overstreet wrote:
> On Wed, Jan 24, 2024 at 02:43:21PM -0500, James Bottomley wrote:
> > On Wed, 2024-01-24 at 13:50 -0500, Kent Overstreet wrote:
> > > > To illustrate the problem with cryptography in rust: just
> > > > because it's rust safe doesn't mean its correct or bug free. 
> > > > Crypto functions are the most difficult to get right
> > > > (algorithmically, regardless of memory safety).  Look at this
> > > > Medium report on the top ten bugs in blockchain:
> > > > 
> > > > https://medium.com/rektoff/top-10-vulnerabilities-in-substrate-based-blockchains-using-rust-d454279521ff
> > > > 
> > > > Number 1 is a rust crypto vulnerability due to insecure
> > > > randomness in a random number generating function (note it was
> > > > rust safe code just not properly checked for algorithmic issues
> > > > by a cryptographer).
> > > > 
> > > > The reason for using the kernel functions is that they are
> > > > vetted by cryptographers and crafted for our environment.
> > > 
> > > Are you arguing that typical kernel code is more secure than
> > > typical Rust code?
> > 
> > For crypto code?  Absolutely, that's what the example above showed.
> > It's pretty much impossible to use an insecure rng in the kernel if
> > you plug into one of our existing APIs.  That's obviously not
> > necessarily true if you pull a random one from crates.io.
> > 
> > James
> 
> I can just as easily use prandom.h instead of random.h in the kernel;

Neither of which would be insecure ...

> this just comes down to Rust not being able to save you from
> arbitrary logic errors. But all the data we have so far from CVEs and
> bug reports shows that Rust code is _dramatically_ more secure than
> any C code, even kernel code.

I've said it thrice the bellman cried and what I tell you three times
is true.

Back in the real world, the literature seems to show that rust code has
about the same bug density as any other code (including C). 
Ironically, memory safety is still an issue because of the inability to
reduce unsafe areas in rust code.  I suspect the density is high simply
because the rust code is newer (bug density in new code tends to be
higher simply due to the human input rate of algorithmic defects), so
this may evolve better over time, but it doesn't change the calculus
that older more vetted code is better than rewriting that code in rust
because the rewrite tends to introduce new bugs.

James

Re: [LSF/MM TOPIC] Rust

[Kent Overstreet]

On Wed, Jan 24, 2024 at 10:47:22PM -0500, James Bottomley wrote:
> On Wed, 2024-01-24 at 14:57 -0500, Kent Overstreet wrote:
> > On Wed, Jan 24, 2024 at 02:43:21PM -0500, James Bottomley wrote:
> > > On Wed, 2024-01-24 at 13:50 -0500, Kent Overstreet wrote:
> > > > > To illustrate the problem with cryptography in rust: just
> > > > > because it's rust safe doesn't mean its correct or bug free. 
> > > > > Crypto functions are the most difficult to get right
> > > > > (algorithmically, regardless of memory safety).  Look at this
> > > > > Medium report on the top ten bugs in blockchain:
> > > > > 
> > > > > https://medium.com/rektoff/top-10-vulnerabilities-in-substrate-based-blockchains-using-rust-d454279521ff
> > > > > 
> > > > > Number 1 is a rust crypto vulnerability due to insecure
> > > > > randomness in a random number generating function (note it was
> > > > > rust safe code just not properly checked for algorithmic issues
> > > > > by a cryptographer).
> > > > > 
> > > > > The reason for using the kernel functions is that they are
> > > > > vetted by cryptographers and crafted for our environment.
> > > > 
> > > > Are you arguing that typical kernel code is more secure than
> > > > typical Rust code?
> > > 
> > > For crypto code?  Absolutely, that's what the example above showed.
> > > It's pretty much impossible to use an insecure rng in the kernel if
> > > you plug into one of our existing APIs.  That's obviously not
> > > necessarily true if you pull a random one from crates.io.
> > > 
> > > James
> > 
> > I can just as easily use prandom.h instead of random.h in the kernel;
> 
> Neither of which would be insecure ...

Are you claiming that
        
/* Pseudo random number generator from numerical recipes. */
static inline u32 next_pseudo_random32(u32 seed)
{
        return seed * 1664525 + 1013904223;
}

is a secure RNG?

> 
> > this just comes down to Rust not being able to save you from
> > arbitrary logic errors. But all the data we have so far from CVEs and
> > bug reports shows that Rust code is _dramatically_ more secure than
> > any C code, even kernel code.
> 
> I've said it thrice the bellman cried and what I tell you three times
> is true.
> 
> Back in the real world, the literature seems to show that rust code has
> about the same bug density as any other code (including C). 

You might want to re-read that literature...

Re: [LSF/MM TOPIC] Rust

[James Bottomley]

On Thu, 2024-01-25 at 04:24 -0500, Kent Overstreet wrote:
> On Wed, Jan 24, 2024 at 10:47:22PM -0500, James Bottomley wrote:
> > On Wed, 2024-01-24 at 14:57 -0500, Kent Overstreet wrote:
> > > On Wed, Jan 24, 2024 at 02:43:21PM -0500, James Bottomley wrote:
> > > > On Wed, 2024-01-24 at 13:50 -0500, Kent Overstreet wrote:
> > > > > > To illustrate the problem with cryptography in rust: just
> > > > > > because it's rust safe doesn't mean its correct or bug
> > > > > > free.  Crypto functions are the most difficult to get right
> > > > > > (algorithmically, regardless of memory safety).  Look at
> > > > > > this Medium report on the top ten bugs in blockchain:
> > > > > > 
> > > > > > https://medium.com/rektoff/top-10-vulnerabilities-in-substrate-based-blockchains-using-rust-d454279521ff
> > > > > > 
> > > > > > Number 1 is a rust crypto vulnerability due to insecure
> > > > > > randomness in a random number generating function (note it
> > > > > > was rust safe code just not properly checked for
> > > > > > algorithmic issues by a cryptographer).
> > > > > > 
> > > > > > The reason for using the kernel functions is that they are
> > > > > > vetted by cryptographers and crafted for our environment.
> > > > > 
> > > > > Are you arguing that typical kernel code is more secure than
> > > > > typical Rust code?
> > > > 
> > > > For crypto code?  Absolutely, that's what the example above
> > > > showed. It's pretty much impossible to use an insecure rng in
> > > > the kernel if you plug into one of our existing APIs.  That's
> > > > obviously not necessarily true if you pull a random one from
> > > > crates.io.
> > > > 
> > > > James
> > > 
> > > I can just as easily use prandom.h instead of random.h in the
> > > kernel;
> > 
> > Neither of which would be insecure ...
> 
> Are you claiming that
>         
> /* Pseudo random number generator from numerical recipes. */
> static inline u32 next_pseudo_random32(u32 seed)
> {
>         return seed * 1664525 + 1013904223;
> }
> 
> is a secure RNG?

It's the best linear congruence generator for 32 bits, yes; it's
straight out of Knuth.   However, I assume you picked that one becuase
you know it's only used in testing to generate a repeatable series of
numbers.  The actual prandom_ API is based on an internally seeded
linear feedback shift register algorithm.

The trick to using PRNGs correctly is 1) knowing you can use them and
don't need cryptographic randomness and 2) seeding them correctly,
which is where the rust problem came from and which you can't get wrong
in Linux because it's done directly from the entropy pool.

James