Rust in Qemu BoF followup 2: Rust toolchain availability

Rust in Qemu BoF followup 2: Rust toolchain availability

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1364 bytes --]

Hi again all,

I've now done.. or at least started... the second part of my followup
from the KVM Forum BoF on Rust in Qemu.

I've extended the page at https://wiki.qemu.org/RustInQemu with
information on Rust toolchain availability.  However, I found I had a
lot more open questions on this one, so there are quite a lot of gaps.

In particular:
 * I haven't so far figured out how to definitively check package
   information for RHEL & SLES (they're not covered by repology, and
   RHEL module structure confuses me, even as a RedHatter)
 * I'm not at all sure what criteria to use to consider something as
   having "good enough" rustup support, so that information is all
   blank so far
 * I've taken a bit of a stab in the dark about what Rust version is
   recent enough for our purposes (1.31.0).  I strongly suspect we're
   going to want to move that to something more recent, but I don't
   know what, which will mean revising a bunch of stuff
 * I'm not really convinced that the way I've formatted it is
    particularly good, but I haven't though of a better alternative.

Contributions to improving this would be most welcomed.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Rust in Qemu BoF followup 2: Rust toolchain availability

[Peter Maydell]

On Thu, 30 Sept 2021 at 03:21, David Gibson <david@gibson.dropbear.id.au> wrote:
>
> Hi again all,
>
> I've now done.. or at least started... the second part of my followup
> from the KVM Forum BoF on Rust in Qemu.
>
> I've extended the page at https://wiki.qemu.org/RustInQemu with
> information on Rust toolchain availability.  However, I found I had a
> lot more open questions on this one, so there are quite a lot of gaps.

Thanks for doing this work.

I note that we have a new host architecture loongarch64 currently
circling to land. It looks like this is in the "Rust does not have
support for this target" category at the moment.

> In particular:
>  * I haven't so far figured out how to definitively check package
>    information for RHEL & SLES (they're not covered by repology, and
>    RHEL module structure confuses me, even as a RedHatter)
>  * I'm not at all sure what criteria to use to consider something as
>    having "good enough" rustup support, so that information is all
>    blank so far

I guess the answer here is probably "if it has at least the
glibc and kernel minimum versions that are documented as
required for the target-triple in the lists at
https://doc.rust-lang.org/rustc/platform-support.html then
that's 'good enough'".

>  * I've taken a bit of a stab in the dark about what Rust version is
>    recent enough for our purposes (1.31.0).  I strongly suspect we're
>    going to want to move that to something more recent, but I don't
>    know what, which will mean revising a bunch of stuff

Hmm. The more we feel we need features that are from a new
version of Rust the more nervous I get about whether this is
really something we want to jump to just yet. 1.31 is less than
3 years old, which is pretty recent in the context of QEMU's
own distro support policy.

>  * I'm not really convinced that the way I've formatted it is
>     particularly good, but I haven't though of a better alternative.

I guess we might also care about the CPU architecture per distro,
in that even if old version X of the distro supports architecture Y
and rust today supports architecture Y, that doesn't mean that
distro version X's shipped rust supported architecture Y.
But we can probably reasonably say "use rustup" if there are
corner cases like that.

-- PMM

Re: Rust in Qemu BoF followup 2: Rust toolchain availability

[Daniel P. Berrangé]

On Thu, Sep 30, 2021 at 11:59:42AM +1000, David Gibson wrote:
> Hi again all,
> 
> I've now done.. or at least started... the second part of my followup
> from the KVM Forum BoF on Rust in Qemu.
> 
> I've extended the page at https://wiki.qemu.org/RustInQemu with
> information on Rust toolchain availability.  However, I found I had a
> lot more open questions on this one, so there are quite a lot of gaps.
> 
> In particular:
>  * I haven't so far figured out how to definitively check package
>    information for RHEL & SLES (they're not covered by repology, and
>    RHEL module structure confuses me, even as a RedHatter)

Don't worry about RHEL/SLES directly wrt repology.

For RHEL, just use corresponding CentOS as a proxy

For SLES, just use corresponding openSUSE version as a proxy

>  * I'm not at all sure what criteria to use to consider something as
>    having "good enough" rustup support, so that information is all
>    blank so far
>  * I've taken a bit of a stab in the dark about what Rust version is
>    recent enough for our purposes (1.31.0).  I strongly suspect we're
>    going to want to move that to something more recent, but I don't
>    know what, which will mean revising a bunch of stuff

Our platform support matrix defines what distros we target.

IOW we would have a strong preference for a rust version that is present
in those distros. Essentially tests/docker/dockerfiles/*.Dockerfile
need to be able to pull in the rust packages from the distro, if
we are to make it mandatory.

If rust is to be strictly optional, then we have way more flexibility
in choosing the version. We do not need to cover all distros in the
support matrixk *provided* the rust is only used for new functionality
and we're not introducing it as a dependancy for existing functionality.

ie existing features previously released must keep working across all
distros in the matrix. new features from a release can set a higher
bar, since by definition it can't regress existing usage.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Rust in Qemu BoF followup 2: Rust toolchain availability

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3408 bytes --]

On Thu, Sep 30, 2021 at 11:30:20AM +0100, Peter Maydell wrote:
> On Thu, 30 Sept 2021 at 03:21, David Gibson <david@gibson.dropbear.id.au> wrote:
> >
> > Hi again all,
> >
> > I've now done.. or at least started... the second part of my followup
> > from the KVM Forum BoF on Rust in Qemu.
> >
> > I've extended the page at https://wiki.qemu.org/RustInQemu with
> > information on Rust toolchain availability.  However, I found I had a
> > lot more open questions on this one, so there are quite a lot of gaps.
> 
> Thanks for doing this work.
> 
> I note that we have a new host architecture loongarch64 currently
> circling to land. It looks like this is in the "Rust does not have
> support for this target" category at the moment.

AIUI this would be the new Loongson which is no longer MIPS
compatible, yes?  If so, then yes it appears not to have Rust support
at this time.

> > In particular:
> >  * I haven't so far figured out how to definitively check package
> >    information for RHEL & SLES (they're not covered by repology, and
> >    RHEL module structure confuses me, even as a RedHatter)
> >  * I'm not at all sure what criteria to use to consider something as
> >    having "good enough" rustup support, so that information is all
> >    blank so far
> 
> I guess the answer here is probably "if it has at least the
> glibc and kernel minimum versions that are documented as
> required for the target-triple in the lists at
> https://doc.rust-lang.org/rustc/platform-support.html then
> that's 'good enough'".

Seems reasonable.  I've updated by table based on that criterion,
though for now I haven't considered architectures - basically if the
distro matches the criterion for any architecture I've marked it as ok.

> >  * I've taken a bit of a stab in the dark about what Rust version is
> >    recent enough for our purposes (1.31.0).  I strongly suspect we're
> >    going to want to move that to something more recent, but I don't
> >    know what, which will mean revising a bunch of stuff
> 
> Hmm. The more we feel we need features that are from a new
> version of Rust the more nervous I get about whether this is
> really something we want to jump to just yet. 1.31 is less than
> 3 years old, which is pretty recent in the context of QEMU's
> own distro support policy.

That's a fair point.  I could be wrong about that.  I'm pretty sure
we'll want Rust 2018 edition support, which is why I used 1.31.0 as
the cutoff.  I know there are a bunch of things that have been
stabilized since then, but I don't know off hand of anything specific
we'll definitely want.

> >  * I'm not really convinced that the way I've formatted it is
> >     particularly good, but I haven't though of a better alternative.
> 
> I guess we might also care about the CPU architecture per distro,
> in that even if old version X of the distro supports architecture Y
> and rust today supports architecture Y, that doesn't mean that
> distro version X's shipped rust supported architecture Y.
> But we can probably reasonably say "use rustup" if there are
> corner cases like that.

Yeah.. that's where it starts to get very confusing.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Rust in Qemu BoF followup 2: Rust toolchain availability

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2782 bytes --]

On Thu, Sep 30, 2021 at 11:45:11AM +0100, Daniel P. Berrangé wrote:
> On Thu, Sep 30, 2021 at 11:59:42AM +1000, David Gibson wrote:
> > Hi again all,
> > 
> > I've now done.. or at least started... the second part of my followup
> > from the KVM Forum BoF on Rust in Qemu.
> > 
> > I've extended the page at https://wiki.qemu.org/RustInQemu with
> > information on Rust toolchain availability.  However, I found I had a
> > lot more open questions on this one, so there are quite a lot of gaps.
> > 
> > In particular:
> >  * I haven't so far figured out how to definitively check package
> >    information for RHEL & SLES (they're not covered by repology, and
> >    RHEL module structure confuses me, even as a RedHatter)
> 
> Don't worry about RHEL/SLES directly wrt repology.
> 
> For RHEL, just use corresponding CentOS as a proxy
> 
> For SLES, just use corresponding openSUSE version as a proxy

That makes sense, I've updated the table accordingly.

> >  * I'm not at all sure what criteria to use to consider something as
> >    having "good enough" rustup support, so that information is all
> >    blank so far
> >  * I've taken a bit of a stab in the dark about what Rust version is
> >    recent enough for our purposes (1.31.0).  I strongly suspect we're
> >    going to want to move that to something more recent, but I don't
> >    know what, which will mean revising a bunch of stuff
> 
> Our platform support matrix defines what distros we target.
> 
> IOW we would have a strong preference for a rust version that is present
> in those distros. Essentially tests/docker/dockerfiles/*.Dockerfile
> need to be able to pull in the rust packages from the distro, if
> we are to make it mandatory.

Works for me.  Let's definitely *not* be like Kata, which installs Go
& Rust from upstream binaries, and builds qemu & a guest kernel from
source in its CI runs.

> If rust is to be strictly optional, then we have way more flexibility
> in choosing the version. We do not need to cover all distros in the
> support matrixk *provided* the rust is only used for new functionality
> and we're not introducing it as a dependancy for existing functionality.

Right, but part of the point of this exercise is seeing if we can make
this mandatory, so that's the perspective I'm working from.

> ie existing features previously released must keep working across all
> distros in the matrix. new features from a release can set a higher
> bar, since by definition it can't regress existing usage.
> 
> 
> Regards,
> Daniel

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]