Re: [RFC v3 13/32] rust: use vendored-sources

From: Ian Jackson <iwj@xenproject.org>
To: "Daniel P. Berrangé" <berrange@redhat.com>
Cc: "Peter Maydell" <peter.maydell@linaro.org>,
	"QEMU Developers" <qemu-devel@nongnu.org>,
	"Markus Armbruster" <armbru@redhat.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	"Marc-André Lureau" <marcandre.lureau@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>
Subject: Re: [RFC v3 13/32] rust: use vendored-sources
Date: Thu, 9 Sep 2021 18:02:28 +0100	[thread overview]
Message-ID: <24890.15908.689806.862379@mariner.uk.xensource.com> (raw)
In-Reply-To: <YTo7C9h6DxZDYdL9@redhat.com>

Daniel P. Berrangé writes ("Re: [RFC v3 13/32] rust: use vendored-sources"):
> Yes, distros do have machinery for this, although it is often
> hard to fit in with it when you have a mixed language project.
> Their machinery typically assumes pure single language project,
> so would work nicer if any QEMU rust pieces were separately
> released from the rest of QEMU. Obviously this is easier said
> than done since QEMU tends towards a monolothic repo approach
> historically.

Right.

However, for a project that has Rust dependencies, the distros will
(or will soon need) machinery to divert the
langage-specific-package-manager downloads to their own repo.  For
example, the Debian Rust team provide a .cargo/config.toml to replace
crates.io with the local Debian Rust packages, which the Debian
package management system has provided via the (translated)
build-dependencies.

Debian certainly wouldn't want to use any vendored crates bundled with
Qemu.  Indeed Debian people hate vendoring more than they hate
language specific package managers.  At least with the LSPM you can
usually nobble it in one place - ie many of the problems can be solved
automatically.  Vendoring typically involves playing whack-a-mole with
compatibility problems, actually-modified versions, etc. - much human
work (and quite annoying work too!)  (Of course this is less of an
issue if you don't actually modify the vendored code, but anyone who
knows Rust and sees a vendored Rust crate will think it's been
modified.)

> > (I'm not personally a fan of the "download everything from crates.io"
> > Rust ecosystem, but it is what it is, and wishing the Rust world
> > worked more like a trad Linux-distro-provides-all-your-dependencies
> > isn't, alas, going to make it so :-))
> 
> Yes, I'm inclined to agree here. For better or worse the battle is
> over and "download everything from <repo> on the fly" is the accepted
> approach for pretty much all modern languages. The language specific
> repo essentially is the OS distro from their POV.

To be honest, I am of the same mind as you about cargo and crates.io
(and things like it) But I think the ship has sailed.

At least, committing the Cargo.lock file will ensure that the same
versions of the dependencies are used - at least by people who don't
know anything much about Rust.

Marc-André Lureau writes ("Re: [RFC v3 13/32] rust: use vendored-sources"):
> A nice alternative to vendoring that could work well for QEMU is to
> mirror the Rust crate we use, so we have similar control and
> guarantee over them as submodules, and use `[patch.crates-io]` to
> point at qemu-project locations.

This is a very reasonable suggestion.  In my experience crates.io is
very reliable - but (as a test system onwer myself) I know how much
you want to reduce the number of different sites whose upness your CI
depends on, no matter how good their communities think they are :-).

I think there should be a documented config option to disable this.
People who know enough Rust to run `cargo update` etc. will need it,
and having them hand-hack the config files is not really desirable.

Ian.