Re: iotests and python dependencies

From: "Daniel P. Berrangé" <berrange@redhat.com>
To: John Snow <jsnow@redhat.com>
Cc: Qemu-block <qemu-block@nongnu.org>,
	qemu-devel <qemu-devel@nongnu.org>, Kevin Wolf <kwolf@redhat.com>,
	Hanna Reitz <hreitz@redhat.com>,
	Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Cleber Rosa <crosa@redhat.com>
Subject: Re: iotests and python dependencies
Date: Thu, 5 May 2022 13:33:30 +0100	[thread overview]
Message-ID: <YnPEGrwa9KVyup6T@redhat.com> (raw)
In-Reply-To: <CAFn=p-bBCbokmZ8FeHon6FBGxp38_z4=vmDNMi5vKKF_1KjQLQ@mail.gmail.com>

On Thu, May 05, 2022 at 08:08:42AM -0400, John Snow wrote:
> On Thu, May 5, 2022, 4:09 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
> 
> > On Wed, May 04, 2022 at 03:38:45PM -0400, John Snow wrote:
> > > Howdy!
> > >
> > > So, I want to finally delete python/qemu/qmp from qemu.git, and this
> > > creates a small problem -- namely, iotests needs access to it in order
> > > to run the python-based tests.
> > >
> > > What I think needs to happen is that we create a virtual environment
> > > that installs python/qemu/. The reason this cannot be done with
> > > PYTHONPATH alone anymore is because the qmp package itself won't be
> > > there anymore, we need an installer like `pip` to actually fetch it
> > > for us and put it somewhere. (i.e., we need to process the
> > > dependencies of python/qemu now and can't treat it as a pre-installed
> > > location.)
> >
> > Having pip fetch it on the fly creates a problem for RPM builds,
> > because the koji build env has no network access. We will, however,
> > have an RPM of python-qemu-qmp installed on the host system though.
> > IOW we need to be able to run iotests using system python and its
> > installed libs, not a virtual env.  So if we do anything with a
> > virtual env, it will need to be optional I believe.
> >
> 
> Hm, do we need iotests during an rpm build? Is it because of "make check"?

Yes, and this is good, because it prevents us from outputting an
RPM build that has a broken QEMU in it. 

> It's possible to create a venv and run pip in no-network mode, too. If the
> package we want is installed on the system or otherwise in pip's cache,
> it'll succeed without network. If the dependencies require a qemu.qmp
> that's too new, the pip install will just fail instead.
> 
> I have to test a way to craft a pip statement that's network *optional*
> though. i.e. try to fetch and fall back to local otherwise. I think it's
> worth trying to keep the environment setup code unified, and always use a
> venv.

As long as it is no-network, that's good enough.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|