All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Snow <jsnow@redhat.com>
To: Kevin Wolf <kwolf@redhat.com>
Cc: Qemu-block <qemu-block@nongnu.org>,
	qemu-devel <qemu-devel@nongnu.org>,
	 Hanna Reitz <hreitz@redhat.com>,
	Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>,
	 Paolo Bonzini <pbonzini@redhat.com>,
	Cleber Rosa <crosa@redhat.com>
Subject: Re: iotests and python dependencies
Date: Thu, 5 May 2022 07:55:22 -0400	[thread overview]
Message-ID: <CAFn=p-adLFUCLvFgKtrKv3wzqWGTMJLUaE9v9_h-fBCbEddqwg@mail.gmail.com> (raw)
In-Reply-To: <YnOQAP1J94zH1pEK@redhat.com>

[-- Attachment #1: Type: text/plain, Size: 5289 bytes --]

On Thu, May 5, 2022, 4:51 AM Kevin Wolf <kwolf@redhat.com> wrote:

> Am 04.05.2022 um 21:38 hat John Snow geschrieben:
> > Howdy!
> >
> > So, I want to finally delete python/qemu/qmp from qemu.git, and this
> > creates a small problem -- namely, iotests needs access to it in order
> > to run the python-based tests.
> >
> > What I think needs to happen is that we create a virtual environment
> > that installs python/qemu/. The reason this cannot be done with
> > PYTHONPATH alone anymore is because the qmp package itself won't be
> > there anymore, we need an installer like `pip` to actually fetch it
> > for us and put it somewhere. (i.e., we need to process the
> > dependencies of python/qemu now and can't treat it as a pre-installed
> > location.)
> >
> > Avocado tests are already creating a venv for the purposes of
> > installing and running Avocado. We can amend e.g. "../../python" to
> > tests/requirements.txt and the Avocado environment is A-OK good-to-go.
> > The Makefile magic for avocado tests creates a venv-per-build.
>
> "per build" sounded pretty bad, because I thought it meant building a
> new venv every time I run either 'make' or the tests. This would
> obviously be very noticable for short-running tests like some iotests.
> But fortunately this is not what it does, it keeps the venv around in
> the build directory. So it's only per build directory really, which I
> guess is fine.
>

Whoops, yeah. I meant per build directory. In contrast to the Python unit
tests themselves, which create some test venvs tied directly to the source
directory and are build-agnostic.


> > It seems to work well enough. One thing to note here is that the
> > supported invocation for avocado tests is only through the Makefile,
> > which handles creating and entering the venv to make the command
> > seamless.
> >
> > iotests, however, manages its own execution environment with
> > testenv.py, and we support running iotests from outside of the
> > Makefile, for example by going to $build/tests/qemu-iotests and
> > running ./check.
>
> Yes, and I agree that this is important.
>

Figured as much. Not plotting to take this away, I promise. Just getting my
requirements straight before I spend time concocting something.


> > Now ... I could update testenv.py to be smart enough to create and
> > enter a python venv, and have even prototyped this. It seems to work
> > pretty well! This approach seemed like the least invasive to how
> > iotests are expected to be run and used. But a downside with this
> > approach is that now avocado tests and iotests are each managing their
> > own python venv. Worse, vm-tests and device-crash-test are still
> > unhandled entirely.
>
> Is there a reason why testenv.py couldn't enter just the shared venv in
> build/tests/venv?
>

It can, but it'd have to be made first so it can enter it. I figured this
would only happen "on-demand", like how check-avocado works, so I'd need
some way for iotests and the Makefile to share the venv creation code,
which is certainly an option.


> If not, I guess it would be enough if iotests just checks that the venv
> exists and all of the dependencies are there in the right version and
> error out if not, telling the user to run 'make check-venv'.
>

I will spend unreasonable engineering hours avoiding this, if only for
pride. I want everything to be as seamless and easy as it's always been.


> Or actually, it could just unconditionally run 'make check-venv' by
> itself, which is probably easier to implement than checking the
> dependencies and more convenient for the user, too.
>

Oh, that's one way to get them to share the venv-creation pathway. Hadn't
occurred to me, but this might be easy to do.


> (One more detail: 'make check-venv GIT_SUBMODULES_ACTION=ignore' seems
> to make it pretty much instantaneous if the venv is current, but leaving
> the submodule mechanism enabled adds a noticable delay.)
>

Noted.


> > I'd like to find a solution where I create a unified python testing
> > venv tied to the build shared by avocado, iotests, vm-tests and
> > device-crash-test. I'm not completely sure how exactly I'll manage
> > that right now, but I wanted to throw this out there in case there are
> > some requirements I might be overlooking.
> >
> > I think vm-tests and avocado-tests can both have a venv created for
> > them and activated before the test runs. device-crash-test I believe
> > will need a script change in the gitlab ci yaml. iotests is somewhat
> > unique in that it needs to run both by manual invocation and from
> > makefile invocations. If I want a shared VM between all of these, I'll
> > need to isolate the create-and-enter-venv logic somewhere where it can
> > be shared both inside and outside of a Makefile.
>
> If just calling 'make' isn't an option, then moving that part out into a
> separate script probably wouldn't be too hard either. But 'make' has the
> advantage that it already contains the check if the venv is already
> there and requirements.txt hasn't changed, which you would have to
> replicate otherwise.
>

Yeah, exactly.


> Kevin
>

Thanks for the ideas!

As of now, I've got a branch where I've moved vm-tests over onto the same
venv with avocado-tests. If I can bridge the gap to iotests, I'll be pretty
happy.

[-- Attachment #2: Type: text/html, Size: 8102 bytes --]

      parent reply	other threads:[~2022-05-05 11:57 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-04 19:38 iotests and python dependencies John Snow
2022-05-05  8:09 ` Daniel P. Berrangé
2022-05-05 12:08   ` John Snow
2022-05-05 12:33     ` Daniel P. Berrangé
2022-05-05 13:10       ` John Snow
2022-05-05 13:15         ` Paolo Bonzini
2022-05-05 14:13           ` John Snow
2022-05-05 15:50             ` Paolo Bonzini
2022-05-05 15:57               ` Daniel P. Berrangé
2022-05-08 14:23                 ` Paolo Bonzini
2022-05-05  8:51 ` Kevin Wolf
2022-05-05  9:28   ` Paolo Bonzini
2022-05-05 10:59     ` Kevin Wolf
2022-05-05 12:24       ` Paolo Bonzini
2022-05-05 12:39         ` Kevin Wolf
2022-05-10 17:40     ` John Snow
2022-05-05 11:55   ` John Snow [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAFn=p-adLFUCLvFgKtrKv3wzqWGTMJLUaE9v9_h-fBCbEddqwg@mail.gmail.com' \
    --to=jsnow@redhat.com \
    --cc=crosa@redhat.com \
    --cc=hreitz@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=v.sementsov-og@mail.ru \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.