* iotests and python dependencies @ 2022-05-04 19:38 John Snow 2022-05-05 8:09 ` Daniel P. Berrangé 2022-05-05 8:51 ` Kevin Wolf 0 siblings, 2 replies; 17+ messages in thread From: John Snow @ 2022-05-04 19:38 UTC (permalink / raw) To: Qemu-block Cc: qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Paolo Bonzini, Cleber Rosa Howdy! So, I want to finally delete python/qemu/qmp from qemu.git, and this creates a small problem -- namely, iotests needs access to it in order to run the python-based tests. What I think needs to happen is that we create a virtual environment that installs python/qemu/. The reason this cannot be done with PYTHONPATH alone anymore is because the qmp package itself won't be there anymore, we need an installer like `pip` to actually fetch it for us and put it somewhere. (i.e., we need to process the dependencies of python/qemu now and can't treat it as a pre-installed location.) Avocado tests are already creating a venv for the purposes of installing and running Avocado. We can amend e.g. "../../python" to tests/requirements.txt and the Avocado environment is A-OK good-to-go. The Makefile magic for avocado tests creates a venv-per-build. It seems to work well enough. One thing to note here is that the supported invocation for avocado tests is only through the Makefile, which handles creating and entering the venv to make the command seamless. iotests, however, manages its own execution environment with testenv.py, and we support running iotests from outside of the Makefile, for example by going to $build/tests/qemu-iotests and running ./check. Now ... I could update testenv.py to be smart enough to create and enter a python venv, and have even prototyped this. It seems to work pretty well! This approach seemed like the least invasive to how iotests are expected to be run and used. But a downside with this approach is that now avocado tests and iotests are each managing their own python venv. Worse, vm-tests and device-crash-test are still unhandled entirely. I'd like to find a solution where I create a unified python testing venv tied to the build shared by avocado, iotests, vm-tests and device-crash-test. I'm not completely sure how exactly I'll manage that right now, but I wanted to throw this out there in case there are some requirements I might be overlooking. I think vm-tests and avocado-tests can both have a venv created for them and activated before the test runs. device-crash-test I believe will need a script change in the gitlab ci yaml. iotests is somewhat unique in that it needs to run both by manual invocation and from makefile invocations. If I want a shared VM between all of these, I'll need to isolate the create-and-enter-venv logic somewhere where it can be shared both inside and outside of a Makefile. I'll see what I can cook up, but if you have any concerns or Cool Ideas, lemme know. I want to make sure this is as painless as I can think to make it. Thanks, --js ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-04 19:38 iotests and python dependencies John Snow @ 2022-05-05 8:09 ` Daniel P. Berrangé 2022-05-05 12:08 ` John Snow 2022-05-05 8:51 ` Kevin Wolf 1 sibling, 1 reply; 17+ messages in thread From: Daniel P. Berrangé @ 2022-05-05 8:09 UTC (permalink / raw) To: John Snow Cc: Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Paolo Bonzini, Cleber Rosa On Wed, May 04, 2022 at 03:38:45PM -0400, John Snow wrote: > Howdy! > > So, I want to finally delete python/qemu/qmp from qemu.git, and this > creates a small problem -- namely, iotests needs access to it in order > to run the python-based tests. > > What I think needs to happen is that we create a virtual environment > that installs python/qemu/. The reason this cannot be done with > PYTHONPATH alone anymore is because the qmp package itself won't be > there anymore, we need an installer like `pip` to actually fetch it > for us and put it somewhere. (i.e., we need to process the > dependencies of python/qemu now and can't treat it as a pre-installed > location.) Having pip fetch it on the fly creates a problem for RPM builds, because the koji build env has no network access. We will, however, have an RPM of python-qemu-qmp installed on the host system though. IOW we need to be able to run iotests using system python and its installed libs, not a virtual env. So if we do anything with a virtual env, it will need to be optional I believe. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 8:09 ` Daniel P. Berrangé @ 2022-05-05 12:08 ` John Snow 2022-05-05 12:33 ` Daniel P. Berrangé 0 siblings, 1 reply; 17+ messages in thread From: John Snow @ 2022-05-05 12:08 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Paolo Bonzini, Cleber Rosa [-- Attachment #1: Type: text/plain, Size: 1828 bytes --] On Thu, May 5, 2022, 4:09 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > On Wed, May 04, 2022 at 03:38:45PM -0400, John Snow wrote: > > Howdy! > > > > So, I want to finally delete python/qemu/qmp from qemu.git, and this > > creates a small problem -- namely, iotests needs access to it in order > > to run the python-based tests. > > > > What I think needs to happen is that we create a virtual environment > > that installs python/qemu/. The reason this cannot be done with > > PYTHONPATH alone anymore is because the qmp package itself won't be > > there anymore, we need an installer like `pip` to actually fetch it > > for us and put it somewhere. (i.e., we need to process the > > dependencies of python/qemu now and can't treat it as a pre-installed > > location.) > > Having pip fetch it on the fly creates a problem for RPM builds, > because the koji build env has no network access. We will, however, > have an RPM of python-qemu-qmp installed on the host system though. > IOW we need to be able to run iotests using system python and its > installed libs, not a virtual env. So if we do anything with a > virtual env, it will need to be optional I believe. > Hm, do we need iotests during an rpm build? Is it because of "make check"? It's possible to create a venv and run pip in no-network mode, too. If the package we want is installed on the system or otherwise in pip's cache, it'll succeed without network. If the dependencies require a qemu.qmp that's too new, the pip install will just fail instead. I have to test a way to craft a pip statement that's network *optional* though. i.e. try to fetch and fall back to local otherwise. I think it's worth trying to keep the environment setup code unified, and always use a venv. I think this is tractable, though. [-- Attachment #2: Type: text/html, Size: 2422 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 12:08 ` John Snow @ 2022-05-05 12:33 ` Daniel P. Berrangé 2022-05-05 13:10 ` John Snow 0 siblings, 1 reply; 17+ messages in thread From: Daniel P. Berrangé @ 2022-05-05 12:33 UTC (permalink / raw) To: John Snow Cc: Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Paolo Bonzini, Cleber Rosa On Thu, May 05, 2022 at 08:08:42AM -0400, John Snow wrote: > On Thu, May 5, 2022, 4:09 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > On Wed, May 04, 2022 at 03:38:45PM -0400, John Snow wrote: > > > Howdy! > > > > > > So, I want to finally delete python/qemu/qmp from qemu.git, and this > > > creates a small problem -- namely, iotests needs access to it in order > > > to run the python-based tests. > > > > > > What I think needs to happen is that we create a virtual environment > > > that installs python/qemu/. The reason this cannot be done with > > > PYTHONPATH alone anymore is because the qmp package itself won't be > > > there anymore, we need an installer like `pip` to actually fetch it > > > for us and put it somewhere. (i.e., we need to process the > > > dependencies of python/qemu now and can't treat it as a pre-installed > > > location.) > > > > Having pip fetch it on the fly creates a problem for RPM builds, > > because the koji build env has no network access. We will, however, > > have an RPM of python-qemu-qmp installed on the host system though. > > IOW we need to be able to run iotests using system python and its > > installed libs, not a virtual env. So if we do anything with a > > virtual env, it will need to be optional I believe. > > > > Hm, do we need iotests during an rpm build? Is it because of "make check"? Yes, and this is good, because it prevents us from outputting an RPM build that has a broken QEMU in it. > It's possible to create a venv and run pip in no-network mode, too. If the > package we want is installed on the system or otherwise in pip's cache, > it'll succeed without network. If the dependencies require a qemu.qmp > that's too new, the pip install will just fail instead. > > I have to test a way to craft a pip statement that's network *optional* > though. i.e. try to fetch and fall back to local otherwise. I think it's > worth trying to keep the environment setup code unified, and always use a > venv. As long as it is no-network, that's good enough. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 12:33 ` Daniel P. Berrangé @ 2022-05-05 13:10 ` John Snow 2022-05-05 13:15 ` Paolo Bonzini 0 siblings, 1 reply; 17+ messages in thread From: John Snow @ 2022-05-05 13:10 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Paolo Bonzini, Cleber Rosa [-- Attachment #1: Type: text/plain, Size: 1755 bytes --] On Thu, May 5, 2022, 8:33 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > On Thu, May 05, 2022 at 08:08:42AM -0400, John Snow wrote: > > On Thu, May 5, 2022, 4:09 AM Daniel P. Berrangé <berrange@redhat.com> > wrote: > > > > > On Wed, May 04, 2022 at 03:38:45PM -0400, John Snow wrote: > > > > Howdy! > > > > > > > > So, I want to finally delete python/qemu/qmp from qemu.git, and this > > > > creates a small problem -- namely, iotests needs access to it in > order > > > > to run the python-based tests. > > > > > > > > What I think needs to happen is that we create a virtual environment > > > > that installs python/qemu/. The reason this cannot be done with > > > > PYTHONPATH alone anymore is because the qmp package itself won't be > > > > there anymore, we need an installer like `pip` to actually fetch it > > > > for us and put it somewhere. (i.e., we need to process the > > > > dependencies of python/qemu now and can't treat it as a pre-installed > > > > location.) > > > > > > Having pip fetch it on the fly creates a problem for RPM builds, > > > because the koji build env has no network access. We will, however, > > > have an RPM of python-qemu-qmp installed on the host system though. > > > IOW we need to be able to run iotests using system python and its > > > installed libs, not a virtual env. So if we do anything with a > > > virtual env, it will need to be optional I believe. > > > > > > > Hm, do we need iotests during an rpm build? Is it because of "make > check"? > > Yes, and this is good, because it prevents us from outputting an > RPM build that has a broken QEMU in it. Guess this means I need to make a Fedora package too, though. My hubris. OK, plenty of work to do. [-- Attachment #2: Type: text/html, Size: 2491 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 13:10 ` John Snow @ 2022-05-05 13:15 ` Paolo Bonzini 2022-05-05 14:13 ` John Snow 0 siblings, 1 reply; 17+ messages in thread From: Paolo Bonzini @ 2022-05-05 13:15 UTC (permalink / raw) To: John Snow, Daniel P. Berrangé Cc: Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa On 5/5/22 15:10, John Snow wrote: > > > Hm, do we need iotests during an rpm build? Is it because of > "make check"? > > Yes, and this is good, because it prevents us from outputting an > RPM build that has a broken QEMU in it. > > Guess this means I need to make a Fedora package too, though. My hubris. I would rather keep python/qemu/qmp as a submodule for a longer time, and still go through a virtual environment that installs it together with its pip dependencies. Paolo ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 13:15 ` Paolo Bonzini @ 2022-05-05 14:13 ` John Snow 2022-05-05 15:50 ` Paolo Bonzini 0 siblings, 1 reply; 17+ messages in thread From: John Snow @ 2022-05-05 14:13 UTC (permalink / raw) To: Paolo Bonzini Cc: Daniel P. Berrangé, Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa [-- Attachment #1: Type: text/plain, Size: 972 bytes --] On Thu, May 5, 2022, 9:16 AM Paolo Bonzini <pbonzini@redhat.com> wrote: > On 5/5/22 15:10, John Snow wrote: > > > > > Hm, do we need iotests during an rpm build? Is it because of > > "make check"? > > > > Yes, and this is good, because it prevents us from outputting an > > RPM build that has a broken QEMU in it. > > > > Guess this means I need to make a Fedora package too, though. My hubris. > > I would rather keep python/qemu/qmp as a submodule for a longer time, > and still go through a virtual environment that installs it together > with its pip dependencies. > A small headache relating fixes to both locations, but if you'd like to see it to prove that the installation mechanism works in general, then OK. I'm willing to deal with the pain until the next release to let us go through a testing cycle. Reluctantly. Maybe. I'm assuming you mean as a subpackage and not a [git] submodule. If you do mean git, then ... uh. That might be messy. [-- Attachment #2: Type: text/html, Size: 1640 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 14:13 ` John Snow @ 2022-05-05 15:50 ` Paolo Bonzini 2022-05-05 15:57 ` Daniel P. Berrangé 0 siblings, 1 reply; 17+ messages in thread From: Paolo Bonzini @ 2022-05-05 15:50 UTC (permalink / raw) To: John Snow Cc: Daniel P. Berrangé, Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa On 5/5/22 16:13, John Snow wrote: > > I would rather keep python/qemu/qmp as a submodule for a longer time, > and still go through a virtual environment that installs it together > with its pip dependencies. > > > A small headache relating fixes to both locations, but if you'd like to > see it to prove that the installation mechanism works in general, then > OK. I'm willing to deal with the pain until the next release to let us > go through a testing cycle. Reluctantly. Maybe. > > I'm assuming you mean as a subpackage and not a [git] submodule. If you > do mean git, then ... uh. That might be messy. Yeah, I meant a git submodule in qemu.git... It would also be the easiest way to build a subpackage in Fedora, since it would be part of the QEMU tarballs. Paolo ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 15:50 ` Paolo Bonzini @ 2022-05-05 15:57 ` Daniel P. Berrangé 2022-05-08 14:23 ` Paolo Bonzini 0 siblings, 1 reply; 17+ messages in thread From: Daniel P. Berrangé @ 2022-05-05 15:57 UTC (permalink / raw) To: Paolo Bonzini Cc: John Snow, Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa On Thu, May 05, 2022 at 05:50:00PM +0200, Paolo Bonzini wrote: > On 5/5/22 16:13, John Snow wrote: > > > > I would rather keep python/qemu/qmp as a submodule for a longer time, > > and still go through a virtual environment that installs it together > > with its pip dependencies. > > > > > > A small headache relating fixes to both locations, but if you'd like to > > see it to prove that the installation mechanism works in general, then > > OK. I'm willing to deal with the pain until the next release to let us > > go through a testing cycle. Reluctantly. Maybe. > > > > I'm assuming you mean as a subpackage and not a [git] submodule. If you > > do mean git, then ... uh. That might be messy. > > Yeah, I meant a git submodule in qemu.git... It would also be the easiest > way to build a subpackage in Fedora, since it would be part of the QEMU > tarballs. When qemu.qmp is uploaded to PyPi, then Fedora packaging guidelines on unbundling will expect us to create a dedicated python-qemu.qmp src.rpm, and use that, not anything QEMU might bundle. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 15:57 ` Daniel P. Berrangé @ 2022-05-08 14:23 ` Paolo Bonzini 0 siblings, 0 replies; 17+ messages in thread From: Paolo Bonzini @ 2022-05-08 14:23 UTC (permalink / raw) To: Daniel P. Berrangé Cc: John Snow, Qemu-block, qemu-devel, Kevin Wolf, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa If we want to provide an installable rpm in Fedora then yes; but we can leave distribution to pypi and use a bundled copy in the virtual environment that is used to run tests. Paolo Il 5 maggio 2022 17:57:24 CEST, "Daniel P. Berrangé" <berrange@redhat.com> ha scritto: >On Thu, May 05, 2022 at 05:50:00PM +0200, Paolo Bonzini wrote: >> On 5/5/22 16:13, John Snow wrote: >> > >> > I would rather keep python/qemu/qmp as a submodule for a longer time, >> > and still go through a virtual environment that installs it together >> > with its pip dependencies. >> > >> > >> > A small headache relating fixes to both locations, but if you'd like to >> > see it to prove that the installation mechanism works in general, then >> > OK. I'm willing to deal with the pain until the next release to let us >> > go through a testing cycle. Reluctantly. Maybe. >> > >> > I'm assuming you mean as a subpackage and not a [git] submodule. If you >> > do mean git, then ... uh. That might be messy. >> >> Yeah, I meant a git submodule in qemu.git... It would also be the easiest >> way to build a subpackage in Fedora, since it would be part of the QEMU >> tarballs. > >When qemu.qmp is uploaded to PyPi, then Fedora packaging guidelines on >unbundling will expect us to create a dedicated python-qemu.qmp src.rpm, >and use that, not anything QEMU might bundle. > >With regards, >Daniel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-04 19:38 iotests and python dependencies John Snow 2022-05-05 8:09 ` Daniel P. Berrangé @ 2022-05-05 8:51 ` Kevin Wolf 2022-05-05 9:28 ` Paolo Bonzini 2022-05-05 11:55 ` John Snow 1 sibling, 2 replies; 17+ messages in thread From: Kevin Wolf @ 2022-05-05 8:51 UTC (permalink / raw) To: John Snow Cc: Qemu-block, qemu-devel, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Paolo Bonzini, Cleber Rosa Am 04.05.2022 um 21:38 hat John Snow geschrieben: > Howdy! > > So, I want to finally delete python/qemu/qmp from qemu.git, and this > creates a small problem -- namely, iotests needs access to it in order > to run the python-based tests. > > What I think needs to happen is that we create a virtual environment > that installs python/qemu/. The reason this cannot be done with > PYTHONPATH alone anymore is because the qmp package itself won't be > there anymore, we need an installer like `pip` to actually fetch it > for us and put it somewhere. (i.e., we need to process the > dependencies of python/qemu now and can't treat it as a pre-installed > location.) > > Avocado tests are already creating a venv for the purposes of > installing and running Avocado. We can amend e.g. "../../python" to > tests/requirements.txt and the Avocado environment is A-OK good-to-go. > The Makefile magic for avocado tests creates a venv-per-build. "per build" sounded pretty bad, because I thought it meant building a new venv every time I run either 'make' or the tests. This would obviously be very noticable for short-running tests like some iotests. But fortunately this is not what it does, it keeps the venv around in the build directory. So it's only per build directory really, which I guess is fine. > It seems to work well enough. One thing to note here is that the > supported invocation for avocado tests is only through the Makefile, > which handles creating and entering the venv to make the command > seamless. > > iotests, however, manages its own execution environment with > testenv.py, and we support running iotests from outside of the > Makefile, for example by going to $build/tests/qemu-iotests and > running ./check. Yes, and I agree that this is important. > Now ... I could update testenv.py to be smart enough to create and > enter a python venv, and have even prototyped this. It seems to work > pretty well! This approach seemed like the least invasive to how > iotests are expected to be run and used. But a downside with this > approach is that now avocado tests and iotests are each managing their > own python venv. Worse, vm-tests and device-crash-test are still > unhandled entirely. Is there a reason why testenv.py couldn't enter just the shared venv in build/tests/venv? If not, I guess it would be enough if iotests just checks that the venv exists and all of the dependencies are there in the right version and error out if not, telling the user to run 'make check-venv'. Or actually, it could just unconditionally run 'make check-venv' by itself, which is probably easier to implement than checking the dependencies and more convenient for the user, too. (One more detail: 'make check-venv GIT_SUBMODULES_ACTION=ignore' seems to make it pretty much instantaneous if the venv is current, but leaving the submodule mechanism enabled adds a noticable delay.) > I'd like to find a solution where I create a unified python testing > venv tied to the build shared by avocado, iotests, vm-tests and > device-crash-test. I'm not completely sure how exactly I'll manage > that right now, but I wanted to throw this out there in case there are > some requirements I might be overlooking. > > I think vm-tests and avocado-tests can both have a venv created for > them and activated before the test runs. device-crash-test I believe > will need a script change in the gitlab ci yaml. iotests is somewhat > unique in that it needs to run both by manual invocation and from > makefile invocations. If I want a shared VM between all of these, I'll > need to isolate the create-and-enter-venv logic somewhere where it can > be shared both inside and outside of a Makefile. If just calling 'make' isn't an option, then moving that part out into a separate script probably wouldn't be too hard either. But 'make' has the advantage that it already contains the check if the venv is already there and requirements.txt hasn't changed, which you would have to replicate otherwise. Kevin ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 8:51 ` Kevin Wolf @ 2022-05-05 9:28 ` Paolo Bonzini 2022-05-05 10:59 ` Kevin Wolf 2022-05-10 17:40 ` John Snow 2022-05-05 11:55 ` John Snow 1 sibling, 2 replies; 17+ messages in thread From: Paolo Bonzini @ 2022-05-05 9:28 UTC (permalink / raw) To: Kevin Wolf, John Snow Cc: Qemu-block, qemu-devel, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa On 5/5/22 10:51, Kevin Wolf wrote: > If not, I guess it would be enough if iotests just checks that the venv > exists and all of the dependencies are there in the right version and > error out if not, telling the user to run 'make check-venv'. > > Or actually, it could just unconditionally run 'make check-venv' by > itself, which is probably easier to implement than checking the > dependencies and more convenient for the user, too. Note that you would still have to add a 'check-block: check-venv' dependency in the Makefile, otherwise two "instances" of check-venv could run in parallel. One small complication is that on BSD systems the binary is actually called "gmake", so you'd have to pass the variable somehow Paolo ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 9:28 ` Paolo Bonzini @ 2022-05-05 10:59 ` Kevin Wolf 2022-05-05 12:24 ` Paolo Bonzini 2022-05-10 17:40 ` John Snow 1 sibling, 1 reply; 17+ messages in thread From: Kevin Wolf @ 2022-05-05 10:59 UTC (permalink / raw) To: Paolo Bonzini Cc: John Snow, Qemu-block, qemu-devel, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa Am 05.05.2022 um 11:28 hat Paolo Bonzini geschrieben: > On 5/5/22 10:51, Kevin Wolf wrote: > > If not, I guess it would be enough if iotests just checks that the venv > > exists and all of the dependencies are there in the right version and > > error out if not, telling the user to run 'make check-venv'. > > > > Or actually, it could just unconditionally run 'make check-venv' by > > itself, which is probably easier to implement than checking the > > dependencies and more convenient for the user, too. > > Note that you would still have to add a 'check-block: check-venv' > dependency in the Makefile, otherwise two "instances" of check-venv > could run in parallel. Good point. I only considered manual invocations, but for 'make check-block' that sounds right. > One small complication is that on BSD systems the binary is actually > called "gmake", so you'd have to pass the variable somehow I guess we could just export $MAKE as an environment variable? Kevin ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 10:59 ` Kevin Wolf @ 2022-05-05 12:24 ` Paolo Bonzini 2022-05-05 12:39 ` Kevin Wolf 0 siblings, 1 reply; 17+ messages in thread From: Paolo Bonzini @ 2022-05-05 12:24 UTC (permalink / raw) To: Kevin Wolf Cc: John Snow, Qemu-block, qemu-devel, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa On 5/5/22 12:59, Kevin Wolf wrote: > Am 05.05.2022 um 11:28 hat Paolo Bonzini geschrieben: >>> Or actually, it could just unconditionally run 'make check-venv' by >>> itself, which is probably easier to implement than checking the >>> dependencies and more convenient for the user, too. >> >> One small complication is that on BSD systems the binary is actually >> called "gmake", so you'd have to pass the variable somehow > > I guess we could just export $MAKE as an environment variable? That would work when invoked by "make", but then that's the case in which the venv would be there anyway. For the other case, it would have to parse config-host.mak and/or reintroduce something like tests/qemu-iotests/common.env. All in all it seems like an unnecessary complication over just printing a clear and polite error message. Paolo ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 12:24 ` Paolo Bonzini @ 2022-05-05 12:39 ` Kevin Wolf 0 siblings, 0 replies; 17+ messages in thread From: Kevin Wolf @ 2022-05-05 12:39 UTC (permalink / raw) To: Paolo Bonzini Cc: John Snow, Qemu-block, qemu-devel, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa Am 05.05.2022 um 14:24 hat Paolo Bonzini geschrieben: > On 5/5/22 12:59, Kevin Wolf wrote: > > Am 05.05.2022 um 11:28 hat Paolo Bonzini geschrieben: > > > > Or actually, it could just unconditionally run 'make check-venv' by > > > > itself, which is probably easier to implement than checking the > > > > dependencies and more convenient for the user, too. > > > > > > One small complication is that on BSD systems the binary is actually > > > called "gmake", so you'd have to pass the variable somehow > > > > I guess we could just export $MAKE as an environment variable? > > That would work when invoked by "make", but then that's the case in which > the venv would be there anyway. > > For the other case, it would have to parse config-host.mak and/or > reintroduce something like tests/qemu-iotests/common.env. All in all it > seems like an unnecessary complication over just printing a clear and polite > error message. Or try 'make' and print the error message only if that fails. Kevin ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 9:28 ` Paolo Bonzini 2022-05-05 10:59 ` Kevin Wolf @ 2022-05-10 17:40 ` John Snow 1 sibling, 0 replies; 17+ messages in thread From: John Snow @ 2022-05-10 17:40 UTC (permalink / raw) To: Paolo Bonzini Cc: Kevin Wolf, Qemu-block, qemu-devel, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Cleber Rosa On Thu, May 5, 2022 at 5:28 AM Paolo Bonzini <pbonzini@redhat.com> wrote: > > On 5/5/22 10:51, Kevin Wolf wrote: > > If not, I guess it would be enough if iotests just checks that the venv > > exists and all of the dependencies are there in the right version and > > error out if not, telling the user to run 'make check-venv'. > > > > Or actually, it could just unconditionally run 'make check-venv' by > > itself, which is probably easier to implement than checking the > > dependencies and more convenient for the user, too. > > Note that you would still have to add a 'check-block: check-venv' > dependency in the Makefile, otherwise two "instances" of check-venv > could run in parallel. > > One small complication is that on BSD systems the binary is actually > called "gmake", so you'd have to pass the variable somehow > > Paolo > Dumb question: where would I express this dependency? I don't know where the top-level "check-block" recipe gets defined. --js ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: iotests and python dependencies 2022-05-05 8:51 ` Kevin Wolf 2022-05-05 9:28 ` Paolo Bonzini @ 2022-05-05 11:55 ` John Snow 1 sibling, 0 replies; 17+ messages in thread From: John Snow @ 2022-05-05 11:55 UTC (permalink / raw) To: Kevin Wolf Cc: Qemu-block, qemu-devel, Hanna Reitz, Vladimir Sementsov-Ogievskiy, Paolo Bonzini, Cleber Rosa [-- Attachment #1: Type: text/plain, Size: 5289 bytes --] On Thu, May 5, 2022, 4:51 AM Kevin Wolf <kwolf@redhat.com> wrote: > Am 04.05.2022 um 21:38 hat John Snow geschrieben: > > Howdy! > > > > So, I want to finally delete python/qemu/qmp from qemu.git, and this > > creates a small problem -- namely, iotests needs access to it in order > > to run the python-based tests. > > > > What I think needs to happen is that we create a virtual environment > > that installs python/qemu/. The reason this cannot be done with > > PYTHONPATH alone anymore is because the qmp package itself won't be > > there anymore, we need an installer like `pip` to actually fetch it > > for us and put it somewhere. (i.e., we need to process the > > dependencies of python/qemu now and can't treat it as a pre-installed > > location.) > > > > Avocado tests are already creating a venv for the purposes of > > installing and running Avocado. We can amend e.g. "../../python" to > > tests/requirements.txt and the Avocado environment is A-OK good-to-go. > > The Makefile magic for avocado tests creates a venv-per-build. > > "per build" sounded pretty bad, because I thought it meant building a > new venv every time I run either 'make' or the tests. This would > obviously be very noticable for short-running tests like some iotests. > But fortunately this is not what it does, it keeps the venv around in > the build directory. So it's only per build directory really, which I > guess is fine. > Whoops, yeah. I meant per build directory. In contrast to the Python unit tests themselves, which create some test venvs tied directly to the source directory and are build-agnostic. > > It seems to work well enough. One thing to note here is that the > > supported invocation for avocado tests is only through the Makefile, > > which handles creating and entering the venv to make the command > > seamless. > > > > iotests, however, manages its own execution environment with > > testenv.py, and we support running iotests from outside of the > > Makefile, for example by going to $build/tests/qemu-iotests and > > running ./check. > > Yes, and I agree that this is important. > Figured as much. Not plotting to take this away, I promise. Just getting my requirements straight before I spend time concocting something. > > Now ... I could update testenv.py to be smart enough to create and > > enter a python venv, and have even prototyped this. It seems to work > > pretty well! This approach seemed like the least invasive to how > > iotests are expected to be run and used. But a downside with this > > approach is that now avocado tests and iotests are each managing their > > own python venv. Worse, vm-tests and device-crash-test are still > > unhandled entirely. > > Is there a reason why testenv.py couldn't enter just the shared venv in > build/tests/venv? > It can, but it'd have to be made first so it can enter it. I figured this would only happen "on-demand", like how check-avocado works, so I'd need some way for iotests and the Makefile to share the venv creation code, which is certainly an option. > If not, I guess it would be enough if iotests just checks that the venv > exists and all of the dependencies are there in the right version and > error out if not, telling the user to run 'make check-venv'. > I will spend unreasonable engineering hours avoiding this, if only for pride. I want everything to be as seamless and easy as it's always been. > Or actually, it could just unconditionally run 'make check-venv' by > itself, which is probably easier to implement than checking the > dependencies and more convenient for the user, too. > Oh, that's one way to get them to share the venv-creation pathway. Hadn't occurred to me, but this might be easy to do. > (One more detail: 'make check-venv GIT_SUBMODULES_ACTION=ignore' seems > to make it pretty much instantaneous if the venv is current, but leaving > the submodule mechanism enabled adds a noticable delay.) > Noted. > > I'd like to find a solution where I create a unified python testing > > venv tied to the build shared by avocado, iotests, vm-tests and > > device-crash-test. I'm not completely sure how exactly I'll manage > > that right now, but I wanted to throw this out there in case there are > > some requirements I might be overlooking. > > > > I think vm-tests and avocado-tests can both have a venv created for > > them and activated before the test runs. device-crash-test I believe > > will need a script change in the gitlab ci yaml. iotests is somewhat > > unique in that it needs to run both by manual invocation and from > > makefile invocations. If I want a shared VM between all of these, I'll > > need to isolate the create-and-enter-venv logic somewhere where it can > > be shared both inside and outside of a Makefile. > > If just calling 'make' isn't an option, then moving that part out into a > separate script probably wouldn't be too hard either. But 'make' has the > advantage that it already contains the check if the venv is already > there and requirements.txt hasn't changed, which you would have to > replicate otherwise. > Yeah, exactly. > Kevin > Thanks for the ideas! As of now, I've got a branch where I've moved vm-tests over onto the same venv with avocado-tests. If I can bridge the gap to iotests, I'll be pretty happy. [-- Attachment #2: Type: text/html, Size: 8102 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2022-05-10 17:44 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-05-04 19:38 iotests and python dependencies John Snow 2022-05-05 8:09 ` Daniel P. Berrangé 2022-05-05 12:08 ` John Snow 2022-05-05 12:33 ` Daniel P. Berrangé 2022-05-05 13:10 ` John Snow 2022-05-05 13:15 ` Paolo Bonzini 2022-05-05 14:13 ` John Snow 2022-05-05 15:50 ` Paolo Bonzini 2022-05-05 15:57 ` Daniel P. Berrangé 2022-05-08 14:23 ` Paolo Bonzini 2022-05-05 8:51 ` Kevin Wolf 2022-05-05 9:28 ` Paolo Bonzini 2022-05-05 10:59 ` Kevin Wolf 2022-05-05 12:24 ` Paolo Bonzini 2022-05-05 12:39 ` Kevin Wolf 2022-05-10 17:40 ` John Snow 2022-05-05 11:55 ` John Snow
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.