* Can we run QEMU Avocado tests without root access?
@ 2021-02-21 23:23 Philippe Mathieu-Daudé
2021-02-22 16:28 ` Cleber Rosa
0 siblings, 1 reply; 3+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-02-21 23:23 UTC (permalink / raw)
To: Cleber Rosa, Wainer dos Santos Moschetta, Willian Rampazzo
Cc: avocado-devel, qemu-devel
Hi,
On a system I want to run the QEMU integration tests I got:
$ make check-venv
GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc capstone slirp
VENV build/tests/venv
The virtual environment was not created successfully because ensurepip
is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
apt-get install python3-venv
You may need to use sudo with that command. After installing the
python3-venv
package, recreate your virtual environment.
Failing command: ['build/tests/venv/bin/python3', '-Im', 'ensurepip',
'--upgrade', '--default-pip']
make: *** [/home/philmd/qemu/tests/Makefile.include:98:
build/tests/venv] Error 1
However I could do:
$ python3 -m pip install --user virtualenv
Collecting virtualenv
Downloading virtualenv-20.4.2-py2.py3-none-any.whl (7.2 MB)
|████████████████████████████████| 7.2 MB 6.4 MB/s
Collecting distlib<1,>=0.3.1
Downloading distlib-0.3.1-py2.py3-none-any.whl (335 kB)
|████████████████████████████████| 335 kB 6.4 MB/s
Collecting appdirs<2,>=1.4.3
Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Requirement already satisfied: six<2,>=1.9.0 in
/usr/lib/python3/dist-packages (from virtualenv) (1.15.0)
Collecting filelock<4,>=3.0.0
Downloading filelock-3.0.12-py3-none-any.whl (7.6 kB)
Installing collected packages: distlib, appdirs, filelock, virtualenv
Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12
virtualenv-20.4.2
$ virtualenv --version
virtualenv 20.4.2 from
/home/philmd/.local/lib/python3.9/site-packages/virtualenv/__init__.py
Note, there is still the old tests/venv/ dir created bug:
$ make check-venv
GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc capstone slirp
make: Nothing to be done for 'check-venv'.
Although it might be true... If I don't have root access, there is
nothing to be done ¯\_(ツ)_/¯
Thanks,
Phil.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Can we run QEMU Avocado tests without root access?
2021-02-21 23:23 Can we run QEMU Avocado tests without root access? Philippe Mathieu-Daudé
@ 2021-02-22 16:28 ` Cleber Rosa
2021-02-22 16:52 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 3+ messages in thread
From: Cleber Rosa @ 2021-02-22 16:28 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: avocado-devel, Willian Rampazzo, Wainer dos Santos Moschetta, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 4537 bytes --]
On Mon, Feb 22, 2021 at 12:23:17AM +0100, Philippe Mathieu-Daudé wrote:
> Hi,
>
> On a system I want to run the QEMU integration tests I got:
>
Would you care to give more information about this system? Is it
possible that it's a Debian-like system with "python3-minimal"
installed instead of "python3"?
> $ make check-venv
> GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3
> tests/fp/berkeley-softfloat-3 dtc capstone slirp
> VENV build/tests/venv
> The virtual environment was not created successfully because ensurepip
> is not
> available. On Debian/Ubuntu systems, you need to install the python3-venv
> package using the following command.
>
> apt-get install python3-venv
>
> You may need to use sudo with that command. After installing the
> python3-venv
> package, recreate your virtual environment.
>
> Failing command: ['build/tests/venv/bin/python3', '-Im', 'ensurepip',
> '--upgrade', '--default-pip']
>
make check-{venv,acceptance} won't attempt to install pip, so I'm
guessing this message is coming from the Python distribution on
your system. And notice that the bare venv *has* been created.
> make: *** [/home/philmd/qemu/tests/Makefile.include:98:
> build/tests/venv] Error 1
>
> However I could do:
>
> $ python3 -m pip install --user virtualenv
> Collecting virtualenv
> Downloading virtualenv-20.4.2-py2.py3-none-any.whl (7.2 MB)
> |████████████████████████████████| 7.2 MB 6.4 MB/s
> Collecting distlib<1,>=0.3.1
> Downloading distlib-0.3.1-py2.py3-none-any.whl (335 kB)
> |████████████████████████████████| 335 kB 6.4 MB/s
> Collecting appdirs<2,>=1.4.3
> Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
> Requirement already satisfied: six<2,>=1.9.0 in
> /usr/lib/python3/dist-packages (from virtualenv) (1.15.0)
> Collecting filelock<4,>=3.0.0
> Downloading filelock-3.0.12-py3-none-any.whl (7.6 kB)
> Installing collected packages: distlib, appdirs, filelock, virtualenv
> Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12
> virtualenv-20.4.2
>
> $ virtualenv --version
> virtualenv 20.4.2 from
> /home/philmd/.local/lib/python3.9/site-packages/virtualenv/__init__.py
>
> Note, there is still the old tests/venv/ dir created bug:
>
OK, noted. We not have a formal bug report to work on:
https://bugs.launchpad.net/qemu/+bug/1916506
> $ make check-venv
> GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3
> tests/fp/berkeley-softfloat-3 dtc capstone slirp
> make: Nothing to be done for 'check-venv'.
>
> Although it might be true... If I don't have root access, there is
> nothing to be done ¯\_(ツ)_/¯
>
With regards to your question on the subject line, one could attempt
to have a secondary Python installation with the *required packages*
(as per the documentation) installed as a non-root user. And then,
you could give that Python path to configure. The current
documentation states:
Note: the build environment must be using a Python 3 stack, and have
the ``venv`` and ``pip`` packages installed. If necessary, make sure
``configure`` is called with ``--python=`` and that those modules are
available. On Debian and Ubuntu based systems, depending on the
specific version, they may be on packages named ``python3-venv`` and
``python3-pip``.
IIUC, the improvement you suggest is to not require "python-venv" as a
package, but install it via "python3 -m pip". One possible way to
rely *only* on Python 3 and setuptools (dropping venv and pip
requirements) would be to do something like:
$ python3 -m ensurepip
$ python3 -m pip install venv
$ make check-venv
It's a valid approach, in theory. In practice, depending on your
distro, you may not have "ensurepip" at all, even if it is a *standard
Python library*:
https://docs.python.org/3/library/ensurepip.html
You can read about how those packaging decisions can become hell
in bug reports such as:
https://bugs.launchpad.net/ubuntu/+source/python3.4/+bug/1290847
In my experience, venv and pip have caveats and using the distro's
(complete) packages will give you an overall better experience.
While *one* could setup the dependencies as non-root, but I don't
think it should be used in the current check-venv rule.
Regards,
- Cleber.
> Thanks,
>
> Phil.
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Can we run QEMU Avocado tests without root access?
2021-02-22 16:28 ` Cleber Rosa
@ 2021-02-22 16:52 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 3+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-02-22 16:52 UTC (permalink / raw)
To: Cleber Rosa
Cc: avocado-devel, Willian Rampazzo, Wainer dos Santos Moschetta, qemu-devel
On 2/22/21 5:28 PM, Cleber Rosa wrote:
> On Mon, Feb 22, 2021 at 12:23:17AM +0100, Philippe Mathieu-Daudé wrote:
>> Hi,
>>
>> On a system I want to run the QEMU integration tests I got:
>>
>
> Would you care to give more information about this system? Is it
> possible that it's a Debian-like system with "python3-minimal"
> installed instead of "python3"?
Debian GNU/Linux bullseye/sid
$ dpkg -S $(which python3)
python3-minimal: /usr/bin/python3
>> $ make check-venv
>> GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3
>> tests/fp/berkeley-softfloat-3 dtc capstone slirp
>> VENV build/tests/venv
>> The virtual environment was not created successfully because ensurepip
>> is not
>> available. On Debian/Ubuntu systems, you need to install the python3-venv
>> package using the following command.
>>
>> apt-get install python3-venv
>>
>> You may need to use sudo with that command. After installing the
>> python3-venv
>> package, recreate your virtual environment.
>>
>> Failing command: ['build/tests/venv/bin/python3', '-Im', 'ensurepip',
>> '--upgrade', '--default-pip']
>>
>
> make check-{venv,acceptance} won't attempt to install pip, so I'm
> guessing this message is coming from the Python distribution on
> your system. And notice that the bare venv *has* been created.
>
>> make: *** [/home/philmd/qemu/tests/Makefile.include:98:
>> build/tests/venv] Error 1
>>
>> However I could do:
>>
>> $ python3 -m pip install --user virtualenv
>> Collecting virtualenv
>> Downloading virtualenv-20.4.2-py2.py3-none-any.whl (7.2 MB)
>> |████████████████████████████████| 7.2 MB 6.4 MB/s
>> Collecting distlib<1,>=0.3.1
>> Downloading distlib-0.3.1-py2.py3-none-any.whl (335 kB)
>> |████████████████████████████████| 335 kB 6.4 MB/s
>> Collecting appdirs<2,>=1.4.3
>> Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
>> Requirement already satisfied: six<2,>=1.9.0 in
>> /usr/lib/python3/dist-packages (from virtualenv) (1.15.0)
>> Collecting filelock<4,>=3.0.0
>> Downloading filelock-3.0.12-py3-none-any.whl (7.6 kB)
>> Installing collected packages: distlib, appdirs, filelock, virtualenv
>> Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12
>> virtualenv-20.4.2
>>
>> $ virtualenv --version
>> virtualenv 20.4.2 from
>> /home/philmd/.local/lib/python3.9/site-packages/virtualenv/__init__.py
>>
>> Note, there is still the old tests/venv/ dir created bug:
>>
>
> OK, noted. We not have a formal bug report to work on:
>
> https://bugs.launchpad.net/qemu/+bug/1916506
Thanks!
>> $ make check-venv
>> GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3
>> tests/fp/berkeley-softfloat-3 dtc capstone slirp
>> make: Nothing to be done for 'check-venv'.
>>
>> Although it might be true... If I don't have root access, there is
>> nothing to be done ¯\_(ツ)_/¯
>>
>
> With regards to your question on the subject line, one could attempt
> to have a secondary Python installation with the *required packages*
> (as per the documentation) installed as a non-root user. And then,
> you could give that Python path to configure. The current
> documentation states:
>
> Note: the build environment must be using a Python 3 stack, and have
> the ``venv`` and ``pip`` packages installed. If necessary, make sure
> ``configure`` is called with ``--python=`` and that those modules are
> available. On Debian and Ubuntu based systems, depending on the
> specific version, they may be on packages named ``python3-venv`` and
> ``python3-pip``.
>
> IIUC, the improvement you suggest is to not require "python-venv" as a
> package, but install it via "python3 -m pip". One possible way to
> rely *only* on Python 3 and setuptools (dropping venv and pip
> requirements) would be to do something like:
>
> $ python3 -m ensurepip
$ python3 -m ensurepip
/usr/bin/python3: No module named ensurepip
> $ python3 -m pip install venv
$ python3 -m pip install venv
ERROR: Could not find a version that satisfies the requirement venv
(from versions: none)
ERROR: No matching distribution found for venv
> $ make check-venv
>
> It's a valid approach, in theory. In practice, depending on your
> distro, you may not have "ensurepip" at all, even if it is a *standard
> Python library*:
>
> https://docs.python.org/3/library/ensurepip.html
>
> You can read about how those packaging decisions can become hell
> in bug reports such as:
>
> https://bugs.launchpad.net/ubuntu/+source/python3.4/+bug/1290847
>
> In my experience, venv and pip have caveats and using the distro's
> (complete) packages will give you an overall better experience.
>
> While *one* could setup the dependencies as non-root, but I don't
> think it should be used in the current check-venv rule.
OK. Well if it isn't fixable, then we can simply add an error message
such: "Missing .... Ask the super administrator to install ...", right?
Thanks for looking at this,
Phil.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-02-22 16:55 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-21 23:23 Can we run QEMU Avocado tests without root access? Philippe Mathieu-Daudé
2021-02-22 16:28 ` Cleber Rosa
2021-02-22 16:52 ` Philippe Mathieu-Daudé
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).