Re: [PATCH v3 1/4] tests/acceptance: avocado_qemu: Introduce the 'accel' test parameter

[Wainer dos Santos Moschetta <wainersm@redhat.com>]

On 1/24/20 7:36 AM, Philippe Mathieu-Daudé wrote:
> On 1/22/20 2:27 AM, Wainer dos Santos Moschetta wrote:
>> The test case may need to boot the VM with an accelerator that
>> isn't actually enabled on the QEMU binary and/or present in the host. In
>> this case the test behavior is undefined, and the best course of
>> action is to skip its execution.
>>
>> This change introduced the 'accel' parameter (and the handler of
>> tag with same name) used to indicate the test case requires a
>> given accelerator available. It was implemented a mechanism to
>> skip the test case if the accelerator is not available. Moreover,
>>   the QEMU -accel argument is set automatically to any VM
>> launched if the parameter is present.
>>
>> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>> ---
>>   docs/devel/testing.rst                    | 16 ++++++++++++++++
>>   tests/acceptance/avocado_qemu/__init__.py | 23 +++++++++++++++++++++++
>>   2 files changed, 39 insertions(+)
>>
>> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
>> index ab5be0c729..d17d0e90aa 100644
>> --- a/docs/devel/testing.rst
>> +++ b/docs/devel/testing.rst
>> @@ -759,6 +759,17 @@ name.  If one is not given explicitly, it will 
>> either be set to
>>   ``None``, or, if the test is tagged with one (and only one)
>>   ``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
>>   +accel
>> +~~~~~
>> +The accelerator that will be set to all QEMUMachine instances created
>> +by the test.
>> +
>> +The ``accel`` attribute will be set to the test parameter of the same
>> +name.  If one is not given explicitly, it will either be set to
>> +``None``, or, if the test is tagged with one (and only one)
>> +``:avocado: tags=accel:VALUE`` tag, it will be set to ``VALUE``. 
>> Currently
>> +``VALUE`` should be either ``kvm`` or ``tcg``.
>> +
>>   qemu_bin
>>   ~~~~~~~~
>>   @@ -800,6 +811,11 @@ machine
>>   The machine type that will be set to all QEMUMachine instances created
>>   by the test.
>>   +accel
>> +~~~~~
>> +The accelerator that will be set to all QEMUMachine instances created
>> +by the test. In case the accelerator is not available (both QEMU
>> +binary and the host system are checked) then the test is canceled.
>>     qemu_bin
>>   ~~~~~~~~
>> diff --git a/tests/acceptance/avocado_qemu/__init__.py 
>> b/tests/acceptance/avocado_qemu/__init__.py
>> index 6618ea67c1..c83a75ccbc 100644
>> --- a/tests/acceptance/avocado_qemu/__init__.py
>> +++ b/tests/acceptance/avocado_qemu/__init__.py
>> @@ -20,6 +20,7 @@ SRC_ROOT_DIR = 
>> os.path.join(os.path.dirname(__file__), '..', '..', '..')
>>   sys.path.append(os.path.join(SRC_ROOT_DIR, 'python'))
>>     from qemu.machine import QEMUMachine
>> +from qemu.accel import kvm_available, tcg_available
>>     def is_readable_executable_file(path):
>>       return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
>> @@ -111,6 +112,8 @@ class Test(avocado.Test):
>>         def setUp(self):
>>           self._vms = {}
>> +        # VM argumments that are mapped from parameters
>> +        self._param_to_vm_args = []
>>             self.arch = self.params.get('arch',
>> default=self._get_unique_tag_val('arch'))
>> @@ -124,10 +127,30 @@ class Test(avocado.Test):
>>           if self.qemu_bin is None:
>>               self.cancel("No QEMU binary defined or found in the 
>> source tree")
>>   +        self.accel = self.params.get('accel',
>> + default=self._get_unique_tag_val('accel'))
>> +        if self.accel:
>> +            avail = False
>> +            if self.accel == 'kvm':
>> +                if kvm_available(self.arch, self.qemu_bin):
>> +                    avail = True
>> +            elif self.accel == 'tcg':
>> +                if tcg_available(self.qemu_bin):
>> +                    avail = True
>> +            else:
>> +                self.cancel("Unknown accelerator: %s" % self.accel)
>> +
>> +            if avail:
>> +                self._param_to_vm_args.extend(['-accel', self.accel])
>> +            else:
>> +                self.cancel("%s is not available" % self.accel)
>
> Why refuse to test the other accelerators?
>
> Isn't it better to QMP-ask QEMU which accelerator it supports, and 
> SKIP if it isn't available?


python/qemu/accel.py:list_accel(qemu_bin) can be used for that end. For 
example:

if self.accel not in list_accel(self.qemu_bin):

     self.cancel("%s is not available" % self.accel)

However checking the support only on QEMU's binary may be very weak 
(take KVM as an example). I implemented checkers for kvm and tcg on 
python/qemu/accel.py. Given that I have zero knowledge on the other 
accelerators,  I simply did not touch on them.

That said, IMHO it needs to implement checkers for those others 
accelerator before they get reliably handled by avocado_qemu 
automatically. And test writers can still run tests with those 
accelerators as long as 'accel' tag is not used.

What do you think Philippe?

Thanks, good point!

- Wainer


>
>
>> +
>>       def _new_vm(self, *args):
>>           vm = QEMUMachine(self.qemu_bin, sock_dir=tempfile.mkdtemp())
>>           if args:
>>               vm.add_args(*args)
>> +        if self._param_to_vm_args:
>> +            vm.add_args(*self._param_to_vm_args)
>>           return vm
>>         @property
>>
>