[PATCH v2 00/11] Record/replay acceptance tests

[PATCH v2 00/11] Record/replay acceptance tests

[Pavel Dovgalyuk]

The following series adds record/replay tests to the acceptance group.
Test pass successfully with the latest submitted record/replay fixes:
 - replay: notify the main loop when there are no instructions
 - replay: synchronize on every virtual timer callback

The provided tests perform kernel boot and disk image boot scenarios.
For all of them recording and replaying phases are executed.
Tests were borrowed from existing boot_linux*.py tests. But some
of the platforms and images were excluded, because icount for them
still has some issues.

v2 changes:
 - Some test structure refactoring (suggested by Willian Rampazzo)

---

Pavel Dovgaluk (11):
      tests/acceptance: allow console interaction with specific VMs
      tests/acceptance: refactor boot_linux_console test to allow code reuse
      tests/acceptance: add base class record/replay kernel tests
      tests/acceptance: add kernel record/replay test for x86_64
      tests/acceptance: add record/replay test for aarch64
      tests/acceptance: add record/replay test for arm
      tests/acceptance: add record/replay test for ppc64
      tests/acceptance: add record/replay test for m68k
      tests/acceptance: record/replay tests with advcal images
      tests/acceptance: refactor boot_linux to allow code reuse
      tests/acceptance: Linux boot test for record/replay


 0 files changed

--
Pavel Dovgalyuk

[PATCH v2 01/11] tests/acceptance: allow console interaction with specific VMs

[Pavel Dovgalyuk]

Console interaction in avocado scripts was possible only with single
default VM.
This patch modifies the function parameters to allow passing a specific
VM as a parameter to interact with it.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
---
 0 files changed

diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index 59e7b4f763..77d1c1d9ff 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -69,13 +69,15 @@ def pick_default_qemu_bin(arch=None):
 
 
 def _console_interaction(test, success_message, failure_message,
-                         send_string, keep_sending=False):
+                         send_string, keep_sending=False, vm=None):
     assert not keep_sending or send_string
-    console = test.vm.console_socket.makefile()
+    if vm is None:
+        vm = test.vm
+    console = vm.console_socket.makefile()
     console_logger = logging.getLogger('console')
     while True:
         if send_string:
-            test.vm.console_socket.sendall(send_string.encode())
+            vm.console_socket.sendall(send_string.encode())
             if not keep_sending:
                 send_string = None # send only once
         msg = console.readline().strip()
@@ -115,7 +117,8 @@ def interrupt_interactive_console_until_pattern(test, success_message,
     _console_interaction(test, success_message, failure_message,
                          interrupt_string, True)
 
-def wait_for_console_pattern(test, success_message, failure_message=None):
+def wait_for_console_pattern(test, success_message, failure_message=None,
+                             vm=None):
     """
     Waits for messages to appear on the console, while logging the content
 
@@ -125,7 +128,7 @@ def wait_for_console_pattern(test, success_message, failure_message=None):
     :param success_message: if this message appears, test succeeds
     :param failure_message: if this message appears, test fails
     """
-    _console_interaction(test, success_message, failure_message, None)
+    _console_interaction(test, success_message, failure_message, None, vm=vm)
 
 def exec_command_and_wait_for_pattern(test, command,
                                       success_message, failure_message=None):

Re: [PATCH v2 01/11] tests/acceptance: allow console interaction with specific VMs

[Alex Bennée]

Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:

> Console interaction in avocado scripts was possible only with single
> default VM.
> This patch modifies the function parameters to allow passing a specific
> VM as a parameter to interact with it.
>
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> Reviewed-by: Willian Rampazzo <willianr@redhat.com>
> ---
>  0 files changed
>
> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> index 59e7b4f763..77d1c1d9ff 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -69,13 +69,15 @@ def pick_default_qemu_bin(arch=None):
>  
>  
>  def _console_interaction(test, success_message, failure_message,
> -                         send_string, keep_sending=False):
> +                         send_string, keep_sending=False, vm=None):

is it not possible to make vm=test.vm to avoid having...

>      assert not keep_sending or send_string
> -    console = test.vm.console_socket.makefile()
> +    if vm is None:
> +        vm = test.vm

to do this here?

> +    console = vm.console_socket.makefile()
>      console_logger = logging.getLogger('console')
>      while True:
>          if send_string:
> -            test.vm.console_socket.sendall(send_string.encode())
> +            vm.console_socket.sendall(send_string.encode())
>              if not keep_sending:
>                  send_string = None # send only once
>          msg = console.readline().strip()
> @@ -115,7 +117,8 @@ def interrupt_interactive_console_until_pattern(test, success_message,
>      _console_interaction(test, success_message, failure_message,
>                           interrupt_string, True)
>  
> -def wait_for_console_pattern(test, success_message, failure_message=None):
> +def wait_for_console_pattern(test, success_message, failure_message=None,
> +                             vm=None):
>      """
>      Waits for messages to appear on the console, while logging the content
>  
> @@ -125,7 +128,7 @@ def wait_for_console_pattern(test, success_message, failure_message=None):
>      :param success_message: if this message appears, test succeeds
>      :param failure_message: if this message appears, test fails
>      """
> -    _console_interaction(test, success_message, failure_message, None)
> +    _console_interaction(test, success_message, failure_message, None, vm=vm)
>  
>  def exec_command_and_wait_for_pattern(test, command,
>                                        success_message, failure_message=None):

Otherwise:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [PATCH v2 01/11] tests/acceptance: allow console interaction with specific VMs

[Willian Rampazzo]

On Wed, May 27, 2020 at 11:20 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>
> > Console interaction in avocado scripts was possible only with single
> > default VM.
> > This patch modifies the function parameters to allow passing a specific
> > VM as a parameter to interact with it.
> >
> > Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> > Reviewed-by: Willian Rampazzo <willianr@redhat.com>
> > ---
> >  0 files changed
> >
> > diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> > index 59e7b4f763..77d1c1d9ff 100644
> > --- a/tests/acceptance/avocado_qemu/__init__.py
> > +++ b/tests/acceptance/avocado_qemu/__init__.py
> > @@ -69,13 +69,15 @@ def pick_default_qemu_bin(arch=None):
> >
> >
> >  def _console_interaction(test, success_message, failure_message,
> > -                         send_string, keep_sending=False):
> > +                         send_string, keep_sending=False, vm=None):
>
> is it not possible to make vm=test.vm to avoid having...

This will cause a NameError as `test` is also a parameter here.

>
> >      assert not keep_sending or send_string
> > -    console = test.vm.console_socket.makefile()
> > +    if vm is None:
> > +        vm = test.vm
>
> to do this here?
>
> > +    console = vm.console_socket.makefile()
> >      console_logger = logging.getLogger('console')
> >      while True:
> >          if send_string:
> > -            test.vm.console_socket.sendall(send_string.encode())
> > +            vm.console_socket.sendall(send_string.encode())
> >              if not keep_sending:
> >                  send_string = None # send only once
> >          msg = console.readline().strip()
> > @@ -115,7 +117,8 @@ def interrupt_interactive_console_until_pattern(test, success_message,
> >      _console_interaction(test, success_message, failure_message,
> >                           interrupt_string, True)
> >
> > -def wait_for_console_pattern(test, success_message, failure_message=None):
> > +def wait_for_console_pattern(test, success_message, failure_message=None,
> > +                             vm=None):
> >      """
> >      Waits for messages to appear on the console, while logging the content
> >
> > @@ -125,7 +128,7 @@ def wait_for_console_pattern(test, success_message, failure_message=None):
> >      :param success_message: if this message appears, test succeeds
> >      :param failure_message: if this message appears, test fails
> >      """
> > -    _console_interaction(test, success_message, failure_message, None)
> > +    _console_interaction(test, success_message, failure_message, None, vm=vm)
> >
> >  def exec_command_and_wait_for_pattern(test, command,
> >                                        success_message, failure_message=None):
>
> Otherwise:
>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> --
> Alex Bennée
>

[PATCH v2 02/11] tests/acceptance: refactor boot_linux_console test to allow code reuse

[Pavel Dovgalyuk]

This patch splits code in BootLinuxConsole class into two different
classes to allow reusing it by record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
---
 0 files changed

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index c6b06a1a13..f23c5833b7 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -28,19 +28,13 @@ try:
 except CmdNotFoundError:
     P7ZIP_AVAILABLE = False
 
-class BootLinuxConsole(Test):
-    """
-    Boots a Linux kernel and checks that the console is operational and the
-    kernel command line is properly passed from QEMU to the kernel
-    """
-
-    timeout = 90
-
+class LinuxKernelUtils(Test):
     KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
 
-    def wait_for_console_pattern(self, success_message):
+    def wait_for_console_pattern(self, success_message, vm=None):
         wait_for_console_pattern(self, success_message,
-                                 failure_message='Kernel panic - not syncing')
+                                 failure_message='Kernel panic - not syncing',
+                                 vm=vm)
 
     def extract_from_deb(self, deb, path):
         """
@@ -79,6 +73,13 @@ class BootLinuxConsole(Test):
         os.chdir(cwd)
         return os.path.normpath(os.path.join(self.workdir, path))
 
+class BootLinuxConsole(LinuxKernelUtils):
+    """
+    Boots a Linux kernel and checks that the console is operational and the
+    kernel command line is properly passed from QEMU to the kernel
+    """
+    timeout = 90
+
     def test_x86_64_pc(self):
         """
         :avocado: tags=arch:x86_64

Re: [PATCH v2 02/11] tests/acceptance: refactor boot_linux_console test to allow code reuse

[Alex Bennée]

Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:

> This patch splits code in BootLinuxConsole class into two different
> classes to allow reusing it by record/replay tests.
>
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> ---
>  0 files changed
>
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index c6b06a1a13..f23c5833b7 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -28,19 +28,13 @@ try:
>  except CmdNotFoundError:
>      P7ZIP_AVAILABLE = False
>  
> -class BootLinuxConsole(Test):
> -    """
> -    Boots a Linux kernel and checks that the console is operational and the
> -    kernel command line is properly passed from QEMU to the kernel
> -    """
> -
> -    timeout = 90
> -
> +class LinuxKernelUtils(Test):

I'm not super keen on the name given it is being sub-classed. Maybe:

  class LinuxKernelTest(Test):

and then:

  class BootLinuxConsole(LinuxKernelTest)

?


Otherwise:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

[PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

[Pavel Dovgalyuk]

This patch adds a base for testing kernel boot recording and replaying.
Each test has the phase of recording and phase of replaying.
Virtual machines just boot the kernel and do not interact with
the network.
Structure and image links for the tests are borrowed from boot_linux_console.py
Testing controls the message pattern at the end of the kernel
boot for both record and replay modes. In replay mode QEMU is also
intended to finish the execution automatically.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>

--

v2:
 - changed default value of args to None (suggested by Willian Rampazzo)
 - inherited common functions from boot_linux_console (suggested by Willian Rampazzo)
---
 0 files changed

diff --git a/MAINTAINERS b/MAINTAINERS
index 47ef3139e6..e9a9ce4f66 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2497,6 +2497,7 @@ F: net/filter-replay.c
 F: include/sysemu/replay.h
 F: docs/replay.txt
 F: stubs/replay.c
+F: tests/acceptance/replay_kernel.py
 
 IOVA Tree
 M: Peter Xu <peterx@redhat.com>
diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
new file mode 100644
index 0000000000..b8b277ad2f
--- /dev/null
+++ b/tests/acceptance/replay_kernel.py
@@ -0,0 +1,57 @@
+# Record/replay test that boots a Linux kernel
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+import gzip
+
+from avocado_qemu import wait_for_console_pattern
+from avocado.utils import process
+from avocado.utils import archive
+from boot_linux_console import LinuxKernelUtils
+
+class ReplayKernel(LinuxKernelUtils):
+    """
+    Boots a Linux kernel in record mode and checks that the console
+    is operational and the kernel command line is properly passed
+    from QEMU to the kernel.
+    Then replays the same scenario and verifies, that QEMU correctly
+    terminates.
+    """
+
+    timeout = 90
+
+    def run_vm(self, kernel_path, kernel_command_line, console_pattern,
+               record, shift, args):
+        vm = self.get_vm()
+        vm.set_console()
+        if record:
+            mode = 'record'
+        else:
+            mode = 'replay'
+        vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
+                    (shift, mode, os.path.join(self.workdir, 'replay.bin')),
+                    '-kernel', kernel_path,
+                    '-append', kernel_command_line,
+                    '-net', 'none')
+        if args:
+            vm.add_args(*args)
+        vm.launch()
+        self.wait_for_console_pattern(console_pattern, vm)
+        if record:
+            vm.shutdown()
+        else:
+            vm.wait()
+
+    def run_rr(self, kernel_path, kernel_command_line, console_pattern,
+        shift=7, args=None):
+        self.run_vm(kernel_path, kernel_command_line, console_pattern,
+                    True, shift, args)
+        self.run_vm(kernel_path, kernel_command_line, console_pattern,
+                    False, shift, args)

Re: [PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

[Philippe Mathieu-Daudé]

On 5/27/20 12:30 PM, Pavel Dovgalyuk wrote:
> This patch adds a base for testing kernel boot recording and replaying.
> Each test has the phase of recording and phase of replaying.
> Virtual machines just boot the kernel and do not interact with
> the network.
> Structure and image links for the tests are borrowed from boot_linux_console.py
> Testing controls the message pattern at the end of the kernel
> boot for both record and replay modes. In replay mode QEMU is also
> intended to finish the execution automatically.
> 
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> 
> --
> 
> v2:
>  - changed default value of args to None (suggested by Willian Rampazzo)
>  - inherited common functions from boot_linux_console (suggested by Willian Rampazzo)
> ---
>  0 files changed
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 47ef3139e6..e9a9ce4f66 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2497,6 +2497,7 @@ F: net/filter-replay.c
>  F: include/sysemu/replay.h
>  F: docs/replay.txt
>  F: stubs/replay.c
> +F: tests/acceptance/replay_kernel.py
>  
>  IOVA Tree
>  M: Peter Xu <peterx@redhat.com>
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> new file mode 100644
> index 0000000000..b8b277ad2f
> --- /dev/null
> +++ b/tests/acceptance/replay_kernel.py
> @@ -0,0 +1,57 @@
> +# Record/replay test that boots a Linux kernel
> +#
> +# Copyright (c) 2020 ISP RAS
> +#
> +# Author:
> +#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import os
> +import gzip
> +
> +from avocado_qemu import wait_for_console_pattern
> +from avocado.utils import process
> +from avocado.utils import archive
> +from boot_linux_console import LinuxKernelUtils
> +
> +class ReplayKernel(LinuxKernelUtils):
> +    """
> +    Boots a Linux kernel in record mode and checks that the console
> +    is operational and the kernel command line is properly passed
> +    from QEMU to the kernel.
> +    Then replays the same scenario and verifies, that QEMU correctly
> +    terminates.
> +    """
> +
> +    timeout = 90
> +
> +    def run_vm(self, kernel_path, kernel_command_line, console_pattern,
> +               record, shift, args):
> +        vm = self.get_vm()
> +        vm.set_console()
> +        if record:
> +            mode = 'record'
> +        else:
> +            mode = 'replay'
> +        vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
> +                    (shift, mode, os.path.join(self.workdir, 'replay.bin')),

Please make replay_filename an argument, so we can add tests replaying
without having to record, and or do verification on the replay_filename
(check size, check header, run with incorrect format...).

> +                    '-kernel', kernel_path,
> +                    '-append', kernel_command_line,
> +                    '-net', 'none')
> +        if args:
> +            vm.add_args(*args)
> +        vm.launch()
> +        self.wait_for_console_pattern(console_pattern, vm)
> +        if record:
> +            vm.shutdown()
> +        else:
> +            vm.wait()
> +
> +    def run_rr(self, kernel_path, kernel_command_line, console_pattern,
> +        shift=7, args=None):
> +        self.run_vm(kernel_path, kernel_command_line, console_pattern,
> +                    True, shift, args)
> +        self.run_vm(kernel_path, kernel_command_line, console_pattern,
> +                    False, shift, args)
> 

Also please consider this snippet for logging:

-- >8 --
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -10,6 +10,7 @@

 import os
 import gzip
+import logging

 from avocado_qemu import wait_for_console_pattern
 from avocado.utils import process
@@ -51,8 +52,11 @@ def run_vm(self, kernel_path, kernel_command_line,
console_pattern,

     def run_rr(self, kernel_path, kernel_command_line, console_pattern,
         shift=7, args=None):
+        logger = logging.getLogger('replay')
+        logger.info('recording...')
         self.run_vm(kernel_path, kernel_command_line, console_pattern,
                     True, shift, args)
+        logger.info('replaying...')
         self.run_vm(kernel_path, kernel_command_line, console_pattern,
                     False, shift, args)
---

Then you can use the logger using:

avocado --show=app,replay run -t machine:vexpress-a9 \
  tests/acceptance/replay_kernel.py

Ideally I'd like to see timestamp (Avocado surely provide the option)
and record.bin size between.

Re: [PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

[Alex Bennée]

Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:

> This patch adds a base for testing kernel boot recording and replaying.
> Each test has the phase of recording and phase of replaying.
> Virtual machines just boot the kernel and do not interact with
> the network.
> Structure and image links for the tests are borrowed from boot_linux_console.py
> Testing controls the message pattern at the end of the kernel
> boot for both record and replay modes. In replay mode QEMU is also
> intended to finish the execution automatically.
>
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>

diff --git a/MAINTAINERS b/MAINTAINERS
index 47ef3139e6..e9a9ce4f66 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2497,6 +2497,7 @@ F: net/filter-replay.c
 F: include/sysemu/replay.h
 F: docs/replay.txt
 F: stubs/replay.c
+F: tests/acceptance/replay_kernel.py
 
 IOVA Tree
 M: Peter Xu <peterx@redhat.com>
diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
new file mode 100644
index 0000000000..b8b277ad2f
--- /dev/null
+++ b/tests/acceptance/replay_kernel.py
@@ -0,0 +1,57 @@
+# Record/replay test that boots a Linux kernel
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+import gzip

Do we actually use gzip in this test?

+
+from avocado_qemu import wait_for_console_pattern
+from avocado.utils import process
+from avocado.utils import archive
+from boot_linux_console import LinuxKernelUtils
+
+class ReplayKernel(LinuxKernelUtils):
+    """
+    Boots a Linux kernel in record mode and checks that the console
+    is operational and the kernel command line is properly passed
+    from QEMU to the kernel.
+    Then replays the same scenario and verifies, that QEMU correctly
+    terminates.

Shouldn't we be doing more to verify the replay behaved the same as the
recorded session? What happens if things go wrong? Does QEMU barf out or
just deviate from the previous run?

+    """
+
+    timeout = 90
+
+    def run_vm(self, kernel_path, kernel_command_line, console_pattern,
+               record, shift, args):
+        vm = self.get_vm()
+        vm.set_console()
+        if record:
+            mode = 'record'
+        else:
+            mode = 'replay'
+        vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
+                    (shift, mode, os.path.join(self.workdir, 'replay.bin')),
+                    '-kernel', kernel_path,
+                    '-append', kernel_command_line,
+                    '-net', 'none')
+        if args:
+            vm.add_args(*args)
+        vm.launch()
+        self.wait_for_console_pattern(console_pattern, vm)
+        if record:
+            vm.shutdown()
+        else:
+            vm.wait()
+
+    def run_rr(self, kernel_path, kernel_command_line, console_pattern,
+        shift=7, args=None):
+        self.run_vm(kernel_path, kernel_command_line, console_pattern,
+                    True, shift, args)
+        self.run_vm(kernel_path, kernel_command_line, console_pattern,
+                    False, shift, args)



-- 
Alex Bennée

Re: [PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

[Pavel Dovgalyuk]

On 27.05.2020 18:20, Alex Bennée wrote:
> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>
>> This patch adds a base for testing kernel boot recording and replaying.
>> Each test has the phase of recording and phase of replaying.
>> Virtual machines just boot the kernel and do not interact with
>> the network.
>> Structure and image links for the tests are borrowed from boot_linux_console.py
>> Testing controls the message pattern at the end of the kernel
>> boot for both record and replay modes. In replay mode QEMU is also
>> intended to finish the execution automatically.
>>
>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 47ef3139e6..e9a9ce4f66 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2497,6 +2497,7 @@ F: net/filter-replay.c
>   F: include/sysemu/replay.h
>   F: docs/replay.txt
>   F: stubs/replay.c
> +F: tests/acceptance/replay_kernel.py
>   
>   IOVA Tree
>   M: Peter Xu <peterx@redhat.com>
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> new file mode 100644
> index 0000000000..b8b277ad2f
> --- /dev/null
> +++ b/tests/acceptance/replay_kernel.py
> @@ -0,0 +1,57 @@
> +# Record/replay test that boots a Linux kernel
> +#
> +# Copyright (c) 2020 ISP RAS
> +#
> +# Author:
> +#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import os
> +import gzip
>
> Do we actually use gzip in this test?

Removed that, thanks.

>
> +
> +from avocado_qemu import wait_for_console_pattern
> +from avocado.utils import process
> +from avocado.utils import archive
> +from boot_linux_console import LinuxKernelUtils
> +
> +class ReplayKernel(LinuxKernelUtils):
> +    """
> +    Boots a Linux kernel in record mode and checks that the console
> +    is operational and the kernel command line is properly passed
> +    from QEMU to the kernel.
> +    Then replays the same scenario and verifies, that QEMU correctly
> +    terminates.
>
> Shouldn't we be doing more to verify the replay behaved the same as the
> recorded session? What happens if things go wrong? Does QEMU barf out or
> just deviate from the previous run?

We hardly can compare vCPU states during record and replay.

But in the most cases it is not needed. When control flow goes in the 
wrong direction, it affects the interrupts and exceptions.

And interrupts and exceptions are the synchronization points in the 
replay log. Therefore when the executions differ, QEMU replay just hangs.

>
> +    """
> +
> +    timeout = 90
> +
> +    def run_vm(self, kernel_path, kernel_command_line, console_pattern,
> +               record, shift, args):
> +        vm = self.get_vm()
> +        vm.set_console()
> +        if record:
> +            mode = 'record'
> +        else:
> +            mode = 'replay'
> +        vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
> +                    (shift, mode, os.path.join(self.workdir, 'replay.bin')),
> +                    '-kernel', kernel_path,
> +                    '-append', kernel_command_line,
> +                    '-net', 'none')
> +        if args:
> +            vm.add_args(*args)
> +        vm.launch()
> +        self.wait_for_console_pattern(console_pattern, vm)
> +        if record:
> +            vm.shutdown()
> +        else:
> +            vm.wait()
> +
> +    def run_rr(self, kernel_path, kernel_command_line, console_pattern,
> +        shift=7, args=None):
> +        self.run_vm(kernel_path, kernel_command_line, console_pattern,
> +                    True, shift, args)
> +        self.run_vm(kernel_path, kernel_command_line, console_pattern,
> +                    False, shift, args)
>
>
>

Re: [PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

[Alex Bennée]

Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:

> On 27.05.2020 18:20, Alex Bennée wrote:
>> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>>
>>> This patch adds a base for testing kernel boot recording and replaying.
>>> Each test has the phase of recording and phase of replaying.
>>> Virtual machines just boot the kernel and do not interact with
>>> the network.
>>> Structure and image links for the tests are borrowed from boot_linux_console.py
>>> Testing controls the message pattern at the end of the kernel
>>> boot for both record and replay modes. In replay mode QEMU is also
>>> intended to finish the execution automatically.
>>>
>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 47ef3139e6..e9a9ce4f66 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -2497,6 +2497,7 @@ F: net/filter-replay.c
>>   F: include/sysemu/replay.h
>>   F: docs/replay.txt
>>   F: stubs/replay.c
>> +F: tests/acceptance/replay_kernel.py
>>     IOVA Tree
>>   M: Peter Xu <peterx@redhat.com>
>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>> new file mode 100644
>> index 0000000000..b8b277ad2f
>> --- /dev/null
>> +++ b/tests/acceptance/replay_kernel.py
>> @@ -0,0 +1,57 @@
>> +# Record/replay test that boots a Linux kernel
>> +#
>> +# Copyright (c) 2020 ISP RAS
>> +#
>> +# Author:
>> +#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later.  See the COPYING file in the top-level directory.
>> +
>> +import os
>> +import gzip
>>
>> Do we actually use gzip in this test?
>
> Removed that, thanks.
>
>>
>> +
>> +from avocado_qemu import wait_for_console_pattern
>> +from avocado.utils import process
>> +from avocado.utils import archive
>> +from boot_linux_console import LinuxKernelUtils
>> +
>> +class ReplayKernel(LinuxKernelUtils):
>> +    """
>> +    Boots a Linux kernel in record mode and checks that the console
>> +    is operational and the kernel command line is properly passed
>> +    from QEMU to the kernel.
>> +    Then replays the same scenario and verifies, that QEMU correctly
>> +    terminates.
>>
>> Shouldn't we be doing more to verify the replay behaved the same as the
>> recorded session? What happens if things go wrong? Does QEMU barf out or
>> just deviate from the previous run?
>
> We hardly can compare vCPU states during record and replay.
>
> But in the most cases it is not needed. When control flow goes in the
> wrong direction, it affects the interrupts and exceptions.
>
> And interrupts and exceptions are the synchronization points in the
> replay log. Therefore when the executions differ, QEMU replay just
> hangs.

Maybe we should fix that and exit with a more definitive error? Hangs
are just plain ugly to debug because your first step has to be to start
poking around with a debugger.

-- 
Alex Bennée

Re: [PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

[Pavel Dovgalyuk]

On 28.05.2020 11:28, Alex Bennée wrote:
> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
>
>> On 27.05.2020 18:20, Alex Bennée wrote:
>>> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>>>
>>>> This patch adds a base for testing kernel boot recording and replaying.
>>>> Each test has the phase of recording and phase of replaying.
>>>> Virtual machines just boot the kernel and do not interact with
>>>> the network.
>>>> Structure and image links for the tests are borrowed from boot_linux_console.py
>>>> Testing controls the message pattern at the end of the kernel
>>>> boot for both record and replay modes. In replay mode QEMU is also
>>>> intended to finish the execution automatically.
>>>>
>>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 47ef3139e6..e9a9ce4f66 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -2497,6 +2497,7 @@ F: net/filter-replay.c
>>>    F: include/sysemu/replay.h
>>>    F: docs/replay.txt
>>>    F: stubs/replay.c
>>> +F: tests/acceptance/replay_kernel.py
>>>      IOVA Tree
>>>    M: Peter Xu <peterx@redhat.com>
>>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>>> new file mode 100644
>>> index 0000000000..b8b277ad2f
>>> --- /dev/null
>>> +++ b/tests/acceptance/replay_kernel.py
>>> @@ -0,0 +1,57 @@
>>> +# Record/replay test that boots a Linux kernel
>>> +#
>>> +# Copyright (c) 2020 ISP RAS
>>> +#
>>> +# Author:
>>> +#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>>> +#
>>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>>> +# later.  See the COPYING file in the top-level directory.
>>> +
>>> +import os
>>> +import gzip
>>>
>>> Do we actually use gzip in this test?
>> Removed that, thanks.
>>
>>> +
>>> +from avocado_qemu import wait_for_console_pattern
>>> +from avocado.utils import process
>>> +from avocado.utils import archive
>>> +from boot_linux_console import LinuxKernelUtils
>>> +
>>> +class ReplayKernel(LinuxKernelUtils):
>>> +    """
>>> +    Boots a Linux kernel in record mode and checks that the console
>>> +    is operational and the kernel command line is properly passed
>>> +    from QEMU to the kernel.
>>> +    Then replays the same scenario and verifies, that QEMU correctly
>>> +    terminates.
>>>
>>> Shouldn't we be doing more to verify the replay behaved the same as the
>>> recorded session? What happens if things go wrong? Does QEMU barf out or
>>> just deviate from the previous run?
>> We hardly can compare vCPU states during record and replay.
>>
>> But in the most cases it is not needed. When control flow goes in the
>> wrong direction, it affects the interrupts and exceptions.
>>
>> And interrupts and exceptions are the synchronization points in the
>> replay log. Therefore when the executions differ, QEMU replay just
>> hangs.
> Maybe we should fix that and exit with a more definitive error? Hangs
> are just plain ugly to debug because your first step has to be to start
> poking around with a debugger.
>
Good point, I'll thinks about it.

[PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Pavel Dovgalyuk]

This patch adds a test for record/replay an execution of x86_64 machine.
Execution scenario includes simple kernel boot, which allows testing
basic hardware interaction in RR mode.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
---
 0 files changed

diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index b8b277ad2f..c7526f1aba 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
                     True, shift, args)
         self.run_vm(kernel_path, kernel_command_line, console_pattern,
                     False, shift, args)
+
+    def test_x86_64_pc(self):
+        """
+        :avocado: tags=arch:x86_64
+        :avocado: tags=machine:pc
+        """
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
+                      '/vmlinuz')
+        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+
+        self.run_rr(kernel_path, kernel_command_line, console_pattern)

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Philippe Mathieu-Daudé]

On 5/27/20 12:31 PM, Pavel Dovgalyuk wrote:
> This patch adds a test for record/replay an execution of x86_64 machine.
> Execution scenario includes simple kernel boot, which allows testing
> basic hardware interaction in RR mode.
> 
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> ---
>  0 files changed
> 
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index b8b277ad2f..c7526f1aba 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>                      True, shift, args)
>          self.run_vm(kernel_path, kernel_command_line, console_pattern,
>                      False, shift, args)
> +
> +    def test_x86_64_pc(self):
> +        """
> +        :avocado: tags=arch:x86_64
> +        :avocado: tags=machine:pc
> +        """
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
> +                      '/vmlinuz')
> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +
> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
> +        console_pattern = 'Kernel command line: %s' % kernel_command_line
> +
> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)
> 

This one timeouted (I build with --enable-debug):

 (1/1) tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc:
replay: recording...
replay: replaying...
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout
reached\nOriginal status: ERROR\n{'name':
'1-tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc',
'logdir':
'avocado/job-results/job-2020-05-27T16.48-71d7bf4/test-results/1-tes...
(90.68 s)

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Pavel Dovgalyuk]

On 27.05.2020 17:53, Philippe Mathieu-Daudé wrote:
> On 5/27/20 12:31 PM, Pavel Dovgalyuk wrote:
>> This patch adds a test for record/replay an execution of x86_64 machine.
>> Execution scenario includes simple kernel boot, which allows testing
>> basic hardware interaction in RR mode.
>>
>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> ---
>>   0 files changed
>>
>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>> index b8b277ad2f..c7526f1aba 100644
>> --- a/tests/acceptance/replay_kernel.py
>> +++ b/tests/acceptance/replay_kernel.py
>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>                       True, shift, args)
>>           self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>                       False, shift, args)
>> +
>> +    def test_x86_64_pc(self):
>> +        """
>> +        :avocado: tags=arch:x86_64
>> +        :avocado: tags=machine:pc
>> +        """
>> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>> +                      '/vmlinuz')
>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>> +
>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
>> +        console_pattern = 'Kernel command line: %s' % kernel_command_line
>> +
>> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)
>>
> This one timeouted (I build with --enable-debug):

I've got the strange behavior for the couple of times.

Console output was correct (I saw 'Kernel command line' in logs), but 
_console_interation function didn't notice it.

Therefore the test finished with timeout.

How this could be possible?


>   (1/1) tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc:
> replay: recording...
> replay: replaying...
> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout
> reached\nOriginal status: ERROR\n{'name':
> '1-tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc',
> 'logdir':
> 'avocado/job-results/job-2020-05-27T16.48-71d7bf4/test-results/1-tes...
> (90.68 s)
>

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Philippe Mathieu-Daudé]

On 5/28/20 9:12 AM, Pavel Dovgalyuk wrote:
> 
> On 27.05.2020 17:53, Philippe Mathieu-Daudé wrote:
>> On 5/27/20 12:31 PM, Pavel Dovgalyuk wrote:
>>> This patch adds a test for record/replay an execution of x86_64 machine.
>>> Execution scenario includes simple kernel boot, which allows testing
>>> basic hardware interaction in RR mode.
>>>
>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>>> ---
>>>   0 files changed
>>>
>>> diff --git a/tests/acceptance/replay_kernel.py
>>> b/tests/acceptance/replay_kernel.py
>>> index b8b277ad2f..c7526f1aba 100644
>>> --- a/tests/acceptance/replay_kernel.py
>>> +++ b/tests/acceptance/replay_kernel.py
>>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>>                       True, shift, args)
>>>           self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>>                       False, shift, args)
>>> +
>>> +    def test_x86_64_pc(self):
>>> +        """
>>> +        :avocado: tags=arch:x86_64
>>> +        :avocado: tags=machine:pc
>>> +        """
>>> +        kernel_url =
>>> ('https://archives.fedoraproject.org/pub/archive/fedora'
>>> +                     
>>> '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>>> +                      '/vmlinuz')
>>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>>> +        kernel_path = self.fetch_asset(kernel_url,
>>> asset_hash=kernel_hash)
>>> +
>>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE +
>>> 'console=ttyS0'
>>> +        console_pattern = 'Kernel command line: %s' %
>>> kernel_command_line
>>> +
>>> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)
>>>
>> This one timeouted (I build with --enable-debug):
> 
> I've got the strange behavior for the couple of times.
> 
> Console output was correct (I saw 'Kernel command line' in logs), but
> _console_interation function didn't notice it.
> 
> Therefore the test finished with timeout.
> 
> How this could be possible?

IIRC there is a problem in how Avocado consume the chardev output.

Cleber has been working on some PoC / kludge.

Cleber/Eduardo do you remember the problem?

> 
>>   (1/1) tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc:
>> replay: recording...
>> replay: replaying...
>> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout
>> reached\nOriginal status: ERROR\n{'name':
>> '1-tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc',
>> 'logdir':
>> 'avocado/job-results/job-2020-05-27T16.48-71d7bf4/test-results/1-tes...
>> (90.68 s)
>>
> 

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@redhat.com> writes:

> On 5/28/20 9:12 AM, Pavel Dovgalyuk wrote:
>> 
>> On 27.05.2020 17:53, Philippe Mathieu-Daudé wrote:
>>> On 5/27/20 12:31 PM, Pavel Dovgalyuk wrote:
>>>> This patch adds a test for record/replay an execution of x86_64 machine.
>>>> Execution scenario includes simple kernel boot, which allows testing
>>>> basic hardware interaction in RR mode.
>>>>
>>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>>>> ---
>>>>   0 files changed
>>>>
>>>> diff --git a/tests/acceptance/replay_kernel.py
>>>> b/tests/acceptance/replay_kernel.py
>>>> index b8b277ad2f..c7526f1aba 100644
>>>> --- a/tests/acceptance/replay_kernel.py
>>>> +++ b/tests/acceptance/replay_kernel.py
>>>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>>>                       True, shift, args)
>>>>           self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>>>                       False, shift, args)
>>>> +
>>>> +    def test_x86_64_pc(self):
>>>> +        """
>>>> +        :avocado: tags=arch:x86_64
>>>> +        :avocado: tags=machine:pc
>>>> +        """
>>>> +        kernel_url =
>>>> ('https://archives.fedoraproject.org/pub/archive/fedora'
>>>> +                     
>>>> '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>>>> +                      '/vmlinuz')
>>>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>>>> +        kernel_path = self.fetch_asset(kernel_url,
>>>> asset_hash=kernel_hash)
>>>> +
>>>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE +
>>>> 'console=ttyS0'
>>>> +        console_pattern = 'Kernel command line: %s' %
>>>> kernel_command_line
>>>> +
>>>> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)
>>>>
>>> This one timeouted (I build with --enable-debug):
>> 
>> I've got the strange behavior for the couple of times.
>> 
>> Console output was correct (I saw 'Kernel command line' in logs), but
>> _console_interation function didn't notice it.
>> 
>> Therefore the test finished with timeout.
>> 
>> How this could be possible?
>
> IIRC there is a problem in how Avocado consume the chardev output.

Is this the same problem Robert has tried to work around in tests/vm?

  From: Robert Foley <robert.foley@linaro.org>
  Subject: [PATCH v7 12/12] tests/vm: Add workaround to consume console
  Date: Tue, 19 May 2020 09:22:59 -0400
  Message-Id: <20200519132259.405-13-robert.foley@linaro.org>

>
> Cleber has been working on some PoC / kludge.
>
> Cleber/Eduardo do you remember the problem?
>
>> 
>>>   (1/1) tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc:
>>> replay: recording...
>>> replay: replaying...
>>> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout
>>> reached\nOriginal status: ERROR\n{'name':
>>> '1-tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc',
>>> 'logdir':
>>> 'avocado/job-results/job-2020-05-27T16.48-71d7bf4/test-results/1-tes...
>>> (90.68 s)
>>>
>> 


-- 
Alex Bennée

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Philippe Mathieu-Daudé]

On 5/28/20 1:07 PM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <philmd@redhat.com> writes:
> 
>> On 5/28/20 9:12 AM, Pavel Dovgalyuk wrote:
>>>
>>> On 27.05.2020 17:53, Philippe Mathieu-Daudé wrote:
>>>> On 5/27/20 12:31 PM, Pavel Dovgalyuk wrote:
>>>>> This patch adds a test for record/replay an execution of x86_64 machine.
>>>>> Execution scenario includes simple kernel boot, which allows testing
>>>>> basic hardware interaction in RR mode.
>>>>>
>>>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>>>>> ---
>>>>>   0 files changed
>>>>>
>>>>> diff --git a/tests/acceptance/replay_kernel.py
>>>>> b/tests/acceptance/replay_kernel.py
>>>>> index b8b277ad2f..c7526f1aba 100644
>>>>> --- a/tests/acceptance/replay_kernel.py
>>>>> +++ b/tests/acceptance/replay_kernel.py
>>>>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>>>>                       True, shift, args)
>>>>>           self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>>>>                       False, shift, args)
>>>>> +
>>>>> +    def test_x86_64_pc(self):
>>>>> +        """
>>>>> +        :avocado: tags=arch:x86_64
>>>>> +        :avocado: tags=machine:pc
>>>>> +        """
>>>>> +        kernel_url =
>>>>> ('https://archives.fedoraproject.org/pub/archive/fedora'
>>>>> +                     
>>>>> '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>>>>> +                      '/vmlinuz')
>>>>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>>>>> +        kernel_path = self.fetch_asset(kernel_url,
>>>>> asset_hash=kernel_hash)
>>>>> +
>>>>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE +
>>>>> 'console=ttyS0'
>>>>> +        console_pattern = 'Kernel command line: %s' %
>>>>> kernel_command_line
>>>>> +
>>>>> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)
>>>>>
>>>> This one timeouted (I build with --enable-debug):
>>>
>>> I've got the strange behavior for the couple of times.
>>>
>>> Console output was correct (I saw 'Kernel command line' in logs), but
>>> _console_interation function didn't notice it.
>>>
>>> Therefore the test finished with timeout.
>>>
>>> How this could be possible?
>>
>> IIRC there is a problem in how Avocado consume the chardev output.
> 
> Is this the same problem Robert has tried to work around in tests/vm?
> 
>   From: Robert Foley <robert.foley@linaro.org>
>   Subject: [PATCH v7 12/12] tests/vm: Add workaround to consume console
>   Date: Tue, 19 May 2020 09:22:59 -0400
>   Message-Id: <20200519132259.405-13-robert.foley@linaro.org>

Exactly! I guess Marc-André or Eduardo suggested the same fix (use a
dedicated polling thread).

> 
>>
>> Cleber has been working on some PoC / kludge.
>>
>> Cleber/Eduardo do you remember the problem?
>>
>>>
>>>>   (1/1) tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc:
>>>> replay: recording...
>>>> replay: replaying...
>>>> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout
>>>> reached\nOriginal status: ERROR\n{'name':
>>>> '1-tests/acceptance/replay_kernel.py:ReplayKernel.test_x86_64_pc',
>>>> 'logdir':
>>>> 'avocado/job-results/job-2020-05-27T16.48-71d7bf4/test-results/1-tes...
>>>> (90.68 s)
>>>>
>>>
> 
> 

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Alex Bennée]

Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:

> This patch adds a test for record/replay an execution of x86_64 machine.
> Execution scenario includes simple kernel boot, which allows testing
> basic hardware interaction in RR mode.
>
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> ---
>  0 files changed
>
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index b8b277ad2f..c7526f1aba 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>                      True, shift, args)
>          self.run_vm(kernel_path, kernel_command_line, console_pattern,
>                      False, shift, args)
> +
> +    def test_x86_64_pc(self):
> +        """
> +        :avocado: tags=arch:x86_64
> +        :avocado: tags=machine:pc
> +        """
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
> +                      '/vmlinuz')
> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +
> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE +
> 'console=ttyS0'

I note that:

  KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '

and given we are looking for repeatability here maybe we should use our
own command line so we can compare the recorded and replayed boot?

> +        console_pattern = 'Kernel command line: %s' % kernel_command_line
> +
> +        self.run_rr(kernel_path, kernel_command_line,
> console_pattern)

-- 
Alex Bennée

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Alex Bennée]

Alex Bennée <alex.bennee@linaro.org> writes:

> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>
>> This patch adds a test for record/replay an execution of x86_64 machine.
>> Execution scenario includes simple kernel boot, which allows testing
>> basic hardware interaction in RR mode.
>>
>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> ---
>>  0 files changed
>>
>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>> index b8b277ad2f..c7526f1aba 100644
>> --- a/tests/acceptance/replay_kernel.py
>> +++ b/tests/acceptance/replay_kernel.py
>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>                      True, shift, args)
>>          self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>                      False, shift, args)
>> +
>> +    def test_x86_64_pc(self):
>> +        """
>> +        :avocado: tags=arch:x86_64
>> +        :avocado: tags=machine:pc
>> +        """
>> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>> +                      '/vmlinuz')
>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>> +
>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE +
>> 'console=ttyS0'
>
> I note that:
>
>   KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>
> and given we are looking for repeatability here maybe we should use our
> own command line so we can compare the recorded and replayed boot?

To build on that I think a command line like:

  KERNEL_COMMON_COMMAND_LINE = 'printk.time=1 panic=-1 '

called with --no-reboot and a pattern:

  console_pattern = 'VFS: Cannot open root device'

You will run more of the kernel (importantly with timestamps > 0.000) so
we can have a better compare between the recorded and replayed run.

-- 
Alex Bennée

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Pavel Dovgalyuk]

On 27.05.2020 19:20, Alex Bennée wrote:
> Alex Bennée <alex.bennee@linaro.org> writes:
>
>> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>>
>>> This patch adds a test for record/replay an execution of x86_64 machine.
>>> Execution scenario includes simple kernel boot, which allows testing
>>> basic hardware interaction in RR mode.
>>>
>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>>> ---
>>>   0 files changed
>>>
>>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>>> index b8b277ad2f..c7526f1aba 100644
>>> --- a/tests/acceptance/replay_kernel.py
>>> +++ b/tests/acceptance/replay_kernel.py
>>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>>                       True, shift, args)
>>>           self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>>                       False, shift, args)
>>> +
>>> +    def test_x86_64_pc(self):
>>> +        """
>>> +        :avocado: tags=arch:x86_64
>>> +        :avocado: tags=machine:pc
>>> +        """
>>> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>>> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>>> +                      '/vmlinuz')
>>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>>> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>>> +
>>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE +
>>> 'console=ttyS0'
>> I note that:
>>
>>    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>>
>> and given we are looking for repeatability here maybe we should use our
>> own command line so we can compare the recorded and replayed boot?
> To build on that I think a command line like:
>
>    KERNEL_COMMON_COMMAND_LINE = 'printk.time=1 panic=-1 '
>
> called with --no-reboot and a pattern:
>
>    console_pattern = 'VFS: Cannot open root device'
>
> You will run more of the kernel (importantly with timestamps > 0.000) so
> we can have a better compare between the recorded and replayed run.
>
This is reasonable, thank you.

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Alex Bennée]

Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:

> This patch adds a test for record/replay an execution of x86_64 machine.
> Execution scenario includes simple kernel boot, which allows testing
> basic hardware interaction in RR mode.
>
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> ---
>  0 files changed
>
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index b8b277ad2f..c7526f1aba 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>                      True, shift, args)
>          self.run_vm(kernel_path, kernel_command_line, console_pattern,
>                      False, shift, args)
> +
> +    def test_x86_64_pc(self):
> +        """
> +        :avocado: tags=arch:x86_64
> +        :avocado: tags=machine:pc
> +        """
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
> +                      '/vmlinuz')
> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +
> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
> +        console_pattern = 'Kernel command line: %s' % kernel_command_line
> +
> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)

This test fails for me on the replay:

  2020-05-27 16:22:21,658 machine          L0326 DEBUG| VM launch command: 'x86_64-softmmu/qemu-system-x86_64 -display none -vga none -chardev socket,id=mon,path=/var/tmp/tmp4n_geosi/qemu-9516-monitor.sock -mon chardev=mon,mode=control -machine pc -chardev socket,id=console,path=/var/tmp/tmp4n_geosi/qemu-9516-console.sock,server,nowait -serial chardev:console -icount shift=7,rr=replay,rrfile=/var/tmp/avocado_b85h3ycg/avocado_job_8xrxksgj/1-._tests_acceptance_replay_kernel.py_ReplayKernel.test_x86_64_pc/replay.bin -kernel /home/alex/avocado/data/cache/by_location/df533120a0e0ffda2626bed6e8a975d3b07e3f05/vmlinuz -append printk.time=0 console=ttyS0 -net none'
  2020-05-27 16:22:21,725 qmp              L0194 DEBUG| >>> {'execute': 'qmp_capabilities'}
  2020-05-27 16:22:21,736 qmp              L0202 DEBUG| <<< {'return': {}}
  2020-05-27 16:23:49,372 stacktrace       L0039 ERROR|
  2020-05-27 16:23:49,372 stacktrace       L0042 ERROR| Reproduced traceback from: /home/alex/lsrc/qemu.git/builds/all/tests/venv/lib/python3.7/site-packages/avocado/core/test.py:860
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR| Traceback (most recent call last):
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 73, in test_x86_64_pc
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     self.run_rr(kernel_path, kernel_command_line, console_pattern)
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 57, in run_rr
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     False, shift, args)
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 46, in run_vm
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     self.wait_for_console_pattern(console_pattern, vm)
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/boot_linux_console.py", line 37, in wait_for_console_pattern
  2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     vm=vm)
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/avocado_qemu/__init__.py", line 131, in wait_for_console_pattern
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     _console_interaction(test, success_message, failure_message, None, vm=vm)
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/avocado_qemu/__init__.py", line 83, in _console_interaction
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     msg = console.readline().strip()
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/usr/lib/python3.7/socket.py", line 589, in readinto
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     return self._sock.recv_into(b)
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/venv/lib/python3.7/site-packages/avocado/plugins/runner.py", line 89, in sigterm_handler
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     raise RuntimeError("Test interrupted by SIGTERM")
  2020-05-27 16:23:49,374 stacktrace       L0045 ERROR| RuntimeError: Test interrupted by SIGTERM


-- 
Alex Bennée

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Pavel Dovgalyuk]

On 27.05.2020 18:41, Alex Bennée wrote:
> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>
>> This patch adds a test for record/replay an execution of x86_64 machine.
>> Execution scenario includes simple kernel boot, which allows testing
>> basic hardware interaction in RR mode.
>>
>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> ---
>>   0 files changed
>>
>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>> index b8b277ad2f..c7526f1aba 100644
>> --- a/tests/acceptance/replay_kernel.py
>> +++ b/tests/acceptance/replay_kernel.py
>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>                       True, shift, args)
>>           self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>                       False, shift, args)
>> +
>> +    def test_x86_64_pc(self):
>> +        """
>> +        :avocado: tags=arch:x86_64
>> +        :avocado: tags=machine:pc
>> +        """
>> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>> +                      '/vmlinuz')
>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>> +
>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
>> +        console_pattern = 'Kernel command line: %s' % kernel_command_line
>> +
>> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)
> This test fails for me on the replay:

Have you applied latest RR patches?


>
>    2020-05-27 16:22:21,658 machine          L0326 DEBUG| VM launch command: 'x86_64-softmmu/qemu-system-x86_64 -display none -vga none -chardev socket,id=mon,path=/var/tmp/tmp4n_geosi/qemu-9516-monitor.sock -mon chardev=mon,mode=control -machine pc -chardev socket,id=console,path=/var/tmp/tmp4n_geosi/qemu-9516-console.sock,server,nowait -serial chardev:console -icount shift=7,rr=replay,rrfile=/var/tmp/avocado_b85h3ycg/avocado_job_8xrxksgj/1-._tests_acceptance_replay_kernel.py_ReplayKernel.test_x86_64_pc/replay.bin -kernel /home/alex/avocado/data/cache/by_location/df533120a0e0ffda2626bed6e8a975d3b07e3f05/vmlinuz -append printk.time=0 console=ttyS0 -net none'
>    2020-05-27 16:22:21,725 qmp              L0194 DEBUG| >>> {'execute': 'qmp_capabilities'}
>    2020-05-27 16:22:21,736 qmp              L0202 DEBUG| <<< {'return': {}}
>    2020-05-27 16:23:49,372 stacktrace       L0039 ERROR|
>    2020-05-27 16:23:49,372 stacktrace       L0042 ERROR| Reproduced traceback from: /home/alex/lsrc/qemu.git/builds/all/tests/venv/lib/python3.7/site-packages/avocado/core/test.py:860
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR| Traceback (most recent call last):
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 73, in test_x86_64_pc
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     self.run_rr(kernel_path, kernel_command_line, console_pattern)
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 57, in run_rr
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     False, shift, args)
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 46, in run_vm
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     self.wait_for_console_pattern(console_pattern, vm)
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/boot_linux_console.py", line 37, in wait_for_console_pattern
>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     vm=vm)
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/avocado_qemu/__init__.py", line 131, in wait_for_console_pattern
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     _console_interaction(test, success_message, failure_message, None, vm=vm)
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/avocado_qemu/__init__.py", line 83, in _console_interaction
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     msg = console.readline().strip()
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/usr/lib/python3.7/socket.py", line 589, in readinto
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     return self._sock.recv_into(b)
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/venv/lib/python3.7/site-packages/avocado/plugins/runner.py", line 89, in sigterm_handler
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     raise RuntimeError("Test interrupted by SIGTERM")
>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR| RuntimeError: Test interrupted by SIGTERM
>
>

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Alex Bennée]

Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:

> On 27.05.2020 18:41, Alex Bennée wrote:
>> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>>
>>> This patch adds a test for record/replay an execution of x86_64 machine.
>>> Execution scenario includes simple kernel boot, which allows testing
>>> basic hardware interaction in RR mode.
>>>
>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>>> ---
>>>   0 files changed
>>>
>>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>>> index b8b277ad2f..c7526f1aba 100644
>>> --- a/tests/acceptance/replay_kernel.py
>>> +++ b/tests/acceptance/replay_kernel.py
>>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>>                       True, shift, args)
>>>           self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>>                       False, shift, args)
>>> +
>>> +    def test_x86_64_pc(self):
>>> +        """
>>> +        :avocado: tags=arch:x86_64
>>> +        :avocado: tags=machine:pc
>>> +        """
>>> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>>> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>>> +                      '/vmlinuz')
>>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>>> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>>> +
>>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
>>> +        console_pattern = 'Kernel command line: %s' % kernel_command_line
>>> +
>>> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)
>> This test fails for me on the replay:
>
> Have you applied latest RR patches?

I have the following on top of the acceptance patches:

a36c23042fe * review/record-replay-acceptance-v2 icount: fix shift=auto for record/replay
4ab2164c10b * replay: synchronize on every virtual timer callback
66104ce6e4b * replay: notify the main loop when there are no instructions

>
>
>>
>>    2020-05-27 16:22:21,658 machine          L0326 DEBUG| VM launch command: 'x86_64-softmmu/qemu-system-x86_64 -display none -vga none -chardev socket,id=mon,path=/var/tmp/tmp4n_geosi/qemu-9516-monitor.sock -mon chardev=mon,mode=control -machine pc -chardev socket,id=console,path=/var/tmp/tmp4n_geosi/qemu-9516-console.sock,server,nowait -serial chardev:console -icount shift=7,rr=replay,rrfile=/var/tmp/avocado_b85h3ycg/avocado_job_8xrxksgj/1-._tests_acceptance_replay_kernel.py_ReplayKernel.test_x86_64_pc/replay.bin -kernel /home/alex/avocado/data/cache/by_location/df533120a0e0ffda2626bed6e8a975d3b07e3f05/vmlinuz -append printk.time=0 console=ttyS0 -net none'
>>    2020-05-27 16:22:21,725 qmp              L0194 DEBUG| >>> {'execute': 'qmp_capabilities'}
>>    2020-05-27 16:22:21,736 qmp              L0202 DEBUG| <<< {'return': {}}
>>    2020-05-27 16:23:49,372 stacktrace       L0039 ERROR|
>>    2020-05-27 16:23:49,372 stacktrace       L0042 ERROR| Reproduced traceback from: /home/alex/lsrc/qemu.git/builds/all/tests/venv/lib/python3.7/site-packages/avocado/core/test.py:860
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR| Traceback (most recent call last):
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 73, in test_x86_64_pc
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     self.run_rr(kernel_path, kernel_command_line, console_pattern)
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 57, in run_rr
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     False, shift, args)
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/replay_kernel.py", line 46, in run_vm
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     self.wait_for_console_pattern(console_pattern, vm)
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/boot_linux_console.py", line 37, in wait_for_console_pattern
>>    2020-05-27 16:23:49,373 stacktrace       L0045 ERROR|     vm=vm)
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/avocado_qemu/__init__.py", line 131, in wait_for_console_pattern
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     _console_interaction(test, success_message, failure_message, None, vm=vm)
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/acceptance/avocado_qemu/__init__.py", line 83, in _console_interaction
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     msg = console.readline().strip()
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/usr/lib/python3.7/socket.py", line 589, in readinto
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     return self._sock.recv_into(b)
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/all/tests/venv/lib/python3.7/site-packages/avocado/plugins/runner.py", line 89, in sigterm_handler
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR|     raise RuntimeError("Test interrupted by SIGTERM")
>>    2020-05-27 16:23:49,374 stacktrace       L0045 ERROR| RuntimeError: Test interrupted by SIGTERM
>>
>>


-- 
Alex Bennée

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Pavel Dovgalyuk]

On 28.05.2020 16:26, Alex Bennée wrote:
> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
>
>> On 27.05.2020 18:41, Alex Bennée wrote:
>>> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>>>
>>>> This patch adds a test for record/replay an execution of x86_64 machine.
>>>> Execution scenario includes simple kernel boot, which allows testing
>>>> basic hardware interaction in RR mode.
>>>>
>>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>>>> ---
>>>>    0 files changed
>>>>
>>>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>>>> index b8b277ad2f..c7526f1aba 100644
>>>> --- a/tests/acceptance/replay_kernel.py
>>>> +++ b/tests/acceptance/replay_kernel.py
>>>> @@ -55,3 +55,19 @@ class ReplayKernel(LinuxKernelUtils):
>>>>                        True, shift, args)
>>>>            self.run_vm(kernel_path, kernel_command_line, console_pattern,
>>>>                        False, shift, args)
>>>> +
>>>> +    def test_x86_64_pc(self):
>>>> +        """
>>>> +        :avocado: tags=arch:x86_64
>>>> +        :avocado: tags=machine:pc
>>>> +        """
>>>> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>>>> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>>>> +                      '/vmlinuz')
>>>> +        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>>>> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>>>> +
>>>> +        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
>>>> +        console_pattern = 'Kernel command line: %s' % kernel_command_line
>>>> +
>>>> +        self.run_rr(kernel_path, kernel_command_line, console_pattern)
>>> This test fails for me on the replay:
>> Have you applied latest RR patches?
> I have the following on top of the acceptance patches:
>
> a36c23042fe * review/record-replay-acceptance-v2 icount: fix shift=auto for record/replay
> 4ab2164c10b * replay: synchronize on every virtual timer callback
> 66104ce6e4b * replay: notify the main loop when there are no instructions
>
Please also try adding "replay: implement fair mutex"

Re: [PATCH v2 04/11] tests/acceptance: add kernel record/replay test for x86_64

[Alex Bennée]

[-- Attachment #1: Type: text/plain, Size: 1794 bytes --]


Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:

> On 28.05.2020 16:26, Alex Bennée wrote:
>> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
>>
>>> On 27.05.2020 18:41, Alex Bennée wrote:
>>>> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>>>>
>>>>> This patch adds a test for record/replay an execution of x86_64 machine.
>>>>> Execution scenario includes simple kernel boot, which allows testing
>>>>> basic hardware interaction in RR mode.
<snip>
>>>> This test fails for me on the replay:
>>> Have you applied latest RR patches?
>> I have the following on top of the acceptance patches:
>>
>> a36c23042fe * review/record-replay-acceptance-v2 icount: fix shift=auto for record/replay
>> 4ab2164c10b * replay: synchronize on every virtual timer callback
>> 66104ce6e4b * replay: notify the main loop when there are no instructions
>>
> Please also try adding "replay: implement fair mutex"

OK that was enough to get the pc_i440fx to pass:

  JOB ID     : 33f94542f1f2d1e98c8dad820f66ec5ea9bd9fc1
  JOB LOG    : /home/alex/avocado/job-results/job-2020-05-29T09.36-33f9454/job.log
   (1/2) ./tests/acceptance/replay_linux.py:ReplayLinuxX8664.test_pc_i440fx: PASS (502.55 s)
   (2/2) ./tests/acceptance/replay_linux.py:ReplayLinuxX8664.test_pc_q35: INTERRUPTED: Test died without reporting the status.\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n{'name': '2-./tests/acceptance/replay_linux.py:ReplayLinuxX8664.test_pc_q35', 'logdir': '/home/alex/avocado/job-results/job-2020-05-29T09.36-33f9454/test... (1811.23 s)
  RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 1 | CANCEL 0
  JOB TIME   : 2313.96 s

I've attached the log although it's a bit messy because the console
seems to have stripped out any newlines.


[-- Attachment #2: log file of failure --]
[-- Type: application/octet-stream, Size: 585179 bytes --]

[-- Attachment #3: Type: text/plain, Size: 23 bytes --]



-- 
Alex Bennée

[PATCH v2 05/11] tests/acceptance: add record/replay test for aarch64

[Pavel Dovgalyuk]

This patch adds a test for record/replay of the kernel
image boot for aarch64 platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
---
 0 files changed

diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index c7526f1aba..deca69d6cb 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -71,3 +71,21 @@ class ReplayKernel(LinuxKernelUtils):
         console_pattern = 'Kernel command line: %s' % kernel_command_line
 
         self.run_rr(kernel_path, kernel_command_line, console_pattern)
+
+    def test_aarch64_virt(self):
+        """
+        :avocado: tags=arch:aarch64
+        :avocado: tags=machine:virt
+        """
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
+                      '/vmlinuz')
+        kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+                               'console=ttyAMA0')
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+
+        self.run_rr(kernel_path, kernel_command_line, console_pattern,
+            args=('-cpu', 'cortex-a53'))

Re: [PATCH v2 05/11] tests/acceptance: add record/replay test for aarch64

[Philippe Mathieu-Daudé]

On 5/27/20 12:31 PM, Pavel Dovgalyuk wrote:
> This patch adds a test for record/replay of the kernel
> image boot for aarch64 platform.
> 
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> ---
>  0 files changed
> 
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index c7526f1aba..deca69d6cb 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -71,3 +71,21 @@ class ReplayKernel(LinuxKernelUtils):
>          console_pattern = 'Kernel command line: %s' % kernel_command_line
>  
>          self.run_rr(kernel_path, kernel_command_line, console_pattern)
> +
> +    def test_aarch64_virt(self):
> +        """
> +        :avocado: tags=arch:aarch64
> +        :avocado: tags=machine:virt
> +        """
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
> +                      '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
> +                      '/vmlinuz')
> +        kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +
> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> +                               'console=ttyAMA0')
> +        console_pattern = 'Kernel command line: %s' % kernel_command_line
> +
> +        self.run_rr(kernel_path, kernel_command_line, console_pattern,
> +            args=('-cpu', 'cortex-a53'))

I thought this would be ':avocado: tags=cpu:cortex-a53' consumed by the
base class before starting the machine, but not (yet). Still, please add
the tag (for filtering).

Thanks.

[PATCH v2 06/11] tests/acceptance: add record/replay test for arm

[Pavel Dovgalyuk]

This patch adds a test for record/replay of the kernel
image boot for two different arm platforms.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
---
 0 files changed

diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index deca69d6cb..d715ee7ca9 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -89,3 +89,49 @@ class ReplayKernel(LinuxKernelUtils):
 
         self.run_rr(kernel_path, kernel_command_line, console_pattern,
             args=('-cpu', 'cortex-a53'))
+
+    def test_arm_virt(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:virt
+        """
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
+                      '/vmlinuz')
+        kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+                               'console=ttyAMA0')
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+
+        self.run_rr(kernel_path, kernel_command_line, console_pattern)
+
+    def test_arm_cubieboard_initrd(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:cubieboard
+        """
+        deb_url = ('https://apt.armbian.com/pool/main/l/'
+                   'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
+        deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+        kernel_path = self.extract_from_deb(deb_path,
+                                            '/boot/vmlinuz-4.20.7-sunxi')
+        dtb_path = '/usr/lib/linux-image-dev-sunxi/sun4i-a10-cubieboard.dtb'
+        dtb_path = self.extract_from_deb(deb_path, dtb_path)
+        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
+                      '2eb0a73b5d5a28df3170c546ddaaa9757e1e0848/rootfs/'
+                      'arm/rootfs-armv5.cpio.gz')
+        initrd_hash = '2b50f1873e113523967806f4da2afe385462ff9b'
+        initrd_path_gz = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+        initrd_path = os.path.join(self.workdir, 'rootfs.cpio')
+        archive.gzip_uncompress(initrd_path_gz, initrd_path)
+
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+                               'console=ttyS0,115200 '
+                               'usbcore.nousb '
+                               'panic=-1 noreboot')
+        console_pattern = 'Boot successful.'
+        self.run_rr(kernel_path, kernel_command_line, console_pattern,
+            args=('-dtb', dtb_path, '-initrd', initrd_path, '-no-reboot'))

[PATCH v2 07/11] tests/acceptance: add record/replay test for ppc64

[Pavel Dovgalyuk]

This patch adds a test for record/replay of the kernel
image boot for ppc64 platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
---
 0 files changed

diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index d715ee7ca9..7ac046b822 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -135,3 +135,18 @@ class ReplayKernel(LinuxKernelUtils):
         console_pattern = 'Boot successful.'
         self.run_rr(kernel_path, kernel_command_line, console_pattern,
             args=('-dtb', dtb_path, '-initrd', initrd_path, '-no-reboot'))
+
+    def test_ppc64_pseries(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
+        kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=hvc0'
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.run_rr(kernel_path, kernel_command_line, console_pattern)

[PATCH v2 08/11] tests/acceptance: add record/replay test for m68k

[Pavel Dovgalyuk]

This patch adds a test for record/replay of the kernel
image boot for m68k platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
---
 0 files changed

diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index 7ac046b822..0d629d1ee7 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -150,3 +150,21 @@ class ReplayKernel(LinuxKernelUtils):
         kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=hvc0'
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.run_rr(kernel_path, kernel_command_line, console_pattern)
+
+    def test_m68k_q800(self):
+        """
+        :avocado: tags=arch:m68k
+        :avocado: tags=machine:q800
+        """
+        deb_url = ('https://snapshot.debian.org/archive/debian-ports'
+                   '/20191021T083923Z/pool-m68k/main'
+                   '/l/linux/kernel-image-5.3.0-1-m68k-di_5.3.7-1_m68k.udeb')
+        deb_hash = '044954bb9be4160a3ce81f8bc1b5e856b75cccd1'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+        kernel_path = self.extract_from_deb(deb_path,
+                                            '/boot/vmlinux-5.3.0-1-m68k')
+
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+                               'console=ttyS0 vga=off')
+        console_pattern = 'No filesystem could mount root'
+        self.run_rr(kernel_path, kernel_command_line, console_pattern)

[PATCH v2 09/11] tests/acceptance: record/replay tests with advcal images

[Pavel Dovgalyuk]

This patch adds more record/replay tests with kernel images.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>

--

v2:
 - make download path fixed to allow pre-test downloading (suggested by Willian Rampazzo)
---
 0 files changed

diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index 0d629d1ee7..cbc6e9742e 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -168,3 +168,108 @@ class ReplayKernel(LinuxKernelUtils):
                                'console=ttyS0 vga=off')
         console_pattern = 'No filesystem could mount root'
         self.run_rr(kernel_path, kernel_command_line, console_pattern)
+
+    def do_test_advcal_2018(self, file_path, kernel_name, args=None):
+        archive.extract(file_path, self.workdir)
+
+        for entry in os.scandir(self.workdir):
+            if entry.name.startswith('day') and entry.is_dir():
+                kernel_path = entry.path + '/' + kernel_name
+                break
+
+        kernel_command_line = ''
+        console_pattern = 'QEMU advent calendar'
+        self.run_rr(kernel_path, kernel_command_line, console_pattern,
+            args=args)
+
+    def test_arm_vexpressa9(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:vexpress-a9
+        """
+        tar_hash = '32b7677ce8b6f1471fb0059865f451169934245b'
+        tar_url = ('https://www.qemu-advent-calendar.org'
+                   '/2018/download/day16.tar.xz')
+        file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
+        self.do_test_advcal_2018(file_path, 'winter.zImage',
+            ('-dtb', self.workdir + '/day16/vexpress-v2p-ca9.dtb'))
+
+    def test_m68k_mcf5208evb(self):
+        """
+        :avocado: tags=arch:m68k
+        :avocado: tags=machine:mcf5208evb
+        """
+        tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
+        tar_url = ('https://www.qemu-advent-calendar.org'
+                   '/2018/download/day07.tar.xz')
+        file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
+        self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
+
+    def test_microblaze_s3adsp1800(self):
+        """
+        :avocado: tags=arch:microblaze
+        :avocado: tags=machine:petalogix-s3adsp1800
+        """
+        tar_hash = '08bf3e3bfb6b6c7ce1e54ab65d54e189f2caf13f'
+        tar_url = ('https://www.qemu-advent-calendar.org'
+                   '/2018/download/day17.tar.xz')
+        file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
+        self.do_test_advcal_2018(file_path, 'ballerina.bin')
+
+    def test_ppc64_e500(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:ppce500
+        """
+        tar_hash = '6951d86d644b302898da2fd701739c9406527fe1'
+        tar_url = ('https://www.qemu-advent-calendar.org'
+                   '/2018/download/day19.tar.xz')
+        file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
+        self.do_test_advcal_2018(file_path, 'uImage', ('-cpu', 'e5500'))
+
+    def test_ppc_g3beige(self):
+        """
+        :avocado: tags=arch:ppc
+        :avocado: tags=machine:g3beige
+        """
+        tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
+        tar_url = ('https://www.qemu-advent-calendar.org'
+                   '/2018/download/day15.tar.xz')
+        file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
+        self.do_test_advcal_2018(file_path, 'invaders.elf',
+            ('-M', 'graphics=off'))
+
+    def test_ppc_mac99(self):
+        """
+        :avocado: tags=arch:ppc
+        :avocado: tags=machine:mac99
+        """
+        tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
+        tar_url = ('https://www.qemu-advent-calendar.org'
+                   '/2018/download/day15.tar.xz')
+        file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
+        self.do_test_advcal_2018(file_path, 'invaders.elf',
+            ('-M', 'graphics=off'))
+
+    def test_sparc_ss20(self):
+        """
+        :avocado: tags=arch:sparc
+        :avocado: tags=machine:SS-20
+        """
+        tar_hash = 'b18550d5d61c7615d989a06edace051017726a9f'
+        tar_url = ('https://www.qemu-advent-calendar.org'
+                   '/2018/download/day11.tar.xz')
+        file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
+        self.do_test_advcal_2018(file_path, 'zImage.elf')
+
+    def test_xtensa_lx60(self):
+        """
+        :avocado: tags=arch:xtensa
+        :avocado: tags=machine:lx60
+        """
+        tar_hash = '49e88d9933742f0164b60839886c9739cb7a0d34'
+        tar_url = ('https://www.qemu-advent-calendar.org'
+                   '/2018/download/day02.tar.xz')
+        file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
+        self.do_test_advcal_2018(file_path, 'santas-sleigh-ride.elf',
+            ('-cpu', 'dc233c'))

[PATCH v2 10/11] tests/acceptance: refactor boot_linux to allow code reuse

[Pavel Dovgalyuk]

This patch moves image downloading functions to the separate class to allow
reusing them from record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
---
 0 files changed

diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
index 075a386300..3aa57e88b0 100644
--- a/tests/acceptance/boot_linux.py
+++ b/tests/acceptance/boot_linux.py
@@ -26,22 +26,8 @@ KVM_NOT_AVAILABLE = ACCEL_NOT_AVAILABLE_FMT % "KVM"
 TCG_NOT_AVAILABLE = ACCEL_NOT_AVAILABLE_FMT % "TCG"
 
 
-class BootLinux(Test):
-    """
-    Boots a Linux system, checking for a successful initialization
-    """
-
-    timeout = 900
-    chksum = None
-
-    def setUp(self):
-        super(BootLinux, self).setUp()
-        self.vm.add_args('-smp', '2')
-        self.vm.add_args('-m', '1024')
-        self.prepare_boot()
-        self.prepare_cloudinit()
-
-    def prepare_boot(self):
+class BootLinuxBase(Test):
+    def download_boot(self):
         self.log.debug('Looking for and selecting a qemu-img binary to be '
                        'used to create the bootable snapshot image')
         # If qemu-img has been built, use it, otherwise the system wide one
@@ -60,17 +46,17 @@ class BootLinux(Test):
         if image_arch == 'ppc64':
             image_arch = 'ppc64le'
         try:
-            self.boot = vmimage.get(
+            boot = vmimage.get(
                 'fedora', arch=image_arch, version='31',
                 checksum=self.chksum,
                 algorithm='sha256',
                 cache_dir=self.cache_dirs[0],
                 snapshot_dir=self.workdir)
-            self.vm.add_args('-drive', 'file=%s' % self.boot.path)
         except:
             self.cancel('Failed to download/prepare boot image')
+        return boot.path
 
-    def prepare_cloudinit(self):
+    def download_cloudinit(self):
         self.log.info('Preparing cloudinit image')
         try:
             cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
@@ -81,9 +67,32 @@ class BootLinux(Test):
                           # QEMU's hard coded usermode router address
                           phone_home_host='10.0.2.2',
                           phone_home_port=self.phone_home_port)
-            self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
         except Exception:
             self.cancel('Failed to prepared cloudinit image')
+        return cloudinit_iso
+
+class BootLinux(BootLinuxBase):
+    """
+    Boots a Linux system, checking for a successful initialization
+    """
+
+    timeout = 900
+    chksum = None
+
+    def setUp(self):
+        super(BootLinux, self).setUp()
+        self.vm.add_args('-smp', '2')
+        self.vm.add_args('-m', '1024')
+        self.prepare_boot()
+        self.prepare_cloudinit()
+
+    def prepare_boot(self):
+        path = self.download_boot()
+        self.vm.add_args('-drive', 'file=%s' % path)
+
+    def prepare_cloudinit(self):
+        cloudinit_iso = self.download_cloudinit()
+        self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
 
     def launch_and_wait(self):
         self.vm.set_console()

[PATCH v2 11/11] tests/acceptance: Linux boot test for record/replay

[Pavel Dovgalyuk]

This patch adds a test for record/replay, which boots Linux
image from the disk and interacts with the network.
The idea and code of this test is borrowed from boot_linux.py
However, currently record/replay works only for x86_64,
therefore other tests were excluded.

Each test consists of the following phases:
 - downloading the disk image
 - recording the execution
 - replaying the execution

Replay does not validates the output, but waits until QEMU
finishes the execution. This is reasonable, because
QEMU usually hangs when replay goes wrong.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
---
 0 files changed

diff --git a/MAINTAINERS b/MAINTAINERS
index e9a9ce4f66..97f066a9b2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2498,6 +2498,7 @@ F: include/sysemu/replay.h
 F: docs/replay.txt
 F: stubs/replay.c
 F: tests/acceptance/replay_kernel.py
+F: tests/acceptance/replay_linux.py
 
 IOVA Tree
 M: Peter Xu <peterx@redhat.com>
diff --git a/tests/acceptance/replay_linux.py b/tests/acceptance/replay_linux.py
new file mode 100644
index 0000000000..7c5971f156
--- /dev/null
+++ b/tests/acceptance/replay_linux.py
@@ -0,0 +1,97 @@
+# Record/replay test that boots a complete Linux system via a cloud image
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+
+from avocado.utils import cloudinit
+from avocado.utils import network
+from avocado.utils import vmimage
+from avocado.utils import datadrainer
+from avocado.utils.path import find_command
+from boot_linux import BootLinuxBase
+
+class ReplayLinux(BootLinuxBase):
+    """
+    Boots a Linux system, checking for a successful initialization
+    """
+
+    timeout = 1800
+    chksum = None
+    hdd = 'ide-hd'
+    cd = 'ide-cd'
+    bus = 'ide'
+
+    def setUp(self):
+        super(ReplayLinux, self).setUp()
+        self.boot_path = self.download_boot()
+        self.cloudinit_path = self.download_cloudinit()
+
+    def vm_add_disk(self, vm, path, id, device):
+        bus_string = ''
+        if self.bus:
+            bus_string = ',bus=%s.%d' % (self.bus, id,)
+        vm.add_args('-drive', 'file=%s,snapshot,id=disk%s,if=none' % (path, id))
+        vm.add_args('-drive', 'driver=blkreplay,id=disk%s-rr,if=none,image=disk%s' % (id, id))
+        vm.add_args('-device', '%s,drive=disk%s-rr%s' % (device, id, bus_string))
+
+    def launch_and_wait(self, record, args, shift):
+        vm = self.get_vm()
+        vm.add_args('-smp', '1')
+        vm.add_args('-m', '1024')
+        vm.add_args('-object', 'filter-replay,id=replay,netdev=hub0port0')
+        if args:
+            vm.add_args(*args)
+        self.vm_add_disk(vm, self.boot_path, 0, self.hdd)
+        self.vm_add_disk(vm, self.cloudinit_path, 1, self.cd)
+        if record:
+            mode = 'record'
+        else:
+            mode = 'replay'
+        vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
+                    (shift, mode, os.path.join(self.workdir, 'replay.bin')))
+
+        vm.set_console()
+        vm.launch()
+        console_drainer = datadrainer.LineLogger(vm.console_socket.fileno(),
+                                                 logger=self.log.getChild('console'),
+                                                 stop_check=(lambda : not vm.is_running()))
+        console_drainer.start()
+        if record:
+            self.log.info('VM launched, waiting for boot confirmation from guest')
+            cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port), self.name)
+            vm.shutdown()
+        else:
+            self.log.info('VM launched, waiting the recorded execution to be replayed')
+            vm.wait()
+
+    def run_rr(self, args=None, shift=7):
+        self.launch_and_wait(True, args, shift)
+        self.launch_and_wait(False, args, shift)
+
+class ReplayLinuxX8664(ReplayLinux):
+    """
+    :avocado: tags=arch:x86_64
+    """
+
+    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
+
+    def test_pc_i440fx(self):
+        """
+        :avocado: tags=machine:pc
+        :avocado: tags=accel:tcg
+        """
+        self.run_rr(shift=1)
+
+    def test_pc_q35(self):
+        """
+        :avocado: tags=machine:q35
+        :avocado: tags=accel:tcg
+        """
+        self.run_rr(shift=3)

Re: [PATCH v2 11/11] tests/acceptance: Linux boot test for record/replay

[Alex Bennée]

Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:

> This patch adds a test for record/replay, which boots Linux
> image from the disk and interacts with the network.
> The idea and code of this test is borrowed from boot_linux.py
> However, currently record/replay works only for x86_64,
> therefore other tests were excluded.
>
> Each test consists of the following phases:
>  - downloading the disk image
>  - recording the execution
>  - replaying the execution
>
> Replay does not validates the output, but waits until QEMU
> finishes the execution. This is reasonable, because
> QEMU usually hangs when replay goes wrong.

Two things:

 - We need to tag these tests as slow so they aren't run by default
 - 1800s is a long timeout to wait for to know it's a problem

Looking at the log shows my test is still running? Maybe we can check
the output as we go?

>
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> ---
>  0 files changed
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e9a9ce4f66..97f066a9b2 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2498,6 +2498,7 @@ F: include/sysemu/replay.h
>  F: docs/replay.txt
>  F: stubs/replay.c
>  F: tests/acceptance/replay_kernel.py
> +F: tests/acceptance/replay_linux.py
>  
>  IOVA Tree
>  M: Peter Xu <peterx@redhat.com>
> diff --git a/tests/acceptance/replay_linux.py b/tests/acceptance/replay_linux.py
> new file mode 100644
> index 0000000000..7c5971f156
> --- /dev/null
> +++ b/tests/acceptance/replay_linux.py
> @@ -0,0 +1,97 @@
> +# Record/replay test that boots a complete Linux system via a cloud image
> +#
> +# Copyright (c) 2020 ISP RAS
> +#
> +# Author:
> +#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import os
> +
> +from avocado.utils import cloudinit
> +from avocado.utils import network
> +from avocado.utils import vmimage
> +from avocado.utils import datadrainer
> +from avocado.utils.path import find_command
> +from boot_linux import BootLinuxBase
> +
> +class ReplayLinux(BootLinuxBase):
> +    """
> +    Boots a Linux system, checking for a successful initialization
> +    """
> +
> +    timeout = 1800
> +    chksum = None
> +    hdd = 'ide-hd'
> +    cd = 'ide-cd'
> +    bus = 'ide'
> +
> +    def setUp(self):
> +        super(ReplayLinux, self).setUp()
> +        self.boot_path = self.download_boot()
> +        self.cloudinit_path = self.download_cloudinit()
> +
> +    def vm_add_disk(self, vm, path, id, device):
> +        bus_string = ''
> +        if self.bus:
> +            bus_string = ',bus=%s.%d' % (self.bus, id,)
> +        vm.add_args('-drive', 'file=%s,snapshot,id=disk%s,if=none' % (path, id))
> +        vm.add_args('-drive', 'driver=blkreplay,id=disk%s-rr,if=none,image=disk%s' % (id, id))
> +        vm.add_args('-device', '%s,drive=disk%s-rr%s' % (device, id, bus_string))
> +
> +    def launch_and_wait(self, record, args, shift):
> +        vm = self.get_vm()
> +        vm.add_args('-smp', '1')
> +        vm.add_args('-m', '1024')
> +        vm.add_args('-object', 'filter-replay,id=replay,netdev=hub0port0')
> +        if args:
> +            vm.add_args(*args)
> +        self.vm_add_disk(vm, self.boot_path, 0, self.hdd)
> +        self.vm_add_disk(vm, self.cloudinit_path, 1, self.cd)
> +        if record:
> +            mode = 'record'
> +        else:
> +            mode = 'replay'
> +        vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
> +                    (shift, mode, os.path.join(self.workdir, 'replay.bin')))
> +
> +        vm.set_console()
> +        vm.launch()
> +        console_drainer = datadrainer.LineLogger(vm.console_socket.fileno(),
> +                                                 logger=self.log.getChild('console'),
> +                                                 stop_check=(lambda : not vm.is_running()))
> +        console_drainer.start()
> +        if record:
> +            self.log.info('VM launched, waiting for boot confirmation from guest')
> +            cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port), self.name)
> +            vm.shutdown()
> +        else:
> +            self.log.info('VM launched, waiting the recorded execution to be replayed')
> +            vm.wait()
> +
> +    def run_rr(self, args=None, shift=7):
> +        self.launch_and_wait(True, args, shift)
> +        self.launch_and_wait(False, args, shift)
> +
> +class ReplayLinuxX8664(ReplayLinux):
> +    """
> +    :avocado: tags=arch:x86_64
> +    """
> +
> +    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
> +
> +    def test_pc_i440fx(self):
> +        """
> +        :avocado: tags=machine:pc
> +        :avocado: tags=accel:tcg
> +        """
> +        self.run_rr(shift=1)
> +
> +    def test_pc_q35(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=accel:tcg
> +        """
> +        self.run_rr(shift=3)


-- 
Alex Bennée

Re: [PATCH v2 11/11] tests/acceptance: Linux boot test for record/replay

[Pavel Dovgalyuk]

On 27.05.2020 19:44, Alex Bennée wrote:
> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>
>> This patch adds a test for record/replay, which boots Linux
>> image from the disk and interacts with the network.
>> The idea and code of this test is borrowed from boot_linux.py
>> However, currently record/replay works only for x86_64,
>> therefore other tests were excluded.
>>
>> Each test consists of the following phases:
>>   - downloading the disk image
>>   - recording the execution
>>   - replaying the execution
>>
>> Replay does not validates the output, but waits until QEMU
>> finishes the execution. This is reasonable, because
>> QEMU usually hangs when replay goes wrong.
> Two things:
>
>   - We need to tag these tests as slow so they aren't run by default

Which flag is responsible for this?

>   - 1800s is a long timeout to wait for to know it's a problem

Right, I just doubled boot_linux timeout. I think, that it could be reduced.

>
> Looking at the log shows my test is still running? Maybe we can check
> the output as we go?

How this could look like?

>
>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> ---
>>   0 files changed
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index e9a9ce4f66..97f066a9b2 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -2498,6 +2498,7 @@ F: include/sysemu/replay.h
>>   F: docs/replay.txt
>>   F: stubs/replay.c
>>   F: tests/acceptance/replay_kernel.py
>> +F: tests/acceptance/replay_linux.py
>>   
>>   IOVA Tree
>>   M: Peter Xu <peterx@redhat.com>
>> diff --git a/tests/acceptance/replay_linux.py b/tests/acceptance/replay_linux.py
>> new file mode 100644
>> index 0000000000..7c5971f156
>> --- /dev/null
>> +++ b/tests/acceptance/replay_linux.py
>> @@ -0,0 +1,97 @@
>> +# Record/replay test that boots a complete Linux system via a cloud image
>> +#
>> +# Copyright (c) 2020 ISP RAS
>> +#
>> +# Author:
>> +#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later.  See the COPYING file in the top-level directory.
>> +
>> +import os
>> +
>> +from avocado.utils import cloudinit
>> +from avocado.utils import network
>> +from avocado.utils import vmimage
>> +from avocado.utils import datadrainer
>> +from avocado.utils.path import find_command
>> +from boot_linux import BootLinuxBase
>> +
>> +class ReplayLinux(BootLinuxBase):
>> +    """
>> +    Boots a Linux system, checking for a successful initialization
>> +    """
>> +
>> +    timeout = 1800
>> +    chksum = None
>> +    hdd = 'ide-hd'
>> +    cd = 'ide-cd'
>> +    bus = 'ide'
>> +
>> +    def setUp(self):
>> +        super(ReplayLinux, self).setUp()
>> +        self.boot_path = self.download_boot()
>> +        self.cloudinit_path = self.download_cloudinit()
>> +
>> +    def vm_add_disk(self, vm, path, id, device):
>> +        bus_string = ''
>> +        if self.bus:
>> +            bus_string = ',bus=%s.%d' % (self.bus, id,)
>> +        vm.add_args('-drive', 'file=%s,snapshot,id=disk%s,if=none' % (path, id))
>> +        vm.add_args('-drive', 'driver=blkreplay,id=disk%s-rr,if=none,image=disk%s' % (id, id))
>> +        vm.add_args('-device', '%s,drive=disk%s-rr%s' % (device, id, bus_string))
>> +
>> +    def launch_and_wait(self, record, args, shift):
>> +        vm = self.get_vm()
>> +        vm.add_args('-smp', '1')
>> +        vm.add_args('-m', '1024')
>> +        vm.add_args('-object', 'filter-replay,id=replay,netdev=hub0port0')
>> +        if args:
>> +            vm.add_args(*args)
>> +        self.vm_add_disk(vm, self.boot_path, 0, self.hdd)
>> +        self.vm_add_disk(vm, self.cloudinit_path, 1, self.cd)
>> +        if record:
>> +            mode = 'record'
>> +        else:
>> +            mode = 'replay'
>> +        vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
>> +                    (shift, mode, os.path.join(self.workdir, 'replay.bin')))
>> +
>> +        vm.set_console()
>> +        vm.launch()
>> +        console_drainer = datadrainer.LineLogger(vm.console_socket.fileno(),
>> +                                                 logger=self.log.getChild('console'),
>> +                                                 stop_check=(lambda : not vm.is_running()))
>> +        console_drainer.start()
>> +        if record:
>> +            self.log.info('VM launched, waiting for boot confirmation from guest')
>> +            cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port), self.name)
>> +            vm.shutdown()
>> +        else:
>> +            self.log.info('VM launched, waiting the recorded execution to be replayed')
>> +            vm.wait()
>> +
>> +    def run_rr(self, args=None, shift=7):
>> +        self.launch_and_wait(True, args, shift)
>> +        self.launch_and_wait(False, args, shift)
>> +
>> +class ReplayLinuxX8664(ReplayLinux):
>> +    """
>> +    :avocado: tags=arch:x86_64
>> +    """
>> +
>> +    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
>> +
>> +    def test_pc_i440fx(self):
>> +        """
>> +        :avocado: tags=machine:pc
>> +        :avocado: tags=accel:tcg
>> +        """
>> +        self.run_rr(shift=1)
>> +
>> +    def test_pc_q35(self):
>> +        """
>> +        :avocado: tags=machine:q35
>> +        :avocado: tags=accel:tcg
>> +        """
>> +        self.run_rr(shift=3)
>

Re: [PATCH v2 00/11] Record/replay acceptance tests

[no-reply]

Patchew URL: https://patchew.org/QEMU/159057543840.16818.14393433996899521784.stgit@pasha-ThinkPad-X280/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 159057543840.16818.14393433996899521784.stgit@pasha-ThinkPad-X280
Subject: [PATCH v2 00/11] Record/replay acceptance tests
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]      patchew/cover.1590429901.git.berto@igalia.com -> patchew/cover.1590429901.git.berto@igalia.com
Switched to a new branch 'test'
ceed53c tests/acceptance: Linux boot test for record/replay
14bf4bc tests/acceptance: refactor boot_linux to allow code reuse
e192fb0 tests/acceptance: record/replay tests with advcal images
0d48aac tests/acceptance: add record/replay test for m68k
1660395 tests/acceptance: add record/replay test for ppc64
881e1cf tests/acceptance: add record/replay test for arm
91c3c1f tests/acceptance: add record/replay test for aarch64
2a236a0 tests/acceptance: add kernel record/replay test for x86_64
bd68b78 tests/acceptance: add base class record/replay kernel tests
69a18b9 tests/acceptance: refactor boot_linux_console test to allow code reuse
f7db49e tests/acceptance: allow console interaction with specific VMs

=== OUTPUT BEGIN ===
1/11 Checking commit f7db49ef403b (tests/acceptance: allow console interaction with specific VMs)
2/11 Checking commit 69a18b92426f (tests/acceptance: refactor boot_linux_console test to allow code reuse)
3/11 Checking commit bd68b7890ae4 (tests/acceptance: add base class record/replay kernel tests)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#38: 
new file mode 100644

total: 0 errors, 1 warnings, 64 lines checked

Patch 3/11 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
4/11 Checking commit 2a236a011a1b (tests/acceptance: add kernel record/replay test for x86_64)
5/11 Checking commit 91c3c1f38968 (tests/acceptance: add record/replay test for aarch64)
6/11 Checking commit 881e1cfd51e9 (tests/acceptance: add record/replay test for arm)
7/11 Checking commit 1660395b5645 (tests/acceptance: add record/replay test for ppc64)
8/11 Checking commit 0d48aac0dbfa (tests/acceptance: add record/replay test for m68k)
9/11 Checking commit e192fb06abc8 (tests/acceptance: record/replay tests with advcal images)
10/11 Checking commit 14bf4bc65656 (tests/acceptance: refactor boot_linux to allow code reuse)
11/11 Checking commit ceed53c0bc8d (tests/acceptance: Linux boot test for record/replay)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#38: 
new file mode 100644

ERROR: line over 90 characters
#83: FILE: tests/acceptance/replay_linux.py:41:
+        vm.add_args('-drive', 'driver=blkreplay,id=disk%s-rr,if=none,image=disk%s' % (id, id))

WARNING: line over 80 characters
#84: FILE: tests/acceptance/replay_linux.py:42:
+        vm.add_args('-device', '%s,drive=disk%s-rr%s' % (device, id, bus_string))

WARNING: line over 80 characters
#105: FILE: tests/acceptance/replay_linux.py:63:
+                                                 logger=self.log.getChild('console'),

ERROR: line over 90 characters
#106: FILE: tests/acceptance/replay_linux.py:64:
+                                                 stop_check=(lambda : not vm.is_running()))

WARNING: line over 80 characters
#109: FILE: tests/acceptance/replay_linux.py:67:
+            self.log.info('VM launched, waiting for boot confirmation from guest')

WARNING: line over 80 characters
#110: FILE: tests/acceptance/replay_linux.py:68:
+            cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port), self.name)

WARNING: line over 80 characters
#113: FILE: tests/acceptance/replay_linux.py:71:
+            self.log.info('VM launched, waiting the recorded execution to be replayed')

total: 2 errors, 6 warnings, 104 lines checked

Patch 11/11 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/159057543840.16818.14393433996899521784.stgit@pasha-ThinkPad-X280/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [PATCH v2 00/11] Record/replay acceptance tests

[Philippe Mathieu-Daudé]

On 5/27/20 3:41 PM, no-reply@patchew.org wrote:
> Patchew URL: https://patchew.org/QEMU/159057543840.16818.14393433996899521784.stgit@pasha-ThinkPad-X280/
> 
> 
> 
> Hi,
> 
> This series seems to have some coding style problems. See output below for
> more information:
> 
[...]>
> ERROR: line over 90 characters
> #83: FILE: tests/acceptance/replay_linux.py:41:
> +        vm.add_args('-drive', 'driver=blkreplay,id=disk%s-rr,if=none,image=disk%s' % (id, id))
> 
> WARNING: line over 80 characters
> #84: FILE: tests/acceptance/replay_linux.py:42:
> +        vm.add_args('-device', '%s,drive=disk%s-rr%s' % (device, id, bus_string))

Please try to respect the QEMU 80 chars limit (it eases review by blind
developers) and align methods & arguments to QEMU style.

Otherwise I'm glad to see this series.

So far:
Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>

I plan to add R-b on v3 once you addressed Alex's comments.

Good job!