[PATCH 0/2] docs/devel/qgraph: add troubleshooting information

[PATCH 0/2] docs/devel/qgraph: add troubleshooting information

[Stefan Hajnoczi]

I recently needed to troubleshoot a case where qos-test terminated immediately
with no output. In other words, qos-test decided that no tests are runnable.

After lots of head scratching and some help from Emanuele it turned out that
the machine types weren't being detected as expected.

These patches add documentation about how to troubleshoot similar cases in the
future.

Stefan Hajnoczi (2):
  libqos/qgraph: fix "UNAVAILBLE" typo
  docs/devel/qgraph: add troubleshooting information

 docs/devel/qgraph.rst       | 58 +++++++++++++++++++++++++++++++++++++
 tests/qtest/libqos/qgraph.c |  2 +-
 2 files changed, 59 insertions(+), 1 deletion(-)

-- 
2.30.2

[PATCH 1/2] libqos/qgraph: fix "UNAVAILBLE" typo

[Stefan Hajnoczi]

Cc: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 tests/qtest/libqos/qgraph.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qtest/libqos/qgraph.c b/tests/qtest/libqos/qgraph.c
index b3b1a31f81..d1dc491930 100644
--- a/tests/qtest/libqos/qgraph.c
+++ b/tests/qtest/libqos/qgraph.c
@@ -844,7 +844,7 @@ void qos_dump_graph(void)
         }
         qos_printf_literal("type=%d cmd_line='%s' [%s]\n",
                            node->type, node->command_line,
-                           node->available ? "available" : "UNAVAILBLE"
+                           node->available ? "available" : "UNAVAILABLE"
         );
     }
     g_list_free(keys);
-- 
2.30.2

[PATCH 2/2] docs/devel/qgraph: add troubleshooting information

[Stefan Hajnoczi]

It can be tricky to troubleshoot qos-test when a test won't execute. Add
an explanation of how to trace qgraph node connectivity and find which
node has the problem.

Cc: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/devel/qgraph.rst | 58 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/docs/devel/qgraph.rst b/docs/devel/qgraph.rst
index a9aff167ad..4635efb2c2 100644
--- a/docs/devel/qgraph.rst
+++ b/docs/devel/qgraph.rst
@@ -92,6 +92,64 @@ The basic framework steps are the following:
 Depending on the QEMU binary used, only some drivers/machines will be
 available and only test that are reached by them will be executed.
 
+Troubleshooting unavailable tests
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+If there is no path from an available machine to a test then that test will
+unavailable and cannot execute. This can happen if a test or driver did not set
+up its qgraph node correctly. It can also happen if the necessary machine type
+or device is missing from the QEMU binary because it was compiled out or
+otherwise.
+
+It is possible to troubleshoot unavailable tests by running::
+
+  $ QTEST_QEMU_BINARY=build/qemu-system-x86_64 build/tests/qtest/qos-test --verbose
+  # ALL QGRAPH EDGES: {
+  #   src='virtio-net'
+  #      |-> dest='virtio-net-tests/vhost-user/multiqueue' type=2 (node=0x559142109e30)
+  #      |-> dest='virtio-net-tests/vhost-user/migrate' type=2 (node=0x559142109d00)
+  #   src='virtio-net-pci'
+  #      |-> dest='virtio-net' type=1 (node=0x55914210d740)
+  #   src='pci-bus'
+  #      |-> dest='virtio-net-pci' type=2 (node=0x55914210d880)
+  #   src='pci-bus-pc'
+  #      |-> dest='pci-bus' type=1 (node=0x559142103f40)
+  #   src='i440FX-pcihost'
+  #      |-> dest='pci-bus-pc' type=0 (node=0x55914210ac70)
+  #   src='x86_64/pc'
+  #      |-> dest='i440FX-pcihost' type=0 (node=0x5591421117f0)
+  #   src=''
+  #      |-> dest='x86_64/pc' type=0 (node=0x559142111600)
+  #      |-> dest='arm/raspi2' type=0 (node=0x559142110740)
+  ...
+  # }
+  # ALL QGRAPH NODES: {
+  #   name='virtio-net-tests/announce-self' type=3 cmd_line='(null)' [available]
+  #   name='arm/raspi2' type=0 cmd_line='-M raspi2 ' [UNAVAILABLE]
+  ...
+  # }
+
+The ``virtio-net-tests/announce-self`` test is listed as "available" in the
+"ALL QGRAPH NODES" output. This means the test will execute. We can follow the
+qgraph path in the "ALL QGRAPH EDGES" output as follows: '' -> 'x86_64/pc' ->
+'i440FX-pcihost' -> 'pci-bus-pc' -> 'pci-bus' -> 'virtio-net-pci' ->
+'virtio-net'. The root of the qgraph is '' and the depth first search begins
+there.
+
+The ``arm/raspi`` machine node is listed as "UNAVAILABLE". Although it is
+reachable from the root via '' -> 'arm/raspi2' the node is unavailable because
+the QEMU binary did not list it when queried by the framework. This is expected
+because we used the ``qemu-system-x86_64`` binary which does not support ARM
+machine types.
+
+If a test is unexpectedly listed as "UNAVAILABLE", first check that the "ALL
+QGRAPH EDGES" output reports edge connectivity from the root ('') to the test.
+If there is no connectivity then the qgraph nodes were not set up correctly and
+the driver or test code is incorrect. If there is connectivity, check the
+availability of each node in the path in the "ALL QGRAPH NODES" output. The
+first unavailable node in the path is the reason why the test is unavailable.
+Typically this is because the QEMU binary lacks support for the necessary
+machine type or device.
+
 Creating a new driver and its interface
 """""""""""""""""""""""""""""""""""""""""
 
-- 
2.30.2

Re: [PATCH 1/2] libqos/qgraph: fix "UNAVAILBLE" typo

[Philippe Mathieu-Daudé]

On 4/9/21 9:01 PM, Stefan Hajnoczi wrote:
> Cc: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  tests/qtest/libqos/qgraph.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH 2/2] docs/devel/qgraph: add troubleshooting information

[Thomas Huth]

On 09/04/2021 21.01, Stefan Hajnoczi wrote:
> It can be tricky to troubleshoot qos-test when a test won't execute. Add
> an explanation of how to trace qgraph node connectivity and find which
> node has the problem.
> 
> Cc: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>   docs/devel/qgraph.rst | 58 +++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 58 insertions(+)
> 
> diff --git a/docs/devel/qgraph.rst b/docs/devel/qgraph.rst
> index a9aff167ad..4635efb2c2 100644
> --- a/docs/devel/qgraph.rst
> +++ b/docs/devel/qgraph.rst
> @@ -92,6 +92,64 @@ The basic framework steps are the following:
>   Depending on the QEMU binary used, only some drivers/machines will be
>   available and only test that are reached by them will be executed.
>   
> +Troubleshooting unavailable tests
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +If there is no path from an available machine to a test then that test will
> +unavailable and cannot execute. This can happen if a test or driver did not set

"will be unavailable" ? or "will be marked as unavailable" ?

Apart from that, patch looks fine to me.

  Thomas

Re: [PATCH 2/2] docs/devel/qgraph: add troubleshooting information

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1376 bytes --]

On Sat, Apr 10, 2021 at 06:41:03AM +0200, Thomas Huth wrote:
> On 09/04/2021 21.01, Stefan Hajnoczi wrote:
> > It can be tricky to troubleshoot qos-test when a test won't execute. Add
> > an explanation of how to trace qgraph node connectivity and find which
> > node has the problem.
> > 
> > Cc: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> > ---
> >   docs/devel/qgraph.rst | 58 +++++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 58 insertions(+)
> > 
> > diff --git a/docs/devel/qgraph.rst b/docs/devel/qgraph.rst
> > index a9aff167ad..4635efb2c2 100644
> > --- a/docs/devel/qgraph.rst
> > +++ b/docs/devel/qgraph.rst
> > @@ -92,6 +92,64 @@ The basic framework steps are the following:
> >   Depending on the QEMU binary used, only some drivers/machines will be
> >   available and only test that are reached by them will be executed.
> > +Troubleshooting unavailable tests
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +If there is no path from an available machine to a test then that test will
> > +unavailable and cannot execute. This can happen if a test or driver did not set
> 
> "will be unavailable" ? or "will be marked as unavailable" ?
> 
> Apart from that, patch looks fine to me.

Thanks, will fix.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]