All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
@ 2013-03-22  8:25 Kazuya Saito
  2013-03-22  8:26 ` [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint Kazuya Saito
                   ` (5 more replies)
  0 siblings, 6 replies; 21+ messages in thread
From: Kazuya Saito @ 2013-03-22  8:25 UTC (permalink / raw)
  To: qemu-devel

This series adds tracepoints for helping us clarify the cause of
troubles. Virtualization on Linux is composed of some components such
as qemu, kvm, libvirt, and so on. So it is very important to clarify
firstly and swiftly the cause of troubles is on what component of
them. Although qemu has useful information of this because it stands
among kvm, libvirt and guest, it doesn't output the information by
trace or log system.
These patches add tracepoints which lead to reduce the time of the
clarification. We'd like to add the tracepoints as the first set
because, based on our experience, we've found out they must be useful
for an investigation in the future. Without those tracepoints,
we had a really hard time investigating a problem since the problem's
reproducibility was quite low and there was no clue in the dump of
qemu.

Kazuya Saito (5):
  vl: add runstate_set tracepoint
  kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints
  kvm-all: add kvm_run_exit tracepoint
  qdev: add qdev_{create,free} tracepoints
  qdev-monitor: add device_add tracepoint

 hw/qdev.c      |    3 +++
 kvm-all.c      |    5 +++++
 qdev-monitor.c |    2 ++
 trace-events   |   14 ++++++++++++++
 vl.c           |    2 +-
 5 files changed, 25 insertions(+), 1 deletions(-)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint
  2013-03-22  8:25 [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Kazuya Saito
@ 2013-03-22  8:26 ` Kazuya Saito
  2013-03-22 11:13   ` Paolo Bonzini
  2013-03-28 13:21   ` Stefan Hajnoczi
  2013-03-22  8:27 ` [Qemu-devel] [PATCH 2/5] kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints Kazuya Saito
                   ` (4 subsequent siblings)
  5 siblings, 2 replies; 21+ messages in thread
From: Kazuya Saito @ 2013-03-22  8:26 UTC (permalink / raw)
  To: qemu-devel

This patch enables us to know RunState transition. It will be userful
for investigation when the trouble occured in special event such like
live migration, shutdown, suspend, and so on.

Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
---
 trace-events |    1 +
 vl.c         |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/trace-events b/trace-events
index d6a847d..1e352a7 100644
--- a/trace-events
+++ b/trace-events
@@ -474,6 +474,7 @@ scsi_request_sense(int target, int lun, int tag) "target %d lun %d tag %d"
 # vl.c
 vm_state_notify(int running, int reason) "running %d reason %d"
 load_file(const char *name, const char *path) "name %s location %s"
+runstate_set(int new_state) "new state %d"

 # block/qcow2.c
 qcow2_writev_start_req(void *co, int64_t sector, int nb_sectors) "co %p sector %" PRIx64 " nb_sectors %d"
diff --git a/vl.c b/vl.c
index a621aec..4bd3671 100644
--- a/vl.c
+++ b/vl.c
@@ -638,7 +638,7 @@ void runstate_set(RunState new_state)
                 RunState_lookup[new_state]);
         abort();
     }
-
+    trace_runstate_set(new_state);
     current_run_state = new_state;
 }

-- 
1.7.1

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 2/5] kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints
  2013-03-22  8:25 [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Kazuya Saito
  2013-03-22  8:26 ` [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint Kazuya Saito
@ 2013-03-22  8:27 ` Kazuya Saito
  2013-03-22 11:12   ` Paolo Bonzini
  2013-03-22  8:28 ` [Qemu-devel] [PATCH 3/5] kvm-all: add kvm_run_exit tracepoint Kazuya Saito
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 21+ messages in thread
From: Kazuya Saito @ 2013-03-22  8:27 UTC (permalink / raw)
  To: qemu-devel

This patch adds tracepoints at ioctl to kvm. Tracing these ioctl is
useful for clarification whether the cause of troubles is qemu or kvm.

Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
---
 kvm-all.c    |    4 ++++
 trace-events |    5 +++++
 2 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index 4decfdc..da15623 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -33,6 +33,7 @@
 #include "exec/memory.h"
 #include "exec/address-spaces.h"
 #include "qemu/event_notifier.h"
+#include "trace.h"

 /* This check must be after config-host.h is included */
 #ifdef CONFIG_EVENTFD
@@ -1636,6 +1637,7 @@ int kvm_ioctl(KVMState *s, int type, ...)
     arg = va_arg(ap, void *);
     va_end(ap);

+    trace_kvm_ioctl(type);
     ret = ioctl(s->fd, type, arg);
     if (ret == -1) {
         ret = -errno;
@@ -1653,6 +1655,7 @@ int kvm_vm_ioctl(KVMState *s, int type, ...)
     arg = va_arg(ap, void *);
     va_end(ap);

+    trace_kvm_vm_ioctl(type);
     ret = ioctl(s->vmfd, type, arg);
     if (ret == -1) {
         ret = -errno;
@@ -1670,6 +1673,7 @@ int kvm_vcpu_ioctl(CPUState *cpu, int type, ...)
     arg = va_arg(ap, void *);
     va_end(ap);

+    trace_kvm_vcpu_ioctl(type);
     ret = ioctl(cpu->kvm_fd, type, arg);
     if (ret == -1) {
         ret = -errno;
diff --git a/trace-events b/trace-events
index 1e352a7..1af7060 100644
--- a/trace-events
+++ b/trace-events
@@ -1096,3 +1096,8 @@ virtio_ccw_new_device(int cssid, int ssid, int schid, int devno, const char *dev

 # migration.c
 migrate_set_state(int new_state) "new state %d"
+
+# kvm-all.c
+kvm_ioctl(int type) "type %d"
+kvm_vm_ioctl(int type) "type %d"
+kvm_vcpu_ioctl(int type) "type %d"
-- 
1.7.1

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 3/5] kvm-all: add kvm_run_exit tracepoint
  2013-03-22  8:25 [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Kazuya Saito
  2013-03-22  8:26 ` [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint Kazuya Saito
  2013-03-22  8:27 ` [Qemu-devel] [PATCH 2/5] kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints Kazuya Saito
@ 2013-03-22  8:28 ` Kazuya Saito
  2013-03-22 11:12   ` Paolo Bonzini
  2013-03-22  8:29 ` [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create,free} tracepoints Kazuya Saito
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 21+ messages in thread
From: Kazuya Saito @ 2013-03-22  8:28 UTC (permalink / raw)
  To: qemu-devel

This patch enable us to know exit reason of KVM_RUN. It will help us
know where the trouble is caused.

Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
---
 kvm-all.c    |    1 +
 trace-events |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index da15623..29fca13 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -1576,6 +1576,7 @@ int kvm_cpu_exec(CPUArchState *env)
             abort();
         }

+        trace_kvm_run_exit(run->exit_reason);
         switch (run->exit_reason) {
         case KVM_EXIT_IO:
             DPRINTF("handle_io\n");
diff --git a/trace-events b/trace-events
index 1af7060..c691ce4 100644
--- a/trace-events
+++ b/trace-events
@@ -1101,3 +1101,4 @@ migrate_set_state(int new_state) "new state %d"
 kvm_ioctl(int type) "type %d"
 kvm_vm_ioctl(int type) "type %d"
 kvm_vcpu_ioctl(int type) "type %d"
+kvm_run_exit(uint32_t reason) "reason %d"
-- 
1.7.1

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create,free} tracepoints
  2013-03-22  8:25 [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Kazuya Saito
                   ` (2 preceding siblings ...)
  2013-03-22  8:28 ` [Qemu-devel] [PATCH 3/5] kvm-all: add kvm_run_exit tracepoint Kazuya Saito
@ 2013-03-22  8:29 ` Kazuya Saito
  2013-03-27 11:03   ` [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create, free} tracepoints Andreas Färber
  2013-03-22  8:29 ` [Qemu-devel] [PATCH 5/5] qdev-monitor: add device_add tracepoint Kazuya Saito
  2013-03-22 11:13 ` [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Paolo Bonzini
  5 siblings, 1 reply; 21+ messages in thread
From: Kazuya Saito @ 2013-03-22  8:29 UTC (permalink / raw)
  To: qemu-devel

This patch adds tracepoints at creating and removing virtual
devices. It is useful for investigation of trouble related to virtual
devices.

Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
---
 hw/qdev.c    |    3 +++
 trace-events |    4 ++++
 2 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/hw/qdev.c b/hw/qdev.c
index 0b20280..0fda23e 100644
--- a/hw/qdev.c
+++ b/hw/qdev.c
@@ -30,6 +30,7 @@
 #include "qapi/error.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/visitor.h"
+#include "trace.h"

 int qdev_hotplug = 0;
 static bool qdev_hot_added = false;
@@ -124,6 +125,7 @@ DeviceState *qdev_create(BusState *bus, const char *name)
         }
     }

+    trace_qdev_create(dev, dev->parent_bus);
     return dev;
 }

@@ -268,6 +270,7 @@ void qdev_init_nofail(DeviceState *dev)
 /* Unlink device from bus and free the structure.  */
 void qdev_free(DeviceState *dev)
 {
+    trace_qdev_free(dev, dev->parent_bus);
     object_unparent(OBJECT(dev));
 }

diff --git a/trace-events b/trace-events
index c691ce4..235b978 100644
--- a/trace-events
+++ b/trace-events
@@ -1102,3 +1102,7 @@ kvm_ioctl(int type) "type %d"
 kvm_vm_ioctl(int type) "type %d"
 kvm_vcpu_ioctl(int type) "type %d"
 kvm_run_exit(uint32_t reason) "reason %d"
+
+# qdev.c
+qdev_create(void *dev, void *bus) "dev %p, bus %p"
+qdev_free(void *dev, void *bus) "dev %p, bus %p"
-- 
1.7.1

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 5/5] qdev-monitor: add device_add tracepoint
  2013-03-22  8:25 [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Kazuya Saito
                   ` (3 preceding siblings ...)
  2013-03-22  8:29 ` [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create,free} tracepoints Kazuya Saito
@ 2013-03-22  8:29 ` Kazuya Saito
  2013-03-22 11:13 ` [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Paolo Bonzini
  5 siblings, 0 replies; 21+ messages in thread
From: Kazuya Saito @ 2013-03-22  8:29 UTC (permalink / raw)
  To: qemu-devel

This patch adds tracepoints at device-add command. It enables us to
clarify whether the trouble is caused by hot-plug through commands of
virsh (libvirt).

Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
---
 qdev-monitor.c |    2 ++
 trace-events   |    3 +++
 2 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/qdev-monitor.c b/qdev-monitor.c
index 9a78ccf..80881f5 100644
--- a/qdev-monitor.c
+++ b/qdev-monitor.c
@@ -23,6 +23,7 @@
 #include "qmp-commands.h"
 #include "sysemu/arch_init.h"
 #include "qemu/config-file.h"
+#include "trace.h"

 /*
  * Aliases were a bad idea from the start.  Let's keep them
@@ -473,6 +474,7 @@ DeviceState *qdev_device_add(QemuOpts *opts)
     /* create device, set properties */
     qdev = DEVICE(object_new(driver));
     qdev_set_parent_bus(qdev, bus);
+    trace_device_add(qdev, bus);

     id = qemu_opts_id(opts);
     if (id) {
diff --git a/trace-events b/trace-events
index 235b978..f23b820 100644
--- a/trace-events
+++ b/trace-events
@@ -1106,3 +1106,6 @@ kvm_run_exit(uint32_t reason) "reason %d"
 # qdev.c
 qdev_create(void *dev, void *bus) "dev %p, bus %p"
 qdev_free(void *dev, void *bus) "dev %p, bus %p"
+
+# qdev-monitor.c
+device_add(void *dev, void *bus) "dev %p, bus %p"
-- 
1.7.1

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 2/5] kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints
  2013-03-22  8:27 ` [Qemu-devel] [PATCH 2/5] kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints Kazuya Saito
@ 2013-03-22 11:12   ` Paolo Bonzini
  0 siblings, 0 replies; 21+ messages in thread
From: Paolo Bonzini @ 2013-03-22 11:12 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: qemu-devel

Il 22/03/2013 09:27, Kazuya Saito ha scritto:
> This patch adds tracepoints at ioctl to kvm. Tracing these ioctl is
> useful for clarification whether the cause of troubles is qemu or kvm.
> 
> Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
> ---
>  kvm-all.c    |    4 ++++
>  trace-events |    5 +++++
>  2 files changed, 9 insertions(+), 0 deletions(-)
> 
> diff --git a/kvm-all.c b/kvm-all.c
> index 4decfdc..da15623 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -33,6 +33,7 @@
>  #include "exec/memory.h"
>  #include "exec/address-spaces.h"
>  #include "qemu/event_notifier.h"
> +#include "trace.h"
> 
>  /* This check must be after config-host.h is included */
>  #ifdef CONFIG_EVENTFD
> @@ -1636,6 +1637,7 @@ int kvm_ioctl(KVMState *s, int type, ...)
>      arg = va_arg(ap, void *);
>      va_end(ap);
> 
> +    trace_kvm_ioctl(type);
>      ret = ioctl(s->fd, type, arg);
>      if (ret == -1) {
>          ret = -errno;
> @@ -1653,6 +1655,7 @@ int kvm_vm_ioctl(KVMState *s, int type, ...)
>      arg = va_arg(ap, void *);
>      va_end(ap);
> 
> +    trace_kvm_vm_ioctl(type);
>      ret = ioctl(s->vmfd, type, arg);
>      if (ret == -1) {
>          ret = -errno;
> @@ -1670,6 +1673,7 @@ int kvm_vcpu_ioctl(CPUState *cpu, int type, ...)
>      arg = va_arg(ap, void *);
>      va_end(ap);
> 
> +    trace_kvm_vcpu_ioctl(type);
>      ret = ioctl(cpu->kvm_fd, type, arg);
>      if (ret == -1) {
>          ret = -errno;
> diff --git a/trace-events b/trace-events
> index 1e352a7..1af7060 100644
> --- a/trace-events
> +++ b/trace-events
> @@ -1096,3 +1096,8 @@ virtio_ccw_new_device(int cssid, int ssid, int schid, int devno, const char *dev
> 
>  # migration.c
>  migrate_set_state(int new_state) "new state %d"
> +
> +# kvm-all.c
> +kvm_ioctl(int type) "type %d"
> +kvm_vm_ioctl(int type) "type %d"
> +kvm_vcpu_ioctl(int type) "type %d"
> 

Please add arg as well and, for kvm_vcpu_ioctl, the CPU number too.

Thanks!

Paolo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 3/5] kvm-all: add kvm_run_exit tracepoint
  2013-03-22  8:28 ` [Qemu-devel] [PATCH 3/5] kvm-all: add kvm_run_exit tracepoint Kazuya Saito
@ 2013-03-22 11:12   ` Paolo Bonzini
  0 siblings, 0 replies; 21+ messages in thread
From: Paolo Bonzini @ 2013-03-22 11:12 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: qemu-devel

Il 22/03/2013 09:28, Kazuya Saito ha scritto:
> This patch enable us to know exit reason of KVM_RUN. It will help us
> know where the trouble is caused.
> 
> Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
> ---
>  kvm-all.c    |    1 +
>  trace-events |    1 +
>  2 files changed, 2 insertions(+), 0 deletions(-)
> 
> diff --git a/kvm-all.c b/kvm-all.c
> index da15623..29fca13 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -1576,6 +1576,7 @@ int kvm_cpu_exec(CPUArchState *env)
>              abort();
>          }
> 
> +        trace_kvm_run_exit(run->exit_reason);
>          switch (run->exit_reason) {
>          case KVM_EXIT_IO:
>              DPRINTF("handle_io\n");
> diff --git a/trace-events b/trace-events
> index 1af7060..c691ce4 100644
> --- a/trace-events
> +++ b/trace-events
> @@ -1101,3 +1101,4 @@ migrate_set_state(int new_state) "new state %d"
>  kvm_ioctl(int type) "type %d"
>  kvm_vm_ioctl(int type) "type %d"
>  kvm_vcpu_ioctl(int type) "type %d"
> +kvm_run_exit(uint32_t reason) "reason %d"
> 

Please add the CPU number here, too.

Paolo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint
  2013-03-22  8:26 ` [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint Kazuya Saito
@ 2013-03-22 11:13   ` Paolo Bonzini
  2013-03-28 13:21   ` Stefan Hajnoczi
  1 sibling, 0 replies; 21+ messages in thread
From: Paolo Bonzini @ 2013-03-22 11:13 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: qemu-devel

Il 22/03/2013 09:26, Kazuya Saito ha scritto:
> This patch enables us to know RunState transition. It will be userful
> for investigation when the trouble occured in special event such like
> live migration, shutdown, suspend, and so on.
> 
> Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
> ---
>  trace-events |    1 +
>  vl.c         |    2 +-
>  2 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/trace-events b/trace-events
> index d6a847d..1e352a7 100644
> --- a/trace-events
> +++ b/trace-events
> @@ -474,6 +474,7 @@ scsi_request_sense(int target, int lun, int tag) "target %d lun %d tag %d"
>  # vl.c
>  vm_state_notify(int running, int reason) "running %d reason %d"
>  load_file(const char *name, const char *path) "name %s location %s"
> +runstate_set(int new_state) "new state %d"
> 
>  # block/qcow2.c
>  qcow2_writev_start_req(void *co, int64_t sector, int nb_sectors) "co %p sector %" PRIx64 " nb_sectors %d"
> diff --git a/vl.c b/vl.c
> index a621aec..4bd3671 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -638,7 +638,7 @@ void runstate_set(RunState new_state)
>                  RunState_lookup[new_state]);
>          abort();
>      }
> -
> +    trace_runstate_set(new_state);
>      current_run_state = new_state;
>  }
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-22  8:25 [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Kazuya Saito
                   ` (4 preceding siblings ...)
  2013-03-22  8:29 ` [Qemu-devel] [PATCH 5/5] qdev-monitor: add device_add tracepoint Kazuya Saito
@ 2013-03-22 11:13 ` Paolo Bonzini
  2013-03-26  7:13   ` Kazuya Saito
  5 siblings, 1 reply; 21+ messages in thread
From: Paolo Bonzini @ 2013-03-22 11:13 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: qemu-devel

Il 22/03/2013 09:25, Kazuya Saito ha scritto:
> This series adds tracepoints for helping us clarify the cause of
> troubles. Virtualization on Linux is composed of some components such
> as qemu, kvm, libvirt, and so on. So it is very important to clarify
> firstly and swiftly the cause of troubles is on what component of
> them. Although qemu has useful information of this because it stands
> among kvm, libvirt and guest, it doesn't output the information by
> trace or log system.
> These patches add tracepoints which lead to reduce the time of the
> clarification. We'd like to add the tracepoints as the first set
> because, based on our experience, we've found out they must be useful
> for an investigation in the future. Without those tracepoints,
> we had a really hard time investigating a problem since the problem's
> reproducibility was quite low and there was no clue in the dump of
> qemu.
> 
> Kazuya Saito (5):
>   vl: add runstate_set tracepoint
>   kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints
>   kvm-all: add kvm_run_exit tracepoint
>   qdev: add qdev_{create,free} tracepoints
>   qdev-monitor: add device_add tracepoint

I'm not sure 4-5 are that useful, but the first 3 patches are definitely
good stuff.

Paolo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-22 11:13 ` [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Paolo Bonzini
@ 2013-03-26  7:13   ` Kazuya Saito
  2013-03-26  7:15     ` Paolo Bonzini
  0 siblings, 1 reply; 21+ messages in thread
From: Kazuya Saito @ 2013-03-26  7:13 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

(2013/03/22 20:13), Paolo Bonzini wrote:
> Il 22/03/2013 09:25, Kazuya Saito ha scritto:
>> This series adds tracepoints for helping us clarify the cause of
>> troubles. Virtualization on Linux is composed of some components such
>> as qemu, kvm, libvirt, and so on. So it is very important to clarify
>> firstly and swiftly the cause of troubles is on what component of
>> them. Although qemu has useful information of this because it stands
>> among kvm, libvirt and guest, it doesn't output the information by
>> trace or log system.
>> These patches add tracepoints which lead to reduce the time of the
>> clarification. We'd like to add the tracepoints as the first set
>> because, based on our experience, we've found out they must be useful
>> for an investigation in the future. Without those tracepoints,
>> we had a really hard time investigating a problem since the problem's
>> reproducibility was quite low and there was no clue in the dump of
>> qemu.
>>
>> Kazuya Saito (5):
>>   vl: add runstate_set tracepoint
>>   kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints
>>   kvm-all: add kvm_run_exit tracepoint
>>   qdev: add qdev_{create,free} tracepoints
>>   qdev-monitor: add device_add tracepoint
> 
> I'm not sure 4-5 are that useful, but the first 3 patches are definitely
> good stuff.

Thanks. I'll modify the patch you pointed out about CPU number and
re-post it.
I'd like to add tracepoints to the virtual device creation/deletion
part.  I had an issue that when a Windows guest OS booted up, it didn't
have a virtual NIC device which it should have had.  It took us time to
figure out which the guest OS or the QEMU had the issue.  If I have
had the tracepoints in the virtual device creation/deletion should have
eased the situation much better.

Kazuya

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-26  7:13   ` Kazuya Saito
@ 2013-03-26  7:15     ` Paolo Bonzini
  2013-03-27  6:21       ` Kazuya Saito
  0 siblings, 1 reply; 21+ messages in thread
From: Paolo Bonzini @ 2013-03-26  7:15 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: qemu-devel

Il 26/03/2013 08:13, Kazuya Saito ha scritto:
>> > 
>> > I'm not sure 4-5 are that useful, but the first 3 patches are definitely
>> > good stuff.
> Thanks. I'll modify the patch you pointed out about CPU number and
> re-post it.
> I'd like to add tracepoints to the virtual device creation/deletion
> part.  I had an issue that when a Windows guest OS booted up, it didn't
> have a virtual NIC device which it should have had.  It took us time to
> figure out which the guest OS or the QEMU had the issue.  If I have
> had the tracepoints in the virtual device creation/deletion should have
> eased the situation much better.

Wouldn't you get the same information from the command line?

Paolo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-26  7:15     ` Paolo Bonzini
@ 2013-03-27  6:21       ` Kazuya Saito
  2013-03-27  8:45         ` Paolo Bonzini
  0 siblings, 1 reply; 21+ messages in thread
From: Kazuya Saito @ 2013-03-27  6:21 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

(2013/03/26 16:15), Paolo Bonzini wrote:
> Il 26/03/2013 08:13, Kazuya Saito ha scritto:
>>>>
>>>> I'm not sure 4-5 are that useful, but the first 3 patches are definitely
>>>> good stuff.
>> Thanks. I'll modify the patch you pointed out about CPU number and
>> re-post it.
>> I'd like to add tracepoints to the virtual device creation/deletion
>> part.  I had an issue that when a Windows guest OS booted up, it didn't
>> have a virtual NIC device which it should have had.  It took us time to
>> figure out which the guest OS or the QEMU had the issue.  If I have
>> had the tracepoints in the virtual device creation/deletion should have
>> eased the situation much better.
> 
> Wouldn't you get the same information from the command line?

I think the information you said is different from what I meant. The
information I wanted to know is whether QEMU creates/deletes a device
successfully or not. We cannot get it from the command line.
I was sure I specified a NIC device for the guest, but the Windows guest
didn't have it when it booted.  There were two possibilities.  One was
QEMU failed to create the NIC device, and the other was QEMU created it
successfully and the Windows guest failed to detect it.  Since QEMU
output limited message at the moment, it was difficult to prove that
it was not QEMU's issue.  I took a coredump of the QEMU and proved the
QEMU must have a valid structures for the NIC device.  I believe the
tracepoints could have allowed me to figure out where the issue existed
a lot faster and easier.  (And it turned out it was the Windows' issue.)

Kazuya

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-27  6:21       ` Kazuya Saito
@ 2013-03-27  8:45         ` Paolo Bonzini
  2013-03-28  5:53           ` Kazuya Saito
  0 siblings, 1 reply; 21+ messages in thread
From: Paolo Bonzini @ 2013-03-27  8:45 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: qemu-devel


> > Wouldn't you get the same information from the command line?
> 
> I think the information you said is different from what I meant. The
> information I wanted to know is whether QEMU creates/deletes a device
> successfully or not.

Failing to create a device will always exit QEMU.  If you cannot assume
that, you're really in debugging territory and your tools should
be the "info" monitor commands (info qtree, info pci) or gdb...


> We cannot get it from the command line.
> I was sure I specified a NIC device for the guest, but the Windows
> guest didn't have it when it booted.  There were two possibilities.  One
> was QEMU failed to create the NIC device, and the other was QEMU created
> it successfully and the Windows guest failed to detect it.  Since QEMU
> output limited message at the moment, it was difficult to prove that
> it was not QEMU's issue.  I took a coredump of the QEMU and proved
> the QEMU must have a valid structures for the NIC device.  I believe the
> tracepoints could have allowed me to figure out where the issue
> existed a lot faster and easier. 

Had you tried "info qtree" or "info pci"?

Paolo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create, free} tracepoints
  2013-03-22  8:29 ` [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create,free} tracepoints Kazuya Saito
@ 2013-03-27 11:03   ` Andreas Färber
  2013-03-28  5:56     ` Kazuya Saito
  0 siblings, 1 reply; 21+ messages in thread
From: Andreas Färber @ 2013-03-27 11:03 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: Paolo Bonzini, qemu-devel

Am 22.03.2013 09:29, schrieb Kazuya Saito:
> This patch adds tracepoints at creating and removing virtual
> devices. It is useful for investigation of trouble related to virtual
> devices.
> 
> Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>

I would prefer not to do this. I had previously posted a patch to remove
qdev_free() in favor of using the QOM function object_unparent()
directly, which adding stuff to qdev_free() would interfere with. And
you should rather add a tracepoint to object_new() or better to
object_initialize() than into the legacy qdev_create() - which doesn't
cover qdev_try_create() btw. Either way, adding new tracepoints with the
legacy "qdev" in the name is ugly.

Regards,
Andreas

P.S. Your patches arrived in HTML format, please check your workflow.

> ---
>  hw/qdev.c    |    3 +++
>  trace-events |    4 ++++
>  2 files changed, 7 insertions(+), 0 deletions(-)
> 
> diff --git a/hw/qdev.c b/hw/qdev.c
> index 0b20280..0fda23e 100644
> --- a/hw/qdev.c
> +++ b/hw/qdev.c
> @@ -30,6 +30,7 @@
>  #include "qapi/error.h"
>  #include "qapi/qmp/qerror.h"
>  #include "qapi/visitor.h"
> +#include "trace.h"
> 
>  int qdev_hotplug = 0;
>  static bool qdev_hot_added = false;
> @@ -124,6 +125,7 @@ DeviceState *qdev_create(BusState *bus, const char *name)
>          }
>      }
> 
> +    trace_qdev_create(dev, dev->parent_bus);
>      return dev;
>  }
> 
> @@ -268,6 +270,7 @@ void qdev_init_nofail(DeviceState *dev)
>  /* Unlink device from bus and free the structure.  */
>  void qdev_free(DeviceState *dev)
>  {
> +    trace_qdev_free(dev, dev->parent_bus);
>      object_unparent(OBJECT(dev));
>  }
> 
> diff --git a/trace-events b/trace-events
> index c691ce4..235b978 100644
> --- a/trace-events
> +++ b/trace-events
> @@ -1102,3 +1102,7 @@ kvm_ioctl(int type) "type %d"
>  kvm_vm_ioctl(int type) "type %d"
>  kvm_vcpu_ioctl(int type) "type %d"
>  kvm_run_exit(uint32_t reason) "reason %d"
> +
> +# qdev.c
> +qdev_create(void *dev, void *bus) "dev %p, bus %p"
> +qdev_free(void *dev, void *bus) "dev %p, bus %p"
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-27  8:45         ` Paolo Bonzini
@ 2013-03-28  5:53           ` Kazuya Saito
  2013-03-28  8:52             ` Paolo Bonzini
  0 siblings, 1 reply; 21+ messages in thread
From: Kazuya Saito @ 2013-03-28  5:53 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

(2013/03/27 17:45), Paolo Bonzini wrote:>
>>> Wouldn't you get the same information from the command line?
>>
>> I think the information you said is different from what I meant. The
>> information I wanted to know is whether QEMU creates/deletes a device
>> successfully or not.
>
> Failing to create a device will always exit QEMU.  If you cannot assume
> that, you're really in debugging territory and your tools should
> be the "info" monitor commands (info qtree, info pci) or gdb...

That's a good point.

>> We cannot get it from the command line.
>> I was sure I specified a NIC device for the guest, but the Windows
>> guest didn't have it when it booted.  There were two possibilities.  One
>> was QEMU failed to create the NIC device, and the other was QEMU created
>> it successfully and the Windows guest failed to detect it.  Since QEMU
>> output limited message at the moment, it was difficult to prove that
>> it was not QEMU's issue.  I took a coredump of the QEMU and proved
>> the QEMU must have a valid structures for the NIC device.  I believe the
>> tracepoints could have allowed me to figure out where the issue
>> existed a lot faster and easier.
>
> Had you tried "info qtree" or "info pci"?

I didn't try these commands. I tried them just now and it seems to solve
the problem. So I think no need to add the tracepoints I posted.
Thank you for your kindness.

Kazuya

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create, free} tracepoints
  2013-03-27 11:03   ` [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create, free} tracepoints Andreas Färber
@ 2013-03-28  5:56     ` Kazuya Saito
  0 siblings, 0 replies; 21+ messages in thread
From: Kazuya Saito @ 2013-03-28  5:56 UTC (permalink / raw)
  To: Andreas Färber; +Cc: Paolo Bonzini, qemu-devel

(2013/03/27 20:03), Andreas Färber wrote:> Am 22.03.2013 09:29, schrieb Kazuya Saito:
>> This patch adds tracepoints at creating and removing virtual
>> devices. It is useful for investigation of trouble related to virtual
>> devices.
>>
>> Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
>
> I would prefer not to do this. I had previously posted a patch to remove
> qdev_free() in favor of using the QOM function object_unparent()
> directly, which adding stuff to qdev_free() would interfere with. And
> you should rather add a tracepoint to object_new() or better to
> object_initialize() than into the legacy qdev_create() - which doesn't
> cover qdev_try_create() btw. Either way, adding new tracepoints with the
> legacy "qdev" in the name is ugly.

Just as I replied to Paolo, I won't add these tracepoints. Thank you for
your good information.

> Regards,
> Andreas
>
> P.S. Your patches arrived in HTML format, please check your workflow.

I checked and modified the setting. Did this mail arrive in plain-text?

Kazuya

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-28  5:53           ` Kazuya Saito
@ 2013-03-28  8:52             ` Paolo Bonzini
  2013-03-28 12:49               ` Anthony Liguori
  0 siblings, 1 reply; 21+ messages in thread
From: Paolo Bonzini @ 2013-03-28  8:52 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: Anthony Liguori, qemu-devel, Stefan Hajnoczi

Il 28/03/2013 06:53, Kazuya Saito ha scritto:
> I didn't try these commands. I tried them just now and it seems to solve
> the problem. So I think no need to add the tracepoints I posted.
> Thank you for your kindness.

Thanks to you!

Anthony or Stefan, can you pick up patch 1
(http://article.gmane.org/gmane.comp.emulators.qemu/201873/raw)?

Kazuya, please resubmit patches 2 and 3.  Label them [PATCH uq/master]
so that the KVM maintainers see them, and CC kvm@vger.kernel.org too.

Paolo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-28  8:52             ` Paolo Bonzini
@ 2013-03-28 12:49               ` Anthony Liguori
  2013-03-28 13:21                 ` Stefan Hajnoczi
  0 siblings, 1 reply; 21+ messages in thread
From: Anthony Liguori @ 2013-03-28 12:49 UTC (permalink / raw)
  To: Paolo Bonzini, Kazuya Saito; +Cc: qemu-devel, Stefan Hajnoczi

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il 28/03/2013 06:53, Kazuya Saito ha scritto:
>> I didn't try these commands. I tried them just now and it seems to solve
>> the problem. So I think no need to add the tracepoints I posted.
>> Thank you for your kindness.
>
> Thanks to you!
>
> Anthony or Stefan, can you pick up patch 1
> (http://article.gmane.org/gmane.comp.emulators.qemu/201873/raw)?

Stefan, please take this through the tracing tree.

Regards,

Anthony Liguori

>
> Kazuya, please resubmit patches 2 and 3.  Label them [PATCH uq/master]
> so that the KVM maintainers see them, and CC kvm@vger.kernel.org too.
>
> Paolo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint
  2013-03-22  8:26 ` [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint Kazuya Saito
  2013-03-22 11:13   ` Paolo Bonzini
@ 2013-03-28 13:21   ` Stefan Hajnoczi
  1 sibling, 0 replies; 21+ messages in thread
From: Stefan Hajnoczi @ 2013-03-28 13:21 UTC (permalink / raw)
  To: Kazuya Saito; +Cc: qemu-devel

On Fri, Mar 22, 2013 at 05:26:59PM +0900, Kazuya Saito wrote:
> This patch enables us to know RunState transition. It will be userful
> for investigation when the trouble occured in special event such like
> live migration, shutdown, suspend, and so on.
> 
> Signed-off-by: Kazuya Saito <saito.kazuya@jp.fujitsu.com>
> ---
>  trace-events |    1 +
>  vl.c         |    2 +-
>  2 files changed, 2 insertions(+), 1 deletions(-)

Thanks, applied to my tracing tree:
https://github.com/stefanha/qemu/commits/tracing

Stefan

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles
  2013-03-28 12:49               ` Anthony Liguori
@ 2013-03-28 13:21                 ` Stefan Hajnoczi
  0 siblings, 0 replies; 21+ messages in thread
From: Stefan Hajnoczi @ 2013-03-28 13:21 UTC (permalink / raw)
  To: Anthony Liguori; +Cc: Paolo Bonzini, Kazuya Saito, Stefan Hajnoczi, qemu-devel

On Thu, Mar 28, 2013 at 07:49:54AM -0500, Anthony Liguori wrote:
> Paolo Bonzini <pbonzini@redhat.com> writes:
> 
> > Il 28/03/2013 06:53, Kazuya Saito ha scritto:
> >> I didn't try these commands. I tried them just now and it seems to solve
> >> the problem. So I think no need to add the tracepoints I posted.
> >> Thank you for your kindness.
> >
> > Thanks to you!
> >
> > Anthony or Stefan, can you pick up patch 1
> > (http://article.gmane.org/gmane.comp.emulators.qemu/201873/raw)?
> 
> Stefan, please take this through the tracing tree.

Sure.

Stefan

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2013-03-28 13:21 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-03-22  8:25 [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Kazuya Saito
2013-03-22  8:26 ` [Qemu-devel] [PATCH 1/5] vl: add runstate_set tracepoint Kazuya Saito
2013-03-22 11:13   ` Paolo Bonzini
2013-03-28 13:21   ` Stefan Hajnoczi
2013-03-22  8:27 ` [Qemu-devel] [PATCH 2/5] kvm-all: add kvm_ioctl, kvm_vm_ioctl, kvm_vcpu_ioctl tracepoints Kazuya Saito
2013-03-22 11:12   ` Paolo Bonzini
2013-03-22  8:28 ` [Qemu-devel] [PATCH 3/5] kvm-all: add kvm_run_exit tracepoint Kazuya Saito
2013-03-22 11:12   ` Paolo Bonzini
2013-03-22  8:29 ` [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create,free} tracepoints Kazuya Saito
2013-03-27 11:03   ` [Qemu-devel] [PATCH 4/5] qdev: add qdev_{create, free} tracepoints Andreas Färber
2013-03-28  5:56     ` Kazuya Saito
2013-03-22  8:29 ` [Qemu-devel] [PATCH 5/5] qdev-monitor: add device_add tracepoint Kazuya Saito
2013-03-22 11:13 ` [Qemu-devel] [PATCH 0/5] Add some tracepoints for clarification of the cause of troubles Paolo Bonzini
2013-03-26  7:13   ` Kazuya Saito
2013-03-26  7:15     ` Paolo Bonzini
2013-03-27  6:21       ` Kazuya Saito
2013-03-27  8:45         ` Paolo Bonzini
2013-03-28  5:53           ` Kazuya Saito
2013-03-28  8:52             ` Paolo Bonzini
2013-03-28 12:49               ` Anthony Liguori
2013-03-28 13:21                 ` Stefan Hajnoczi

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.