[Qemu-devel] [PATCH 0/6] Fix multifd with big number of channels

[Qemu-devel] [PATCH 0/6] Fix multifd with big number of channels

[Juan Quintela]

Hi

When we have much more channels than cpus, we end having failures when
writting to sockets. This series:
- add some traces
- fix some of the trouble with serialization of creating the
  threads/channels in proper order.
- Ask for help with the last patch.  See documentation there.

Please, review.

Juan Quintela (6):
  migration: Add traces for multifd terminate threads
  migration: Make global sem_sync semaphore by channel
  migration: Make sure that all multifd channels have been created
  migration: Make multifd threads wait until all have been created
  migration: add some multifd traces
  RFH: We lost "connect" events

 migration/ram.c        | 60 +++++++++++++++++++++++++++++++++++-------
 migration/trace-events |  8 ++++++
 2 files changed, 59 insertions(+), 9 deletions(-)

-- 
2.21.0

[Qemu-devel] [PATCH 1/6] migration: Add traces for multifd terminate threads

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c        | 4 ++++
 migration/trace-events | 2 ++
 2 files changed, 6 insertions(+)

diff --git a/migration/ram.c b/migration/ram.c
index 889148dd84..ca11d43e30 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -996,6 +996,8 @@ static void multifd_send_terminate_threads(Error *err)
 {
     int i;
 
+    trace_multifd_send_terminate_threads(err != NULL);
+
     if (err) {
         MigrationState *s = migrate_get_current();
         migrate_set_error(s, err);
@@ -1254,6 +1256,8 @@ static void multifd_recv_terminate_threads(Error *err)
 {
     int i;
 
+    trace_multifd_recv_terminate_threads(err != NULL);
+
     if (err) {
         MigrationState *s = migrate_get_current();
         migrate_set_error(s, err);
diff --git a/migration/trace-events b/migration/trace-events
index d8e54c367a..886ce70ca0 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -85,12 +85,14 @@ multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uin
 multifd_recv_sync_main(long packet_num) "packet num %ld"
 multifd_recv_sync_main_signal(uint8_t id) "channel %d"
 multifd_recv_sync_main_wait(uint8_t id) "channel %d"
+multifd_recv_terminate_threads(bool error) "error %d"
 multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64
 multifd_recv_thread_start(uint8_t id) "%d"
 multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
 multifd_send_sync_main(long packet_num) "packet num %ld"
 multifd_send_sync_main_signal(uint8_t id) "channel %d"
 multifd_send_sync_main_wait(uint8_t id) "channel %d"
+multifd_send_terminate_threads(bool error) "error %d"
 multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %"  PRIu64
 multifd_send_thread_start(uint8_t id) "%d"
 ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
-- 
2.21.0

Re: [Qemu-devel] [PATCH 1/6] migration: Add traces for multifd terminate threads

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

and queued

> ---
>  migration/ram.c        | 4 ++++
>  migration/trace-events | 2 ++
>  2 files changed, 6 insertions(+)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index 889148dd84..ca11d43e30 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -996,6 +996,8 @@ static void multifd_send_terminate_threads(Error *err)
>  {
>      int i;
>  
> +    trace_multifd_send_terminate_threads(err != NULL);
> +
>      if (err) {
>          MigrationState *s = migrate_get_current();
>          migrate_set_error(s, err);
> @@ -1254,6 +1256,8 @@ static void multifd_recv_terminate_threads(Error *err)
>  {
>      int i;
>  
> +    trace_multifd_recv_terminate_threads(err != NULL);
> +
>      if (err) {
>          MigrationState *s = migrate_get_current();
>          migrate_set_error(s, err);
> diff --git a/migration/trace-events b/migration/trace-events
> index d8e54c367a..886ce70ca0 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -85,12 +85,14 @@ multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uin
>  multifd_recv_sync_main(long packet_num) "packet num %ld"
>  multifd_recv_sync_main_signal(uint8_t id) "channel %d"
>  multifd_recv_sync_main_wait(uint8_t id) "channel %d"
> +multifd_recv_terminate_threads(bool error) "error %d"
>  multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64
>  multifd_recv_thread_start(uint8_t id) "%d"
>  multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
>  multifd_send_sync_main(long packet_num) "packet num %ld"
>  multifd_send_sync_main_signal(uint8_t id) "channel %d"
>  multifd_send_sync_main_wait(uint8_t id) "channel %d"
> +multifd_send_terminate_threads(bool error) "error %d"
>  multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %"  PRIu64
>  multifd_send_thread_start(uint8_t id) "%d"
>  ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
> -- 
> 2.21.0
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/6] migration: Add traces for multifd terminate threads

[Philippe Mathieu-Daudé]

On 8/14/19 4:02 AM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  migration/ram.c        | 4 ++++
>  migration/trace-events | 2 ++
>  2 files changed, 6 insertions(+)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index 889148dd84..ca11d43e30 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -996,6 +996,8 @@ static void multifd_send_terminate_threads(Error *err)
>  {
>      int i;
>  
> +    trace_multifd_send_terminate_threads(err != NULL);
> +
>      if (err) {
>          MigrationState *s = migrate_get_current();
>          migrate_set_error(s, err);
> @@ -1254,6 +1256,8 @@ static void multifd_recv_terminate_threads(Error *err)
>  {
>      int i;
>  
> +    trace_multifd_recv_terminate_threads(err != NULL);
> +
>      if (err) {
>          MigrationState *s = migrate_get_current();
>          migrate_set_error(s, err);
> diff --git a/migration/trace-events b/migration/trace-events
> index d8e54c367a..886ce70ca0 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -85,12 +85,14 @@ multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uin
>  multifd_recv_sync_main(long packet_num) "packet num %ld"
>  multifd_recv_sync_main_signal(uint8_t id) "channel %d"
>  multifd_recv_sync_main_wait(uint8_t id) "channel %d"
> +multifd_recv_terminate_threads(bool error) "error %d"
>  multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64
>  multifd_recv_thread_start(uint8_t id) "%d"
>  multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
>  multifd_send_sync_main(long packet_num) "packet num %ld"
>  multifd_send_sync_main_signal(uint8_t id) "channel %d"
>  multifd_send_sync_main_wait(uint8_t id) "channel %d"
> +multifd_send_terminate_threads(bool error) "error %d"
>  multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %"  PRIu64
>  multifd_send_thread_start(uint8_t id) "%d"
>  ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

[Qemu-devel] [PATCH 2/6] migration: Make global sem_sync semaphore by channel

[Juan Quintela]

This makes easy to debug things because when you want for all threads
to arrive at that semaphore, you know which one your are waiting for.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index ca11d43e30..4bdd201a4e 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -661,6 +661,8 @@ typedef struct {
     uint64_t num_packets;
     /* pages sent through this channel */
     uint64_t num_pages;
+    /* syncs main thread and channels */
+    QemuSemaphore sem_sync;
 }  MultiFDSendParams;
 
 typedef struct {
@@ -896,8 +898,6 @@ struct {
     MultiFDSendParams *params;
     /* array of pages to sent */
     MultiFDPages_t *pages;
-    /* syncs main thread and channels */
-    QemuSemaphore sem_sync;
     /* global number of generated multifd packets */
     uint64_t packet_num;
     /* send channels ready */
@@ -1038,6 +1038,7 @@ void multifd_save_cleanup(void)
         p->c = NULL;
         qemu_mutex_destroy(&p->mutex);
         qemu_sem_destroy(&p->sem);
+        qemu_sem_destroy(&p->sem_sync);
         g_free(p->name);
         p->name = NULL;
         multifd_pages_clear(p->pages);
@@ -1047,7 +1048,6 @@ void multifd_save_cleanup(void)
         p->packet = NULL;
     }
     qemu_sem_destroy(&multifd_send_state->channels_ready);
-    qemu_sem_destroy(&multifd_send_state->sem_sync);
     g_free(multifd_send_state->params);
     multifd_send_state->params = NULL;
     multifd_pages_clear(multifd_send_state->pages);
@@ -1092,7 +1092,7 @@ static void multifd_send_sync_main(void)
         MultiFDSendParams *p = &multifd_send_state->params[i];
 
         trace_multifd_send_sync_main_wait(p->id);
-        qemu_sem_wait(&multifd_send_state->sem_sync);
+        qemu_sem_wait(&p->sem_sync);
     }
     trace_multifd_send_sync_main(multifd_send_state->packet_num);
 }
@@ -1152,7 +1152,7 @@ static void *multifd_send_thread(void *opaque)
             qemu_mutex_unlock(&p->mutex);
 
             if (flags & MULTIFD_FLAG_SYNC) {
-                qemu_sem_post(&multifd_send_state->sem_sync);
+                qemu_sem_post(&p->sem_sync);
             }
             qemu_sem_post(&multifd_send_state->channels_ready);
         } else if (p->quit) {
@@ -1175,7 +1175,7 @@ out:
      */
     if (ret != 0) {
         if (flags & MULTIFD_FLAG_SYNC) {
-            qemu_sem_post(&multifd_send_state->sem_sync);
+            qemu_sem_post(&p->sem_sync);
         }
         qemu_sem_post(&multifd_send_state->channels_ready);
     }
@@ -1221,7 +1221,6 @@ int multifd_save_setup(void)
     multifd_send_state = g_malloc0(sizeof(*multifd_send_state));
     multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
     multifd_send_state->pages = multifd_pages_init(page_count);
-    qemu_sem_init(&multifd_send_state->sem_sync, 0);
     qemu_sem_init(&multifd_send_state->channels_ready, 0);
 
     for (i = 0; i < thread_count; i++) {
@@ -1229,6 +1228,7 @@ int multifd_save_setup(void)
 
         qemu_mutex_init(&p->mutex);
         qemu_sem_init(&p->sem, 0);
+        qemu_sem_init(&p->sem_sync, 0);
         p->quit = false;
         p->pending_job = 0;
         p->id = i;
-- 
2.21.0

Re: [Qemu-devel] [PATCH 2/6] migration: Make global sem_sync semaphore by channel

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> This makes easy to debug things because when you want for all threads
> to arrive at that semaphore, you know which one your are waiting for.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

and queued.

> ---
>  migration/ram.c | 14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index ca11d43e30..4bdd201a4e 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -661,6 +661,8 @@ typedef struct {
>      uint64_t num_packets;
>      /* pages sent through this channel */
>      uint64_t num_pages;
> +    /* syncs main thread and channels */
> +    QemuSemaphore sem_sync;
>  }  MultiFDSendParams;
>  
>  typedef struct {
> @@ -896,8 +898,6 @@ struct {
>      MultiFDSendParams *params;
>      /* array of pages to sent */
>      MultiFDPages_t *pages;
> -    /* syncs main thread and channels */
> -    QemuSemaphore sem_sync;
>      /* global number of generated multifd packets */
>      uint64_t packet_num;
>      /* send channels ready */
> @@ -1038,6 +1038,7 @@ void multifd_save_cleanup(void)
>          p->c = NULL;
>          qemu_mutex_destroy(&p->mutex);
>          qemu_sem_destroy(&p->sem);
> +        qemu_sem_destroy(&p->sem_sync);
>          g_free(p->name);
>          p->name = NULL;
>          multifd_pages_clear(p->pages);
> @@ -1047,7 +1048,6 @@ void multifd_save_cleanup(void)
>          p->packet = NULL;
>      }
>      qemu_sem_destroy(&multifd_send_state->channels_ready);
> -    qemu_sem_destroy(&multifd_send_state->sem_sync);
>      g_free(multifd_send_state->params);
>      multifd_send_state->params = NULL;
>      multifd_pages_clear(multifd_send_state->pages);
> @@ -1092,7 +1092,7 @@ static void multifd_send_sync_main(void)
>          MultiFDSendParams *p = &multifd_send_state->params[i];
>  
>          trace_multifd_send_sync_main_wait(p->id);
> -        qemu_sem_wait(&multifd_send_state->sem_sync);
> +        qemu_sem_wait(&p->sem_sync);
>      }
>      trace_multifd_send_sync_main(multifd_send_state->packet_num);
>  }
> @@ -1152,7 +1152,7 @@ static void *multifd_send_thread(void *opaque)
>              qemu_mutex_unlock(&p->mutex);
>  
>              if (flags & MULTIFD_FLAG_SYNC) {
> -                qemu_sem_post(&multifd_send_state->sem_sync);
> +                qemu_sem_post(&p->sem_sync);
>              }
>              qemu_sem_post(&multifd_send_state->channels_ready);
>          } else if (p->quit) {
> @@ -1175,7 +1175,7 @@ out:
>       */
>      if (ret != 0) {
>          if (flags & MULTIFD_FLAG_SYNC) {
> -            qemu_sem_post(&multifd_send_state->sem_sync);
> +            qemu_sem_post(&p->sem_sync);
>          }
>          qemu_sem_post(&multifd_send_state->channels_ready);
>      }
> @@ -1221,7 +1221,6 @@ int multifd_save_setup(void)
>      multifd_send_state = g_malloc0(sizeof(*multifd_send_state));
>      multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
>      multifd_send_state->pages = multifd_pages_init(page_count);
> -    qemu_sem_init(&multifd_send_state->sem_sync, 0);
>      qemu_sem_init(&multifd_send_state->channels_ready, 0);
>  
>      for (i = 0; i < thread_count; i++) {
> @@ -1229,6 +1228,7 @@ int multifd_save_setup(void)
>  
>          qemu_mutex_init(&p->mutex);
>          qemu_sem_init(&p->sem, 0);
> +        qemu_sem_init(&p->sem_sync, 0);
>          p->quit = false;
>          p->pending_job = 0;
>          p->id = i;
> -- 
> 2.21.0
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

[Qemu-devel] [PATCH 3/6] migration: Make sure that all multifd channels have been created

[Juan Quintela]

If we start the migration before all have been created, we have to
handle the case that one channel still don't exist.  This way it is
easier.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c        | 14 ++++++++++++++
 migration/trace-events |  1 +
 2 files changed, 15 insertions(+)

diff --git a/migration/ram.c b/migration/ram.c
index 4bdd201a4e..4a6ae677a9 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -663,6 +663,8 @@ typedef struct {
     uint64_t num_pages;
     /* syncs main thread and channels */
     QemuSemaphore sem_sync;
+    /* thread has started and setup is done */
+    QemuSemaphore started;
 }  MultiFDSendParams;
 
 typedef struct {
@@ -1039,6 +1041,7 @@ void multifd_save_cleanup(void)
         qemu_mutex_destroy(&p->mutex);
         qemu_sem_destroy(&p->sem);
         qemu_sem_destroy(&p->sem_sync);
+        qemu_sem_destroy(&p->started);
         g_free(p->name);
         p->name = NULL;
         multifd_pages_clear(p->pages);
@@ -1113,6 +1116,8 @@ static void *multifd_send_thread(void *opaque)
     /* initial packet */
     p->num_packets = 1;
 
+    qemu_sem_post(&p->started);
+
     while (true) {
         qemu_sem_wait(&p->sem);
         qemu_mutex_lock(&p->mutex);
@@ -1229,6 +1234,7 @@ int multifd_save_setup(void)
         qemu_mutex_init(&p->mutex);
         qemu_sem_init(&p->sem, 0);
         qemu_sem_init(&p->sem_sync, 0);
+        qemu_sem_init(&p->started, 0);
         p->quit = false;
         p->pending_job = 0;
         p->id = i;
@@ -3486,6 +3492,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
     ram_control_before_iterate(f, RAM_CONTROL_SETUP);
     ram_control_after_iterate(f, RAM_CONTROL_SETUP);
 
+    /* We want to wait for all threads to have started before doing
+     * anything else */
+    for (int i = 0; i < migrate_multifd_channels(); i++) {
+        MultiFDSendParams *p = &multifd_send_state->params[i];
+
+        qemu_sem_wait(&p->started);
+        trace_multifd_send_thread_started(p->id);
+    }
     multifd_send_sync_main();
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
     qemu_fflush(f);
diff --git a/migration/trace-events b/migration/trace-events
index 886ce70ca0..dd13a5c4b1 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -95,6 +95,7 @@ multifd_send_sync_main_wait(uint8_t id) "channel %d"
 multifd_send_terminate_threads(bool error) "error %d"
 multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %"  PRIu64
 multifd_send_thread_start(uint8_t id) "%d"
+multifd_send_thread_started(uint8_t id) "channel %d"
 ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
 ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%s: addr: 0x%" PRIx64 " flags: 0x%x host: %p"
 ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x"
-- 
2.21.0

Re: [Qemu-devel] [PATCH 3/6] migration: Make sure that all multifd channels have been created

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> If we start the migration before all have been created, we have to
> handle the case that one channel still don't exist.  This way it is
> easier.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  migration/ram.c        | 14 ++++++++++++++
>  migration/trace-events |  1 +
>  2 files changed, 15 insertions(+)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index 4bdd201a4e..4a6ae677a9 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -663,6 +663,8 @@ typedef struct {
>      uint64_t num_pages;
>      /* syncs main thread and channels */
>      QemuSemaphore sem_sync;
> +    /* thread has started and setup is done */
> +    QemuSemaphore started;
>  }  MultiFDSendParams;
>  
>  typedef struct {
> @@ -1039,6 +1041,7 @@ void multifd_save_cleanup(void)
>          qemu_mutex_destroy(&p->mutex);
>          qemu_sem_destroy(&p->sem);
>          qemu_sem_destroy(&p->sem_sync);
> +        qemu_sem_destroy(&p->started);
>          g_free(p->name);
>          p->name = NULL;
>          multifd_pages_clear(p->pages);
> @@ -1113,6 +1116,8 @@ static void *multifd_send_thread(void *opaque)
>      /* initial packet */
>      p->num_packets = 1;
>  
> +    qemu_sem_post(&p->started);
> +
>      while (true) {
>          qemu_sem_wait(&p->sem);
>          qemu_mutex_lock(&p->mutex);
> @@ -1229,6 +1234,7 @@ int multifd_save_setup(void)
>          qemu_mutex_init(&p->mutex);
>          qemu_sem_init(&p->sem, 0);
>          qemu_sem_init(&p->sem_sync, 0);
> +        qemu_sem_init(&p->started, 0);
>          p->quit = false;
>          p->pending_job = 0;
>          p->id = i;
> @@ -3486,6 +3492,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
>      ram_control_before_iterate(f, RAM_CONTROL_SETUP);
>      ram_control_after_iterate(f, RAM_CONTROL_SETUP);
>  
> +    /* We want to wait for all threads to have started before doing
> +     * anything else */
> +    for (int i = 0; i < migrate_multifd_channels(); i++) {
> +        MultiFDSendParams *p = &multifd_send_state->params[i];
> +
> +        qemu_sem_wait(&p->started);
> +        trace_multifd_send_thread_started(p->id);
> +    }

What happens if there's an error during startup and either we cancel or
the migration fails and we try and cleanup - how do we get out of this
loop?

Dave

>      multifd_send_sync_main();
>      qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
>      qemu_fflush(f);
> diff --git a/migration/trace-events b/migration/trace-events
> index 886ce70ca0..dd13a5c4b1 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -95,6 +95,7 @@ multifd_send_sync_main_wait(uint8_t id) "channel %d"
>  multifd_send_terminate_threads(bool error) "error %d"
>  multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %"  PRIu64
>  multifd_send_thread_start(uint8_t id) "%d"
> +multifd_send_thread_started(uint8_t id) "channel %d"
>  ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
>  ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%s: addr: 0x%" PRIx64 " flags: 0x%x host: %p"
>  ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x"
> -- 
> 2.21.0
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 3/6] migration: Make sure that all multifd channels have been created

[Juan Quintela]

"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> If we start the migration before all have been created, we have to
>> handle the case that one channel still don't exist.  This way it is
>> easier.
>> 
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> @@ -3486,6 +3492,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
>>      ram_control_before_iterate(f, RAM_CONTROL_SETUP);
>>      ram_control_after_iterate(f, RAM_CONTROL_SETUP);
>>  
>> +    /* We want to wait for all threads to have started before doing
>> +     * anything else */
>> +    for (int i = 0; i < migrate_multifd_channels(); i++) {
>> +        MultiFDSendParams *p = &multifd_send_state->params[i];
>> +
>> +        qemu_sem_wait(&p->started);
>> +        trace_multifd_send_thread_started(p->id);
>> +    }
>
> What happens if there's an error during startup and either we cancel or
> the migration fails and we try and cleanup - how do we get out of this
> loop?

Good catch.

Will think a way to do it.

Thanks.


> Dave
>
>>      multifd_send_sync_main();
>>      qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
>>      qemu_fflush(f);
>> diff --git a/migration/trace-events b/migration/trace-events
>> index 886ce70ca0..dd13a5c4b1 100644
>> --- a/migration/trace-events
>> +++ b/migration/trace-events
>> @@ -95,6 +95,7 @@ multifd_send_sync_main_wait(uint8_t id) "channel %d"
>>  multifd_send_terminate_threads(bool error) "error %d"
>>  multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %"  PRIu64
>>  multifd_send_thread_start(uint8_t id) "%d"
>> +multifd_send_thread_started(uint8_t id) "channel %d"
>>  ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
>>  ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%s: addr: 0x%" PRIx64 " flags: 0x%x host: %p"
>>  ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x"
>> -- 
>> 2.21.0
>> 
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

[Qemu-devel] [PATCH 4/6] migration: Make multifd threads wait until all have been created

[Juan Quintela]

This makes it clear that no thread handles any incoming message until
all threads have been created.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c        | 24 ++++++++++++++++++++++--
 migration/trace-events |  1 +
 2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 4a6ae677a9..f1aec95f83 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -702,6 +702,8 @@ typedef struct {
     uint64_t num_pages;
     /* syncs main thread and channels */
     QemuSemaphore sem_sync;
+    /* thread can continue */
+    QemuSemaphore can_start;
 } MultiFDRecvParams;
 
 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -1313,6 +1315,7 @@ int multifd_load_cleanup(Error **errp)
         p->c = NULL;
         qemu_mutex_destroy(&p->mutex);
         qemu_sem_destroy(&p->sem_sync);
+        qemu_sem_destroy(&p->can_start);
         g_free(p->name);
         p->name = NULL;
         multifd_pages_clear(p->pages);
@@ -1366,6 +1369,9 @@ static void *multifd_recv_thread(void *opaque)
     trace_multifd_recv_thread_start(p->id);
     rcu_register_thread();
 
+    qemu_sem_wait(&p->can_start);
+    trace_multifd_recv_thread_can_start(p->id);
+
     while (true) {
         uint32_t used;
         uint32_t flags;
@@ -1445,6 +1451,7 @@ int multifd_load_setup(void)
 
         qemu_mutex_init(&p->mutex);
         qemu_sem_init(&p->sem_sync, 0);
+        qemu_sem_init(&p->can_start, 0);
         p->quit = false;
         p->id = i;
         p->pages = multifd_pages_init(page_count);
@@ -1477,6 +1484,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp)
 {
     MultiFDRecvParams *p;
     Error *local_err = NULL;
+    bool last_one;
     int id;
 
     id = multifd_recv_initial_packet(ioc, &local_err);
@@ -1506,8 +1514,20 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp)
     qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p,
                        QEMU_THREAD_JOINABLE);
     atomic_inc(&multifd_recv_state->count);
-    return atomic_read(&multifd_recv_state->count) ==
-           migrate_multifd_channels();
+
+    last_one =  atomic_read(&multifd_recv_state->count)
+        == migrate_multifd_channels();
+
+    if (last_one) {
+        int i;
+
+        for (i = 0; i < migrate_multifd_channels(); i++) {
+            MultiFDRecvParams *p = &multifd_recv_state->params[i];
+
+            qemu_sem_post(&p->can_start);
+        }
+    }
+    return last_one;
 }
 
 /**
diff --git a/migration/trace-events b/migration/trace-events
index dd13a5c4b1..9fbef614ab 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -86,6 +86,7 @@ multifd_recv_sync_main(long packet_num) "packet num %ld"
 multifd_recv_sync_main_signal(uint8_t id) "channel %d"
 multifd_recv_sync_main_wait(uint8_t id) "channel %d"
 multifd_recv_terminate_threads(bool error) "error %d"
+multifd_recv_thread_can_start(uint8_t id) "channel %d"
 multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64
 multifd_recv_thread_start(uint8_t id) "%d"
 multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
-- 
2.21.0

[Qemu-devel] [PATCH 5/6] migration: add some multifd traces

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c        | 3 +++
 migration/trace-events | 4 ++++
 2 files changed, 7 insertions(+)

diff --git a/migration/ram.c b/migration/ram.c
index f1aec95f83..25a211c3fb 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1173,6 +1173,7 @@ static void *multifd_send_thread(void *opaque)
 
 out:
     if (local_err) {
+        trace_multifd_send_error(p->id);
         multifd_send_terminate_threads(local_err);
     }
 
@@ -1203,6 +1204,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
     QIOChannel *sioc = QIO_CHANNEL(qio_task_get_source(task));
     Error *local_err = NULL;
 
+    trace_multifd_new_send_channel_async(p->id);
     if (qio_task_propagate_error(task, &local_err)) {
         migrate_set_error(migrate_get_current(), local_err);
         multifd_save_cleanup();
@@ -1496,6 +1498,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp)
                                 atomic_read(&multifd_recv_state->count));
         return false;
     }
+    trace_multifd_recv_new_channel(id);
 
     p = &multifd_recv_state->params[id];
     if (p->c != NULL) {
diff --git a/migration/trace-events b/migration/trace-events
index 9fbef614ab..5d85f8bf83 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -81,7 +81,9 @@ migration_bitmap_sync_start(void) ""
 migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
 migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, unsigned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx"
 migration_throttle(void) ""
+multifd_new_send_channel_async(uint8_t id) "channel %d"
 multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
+multifd_recv_new_channel(uint8_t id) "channel %d"
 multifd_recv_sync_main(long packet_num) "packet num %ld"
 multifd_recv_sync_main_signal(uint8_t id) "channel %d"
 multifd_recv_sync_main_wait(uint8_t id) "channel %d"
@@ -89,7 +91,9 @@ multifd_recv_terminate_threads(bool error) "error %d"
 multifd_recv_thread_can_start(uint8_t id) "channel %d"
 multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64
 multifd_recv_thread_start(uint8_t id) "%d"
+multifd_save_setup_wait(uint8_t id) "%d"
 multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
+multifd_send_error(uint8_t id) "channel %d"
 multifd_send_sync_main(long packet_num) "packet num %ld"
 multifd_send_sync_main_signal(uint8_t id) "channel %d"
 multifd_send_sync_main_wait(uint8_t id) "channel %d"
-- 
2.21.0

Re: [Qemu-devel] [PATCH 5/6] migration: add some multifd traces

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>


Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Queued
> ---
>  migration/ram.c        | 3 +++
>  migration/trace-events | 4 ++++
>  2 files changed, 7 insertions(+)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index f1aec95f83..25a211c3fb 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -1173,6 +1173,7 @@ static void *multifd_send_thread(void *opaque)
>  
>  out:
>      if (local_err) {
> +        trace_multifd_send_error(p->id);
>          multifd_send_terminate_threads(local_err);
>      }
>  
> @@ -1203,6 +1204,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
>      QIOChannel *sioc = QIO_CHANNEL(qio_task_get_source(task));
>      Error *local_err = NULL;
>  
> +    trace_multifd_new_send_channel_async(p->id);
>      if (qio_task_propagate_error(task, &local_err)) {
>          migrate_set_error(migrate_get_current(), local_err);
>          multifd_save_cleanup();
> @@ -1496,6 +1498,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp)
>                                  atomic_read(&multifd_recv_state->count));
>          return false;
>      }
> +    trace_multifd_recv_new_channel(id);
>  
>      p = &multifd_recv_state->params[id];
>      if (p->c != NULL) {
> diff --git a/migration/trace-events b/migration/trace-events
> index 9fbef614ab..5d85f8bf83 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -81,7 +81,9 @@ migration_bitmap_sync_start(void) ""
>  migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
>  migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, unsigned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx"
>  migration_throttle(void) ""
> +multifd_new_send_channel_async(uint8_t id) "channel %d"
>  multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
> +multifd_recv_new_channel(uint8_t id) "channel %d"
>  multifd_recv_sync_main(long packet_num) "packet num %ld"
>  multifd_recv_sync_main_signal(uint8_t id) "channel %d"
>  multifd_recv_sync_main_wait(uint8_t id) "channel %d"
> @@ -89,7 +91,9 @@ multifd_recv_terminate_threads(bool error) "error %d"
>  multifd_recv_thread_can_start(uint8_t id) "channel %d"
>  multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64
>  multifd_recv_thread_start(uint8_t id) "%d"
> +multifd_save_setup_wait(uint8_t id) "%d"
>  multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
> +multifd_send_error(uint8_t id) "channel %d"
>  multifd_send_sync_main(long packet_num) "packet num %ld"
>  multifd_send_sync_main_signal(uint8_t id) "channel %d"
>  multifd_send_sync_main_wait(uint8_t id) "channel %d"
> -- 
> 2.21.0
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 5/6] migration: add some multifd traces

[Philippe Mathieu-Daudé]

On 8/14/19 4:02 AM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  migration/ram.c        | 3 +++
>  migration/trace-events | 4 ++++
>  2 files changed, 7 insertions(+)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index f1aec95f83..25a211c3fb 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -1173,6 +1173,7 @@ static void *multifd_send_thread(void *opaque)
>  
>  out:
>      if (local_err) {
> +        trace_multifd_send_error(p->id);
>          multifd_send_terminate_threads(local_err);
>      }
>  
> @@ -1203,6 +1204,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
>      QIOChannel *sioc = QIO_CHANNEL(qio_task_get_source(task));
>      Error *local_err = NULL;
>  
> +    trace_multifd_new_send_channel_async(p->id);
>      if (qio_task_propagate_error(task, &local_err)) {
>          migrate_set_error(migrate_get_current(), local_err);
>          multifd_save_cleanup();
> @@ -1496,6 +1498,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp)
>                                  atomic_read(&multifd_recv_state->count));
>          return false;
>      }
> +    trace_multifd_recv_new_channel(id);
>  
>      p = &multifd_recv_state->params[id];
>      if (p->c != NULL) {
> diff --git a/migration/trace-events b/migration/trace-events
> index 9fbef614ab..5d85f8bf83 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -81,7 +81,9 @@ migration_bitmap_sync_start(void) ""
>  migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
>  migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, unsigned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx"
>  migration_throttle(void) ""
> +multifd_new_send_channel_async(uint8_t id) "channel %d"
>  multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
> +multifd_recv_new_channel(uint8_t id) "channel %d"
>  multifd_recv_sync_main(long packet_num) "packet num %ld"
>  multifd_recv_sync_main_signal(uint8_t id) "channel %d"
>  multifd_recv_sync_main_wait(uint8_t id) "channel %d"
> @@ -89,7 +91,9 @@ multifd_recv_terminate_threads(bool error) "error %d"
>  multifd_recv_thread_can_start(uint8_t id) "channel %d"
>  multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64
>  multifd_recv_thread_start(uint8_t id) "%d"
> +multifd_save_setup_wait(uint8_t id) "%d"

This one is not used.

For the rest:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

>  multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
> +multifd_send_error(uint8_t id) "channel %d"
>  multifd_send_sync_main(long packet_num) "packet num %ld"
>  multifd_send_sync_main_signal(uint8_t id) "channel %d"
>  multifd_send_sync_main_wait(uint8_t id) "channel %d"
> 

Re: [Qemu-devel] [PATCH 5/6] migration: add some multifd traces

[Philippe Mathieu-Daudé]

On 8/19/19 1:10 PM, Philippe Mathieu-Daudé wrote:
> On 8/14/19 4:02 AM, Juan Quintela wrote:
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  migration/ram.c        | 3 +++
>>  migration/trace-events | 4 ++++
>>  2 files changed, 7 insertions(+)
>>
>> diff --git a/migration/ram.c b/migration/ram.c
>> index f1aec95f83..25a211c3fb 100644
>> --- a/migration/ram.c
>> +++ b/migration/ram.c
>> @@ -1173,6 +1173,7 @@ static void *multifd_send_thread(void *opaque)
>>  
>>  out:
>>      if (local_err) {
>> +        trace_multifd_send_error(p->id);
>>          multifd_send_terminate_threads(local_err);
>>      }
>>  
>> @@ -1203,6 +1204,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
>>      QIOChannel *sioc = QIO_CHANNEL(qio_task_get_source(task));
>>      Error *local_err = NULL;
>>  
>> +    trace_multifd_new_send_channel_async(p->id);
>>      if (qio_task_propagate_error(task, &local_err)) {
>>          migrate_set_error(migrate_get_current(), local_err);
>>          multifd_save_cleanup();
>> @@ -1496,6 +1498,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp)
>>                                  atomic_read(&multifd_recv_state->count));
>>          return false;
>>      }
>> +    trace_multifd_recv_new_channel(id);
>>  
>>      p = &multifd_recv_state->params[id];
>>      if (p->c != NULL) {
>> diff --git a/migration/trace-events b/migration/trace-events
>> index 9fbef614ab..5d85f8bf83 100644
>> --- a/migration/trace-events
>> +++ b/migration/trace-events
>> @@ -81,7 +81,9 @@ migration_bitmap_sync_start(void) ""
>>  migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
>>  migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, unsigned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx"
>>  migration_throttle(void) ""
>> +multifd_new_send_channel_async(uint8_t id) "channel %d"
>>  multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
>> +multifd_recv_new_channel(uint8_t id) "channel %d"
>>  multifd_recv_sync_main(long packet_num) "packet num %ld"
>>  multifd_recv_sync_main_signal(uint8_t id) "channel %d"
>>  multifd_recv_sync_main_wait(uint8_t id) "channel %d"
>> @@ -89,7 +91,9 @@ multifd_recv_terminate_threads(bool error) "error %d"
>>  multifd_recv_thread_can_start(uint8_t id) "channel %d"
>>  multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64
>>  multifd_recv_thread_start(uint8_t id) "%d"
>> +multifd_save_setup_wait(uint8_t id) "%d"
> 
> This one is not used.
> 
> For the rest:
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Oh I forgot, uint8_t format is "%u", we don't care so far but it could
be useful to enable the -Wformat-signedness cflag one day.

>>  multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
>> +multifd_send_error(uint8_t id) "channel %d"
>>  multifd_send_sync_main(long packet_num) "packet num %ld"
>>  multifd_send_sync_main_signal(uint8_t id) "channel %d"
>>  multifd_send_sync_main_wait(uint8_t id) "channel %d"
>>

[Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Juan Quintela]

When we have lots of channels, sometimes multifd migration fails
with the following error:

(qemu) migrate -d tcp:0:4444
(qemu) qemu-system-x86_64: multifd_send_pages: channel 17 has already quit!
qemu-system-x86_64: multifd_send_pages: channel 17 has already quit!
qemu-system-x86_64: multifd_send_sync_main: multifd_send_pages fail
qemu-system-x86_64: Unable to write to socket: Connection reset by peer
info migrate
globals:
store-global-state: on
only-migratable: off
send-configuration: on
send-section-footer: on
decompress-error-check: on
clear-bitmap-shift: 18
capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off compress: off events: off postcopy-ram: off x-colo: off release-ram: off block: off return-path: off pause-before-switchover: off multifd: on dirty-bitmaps: off postcopy-blocktime: off late-block-activate: off x-ignore-shared: off
Migration status: failed (Unable to write to socket: Connection reset by peer)
total time: 0 milliseconds

On this particular example I am using 100 channels.  The bigger the
number of channels, the easier that it is to reproduce.  That don't
mean that it is a good idea to use so many channels.

With the previous patches on this series, I can run "reliabely" on my
hardware with until 10 channels.  Most of the time.  Until it fails.
With 100 channels, it fails almost always.

I thought that the problem was on the send side, so I tried to debug
there.  As you can see for the delay, if you put any
printf()/error_report/trace, you can get that the error goes away, it
is very timing sensitive.  With a delay of 10000 microseconds, it only
works sometimes.

What have I discovered so far:

- send side calls qemu_socket() on all the channels.  So it appears
  that it gets created correctly.
- on the destination side, it appears that "somehowe" some of the
  connections are lost by the listener.  This error happens when the
  destination side socket hasn't been "accepted", and it is not
  properly created.  As far as I can see, we have several options:

  1- I don't know how to use properly qio asynchronously
     (this is one big posiblity).

  2- glib has one error in this case?  or how qio listener is
     implemented on top of glib.  I put lots of printf() and other
     instrumentation, and it appears that the listener io_func is not
     called at all for the connections that are missing.

  3- it is always possible that we are missing some g_main_loop_run()
     somewhere.  Notice how test/test-io-channel-socket.c calls it
     "creatively".

  4- It is enterely possible that I should be using the sockets as
     blocking instead of non-blocking.  But I am not sure about that
     one yet.

- on the sending side, what happens is:

  eventually it call socket_connect() after all the async dance with
  thread creation, etc, etc. Source side creates all the channels, it
  is the destination side which is missing some of them.

  sending side sends the first packet by that channel, it "sucheeds"
  and didn't give any error.

  after some time, sending side decides to send another packet through
  that channel, and it is now when we get the above error.

Any good ideas?

Later, Juan.

PD: Command line used is attached:

Imortant bits:
- multifd is set
- multifd_channels is set to 100

/scratch/qemu/fail/x64/x86_64-softmmu/qemu-system-x86_64 -M
pc-i440fx-3.1,accel=kvm,usb=off,vmport=off,nvdimm -L
/mnt/code/qemu/check/pc-bios/ -smp 2 -name t1,debug-threads=on -m 3G
-uuid 113100f9-6c99-4a7a-9b78-eb1c088d1087 -monitor stdio -boot
strict=on -drive
file=/mnt/images/test.img,format=qcow2,if=none,id=disk0 -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=disk0,id=virtio-disk0,bootindex=1
-netdev tap,id=hostnet0,script=/etc/kvm-ifup,downscript= -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:9d:10:51,bus=pci.0,addr=0x3
-serial pty -parallel none -usb -device usb-tablet -k es -vga cirrus
--global migration.x-multifd=on --global
migration.multifd-channels=100 -trace events=/home/quintela/tmp/events

CC: Daniel P. Berrangé <berrange@redhat.com>

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/migration/ram.c b/migration/ram.c
index 25a211c3fb..50586304a0 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1248,6 +1248,7 @@ int multifd_save_setup(void)
         p->packet = g_malloc0(p->packet_len);
         p->name = g_strdup_printf("multifdsend_%d", i);
         socket_send_channel_create(multifd_new_send_channel_async, p);
+        usleep(100000);
     }
     return 0;
 }
-- 
2.21.0

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Daniel P. Berrangé]

On Wed, Aug 14, 2019 at 04:02:18AM +0200, Juan Quintela wrote:
> When we have lots of channels, sometimes multifd migration fails
> with the following error:
> 
> (qemu) migrate -d tcp:0:4444
> (qemu) qemu-system-x86_64: multifd_send_pages: channel 17 has already quit!
> qemu-system-x86_64: multifd_send_pages: channel 17 has already quit!
> qemu-system-x86_64: multifd_send_sync_main: multifd_send_pages fail
> qemu-system-x86_64: Unable to write to socket: Connection reset by peer
> info migrate
> globals:
> store-global-state: on
> only-migratable: off
> send-configuration: on
> send-section-footer: on
> decompress-error-check: on
> clear-bitmap-shift: 18
> capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off compress: off events: off postcopy-ram: off x-colo: off release-ram: off block: off return-path: off pause-before-switchover: off multifd: on dirty-bitmaps: off postcopy-blocktime: off late-block-activate: off x-ignore-shared: off
> Migration status: failed (Unable to write to socket: Connection reset by peer)
> total time: 0 milliseconds
> 
> On this particular example I am using 100 channels.  The bigger the
> number of channels, the easier that it is to reproduce.  That don't
> mean that it is a good idea to use so many channels.
> 
> With the previous patches on this series, I can run "reliabely" on my
> hardware with until 10 channels.  Most of the time.  Until it fails.
> With 100 channels, it fails almost always.
> 
> I thought that the problem was on the send side, so I tried to debug
> there.  As you can see for the delay, if you put any
> printf()/error_report/trace, you can get that the error goes away, it
> is very timing sensitive.  With a delay of 10000 microseconds, it only
> works sometimes.
> 
> What have I discovered so far:
> 
> - send side calls qemu_socket() on all the channels.  So it appears
>   that it gets created correctly.
> - on the destination side, it appears that "somehowe" some of the
>   connections are lost by the listener.  This error happens when the
>   destination side socket hasn't been "accepted", and it is not
>   properly created.  As far as I can see, we have several options:
> 
>   1- I don't know how to use properly qio asynchronously
>      (this is one big posiblity).
> 
>   2- glib has one error in this case?  or how qio listener is
>      implemented on top of glib.  I put lots of printf() and other
>      instrumentation, and it appears that the listener io_func is not
>      called at all for the connections that are missing.
> 
>   3- it is always possible that we are missing some g_main_loop_run()
>      somewhere.  Notice how test/test-io-channel-socket.c calls it
>      "creatively".
> 
>   4- It is enterely possible that I should be using the sockets as
>      blocking instead of non-blocking.  But I am not sure about that
>      one yet.
> 
> - on the sending side, what happens is:
> 
>   eventually it call socket_connect() after all the async dance with
>   thread creation, etc, etc. Source side creates all the channels, it
>   is the destination side which is missing some of them.
> 
>   sending side sends the first packet by that channel, it "sucheeds"
>   and didn't give any error.
> 
>   after some time, sending side decides to send another packet through
>   that channel, and it is now when we get the above error.
> 
> Any good ideas?

In inet_listen_saddr() we call

    if (!listen(slisten, 1)) {

note the second parameter sets the socket backlog, which is the max
number of pending socket connections we allow. My guess is that the
target QEMU is not accepting incoming connections quickly enough and
thus you hit the limit & the kernel starts dropping the incoming
connections.

As a quick test, just hack this code to pass a value of 100 and see
if it makes your test reliable. If it does, then we'll need to figure
out a nice way to handle backlog instead of hardcoding it at 1.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Juan Quintela]

Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Wed, Aug 14, 2019 at 04:02:18AM +0200, Juan Quintela wrote:
>> When we have lots of channels, sometimes multifd migration fails
>> with the following error:
>>   after some time, sending side decides to send another packet through
>>   that channel, and it is now when we get the above error.
>> 
>> Any good ideas?
>
> In inet_listen_saddr() we call
>
>     if (!listen(slisten, 1)) {
>
> note the second parameter sets the socket backlog, which is the max
> number of pending socket connections we allow. My guess is that the
> target QEMU is not accepting incoming connections quickly enough and
> thus you hit the limit & the kernel starts dropping the incoming
> connections.
>
> As a quick test, just hack this code to pass a value of 100 and see
> if it makes your test reliable. If it does, then we'll need to figure
> out a nice way to handle backlog instead of hardcoding it at 1.

I will test.

But notice that the qemu_connect() on source side says that things went
right.  It is the destination what is *not* calling the callback.  Or
at least that is what I think it is happening.

Later, Juan.

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Daniel P. Berrangé]

On Mon, Aug 19, 2019 at 12:33:45PM +0200, Juan Quintela wrote:
> Daniel P. Berrangé <berrange@redhat.com> wrote:
> > On Wed, Aug 14, 2019 at 04:02:18AM +0200, Juan Quintela wrote:
> >> When we have lots of channels, sometimes multifd migration fails
> >> with the following error:
> >>   after some time, sending side decides to send another packet through
> >>   that channel, and it is now when we get the above error.
> >> 
> >> Any good ideas?
> >
> > In inet_listen_saddr() we call
> >
> >     if (!listen(slisten, 1)) {
> >
> > note the second parameter sets the socket backlog, which is the max
> > number of pending socket connections we allow. My guess is that the
> > target QEMU is not accepting incoming connections quickly enough and
> > thus you hit the limit & the kernel starts dropping the incoming
> > connections.
> >
> > As a quick test, just hack this code to pass a value of 100 and see
> > if it makes your test reliable. If it does, then we'll need to figure
> > out a nice way to handle backlog instead of hardcoding it at 1.
> 
> I will test.
> 
> But notice that the qemu_connect() on source side says that things went
> right.  It is the destination what is *not* calling the callback.  Or
> at least that is what I think it is happening.

IIRC, the connect() can succeed on the source host, even if the target host
has not called accept(), because the kernel will complete the connection at
the protocol level regardless. IOW, don't assume that the destination QEMU
has seen the connection. If you turn on tracing, we have a trace point
"qio_channel_socket_accept_complete" that is emitted after we have done
an accept(). So you can see if you get 100 of those trace points emitted
on the target, when you make 100 connects on the source.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Juan Quintela]

Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Wed, Aug 14, 2019 at 04:02:18AM +0200, Juan Quintela wrote:
>> When we have lots of channels, sometimes multifd migration fails
>> with the following error:
>> 
>> Any good ideas?
>
> In inet_listen_saddr() we call
>
>     if (!listen(slisten, 1)) {
>
> note the second parameter sets the socket backlog, which is the max
> number of pending socket connections we allow. My guess is that the
> target QEMU is not accepting incoming connections quickly enough and
> thus you hit the limit & the kernel starts dropping the incoming
> connections.
>
> As a quick test, just hack this code to pass a value of 100 and see
> if it makes your test reliable. If it does, then we'll need to figure
> out a nice way to handle backlog instead of hardcoding it at 1.

Nice.

With this change I can create 100 channels on a 4 core machine without
any trouble.

How can we proceed from here?

Thanks, Juan.

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Daniel P. Berrangé]

On Mon, Aug 19, 2019 at 12:46:20PM +0200, Juan Quintela wrote:
> Daniel P. Berrangé <berrange@redhat.com> wrote:
> > On Wed, Aug 14, 2019 at 04:02:18AM +0200, Juan Quintela wrote:
> >> When we have lots of channels, sometimes multifd migration fails
> >> with the following error:
> >> 
> >> Any good ideas?
> >
> > In inet_listen_saddr() we call
> >
> >     if (!listen(slisten, 1)) {
> >
> > note the second parameter sets the socket backlog, which is the max
> > number of pending socket connections we allow. My guess is that the
> > target QEMU is not accepting incoming connections quickly enough and
> > thus you hit the limit & the kernel starts dropping the incoming
> > connections.
> >
> > As a quick test, just hack this code to pass a value of 100 and see
> > if it makes your test reliable. If it does, then we'll need to figure
> > out a nice way to handle backlog instead of hardcoding it at 1.
> 
> Nice.
> 
> With this change I can create 100 channels on a 4 core machine without
> any trouble.
> 
> How can we proceed from here?

I don't think we want to expose this in the QAPI schema for the socket
address, since the correct value is really something that QEMU should
figure out based on usage context.

Thus, I think we'll have to make it an explicit parameter to the
qio_channel_socket_listen_{sync,async} APIs, and socket_listen()
and inet_listen_saddr(), etc. Then the migration code can pass in
a sensible value based on multifd usage.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Juan Quintela]

Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Mon, Aug 19, 2019 at 12:46:20PM +0200, Juan Quintela wrote:
>> Daniel P. Berrangé <berrange@redhat.com> wrote:
>> > On Wed, Aug 14, 2019 at 04:02:18AM +0200, Juan Quintela wrote:
>> >> When we have lots of channels, sometimes multifd migration fails
>> >> with the following error:
>> >> 
>> >> Any good ideas?
>> >
>> > In inet_listen_saddr() we call
>> >
>> >     if (!listen(slisten, 1)) {
>> >
>> > note the second parameter sets the socket backlog, which is the max
>> > number of pending socket connections we allow. My guess is that the
>> > target QEMU is not accepting incoming connections quickly enough and
>> > thus you hit the limit & the kernel starts dropping the incoming
>> > connections.
>> >
>> > As a quick test, just hack this code to pass a value of 100 and see
>> > if it makes your test reliable. If it does, then we'll need to figure
>> > out a nice way to handle backlog instead of hardcoding it at 1.
>> 
>> Nice.
>> 
>> With this change I can create 100 channels on a 4 core machine without
>> any trouble.
>> 
>> How can we proceed from here?
>
> I don't think we want to expose this in the QAPI schema for the socket
> address, since the correct value is really something that QEMU should
> figure out based on usage context.
>
> Thus, I think we'll have to make it an explicit parameter to the
> qio_channel_socket_listen_{sync,async} APIs, and socket_listen()
> and inet_listen_saddr(), etc. Then the migration code can pass in
> a sensible value based on multifd usage.

ok with me.  I will give it a try.

Thanks for the tip.

Later, Juan.

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Peter Maydell]

On Mon, 19 Aug 2019 at 11:50, Daniel P. Berrangé <berrange@redhat.com> wrote:
> I don't think we want to expose this in the QAPI schema for the socket
> address, since the correct value is really something that QEMU should
> figure out based on usage context.
>
> Thus, I think we'll have to make it an explicit parameter to the
> qio_channel_socket_listen_{sync,async} APIs, and socket_listen()
> and inet_listen_saddr(), etc. Then the migration code can pass in
> a sensible value based on multifd usage.

How bad would it be if we just passed SOMAXCONN for the backlog
value always?

thanks
-- PMM

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Daniel P. Berrangé]

On Mon, Aug 19, 2019 at 12:00:15PM +0100, Peter Maydell wrote:
> On Mon, 19 Aug 2019 at 11:50, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > I don't think we want to expose this in the QAPI schema for the socket
> > address, since the correct value is really something that QEMU should
> > figure out based on usage context.
> >
> > Thus, I think we'll have to make it an explicit parameter to the
> > qio_channel_socket_listen_{sync,async} APIs, and socket_listen()
> > and inet_listen_saddr(), etc. Then the migration code can pass in
> > a sensible value based on multifd usage.
> 
> How bad would it be if we just passed SOMAXCONN for the backlog
> value always?

I'm not 100% clear to be honest, but my feeling is that this will waste
resources for much of QEMU usage where we only ever accept a single
client connection on an incoming socket.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 6/6] RFH: We lost "connect" events

[Juan Quintela]

Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Wed, Aug 14, 2019 at 04:02:18AM +0200, Juan Quintela wrote:
>> When we have lots of channels, sometimes multifd migration fails
>> with the following error:
>> 
>> Any good ideas?
>
> In inet_listen_saddr() we call
>
>     if (!listen(slisten, 1)) {
>
> note the second parameter sets the socket backlog, which is the max
> number of pending socket connections we allow. My guess is that the
> target QEMU is not accepting incoming connections quickly enough and
> thus you hit the limit & the kernel starts dropping the incoming
> connections.
>
> As a quick test, just hack this code to pass a value of 100 and see
> if it makes your test reliable. If it does, then we'll need to figure
> out a nice way to handle backlog instead of hardcoding it at 1.

Nice.

With this change I can create 100 channels on a 4 core machine without
any trouble (5 tries so far).  Previously with 10 channels on the same
machine failed around 50% of the time, and 100 channels failed 100% of
the time.

How can we proceed from here?

Thanks, Juan.

Re: [Qemu-devel] [PATCH 0/6] Fix multifd with big number of channels

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190814020218.1868-1-quintela@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

PASS 1 fdc-test /x86_64/fdc/cmos
PASS 2 fdc-test /x86_64/fdc/no_media_on_start
PASS 3 fdc-test /x86_64/fdc/read_without_media
==8022==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 4 fdc-test /x86_64/fdc/media_change
PASS 5 fdc-test /x86_64/fdc/sense_interrupt
PASS 6 fdc-test /x86_64/fdc/relative_seek
---
PASS 32 test-opts-visitor /visitor/opts/range/beyond
PASS 33 test-opts-visitor /visitor/opts/dict/unvisited
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-coroutine -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-coroutine" 
==8066==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8066==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffce27db000; bottom 0x7f337a3f8000; size: 0x00c9683e3000 (865037332480)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-coroutine /basic/no-dangling-access
---
PASS 13 test-aio /aio/event/wait/no-flush-cb
PASS 12 fdc-test /x86_64/fdc/read_no_dma_19
PASS 14 test-aio /aio/timer/schedule
==8085==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 13 fdc-test /x86_64/fdc/fuzz-registers
PASS 15 test-aio /aio/coroutine/queue-chaining
PASS 16 test-aio /aio-gsource/flush
---
PASS 26 test-aio /aio-gsource/event/flush
PASS 27 test-aio /aio-gsource/event/wait/no-flush-cb
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img tests/ide-test -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="ide-test" 
==8094==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 28 test-aio /aio-gsource/timer/schedule
PASS 1 ide-test /x86_64/ide/identify
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-aio-multithread -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-aio-multithread" 
==8103==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-aio-multithread /aio/multi/lifecycle
==8101==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 ide-test /x86_64/ide/flush
PASS 2 test-aio-multithread /aio/multi/schedule
==8121==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 ide-test /x86_64/ide/bmdma/simple_rw
PASS 3 test-aio-multithread /aio/multi/mutex/contended
==8132==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 4 ide-test /x86_64/ide/bmdma/trim
==8143==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 5 ide-test /x86_64/ide/bmdma/short_prdt
==8149==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 6 ide-test /x86_64/ide/bmdma/one_sector_short_prdt
PASS 4 test-aio-multithread /aio/multi/mutex/handoff
==8155==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 7 ide-test /x86_64/ide/bmdma/long_prdt
PASS 5 test-aio-multithread /aio/multi/mutex/mcs
==8166==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8166==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fff3d833000; bottom 0x7efe37d76000; size: 0x010105abd000 (1103901741056)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 8 ide-test /x86_64/ide/bmdma/no_busmaster
PASS 6 test-aio-multithread /aio/multi/mutex/pthread
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-throttle -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-throttle" 
==8183==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-throttle /throttle/leak_bucket
PASS 2 test-throttle /throttle/compute_wait
PASS 3 test-throttle /throttle/init
---
PASS 15 test-throttle /throttle/config/iops_size
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-thread-pool -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-thread-pool" 
PASS 9 ide-test /x86_64/ide/flush/nodev
==8190==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-thread-pool /thread-pool/submit
PASS 2 test-thread-pool /thread-pool/submit-aio
PASS 3 test-thread-pool /thread-pool/submit-co
PASS 4 test-thread-pool /thread-pool/submit-many
==8192==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 10 ide-test /x86_64/ide/flush/empty_drive
==8262==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 5 test-thread-pool /thread-pool/cancel
PASS 11 ide-test /x86_64/ide/flush/retry_pci
==8268==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 12 ide-test /x86_64/ide/flush/retry_isa
PASS 6 test-thread-pool /thread-pool/cancel-async
==8274==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-hbitmap -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-hbitmap" 
PASS 13 ide-test /x86_64/ide/cdrom/pio
PASS 1 test-hbitmap /hbitmap/granularity
---
PASS 3 test-hbitmap /hbitmap/size/unaligned
PASS 4 test-hbitmap /hbitmap/iter/empty
PASS 5 test-hbitmap /hbitmap/iter/partial
==8285==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 6 test-hbitmap /hbitmap/iter/granularity
PASS 7 test-hbitmap /hbitmap/iter/iter_and_reset
PASS 8 test-hbitmap /hbitmap/get/all
---
PASS 28 test-hbitmap /hbitmap/truncate/shrink/medium
PASS 29 test-hbitmap /hbitmap/truncate/shrink/large
PASS 30 test-hbitmap /hbitmap/meta/zero
==8291==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 15 ide-test /x86_64/ide/cdrom/dma
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img tests/ahci-test -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="ahci-test" 
==8305==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 ahci-test /x86_64/ahci/sanity
==8311==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 ahci-test /x86_64/ahci/pci_spec
==8317==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 31 test-hbitmap /hbitmap/meta/one
PASS 32 test-hbitmap /hbitmap/meta/byte
PASS 33 test-hbitmap /hbitmap/meta/word
PASS 3 ahci-test /x86_64/ahci/pci_enable
PASS 34 test-hbitmap /hbitmap/meta/sector
PASS 35 test-hbitmap /hbitmap/serialize/align
==8323==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 4 ahci-test /x86_64/ahci/hba_spec
==8329==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 36 test-hbitmap /hbitmap/serialize/basic
PASS 37 test-hbitmap /hbitmap/serialize/part
PASS 38 test-hbitmap /hbitmap/serialize/zeroes
---
PASS 43 test-hbitmap /hbitmap/next_dirty_area/next_dirty_area_4
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-bdrv-drain -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-bdrv-drain" 
PASS 5 ahci-test /x86_64/ahci/hba_enable
==8337==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-bdrv-drain /bdrv-drain/nested
PASS 2 test-bdrv-drain /bdrv-drain/multiparent
PASS 3 test-bdrv-drain /bdrv-drain/set_aio_context
---
PASS 24 test-bdrv-drain /bdrv-drain/blockjob/commit_by_drained_end
PASS 25 test-bdrv-drain /bdrv-drain/blockjob/error/drain_all
PASS 26 test-bdrv-drain /bdrv-drain/blockjob/error/drain
==8339==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 27 test-bdrv-drain /bdrv-drain/blockjob/error/drain_subtree
PASS 28 test-bdrv-drain /bdrv-drain/blockjob/iothread/drain_all
PASS 29 test-bdrv-drain /bdrv-drain/blockjob/iothread/drain
---
PASS 40 test-bdrv-drain /bdrv-drain/attach/drain
PASS 6 ahci-test /x86_64/ahci/identify
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-bdrv-graph-mod -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-bdrv-graph-mod" 
==8384==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-bdrv-graph-mod /bdrv-graph-mod/update-perm-tree
PASS 2 test-bdrv-graph-mod /bdrv-graph-mod/should-update-child
==8382==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-blockjob -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-blockjob" 
==8393==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-blockjob /blockjob/ids
PASS 2 test-blockjob /blockjob/cancel/created
PASS 3 test-blockjob /blockjob/cancel/running
---
PASS 8 test-blockjob /blockjob/cancel/concluded
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-blockjob-txn -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-blockjob-txn" 
PASS 7 ahci-test /x86_64/ahci/max
==8399==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-blockjob-txn /single/success
PASS 2 test-blockjob-txn /single/failure
PASS 3 test-blockjob-txn /single/cancel
---
PASS 6 test-blockjob-txn /pair/cancel
PASS 7 test-blockjob-txn /pair/fail-cancel-race
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-block-backend -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-block-backend" 
==8401==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8405==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-block-backend /block-backend/drain_aio_error
PASS 2 test-block-backend /block-backend/drain_all_aio_error
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-block-iothread -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-block-iothread" 
==8414==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-block-iothread /sync-op/pread
PASS 2 test-block-iothread /sync-op/pwrite
PASS 3 test-block-iothread /sync-op/load_vmstate
---
PASS 16 test-block-iothread /propagate/mirror
PASS 8 ahci-test /x86_64/ahci/reset
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-image-locking -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-image-locking" 
==8435==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8437==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-image-locking /image-locking/basic
PASS 2 test-image-locking /image-locking/set-perm-abort
==8435==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffc6e4dc000; bottom 0x7ff2e9bfe000; size: 0x0009848de000 (40878596096)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-x86-cpuid -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-x86-cpuid" 
---
PASS 3 test-xbzrle /xbzrle/encode_decode_unchanged
PASS 4 test-xbzrle /xbzrle/encode_decode_1_byte
PASS 5 test-xbzrle /xbzrle/encode_decode_overflow
==8450==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8450==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffffe158000; bottom 0x7fec6effe000; size: 0x00138f15a000 (84004937728)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 6 test-xbzrle /xbzrle/encode_decode
---
PASS 1 test-shift128 /host-utils/test_lshift
PASS 2 test-shift128 /host-utils/test_rshift
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-mul64 -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-mul64" 
==8468==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-mul64 /host-utils/mulu64
PASS 2 test-mul64 /host-utils/muls64
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-int128 -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-int128" 
==8468==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffd369e3000; bottom 0x7f8f527fe000; size: 0x006de41e5000 (471978627072)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-int128 /int128/int128_and
---
PASS 10 test-int128 /int128/int128_rshift
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/rcutorture -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="rcutorture" 
PASS 11 ahci-test /x86_64/ahci/io/pio/lba28/simple/high
==8498==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8498==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fff893e4000; bottom 0x7f2f24bfe000; size: 0x00d0647e6000 (895039201280)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 rcutorture /rcu/torture/1reader
PASS 12 ahci-test /x86_64/ahci/io/pio/lba28/double/zero
==8520==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 rcutorture /rcu/torture/10readers
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-rcu-list -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-rcu-list" 
==8520==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe570f8000; bottom 0x7f41a2ffe000; size: 0x00bcb40fa000 (810474774528)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 13 ahci-test /x86_64/ahci/io/pio/lba28/double/low
PASS 1 test-rcu-list /rcu/qlist/single-threaded
==8533==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8533==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fffcbebd000; bottom 0x7f2ea0dfe000; size: 0x00d12b0bf000 (898370367488)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 2 test-rcu-list /rcu/qlist/short-few
PASS 14 ahci-test /x86_64/ahci/io/pio/lba28/double/high
==8566==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8566==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fff49e0a000; bottom 0x7f95e3d24000; size: 0x0069660e6000 (452683784192)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 15 ahci-test /x86_64/ahci/io/pio/lba28/long/zero
PASS 3 test-rcu-list /rcu/qlist/long-many
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-rcu-simpleq -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-rcu-simpleq" 
==8572==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8572==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffdfe654000; bottom 0x7f9fe8bfe000; size: 0x005e15a56000 (404090085376)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-rcu-simpleq /rcu/qsimpleq/single-threaded
PASS 16 ahci-test /x86_64/ahci/io/pio/lba28/long/low
==8591==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 test-rcu-simpleq /rcu/qsimpleq/short-few
==8591==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffee0f22000; bottom 0x7fab7e524000; size: 0x0053629fe000 (358136930304)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 17 ahci-test /x86_64/ahci/io/pio/lba28/long/high
==8618==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-rcu-simpleq /rcu/qsimpleq/long-many
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-rcu-tailq -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-rcu-tailq" 
PASS 18 ahci-test /x86_64/ahci/io/pio/lba28/short/zero
==8628==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-rcu-tailq /rcu/qtailq/single-threaded
PASS 19 ahci-test /x86_64/ahci/io/pio/lba28/short/low
==8643==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 test-rcu-tailq /rcu/qtailq/short-few
PASS 20 ahci-test /x86_64/ahci/io/pio/lba28/short/high
==8670==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8670==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffcf6892000; bottom 0x7f3b9c9fe000; size: 0x00c159e94000 (830437146624)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 21 ahci-test /x86_64/ahci/io/pio/lba48/simple/zero
==8676==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-rcu-tailq /rcu/qtailq/long-many
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-qdist -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-qdist" 
==8676==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffd6fd68000; bottom 0x7f895c5fe000; size: 0x00741376a000 (498542747648)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-qdist /qdist/none
---
PASS 8 test-qdist /qdist/binning/shrink
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-qht -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-qht" 
PASS 22 ahci-test /x86_64/ahci/io/pio/lba48/simple/low
==8691==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8691==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fff1a6fc000; bottom 0x7fa1267fe000; size: 0x005df3efe000 (403524542464)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 23 ahci-test /x86_64/ahci/io/pio/lba48/simple/high
==8697==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8697==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffc3cbe0000; bottom 0x7fb26fbfe000; size: 0x0049ccfe2000 (316971819008)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 24 ahci-test /x86_64/ahci/io/pio/lba48/double/zero
==8703==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8703==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fffc9de0000; bottom 0x7f0bebffe000; size: 0x00f3ddde2000 (1047399374848)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 25 ahci-test /x86_64/ahci/io/pio/lba48/double/low
==8709==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8709==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe8cc1a000; bottom 0x7fe66bbfe000; size: 0x00182101c000 (103632977920)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 26 ahci-test /x86_64/ahci/io/pio/lba48/double/high
==8715==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8715==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffd3ef76000; bottom 0x7f64fe1fe000; size: 0x009840d78000 (653922893824)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 27 ahci-test /x86_64/ahci/io/pio/lba48/long/zero
==8721==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8721==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe14f82000; bottom 0x7f51515fe000; size: 0x00acc3984000 (742015909888)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 28 ahci-test /x86_64/ahci/io/pio/lba48/long/low
==8727==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8727==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffded245000; bottom 0x7f3a95d24000; size: 0x00c357521000 (838983618560)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 29 ahci-test /x86_64/ahci/io/pio/lba48/long/high
==8733==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 30 ahci-test /x86_64/ahci/io/pio/lba48/short/zero
==8739==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 31 ahci-test /x86_64/ahci/io/pio/lba48/short/low
==8745==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-qht /qht/mode/default
PASS 2 test-qht /qht/mode/resize
PASS 32 ahci-test /x86_64/ahci/io/pio/lba48/short/high
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-qht-par -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-qht-par" 
==8752==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-qht-par /qht/parallel/2threads-0%updates-1s
PASS 33 ahci-test /x86_64/ahci/io/dma/lba28/fragmented
==8774==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 test-qht-par /qht/parallel/2threads-20%updates-1s
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-bitops -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-bitops" 
PASS 34 ahci-test /x86_64/ahci/io/dma/lba28/retry
---
PASS 2 test-qdev-global-props /qdev/properties/static/global
PASS 3 test-qdev-global-props /qdev/properties/dynamic/global
PASS 4 test-qdev-global-props /qdev/properties/global/subclass
==8786==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/check-qom-interface -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="check-qom-interface" 
PASS 1 check-qom-interface /qom/interface/direct_impl
PASS 2 check-qom-interface /qom/interface/intermediate_impl
---
PASS 9 test-keyval /keyval/visit/alternate
PASS 10 test-keyval /keyval/visit/any
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-write-threshold -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-write-threshold" 
==8823==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-write-threshold /write-threshold/not-set-on-init
PASS 2 test-write-threshold /write-threshold/set-get
PASS 3 test-write-threshold /write-threshold/multi-set-get
---
PASS 15 test-crypto-secret /crypto/secret/crypt/missingiv
PASS 16 test-crypto-secret /crypto/secret/crypt/badiv
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-crypto-tlscredsx509 -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-crypto-tlscredsx509" 
==8854==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/perfectserver
PASS 2 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/perfectclient
PASS 3 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodca1
---
PASS 7 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/badca2
PASS 8 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/badca3
PASS 9 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver1
==8864==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 10 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver2
PASS 38 ahci-test /x86_64/ahci/io/dma/lba28/double/zero
==8870==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 11 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver3
PASS 39 ahci-test /x86_64/ahci/io/dma/lba28/double/low
PASS 12 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver4
==8876==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 13 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver5
PASS 40 ahci-test /x86_64/ahci/io/dma/lba28/double/high
==8882==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 14 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver6
PASS 15 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver7
PASS 16 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/badserver1
---
PASS 39 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/missingclient
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-crypto-tlssession -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-crypto-tlssession" 
PASS 41 ahci-test /x86_64/ahci/io/dma/lba28/long/zero
==8893==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-crypto-tlssession /qcrypto/tlssession/psk
PASS 42 ahci-test /x86_64/ahci/io/dma/lba28/long/low
PASS 2 test-crypto-tlssession /qcrypto/tlssession/basicca
==8899==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-crypto-tlssession /qcrypto/tlssession/differentca
PASS 43 ahci-test /x86_64/ahci/io/dma/lba28/long/high
==8905==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 4 test-crypto-tlssession /qcrypto/tlssession/altname1
PASS 5 test-crypto-tlssession /qcrypto/tlssession/altname2
PASS 6 test-crypto-tlssession /qcrypto/tlssession/altname3
PASS 44 ahci-test /x86_64/ahci/io/dma/lba28/short/zero
==8911==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 7 test-crypto-tlssession /qcrypto/tlssession/altname4
PASS 45 ahci-test /x86_64/ahci/io/dma/lba28/short/low
PASS 8 test-crypto-tlssession /qcrypto/tlssession/altname5
==8917==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 9 test-crypto-tlssession /qcrypto/tlssession/altname6
PASS 46 ahci-test /x86_64/ahci/io/dma/lba28/short/high
==8923==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 10 test-crypto-tlssession /qcrypto/tlssession/wildcard1
PASS 47 ahci-test /x86_64/ahci/io/dma/lba48/simple/zero
PASS 11 test-crypto-tlssession /qcrypto/tlssession/wildcard2
==8929==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 12 test-crypto-tlssession /qcrypto/tlssession/wildcard3
PASS 13 test-crypto-tlssession /qcrypto/tlssession/wildcard4
PASS 48 ahci-test /x86_64/ahci/io/dma/lba48/simple/low
PASS 14 test-crypto-tlssession /qcrypto/tlssession/wildcard5
PASS 15 test-crypto-tlssession /qcrypto/tlssession/wildcard6
==8935==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 16 test-crypto-tlssession /qcrypto/tlssession/cachain
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-qga -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-qga" 
PASS 49 ahci-test /x86_64/ahci/io/dma/lba48/simple/high
==8947==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-qga /qga/sync-delimited
PASS 2 test-qga /qga/sync
PASS 3 test-qga /qga/ping
---
PASS 16 test-qga /qga/invalid-args
PASS 17 test-qga /qga/fsfreeze-status
PASS 50 ahci-test /x86_64/ahci/io/dma/lba48/double/zero
==8955==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 18 test-qga /qga/blacklist
PASS 19 test-qga /qga/config
PASS 20 test-qga /qga/guest-exec
PASS 21 test-qga /qga/guest-exec-invalid
PASS 51 ahci-test /x86_64/ahci/io/dma/lba48/double/low
==8968==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 22 test-qga /qga/guest-get-osinfo
PASS 23 test-qga /qga/guest-get-host-name
PASS 24 test-qga /qga/guest-get-timezone
---
PASS 7 test-util-sockets /socket/fd-pass/num/bad
PASS 8 test-util-sockets /socket/fd-pass/num/nocli
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-authz-simple -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-authz-simple" 
==8988==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-authz-simple /authz/simple
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-authz-list -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-authz-list" 
PASS 1 test-authz-list /auth/list/complex
---
PASS 4 test-io-channel-file /io/channel/pipe/sync
PASS 5 test-io-channel-file /io/channel/pipe/async
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-io-channel-tls -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-io-channel-tls" 
==9023==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 54 ahci-test /x86_64/ahci/io/dma/lba48/long/low
==9080==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-io-channel-tls /qio/channel/tls/basic
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-io-channel-command -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-io-channel-command" 
PASS 1 test-io-channel-command /io/channel/command/fifo/sync
---
PASS 55 ahci-test /x86_64/ahci/io/dma/lba48/long/high
PASS 1 test-io-channel-buffer /io/channel/buf
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-base64 -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-base64" 
==9101==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-base64 /util/base64/good
PASS 2 test-base64 /util/base64/embedded-nul
PASS 3 test-base64 /util/base64/not-nul-terminated
---
PASS 17 test-crypto-xts /crypto/xts/t-21-key-32-ptx-31/basic
PASS 18 test-crypto-xts /crypto/xts/t-21-key-32-ptx-31/unaligned
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-crypto-block -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-crypto-block" 
==9131==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-crypto-block /crypto/block/qcow
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-logging -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-logging" 
PASS 1 test-logging /logging/parse_range
PASS 2 test-logging /logging/parse_path
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-replication -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-replication" 
==9147==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 57 ahci-test /x86_64/ahci/io/dma/lba48/short/low
PASS 1 test-replication /replication/primary/read
PASS 2 test-replication /replication/primary/write
==9151==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-replication /replication/primary/start
PASS 4 test-replication /replication/primary/stop
PASS 5 test-replication /replication/primary/do_checkpoint
PASS 6 test-replication /replication/primary/get_error_all
PASS 58 ahci-test /x86_64/ahci/io/dma/lba48/short/high
==9157==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 7 test-replication /replication/secondary/read
PASS 59 ahci-test /x86_64/ahci/io/ncq/simple
PASS 8 test-replication /replication/secondary/write
==9163==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 60 ahci-test /x86_64/ahci/io/ncq/retry
==9169==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==9147==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe429f8000; bottom 0x7f4af5ffc000; size: 0x00b34c9fc000 (770084683776)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 61 ahci-test /x86_64/ahci/flush/simple
PASS 9 test-replication /replication/secondary/start
==9196==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 62 ahci-test /x86_64/ahci/flush/retry
==9202==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==9207==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
/tmp/qemu-test/src/migration/ram.c:3522:53: runtime error: member access within null pointer of type 'struct (anonymous struct at /tmp/qemu-test/src/migration/ram.c:901:1)'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /tmp/qemu-test/src/migration/ram.c:3522:53 in 
/tmp/qemu-test/src/migration/ram.c:3522:53: runtime error: load of null pointer of type 'MultiFDSendParams *'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /tmp/qemu-test/src/migration/ram.c:3522:53 in 
AddressSanitizer:DEADLYSIGNAL
=================================================================
==9202==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x560adc79b7ca bp 0x7f76d1dfb090 sp 0x7f76d1dfae60 T6)
==9202==The signal is caused by a READ memory access.
==9202==Hint: address points to the zero page.
PASS 10 test-replication /replication/secondary/stop
---
qemu-system-x86_64: Not a migration stream
qemu-system-x86_64: load of migration failed: Invalid argument
/tmp/qemu-test/src/tests/libqtest.c:137: kill_qemu() tried to terminate QEMU process but encountered exit status 1
ERROR - too few tests run (expected 74, got 62)
make: *** [/tmp/qemu-test/src/tests/Makefile.include:899: check-qtest-x86_64] Error 1
make: *** Waiting for unfinished jobs....
PASS 11 test-replication /replication/secondary/do_checkpoint


The full log is available at
http://patchew.org/logs/20190814020218.1868-1-quintela@redhat.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [Qemu-devel] [PATCH 0/6] Fix multifd with big number of channels

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> Hi
> 
> When we have much more channels than cpus, we end having failures when
> writting to sockets. This series:
> - add some traces
> - fix some of the trouble with serialization of creating the
>   threads/channels in proper order.
> - Ask for help with the last patch.  See documentation there.
> 
> Please, review.

patchew is reporting a failure that looks like it came from one of
these; but I'm not sure which one yet!

> 
> Juan Quintela (6):
>   migration: Add traces for multifd terminate threads
>   migration: Make global sem_sync semaphore by channel
>   migration: Make sure that all multifd channels have been created
>   migration: Make multifd threads wait until all have been created
>   migration: add some multifd traces
>   RFH: We lost "connect" events
> 
>  migration/ram.c        | 60 +++++++++++++++++++++++++++++++++++-------
>  migration/trace-events |  8 ++++++
>  2 files changed, 59 insertions(+), 9 deletions(-)
> 
> -- 
> 2.21.0
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 0/6] Fix multifd with big number of channels

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190814020218.1868-1-quintela@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

PASS 1 fdc-test /x86_64/fdc/cmos
PASS 2 fdc-test /x86_64/fdc/no_media_on_start
PASS 3 fdc-test /x86_64/fdc/read_without_media
==7932==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 4 fdc-test /x86_64/fdc/media_change
PASS 5 fdc-test /x86_64/fdc/sense_interrupt
PASS 6 fdc-test /x86_64/fdc/relative_seek
---
PASS 32 test-opts-visitor /visitor/opts/range/beyond
PASS 33 test-opts-visitor /visitor/opts/dict/unvisited
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-coroutine -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-coroutine" 
==7977==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==7977==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffec6185000; bottom 0x7fb9ac9f8000; size: 0x00451978d000 (296780091392)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-coroutine /basic/no-dangling-access
---
PASS 11 test-aio /aio/event/wait
PASS 12 test-aio /aio/event/flush
PASS 13 test-aio /aio/event/wait/no-flush-cb
==7996==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 14 test-aio /aio/timer/schedule
PASS 15 test-aio /aio/coroutine/queue-chaining
PASS 16 test-aio /aio-gsource/flush
---
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img tests/ide-test -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="ide-test" 
PASS 28 test-aio /aio-gsource/timer/schedule
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-aio-multithread -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-aio-multithread" 
==8005==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8011==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-aio-multithread /aio/multi/lifecycle
PASS 1 ide-test /x86_64/ide/identify
==8026==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 test-aio-multithread /aio/multi/schedule
PASS 2 ide-test /x86_64/ide/flush
==8037==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-aio-multithread /aio/multi/mutex/contended
PASS 3 ide-test /x86_64/ide/bmdma/simple_rw
==8048==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 4 ide-test /x86_64/ide/bmdma/trim
==8054==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 5 ide-test /x86_64/ide/bmdma/short_prdt
==8060==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 4 test-aio-multithread /aio/multi/mutex/handoff
PASS 6 ide-test /x86_64/ide/bmdma/one_sector_short_prdt
==8071==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 5 test-aio-multithread /aio/multi/mutex/mcs
PASS 7 ide-test /x86_64/ide/bmdma/long_prdt
==8082==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8082==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe85f30000; bottom 0x7f578fb66000; size: 0x00a6f63ca000 (717095739392)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 6 test-aio-multithread /aio/multi/mutex/pthread
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-throttle -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-throttle" 
PASS 8 ide-test /x86_64/ide/bmdma/no_busmaster
==8090==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-throttle /throttle/leak_bucket
PASS 2 test-throttle /throttle/compute_wait
PASS 3 test-throttle /throttle/init
---
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-thread-pool -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-thread-pool" 
PASS 1 test-thread-pool /thread-pool/submit
PASS 2 test-thread-pool /thread-pool/submit-aio
==8096==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-thread-pool /thread-pool/submit-co
PASS 4 test-thread-pool /thread-pool/submit-many
PASS 9 ide-test /x86_64/ide/flush/nodev
==8167==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 10 ide-test /x86_64/ide/flush/empty_drive
PASS 5 test-thread-pool /thread-pool/cancel
==8172==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 11 ide-test /x86_64/ide/flush/retry_pci
==8179==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 12 ide-test /x86_64/ide/flush/retry_isa
PASS 6 test-thread-pool /thread-pool/cancel-async
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-hbitmap -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-hbitmap" 
==8185==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-hbitmap /hbitmap/granularity
PASS 2 test-hbitmap /hbitmap/size/0
PASS 3 test-hbitmap /hbitmap/size/unaligned
---
PASS 10 test-hbitmap /hbitmap/set/all
PASS 11 test-hbitmap /hbitmap/set/one
PASS 12 test-hbitmap /hbitmap/set/two-elem
==8196==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 13 test-hbitmap /hbitmap/set/general
PASS 14 test-hbitmap /hbitmap/set/twice
PASS 15 test-hbitmap /hbitmap/set/overlap
---
PASS 29 test-hbitmap /hbitmap/truncate/shrink/large
PASS 30 test-hbitmap /hbitmap/meta/zero
PASS 14 ide-test /x86_64/ide/cdrom/pio_large
==8202==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 15 ide-test /x86_64/ide/cdrom/dma
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img tests/ahci-test -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="ahci-test" 
==8216==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 ahci-test /x86_64/ahci/sanity
==8222==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 ahci-test /x86_64/ahci/pci_spec
==8228==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 31 test-hbitmap /hbitmap/meta/one
PASS 32 test-hbitmap /hbitmap/meta/byte
PASS 33 test-hbitmap /hbitmap/meta/word
PASS 3 ahci-test /x86_64/ahci/pci_enable
==8234==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 34 test-hbitmap /hbitmap/meta/sector
PASS 35 test-hbitmap /hbitmap/serialize/align
PASS 4 ahci-test /x86_64/ahci/hba_spec
==8240==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 5 ahci-test /x86_64/ahci/hba_enable
PASS 36 test-hbitmap /hbitmap/serialize/basic
==8246==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 37 test-hbitmap /hbitmap/serialize/part
PASS 38 test-hbitmap /hbitmap/serialize/zeroes
PASS 39 test-hbitmap /hbitmap/next_zero/next_zero_0
---
PASS 42 test-hbitmap /hbitmap/next_dirty_area/next_dirty_area_1
PASS 43 test-hbitmap /hbitmap/next_dirty_area/next_dirty_area_4
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-bdrv-drain -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-bdrv-drain" 
==8254==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-bdrv-drain /bdrv-drain/nested
PASS 2 test-bdrv-drain /bdrv-drain/multiparent
PASS 3 test-bdrv-drain /bdrv-drain/set_aio_context
---
PASS 38 test-bdrv-drain /bdrv-drain/detach/parent_cb
PASS 39 test-bdrv-drain /bdrv-drain/detach/driver_cb
PASS 40 test-bdrv-drain /bdrv-drain/attach/drain
==8272==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-bdrv-graph-mod -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-bdrv-graph-mod" 
==8299==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-bdrv-graph-mod /bdrv-graph-mod/update-perm-tree
PASS 2 test-bdrv-graph-mod /bdrv-graph-mod/should-update-child
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-blockjob -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-blockjob" 
PASS 7 ahci-test /x86_64/ahci/max
==8305==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-blockjob /blockjob/ids
PASS 2 test-blockjob /blockjob/cancel/created
PASS 3 test-blockjob /blockjob/cancel/running
---
PASS 7 test-blockjob /blockjob/cancel/pending
PASS 8 test-blockjob /blockjob/cancel/concluded
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-blockjob-txn -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-blockjob-txn" 
==8311==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8307==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-blockjob-txn /single/success
PASS 2 test-blockjob-txn /single/failure
PASS 3 test-blockjob-txn /single/cancel
---
PASS 6 test-blockjob-txn /pair/cancel
PASS 7 test-blockjob-txn /pair/fail-cancel-race
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-block-backend -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-block-backend" 
==8320==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-block-backend /block-backend/drain_aio_error
PASS 2 test-block-backend /block-backend/drain_all_aio_error
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-block-iothread -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-block-iothread" 
==8325==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-block-iothread /sync-op/pread
PASS 2 test-block-iothread /sync-op/pwrite
PASS 3 test-block-iothread /sync-op/load_vmstate
---
PASS 16 test-block-iothread /propagate/mirror
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-image-locking -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-image-locking" 
PASS 8 ahci-test /x86_64/ahci/reset
==8347==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-image-locking /image-locking/basic
PASS 2 test-image-locking /image-locking/set-perm-abort
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-x86-cpuid -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-x86-cpuid" 
==8349==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-x86-cpuid /cpuid/topology/basic
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-xbzrle -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-xbzrle" 
==8349==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffd0de5a000; bottom 0x7fa797ffe000; size: 0x005575e5c000 (367050211328)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-xbzrle /xbzrle/uleb
---
PASS 16 test-vmstate /vmstate/qtailq/save/saveq
PASS 17 test-vmstate /vmstate/qtailq/load/loadq
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-cutils -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-cutils" 
==8364==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-cutils /cutils/parse_uint/null
PASS 2 test-cutils /cutils/parse_uint/empty
PASS 3 test-cutils /cutils/parse_uint/whitespace
---
PASS 133 test-cutils /cutils/strtosz/erange
PASS 134 test-cutils /cutils/strtosz/metric
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-shift128 -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-shift128" 
==8364==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fffacd18000; bottom 0x7fbdb23fe000; size: 0x0041fa91a000 (283376721920)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-shift128 /host-utils/test_lshift
---
PASS 9 test-int128 /int128/int128_gt
PASS 10 test-int128 /int128/int128_rshift
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/rcutorture -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="rcutorture" 
==8388==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8388==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe0e1bd000; bottom 0x7fbd0cbfe000; size: 0x0041015bf000 (279195676672)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 11 ahci-test /x86_64/ahci/io/pio/lba28/simple/high
PASS 1 rcutorture /rcu/torture/1reader
==8409==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8409==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffd95562000; bottom 0x7f95963fe000; size: 0x0067ff164000 (446661279744)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 2 rcutorture /rcu/torture/10readers
PASS 12 ahci-test /x86_64/ahci/io/pio/lba28/double/zero
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-rcu-list -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-rcu-list" 
==8432==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8432==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffede9c0000; bottom 0x7fc2d3dfe000; size: 0x003c0abc2000 (257878138880)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 13 ahci-test /x86_64/ahci/io/pio/lba28/double/low
PASS 1 test-rcu-list /rcu/qlist/single-threaded
==8444==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8444==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffd71ba7000; bottom 0x7f0c0d9fe000; size: 0x00f1641a9000 (1036766580736)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 14 ahci-test /x86_64/ahci/io/pio/lba28/double/high
PASS 2 test-rcu-list /rcu/qlist/short-few
==8456==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8456==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe8a00e000; bottom 0x7f8c9e3fe000; size: 0x0071ebc10000 (489286598656)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 15 ahci-test /x86_64/ahci/io/pio/lba28/long/zero
==8483==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-rcu-list /rcu/qlist/long-many
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-rcu-simpleq -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-rcu-simpleq" 
==8483==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe47c79000; bottom 0x7fa58cbfe000; size: 0x0058bb07b000 (381094965248)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 16 ahci-test /x86_64/ahci/io/pio/lba28/long/low
PASS 1 test-rcu-simpleq /rcu/qsimpleq/single-threaded
==8496==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8496==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffe1ba6b000; bottom 0x7f787677c000; size: 0x0085a52ef000 (574001967104)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 2 test-rcu-simpleq /rcu/qsimpleq/short-few
PASS 17 ahci-test /x86_64/ahci/io/pio/lba28/long/high
==8529==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 18 ahci-test /x86_64/ahci/io/pio/lba28/short/zero
==8535==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-rcu-simpleq /rcu/qsimpleq/long-many
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-rcu-tailq -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-rcu-tailq" 
PASS 19 ahci-test /x86_64/ahci/io/pio/lba28/short/low
==8548==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-rcu-tailq /rcu/qtailq/single-threaded
PASS 20 ahci-test /x86_64/ahci/io/pio/lba28/short/high
==8560==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 test-rcu-tailq /rcu/qtailq/short-few
==8560==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffcbb7ce000; bottom 0x7fb0221fe000; size: 0x004c995d0000 (328990523392)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 21 ahci-test /x86_64/ahci/io/pio/lba48/simple/zero
==8587==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8587==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffd5bd35000; bottom 0x7f3c24bfe000; size: 0x00c137137000 (829852708864)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 22 ahci-test /x86_64/ahci/io/pio/lba48/simple/low
PASS 3 test-rcu-tailq /rcu/qtailq/long-many
==8593==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-qdist -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-qdist" 
==8593==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffcab196000; bottom 0x7f50bb3fe000; size: 0x00abefd98000 (738463416320)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-qdist /qdist/none
---
PASS 8 test-qdist /qdist/binning/shrink
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-qht -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-qht" 
PASS 23 ahci-test /x86_64/ahci/io/pio/lba48/simple/high
==8608==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8608==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffca2554000; bottom 0x7f2bef9fe000; size: 0x00d0b2b56000 (896351428608)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 24 ahci-test /x86_64/ahci/io/pio/lba48/double/zero
==8614==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8614==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffeae3be000; bottom 0x7ff5817fe000; size: 0x00092cbc0000 (39405223936)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 25 ahci-test /x86_64/ahci/io/pio/lba48/double/low
==8620==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8620==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffc3caeb000; bottom 0x7f87257fe000; size: 0x0075172ed000 (502900117504)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 26 ahci-test /x86_64/ahci/io/pio/lba48/double/high
==8626==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8626==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffdd39f9000; bottom 0x7f77d9ffe000; size: 0x0085f99fb000 (575418642432)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 27 ahci-test /x86_64/ahci/io/pio/lba48/long/zero
==8632==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8632==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fff9b022000; bottom 0x7f717397c000; size: 0x008e276a6000 (610546638848)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 28 ahci-test /x86_64/ahci/io/pio/lba48/long/low
==8638==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==8638==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffc5b44f000; bottom 0x7f8eb8dfe000; size: 0x006da2651000 (470875967488)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 29 ahci-test /x86_64/ahci/io/pio/lba48/long/high
==8644==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 30 ahci-test /x86_64/ahci/io/pio/lba48/short/zero
==8650==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 31 ahci-test /x86_64/ahci/io/pio/lba48/short/low
==8656==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 32 ahci-test /x86_64/ahci/io/pio/lba48/short/high
==8662==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 33 ahci-test /x86_64/ahci/io/dma/lba28/fragmented
==8668==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-qht /qht/mode/default
PASS 34 ahci-test /x86_64/ahci/io/dma/lba28/retry
PASS 2 test-qht /qht/mode/resize
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-qht-par -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-qht-par" 
==8674==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 35 ahci-test /x86_64/ahci/io/dma/lba28/simple/zero
PASS 1 test-qht-par /qht/parallel/2threads-0%updates-1s
==8690==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 36 ahci-test /x86_64/ahci/io/dma/lba28/simple/low
PASS 2 test-qht-par /qht/parallel/2threads-20%updates-1s
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-bitops -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-bitops" 
==8703==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-bitops /bitops/sextract32
PASS 2 test-bitops /bitops/sextract64
PASS 3 test-bitops /bitops/half_shuffle32
---
PASS 1 check-qom-interface /qom/interface/direct_impl
PASS 2 check-qom-interface /qom/interface/intermediate_impl
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/check-qom-proplist -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="check-qom-proplist" 
==8733==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 check-qom-proplist /qom/proplist/createlist
PASS 2 check-qom-proplist /qom/proplist/createv
PASS 3 check-qom-proplist /qom/proplist/createcmdline
---
PASS 3 test-write-threshold /write-threshold/multi-set-get
PASS 4 test-write-threshold /write-threshold/not-trigger
PASS 5 test-write-threshold /write-threshold/trigger
==8752==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-crypto-hash -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-crypto-hash" 
PASS 1 test-crypto-hash /crypto/hash/iov
PASS 2 test-crypto-hash /crypto/hash/alloc
---
PASS 15 test-crypto-secret /crypto/secret/crypt/missingiv
PASS 16 test-crypto-secret /crypto/secret/crypt/badiv
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-crypto-tlscredsx509 -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-crypto-tlscredsx509" 
==8781==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/perfectserver
PASS 2 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/perfectclient
PASS 40 ahci-test /x86_64/ahci/io/dma/lba28/double/high
==8793==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 3 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodca1
PASS 41 ahci-test /x86_64/ahci/io/dma/lba28/long/zero
==8799==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 4 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodca2
PASS 5 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodca3
PASS 6 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/badca1
PASS 7 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/badca2
PASS 8 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/badca3
PASS 42 ahci-test /x86_64/ahci/io/dma/lba28/long/low
==8805==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 9 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver1
PASS 10 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver2
PASS 43 ahci-test /x86_64/ahci/io/dma/lba28/long/high
==8811==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 11 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver3
PASS 12 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver4
PASS 44 ahci-test /x86_64/ahci/io/dma/lba28/short/zero
==8817==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 45 ahci-test /x86_64/ahci/io/dma/lba28/short/low
==8823==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 13 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver5
PASS 46 ahci-test /x86_64/ahci/io/dma/lba28/short/high
PASS 14 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/goodserver6
---
PASS 32 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/inactive1
PASS 33 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/inactive2
PASS 34 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/inactive3
==8829==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 35 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/chain1
PASS 36 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/chain2
PASS 37 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/missingca
---
PASS 39 test-crypto-tlscredsx509 /qcrypto/tlscredsx509/missingclient
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-crypto-tlssession -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-crypto-tlssession" 
PASS 47 ahci-test /x86_64/ahci/io/dma/lba48/simple/zero
==8840==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 48 ahci-test /x86_64/ahci/io/dma/lba48/simple/low
PASS 1 test-crypto-tlssession /qcrypto/tlssession/psk
==8846==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 2 test-crypto-tlssession /qcrypto/tlssession/basicca
PASS 3 test-crypto-tlssession /qcrypto/tlssession/differentca
PASS 49 ahci-test /x86_64/ahci/io/dma/lba48/simple/high
PASS 4 test-crypto-tlssession /qcrypto/tlssession/altname1
==8852==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 50 ahci-test /x86_64/ahci/io/dma/lba48/double/zero
PASS 5 test-crypto-tlssession /qcrypto/tlssession/altname2
==8858==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 6 test-crypto-tlssession /qcrypto/tlssession/altname3
PASS 7 test-crypto-tlssession /qcrypto/tlssession/altname4
PASS 51 ahci-test /x86_64/ahci/io/dma/lba48/double/low
==8864==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 8 test-crypto-tlssession /qcrypto/tlssession/altname5
PASS 9 test-crypto-tlssession /qcrypto/tlssession/altname6
PASS 52 ahci-test /x86_64/ahci/io/dma/lba48/double/high
==8870==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 10 test-crypto-tlssession /qcrypto/tlssession/wildcard1
PASS 11 test-crypto-tlssession /qcrypto/tlssession/wildcard2
PASS 53 ahci-test /x86_64/ahci/io/dma/lba48/long/zero
PASS 12 test-crypto-tlssession /qcrypto/tlssession/wildcard3
==8876==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 13 test-crypto-tlssession /qcrypto/tlssession/wildcard4
PASS 54 ahci-test /x86_64/ahci/io/dma/lba48/long/low
==8882==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 14 test-crypto-tlssession /qcrypto/tlssession/wildcard5
PASS 15 test-crypto-tlssession /qcrypto/tlssession/wildcard6
PASS 55 ahci-test /x86_64/ahci/io/dma/lba48/long/high
==8888==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 56 ahci-test /x86_64/ahci/io/dma/lba48/short/zero
PASS 16 test-crypto-tlssession /qcrypto/tlssession/cachain
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-qga -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-qga" 
==8894==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 57 ahci-test /x86_64/ahci/io/dma/lba48/short/low
PASS 1 test-qga /qga/sync-delimited
PASS 2 test-qga /qga/sync
---
PASS 7 test-qga /qga/get-fsinfo
PASS 8 test-qga /qga/get-memory-block-info
PASS 9 test-qga /qga/get-memory-blocks
==8906==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 10 test-qga /qga/file-ops
PASS 11 test-qga /qga/file-write-read
PASS 12 test-qga /qga/get-time
---
PASS 58 ahci-test /x86_64/ahci/io/dma/lba48/short/high
PASS 18 test-qga /qga/blacklist
PASS 19 test-qga /qga/config
==8913==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 20 test-qga /qga/guest-exec
PASS 21 test-qga /qga/guest-exec-invalid
PASS 59 ahci-test /x86_64/ahci/io/ncq/simple
==8927==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 22 test-qga /qga/guest-get-osinfo
PASS 23 test-qga /qga/guest-get-host-name
PASS 24 test-qga /qga/guest-get-timezone
---
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-authz-simple -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-authz-simple" 
PASS 1 test-authz-simple /authz/simple
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-authz-list -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-authz-list" 
==8951==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-authz-list /auth/list/complex
PASS 2 test-authz-list /auth/list/add-remove
PASS 3 test-authz-list /auth/list/default/deny
---
PASS 8 test-io-channel-socket /io/channel/socket/unix-fd-pass
PASS 9 test-io-channel-socket /io/channel/socket/unix-listen-cleanup
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-io-channel-file -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-io-channel-file" 
==8972==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-io-channel-file /io/channel/file
PASS 2 test-io-channel-file /io/channel/file/rdwr
PASS 3 test-io-channel-file /io/channel/file/fd
---
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-io-channel-buffer -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-io-channel-buffer" 
PASS 1 test-io-channel-buffer /io/channel/buf
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-base64 -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-base64" 
==9047==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-base64 /util/base64/good
PASS 2 test-base64 /util/base64/embedded-nul
PASS 3 test-base64 /util/base64/not-nul-terminated
---
PASS 8 test-crypto-ivgen /crypto/ivgen/essiv/1f2e3d4c
PASS 9 test-crypto-ivgen /crypto/ivgen/essiv/1f2e3d4c5b6a7988
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-crypto-afsplit -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-crypto-afsplit" 
==9066==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
PASS 1 test-crypto-afsplit /crypto/afsplit/sha256/5
PASS 2 test-crypto-afsplit /crypto/afsplit/sha256/5000
PASS 3 test-crypto-afsplit /crypto/afsplit/sha256/big
---
PASS 17 test-crypto-xts /crypto/xts/t-21-key-32-ptx-31/basic
PASS 18 test-crypto-xts /crypto/xts/t-21-key-32-ptx-31/unaligned
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-crypto-block -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-crypto-block" 
/tmp/qemu-test/src/migration/ram.c:3522:53: runtime error: member access within null pointer of type 'struct (anonymous struct at /tmp/qemu-test/src/migration/ram.c:901:1)'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /tmp/qemu-test/src/migration/ram.c:3522:53 in 
/tmp/qemu-test/src/migration/ram.c:3522:53: runtime error: load of null pointer of type 'MultiFDSendParams *'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /tmp/qemu-test/src/migration/ram.c:3522:53 in 
AddressSanitizer:DEADLYSIGNAL
=================================================================
==9047==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x5622032f87ca bp 0x7f71c9bfb090 sp 0x7f71c9bfae60 T6)
==9047==The signal is caused by a READ memory access.
==9047==Hint: address points to the zero page.
PASS 1 test-crypto-block /crypto/block/qcow
---
PASS 1 test-logging /logging/parse_range
PASS 2 test-logging /logging/parse_path
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-replication -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-replication" 
==9101==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
    #0 0x5622032f87c9 in ram_save_setup /tmp/qemu-test/src/migration/ram.c
    #1 0x562204c4630d in qemu_savevm_state_setup /tmp/qemu-test/src/migration/savevm.c:1106:15
    #2 0x562204c1edd0 in migration_thread /tmp/qemu-test/src/migration/migration.c:3219:5
---
PASS 1 test-replication /replication/primary/read
PASS 2 test-replication /replication/primary/write
/tmp/qemu-test/src/tests/libqtest.c:137: kill_qemu() tried to terminate QEMU process but encountered exit status 1
ERROR - too few tests run (expected 74, got 62)
make: *** [/tmp/qemu-test/src/tests/Makefile.include:899: check-qtest-x86_64] Error 1
make: *** Waiting for unfinished jobs....
PASS 3 test-replication /replication/primary/start
---
PASS 6 test-replication /replication/primary/get_error_all
PASS 7 test-replication /replication/secondary/read
PASS 8 test-replication /replication/secondary/write
==9101==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffd4d1f1000; bottom 0x7fe7c97fc000; size: 0x0015839f5000 (92402569216)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 9 test-replication /replication/secondary/start


The full log is available at
http://patchew.org/logs/20190814020218.1868-1-quintela@redhat.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com