All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v14 00/12] Multifd
@ 2018-06-20  8:15 Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 01/12] migration: Create multipage support Juan Quintela
                   ` (12 more replies)
  0 siblings, 13 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

Hi

THis iv v14 multifd patches: Changes from previous submit:
- rename seq -> packet_num: make things easier to understand
- packet_num is now 64bit wide
- include the size of the packet headers in the transfer stats (dave noticed it)
- improve comments here and there.

All the patches except two are already reviewed-by.  And my
understanding is that I fixed last issues with two remaining ones, so
I am expect to pull this when this two patches are reviewed.

Please review.

Thanks, Juan.



This is v13 of multifd patches:

- several patches already integrated
- rebased to latests upstreams
- addressed all the reviews comments around.

Please review.

Thanks, Juan.

[v12]

Big news, it is not RFC anymore, it works reliabely for me.

Changes:
- Locknig changed completely (several times)
- We now send  all pages through the channels.  In a 2GB guest with 1 disk and a network card, the amount of data send for RAM was 80KB.
- This is not optimized yet, but it shouws clear improvements over precopy.  testing over localhost networking I can guet:
  - 2 VCPUs guest
  - 2GB RAM
  - runn stress --vm 4 --vm 500GB (i.e. dirtying 2GB or RAM each second)

  - Total time: precopy ~50seconds, multifd  around 11seconds
  - Bandwidth usage is around 273MB/s vs 71MB/s on the same hardware

This is very preleminary testing, will send more numbers when I got them.  But looks promissing.

Things that will be improved later:
- Initial synchronization is too slow (around 1s)
- We synchronize all threads after each RAM section, we can move to only
  synchronize them after we have done a bitmap syncrhronization
- We can improve bitmap walking (but that is independent of multifd)

Please review.

Later, Juan.


[v11]

Changes on top of previous sumbimission:
- Now on top of migration-tests/v6 that I sent on Wednesday
- Rebased to latest upstream
- Everything that is sent through the network should be converted correctly
  (famous last words)
- Still on RFC (sometimes it ends some packets at the end), just to
  show how things are going on.  Problems are only on the last patch.

- Redo some locking (again) Now the problem is being able te send the
  synchronization through the multifd channels.  I end the migration
  _before_ all the channels have recevied all the packets.

- Trying to get a flags argument into each packet, to be able to synchronze
  through the network, not from the "main" incoming corroutine.

- Related to the network-safe fields, now everything is in its own
  routine, it should be easier to understand/review.  Once there, I
  check that all values are inside range.

So, please comment.

Thanks, Juan.


[v10]
Lots of changes from previous versions:
a - everything is sent now through the multifd channels, nothing is sent through main channel
b - locking is band new, I was getting into a hole with the previous approach, right now, there is a single way to
    do locking (both source and destination)
       main thread : sets a ->sync variable for each thread and wakeps it
       multifd threads: clean the variable and signal (sem) back to main thread

    using this for either:
    - all threads have started
    - we need to synchronize after each round through memory
    - all threads have finished

c - I have to use a qio watcher for a thread to wait for ready data to read

d - lots of cleanups

e - to make things easier, I have included the missing tests stuff on
    this round of patches, because they build on top of them

f - lots of traces, it is now much easier to follow what is happening

Now, why it is an RFC:

- in the last patch, there is still race between the whatcher, the
  ->quit of the threads and the last synchronization.  Techinically they
  are done in oder, but in practice, they are hanging sometimes.

- I *know* I can optimize the synchronization of the threads sending
  the "we start a new round" through the multifd channels, have to add a flag here.

- Not having a thread on the incoming side  is a mess, I can't block waiting for things to happen :-(

- When doing the synchronization, I need to optimize the sending of the "not finished packet" of pages, working on that.

please, take a look and review.

Thanks, Juan.

[v9]

This series is on top of my migration test series just sent, only reject should be on the test system, though.

On v9 series for you:
- qobject_unref() as requested by dan

  Yes he was right, I had a reference leak for _non_ multifd, I
  *thought* he mean for multifd, and that took a while to understand
  (and then find when/where).

- multifd page count: it is dropped for good
- uuid handling: we use the default qemu uuid of 0000...
- uuid handling: using and struct and sending the struct
  * idea is to add a size field and add more parameter after that
  * anyone has a good idea how to "ouptut" info
    migrate_capabilities/parameters json into a string and how to read it back?
- changed how we test that all threads/channels are already created.
  Should be more robust.
- Add tests multifd.  Still not ported on top of migration-tests series sent early
  waiting for review on the ideas there.
- Rebase and remove al the integrated patches (back at 12)

Please, review.

Later, Juan.

[v8]
Things NOT done yet:

- drop x-multifd-page-count?  We can use performance to set a default value
- paolo suggestion of not having a control channel
  needs iyet more cleanups to be able to have more than one ramstate, trying it.
- still not performance done, but it has been very stable

On v8:
- use connect_async
- rename multifd-group to multifd-page-count (danp suggestion)
- rename multifd-threads to multifd-channels (danp suggestion)
- use new qio*channel functions
- Address rest of comments left


So, please review.

My idea will be to pull this changes and continue performance changes
for inside, basically everything is already reviewed.

Thanks, Juan.

On v7:
- tests fixed as danp wanted
- have to revert danp qio_*_all patches, as they break multifd, I have to investigate why.
- error_abort is gone.  After several tries about getting errors, I ended having a single error
  proceted by a lock and first error wins.
- Addressed basically all reviews (see on ToDo)
- Pointers to struct are done now
- fix lots of leaks
- lots of small fixes


[v6]
- Improve migration_ioc_porcess_incoming
- teach about G_SOURCE_REMOVE/CONTINUE
- Add test for migration_has_all_channels
- use DEFIN_PROP*
- change recv_state to use pointers to parameters
  make easier to receive channels out of order
- use g_strdup_printf()
- improve count of threads to know when we have to finish
- report channel id's on errors
- Use last_page parameter for multifd_send_page() sooner
- Improve commets for address
- use g_new0() instead of g_malloc()
- create MULTIFD_CONTINUE instead of using UINT16_MAX
- clear memory used by group of pages
  once there, pass everything to the global state variables instead of being
  local to the function.  This way it works if we cancel migration and start
  a new one
- Really wait to create the migration_thread until all channels are created
- split initial_bytes setup to make clearer following patches.
- createRAM_SAVE_FLAG_MULTIFD_SYNC macro, to make clear what we are doing
- move setting of need_flush to inside bitmap_sync
- Lots of other small changes & reorderings

Please, comment.


[v5]

- tests from qio functions (a.k.a. make danp happy)
- 1st message from one channel to the other contains:
   <uuid> multifd <channel number>
   This would allow us to create more channels as we want them.
   a.k.a. Making dave happy
- Waiting in reception for new channels using qio listeners
  Getting threads, qio and reference counters working at the same time
  was interesing.
  Another make danp happy.

- Lots and lots of small changes and fixes.  Notice that the last 70 patches
  that I merged or so what to make this series easier/smaller.

- NOT DONE: I haven't been woring on measuring performance
  differences, this was about getting the creation of the
  threads/channels right.

So, what I want:

- Are people happy with how I have (ab)used qio channels? (yes danp,
  that is you).
- My understanding is th

ToDo:

- Make paolo happy: He wanted to test using control information
  through each channel, not only pages.  This requires yet more
  cleanups to be able to have more than one QEMUFile/RAMState open at
  the same time.

- How I create multiple channels.  Things I know:
  * with current changes, it should work with fd/channels (the multifd bits),
    but we don;t have a way to pass multiple fd;s or exec files.
    Danp, any idea about how to create an UI for it?
  * My idea is that we would split current code to be:
    + channel creation at migration.c
    + rest of bits at ram.c
    + change format to:
      <uuid> main <rest of migration capabilities/paramentes> so we can check
      <uuid> postcopy <no clue what parameters are needed>
          Dave wanted a way to create a new fd for postcopy for some time
    + Adding new channels is easy

- Performance data/numbers: Yes, I wanted to get this out at once, I
  would continue with this.


Please, review.


[v4]
This is the 4th version of multifd. Changes:
- XBZRLE don't need to be checked for
- Documentation and defaults are consistent
- split socketArgs
- use iovec instead of creating something similar.
- We use now the exported size of target page (another HACK removal)
- created qio_chanel_{wirtev,readv}_all functions.  the _full() name
  was already taken.
  What they do is the same that the without _all() function, but if it
  returns due to blocking it redo the call.
- it is checkpatch.pl clean now.

Please comment, Juan.

Juan Quintela (12):
  migration: Create multipage support
  migration: Create multifd packet
  migration: Add multifd traces for start/end thread
  migration: Calculate transferred ram correctly
  migration: Multifd channels always wait on the sem
  migration: Add block where to send/receive packets
  migration: Synchronize multifd threads with main thread
  migration: Create ram_save_multifd_page
  migration: Start sending messages
  migration: Wait for blocking IO
  migration: Remove not needed semaphore and quit
  migration: Stop sending whole pages through main channel

 migration/migration.c  |  12 +-
 migration/ram.c        | 498 +++++++++++++++++++++++++++++++++++++++--
 migration/ram.h        |   1 +
 migration/trace-events |  12 +
 4 files changed, 505 insertions(+), 18 deletions(-)

-- 
2.17.1

^ permalink raw reply	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 01/12] migration: Create multipage support
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 02/12] migration: Create multifd packet Juan Quintela
                   ` (11 subsequent siblings)
  12 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

We only create/destry the page list here.  We will use it later.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/ram.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/migration/ram.c b/migration/ram.c
index cd5f55117d..ed4401ee46 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -517,6 +517,20 @@ typedef struct {
     uint8_t id;
 } __attribute__((packed)) MultiFDInit_t;
 
+typedef struct {
+    /* number of used pages */
+    uint32_t used;
+    /* number of allocated pages */
+    uint32_t allocated;
+    /* global number of generated multifd packets */
+    uint64_t packet_num;
+    /* offset of each page */
+    ram_addr_t *offset;
+    /* pointer to each page */
+    struct iovec *iov;
+    RAMBlock *block;
+} MultiFDPages_t;
+
 typedef struct {
     /* this fields are not changed once the thread is created */
     /* channel number */
@@ -535,6 +549,8 @@ typedef struct {
     bool running;
     /* should this thread finish */
     bool quit;
+    /* array of pages to sent */
+    MultiFDPages_t *pages;
 }  MultiFDSendParams;
 
 typedef struct {
@@ -555,6 +571,8 @@ typedef struct {
     bool running;
     /* should this thread finish */
     bool quit;
+    /* array of pages to receive */
+    MultiFDPages_t *pages;
 } MultiFDRecvParams;
 
 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -619,10 +637,36 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp)
     return msg.id;
 }
 
+static MultiFDPages_t *multifd_pages_init(size_t size)
+{
+    MultiFDPages_t *pages = g_new0(MultiFDPages_t, 1);
+
+    pages->allocated = size;
+    pages->iov = g_new0(struct iovec, size);
+    pages->offset = g_new0(ram_addr_t, size);
+
+    return pages;
+}
+
+static void multifd_pages_clear(MultiFDPages_t *pages)
+{
+    pages->used = 0;
+    pages->allocated = 0;
+    pages->packet_num = 0;
+    pages->block = NULL;
+    g_free(pages->iov);
+    pages->iov = NULL;
+    g_free(pages->offset);
+    pages->offset = NULL;
+    g_free(pages);
+}
+
 struct {
     MultiFDSendParams *params;
     /* number of created threads */
     int count;
+    /* array of pages to sent */
+    MultiFDPages_t *pages;
 } *multifd_send_state;
 
 static void multifd_send_terminate_threads(Error *err)
@@ -672,9 +716,13 @@ int multifd_save_cleanup(Error **errp)
         qemu_sem_destroy(&p->sem);
         g_free(p->name);
         p->name = NULL;
+        multifd_pages_clear(p->pages);
+        p->pages = NULL;
     }
     g_free(multifd_send_state->params);
     multifd_send_state->params = NULL;
+    multifd_pages_clear(multifd_send_state->pages);
+    multifd_send_state->pages = NULL;
     g_free(multifd_send_state);
     multifd_send_state = NULL;
     return ret;
@@ -735,6 +783,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
 int multifd_save_setup(void)
 {
     int thread_count;
+    uint32_t page_count = migrate_multifd_page_count();
     uint8_t i;
 
     if (!migrate_use_multifd()) {
@@ -744,6 +793,8 @@ int multifd_save_setup(void)
     multifd_send_state = g_malloc0(sizeof(*multifd_send_state));
     multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
     atomic_set(&multifd_send_state->count, 0);
+    multifd_send_state->pages = multifd_pages_init(page_count);
+
     for (i = 0; i < thread_count; i++) {
         MultiFDSendParams *p = &multifd_send_state->params[i];
 
@@ -751,6 +802,7 @@ int multifd_save_setup(void)
         qemu_sem_init(&p->sem, 0);
         p->quit = false;
         p->id = i;
+        p->pages = multifd_pages_init(page_count);
         p->name = g_strdup_printf("multifdsend_%d", i);
         socket_send_channel_create(multifd_new_send_channel_async, p);
     }
@@ -808,6 +860,8 @@ int multifd_load_cleanup(Error **errp)
         qemu_sem_destroy(&p->sem);
         g_free(p->name);
         p->name = NULL;
+        multifd_pages_clear(p->pages);
+        p->pages = NULL;
     }
     g_free(multifd_recv_state->params);
     multifd_recv_state->params = NULL;
@@ -841,6 +895,7 @@ static void *multifd_recv_thread(void *opaque)
 int multifd_load_setup(void)
 {
     int thread_count;
+    uint32_t page_count = migrate_multifd_page_count();
     uint8_t i;
 
     if (!migrate_use_multifd()) {
@@ -850,6 +905,7 @@ int multifd_load_setup(void)
     multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state));
     multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count);
     atomic_set(&multifd_recv_state->count, 0);
+
     for (i = 0; i < thread_count; i++) {
         MultiFDRecvParams *p = &multifd_recv_state->params[i];
 
@@ -857,6 +913,7 @@ int multifd_load_setup(void)
         qemu_sem_init(&p->sem, 0);
         p->quit = false;
         p->id = i;
+        p->pages = multifd_pages_init(page_count);
         p->name = g_strdup_printf("multifdrecv_%d", i);
     }
     return 0;
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 02/12] migration: Create multifd packet
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 01/12] migration: Create multipage support Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  9:40   ` Dr. David Alan Gilbert
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 03/12] migration: Add multifd traces for start/end thread Juan Quintela
                   ` (10 subsequent siblings)
  12 siblings, 1 reply; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

We still don't put anything there.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>

--
fix magic (dave)
check offset/ramblock  (dave)
s/seq/packet_num/ and make it 64bit
---
 migration/ram.c | 145 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 144 insertions(+), 1 deletion(-)

diff --git a/migration/ram.c b/migration/ram.c
index ed4401ee46..6504b492da 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -517,6 +517,17 @@ typedef struct {
     uint8_t id;
 } __attribute__((packed)) MultiFDInit_t;
 
+typedef struct {
+    uint32_t magic;
+    uint32_t version;
+    uint32_t flags;
+    uint32_t size;
+    uint32_t used;
+    uint64_t packet_num;
+    char ramblock[256];
+    uint64_t offset[];
+} __attribute__((packed)) MultiFDPacket_t;
+
 typedef struct {
     /* number of used pages */
     uint32_t used;
@@ -551,6 +562,14 @@ typedef struct {
     bool quit;
     /* array of pages to sent */
     MultiFDPages_t *pages;
+    /* packet allocated len */
+    uint32_t packet_len;
+    /* pointer to the packet */
+    MultiFDPacket_t *packet;
+    /* multifd flags for each packet */
+    uint32_t flags;
+    /* global number of generated multifd packets */
+    uint64_t packet_num;
 }  MultiFDSendParams;
 
 typedef struct {
@@ -573,6 +592,14 @@ typedef struct {
     bool quit;
     /* array of pages to receive */
     MultiFDPages_t *pages;
+    /* packet allocated len */
+    uint32_t packet_len;
+    /* pointer to the packet */
+    MultiFDPacket_t *packet;
+    /* multifd flags for each packet */
+    uint32_t flags;
+    /* global number of generated multifd packets */
+    uint64_t packet_num;
 } MultiFDRecvParams;
 
 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -661,6 +688,99 @@ static void multifd_pages_clear(MultiFDPages_t *pages)
     g_free(pages);
 }
 
+static void multifd_send_fill_packet(MultiFDSendParams *p)
+{
+    MultiFDPacket_t *packet = p->packet;
+    int i;
+
+    packet->magic = cpu_to_be32(MULTIFD_MAGIC);
+    packet->version = cpu_to_be32(MULTIFD_VERSION);
+    packet->flags = cpu_to_be32(p->flags);
+    packet->size = cpu_to_be32(migrate_multifd_page_count());
+    packet->used = cpu_to_be32(p->pages->used);
+    packet->packet_num = cpu_to_be64(p->packet_num);
+
+    if (p->pages->block) {
+        strncpy(packet->ramblock, p->pages->block->idstr, 256);
+    }
+
+    for (i = 0; i < p->pages->used; i++) {
+        packet->offset[i] = cpu_to_be64(p->pages->offset[i]);
+    }
+}
+
+static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp)
+{
+    MultiFDPacket_t *packet = p->packet;
+    RAMBlock *block;
+    int i;
+
+    /* ToDo: We can't use it until we haven't received a message */
+    return 0;
+
+    be32_to_cpus(&packet->magic);
+    if (packet->magic != MULTIFD_MAGIC) {
+        error_setg(errp, "multifd: received packet "
+                   "magic %x and expected magic %x",
+                   packet->magic, MULTIFD_MAGIC);
+        return -1;
+    }
+
+    be32_to_cpus(&packet->version);
+    if (packet->version != MULTIFD_VERSION) {
+        error_setg(errp, "multifd: received packet "
+                   "version %d and expected version %d",
+                   packet->version, MULTIFD_VERSION);
+        return -1;
+    }
+
+    p->flags = be32_to_cpu(packet->flags);
+
+    be32_to_cpus(&packet->size);
+    if (packet->size > migrate_multifd_page_count()) {
+        error_setg(errp, "multifd: received packet "
+                   "with size %d and expected maximum size %d",
+                   packet->size, migrate_multifd_page_count()) ;
+        return -1;
+    }
+
+    p->pages->used = be32_to_cpu(packet->used);
+    if (p->pages->used > packet->size) {
+        error_setg(errp, "multifd: received packet "
+                   "with size %d and expected maximum size %d",
+                   p->pages->used, packet->size) ;
+        return -1;
+    }
+
+    p->packet_num = be64_to_cpu(packet->packet_num);
+
+    if (p->pages->used) {
+        /* make sure that ramblock is 0 terminated */
+        packet->ramblock[255] = 0;
+        block = qemu_ram_block_by_name(packet->ramblock);
+        if (!block) {
+            error_setg(errp, "multifd: unknown ram block %s",
+                       packet->ramblock);
+            return -1;
+        }
+    }
+
+    for (i = 0; i < p->pages->used; i++) {
+        ram_addr_t offset = be64_to_cpu(packet->offset[i]);
+
+        if (offset > (block->used_length - TARGET_PAGE_SIZE)) {
+            error_setg(errp, "multifd: offset too long %" PRId64
+                       " (max %" PRId64 ")",
+                       offset, block->max_length);
+            return -1;
+        }
+        p->pages->iov[i].iov_base = block->host + offset;
+        p->pages->iov[i].iov_len = TARGET_PAGE_SIZE;
+    }
+
+    return 0;
+}
+
 struct {
     MultiFDSendParams *params;
     /* number of created threads */
@@ -718,6 +838,9 @@ int multifd_save_cleanup(Error **errp)
         p->name = NULL;
         multifd_pages_clear(p->pages);
         p->pages = NULL;
+        p->packet_len = 0;
+        g_free(p->packet);
+        p->packet = NULL;
     }
     g_free(multifd_send_state->params);
     multifd_send_state->params = NULL;
@@ -739,6 +862,7 @@ static void *multifd_send_thread(void *opaque)
 
     while (true) {
         qemu_mutex_lock(&p->mutex);
+        multifd_send_fill_packet(p);
         if (p->quit) {
             qemu_mutex_unlock(&p->mutex);
             break;
@@ -803,6 +927,9 @@ int multifd_save_setup(void)
         p->quit = false;
         p->id = i;
         p->pages = multifd_pages_init(page_count);
+        p->packet_len = sizeof(MultiFDPacket_t)
+                      + sizeof(ram_addr_t) * page_count;
+        p->packet = g_malloc0(p->packet_len);
         p->name = g_strdup_printf("multifdsend_%d", i);
         socket_send_channel_create(multifd_new_send_channel_async, p);
     }
@@ -862,6 +989,9 @@ int multifd_load_cleanup(Error **errp)
         p->name = NULL;
         multifd_pages_clear(p->pages);
         p->pages = NULL;
+        p->packet_len = 0;
+        g_free(p->packet);
+        p->packet = NULL;
     }
     g_free(multifd_recv_state->params);
     multifd_recv_state->params = NULL;
@@ -874,10 +1004,20 @@ int multifd_load_cleanup(Error **errp)
 static void *multifd_recv_thread(void *opaque)
 {
     MultiFDRecvParams *p = opaque;
+    Error *local_err = NULL;
+    int ret;
 
     while (true) {
         qemu_mutex_lock(&p->mutex);
-        if (p->quit) {
+        if (false)  {
+            /* ToDo: Packet reception goes here */
+
+            ret = multifd_recv_unfill_packet(p, &local_err);
+            qemu_mutex_unlock(&p->mutex);
+            if (ret) {
+                break;
+            }
+        } else if (p->quit) {
             qemu_mutex_unlock(&p->mutex);
             break;
         }
@@ -914,6 +1054,9 @@ int multifd_load_setup(void)
         p->quit = false;
         p->id = i;
         p->pages = multifd_pages_init(page_count);
+        p->packet_len = sizeof(MultiFDPacket_t)
+                      + sizeof(ram_addr_t) * page_count;
+        p->packet = g_malloc0(p->packet_len);
         p->name = g_strdup_printf("multifdrecv_%d", i);
     }
     return 0;
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 03/12] migration: Add multifd traces for start/end thread
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 01/12] migration: Create multipage support Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 02/12] migration: Create multifd packet Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  9:51   ` Dr. David Alan Gilbert
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 04/12] migration: Calculate transferred ram correctly Juan Quintela
                   ` (9 subsequent siblings)
  12 siblings, 1 reply; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

We want to know how many pages/packets each channel has sent.  Add
counters for those.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
--
sort trace-events (dave)
---
 migration/ram.c        | 22 ++++++++++++++++++++++
 migration/trace-events |  4 ++++
 2 files changed, 26 insertions(+)

diff --git a/migration/ram.c b/migration/ram.c
index 6504b492da..d146689d3a 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -570,6 +570,11 @@ typedef struct {
     uint32_t flags;
     /* global number of generated multifd packets */
     uint64_t packet_num;
+    /* thread local variables */
+    /* packets sent through this channel */
+    uint32_t num_packets;
+    /* pages sent through this channel */
+    uint32_t num_pages;
 }  MultiFDSendParams;
 
 typedef struct {
@@ -600,6 +605,11 @@ typedef struct {
     uint32_t flags;
     /* global number of generated multifd packets */
     uint64_t packet_num;
+    /* thread local variables */
+    /* packets sent through this channel */
+    uint32_t num_packets;
+    /* pages sent through this channel */
+    uint32_t num_pages;
 } MultiFDRecvParams;
 
 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -856,9 +866,13 @@ static void *multifd_send_thread(void *opaque)
     MultiFDSendParams *p = opaque;
     Error *local_err = NULL;
 
+    trace_multifd_send_thread_start(p->id);
+
     if (multifd_send_initial_packet(p, &local_err) < 0) {
         goto out;
     }
+    /* initial packet */
+    p->num_packets = 1;
 
     while (true) {
         qemu_mutex_lock(&p->mutex);
@@ -880,6 +894,8 @@ out:
     p->running = false;
     qemu_mutex_unlock(&p->mutex);
 
+    trace_multifd_send_thread_end(p->id, p->num_packets, p->num_pages);
+
     return NULL;
 }
 
@@ -1007,6 +1023,8 @@ static void *multifd_recv_thread(void *opaque)
     Error *local_err = NULL;
     int ret;
 
+    trace_multifd_recv_thread_start(p->id);
+
     while (true) {
         qemu_mutex_lock(&p->mutex);
         if (false)  {
@@ -1029,6 +1047,8 @@ static void *multifd_recv_thread(void *opaque)
     p->running = false;
     qemu_mutex_unlock(&p->mutex);
 
+    trace_multifd_recv_thread_end(p->id, p->num_packets, p->num_pages);
+
     return NULL;
 }
 
@@ -1094,6 +1114,8 @@ void multifd_recv_new_channel(QIOChannel *ioc)
     }
     p->c = ioc;
     object_ref(OBJECT(ioc));
+    /* initial packet */
+    p->num_packets = 1;
 
     p->running = true;
     qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p,
diff --git a/migration/trace-events b/migration/trace-events
index 3f67758893..6d499448b3 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -76,6 +76,10 @@ get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned
 migration_bitmap_sync_start(void) ""
 migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
 migration_throttle(void) ""
+multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
+multifd_recv_thread_start(uint8_t id) "%d"
+multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
+multifd_send_thread_start(uint8_t id) "%d"
 ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
 ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%s: addr: 0x%" PRIx64 " flags: 0x%x host: %p"
 ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x"
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 04/12] migration: Calculate transferred ram correctly
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (2 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 03/12] migration: Add multifd traces for start/end thread Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  9:58   ` Dr. David Alan Gilbert
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 05/12] migration: Multifd channels always wait on the sem Juan Quintela
                   ` (8 subsequent siblings)
  12 siblings, 1 reply; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

On multifd we send data from more places that main channel.

Signed-off-by: Juan Quintela <quintela@redhat.com>
--
Add placeholder for packets size
---
 migration/migration.c | 12 ++++++++++--
 migration/ram.c       |  7 +++++++
 migration/ram.h       |  1 +
 3 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index e1eaa97df4..224629533b 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2728,12 +2728,20 @@ static void migration_update_counters(MigrationState *s,
 {
     uint64_t transferred, time_spent;
     double bandwidth;
+    uint64_t now;
 
     if (current_time < s->iteration_start_time + BUFFER_DELAY) {
         return;
     }
 
-    transferred = qemu_ftell(s->to_dst_file) - s->iteration_initial_bytes;
+    if (migrate_use_multifd()) {
+        now = ram_counters.normal * qemu_target_page_size()
+            + multifd_packets_size()
+            + qemu_ftell(s->to_dst_file);
+    } else {
+        now = qemu_ftell(s->to_dst_file);
+    }
+    transferred = now - s->iteration_initial_bytes;
     time_spent = current_time - s->iteration_start_time;
     bandwidth = (double)transferred / time_spent;
     s->threshold_size = bandwidth * s->parameters.downtime_limit;
@@ -2752,7 +2760,7 @@ static void migration_update_counters(MigrationState *s,
     qemu_file_reset_rate_limit(s->to_dst_file);
 
     s->iteration_start_time = current_time;
-    s->iteration_initial_bytes = qemu_ftell(s->to_dst_file);
+    s->iteration_initial_bytes = now;
 
     trace_migrate_transferred(transferred, time_spent,
                               bandwidth, s->threshold_size);
diff --git a/migration/ram.c b/migration/ram.c
index d146689d3a..d7f8b0d989 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -952,6 +952,13 @@ int multifd_save_setup(void)
     return 0;
 }
 
+/* Size in bytes of the page headers */
+int multifd_packets_size(void)
+{
+    /* We are not yet sending any data through channels */
+    return 0;
+}
+
 struct {
     MultiFDRecvParams *params;
     /* number of created threads */
diff --git a/migration/ram.h b/migration/ram.h
index d386f4d641..e1decb7418 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -47,6 +47,7 @@ int multifd_load_setup(void);
 int multifd_load_cleanup(Error **errp);
 bool multifd_recv_all_channels_created(void);
 void multifd_recv_new_channel(QIOChannel *ioc);
+int multifd_packets_size(void);
 
 uint64_t ram_pagesize_summary(void);
 int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len);
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 05/12] migration: Multifd channels always wait on the sem
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (3 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 04/12] migration: Calculate transferred ram correctly Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 06/12] migration: Add block where to send/receive packets Juan Quintela
                   ` (7 subsequent siblings)
  12 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

Either for quit, sync or packet, we first wake them.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/ram.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index d7f8b0d989..617da76a2e 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -875,6 +875,7 @@ static void *multifd_send_thread(void *opaque)
     p->num_packets = 1;
 
     while (true) {
+        qemu_sem_wait(&p->sem);
         qemu_mutex_lock(&p->mutex);
         multifd_send_fill_packet(p);
         if (p->quit) {
@@ -882,7 +883,9 @@ static void *multifd_send_thread(void *opaque)
             break;
         }
         qemu_mutex_unlock(&p->mutex);
-        qemu_sem_wait(&p->sem);
+        /* this is impossible */
+        error_setg(&local_err, "multifd_send_thread: Unknown command");
+        break;
     }
 
 out:
@@ -1033,6 +1036,7 @@ static void *multifd_recv_thread(void *opaque)
     trace_multifd_recv_thread_start(p->id);
 
     while (true) {
+        qemu_sem_wait(&p->sem);
         qemu_mutex_lock(&p->mutex);
         if (false)  {
             /* ToDo: Packet reception goes here */
@@ -1047,9 +1051,14 @@ static void *multifd_recv_thread(void *opaque)
             break;
         }
         qemu_mutex_unlock(&p->mutex);
-        qemu_sem_wait(&p->sem);
+        /* this is impossible */
+        error_setg(&local_err, "multifd_recv_thread: Unknown command");
+        break;
     }
 
+    if (local_err) {
+        multifd_recv_terminate_threads(local_err);
+    }
     qemu_mutex_lock(&p->mutex);
     p->running = false;
     qemu_mutex_unlock(&p->mutex);
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 06/12] migration: Add block where to send/receive packets
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (4 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 05/12] migration: Multifd channels always wait on the sem Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 07/12] migration: Synchronize multifd threads with main thread Juan Quintela
                   ` (6 subsequent siblings)
  12 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

Once there add tracepoints.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/ram.c        | 49 +++++++++++++++++++++++++++++++++++++-----
 migration/trace-events |  2 ++
 2 files changed, 46 insertions(+), 5 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 617da76a2e..793f0dc5d3 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -560,6 +560,8 @@ typedef struct {
     bool running;
     /* should this thread finish */
     bool quit;
+    /* thread has work to do */
+    int pending_job;
     /* array of pages to sent */
     MultiFDPages_t *pages;
     /* packet allocated len */
@@ -595,6 +597,8 @@ typedef struct {
     bool running;
     /* should this thread finish */
     bool quit;
+    /* thread has work to do */
+    bool pending_job;
     /* array of pages to receive */
     MultiFDPages_t *pages;
     /* packet allocated len */
@@ -877,8 +881,28 @@ static void *multifd_send_thread(void *opaque)
     while (true) {
         qemu_sem_wait(&p->sem);
         qemu_mutex_lock(&p->mutex);
-        multifd_send_fill_packet(p);
-        if (p->quit) {
+
+        if (p->pending_job) {
+            uint32_t used = p->pages->used;
+            uint32_t packet_num = p->packet_num;
+            uint32_t flags = p->flags;
+
+            multifd_send_fill_packet(p);
+            p->flags = 0;
+            p->num_packets++;
+            p->num_pages += used;
+            p->pages->used = 0;
+            qemu_mutex_unlock(&p->mutex);
+
+            trace_multifd_send(p->id, packet_num, used, flags);
+
+            /* ToDo: send packet here */
+
+            qemu_mutex_lock(&p->mutex);
+            p->pending_job--;
+            qemu_mutex_unlock(&p->mutex);
+            continue;
+        } else if (p->quit) {
             qemu_mutex_unlock(&p->mutex);
             break;
         }
@@ -944,6 +968,7 @@ int multifd_save_setup(void)
         qemu_mutex_init(&p->mutex);
         qemu_sem_init(&p->sem, 0);
         p->quit = false;
+        p->pending_job = 0;
         p->id = i;
         p->pages = multifd_pages_init(page_count);
         p->packet_len = sizeof(MultiFDPacket_t)
@@ -1038,14 +1063,27 @@ static void *multifd_recv_thread(void *opaque)
     while (true) {
         qemu_sem_wait(&p->sem);
         qemu_mutex_lock(&p->mutex);
-        if (false)  {
-            /* ToDo: Packet reception goes here */
+        if (p->pending_job) {
+            uint32_t used;
+            uint32_t flags;
+            qemu_mutex_unlock(&p->mutex);
 
+            /* ToDo: recv packet here */
+
+            qemu_mutex_lock(&p->mutex);
             ret = multifd_recv_unfill_packet(p, &local_err);
-            qemu_mutex_unlock(&p->mutex);
             if (ret) {
+                qemu_mutex_unlock(&p->mutex);
                 break;
             }
+
+            used = p->pages->used;
+            flags = p->flags;
+            trace_multifd_recv(p->id, p->packet_num, used, flags);
+            p->pending_job = false;
+            p->num_packets++;
+            p->num_pages += used;
+            qemu_mutex_unlock(&p->mutex);
         } else if (p->quit) {
             qemu_mutex_unlock(&p->mutex);
             break;
@@ -1088,6 +1126,7 @@ int multifd_load_setup(void)
         qemu_mutex_init(&p->mutex);
         qemu_sem_init(&p->sem, 0);
         p->quit = false;
+        p->pending_job = false;
         p->id = i;
         p->pages = multifd_pages_init(page_count);
         p->packet_len = sizeof(MultiFDPacket_t)
diff --git a/migration/trace-events b/migration/trace-events
index 6d499448b3..c667d98529 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -76,8 +76,10 @@ get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned
 migration_bitmap_sync_start(void) ""
 migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
 migration_throttle(void) ""
+multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags) "channel %d packet number %ld pages %d flags 0x%x"
 multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
 multifd_recv_thread_start(uint8_t id) "%d"
+multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags) "channel %d packet_num %ld pages %d flags 0x%x"
 multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
 multifd_send_thread_start(uint8_t id) "%d"
 ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 07/12] migration: Synchronize multifd threads with main thread
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (5 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 06/12] migration: Add block where to send/receive packets Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20 10:30   ` Dr. David Alan Gilbert
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 08/12] migration: Create ram_save_multifd_page Juan Quintela
                   ` (5 subsequent siblings)
  12 siblings, 1 reply; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

We synchronize all threads each RAM_SAVE_FLAG_EOS.  Bitmap
synchronizations don't happen inside a  ram section, so we are safe
about two channels trying to overwrite the same memory.

Signed-off-by: Juan Quintela <quintela@redhat.com>
--
seq needs to be atomic now, will also be accessed from main thread.
Fix the if (true || ...) leftover
---
 migration/ram.c        | 147 ++++++++++++++++++++++++++++++++---------
 migration/trace-events |   6 ++
 2 files changed, 122 insertions(+), 31 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 793f0dc5d3..516f347d24 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -510,6 +510,8 @@ exit:
 #define MULTIFD_MAGIC 0x11223344U
 #define MULTIFD_VERSION 1
 
+#define MULTIFD_FLAG_SYNC (1 << 0)
+
 typedef struct {
     uint32_t magic;
     uint32_t version;
@@ -577,6 +579,8 @@ typedef struct {
     uint32_t num_packets;
     /* pages sent through this channel */
     uint32_t num_pages;
+    /* syncs main thread and channels */
+    QemuSemaphore sem_sync;
 }  MultiFDSendParams;
 
 typedef struct {
@@ -614,6 +618,8 @@ typedef struct {
     uint32_t num_packets;
     /* pages sent through this channel */
     uint32_t num_pages;
+    /* syncs main thread and channels */
+    QemuSemaphore sem_sync;
 } MultiFDRecvParams;
 
 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -801,6 +807,10 @@ struct {
     int count;
     /* array of pages to sent */
     MultiFDPages_t *pages;
+    /* syncs main thread and channels */
+    QemuSemaphore sem_sync;
+    /* global number of generated multifd packets */
+    uint64_t packet_num;
 } *multifd_send_state;
 
 static void multifd_send_terminate_threads(Error *err)
@@ -848,6 +858,7 @@ int multifd_save_cleanup(Error **errp)
         p->c = NULL;
         qemu_mutex_destroy(&p->mutex);
         qemu_sem_destroy(&p->sem);
+        qemu_sem_destroy(&p->sem_sync);
         g_free(p->name);
         p->name = NULL;
         multifd_pages_clear(p->pages);
@@ -856,6 +867,7 @@ int multifd_save_cleanup(Error **errp)
         g_free(p->packet);
         p->packet = NULL;
     }
+    qemu_sem_destroy(&multifd_send_state->sem_sync);
     g_free(multifd_send_state->params);
     multifd_send_state->params = NULL;
     multifd_pages_clear(multifd_send_state->pages);
@@ -865,6 +877,33 @@ int multifd_save_cleanup(Error **errp)
     return ret;
 }
 
+static void multifd_send_sync_main(void)
+{
+    int i;
+
+    if (!migrate_use_multifd()) {
+        return;
+    }
+    for (i = 0; i < migrate_multifd_channels(); i++) {
+        MultiFDSendParams *p = &multifd_send_state->params[i];
+
+        trace_multifd_send_sync_main_signal(p->id);
+
+        qemu_mutex_lock(&p->mutex);
+        p->flags |= MULTIFD_FLAG_SYNC;
+        p->pending_job++;
+        qemu_mutex_unlock(&p->mutex);
+        qemu_sem_post(&p->sem);
+    }
+    for (i = 0; i < migrate_multifd_channels(); i++) {
+        MultiFDSendParams *p = &multifd_send_state->params[i];
+
+        trace_multifd_send_sync_main_wait(p->id);
+        qemu_sem_wait(&multifd_send_state->sem_sync);
+    }
+    trace_multifd_send_sync_main(atomic_read(&multifd_send_state->packet_num));
+}
+
 static void *multifd_send_thread(void *opaque)
 {
     MultiFDSendParams *p = opaque;
@@ -901,15 +940,17 @@ static void *multifd_send_thread(void *opaque)
             qemu_mutex_lock(&p->mutex);
             p->pending_job--;
             qemu_mutex_unlock(&p->mutex);
-            continue;
+
+            if (flags & MULTIFD_FLAG_SYNC) {
+                qemu_sem_post(&multifd_send_state->sem_sync);
+            }
         } else if (p->quit) {
             qemu_mutex_unlock(&p->mutex);
             break;
+        } else {
+            qemu_mutex_unlock(&p->mutex);
+            /* sometimes there are spurious wakeups */
         }
-        qemu_mutex_unlock(&p->mutex);
-        /* this is impossible */
-        error_setg(&local_err, "multifd_send_thread: Unknown command");
-        break;
     }
 
 out:
@@ -961,12 +1002,14 @@ int multifd_save_setup(void)
     multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
     atomic_set(&multifd_send_state->count, 0);
     multifd_send_state->pages = multifd_pages_init(page_count);
+    qemu_sem_init(&multifd_send_state->sem_sync, 0);
 
     for (i = 0; i < thread_count; i++) {
         MultiFDSendParams *p = &multifd_send_state->params[i];
 
         qemu_mutex_init(&p->mutex);
         qemu_sem_init(&p->sem, 0);
+        qemu_sem_init(&p->sem_sync, 0);
         p->quit = false;
         p->pending_job = 0;
         p->id = i;
@@ -991,6 +1034,10 @@ struct {
     MultiFDRecvParams *params;
     /* number of created threads */
     int count;
+    /* syncs main thread and channels */
+    QemuSemaphore sem_sync;
+    /* global number of generated multifd packets */
+    uint64_t packet_num;
 } *multifd_recv_state;
 
 static void multifd_recv_terminate_threads(Error *err)
@@ -1036,6 +1083,7 @@ int multifd_load_cleanup(Error **errp)
         p->c = NULL;
         qemu_mutex_destroy(&p->mutex);
         qemu_sem_destroy(&p->sem);
+        qemu_sem_destroy(&p->sem_sync);
         g_free(p->name);
         p->name = NULL;
         multifd_pages_clear(p->pages);
@@ -1044,6 +1092,7 @@ int multifd_load_cleanup(Error **errp)
         g_free(p->packet);
         p->packet = NULL;
     }
+    qemu_sem_destroy(&multifd_recv_state->sem_sync);
     g_free(multifd_recv_state->params);
     multifd_recv_state->params = NULL;
     g_free(multifd_recv_state);
@@ -1052,6 +1101,42 @@ int multifd_load_cleanup(Error **errp)
     return ret;
 }
 
+static void multifd_recv_sync_main(void)
+{
+    int i;
+
+    if (!migrate_use_multifd()) {
+        return;
+    }
+    for (i = 0; i < migrate_multifd_channels(); i++) {
+        MultiFDRecvParams *p = &multifd_recv_state->params[i];
+
+        trace_multifd_recv_sync_main_signal(p->id);
+        qemu_mutex_lock(&p->mutex);
+        p->pending_job = true;
+        qemu_mutex_unlock(&p->mutex);
+    }
+    for (i = 0; i < migrate_multifd_channels(); i++) {
+        MultiFDRecvParams *p = &multifd_recv_state->params[i];
+
+        trace_multifd_recv_sync_main_wait(p->id);
+        qemu_sem_wait(&multifd_recv_state->sem_sync);
+        qemu_mutex_lock(&p->mutex);
+        if (atomic_read(&multifd_recv_state->packet_num) < p->packet_num) {
+            atomic_set(&multifd_recv_state->packet_num, p->packet_num);
+        }
+        qemu_mutex_unlock(&p->mutex);
+    }
+    for (i = 0; i < migrate_multifd_channels(); i++) {
+        MultiFDRecvParams *p = &multifd_recv_state->params[i];
+
+        trace_multifd_recv_sync_main_signal(p->id);
+
+        qemu_sem_post(&p->sem_sync);
+    }
+    trace_multifd_recv_sync_main(atomic_read(&multifd_recv_state->packet_num));
+}
+
 static void *multifd_recv_thread(void *opaque)
 {
     MultiFDRecvParams *p = opaque;
@@ -1061,37 +1146,30 @@ static void *multifd_recv_thread(void *opaque)
     trace_multifd_recv_thread_start(p->id);
 
     while (true) {
-        qemu_sem_wait(&p->sem);
+        uint32_t used;
+        uint32_t flags;
+
+        /* ToDo: recv packet here */
+
         qemu_mutex_lock(&p->mutex);
-        if (p->pending_job) {
-            uint32_t used;
-            uint32_t flags;
-            qemu_mutex_unlock(&p->mutex);
-
-            /* ToDo: recv packet here */
-
-            qemu_mutex_lock(&p->mutex);
-            ret = multifd_recv_unfill_packet(p, &local_err);
-            if (ret) {
-                qemu_mutex_unlock(&p->mutex);
-                break;
-            }
-
-            used = p->pages->used;
-            flags = p->flags;
-            trace_multifd_recv(p->id, p->packet_num, used, flags);
-            p->pending_job = false;
-            p->num_packets++;
-            p->num_pages += used;
-            qemu_mutex_unlock(&p->mutex);
-        } else if (p->quit) {
+        ret = multifd_recv_unfill_packet(p, &local_err);
+        if (ret) {
             qemu_mutex_unlock(&p->mutex);
             break;
         }
+
+        used = p->pages->used;
+        flags = p->flags;
+        trace_multifd_recv(p->id, p->packet_num, used, flags);
+        p->pending_job = false;
+        p->num_packets++;
+        p->num_pages += used;
         qemu_mutex_unlock(&p->mutex);
-        /* this is impossible */
-        error_setg(&local_err, "multifd_recv_thread: Unknown command");
-        break;
+
+        if (flags & MULTIFD_FLAG_SYNC) {
+            qemu_sem_post(&multifd_recv_state->sem_sync);
+            qemu_sem_wait(&p->sem_sync);
+        }
     }
 
     if (local_err) {
@@ -1119,12 +1197,14 @@ int multifd_load_setup(void)
     multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state));
     multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count);
     atomic_set(&multifd_recv_state->count, 0);
+    qemu_sem_init(&multifd_recv_state->sem_sync, 0);
 
     for (i = 0; i < thread_count; i++) {
         MultiFDRecvParams *p = &multifd_recv_state->params[i];
 
         qemu_mutex_init(&p->mutex);
         qemu_sem_init(&p->sem, 0);
+        qemu_sem_init(&p->sem_sync, 0);
         p->quit = false;
         p->pending_job = false;
         p->id = i;
@@ -2882,6 +2962,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
     ram_control_before_iterate(f, RAM_CONTROL_SETUP);
     ram_control_after_iterate(f, RAM_CONTROL_SETUP);
 
+    multifd_send_sync_main();
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
 
     return 0;
@@ -2962,6 +3043,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
      */
     ram_control_after_iterate(f, RAM_CONTROL_ROUND);
 
+    multifd_send_sync_main();
 out:
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
     ram_counters.transferred += 8;
@@ -3015,6 +3097,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
 
     rcu_read_unlock();
 
+    multifd_send_sync_main();
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
 
     return 0;
@@ -3504,6 +3587,7 @@ static int ram_load_postcopy(QEMUFile *f)
             break;
         case RAM_SAVE_FLAG_EOS:
             /* normal exit */
+            multifd_recv_sync_main();
             break;
         default:
             error_report("Unknown combination of migration flags: %#x"
@@ -3692,6 +3776,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
             break;
         case RAM_SAVE_FLAG_EOS:
             /* normal exit */
+            multifd_recv_sync_main();
             break;
         default:
             if (flags & RAM_SAVE_FLAG_HOOK) {
diff --git a/migration/trace-events b/migration/trace-events
index c667d98529..ea39bb2bc5 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -77,9 +77,15 @@ migration_bitmap_sync_start(void) ""
 migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
 migration_throttle(void) ""
 multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags) "channel %d packet number %ld pages %d flags 0x%x"
+multifd_recv_sync_main(uint32_t seq) "seq %d"
+multifd_recv_sync_main_signal(uint8_t id) "channel %d"
+multifd_recv_sync_main_wait(uint8_t id) "channel %d"
 multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
 multifd_recv_thread_start(uint8_t id) "%d"
 multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags) "channel %d packet_num %ld pages %d flags 0x%x"
+multifd_send_sync_main(uint32_t seq) "seq %d"
+multifd_send_sync_main_signal(uint8_t id) "channel %d"
+multifd_send_sync_main_wait(uint8_t id) "channel %d"
 multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
 multifd_send_thread_start(uint8_t id) "%d"
 ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 08/12] migration: Create ram_save_multifd_page
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (6 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 07/12] migration: Synchronize multifd threads with main thread Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 09/12] migration: Start sending messages Juan Quintela
                   ` (4 subsequent siblings)
  12 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

The function still don't use multifd, but we have simplified
ram_save_page, xbzrle and RDMA stuff is gone.  We have added a new
counter.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

--
Add last_page parameter
Add commets for done and address
Remove multifd field, it is the same than normal pages
Merge next patch, now we send multiple pages at a time
Remove counter for multifd pages, it is identical to normal pages
Use iovec's instead of creating the equivalent.
Clear memory used by pages (dave)
Use g_new0(danp)
define MULTIFD_CONTINUE
now pages member is a pointer
Fix off-by-one in number of pages in one packet
Remove RAM_SAVE_FLAG_MULTIFD_PAGE
s/multifd_pages_t/MultiFDPages_t/
add comment explaining what it means
---
 migration/ram.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 110 insertions(+), 2 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 516f347d24..71a33b73e7 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -55,6 +55,7 @@
 #include "sysemu/sysemu.h"
 #include "qemu/uuid.h"
 #include "savevm.h"
+#include "qemu/iov.h"
 
 /***********************************************************/
 /* ram save/restore */
@@ -811,8 +812,83 @@ struct {
     QemuSemaphore sem_sync;
     /* global number of generated multifd packets */
     uint64_t packet_num;
+    /* send channels ready */
+    QemuSemaphore channels_ready;
 } *multifd_send_state;
 
+/*
+ * How we use multifd_send_state->pages and channel->pages?
+ *
+ * We create a pages for each channel, and a main one.  Each time that
+ * we need to send a batch of pages we interchange the ones between
+ * multifd_send_state and the channel that is sending it.  There are
+ * two reasons for that:
+ *    - to not have to do so many mallocs during migration
+ *    - to make easier to know what to free at the end of migration
+ *
+ * This way we always know who is the owner of each "pages" struct,
+ * and we don't need any loocking.  It belongs to the migration thread
+ * or to the channel thread.  Switching is safe because the migration
+ * thread is using the channel mutex when changing it, and the channel
+ * have to had finish with its own, otherwise pending_job can't be
+ * false.
+ */
+
+static void multifd_send_pages(void)
+{
+    int i;
+    static int next_channel;
+    MultiFDSendParams *p = NULL; /* make happy gcc */
+    MultiFDPages_t *pages = multifd_send_state->pages;
+
+    qemu_sem_wait(&multifd_send_state->channels_ready);
+    for (i = next_channel;; i = (i + 1) % migrate_multifd_channels()) {
+        p = &multifd_send_state->params[i];
+
+        qemu_mutex_lock(&p->mutex);
+        if (!p->pending_job) {
+            p->pending_job++;
+            next_channel = (i + 1) % migrate_multifd_channels();
+            break;
+        }
+        qemu_mutex_unlock(&p->mutex);
+    }
+    p->pages->used = 0;
+
+    p->packet_num = atomic_inc_fetch(&multifd_send_state->packet_num);
+    p->pages->block = NULL;
+    multifd_send_state->pages = p->pages;
+    p->pages = pages;
+    qemu_mutex_unlock(&p->mutex);
+    qemu_sem_post(&p->sem);
+}
+
+static void multifd_queue_page(RAMBlock *block, ram_addr_t offset)
+{
+    MultiFDPages_t *pages = multifd_send_state->pages;
+
+    if (!pages->block) {
+        pages->block = block;
+    }
+
+    if (pages->block == block) {
+        pages->offset[pages->used] = offset;
+        pages->iov[pages->used].iov_base = block->host + offset;
+        pages->iov[pages->used].iov_len = TARGET_PAGE_SIZE;
+        pages->used++;
+
+        if (pages->used < pages->allocated) {
+            return;
+        }
+    }
+
+    multifd_send_pages();
+
+    if (pages->block != block) {
+        multifd_queue_page(block, offset);
+    }
+}
+
 static void multifd_send_terminate_threads(Error *err)
 {
     int i;
@@ -867,6 +943,7 @@ int multifd_save_cleanup(Error **errp)
         g_free(p->packet);
         p->packet = NULL;
     }
+    qemu_sem_destroy(&multifd_send_state->channels_ready);
     qemu_sem_destroy(&multifd_send_state->sem_sync);
     g_free(multifd_send_state->params);
     multifd_send_state->params = NULL;
@@ -884,12 +961,17 @@ static void multifd_send_sync_main(void)
     if (!migrate_use_multifd()) {
         return;
     }
+    if (multifd_send_state->pages->used) {
+        multifd_send_pages();
+    }
     for (i = 0; i < migrate_multifd_channels(); i++) {
         MultiFDSendParams *p = &multifd_send_state->params[i];
 
         trace_multifd_send_sync_main_signal(p->id);
 
         qemu_mutex_lock(&p->mutex);
+
+        p->packet_num = atomic_inc_fetch(&multifd_send_state->packet_num);
         p->flags |= MULTIFD_FLAG_SYNC;
         p->pending_job++;
         qemu_mutex_unlock(&p->mutex);
@@ -944,6 +1026,7 @@ static void *multifd_send_thread(void *opaque)
             if (flags & MULTIFD_FLAG_SYNC) {
                 qemu_sem_post(&multifd_send_state->sem_sync);
             }
+            qemu_sem_post(&multifd_send_state->channels_ready);
         } else if (p->quit) {
             qemu_mutex_unlock(&p->mutex);
             break;
@@ -1003,6 +1086,7 @@ int multifd_save_setup(void)
     atomic_set(&multifd_send_state->count, 0);
     multifd_send_state->pages = multifd_pages_init(page_count);
     qemu_sem_init(&multifd_send_state->sem_sync, 0);
+    qemu_sem_init(&multifd_send_state->channels_ready, 0);
 
     for (i = 0; i < thread_count; i++) {
         MultiFDSendParams *p = &multifd_send_state->params[i];
@@ -1026,8 +1110,13 @@ int multifd_save_setup(void)
 /* Size in bytes of the page headers */
 int multifd_packets_size(void)
 {
-    /* We are not yet sending any data through channels */
-    return 0;
+    int packet_num = atomic_read(&multifd_send_state->packet_num);
+    int packet_size = sizeof(MultiFDPacket_t);
+
+    /* Add sincronization packets */
+    packet_num += ram_counters.dirty_sync_count * migrate_multifd_channels();
+
+    return packet_num * packet_size;
 }
 
 struct {
@@ -1731,6 +1820,23 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage)
     return pages;
 }
 
+static int ram_save_multifd_page(RAMState *rs, RAMBlock *block,
+                                 ram_addr_t offset)
+{
+    uint8_t *p;
+
+    p = block->host + offset;
+
+    ram_counters.transferred += save_page_header(rs, rs->f, block,
+                                                 offset | RAM_SAVE_FLAG_PAGE);
+    multifd_queue_page(block, offset);
+    qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE);
+    ram_counters.transferred += TARGET_PAGE_SIZE;
+    ram_counters.normal++;
+
+    return 1;
+}
+
 static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block,
                                 ram_addr_t offset, uint8_t *source_buf)
 {
@@ -2136,6 +2242,8 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss,
      */
     if (block == rs->last_sent_block && save_page_use_compression(rs)) {
         return compress_page_with_multi_thread(rs, block, offset);
+    } else if (migrate_use_multifd()) {
+        return ram_save_multifd_page(rs, block, offset);
     }
 
     return ram_save_page(rs, pss, last_stage);
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 09/12] migration: Start sending messages
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (7 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 08/12] migration: Create ram_save_multifd_page Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 10/12] migration: Wait for blocking IO Juan Quintela
                   ` (3 subsequent siblings)
  12 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/ram.c | 29 ++++++++++++++++++++++++-----
 1 file changed, 24 insertions(+), 5 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 71a33b73e7..09df573441 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -736,9 +736,6 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp)
     RAMBlock *block;
     int i;
 
-    /* ToDo: We can't use it until we haven't received a message */
-    return 0;
-
     be32_to_cpus(&packet->magic);
     if (packet->magic != MULTIFD_MAGIC) {
         error_setg(errp, "multifd: received packet "
@@ -990,6 +987,7 @@ static void *multifd_send_thread(void *opaque)
 {
     MultiFDSendParams *p = opaque;
     Error *local_err = NULL;
+    int ret;
 
     trace_multifd_send_thread_start(p->id);
 
@@ -1017,7 +1015,16 @@ static void *multifd_send_thread(void *opaque)
 
             trace_multifd_send(p->id, packet_num, used, flags);
 
-            /* ToDo: send packet here */
+            ret = qio_channel_write_all(p->c, (void *)p->packet,
+                                        p->packet_len, &local_err);
+            if (ret != 0) {
+                break;
+            }
+
+            ret = qio_channel_writev_all(p->c, p->pages->iov, used, &local_err);
+            if (ret != 0) {
+                break;
+            }
 
             qemu_mutex_lock(&p->mutex);
             p->pending_job--;
@@ -1238,7 +1245,14 @@ static void *multifd_recv_thread(void *opaque)
         uint32_t used;
         uint32_t flags;
 
-        /* ToDo: recv packet here */
+        ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
+                                       p->packet_len, &local_err);
+        if (ret == 0) {   /* EOF */
+            break;
+        }
+        if (ret == -1) {   /* Error */
+            break;
+        }
 
         qemu_mutex_lock(&p->mutex);
         ret = multifd_recv_unfill_packet(p, &local_err);
@@ -1255,6 +1269,11 @@ static void *multifd_recv_thread(void *opaque)
         p->num_pages += used;
         qemu_mutex_unlock(&p->mutex);
 
+        ret = qio_channel_readv_all(p->c, p->pages->iov, used, &local_err);
+        if (ret != 0) {
+            break;
+        }
+
         if (flags & MULTIFD_FLAG_SYNC) {
             qemu_sem_post(&multifd_recv_state->sem_sync);
             qemu_sem_wait(&p->sem_sync);
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 10/12] migration: Wait for blocking IO
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (8 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 09/12] migration: Start sending messages Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 11/12] migration: Remove not needed semaphore and quit Juan Quintela
                   ` (2 subsequent siblings)
  12 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

We have three conditions here:
- channel fails -> error
- we have to quit: we close the channel and reads fails
- normal read that success, we are in bussiness

So forget the complications of waiting in a semaphore.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/ram.c | 13 -------------
 1 file changed, 13 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 09df573441..2c3a452a7d 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -602,8 +602,6 @@ typedef struct {
     bool running;
     /* should this thread finish */
     bool quit;
-    /* thread has work to do */
-    bool pending_job;
     /* array of pages to receive */
     MultiFDPages_t *pages;
     /* packet allocated len */
@@ -1207,14 +1205,6 @@ static void multifd_recv_sync_main(void)
     for (i = 0; i < migrate_multifd_channels(); i++) {
         MultiFDRecvParams *p = &multifd_recv_state->params[i];
 
-        trace_multifd_recv_sync_main_signal(p->id);
-        qemu_mutex_lock(&p->mutex);
-        p->pending_job = true;
-        qemu_mutex_unlock(&p->mutex);
-    }
-    for (i = 0; i < migrate_multifd_channels(); i++) {
-        MultiFDRecvParams *p = &multifd_recv_state->params[i];
-
         trace_multifd_recv_sync_main_wait(p->id);
         qemu_sem_wait(&multifd_recv_state->sem_sync);
         qemu_mutex_lock(&p->mutex);
@@ -1227,7 +1217,6 @@ static void multifd_recv_sync_main(void)
         MultiFDRecvParams *p = &multifd_recv_state->params[i];
 
         trace_multifd_recv_sync_main_signal(p->id);
-
         qemu_sem_post(&p->sem_sync);
     }
     trace_multifd_recv_sync_main(atomic_read(&multifd_recv_state->packet_num));
@@ -1264,7 +1253,6 @@ static void *multifd_recv_thread(void *opaque)
         used = p->pages->used;
         flags = p->flags;
         trace_multifd_recv(p->id, p->packet_num, used, flags);
-        p->pending_job = false;
         p->num_packets++;
         p->num_pages += used;
         qemu_mutex_unlock(&p->mutex);
@@ -1314,7 +1302,6 @@ int multifd_load_setup(void)
         qemu_sem_init(&p->sem, 0);
         qemu_sem_init(&p->sem_sync, 0);
         p->quit = false;
-        p->pending_job = false;
         p->id = i;
         p->pages = multifd_pages_init(page_count);
         p->packet_len = sizeof(MultiFDPacket_t)
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 11/12] migration: Remove not needed semaphore and quit
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (9 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 10/12] migration: Wait for blocking IO Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 12/12] migration: Stop sending whole pages through main channel Juan Quintela
  2018-06-20 10:44 ` [Qemu-devel] [PATCH v14 00/12] Multifd no-reply
  12 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

We know quit closing the QIO.

Signed-off-by: Juan Quintela <quintela@redhat.com>
--
Add comment
don't object_ref() twice.
---
 migration/ram.c | 17 ++++++-----------
 1 file changed, 6 insertions(+), 11 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 2c3a452a7d..57d3ad1c45 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -594,14 +594,10 @@ typedef struct {
     QemuThread thread;
     /* communication channel */
     QIOChannel *c;
-    /* sem where to wait for more work */
-    QemuSemaphore sem;
     /* this mutex protects the following parameters */
     QemuMutex mutex;
     /* is this channel thread running */
     bool running;
-    /* should this thread finish */
-    bool quit;
     /* array of pages to receive */
     MultiFDPages_t *pages;
     /* packet allocated len */
@@ -1152,8 +1148,12 @@ static void multifd_recv_terminate_threads(Error *err)
         MultiFDRecvParams *p = &multifd_recv_state->params[i];
 
         qemu_mutex_lock(&p->mutex);
-        p->quit = true;
-        qemu_sem_post(&p->sem);
+        /* We could arrive here for two reasons:
+           - normal quit, i.e. everything went fine, just finished
+           - error quit: We close the channels so the channel threads
+             finish the qio_channel_read_all_eof() */
+        object_unref(OBJECT(p->c));
+        p->c = NULL;
         qemu_mutex_unlock(&p->mutex);
     }
 }
@@ -1173,10 +1173,7 @@ int multifd_load_cleanup(Error **errp)
         if (p->running) {
             qemu_thread_join(&p->thread);
         }
-        object_unref(OBJECT(p->c));
-        p->c = NULL;
         qemu_mutex_destroy(&p->mutex);
-        qemu_sem_destroy(&p->sem);
         qemu_sem_destroy(&p->sem_sync);
         g_free(p->name);
         p->name = NULL;
@@ -1299,9 +1296,7 @@ int multifd_load_setup(void)
         MultiFDRecvParams *p = &multifd_recv_state->params[i];
 
         qemu_mutex_init(&p->mutex);
-        qemu_sem_init(&p->sem, 0);
         qemu_sem_init(&p->sem_sync, 0);
-        p->quit = false;
         p->id = i;
         p->pages = multifd_pages_init(page_count);
         p->packet_len = sizeof(MultiFDPacket_t)
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Qemu-devel] [PATCH v14 12/12] migration: Stop sending whole pages through main channel
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (10 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 11/12] migration: Remove not needed semaphore and quit Juan Quintela
@ 2018-06-20  8:15 ` Juan Quintela
  2018-06-20 10:44 ` [Qemu-devel] [PATCH v14 00/12] Multifd no-reply
  12 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  8:15 UTC (permalink / raw)
  To: qemu-devel; +Cc: dgilbert, lvivier, peterx

We have to flush() the QEMUFile because now we sent really few data
through that channel.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/ram.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 57d3ad1c45..6d0782623c 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1824,15 +1824,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage)
 static int ram_save_multifd_page(RAMState *rs, RAMBlock *block,
                                  ram_addr_t offset)
 {
-    uint8_t *p;
-
-    p = block->host + offset;
-
-    ram_counters.transferred += save_page_header(rs, rs->f, block,
-                                                 offset | RAM_SAVE_FLAG_PAGE);
     multifd_queue_page(block, offset);
-    qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE);
-    ram_counters.transferred += TARGET_PAGE_SIZE;
     ram_counters.normal++;
 
     return 1;
@@ -3073,6 +3065,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
 
     multifd_send_sync_main();
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+    qemu_fflush(f);
 
     return 0;
 }
@@ -3155,6 +3148,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
     multifd_send_sync_main();
 out:
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+    qemu_fflush(f);
     ram_counters.transferred += 8;
 
     ret = qemu_file_get_error(f);
@@ -3208,6 +3202,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
 
     multifd_send_sync_main();
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+    qemu_fflush(f);
 
     return 0;
 }
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [Qemu-devel] [PATCH v14 02/12] migration: Create multifd packet
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 02/12] migration: Create multifd packet Juan Quintela
@ 2018-06-20  9:40   ` Dr. David Alan Gilbert
  2018-06-20  9:43     ` Juan Quintela
  0 siblings, 1 reply; 19+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-20  9:40 UTC (permalink / raw)
  To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx

* Juan Quintela (quintela@redhat.com) wrote:
> We still don't put anything there.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Juan Quintela <quintela@redhat.com>

Wrong r-b; that's:


Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> --
> fix magic (dave)
> check offset/ramblock  (dave)
> s/seq/packet_num/ and make it 64bit
> ---
>  migration/ram.c | 145 +++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 144 insertions(+), 1 deletion(-)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index ed4401ee46..6504b492da 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -517,6 +517,17 @@ typedef struct {
>      uint8_t id;
>  } __attribute__((packed)) MultiFDInit_t;
>  
> +typedef struct {
> +    uint32_t magic;
> +    uint32_t version;
> +    uint32_t flags;
> +    uint32_t size;
> +    uint32_t used;
> +    uint64_t packet_num;
> +    char ramblock[256];
> +    uint64_t offset[];
> +} __attribute__((packed)) MultiFDPacket_t;
> +
>  typedef struct {
>      /* number of used pages */
>      uint32_t used;
> @@ -551,6 +562,14 @@ typedef struct {
>      bool quit;
>      /* array of pages to sent */
>      MultiFDPages_t *pages;
> +    /* packet allocated len */
> +    uint32_t packet_len;
> +    /* pointer to the packet */
> +    MultiFDPacket_t *packet;
> +    /* multifd flags for each packet */
> +    uint32_t flags;
> +    /* global number of generated multifd packets */
> +    uint64_t packet_num;
>  }  MultiFDSendParams;
>  
>  typedef struct {
> @@ -573,6 +592,14 @@ typedef struct {
>      bool quit;
>      /* array of pages to receive */
>      MultiFDPages_t *pages;
> +    /* packet allocated len */
> +    uint32_t packet_len;
> +    /* pointer to the packet */
> +    MultiFDPacket_t *packet;
> +    /* multifd flags for each packet */
> +    uint32_t flags;
> +    /* global number of generated multifd packets */
> +    uint64_t packet_num;
>  } MultiFDRecvParams;
>  
>  static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
> @@ -661,6 +688,99 @@ static void multifd_pages_clear(MultiFDPages_t *pages)
>      g_free(pages);
>  }
>  
> +static void multifd_send_fill_packet(MultiFDSendParams *p)
> +{
> +    MultiFDPacket_t *packet = p->packet;
> +    int i;
> +
> +    packet->magic = cpu_to_be32(MULTIFD_MAGIC);
> +    packet->version = cpu_to_be32(MULTIFD_VERSION);
> +    packet->flags = cpu_to_be32(p->flags);
> +    packet->size = cpu_to_be32(migrate_multifd_page_count());
> +    packet->used = cpu_to_be32(p->pages->used);
> +    packet->packet_num = cpu_to_be64(p->packet_num);
> +
> +    if (p->pages->block) {
> +        strncpy(packet->ramblock, p->pages->block->idstr, 256);
> +    }
> +
> +    for (i = 0; i < p->pages->used; i++) {
> +        packet->offset[i] = cpu_to_be64(p->pages->offset[i]);
> +    }
> +}
> +
> +static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp)
> +{
> +    MultiFDPacket_t *packet = p->packet;
> +    RAMBlock *block;
> +    int i;
> +
> +    /* ToDo: We can't use it until we haven't received a message */
> +    return 0;
> +
> +    be32_to_cpus(&packet->magic);
> +    if (packet->magic != MULTIFD_MAGIC) {
> +        error_setg(errp, "multifd: received packet "
> +                   "magic %x and expected magic %x",
> +                   packet->magic, MULTIFD_MAGIC);
> +        return -1;
> +    }
> +
> +    be32_to_cpus(&packet->version);
> +    if (packet->version != MULTIFD_VERSION) {
> +        error_setg(errp, "multifd: received packet "
> +                   "version %d and expected version %d",
> +                   packet->version, MULTIFD_VERSION);
> +        return -1;
> +    }
> +
> +    p->flags = be32_to_cpu(packet->flags);
> +
> +    be32_to_cpus(&packet->size);
> +    if (packet->size > migrate_multifd_page_count()) {
> +        error_setg(errp, "multifd: received packet "
> +                   "with size %d and expected maximum size %d",
> +                   packet->size, migrate_multifd_page_count()) ;
> +        return -1;
> +    }
> +
> +    p->pages->used = be32_to_cpu(packet->used);
> +    if (p->pages->used > packet->size) {
> +        error_setg(errp, "multifd: received packet "
> +                   "with size %d and expected maximum size %d",
> +                   p->pages->used, packet->size) ;
> +        return -1;
> +    }
> +
> +    p->packet_num = be64_to_cpu(packet->packet_num);
> +
> +    if (p->pages->used) {
> +        /* make sure that ramblock is 0 terminated */
> +        packet->ramblock[255] = 0;
> +        block = qemu_ram_block_by_name(packet->ramblock);
> +        if (!block) {
> +            error_setg(errp, "multifd: unknown ram block %s",
> +                       packet->ramblock);
> +            return -1;
> +        }
> +    }
> +
> +    for (i = 0; i < p->pages->used; i++) {
> +        ram_addr_t offset = be64_to_cpu(packet->offset[i]);
> +
> +        if (offset > (block->used_length - TARGET_PAGE_SIZE)) {
> +            error_setg(errp, "multifd: offset too long %" PRId64
> +                       " (max %" PRId64 ")",
> +                       offset, block->max_length);
> +            return -1;
> +        }
> +        p->pages->iov[i].iov_base = block->host + offset;
> +        p->pages->iov[i].iov_len = TARGET_PAGE_SIZE;
> +    }
> +
> +    return 0;
> +}
> +
>  struct {
>      MultiFDSendParams *params;
>      /* number of created threads */
> @@ -718,6 +838,9 @@ int multifd_save_cleanup(Error **errp)
>          p->name = NULL;
>          multifd_pages_clear(p->pages);
>          p->pages = NULL;
> +        p->packet_len = 0;
> +        g_free(p->packet);
> +        p->packet = NULL;
>      }
>      g_free(multifd_send_state->params);
>      multifd_send_state->params = NULL;
> @@ -739,6 +862,7 @@ static void *multifd_send_thread(void *opaque)
>  
>      while (true) {
>          qemu_mutex_lock(&p->mutex);
> +        multifd_send_fill_packet(p);
>          if (p->quit) {
>              qemu_mutex_unlock(&p->mutex);
>              break;
> @@ -803,6 +927,9 @@ int multifd_save_setup(void)
>          p->quit = false;
>          p->id = i;
>          p->pages = multifd_pages_init(page_count);
> +        p->packet_len = sizeof(MultiFDPacket_t)
> +                      + sizeof(ram_addr_t) * page_count;
> +        p->packet = g_malloc0(p->packet_len);
>          p->name = g_strdup_printf("multifdsend_%d", i);
>          socket_send_channel_create(multifd_new_send_channel_async, p);
>      }
> @@ -862,6 +989,9 @@ int multifd_load_cleanup(Error **errp)
>          p->name = NULL;
>          multifd_pages_clear(p->pages);
>          p->pages = NULL;
> +        p->packet_len = 0;
> +        g_free(p->packet);
> +        p->packet = NULL;
>      }
>      g_free(multifd_recv_state->params);
>      multifd_recv_state->params = NULL;
> @@ -874,10 +1004,20 @@ int multifd_load_cleanup(Error **errp)
>  static void *multifd_recv_thread(void *opaque)
>  {
>      MultiFDRecvParams *p = opaque;
> +    Error *local_err = NULL;
> +    int ret;
>  
>      while (true) {
>          qemu_mutex_lock(&p->mutex);
> -        if (p->quit) {
> +        if (false)  {
> +            /* ToDo: Packet reception goes here */
> +
> +            ret = multifd_recv_unfill_packet(p, &local_err);
> +            qemu_mutex_unlock(&p->mutex);
> +            if (ret) {
> +                break;
> +            }
> +        } else if (p->quit) {
>              qemu_mutex_unlock(&p->mutex);
>              break;
>          }
> @@ -914,6 +1054,9 @@ int multifd_load_setup(void)
>          p->quit = false;
>          p->id = i;
>          p->pages = multifd_pages_init(page_count);
> +        p->packet_len = sizeof(MultiFDPacket_t)
> +                      + sizeof(ram_addr_t) * page_count;
> +        p->packet = g_malloc0(p->packet_len);
>          p->name = g_strdup_printf("multifdrecv_%d", i);
>      }
>      return 0;
> -- 
> 2.17.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Qemu-devel] [PATCH v14 02/12] migration: Create multifd packet
  2018-06-20  9:40   ` Dr. David Alan Gilbert
@ 2018-06-20  9:43     ` Juan Quintela
  0 siblings, 0 replies; 19+ messages in thread
From: Juan Quintela @ 2018-06-20  9:43 UTC (permalink / raw)
  To: Dr. David Alan Gilbert; +Cc: qemu-devel, lvivier, peterx

"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> We still don't put anything there.
>> 
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> Reviewed-by: Juan Quintela <quintela@redhat.com>
>
> Wrong r-b; that's:
>
>
> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Fixed, thanks.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Qemu-devel] [PATCH v14 03/12] migration: Add multifd traces for start/end thread
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 03/12] migration: Add multifd traces for start/end thread Juan Quintela
@ 2018-06-20  9:51   ` Dr. David Alan Gilbert
  0 siblings, 0 replies; 19+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-20  9:51 UTC (permalink / raw)
  To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx

* Juan Quintela (quintela@redhat.com) wrote:
> We want to know how many pages/packets each channel has sent.  Add
> counters for those.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Juan Quintela <quintela@redhat.com>

You mean

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> --
> sort trace-events (dave)
> ---
>  migration/ram.c        | 22 ++++++++++++++++++++++
>  migration/trace-events |  4 ++++
>  2 files changed, 26 insertions(+)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index 6504b492da..d146689d3a 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -570,6 +570,11 @@ typedef struct {
>      uint32_t flags;
>      /* global number of generated multifd packets */
>      uint64_t packet_num;
> +    /* thread local variables */
> +    /* packets sent through this channel */
> +    uint32_t num_packets;
> +    /* pages sent through this channel */
> +    uint32_t num_pages;
>  }  MultiFDSendParams;
>  
>  typedef struct {
> @@ -600,6 +605,11 @@ typedef struct {
>      uint32_t flags;
>      /* global number of generated multifd packets */
>      uint64_t packet_num;
> +    /* thread local variables */
> +    /* packets sent through this channel */
> +    uint32_t num_packets;
> +    /* pages sent through this channel */
> +    uint32_t num_pages;
>  } MultiFDRecvParams;
>  
>  static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
> @@ -856,9 +866,13 @@ static void *multifd_send_thread(void *opaque)
>      MultiFDSendParams *p = opaque;
>      Error *local_err = NULL;
>  
> +    trace_multifd_send_thread_start(p->id);
> +
>      if (multifd_send_initial_packet(p, &local_err) < 0) {
>          goto out;
>      }
> +    /* initial packet */
> +    p->num_packets = 1;
>  
>      while (true) {
>          qemu_mutex_lock(&p->mutex);
> @@ -880,6 +894,8 @@ out:
>      p->running = false;
>      qemu_mutex_unlock(&p->mutex);
>  
> +    trace_multifd_send_thread_end(p->id, p->num_packets, p->num_pages);
> +
>      return NULL;
>  }
>  
> @@ -1007,6 +1023,8 @@ static void *multifd_recv_thread(void *opaque)
>      Error *local_err = NULL;
>      int ret;
>  
> +    trace_multifd_recv_thread_start(p->id);
> +
>      while (true) {
>          qemu_mutex_lock(&p->mutex);
>          if (false)  {
> @@ -1029,6 +1047,8 @@ static void *multifd_recv_thread(void *opaque)
>      p->running = false;
>      qemu_mutex_unlock(&p->mutex);
>  
> +    trace_multifd_recv_thread_end(p->id, p->num_packets, p->num_pages);
> +
>      return NULL;
>  }
>  
> @@ -1094,6 +1114,8 @@ void multifd_recv_new_channel(QIOChannel *ioc)
>      }
>      p->c = ioc;
>      object_ref(OBJECT(ioc));
> +    /* initial packet */
> +    p->num_packets = 1;
>  
>      p->running = true;
>      qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p,
> diff --git a/migration/trace-events b/migration/trace-events
> index 3f67758893..6d499448b3 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -76,6 +76,10 @@ get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned
>  migration_bitmap_sync_start(void) ""
>  migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
>  migration_throttle(void) ""
> +multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
> +multifd_recv_thread_start(uint8_t id) "%d"
> +multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
> +multifd_send_thread_start(uint8_t id) "%d"
>  ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
>  ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%s: addr: 0x%" PRIx64 " flags: 0x%x host: %p"
>  ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x"
> -- 
> 2.17.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Qemu-devel] [PATCH v14 04/12] migration: Calculate transferred ram correctly
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 04/12] migration: Calculate transferred ram correctly Juan Quintela
@ 2018-06-20  9:58   ` Dr. David Alan Gilbert
  0 siblings, 0 replies; 19+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-20  9:58 UTC (permalink / raw)
  To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx

* Juan Quintela (quintela@redhat.com) wrote:
> On multifd we send data from more places that main channel.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> --
> Add placeholder for packets size
> ---
>  migration/migration.c | 12 ++++++++++--
>  migration/ram.c       |  7 +++++++
>  migration/ram.h       |  1 +
>  3 files changed, 18 insertions(+), 2 deletions(-)
> 
> diff --git a/migration/migration.c b/migration/migration.c
> index e1eaa97df4..224629533b 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -2728,12 +2728,20 @@ static void migration_update_counters(MigrationState *s,
>  {
>      uint64_t transferred, time_spent;
>      double bandwidth;
> +    uint64_t now;
>  
>      if (current_time < s->iteration_start_time + BUFFER_DELAY) {
>          return;
>      }
>  
> -    transferred = qemu_ftell(s->to_dst_file) - s->iteration_initial_bytes;
> +    if (migrate_use_multifd()) {
> +        now = ram_counters.normal * qemu_target_page_size()
> +            + multifd_packets_size()
> +            + qemu_ftell(s->to_dst_file);
> +    } else {
> +        now = qemu_ftell(s->to_dst_file);
> +    }
> +    transferred = now - s->iteration_initial_bytes;
>      time_spent = current_time - s->iteration_start_time;
>      bandwidth = (double)transferred / time_spent;
>      s->threshold_size = bandwidth * s->parameters.downtime_limit;
> @@ -2752,7 +2760,7 @@ static void migration_update_counters(MigrationState *s,
>      qemu_file_reset_rate_limit(s->to_dst_file);
>  
>      s->iteration_start_time = current_time;
> -    s->iteration_initial_bytes = qemu_ftell(s->to_dst_file);
> +    s->iteration_initial_bytes = now;
>  
>      trace_migrate_transferred(transferred, time_spent,
>                                bandwidth, s->threshold_size);
> diff --git a/migration/ram.c b/migration/ram.c
> index d146689d3a..d7f8b0d989 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -952,6 +952,13 @@ int multifd_save_setup(void)
>      return 0;
>  }
>  
> +/* Size in bytes of the page headers */
> +int multifd_packets_size(void)
> +{
> +    /* We are not yet sending any data through channels */
> +    return 0;
> +}
> +
>  struct {
>      MultiFDRecvParams *params;
>      /* number of created threads */
> diff --git a/migration/ram.h b/migration/ram.h
> index d386f4d641..e1decb7418 100644
> --- a/migration/ram.h
> +++ b/migration/ram.h
> @@ -47,6 +47,7 @@ int multifd_load_setup(void);
>  int multifd_load_cleanup(Error **errp);
>  bool multifd_recv_all_channels_created(void);
>  void multifd_recv_new_channel(QIOChannel *ioc);
> +int multifd_packets_size(void);
>  
>  uint64_t ram_pagesize_summary(void);
>  int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len);
> -- 
> 2.17.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Qemu-devel] [PATCH v14 07/12] migration: Synchronize multifd threads with main thread
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 07/12] migration: Synchronize multifd threads with main thread Juan Quintela
@ 2018-06-20 10:30   ` Dr. David Alan Gilbert
  0 siblings, 0 replies; 19+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-20 10:30 UTC (permalink / raw)
  To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx

* Juan Quintela (quintela@redhat.com) wrote:
> We synchronize all threads each RAM_SAVE_FLAG_EOS.  Bitmap
> synchronizations don't happen inside a  ram section, so we are safe
> about two channels trying to overwrite the same memory.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> --
> seq needs to be atomic now, will also be accessed from main thread.
> Fix the if (true || ...) leftover
> ---
>  migration/ram.c        | 147 ++++++++++++++++++++++++++++++++---------
>  migration/trace-events |   6 ++
>  2 files changed, 122 insertions(+), 31 deletions(-)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index 793f0dc5d3..516f347d24 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -510,6 +510,8 @@ exit:
>  #define MULTIFD_MAGIC 0x11223344U
>  #define MULTIFD_VERSION 1
>  
> +#define MULTIFD_FLAG_SYNC (1 << 0)
> +
>  typedef struct {
>      uint32_t magic;
>      uint32_t version;
> @@ -577,6 +579,8 @@ typedef struct {
>      uint32_t num_packets;
>      /* pages sent through this channel */
>      uint32_t num_pages;
> +    /* syncs main thread and channels */
> +    QemuSemaphore sem_sync;
>  }  MultiFDSendParams;
>  
>  typedef struct {
> @@ -614,6 +618,8 @@ typedef struct {
>      uint32_t num_packets;
>      /* pages sent through this channel */
>      uint32_t num_pages;
> +    /* syncs main thread and channels */
> +    QemuSemaphore sem_sync;
>  } MultiFDRecvParams;
>  
>  static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
> @@ -801,6 +807,10 @@ struct {
>      int count;
>      /* array of pages to sent */
>      MultiFDPages_t *pages;
> +    /* syncs main thread and channels */
> +    QemuSemaphore sem_sync;
> +    /* global number of generated multifd packets */
> +    uint64_t packet_num;
>  } *multifd_send_state;
>  
>  static void multifd_send_terminate_threads(Error *err)
> @@ -848,6 +858,7 @@ int multifd_save_cleanup(Error **errp)
>          p->c = NULL;
>          qemu_mutex_destroy(&p->mutex);
>          qemu_sem_destroy(&p->sem);
> +        qemu_sem_destroy(&p->sem_sync);
>          g_free(p->name);
>          p->name = NULL;
>          multifd_pages_clear(p->pages);
> @@ -856,6 +867,7 @@ int multifd_save_cleanup(Error **errp)
>          g_free(p->packet);
>          p->packet = NULL;
>      }
> +    qemu_sem_destroy(&multifd_send_state->sem_sync);
>      g_free(multifd_send_state->params);
>      multifd_send_state->params = NULL;
>      multifd_pages_clear(multifd_send_state->pages);
> @@ -865,6 +877,33 @@ int multifd_save_cleanup(Error **errp)
>      return ret;
>  }
>  
> +static void multifd_send_sync_main(void)
> +{
> +    int i;
> +
> +    if (!migrate_use_multifd()) {
> +        return;
> +    }
> +    for (i = 0; i < migrate_multifd_channels(); i++) {
> +        MultiFDSendParams *p = &multifd_send_state->params[i];
> +
> +        trace_multifd_send_sync_main_signal(p->id);
> +
> +        qemu_mutex_lock(&p->mutex);
> +        p->flags |= MULTIFD_FLAG_SYNC;
> +        p->pending_job++;
> +        qemu_mutex_unlock(&p->mutex);
> +        qemu_sem_post(&p->sem);
> +    }
> +    for (i = 0; i < migrate_multifd_channels(); i++) {
> +        MultiFDSendParams *p = &multifd_send_state->params[i];
> +
> +        trace_multifd_send_sync_main_wait(p->id);
> +        qemu_sem_wait(&multifd_send_state->sem_sync);
> +    }
> +    trace_multifd_send_sync_main(atomic_read(&multifd_send_state->packet_num));
> +}
> +
>  static void *multifd_send_thread(void *opaque)
>  {
>      MultiFDSendParams *p = opaque;
> @@ -901,15 +940,17 @@ static void *multifd_send_thread(void *opaque)
>              qemu_mutex_lock(&p->mutex);
>              p->pending_job--;
>              qemu_mutex_unlock(&p->mutex);
> -            continue;
> +
> +            if (flags & MULTIFD_FLAG_SYNC) {
> +                qemu_sem_post(&multifd_send_state->sem_sync);
> +            }
>          } else if (p->quit) {
>              qemu_mutex_unlock(&p->mutex);
>              break;
> +        } else {
> +            qemu_mutex_unlock(&p->mutex);
> +            /* sometimes there are spurious wakeups */
>          }
> -        qemu_mutex_unlock(&p->mutex);
> -        /* this is impossible */
> -        error_setg(&local_err, "multifd_send_thread: Unknown command");
> -        break;
>      }
>  
>  out:
> @@ -961,12 +1002,14 @@ int multifd_save_setup(void)
>      multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
>      atomic_set(&multifd_send_state->count, 0);
>      multifd_send_state->pages = multifd_pages_init(page_count);
> +    qemu_sem_init(&multifd_send_state->sem_sync, 0);
>  
>      for (i = 0; i < thread_count; i++) {
>          MultiFDSendParams *p = &multifd_send_state->params[i];
>  
>          qemu_mutex_init(&p->mutex);
>          qemu_sem_init(&p->sem, 0);
> +        qemu_sem_init(&p->sem_sync, 0);
>          p->quit = false;
>          p->pending_job = 0;
>          p->id = i;
> @@ -991,6 +1034,10 @@ struct {
>      MultiFDRecvParams *params;
>      /* number of created threads */
>      int count;
> +    /* syncs main thread and channels */
> +    QemuSemaphore sem_sync;
> +    /* global number of generated multifd packets */
> +    uint64_t packet_num;
>  } *multifd_recv_state;
>  
>  static void multifd_recv_terminate_threads(Error *err)
> @@ -1036,6 +1083,7 @@ int multifd_load_cleanup(Error **errp)
>          p->c = NULL;
>          qemu_mutex_destroy(&p->mutex);
>          qemu_sem_destroy(&p->sem);
> +        qemu_sem_destroy(&p->sem_sync);
>          g_free(p->name);
>          p->name = NULL;
>          multifd_pages_clear(p->pages);
> @@ -1044,6 +1092,7 @@ int multifd_load_cleanup(Error **errp)
>          g_free(p->packet);
>          p->packet = NULL;
>      }
> +    qemu_sem_destroy(&multifd_recv_state->sem_sync);
>      g_free(multifd_recv_state->params);
>      multifd_recv_state->params = NULL;
>      g_free(multifd_recv_state);
> @@ -1052,6 +1101,42 @@ int multifd_load_cleanup(Error **errp)
>      return ret;
>  }
>  
> +static void multifd_recv_sync_main(void)
> +{
> +    int i;
> +
> +    if (!migrate_use_multifd()) {
> +        return;
> +    }
> +    for (i = 0; i < migrate_multifd_channels(); i++) {
> +        MultiFDRecvParams *p = &multifd_recv_state->params[i];
> +
> +        trace_multifd_recv_sync_main_signal(p->id);
> +        qemu_mutex_lock(&p->mutex);
> +        p->pending_job = true;
> +        qemu_mutex_unlock(&p->mutex);
> +    }
> +    for (i = 0; i < migrate_multifd_channels(); i++) {
> +        MultiFDRecvParams *p = &multifd_recv_state->params[i];
> +
> +        trace_multifd_recv_sync_main_wait(p->id);
> +        qemu_sem_wait(&multifd_recv_state->sem_sync);
> +        qemu_mutex_lock(&p->mutex);
> +        if (atomic_read(&multifd_recv_state->packet_num) < p->packet_num) {
> +            atomic_set(&multifd_recv_state->packet_num, p->packet_num);
> +        }
> +        qemu_mutex_unlock(&p->mutex);
> +    }
> +    for (i = 0; i < migrate_multifd_channels(); i++) {
> +        MultiFDRecvParams *p = &multifd_recv_state->params[i];
> +
> +        trace_multifd_recv_sync_main_signal(p->id);
> +
> +        qemu_sem_post(&p->sem_sync);
> +    }
> +    trace_multifd_recv_sync_main(atomic_read(&multifd_recv_state->packet_num));
> +}
> +
>  static void *multifd_recv_thread(void *opaque)
>  {
>      MultiFDRecvParams *p = opaque;
> @@ -1061,37 +1146,30 @@ static void *multifd_recv_thread(void *opaque)
>      trace_multifd_recv_thread_start(p->id);
>  
>      while (true) {
> -        qemu_sem_wait(&p->sem);
> +        uint32_t used;
> +        uint32_t flags;
> +
> +        /* ToDo: recv packet here */
> +
>          qemu_mutex_lock(&p->mutex);
> -        if (p->pending_job) {
> -            uint32_t used;
> -            uint32_t flags;
> -            qemu_mutex_unlock(&p->mutex);
> -
> -            /* ToDo: recv packet here */
> -
> -            qemu_mutex_lock(&p->mutex);
> -            ret = multifd_recv_unfill_packet(p, &local_err);
> -            if (ret) {
> -                qemu_mutex_unlock(&p->mutex);
> -                break;
> -            }
> -
> -            used = p->pages->used;
> -            flags = p->flags;
> -            trace_multifd_recv(p->id, p->packet_num, used, flags);
> -            p->pending_job = false;
> -            p->num_packets++;
> -            p->num_pages += used;
> -            qemu_mutex_unlock(&p->mutex);
> -        } else if (p->quit) {
> +        ret = multifd_recv_unfill_packet(p, &local_err);
> +        if (ret) {
>              qemu_mutex_unlock(&p->mutex);
>              break;
>          }
> +
> +        used = p->pages->used;
> +        flags = p->flags;
> +        trace_multifd_recv(p->id, p->packet_num, used, flags);
> +        p->pending_job = false;
> +        p->num_packets++;
> +        p->num_pages += used;
>          qemu_mutex_unlock(&p->mutex);
> -        /* this is impossible */
> -        error_setg(&local_err, "multifd_recv_thread: Unknown command");
> -        break;
> +
> +        if (flags & MULTIFD_FLAG_SYNC) {
> +            qemu_sem_post(&multifd_recv_state->sem_sync);
> +            qemu_sem_wait(&p->sem_sync);
> +        }
>      }
>  
>      if (local_err) {
> @@ -1119,12 +1197,14 @@ int multifd_load_setup(void)
>      multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state));
>      multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count);
>      atomic_set(&multifd_recv_state->count, 0);
> +    qemu_sem_init(&multifd_recv_state->sem_sync, 0);
>  
>      for (i = 0; i < thread_count; i++) {
>          MultiFDRecvParams *p = &multifd_recv_state->params[i];
>  
>          qemu_mutex_init(&p->mutex);
>          qemu_sem_init(&p->sem, 0);
> +        qemu_sem_init(&p->sem_sync, 0);
>          p->quit = false;
>          p->pending_job = false;
>          p->id = i;
> @@ -2882,6 +2962,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
>      ram_control_before_iterate(f, RAM_CONTROL_SETUP);
>      ram_control_after_iterate(f, RAM_CONTROL_SETUP);
>  
> +    multifd_send_sync_main();
>      qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
>  
>      return 0;
> @@ -2962,6 +3043,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
>       */
>      ram_control_after_iterate(f, RAM_CONTROL_ROUND);
>  
> +    multifd_send_sync_main();
>  out:
>      qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
>      ram_counters.transferred += 8;
> @@ -3015,6 +3097,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
>  
>      rcu_read_unlock();
>  
> +    multifd_send_sync_main();
>      qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
>  
>      return 0;
> @@ -3504,6 +3587,7 @@ static int ram_load_postcopy(QEMUFile *f)
>              break;
>          case RAM_SAVE_FLAG_EOS:
>              /* normal exit */
> +            multifd_recv_sync_main();
>              break;
>          default:
>              error_report("Unknown combination of migration flags: %#x"
> @@ -3692,6 +3776,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
>              break;
>          case RAM_SAVE_FLAG_EOS:
>              /* normal exit */
> +            multifd_recv_sync_main();
>              break;
>          default:
>              if (flags & RAM_SAVE_FLAG_HOOK) {
> diff --git a/migration/trace-events b/migration/trace-events
> index c667d98529..ea39bb2bc5 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -77,9 +77,15 @@ migration_bitmap_sync_start(void) ""
>  migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
>  migration_throttle(void) ""
>  multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags) "channel %d packet number %ld pages %d flags 0x%x"
> +multifd_recv_sync_main(uint32_t seq) "seq %d"
> +multifd_recv_sync_main_signal(uint8_t id) "channel %d"
> +multifd_recv_sync_main_wait(uint8_t id) "channel %d"
>  multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
>  multifd_recv_thread_start(uint8_t id) "%d"
>  multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags) "channel %d packet_num %ld pages %d flags 0x%x"
> +multifd_send_sync_main(uint32_t seq) "seq %d"
> +multifd_send_sync_main_signal(uint8_t id) "channel %d"
> +multifd_send_sync_main_wait(uint8_t id) "channel %d"
>  multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
>  multifd_send_thread_start(uint8_t id) "%d"
>  ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
> -- 
> 2.17.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Qemu-devel] [PATCH v14 00/12] Multifd
  2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
                   ` (11 preceding siblings ...)
  2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 12/12] migration: Stop sending whole pages through main channel Juan Quintela
@ 2018-06-20 10:44 ` no-reply
  12 siblings, 0 replies; 19+ messages in thread
From: no-reply @ 2018-06-20 10:44 UTC (permalink / raw)
  To: quintela; +Cc: famz, qemu-devel, lvivier, dgilbert, peterx

Hi,

This series failed docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

Type: series
Message-id: 20180620081524.5751-1-quintela@redhat.com
Subject: [Qemu-devel] [PATCH v14 00/12] Multifd

=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
d6d2762f4e migration: Stop sending whole pages through main channel
952c599f34 migration: Remove not needed semaphore and quit
1e3957bd5f migration: Wait for blocking IO
abb8fc1459 migration: Start sending messages
60ca501756 migration: Create ram_save_multifd_page
21b4e5212e migration: Synchronize multifd threads with main thread
d15b64ae2f migration: Add block where to send/receive packets
4fe7a1ff28 migration: Multifd channels always wait on the sem
b3170dd2e0 migration: Calculate transferred ram correctly
b78e868dbf migration: Add multifd traces for start/end thread
188376343e migration: Create multifd packet
f098ee2927 migration: Create multipage support

=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-d_u5sy7x/src/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
  BUILD   fedora
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-d_u5sy7x/src'
  GEN     /var/tmp/patchew-tester-tmp-d_u5sy7x/src/docker-src.2018-06-20-06.39.46.31712/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-d_u5sy7x/src/docker-src.2018-06-20-06.39.46.31712/qemu.tar.vroot'...
done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-d_u5sy7x/src/docker-src.2018-06-20-06.39.46.31712/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-d_u5sy7x/src/docker-src.2018-06-20-06.39.46.31712/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
  COPY    RUNNER
    RUN test-mingw in qemu:fedora 
Packages installed:
SDL2-devel-2.0.8-5.fc28.x86_64
bc-1.07.1-5.fc28.x86_64
bison-3.0.4-9.fc28.x86_64
bluez-libs-devel-5.49-3.fc28.x86_64
brlapi-devel-0.6.7-12.fc28.x86_64
bzip2-1.0.6-26.fc28.x86_64
bzip2-devel-1.0.6-26.fc28.x86_64
ccache-3.4.2-2.fc28.x86_64
clang-6.0.0-5.fc28.x86_64
device-mapper-multipath-devel-0.7.4-2.git07e7bd5.fc28.x86_64
findutils-4.6.0-19.fc28.x86_64
flex-2.6.1-7.fc28.x86_64
gcc-8.1.1-1.fc28.x86_64
gcc-c++-8.1.1-1.fc28.x86_64
gettext-0.19.8.1-14.fc28.x86_64
git-2.17.1-2.fc28.x86_64
glib2-devel-2.56.1-3.fc28.x86_64
glusterfs-api-devel-4.0.2-1.fc28.x86_64
gnutls-devel-3.6.2-1.fc28.x86_64
gtk3-devel-3.22.30-1.fc28.x86_64
hostname-3.20-3.fc28.x86_64
libaio-devel-0.3.110-11.fc28.x86_64
libasan-8.1.1-1.fc28.x86_64
libattr-devel-2.4.47-23.fc28.x86_64
libcap-devel-2.25-9.fc28.x86_64
libcap-ng-devel-0.7.9-1.fc28.x86_64
libcurl-devel-7.59.0-3.fc28.x86_64
libfdt-devel-1.4.6-4.fc28.x86_64
libpng-devel-1.6.34-3.fc28.x86_64
librbd-devel-12.2.5-1.fc28.x86_64
libssh2-devel-1.8.0-7.fc28.x86_64
libubsan-8.1.1-1.fc28.x86_64
libusbx-devel-1.0.21-6.fc28.x86_64
libxml2-devel-2.9.7-4.fc28.x86_64
llvm-6.0.0-11.fc28.x86_64
lzo-devel-2.08-12.fc28.x86_64
make-4.2.1-6.fc28.x86_64
mingw32-SDL2-2.0.5-3.fc27.noarch
mingw32-bzip2-1.0.6-9.fc27.noarch
mingw32-curl-7.57.0-1.fc28.noarch
mingw32-glib2-2.54.1-1.fc28.noarch
mingw32-gmp-6.1.2-2.fc27.noarch
mingw32-gnutls-3.5.13-2.fc27.noarch
mingw32-gtk3-3.22.16-1.fc27.noarch
mingw32-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw32-libpng-1.6.29-2.fc27.noarch
mingw32-libssh2-1.8.0-3.fc27.noarch
mingw32-libtasn1-4.13-1.fc28.noarch
mingw32-nettle-3.3-3.fc27.noarch
mingw32-pixman-0.34.0-3.fc27.noarch
mingw32-pkg-config-0.28-9.fc27.x86_64
mingw64-SDL2-2.0.5-3.fc27.noarch
mingw64-bzip2-1.0.6-9.fc27.noarch
mingw64-curl-7.57.0-1.fc28.noarch
mingw64-glib2-2.54.1-1.fc28.noarch
mingw64-gmp-6.1.2-2.fc27.noarch
mingw64-gnutls-3.5.13-2.fc27.noarch
mingw64-gtk3-3.22.16-1.fc27.noarch
mingw64-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw64-libpng-1.6.29-2.fc27.noarch
mingw64-libssh2-1.8.0-3.fc27.noarch
mingw64-libtasn1-4.13-1.fc28.noarch
mingw64-nettle-3.3-3.fc27.noarch
mingw64-pixman-0.34.0-3.fc27.noarch
mingw64-pkg-config-0.28-9.fc27.x86_64
ncurses-devel-6.1-5.20180224.fc28.x86_64
nettle-devel-3.4-2.fc28.x86_64
nss-devel-3.36.1-1.1.fc28.x86_64
numactl-devel-2.0.11-8.fc28.x86_64
package PyYAML is not installed
package libjpeg-devel is not installed
perl-5.26.2-411.fc28.x86_64
pixman-devel-0.34.0-8.fc28.x86_64
python3-3.6.5-1.fc28.x86_64
snappy-devel-1.1.7-5.fc28.x86_64
sparse-0.5.2-1.fc28.x86_64
spice-server-devel-0.14.0-4.fc28.x86_64
systemtap-sdt-devel-3.2-11.fc28.x86_64
tar-1.30-3.fc28.x86_64
usbredir-devel-0.7.1-7.fc28.x86_64
virglrenderer-devel-0.6.0-4.20170210git76b3da97b.fc28.x86_64
vte3-devel-0.36.5-6.fc28.x86_64
which-2.21-8.fc28.x86_64
xen-devel-4.10.1-3.fc28.x86_64
zlib-devel-1.2.11-8.fc28.x86_64

Environment variables:
TARGET_LIST=
PACKAGES=ccache gettext git tar PyYAML sparse flex bison python3 bzip2 hostname     gcc gcc-c++ llvm clang make perl which bc findutils glib2-devel     libaio-devel pixman-devel zlib-devel libfdt-devel libasan libubsan     bluez-libs-devel brlapi-devel bzip2-devel     device-mapper-multipath-devel glusterfs-api-devel gnutls-devel     gtk3-devel libattr-devel libcap-devel libcap-ng-devel libcurl-devel     libjpeg-devel libpng-devel librbd-devel libssh2-devel libusbx-devel     libxml2-devel lzo-devel ncurses-devel nettle-devel nss-devel     numactl-devel SDL2-devel snappy-devel spice-server-devel     systemtap-sdt-devel usbredir-devel virglrenderer-devel vte3-devel     xen-devel     mingw32-pixman mingw32-glib2 mingw32-gmp mingw32-SDL2 mingw32-pkg-config     mingw32-gtk3 mingw32-gnutls mingw32-nettle mingw32-libtasn1     mingw32-libjpeg-turbo mingw32-libpng mingw32-curl mingw32-libssh2     mingw32-bzip2     mingw64-pixman mingw64-glib2 mingw64-gmp mingw64-SDL2 mingw64-pkg-config     mingw64-gtk3 mingw64-gnutls mingw64-nettle mingw64-libtasn1     mingw64-libjpeg-turbo mingw64-libpng mingw64-curl mingw64-libssh2     mingw64-bzip2
J=8
V=
HOSTNAME=25d1de98df32
DEBUG=
SHOW_ENV=1
PWD=/
HOME=/root
CCACHE_DIR=/var/tmp/ccache
DISTTAG=f28container
QEMU_CONFIGURE_OPTS=--python=/usr/bin/python3
FGC=f28
TEST_DIR=/tmp/qemu-test
SHLVL=1
FEATURES=mingw clang pyyaml asan dtc
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAKEFLAGS= -j8
EXTRA_CONFIGURE_OPTS=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=x86_64-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=2.0 --with-gtkabi=3.0
Install prefix    /tmp/qemu-test/install
BIOS directory    /tmp/qemu-test/install
firmware path     /tmp/qemu-test/install/share/qemu-firmware
binary directory  /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory  /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory  /tmp/qemu-test/install
local state directory   queried at runtime
Windows SDK       no
Source path       /tmp/qemu-test/src
GIT binary        git
GIT submodules    
C compiler        x86_64-w64-mingw32-gcc
Host C compiler   cc
C++ compiler      x86_64-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS           rv
CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS       -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/pixman-1  -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -mms-bitfields -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/x86_64-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include  -m64 -mcx16 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv  -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/usr/x86_64-w64-mingw32/sys-root/mingw/include  -I/usr/x86_64-w64-mingw32/sys-root/mingw/include   -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/libpng16 
LDFLAGS           -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g 
QEMU_LDFLAGS      -L$(BUILD_DIR)/dtc/libfdt 
make              make
install           install
python            /usr/bin/python3 -B
smbd              /usr/sbin/smbd
module support    no
host CPU          x86_64
host big endian   no
target list       x86_64-softmmu aarch64-softmmu
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
SDL support       yes (2.0.5)
GTK support       yes (3.22.16)
GTK GL support    no
VTE support       no 
TLS priority      NORMAL
GNUTLS support    yes
GNUTLS rnd        yes
libgcrypt         no
libgcrypt kdf     no
nettle            yes (3.3)
nettle kdf        yes
libtasn1          yes
curses support    no
virgl support     no 
curl support      yes
mingw32 support   yes
Audio drivers     dsound
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS support    no
Multipath support no
VNC support       yes
VNC SASL support  no
VNC JPEG support  yes
VNC PNG support   yes
xen support       no
brlapi support    no
bluez  support    no
Documentation     no
PIE               no
vde support       no
netmap support    no
Linux AIO support no
ATTR/XATTR support no
Install blobs     yes
KVM support       no
HAX support       yes
HVF support       no
WHPX support      no
TCG support       yes
TCG debug enabled no
TCG interpreter   no
malloc trim support no
RDMA support      no
fdt support       git
membarrier        no
preadv support    no
fdatasync         no
madvise           no
posix_madvise     no
posix_memalign    no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends    simple
Trace output file trace-<pid>
spice support     no 
rbd support       no
xfsctl support    no
smartcard support no
libusb            no
usb net redir     no
OpenGL support    no
OpenGL dmabufs    no
libiscsi support  no
libnfs support    no
build guest agent yes
QGA VSS support   no
QGA w32 disk info yes
QGA MSI support   no
seccomp support   no
coroutine backend win32
coroutine pool    yes
debug stack usage no
crypto afalg      no
GlusterFS support no
gcov              gcov
gcov enabled      no
TPM support       yes
libssh2 support   yes
TPM passthrough   no
TPM emulator      no
QOM debugging     yes
Live block migration yes
lzo support       no
snappy support    no
bzip2 support     yes
NUMA host support no
libxml2           no
tcmalloc support  no
jemalloc support  no
avx2 optimization yes
replication support yes
VxHS block device no
capstone          no
docker            no
  GEN     x86_64-softmmu/config-devices.mak.tmp
  GEN     aarch64-softmmu/config-devices.mak.tmp
  GEN     config-host.h
  GEN     qemu-options.def
  GEN     qapi-gen
  GEN     trace/generated-tcg-tracers.h
  GEN     trace/generated-helpers-wrappers.h
  GEN     trace/generated-helpers.h
  GEN     trace/generated-helpers.c
  GEN     module_block.h
  GEN     x86_64-softmmu/config-devices.mak
  GEN     ui/input-keymap-atset1-to-qcode.c
  GEN     aarch64-softmmu/config-devices.mak
  GEN     ui/input-keymap-linux-to-qcode.c
  GEN     ui/input-keymap-qcode-to-atset1.c
  GEN     ui/input-keymap-qcode-to-atset2.c
  GEN     ui/input-keymap-qcode-to-atset3.c
  GEN     ui/input-keymap-qcode-to-linux.c
  GEN     ui/input-keymap-qcode-to-qnum.c
  GEN     ui/input-keymap-qcode-to-sun.c
  GEN     ui/input-keymap-qnum-to-qcode.c
  GEN     ui/input-keymap-usb-to-qcode.c
  GEN     ui/input-keymap-win32-to-qcode.c
  GEN     ui/input-keymap-x11-to-qcode.c
  GEN     ui/input-keymap-xorgevdev-to-qcode.c
  GEN     ui/input-keymap-xorgkbd-to-qcode.c
  GEN     ui/input-keymap-xorgxwin-to-qcode.c
  GEN     ui/input-keymap-xorgxquartz-to-qcode.c
  GEN     ui/input-keymap-osx-to-qcode.c
  GEN     tests/test-qapi-gen
  GEN     trace-root.h
  GEN     accel/kvm/trace.h
  GEN     accel/tcg/trace.h
  GEN     audio/trace.h
  GEN     block/trace.h
  GEN     chardev/trace.h
  GEN     crypto/trace.h
  GEN     hw/9pfs/trace.h
  GEN     hw/acpi/trace.h
  GEN     hw/alpha/trace.h
  GEN     hw/arm/trace.h
  GEN     hw/audio/trace.h
  GEN     hw/block/trace.h
  GEN     hw/block/dataplane/trace.h
  GEN     hw/char/trace.h
  GEN     hw/display/trace.h
  GEN     hw/dma/trace.h
  GEN     hw/hppa/trace.h
  GEN     hw/i2c/trace.h
  GEN     hw/i386/trace.h
  GEN     hw/i386/xen/trace.h
  GEN     hw/ide/trace.h
  GEN     hw/input/trace.h
  GEN     hw/intc/trace.h
  GEN     hw/isa/trace.h
  GEN     hw/mem/trace.h
  GEN     hw/misc/trace.h
  GEN     hw/misc/macio/trace.h
  GEN     hw/net/trace.h
  GEN     hw/nvram/trace.h
  GEN     hw/pci/trace.h
  GEN     hw/pci-host/trace.h
  GEN     hw/ppc/trace.h
  GEN     hw/rdma/trace.h
  GEN     hw/rdma/vmw/trace.h
  GEN     hw/s390x/trace.h
  GEN     hw/scsi/trace.h
  GEN     hw/sd/trace.h
  GEN     hw/sparc/trace.h
  GEN     hw/sparc64/trace.h
  GEN     hw/timer/trace.h
  GEN     hw/tpm/trace.h
  GEN     hw/usb/trace.h
  GEN     hw/vfio/trace.h
  GEN     hw/virtio/trace.h
  GEN     hw/xen/trace.h
  GEN     io/trace.h
  GEN     linux-user/trace.h
  GEN     migration/trace.h
  GEN     nbd/trace.h
  GEN     net/trace.h
  GEN     qapi/trace.h
  GEN     qom/trace.h
  GEN     scsi/trace.h
  GEN     target/arm/trace.h
  GEN     target/i386/trace.h
  GEN     target/mips/trace.h
  GEN     target/ppc/trace.h
  GEN     target/s390x/trace.h
  GEN     target/sparc/trace.h
  GEN     ui/trace.h
  GEN     util/trace.h
  GEN     trace-root.c
  GEN     accel/kvm/trace.c
  GEN     accel/tcg/trace.c
  GEN     audio/trace.c
  GEN     block/trace.c
  GEN     chardev/trace.c
  GEN     crypto/trace.c
  GEN     hw/9pfs/trace.c
  GEN     hw/acpi/trace.c
  GEN     hw/alpha/trace.c
  GEN     hw/arm/trace.c
  GEN     hw/audio/trace.c
  GEN     hw/block/trace.c
  GEN     hw/block/dataplane/trace.c
  GEN     hw/char/trace.c
  GEN     hw/display/trace.c
  GEN     hw/dma/trace.c
  GEN     hw/hppa/trace.c
  GEN     hw/i2c/trace.c
  GEN     hw/i386/trace.c
  GEN     hw/i386/xen/trace.c
  GEN     hw/ide/trace.c
  GEN     hw/input/trace.c
  GEN     hw/intc/trace.c
  GEN     hw/isa/trace.c
  GEN     hw/mem/trace.c
  GEN     hw/misc/trace.c
  GEN     hw/misc/macio/trace.c
  GEN     hw/net/trace.c
  GEN     hw/nvram/trace.c
  GEN     hw/pci/trace.c
  GEN     hw/pci-host/trace.c
  GEN     hw/ppc/trace.c
  GEN     hw/rdma/trace.c
  GEN     hw/rdma/vmw/trace.c
  GEN     hw/s390x/trace.c
  GEN     hw/scsi/trace.c
  GEN     hw/sd/trace.c
  GEN     hw/sparc/trace.c
  GEN     hw/sparc64/trace.c
  GEN     hw/timer/trace.c
  GEN     hw/tpm/trace.c
  GEN     hw/usb/trace.c
  GEN     hw/vfio/trace.c
  GEN     hw/virtio/trace.c
  GEN     hw/xen/trace.c
  GEN     io/trace.c
  GEN     linux-user/trace.c
  GEN     migration/trace.c
  GEN     nbd/trace.c
  GEN     net/trace.c
  GEN     qapi/trace.c
  GEN     qom/trace.c
  GEN     scsi/trace.c
  GEN     target/arm/trace.c
  GEN     target/i386/trace.c
  GEN     target/mips/trace.c
  GEN     target/ppc/trace.c
  GEN     target/s390x/trace.c
  GEN     target/sparc/trace.c
  GEN     ui/trace.c
  GEN     util/trace.c
  GEN     config-all-devices.mak
	 DEP /tmp/qemu-test/src/dtc/tests/dumptrees.c
	 DEP /tmp/qemu-test/src/dtc/tests/trees.S
	 DEP /tmp/qemu-test/src/dtc/tests/testutils.c
	 DEP /tmp/qemu-test/src/dtc/tests/value-labels.c
	 DEP /tmp/qemu-test/src/dtc/tests/asm_tree_dump.c
	 DEP /tmp/qemu-test/src/dtc/tests/truncated_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/check_path.c
	 DEP /tmp/qemu-test/src/dtc/tests/overlay_bad_fixup.c
	 DEP /tmp/qemu-test/src/dtc/tests/subnode_iterate.c
	 DEP /tmp/qemu-test/src/dtc/tests/overlay.c
	 DEP /tmp/qemu-test/src/dtc/tests/property_iterate.c
	 DEP /tmp/qemu-test/src/dtc/tests/integer-expressions.c
	 DEP /tmp/qemu-test/src/dtc/tests/utilfdt_test.c
	 DEP /tmp/qemu-test/src/dtc/tests/path_offset_aliases.c
	 DEP /tmp/qemu-test/src/dtc/tests/add_subnode_with_nops.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_unordered.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtb_reverse.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_ordered.c
	 DEP /tmp/qemu-test/src/dtc/tests/extra-terminating-null.c
	 DEP /tmp/qemu-test/src/dtc/tests/incbin.c
	 DEP /tmp/qemu-test/src/dtc/tests/boot-cpuid.c
	 DEP /tmp/qemu-test/src/dtc/tests/phandle_format.c
	 DEP /tmp/qemu-test/src/dtc/tests/path-references.c
	 DEP /tmp/qemu-test/src/dtc/tests/references.c
	 DEP /tmp/qemu-test/src/dtc/tests/string_escapes.c
	 DEP /tmp/qemu-test/src/dtc/tests/propname_escapes.c
	 DEP /tmp/qemu-test/src/dtc/tests/appendprop2.c
	 DEP /tmp/qemu-test/src/dtc/tests/appendprop1.c
	 DEP /tmp/qemu-test/src/dtc/tests/del_node.c
	 DEP /tmp/qemu-test/src/dtc/tests/del_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/setprop.c
	 DEP /tmp/qemu-test/src/dtc/tests/set_name.c
	 DEP /tmp/qemu-test/src/dtc/tests/rw_tree1.c
	 DEP /tmp/qemu-test/src/dtc/tests/open_pack.c
	 DEP /tmp/qemu-test/src/dtc/tests/nopulate.c
	 DEP /tmp/qemu-test/src/dtc/tests/mangle-layout.c
	 DEP /tmp/qemu-test/src/dtc/tests/move_and_save.c
	 DEP /tmp/qemu-test/src/dtc/tests/sw_tree1.c
	 DEP /tmp/qemu-test/src/dtc/tests/nop_node.c
	 DEP /tmp/qemu-test/src/dtc/tests/nop_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/setprop_inplace.c
	 DEP /tmp/qemu-test/src/dtc/tests/stringlist.c
	 DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells.c
	 DEP /tmp/qemu-test/src/dtc/tests/notfound.c
	 DEP /tmp/qemu-test/src/dtc/tests/char_literal.c
	 DEP /tmp/qemu-test/src/dtc/tests/sized_cells.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_alias.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_compatible.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_check_compatible.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_phandle.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_prop_value.c
	 DEP /tmp/qemu-test/src/dtc/tests/parent_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/supernode_atdepth_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_path.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_phandle.c
	 DEP /tmp/qemu-test/src/dtc/tests/getprop.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_name.c
	 DEP /tmp/qemu-test/src/dtc/tests/path_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/subnode_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/find_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/root_node.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_mem_rsv.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_overlay.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_addresses.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_empty_tree.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_strerror.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_rw.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_wip.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_sw.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_ro.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt.c
	 DEP /tmp/qemu-test/src/dtc/util.c
	 DEP /tmp/qemu-test/src/dtc/fdtput.c
	 DEP /tmp/qemu-test/src/dtc/fdtoverlay.c
	 DEP /tmp/qemu-test/src/dtc/fdtget.c
	 DEP /tmp/qemu-test/src/dtc/fdtdump.c
	 LEX convert-dtsv0-lexer.lex.c
	 DEP /tmp/qemu-test/src/dtc/srcpos.c
	 BISON dtc-parser.tab.c
	 LEX dtc-lexer.lex.c
	 DEP /tmp/qemu-test/src/dtc/treesource.c
	 DEP /tmp/qemu-test/src/dtc/livetree.c
	 DEP /tmp/qemu-test/src/dtc/fstree.c
	 DEP /tmp/qemu-test/src/dtc/flattree.c
	 DEP /tmp/qemu-test/src/dtc/dtc.c
	 DEP /tmp/qemu-test/src/dtc/data.c
	 DEP /tmp/qemu-test/src/dtc/checks.c
	 DEP convert-dtsv0-lexer.lex.c
	 DEP dtc-parser.tab.c
	 DEP dtc-lexer.lex.c
	CHK version_gen.h
	UPD version_gen.h
	 DEP /tmp/qemu-test/src/dtc/util.c
	 CC libfdt/fdt.o
	 CC libfdt/fdt_ro.o
	 CC libfdt/fdt_wip.o
	 CC libfdt/fdt_sw.o
	 CC libfdt/fdt_empty_tree.o
	 CC libfdt/fdt_rw.o
	 CC libfdt/fdt_strerror.o
	 CC libfdt/fdt_addresses.o
	 CC libfdt/fdt_overlay.o
	 AR libfdt/libfdt.a
x86_64-w64-mingw32-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
  RC      version.o
  GEN     qga/qapi-generated/qapi-gen
  CC      qapi/qapi-builtin-types.o
  CC      qapi/qapi-types-block-core.o
  CC      qapi/qapi-types.o
  CC      qapi/qapi-types-char.o
  CC      qapi/qapi-types-block.o
  CC      qapi/qapi-types-common.o
  CC      qapi/qapi-types-crypto.o
  CC      qapi/qapi-types-introspect.o
  CC      qapi/qapi-types-job.o
  CC      qapi/qapi-types-migration.o
  CC      qapi/qapi-types-misc.o
  CC      qapi/qapi-types-net.o
  CC      qapi/qapi-types-rocker.o
  CC      qapi/qapi-types-run-state.o
  CC      qapi/qapi-types-sockets.o
  CC      qapi/qapi-types-tpm.o
  CC      qapi/qapi-types-trace.o
  CC      qapi/qapi-types-transaction.o
  CC      qapi/qapi-types-ui.o
  CC      qapi/qapi-builtin-visit.o
  CC      qapi/qapi-visit.o
  CC      qapi/qapi-visit-block-core.o
  CC      qapi/qapi-visit-block.o
  CC      qapi/qapi-visit-char.o
  CC      qapi/qapi-visit-common.o
  CC      qapi/qapi-visit-crypto.o
  CC      qapi/qapi-visit-introspect.o
  CC      qapi/qapi-visit-job.o
  CC      qapi/qapi-visit-migration.o
  CC      qapi/qapi-visit-misc.o
  CC      qapi/qapi-visit-net.o
  CC      qapi/qapi-visit-rocker.o
  CC      qapi/qapi-visit-run-state.o
  CC      qapi/qapi-visit-sockets.o
  CC      qapi/qapi-visit-tpm.o
  CC      qapi/qapi-visit-trace.o
  CC      qapi/qapi-visit-transaction.o
  CC      qapi/qapi-visit-ui.o
  CC      qapi/qapi-events.o
  CC      qapi/qapi-events-block-core.o
  CC      qapi/qapi-events-block.o
  CC      qapi/qapi-events-char.o
  CC      qapi/qapi-events-common.o
  CC      qapi/qapi-events-crypto.o
  CC      qapi/qapi-events-introspect.o
  CC      qapi/qapi-events-job.o
  CC      qapi/qapi-events-migration.o
  CC      qapi/qapi-events-misc.o
  CC      qapi/qapi-events-net.o
  CC      qapi/qapi-events-rocker.o
  CC      qapi/qapi-events-run-state.o
  CC      qapi/qapi-events-sockets.o
  CC      qapi/qapi-events-tpm.o
  CC      qapi/qapi-events-trace.o
  CC      qapi/qapi-events-transaction.o
  CC      qapi/qapi-events-ui.o
  CC      qapi/qapi-introspect.o
  CC      qapi/qapi-visit-core.o
  CC      qapi/qapi-dealloc-visitor.o
  CC      qapi/qobject-input-visitor.o
  CC      qapi/qobject-output-visitor.o
  CC      qapi/qmp-registry.o
  CC      qapi/qmp-dispatch.o
  CC      qapi/string-output-visitor.o
  CC      qapi/string-input-visitor.o
  CC      qapi/opts-visitor.o
  CC      qapi/qapi-clone-visitor.o
  CC      qapi/qmp-event.o
  CC      qapi/qapi-util.o
  CC      qobject/qnull.o
  CC      qobject/qnum.o
  CC      qobject/qstring.o
  CC      qobject/qdict.o
  CC      qobject/qlist.o
  CC      qobject/qbool.o
  CC      qobject/qlit.o
  CC      qobject/qjson.o
  CC      qobject/qobject.o
  CC      qobject/json-lexer.o
  CC      qobject/json-streamer.o
  CC      qobject/json-parser.o
  CC      qobject/block-qdict.o
  CC      trace/simple.o
  CC      trace/control.o
  CC      trace/qmp.o
  CC      util/osdep.o
  CC      util/cutils.o
  CC      util/unicode.o
  CC      util/qemu-timer-common.o
  CC      util/bufferiszero.o
  CC      util/lockcnt.o
  CC      util/aiocb.o
  CC      util/async.o
  CC      util/aio-wait.o
  CC      util/thread-pool.o
  CC      util/qemu-timer.o
  CC      util/main-loop.o
  CC      util/iohandler.o
  CC      util/aio-win32.o
  CC      util/event_notifier-win32.o
  CC      util/oslib-win32.o
  CC      util/envlist.o
  CC      util/qemu-thread-win32.o
  CC      util/path.o
  CC      util/module.o
  CC      util/host-utils.o
  CC      util/bitops.o
  CC      util/bitmap.o
  CC      util/hbitmap.o
  CC      util/fifo8.o
  CC      util/acl.o
  CC      util/cacheinfo.o
  CC      util/error.o
  CC      util/qemu-error.o
  CC      util/id.o
  CC      util/iov.o
  CC      util/qemu-config.o
  CC      util/qemu-sockets.o
  CC      util/uri.o
  CC      util/notify.o
  CC      util/qemu-option.o
  CC      util/qemu-progress.o
  CC      util/keyval.o
  CC      util/hexdump.o
  CC      util/crc32c.o
  CC      util/uuid.o
  CC      util/getauxval.o
  CC      util/throttle.o
  CC      util/readline.o
  CC      util/rcu.o
  CC      util/qemu-coroutine.o
  CC      util/qemu-coroutine-lock.o
  CC      util/qemu-coroutine-io.o
  CC      util/qemu-coroutine-sleep.o
  CC      util/coroutine-win32.o
  CC      util/buffer.o
  CC      util/timed-average.o
  CC      util/base64.o
  CC      util/log.o
  CC      util/pagesize.o
  CC      util/qdist.o
  CC      util/qht.o
  CC      util/range.o
  CC      util/stats64.o
  CC      util/systemd.o
  CC      util/iova-tree.o
  CC      trace-root.o
  CC      accel/kvm/trace.o
  CC      accel/tcg/trace.o
  CC      audio/trace.o
  CC      block/trace.o
  CC      crypto/trace.o
  CC      chardev/trace.o
  CC      hw/9pfs/trace.o
  CC      hw/acpi/trace.o
  CC      hw/alpha/trace.o
  CC      hw/arm/trace.o
  CC      hw/audio/trace.o
  CC      hw/block/trace.o
  CC      hw/block/dataplane/trace.o
  CC      hw/char/trace.o
  CC      hw/display/trace.o
  CC      hw/dma/trace.o
  CC      hw/hppa/trace.o
  CC      hw/i2c/trace.o
  CC      hw/i386/trace.o
  CC      hw/i386/xen/trace.o
  CC      hw/ide/trace.o
  CC      hw/input/trace.o
  CC      hw/intc/trace.o
  CC      hw/isa/trace.o
  CC      hw/mem/trace.o
  CC      hw/misc/trace.o
  CC      hw/misc/macio/trace.o
  CC      hw/net/trace.o
  CC      hw/nvram/trace.o
  CC      hw/pci/trace.o
  CC      hw/pci-host/trace.o
  CC      hw/ppc/trace.o
  CC      hw/rdma/trace.o
  CC      hw/rdma/vmw/trace.o
  CC      hw/s390x/trace.o
  CC      hw/scsi/trace.o
  CC      hw/sd/trace.o
  CC      hw/sparc/trace.o
  CC      hw/sparc64/trace.o
  CC      hw/timer/trace.o
  CC      hw/tpm/trace.o
  CC      hw/usb/trace.o
  CC      hw/vfio/trace.o
  CC      hw/virtio/trace.o
  CC      io/trace.o
  CC      hw/xen/trace.o
  CC      linux-user/trace.o
  CC      migration/trace.o
  CC      nbd/trace.o
  CC      net/trace.o
  CC      qapi/trace.o
  CC      qom/trace.o
  CC      scsi/trace.o
  CC      target/arm/trace.o
  CC      target/i386/trace.o
  CC      target/mips/trace.o
  CC      target/ppc/trace.o
  CC      target/s390x/trace.o
  CC      target/sparc/trace.o
  CC      ui/trace.o
  CC      util/trace.o
  CC      crypto/pbkdf-stub.o
  CC      stubs/arch-query-cpu-def.o
  CC      stubs/arch-query-cpu-model-expansion.o
  CC      stubs/arch-query-cpu-model-comparison.o
  CC      stubs/arch-query-cpu-model-baseline.o
  CC      stubs/bdrv-next-monitor-owned.o
  CC      stubs/blk-commit-all.o
  CC      stubs/blockdev-close-all-bdrv-states.o
  CC      stubs/clock-warp.o
  CC      stubs/cpu-get-clock.o
  CC      stubs/dump.o
  CC      stubs/cpu-get-icount.o
  CC      stubs/error-printf.o
  CC      stubs/fdset.o
  CC      stubs/gdbstub.o
  CC      stubs/get-vm-name.o
  CC      stubs/iothread.o
  CC      stubs/iothread-lock.o
  CC      stubs/is-daemonized.o
  CC      stubs/machine-init-done.o
  CC      stubs/migr-blocker.o
  CC      stubs/change-state-handler.o
  CC      stubs/monitor.o
  CC      stubs/notify-event.o
  CC      stubs/qtest.o
  CC      stubs/replay.o
  CC      stubs/runstate-check.o
  CC      stubs/set-fd-handler.o
  CC      stubs/slirp.o
  CC      stubs/sysbus.o
  CC      stubs/tpm.o
  CC      stubs/trace-control.o
  CC      stubs/uuid.o
  CC      stubs/vm-stop.o
  CC      stubs/vmstate.o
  CC      stubs/fd-register.o
  CC      stubs/qmp_memory_device.o
  CC      stubs/target-monitor-defs.o
  CC      stubs/target-get-monitor-def.o
  CC      stubs/pc_madt_cpu_entry.o
  CC      stubs/vmgenid.o
  CC      stubs/xen-common.o
  CC      stubs/xen-hvm.o
  CC      stubs/pci-host-piix.o
  CC      stubs/ram-block.o
  GEN     qemu-img-cmds.h
  CC      block.o
  CC      blockjob.o
  CC      job.o
  CC      qemu-io-cmds.o
  CC      replication.o
  CC      block/raw-format.o
  CC      block/qcow.o
  CC      block/vmdk.o
  CC      block/vdi.o
  CC      block/cloop.o
  CC      block/bochs.o
  CC      block/vpc.o
  CC      block/vvfat.o
  CC      block/dmg.o
  CC      block/qcow2.o
  CC      block/qcow2-refcount.o
  CC      block/qcow2-cluster.o
  CC      block/qcow2-snapshot.o
  CC      block/qcow2-cache.o
  CC      block/qcow2-bitmap.o
  CC      block/qed.o
  CC      block/qed-l2-cache.o
  CC      block/qed-table.o
  CC      block/qed-cluster.o
  CC      block/qed-check.o
  CC      block/vhdx.o
  CC      block/vhdx-endian.o
  CC      block/vhdx-log.o
  CC      block/quorum.o
  CC      block/parallels.o
  CC      block/blkdebug.o
  CC      block/blkverify.o
  CC      block/blkreplay.o
  CC      block/block-backend.o
  CC      block/snapshot.o
  CC      block/file-win32.o
  CC      block/qapi.o
  CC      block/win32-aio.o
  CC      block/null.o
  CC      block/mirror.o
  CC      block/commit.o
  CC      block/io.o
  CC      block/create.o
  CC      block/throttle-groups.o
  CC      block/nbd.o
  CC      block/nbd-client.o
  CC      block/sheepdog.o
  CC      block/accounting.o
  CC      block/dirty-bitmap.o
  CC      block/write-threshold.o
  CC      block/backup.o
  CC      block/replication.o
  CC      block/throttle.o
  CC      block/copy-on-read.o
  CC      block/crypto.o
  CC      nbd/server.o
  CC      nbd/client.o
  CC      nbd/common.o
  CC      scsi/utils.o
  CC      block/curl.o
  CC      block/ssh.o
  CC      block/dmg-bz2.o
  CC      crypto/init.o
  CC      crypto/hash.o
  CC      crypto/hash-nettle.o
  CC      crypto/hmac.o
  CC      crypto/hmac-nettle.o
  CC      crypto/aes.o
  CC      crypto/desrfb.o
  CC      crypto/cipher.o
  CC      crypto/tlscreds.o
  CC      crypto/tlscredsanon.o
  CC      crypto/tlscredsx509.o
  CC      crypto/tlssession.o
  CC      crypto/secret.o
  CC      crypto/random-gnutls.o
  CC      crypto/pbkdf.o
  CC      crypto/ivgen.o
  CC      crypto/pbkdf-nettle.o
  CC      crypto/ivgen-essiv.o
  CC      crypto/ivgen-plain.o
  CC      crypto/ivgen-plain64.o
  CC      crypto/afsplit.o
  CC      crypto/xts.o
  CC      crypto/block-qcow.o
  CC      crypto/block.o
  CC      crypto/block-luks.o
  CC      io/channel.o
  CC      io/channel-buffer.o
  CC      io/channel-command.o
  CC      io/channel-socket.o
  CC      io/channel-file.o
  CC      io/channel-tls.o
  CC      io/channel-watch.o
  CC      io/channel-websock.o
  CC      io/channel-util.o
  CC      io/dns-resolver.o
  CC      io/net-listener.o
  CC      io/task.o
  CC      qom/object.o
  CC      qom/qom-qobject.o
  CC      qom/container.o
  CC      qom/object_interfaces.o
  CC      qemu-io.o
  CC      blockdev.o
  CC      blockdev-nbd.o
  CC      bootdevice.o
  CC      iothread.o
  CC      job-qmp.o
  CC      qdev-monitor.o
  CC      device-hotplug.o
  CC      os-win32.o
  CC      bt-vhci.o
  CC      bt-host.o
  CC      dma-helpers.o
  CC      vl.o
  CC      tpm.o
  CC      device_tree.o
  CC      qapi/qapi-commands.o
  CC      qapi/qapi-commands-block-core.o
  CC      qapi/qapi-commands-block.o
  CC      qapi/qapi-commands-char.o
  CC      qapi/qapi-commands-common.o
  CC      qapi/qapi-commands-crypto.o
  CC      qapi/qapi-commands-job.o
  CC      qapi/qapi-commands-introspect.o
  CC      qapi/qapi-commands-migration.o
  CC      qapi/qapi-commands-misc.o
  CC      qapi/qapi-commands-net.o
  CC      qapi/qapi-commands-rocker.o
  CC      qapi/qapi-commands-run-state.o
  CC      qapi/qapi-commands-sockets.o
  CC      qapi/qapi-commands-tpm.o
  CC      qapi/qapi-commands-trace.o
  CC      qapi/qapi-commands-transaction.o
  CC      qapi/qapi-commands-ui.o
  CC      qmp.o
  CC      hmp.o
  CC      cpus-common.o
  CC      audio/audio.o
  CC      audio/noaudio.o
  CC      audio/wavaudio.o
  CC      audio/mixeng.o
  CC      audio/dsoundaudio.o
  CC      audio/audio_win_int.o
  CC      audio/wavcapture.o
  CC      backends/rng.o
  CC      backends/rng-egd.o
  CC      backends/tpm.o
  CC      backends/hostmem.o
  CC      backends/hostmem-ram.o
  CC      backends/cryptodev.o
  CC      backends/cryptodev-builtin.o
  CC      backends/cryptodev-vhost.o
  CC      block/stream.o
  CC      chardev/msmouse.o
  CC      chardev/wctablet.o
  CC      chardev/testdev.o
  CXX     disas/arm-a64.o
  CC      disas/arm.o
  CC      disas/i386.o
  CXX     disas/libvixl/vixl/utils.o
  CXX     disas/libvixl/vixl/compiler-intrinsics.o
  CXX     disas/libvixl/vixl/a64/instructions-a64.o
  CXX     disas/libvixl/vixl/a64/decoder-a64.o
  CXX     disas/libvixl/vixl/a64/disasm-a64.o
  CC      hw/acpi/core.o
  CC      hw/acpi/piix4.o
  CC      hw/acpi/pcihp.o
  CC      hw/acpi/ich9.o
  CC      hw/acpi/tco.o
  CC      hw/acpi/cpu_hotplug.o
  CC      hw/acpi/memory_hotplug.o
  CC      hw/acpi/cpu.o
  CC      hw/acpi/vmgenid.o
  CC      hw/acpi/nvdimm.o
  CC      hw/acpi/acpi_interface.o
  CC      hw/acpi/bios-linker-loader.o
  CC      hw/acpi/aml-build.o
  CC      hw/acpi/ipmi.o
  CC      hw/acpi/acpi-stub.o
  CC      hw/acpi/ipmi-stub.o
  CC      hw/audio/sb16.o
  CC      hw/audio/es1370.o
  CC      hw/audio/ac97.o
  CC      hw/audio/fmopl.o
  CC      hw/audio/adlib.o
  CC      hw/audio/gus.o
  CC      hw/audio/gusemu_hal.o
  CC      hw/audio/gusemu_mixer.o
  CC      hw/audio/cs4231a.o
  CC      hw/audio/intel-hda.o
  CC      hw/audio/hda-codec.o
  CC      hw/audio/pcspk.o
  CC      hw/audio/wm8750.o
  CC      hw/audio/pl041.o
  CC      hw/audio/marvell_88w8618.o
  CC      hw/block/block.o
  CC      hw/audio/lm4549.o
  CC      hw/audio/soundhw.o
  CC      hw/block/cdrom.o
  CC      hw/block/hd-geometry.o
  CC      hw/block/fdc.o
  CC      hw/block/m25p80.o
  CC      hw/block/nand.o
  CC      hw/block/pflash_cfi01.o
  CC      hw/block/pflash_cfi02.o
  CC      hw/block/ecc.o
  CC      hw/block/onenand.o
  CC      hw/block/nvme.o
  CC      hw/bt/core.o
  CC      hw/bt/l2cap.o
  CC      hw/bt/sdp.o
  CC      hw/bt/hci.o
  CC      hw/bt/hid.o
  CC      hw/bt/hci-csr.o
  CC      hw/char/ipoctal232.o
  CC      hw/char/parallel.o
  CC      hw/char/parallel-isa.o
  CC      hw/char/pl011.o
  CC      hw/char/serial.o
  CC      hw/char/serial-isa.o
  CC      hw/char/serial-pci.o
  CC      hw/char/virtio-console.o
  CC      hw/char/cadence_uart.o
  CC      hw/char/cmsdk-apb-uart.o
  CC      hw/char/debugcon.o
  CC      hw/char/imx_serial.o
  CC      hw/core/qdev.o
  CC      hw/core/qdev-properties.o
  CC      hw/core/bus.o
  CC      hw/core/reset.o
  CC      hw/core/qdev-fw.o
  CC      hw/core/fw-path-provider.o
  CC      hw/core/irq.o
  CC      hw/core/hotplug.o
  CC      hw/core/nmi.o
  CC      hw/core/stream.o
  CC      hw/core/ptimer.o
  CC      hw/core/sysbus.o
  CC      hw/core/machine.o
  CC      hw/core/loader.o
  CC      hw/core/qdev-properties-system.o
  CC      hw/core/register.o
  CC      hw/core/or-irq.o
  CC      hw/core/split-irq.o
  CC      hw/core/platform-bus.o
  CC      hw/cpu/core.o
  CC      hw/display/ramfb.o
  CC      hw/display/ramfb-standalone.o
  CC      hw/display/ads7846.o
  CC      hw/display/cirrus_vga.o
  CC      hw/display/pl110.o
  CC      hw/display/sii9022.o
  CC      hw/display/ssd0303.o
  CC      hw/display/ssd0323.o
  CC      hw/display/vga-pci.o
  CC      hw/display/bochs-display.o
  CC      hw/display/vga-isa.o
  CC      hw/display/vmware_vga.o
  CC      hw/display/blizzard.o
  CC      hw/display/exynos4210_fimd.o
  CC      hw/display/framebuffer.o
  CC      hw/dma/pl080.o
  CC      hw/display/tc6393xb.o
  CC      hw/dma/pl330.o
  CC      hw/dma/i8257.o
  CC      hw/dma/xilinx_axidma.o
  CC      hw/dma/xlnx-zynq-devcfg.o
  CC      hw/dma/xlnx-zdma.o
  CC      hw/gpio/max7310.o
  CC      hw/gpio/pl061.o
  CC      hw/gpio/zaurus.o
  CC      hw/gpio/gpio_key.o
  CC      hw/i2c/core.o
  CC      hw/i2c/smbus.o
  CC      hw/i2c/smbus_eeprom.o
  CC      hw/i2c/i2c-ddc.o
  CC      hw/i2c/versatile_i2c.o
  CC      hw/i2c/smbus_ich9.o
  CC      hw/i2c/pm_smbus.o
  CC      hw/i2c/bitbang_i2c.o
  CC      hw/i2c/exynos4210_i2c.o
  CC      hw/i2c/imx_i2c.o
  CC      hw/i2c/aspeed_i2c.o
  CC      hw/ide/core.o
  CC      hw/ide/atapi.o
  CC      hw/ide/qdev.o
  CC      hw/ide/pci.o
  CC      hw/ide/isa.o
  CC      hw/ide/microdrive.o
  CC      hw/ide/piix.o
  CC      hw/ide/ahci.o
  CC      hw/ide/ich.o
  CC      hw/ide/ahci-allwinner.o
  CC      hw/input/hid.o
  CC      hw/input/lm832x.o
  CC      hw/input/pckbd.o
  CC      hw/input/pl050.o
  CC      hw/input/ps2.o
  CC      hw/input/stellaris_input.o
  CC      hw/input/tsc2005.o
  CC      hw/input/virtio-input.o
  CC      hw/input/virtio-input-hid.o
  CC      hw/intc/i8259_common.o
  CC      hw/intc/i8259.o
  CC      hw/intc/pl190.o
  CC      hw/intc/xlnx-pmu-iomod-intc.o
  CC      hw/intc/xlnx-zynqmp-ipi.o
  CC      hw/intc/imx_avic.o
  CC      hw/intc/imx_gpcv2.o
  CC      hw/intc/realview_gic.o
  CC      hw/intc/ioapic_common.o
  CC      hw/intc/arm_gic_common.o
  CC      hw/intc/arm_gic.o
  CC      hw/intc/arm_gicv2m.o
  CC      hw/intc/arm_gicv3_common.o
  CC      hw/intc/arm_gicv3.o
  CC      hw/intc/arm_gicv3_dist.o
  CC      hw/intc/arm_gicv3_redist.o
  CC      hw/intc/arm_gicv3_its_common.o
  CC      hw/intc/intc.o
  CC      hw/ipack/ipack.o
  CC      hw/ipack/tpci200.o
  CC      hw/ipmi/ipmi.o
  CC      hw/ipmi/ipmi_bmc_sim.o
  CC      hw/ipmi/ipmi_bmc_extern.o
  CC      hw/ipmi/isa_ipmi_kcs.o
  CC      hw/ipmi/isa_ipmi_bt.o
  CC      hw/isa/isa-bus.o
  CC      hw/isa/smc37c669-superio.o
  CC      hw/isa/isa-superio.o
  CC      hw/isa/apm.o
  CC      hw/mem/pc-dimm.o
  CC      hw/mem/memory-device.o
  CC      hw/mem/nvdimm.o
  CC      hw/misc/applesmc.o
  CC      hw/misc/max111x.o
  CC      hw/misc/tmp105.o
  CC      hw/misc/tmp421.o
  CC      hw/misc/debugexit.o
  CC      hw/misc/sga.o
  CC      hw/misc/pc-testdev.o
  CC      hw/misc/pci-testdev.o
  CC      hw/misc/edu.o
  CC      hw/misc/pca9552.o
  CC      hw/misc/unimp.o
  CC      hw/misc/vmcoreinfo.o
  CC      hw/misc/arm_l2x0.o
  CC      hw/misc/arm_integrator_debug.o
  CC      hw/misc/a9scu.o
  CC      hw/misc/arm11scu.o
  CC      hw/net/ne2000.o
  CC      hw/net/eepro100.o
  CC      hw/net/pcnet-pci.o
  CC      hw/net/pcnet.o
  CC      hw/net/e1000.o
  CC      hw/net/e1000x_common.o
  CC      hw/net/net_tx_pkt.o
  CC      hw/net/net_rx_pkt.o
  CC      hw/net/e1000e.o
  CC      hw/net/e1000e_core.o
  CC      hw/net/rtl8139.o
  CC      hw/net/vmxnet3.o
  CC      hw/net/smc91c111.o
  CC      hw/net/lan9118.o
  CC      hw/net/ne2000-isa.o
  CC      hw/net/xgmac.o
  CC      hw/net/xilinx_axienet.o
  CC      hw/net/allwinner_emac.o
  CC      hw/net/imx_fec.o
  CC      hw/net/cadence_gem.o
  CC      hw/net/stellaris_enet.o
  CC      hw/net/ftgmac100.o
  CC      hw/net/rocker/rocker.o
  CC      hw/net/rocker/rocker_fp.o
  CC      hw/net/rocker/rocker_desc.o
  CC      hw/net/rocker/rocker_world.o
  CC      hw/net/rocker/rocker_of_dpa.o
  CC      hw/net/can/can_sja1000.o
  CC      hw/net/can/can_kvaser_pci.o
  CC      hw/net/can/can_pcm3680_pci.o
  CC      hw/net/can/can_mioe3680_pci.o
  CC      hw/nvram/eeprom93xx.o
  CC      hw/nvram/eeprom_at24c.o
  CC      hw/nvram/fw_cfg.o
  CC      hw/nvram/chrp_nvram.o
  CC      hw/pci-bridge/pci_bridge_dev.o
  CC      hw/pci-bridge/pcie_root_port.o
  CC      hw/pci-bridge/gen_pcie_root_port.o
  CC      hw/pci-bridge/pcie_pci_bridge.o
  CC      hw/pci-bridge/pci_expander_bridge.o
  CC      hw/pci-bridge/xio3130_upstream.o
  CC      hw/pci-bridge/xio3130_downstream.o
  CC      hw/pci-bridge/ioh3420.o
  CC      hw/pci-bridge/i82801b11.o
  CC      hw/pci-host/pam.o
  CC      hw/pci-host/versatile.o
  CC      hw/pci-host/piix.o
  CC      hw/pci-host/q35.o
  CC      hw/pci-host/gpex.o
  CC      hw/pci-host/designware.o
  CC      hw/pci/pci.o
  CC      hw/pci/pci_bridge.o
  CC      hw/pci/msix.o
  CC      hw/pci/msi.o
  CC      hw/pci/shpc.o
  CC      hw/pci/slotid_cap.o
  CC      hw/pci/pci_host.o
  CC      hw/pci/pcie_host.o
  CC      hw/pci/pcie.o
  CC      hw/pci/pcie_aer.o
  CC      hw/pci/pcie_port.o
  CC      hw/pci/pci-stub.o
  CC      hw/pcmcia/pcmcia.o
  CC      hw/scsi/scsi-disk.o
  CC      hw/scsi/scsi-generic.o
  CC      hw/scsi/scsi-bus.o
  CC      hw/scsi/lsi53c895a.o
  CC      hw/scsi/mptsas.o
  CC      hw/scsi/mptconfig.o
  CC      hw/scsi/mptendian.o
  CC      hw/scsi/megasas.o
  CC      hw/scsi/vmw_pvscsi.o
  CC      hw/scsi/esp.o
  CC      hw/scsi/esp-pci.o
  CC      hw/sd/pl181.o
  CC      hw/sd/ssi-sd.o
  CC      hw/sd/sd.o
  CC      hw/sd/core.o
  CC      hw/sd/sdmmc-internal.o
  CC      hw/sd/sdhci.o
  CC      hw/smbios/smbios.o
  CC      hw/smbios/smbios_type_38.o
  CC      hw/smbios/smbios-stub.o
  CC      hw/smbios/smbios_type_38-stub.o
  CC      hw/ssi/pl022.o
  CC      hw/ssi/ssi.o
  CC      hw/ssi/xilinx_spips.o
  CC      hw/ssi/aspeed_smc.o
  CC      hw/ssi/stm32f2xx_spi.o
  CC      hw/ssi/mss-spi.o
  CC      hw/timer/arm_timer.o
  CC      hw/timer/arm_mptimer.o
  CC      hw/timer/armv7m_systick.o
  CC      hw/timer/a9gtimer.o
  CC      hw/timer/cadence_ttc.o
  CC      hw/timer/ds1338.o
  CC      hw/timer/hpet.o
  CC      hw/timer/i8254_common.o
  CC      hw/timer/i8254.o
  CC      hw/timer/pl031.o
  CC      hw/timer/twl92230.o
  CC      hw/timer/imx_epit.o
  CC      hw/timer/imx_gpt.o
  CC      hw/timer/xlnx-zynqmp-rtc.o
  CC      hw/timer/stm32f2xx_timer.o
  CC      hw/timer/aspeed_timer.o
  CC      hw/timer/cmsdk-apb-timer.o
  CC      hw/timer/mss-timer.o
  CC      hw/tpm/tpm_util.o
  CC      hw/tpm/tpm_tis.o
  CC      hw/tpm/tpm_crb.o
  CC      hw/usb/core.o
  CC      hw/usb/combined-packet.o
  CC      hw/usb/bus.o
  CC      hw/usb/libhw.o
  CC      hw/usb/desc.o
  CC      hw/usb/desc-msos.o
  CC      hw/usb/hcd-uhci.o
  CC      hw/usb/hcd-ohci.o
  CC      hw/usb/hcd-ehci.o
  CC      hw/usb/hcd-ehci-pci.o
  CC      hw/usb/hcd-ehci-sysbus.o
  CC      hw/usb/hcd-xhci.o
  CC      hw/usb/hcd-xhci-nec.o
  CC      hw/usb/hcd-musb.o
  CC      hw/usb/dev-hub.o
  CC      hw/usb/dev-hid.o
  CC      hw/usb/dev-wacom.o
  CC      hw/usb/dev-storage.o
  CC      hw/usb/dev-uas.o
  CC      hw/usb/dev-audio.o
  CC      hw/usb/dev-serial.o
  CC      hw/usb/dev-network.o
  CC      hw/usb/dev-bluetooth.o
  CC      hw/usb/dev-smartcard-reader.o
  CC      hw/usb/host-stub.o
  CC      hw/virtio/virtio-bus.o
  CC      hw/virtio/virtio-rng.o
  CC      hw/virtio/virtio-pci.o
  CC      hw/virtio/virtio-mmio.o
  CC      hw/virtio/vhost-stub.o
  CC      hw/watchdog/watchdog.o
  CC      hw/watchdog/wdt_i6300esb.o
  CC      hw/watchdog/wdt_ib700.o
  CC      hw/watchdog/wdt_aspeed.o
  CC      migration/migration.o
  CC      migration/socket.o
  CC      migration/fd.o
  CC      migration/exec.o
  CC      migration/tls.o
  CC      migration/channel.o
  CC      migration/savevm.o
  CC      migration/colo-comm.o
  CC      migration/colo.o
  CC      migration/colo-failover.o
  CC      migration/vmstate.o
  CC      migration/vmstate-types.o
  CC      migration/page_cache.o
  CC      migration/qemu-file.o
  CC      migration/global_state.o
  CC      migration/qemu-file-channel.o
  CC      migration/xbzrle.o
  CC      migration/postcopy-ram.o
  CC      migration/qjson.o
  CC      migration/block-dirty-bitmap.o
  CC      migration/block.o
  CC      net/net.o
  CC      net/queue.o
  CC      net/checksum.o
  CC      net/hub.o
  CC      net/util.o
  CC      net/socket.o
  CC      net/dump.o
  CC      net/eth.o
  CC      net/slirp.o
  CC      net/filter.o
  CC      net/filter-buffer.o
  CC      net/filter-mirror.o
  CC      net/colo-compare.o
  CC      net/colo.o
  CC      net/filter-rewriter.o
  CC      net/filter-replay.o
  CC      net/tap-win32.o
  CC      net/can/can_core.o
  CC      net/can/can_host.o
  CC      qom/cpu.o
  CC      replay/replay.o
  CC      replay/replay-internal.o
  CC      replay/replay-events.o
  CC      replay/replay-time.o
  CC      replay/replay-input.o
  CC      replay/replay-char.o
  CC      replay/replay-snapshot.o
  CC      replay/replay-audio.o
  CC      replay/replay-net.o
  CC      slirp/cksum.o
  CC      slirp/if.o
  CC      slirp/ip_icmp.o
  CC      slirp/ip6_input.o
  CC      slirp/ip6_icmp.o
  CC      slirp/ip6_output.o
  CC      slirp/ip_input.o
  CC      slirp/ip_output.o
  CC      slirp/dnssearch.o
  CC      slirp/dhcpv6.o
  CC      slirp/slirp.o
  CC      slirp/mbuf.o
  CC      slirp/misc.o
  CC      slirp/sbuf.o
  CC      slirp/socket.o
  CC      slirp/tcp_input.o
  CC      slirp/tcp_output.o
  CC      slirp/tcp_subr.o
  CC      slirp/tcp_timer.o
  CC      slirp/udp.o
  CC      slirp/udp6.o
  CC      slirp/bootp.o
  CC      slirp/tftp.o
  CC      slirp/arp_table.o
  CC      slirp/ndp_table.o
  CC      slirp/ncsi.o
  CC      ui/keymaps.o
  CC      ui/console.o
  CC      ui/cursor.o
  CC      ui/qemu-pixman.o
  CC      ui/input.o
  CC      ui/input-keymap.o
  CC      ui/input-legacy.o
  CC      ui/vnc.o
  CC      ui/vnc-enc-zlib.o
  CC      ui/vnc-enc-hextile.o
  CC      ui/vnc-palette.o
  CC      ui/vnc-enc-tight.o
  CC      ui/vnc-enc-zrle.o
  CC      ui/vnc-auth-vencrypt.o
  CC      ui/vnc-ws.o
  CC      ui/vnc-jobs.o
  CC      ui/sdl2.o
  CC      ui/sdl2-input.o
  CC      ui/sdl2-2d.o
  CC      ui/gtk.o
  CC      chardev/char.o
  CC      chardev/char-console.o
  CC      chardev/char-fe.o
  CC      chardev/char-file.o
  CC      chardev/char-io.o
  CC      chardev/char-mux.o
  CC      chardev/char-null.o
  CC      chardev/char-pipe.o
  CC      chardev/char-ringbuf.o
  CC      chardev/char-serial.o
  CC      chardev/char-socket.o
  CC      chardev/char-stdio.o
  CC      chardev/char-udp.o
  CC      chardev/char-win-stdio.o
  CC      chardev/char-win.o
  CC      qga/commands.o
  CC      qga/guest-agent-command-state.o
  CC      qga/main.o
  CC      qga/commands-win32.o
  CC      qga/channel-win32.o
  CC      qga/service-win32.o
  CC      qga/vss-win32.o
  CC      qga/qapi-generated/qga-qapi-types.o
  AS      optionrom/multiboot.o
  AS      optionrom/linuxboot.o
  CC      qga/qapi-generated/qga-qapi-visit.o
  CC      optionrom/linuxboot_dma.o
  CC      qga/qapi-generated/qga-qapi-commands.o
  AS      optionrom/kvmvapic.o
  BUILD   optionrom/multiboot.img
  BUILD   optionrom/linuxboot.img
  BUILD   optionrom/linuxboot_dma.img
  BUILD   optionrom/kvmvapic.img
  AR      libqemuutil.a
  BUILD   optionrom/multiboot.raw
  CC      qemu-img.o
  BUILD   optionrom/linuxboot.raw
  BUILD   optionrom/linuxboot_dma.raw
  SIGN    optionrom/multiboot.bin
  BUILD   optionrom/kvmvapic.raw
  SIGN    optionrom/linuxboot.bin
  SIGN    optionrom/linuxboot_dma.bin
  SIGN    optionrom/kvmvapic.bin
  LINK    qemu-io.exe
  LINK    qemu-img.exe
  LINK    qemu-ga.exe
  GEN     x86_64-softmmu/hmp-commands.h
  GEN     x86_64-softmmu/config-target.h
  GEN     x86_64-softmmu/hmp-commands-info.h
  CC      x86_64-softmmu/tcg/tcg.o
  CC      x86_64-softmmu/tcg/tcg-op-gvec.o
  CC      x86_64-softmmu/tcg/tcg-op.o
  CC      x86_64-softmmu/tcg/tcg-op-vec.o
  CC      x86_64-softmmu/exec.o
  CC      x86_64-softmmu/tcg/tcg-common.o
  CC      x86_64-softmmu/tcg/optimize.o
  CC      x86_64-softmmu/fpu/softfloat.o
  CC      x86_64-softmmu/disas.o
  GEN     x86_64-softmmu/gdbstub-xml.c
  CC      x86_64-softmmu/arch_init.o
  CC      x86_64-softmmu/cpus.o
  CC      x86_64-softmmu/monitor.o
  GEN     aarch64-softmmu/hmp-commands.h
  GEN     aarch64-softmmu/hmp-commands-info.h
  GEN     aarch64-softmmu/config-target.h
  CC      aarch64-softmmu/exec.o
  CC      aarch64-softmmu/tcg/tcg.o
  CC      x86_64-softmmu/gdbstub.o
  CC      x86_64-softmmu/balloon.o
  CC      x86_64-softmmu/ioport.o
  CC      aarch64-softmmu/tcg/tcg-op.o
  CC      x86_64-softmmu/numa.o
  CC      x86_64-softmmu/qtest.o
  CC      x86_64-softmmu/memory.o
  CC      x86_64-softmmu/memory_mapping.o
  CC      x86_64-softmmu/dump.o
  CC      x86_64-softmmu/migration/ram.o
  CC      x86_64-softmmu/accel/accel.o
  CC      x86_64-softmmu/accel/stubs/hvf-stub.o
  CC      x86_64-softmmu/accel/stubs/whpx-stub.o
  CC      x86_64-softmmu/accel/stubs/kvm-stub.o
  CC      x86_64-softmmu/accel/tcg/tcg-all.o
  CC      x86_64-softmmu/accel/tcg/cputlb.o
  CC      x86_64-softmmu/accel/tcg/tcg-runtime.o
  CC      x86_64-softmmu/accel/tcg/tcg-runtime-gvec.o
  CC      x86_64-softmmu/accel/tcg/cpu-exec.o
  CC      x86_64-softmmu/accel/tcg/cpu-exec-common.o
  CC      x86_64-softmmu/accel/tcg/translate-all.o
  CC      x86_64-softmmu/accel/tcg/translator.o
  CC      x86_64-softmmu/hw/block/virtio-blk.o
  CC      x86_64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      x86_64-softmmu/hw/char/virtio-serial-bus.o
  CC      x86_64-softmmu/hw/core/generic-loader.o
  CC      x86_64-softmmu/hw/core/null-machine.o
  CC      x86_64-softmmu/hw/display/vga.o
  CC      x86_64-softmmu/hw/display/virtio-gpu.o
  CC      x86_64-softmmu/hw/display/virtio-gpu-3d.o
  CC      x86_64-softmmu/hw/display/virtio-gpu-pci.o
  CC      aarch64-softmmu/tcg/tcg-op-vec.o
  CC      x86_64-softmmu/hw/display/virtio-vga.o
  CC      aarch64-softmmu/tcg/tcg-op-gvec.o
  CC      x86_64-softmmu/hw/intc/apic.o
  CC      x86_64-softmmu/hw/intc/apic_common.o
  CC      x86_64-softmmu/hw/intc/ioapic.o
  CC      x86_64-softmmu/hw/isa/lpc_ich9.o
  CC      x86_64-softmmu/hw/misc/pvpanic.o
  CC      x86_64-softmmu/hw/misc/mmio_interface.o
  CC      x86_64-softmmu/hw/net/virtio-net.o
  CC      aarch64-softmmu/tcg/tcg-common.o
  CC      aarch64-softmmu/tcg/optimize.o
  CC      x86_64-softmmu/hw/net/vhost_net.o
  CC      x86_64-softmmu/hw/scsi/virtio-scsi.o
  CC      x86_64-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      aarch64-softmmu/fpu/softfloat.o
  CC      x86_64-softmmu/hw/timer/mc146818rtc.o
  GEN     aarch64-softmmu/gdbstub-xml.c
  CC      aarch64-softmmu/disas.o
  CC      x86_64-softmmu/hw/virtio/virtio.o
  CC      x86_64-softmmu/hw/virtio/virtio-balloon.o
  CC      x86_64-softmmu/hw/virtio/virtio-crypto.o
  CC      x86_64-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      x86_64-softmmu/hw/i386/multiboot.o
  CC      x86_64-softmmu/hw/i386/pc.o
  CC      x86_64-softmmu/hw/i386/pc_piix.o
  CC      x86_64-softmmu/hw/i386/pc_q35.o
  CC      x86_64-softmmu/hw/i386/pc_sysfw.o
  CC      x86_64-softmmu/hw/i386/x86-iommu.o
  CC      x86_64-softmmu/hw/i386/intel_iommu.o
  CC      x86_64-softmmu/hw/i386/amd_iommu.o
  CC      x86_64-softmmu/hw/i386/vmport.o
  CC      x86_64-softmmu/hw/i386/vmmouse.o
  CC      x86_64-softmmu/hw/i386/kvmvapic.o
  CC      x86_64-softmmu/hw/i386/acpi-build.o
  CC      x86_64-softmmu/target/i386/helper.o
  CC      aarch64-softmmu/arch_init.o
  CC      x86_64-softmmu/target/i386/cpu.o
  CC      x86_64-softmmu/target/i386/gdbstub.o
  CC      aarch64-softmmu/cpus.o
  CC      aarch64-softmmu/monitor.o
  CC      x86_64-softmmu/target/i386/xsave_helper.o
  CC      x86_64-softmmu/target/i386/translate.o
  CC      x86_64-softmmu/target/i386/bpt_helper.o
  CC      x86_64-softmmu/target/i386/cc_helper.o
  CC      x86_64-softmmu/target/i386/excp_helper.o
  CC      x86_64-softmmu/target/i386/fpu_helper.o
  CC      aarch64-softmmu/gdbstub.o
  CC      x86_64-softmmu/target/i386/int_helper.o
  CC      x86_64-softmmu/target/i386/mem_helper.o
  CC      x86_64-softmmu/target/i386/misc_helper.o
  CC      x86_64-softmmu/target/i386/mpx_helper.o
  CC      x86_64-softmmu/target/i386/seg_helper.o
  CC      x86_64-softmmu/target/i386/smm_helper.o
  CC      x86_64-softmmu/target/i386/svm_helper.o
  CC      x86_64-softmmu/target/i386/machine.o
  CC      x86_64-softmmu/target/i386/arch_memory_mapping.o
  CC      x86_64-softmmu/target/i386/arch_dump.o
  CC      x86_64-softmmu/target/i386/monitor.o
  CC      x86_64-softmmu/target/i386/kvm-stub.o
  CC      x86_64-softmmu/target/i386/sev-stub.o
  CC      x86_64-softmmu/target/i386/hax-mem.o
  CC      x86_64-softmmu/target/i386/hax-all.o
  CC      x86_64-softmmu/target/i386/hax-windows.o
  CC      x86_64-softmmu/trace/control-target.o
  CC      aarch64-softmmu/ioport.o
  GEN     trace/generated-helpers.c
  CC      aarch64-softmmu/balloon.o
  CC      aarch64-softmmu/numa.o
  CC      x86_64-softmmu/gdbstub-xml.o
  CC      aarch64-softmmu/qtest.o
  CC      aarch64-softmmu/memory.o
  CC      aarch64-softmmu/memory_mapping.o
  CC      aarch64-softmmu/dump.o
  CC      aarch64-softmmu/migration/ram.o
  CC      aarch64-softmmu/accel/accel.o
  CC      aarch64-softmmu/accel/stubs/hax-stub.o
  CC      aarch64-softmmu/accel/stubs/hvf-stub.o
  CC      aarch64-softmmu/accel/stubs/whpx-stub.o
  CC      aarch64-softmmu/accel/stubs/kvm-stub.o
  CC      aarch64-softmmu/accel/tcg/tcg-all.o
  CC      x86_64-softmmu/trace/generated-helpers.o
  CC      aarch64-softmmu/accel/tcg/cputlb.o
  CC      aarch64-softmmu/accel/tcg/tcg-runtime.o
  CC      aarch64-softmmu/accel/tcg/tcg-runtime-gvec.o
  CC      aarch64-softmmu/accel/tcg/cpu-exec.o
  CC      aarch64-softmmu/accel/tcg/cpu-exec-common.o
  CC      aarch64-softmmu/accel/tcg/translate-all.o
  CC      aarch64-softmmu/accel/tcg/translator.o
  CC      aarch64-softmmu/hw/block/virtio-blk.o
  CC      aarch64-softmmu/hw/adc/stm32f2xx_adc.o
  CC      aarch64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      aarch64-softmmu/hw/char/exynos4210_uart.o
  CC      aarch64-softmmu/hw/char/omap_uart.o
  CC      aarch64-softmmu/hw/char/digic-uart.o
  CC      aarch64-softmmu/hw/char/stm32f2xx_usart.o
  CC      aarch64-softmmu/hw/char/bcm2835_aux.o
  CC      aarch64-softmmu/hw/char/virtio-serial-bus.o
  CC      aarch64-softmmu/hw/core/generic-loader.o
  CC      aarch64-softmmu/hw/core/null-machine.o
  CC      aarch64-softmmu/hw/cpu/arm11mpcore.o
  CC      aarch64-softmmu/hw/cpu/realview_mpcore.o
  CC      aarch64-softmmu/hw/cpu/a9mpcore.o
  CC      aarch64-softmmu/hw/cpu/a15mpcore.o
  CC      aarch64-softmmu/hw/display/omap_dss.o
  CC      aarch64-softmmu/hw/display/omap_lcdc.o
  CC      aarch64-softmmu/hw/display/pxa2xx_lcd.o
  CC      aarch64-softmmu/hw/display/bcm2835_fb.o
  CC      aarch64-softmmu/hw/display/vga.o
  CC      aarch64-softmmu/hw/display/virtio-gpu.o
  CC      aarch64-softmmu/hw/display/virtio-gpu-3d.o
  CC      aarch64-softmmu/hw/display/virtio-gpu-pci.o
  CC      aarch64-softmmu/hw/display/dpcd.o
  CC      aarch64-softmmu/hw/display/xlnx_dp.o
  CC      aarch64-softmmu/hw/dma/xlnx_dpdma.o
  CC      aarch64-softmmu/hw/dma/omap_dma.o
  CC      aarch64-softmmu/hw/dma/soc_dma.o
  CC      aarch64-softmmu/hw/dma/pxa2xx_dma.o
  CC      aarch64-softmmu/hw/dma/bcm2835_dma.o
  CC      aarch64-softmmu/hw/gpio/omap_gpio.o
  CC      aarch64-softmmu/hw/gpio/imx_gpio.o
  CC      aarch64-softmmu/hw/gpio/bcm2835_gpio.o
  CC      aarch64-softmmu/hw/i2c/omap_i2c.o
  CC      aarch64-softmmu/hw/input/pxa2xx_keypad.o
  CC      aarch64-softmmu/hw/input/tsc210x.o
  CC      aarch64-softmmu/hw/intc/armv7m_nvic.o
  CC      aarch64-softmmu/hw/intc/exynos4210_gic.o
  CC      aarch64-softmmu/hw/intc/exynos4210_combiner.o
  CC      aarch64-softmmu/hw/intc/omap_intc.o
  CC      aarch64-softmmu/hw/intc/bcm2835_ic.o
  CC      aarch64-softmmu/hw/intc/bcm2836_control.o
  CC      aarch64-softmmu/hw/intc/allwinner-a10-pic.o
  CC      aarch64-softmmu/hw/intc/aspeed_vic.o
  CC      aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
  CC      aarch64-softmmu/hw/misc/arm_sysctl.o
  CC      aarch64-softmmu/hw/misc/cbus.o
  CC      aarch64-softmmu/hw/misc/exynos4210_pmu.o
  CC      aarch64-softmmu/hw/misc/exynos4210_clk.o
  CC      aarch64-softmmu/hw/misc/exynos4210_rng.o
  CC      aarch64-softmmu/hw/misc/imx_ccm.o
  CC      aarch64-softmmu/hw/misc/imx31_ccm.o
  CC      aarch64-softmmu/hw/misc/imx25_ccm.o
  CC      aarch64-softmmu/hw/misc/imx6_ccm.o
  LINK    x86_64-softmmu/qemu-system-x86_64w.exe
  CC      aarch64-softmmu/hw/misc/imx6_src.o
  CC      aarch64-softmmu/hw/misc/imx7_ccm.o
  CC      aarch64-softmmu/hw/misc/imx2_wdt.o
  CC      aarch64-softmmu/hw/misc/imx7_snvs.o
  CC      aarch64-softmmu/hw/misc/imx7_gpr.o
  CC      aarch64-softmmu/hw/misc/mst_fpga.o
  CC      aarch64-softmmu/hw/misc/omap_clk.o
  CC      aarch64-softmmu/hw/misc/omap_gpmc.o
  CC      aarch64-softmmu/hw/misc/omap_l4.o
  CC      aarch64-softmmu/hw/misc/omap_sdrc.o
  CC      aarch64-softmmu/hw/misc/omap_tap.o
  CC      aarch64-softmmu/hw/misc/bcm2835_mbox.o
  CC      aarch64-softmmu/hw/misc/bcm2835_property.o
  CC      aarch64-softmmu/hw/misc/bcm2835_rng.o
  CC      aarch64-softmmu/hw/misc/zynq_slcr.o
  CC      aarch64-softmmu/hw/misc/zynq-xadc.o
  CC      aarch64-softmmu/hw/misc/stm32f2xx_syscfg.o
  CC      aarch64-softmmu/hw/misc/mps2-fpgaio.o
  CC      aarch64-softmmu/hw/misc/mps2-scc.o
  CC      aarch64-softmmu/hw/misc/tz-ppc.o
  CC      aarch64-softmmu/hw/misc/iotkit-secctl.o
  CC      aarch64-softmmu/hw/misc/auxbus.o
  CC      aarch64-softmmu/hw/misc/aspeed_scu.o
  CC      aarch64-softmmu/hw/misc/aspeed_sdmc.o
  CC      aarch64-softmmu/hw/misc/mmio_interface.o
  CC      aarch64-softmmu/hw/misc/msf2-sysreg.o
  CC      aarch64-softmmu/hw/net/virtio-net.o
  CC      aarch64-softmmu/hw/net/vhost_net.o
  CC      aarch64-softmmu/hw/pcmcia/pxa2xx.o
  CC      aarch64-softmmu/hw/scsi/virtio-scsi.o
  CC      aarch64-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      aarch64-softmmu/hw/sd/omap_mmc.o
  CC      aarch64-softmmu/hw/sd/pxa2xx_mmci.o
  CC      aarch64-softmmu/hw/sd/bcm2835_sdhost.o
  CC      aarch64-softmmu/hw/ssi/omap_spi.o
  CC      aarch64-softmmu/hw/ssi/imx_spi.o
  CC      aarch64-softmmu/hw/timer/exynos4210_mct.o
  CC      aarch64-softmmu/hw/timer/exynos4210_pwm.o
  CC      aarch64-softmmu/hw/timer/exynos4210_rtc.o
  CC      aarch64-softmmu/hw/timer/omap_gptimer.o
  CC      aarch64-softmmu/hw/timer/omap_synctimer.o
  CC      aarch64-softmmu/hw/timer/pxa2xx_timer.o
  CC      aarch64-softmmu/hw/timer/digic-timer.o
  CC      aarch64-softmmu/hw/timer/allwinner-a10-pit.o
  CC      aarch64-softmmu/hw/usb/tusb6010.o
  CC      aarch64-softmmu/hw/usb/chipidea.o
  CC      aarch64-softmmu/hw/virtio/virtio.o
  CC      aarch64-softmmu/hw/virtio/virtio-balloon.o
  CC      aarch64-softmmu/hw/virtio/virtio-crypto.o
  CC      aarch64-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      aarch64-softmmu/hw/arm/boot.o
  CC      aarch64-softmmu/hw/arm/virt.o
  CC      aarch64-softmmu/hw/arm/sysbus-fdt.o
  CC      aarch64-softmmu/hw/arm/virt-acpi-build.o
  CC      aarch64-softmmu/hw/arm/digic_boards.o
  CC      aarch64-softmmu/hw/arm/exynos4_boards.o
  CC      aarch64-softmmu/hw/arm/highbank.o
  CC      aarch64-softmmu/hw/arm/integratorcp.o
  CC      aarch64-softmmu/hw/arm/mainstone.o
  CC      aarch64-softmmu/hw/arm/musicpal.o
  CC      aarch64-softmmu/hw/arm/netduino2.o
  CC      aarch64-softmmu/hw/arm/nseries.o
  CC      aarch64-softmmu/hw/arm/omap_sx1.o
  CC      aarch64-softmmu/hw/arm/palm.o
  CC      aarch64-softmmu/hw/arm/gumstix.o
  CC      aarch64-softmmu/hw/arm/spitz.o
  CC      aarch64-softmmu/hw/arm/tosa.o
  CC      aarch64-softmmu/hw/arm/z2.o
  CC      aarch64-softmmu/hw/arm/realview.o
  CC      aarch64-softmmu/hw/arm/stellaris.o
  CC      aarch64-softmmu/hw/arm/collie.o
  CC      aarch64-softmmu/hw/arm/vexpress.o
  CC      aarch64-softmmu/hw/arm/versatilepb.o
  CC      aarch64-softmmu/hw/arm/xilinx_zynq.o
  CC      aarch64-softmmu/hw/arm/armv7m.o
  CC      aarch64-softmmu/hw/arm/exynos4210.o
  CC      aarch64-softmmu/hw/arm/pxa2xx.o
  CC      aarch64-softmmu/hw/arm/pxa2xx_gpio.o
  CC      aarch64-softmmu/hw/arm/pxa2xx_pic.o
  CC      aarch64-softmmu/hw/arm/digic.o
  CC      aarch64-softmmu/hw/arm/omap1.o
  CC      aarch64-softmmu/hw/arm/omap2.o
  CC      aarch64-softmmu/hw/arm/strongarm.o
  CC      aarch64-softmmu/hw/arm/allwinner-a10.o
  CC      aarch64-softmmu/hw/arm/cubieboard.o
  CC      aarch64-softmmu/hw/arm/bcm2835_peripherals.o
  CC      aarch64-softmmu/hw/arm/bcm2836.o
  CC      aarch64-softmmu/hw/arm/raspi.o
  CC      aarch64-softmmu/hw/arm/stm32f205_soc.o
  CC      aarch64-softmmu/hw/arm/xlnx-zynqmp.o
  CC      aarch64-softmmu/hw/arm/xlnx-zcu102.o
  CC      aarch64-softmmu/hw/arm/fsl-imx25.o
  CC      aarch64-softmmu/hw/arm/imx25_pdk.o
  CC      aarch64-softmmu/hw/arm/fsl-imx31.o
  CC      aarch64-softmmu/hw/arm/kzm.o
  CC      aarch64-softmmu/hw/arm/fsl-imx6.o
  CC      aarch64-softmmu/hw/arm/sabrelite.o
  CC      aarch64-softmmu/hw/arm/aspeed_soc.o
  CC      aarch64-softmmu/hw/arm/aspeed.o
  CC      aarch64-softmmu/hw/arm/mps2.o
  CC      aarch64-softmmu/hw/arm/mps2-tz.o
  CC      aarch64-softmmu/hw/arm/msf2-soc.o
  CC      aarch64-softmmu/hw/arm/msf2-som.o
  CC      aarch64-softmmu/hw/arm/iotkit.o
  CC      aarch64-softmmu/hw/arm/fsl-imx7.o
  CC      aarch64-softmmu/hw/arm/mcimx7d-sabre.o
  CC      aarch64-softmmu/hw/arm/smmu-common.o
  CC      aarch64-softmmu/hw/arm/smmuv3.o
  CC      aarch64-softmmu/target/arm/arm-semi.o
  CC      aarch64-softmmu/target/arm/machine.o
  CC      aarch64-softmmu/target/arm/psci.o
  CC      aarch64-softmmu/target/arm/arch_dump.o
  CC      aarch64-softmmu/target/arm/monitor.o
  CC      aarch64-softmmu/target/arm/translate.o
  CC      aarch64-softmmu/target/arm/kvm-stub.o
  CC      aarch64-softmmu/target/arm/op_helper.o
  CC      aarch64-softmmu/target/arm/helper.o
  CC      aarch64-softmmu/target/arm/cpu.o
  CC      aarch64-softmmu/target/arm/neon_helper.o
  CC      aarch64-softmmu/target/arm/iwmmxt_helper.o
  CC      aarch64-softmmu/target/arm/vec_helper.o
  CC      aarch64-softmmu/target/arm/gdbstub.o
  CC      aarch64-softmmu/target/arm/cpu64.o
  CC      aarch64-softmmu/target/arm/translate-a64.o
  CC      aarch64-softmmu/target/arm/helper-a64.o
  CC      aarch64-softmmu/target/arm/gdbstub64.o
  CC      aarch64-softmmu/target/arm/arm-powerctl.o
  CC      aarch64-softmmu/target/arm/crypto_helper.o
  GEN     aarch64-softmmu/target/arm/decode-sve.inc.c
  CC      aarch64-softmmu/target/arm/sve_helper.o
  GEN     trace/generated-helpers.c
  CC      aarch64-softmmu/trace/control-target.o
  CC      aarch64-softmmu/gdbstub-xml.o
  CC      aarch64-softmmu/trace/generated-helpers.o
  CC      aarch64-softmmu/target/arm/translate-sve.o
  LINK    aarch64-softmmu/qemu-system-aarch64w.exe
  GEN     x86_64-softmmu/qemu-system-x86_64.exe
  GEN     aarch64-softmmu/qemu-system-aarch64.exe
install -d -m 0755 "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install"
install -d -m 0755 "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install"
install -c -m 0755 qemu-ga.exe qemu-img.exe qemu-io.exe  "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install"
x86_64-w64-mingw32-strip "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/qemu-ga.exe" "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/qemu-img.exe" "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/qemu-io.exe"
set -e; for x in bios.bin bios-256k.bin sgabios.bin vgabios.bin vgabios-cirrus.bin vgabios-stdvga.bin vgabios-vmware.bin vgabios-qxl.bin vgabios-virtio.bin ppc_rom.bin openbios-sparc32 openbios-sparc64 openbios-ppc QEMU,tcx.bin QEMU,cgthree.bin pxe-e1000.rom pxe-eepro100.rom pxe-ne2k_pci.rom pxe-pcnet.rom pxe-rtl8139.rom pxe-virtio.rom efi-e1000.rom efi-eepro100.rom efi-ne2k_pci.rom efi-pcnet.rom efi-rtl8139.rom efi-virtio.rom efi-e1000e.rom efi-vmxnet3.rom qemu-icon.bmp qemu_logo_no_text.svg bamboo.dtb canyonlands.dtb petalogix-s3adsp1800.dtb petalogix-ml605.dtb multiboot.bin linuxboot.bin linuxboot_dma.bin kvmvapic.bin s390-ccw.img s390-netboot.img spapr-rtas.bin slof.bin skiboot.lid palcode-clipper u-boot.e500 u-boot-sam460-20100605.bin qemu_vga.ndrv hppa-firmware.img; do \
	install -c -m 0644 /tmp/qemu-test/src/pc-bios/$x "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install"; \
done
make -C po install
make[1]: Entering directory '/tmp/qemu-test/build/po'
  GEN     hu.mo
  GEN     tr.mo
  GEN     it.mo
  GEN     zh_CN.mo
  GEN     de_DE.mo
  GEN     fr_FR.mo
  GEN     bg.mo
for obj in hu.mo tr.mo it.mo bg.mo fr_FR.mo zh_CN.mo de_DE.mo; do \
    base=`basename $obj .mo`; \
    install -d /tmp/qemu-test/build/=destdir/tmp/qemu-test/install/share/locale/$base/LC_MESSAGES; \
    install -m644 $obj /tmp/qemu-test/build/=destdir/tmp/qemu-test/install/share/locale/$base/LC_MESSAGES/qemu.mo; \
done
make[1]: Leaving directory '/tmp/qemu-test/build/po'
install -d -m 0755 "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/keymaps"
set -e; for x in da     en-gb  et  fr     fr-ch  is  lt  modifiers  no  pt-br  sv ar      de     en-us  fi  fr-be  hr     it  lv  nl         pl  ru     th common  de-ch  es     fo  fr-ca  hu     ja  mk  nl-be      pt  sl     tr bepo    cz; do \
	install -c -m 0644 /tmp/qemu-test/src/pc-bios/keymaps/$x "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/keymaps"; \
done
install -c -m 0644 /tmp/qemu-test/build/trace-events-all "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/trace-events-all"
for d in x86_64-softmmu aarch64-softmmu; do \
make --no-print-directory --quiet BUILD_DIR=/tmp/qemu-test/build TARGET_DIR=$d/ -C $d install || exit 1 ; \
        done
rm -f *.timestamp
make -C tests/tcg clean
make[1]: Entering directory '/tmp/qemu-test/build/tests/tcg'
rm -f *.timestamp
rm -f *~ *.o test-i386.out test-i386.ref \
           test-x86_64.log test-x86_64.ref qruncom test_path hello-i386 linux-test testthread sha1-i386 test-i386 test-i386-fprem test-mmap  run-test-x86_64
make[1]: Leaving directory '/tmp/qemu-test/build/tests/tcg'
rm -rf tests/check-block-qdict.exe tests/test-char.exe tests/check-qnum.exe tests/check-qstring.exe tests/check-qlist.exe tests/check-qnull.exe tests/check-qobject.exe tests/check-qjson.exe tests/check-qlit.exe tests/test-qobject-output-visitor.exe tests/test-clone-visitor.exe tests/test-qobject-input-visitor.exe tests/test-qmp-cmds.exe tests/test-string-input-visitor.exe tests/test-string-output-visitor.exe tests/test-qmp-event.exe tests/test-opts-visitor.exe tests/test-coroutine.exe tests/test-visitor-serialization.exe tests/test-iov.exe tests/test-aio.exe tests/test-aio-multithread.exe tests/test-throttle.exe tests/test-thread-pool.exe tests/test-hbitmap.exe tests/test-bdrv-drain.exe tests/test-blockjob.exe tests/test-blockjob-txn.exe tests/test-block-backend.exe tests/test-x86-cpuid.exe tests/test-xbzrle.exe tests/test-cutils.exe tests/test-shift128.exe tests/test-mul64.exe tests/test-int128.exe tests/rcutorture.exe tests/test-rcu-list.exe tests/test-qdist.exe tests/test-qht.exe tests/test-qht-par.exe tests/test-bitops.exe tests/test-bitcnt.exe tests/test-qdev-global-props.exe tests/check-qom-interface.exe tests/check-qom-proplist.exe tests/test-qemu-opts.exe tests/test-keyval.exe tests/test-write-threshold.exe tests/test-crypto-hash.exe tests/test-crypto-hmac.exe tests/test-crypto-cipher.exe tests/test-crypto-secret.exe tests/test-crypto-tlscredsx509.exe tests/test-crypto-tlssession.exe tests/test-timed-average.exe tests/test-util-sockets.exe tests/test-io-task.exe tests/test-io-channel-socket.exe tests/test-io-channel-file.exe tests/test-io-channel-tls.exe tests/test-io-channel-command.exe tests/test-io-channel-buffer.exe tests/test-base64.exe tests/test-crypto-pbkdf.exe tests/test-crypto-ivgen.exe tests/test-crypto-afsplit.exe tests/test-crypto-xts.exe tests/test-crypto-block.exe tests/test-logging.exe tests/test-replication.exe tests/test-bufferiszero.exe tests/test-uuid.exe tests/ptimer-test.exe tests/test-qapi-util.exe tests/*.o 
rm -rf tests/ac97-test.exe tests/ahci-test.exe tests/bios-tables-test.exe tests/boot-order-test.exe tests/boot-serial-test.exe tests/cdrom-test.exe tests/cpu-plug-test.exe tests/device-introspect-test.exe tests/display-vga-test.exe tests/drive_del-test.exe tests/ds1338-test.exe tests/e1000-test.exe tests/e1000e-test.exe tests/eepro100-test.exe tests/endianness-test.exe tests/es1370-test.exe tests/fdc-test.exe tests/fw_cfg-test.exe tests/hd-geo-test.exe tests/i440fx-test.exe tests/i82801b11-test.exe tests/ide-test.exe tests/intel-hda-test.exe tests/ioh3420-test.exe tests/ipmi-bt-test.exe tests/ipmi-kcs-test.exe tests/ipoctal232-test.exe tests/m25p80-test.exe tests/m48t59-test.exe tests/machine-none-test.exe tests/megasas-test.exe tests/migration-test.exe tests/ne2000-test.exe tests/numa-test.exe tests/nvme-test.exe tests/pca9552-test.exe tests/pcnet-test.exe tests/pnv-xscom-test.exe tests/prom-env-test.exe tests/pvpanic-test.exe tests/pxe-test.exe tests/q35-test.exe tests/qmp-test.exe tests/qom-test.exe tests/rtas-test.exe tests/rtc-test.exe tests/rtl8139-test.exe tests/sdhci-test.exe tests/spapr-phb-test.exe tests/tco-test.exe tests/test-arm-mptimer.exe tests/test-hmp.exe tests/test-netfilter.exe tests/test-x86-cpuid-compat.exe tests/tmp105-test.exe tests/tpci200-test.exe tests/tpm-crb-swtpm-test.exe tests/tpm-crb-test.exe tests/tpm-tis-swtpm-test.exe tests/tpm-tis-test.exe tests/usb-hcd-ehci-test.exe tests/usb-hcd-ohci-test.exe tests/usb-hcd-uhci-test.exe tests/usb-hcd-xhci-test.exe tests/virtio-balloon-test.exe tests/virtio-blk-test.exe tests/virtio-console-test.exe tests/virtio-net-test.exe tests/virtio-rng-test.exe tests/virtio-scsi-test.exe tests/virtio-serial-test.exe tests/vmgenid-test.exe tests/vmxnet3-test.exe tests/wdt_ib700-test.exe
rm -f tests/test-qapi-gen-timestamp
rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h
rm -f qemu-options.def
rm -f *.msi
find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} +
rm -f qemu-ga qemu-img.exe qemu-io.exe  qemu-ga TAGS cscope.* *.pod *~ */*~
rm -f fsdev/*.pod scsi/*.pod
rm -f qemu-img-cmds.h
rm -f ui/shader/*-vert.h ui/shader/*-frag.h
rm -f trace/generated-tracers-dtrace.dtrace*
rm -f trace/generated-tracers-dtrace.h*
rm -f qemu-version.h qemu-version.h-timestamp config-host.h config-host.h-timestamp qemu-options.def qemu-options.def-timestamp qapi/qapi-builtin-types.h qapi/qapi-builtin-types.h-timestamp qapi/qapi-builtin-types.c qapi/qapi-builtin-types.c-timestamp qapi/qapi-types.h qapi/qapi-types.h-timestamp qapi/qapi-types.c qapi/qapi-types.c-timestamp qapi/qapi-types-block-core.h qapi/qapi-types-block-core.h-timestamp qapi/qapi-types-block-core.c qapi/qapi-types-block-core.c-timestamp qapi/qapi-types-block.h qapi/qapi-types-block.h-timestamp qapi/qapi-types-block.c qapi/qapi-types-block.c-timestamp qapi/qapi-types-char.h qapi/qapi-types-char.h-timestamp qapi/qapi-types-char.c qapi/qapi-types-char.c-timestamp qapi/qapi-types-common.h qapi/qapi-types-common.h-timestamp qapi/qapi-types-common.c qapi/qapi-types-common.c-timestamp qapi/qapi-types-crypto.h qapi/qapi-types-crypto.h-timestamp qapi/qapi-types-crypto.c qapi/qapi-types-crypto.c-timestamp qapi/qapi-types-introspect.h qapi/qapi-types-introspect.h-timestamp qapi/qapi-types-introspect.c qapi/qapi-types-introspect.c-timestamp qapi/qapi-types-job.h qapi/qapi-types-job.h-timestamp qapi/qapi-types-job.c qapi/qapi-types-job.c-timestamp qapi/qapi-types-migration.h qapi/qapi-types-migration.h-timestamp qapi/qapi-types-migration.c qapi/qapi-types-migration.c-timestamp qapi/qapi-types-misc.h qapi/qapi-types-misc.h-timestamp qapi/qapi-types-misc.c qapi/qapi-types-misc.c-timestamp qapi/qapi-types-net.h qapi/qapi-types-net.h-timestamp qapi/qapi-types-net.c qapi/qapi-types-net.c-timestamp qapi/qapi-types-rocker.h qapi/qapi-types-rocker.h-timestamp qapi/qapi-types-rocker.c qapi/qapi-types-rocker.c-timestamp qapi/qapi-types-run-state.h qapi/qapi-types-run-state.h-timestamp qapi/qapi-types-run-state.c qapi/qapi-types-run-state.c-timestamp qapi/qapi-types-sockets.h qapi/qapi-types-sockets.h-timestamp qapi/qapi-types-sockets.c qapi/qapi-types-sockets.c-timestamp qapi/qapi-types-tpm.h qapi/qapi-types-tpm.h-timestamp qapi/qapi-types-tpm.c qapi/qapi-types-tpm.c-timestamp qapi/qapi-types-trace.h qapi/qapi-types-trace.h-timestamp qapi/qapi-types-trace.c qapi/qapi-types-trace.c-timestamp qapi/qapi-types-transaction.h qapi/qapi-types-transaction.h-timestamp qapi/qapi-types-transaction.c qapi/qapi-types-transaction.c-timestamp qapi/qapi-types-ui.h qapi/qapi-types-ui.h-timestamp qapi/qapi-types-ui.c qapi/qapi-types-ui.c-timestamp qapi/qapi-builtin-visit.h qapi/qapi-builtin-visit.h-timestamp qapi/qapi-builtin-visit.c qapi/qapi-builtin-visit.c-timestamp qapi/qapi-visit.h qapi/qapi-visit.h-timestamp qapi/qapi-visit.c qapi/qapi-visit.c-timestamp qapi/qapi-visit-block-core.h qapi/qapi-visit-block-core.h-timestamp qapi/qapi-visit-block-core.c qapi/qapi-visit-block-core.c-timestamp qapi/qapi-visit-block.h qapi/qapi-visit-block.h-timestamp qapi/qapi-visit-block.c qapi/qapi-visit-block.c-timestamp qapi/qapi-visit-char.h qapi/qapi-visit-char.h-timestamp qapi/qapi-visit-char.c qapi/qapi-visit-char.c-timestamp qapi/qapi-visit-common.h qapi/qapi-visit-common.h-timestamp qapi/qapi-visit-common.c qapi/qapi-visit-common.c-timestamp qapi/qapi-visit-crypto.h qapi/qapi-visit-crypto.h-timestamp qapi/qapi-visit-crypto.c qapi/qapi-visit-crypto.c-timestamp qapi/qapi-visit-introspect.h qapi/qapi-visit-introspect.h-timestamp qapi/qapi-visit-introspect.c qapi/qapi-visit-introspect.c-timestamp qapi/qapi-visit-job.h qapi/qapi-visit-job.h-timestamp qapi/qapi-visit-job.c qapi/qapi-visit-job.c-timestamp qapi/qapi-visit-migration.h qapi/qapi-visit-migration.h-timestamp qapi/qapi-visit-migration.c qapi/qapi-visit-migration.c-timestamp qapi/qapi-visit-misc.h qapi/qapi-visit-misc.h-timestamp qapi/qapi-visit-misc.c qapi/qapi-visit-misc.c-timestamp qapi/qapi-visit-net.h qapi/qapi-visit-net.h-timestamp qapi/qapi-visit-net.c qapi/qapi-visit-net.c-timestamp qapi/qapi-visit-rocker.h qapi/qapi-visit-rocker.h-timestamp qapi/qapi-visit-rocker.c qapi/qapi-visit-rocker.c-timestamp qapi/qapi-visit-run-state.h qapi/qapi-visit-run-state.h-timestamp qapi/qapi-visit-run-state.c qapi/qapi-visit-run-state.c-timestamp qapi/qapi-visit-sockets.h qapi/qapi-visit-sockets.h-timestamp qapi/qapi-visit-sockets.c qapi/qapi-visit-sockets.c-timestamp qapi/qapi-visit-tpm.h qapi/qapi-visit-tpm.h-timestamp qapi/qapi-visit-tpm.c qapi/qapi-visit-tpm.c-timestamp qapi/qapi-visit-trace.h qapi/qapi-visit-trace.h-timestamp qapi/qapi-visit-trace.c qapi/qapi-visit-trace.c-timestamp qapi/qapi-visit-transaction.h qapi/qapi-visit-transaction.h-timestamp qapi/qapi-visit-transaction.c qapi/qapi-visit-transaction.c-timestamp qapi/qapi-visit-ui.h qapi/qapi-visit-ui.h-timestamp qapi/qapi-visit-ui.c qapi/qapi-visit-ui.c-timestamp qapi/qapi-commands.h qapi/qapi-commands.h-timestamp qapi/qapi-commands.c qapi/qapi-commands.c-timestamp qapi/qapi-commands-block-core.h qapi/qapi-commands-block-core.h-timestamp qapi/qapi-commands-block-core.c qapi/qapi-commands-block-core.c-timestamp qapi/qapi-commands-block.h qapi/qapi-commands-block.h-timestamp qapi/qapi-commands-block.c qapi/qapi-commands-block.c-timestamp qapi/qapi-commands-char.h qapi/qapi-commands-char.h-timestamp qapi/qapi-commands-char.c qapi/qapi-commands-char.c-timestamp qapi/qapi-commands-common.h qapi/qapi-commands-common.h-timestamp qapi/qapi-commands-common.c qapi/qapi-commands-common.c-timestamp qapi/qapi-commands-crypto.h qapi/qapi-commands-crypto.h-timestamp qapi/qapi-commands-crypto.c qapi/qapi-commands-crypto.c-timestamp qapi/qapi-commands-introspect.h qapi/qapi-commands-introspect.h-timestamp qapi/qapi-commands-introspect.c qapi/qapi-commands-introspect.c-timestamp qapi/qapi-commands-job.h qapi/qapi-commands-job.h-timestamp qapi/qapi-commands-job.c qapi/qapi-commands-job.c-timestamp qapi/qapi-commands-migration.h qapi/qapi-commands-migration.h-timestamp qapi/qapi-commands-migration.c qapi/qapi-commands-migration.c-timestamp qapi/qapi-commands-misc.h qapi/qapi-commands-misc.h-timestamp qapi/qapi-commands-misc.c qapi/qapi-commands-misc.c-timestamp qapi/qapi-commands-net.h qapi/qapi-commands-net.h-timestamp qapi/qapi-commands-net.c qapi/qapi-commands-net.c-timestamp qapi/qapi-commands-rocker.h qapi/qapi-commands-rocker.h-timestamp qapi/qapi-commands-rocker.c qapi/qapi-commands-rocker.c-timestamp qapi/qapi-commands-run-state.h qapi/qapi-commands-run-state.h-timestamp qapi/qapi-commands-run-state.c qapi/qapi-commands-run-state.c-timestamp qapi/qapi-commands-sockets.h qapi/qapi-commands-sockets.h-timestamp qapi/qapi-commands-sockets.c qapi/qapi-commands-sockets.c-timestamp qapi/qapi-commands-tpm.h qapi/qapi-commands-tpm.h-timestamp qapi/qapi-commands-tpm.c qapi/qapi-commands-tpm.c-timestamp qapi/qapi-commands-trace.h qapi/qapi-commands-trace.h-timestamp qapi/qapi-commands-trace.c qapi/qapi-commands-trace.c-timestamp qapi/qapi-commands-transaction.h qapi/qapi-commands-transaction.h-timestamp qapi/qapi-commands-transaction.c qapi/qapi-commands-transaction.c-timestamp qapi/qapi-commands-ui.h qapi/qapi-commands-ui.h-timestamp qapi/qapi-commands-ui.c qapi/qapi-commands-ui.c-timestamp qapi/qapi-events.h qapi/qapi-events.h-timestamp qapi/qapi-events.c qapi/qapi-events.c-timestamp qapi/qapi-events-block-core.h qapi/qapi-events-block-core.h-timestamp qapi/qapi-events-block-core.c qapi/qapi-events-block-core.c-timestamp qapi/qapi-events-block.h qapi/qapi-events-block.h-timestamp qapi/qapi-events-block.c qapi/qapi-events-block.c-timestamp qapi/qapi-events-char.h qapi/qapi-events-char.h-timestamp qapi/qapi-events-char.c qapi/qapi-events-char.c-timestamp qapi/qapi-events-common.h qapi/qapi-events-common.h-timestamp qapi/qapi-events-common.c qapi/qapi-events-common.c-timestamp qapi/qapi-events-crypto.h qapi/qapi-events-crypto.h-timestamp qapi/qapi-events-crypto.c qapi/qapi-events-crypto.c-timestamp qapi/qapi-events-introspect.h qapi/qapi-events-introspect.h-timestamp qapi/qapi-events-introspect.c qapi/qapi-events-introspect.c-timestamp qapi/qapi-events-job.h qapi/qapi-events-job.h-timestamp qapi/qapi-events-job.c qapi/qapi-events-job.c-timestamp qapi/qapi-events-migration.h qapi/qapi-events-migration.h-timestamp qapi/qapi-events-migration.c qapi/qapi-events-migration.c-timestamp qapi/qapi-events-misc.h qapi/qapi-events-misc.h-timestamp qapi/qapi-events-misc.c qapi/qapi-events-misc.c-timestamp qapi/qapi-events-net.h qapi/qapi-events-net.h-timestamp qapi/qapi-events-net.c qapi/qapi-events-net.c-timestamp qapi/qapi-events-rocker.h qapi/qapi-events-rocker.h-timestamp qapi/qapi-events-rocker.c qapi/qapi-events-rocker.c-timestamp qapi/qapi-events-run-state.h qapi/qapi-events-run-state.h-timestamp qapi/qapi-events-run-state.c qapi/qapi-events-run-state.c-timestamp qapi/qapi-events-sockets.h qapi/qapi-events-sockets.h-timestamp qapi/qapi-events-sockets.c qapi/qapi-events-sockets.c-timestamp qapi/qapi-events-tpm.h qapi/qapi-events-tpm.h-timestamp qapi/qapi-events-tpm.c qapi/qapi-events-tpm.c-timestamp qapi/qapi-events-trace.h qapi/qapi-events-trace.h-timestamp qapi/qapi-events-trace.c qapi/qapi-events-trace.c-timestamp qapi/qapi-events-transaction.h qapi/qapi-events-transaction.h-timestamp qapi/qapi-events-transaction.c qapi/qapi-events-transaction.c-timestamp qapi/qapi-events-ui.h qapi/qapi-events-ui.h-timestamp qapi/qapi-events-ui.c qapi/qapi-events-ui.c-timestamp qapi/qapi-introspect.c qapi/qapi-introspect.c-timestamp qapi/qapi-introspect.h qapi/qapi-introspect.h-timestamp qapi/qapi-doc.texi qapi/qapi-doc.texi-timestamp trace/generated-tcg-tracers.h trace/generated-tcg-tracers.h-timestamp trace/generated-helpers-wrappers.h trace/generated-helpers-wrappers.h-timestamp trace/generated-helpers.h trace/generated-helpers.h-timestamp trace/generated-helpers.c trace/generated-helpers.c-timestamp module_block.h module_block.h-timestamp trace-root.h trace-root.h-timestamp accel/kvm/trace.h accel/kvm/trace.h-timestamp accel/tcg/trace.h accel/tcg/trace.h-timestamp audio/trace.h audio/trace.h-timestamp block/trace.h block/trace.h-timestamp chardev/trace.h chardev/trace.h-timestamp crypto/trace.h crypto/trace.h-timestamp hw/9pfs/trace.h hw/9pfs/trace.h-timestamp hw/acpi/trace.h hw/acpi/trace.h-timestamp hw/alpha/trace.h hw/alpha/trace.h-timestamp hw/arm/trace.h hw/arm/trace.h-timestamp hw/audio/trace.h hw/audio/trace.h-timestamp hw/block/trace.h hw/block/trace.h-timestamp hw/block/dataplane/trace.h hw/block/dataplane/trace.h-timestamp hw/char/trace.h hw/char/trace.h-timestamp hw/display/trace.h hw/display/trace.h-timestamp hw/dma/trace.h hw/dma/trace.h-timestamp hw/hppa/trace.h hw/hppa/trace.h-timestamp hw/i2c/trace.h hw/i2c/trace.h-timestamp hw/i386/trace.h hw/i386/trace.h-timestamp hw/i386/xen/trace.h hw/i386/xen/trace.h-timestamp hw/ide/trace.h hw/ide/trace.h-timestamp hw/input/trace.h hw/input/trace.h-timestamp hw/intc/trace.h hw/intc/trace.h-timestamp hw/isa/trace.h hw/isa/trace.h-timestamp hw/mem/trace.h hw/mem/trace.h-timestamp hw/misc/trace.h hw/misc/trace.h-timestamp hw/misc/macio/trace.h hw/misc/macio/trace.h-timestamp hw/net/trace.h hw/net/trace.h-timestamp hw/nvram/trace.h hw/nvram/trace.h-timestamp hw/pci/trace.h hw/pci/trace.h-timestamp hw/pci-host/trace.h hw/pci-host/trace.h-timestamp hw/ppc/trace.h hw/ppc/trace.h-timestamp hw/rdma/trace.h hw/rdma/trace.h-timestamp hw/rdma/vmw/trace.h hw/rdma/vmw/trace.h-timestamp hw/s390x/trace.h hw/s390x/trace.h-timestamp hw/scsi/trace.h hw/scsi/trace.h-timestamp hw/sd/trace.h hw/sd/trace.h-timestamp hw/sparc/trace.h hw/sparc/trace.h-timestamp hw/sparc64/trace.h hw/sparc64/trace.h-timestamp hw/timer/trace.h hw/timer/trace.h-timestamp hw/tpm/trace.h hw/tpm/trace.h-timestamp hw/usb/trace.h hw/usb/trace.h-timestamp hw/vfio/trace.h hw/vfio/trace.h-timestamp hw/virtio/trace.h hw/virtio/trace.h-timestamp hw/xen/trace.h hw/xen/trace.h-timestamp io/trace.h io/trace.h-timestamp linux-user/trace.h linux-user/trace.h-timestamp migration/trace.h migration/trace.h-timestamp nbd/trace.h nbd/trace.h-timestamp net/trace.h net/trace.h-timestamp qapi/trace.h qapi/trace.h-timestamp qom/trace.h qom/trace.h-timestamp scsi/trace.h scsi/trace.h-timestamp target/arm/trace.h target/arm/trace.h-timestamp target/i386/trace.h target/i386/trace.h-timestamp target/mips/trace.h target/mips/trace.h-timestamp target/ppc/trace.h target/ppc/trace.h-timestamp target/s390x/trace.h target/s390x/trace.h-timestamp target/sparc/trace.h target/sparc/trace.h-timestamp ui/trace.h ui/trace.h-timestamp util/trace.h util/trace.h-timestamp trace-root.c trace-root.c-timestamp accel/kvm/trace.c accel/kvm/trace.c-timestamp accel/tcg/trace.c accel/tcg/trace.c-timestamp audio/trace.c audio/trace.c-timestamp block/trace.c block/trace.c-timestamp chardev/trace.c chardev/trace.c-timestamp crypto/trace.c crypto/trace.c-timestamp hw/9pfs/trace.c hw/9pfs/trace.c-timestamp hw/acpi/trace.c hw/acpi/trace.c-timestamp hw/alpha/trace.c hw/alpha/trace.c-timestamp hw/arm/trace.c hw/arm/trace.c-timestamp hw/audio/trace.c hw/audio/trace.c-timestamp hw/block/trace.c hw/block/trace.c-timestamp hw/block/dataplane/trace.c hw/block/dataplane/trace.c-timestamp hw/char/trace.c hw/char/trace.c-timestamp hw/display/trace.c hw/display/trace.c-timestamp hw/dma/trace.c hw/dma/trace.c-timestamp hw/hppa/trace.c hw/hppa/trace.c-timestamp hw/i2c/trace.c hw/i2c/trace.c-timestamp hw/i386/trace.c hw/i386/trace.c-timestamp hw/i386/xen/trace.c hw/i386/xen/trace.c-timestamp hw/ide/trace.c hw/ide/trace.c-timestamp hw/input/trace.c hw/input/trace.c-timestamp hw/intc/trace.c hw/intc/trace.c-timestamp hw/isa/trace.c hw/isa/trace.c-timestamp hw/mem/trace.c hw/mem/trace.c-timestamp hw/misc/trace.c hw/misc/trace.c-timestamp hw/misc/macio/trace.c hw/misc/macio/trace.c-timestamp hw/net/trace.c hw/net/trace.c-timestamp hw/nvram/trace.c hw/nvram/trace.c-timestamp hw/pci/trace.c hw/pci/trace.c-timestamp hw/pci-host/trace.c hw/pci-host/trace.c-timestamp hw/ppc/trace.c hw/ppc/trace.c-timestamp hw/rdma/trace.c hw/rdma/trace.c-timestamp hw/rdma/vmw/trace.c hw/rdma/vmw/trace.c-timestamp hw/s390x/trace.c hw/s390x/trace.c-timestamp hw/scsi/trace.c hw/scsi/trace.c-timestamp hw/sd/trace.c hw/sd/trace.c-timestamp hw/sparc/trace.c hw/sparc/trace.c-timestamp hw/sparc64/trace.c hw/sparc64/trace.c-timestamp hw/timer/trace.c hw/timer/trace.c-timestamp hw/tpm/trace.c hw/tpm/trace.c-timestamp hw/usb/trace.c hw/usb/trace.c-timestamp hw/vfio/trace.c hw/vfio/trace.c-timestamp hw/virtio/trace.c hw/virtio/trace.c-timestamp hw/xen/trace.c hw/xen/trace.c-timestamp io/trace.c io/trace.c-timestamp linux-user/trace.c linux-user/trace.c-timestamp migration/trace.c migration/trace.c-timestamp nbd/trace.c nbd/trace.c-timestamp net/trace.c net/trace.c-timestamp qapi/trace.c qapi/trace.c-timestamp qom/trace.c qom/trace.c-timestamp scsi/trace.c scsi/trace.c-timestamp target/arm/trace.c target/arm/trace.c-timestamp target/i386/trace.c target/i386/trace.c-timestamp target/mips/trace.c target/mips/trace.c-timestamp target/ppc/trace.c target/ppc/trace.c-timestamp target/s390x/trace.c target/s390x/trace.c-timestamp target/sparc/trace.c target/sparc/trace.c-timestamp ui/trace.c ui/trace.c-timestamp util/trace.c util/trace.c-timestamp /tmp/qemu-test/build/trace-events-all /tmp/qemu-test/build/trace-events-all-timestamp .git-submodule-status .git-submodule-status-timestamp ui/input-keymap-atset1-to-qcode.c ui/input-keymap-atset1-to-qcode.c-timestamp ui/input-keymap-linux-to-qcode.c ui/input-keymap-linux-to-qcode.c-timestamp ui/input-keymap-qcode-to-atset1.c ui/input-keymap-qcode-to-atset1.c-timestamp ui/input-keymap-qcode-to-atset2.c ui/input-keymap-qcode-to-atset2.c-timestamp ui/input-keymap-qcode-to-atset3.c ui/input-keymap-qcode-to-atset3.c-timestamp ui/input-keymap-qcode-to-linux.c ui/input-keymap-qcode-to-linux.c-timestamp ui/input-keymap-qcode-to-qnum.c ui/input-keymap-qcode-to-qnum.c-timestamp ui/input-keymap-qcode-to-sun.c ui/input-keymap-qcode-to-sun.c-timestamp ui/input-keymap-qnum-to-qcode.c ui/input-keymap-qnum-to-qcode.c-timestamp ui/input-keymap-usb-to-qcode.c ui/input-keymap-usb-to-qcode.c-timestamp ui/input-keymap-win32-to-qcode.c ui/input-keymap-win32-to-qcode.c-timestamp ui/input-keymap-x11-to-qcode.c ui/input-keymap-x11-to-qcode.c-timestamp ui/input-keymap-xorgevdev-to-qcode.c ui/input-keymap-xorgevdev-to-qcode.c-timestamp ui/input-keymap-xorgkbd-to-qcode.c ui/input-keymap-xorgkbd-to-qcode.c-timestamp ui/input-keymap-xorgxquartz-to-qcode.c ui/input-keymap-xorgxquartz-to-qcode.c-timestamp ui/input-keymap-xorgxwin-to-qcode.c ui/input-keymap-xorgxwin-to-qcode.c-timestamp ui/input-keymap-osx-to-qcode.c ui/input-keymap-osx-to-qcode.c-timestamp tests/test-qapi-types.h tests/test-qapi-types.h-timestamp tests/test-qapi-visit.h tests/test-qapi-visit.h-timestamp tests/test-qapi-commands.h tests/test-qapi-commands.h-timestamp tests/test-qapi-events.h tests/test-qapi-events.h-timestamp tests/test-qapi-introspect.h tests/test-qapi-introspect.h-timestamp
rm -f qapi-gen-timestamp
rm -rf qga/qapi-generated
for d in x86_64-softmmu aarch64-softmmu pc-bios/optionrom; do \
if test -d $d; then make -C $d clean || exit 1; fi; \
rm -f $d/qemu-options.def; \
        done
make[1]: Entering directory '/tmp/qemu-test/build/x86_64-softmmu'
rm -f *.timestamp
rm -f *.a *~ qemu-system-x86_64.exe qemu-system-x86_64w.exe
rm -f 
rm -f hmp-commands.h gdbstub-xml.c
make[1]: Leaving directory '/tmp/qemu-test/build/x86_64-softmmu'
make[1]: Entering directory '/tmp/qemu-test/build/aarch64-softmmu'
rm -f *.timestamp
rm -f *.a *~ qemu-system-aarch64.exe qemu-system-aarch64w.exe
rm -f 
rm -f hmp-commands.h gdbstub-xml.c
make[1]: Leaving directory '/tmp/qemu-test/build/aarch64-softmmu'
make[1]: Entering directory '/tmp/qemu-test/build/pc-bios/optionrom'
rm -f *.timestamp
rm -f *.o *.d *.raw *.img *.bin *~
make[1]: Leaving directory '/tmp/qemu-test/build/pc-bios/optionrom'
rm -f  x86_64-softmmu/config-devices.mak  aarch64-softmmu/config-devices.mak config-all-devices.mak
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=i686-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=2.0 --with-gtkabi=3.0
Install prefix    /tmp/qemu-test/install
BIOS directory    /tmp/qemu-test/install
firmware path     /tmp/qemu-test/install/share/qemu-firmware
binary directory  /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory  /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory  /tmp/qemu-test/install
local state directory   queried at runtime
Windows SDK       no
Source path       /tmp/qemu-test/src
GIT binary        git
GIT submodules    
C compiler        i686-w64-mingw32-gcc
Host C compiler   cc
C++ compiler      i686-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS           rv
CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS       -I/usr/i686-w64-mingw32/sys-root/mingw/include/pixman-1  -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -mms-bitfields -I/usr/i686-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/i686-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/i686-w64-mingw32/sys-root/mingw/include  -m32 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv  -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/i686-w64-mingw32/sys-root/mingw/include -I/usr/i686-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/usr/i686-w64-mingw32/sys-root/mingw/include  -I/usr/i686-w64-mingw32/sys-root/mingw/include   -I/usr/i686-w64-mingw32/sys-root/mingw/include/libpng16 
LDFLAGS           -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m32 -g 
QEMU_LDFLAGS      -L$(BUILD_DIR)/dtc/libfdt 
make              make
install           install
python            /usr/bin/python3 -B
smbd              /usr/sbin/smbd
module support    no
host CPU          i386
host big endian   no
target list       x86_64-softmmu aarch64-softmmu
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
SDL support       yes (2.0.5)
GTK support       yes (3.22.16)
GTK GL support    no
VTE support       no 
TLS priority      NORMAL
GNUTLS support    yes
GNUTLS rnd        yes
libgcrypt         no
libgcrypt kdf     no
nettle            yes (3.3)
nettle kdf        yes
libtasn1          yes
curses support    no
virgl support     no 
curl support      yes
mingw32 support   yes
Audio drivers     dsound
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS support    no
Multipath support no
VNC support       yes
VNC SASL support  no
VNC JPEG support  yes
VNC PNG support   yes
xen support       no
brlapi support    no
bluez  support    no
Documentation     no
PIE               no
vde support       no
netmap support    no
Linux AIO support no
ATTR/XATTR support no
Install blobs     yes
KVM support       no
HAX support       yes
HVF support       no
WHPX support      no
TCG support       yes
TCG debug enabled no
TCG interpreter   no
malloc trim support no
RDMA support      no
fdt support       git
membarrier        no
preadv support    no
fdatasync         no
madvise           no
posix_madvise     no
posix_memalign    no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends    simple
Trace output file trace-<pid>
spice support     no 
rbd support       no
xfsctl support    no
smartcard support no
libusb            no
usb net redir     no
OpenGL support    no
OpenGL dmabufs    no
libiscsi support  no
libnfs support    no
build guest agent yes
QGA VSS support   no
QGA w32 disk info yes
QGA MSI support   no
seccomp support   no
coroutine backend win32
coroutine pool    yes
debug stack usage no
crypto afalg      no
GlusterFS support no
gcov              gcov
gcov enabled      no
TPM support       yes
libssh2 support   yes
TPM passthrough   no
TPM emulator      no
QOM debugging     yes
Live block migration yes
lzo support       no
snappy support    no
bzip2 support     yes
NUMA host support no
libxml2           no
tcmalloc support  no
jemalloc support  no
avx2 optimization yes
replication support yes
VxHS block device no
capstone          no
docker            no
  GEN     x86_64-softmmu/config-devices.mak.tmp
  GEN     aarch64-softmmu/config-devices.mak.tmp
  GEN     config-host.h
  GEN     qemu-options.def
  GEN     qapi-gen
  GEN     trace/generated-tcg-tracers.h
  GEN     trace/generated-helpers-wrappers.h
  GEN     trace/generated-helpers.c
  GEN     trace/generated-helpers.h
  GEN     module_block.h
  GEN     ui/input-keymap-atset1-to-qcode.c
  GEN     ui/input-keymap-qcode-to-atset1.c
  GEN     ui/input-keymap-linux-to-qcode.c
  GEN     x86_64-softmmu/config-devices.mak
  GEN     ui/input-keymap-qcode-to-atset2.c
  GEN     aarch64-softmmu/config-devices.mak
  GEN     ui/input-keymap-qcode-to-linux.c
  GEN     ui/input-keymap-qcode-to-atset3.c
  GEN     ui/input-keymap-qcode-to-qnum.c
  GEN     ui/input-keymap-qcode-to-sun.c
  GEN     ui/input-keymap-qnum-to-qcode.c
  GEN     ui/input-keymap-usb-to-qcode.c
  GEN     ui/input-keymap-win32-to-qcode.c
  GEN     ui/input-keymap-x11-to-qcode.c
  GEN     ui/input-keymap-xorgevdev-to-qcode.c
  GEN     ui/input-keymap-xorgkbd-to-qcode.c
  GEN     ui/input-keymap-xorgxquartz-to-qcode.c
  GEN     ui/input-keymap-xorgxwin-to-qcode.c
  GEN     ui/input-keymap-osx-to-qcode.c
  GEN     tests/test-qapi-gen
  GEN     trace-root.h
  GEN     accel/kvm/trace.h
  GEN     accel/tcg/trace.h
  GEN     audio/trace.h
  GEN     block/trace.h
  GEN     chardev/trace.h
  GEN     crypto/trace.h
  GEN     hw/9pfs/trace.h
  GEN     hw/acpi/trace.h
  GEN     hw/alpha/trace.h
  GEN     hw/arm/trace.h
  GEN     hw/audio/trace.h
  GEN     hw/block/trace.h
  GEN     hw/block/dataplane/trace.h
  GEN     hw/char/trace.h
  GEN     hw/display/trace.h
  GEN     hw/dma/trace.h
  GEN     hw/hppa/trace.h
  GEN     hw/i2c/trace.h
  GEN     hw/i386/trace.h
  GEN     hw/i386/xen/trace.h
  GEN     hw/ide/trace.h
  GEN     hw/input/trace.h
  GEN     hw/intc/trace.h
  GEN     hw/isa/trace.h
  GEN     hw/mem/trace.h
  GEN     hw/misc/trace.h
  GEN     hw/misc/macio/trace.h
  GEN     hw/net/trace.h
  GEN     hw/nvram/trace.h
  GEN     hw/pci/trace.h
  GEN     hw/pci-host/trace.h
  GEN     hw/ppc/trace.h
  GEN     hw/rdma/trace.h
  GEN     hw/rdma/vmw/trace.h
  GEN     hw/s390x/trace.h
  GEN     hw/scsi/trace.h
  GEN     hw/sd/trace.h
  GEN     hw/sparc/trace.h
  GEN     hw/sparc64/trace.h
  GEN     hw/timer/trace.h
  GEN     hw/tpm/trace.h
  GEN     hw/usb/trace.h
  GEN     hw/vfio/trace.h
  GEN     hw/virtio/trace.h
  GEN     hw/xen/trace.h
  GEN     io/trace.h
  GEN     linux-user/trace.h
  GEN     migration/trace.h
  GEN     nbd/trace.h
  GEN     net/trace.h
  GEN     qapi/trace.h
  GEN     qom/trace.h
  GEN     scsi/trace.h
  GEN     target/arm/trace.h
  GEN     target/i386/trace.h
  GEN     target/mips/trace.h
  GEN     target/ppc/trace.h
  GEN     target/s390x/trace.h
  GEN     target/sparc/trace.h
  GEN     ui/trace.h
  GEN     util/trace.h
  GEN     trace-root.c
  GEN     accel/kvm/trace.c
  GEN     accel/tcg/trace.c
  GEN     audio/trace.c
  GEN     block/trace.c
  GEN     chardev/trace.c
  GEN     crypto/trace.c
  GEN     hw/9pfs/trace.c
  GEN     hw/acpi/trace.c
  GEN     hw/alpha/trace.c
  GEN     hw/arm/trace.c
  GEN     hw/audio/trace.c
  GEN     hw/block/trace.c
  GEN     hw/block/dataplane/trace.c
  GEN     hw/char/trace.c
  GEN     hw/display/trace.c
  GEN     hw/dma/trace.c
  GEN     hw/hppa/trace.c
  GEN     hw/i2c/trace.c
  GEN     hw/i386/trace.c
  GEN     hw/i386/xen/trace.c
  GEN     hw/ide/trace.c
  GEN     hw/input/trace.c
  GEN     hw/intc/trace.c
  GEN     hw/isa/trace.c
  GEN     hw/mem/trace.c
  GEN     hw/misc/trace.c
  GEN     hw/misc/macio/trace.c
  GEN     hw/net/trace.c
  GEN     hw/nvram/trace.c
  GEN     hw/pci/trace.c
  GEN     hw/pci-host/trace.c
  GEN     hw/ppc/trace.c
  GEN     hw/rdma/trace.c
  GEN     hw/rdma/vmw/trace.c
  GEN     hw/s390x/trace.c
  GEN     hw/scsi/trace.c
  GEN     hw/sd/trace.c
  GEN     hw/sparc/trace.c
  GEN     hw/sparc64/trace.c
  GEN     hw/timer/trace.c
  GEN     hw/tpm/trace.c
  GEN     hw/usb/trace.c
  GEN     hw/vfio/trace.c
  GEN     hw/virtio/trace.c
  GEN     hw/xen/trace.c
  GEN     io/trace.c
  GEN     linux-user/trace.c
  GEN     migration/trace.c
  GEN     nbd/trace.c
  GEN     net/trace.c
  GEN     qapi/trace.c
  GEN     qom/trace.c
  GEN     scsi/trace.c
  GEN     target/arm/trace.c
  GEN     target/i386/trace.c
  GEN     target/mips/trace.c
  GEN     target/ppc/trace.c
  GEN     target/s390x/trace.c
  GEN     target/sparc/trace.c
  GEN     ui/trace.c
  GEN     util/trace.c
  GEN     config-all-devices.mak
	 DEP /tmp/qemu-test/src/dtc/tests/dumptrees.c
	 DEP /tmp/qemu-test/src/dtc/tests/trees.S
	 DEP /tmp/qemu-test/src/dtc/tests/testutils.c
	 DEP /tmp/qemu-test/src/dtc/tests/value-labels.c
	 DEP /tmp/qemu-test/src/dtc/tests/asm_tree_dump.c
	 DEP /tmp/qemu-test/src/dtc/tests/truncated_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/check_path.c
	 DEP /tmp/qemu-test/src/dtc/tests/overlay_bad_fixup.c
	 DEP /tmp/qemu-test/src/dtc/tests/overlay.c
	 DEP /tmp/qemu-test/src/dtc/tests/subnode_iterate.c
	 DEP /tmp/qemu-test/src/dtc/tests/property_iterate.c
	 DEP /tmp/qemu-test/src/dtc/tests/integer-expressions.c
	 DEP /tmp/qemu-test/src/dtc/tests/utilfdt_test.c
	 DEP /tmp/qemu-test/src/dtc/tests/path_offset_aliases.c
	 DEP /tmp/qemu-test/src/dtc/tests/add_subnode_with_nops.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_unordered.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtb_reverse.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_ordered.c
	 DEP /tmp/qemu-test/src/dtc/tests/extra-terminating-null.c
	 DEP /tmp/qemu-test/src/dtc/tests/incbin.c
	 DEP /tmp/qemu-test/src/dtc/tests/boot-cpuid.c
	 DEP /tmp/qemu-test/src/dtc/tests/phandle_format.c
	 DEP /tmp/qemu-test/src/dtc/tests/path-references.c
	 DEP /tmp/qemu-test/src/dtc/tests/references.c
	 DEP /tmp/qemu-test/src/dtc/tests/string_escapes.c
	 DEP /tmp/qemu-test/src/dtc/tests/propname_escapes.c
	 DEP /tmp/qemu-test/src/dtc/tests/appendprop2.c
	 DEP /tmp/qemu-test/src/dtc/tests/appendprop1.c
	 DEP /tmp/qemu-test/src/dtc/tests/del_node.c
	 DEP /tmp/qemu-test/src/dtc/tests/del_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/setprop.c
	 DEP /tmp/qemu-test/src/dtc/tests/rw_tree1.c
	 DEP /tmp/qemu-test/src/dtc/tests/set_name.c
	 DEP /tmp/qemu-test/src/dtc/tests/nopulate.c
	 DEP /tmp/qemu-test/src/dtc/tests/open_pack.c
	 DEP /tmp/qemu-test/src/dtc/tests/mangle-layout.c
	 DEP /tmp/qemu-test/src/dtc/tests/move_and_save.c
	 DEP /tmp/qemu-test/src/dtc/tests/sw_tree1.c
	 DEP /tmp/qemu-test/src/dtc/tests/nop_node.c
	 DEP /tmp/qemu-test/src/dtc/tests/nop_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/setprop_inplace.c
	 DEP /tmp/qemu-test/src/dtc/tests/stringlist.c
	 DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells.c
	 DEP /tmp/qemu-test/src/dtc/tests/notfound.c
	 DEP /tmp/qemu-test/src/dtc/tests/sized_cells.c
	 DEP /tmp/qemu-test/src/dtc/tests/char_literal.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_alias.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_compatible.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_check_compatible.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_phandle.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_prop_value.c
	 DEP /tmp/qemu-test/src/dtc/tests/parent_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/supernode_atdepth_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_path.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_phandle.c
	 DEP /tmp/qemu-test/src/dtc/tests/getprop.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_name.c
	 DEP /tmp/qemu-test/src/dtc/tests/path_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/subnode_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/find_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/root_node.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_overlay.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_mem_rsv.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_empty_tree.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_addresses.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_strerror.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_rw.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_wip.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_sw.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_ro.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt.c
	 DEP /tmp/qemu-test/src/dtc/util.c
	 DEP /tmp/qemu-test/src/dtc/fdtput.c
	 DEP /tmp/qemu-test/src/dtc/fdtoverlay.c
	 DEP /tmp/qemu-test/src/dtc/fdtget.c
	 DEP /tmp/qemu-test/src/dtc/fdtdump.c
	 DEP convert-dtsv0-lexer.lex.c
	 DEP /tmp/qemu-test/src/dtc/srcpos.c
	 DEP dtc-parser.tab.c
	 DEP /tmp/qemu-test/src/dtc/treesource.c
	 DEP dtc-lexer.lex.c
	 DEP /tmp/qemu-test/src/dtc/livetree.c
	 DEP /tmp/qemu-test/src/dtc/fstree.c
	 DEP /tmp/qemu-test/src/dtc/flattree.c
	 DEP /tmp/qemu-test/src/dtc/dtc.c
	 DEP /tmp/qemu-test/src/dtc/data.c
	 DEP /tmp/qemu-test/src/dtc/checks.c
	 CC libfdt/fdt_ro.o
	 CC libfdt/fdt.o
	 CC libfdt/fdt_sw.o
	 CC libfdt/fdt_wip.o
	 CC libfdt/fdt_strerror.o
	 CC libfdt/fdt_rw.o
	 CC libfdt/fdt_empty_tree.o
	 CC libfdt/fdt_addresses.o
	 CC libfdt/fdt_overlay.o
	 AR libfdt/libfdt.a
i686-w64-mingw32-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
  RC      version.o
  GEN     qga/qapi-generated/qapi-gen
  CC      qapi/qapi-types.o
  CC      qapi/qapi-types-block-core.o
  CC      qapi/qapi-types-block.o
  CC      qapi/qapi-types-char.o
  CC      qapi/qapi-builtin-types.o
  CC      qapi/qapi-types-common.o
  CC      qapi/qapi-types-crypto.o
  CC      qapi/qapi-types-introspect.o
  CC      qapi/qapi-types-job.o
  CC      qapi/qapi-types-migration.o
  CC      qapi/qapi-types-misc.o
  CC      qapi/qapi-types-net.o
  CC      qapi/qapi-types-rocker.o
  CC      qapi/qapi-types-run-state.o
  CC      qapi/qapi-types-sockets.o
  CC      qapi/qapi-types-tpm.o
  CC      qapi/qapi-types-trace.o
  CC      qapi/qapi-types-transaction.o
  CC      qapi/qapi-types-ui.o
  CC      qapi/qapi-builtin-visit.o
  CC      qapi/qapi-visit.o
  CC      qapi/qapi-visit-block-core.o
  CC      qapi/qapi-visit-block.o
  CC      qapi/qapi-visit-char.o
  CC      qapi/qapi-visit-common.o
  CC      qapi/qapi-visit-crypto.o
  CC      qapi/qapi-visit-introspect.o
  CC      qapi/qapi-visit-job.o
  CC      qapi/qapi-visit-migration.o
  CC      qapi/qapi-visit-misc.o
  CC      qapi/qapi-visit-net.o
  CC      qapi/qapi-visit-rocker.o
  CC      qapi/qapi-visit-run-state.o
  CC      qapi/qapi-visit-sockets.o
  CC      qapi/qapi-visit-tpm.o
  CC      qapi/qapi-visit-trace.o
  CC      qapi/qapi-visit-transaction.o
  CC      qapi/qapi-visit-ui.o
  CC      qapi/qapi-events.o
  CC      qapi/qapi-events-block-core.o
  CC      qapi/qapi-events-block.o
  CC      qapi/qapi-events-char.o
  CC      qapi/qapi-events-common.o
  CC      qapi/qapi-events-crypto.o
  CC      qapi/qapi-events-introspect.o
  CC      qapi/qapi-events-migration.o
  CC      qapi/qapi-events-job.o
  CC      qapi/qapi-events-misc.o
  CC      qapi/qapi-events-net.o
  CC      qapi/qapi-events-rocker.o
  CC      qapi/qapi-events-run-state.o
  CC      qapi/qapi-events-sockets.o
  CC      qapi/qapi-events-tpm.o
  CC      qapi/qapi-events-trace.o
  CC      qapi/qapi-events-transaction.o
  CC      qapi/qapi-events-ui.o
  CC      qapi/qapi-introspect.o
  CC      qapi/qapi-visit-core.o
  CC      qapi/qapi-dealloc-visitor.o
  CC      qapi/qobject-input-visitor.o
  CC      qapi/qobject-output-visitor.o
  CC      qapi/qmp-registry.o
  CC      qapi/qmp-dispatch.o
  CC      qapi/string-input-visitor.o
  CC      qapi/string-output-visitor.o
  CC      qapi/opts-visitor.o
  CC      qapi/qapi-clone-visitor.o
  CC      qapi/qmp-event.o
  CC      qapi/qapi-util.o
  CC      qobject/qnull.o
  CC      qobject/qnum.o
  CC      qobject/qstring.o
  CC      qobject/qlist.o
  CC      qobject/qdict.o
  CC      qobject/qbool.o
  CC      qobject/qlit.o
  CC      qobject/qjson.o
  CC      qobject/qobject.o
  CC      qobject/json-lexer.o
  CC      qobject/json-streamer.o
  CC      qobject/json-parser.o
  CC      qobject/block-qdict.o
  CC      trace/simple.o
  CC      trace/control.o
  CC      trace/qmp.o
  CC      util/osdep.o
  CC      util/cutils.o
  CC      util/unicode.o
  CC      util/qemu-timer-common.o
  CC      util/bufferiszero.o
  CC      util/lockcnt.o
  CC      util/aiocb.o
  CC      util/async.o
  CC      util/aio-wait.o
  CC      util/thread-pool.o
  CC      util/qemu-timer.o
  CC      util/main-loop.o
  CC      util/iohandler.o
  CC      util/aio-win32.o
  CC      util/event_notifier-win32.o
  CC      util/oslib-win32.o
  CC      util/qemu-thread-win32.o
  CC      util/envlist.o
  CC      util/path.o
  CC      util/module.o
  CC      util/host-utils.o
  CC      util/bitmap.o
  CC      util/bitops.o
  CC      util/hbitmap.o
  CC      util/fifo8.o
  CC      util/acl.o
  CC      util/cacheinfo.o
  CC      util/error.o
  CC      util/qemu-error.o
  CC      util/id.o
  CC      util/iov.o
  CC      util/qemu-config.o
  CC      util/qemu-sockets.o
  CC      util/uri.o
  CC      util/notify.o
  CC      util/qemu-option.o
  CC      util/qemu-progress.o
  CC      util/keyval.o
  CC      util/hexdump.o
  CC      util/crc32c.o
  CC      util/uuid.o
  CC      util/throttle.o
  CC      util/getauxval.o
  CC      util/readline.o
  CC      util/rcu.o
  CC      util/qemu-coroutine.o
  CC      util/qemu-coroutine-lock.o
  CC      util/qemu-coroutine-io.o
  CC      util/qemu-coroutine-sleep.o
  CC      util/buffer.o
  CC      util/coroutine-win32.o
  CC      util/timed-average.o
  CC      util/base64.o
  CC      util/log.o
  CC      util/pagesize.o
  CC      util/qdist.o
  CC      util/qht.o
  CC      util/range.o
  CC      util/stats64.o
  CC      util/systemd.o
  CC      util/iova-tree.o
  CC      trace-root.o
  CC      accel/kvm/trace.o
  CC      accel/tcg/trace.o
  CC      block/trace.o
  CC      audio/trace.o
  CC      chardev/trace.o
  CC      crypto/trace.o
  CC      hw/9pfs/trace.o
  CC      hw/acpi/trace.o
  CC      hw/alpha/trace.o
  CC      hw/arm/trace.o
  CC      hw/audio/trace.o
  CC      hw/block/trace.o
  CC      hw/block/dataplane/trace.o
  CC      hw/display/trace.o
  CC      hw/char/trace.o
  CC      hw/dma/trace.o
  CC      hw/hppa/trace.o
  CC      hw/i2c/trace.o
  CC      hw/i386/trace.o
  CC      hw/i386/xen/trace.o
  CC      hw/ide/trace.o
  CC      hw/input/trace.o
  CC      hw/intc/trace.o
  CC      hw/isa/trace.o
  CC      hw/mem/trace.o
  CC      hw/misc/trace.o
  CC      hw/misc/macio/trace.o
  CC      hw/net/trace.o
  CC      hw/nvram/trace.o
  CC      hw/pci/trace.o
  CC      hw/pci-host/trace.o
  CC      hw/ppc/trace.o
  CC      hw/rdma/trace.o
  CC      hw/rdma/vmw/trace.o
  CC      hw/s390x/trace.o
  CC      hw/scsi/trace.o
  CC      hw/sd/trace.o
  CC      hw/sparc/trace.o
  CC      hw/sparc64/trace.o
  CC      hw/timer/trace.o
  CC      hw/tpm/trace.o
  CC      hw/usb/trace.o
  CC      hw/vfio/trace.o
  CC      hw/virtio/trace.o
  CC      hw/xen/trace.o
  CC      io/trace.o
  CC      linux-user/trace.o
  CC      migration/trace.o
  CC      nbd/trace.o
  CC      net/trace.o
  CC      qapi/trace.o
  CC      qom/trace.o
  CC      scsi/trace.o
  CC      target/arm/trace.o
  CC      target/i386/trace.o
  CC      target/mips/trace.o
  CC      target/ppc/trace.o
  CC      target/s390x/trace.o
  CC      target/sparc/trace.o
  CC      ui/trace.o
  CC      crypto/pbkdf-stub.o
  CC      util/trace.o
  CC      stubs/arch-query-cpu-def.o
  CC      stubs/arch-query-cpu-model-expansion.o
  CC      stubs/arch-query-cpu-model-comparison.o
  CC      stubs/arch-query-cpu-model-baseline.o
  CC      stubs/bdrv-next-monitor-owned.o
  CC      stubs/blk-commit-all.o
  CC      stubs/blockdev-close-all-bdrv-states.o
  CC      stubs/clock-warp.o
  CC      stubs/cpu-get-clock.o
  CC      stubs/cpu-get-icount.o
  CC      stubs/dump.o
  CC      stubs/error-printf.o
  CC      stubs/fdset.o
  CC      stubs/gdbstub.o
  CC      stubs/get-vm-name.o
  CC      stubs/iothread-lock.o
  CC      stubs/iothread.o
  CC      stubs/is-daemonized.o
  CC      stubs/machine-init-done.o
  CC      stubs/migr-blocker.o
  CC      stubs/change-state-handler.o
  CC      stubs/notify-event.o
  CC      stubs/monitor.o
  CC      stubs/qtest.o
  CC      stubs/replay.o
  CC      stubs/runstate-check.o
  CC      stubs/set-fd-handler.o
  CC      stubs/slirp.o
  CC      stubs/sysbus.o
  CC      stubs/tpm.o
  CC      stubs/trace-control.o
  CC      stubs/uuid.o
  CC      stubs/vm-stop.o
  CC      stubs/fd-register.o
  CC      stubs/vmstate.o
  CC      stubs/qmp_memory_device.o
  CC      stubs/target-monitor-defs.o
  CC      stubs/target-get-monitor-def.o
  CC      stubs/pc_madt_cpu_entry.o
  CC      stubs/xen-common.o
  CC      stubs/vmgenid.o
  CC      stubs/xen-hvm.o
  CC      stubs/pci-host-piix.o
  CC      stubs/ram-block.o
  GEN     qemu-img-cmds.h
  CC      block.o
  CC      blockjob.o
  CC      qemu-io-cmds.o
  CC      job.o
  CC      replication.o
  CC      block/raw-format.o
  CC      block/qcow.o
  CC      block/vdi.o
  CC      block/vmdk.o
  CC      block/cloop.o
  CC      block/bochs.o
  CC      block/vpc.o
  CC      block/vvfat.o
  CC      block/dmg.o
  CC      block/qcow2.o
  CC      block/qcow2-refcount.o
  CC      block/qcow2-cluster.o
  CC      block/qcow2-snapshot.o
  CC      block/qcow2-cache.o
  CC      block/qcow2-bitmap.o
  CC      block/qed.o
  CC      block/qed-l2-cache.o
  CC      block/qed-table.o
  CC      block/qed-cluster.o
  CC      block/vhdx.o
  CC      block/qed-check.o
  CC      block/vhdx-endian.o
  CC      block/vhdx-log.o
  CC      block/quorum.o
  CC      block/parallels.o
  CC      block/blkdebug.o
  CC      block/blkverify.o
  CC      block/blkreplay.o
  CC      block/block-backend.o
  CC      block/snapshot.o
  CC      block/qapi.o
  CC      block/file-win32.o
  CC      block/win32-aio.o
  CC      block/null.o
  CC      block/mirror.o
  CC      block/commit.o
  CC      block/create.o
  CC      block/io.o
  CC      block/throttle-groups.o
  CC      block/nbd.o
  CC      block/nbd-client.o
  CC      block/sheepdog.o
  CC      block/accounting.o
  CC      block/dirty-bitmap.o
  CC      block/write-threshold.o
  CC      block/backup.o
  CC      block/replication.o
  CC      block/throttle.o
  CC      block/copy-on-read.o
  CC      block/crypto.o
  CC      nbd/server.o
  CC      nbd/client.o
  CC      nbd/common.o
  CC      scsi/utils.o
  CC      block/curl.o
  CC      block/ssh.o
  CC      block/dmg-bz2.o
  CC      crypto/init.o
  CC      crypto/hash.o
  CC      crypto/hash-nettle.o
  CC      crypto/hmac.o
  CC      crypto/hmac-nettle.o
  CC      crypto/aes.o
  CC      crypto/desrfb.o
  CC      crypto/cipher.o
  CC      crypto/tlscreds.o
  CC      crypto/tlscredsanon.o
  CC      crypto/tlscredsx509.o
  CC      crypto/tlssession.o
  CC      crypto/secret.o
  CC      crypto/random-gnutls.o
  CC      crypto/pbkdf.o
  CC      crypto/pbkdf-nettle.o
  CC      crypto/ivgen.o
  CC      crypto/ivgen-essiv.o
  CC      crypto/ivgen-plain.o
  CC      crypto/ivgen-plain64.o
  CC      crypto/afsplit.o
  CC      crypto/xts.o
  CC      crypto/block.o
  CC      crypto/block-qcow.o
  CC      crypto/block-luks.o
  CC      io/channel.o
  CC      io/channel-buffer.o
  CC      io/channel-command.o
  CC      io/channel-file.o
  CC      io/channel-socket.o
  CC      io/channel-tls.o
  CC      io/channel-watch.o
  CC      io/channel-websock.o
  CC      io/channel-util.o
  CC      io/dns-resolver.o
  CC      io/net-listener.o
  CC      io/task.o
  CC      qom/object.o
  CC      qom/container.o
  CC      qom/qom-qobject.o
  CC      qom/object_interfaces.o
  CC      qemu-io.o
  CC      blockdev.o
  CC      bootdevice.o
  CC      blockdev-nbd.o
  CC      iothread.o
  CC      job-qmp.o
  CC      qdev-monitor.o
  CC      device-hotplug.o
  CC      os-win32.o
  CC      bt-host.o
  CC      bt-vhci.o
  CC      dma-helpers.o
  CC      vl.o
  CC      tpm.o
  CC      device_tree.o
  CC      qapi/qapi-commands-block-core.o
  CC      qapi/qapi-commands.o
  CC      qapi/qapi-commands-block.o
  CC      qapi/qapi-commands-char.o
  CC      qapi/qapi-commands-common.o
  CC      qapi/qapi-commands-crypto.o
  CC      qapi/qapi-commands-introspect.o
  CC      qapi/qapi-commands-migration.o
  CC      qapi/qapi-commands-job.o
  CC      qapi/qapi-commands-misc.o
  CC      qapi/qapi-commands-net.o
  CC      qapi/qapi-commands-rocker.o
  CC      qapi/qapi-commands-run-state.o
  CC      qapi/qapi-commands-sockets.o
  CC      qapi/qapi-commands-tpm.o
  CC      qapi/qapi-commands-trace.o
  CC      qapi/qapi-commands-transaction.o
  CC      qapi/qapi-commands-ui.o
  CC      qmp.o
  CC      cpus-common.o
  CC      hmp.o
  CC      audio/audio.o
  CC      audio/noaudio.o
  CC      audio/wavaudio.o
  CC      audio/mixeng.o
  CC      audio/dsoundaudio.o
  CC      audio/audio_win_int.o
  CC      audio/wavcapture.o
  CC      backends/rng.o
  CC      backends/rng-egd.o
  CC      backends/tpm.o
  CC      backends/hostmem.o
  CC      backends/hostmem-ram.o
  CC      backends/cryptodev.o
  CC      backends/cryptodev-builtin.o
  CC      backends/cryptodev-vhost.o
  CC      block/stream.o
  CC      chardev/msmouse.o
  CC      chardev/wctablet.o
  CC      chardev/testdev.o
  CC      disas/arm.o
  CXX     disas/arm-a64.o
  CC      disas/i386.o
  CXX     disas/libvixl/vixl/utils.o
  CXX     disas/libvixl/vixl/compiler-intrinsics.o
  CXX     disas/libvixl/vixl/a64/decoder-a64.o
  CXX     disas/libvixl/vixl/a64/instructions-a64.o
  CXX     disas/libvixl/vixl/a64/disasm-a64.o
  CC      hw/acpi/core.o
  CC      hw/acpi/piix4.o
  CC      hw/acpi/pcihp.o
  CC      hw/acpi/ich9.o
  CC      hw/acpi/tco.o
  CC      hw/acpi/cpu_hotplug.o
  CC      hw/acpi/memory_hotplug.o
  CC      hw/acpi/cpu.o
  CC      hw/acpi/nvdimm.o
  CC      hw/acpi/vmgenid.o
  CC      hw/acpi/acpi_interface.o
  CC      hw/acpi/bios-linker-loader.o
  CC      hw/acpi/aml-build.o
  CC      hw/acpi/ipmi.o
  CC      hw/acpi/acpi-stub.o
  CC      hw/acpi/ipmi-stub.o
  CC      hw/audio/sb16.o
  CC      hw/audio/es1370.o
  CC      hw/audio/ac97.o
  CC      hw/audio/fmopl.o
  CC      hw/audio/adlib.o
  CC      hw/audio/gus.o
  CC      hw/audio/gusemu_hal.o
  CC      hw/audio/gusemu_mixer.o
  CC      hw/audio/cs4231a.o
  CC      hw/audio/intel-hda.o
  CC      hw/audio/hda-codec.o
  CC      hw/audio/pcspk.o
  CC      hw/audio/wm8750.o
  CC      hw/audio/pl041.o
  CC      hw/audio/lm4549.o
  CC      hw/audio/marvell_88w8618.o
  CC      hw/audio/soundhw.o
  CC      hw/block/block.o
  CC      hw/block/cdrom.o
  CC      hw/block/hd-geometry.o
  CC      hw/block/fdc.o
  CC      hw/block/m25p80.o
  CC      hw/block/nand.o
  CC      hw/block/pflash_cfi01.o
  CC      hw/block/pflash_cfi02.o
  CC      hw/block/ecc.o
  CC      hw/block/onenand.o
  CC      hw/block/nvme.o
  CC      hw/bt/core.o
  CC      hw/bt/l2cap.o
  CC      hw/bt/sdp.o
  CC      hw/bt/hci.o
  CC      hw/bt/hid.o
  CC      hw/bt/hci-csr.o
  CC      hw/char/ipoctal232.o
  CC      hw/char/parallel.o
  CC      hw/char/parallel-isa.o
  CC      hw/char/pl011.o
  CC      hw/char/serial-isa.o
  CC      hw/char/serial.o
  CC      hw/char/serial-pci.o
  CC      hw/char/virtio-console.o
  CC      hw/char/cadence_uart.o
  CC      hw/char/cmsdk-apb-uart.o
  CC      hw/char/debugcon.o
  CC      hw/char/imx_serial.o
  CC      hw/core/qdev.o
  CC      hw/core/qdev-properties.o
  CC      hw/core/bus.o
  CC      hw/core/reset.o
  CC      hw/core/qdev-fw.o
  CC      hw/core/fw-path-provider.o
  CC      hw/core/irq.o
  CC      hw/core/hotplug.o
  CC      hw/core/nmi.o
  CC      hw/core/stream.o
  CC      hw/core/ptimer.o
  CC      hw/core/sysbus.o
  CC      hw/core/machine.o
  CC      hw/core/loader.o
  CC      hw/core/qdev-properties-system.o
  CC      hw/core/register.o
  CC      hw/core/or-irq.o
  CC      hw/core/split-irq.o
  CC      hw/core/platform-bus.o
  CC      hw/cpu/core.o
  CC      hw/display/ramfb.o
  CC      hw/display/ramfb-standalone.o
  CC      hw/display/ads7846.o
  CC      hw/display/cirrus_vga.o
  CC      hw/display/pl110.o
  CC      hw/display/sii9022.o
  CC      hw/display/ssd0303.o
  CC      hw/display/ssd0323.o
  CC      hw/display/vga-pci.o
  CC      hw/display/bochs-display.o
  CC      hw/display/vga-isa.o
  CC      hw/display/vmware_vga.o
  CC      hw/display/blizzard.o
  CC      hw/display/exynos4210_fimd.o
  CC      hw/display/framebuffer.o
  CC      hw/display/tc6393xb.o
  CC      hw/dma/pl080.o
  CC      hw/dma/pl330.o
  CC      hw/dma/i8257.o
  CC      hw/dma/xilinx_axidma.o
  CC      hw/dma/xlnx-zynq-devcfg.o
  CC      hw/dma/xlnx-zdma.o
  CC      hw/gpio/max7310.o
  CC      hw/gpio/pl061.o
  CC      hw/gpio/zaurus.o
  CC      hw/gpio/gpio_key.o
  CC      hw/i2c/core.o
  CC      hw/i2c/smbus.o
  CC      hw/i2c/smbus_eeprom.o
  CC      hw/i2c/i2c-ddc.o
  CC      hw/i2c/versatile_i2c.o
  CC      hw/i2c/smbus_ich9.o
  CC      hw/i2c/pm_smbus.o
  CC      hw/i2c/bitbang_i2c.o
  CC      hw/i2c/exynos4210_i2c.o
  CC      hw/i2c/imx_i2c.o
  CC      hw/i2c/aspeed_i2c.o
  CC      hw/ide/core.o
  CC      hw/ide/atapi.o
  CC      hw/ide/qdev.o
  CC      hw/ide/pci.o
  CC      hw/ide/isa.o
  CC      hw/ide/piix.o
  CC      hw/ide/microdrive.o
  CC      hw/ide/ahci.o
  CC      hw/ide/ich.o
  CC      hw/ide/ahci-allwinner.o
  CC      hw/input/hid.o
  CC      hw/input/lm832x.o
  CC      hw/input/pckbd.o
  CC      hw/input/pl050.o
  CC      hw/input/ps2.o
  CC      hw/input/stellaris_input.o
  CC      hw/input/virtio-input.o
  CC      hw/input/tsc2005.o
  CC      hw/intc/i8259_common.o
  CC      hw/input/virtio-input-hid.o
  CC      hw/intc/pl190.o
  CC      hw/intc/i8259.o
  CC      hw/intc/xlnx-pmu-iomod-intc.o
  CC      hw/intc/xlnx-zynqmp-ipi.o
  CC      hw/intc/imx_avic.o
  CC      hw/intc/imx_gpcv2.o
  CC      hw/intc/ioapic_common.o
  CC      hw/intc/realview_gic.o
  CC      hw/intc/arm_gic_common.o
  CC      hw/intc/arm_gic.o
  CC      hw/intc/arm_gicv2m.o
  CC      hw/intc/arm_gicv3_common.o
  CC      hw/intc/arm_gicv3.o
  CC      hw/intc/arm_gicv3_dist.o
  CC      hw/intc/arm_gicv3_redist.o
  CC      hw/intc/arm_gicv3_its_common.o
  CC      hw/intc/intc.o
  CC      hw/ipack/ipack.o
  CC      hw/ipack/tpci200.o
  CC      hw/ipmi/ipmi.o
  CC      hw/ipmi/ipmi_bmc_sim.o
  CC      hw/ipmi/ipmi_bmc_extern.o
  CC      hw/ipmi/isa_ipmi_kcs.o
  CC      hw/ipmi/isa_ipmi_bt.o
  CC      hw/isa/isa-superio.o
  CC      hw/isa/isa-bus.o
  CC      hw/isa/smc37c669-superio.o
  CC      hw/isa/apm.o
  CC      hw/mem/pc-dimm.o
  CC      hw/mem/memory-device.o
  CC      hw/mem/nvdimm.o
  CC      hw/misc/applesmc.o
  CC      hw/misc/max111x.o
  CC      hw/misc/tmp421.o
  CC      hw/misc/tmp105.o
  CC      hw/misc/debugexit.o
  CC      hw/misc/sga.o
  CC      hw/misc/pc-testdev.o
  CC      hw/misc/pci-testdev.o
  CC      hw/misc/edu.o
  CC      hw/misc/pca9552.o
  CC      hw/misc/vmcoreinfo.o
  CC      hw/misc/arm_l2x0.o
  CC      hw/misc/unimp.o
  CC      hw/misc/arm_integrator_debug.o
  CC      hw/misc/a9scu.o
  CC      hw/misc/arm11scu.o
  CC      hw/net/ne2000.o
  CC      hw/net/eepro100.o
  CC      hw/net/pcnet-pci.o
  CC      hw/net/pcnet.o
  CC      hw/net/e1000.o
  CC      hw/net/e1000x_common.o
  CC      hw/net/net_tx_pkt.o
  CC      hw/net/net_rx_pkt.o
  CC      hw/net/e1000e.o
  CC      hw/net/e1000e_core.o
  CC      hw/net/rtl8139.o
  CC      hw/net/vmxnet3.o
  CC      hw/net/smc91c111.o
  CC      hw/net/lan9118.o
  CC      hw/net/ne2000-isa.o
  CC      hw/net/xilinx_axienet.o
  CC      hw/net/xgmac.o
  CC      hw/net/allwinner_emac.o
  CC      hw/net/imx_fec.o
  CC      hw/net/cadence_gem.o
  CC      hw/net/stellaris_enet.o
  CC      hw/net/ftgmac100.o
  CC      hw/net/rocker/rocker.o
  CC      hw/net/rocker/rocker_fp.o
  CC      hw/net/rocker/rocker_desc.o
  CC      hw/net/rocker/rocker_world.o
  CC      hw/net/rocker/rocker_of_dpa.o
  CC      hw/net/can/can_sja1000.o
  CC      hw/net/can/can_kvaser_pci.o
  CC      hw/net/can/can_pcm3680_pci.o
  CC      hw/net/can/can_mioe3680_pci.o
  CC      hw/nvram/eeprom93xx.o
  CC      hw/nvram/eeprom_at24c.o
  CC      hw/nvram/fw_cfg.o
  CC      hw/nvram/chrp_nvram.o
  CC      hw/pci-bridge/pci_bridge_dev.o
  CC      hw/pci-bridge/pcie_root_port.o
  CC      hw/pci-bridge/gen_pcie_root_port.o
  CC      hw/pci-bridge/pcie_pci_bridge.o
  CC      hw/pci-bridge/pci_expander_bridge.o
  CC      hw/pci-bridge/xio3130_upstream.o
  CC      hw/pci-bridge/xio3130_downstream.o
  CC      hw/pci-bridge/ioh3420.o
  CC      hw/pci-bridge/i82801b11.o
  CC      hw/pci-host/pam.o
  CC      hw/pci-host/versatile.o
  CC      hw/pci-host/piix.o
  CC      hw/pci-host/q35.o
  CC      hw/pci-host/gpex.o
  CC      hw/pci-host/designware.o
  CC      hw/pci/pci.o
  CC      hw/pci/pci_bridge.o
  CC      hw/pci/msix.o
  CC      hw/pci/msi.o
  CC      hw/pci/shpc.o
  CC      hw/pci/pci_host.o
  CC      hw/pci/slotid_cap.o
  CC      hw/pci/pcie_host.o
  CC      hw/pci/pcie.o
  CC      hw/pci/pcie_aer.o
  CC      hw/pci/pcie_port.o
  CC      hw/pcmcia/pcmcia.o
  CC      hw/pci/pci-stub.o
  CC      hw/scsi/scsi-disk.o
  CC      hw/scsi/scsi-generic.o
  CC      hw/scsi/scsi-bus.o
  CC      hw/scsi/lsi53c895a.o
  CC      hw/scsi/mptsas.o
  CC      hw/scsi/mptconfig.o
  CC      hw/scsi/mptendian.o
  CC      hw/scsi/megasas.o
  CC      hw/scsi/vmw_pvscsi.o
  CC      hw/scsi/esp.o
  CC      hw/scsi/esp-pci.o
  CC      hw/sd/pl181.o
  CC      hw/sd/ssi-sd.o
  CC      hw/sd/sd.o
  CC      hw/sd/core.o
  CC      hw/sd/sdmmc-internal.o
  CC      hw/sd/sdhci.o
  CC      hw/smbios/smbios_type_38.o
  CC      hw/smbios/smbios.o
  CC      hw/smbios/smbios-stub.o
  CC      hw/smbios/smbios_type_38-stub.o
  CC      hw/ssi/pl022.o
  CC      hw/ssi/ssi.o
  CC      hw/ssi/xilinx_spips.o
  CC      hw/ssi/aspeed_smc.o
  CC      hw/ssi/stm32f2xx_spi.o
  CC      hw/ssi/mss-spi.o
  CC      hw/timer/arm_mptimer.o
  CC      hw/timer/arm_timer.o
  CC      hw/timer/armv7m_systick.o
  CC      hw/timer/a9gtimer.o
  CC      hw/timer/cadence_ttc.o
  CC      hw/timer/ds1338.o
  CC      hw/timer/hpet.o
  CC      hw/timer/i8254_common.o
  CC      hw/timer/i8254.o
  CC      hw/timer/pl031.o
  CC      hw/timer/twl92230.o
  CC      hw/timer/imx_epit.o
  CC      hw/timer/imx_gpt.o
  CC      hw/timer/xlnx-zynqmp-rtc.o
  CC      hw/timer/stm32f2xx_timer.o
  CC      hw/timer/aspeed_timer.o
  CC      hw/timer/cmsdk-apb-timer.o
  CC      hw/timer/mss-timer.o
  CC      hw/tpm/tpm_util.o
  CC      hw/tpm/tpm_tis.o
  CC      hw/tpm/tpm_crb.o
  CC      hw/usb/core.o
  CC      hw/usb/combined-packet.o
  CC      hw/usb/bus.o
  CC      hw/usb/libhw.o
  CC      hw/usb/desc.o
  CC      hw/usb/desc-msos.o
  CC      hw/usb/hcd-uhci.o
  CC      hw/usb/hcd-ohci.o
  CC      hw/usb/hcd-ehci.o
  CC      hw/usb/hcd-ehci-pci.o
  CC      hw/usb/hcd-ehci-sysbus.o
  CC      hw/usb/hcd-xhci.o
  CC      hw/usb/hcd-xhci-nec.o
  CC      hw/usb/hcd-musb.o
  CC      hw/usb/dev-hub.o
  CC      hw/usb/dev-hid.o
  CC      hw/usb/dev-wacom.o
  CC      hw/usb/dev-storage.o
  CC      hw/usb/dev-uas.o
  CC      hw/usb/dev-audio.o
  CC      hw/usb/dev-serial.o
  CC      hw/usb/dev-network.o
  CC      hw/usb/dev-bluetooth.o
  CC      hw/usb/dev-smartcard-reader.o
  CC      hw/usb/host-stub.o
  CC      hw/virtio/virtio-bus.o
  CC      hw/virtio/virtio-rng.o
  CC      hw/virtio/virtio-pci.o
  CC      hw/virtio/virtio-mmio.o
  CC      hw/virtio/vhost-stub.o
  CC      hw/watchdog/watchdog.o
  CC      hw/watchdog/wdt_i6300esb.o
  CC      hw/watchdog/wdt_ib700.o
  CC      hw/watchdog/wdt_aspeed.o
  CC      migration/socket.o
  CC      migration/migration.o
  CC      migration/exec.o
  CC      migration/fd.o
  CC      migration/channel.o
  CC      migration/tls.o
  CC      migration/savevm.o
  CC      migration/colo-comm.o
  CC      migration/colo.o
  CC      migration/colo-failover.o
  CC      migration/vmstate.o
  CC      migration/vmstate-types.o
  CC      migration/page_cache.o
  CC      migration/qemu-file.o
  CC      migration/global_state.o
  CC      migration/qemu-file-channel.o
  CC      migration/xbzrle.o
  CC      migration/postcopy-ram.o
  CC      migration/qjson.o
  CC      migration/block-dirty-bitmap.o
  CC      migration/block.o
  CC      net/net.o
  CC      net/queue.o
  CC      net/checksum.o
  CC      net/util.o
  CC      net/hub.o
  CC      net/socket.o
  CC      net/dump.o
  CC      net/eth.o
  CC      net/slirp.o
  CC      net/filter.o
  CC      net/filter-buffer.o
  CC      net/filter-mirror.o
  CC      net/colo-compare.o
  CC      net/colo.o
  CC      net/filter-rewriter.o
  CC      net/filter-replay.o
  CC      net/tap-win32.o
  CC      net/can/can_core.o
  CC      net/can/can_host.o
  CC      qom/cpu.o
  CC      replay/replay.o
  CC      replay/replay-internal.o
  CC      replay/replay-events.o
  CC      replay/replay-time.o
  CC      replay/replay-input.o
  CC      replay/replay-char.o
  CC      replay/replay-snapshot.o
  CC      replay/replay-net.o
  CC      replay/replay-audio.o
  CC      slirp/cksum.o
  CC      slirp/if.o
  CC      slirp/ip_icmp.o
  CC      slirp/ip6_icmp.o
  CC      slirp/ip6_output.o
  CC      slirp/ip6_input.o
  CC      slirp/ip_input.o
  CC      slirp/ip_output.o
  CC      slirp/dnssearch.o
  CC      slirp/dhcpv6.o
  CC      slirp/slirp.o
  CC      slirp/mbuf.o
  CC      slirp/misc.o
  CC      slirp/sbuf.o
  CC      slirp/socket.o
  CC      slirp/tcp_input.o
  CC      slirp/tcp_output.o
  CC      slirp/tcp_subr.o
  CC      slirp/tcp_timer.o
  CC      slirp/udp.o
  CC      slirp/udp6.o
  CC      slirp/bootp.o
  CC      slirp/tftp.o
  CC      slirp/arp_table.o
  CC      slirp/ndp_table.o
  CC      slirp/ncsi.o
  CC      ui/keymaps.o
  CC      ui/console.o
  CC      ui/cursor.o
  CC      ui/qemu-pixman.o
  CC      ui/input.o
  CC      ui/input-keymap.o
  CC      ui/input-legacy.o
  CC      ui/vnc.o
  CC      ui/vnc-enc-zlib.o
  CC      ui/vnc-enc-hextile.o
  CC      ui/vnc-enc-tight.o
  CC      ui/vnc-palette.o
  CC      ui/vnc-enc-zrle.o
  CC      ui/vnc-auth-vencrypt.o
  CC      ui/vnc-ws.o
  CC      ui/vnc-jobs.o
  CC      ui/sdl2.o
  CC      ui/sdl2-input.o
  CC      ui/sdl2-2d.o
  CC      ui/gtk.o
  CC      chardev/char.o
  CC      chardev/char-console.o
  CC      chardev/char-fe.o
  CC      chardev/char-file.o
  CC      chardev/char-io.o
  CC      chardev/char-mux.o
  CC      chardev/char-null.o
  CC      chardev/char-pipe.o
  CC      chardev/char-ringbuf.o
  CC      chardev/char-serial.o
  CC      chardev/char-socket.o
  CC      chardev/char-stdio.o
  CC      chardev/char-udp.o
  CC      chardev/char-win.o
  CC      chardev/char-win-stdio.o
  CC      qga/commands.o
  CC      qga/guest-agent-command-state.o
  CC      qga/main.o
  CC      qga/commands-win32.o
  CC      qga/channel-win32.o
  CC      qga/service-win32.o
  CC      qga/vss-win32.o
  CC      qga/qapi-generated/qga-qapi-types.o
  AS      optionrom/multiboot.o
  AS      optionrom/linuxboot.o
  CC      qga/qapi-generated/qga-qapi-visit.o
  CC      optionrom/linuxboot_dma.o
  AS      optionrom/kvmvapic.o
  BUILD   optionrom/multiboot.img
  CC      qga/qapi-generated/qga-qapi-commands.o
  BUILD   optionrom/linuxboot.img
  AR      libqemuutil.a
  BUILD   optionrom/linuxboot_dma.img
  CC      qemu-img.o
  BUILD   optionrom/multiboot.raw
  BUILD   optionrom/linuxboot.raw
  BUILD   optionrom/linuxboot_dma.raw
  BUILD   optionrom/kvmvapic.img
  BUILD   optionrom/kvmvapic.raw
  SIGN    optionrom/multiboot.bin
  SIGN    optionrom/linuxboot_dma.bin
  SIGN    optionrom/linuxboot.bin
  SIGN    optionrom/kvmvapic.bin
  LINK    qemu-img.exe
  LINK    qemu-io.exe
  LINK    qemu-ga.exe
  GEN     x86_64-softmmu/hmp-commands.h
  GEN     x86_64-softmmu/config-target.h
  CC      x86_64-softmmu/tcg/tcg.o
  CC      x86_64-softmmu/tcg/tcg-op.o
  CC      x86_64-softmmu/exec.o
  CC      x86_64-softmmu/tcg/tcg-common.o
  CC      x86_64-softmmu/tcg/tcg-op-gvec.o
  CC      x86_64-softmmu/tcg/tcg-op-vec.o
  CC      x86_64-softmmu/tcg/optimize.o
  GEN     aarch64-softmmu/hmp-commands.h
  GEN     aarch64-softmmu/config-target.h
  CC      x86_64-softmmu/fpu/softfloat.o
  CC      x86_64-softmmu/disas.o
  GEN     x86_64-softmmu/gdbstub-xml.c
  CC      x86_64-softmmu/arch_init.o
  CC      aarch64-softmmu/exec.o
  CC      aarch64-softmmu/tcg/tcg.o
  CC      aarch64-softmmu/tcg/tcg-op.o
  CC      x86_64-softmmu/cpus.o
  CC      x86_64-softmmu/monitor.o
  CC      x86_64-softmmu/gdbstub.o
  CC      aarch64-softmmu/tcg/tcg-op-vec.o
  CC      x86_64-softmmu/balloon.o
  CC      aarch64-softmmu/tcg/tcg-op-gvec.o
  CC      x86_64-softmmu/ioport.o
  CC      aarch64-softmmu/tcg/optimize.o
  CC      aarch64-softmmu/tcg/tcg-common.o
  CC      aarch64-softmmu/fpu/softfloat.o
  CC      x86_64-softmmu/numa.o
  GEN     aarch64-softmmu/gdbstub-xml.c
  CC      aarch64-softmmu/disas.o
  CC      x86_64-softmmu/qtest.o
  CC      aarch64-softmmu/cpus.o
  CC      aarch64-softmmu/arch_init.o
  CC      x86_64-softmmu/memory.o
  CC      aarch64-softmmu/monitor.o
  CC      x86_64-softmmu/memory_mapping.o
  CC      x86_64-softmmu/dump.o
  CC      x86_64-softmmu/migration/ram.o
  CC      x86_64-softmmu/accel/accel.o
  CC      x86_64-softmmu/accel/stubs/hvf-stub.o
  CC      x86_64-softmmu/accel/stubs/whpx-stub.o
  CC      aarch64-softmmu/gdbstub.o
  CC      x86_64-softmmu/accel/stubs/kvm-stub.o
  CC      x86_64-softmmu/accel/tcg/tcg-all.o
  CC      aarch64-softmmu/ioport.o
  CC      aarch64-softmmu/balloon.o
  CC      aarch64-softmmu/numa.o
  CC      x86_64-softmmu/accel/tcg/cputlb.o
  CC      x86_64-softmmu/accel/tcg/tcg-runtime.o
  CC      x86_64-softmmu/accel/tcg/tcg-runtime-gvec.o
  CC      aarch64-softmmu/qtest.o
  CC      aarch64-softmmu/memory.o
  CC      aarch64-softmmu/memory_mapping.o
  CC      x86_64-softmmu/accel/tcg/cpu-exec.o
  CC      x86_64-softmmu/accel/tcg/cpu-exec-common.o
  CC      x86_64-softmmu/accel/tcg/translate-all.o
  CC      aarch64-softmmu/dump.o
  CC      x86_64-softmmu/accel/tcg/translator.o
  CC      aarch64-softmmu/migration/ram.o
  CC      x86_64-softmmu/hw/block/virtio-blk.o
  CC      x86_64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      aarch64-softmmu/accel/accel.o
  CC      aarch64-softmmu/accel/stubs/hvf-stub.o
  CC      aarch64-softmmu/accel/stubs/hax-stub.o
  CC      x86_64-softmmu/hw/char/virtio-serial-bus.o
  CC      x86_64-softmmu/hw/core/generic-loader.o
  CC      aarch64-softmmu/accel/stubs/whpx-stub.o
  CC      x86_64-softmmu/hw/core/null-machine.o
  CC      x86_64-softmmu/hw/display/vga.o
  CC      aarch64-softmmu/accel/stubs/kvm-stub.o
  CC      aarch64-softmmu/accel/tcg/tcg-all.o
  CC      aarch64-softmmu/accel/tcg/cputlb.o
  CC      aarch64-softmmu/accel/tcg/tcg-runtime.o
  CC      aarch64-softmmu/accel/tcg/tcg-runtime-gvec.o
  CC      aarch64-softmmu/accel/tcg/cpu-exec.o
  CC      aarch64-softmmu/accel/tcg/cpu-exec-common.o
  CC      aarch64-softmmu/accel/tcg/translate-all.o
  CC      aarch64-softmmu/accel/tcg/translator.o
  CC      x86_64-softmmu/hw/display/virtio-gpu.o
  CC      aarch64-softmmu/hw/adc/stm32f2xx_adc.o
  CC      x86_64-softmmu/hw/display/virtio-gpu-3d.o
  CC      x86_64-softmmu/hw/display/virtio-gpu-pci.o
  CC      aarch64-softmmu/hw/block/virtio-blk.o
  CC      aarch64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      x86_64-softmmu/hw/display/virtio-vga.o
  CC      x86_64-softmmu/hw/intc/apic.o
  CC      x86_64-softmmu/hw/intc/apic_common.o
  CC      x86_64-softmmu/hw/intc/ioapic.o
  CC      x86_64-softmmu/hw/isa/lpc_ich9.o
  CC      x86_64-softmmu/hw/misc/pvpanic.o
  CC      x86_64-softmmu/hw/misc/mmio_interface.o
  CC      x86_64-softmmu/hw/net/virtio-net.o
  CC      x86_64-softmmu/hw/net/vhost_net.o
  CC      aarch64-softmmu/hw/char/omap_uart.o
  CC      aarch64-softmmu/hw/char/exynos4210_uart.o
  CC      x86_64-softmmu/hw/scsi/virtio-scsi.o
  CC      x86_64-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      x86_64-softmmu/hw/timer/mc146818rtc.o
  CC      aarch64-softmmu/hw/char/digic-uart.o
  CC      x86_64-softmmu/hw/virtio/virtio.o
  CC      aarch64-softmmu/hw/char/stm32f2xx_usart.o
  CC      aarch64-softmmu/hw/char/bcm2835_aux.o
  CC      aarch64-softmmu/hw/char/virtio-serial-bus.o
  CC      x86_64-softmmu/hw/virtio/virtio-balloon.o
  CC      x86_64-softmmu/hw/virtio/virtio-crypto.o
  CC      aarch64-softmmu/hw/core/generic-loader.o
  CC      aarch64-softmmu/hw/core/null-machine.o
  CC      x86_64-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      aarch64-softmmu/hw/cpu/arm11mpcore.o
  CC      aarch64-softmmu/hw/cpu/realview_mpcore.o
In file included from /tmp/qemu-test/src/migration/ram.c:46:0:
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_recv_unfill_packet':
/tmp/qemu-test/src/include/qapi/error.h:162:25: error: format '%lld' expects argument of type 'long long int', but argument 6 has type 'ram_addr_t {aka unsigned int}' [-Werror=format=]
                         (fmt), ## __VA_ARGS__)
                         ^
/tmp/qemu-test/src/migration/ram.c:784:13: note: in expansion of macro 'error_setg'
             error_setg(errp, "multifd: offset too long %" PRId64
             ^~~~~~~~~~
/tmp/qemu-test/src/include/qapi/error.h:162:25: error: format '%lld' expects argument of type 'long long int', but argument 7 has type 'ram_addr_t {aka unsigned int}' [-Werror=format=]
                         (fmt), ## __VA_ARGS__)
                         ^
/tmp/qemu-test/src/migration/ram.c:784:13: note: in expansion of macro 'error_setg'
             error_setg(errp, "multifd: offset too long %" PRId64
             ^~~~~~~~~~
In file included from /tmp/qemu-test/src/include/qemu/osdep.h:36:0,
                 from /tmp/qemu-test/src/migration/ram.c:29:
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_send_sync_main':
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_send_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:125:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:977:34: note: in expansion of macro 'atomic_read'
     trace_multifd_send_sync_main(atomic_read(&multifd_send_state->packet_num));
                                  ^~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_packets_size':
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_send_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:125:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:1114:22: note: in expansion of macro 'atomic_read'
     int packet_num = atomic_read(&multifd_send_state->packet_num);
                      ^~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_recv_sync_main':
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_recv_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:125:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:1208:13: note: in expansion of macro 'atomic_read'
         if (atomic_read(&multifd_recv_state->packet_num) < p->packet_num) {
             ^~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_recv_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:133:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:1209:13: note: in expansion of macro 'atomic_set'
             atomic_set(&multifd_recv_state->packet_num, p->packet_num);
             ^~~~~~~~~~
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_recv_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:125:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:1219:34: note: in expansion of macro 'atomic_read'
     trace_multifd_recv_sync_main(atomic_read(&multifd_recv_state->packet_num));
                                  ^~~~~~~~~~~
cc1: all warnings being treated as errors
  CC      aarch64-softmmu/hw/cpu/a9mpcore.o
make[1]: *** [/tmp/qemu-test/src/rules.mak:69: migration/ram.o] Error 1
make[1]: *** Waiting for unfinished jobs....
  CC      aarch64-softmmu/hw/cpu/a15mpcore.o
  CC      x86_64-softmmu/hw/i386/multiboot.o
  CC      aarch64-softmmu/hw/display/omap_dss.o
  CC      aarch64-softmmu/hw/display/pxa2xx_lcd.o
  CC      aarch64-softmmu/hw/display/omap_lcdc.o
  CC      aarch64-softmmu/hw/display/bcm2835_fb.o
  CC      aarch64-softmmu/hw/display/vga.o
  CC      aarch64-softmmu/hw/display/virtio-gpu.o
  CC      aarch64-softmmu/hw/display/virtio-gpu-pci.o
  CC      aarch64-softmmu/hw/display/virtio-gpu-3d.o
  CC      aarch64-softmmu/hw/display/dpcd.o
  CC      aarch64-softmmu/hw/display/xlnx_dp.o
make: *** [Makefile:483: subdir-x86_64-softmmu] Error 2
make: *** Waiting for unfinished jobs....
  CC      aarch64-softmmu/hw/dma/xlnx_dpdma.o
  CC      aarch64-softmmu/hw/dma/omap_dma.o
  CC      aarch64-softmmu/hw/dma/soc_dma.o
  CC      aarch64-softmmu/hw/dma/pxa2xx_dma.o
  CC      aarch64-softmmu/hw/dma/bcm2835_dma.o
  CC      aarch64-softmmu/hw/gpio/omap_gpio.o
  CC      aarch64-softmmu/hw/gpio/imx_gpio.o
  CC      aarch64-softmmu/hw/gpio/bcm2835_gpio.o
  CC      aarch64-softmmu/hw/i2c/omap_i2c.o
  CC      aarch64-softmmu/hw/input/pxa2xx_keypad.o
  CC      aarch64-softmmu/hw/intc/armv7m_nvic.o
  CC      aarch64-softmmu/hw/input/tsc210x.o
  CC      aarch64-softmmu/hw/intc/exynos4210_gic.o
  CC      aarch64-softmmu/hw/intc/exynos4210_combiner.o
  CC      aarch64-softmmu/hw/intc/omap_intc.o
  CC      aarch64-softmmu/hw/intc/bcm2835_ic.o
  CC      aarch64-softmmu/hw/intc/bcm2836_control.o
  CC      aarch64-softmmu/hw/intc/allwinner-a10-pic.o
  CC      aarch64-softmmu/hw/intc/aspeed_vic.o
  CC      aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
  CC      aarch64-softmmu/hw/misc/arm_sysctl.o
In file included from /tmp/qemu-test/src/migration/ram.c:46:0:
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_recv_unfill_packet':
/tmp/qemu-test/src/include/qapi/error.h:162:25: error: format '%lld' expects argument of type 'long long int', but argument 6 has type 'ram_addr_t {aka unsigned int}' [-Werror=format=]
                         (fmt), ## __VA_ARGS__)
                         ^
/tmp/qemu-test/src/migration/ram.c:784:13: note: in expansion of macro 'error_setg'
             error_setg(errp, "multifd: offset too long %" PRId64
             ^~~~~~~~~~
/tmp/qemu-test/src/include/qapi/error.h:162:25: error: format '%lld' expects argument of type 'long long int', but argument 7 has type 'ram_addr_t {aka unsigned int}' [-Werror=format=]
                         (fmt), ## __VA_ARGS__)
                         ^
/tmp/qemu-test/src/migration/ram.c:784:13: note: in expansion of macro 'error_setg'
             error_setg(errp, "multifd: offset too long %" PRId64
             ^~~~~~~~~~
In file included from /tmp/qemu-test/src/include/qemu/osdep.h:36:0,
                 from /tmp/qemu-test/src/migration/ram.c:29:
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_send_sync_main':
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_send_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:125:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:977:34: note: in expansion of macro 'atomic_read'
     trace_multifd_send_sync_main(atomic_read(&multifd_send_state->packet_num));
                                  ^~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_packets_size':
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_send_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:125:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:1114:22: note: in expansion of macro 'atomic_read'
     int packet_num = atomic_read(&multifd_send_state->packet_num);
                      ^~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_recv_sync_main':
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_recv_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:125:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:1208:13: note: in expansion of macro 'atomic_read'
         if (atomic_read(&multifd_recv_state->packet_num) < p->packet_num) {
             ^~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_recv_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:133:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:1209:13: note: in expansion of macro 'atomic_set'
             atomic_set(&multifd_recv_state->packet_num, p->packet_num);
             ^~~~~~~~~~
/tmp/qemu-test/src/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: sizeof(*&multifd_recv_state->packet_num) > ATOMIC_REG_SIZE"
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
                                    ^
/tmp/qemu-test/src/include/qemu/compiler.h:98:30: note: in expansion of macro 'QEMU_BUILD_BUG_MSG'
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
                              ^~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/include/qemu/atomic.h:125:5: note: in expansion of macro 'QEMU_BUILD_BUG_ON'
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
     ^~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/ram.c:1219:34: note: in expansion of macro 'atomic_read'
     trace_multifd_recv_sync_main(atomic_read(&multifd_recv_state->packet_num));
                                  ^~~~~~~~~~~
cc1: all warnings being treated as errors
make[1]: *** [/tmp/qemu-test/src/rules.mak:69: migration/ram.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:483: subdir-aarch64-softmmu] Error 2
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 426, in <module>
    sys.exit(main())
  File "./tests/docker/docker.py", line 423, in main
    return args.cmdobj.run(args, argv)
  File "./tests/docker/docker.py", line 262, in run
    return Docker().run(argv, args.keep, quiet=args.quiet)
  File "./tests/docker/docker.py", line 230, in run
    quiet=quiet)
  File "./tests/docker/docker.py", line 148, in _do_check
    return subprocess.check_call(self._command + cmd, **kwargs)
  File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--label', 'com.qemu.instance.uuid=447dcf12747611e884d452540069c830', '-u', '0', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/root/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-d_u5sy7x/src/docker-src.2018-06-20-06.39.46.31712:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:133: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-d_u5sy7x/src'
make: *** [tests/docker/Makefile.include:167: docker-run-test-mingw@fedora] Error 2

real	4m43.923s
user	0m4.469s
sys	0m3.616s
=== OUTPUT END ===

Test command exited with code: 2


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2018-06-20 12:30 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-20  8:15 [Qemu-devel] [PATCH v14 00/12] Multifd Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 01/12] migration: Create multipage support Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 02/12] migration: Create multifd packet Juan Quintela
2018-06-20  9:40   ` Dr. David Alan Gilbert
2018-06-20  9:43     ` Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 03/12] migration: Add multifd traces for start/end thread Juan Quintela
2018-06-20  9:51   ` Dr. David Alan Gilbert
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 04/12] migration: Calculate transferred ram correctly Juan Quintela
2018-06-20  9:58   ` Dr. David Alan Gilbert
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 05/12] migration: Multifd channels always wait on the sem Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 06/12] migration: Add block where to send/receive packets Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 07/12] migration: Synchronize multifd threads with main thread Juan Quintela
2018-06-20 10:30   ` Dr. David Alan Gilbert
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 08/12] migration: Create ram_save_multifd_page Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 09/12] migration: Start sending messages Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 10/12] migration: Wait for blocking IO Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 11/12] migration: Remove not needed semaphore and quit Juan Quintela
2018-06-20  8:15 ` [Qemu-devel] [PATCH v14 12/12] migration: Stop sending whole pages through main channel Juan Quintela
2018-06-20 10:44 ` [Qemu-devel] [PATCH v14 00/12] Multifd no-reply

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.