* [Qemu-devel] [PATCH v13 00/12] Multifd
@ 2018-05-23 11:18 Juan Quintela
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 01/12] migration: Create multipage support Juan Quintela
` (12 more replies)
0 siblings, 13 replies; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
Hi
This is v13 of multifd patches:
- several patches already integrated
- rebased to latests upstreams
- addressed all the reviews comments around.
Please review.
Thanks, Juan.
[v12]
Big news, it is not RFC anymore, it works reliabely for me.
Changes:
- Locknig changed completely (several times)
- We now send all pages through the channels. In a 2GB guest with 1 disk and a network card, the amount of data send for RAM was 80KB.
- This is not optimized yet, but it shouws clear improvements over precopy. testing over localhost networking I can guet:
- 2 VCPUs guest
- 2GB RAM
- runn stress --vm 4 --vm 500GB (i.e. dirtying 2GB or RAM each second)
- Total time: precopy ~50seconds, multifd around 11seconds
- Bandwidth usage is around 273MB/s vs 71MB/s on the same hardware
This is very preleminary testing, will send more numbers when I got them. But looks promissing.
Things that will be improved later:
- Initial synchronization is too slow (around 1s)
- We synchronize all threads after each RAM section, we can move to only
synchronize them after we have done a bitmap syncrhronization
- We can improve bitmap walking (but that is independent of multifd)
Please review.
Later, Juan.
[v11]
Changes on top of previous sumbimission:
- Now on top of migration-tests/v6 that I sent on Wednesday
- Rebased to latest upstream
- Everything that is sent through the network should be converted correctly
(famous last words)
- Still on RFC (sometimes it ends some packets at the end), just to
show how things are going on. Problems are only on the last patch.
- Redo some locking (again) Now the problem is being able te send the
synchronization through the multifd channels. I end the migration
_before_ all the channels have recevied all the packets.
- Trying to get a flags argument into each packet, to be able to synchronze
through the network, not from the "main" incoming corroutine.
- Related to the network-safe fields, now everything is in its own
routine, it should be easier to understand/review. Once there, I
check that all values are inside range.
So, please comment.
Thanks, Juan.
[v10]
Lots of changes from previous versions:
a - everything is sent now through the multifd channels, nothing is sent through main channel
b - locking is band new, I was getting into a hole with the previous approach, right now, there is a single way to
do locking (both source and destination)
main thread : sets a ->sync variable for each thread and wakeps it
multifd threads: clean the variable and signal (sem) back to main thread
using this for either:
- all threads have started
- we need to synchronize after each round through memory
- all threads have finished
c - I have to use a qio watcher for a thread to wait for ready data to read
d - lots of cleanups
e - to make things easier, I have included the missing tests stuff on
this round of patches, because they build on top of them
f - lots of traces, it is now much easier to follow what is happening
Now, why it is an RFC:
- in the last patch, there is still race between the whatcher, the
->quit of the threads and the last synchronization. Techinically they
are done in oder, but in practice, they are hanging sometimes.
- I *know* I can optimize the synchronization of the threads sending
the "we start a new round" through the multifd channels, have to add a flag here.
- Not having a thread on the incoming side is a mess, I can't block waiting for things to happen :-(
- When doing the synchronization, I need to optimize the sending of the "not finished packet" of pages, working on that.
please, take a look and review.
Thanks, Juan.
[v9]
This series is on top of my migration test series just sent, only reject should be on the test system, though.
On v9 series for you:
- qobject_unref() as requested by dan
Yes he was right, I had a reference leak for _non_ multifd, I
*thought* he mean for multifd, and that took a while to understand
(and then find when/where).
- multifd page count: it is dropped for good
- uuid handling: we use the default qemu uuid of 0000...
- uuid handling: using and struct and sending the struct
* idea is to add a size field and add more parameter after that
* anyone has a good idea how to "ouptut" info
migrate_capabilities/parameters json into a string and how to read it back?
- changed how we test that all threads/channels are already created.
Should be more robust.
- Add tests multifd. Still not ported on top of migration-tests series sent early
waiting for review on the ideas there.
- Rebase and remove al the integrated patches (back at 12)
Please, review.
Later, Juan.
[v8]
Things NOT done yet:
- drop x-multifd-page-count? We can use performance to set a default value
- paolo suggestion of not having a control channel
needs iyet more cleanups to be able to have more than one ramstate, trying it.
- still not performance done, but it has been very stable
On v8:
- use connect_async
- rename multifd-group to multifd-page-count (danp suggestion)
- rename multifd-threads to multifd-channels (danp suggestion)
- use new qio*channel functions
- Address rest of comments left
So, please review.
My idea will be to pull this changes and continue performance changes
for inside, basically everything is already reviewed.
Thanks, Juan.
On v7:
- tests fixed as danp wanted
- have to revert danp qio_*_all patches, as they break multifd, I have to investigate why.
- error_abort is gone. After several tries about getting errors, I ended having a single error
proceted by a lock and first error wins.
- Addressed basically all reviews (see on ToDo)
- Pointers to struct are done now
- fix lots of leaks
- lots of small fixes
[v6]
- Improve migration_ioc_porcess_incoming
- teach about G_SOURCE_REMOVE/CONTINUE
- Add test for migration_has_all_channels
- use DEFIN_PROP*
- change recv_state to use pointers to parameters
make easier to receive channels out of order
- use g_strdup_printf()
- improve count of threads to know when we have to finish
- report channel id's on errors
- Use last_page parameter for multifd_send_page() sooner
- Improve commets for address
- use g_new0() instead of g_malloc()
- create MULTIFD_CONTINUE instead of using UINT16_MAX
- clear memory used by group of pages
once there, pass everything to the global state variables instead of being
local to the function. This way it works if we cancel migration and start
a new one
- Really wait to create the migration_thread until all channels are created
- split initial_bytes setup to make clearer following patches.
- createRAM_SAVE_FLAG_MULTIFD_SYNC macro, to make clear what we are doing
- move setting of need_flush to inside bitmap_sync
- Lots of other small changes & reorderings
Please, comment.
[v5]
- tests from qio functions (a.k.a. make danp happy)
- 1st message from one channel to the other contains:
<uuid> multifd <channel number>
This would allow us to create more channels as we want them.
a.k.a. Making dave happy
- Waiting in reception for new channels using qio listeners
Getting threads, qio and reference counters working at the same time
was interesing.
Another make danp happy.
- Lots and lots of small changes and fixes. Notice that the last 70 patches
that I merged or so what to make this series easier/smaller.
- NOT DONE: I haven't been woring on measuring performance
differences, this was about getting the creation of the
threads/channels right.
So, what I want:
- Are people happy with how I have (ab)used qio channels? (yes danp,
that is you).
- My understanding is th
ToDo:
- Make paolo happy: He wanted to test using control information
through each channel, not only pages. This requires yet more
cleanups to be able to have more than one QEMUFile/RAMState open at
the same time.
- How I create multiple channels. Things I know:
* with current changes, it should work with fd/channels (the multifd bits),
but we don;t have a way to pass multiple fd;s or exec files.
Danp, any idea about how to create an UI for it?
* My idea is that we would split current code to be:
+ channel creation at migration.c
+ rest of bits at ram.c
+ change format to:
<uuid> main <rest of migration capabilities/paramentes> so we can check
<uuid> postcopy <no clue what parameters are needed>
Dave wanted a way to create a new fd for postcopy for some time
+ Adding new channels is easy
- Performance data/numbers: Yes, I wanted to get this out at once, I
would continue with this.
Please, review.
[v4]
This is the 4th version of multifd. Changes:
- XBZRLE don't need to be checked for
- Documentation and defaults are consistent
- split socketArgs
- use iovec instead of creating something similar.
- We use now the exported size of target page (another HACK removal)
- created qio_chanel_{wirtev,readv}_all functions. the _full() name
was already taken.
What they do is the same that the without _all() function, but if it
returns due to blocking it redo the call.
- it is checkpatch.pl clean now.
Please comment, Juan.
Juan Quintela (12):
migration: Create multipage support
migration: Create multifd packet
migration: Add multifd traces for start/end thread
migration: Calculate transferred ram correctly
migration: Multifd channels always wait on the sem
migration: Add block where to send/receive packets
migration: Synchronize multifd threads with main thread
migration: Create ram_save_multifd_page
migration: Start sending messages
migration: Wait for blocking IO
migration: Remove not needed semaphore and quit
migration: Stop sending whole pages through main channel
migration/migration.c | 11 +-
migration/ram.c | 462 +++++++++++++++++++++++++++++++++++++++--
migration/trace-events | 12 ++
3 files changed, 469 insertions(+), 16 deletions(-)
--
2.17.0
^ permalink raw reply [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 01/12] migration: Create multipage support
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-06-11 16:58 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 02/12] migration: Create multifd packet Juan Quintela
` (11 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
We only create/destry the page list here. We will use it later.
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/ram.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 57 insertions(+)
diff --git a/migration/ram.c b/migration/ram.c
index 5bcbf7a9f9..23cc5625eb 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -510,6 +510,20 @@ typedef struct {
uint8_t id;
} __attribute__((packed)) MultiFDInit_t;
+typedef struct {
+ /* number of used pages */
+ uint32_t used;
+ /* number of allocated pages */
+ uint32_t allocated;
+ /* global number of generated multifd packets */
+ uint32_t seq;
+ /* offset of each page */
+ ram_addr_t *offset;
+ /* pointer to each page */
+ struct iovec *iov;
+ RAMBlock *block;
+} MultiFDPages_t;
+
typedef struct {
/* this fields are not changed once the thread is created */
/* channel number */
@@ -528,6 +542,8 @@ typedef struct {
bool running;
/* should this thread finish */
bool quit;
+ /* array of pages to sent */
+ MultiFDPages_t *pages;
} MultiFDSendParams;
typedef struct {
@@ -548,6 +564,8 @@ typedef struct {
bool running;
/* should this thread finish */
bool quit;
+ /* array of pages to receive */
+ MultiFDPages_t *pages;
} MultiFDRecvParams;
static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -612,10 +630,36 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp)
return msg.id;
}
+static MultiFDPages_t *multifd_pages_init(size_t size)
+{
+ MultiFDPages_t *pages = g_new0(MultiFDPages_t, 1);
+
+ pages->allocated = size;
+ pages->iov = g_new0(struct iovec, size);
+ pages->offset = g_new0(ram_addr_t, size);
+
+ return pages;
+}
+
+static void multifd_pages_clear(MultiFDPages_t *pages)
+{
+ pages->used = 0;
+ pages->allocated = 0;
+ pages->seq = 0;
+ pages->block = NULL;
+ g_free(pages->iov);
+ pages->iov = NULL;
+ g_free(pages->offset);
+ pages->offset = NULL;
+ g_free(pages);
+}
+
struct {
MultiFDSendParams *params;
/* number of created threads */
int count;
+ /* array of pages to sent */
+ MultiFDPages_t *pages;
} *multifd_send_state;
static void multifd_send_terminate_threads(Error *err)
@@ -665,9 +709,13 @@ int multifd_save_cleanup(Error **errp)
qemu_sem_destroy(&p->sem);
g_free(p->name);
p->name = NULL;
+ multifd_pages_clear(p->pages);
+ p->pages = NULL;
}
g_free(multifd_send_state->params);
multifd_send_state->params = NULL;
+ multifd_pages_clear(multifd_send_state->pages);
+ multifd_send_state->pages = NULL;
g_free(multifd_send_state);
multifd_send_state = NULL;
return ret;
@@ -728,6 +776,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
int multifd_save_setup(void)
{
int thread_count;
+ uint32_t page_count = migrate_multifd_page_count();
uint8_t i;
if (!migrate_use_multifd()) {
@@ -737,6 +786,8 @@ int multifd_save_setup(void)
multifd_send_state = g_malloc0(sizeof(*multifd_send_state));
multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
atomic_set(&multifd_send_state->count, 0);
+ multifd_send_state->pages = multifd_pages_init(page_count);
+
for (i = 0; i < thread_count; i++) {
MultiFDSendParams *p = &multifd_send_state->params[i];
@@ -744,6 +795,7 @@ int multifd_save_setup(void)
qemu_sem_init(&p->sem, 0);
p->quit = false;
p->id = i;
+ p->pages = multifd_pages_init(page_count);
p->name = g_strdup_printf("multifdsend_%d", i);
socket_send_channel_create(multifd_new_send_channel_async, p);
}
@@ -801,6 +853,8 @@ int multifd_load_cleanup(Error **errp)
qemu_sem_destroy(&p->sem);
g_free(p->name);
p->name = NULL;
+ multifd_pages_clear(p->pages);
+ p->pages = NULL;
}
g_free(multifd_recv_state->params);
multifd_recv_state->params = NULL;
@@ -834,6 +888,7 @@ static void *multifd_recv_thread(void *opaque)
int multifd_load_setup(void)
{
int thread_count;
+ uint32_t page_count = migrate_multifd_page_count();
uint8_t i;
if (!migrate_use_multifd()) {
@@ -843,6 +898,7 @@ int multifd_load_setup(void)
multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state));
multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count);
atomic_set(&multifd_recv_state->count, 0);
+
for (i = 0; i < thread_count; i++) {
MultiFDRecvParams *p = &multifd_recv_state->params[i];
@@ -850,6 +906,7 @@ int multifd_load_setup(void)
qemu_sem_init(&p->sem, 0);
p->quit = false;
p->id = i;
+ p->pages = multifd_pages_init(page_count);
p->name = g_strdup_printf("multifdrecv_%d", i);
}
return 0;
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 02/12] migration: Create multifd packet
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 01/12] migration: Create multipage support Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-06-11 11:10 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 03/12] migration: Add multifd traces for start/end thread Juan Quintela
` (10 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
We still don't put anything there.
Signed-off-by: Juan Quintela <quintela@redhat.com>
--
fix magic (dave)
check offset/ramblock (dave)
---
migration/ram.c | 145 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 144 insertions(+), 1 deletion(-)
diff --git a/migration/ram.c b/migration/ram.c
index 23cc5625eb..54350db8b0 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -510,6 +510,17 @@ typedef struct {
uint8_t id;
} __attribute__((packed)) MultiFDInit_t;
+typedef struct {
+ uint32_t magic;
+ uint32_t version;
+ uint32_t flags;
+ uint32_t size;
+ uint32_t used;
+ uint32_t seq;
+ char ramblock[256];
+ uint64_t offset[];
+} __attribute__((packed)) MultiFDPacket_t;
+
typedef struct {
/* number of used pages */
uint32_t used;
@@ -544,6 +555,14 @@ typedef struct {
bool quit;
/* array of pages to sent */
MultiFDPages_t *pages;
+ /* packet allocated len */
+ uint32_t packet_len;
+ /* pointer to the packet */
+ MultiFDPacket_t *packet;
+ /* multifd flags for each packet */
+ uint32_t flags;
+ /* global number of generated multifd packets */
+ uint32_t seq;
} MultiFDSendParams;
typedef struct {
@@ -566,6 +585,14 @@ typedef struct {
bool quit;
/* array of pages to receive */
MultiFDPages_t *pages;
+ /* packet allocated len */
+ uint32_t packet_len;
+ /* pointer to the packet */
+ MultiFDPacket_t *packet;
+ /* multifd flags for each packet */
+ uint32_t flags;
+ /* global number of generated multifd packets */
+ uint32_t seq;
} MultiFDRecvParams;
static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -654,6 +681,99 @@ static void multifd_pages_clear(MultiFDPages_t *pages)
g_free(pages);
}
+static void multifd_send_fill_packet(MultiFDSendParams *p)
+{
+ MultiFDPacket_t *packet = p->packet;
+ int i;
+
+ packet->magic = cpu_to_be32(MULTIFD_MAGIC);
+ packet->version = cpu_to_be32(MULTIFD_VERSION);
+ packet->flags = cpu_to_be32(p->flags);
+ packet->size = cpu_to_be32(migrate_multifd_page_count());
+ packet->used = cpu_to_be32(p->pages->used);
+ packet->seq = cpu_to_be32(p->seq);
+
+ if (p->pages->block) {
+ strncpy(packet->ramblock, p->pages->block->idstr, 256);
+ }
+
+ for (i = 0; i < p->pages->used; i++) {
+ packet->offset[i] = cpu_to_be64(p->pages->offset[i]);
+ }
+}
+
+static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp)
+{
+ MultiFDPacket_t *packet = p->packet;
+ RAMBlock *block;
+ int i;
+
+ /* ToDo: We can't use it until we haven't received a message */
+ return 0;
+
+ be32_to_cpus(&packet->magic);
+ if (packet->magic != MULTIFD_MAGIC) {
+ error_setg(errp, "multifd: received packet "
+ "magic %x and expected magic %x",
+ packet->magic, MULTIFD_MAGIC);
+ return -1;
+ }
+
+ be32_to_cpus(&packet->version);
+ if (packet->version != MULTIFD_VERSION) {
+ error_setg(errp, "multifd: received packet "
+ "version %d and expected version %d",
+ packet->version, MULTIFD_VERSION);
+ return -1;
+ }
+
+ p->flags = be32_to_cpu(packet->flags);
+
+ be32_to_cpus(&packet->size);
+ if (packet->size > migrate_multifd_page_count()) {
+ error_setg(errp, "multifd: received packet "
+ "with size %d and expected maximum size %d",
+ packet->size, migrate_multifd_page_count()) ;
+ return -1;
+ }
+
+ p->pages->used = be32_to_cpu(packet->used);
+ if (p->pages->used > packet->size) {
+ error_setg(errp, "multifd: received packet "
+ "with size %d and expected maximum size %d",
+ p->pages->used, packet->size) ;
+ return -1;
+ }
+
+ p->seq = be32_to_cpu(packet->seq);
+
+ if (p->pages->used) {
+ /* make sure that ramblock is 0 terminated */
+ packet->ramblock[255] = 0;
+ block = qemu_ram_block_by_name(packet->ramblock);
+ if (!block) {
+ error_setg(errp, "multifd: unknown ram block %s",
+ packet->ramblock);
+ return -1;
+ }
+ }
+
+ for (i = 0; i < p->pages->used; i++) {
+ ram_addr_t offset = be64_to_cpu(packet->offset[i]);
+
+ if (offset > (block->used_length - TARGET_PAGE_SIZE)) {
+ error_setg(errp, "multifd: offset too long %" PRId64
+ " (max %" PRId64 ")",
+ offset, block->max_length);
+ return -1;
+ }
+ p->pages->iov[i].iov_base = block->host + offset;
+ p->pages->iov[i].iov_len = TARGET_PAGE_SIZE;
+ }
+
+ return 0;
+}
+
struct {
MultiFDSendParams *params;
/* number of created threads */
@@ -711,6 +831,9 @@ int multifd_save_cleanup(Error **errp)
p->name = NULL;
multifd_pages_clear(p->pages);
p->pages = NULL;
+ p->packet_len = 0;
+ g_free(p->packet);
+ p->packet = NULL;
}
g_free(multifd_send_state->params);
multifd_send_state->params = NULL;
@@ -732,6 +855,7 @@ static void *multifd_send_thread(void *opaque)
while (true) {
qemu_mutex_lock(&p->mutex);
+ multifd_send_fill_packet(p);
if (p->quit) {
qemu_mutex_unlock(&p->mutex);
break;
@@ -796,6 +920,9 @@ int multifd_save_setup(void)
p->quit = false;
p->id = i;
p->pages = multifd_pages_init(page_count);
+ p->packet_len = sizeof(MultiFDPacket_t)
+ + sizeof(ram_addr_t) * page_count;
+ p->packet = g_malloc0(p->packet_len);
p->name = g_strdup_printf("multifdsend_%d", i);
socket_send_channel_create(multifd_new_send_channel_async, p);
}
@@ -855,6 +982,9 @@ int multifd_load_cleanup(Error **errp)
p->name = NULL;
multifd_pages_clear(p->pages);
p->pages = NULL;
+ p->packet_len = 0;
+ g_free(p->packet);
+ p->packet = NULL;
}
g_free(multifd_recv_state->params);
multifd_recv_state->params = NULL;
@@ -867,10 +997,20 @@ int multifd_load_cleanup(Error **errp)
static void *multifd_recv_thread(void *opaque)
{
MultiFDRecvParams *p = opaque;
+ Error *local_err = NULL;
+ int ret;
while (true) {
qemu_mutex_lock(&p->mutex);
- if (p->quit) {
+ if (false) {
+ /* ToDo: Packet reception goes here */
+
+ ret = multifd_recv_unfill_packet(p, &local_err);
+ qemu_mutex_unlock(&p->mutex);
+ if (ret) {
+ break;
+ }
+ } else if (p->quit) {
qemu_mutex_unlock(&p->mutex);
break;
}
@@ -907,6 +1047,9 @@ int multifd_load_setup(void)
p->quit = false;
p->id = i;
p->pages = multifd_pages_init(page_count);
+ p->packet_len = sizeof(MultiFDPacket_t)
+ + sizeof(ram_addr_t) * page_count;
+ p->packet = g_malloc0(p->packet_len);
p->name = g_strdup_printf("multifdrecv_%d", i);
}
return 0;
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 03/12] migration: Add multifd traces for start/end thread
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 01/12] migration: Create multipage support Juan Quintela
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 02/12] migration: Create multifd packet Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-05-31 16:11 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 04/12] migration: Calculate transferred ram correctly Juan Quintela
` (9 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
We want to know how many pages/packets each channel has sent. Add
counters for those.
Signed-off-by: Juan Quintela <quintela@redhat.com>
--
sort trace-events (dave)
---
migration/ram.c | 22 ++++++++++++++++++++++
migration/trace-events | 4 ++++
2 files changed, 26 insertions(+)
diff --git a/migration/ram.c b/migration/ram.c
index 54350db8b0..ab715cc148 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -563,6 +563,11 @@ typedef struct {
uint32_t flags;
/* global number of generated multifd packets */
uint32_t seq;
+ /* thread local variables */
+ /* packets sent through this channel */
+ uint32_t num_packets;
+ /* pages sent through this channel */
+ uint32_t num_pages;
} MultiFDSendParams;
typedef struct {
@@ -593,6 +598,11 @@ typedef struct {
uint32_t flags;
/* global number of generated multifd packets */
uint32_t seq;
+ /* thread local variables */
+ /* packets sent through this channel */
+ uint32_t num_packets;
+ /* pages sent through this channel */
+ uint32_t num_pages;
} MultiFDRecvParams;
static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -849,9 +859,13 @@ static void *multifd_send_thread(void *opaque)
MultiFDSendParams *p = opaque;
Error *local_err = NULL;
+ trace_multifd_send_thread_start(p->id);
+
if (multifd_send_initial_packet(p, &local_err) < 0) {
goto out;
}
+ /* initial packet */
+ p->num_packets = 1;
while (true) {
qemu_mutex_lock(&p->mutex);
@@ -873,6 +887,8 @@ out:
p->running = false;
qemu_mutex_unlock(&p->mutex);
+ trace_multifd_send_thread_end(p->id, p->num_packets, p->num_pages);
+
return NULL;
}
@@ -1000,6 +1016,8 @@ static void *multifd_recv_thread(void *opaque)
Error *local_err = NULL;
int ret;
+ trace_multifd_recv_thread_start(p->id);
+
while (true) {
qemu_mutex_lock(&p->mutex);
if (false) {
@@ -1022,6 +1040,8 @@ static void *multifd_recv_thread(void *opaque)
p->running = false;
qemu_mutex_unlock(&p->mutex);
+ trace_multifd_recv_thread_end(p->id, p->num_packets, p->num_pages);
+
return NULL;
}
@@ -1087,6 +1107,8 @@ void multifd_recv_new_channel(QIOChannel *ioc)
}
p->c = ioc;
object_ref(OBJECT(ioc));
+ /* initial packet */
+ p->num_packets = 1;
p->running = true;
qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p,
diff --git a/migration/trace-events b/migration/trace-events
index 3c798ddd11..3c7a9a8332 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -76,6 +76,10 @@ get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned
migration_bitmap_sync_start(void) ""
migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
migration_throttle(void) ""
+multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
+multifd_recv_thread_start(uint8_t id) "%d"
+multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
+multifd_send_thread_start(uint8_t id) "%d"
ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%s: addr: 0x%" PRIx64 " flags: 0x%x host: %p"
ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x"
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 04/12] migration: Calculate transferred ram correctly
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (2 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 03/12] migration: Add multifd traces for start/end thread Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-05-31 17:14 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 05/12] migration: Multifd channels always wait on the sem Juan Quintela
` (8 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
On multifd we send data from more places that main channel.
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/migration.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/migration/migration.c b/migration/migration.c
index 05aec2c905..55d7b64e24 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2679,12 +2679,19 @@ static void migration_update_counters(MigrationState *s,
{
uint64_t transferred, time_spent;
double bandwidth;
+ uint64_t now;
if (current_time < s->iteration_start_time + BUFFER_DELAY) {
return;
}
- transferred = qemu_ftell(s->to_dst_file) - s->iteration_initial_bytes;
+ if (migrate_use_multifd()) {
+ now = ram_counters.normal * qemu_target_page_size()
+ + qemu_ftell(s->to_dst_file);
+ } else {
+ now = qemu_ftell(s->to_dst_file);
+ }
+ transferred = now - s->iteration_initial_bytes;
time_spent = current_time - s->iteration_start_time;
bandwidth = (double)transferred / time_spent;
s->threshold_size = bandwidth * s->parameters.downtime_limit;
@@ -2704,7 +2711,7 @@ static void migration_update_counters(MigrationState *s,
qemu_file_reset_rate_limit(s->to_dst_file);
s->iteration_start_time = current_time;
- s->iteration_initial_bytes = qemu_ftell(s->to_dst_file);
+ s->iteration_initial_bytes = now;
trace_migrate_transferred(transferred, time_spent,
bandwidth, s->threshold_size);
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 05/12] migration: Multifd channels always wait on the sem
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (3 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 04/12] migration: Calculate transferred ram correctly Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-06-11 17:13 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 06/12] migration: Add block where to send/receive packets Juan Quintela
` (7 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
Either for quit, sync or packet, we first wake them.
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/ram.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index ab715cc148..90f62144e2 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -868,6 +868,7 @@ static void *multifd_send_thread(void *opaque)
p->num_packets = 1;
while (true) {
+ qemu_sem_wait(&p->sem);
qemu_mutex_lock(&p->mutex);
multifd_send_fill_packet(p);
if (p->quit) {
@@ -875,7 +876,9 @@ static void *multifd_send_thread(void *opaque)
break;
}
qemu_mutex_unlock(&p->mutex);
- qemu_sem_wait(&p->sem);
+ /* this is impossible */
+ error_setg(&local_err, "multifd_send_thread: Unknown command");
+ break;
}
out:
@@ -1019,6 +1022,7 @@ static void *multifd_recv_thread(void *opaque)
trace_multifd_recv_thread_start(p->id);
while (true) {
+ qemu_sem_wait(&p->sem);
qemu_mutex_lock(&p->mutex);
if (false) {
/* ToDo: Packet reception goes here */
@@ -1033,9 +1037,14 @@ static void *multifd_recv_thread(void *opaque)
break;
}
qemu_mutex_unlock(&p->mutex);
- qemu_sem_wait(&p->sem);
+ /* this is impossible */
+ error_setg(&local_err, "multifd_recv_thread: Unknown command");
+ break;
}
+ if (local_err) {
+ multifd_recv_terminate_threads(local_err);
+ }
qemu_mutex_lock(&p->mutex);
p->running = false;
qemu_mutex_unlock(&p->mutex);
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 06/12] migration: Add block where to send/receive packets
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (4 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 05/12] migration: Multifd channels always wait on the sem Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 07/12] migration: Synchronize multifd threads with main thread Juan Quintela
` (6 subsequent siblings)
12 siblings, 0 replies; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
Once there add tracepoints.
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/ram.c | 49 +++++++++++++++++++++++++++++++++++++-----
migration/trace-events | 2 ++
2 files changed, 46 insertions(+), 5 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 90f62144e2..c9a9bd79f3 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -553,6 +553,8 @@ typedef struct {
bool running;
/* should this thread finish */
bool quit;
+ /* thread has work to do */
+ int pending_job;
/* array of pages to sent */
MultiFDPages_t *pages;
/* packet allocated len */
@@ -588,6 +590,8 @@ typedef struct {
bool running;
/* should this thread finish */
bool quit;
+ /* thread has work to do */
+ bool pending_job;
/* array of pages to receive */
MultiFDPages_t *pages;
/* packet allocated len */
@@ -870,8 +874,28 @@ static void *multifd_send_thread(void *opaque)
while (true) {
qemu_sem_wait(&p->sem);
qemu_mutex_lock(&p->mutex);
- multifd_send_fill_packet(p);
- if (p->quit) {
+
+ if (p->pending_job) {
+ uint32_t used = p->pages->used;
+ uint32_t seq = p->seq;
+ uint32_t flags = p->flags;
+
+ multifd_send_fill_packet(p);
+ p->flags = 0;
+ p->num_packets++;
+ p->num_pages += used;
+ p->pages->used = 0;
+ qemu_mutex_unlock(&p->mutex);
+
+ trace_multifd_send(p->id, seq, used, flags);
+
+ /* ToDo: send packet here */
+
+ qemu_mutex_lock(&p->mutex);
+ p->pending_job--;
+ qemu_mutex_unlock(&p->mutex);
+ continue;
+ } else if (p->quit) {
qemu_mutex_unlock(&p->mutex);
break;
}
@@ -937,6 +961,7 @@ int multifd_save_setup(void)
qemu_mutex_init(&p->mutex);
qemu_sem_init(&p->sem, 0);
p->quit = false;
+ p->pending_job = 0;
p->id = i;
p->pages = multifd_pages_init(page_count);
p->packet_len = sizeof(MultiFDPacket_t)
@@ -1024,14 +1049,27 @@ static void *multifd_recv_thread(void *opaque)
while (true) {
qemu_sem_wait(&p->sem);
qemu_mutex_lock(&p->mutex);
- if (false) {
- /* ToDo: Packet reception goes here */
+ if (p->pending_job) {
+ uint32_t used;
+ uint32_t flags;
+ qemu_mutex_unlock(&p->mutex);
+ /* ToDo: recv packet here */
+
+ qemu_mutex_lock(&p->mutex);
ret = multifd_recv_unfill_packet(p, &local_err);
- qemu_mutex_unlock(&p->mutex);
if (ret) {
+ qemu_mutex_unlock(&p->mutex);
break;
}
+
+ used = p->pages->used;
+ flags = p->flags;
+ trace_multifd_recv(p->id, p->seq, used, flags);
+ p->pending_job = false;
+ p->num_packets++;
+ p->num_pages += used;
+ qemu_mutex_unlock(&p->mutex);
} else if (p->quit) {
qemu_mutex_unlock(&p->mutex);
break;
@@ -1074,6 +1112,7 @@ int multifd_load_setup(void)
qemu_mutex_init(&p->mutex);
qemu_sem_init(&p->sem, 0);
p->quit = false;
+ p->pending_job = false;
p->id = i;
p->pages = multifd_pages_init(page_count);
p->packet_len = sizeof(MultiFDPacket_t)
diff --git a/migration/trace-events b/migration/trace-events
index 3c7a9a8332..36e20b312d 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -76,8 +76,10 @@ get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned
migration_bitmap_sync_start(void) ""
migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
migration_throttle(void) ""
+multifd_recv(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "channel %d seq number %d pages %d flags 0x%x"
multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
multifd_recv_thread_start(uint8_t id) "%d"
+multifd_send(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "channel %d seq number %d pages %d flags 0x%x"
multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
multifd_send_thread_start(uint8_t id) "%d"
ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 07/12] migration: Synchronize multifd threads with main thread
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (5 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 06/12] migration: Add block where to send/receive packets Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-06-11 11:53 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 08/12] migration: Create ram_save_multifd_page Juan Quintela
` (5 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
We synchronize all threads each RAM_SAVE_FLAG_EOS. Bitmap
synchronizations don't happen inside a ram section, so we are safe
about two channels trying to overwrite the same memory.
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/ram.c | 117 +++++++++++++++++++++++++++++++++++++----
migration/trace-events | 6 +++
2 files changed, 112 insertions(+), 11 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index c9a9bd79f3..3e99d48123 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -503,6 +503,8 @@ exit:
#define MULTIFD_MAGIC 0x11223344U
#define MULTIFD_VERSION 1
+#define MULTIFD_FLAG_SYNC (1 << 0)
+
typedef struct {
uint32_t magic;
uint32_t version;
@@ -570,6 +572,8 @@ typedef struct {
uint32_t num_packets;
/* pages sent through this channel */
uint32_t num_pages;
+ /* syncs main thread and channels */
+ QemuSemaphore sem_sync;
} MultiFDSendParams;
typedef struct {
@@ -607,6 +611,8 @@ typedef struct {
uint32_t num_packets;
/* pages sent through this channel */
uint32_t num_pages;
+ /* syncs main thread and channels */
+ QemuSemaphore sem_sync;
} MultiFDRecvParams;
static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
@@ -794,6 +800,10 @@ struct {
int count;
/* array of pages to sent */
MultiFDPages_t *pages;
+ /* syncs main thread and channels */
+ QemuSemaphore sem_sync;
+ /* global number of generated multifd packets */
+ uint32_t seq;
} *multifd_send_state;
static void multifd_send_terminate_threads(Error *err)
@@ -841,6 +851,7 @@ int multifd_save_cleanup(Error **errp)
p->c = NULL;
qemu_mutex_destroy(&p->mutex);
qemu_sem_destroy(&p->sem);
+ qemu_sem_destroy(&p->sem_sync);
g_free(p->name);
p->name = NULL;
multifd_pages_clear(p->pages);
@@ -849,6 +860,7 @@ int multifd_save_cleanup(Error **errp)
g_free(p->packet);
p->packet = NULL;
}
+ qemu_sem_destroy(&multifd_send_state->sem_sync);
g_free(multifd_send_state->params);
multifd_send_state->params = NULL;
multifd_pages_clear(multifd_send_state->pages);
@@ -858,6 +870,33 @@ int multifd_save_cleanup(Error **errp)
return ret;
}
+static void multifd_send_sync_main(void)
+{
+ int i;
+
+ if (!migrate_use_multifd()) {
+ return;
+ }
+ for (i = 0; i < migrate_multifd_channels(); i++) {
+ MultiFDSendParams *p = &multifd_send_state->params[i];
+
+ trace_multifd_send_sync_main_signal(p->id);
+
+ qemu_mutex_lock(&p->mutex);
+ p->flags |= MULTIFD_FLAG_SYNC;
+ p->pending_job++;
+ qemu_mutex_unlock(&p->mutex);
+ qemu_sem_post(&p->sem);
+ }
+ for (i = 0; i < migrate_multifd_channels(); i++) {
+ MultiFDSendParams *p = &multifd_send_state->params[i];
+
+ trace_multifd_send_sync_main_wait(p->id);
+ qemu_sem_wait(&multifd_send_state->sem_sync);
+ }
+ trace_multifd_send_sync_main(multifd_send_state->seq);
+}
+
static void *multifd_send_thread(void *opaque)
{
MultiFDSendParams *p = opaque;
@@ -894,15 +933,17 @@ static void *multifd_send_thread(void *opaque)
qemu_mutex_lock(&p->mutex);
p->pending_job--;
qemu_mutex_unlock(&p->mutex);
- continue;
+
+ if (flags & MULTIFD_FLAG_SYNC) {
+ qemu_sem_post(&multifd_send_state->sem_sync);
+ }
} else if (p->quit) {
qemu_mutex_unlock(&p->mutex);
break;
+ } else {
+ qemu_mutex_unlock(&p->mutex);
+ /* sometimes there are spurious wakeups */
}
- qemu_mutex_unlock(&p->mutex);
- /* this is impossible */
- error_setg(&local_err, "multifd_send_thread: Unknown command");
- break;
}
out:
@@ -954,12 +995,14 @@ int multifd_save_setup(void)
multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
atomic_set(&multifd_send_state->count, 0);
multifd_send_state->pages = multifd_pages_init(page_count);
+ qemu_sem_init(&multifd_send_state->sem_sync, 0);
for (i = 0; i < thread_count; i++) {
MultiFDSendParams *p = &multifd_send_state->params[i];
qemu_mutex_init(&p->mutex);
qemu_sem_init(&p->sem, 0);
+ qemu_sem_init(&p->sem_sync, 0);
p->quit = false;
p->pending_job = 0;
p->id = i;
@@ -977,6 +1020,10 @@ struct {
MultiFDRecvParams *params;
/* number of created threads */
int count;
+ /* syncs main thread and channels */
+ QemuSemaphore sem_sync;
+ /* global number of generated multifd packets */
+ uint32_t seq;
} *multifd_recv_state;
static void multifd_recv_terminate_threads(Error *err)
@@ -1022,6 +1069,7 @@ int multifd_load_cleanup(Error **errp)
p->c = NULL;
qemu_mutex_destroy(&p->mutex);
qemu_sem_destroy(&p->sem);
+ qemu_sem_destroy(&p->sem_sync);
g_free(p->name);
p->name = NULL;
multifd_pages_clear(p->pages);
@@ -1030,6 +1078,7 @@ int multifd_load_cleanup(Error **errp)
g_free(p->packet);
p->packet = NULL;
}
+ qemu_sem_destroy(&multifd_recv_state->sem_sync);
g_free(multifd_recv_state->params);
multifd_recv_state->params = NULL;
g_free(multifd_recv_state);
@@ -1038,6 +1087,42 @@ int multifd_load_cleanup(Error **errp)
return ret;
}
+static void multifd_recv_sync_main(void)
+{
+ int i;
+
+ if (!migrate_use_multifd()) {
+ return;
+ }
+ for (i = 0; i < migrate_multifd_channels(); i++) {
+ MultiFDRecvParams *p = &multifd_recv_state->params[i];
+
+ trace_multifd_recv_sync_main_signal(p->id);
+ qemu_mutex_lock(&p->mutex);
+ p->pending_job = true;
+ qemu_mutex_unlock(&p->mutex);
+ }
+ for (i = 0; i < migrate_multifd_channels(); i++) {
+ MultiFDRecvParams *p = &multifd_recv_state->params[i];
+
+ trace_multifd_recv_sync_main_wait(p->id);
+ qemu_sem_wait(&multifd_recv_state->sem_sync);
+ qemu_mutex_lock(&p->mutex);
+ if (multifd_recv_state->seq < p->seq) {
+ multifd_recv_state->seq = p->seq;
+ }
+ qemu_mutex_unlock(&p->mutex);
+ }
+ for (i = 0; i < migrate_multifd_channels(); i++) {
+ MultiFDRecvParams *p = &multifd_recv_state->params[i];
+
+ trace_multifd_recv_sync_main_signal(p->id);
+
+ qemu_sem_post(&p->sem_sync);
+ }
+ trace_multifd_recv_sync_main(multifd_recv_state->seq);
+}
+
static void *multifd_recv_thread(void *opaque)
{
MultiFDRecvParams *p = opaque;
@@ -1047,9 +1132,8 @@ static void *multifd_recv_thread(void *opaque)
trace_multifd_recv_thread_start(p->id);
while (true) {
- qemu_sem_wait(&p->sem);
qemu_mutex_lock(&p->mutex);
- if (p->pending_job) {
+ if (true || p->pending_job) {
uint32_t used;
uint32_t flags;
qemu_mutex_unlock(&p->mutex);
@@ -1070,14 +1154,18 @@ static void *multifd_recv_thread(void *opaque)
p->num_packets++;
p->num_pages += used;
qemu_mutex_unlock(&p->mutex);
+
+ if (flags & MULTIFD_FLAG_SYNC) {
+ qemu_sem_post(&multifd_recv_state->sem_sync);
+ qemu_sem_wait(&p->sem_sync);
+ }
} else if (p->quit) {
qemu_mutex_unlock(&p->mutex);
break;
+ } else {
+ qemu_mutex_unlock(&p->mutex);
+ /* sometimes there are spurious wakeups */
}
- qemu_mutex_unlock(&p->mutex);
- /* this is impossible */
- error_setg(&local_err, "multifd_recv_thread: Unknown command");
- break;
}
if (local_err) {
@@ -1105,12 +1193,14 @@ int multifd_load_setup(void)
multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state));
multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count);
atomic_set(&multifd_recv_state->count, 0);
+ qemu_sem_init(&multifd_recv_state->sem_sync, 0);
for (i = 0; i < thread_count; i++) {
MultiFDRecvParams *p = &multifd_recv_state->params[i];
qemu_mutex_init(&p->mutex);
qemu_sem_init(&p->sem, 0);
+ qemu_sem_init(&p->sem_sync, 0);
p->quit = false;
p->pending_job = false;
p->id = i;
@@ -2847,6 +2937,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
ram_control_before_iterate(f, RAM_CONTROL_SETUP);
ram_control_after_iterate(f, RAM_CONTROL_SETUP);
+ multifd_send_sync_main();
qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
return 0;
@@ -2922,6 +3013,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
*/
ram_control_after_iterate(f, RAM_CONTROL_ROUND);
+ multifd_send_sync_main();
out:
qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
ram_counters.transferred += 8;
@@ -2975,6 +3067,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
rcu_read_unlock();
+ multifd_send_sync_main();
qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
return 0;
@@ -3459,6 +3552,7 @@ static int ram_load_postcopy(QEMUFile *f)
break;
case RAM_SAVE_FLAG_EOS:
/* normal exit */
+ multifd_recv_sync_main();
break;
default:
error_report("Unknown combination of migration flags: %#x"
@@ -3644,6 +3738,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
break;
case RAM_SAVE_FLAG_EOS:
/* normal exit */
+ multifd_recv_sync_main();
break;
default:
if (flags & RAM_SAVE_FLAG_HOOK) {
diff --git a/migration/trace-events b/migration/trace-events
index 36e20b312d..b821041281 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -77,9 +77,15 @@ migration_bitmap_sync_start(void) ""
migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
migration_throttle(void) ""
multifd_recv(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "channel %d seq number %d pages %d flags 0x%x"
+multifd_recv_sync_main(uint32_t seq) "seq %d"
+multifd_recv_sync_main_signal(uint8_t id) "channel %d"
+multifd_recv_sync_main_wait(uint8_t id) "channel %d"
multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
multifd_recv_thread_start(uint8_t id) "%d"
multifd_send(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "channel %d seq number %d pages %d flags 0x%x"
+multifd_send_sync_main(uint32_t seq) "seq %d"
+multifd_send_sync_main_signal(uint8_t id) "channel %d"
+multifd_send_sync_main_wait(uint8_t id) "channel %d"
multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
multifd_send_thread_start(uint8_t id) "%d"
ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 08/12] migration: Create ram_save_multifd_page
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (6 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 07/12] migration: Synchronize multifd threads with main thread Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-06-11 17:33 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 09/12] migration: Start sending messages Juan Quintela
` (4 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
The function still don't use multifd, but we have simplified
ram_save_page, xbzrle and RDMA stuff is gone. We have added a new
counter.
Signed-off-by: Juan Quintela <quintela@redhat.com>
--
Add last_page parameter
Add commets for done and address
Remove multifd field, it is the same than normal pages
Merge next patch, now we send multiple pages at a time
Remove counter for multifd pages, it is identical to normal pages
Use iovec's instead of creating the equivalent.
Clear memory used by pages (dave)
Use g_new0(danp)
define MULTIFD_CONTINUE
now pages member is a pointer
Fix off-by-one in number of pages in one packet
Remove RAM_SAVE_FLAG_MULTIFD_PAGE
s/multifd_pages_t/MultiFDPages_t/
---
migration/ram.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 85 insertions(+)
diff --git a/migration/ram.c b/migration/ram.c
index 3e99d48123..28f5cea4d8 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -55,6 +55,7 @@
#include "sysemu/sysemu.h"
#include "qemu/uuid.h"
#include "savevm.h"
+#include "qemu/iov.h"
/***********************************************************/
/* ram save/restore */
@@ -804,8 +805,65 @@ struct {
QemuSemaphore sem_sync;
/* global number of generated multifd packets */
uint32_t seq;
+ /* send channels ready */
+ QemuSemaphore channels_ready;
} *multifd_send_state;
+static void multifd_send_pages(void)
+{
+ int i;
+ static int next_channel;
+ MultiFDSendParams *p = NULL; /* make happy gcc */
+ MultiFDPages_t *pages = multifd_send_state->pages;
+
+ qemu_sem_wait(&multifd_send_state->channels_ready);
+ for (i = next_channel;; i = (i + 1) % migrate_multifd_channels()) {
+ p = &multifd_send_state->params[i];
+
+ qemu_mutex_lock(&p->mutex);
+ if (!p->pending_job) {
+ p->pending_job++;
+ next_channel = (i + 1) % migrate_multifd_channels();
+ break;
+ }
+ qemu_mutex_unlock(&p->mutex);
+ }
+ p->pages->used = 0;
+ multifd_send_state->seq++;
+ p->seq = multifd_send_state->seq;
+ p->pages->block = NULL;
+ multifd_send_state->pages = p->pages;
+ p->pages = pages;
+ qemu_mutex_unlock(&p->mutex);
+ qemu_sem_post(&p->sem);
+}
+
+static void multifd_queue_page(RAMBlock *block, ram_addr_t offset)
+{
+ MultiFDPages_t *pages = multifd_send_state->pages;
+
+ if (!pages->block) {
+ pages->block = block;
+ }
+
+ if (pages->block == block) {
+ pages->offset[pages->used] = offset;
+ pages->iov[pages->used].iov_base = block->host + offset;
+ pages->iov[pages->used].iov_len = TARGET_PAGE_SIZE;
+ pages->used++;
+
+ if (pages->used < pages->allocated) {
+ return;
+ }
+ }
+
+ multifd_send_pages();
+
+ if (pages->block != block) {
+ multifd_queue_page(block, offset);
+ }
+}
+
static void multifd_send_terminate_threads(Error *err)
{
int i;
@@ -860,6 +918,7 @@ int multifd_save_cleanup(Error **errp)
g_free(p->packet);
p->packet = NULL;
}
+ qemu_sem_destroy(&multifd_send_state->channels_ready);
qemu_sem_destroy(&multifd_send_state->sem_sync);
g_free(multifd_send_state->params);
multifd_send_state->params = NULL;
@@ -877,12 +936,17 @@ static void multifd_send_sync_main(void)
if (!migrate_use_multifd()) {
return;
}
+ if (multifd_send_state->pages->used) {
+ multifd_send_pages();
+ }
for (i = 0; i < migrate_multifd_channels(); i++) {
MultiFDSendParams *p = &multifd_send_state->params[i];
trace_multifd_send_sync_main_signal(p->id);
qemu_mutex_lock(&p->mutex);
+ multifd_send_state->seq++;
+ p->seq = multifd_send_state->seq;
p->flags |= MULTIFD_FLAG_SYNC;
p->pending_job++;
qemu_mutex_unlock(&p->mutex);
@@ -937,6 +1001,7 @@ static void *multifd_send_thread(void *opaque)
if (flags & MULTIFD_FLAG_SYNC) {
qemu_sem_post(&multifd_send_state->sem_sync);
}
+ qemu_sem_post(&multifd_send_state->channels_ready);
} else if (p->quit) {
qemu_mutex_unlock(&p->mutex);
break;
@@ -996,6 +1061,7 @@ int multifd_save_setup(void)
atomic_set(&multifd_send_state->count, 0);
multifd_send_state->pages = multifd_pages_init(page_count);
qemu_sem_init(&multifd_send_state->sem_sync, 0);
+ qemu_sem_init(&multifd_send_state->channels_ready, 0);
for (i = 0; i < thread_count; i++) {
MultiFDSendParams *p = &multifd_send_state->params[i];
@@ -1713,6 +1779,23 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage)
return pages;
}
+static int ram_save_multifd_page(RAMState *rs, RAMBlock *block,
+ ram_addr_t offset)
+{
+ uint8_t *p;
+
+ p = block->host + offset;
+
+ ram_counters.transferred += save_page_header(rs, rs->f, block,
+ offset | RAM_SAVE_FLAG_PAGE);
+ multifd_queue_page(block, offset);
+ qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE);
+ ram_counters.transferred += TARGET_PAGE_SIZE;
+ ram_counters.normal++;
+
+ return 1;
+}
+
static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block,
ram_addr_t offset, uint8_t *source_buf)
{
@@ -2116,6 +2199,8 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss,
*/
if (block == rs->last_sent_block && save_page_use_compression(rs)) {
return compress_page_with_multi_thread(rs, block, offset);
+ } else if (migrate_use_multifd()) {
+ return ram_save_multifd_page(rs, block, offset);
}
return ram_save_page(rs, pss, last_stage);
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 09/12] migration: Start sending messages
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (7 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 08/12] migration: Create ram_save_multifd_page Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-06-11 15:49 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 10/12] migration: Wait for blocking IO Juan Quintela
` (3 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/ram.c | 29 ++++++++++++++++++++++++-----
1 file changed, 24 insertions(+), 5 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 28f5cea4d8..2584130c85 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -729,9 +729,6 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp)
RAMBlock *block;
int i;
- /* ToDo: We can't use it until we haven't received a message */
- return 0;
-
be32_to_cpus(&packet->magic);
if (packet->magic != MULTIFD_MAGIC) {
error_setg(errp, "multifd: received packet "
@@ -965,6 +962,7 @@ static void *multifd_send_thread(void *opaque)
{
MultiFDSendParams *p = opaque;
Error *local_err = NULL;
+ int ret;
trace_multifd_send_thread_start(p->id);
@@ -992,7 +990,16 @@ static void *multifd_send_thread(void *opaque)
trace_multifd_send(p->id, seq, used, flags);
- /* ToDo: send packet here */
+ ret = qio_channel_write_all(p->c, (void *)p->packet,
+ p->packet_len, &local_err);
+ if (ret != 0) {
+ break;
+ }
+
+ ret = qio_channel_writev_all(p->c, p->pages->iov, used, &local_err);
+ if (ret != 0) {
+ break;
+ }
qemu_mutex_lock(&p->mutex);
p->pending_job--;
@@ -1204,7 +1211,14 @@ static void *multifd_recv_thread(void *opaque)
uint32_t flags;
qemu_mutex_unlock(&p->mutex);
- /* ToDo: recv packet here */
+ ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
+ p->packet_len, &local_err);
+ if (ret == 0) { /* EOF */
+ break;
+ }
+ if (ret == -1) { /* Error */
+ break;
+ }
qemu_mutex_lock(&p->mutex);
ret = multifd_recv_unfill_packet(p, &local_err);
@@ -1221,6 +1235,11 @@ static void *multifd_recv_thread(void *opaque)
p->num_pages += used;
qemu_mutex_unlock(&p->mutex);
+ ret = qio_channel_readv_all(p->c, p->pages->iov, used, &local_err);
+ if (ret != 0) {
+ break;
+ }
+
if (flags & MULTIFD_FLAG_SYNC) {
qemu_sem_post(&multifd_recv_state->sem_sync);
qemu_sem_wait(&p->sem_sync);
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 10/12] migration: Wait for blocking IO
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (8 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 09/12] migration: Start sending messages Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-06-11 15:54 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit Juan Quintela
` (2 subsequent siblings)
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
We have three conditions here:
- channel fails -> error
- we have to quit: we close the channel and reads fails
- normal read that success, we are in bussiness
So forget the complications of waiting in a semaphore.
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/ram.c | 81 ++++++++++++++++++-------------------------------
1 file changed, 29 insertions(+), 52 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 2584130c85..a707d3ae80 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -595,8 +595,6 @@ typedef struct {
bool running;
/* should this thread finish */
bool quit;
- /* thread has work to do */
- bool pending_job;
/* array of pages to receive */
MultiFDPages_t *pages;
/* packet allocated len */
@@ -1170,14 +1168,6 @@ static void multifd_recv_sync_main(void)
for (i = 0; i < migrate_multifd_channels(); i++) {
MultiFDRecvParams *p = &multifd_recv_state->params[i];
- trace_multifd_recv_sync_main_signal(p->id);
- qemu_mutex_lock(&p->mutex);
- p->pending_job = true;
- qemu_mutex_unlock(&p->mutex);
- }
- for (i = 0; i < migrate_multifd_channels(); i++) {
- MultiFDRecvParams *p = &multifd_recv_state->params[i];
-
trace_multifd_recv_sync_main_wait(p->id);
qemu_sem_wait(&multifd_recv_state->sem_sync);
qemu_mutex_lock(&p->mutex);
@@ -1190,7 +1180,6 @@ static void multifd_recv_sync_main(void)
MultiFDRecvParams *p = &multifd_recv_state->params[i];
trace_multifd_recv_sync_main_signal(p->id);
-
qemu_sem_post(&p->sem_sync);
}
trace_multifd_recv_sync_main(multifd_recv_state->seq);
@@ -1205,51 +1194,40 @@ static void *multifd_recv_thread(void *opaque)
trace_multifd_recv_thread_start(p->id);
while (true) {
+ uint32_t used;
+ uint32_t flags;
+
+ ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
+ p->packet_len, &local_err);
+ if (ret == 0) { /* EOF */
+ break;
+ }
+ if (ret == -1) { /* Error */
+ break;
+ }
+
qemu_mutex_lock(&p->mutex);
- if (true || p->pending_job) {
- uint32_t used;
- uint32_t flags;
- qemu_mutex_unlock(&p->mutex);
-
- ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
- p->packet_len, &local_err);
- if (ret == 0) { /* EOF */
- break;
- }
- if (ret == -1) { /* Error */
- break;
- }
-
- qemu_mutex_lock(&p->mutex);
- ret = multifd_recv_unfill_packet(p, &local_err);
- if (ret) {
- qemu_mutex_unlock(&p->mutex);
- break;
- }
-
- used = p->pages->used;
- flags = p->flags;
- trace_multifd_recv(p->id, p->seq, used, flags);
- p->pending_job = false;
- p->num_packets++;
- p->num_pages += used;
+ ret = multifd_recv_unfill_packet(p, &local_err);
+ if (ret) {
qemu_mutex_unlock(&p->mutex);
+ break;
+ }
- ret = qio_channel_readv_all(p->c, p->pages->iov, used, &local_err);
- if (ret != 0) {
- break;
- }
+ used = p->pages->used;
+ flags = p->flags;
+ trace_multifd_recv(p->id, p->seq, used, flags);
+ p->num_packets++;
+ p->num_pages += used;
+ qemu_mutex_unlock(&p->mutex);
- if (flags & MULTIFD_FLAG_SYNC) {
- qemu_sem_post(&multifd_recv_state->sem_sync);
- qemu_sem_wait(&p->sem_sync);
- }
- } else if (p->quit) {
- qemu_mutex_unlock(&p->mutex);
+ ret = qio_channel_readv_all(p->c, p->pages->iov, used, &local_err);
+ if (ret != 0) {
break;
- } else {
- qemu_mutex_unlock(&p->mutex);
- /* sometimes there are spurious wakeups */
+ }
+
+ if (flags & MULTIFD_FLAG_SYNC) {
+ qemu_sem_post(&multifd_recv_state->sem_sync);
+ qemu_sem_wait(&p->sem_sync);
}
}
@@ -1287,7 +1265,6 @@ int multifd_load_setup(void)
qemu_sem_init(&p->sem, 0);
qemu_sem_init(&p->sem_sync, 0);
p->quit = false;
- p->pending_job = false;
p->id = i;
p->pages = multifd_pages_init(page_count);
p->packet_len = sizeof(MultiFDPacket_t)
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (9 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 10/12] migration: Wait for blocking IO Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-06-11 16:45 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 12/12] migration: Stop sending whole pages through main channel Juan Quintela
2018-05-23 11:41 ` [Qemu-devel] [PATCH v13 00/12] Multifd no-reply
12 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
We know quit closing the QIO.
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/ram.c | 11 ++---------
1 file changed, 2 insertions(+), 9 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index a707d3ae80..62eea9e02f 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -587,14 +587,10 @@ typedef struct {
QemuThread thread;
/* communication channel */
QIOChannel *c;
- /* sem where to wait for more work */
- QemuSemaphore sem;
/* this mutex protects the following parameters */
QemuMutex mutex;
/* is this channel thread running */
bool running;
- /* should this thread finish */
- bool quit;
/* array of pages to receive */
MultiFDPages_t *pages;
/* packet allocated len */
@@ -1115,8 +1111,8 @@ static void multifd_recv_terminate_threads(Error *err)
MultiFDRecvParams *p = &multifd_recv_state->params[i];
qemu_mutex_lock(&p->mutex);
- p->quit = true;
- qemu_sem_post(&p->sem);
+ object_unref(OBJECT(p->c));
+ p->c = NULL;
qemu_mutex_unlock(&p->mutex);
}
}
@@ -1139,7 +1135,6 @@ int multifd_load_cleanup(Error **errp)
object_unref(OBJECT(p->c));
p->c = NULL;
qemu_mutex_destroy(&p->mutex);
- qemu_sem_destroy(&p->sem);
qemu_sem_destroy(&p->sem_sync);
g_free(p->name);
p->name = NULL;
@@ -1262,9 +1257,7 @@ int multifd_load_setup(void)
MultiFDRecvParams *p = &multifd_recv_state->params[i];
qemu_mutex_init(&p->mutex);
- qemu_sem_init(&p->sem, 0);
qemu_sem_init(&p->sem_sync, 0);
- p->quit = false;
p->id = i;
p->pages = multifd_pages_init(page_count);
p->packet_len = sizeof(MultiFDPacket_t)
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [Qemu-devel] [PATCH v13 12/12] migration: Stop sending whole pages through main channel
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (10 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit Juan Quintela
@ 2018-05-23 11:18 ` Juan Quintela
2018-05-23 11:41 ` [Qemu-devel] [PATCH v13 00/12] Multifd no-reply
12 siblings, 0 replies; 32+ messages in thread
From: Juan Quintela @ 2018-05-23 11:18 UTC (permalink / raw)
To: qemu-devel; +Cc: dgilbert, lvivier, peterx
We have to flush() the QEMUFile because now we sent really few data
through that channel.
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/ram.c | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 62eea9e02f..b327805403 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1771,15 +1771,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage)
static int ram_save_multifd_page(RAMState *rs, RAMBlock *block,
ram_addr_t offset)
{
- uint8_t *p;
-
- p = block->host + offset;
-
- ram_counters.transferred += save_page_header(rs, rs->f, block,
- offset | RAM_SAVE_FLAG_PAGE);
multifd_queue_page(block, offset);
- qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE);
- ram_counters.transferred += TARGET_PAGE_SIZE;
ram_counters.normal++;
return 1;
@@ -3013,6 +3005,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
multifd_send_sync_main();
qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+ qemu_fflush(f);
return 0;
}
@@ -3090,6 +3083,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
multifd_send_sync_main();
out:
qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+ qemu_fflush(f);
ram_counters.transferred += 8;
ret = qemu_file_get_error(f);
@@ -3143,6 +3137,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
multifd_send_sync_main();
qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+ qemu_fflush(f);
return 0;
}
--
2.17.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 00/12] Multifd
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
` (11 preceding siblings ...)
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 12/12] migration: Stop sending whole pages through main channel Juan Quintela
@ 2018-05-23 11:41 ` no-reply
12 siblings, 0 replies; 32+ messages in thread
From: no-reply @ 2018-05-23 11:41 UTC (permalink / raw)
To: quintela; +Cc: famz, qemu-devel, lvivier, dgilbert, peterx
Hi,
This series failed docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 20180523111817.1463-1-quintela@redhat.com
Subject: [Qemu-devel] [PATCH v13 00/12] Multifd
=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
2ac420358e migration: Stop sending whole pages through main channel
156ed2d4f7 migration: Remove not needed semaphore and quit
129d270d05 migration: Wait for blocking IO
e400c39d21 migration: Start sending messages
bc2e848a5b migration: Create ram_save_multifd_page
5a07f651c9 migration: Synchronize multifd threads with main thread
f1cbafa39f migration: Add block where to send/receive packets
897c4da50d migration: Multifd channels always wait on the sem
c9287370ff migration: Calculate transferred ram correctly
cdd0c20179 migration: Add multifd traces for start/end thread
8b14f57507 migration: Create multifd packet
aa1abb1fb5 migration: Create multipage support
=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-7ulkeqm3/src/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
BUILD fedora
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-7ulkeqm3/src'
GEN /var/tmp/patchew-tester-tmp-7ulkeqm3/src/docker-src.2018-05-23-07.37.40.11348/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-7ulkeqm3/src/docker-src.2018-05-23-07.37.40.11348/qemu.tar.vroot'...
done.
Checking out files: 47% (2921/6141)
Checking out files: 48% (2948/6141)
Checking out files: 49% (3010/6141)
Checking out files: 50% (3071/6141)
Checking out files: 51% (3132/6141)
Checking out files: 52% (3194/6141)
Checking out files: 53% (3255/6141)
Checking out files: 54% (3317/6141)
Checking out files: 55% (3378/6141)
Checking out files: 56% (3439/6141)
Checking out files: 57% (3501/6141)
Checking out files: 58% (3562/6141)
Checking out files: 59% (3624/6141)
Checking out files: 60% (3685/6141)
Checking out files: 61% (3747/6141)
Checking out files: 62% (3808/6141)
Checking out files: 63% (3869/6141)
Checking out files: 64% (3931/6141)
Checking out files: 65% (3992/6141)
Checking out files: 66% (4054/6141)
Checking out files: 67% (4115/6141)
Checking out files: 68% (4176/6141)
Checking out files: 69% (4238/6141)
Checking out files: 70% (4299/6141)
Checking out files: 71% (4361/6141)
Checking out files: 72% (4422/6141)
Checking out files: 73% (4483/6141)
Checking out files: 74% (4545/6141)
Checking out files: 75% (4606/6141)
Checking out files: 76% (4668/6141)
Checking out files: 77% (4729/6141)
Checking out files: 78% (4790/6141)
Checking out files: 79% (4852/6141)
Checking out files: 80% (4913/6141)
Checking out files: 81% (4975/6141)
Checking out files: 82% (5036/6141)
Checking out files: 83% (5098/6141)
Checking out files: 84% (5159/6141)
Checking out files: 85% (5220/6141)
Checking out files: 86% (5282/6141)
Checking out files: 87% (5343/6141)
Checking out files: 88% (5405/6141)
Checking out files: 89% (5466/6141)
Checking out files: 90% (5527/6141)
Checking out files: 91% (5589/6141)
Checking out files: 92% (5650/6141)
Checking out files: 93% (5712/6141)
Checking out files: 94% (5773/6141)
Checking out files: 95% (5834/6141)
Checking out files: 96% (5896/6141)
Checking out files: 97% (5957/6141)
Checking out files: 98% (6019/6141)
Checking out files: 99% (6080/6141)
Checking out files: 100% (6141/6141)
Checking out files: 100% (6141/6141), done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-7ulkeqm3/src/docker-src.2018-05-23-07.37.40.11348/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-7ulkeqm3/src/docker-src.2018-05-23-07.37.40.11348/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
COPY RUNNER
RUN test-mingw in qemu:fedora
Packages installed:
PyYAML-3.12-5.fc27.x86_64
SDL2-devel-2.0.7-2.fc27.x86_64
bc-1.07.1-3.fc27.x86_64
bison-3.0.4-8.fc27.x86_64
bluez-libs-devel-5.48-3.fc27.x86_64
brlapi-devel-0.6.6-8.fc27.x86_64
bzip2-1.0.6-24.fc27.x86_64
bzip2-devel-1.0.6-24.fc27.x86_64
ccache-3.3.6-1.fc27.x86_64
clang-5.0.1-5.fc27.x86_64
device-mapper-multipath-devel-0.7.1-9.git847cc43.fc27.x86_64
findutils-4.6.0-16.fc27.x86_64
flex-2.6.1-5.fc27.x86_64
gcc-7.3.1-5.fc27.x86_64
gcc-c++-7.3.1-5.fc27.x86_64
gettext-0.19.8.1-12.fc27.x86_64
git-2.14.3-3.fc27.x86_64
glib2-devel-2.54.3-2.fc27.x86_64
glusterfs-api-devel-3.12.7-1.fc27.x86_64
gnutls-devel-3.5.18-2.fc27.x86_64
gtk3-devel-3.22.26-2.fc27.x86_64
hostname-3.18-4.fc27.x86_64
libaio-devel-0.3.110-9.fc27.x86_64
libasan-7.3.1-5.fc27.x86_64
libattr-devel-2.4.47-21.fc27.x86_64
libcap-devel-2.25-7.fc27.x86_64
libcap-ng-devel-0.7.8-5.fc27.x86_64
libcurl-devel-7.55.1-10.fc27.x86_64
libfdt-devel-1.4.6-1.fc27.x86_64
libpng-devel-1.6.31-1.fc27.x86_64
librbd-devel-12.2.4-1.fc27.x86_64
libssh2-devel-1.8.0-5.fc27.x86_64
libubsan-7.3.1-5.fc27.x86_64
libusbx-devel-1.0.21-4.fc27.x86_64
libxml2-devel-2.9.7-1.fc27.x86_64
llvm-5.0.1-6.fc27.x86_64
lzo-devel-2.08-11.fc27.x86_64
make-4.2.1-4.fc27.x86_64
mingw32-SDL-1.2.15-9.fc27.noarch
mingw32-bzip2-1.0.6-9.fc27.noarch
mingw32-curl-7.54.1-2.fc27.noarch
mingw32-glib2-2.54.1-1.fc27.noarch
mingw32-gmp-6.1.2-2.fc27.noarch
mingw32-gnutls-3.5.13-2.fc27.noarch
mingw32-gtk2-2.24.31-4.fc27.noarch
mingw32-gtk3-3.22.16-1.fc27.noarch
mingw32-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw32-libpng-1.6.29-2.fc27.noarch
mingw32-libssh2-1.8.0-3.fc27.noarch
mingw32-libtasn1-4.13-1.fc27.noarch
mingw32-nettle-3.3-3.fc27.noarch
mingw32-pixman-0.34.0-3.fc27.noarch
mingw32-pkg-config-0.28-9.fc27.x86_64
mingw64-SDL-1.2.15-9.fc27.noarch
mingw64-bzip2-1.0.6-9.fc27.noarch
mingw64-curl-7.54.1-2.fc27.noarch
mingw64-glib2-2.54.1-1.fc27.noarch
mingw64-gmp-6.1.2-2.fc27.noarch
mingw64-gnutls-3.5.13-2.fc27.noarch
mingw64-gtk2-2.24.31-4.fc27.noarch
mingw64-gtk3-3.22.16-1.fc27.noarch
mingw64-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw64-libpng-1.6.29-2.fc27.noarch
mingw64-libssh2-1.8.0-3.fc27.noarch
mingw64-libtasn1-4.13-1.fc27.noarch
mingw64-nettle-3.3-3.fc27.noarch
mingw64-pixman-0.34.0-3.fc27.noarch
mingw64-pkg-config-0.28-9.fc27.x86_64
ncurses-devel-6.0-13.20170722.fc27.x86_64
nettle-devel-3.4-1.fc27.x86_64
nss-devel-3.36.0-1.0.fc27.x86_64
numactl-devel-2.0.11-5.fc27.x86_64
package libjpeg-devel is not installed
perl-5.26.1-403.fc27.x86_64
pixman-devel-0.34.0-4.fc27.x86_64
python3-3.6.2-13.fc27.x86_64
snappy-devel-1.1.4-5.fc27.x86_64
sparse-0.5.1-2.fc27.x86_64
spice-server-devel-0.14.0-1.fc27.x86_64
systemtap-sdt-devel-3.2-3.fc27.x86_64
tar-1.29-7.fc27.x86_64
usbredir-devel-0.7.1-5.fc27.x86_64
virglrenderer-devel-0.6.0-3.20170210git76b3da97b.fc27.x86_64
vte3-devel-0.36.5-5.fc27.x86_64
which-2.21-4.fc27.x86_64
xen-devel-4.9.1-5.fc27.x86_64
zlib-devel-1.2.11-4.fc27.x86_64
Environment variables:
TARGET_LIST=
PACKAGES=ccache gettext git tar PyYAML sparse flex bison python3 bzip2 hostname gcc gcc-c++ llvm clang make perl which bc findutils glib2-devel libaio-devel pixman-devel zlib-devel libfdt-devel libasan libubsan bluez-libs-devel brlapi-devel bzip2-devel device-mapper-multipath-devel glusterfs-api-devel gnutls-devel gtk3-devel libattr-devel libcap-devel libcap-ng-devel libcurl-devel libjpeg-devel libpng-devel librbd-devel libssh2-devel libusbx-devel libxml2-devel lzo-devel ncurses-devel nettle-devel nss-devel numactl-devel SDL2-devel snappy-devel spice-server-devel systemtap-sdt-devel usbredir-devel virglrenderer-devel vte3-devel xen-devel mingw32-pixman mingw32-glib2 mingw32-gmp mingw32-SDL mingw32-pkg-config mingw32-gtk2 mingw32-gtk3 mingw32-gnutls mingw32-nettle mingw32-libtasn1 mingw32-libjpeg-turbo mingw32-libpng mingw32-curl mingw32-libssh2 mingw32-bzip2 mingw64-pixman mingw64-glib2 mingw64-gmp mingw64-SDL mingw64-pkg-config mingw64-gtk2 mingw64-gtk3 mingw64-gnutls mingw64-nettle mingw64-libtasn1 mingw64-libjpeg-turbo mingw64-libpng mingw64-curl mingw64-libssh2 mingw64-bzip2
J=8
V=
HOSTNAME=9b020e7fcdaa
DEBUG=
SHOW_ENV=1
PWD=/
HOME=/root
CCACHE_DIR=/var/tmp/ccache
DISTTAG=f27container
QEMU_CONFIGURE_OPTS=--python=/usr/bin/python3
FGC=f27
TEST_DIR=/tmp/qemu-test
SHLVL=1
FEATURES=mingw clang pyyaml asan dtc
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAKEFLAGS= -j8
EXTRA_CONFIGURE_OPTS=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=x86_64-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=1.2 --with-gtkabi=2.0
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install
local state directory queried at runtime
Windows SDK no
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules
C compiler x86_64-w64-mingw32-gcc
Host C compiler cc
C++ compiler x86_64-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -mms-bitfields -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/x86_64-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -m64 -mcx16 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/libpng16
LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g
QEMU_LDFLAGS -L$(BUILD_DIR)/dtc/libfdt
make make
install install
python /usr/bin/python3 -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (1.2.15)
GTK support yes (2.24.31)
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support yes
GNUTLS rnd yes
libgcrypt no
libgcrypt kdf no
nettle yes (3.3)
nettle kdf yes
libtasn1 yes
curses support no
virgl support no
curl support yes
mingw32 support yes
Audio drivers dsound
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support yes
VNC PNG support yes
xen support no
brlapi support no
bluez support no
Documentation no
PIE no
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support no
Install blobs yes
KVM support no
HAX support yes
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support no
RDMA support no
fdt support git
membarrier no
preadv support no
fdatasync no
madvise no
posix_madvise no
posix_memalign no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends simple
Trace output file trace-<pid>
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info yes
QGA MSI support no
seccomp support no
coroutine backend win32
coroutine pool yes
debug stack usage no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support yes
TPM passthrough no
TPM emulator no
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone no
WARNING: Use of GTK 2.0 is deprecated and will be removed in
WARNING: future releases. Please switch to using GTK 3.0
WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GEN qemu-options.def
GEN qapi-gen
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN ui/input-keymap-atset1-to-qcode.c
GEN ui/input-keymap-linux-to-qcode.c
GEN ui/input-keymap-qcode-to-atset1.c
GEN ui/input-keymap-qcode-to-atset2.c
GEN aarch64-softmmu/config-devices.mak
GEN x86_64-softmmu/config-devices.mak
GEN ui/input-keymap-qcode-to-atset3.c
GEN ui/input-keymap-qcode-to-linux.c
GEN ui/input-keymap-qcode-to-qnum.c
GEN ui/input-keymap-qcode-to-sun.c
GEN ui/input-keymap-qnum-to-qcode.c
GEN ui/input-keymap-usb-to-qcode.c
GEN ui/input-keymap-win32-to-qcode.c
GEN ui/input-keymap-x11-to-qcode.c
GEN ui/input-keymap-xorgevdev-to-qcode.c
GEN ui/input-keymap-xorgkbd-to-qcode.c
GEN ui/input-keymap-xorgxquartz-to-qcode.c
GEN ui/input-keymap-xorgxwin-to-qcode.c
GEN tests/test-qapi-gen
GEN trace-root.h
GEN util/trace.h
GEN crypto/trace.h
GEN io/trace.h
GEN migration/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/intc/trace.h
GEN hw/net/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/virtio/trace.h
GEN hw/audio/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/usb/trace.h
GEN hw/scsi/trace.h
GEN hw/nvram/trace.h
GEN hw/display/trace.h
GEN hw/input/trace.h
GEN hw/timer/trace.h
GEN hw/dma/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/sd/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/9pfs/trace.h
GEN hw/ppc/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/s390x/trace.h
GEN hw/vfio/trace.h
GEN hw/acpi/trace.h
GEN hw/arm/trace.h
GEN hw/alpha/trace.h
GEN hw/hppa/trace.h
GEN hw/xen/trace.h
GEN hw/ide/trace.h
GEN hw/tpm/trace.h
GEN ui/trace.h
GEN audio/trace.h
GEN net/trace.h
GEN target/arm/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/sparc/trace.h
GEN target/s390x/trace.h
GEN target/ppc/trace.h
GEN qom/trace.h
GEN linux-user/trace.h
GEN qapi/trace.h
GEN accel/tcg/trace.h
GEN accel/kvm/trace.h
GEN nbd/trace.h
GEN scsi/trace.h
GEN trace-root.c
GEN util/trace.c
GEN crypto/trace.c
GEN io/trace.c
GEN migration/trace.c
GEN block/trace.c
GEN chardev/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/intc/trace.c
GEN hw/net/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/virtio/trace.c
GEN hw/audio/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/usb/trace.c
GEN hw/scsi/trace.c
GEN hw/nvram/trace.c
GEN hw/display/trace.c
GEN hw/input/trace.c
GEN hw/timer/trace.c
GEN hw/dma/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/sd/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/9pfs/trace.c
GEN hw/ppc/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/s390x/trace.c
GEN hw/vfio/trace.c
GEN hw/acpi/trace.c
GEN hw/arm/trace.c
GEN hw/alpha/trace.c
GEN hw/hppa/trace.c
GEN hw/xen/trace.c
GEN hw/ide/trace.c
GEN hw/tpm/trace.c
GEN ui/trace.c
GEN audio/trace.c
GEN net/trace.c
GEN target/arm/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/sparc/trace.c
GEN target/s390x/trace.c
GEN target/ppc/trace.c
GEN qom/trace.c
GEN linux-user/trace.c
GEN qapi/trace.c
GEN accel/tcg/trace.c
GEN accel/kvm/trace.c
GEN nbd/trace.c
GEN scsi/trace.c
GEN config-all-devices.mak
DEP /tmp/qemu-test/src/dtc/tests/dumptrees.c
DEP /tmp/qemu-test/src/dtc/tests/trees.S
DEP /tmp/qemu-test/src/dtc/tests/testutils.c
DEP /tmp/qemu-test/src/dtc/tests/value-labels.c
DEP /tmp/qemu-test/src/dtc/tests/asm_tree_dump.c
DEP /tmp/qemu-test/src/dtc/tests/truncated_property.c
DEP /tmp/qemu-test/src/dtc/tests/check_path.c
DEP /tmp/qemu-test/src/dtc/tests/overlay_bad_fixup.c
DEP /tmp/qemu-test/src/dtc/tests/overlay.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/property_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/integer-expressions.c
DEP /tmp/qemu-test/src/dtc/tests/utilfdt_test.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset_aliases.c
DEP /tmp/qemu-test/src/dtc/tests/add_subnode_with_nops.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_unordered.c
DEP /tmp/qemu-test/src/dtc/tests/dtb_reverse.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_ordered.c
DEP /tmp/qemu-test/src/dtc/tests/extra-terminating-null.c
DEP /tmp/qemu-test/src/dtc/tests/incbin.c
DEP /tmp/qemu-test/src/dtc/tests/boot-cpuid.c
DEP /tmp/qemu-test/src/dtc/tests/phandle_format.c
DEP /tmp/qemu-test/src/dtc/tests/path-references.c
DEP /tmp/qemu-test/src/dtc/tests/references.c
DEP /tmp/qemu-test/src/dtc/tests/string_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/propname_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop2.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop1.c
DEP /tmp/qemu-test/src/dtc/tests/del_node.c
DEP /tmp/qemu-test/src/dtc/tests/del_property.c
DEP /tmp/qemu-test/src/dtc/tests/setprop.c
DEP /tmp/qemu-test/src/dtc/tests/set_name.c
DEP /tmp/qemu-test/src/dtc/tests/rw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/nopulate.c
DEP /tmp/qemu-test/src/dtc/tests/mangle-layout.c
DEP /tmp/qemu-test/src/dtc/tests/move_and_save.c
DEP /tmp/qemu-test/src/dtc/tests/open_pack.c
DEP /tmp/qemu-test/src/dtc/tests/sw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/nop_node.c
DEP /tmp/qemu-test/src/dtc/tests/setprop_inplace.c
DEP /tmp/qemu-test/src/dtc/tests/nop_property.c
DEP /tmp/qemu-test/src/dtc/tests/stringlist.c
DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells.c
DEP /tmp/qemu-test/src/dtc/tests/notfound.c
DEP /tmp/qemu-test/src/dtc/tests/sized_cells.c
DEP /tmp/qemu-test/src/dtc/tests/char_literal.c
DEP /tmp/qemu-test/src/dtc/tests/get_alias.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_check_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_prop_value.c
DEP /tmp/qemu-test/src/dtc/tests/parent_offset.c
DEP /tmp/qemu-test/src/dtc/tests/supernode_atdepth_offset.c
DEP /tmp/qemu-test/src/dtc/tests/get_path.c
DEP /tmp/qemu-test/src/dtc/tests/get_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/get_name.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset.c
DEP /tmp/qemu-test/src/dtc/tests/getprop.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_offset.c
DEP /tmp/qemu-test/src/dtc/tests/find_property.c
DEP /tmp/qemu-test/src/dtc/tests/root_node.c
DEP /tmp/qemu-test/src/dtc/tests/get_mem_rsv.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_overlay.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_addresses.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_empty_tree.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_strerror.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_rw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_sw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_wip.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_ro.c
DEP /tmp/qemu-test/src/dtc/util.c
DEP /tmp/qemu-test/src/dtc/fdtoverlay.c
DEP /tmp/qemu-test/src/dtc/fdtput.c
DEP /tmp/qemu-test/src/dtc/fdtget.c
DEP /tmp/qemu-test/src/dtc/fdtdump.c
LEX convert-dtsv0-lexer.lex.c
DEP /tmp/qemu-test/src/dtc/srcpos.c
LEX dtc-lexer.lex.c
DEP /tmp/qemu-test/src/dtc/treesource.c
BISON dtc-parser.tab.c
DEP /tmp/qemu-test/src/dtc/livetree.c
DEP /tmp/qemu-test/src/dtc/fstree.c
DEP /tmp/qemu-test/src/dtc/flattree.c
DEP /tmp/qemu-test/src/dtc/data.c
DEP /tmp/qemu-test/src/dtc/dtc.c
DEP /tmp/qemu-test/src/dtc/checks.c
DEP convert-dtsv0-lexer.lex.c
DEP dtc-parser.tab.c
DEP dtc-lexer.lex.c
CHK version_gen.h
UPD version_gen.h
DEP /tmp/qemu-test/src/dtc/util.c
CC libfdt/fdt.o
CC libfdt/fdt_ro.o
CC libfdt/fdt_sw.o
CC libfdt/fdt_wip.o
CC libfdt/fdt_strerror.o
CC libfdt/fdt_rw.o
CC libfdt/fdt_empty_tree.o
CC libfdt/fdt_addresses.o
CC libfdt/fdt_overlay.o
AR libfdt/libfdt.a
x86_64-w64-mingw32-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
RC version.o
GEN qga/qapi-generated/qapi-gen
CC qapi/qapi-builtin-types.o
CC qapi/qapi-types.o
CC qapi/qapi-types-block.o
CC qapi/qapi-types-char.o
CC qapi/qapi-types-block-core.o
CC qapi/qapi-types-crypto.o
CC qapi/qapi-types-common.o
CC qapi/qapi-types-introspect.o
CC qapi/qapi-types-migration.o
CC qapi/qapi-types-misc.o
CC qapi/qapi-types-net.o
CC qapi/qapi-types-rocker.o
CC qapi/qapi-types-run-state.o
CC qapi/qapi-types-sockets.o
CC qapi/qapi-types-tpm.o
CC qapi/qapi-types-trace.o
CC qapi/qapi-types-ui.o
CC qapi/qapi-types-transaction.o
CC qapi/qapi-builtin-visit.o
CC qapi/qapi-visit.o
CC qapi/qapi-visit-block-core.o
CC qapi/qapi-visit-block.o
CC qapi/qapi-visit-char.o
CC qapi/qapi-visit-common.o
CC qapi/qapi-visit-crypto.o
CC qapi/qapi-visit-introspect.o
CC qapi/qapi-visit-migration.o
CC qapi/qapi-visit-misc.o
CC qapi/qapi-visit-net.o
CC qapi/qapi-visit-rocker.o
CC qapi/qapi-visit-run-state.o
CC qapi/qapi-visit-sockets.o
CC qapi/qapi-visit-tpm.o
CC qapi/qapi-visit-trace.o
CC qapi/qapi-visit-transaction.o
CC qapi/qapi-visit-ui.o
CC qapi/qapi-events.o
CC qapi/qapi-events-block-core.o
CC qapi/qapi-events-block.o
CC qapi/qapi-events-char.o
CC qapi/qapi-events-common.o
CC qapi/qapi-events-crypto.o
CC qapi/qapi-events-introspect.o
CC qapi/qapi-events-migration.o
CC qapi/qapi-events-misc.o
CC qapi/qapi-events-net.o
CC qapi/qapi-events-rocker.o
CC qapi/qapi-events-run-state.o
CC qapi/qapi-events-sockets.o
CC qapi/qapi-events-tpm.o
CC qapi/qapi-events-trace.o
CC qapi/qapi-events-transaction.o
CC qapi/qapi-events-ui.o
CC qapi/qapi-introspect.o
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qnum.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qbool.o
CC qobject/qlit.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
CC trace/simple.o
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/lockcnt.o
CC util/aiocb.o
CC util/async.o
CC util/aio-wait.o
CC util/thread-pool.o
CC util/qemu-timer.o
CC util/main-loop.o
CC util/iohandler.o
CC util/aio-win32.o
CC util/event_notifier-win32.o
CC util/oslib-win32.o
CC util/qemu-thread-win32.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/host-utils.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/fifo8.o
CC util/acl.o
CC util/cacheinfo.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/qemu-config.o
CC util/iov.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/keyval.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/readline.o
CC util/rcu.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-win32.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/pagesize.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC util/stats64.o
CC trace-root.o
CC util/systemd.o
CC util/trace.o
CC crypto/trace.o
CC io/trace.o
CC migration/trace.o
CC block/trace.o
CC chardev/trace.o
CC hw/block/trace.o
CC hw/block/dataplane/trace.o
CC hw/char/trace.o
CC hw/intc/trace.o
CC hw/net/trace.o
CC hw/rdma/vmw/trace.o
CC hw/rdma/trace.o
CC hw/virtio/trace.o
CC hw/audio/trace.o
CC hw/misc/trace.o
CC hw/misc/macio/trace.o
CC hw/usb/trace.o
CC hw/scsi/trace.o
CC hw/nvram/trace.o
CC hw/display/trace.o
CC hw/input/trace.o
CC hw/timer/trace.o
CC hw/dma/trace.o
CC hw/sparc/trace.o
CC hw/sparc64/trace.o
CC hw/sd/trace.o
CC hw/isa/trace.o
CC hw/mem/trace.o
CC hw/i386/trace.o
CC hw/i386/xen/trace.o
CC hw/9pfs/trace.o
CC hw/ppc/trace.o
CC hw/pci/trace.o
CC hw/pci-host/trace.o
CC hw/s390x/trace.o
CC hw/vfio/trace.o
CC hw/acpi/trace.o
CC hw/arm/trace.o
CC hw/alpha/trace.o
CC hw/hppa/trace.o
CC hw/xen/trace.o
CC hw/ide/trace.o
CC hw/tpm/trace.o
CC ui/trace.o
CC audio/trace.o
CC net/trace.o
CC target/arm/trace.o
CC target/i386/trace.o
CC target/mips/trace.o
CC target/sparc/trace.o
CC target/s390x/trace.o
CC target/ppc/trace.o
CC qom/trace.o
CC linux-user/trace.o
CC qapi/trace.o
CC accel/tcg/trace.o
CC accel/kvm/trace.o
CC nbd/trace.o
CC scsi/trace.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-clock.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/error-printf.o
CC stubs/fdset.o
CC stubs/gdbstub.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/iothread-lock.o
CC stubs/is-daemonized.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/change-state-handler.o
CC stubs/monitor.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/tpm.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/fd-register.o
CC stubs/qmp_memory_device.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/vmgenid.o
CC stubs/xen-common.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
CC stubs/ram-block.o
GEN qemu-img-cmds.h
CC block.o
CC qemu-io-cmds.o
CC blockjob.o
CC replication.o
CC block/raw-format.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qcow2-bitmap.o
CC block/qed.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/quorum.o
CC block/vhdx-log.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/file-win32.o
CC block/win32-aio.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
CC block/io.o
CC block/create.o
CC block/throttle-groups.o
CC block/nbd.o
CC block/nbd-client.o
CC block/sheepdog.o
CC block/accounting.o
CC block/dirty-bitmap.o
CC block/write-threshold.o
CC block/backup.o
CC block/replication.o
CC block/throttle.o
CC block/copy-on-read.o
CC block/crypto.o
CC nbd/server.o
CC nbd/client.o
CC nbd/common.o
CC scsi/utils.o
CC block/curl.o
CC block/ssh.o
CC block/dmg-bz2.o
CC crypto/init.o
CC crypto/hash.o
CC crypto/hash-nettle.o
CC crypto/hmac.o
CC crypto/hmac-nettle.o
CC crypto/aes.o
CC crypto/desrfb.o
CC crypto/cipher.o
CC crypto/tlscreds.o
CC crypto/tlscredsanon.o
CC crypto/tlscredsx509.o
CC crypto/tlssession.o
CC crypto/secret.o
CC crypto/random-gnutls.o
CC crypto/pbkdf.o
CC crypto/pbkdf-nettle.o
CC crypto/ivgen.o
CC crypto/ivgen-essiv.o
CC crypto/ivgen-plain.o
CC crypto/ivgen-plain64.o
CC crypto/afsplit.o
CC crypto/xts.o
CC crypto/block.o
CC crypto/block-qcow.o
CC crypto/block-luks.o
CC io/channel.o
CC io/channel-buffer.o
CC io/channel-command.o
CC io/channel-file.o
CC io/channel-socket.o
CC io/channel-tls.o
CC io/channel-watch.o
CC io/channel-websock.o
CC io/channel-util.o
CC io/dns-resolver.o
CC io/net-listener.o
CC io/task.o
CC qom/object.o
CC qom/qom-qobject.o
CC qom/container.o
CC qom/object_interfaces.o
CC qemu-io.o
CC blockdev.o
CC blockdev-nbd.o
CC bootdevice.o
CC iothread.o
CC qdev-monitor.o
CC device-hotplug.o
CC os-win32.o
CC bt-host.o
CC bt-vhci.o
CC dma-helpers.o
CC vl.o
CC tpm.o
CC device_tree.o
CC qapi/qapi-commands.o
CC qapi/qapi-commands-block-core.o
CC qapi/qapi-commands-block.o
CC qapi/qapi-commands-char.o
CC qapi/qapi-commands-common.o
CC qapi/qapi-commands-crypto.o
CC qapi/qapi-commands-introspect.o
CC qapi/qapi-commands-migration.o
CC qapi/qapi-commands-misc.o
CC qapi/qapi-commands-net.o
CC qapi/qapi-commands-rocker.o
CC qapi/qapi-commands-run-state.o
CC qapi/qapi-commands-sockets.o
CC qapi/qapi-commands-tpm.o
CC qapi/qapi-commands-trace.o
CC qapi/qapi-commands-transaction.o
CC qapi/qapi-commands-ui.o
CC qmp.o
CC hmp.o
CC cpus-common.o
CC audio/audio.o
CC audio/noaudio.o
CC audio/wavaudio.o
CC audio/mixeng.o
CC audio/dsoundaudio.o
CC audio/audio_win_int.o
CC audio/wavcapture.o
CC backends/rng.o
CC backends/rng-egd.o
CC backends/tpm.o
CC backends/hostmem.o
CC backends/hostmem-ram.o
CC backends/cryptodev.o
CC backends/cryptodev-builtin.o
CC backends/cryptodev-vhost.o
CC block/stream.o
CC chardev/msmouse.o
CC chardev/wctablet.o
CC chardev/testdev.o
CC disas/arm.o
CXX disas/arm-a64.o
CC disas/i386.o
CXX disas/libvixl/vixl/utils.o
CXX disas/libvixl/vixl/compiler-intrinsics.o
CXX disas/libvixl/vixl/a64/instructions-a64.o
CXX disas/libvixl/vixl/a64/decoder-a64.o
CXX disas/libvixl/vixl/a64/disasm-a64.o
CC hw/acpi/core.o
CC hw/acpi/piix4.o
CC hw/acpi/pcihp.o
CC hw/acpi/ich9.o
CC hw/acpi/tco.o
CC hw/acpi/cpu_hotplug.o
CC hw/acpi/memory_hotplug.o
CC hw/acpi/cpu.o
CC hw/acpi/nvdimm.o
CC hw/acpi/vmgenid.o
CC hw/acpi/acpi_interface.o
CC hw/acpi/bios-linker-loader.o
CC hw/acpi/aml-build.o
CC hw/acpi/ipmi.o
CC hw/acpi/acpi-stub.o
CC hw/acpi/ipmi-stub.o
CC hw/audio/sb16.o
CC hw/audio/es1370.o
CC hw/audio/ac97.o
CC hw/audio/fmopl.o
CC hw/audio/adlib.o
CC hw/audio/gus.o
CC hw/audio/gusemu_hal.o
CC hw/audio/gusemu_mixer.o
CC hw/audio/cs4231a.o
CC hw/audio/intel-hda.o
CC hw/audio/hda-codec.o
CC hw/audio/pcspk.o
CC hw/audio/wm8750.o
CC hw/audio/pl041.o
CC hw/audio/lm4549.o
CC hw/audio/marvell_88w8618.o
CC hw/audio/soundhw.o
CC hw/block/block.o
CC hw/block/cdrom.o
CC hw/block/hd-geometry.o
CC hw/block/fdc.o
CC hw/block/m25p80.o
CC hw/block/nand.o
CC hw/block/pflash_cfi01.o
CC hw/block/pflash_cfi02.o
CC hw/block/ecc.o
CC hw/block/onenand.o
CC hw/block/nvme.o
CC hw/bt/core.o
CC hw/bt/l2cap.o
CC hw/bt/sdp.o
CC hw/bt/hci.o
CC hw/bt/hid.o
CC hw/bt/hci-csr.o
CC hw/char/ipoctal232.o
CC hw/char/parallel.o
CC hw/char/parallel-isa.o
CC hw/char/pl011.o
CC hw/char/serial.o
CC hw/char/serial-isa.o
CC hw/char/serial-pci.o
CC hw/char/virtio-console.o
CC hw/char/cadence_uart.o
CC hw/char/cmsdk-apb-uart.o
CC hw/char/debugcon.o
CC hw/char/imx_serial.o
CC hw/core/qdev.o
CC hw/core/qdev-properties.o
CC hw/core/bus.o
CC hw/core/reset.o
CC hw/core/qdev-fw.o
CC hw/core/fw-path-provider.o
CC hw/core/irq.o
CC hw/core/hotplug.o
CC hw/core/nmi.o
CC hw/core/stream.o
CC hw/core/ptimer.o
CC hw/core/sysbus.o
CC hw/core/machine.o
CC hw/core/loader.o
CC hw/core/qdev-properties-system.o
CC hw/core/register.o
CC hw/core/or-irq.o
CC hw/core/split-irq.o
CC hw/core/platform-bus.o
CC hw/cpu/core.o
CC hw/display/ads7846.o
CC hw/display/cirrus_vga.o
CC hw/display/pl110.o
CC hw/display/ssd0303.o
CC hw/display/sii9022.o
CC hw/display/ssd0323.o
CC hw/display/vga-pci.o
CC hw/display/vga-isa.o
CC hw/display/vmware_vga.o
CC hw/display/blizzard.o
CC hw/display/exynos4210_fimd.o
CC hw/display/framebuffer.o
CC hw/display/tc6393xb.o
CC hw/dma/pl080.o
CC hw/dma/pl330.o
CC hw/dma/i8257.o
CC hw/dma/xilinx_axidma.o
CC hw/dma/xlnx-zynq-devcfg.o
CC hw/dma/xlnx-zdma.o
CC hw/gpio/max7310.o
CC hw/gpio/pl061.o
CC hw/gpio/zaurus.o
CC hw/gpio/gpio_key.o
CC hw/i2c/core.o
CC hw/i2c/smbus.o
CC hw/i2c/smbus_eeprom.o
CC hw/i2c/i2c-ddc.o
CC hw/i2c/versatile_i2c.o
CC hw/i2c/smbus_ich9.o
CC hw/i2c/pm_smbus.o
CC hw/i2c/bitbang_i2c.o
CC hw/i2c/exynos4210_i2c.o
CC hw/i2c/imx_i2c.o
CC hw/i2c/aspeed_i2c.o
CC hw/ide/core.o
CC hw/ide/atapi.o
CC hw/ide/qdev.o
CC hw/ide/pci.o
CC hw/ide/isa.o
CC hw/ide/piix.o
CC hw/ide/microdrive.o
CC hw/ide/ahci.o
CC hw/ide/ich.o
CC hw/ide/ahci-allwinner.o
CC hw/input/hid.o
CC hw/input/lm832x.o
CC hw/input/pckbd.o
CC hw/input/pl050.o
CC hw/input/ps2.o
CC hw/input/stellaris_input.o
CC hw/input/tsc2005.o
CC hw/input/virtio-input.o
CC hw/input/virtio-input-hid.o
CC hw/intc/i8259_common.o
CC hw/intc/i8259.o
CC hw/intc/pl190.o
CC hw/intc/xlnx-pmu-iomod-intc.o
CC hw/intc/xlnx-zynqmp-ipi.o
CC hw/intc/imx_avic.o
CC hw/intc/imx_gpcv2.o
CC hw/intc/realview_gic.o
CC hw/intc/ioapic_common.o
CC hw/intc/arm_gic_common.o
CC hw/intc/arm_gic.o
CC hw/intc/arm_gicv2m.o
CC hw/intc/arm_gicv3_common.o
CC hw/intc/arm_gicv3.o
CC hw/intc/arm_gicv3_dist.o
CC hw/intc/arm_gicv3_redist.o
CC hw/intc/arm_gicv3_its_common.o
CC hw/intc/intc.o
CC hw/ipack/ipack.o
CC hw/ipack/tpci200.o
CC hw/ipmi/ipmi.o
CC hw/ipmi/ipmi_bmc_sim.o
CC hw/ipmi/ipmi_bmc_extern.o
CC hw/ipmi/isa_ipmi_kcs.o
CC hw/ipmi/isa_ipmi_bt.o
CC hw/isa/isa-bus.o
CC hw/isa/isa-superio.o
CC hw/isa/smc37c669-superio.o
CC hw/isa/apm.o
CC hw/mem/pc-dimm.o
CC hw/mem/memory-device.o
CC hw/mem/nvdimm.o
CC hw/misc/applesmc.o
CC hw/misc/max111x.o
CC hw/misc/tmp105.o
CC hw/misc/tmp421.o
CC hw/misc/debugexit.o
CC hw/misc/sga.o
CC hw/misc/pc-testdev.o
CC hw/misc/pci-testdev.o
CC hw/misc/unimp.o
CC hw/misc/edu.o
CC hw/misc/vmcoreinfo.o
CC hw/misc/arm_l2x0.o
CC hw/misc/arm_integrator_debug.o
CC hw/misc/a9scu.o
CC hw/misc/arm11scu.o
CC hw/net/eepro100.o
CC hw/net/ne2000.o
CC hw/net/pcnet-pci.o
CC hw/net/pcnet.o
CC hw/net/e1000.o
CC hw/net/e1000x_common.o
CC hw/net/net_tx_pkt.o
CC hw/net/net_rx_pkt.o
CC hw/net/e1000e.o
CC hw/net/e1000e_core.o
CC hw/net/rtl8139.o
CC hw/net/vmxnet3.o
CC hw/net/smc91c111.o
CC hw/net/lan9118.o
CC hw/net/ne2000-isa.o
CC hw/net/xgmac.o
CC hw/net/xilinx_axienet.o
CC hw/net/allwinner_emac.o
CC hw/net/imx_fec.o
CC hw/net/cadence_gem.o
CC hw/net/stellaris_enet.o
CC hw/net/ftgmac100.o
CC hw/net/rocker/rocker.o
CC hw/net/rocker/rocker_fp.o
CC hw/net/rocker/rocker_desc.o
CC hw/net/rocker/rocker_world.o
CC hw/net/rocker/rocker_of_dpa.o
CC hw/net/can/can_sja1000.o
CC hw/net/can/can_kvaser_pci.o
CC hw/net/can/can_pcm3680_pci.o
CC hw/net/can/can_mioe3680_pci.o
CC hw/nvram/eeprom93xx.o
CC hw/nvram/eeprom_at24c.o
CC hw/nvram/fw_cfg.o
CC hw/nvram/chrp_nvram.o
CC hw/pci-bridge/pci_bridge_dev.o
CC hw/pci-bridge/pcie_root_port.o
CC hw/pci-bridge/gen_pcie_root_port.o
CC hw/pci-bridge/pcie_pci_bridge.o
CC hw/pci-bridge/pci_expander_bridge.o
CC hw/pci-bridge/xio3130_downstream.o
CC hw/pci-bridge/xio3130_upstream.o
CC hw/pci-bridge/ioh3420.o
CC hw/pci-bridge/i82801b11.o
CC hw/pci-host/pam.o
CC hw/pci-host/versatile.o
CC hw/pci-host/piix.o
CC hw/pci-host/q35.o
CC hw/pci-host/gpex.o
CC hw/pci-host/designware.o
CC hw/pci/pci.o
CC hw/pci/pci_bridge.o
CC hw/pci/msix.o
CC hw/pci/msi.o
CC hw/pci/shpc.o
CC hw/pci/slotid_cap.o
CC hw/pci/pci_host.o
CC hw/pci/pcie_host.o
CC hw/pci/pcie_aer.o
CC hw/pci/pcie.o
CC hw/pci/pcie_port.o
CC hw/pci/pci-stub.o
CC hw/pcmcia/pcmcia.o
CC hw/scsi/scsi-disk.o
CC hw/scsi/scsi-generic.o
CC hw/scsi/scsi-bus.o
CC hw/scsi/lsi53c895a.o
CC hw/scsi/mptsas.o
CC hw/scsi/mptconfig.o
CC hw/scsi/mptendian.o
CC hw/scsi/megasas.o
CC hw/scsi/vmw_pvscsi.o
CC hw/scsi/esp.o
CC hw/scsi/esp-pci.o
CC hw/sd/pl181.o
CC hw/sd/ssi-sd.o
CC hw/sd/sd.o
CC hw/sd/core.o
CC hw/sd/sdmmc-internal.o
CC hw/sd/sdhci.o
CC hw/smbios/smbios.o
CC hw/smbios/smbios_type_38.o
CC hw/smbios/smbios-stub.o
CC hw/smbios/smbios_type_38-stub.o
CC hw/ssi/pl022.o
CC hw/ssi/ssi.o
CC hw/ssi/xilinx_spips.o
CC hw/ssi/aspeed_smc.o
CC hw/ssi/stm32f2xx_spi.o
CC hw/ssi/mss-spi.o
CC hw/timer/arm_timer.o
CC hw/timer/arm_mptimer.o
CC hw/timer/armv7m_systick.o
CC hw/timer/a9gtimer.o
CC hw/timer/cadence_ttc.o
CC hw/timer/ds1338.o
CC hw/timer/hpet.o
CC hw/timer/i8254_common.o
CC hw/timer/i8254.o
CC hw/timer/pl031.o
CC hw/timer/twl92230.o
CC hw/timer/imx_epit.o
CC hw/timer/imx_gpt.o
CC hw/timer/xlnx-zynqmp-rtc.o
CC hw/timer/stm32f2xx_timer.o
CC hw/timer/aspeed_timer.o
CC hw/timer/cmsdk-apb-timer.o
CC hw/timer/mss-timer.o
CC hw/tpm/tpm_util.o
CC hw/tpm/tpm_tis.o
CC hw/tpm/tpm_crb.o
CC hw/usb/core.o
CC hw/usb/combined-packet.o
CC hw/usb/bus.o
CC hw/usb/libhw.o
CC hw/usb/desc.o
CC hw/usb/desc-msos.o
CC hw/usb/hcd-uhci.o
CC hw/usb/hcd-ohci.o
CC hw/usb/hcd-ehci.o
CC hw/usb/hcd-ehci-pci.o
CC hw/usb/hcd-ehci-sysbus.o
CC hw/usb/hcd-xhci.o
CC hw/usb/hcd-xhci-nec.o
CC hw/usb/hcd-musb.o
CC hw/usb/dev-hub.o
CC hw/usb/dev-hid.o
CC hw/usb/dev-wacom.o
CC hw/usb/dev-storage.o
CC hw/usb/dev-uas.o
CC hw/usb/dev-audio.o
CC hw/usb/dev-serial.o
CC hw/usb/dev-network.o
CC hw/usb/dev-bluetooth.o
CC hw/usb/dev-smartcard-reader.o
CC hw/usb/host-stub.o
CC hw/virtio/virtio-rng.o
CC hw/virtio/virtio-pci.o
CC hw/virtio/virtio-bus.o
CC hw/virtio/virtio-mmio.o
CC hw/virtio/vhost-stub.o
CC hw/watchdog/watchdog.o
CC hw/watchdog/wdt_i6300esb.o
CC hw/watchdog/wdt_ib700.o
CC hw/watchdog/wdt_aspeed.o
CC migration/migration.o
CC migration/socket.o
CC migration/fd.o
CC migration/exec.o
CC migration/tls.o
CC migration/channel.o
CC migration/savevm.o
CC migration/colo-comm.o
CC migration/colo.o
CC migration/colo-failover.o
CC migration/vmstate.o
CC migration/vmstate-types.o
CC migration/page_cache.o
CC migration/qemu-file.o
CC migration/global_state.o
CC migration/qemu-file-channel.o
CC migration/xbzrle.o
CC migration/postcopy-ram.o
CC migration/qjson.o
CC migration/block-dirty-bitmap.o
CC migration/block.o
CC net/net.o
CC net/queue.o
CC net/checksum.o
CC net/util.o
CC net/hub.o
CC net/socket.o
CC net/dump.o
CC net/slirp.o
CC net/eth.o
CC net/filter.o
CC net/filter-buffer.o
CC net/filter-mirror.o
CC net/colo-compare.o
CC net/colo.o
CC net/filter-rewriter.o
CC net/filter-replay.o
CC net/tap-win32.o
CC net/can/can_core.o
CC net/can/can_host.o
CC qom/cpu.o
CC replay/replay.o
CC replay/replay-internal.o
CC replay/replay-events.o
CC replay/replay-time.o
CC replay/replay-input.o
CC replay/replay-char.o
CC replay/replay-snapshot.o
CC replay/replay-net.o
CC replay/replay-audio.o
CC slirp/cksum.o
CC slirp/if.o
CC slirp/ip_icmp.o
CC slirp/ip6_icmp.o
CC slirp/ip6_input.o
CC slirp/ip6_output.o
CC slirp/ip_input.o
CC slirp/ip_output.o
CC slirp/dnssearch.o
CC slirp/dhcpv6.o
CC slirp/slirp.o
CC slirp/mbuf.o
CC slirp/misc.o
CC slirp/sbuf.o
CC slirp/socket.o
CC slirp/tcp_input.o
CC slirp/tcp_output.o
CC slirp/tcp_subr.o
CC slirp/tcp_timer.o
CC slirp/udp.o
CC slirp/udp6.o
CC slirp/bootp.o
CC slirp/tftp.o
CC slirp/arp_table.o
CC slirp/ndp_table.o
CC slirp/ncsi.o
CC ui/keymaps.o
CC ui/console.o
CC ui/cursor.o
CC ui/qemu-pixman.o
CC ui/input.o
CC ui/input-keymap.o
CC ui/input-legacy.o
CC ui/vnc.o
CC ui/vnc-enc-zlib.o
CC ui/vnc-enc-hextile.o
CC ui/vnc-enc-tight.o
CC ui/vnc-palette.o
CC ui/vnc-enc-zrle.o
CC ui/vnc-auth-vencrypt.o
CC ui/vnc-ws.o
CC ui/vnc-jobs.o
CC ui/sdl.o
CC ui/sdl_zoom.o
CC ui/gtk.o
CC chardev/char.o
CC chardev/char-console.o
CC chardev/char-fe.o
CC chardev/char-file.o
CC chardev/char-io.o
CC chardev/char-mux.o
CC chardev/char-null.o
CC chardev/char-pipe.o
CC chardev/char-ringbuf.o
CC chardev/char-serial.o
CC chardev/char-socket.o
CC chardev/char-stdio.o
CC chardev/char-udp.o
CC chardev/char-win.o
CC chardev/char-win-stdio.o
CC qga/commands.o
CC qga/guest-agent-command-state.o
CC qga/main.o
CC qga/commands-win32.o
CC qga/channel-win32.o
CC qga/service-win32.o
CC qga/vss-win32.o
CC qga/qapi-generated/qga-qapi-types.o
CC qga/qapi-generated/qga-qapi-visit.o
CC qga/qapi-generated/qga-qapi-commands.o
AR libqemuutil.a
CC qemu-img.o
AS optionrom/multiboot.o
AS optionrom/linuxboot.o
CC optionrom/linuxboot_dma.o
AS optionrom/kvmvapic.o
BUILD optionrom/multiboot.img
BUILD optionrom/linuxboot.img
BUILD optionrom/linuxboot_dma.img
BUILD optionrom/kvmvapic.img
BUILD optionrom/multiboot.raw
BUILD optionrom/linuxboot.raw
BUILD optionrom/linuxboot_dma.raw
BUILD optionrom/kvmvapic.raw
SIGN optionrom/kvmvapic.bin
SIGN optionrom/linuxboot.bin
SIGN optionrom/multiboot.bin
SIGN optionrom/linuxboot_dma.bin
LINK qemu-img.exe
LINK qemu-io.exe
LINK qemu-ga.exe
GEN x86_64-softmmu/hmp-commands.h
GEN x86_64-softmmu/hmp-commands-info.h
GEN x86_64-softmmu/config-target.h
GEN aarch64-softmmu/config-target.h
GEN aarch64-softmmu/hmp-commands.h
GEN aarch64-softmmu/hmp-commands-info.h
CC x86_64-softmmu/tcg/tcg.o
CC x86_64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/exec.o
CC x86_64-softmmu/tcg/tcg-op-vec.o
CC x86_64-softmmu/tcg/tcg-op-gvec.o
CC x86_64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/tcg/optimize.o
CC aarch64-softmmu/exec.o
CC x86_64-softmmu/fpu/softfloat.o
CC x86_64-softmmu/disas.o
GEN x86_64-softmmu/gdbstub-xml.c
CC x86_64-softmmu/arch_init.o
CC aarch64-softmmu/tcg/tcg.o
CC aarch64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/cpus.o
CC x86_64-softmmu/monitor.o
CC x86_64-softmmu/gdbstub.o
CC aarch64-softmmu/tcg/tcg-op-vec.o
CC x86_64-softmmu/balloon.o
CC aarch64-softmmu/tcg/tcg-op-gvec.o
CC aarch64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/ioport.o
CC x86_64-softmmu/numa.o
CC x86_64-softmmu/qtest.o
CC x86_64-softmmu/memory.o
CC x86_64-softmmu/memory_mapping.o
CC x86_64-softmmu/dump.o
CC x86_64-softmmu/migration/ram.o
CC x86_64-softmmu/accel/accel.o
CC x86_64-softmmu/accel/stubs/hvf-stub.o
CC x86_64-softmmu/accel/stubs/whpx-stub.o
CC x86_64-softmmu/accel/stubs/kvm-stub.o
CC aarch64-softmmu/tcg/optimize.o
CC aarch64-softmmu/fpu/softfloat.o
CC x86_64-softmmu/accel/tcg/tcg-all.o
CC x86_64-softmmu/accel/tcg/tcg-runtime.o
CC x86_64-softmmu/accel/tcg/cputlb.o
CC x86_64-softmmu/accel/tcg/tcg-runtime-gvec.o
CC x86_64-softmmu/accel/tcg/cpu-exec.o
CC aarch64-softmmu/disas.o
CC x86_64-softmmu/accel/tcg/cpu-exec-common.o
CC x86_64-softmmu/accel/tcg/translate-all.o
GEN aarch64-softmmu/gdbstub-xml.c
CC aarch64-softmmu/arch_init.o
CC x86_64-softmmu/accel/tcg/translator.o
CC x86_64-softmmu/hw/block/virtio-blk.o
CC aarch64-softmmu/cpus.o
CC aarch64-softmmu/monitor.o
CC aarch64-softmmu/gdbstub.o
CC x86_64-softmmu/hw/block/dataplane/virtio-blk.o
CC x86_64-softmmu/hw/char/virtio-serial-bus.o
CC aarch64-softmmu/balloon.o
CC aarch64-softmmu/ioport.o
CC aarch64-softmmu/numa.o
CC x86_64-softmmu/hw/core/generic-loader.o
CC x86_64-softmmu/hw/core/null-machine.o
CC aarch64-softmmu/qtest.o
CC x86_64-softmmu/hw/display/vga.o
CC x86_64-softmmu/hw/display/virtio-gpu.o
CC aarch64-softmmu/memory.o
CC x86_64-softmmu/hw/display/virtio-gpu-3d.o
CC aarch64-softmmu/memory_mapping.o
CC x86_64-softmmu/hw/display/virtio-gpu-pci.o
CC aarch64-softmmu/dump.o
CC x86_64-softmmu/hw/display/virtio-vga.o
CC x86_64-softmmu/hw/intc/apic.o
CC x86_64-softmmu/hw/intc/apic_common.o
CC x86_64-softmmu/hw/intc/ioapic.o
CC x86_64-softmmu/hw/isa/lpc_ich9.o
CC x86_64-softmmu/hw/misc/pvpanic.o
CC x86_64-softmmu/hw/misc/mmio_interface.o
CC x86_64-softmmu/hw/net/virtio-net.o
CC aarch64-softmmu/migration/ram.o
CC aarch64-softmmu/accel/accel.o
CC x86_64-softmmu/hw/net/vhost_net.o
CC x86_64-softmmu/hw/scsi/virtio-scsi.o
CC aarch64-softmmu/accel/stubs/hax-stub.o
CC x86_64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC x86_64-softmmu/hw/timer/mc146818rtc.o
CC x86_64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/accel/stubs/hvf-stub.o
CC x86_64-softmmu/hw/virtio/virtio-balloon.o
CC x86_64-softmmu/hw/virtio/virtio-crypto.o
CC aarch64-softmmu/accel/stubs/whpx-stub.o
CC x86_64-softmmu/hw/virtio/virtio-crypto-pci.o
CC aarch64-softmmu/accel/stubs/kvm-stub.o
CC aarch64-softmmu/accel/tcg/tcg-all.o
CC x86_64-softmmu/hw/i386/multiboot.o
CC aarch64-softmmu/accel/tcg/cputlb.o
CC x86_64-softmmu/hw/i386/pc.o
CC aarch64-softmmu/accel/tcg/tcg-runtime.o
CC aarch64-softmmu/accel/tcg/tcg-runtime-gvec.o
CC aarch64-softmmu/accel/tcg/cpu-exec.o
CC x86_64-softmmu/hw/i386/pc_piix.o
CC x86_64-softmmu/hw/i386/pc_q35.o
CC aarch64-softmmu/accel/tcg/cpu-exec-common.o
CC x86_64-softmmu/hw/i386/pc_sysfw.o
CC aarch64-softmmu/accel/tcg/translate-all.o
CC aarch64-softmmu/accel/tcg/translator.o
CC aarch64-softmmu/hw/adc/stm32f2xx_adc.o
CC aarch64-softmmu/hw/block/virtio-blk.o
CC aarch64-softmmu/hw/block/dataplane/virtio-blk.o
CC aarch64-softmmu/hw/char/exynos4210_uart.o
CC aarch64-softmmu/hw/char/omap_uart.o
CC x86_64-softmmu/hw/i386/x86-iommu.o
CC aarch64-softmmu/hw/char/digic-uart.o
CC x86_64-softmmu/hw/i386/intel_iommu.o
CC x86_64-softmmu/hw/i386/amd_iommu.o
CC x86_64-softmmu/hw/i386/vmport.o
CC x86_64-softmmu/hw/i386/vmmouse.o
CC x86_64-softmmu/hw/i386/kvmvapic.o
CC x86_64-softmmu/hw/i386/acpi-build.o
CC x86_64-softmmu/target/i386/helper.o
CC x86_64-softmmu/target/i386/cpu.o
CC x86_64-softmmu/target/i386/gdbstub.o
CC x86_64-softmmu/target/i386/xsave_helper.o
CC aarch64-softmmu/hw/char/stm32f2xx_usart.o
CC x86_64-softmmu/target/i386/translate.o
CC aarch64-softmmu/hw/char/virtio-serial-bus.o
CC x86_64-softmmu/target/i386/bpt_helper.o
CC aarch64-softmmu/hw/char/bcm2835_aux.o
CC aarch64-softmmu/hw/core/generic-loader.o
CC x86_64-softmmu/target/i386/cc_helper.o
CC x86_64-softmmu/target/i386/excp_helper.o
CC x86_64-softmmu/target/i386/fpu_helper.o
CC x86_64-softmmu/target/i386/int_helper.o
CC x86_64-softmmu/target/i386/mem_helper.o
CC x86_64-softmmu/target/i386/misc_helper.o
CC x86_64-softmmu/target/i386/mpx_helper.o
CC x86_64-softmmu/target/i386/seg_helper.o
CC aarch64-softmmu/hw/core/null-machine.o
CC aarch64-softmmu/hw/cpu/arm11mpcore.o
CC aarch64-softmmu/hw/cpu/realview_mpcore.o
CC x86_64-softmmu/target/i386/smm_helper.o
CC aarch64-softmmu/hw/cpu/a9mpcore.o
CC aarch64-softmmu/hw/cpu/a15mpcore.o
CC aarch64-softmmu/hw/display/omap_dss.o
CC aarch64-softmmu/hw/display/omap_lcdc.o
CC aarch64-softmmu/hw/display/pxa2xx_lcd.o
CC aarch64-softmmu/hw/display/bcm2835_fb.o
CC aarch64-softmmu/hw/display/vga.o
CC aarch64-softmmu/hw/display/virtio-gpu.o
CC aarch64-softmmu/hw/display/virtio-gpu-3d.o
CC aarch64-softmmu/hw/display/virtio-gpu-pci.o
CC aarch64-softmmu/hw/display/dpcd.o
CC aarch64-softmmu/hw/display/xlnx_dp.o
CC x86_64-softmmu/target/i386/machine.o
CC x86_64-softmmu/target/i386/svm_helper.o
CC aarch64-softmmu/hw/dma/xlnx_dpdma.o
CC x86_64-softmmu/target/i386/arch_memory_mapping.o
CC aarch64-softmmu/hw/dma/omap_dma.o
CC aarch64-softmmu/hw/dma/soc_dma.o
CC x86_64-softmmu/target/i386/arch_dump.o
CC x86_64-softmmu/target/i386/monitor.o
CC x86_64-softmmu/target/i386/kvm-stub.o
CC aarch64-softmmu/hw/dma/pxa2xx_dma.o
CC aarch64-softmmu/hw/dma/bcm2835_dma.o
CC aarch64-softmmu/hw/gpio/omap_gpio.o
CC aarch64-softmmu/hw/gpio/imx_gpio.o
CC aarch64-softmmu/hw/gpio/bcm2835_gpio.o
CC x86_64-softmmu/target/i386/sev-stub.o
CC aarch64-softmmu/hw/input/pxa2xx_keypad.o
CC x86_64-softmmu/target/i386/hax-all.o
CC aarch64-softmmu/hw/i2c/omap_i2c.o
CC aarch64-softmmu/hw/input/tsc210x.o
CC aarch64-softmmu/hw/intc/armv7m_nvic.o
CC aarch64-softmmu/hw/intc/exynos4210_gic.o
CC x86_64-softmmu/target/i386/hax-mem.o
CC x86_64-softmmu/target/i386/hax-windows.o
CC aarch64-softmmu/hw/intc/exynos4210_combiner.o
GEN trace/generated-helpers.c
CC aarch64-softmmu/hw/intc/omap_intc.o
CC x86_64-softmmu/trace/control-target.o
CC aarch64-softmmu/hw/intc/bcm2835_ic.o
CC x86_64-softmmu/gdbstub-xml.o
CC aarch64-softmmu/hw/intc/bcm2836_control.o
CC aarch64-softmmu/hw/intc/allwinner-a10-pic.o
CC aarch64-softmmu/hw/intc/aspeed_vic.o
CC aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
CC aarch64-softmmu/hw/misc/arm_sysctl.o
CC x86_64-softmmu/trace/generated-helpers.o
CC aarch64-softmmu/hw/misc/cbus.o
CC aarch64-softmmu/hw/misc/exynos4210_pmu.o
CC aarch64-softmmu/hw/misc/exynos4210_clk.o
CC aarch64-softmmu/hw/misc/exynos4210_rng.o
CC aarch64-softmmu/hw/misc/imx_ccm.o
CC aarch64-softmmu/hw/misc/imx31_ccm.o
CC aarch64-softmmu/hw/misc/imx25_ccm.o
CC aarch64-softmmu/hw/misc/imx6_ccm.o
CC aarch64-softmmu/hw/misc/imx6_src.o
CC aarch64-softmmu/hw/misc/imx7_ccm.o
CC aarch64-softmmu/hw/misc/imx2_wdt.o
CC aarch64-softmmu/hw/misc/imx7_snvs.o
LINK x86_64-softmmu/qemu-system-x86_64w.exe
CC aarch64-softmmu/hw/misc/imx7_gpr.o
CC aarch64-softmmu/hw/misc/mst_fpga.o
CC aarch64-softmmu/hw/misc/omap_clk.o
CC aarch64-softmmu/hw/misc/omap_gpmc.o
CC aarch64-softmmu/hw/misc/omap_l4.o
CC aarch64-softmmu/hw/misc/omap_sdrc.o
CC aarch64-softmmu/hw/misc/omap_tap.o
CC aarch64-softmmu/hw/misc/bcm2835_mbox.o
CC aarch64-softmmu/hw/misc/bcm2835_property.o
CC aarch64-softmmu/hw/misc/bcm2835_rng.o
CC aarch64-softmmu/hw/misc/zynq_slcr.o
CC aarch64-softmmu/hw/misc/zynq-xadc.o
CC aarch64-softmmu/hw/misc/stm32f2xx_syscfg.o
CC aarch64-softmmu/hw/misc/mps2-fpgaio.o
CC aarch64-softmmu/hw/misc/mps2-scc.o
CC aarch64-softmmu/hw/misc/tz-ppc.o
CC aarch64-softmmu/hw/misc/iotkit-secctl.o
CC aarch64-softmmu/hw/misc/auxbus.o
CC aarch64-softmmu/hw/misc/aspeed_scu.o
CC aarch64-softmmu/hw/misc/aspeed_sdmc.o
CC aarch64-softmmu/hw/misc/msf2-sysreg.o
CC aarch64-softmmu/hw/misc/mmio_interface.o
CC aarch64-softmmu/hw/net/virtio-net.o
CC aarch64-softmmu/hw/net/vhost_net.o
CC aarch64-softmmu/hw/pcmcia/pxa2xx.o
CC aarch64-softmmu/hw/scsi/virtio-scsi.o
CC aarch64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC aarch64-softmmu/hw/sd/omap_mmc.o
CC aarch64-softmmu/hw/sd/pxa2xx_mmci.o
CC aarch64-softmmu/hw/sd/bcm2835_sdhost.o
CC aarch64-softmmu/hw/ssi/omap_spi.o
CC aarch64-softmmu/hw/ssi/imx_spi.o
CC aarch64-softmmu/hw/timer/exynos4210_mct.o
CC aarch64-softmmu/hw/timer/exynos4210_pwm.o
CC aarch64-softmmu/hw/timer/exynos4210_rtc.o
CC aarch64-softmmu/hw/timer/omap_gptimer.o
CC aarch64-softmmu/hw/timer/pxa2xx_timer.o
CC aarch64-softmmu/hw/timer/omap_synctimer.o
CC aarch64-softmmu/hw/timer/digic-timer.o
CC aarch64-softmmu/hw/timer/allwinner-a10-pit.o
CC aarch64-softmmu/hw/usb/tusb6010.o
CC aarch64-softmmu/hw/usb/chipidea.o
CC aarch64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/hw/virtio/virtio-balloon.o
CC aarch64-softmmu/hw/virtio/virtio-crypto.o
CC aarch64-softmmu/hw/virtio/virtio-crypto-pci.o
CC aarch64-softmmu/hw/arm/boot.o
CC aarch64-softmmu/hw/arm/virt.o
CC aarch64-softmmu/hw/arm/sysbus-fdt.o
CC aarch64-softmmu/hw/arm/virt-acpi-build.o
CC aarch64-softmmu/hw/arm/digic_boards.o
CC aarch64-softmmu/hw/arm/exynos4_boards.o
CC aarch64-softmmu/hw/arm/highbank.o
CC aarch64-softmmu/hw/arm/mainstone.o
CC aarch64-softmmu/hw/arm/integratorcp.o
CC aarch64-softmmu/hw/arm/musicpal.o
CC aarch64-softmmu/hw/arm/netduino2.o
CC aarch64-softmmu/hw/arm/nseries.o
CC aarch64-softmmu/hw/arm/omap_sx1.o
CC aarch64-softmmu/hw/arm/gumstix.o
CC aarch64-softmmu/hw/arm/palm.o
CC aarch64-softmmu/hw/arm/spitz.o
CC aarch64-softmmu/hw/arm/tosa.o
CC aarch64-softmmu/hw/arm/z2.o
CC aarch64-softmmu/hw/arm/realview.o
CC aarch64-softmmu/hw/arm/stellaris.o
CC aarch64-softmmu/hw/arm/collie.o
CC aarch64-softmmu/hw/arm/vexpress.o
CC aarch64-softmmu/hw/arm/versatilepb.o
CC aarch64-softmmu/hw/arm/xilinx_zynq.o
CC aarch64-softmmu/hw/arm/armv7m.o
CC aarch64-softmmu/hw/arm/exynos4210.o
CC aarch64-softmmu/hw/arm/pxa2xx.o
CC aarch64-softmmu/hw/arm/pxa2xx_gpio.o
CC aarch64-softmmu/hw/arm/pxa2xx_pic.o
CC aarch64-softmmu/hw/arm/digic.o
CC aarch64-softmmu/hw/arm/omap1.o
CC aarch64-softmmu/hw/arm/omap2.o
CC aarch64-softmmu/hw/arm/strongarm.o
CC aarch64-softmmu/hw/arm/allwinner-a10.o
CC aarch64-softmmu/hw/arm/cubieboard.o
CC aarch64-softmmu/hw/arm/bcm2835_peripherals.o
CC aarch64-softmmu/hw/arm/bcm2836.o
CC aarch64-softmmu/hw/arm/raspi.o
CC aarch64-softmmu/hw/arm/stm32f205_soc.o
CC aarch64-softmmu/hw/arm/xlnx-zynqmp.o
CC aarch64-softmmu/hw/arm/xlnx-zcu102.o
CC aarch64-softmmu/hw/arm/fsl-imx25.o
CC aarch64-softmmu/hw/arm/imx25_pdk.o
CC aarch64-softmmu/hw/arm/fsl-imx31.o
CC aarch64-softmmu/hw/arm/kzm.o
CC aarch64-softmmu/hw/arm/fsl-imx6.o
CC aarch64-softmmu/hw/arm/sabrelite.o
CC aarch64-softmmu/hw/arm/aspeed_soc.o
CC aarch64-softmmu/hw/arm/aspeed.o
CC aarch64-softmmu/hw/arm/mps2.o
CC aarch64-softmmu/hw/arm/mps2-tz.o
CC aarch64-softmmu/hw/arm/msf2-soc.o
CC aarch64-softmmu/hw/arm/msf2-som.o
CC aarch64-softmmu/hw/arm/iotkit.o
CC aarch64-softmmu/hw/arm/fsl-imx7.o
CC aarch64-softmmu/hw/arm/mcimx7d-sabre.o
CC aarch64-softmmu/hw/arm/smmu-common.o
CC aarch64-softmmu/hw/arm/smmuv3.o
CC aarch64-softmmu/target/arm/arm-semi.o
CC aarch64-softmmu/target/arm/machine.o
CC aarch64-softmmu/target/arm/psci.o
CC aarch64-softmmu/target/arm/arch_dump.o
CC aarch64-softmmu/target/arm/monitor.o
CC aarch64-softmmu/target/arm/kvm-stub.o
CC aarch64-softmmu/target/arm/translate.o
CC aarch64-softmmu/target/arm/op_helper.o
CC aarch64-softmmu/target/arm/helper.o
CC aarch64-softmmu/target/arm/cpu.o
CC aarch64-softmmu/target/arm/neon_helper.o
CC aarch64-softmmu/target/arm/iwmmxt_helper.o
CC aarch64-softmmu/target/arm/vec_helper.o
CC aarch64-softmmu/target/arm/gdbstub.o
CC aarch64-softmmu/target/arm/cpu64.o
CC aarch64-softmmu/target/arm/translate-a64.o
CC aarch64-softmmu/target/arm/helper-a64.o
CC aarch64-softmmu/target/arm/gdbstub64.o
CC aarch64-softmmu/target/arm/crypto_helper.o
CC aarch64-softmmu/target/arm/arm-powerctl.o
GEN aarch64-softmmu/target/arm/decode-sve.inc.c
CC aarch64-softmmu/target/arm/sve_helper.o
GEN trace/generated-helpers.c
CC aarch64-softmmu/trace/control-target.o
CC aarch64-softmmu/gdbstub-xml.o
CC aarch64-softmmu/trace/generated-helpers.o
CC aarch64-softmmu/target/arm/translate-sve.o
LINK aarch64-softmmu/qemu-system-aarch64w.exe
GEN x86_64-softmmu/qemu-system-x86_64.exe
GEN aarch64-softmmu/qemu-system-aarch64.exe
install -d -m 0755 "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install"
install -d -m 0755 "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install"
install -c -m 0755 qemu-ga.exe qemu-img.exe qemu-io.exe "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install"
x86_64-w64-mingw32-strip "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/qemu-ga.exe" "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/qemu-img.exe" "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/qemu-io.exe"
set -e; for x in bios.bin bios-256k.bin sgabios.bin vgabios.bin vgabios-cirrus.bin vgabios-stdvga.bin vgabios-vmware.bin vgabios-qxl.bin vgabios-virtio.bin ppc_rom.bin openbios-sparc32 openbios-sparc64 openbios-ppc QEMU,tcx.bin QEMU,cgthree.bin pxe-e1000.rom pxe-eepro100.rom pxe-ne2k_pci.rom pxe-pcnet.rom pxe-rtl8139.rom pxe-virtio.rom efi-e1000.rom efi-eepro100.rom efi-ne2k_pci.rom efi-pcnet.rom efi-rtl8139.rom efi-virtio.rom efi-e1000e.rom efi-vmxnet3.rom qemu-icon.bmp qemu_logo_no_text.svg bamboo.dtb canyonlands.dtb petalogix-s3adsp1800.dtb petalogix-ml605.dtb multiboot.bin linuxboot.bin linuxboot_dma.bin kvmvapic.bin s390-ccw.img s390-netboot.img spapr-rtas.bin slof.bin skiboot.lid palcode-clipper u-boot.e500 u-boot-sam460-20100605.bin qemu_vga.ndrv hppa-firmware.img; do \
install -c -m 0644 /tmp/qemu-test/src/pc-bios/$x "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install"; \
done
make -C po install
make[1]: Entering directory '/tmp/qemu-test/build/po'
GEN /tmp/qemu-test/src/po/messages.po
GEN /tmp/qemu-test/src/po/tr.po
GEN /tmp/qemu-test/src/po/it.po
GEN /tmp/qemu-test/src/po/bg.po
GEN /tmp/qemu-test/src/po/hu.po
GEN /tmp/qemu-test/src/po/fr_FR.po
GEN /tmp/qemu-test/src/po/zh_CN.po
GEN /tmp/qemu-test/src/po/de_DE.po
GEN hu.mo
GEN bg.mo
GEN it.mo
GEN zh_CN.mo
GEN fr_FR.mo
GEN de_DE.mo
GEN tr.mo
for obj in hu.mo tr.mo it.mo bg.mo fr_FR.mo zh_CN.mo de_DE.mo; do \
base=`basename $obj .mo`; \
install -d /tmp/qemu-test/build/=destdir/tmp/qemu-test/install/share/locale/$base/LC_MESSAGES; \
install -m644 $obj /tmp/qemu-test/build/=destdir/tmp/qemu-test/install/share/locale/$base/LC_MESSAGES/qemu.mo; \
done
make[1]: Leaving directory '/tmp/qemu-test/build/po'
install -d -m 0755 "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/keymaps"
set -e; for x in da en-gb et fr fr-ch is lt modifiers no pt-br sv ar de en-us fi fr-be hr it lv nl pl ru th common de-ch es fo fr-ca hu ja mk nl-be pt sl tr bepo cz; do \
install -c -m 0644 /tmp/qemu-test/src/pc-bios/keymaps/$x "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/keymaps"; \
done
install -c -m 0644 /tmp/qemu-test/build/trace-events-all "/tmp/qemu-test/build/=destdir/tmp/qemu-test/install/trace-events-all"
for d in x86_64-softmmu aarch64-softmmu; do \
make --no-print-directory --quiet BUILD_DIR=/tmp/qemu-test/build TARGET_DIR=$d/ -C $d install || exit 1 ; \
done
rm -f *.timestamp
make -C tests/tcg clean
make[1]: Entering directory '/tmp/qemu-test/build/tests/tcg'
rm -f *.timestamp
rm -f *~ *.o test-i386.out test-i386.ref \
test-x86_64.log test-x86_64.ref qruncom test_path hello-i386 linux-test testthread sha1-i386 test-i386 test-i386-fprem test-mmap run-test-x86_64
make[1]: Leaving directory '/tmp/qemu-test/build/tests/tcg'
rm -rf tests/check-qdict.exe tests/test-char.exe tests/check-qnum.exe tests/check-qstring.exe tests/check-qlist.exe tests/check-qnull.exe tests/check-qobject.exe tests/check-qjson.exe tests/check-qlit.exe tests/test-qobject-output-visitor.exe tests/test-clone-visitor.exe tests/test-qobject-input-visitor.exe tests/test-qmp-cmds.exe tests/test-string-input-visitor.exe tests/test-string-output-visitor.exe tests/test-qmp-event.exe tests/test-opts-visitor.exe tests/test-coroutine.exe tests/test-visitor-serialization.exe tests/test-iov.exe tests/test-aio.exe tests/test-aio-multithread.exe tests/test-throttle.exe tests/test-thread-pool.exe tests/test-hbitmap.exe tests/test-bdrv-drain.exe tests/test-blockjob.exe tests/test-blockjob-txn.exe tests/test-block-backend.exe tests/test-x86-cpuid.exe tests/test-xbzrle.exe tests/test-cutils.exe tests/test-shift128.exe tests/test-mul64.exe tests/test-int128.exe tests/rcutorture.exe tests/test-rcu-list.exe tests/test-qdist.exe tests/test-qht.exe tests/test-qht-par.exe tests/test-bitops.exe tests/test-bitcnt.exe tests/test-qdev-global-props.exe tests/check-qom-interface.exe tests/check-qom-proplist.exe tests/test-qemu-opts.exe tests/test-keyval.exe tests/test-write-threshold.exe tests/test-crypto-hash.exe tests/test-crypto-hmac.exe tests/test-crypto-cipher.exe tests/test-crypto-secret.exe tests/test-crypto-tlscredsx509.exe tests/test-crypto-tlssession.exe tests/test-timed-average.exe tests/test-util-sockets.exe tests/test-io-task.exe tests/test-io-channel-socket.exe tests/test-io-channel-file.exe tests/test-io-channel-tls.exe tests/test-io-channel-command.exe tests/test-io-channel-buffer.exe tests/test-base64.exe tests/test-crypto-pbkdf.exe tests/test-crypto-ivgen.exe tests/test-crypto-afsplit.exe tests/test-crypto-xts.exe tests/test-crypto-block.exe tests/test-logging.exe tests/test-replication.exe tests/test-bufferiszero.exe tests/test-uuid.exe tests/ptimer-test.exe tests/test-qapi-util.exe tests/*.o
rm -rf tests/ac97-test.exe tests/ahci-test.exe tests/bios-tables-test.exe tests/boot-order-test.exe tests/boot-serial-test.exe tests/cpu-plug-test.exe tests/device-introspect-test.exe tests/display-vga-test.exe tests/drive_del-test.exe tests/ds1338-test.exe tests/e1000-test.exe tests/e1000e-test.exe tests/eepro100-test.exe tests/endianness-test.exe tests/es1370-test.exe tests/fdc-test.exe tests/fw_cfg-test.exe tests/hd-geo-test.exe tests/i440fx-test.exe tests/i82801b11-test.exe tests/ide-test.exe tests/intel-hda-test.exe tests/ioh3420-test.exe tests/ipmi-bt-test.exe tests/ipmi-kcs-test.exe tests/ipoctal232-test.exe tests/m25p80-test.exe tests/m48t59-test.exe tests/machine-none-test.exe tests/megasas-test.exe tests/migration-test.exe tests/ne2000-test.exe tests/numa-test.exe tests/nvme-test.exe tests/pcnet-test.exe tests/pnv-xscom-test.exe tests/prom-env-test.exe tests/pvpanic-test.exe tests/pxe-test.exe tests/q35-test.exe tests/qmp-test.exe tests/qom-test.exe tests/rtas-test.exe tests/rtc-test.exe tests/rtl8139-test.exe tests/sdhci-test.exe tests/spapr-phb-test.exe tests/tco-test.exe tests/test-arm-mptimer.exe tests/test-hmp.exe tests/test-netfilter.exe tests/test-x86-cpuid-compat.exe tests/tmp105-test.exe tests/tpci200-test.exe tests/tpm-crb-test.exe tests/tpm-tis-test.exe tests/usb-hcd-ehci-test.exe tests/usb-hcd-ohci-test.exe tests/usb-hcd-uhci-test.exe tests/usb-hcd-xhci-test.exe tests/virtio-balloon-test.exe tests/virtio-blk-test.exe tests/virtio-console-test.exe tests/virtio-net-test.exe tests/virtio-rng-test.exe tests/virtio-scsi-test.exe tests/virtio-serial-test.exe tests/vmgenid-test.exe tests/vmxnet3-test.exe tests/wdt_ib700-test.exe
rm -f tests/test-qapi-gen-timestamp
rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h
rm -f qemu-options.def
rm -f *.msi
find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} +
rm -f qemu-ga qemu-img.exe qemu-io.exe qemu-ga TAGS cscope.* *.pod *~ */*~
rm -f fsdev/*.pod scsi/*.pod
rm -f qemu-img-cmds.h
rm -f ui/shader/*-vert.h ui/shader/*-frag.h
rm -f trace/generated-tracers-dtrace.dtrace*
rm -f trace/generated-tracers-dtrace.h*
rm -f qemu-version.h qemu-version.h-timestamp config-host.h config-host.h-timestamp qemu-options.def qemu-options.def-timestamp qapi/qapi-builtin-types.h qapi/qapi-builtin-types.h-timestamp qapi/qapi-builtin-types.c qapi/qapi-builtin-types.c-timestamp qapi/qapi-types.h qapi/qapi-types.h-timestamp qapi/qapi-types.c qapi/qapi-types.c-timestamp qapi/qapi-types-block-core.h qapi/qapi-types-block-core.h-timestamp qapi/qapi-types-block-core.c qapi/qapi-types-block-core.c-timestamp qapi/qapi-types-block.h qapi/qapi-types-block.h-timestamp qapi/qapi-types-block.c qapi/qapi-types-block.c-timestamp qapi/qapi-types-char.h qapi/qapi-types-char.h-timestamp qapi/qapi-types-char.c qapi/qapi-types-char.c-timestamp qapi/qapi-types-common.h qapi/qapi-types-common.h-timestamp qapi/qapi-types-common.c qapi/qapi-types-common.c-timestamp qapi/qapi-types-crypto.h qapi/qapi-types-crypto.h-timestamp qapi/qapi-types-crypto.c qapi/qapi-types-crypto.c-timestamp qapi/qapi-types-introspect.h qapi/qapi-types-introspect.h-timestamp qapi/qapi-types-introspect.c qapi/qapi-types-introspect.c-timestamp qapi/qapi-types-migration.h qapi/qapi-types-migration.h-timestamp qapi/qapi-types-migration.c qapi/qapi-types-migration.c-timestamp qapi/qapi-types-misc.h qapi/qapi-types-misc.h-timestamp qapi/qapi-types-misc.c qapi/qapi-types-misc.c-timestamp qapi/qapi-types-net.h qapi/qapi-types-net.h-timestamp qapi/qapi-types-net.c qapi/qapi-types-net.c-timestamp qapi/qapi-types-rocker.h qapi/qapi-types-rocker.h-timestamp qapi/qapi-types-rocker.c qapi/qapi-types-rocker.c-timestamp qapi/qapi-types-run-state.h qapi/qapi-types-run-state.h-timestamp qapi/qapi-types-run-state.c qapi/qapi-types-run-state.c-timestamp qapi/qapi-types-sockets.h qapi/qapi-types-sockets.h-timestamp qapi/qapi-types-sockets.c qapi/qapi-types-sockets.c-timestamp qapi/qapi-types-tpm.h qapi/qapi-types-tpm.h-timestamp qapi/qapi-types-tpm.c qapi/qapi-types-tpm.c-timestamp qapi/qapi-types-trace.h qapi/qapi-types-trace.h-timestamp qapi/qapi-types-trace.c qapi/qapi-types-trace.c-timestamp qapi/qapi-types-transaction.h qapi/qapi-types-transaction.h-timestamp qapi/qapi-types-transaction.c qapi/qapi-types-transaction.c-timestamp qapi/qapi-types-ui.h qapi/qapi-types-ui.h-timestamp qapi/qapi-types-ui.c qapi/qapi-types-ui.c-timestamp qapi/qapi-builtin-visit.h qapi/qapi-builtin-visit.h-timestamp qapi/qapi-builtin-visit.c qapi/qapi-builtin-visit.c-timestamp qapi/qapi-visit.h qapi/qapi-visit.h-timestamp qapi/qapi-visit.c qapi/qapi-visit.c-timestamp qapi/qapi-visit-block-core.h qapi/qapi-visit-block-core.h-timestamp qapi/qapi-visit-block-core.c qapi/qapi-visit-block-core.c-timestamp qapi/qapi-visit-block.h qapi/qapi-visit-block.h-timestamp qapi/qapi-visit-block.c qapi/qapi-visit-block.c-timestamp qapi/qapi-visit-char.h qapi/qapi-visit-char.h-timestamp qapi/qapi-visit-char.c qapi/qapi-visit-char.c-timestamp qapi/qapi-visit-common.h qapi/qapi-visit-common.h-timestamp qapi/qapi-visit-common.c qapi/qapi-visit-common.c-timestamp qapi/qapi-visit-crypto.h qapi/qapi-visit-crypto.h-timestamp qapi/qapi-visit-crypto.c qapi/qapi-visit-crypto.c-timestamp qapi/qapi-visit-introspect.h qapi/qapi-visit-introspect.h-timestamp qapi/qapi-visit-introspect.c qapi/qapi-visit-introspect.c-timestamp qapi/qapi-visit-migration.h qapi/qapi-visit-migration.h-timestamp qapi/qapi-visit-migration.c qapi/qapi-visit-migration.c-timestamp qapi/qapi-visit-misc.h qapi/qapi-visit-misc.h-timestamp qapi/qapi-visit-misc.c qapi/qapi-visit-misc.c-timestamp qapi/qapi-visit-net.h qapi/qapi-visit-net.h-timestamp qapi/qapi-visit-net.c qapi/qapi-visit-net.c-timestamp qapi/qapi-visit-rocker.h qapi/qapi-visit-rocker.h-timestamp qapi/qapi-visit-rocker.c qapi/qapi-visit-rocker.c-timestamp qapi/qapi-visit-run-state.h qapi/qapi-visit-run-state.h-timestamp qapi/qapi-visit-run-state.c qapi/qapi-visit-run-state.c-timestamp qapi/qapi-visit-sockets.h qapi/qapi-visit-sockets.h-timestamp qapi/qapi-visit-sockets.c qapi/qapi-visit-sockets.c-timestamp qapi/qapi-visit-tpm.h qapi/qapi-visit-tpm.h-timestamp qapi/qapi-visit-tpm.c qapi/qapi-visit-tpm.c-timestamp qapi/qapi-visit-trace.h qapi/qapi-visit-trace.h-timestamp qapi/qapi-visit-trace.c qapi/qapi-visit-trace.c-timestamp qapi/qapi-visit-transaction.h qapi/qapi-visit-transaction.h-timestamp qapi/qapi-visit-transaction.c qapi/qapi-visit-transaction.c-timestamp qapi/qapi-visit-ui.h qapi/qapi-visit-ui.h-timestamp qapi/qapi-visit-ui.c qapi/qapi-visit-ui.c-timestamp qapi/qapi-commands.h qapi/qapi-commands.h-timestamp qapi/qapi-commands.c qapi/qapi-commands.c-timestamp qapi/qapi-commands-block-core.h qapi/qapi-commands-block-core.h-timestamp qapi/qapi-commands-block-core.c qapi/qapi-commands-block-core.c-timestamp qapi/qapi-commands-block.h qapi/qapi-commands-block.h-timestamp qapi/qapi-commands-block.c qapi/qapi-commands-block.c-timestamp qapi/qapi-commands-char.h qapi/qapi-commands-char.h-timestamp qapi/qapi-commands-char.c qapi/qapi-commands-char.c-timestamp qapi/qapi-commands-common.h qapi/qapi-commands-common.h-timestamp qapi/qapi-commands-common.c qapi/qapi-commands-common.c-timestamp qapi/qapi-commands-crypto.h qapi/qapi-commands-crypto.h-timestamp qapi/qapi-commands-crypto.c qapi/qapi-commands-crypto.c-timestamp qapi/qapi-commands-introspect.h qapi/qapi-commands-introspect.h-timestamp qapi/qapi-commands-introspect.c qapi/qapi-commands-introspect.c-timestamp qapi/qapi-commands-migration.h qapi/qapi-commands-migration.h-timestamp qapi/qapi-commands-migration.c qapi/qapi-commands-migration.c-timestamp qapi/qapi-commands-misc.h qapi/qapi-commands-misc.h-timestamp qapi/qapi-commands-misc.c qapi/qapi-commands-misc.c-timestamp qapi/qapi-commands-net.h qapi/qapi-commands-net.h-timestamp qapi/qapi-commands-net.c qapi/qapi-commands-net.c-timestamp qapi/qapi-commands-rocker.h qapi/qapi-commands-rocker.h-timestamp qapi/qapi-commands-rocker.c qapi/qapi-commands-rocker.c-timestamp qapi/qapi-commands-run-state.h qapi/qapi-commands-run-state.h-timestamp qapi/qapi-commands-run-state.c qapi/qapi-commands-run-state.c-timestamp qapi/qapi-commands-sockets.h qapi/qapi-commands-sockets.h-timestamp qapi/qapi-commands-sockets.c qapi/qapi-commands-sockets.c-timestamp qapi/qapi-commands-tpm.h qapi/qapi-commands-tpm.h-timestamp qapi/qapi-commands-tpm.c qapi/qapi-commands-tpm.c-timestamp qapi/qapi-commands-trace.h qapi/qapi-commands-trace.h-timestamp qapi/qapi-commands-trace.c qapi/qapi-commands-trace.c-timestamp qapi/qapi-commands-transaction.h qapi/qapi-commands-transaction.h-timestamp qapi/qapi-commands-transaction.c qapi/qapi-commands-transaction.c-timestamp qapi/qapi-commands-ui.h qapi/qapi-commands-ui.h-timestamp qapi/qapi-commands-ui.c qapi/qapi-commands-ui.c-timestamp qapi/qapi-events.h qapi/qapi-events.h-timestamp qapi/qapi-events.c qapi/qapi-events.c-timestamp qapi/qapi-events-block-core.h qapi/qapi-events-block-core.h-timestamp qapi/qapi-events-block-core.c qapi/qapi-events-block-core.c-timestamp qapi/qapi-events-block.h qapi/qapi-events-block.h-timestamp qapi/qapi-events-block.c qapi/qapi-events-block.c-timestamp qapi/qapi-events-char.h qapi/qapi-events-char.h-timestamp qapi/qapi-events-char.c qapi/qapi-events-char.c-timestamp qapi/qapi-events-common.h qapi/qapi-events-common.h-timestamp qapi/qapi-events-common.c qapi/qapi-events-common.c-timestamp qapi/qapi-events-crypto.h qapi/qapi-events-crypto.h-timestamp qapi/qapi-events-crypto.c qapi/qapi-events-crypto.c-timestamp qapi/qapi-events-introspect.h qapi/qapi-events-introspect.h-timestamp qapi/qapi-events-introspect.c qapi/qapi-events-introspect.c-timestamp qapi/qapi-events-migration.h qapi/qapi-events-migration.h-timestamp qapi/qapi-events-migration.c qapi/qapi-events-migration.c-timestamp qapi/qapi-events-misc.h qapi/qapi-events-misc.h-timestamp qapi/qapi-events-misc.c qapi/qapi-events-misc.c-timestamp qapi/qapi-events-net.h qapi/qapi-events-net.h-timestamp qapi/qapi-events-net.c qapi/qapi-events-net.c-timestamp qapi/qapi-events-rocker.h qapi/qapi-events-rocker.h-timestamp qapi/qapi-events-rocker.c qapi/qapi-events-rocker.c-timestamp qapi/qapi-events-run-state.h qapi/qapi-events-run-state.h-timestamp qapi/qapi-events-run-state.c qapi/qapi-events-run-state.c-timestamp qapi/qapi-events-sockets.h qapi/qapi-events-sockets.h-timestamp qapi/qapi-events-sockets.c qapi/qapi-events-sockets.c-timestamp qapi/qapi-events-tpm.h qapi/qapi-events-tpm.h-timestamp qapi/qapi-events-tpm.c qapi/qapi-events-tpm.c-timestamp qapi/qapi-events-trace.h qapi/qapi-events-trace.h-timestamp qapi/qapi-events-trace.c qapi/qapi-events-trace.c-timestamp qapi/qapi-events-transaction.h qapi/qapi-events-transaction.h-timestamp qapi/qapi-events-transaction.c qapi/qapi-events-transaction.c-timestamp qapi/qapi-events-ui.h qapi/qapi-events-ui.h-timestamp qapi/qapi-events-ui.c qapi/qapi-events-ui.c-timestamp qapi/qapi-introspect.c qapi/qapi-introspect.c-timestamp qapi/qapi-introspect.h qapi/qapi-introspect.h-timestamp qapi/qapi-doc.texi qapi/qapi-doc.texi-timestamp trace/generated-tcg-tracers.h trace/generated-tcg-tracers.h-timestamp trace/generated-helpers-wrappers.h trace/generated-helpers-wrappers.h-timestamp trace/generated-helpers.h trace/generated-helpers.h-timestamp trace/generated-helpers.c trace/generated-helpers.c-timestamp module_block.h module_block.h-timestamp trace-root.h trace-root.h-timestamp util/trace.h util/trace.h-timestamp crypto/trace.h crypto/trace.h-timestamp io/trace.h io/trace.h-timestamp migration/trace.h migration/trace.h-timestamp block/trace.h block/trace.h-timestamp chardev/trace.h chardev/trace.h-timestamp hw/block/trace.h hw/block/trace.h-timestamp hw/block/dataplane/trace.h hw/block/dataplane/trace.h-timestamp hw/char/trace.h hw/char/trace.h-timestamp hw/intc/trace.h hw/intc/trace.h-timestamp hw/net/trace.h hw/net/trace.h-timestamp hw/rdma/trace.h hw/rdma/trace.h-timestamp hw/rdma/vmw/trace.h hw/rdma/vmw/trace.h-timestamp hw/virtio/trace.h hw/virtio/trace.h-timestamp hw/audio/trace.h hw/audio/trace.h-timestamp hw/misc/trace.h hw/misc/trace.h-timestamp hw/misc/macio/trace.h hw/misc/macio/trace.h-timestamp hw/usb/trace.h hw/usb/trace.h-timestamp hw/scsi/trace.h hw/scsi/trace.h-timestamp hw/nvram/trace.h hw/nvram/trace.h-timestamp hw/display/trace.h hw/display/trace.h-timestamp hw/input/trace.h hw/input/trace.h-timestamp hw/timer/trace.h hw/timer/trace.h-timestamp hw/dma/trace.h hw/dma/trace.h-timestamp hw/sparc/trace.h hw/sparc/trace.h-timestamp hw/sparc64/trace.h hw/sparc64/trace.h-timestamp hw/sd/trace.h hw/sd/trace.h-timestamp hw/isa/trace.h hw/isa/trace.h-timestamp hw/mem/trace.h hw/mem/trace.h-timestamp hw/i386/trace.h hw/i386/trace.h-timestamp hw/i386/xen/trace.h hw/i386/xen/trace.h-timestamp hw/9pfs/trace.h hw/9pfs/trace.h-timestamp hw/ppc/trace.h hw/ppc/trace.h-timestamp hw/pci/trace.h hw/pci/trace.h-timestamp hw/pci-host/trace.h hw/pci-host/trace.h-timestamp hw/s390x/trace.h hw/s390x/trace.h-timestamp hw/vfio/trace.h hw/vfio/trace.h-timestamp hw/acpi/trace.h hw/acpi/trace.h-timestamp hw/arm/trace.h hw/arm/trace.h-timestamp hw/alpha/trace.h hw/alpha/trace.h-timestamp hw/hppa/trace.h hw/hppa/trace.h-timestamp hw/xen/trace.h hw/xen/trace.h-timestamp hw/ide/trace.h hw/ide/trace.h-timestamp hw/tpm/trace.h hw/tpm/trace.h-timestamp ui/trace.h ui/trace.h-timestamp audio/trace.h audio/trace.h-timestamp net/trace.h net/trace.h-timestamp target/arm/trace.h target/arm/trace.h-timestamp target/i386/trace.h target/i386/trace.h-timestamp target/mips/trace.h target/mips/trace.h-timestamp target/sparc/trace.h target/sparc/trace.h-timestamp target/s390x/trace.h target/s390x/trace.h-timestamp target/ppc/trace.h target/ppc/trace.h-timestamp qom/trace.h qom/trace.h-timestamp linux-user/trace.h linux-user/trace.h-timestamp qapi/trace.h qapi/trace.h-timestamp accel/tcg/trace.h accel/tcg/trace.h-timestamp accel/kvm/trace.h accel/kvm/trace.h-timestamp nbd/trace.h nbd/trace.h-timestamp scsi/trace.h scsi/trace.h-timestamp trace-root.c trace-root.c-timestamp util/trace.c util/trace.c-timestamp crypto/trace.c crypto/trace.c-timestamp io/trace.c io/trace.c-timestamp migration/trace.c migration/trace.c-timestamp block/trace.c block/trace.c-timestamp chardev/trace.c chardev/trace.c-timestamp hw/block/trace.c hw/block/trace.c-timestamp hw/block/dataplane/trace.c hw/block/dataplane/trace.c-timestamp hw/char/trace.c hw/char/trace.c-timestamp hw/intc/trace.c hw/intc/trace.c-timestamp hw/net/trace.c hw/net/trace.c-timestamp hw/rdma/trace.c hw/rdma/trace.c-timestamp hw/rdma/vmw/trace.c hw/rdma/vmw/trace.c-timestamp hw/virtio/trace.c hw/virtio/trace.c-timestamp hw/audio/trace.c hw/audio/trace.c-timestamp hw/misc/trace.c hw/misc/trace.c-timestamp hw/misc/macio/trace.c hw/misc/macio/trace.c-timestamp hw/usb/trace.c hw/usb/trace.c-timestamp hw/scsi/trace.c hw/scsi/trace.c-timestamp hw/nvram/trace.c hw/nvram/trace.c-timestamp hw/display/trace.c hw/display/trace.c-timestamp hw/input/trace.c hw/input/trace.c-timestamp hw/timer/trace.c hw/timer/trace.c-timestamp hw/dma/trace.c hw/dma/trace.c-timestamp hw/sparc/trace.c hw/sparc/trace.c-timestamp hw/sparc64/trace.c hw/sparc64/trace.c-timestamp hw/sd/trace.c hw/sd/trace.c-timestamp hw/isa/trace.c hw/isa/trace.c-timestamp hw/mem/trace.c hw/mem/trace.c-timestamp hw/i386/trace.c hw/i386/trace.c-timestamp hw/i386/xen/trace.c hw/i386/xen/trace.c-timestamp hw/9pfs/trace.c hw/9pfs/trace.c-timestamp hw/ppc/trace.c hw/ppc/trace.c-timestamp hw/pci/trace.c hw/pci/trace.c-timestamp hw/pci-host/trace.c hw/pci-host/trace.c-timestamp hw/s390x/trace.c hw/s390x/trace.c-timestamp hw/vfio/trace.c hw/vfio/trace.c-timestamp hw/acpi/trace.c hw/acpi/trace.c-timestamp hw/arm/trace.c hw/arm/trace.c-timestamp hw/alpha/trace.c hw/alpha/trace.c-timestamp hw/hppa/trace.c hw/hppa/trace.c-timestamp hw/xen/trace.c hw/xen/trace.c-timestamp hw/ide/trace.c hw/ide/trace.c-timestamp hw/tpm/trace.c hw/tpm/trace.c-timestamp ui/trace.c ui/trace.c-timestamp audio/trace.c audio/trace.c-timestamp net/trace.c net/trace.c-timestamp target/arm/trace.c target/arm/trace.c-timestamp target/i386/trace.c target/i386/trace.c-timestamp target/mips/trace.c target/mips/trace.c-timestamp target/sparc/trace.c target/sparc/trace.c-timestamp target/s390x/trace.c target/s390x/trace.c-timestamp target/ppc/trace.c target/ppc/trace.c-timestamp qom/trace.c qom/trace.c-timestamp linux-user/trace.c linux-user/trace.c-timestamp qapi/trace.c qapi/trace.c-timestamp accel/tcg/trace.c accel/tcg/trace.c-timestamp accel/kvm/trace.c accel/kvm/trace.c-timestamp nbd/trace.c nbd/trace.c-timestamp scsi/trace.c scsi/trace.c-timestamp /tmp/qemu-test/build/trace-events-all /tmp/qemu-test/build/trace-events-all-timestamp .git-submodule-status .git-submodule-status-timestamp ui/input-keymap-atset1-to-qcode.c ui/input-keymap-atset1-to-qcode.c-timestamp ui/input-keymap-linux-to-qcode.c ui/input-keymap-linux-to-qcode.c-timestamp ui/input-keymap-qcode-to-atset1.c ui/input-keymap-qcode-to-atset1.c-timestamp ui/input-keymap-qcode-to-atset2.c ui/input-keymap-qcode-to-atset2.c-timestamp ui/input-keymap-qcode-to-atset3.c ui/input-keymap-qcode-to-atset3.c-timestamp ui/input-keymap-qcode-to-linux.c ui/input-keymap-qcode-to-linux.c-timestamp ui/input-keymap-qcode-to-qnum.c ui/input-keymap-qcode-to-qnum.c-timestamp ui/input-keymap-qcode-to-sun.c ui/input-keymap-qcode-to-sun.c-timestamp ui/input-keymap-qnum-to-qcode.c ui/input-keymap-qnum-to-qcode.c-timestamp ui/input-keymap-usb-to-qcode.c ui/input-keymap-usb-to-qcode.c-timestamp ui/input-keymap-win32-to-qcode.c ui/input-keymap-win32-to-qcode.c-timestamp ui/input-keymap-x11-to-qcode.c ui/input-keymap-x11-to-qcode.c-timestamp ui/input-keymap-xorgevdev-to-qcode.c ui/input-keymap-xorgevdev-to-qcode.c-timestamp ui/input-keymap-xorgkbd-to-qcode.c ui/input-keymap-xorgkbd-to-qcode.c-timestamp ui/input-keymap-xorgxquartz-to-qcode.c ui/input-keymap-xorgxquartz-to-qcode.c-timestamp ui/input-keymap-xorgxwin-to-qcode.c ui/input-keymap-xorgxwin-to-qcode.c-timestamp tests/test-qapi-types.h tests/test-qapi-types.h-timestamp tests/test-qapi-visit.h tests/test-qapi-visit.h-timestamp tests/test-qapi-commands.h tests/test-qapi-commands.h-timestamp tests/test-qapi-events.h tests/test-qapi-events.h-timestamp tests/test-qapi-introspect.h tests/test-qapi-introspect.h-timestamp
rm -f qapi-gen-timestamp
rm -rf qga/qapi-generated
for d in x86_64-softmmu aarch64-softmmu pc-bios/optionrom; do \
if test -d $d; then make -C $d clean || exit 1; fi; \
rm -f $d/qemu-options.def; \
done
make[1]: Entering directory '/tmp/qemu-test/build/x86_64-softmmu'
rm -f *.timestamp
rm -f *.a *~ qemu-system-x86_64.exe qemu-system-x86_64w.exe
rm -f
rm -f hmp-commands.h gdbstub-xml.c
make[1]: Leaving directory '/tmp/qemu-test/build/x86_64-softmmu'
make[1]: Entering directory '/tmp/qemu-test/build/aarch64-softmmu'
rm -f *.timestamp
rm -f *.a *~ qemu-system-aarch64.exe qemu-system-aarch64w.exe
rm -f
rm -f hmp-commands.h gdbstub-xml.c
make[1]: Leaving directory '/tmp/qemu-test/build/aarch64-softmmu'
make[1]: Entering directory '/tmp/qemu-test/build/pc-bios/optionrom'
rm -f *.timestamp
rm -f *.o *.d *.raw *.img *.bin *~
make[1]: Leaving directory '/tmp/qemu-test/build/pc-bios/optionrom'
rm -f x86_64-softmmu/config-devices.mak aarch64-softmmu/config-devices.mak config-all-devices.mak
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=i686-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=1.2 --with-gtkabi=2.0
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install
local state directory queried at runtime
Windows SDK no
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules
C compiler i686-w64-mingw32-gcc
Host C compiler cc
C++ compiler i686-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/i686-w64-mingw32/sys-root/mingw/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -mms-bitfields -I/usr/i686-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/i686-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/i686-w64-mingw32/sys-root/mingw/include -m32 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/i686-w64-mingw32/sys-root/mingw/include -I/usr/i686-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/usr/i686-w64-mingw32/sys-root/mingw/include -I/usr/i686-w64-mingw32/sys-root/mingw/include -I/usr/i686-w64-mingw32/sys-root/mingw/include/libpng16
LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m32 -g
QEMU_LDFLAGS -L$(BUILD_DIR)/dtc/libfdt
make make
install install
python /usr/bin/python3 -B
smbd /usr/sbin/smbd
module support no
host CPU i386
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (1.2.15)
GTK support yes (2.24.31)
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support yes
GNUTLS rnd yes
libgcrypt no
libgcrypt kdf no
nettle yes (3.3)
nettle kdf yes
libtasn1 yes
curses support no
virgl support no
curl support yes
mingw32 support yes
Audio drivers dsound
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support yes
VNC PNG support yes
xen support no
brlapi support no
bluez support no
Documentation no
PIE no
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support no
Install blobs yes
KVM support no
HAX support yes
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support no
RDMA support no
fdt support git
membarrier no
preadv support no
fdatasync no
madvise no
posix_madvise no
posix_memalign no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends simple
Trace output file trace-<pid>
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info yes
QGA MSI support no
seccomp support no
coroutine backend win32
coroutine pool yes
debug stack usage no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support yes
TPM passthrough no
TPM emulator no
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone no
WARNING: Use of GTK 2.0 is deprecated and will be removed in
WARNING: future releases. Please switch to using GTK 3.0
WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0
GEN x86_64-softmmu/config-devices.mak.tmp
GEN config-host.h
GEN aarch64-softmmu/config-devices.mak.tmp
GEN qapi-gen
GEN qemu-options.def
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
GEN x86_64-softmmu/config-devices.mak
GEN module_block.h
GEN trace/generated-helpers.c
GEN aarch64-softmmu/config-devices.mak
GEN ui/input-keymap-atset1-to-qcode.c
GEN ui/input-keymap-linux-to-qcode.c
GEN ui/input-keymap-qcode-to-atset1.c
GEN ui/input-keymap-qcode-to-atset3.c
GEN ui/input-keymap-qcode-to-atset2.c
GEN ui/input-keymap-qcode-to-linux.c
GEN ui/input-keymap-qcode-to-qnum.c
GEN ui/input-keymap-qcode-to-sun.c
GEN ui/input-keymap-qnum-to-qcode.c
GEN ui/input-keymap-usb-to-qcode.c
GEN ui/input-keymap-win32-to-qcode.c
GEN ui/input-keymap-x11-to-qcode.c
GEN ui/input-keymap-xorgevdev-to-qcode.c
GEN ui/input-keymap-xorgkbd-to-qcode.c
GEN ui/input-keymap-xorgxquartz-to-qcode.c
GEN ui/input-keymap-xorgxwin-to-qcode.c
GEN tests/test-qapi-gen
GEN trace-root.h
GEN util/trace.h
GEN crypto/trace.h
GEN io/trace.h
GEN migration/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/intc/trace.h
GEN hw/net/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/virtio/trace.h
GEN hw/audio/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/usb/trace.h
GEN hw/scsi/trace.h
GEN hw/nvram/trace.h
GEN hw/display/trace.h
GEN hw/input/trace.h
GEN hw/timer/trace.h
GEN hw/dma/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/sd/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/9pfs/trace.h
GEN hw/ppc/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/s390x/trace.h
GEN hw/vfio/trace.h
GEN hw/acpi/trace.h
GEN hw/arm/trace.h
GEN hw/alpha/trace.h
GEN hw/hppa/trace.h
GEN hw/xen/trace.h
GEN hw/ide/trace.h
GEN hw/tpm/trace.h
GEN ui/trace.h
GEN audio/trace.h
GEN net/trace.h
GEN target/arm/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/sparc/trace.h
GEN target/s390x/trace.h
GEN target/ppc/trace.h
GEN qom/trace.h
GEN linux-user/trace.h
GEN qapi/trace.h
GEN accel/tcg/trace.h
GEN accel/kvm/trace.h
GEN nbd/trace.h
GEN scsi/trace.h
GEN trace-root.c
GEN util/trace.c
GEN crypto/trace.c
GEN io/trace.c
GEN migration/trace.c
GEN block/trace.c
GEN chardev/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/intc/trace.c
GEN hw/net/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/virtio/trace.c
GEN hw/audio/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/usb/trace.c
GEN hw/scsi/trace.c
GEN hw/nvram/trace.c
GEN hw/display/trace.c
GEN hw/input/trace.c
GEN hw/timer/trace.c
GEN hw/dma/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/sd/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/9pfs/trace.c
GEN hw/ppc/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/s390x/trace.c
GEN hw/vfio/trace.c
GEN hw/acpi/trace.c
GEN hw/arm/trace.c
GEN hw/alpha/trace.c
GEN hw/hppa/trace.c
GEN hw/xen/trace.c
GEN hw/ide/trace.c
GEN hw/tpm/trace.c
GEN ui/trace.c
GEN audio/trace.c
GEN net/trace.c
GEN target/arm/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/sparc/trace.c
GEN target/s390x/trace.c
GEN target/ppc/trace.c
GEN qom/trace.c
GEN linux-user/trace.c
GEN qapi/trace.c
GEN accel/tcg/trace.c
GEN accel/kvm/trace.c
GEN nbd/trace.c
GEN scsi/trace.c
GEN config-all-devices.mak
DEP /tmp/qemu-test/src/dtc/tests/dumptrees.c
DEP /tmp/qemu-test/src/dtc/tests/trees.S
DEP /tmp/qemu-test/src/dtc/tests/testutils.c
DEP /tmp/qemu-test/src/dtc/tests/value-labels.c
DEP /tmp/qemu-test/src/dtc/tests/asm_tree_dump.c
DEP /tmp/qemu-test/src/dtc/tests/truncated_property.c
DEP /tmp/qemu-test/src/dtc/tests/check_path.c
DEP /tmp/qemu-test/src/dtc/tests/overlay_bad_fixup.c
DEP /tmp/qemu-test/src/dtc/tests/overlay.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/property_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/integer-expressions.c
DEP /tmp/qemu-test/src/dtc/tests/utilfdt_test.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset_aliases.c
DEP /tmp/qemu-test/src/dtc/tests/add_subnode_with_nops.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_unordered.c
DEP /tmp/qemu-test/src/dtc/tests/dtb_reverse.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_ordered.c
DEP /tmp/qemu-test/src/dtc/tests/extra-terminating-null.c
DEP /tmp/qemu-test/src/dtc/tests/incbin.c
DEP /tmp/qemu-test/src/dtc/tests/boot-cpuid.c
DEP /tmp/qemu-test/src/dtc/tests/phandle_format.c
DEP /tmp/qemu-test/src/dtc/tests/path-references.c
DEP /tmp/qemu-test/src/dtc/tests/references.c
DEP /tmp/qemu-test/src/dtc/tests/string_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/propname_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop2.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop1.c
DEP /tmp/qemu-test/src/dtc/tests/del_node.c
DEP /tmp/qemu-test/src/dtc/tests/del_property.c
DEP /tmp/qemu-test/src/dtc/tests/setprop.c
DEP /tmp/qemu-test/src/dtc/tests/set_name.c
DEP /tmp/qemu-test/src/dtc/tests/rw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/open_pack.c
DEP /tmp/qemu-test/src/dtc/tests/nopulate.c
DEP /tmp/qemu-test/src/dtc/tests/mangle-layout.c
DEP /tmp/qemu-test/src/dtc/tests/move_and_save.c
DEP /tmp/qemu-test/src/dtc/tests/sw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/nop_node.c
DEP /tmp/qemu-test/src/dtc/tests/nop_property.c
DEP /tmp/qemu-test/src/dtc/tests/stringlist.c
DEP /tmp/qemu-test/src/dtc/tests/setprop_inplace.c
DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells.c
DEP /tmp/qemu-test/src/dtc/tests/notfound.c
DEP /tmp/qemu-test/src/dtc/tests/sized_cells.c
DEP /tmp/qemu-test/src/dtc/tests/get_alias.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/char_literal.c
DEP /tmp/qemu-test/src/dtc/tests/node_check_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_prop_value.c
DEP /tmp/qemu-test/src/dtc/tests/parent_offset.c
DEP /tmp/qemu-test/src/dtc/tests/get_path.c
DEP /tmp/qemu-test/src/dtc/tests/supernode_atdepth_offset.c
DEP /tmp/qemu-test/src/dtc/tests/get_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/getprop.c
DEP /tmp/qemu-test/src/dtc/tests/get_name.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_offset.c
DEP /tmp/qemu-test/src/dtc/tests/find_property.c
DEP /tmp/qemu-test/src/dtc/tests/root_node.c
DEP /tmp/qemu-test/src/dtc/tests/get_mem_rsv.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_overlay.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_addresses.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_empty_tree.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_strerror.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_rw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_sw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_ro.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_wip.c
DEP /tmp/qemu-test/src/dtc/util.c
DEP /tmp/qemu-test/src/dtc/fdtoverlay.c
DEP /tmp/qemu-test/src/dtc/fdtput.c
DEP /tmp/qemu-test/src/dtc/fdtget.c
DEP /tmp/qemu-test/src/dtc/fdtdump.c
DEP convert-dtsv0-lexer.lex.c
DEP /tmp/qemu-test/src/dtc/srcpos.c
DEP dtc-parser.tab.c
DEP dtc-lexer.lex.c
DEP /tmp/qemu-test/src/dtc/treesource.c
DEP /tmp/qemu-test/src/dtc/livetree.c
DEP /tmp/qemu-test/src/dtc/fstree.c
DEP /tmp/qemu-test/src/dtc/flattree.c
DEP /tmp/qemu-test/src/dtc/dtc.c
DEP /tmp/qemu-test/src/dtc/data.c
DEP /tmp/qemu-test/src/dtc/checks.c
CC libfdt/fdt.o
CC libfdt/fdt_ro.o
CC libfdt/fdt_sw.o
CC libfdt/fdt_wip.o
CC libfdt/fdt_rw.o
CC libfdt/fdt_strerror.o
CC libfdt/fdt_addresses.o
CC libfdt/fdt_empty_tree.o
CC libfdt/fdt_overlay.o
AR libfdt/libfdt.a
i686-w64-mingw32-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
RC version.o
GEN qga/qapi-generated/qapi-gen
CC qapi/qapi-builtin-types.o
CC qapi/qapi-types-block-core.o
CC qapi/qapi-types.o
CC qapi/qapi-types-block.o
CC qapi/qapi-types-char.o
CC qapi/qapi-types-crypto.o
CC qapi/qapi-types-common.o
CC qapi/qapi-types-introspect.o
CC qapi/qapi-types-migration.o
CC qapi/qapi-types-misc.o
CC qapi/qapi-types-net.o
CC qapi/qapi-types-rocker.o
CC qapi/qapi-types-sockets.o
CC qapi/qapi-types-run-state.o
CC qapi/qapi-types-tpm.o
CC qapi/qapi-types-trace.o
CC qapi/qapi-types-transaction.o
CC qapi/qapi-types-ui.o
CC qapi/qapi-builtin-visit.o
CC qapi/qapi-visit.o
CC qapi/qapi-visit-block-core.o
CC qapi/qapi-visit-block.o
CC qapi/qapi-visit-char.o
CC qapi/qapi-visit-common.o
CC qapi/qapi-visit-crypto.o
CC qapi/qapi-visit-introspect.o
CC qapi/qapi-visit-migration.o
CC qapi/qapi-visit-misc.o
CC qapi/qapi-visit-net.o
CC qapi/qapi-visit-rocker.o
CC qapi/qapi-visit-run-state.o
CC qapi/qapi-visit-sockets.o
CC qapi/qapi-visit-tpm.o
CC qapi/qapi-visit-trace.o
CC qapi/qapi-visit-transaction.o
CC qapi/qapi-visit-ui.o
CC qapi/qapi-events.o
CC qapi/qapi-events-block-core.o
CC qapi/qapi-events-block.o
CC qapi/qapi-events-char.o
CC qapi/qapi-events-common.o
CC qapi/qapi-events-crypto.o
CC qapi/qapi-events-introspect.o
CC qapi/qapi-events-migration.o
CC qapi/qapi-events-misc.o
CC qapi/qapi-events-net.o
CC qapi/qapi-events-rocker.o
CC qapi/qapi-events-run-state.o
CC qapi/qapi-events-sockets.o
CC qapi/qapi-events-trace.o
CC qapi/qapi-events-transaction.o
CC qapi/qapi-events-ui.o
CC qapi/qapi-introspect.o
CC qapi/qapi-events-tpm.o
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qnum.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qbool.o
CC qobject/qlist.o
CC qobject/qlit.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
CC trace/simple.o
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/lockcnt.o
CC util/aiocb.o
CC util/async.o
CC util/aio-wait.o
CC util/thread-pool.o
CC util/main-loop.o
CC util/qemu-timer.o
CC util/iohandler.o
CC util/aio-win32.o
CC util/event_notifier-win32.o
CC util/oslib-win32.o
CC util/qemu-thread-win32.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/host-utils.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/fifo8.o
CC util/acl.o
CC util/cacheinfo.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/keyval.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/hexdump.o
CC util/getauxval.o
CC util/readline.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/rcu.o
CC util/qemu-coroutine-io.o
CC util/coroutine-win32.o
CC util/qemu-coroutine-sleep.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/pagesize.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC util/stats64.o
CC util/systemd.o
CC trace-root.o
CC util/trace.o
CC crypto/trace.o
CC io/trace.o
CC migration/trace.o
CC block/trace.o
CC chardev/trace.o
CC hw/block/trace.o
CC hw/block/dataplane/trace.o
CC hw/char/trace.o
CC hw/intc/trace.o
CC hw/net/trace.o
CC hw/rdma/trace.o
CC hw/rdma/vmw/trace.o
CC hw/virtio/trace.o
CC hw/audio/trace.o
CC hw/misc/trace.o
CC hw/misc/macio/trace.o
CC hw/scsi/trace.o
CC hw/usb/trace.o
CC hw/nvram/trace.o
CC hw/display/trace.o
CC hw/input/trace.o
CC hw/timer/trace.o
CC hw/dma/trace.o
CC hw/sparc/trace.o
CC hw/sparc64/trace.o
CC hw/sd/trace.o
CC hw/isa/trace.o
CC hw/mem/trace.o
CC hw/i386/trace.o
CC hw/i386/xen/trace.o
CC hw/9pfs/trace.o
CC hw/ppc/trace.o
CC hw/pci/trace.o
CC hw/pci-host/trace.o
CC hw/s390x/trace.o
CC hw/vfio/trace.o
CC hw/acpi/trace.o
CC hw/arm/trace.o
CC hw/alpha/trace.o
CC hw/hppa/trace.o
CC hw/xen/trace.o
CC hw/ide/trace.o
CC hw/tpm/trace.o
CC ui/trace.o
CC net/trace.o
CC audio/trace.o
CC target/arm/trace.o
CC target/i386/trace.o
CC target/sparc/trace.o
CC target/mips/trace.o
CC target/s390x/trace.o
CC target/ppc/trace.o
CC qom/trace.o
CC linux-user/trace.o
CC qapi/trace.o
CC accel/tcg/trace.o
CC accel/kvm/trace.o
CC nbd/trace.o
CC scsi/trace.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-icount.o
CC stubs/cpu-get-clock.o
CC stubs/dump.o
CC stubs/error-printf.o
CC stubs/fdset.o
CC stubs/gdbstub.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/is-daemonized.o
CC stubs/iothread-lock.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/change-state-handler.o
CC stubs/monitor.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/tpm.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/fd-register.o
CC stubs/qmp_memory_device.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/vmgenid.o
CC stubs/xen-common.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
CC stubs/ram-block.o
GEN qemu-img-cmds.h
CC block.o
CC qemu-io-cmds.o
CC blockjob.o
CC replication.o
CC block/raw-format.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qcow2-bitmap.o
CC block/qed.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/vhdx-log.o
CC block/quorum.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/file-win32.o
CC block/win32-aio.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
CC block/io.o
CC block/create.o
CC block/throttle-groups.o
CC block/nbd.o
CC block/nbd-client.o
CC block/sheepdog.o
CC block/accounting.o
CC block/dirty-bitmap.o
CC block/write-threshold.o
CC block/backup.o
CC block/replication.o
CC block/throttle.o
CC block/copy-on-read.o
CC block/crypto.o
CC nbd/server.o
CC nbd/client.o
CC nbd/common.o
CC scsi/utils.o
CC block/curl.o
CC block/ssh.o
CC block/dmg-bz2.o
CC crypto/init.o
CC crypto/hash.o
CC crypto/hash-nettle.o
CC crypto/hmac.o
CC crypto/hmac-nettle.o
CC crypto/aes.o
CC crypto/desrfb.o
CC crypto/cipher.o
CC crypto/tlscreds.o
CC crypto/tlscredsanon.o
CC crypto/tlscredsx509.o
CC crypto/tlssession.o
CC crypto/secret.o
CC crypto/random-gnutls.o
CC crypto/pbkdf.o
CC crypto/pbkdf-nettle.o
CC crypto/ivgen.o
CC crypto/ivgen-essiv.o
CC crypto/ivgen-plain.o
CC crypto/ivgen-plain64.o
CC crypto/afsplit.o
CC crypto/xts.o
CC crypto/block.o
CC crypto/block-qcow.o
CC crypto/block-luks.o
CC io/channel.o
CC io/channel-buffer.o
CC io/channel-command.o
CC io/channel-file.o
CC io/channel-socket.o
CC io/channel-tls.o
CC io/channel-watch.o
CC io/channel-websock.o
CC io/channel-util.o
CC io/dns-resolver.o
CC io/net-listener.o
CC io/task.o
CC qom/object.o
CC qom/container.o
CC qom/qom-qobject.o
CC qom/object_interfaces.o
CC qemu-io.o
CC blockdev.o
CC bootdevice.o
CC blockdev-nbd.o
CC iothread.o
CC qdev-monitor.o
CC device-hotplug.o
CC os-win32.o
CC bt-host.o
CC bt-vhci.o
CC dma-helpers.o
CC vl.o
CC tpm.o
CC device_tree.o
CC qapi/qapi-commands.o
CC qapi/qapi-commands-block-core.o
CC qapi/qapi-commands-block.o
CC qapi/qapi-commands-char.o
CC qapi/qapi-commands-common.o
CC qapi/qapi-commands-crypto.o
CC qapi/qapi-commands-introspect.o
CC qapi/qapi-commands-migration.o
CC qapi/qapi-commands-misc.o
CC qapi/qapi-commands-net.o
CC qapi/qapi-commands-rocker.o
CC qapi/qapi-commands-run-state.o
CC qapi/qapi-commands-sockets.o
CC qapi/qapi-commands-tpm.o
CC qapi/qapi-commands-trace.o
CC qapi/qapi-commands-transaction.o
CC qapi/qapi-commands-ui.o
CC qmp.o
CC hmp.o
CC cpus-common.o
CC audio/audio.o
CC audio/noaudio.o
CC audio/wavaudio.o
CC audio/mixeng.o
CC audio/dsoundaudio.o
CC audio/audio_win_int.o
CC audio/wavcapture.o
CC backends/rng.o
CC backends/rng-egd.o
CC backends/tpm.o
CC backends/hostmem.o
CC backends/hostmem-ram.o
CC backends/cryptodev.o
CC backends/cryptodev-builtin.o
CC backends/cryptodev-vhost.o
CC block/stream.o
CC chardev/msmouse.o
CC chardev/wctablet.o
CC chardev/testdev.o
CC disas/arm.o
CXX disas/arm-a64.o
CC disas/i386.o
CXX disas/libvixl/vixl/utils.o
CXX disas/libvixl/vixl/compiler-intrinsics.o
CXX disas/libvixl/vixl/a64/instructions-a64.o
CXX disas/libvixl/vixl/a64/decoder-a64.o
CXX disas/libvixl/vixl/a64/disasm-a64.o
CC hw/acpi/core.o
CC hw/acpi/piix4.o
CC hw/acpi/pcihp.o
CC hw/acpi/ich9.o
CC hw/acpi/tco.o
CC hw/acpi/cpu_hotplug.o
CC hw/acpi/memory_hotplug.o
CC hw/acpi/cpu.o
CC hw/acpi/nvdimm.o
CC hw/acpi/vmgenid.o
CC hw/acpi/acpi_interface.o
CC hw/acpi/bios-linker-loader.o
CC hw/acpi/aml-build.o
CC hw/acpi/ipmi.o
CC hw/acpi/acpi-stub.o
CC hw/acpi/ipmi-stub.o
CC hw/audio/sb16.o
CC hw/audio/es1370.o
CC hw/audio/ac97.o
CC hw/audio/fmopl.o
CC hw/audio/adlib.o
CC hw/audio/gus.o
CC hw/audio/gusemu_hal.o
CC hw/audio/gusemu_mixer.o
CC hw/audio/cs4231a.o
CC hw/audio/intel-hda.o
CC hw/audio/hda-codec.o
CC hw/audio/pcspk.o
CC hw/audio/wm8750.o
CC hw/audio/pl041.o
CC hw/audio/lm4549.o
CC hw/audio/marvell_88w8618.o
CC hw/audio/soundhw.o
CC hw/block/block.o
CC hw/block/cdrom.o
CC hw/block/hd-geometry.o
CC hw/block/fdc.o
CC hw/block/m25p80.o
CC hw/block/nand.o
CC hw/block/pflash_cfi01.o
CC hw/block/pflash_cfi02.o
CC hw/block/ecc.o
CC hw/block/onenand.o
CC hw/block/nvme.o
CC hw/bt/core.o
CC hw/bt/l2cap.o
CC hw/bt/sdp.o
CC hw/bt/hci.o
CC hw/bt/hid.o
CC hw/bt/hci-csr.o
CC hw/char/ipoctal232.o
CC hw/char/parallel.o
CC hw/char/parallel-isa.o
CC hw/char/pl011.o
CC hw/char/serial.o
CC hw/char/serial-isa.o
CC hw/char/serial-pci.o
CC hw/char/virtio-console.o
CC hw/char/cadence_uart.o
CC hw/char/cmsdk-apb-uart.o
CC hw/char/debugcon.o
CC hw/char/imx_serial.o
CC hw/core/qdev.o
CC hw/core/qdev-properties.o
CC hw/core/bus.o
CC hw/core/reset.o
CC hw/core/qdev-fw.o
CC hw/core/fw-path-provider.o
CC hw/core/irq.o
CC hw/core/hotplug.o
CC hw/core/nmi.o
CC hw/core/stream.o
CC hw/core/ptimer.o
CC hw/core/sysbus.o
CC hw/core/machine.o
CC hw/core/loader.o
CC hw/core/qdev-properties-system.o
CC hw/core/register.o
CC hw/core/or-irq.o
CC hw/core/split-irq.o
CC hw/core/platform-bus.o
CC hw/cpu/core.o
CC hw/display/ads7846.o
CC hw/display/cirrus_vga.o
CC hw/display/pl110.o
CC hw/display/sii9022.o
CC hw/display/ssd0303.o
CC hw/display/ssd0323.o
CC hw/display/vga-pci.o
CC hw/display/vga-isa.o
CC hw/display/vmware_vga.o
CC hw/display/exynos4210_fimd.o
CC hw/display/blizzard.o
CC hw/display/framebuffer.o
CC hw/display/tc6393xb.o
CC hw/dma/pl080.o
CC hw/dma/pl330.o
CC hw/dma/i8257.o
CC hw/dma/xilinx_axidma.o
CC hw/dma/xlnx-zynq-devcfg.o
CC hw/dma/xlnx-zdma.o
CC hw/gpio/max7310.o
CC hw/gpio/pl061.o
CC hw/gpio/zaurus.o
CC hw/gpio/gpio_key.o
CC hw/i2c/core.o
CC hw/i2c/smbus.o
CC hw/i2c/smbus_eeprom.o
CC hw/i2c/i2c-ddc.o
CC hw/i2c/versatile_i2c.o
CC hw/i2c/smbus_ich9.o
CC hw/i2c/pm_smbus.o
CC hw/i2c/bitbang_i2c.o
CC hw/i2c/exynos4210_i2c.o
CC hw/i2c/imx_i2c.o
CC hw/i2c/aspeed_i2c.o
CC hw/ide/core.o
CC hw/ide/atapi.o
CC hw/ide/qdev.o
CC hw/ide/pci.o
CC hw/ide/isa.o
CC hw/ide/piix.o
CC hw/ide/microdrive.o
CC hw/ide/ahci.o
CC hw/ide/ich.o
CC hw/ide/ahci-allwinner.o
CC hw/input/hid.o
CC hw/input/lm832x.o
CC hw/input/pckbd.o
CC hw/input/pl050.o
CC hw/input/ps2.o
CC hw/input/stellaris_input.o
CC hw/input/tsc2005.o
CC hw/input/virtio-input.o
CC hw/input/virtio-input-hid.o
CC hw/intc/i8259_common.o
CC hw/intc/i8259.o
CC hw/intc/pl190.o
CC hw/intc/xlnx-pmu-iomod-intc.o
CC hw/intc/xlnx-zynqmp-ipi.o
CC hw/intc/imx_avic.o
CC hw/intc/imx_gpcv2.o
CC hw/intc/realview_gic.o
CC hw/intc/ioapic_common.o
CC hw/intc/arm_gic_common.o
CC hw/intc/arm_gic.o
CC hw/intc/arm_gicv2m.o
CC hw/intc/arm_gicv3_common.o
CC hw/intc/arm_gicv3.o
CC hw/intc/arm_gicv3_dist.o
CC hw/intc/arm_gicv3_redist.o
CC hw/intc/arm_gicv3_its_common.o
CC hw/intc/intc.o
CC hw/ipack/ipack.o
CC hw/ipack/tpci200.o
CC hw/ipmi/ipmi.o
CC hw/ipmi/ipmi_bmc_sim.o
CC hw/ipmi/ipmi_bmc_extern.o
CC hw/ipmi/isa_ipmi_kcs.o
CC hw/ipmi/isa_ipmi_bt.o
CC hw/isa/isa-bus.o
CC hw/isa/isa-superio.o
CC hw/isa/smc37c669-superio.o
CC hw/isa/apm.o
CC hw/mem/pc-dimm.o
CC hw/mem/memory-device.o
CC hw/mem/nvdimm.o
CC hw/misc/applesmc.o
CC hw/misc/max111x.o
CC hw/misc/tmp421.o
CC hw/misc/tmp105.o
CC hw/misc/debugexit.o
CC hw/misc/sga.o
CC hw/misc/pci-testdev.o
CC hw/misc/pc-testdev.o
CC hw/misc/edu.o
CC hw/misc/unimp.o
CC hw/misc/vmcoreinfo.o
CC hw/misc/arm_l2x0.o
CC hw/misc/arm_integrator_debug.o
CC hw/misc/a9scu.o
CC hw/misc/arm11scu.o
CC hw/net/ne2000.o
CC hw/net/eepro100.o
CC hw/net/pcnet-pci.o
CC hw/net/pcnet.o
CC hw/net/e1000.o
CC hw/net/e1000x_common.o
CC hw/net/net_tx_pkt.o
CC hw/net/e1000e.o
CC hw/net/net_rx_pkt.o
CC hw/net/e1000e_core.o
CC hw/net/rtl8139.o
CC hw/net/vmxnet3.o
CC hw/net/smc91c111.o
CC hw/net/lan9118.o
CC hw/net/ne2000-isa.o
CC hw/net/xgmac.o
CC hw/net/xilinx_axienet.o
CC hw/net/allwinner_emac.o
CC hw/net/imx_fec.o
CC hw/net/cadence_gem.o
CC hw/net/stellaris_enet.o
CC hw/net/ftgmac100.o
CC hw/net/rocker/rocker.o
CC hw/net/rocker/rocker_fp.o
CC hw/net/rocker/rocker_desc.o
CC hw/net/rocker/rocker_world.o
CC hw/net/rocker/rocker_of_dpa.o
CC hw/net/can/can_sja1000.o
CC hw/net/can/can_kvaser_pci.o
CC hw/net/can/can_pcm3680_pci.o
CC hw/net/can/can_mioe3680_pci.o
CC hw/nvram/eeprom93xx.o
CC hw/nvram/eeprom_at24c.o
CC hw/nvram/fw_cfg.o
CC hw/nvram/chrp_nvram.o
CC hw/pci-bridge/pci_bridge_dev.o
CC hw/pci-bridge/pcie_root_port.o
CC hw/pci-bridge/gen_pcie_root_port.o
CC hw/pci-bridge/pcie_pci_bridge.o
CC hw/pci-bridge/pci_expander_bridge.o
CC hw/pci-bridge/xio3130_upstream.o
CC hw/pci-bridge/xio3130_downstream.o
CC hw/pci-bridge/ioh3420.o
CC hw/pci-bridge/i82801b11.o
CC hw/pci-host/pam.o
CC hw/pci-host/versatile.o
CC hw/pci-host/piix.o
CC hw/pci-host/q35.o
CC hw/pci-host/gpex.o
CC hw/pci-host/designware.o
CC hw/pci/pci.o
CC hw/pci/pci_bridge.o
CC hw/pci/msix.o
CC hw/pci/msi.o
CC hw/pci/shpc.o
CC hw/pci/slotid_cap.o
CC hw/pci/pci_host.o
CC hw/pci/pcie_host.o
CC hw/pci/pcie.o
CC hw/pci/pcie_aer.o
CC hw/pci/pcie_port.o
CC hw/pci/pci-stub.o
CC hw/pcmcia/pcmcia.o
CC hw/scsi/scsi-disk.o
CC hw/scsi/scsi-generic.o
CC hw/scsi/scsi-bus.o
CC hw/scsi/lsi53c895a.o
CC hw/scsi/mptsas.o
CC hw/scsi/mptconfig.o
CC hw/scsi/mptendian.o
CC hw/scsi/megasas.o
CC hw/scsi/vmw_pvscsi.o
CC hw/scsi/esp.o
CC hw/scsi/esp-pci.o
CC hw/sd/pl181.o
CC hw/sd/ssi-sd.o
CC hw/sd/sd.o
CC hw/sd/core.o
CC hw/sd/sdmmc-internal.o
CC hw/sd/sdhci.o
CC hw/smbios/smbios.o
CC hw/smbios/smbios_type_38.o
CC hw/smbios/smbios-stub.o
CC hw/smbios/smbios_type_38-stub.o
CC hw/ssi/pl022.o
CC hw/ssi/ssi.o
CC hw/ssi/xilinx_spips.o
CC hw/ssi/aspeed_smc.o
CC hw/ssi/stm32f2xx_spi.o
CC hw/ssi/mss-spi.o
CC hw/timer/arm_timer.o
CC hw/timer/armv7m_systick.o
CC hw/timer/arm_mptimer.o
CC hw/timer/a9gtimer.o
CC hw/timer/cadence_ttc.o
CC hw/timer/ds1338.o
CC hw/timer/hpet.o
CC hw/timer/i8254_common.o
CC hw/timer/i8254.o
CC hw/timer/pl031.o
CC hw/timer/twl92230.o
CC hw/timer/imx_epit.o
CC hw/timer/imx_gpt.o
CC hw/timer/xlnx-zynqmp-rtc.o
CC hw/timer/stm32f2xx_timer.o
CC hw/timer/aspeed_timer.o
CC hw/timer/cmsdk-apb-timer.o
CC hw/timer/mss-timer.o
CC hw/tpm/tpm_util.o
CC hw/tpm/tpm_tis.o
CC hw/tpm/tpm_crb.o
CC hw/usb/core.o
CC hw/usb/combined-packet.o
CC hw/usb/bus.o
CC hw/usb/libhw.o
CC hw/usb/desc.o
CC hw/usb/desc-msos.o
CC hw/usb/hcd-uhci.o
CC hw/usb/hcd-ohci.o
CC hw/usb/hcd-ehci.o
CC hw/usb/hcd-ehci-pci.o
CC hw/usb/hcd-ehci-sysbus.o
CC hw/usb/hcd-xhci.o
CC hw/usb/hcd-xhci-nec.o
CC hw/usb/hcd-musb.o
CC hw/usb/dev-hub.o
CC hw/usb/dev-hid.o
CC hw/usb/dev-wacom.o
CC hw/usb/dev-storage.o
CC hw/usb/dev-uas.o
CC hw/usb/dev-audio.o
CC hw/usb/dev-serial.o
CC hw/usb/dev-network.o
CC hw/usb/dev-bluetooth.o
CC hw/usb/dev-smartcard-reader.o
CC hw/usb/host-stub.o
CC hw/virtio/virtio-rng.o
CC hw/virtio/virtio-pci.o
CC hw/virtio/virtio-bus.o
CC hw/virtio/virtio-mmio.o
CC hw/virtio/vhost-stub.o
CC hw/watchdog/watchdog.o
CC hw/watchdog/wdt_i6300esb.o
CC hw/watchdog/wdt_ib700.o
CC hw/watchdog/wdt_aspeed.o
CC migration/migration.o
CC migration/socket.o
CC migration/fd.o
CC migration/exec.o
CC migration/tls.o
CC migration/channel.o
CC migration/savevm.o
CC migration/colo-comm.o
CC migration/colo.o
CC migration/colo-failover.o
CC migration/vmstate.o
CC migration/vmstate-types.o
CC migration/page_cache.o
CC migration/qemu-file.o
CC migration/global_state.o
CC migration/qemu-file-channel.o
CC migration/xbzrle.o
CC migration/postcopy-ram.o
CC migration/qjson.o
CC migration/block-dirty-bitmap.o
CC migration/block.o
CC net/net.o
CC net/queue.o
CC net/checksum.o
CC net/util.o
CC net/hub.o
CC net/socket.o
CC net/dump.o
CC net/eth.o
CC net/slirp.o
CC net/filter.o
CC net/filter-buffer.o
CC net/filter-mirror.o
CC net/colo-compare.o
CC net/colo.o
CC net/filter-rewriter.o
CC net/filter-replay.o
CC net/tap-win32.o
CC net/can/can_core.o
CC net/can/can_host.o
CC qom/cpu.o
CC replay/replay.o
CC replay/replay-internal.o
CC replay/replay-events.o
CC replay/replay-time.o
CC replay/replay-input.o
CC replay/replay-char.o
CC replay/replay-snapshot.o
CC replay/replay-net.o
CC replay/replay-audio.o
CC slirp/cksum.o
CC slirp/if.o
CC slirp/ip_icmp.o
CC slirp/ip6_icmp.o
CC slirp/ip6_input.o
CC slirp/ip6_output.o
CC slirp/ip_input.o
CC slirp/ip_output.o
CC slirp/dnssearch.o
CC slirp/dhcpv6.o
CC slirp/slirp.o
CC slirp/mbuf.o
CC slirp/misc.o
CC slirp/sbuf.o
CC slirp/socket.o
CC slirp/tcp_input.o
CC slirp/tcp_output.o
CC slirp/tcp_subr.o
CC slirp/tcp_timer.o
CC slirp/udp.o
CC slirp/udp6.o
CC slirp/bootp.o
CC slirp/tftp.o
CC slirp/arp_table.o
CC slirp/ndp_table.o
CC slirp/ncsi.o
CC ui/keymaps.o
CC ui/console.o
CC ui/cursor.o
CC ui/qemu-pixman.o
CC ui/input.o
CC ui/input-keymap.o
CC ui/input-legacy.o
CC ui/vnc.o
CC ui/vnc-enc-zlib.o
CC ui/vnc-enc-hextile.o
CC ui/vnc-enc-tight.o
CC ui/vnc-palette.o
CC ui/vnc-enc-zrle.o
CC ui/vnc-auth-vencrypt.o
CC ui/vnc-ws.o
CC ui/vnc-jobs.o
CC ui/sdl.o
CC ui/sdl_zoom.o
CC ui/gtk.o
CC chardev/char-console.o
CC chardev/char.o
CC chardev/char-file.o
CC chardev/char-fe.o
CC chardev/char-io.o
CC chardev/char-mux.o
CC chardev/char-null.o
CC chardev/char-pipe.o
CC chardev/char-ringbuf.o
CC chardev/char-serial.o
CC chardev/char-socket.o
CC chardev/char-udp.o
CC chardev/char-stdio.o
CC chardev/char-win.o
CC chardev/char-win-stdio.o
CC qga/commands.o
CC qga/guest-agent-command-state.o
CC qga/main.o
CC qga/commands-win32.o
CC qga/channel-win32.o
CC qga/service-win32.o
CC qga/vss-win32.o
CC qga/qapi-generated/qga-qapi-types.o
CC qga/qapi-generated/qga-qapi-visit.o
CC qga/qapi-generated/qga-qapi-commands.o
AR libqemuutil.a
AS optionrom/multiboot.o
AS optionrom/linuxboot.o
CC optionrom/linuxboot_dma.o
AS optionrom/kvmvapic.o
CC qemu-img.o
BUILD optionrom/multiboot.img
BUILD optionrom/linuxboot_dma.img
BUILD optionrom/multiboot.raw
BUILD optionrom/linuxboot_dma.raw
BUILD optionrom/linuxboot.img
BUILD optionrom/linuxboot.raw
BUILD optionrom/kvmvapic.img
BUILD optionrom/kvmvapic.raw
SIGN optionrom/multiboot.bin
SIGN optionrom/linuxboot_dma.bin
SIGN optionrom/linuxboot.bin
SIGN optionrom/kvmvapic.bin
LINK qemu-io.exe
LINK qemu-img.exe
LINK qemu-ga.exe
GEN x86_64-softmmu/hmp-commands.h
GEN x86_64-softmmu/config-target.h
CC x86_64-softmmu/exec.o
CC x86_64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/tcg/tcg.o
CC x86_64-softmmu/tcg/optimize.o
CC x86_64-softmmu/tcg/tcg-op-vec.o
CC x86_64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/tcg/tcg-op-gvec.o
GEN aarch64-softmmu/hmp-commands.h
CC x86_64-softmmu/fpu/softfloat.o
GEN aarch64-softmmu/config-target.h
CC x86_64-softmmu/disas.o
GEN x86_64-softmmu/gdbstub-xml.c
CC x86_64-softmmu/arch_init.o
CC x86_64-softmmu/cpus.o
CC x86_64-softmmu/monitor.o
CC x86_64-softmmu/gdbstub.o
CC x86_64-softmmu/balloon.o
CC x86_64-softmmu/ioport.o
CC x86_64-softmmu/numa.o
CC x86_64-softmmu/qtest.o
CC aarch64-softmmu/exec.o
CC aarch64-softmmu/tcg/tcg.o
CC aarch64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/memory.o
CC aarch64-softmmu/tcg/tcg-op-vec.o
CC x86_64-softmmu/memory_mapping.o
CC aarch64-softmmu/tcg/tcg-op-gvec.o
CC x86_64-softmmu/dump.o
CC aarch64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/migration/ram.o
CC aarch64-softmmu/tcg/optimize.o
CC aarch64-softmmu/fpu/softfloat.o
CC aarch64-softmmu/disas.o
GEN aarch64-softmmu/gdbstub-xml.c
CC aarch64-softmmu/arch_init.o
CC aarch64-softmmu/cpus.o
CC aarch64-softmmu/monitor.o
CC x86_64-softmmu/accel/accel.o
CC aarch64-softmmu/gdbstub.o
CC aarch64-softmmu/balloon.o
CC aarch64-softmmu/ioport.o
CC aarch64-softmmu/numa.o
CC aarch64-softmmu/qtest.o
CC aarch64-softmmu/memory.o
CC x86_64-softmmu/accel/stubs/hvf-stub.o
CC x86_64-softmmu/accel/stubs/whpx-stub.o
CC aarch64-softmmu/memory_mapping.o
CC aarch64-softmmu/dump.o
CC x86_64-softmmu/accel/stubs/kvm-stub.o
CC x86_64-softmmu/accel/tcg/tcg-all.o
CC x86_64-softmmu/accel/tcg/cputlb.o
CC x86_64-softmmu/accel/tcg/tcg-runtime.o
CC x86_64-softmmu/accel/tcg/tcg-runtime-gvec.o
CC x86_64-softmmu/accel/tcg/cpu-exec.o
CC x86_64-softmmu/accel/tcg/cpu-exec-common.o
CC x86_64-softmmu/accel/tcg/translate-all.o
CC x86_64-softmmu/accel/tcg/translator.o
CC x86_64-softmmu/hw/block/virtio-blk.o
CC x86_64-softmmu/hw/block/dataplane/virtio-blk.o
CC aarch64-softmmu/accel/accel.o
CC x86_64-softmmu/hw/char/virtio-serial-bus.o
CC aarch64-softmmu/migration/ram.o
CC aarch64-softmmu/accel/stubs/hax-stub.o
CC aarch64-softmmu/accel/stubs/hvf-stub.o
CC x86_64-softmmu/hw/core/generic-loader.o
CC aarch64-softmmu/accel/stubs/whpx-stub.o
CC aarch64-softmmu/accel/stubs/kvm-stub.o
CC x86_64-softmmu/hw/core/null-machine.o
CC aarch64-softmmu/accel/tcg/tcg-all.o
CC x86_64-softmmu/hw/display/vga.o
CC x86_64-softmmu/hw/display/virtio-gpu.o
CC aarch64-softmmu/accel/tcg/cputlb.o
CC aarch64-softmmu/accel/tcg/tcg-runtime.o
CC x86_64-softmmu/hw/display/virtio-gpu-3d.o
CC aarch64-softmmu/accel/tcg/tcg-runtime-gvec.o
CC x86_64-softmmu/hw/display/virtio-gpu-pci.o
CC x86_64-softmmu/hw/display/virtio-vga.o
CC aarch64-softmmu/accel/tcg/cpu-exec.o
CC aarch64-softmmu/accel/tcg/cpu-exec-common.o
CC aarch64-softmmu/accel/tcg/translate-all.o
CC aarch64-softmmu/accel/tcg/translator.o
CC aarch64-softmmu/hw/adc/stm32f2xx_adc.o
CC aarch64-softmmu/hw/block/virtio-blk.o
CC x86_64-softmmu/hw/intc/apic.o
CC aarch64-softmmu/hw/block/dataplane/virtio-blk.o
CC aarch64-softmmu/hw/char/exynos4210_uart.o
CC x86_64-softmmu/hw/intc/apic_common.o
CC x86_64-softmmu/hw/intc/ioapic.o
CC aarch64-softmmu/hw/char/omap_uart.o
CC aarch64-softmmu/hw/char/digic-uart.o
CC aarch64-softmmu/hw/char/stm32f2xx_usart.o
CC x86_64-softmmu/hw/misc/pvpanic.o
CC x86_64-softmmu/hw/isa/lpc_ich9.o
CC aarch64-softmmu/hw/char/bcm2835_aux.o
CC aarch64-softmmu/hw/char/virtio-serial-bus.o
CC x86_64-softmmu/hw/misc/mmio_interface.o
CC aarch64-softmmu/hw/core/generic-loader.o
CC x86_64-softmmu/hw/net/virtio-net.o
CC aarch64-softmmu/hw/core/null-machine.o
CC x86_64-softmmu/hw/net/vhost_net.o
CC aarch64-softmmu/hw/cpu/arm11mpcore.o
CC x86_64-softmmu/hw/scsi/virtio-scsi.o
CC aarch64-softmmu/hw/cpu/realview_mpcore.o
CC x86_64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC aarch64-softmmu/hw/cpu/a9mpcore.o
CC x86_64-softmmu/hw/timer/mc146818rtc.o
CC aarch64-softmmu/hw/cpu/a15mpcore.o
CC x86_64-softmmu/hw/virtio/virtio.o
CC x86_64-softmmu/hw/virtio/virtio-balloon.o
CC aarch64-softmmu/hw/display/omap_dss.o
CC x86_64-softmmu/hw/virtio/virtio-crypto.o
CC x86_64-softmmu/hw/virtio/virtio-crypto-pci.o
CC x86_64-softmmu/hw/i386/multiboot.o
CC aarch64-softmmu/hw/display/omap_lcdc.o
CC aarch64-softmmu/hw/display/pxa2xx_lcd.o
CC x86_64-softmmu/hw/i386/pc.o
CC x86_64-softmmu/hw/i386/pc_piix.o
CC x86_64-softmmu/hw/i386/pc_q35.o
CC x86_64-softmmu/hw/i386/pc_sysfw.o
CC aarch64-softmmu/hw/display/bcm2835_fb.o
CC aarch64-softmmu/hw/display/vga.o
CC x86_64-softmmu/hw/i386/x86-iommu.o
CC aarch64-softmmu/hw/display/virtio-gpu.o
CC aarch64-softmmu/hw/display/virtio-gpu-3d.o
CC x86_64-softmmu/hw/i386/intel_iommu.o
CC x86_64-softmmu/hw/i386/amd_iommu.o
CC x86_64-softmmu/hw/i386/vmport.o
CC x86_64-softmmu/hw/i386/vmmouse.o
CC x86_64-softmmu/hw/i386/kvmvapic.o
CC x86_64-softmmu/hw/i386/acpi-build.o
CC x86_64-softmmu/target/i386/helper.o
CC x86_64-softmmu/target/i386/cpu.o
CC x86_64-softmmu/target/i386/gdbstub.o
CC x86_64-softmmu/target/i386/xsave_helper.o
CC aarch64-softmmu/hw/display/virtio-gpu-pci.o
CC x86_64-softmmu/target/i386/translate.o
CC aarch64-softmmu/hw/display/dpcd.o
CC aarch64-softmmu/hw/display/xlnx_dp.o
CC aarch64-softmmu/hw/dma/xlnx_dpdma.o
CC aarch64-softmmu/hw/dma/omap_dma.o
CC x86_64-softmmu/target/i386/cc_helper.o
CC x86_64-softmmu/target/i386/bpt_helper.o
CC aarch64-softmmu/hw/dma/pxa2xx_dma.o
CC aarch64-softmmu/hw/dma/soc_dma.o
CC aarch64-softmmu/hw/dma/bcm2835_dma.o
CC aarch64-softmmu/hw/gpio/omap_gpio.o
CC aarch64-softmmu/hw/gpio/imx_gpio.o
CC x86_64-softmmu/target/i386/excp_helper.o
CC aarch64-softmmu/hw/gpio/bcm2835_gpio.o
CC aarch64-softmmu/hw/i2c/omap_i2c.o
CC aarch64-softmmu/hw/input/pxa2xx_keypad.o
CC aarch64-softmmu/hw/input/tsc210x.o
CC aarch64-softmmu/hw/intc/armv7m_nvic.o
CC aarch64-softmmu/hw/intc/exynos4210_gic.o
CC x86_64-softmmu/target/i386/fpu_helper.o
CC aarch64-softmmu/hw/intc/exynos4210_combiner.o
CC aarch64-softmmu/hw/intc/omap_intc.o
CC aarch64-softmmu/hw/intc/bcm2835_ic.o
CC x86_64-softmmu/target/i386/int_helper.o
CC aarch64-softmmu/hw/intc/bcm2836_control.o
CC aarch64-softmmu/hw/intc/allwinner-a10-pic.o
CC aarch64-softmmu/hw/intc/aspeed_vic.o
CC aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
CC x86_64-softmmu/target/i386/mem_helper.o
CC x86_64-softmmu/target/i386/misc_helper.o
CC aarch64-softmmu/hw/misc/arm_sysctl.o
CC aarch64-softmmu/hw/misc/cbus.o
CC aarch64-softmmu/hw/misc/exynos4210_pmu.o
CC aarch64-softmmu/hw/misc/exynos4210_clk.o
CC x86_64-softmmu/target/i386/mpx_helper.o
In file included from /tmp/qemu-test/src/migration/ram.c:46:0:
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_recv_unfill_packet':
/tmp/qemu-test/src/include/qapi/error.h:162:25: error: format '%lld' expects argument of type 'long long int', but argument 6 has type 'ram_addr_t {aka unsigned int}' [-Werror=format=]
(fmt), ## __VA_ARGS__)
^
/tmp/qemu-test/src/migration/ram.c:777:13: note: in expansion of macro 'error_setg'
error_setg(errp, "multifd: offset too long %" PRId64
^~~~~~~~~~
/tmp/qemu-test/src/include/qapi/error.h:162:25: error: format '%lld' expects argument of type 'long long int', but argument 7 has type 'ram_addr_t {aka unsigned int}' [-Werror=format=]
(fmt), ## __VA_ARGS__)
^
/tmp/qemu-test/src/migration/ram.c:777:13: note: in expansion of macro 'error_setg'
error_setg(errp, "multifd: offset too long %" PRId64
^~~~~~~~~~
cc1: all warnings being treated as errors
make[1]: *** [/tmp/qemu-test/src/rules.mak:66: migration/ram.o] Error 1
make[1]: *** Waiting for unfinished jobs....
CC aarch64-softmmu/hw/misc/exynos4210_rng.o
CC aarch64-softmmu/hw/misc/imx_ccm.o
CC aarch64-softmmu/hw/misc/imx25_ccm.o
CC aarch64-softmmu/hw/misc/imx31_ccm.o
CC aarch64-softmmu/hw/misc/imx6_ccm.o
CC aarch64-softmmu/hw/misc/imx6_src.o
make: *** [Makefile:478: subdir-x86_64-softmmu] Error 2
make: *** Waiting for unfinished jobs....
CC aarch64-softmmu/hw/misc/imx7_ccm.o
CC aarch64-softmmu/hw/misc/imx2_wdt.o
CC aarch64-softmmu/hw/misc/imx7_snvs.o
CC aarch64-softmmu/hw/misc/imx7_gpr.o
CC aarch64-softmmu/hw/misc/mst_fpga.o
CC aarch64-softmmu/hw/misc/omap_clk.o
CC aarch64-softmmu/hw/misc/omap_gpmc.o
CC aarch64-softmmu/hw/misc/omap_l4.o
CC aarch64-softmmu/hw/misc/omap_sdrc.o
CC aarch64-softmmu/hw/misc/omap_tap.o
CC aarch64-softmmu/hw/misc/bcm2835_property.o
CC aarch64-softmmu/hw/misc/bcm2835_mbox.o
CC aarch64-softmmu/hw/misc/bcm2835_rng.o
CC aarch64-softmmu/hw/misc/zynq_slcr.o
CC aarch64-softmmu/hw/misc/zynq-xadc.o
CC aarch64-softmmu/hw/misc/mps2-fpgaio.o
CC aarch64-softmmu/hw/misc/stm32f2xx_syscfg.o
CC aarch64-softmmu/hw/misc/mps2-scc.o
CC aarch64-softmmu/hw/misc/tz-ppc.o
CC aarch64-softmmu/hw/misc/iotkit-secctl.o
CC aarch64-softmmu/hw/misc/auxbus.o
CC aarch64-softmmu/hw/misc/aspeed_scu.o
CC aarch64-softmmu/hw/misc/aspeed_sdmc.o
CC aarch64-softmmu/hw/misc/mmio_interface.o
CC aarch64-softmmu/hw/misc/msf2-sysreg.o
CC aarch64-softmmu/hw/net/virtio-net.o
CC aarch64-softmmu/hw/net/vhost_net.o
CC aarch64-softmmu/hw/pcmcia/pxa2xx.o
CC aarch64-softmmu/hw/scsi/virtio-scsi.o
CC aarch64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC aarch64-softmmu/hw/sd/omap_mmc.o
CC aarch64-softmmu/hw/sd/pxa2xx_mmci.o
CC aarch64-softmmu/hw/sd/bcm2835_sdhost.o
CC aarch64-softmmu/hw/ssi/omap_spi.o
CC aarch64-softmmu/hw/ssi/imx_spi.o
CC aarch64-softmmu/hw/timer/exynos4210_mct.o
CC aarch64-softmmu/hw/timer/exynos4210_pwm.o
CC aarch64-softmmu/hw/timer/exynos4210_rtc.o
CC aarch64-softmmu/hw/timer/omap_gptimer.o
CC aarch64-softmmu/hw/timer/omap_synctimer.o
CC aarch64-softmmu/hw/timer/pxa2xx_timer.o
CC aarch64-softmmu/hw/timer/digic-timer.o
CC aarch64-softmmu/hw/timer/allwinner-a10-pit.o
CC aarch64-softmmu/hw/usb/tusb6010.o
CC aarch64-softmmu/hw/usb/chipidea.o
CC aarch64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/hw/virtio/virtio-balloon.o
CC aarch64-softmmu/hw/virtio/virtio-crypto.o
CC aarch64-softmmu/hw/virtio/virtio-crypto-pci.o
CC aarch64-softmmu/hw/arm/boot.o
CC aarch64-softmmu/hw/arm/virt.o
CC aarch64-softmmu/hw/arm/sysbus-fdt.o
CC aarch64-softmmu/hw/arm/virt-acpi-build.o
CC aarch64-softmmu/hw/arm/digic_boards.o
CC aarch64-softmmu/hw/arm/exynos4_boards.o
CC aarch64-softmmu/hw/arm/highbank.o
CC aarch64-softmmu/hw/arm/integratorcp.o
CC aarch64-softmmu/hw/arm/mainstone.o
CC aarch64-softmmu/hw/arm/musicpal.o
CC aarch64-softmmu/hw/arm/netduino2.o
CC aarch64-softmmu/hw/arm/nseries.o
CC aarch64-softmmu/hw/arm/omap_sx1.o
CC aarch64-softmmu/hw/arm/palm.o
CC aarch64-softmmu/hw/arm/gumstix.o
CC aarch64-softmmu/hw/arm/spitz.o
CC aarch64-softmmu/hw/arm/tosa.o
CC aarch64-softmmu/hw/arm/z2.o
CC aarch64-softmmu/hw/arm/realview.o
CC aarch64-softmmu/hw/arm/stellaris.o
CC aarch64-softmmu/hw/arm/collie.o
CC aarch64-softmmu/hw/arm/vexpress.o
CC aarch64-softmmu/hw/arm/versatilepb.o
CC aarch64-softmmu/hw/arm/xilinx_zynq.o
CC aarch64-softmmu/hw/arm/armv7m.o
CC aarch64-softmmu/hw/arm/exynos4210.o
CC aarch64-softmmu/hw/arm/pxa2xx.o
CC aarch64-softmmu/hw/arm/pxa2xx_gpio.o
CC aarch64-softmmu/hw/arm/pxa2xx_pic.o
CC aarch64-softmmu/hw/arm/omap1.o
CC aarch64-softmmu/hw/arm/digic.o
CC aarch64-softmmu/hw/arm/omap2.o
CC aarch64-softmmu/hw/arm/strongarm.o
CC aarch64-softmmu/hw/arm/allwinner-a10.o
CC aarch64-softmmu/hw/arm/cubieboard.o
CC aarch64-softmmu/hw/arm/bcm2835_peripherals.o
CC aarch64-softmmu/hw/arm/bcm2836.o
CC aarch64-softmmu/hw/arm/raspi.o
CC aarch64-softmmu/hw/arm/stm32f205_soc.o
CC aarch64-softmmu/hw/arm/xlnx-zynqmp.o
CC aarch64-softmmu/hw/arm/xlnx-zcu102.o
CC aarch64-softmmu/hw/arm/fsl-imx25.o
CC aarch64-softmmu/hw/arm/imx25_pdk.o
CC aarch64-softmmu/hw/arm/fsl-imx31.o
CC aarch64-softmmu/hw/arm/kzm.o
CC aarch64-softmmu/hw/arm/fsl-imx6.o
CC aarch64-softmmu/hw/arm/sabrelite.o
CC aarch64-softmmu/hw/arm/aspeed_soc.o
CC aarch64-softmmu/hw/arm/mps2.o
CC aarch64-softmmu/hw/arm/aspeed.o
CC aarch64-softmmu/hw/arm/mps2-tz.o
CC aarch64-softmmu/hw/arm/msf2-soc.o
CC aarch64-softmmu/hw/arm/msf2-som.o
In file included from /tmp/qemu-test/src/migration/ram.c:46:0:
/tmp/qemu-test/src/migration/ram.c: In function 'multifd_recv_unfill_packet':
/tmp/qemu-test/src/include/qapi/error.h:162:25: error: format '%lld' expects argument of type 'long long int', but argument 6 has type 'ram_addr_t {aka unsigned int}' [-Werror=format=]
(fmt), ## __VA_ARGS__)
^
/tmp/qemu-test/src/migration/ram.c:777:13: note: in expansion of macro 'error_setg'
error_setg(errp, "multifd: offset too long %" PRId64
^~~~~~~~~~
/tmp/qemu-test/src/include/qapi/error.h:162:25: error: format '%lld' expects argument of type 'long long int', but argument 7 has type 'ram_addr_t {aka unsigned int}' [-Werror=format=]
(fmt), ## __VA_ARGS__)
^
/tmp/qemu-test/src/migration/ram.c:777:13: note: in expansion of macro 'error_setg'
error_setg(errp, "multifd: offset too long %" PRId64
^~~~~~~~~~
cc1: all warnings being treated as errors
CC aarch64-softmmu/hw/arm/iotkit.o
make[1]: *** [/tmp/qemu-test/src/rules.mak:66: migration/ram.o] Error 1
make[1]: *** Waiting for unfinished jobs....
CC aarch64-softmmu/hw/arm/fsl-imx7.o
make: *** [Makefile:478: subdir-aarch64-softmmu] Error 2
Traceback (most recent call last):
File "./tests/docker/docker.py", line 407, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 404, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 261, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 229, in run
quiet=quiet)
File "./tests/docker/docker.py", line 147, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--label', 'com.qemu.instance.uuid=ba86b3185e7d11e894d452540069c830', '-u', '0', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/root/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-7ulkeqm3/src/docker-src.2018-05-23-07.37.40.11348:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:129: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-7ulkeqm3/src'
make: *** [tests/docker/Makefile.include:163: docker-run-test-mingw@fedora] Error 2
real 3m48.738s
user 0m4.885s
sys 0m4.231s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 03/12] migration: Add multifd traces for start/end thread
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 03/12] migration: Add multifd traces for start/end thread Juan Quintela
@ 2018-05-31 16:11 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-05-31 16:11 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> We want to know how many pages/packets each channel has sent. Add
> counters for those.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> --
> sort trace-events (dave)
> ---
> migration/ram.c | 22 ++++++++++++++++++++++
> migration/trace-events | 4 ++++
> 2 files changed, 26 insertions(+)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index 54350db8b0..ab715cc148 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -563,6 +563,11 @@ typedef struct {
> uint32_t flags;
> /* global number of generated multifd packets */
> uint32_t seq;
> + /* thread local variables */
> + /* packets sent through this channel */
> + uint32_t num_packets;
> + /* pages sent through this channel */
> + uint32_t num_pages;
> } MultiFDSendParams;
>
> typedef struct {
> @@ -593,6 +598,11 @@ typedef struct {
> uint32_t flags;
> /* global number of generated multifd packets */
> uint32_t seq;
> + /* thread local variables */
> + /* packets sent through this channel */
> + uint32_t num_packets;
> + /* pages sent through this channel */
> + uint32_t num_pages;
> } MultiFDRecvParams;
>
> static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
> @@ -849,9 +859,13 @@ static void *multifd_send_thread(void *opaque)
> MultiFDSendParams *p = opaque;
> Error *local_err = NULL;
>
> + trace_multifd_send_thread_start(p->id);
> +
> if (multifd_send_initial_packet(p, &local_err) < 0) {
> goto out;
> }
> + /* initial packet */
> + p->num_packets = 1;
>
> while (true) {
> qemu_mutex_lock(&p->mutex);
> @@ -873,6 +887,8 @@ out:
> p->running = false;
> qemu_mutex_unlock(&p->mutex);
>
> + trace_multifd_send_thread_end(p->id, p->num_packets, p->num_pages);
> +
> return NULL;
> }
>
> @@ -1000,6 +1016,8 @@ static void *multifd_recv_thread(void *opaque)
> Error *local_err = NULL;
> int ret;
>
> + trace_multifd_recv_thread_start(p->id);
> +
> while (true) {
> qemu_mutex_lock(&p->mutex);
> if (false) {
> @@ -1022,6 +1040,8 @@ static void *multifd_recv_thread(void *opaque)
> p->running = false;
> qemu_mutex_unlock(&p->mutex);
>
> + trace_multifd_recv_thread_end(p->id, p->num_packets, p->num_pages);
> +
> return NULL;
> }
>
> @@ -1087,6 +1107,8 @@ void multifd_recv_new_channel(QIOChannel *ioc)
> }
> p->c = ioc;
> object_ref(OBJECT(ioc));
> + /* initial packet */
> + p->num_packets = 1;
>
> p->running = true;
> qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p,
> diff --git a/migration/trace-events b/migration/trace-events
> index 3c798ddd11..3c7a9a8332 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -76,6 +76,10 @@ get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned
> migration_bitmap_sync_start(void) ""
> migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
> migration_throttle(void) ""
> +multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
> +multifd_recv_thread_start(uint8_t id) "%d"
> +multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
> +multifd_send_thread_start(uint8_t id) "%d"
> ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
> ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%s: addr: 0x%" PRIx64 " flags: 0x%x host: %p"
> ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x"
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 04/12] migration: Calculate transferred ram correctly
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 04/12] migration: Calculate transferred ram correctly Juan Quintela
@ 2018-05-31 17:14 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-05-31 17:14 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> On multifd we send data from more places that main channel.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
> migration/migration.c | 11 +++++++++--
> 1 file changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/migration/migration.c b/migration/migration.c
> index 05aec2c905..55d7b64e24 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -2679,12 +2679,19 @@ static void migration_update_counters(MigrationState *s,
> {
> uint64_t transferred, time_spent;
> double bandwidth;
> + uint64_t now;
>
> if (current_time < s->iteration_start_time + BUFFER_DELAY) {
> return;
> }
>
> - transferred = qemu_ftell(s->to_dst_file) - s->iteration_initial_bytes;
> + if (migrate_use_multifd()) {
> + now = ram_counters.normal * qemu_target_page_size()
> + + qemu_ftell(s->to_dst_file);
Didn't we decide after the last set that this had to include the headers
as well?
Dave
> + } else {
> + now = qemu_ftell(s->to_dst_file);
> + }
> + transferred = now - s->iteration_initial_bytes;
> time_spent = current_time - s->iteration_start_time;
> bandwidth = (double)transferred / time_spent;
> s->threshold_size = bandwidth * s->parameters.downtime_limit;
> @@ -2704,7 +2711,7 @@ static void migration_update_counters(MigrationState *s,
> qemu_file_reset_rate_limit(s->to_dst_file);
>
> s->iteration_start_time = current_time;
> - s->iteration_initial_bytes = qemu_ftell(s->to_dst_file);
> + s->iteration_initial_bytes = now;
>
> trace_migrate_transferred(transferred, time_spent,
> bandwidth, s->threshold_size);
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 02/12] migration: Create multifd packet
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 02/12] migration: Create multifd packet Juan Quintela
@ 2018-06-11 11:10 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-11 11:10 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> We still don't put anything there.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
>
> --
> fix magic (dave)
> check offset/ramblock (dave)
> ---
> migration/ram.c | 145 +++++++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 144 insertions(+), 1 deletion(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index 23cc5625eb..54350db8b0 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -510,6 +510,17 @@ typedef struct {
> uint8_t id;
> } __attribute__((packed)) MultiFDInit_t;
>
> +typedef struct {
> + uint32_t magic;
> + uint32_t version;
> + uint32_t flags;
> + uint32_t size;
> + uint32_t used;
> + uint32_t seq;
> + char ramblock[256];
> + uint64_t offset[];
> +} __attribute__((packed)) MultiFDPacket_t;
> +
> typedef struct {
> /* number of used pages */
> uint32_t used;
> @@ -544,6 +555,14 @@ typedef struct {
> bool quit;
> /* array of pages to sent */
> MultiFDPages_t *pages;
> + /* packet allocated len */
> + uint32_t packet_len;
> + /* pointer to the packet */
> + MultiFDPacket_t *packet;
> + /* multifd flags for each packet */
> + uint32_t flags;
> + /* global number of generated multifd packets */
> + uint32_t seq;
> } MultiFDSendParams;
>
> typedef struct {
> @@ -566,6 +585,14 @@ typedef struct {
> bool quit;
> /* array of pages to receive */
> MultiFDPages_t *pages;
> + /* packet allocated len */
> + uint32_t packet_len;
> + /* pointer to the packet */
> + MultiFDPacket_t *packet;
> + /* multifd flags for each packet */
> + uint32_t flags;
> + /* global number of generated multifd packets */
> + uint32_t seq;
> } MultiFDRecvParams;
>
> static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
> @@ -654,6 +681,99 @@ static void multifd_pages_clear(MultiFDPages_t *pages)
> g_free(pages);
> }
>
> +static void multifd_send_fill_packet(MultiFDSendParams *p)
> +{
> + MultiFDPacket_t *packet = p->packet;
> + int i;
> +
> + packet->magic = cpu_to_be32(MULTIFD_MAGIC);
> + packet->version = cpu_to_be32(MULTIFD_VERSION);
> + packet->flags = cpu_to_be32(p->flags);
> + packet->size = cpu_to_be32(migrate_multifd_page_count());
> + packet->used = cpu_to_be32(p->pages->used);
> + packet->seq = cpu_to_be32(p->seq);
> +
> + if (p->pages->block) {
> + strncpy(packet->ramblock, p->pages->block->idstr, 256);
> + }
> +
> + for (i = 0; i < p->pages->used; i++) {
> + packet->offset[i] = cpu_to_be64(p->pages->offset[i]);
> + }
> +}
> +
> +static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp)
> +{
> + MultiFDPacket_t *packet = p->packet;
> + RAMBlock *block;
> + int i;
> +
> + /* ToDo: We can't use it until we haven't received a message */
> + return 0;
> +
> + be32_to_cpus(&packet->magic);
> + if (packet->magic != MULTIFD_MAGIC) {
> + error_setg(errp, "multifd: received packet "
> + "magic %x and expected magic %x",
> + packet->magic, MULTIFD_MAGIC);
> + return -1;
> + }
> +
> + be32_to_cpus(&packet->version);
> + if (packet->version != MULTIFD_VERSION) {
> + error_setg(errp, "multifd: received packet "
> + "version %d and expected version %d",
> + packet->version, MULTIFD_VERSION);
> + return -1;
> + }
> +
> + p->flags = be32_to_cpu(packet->flags);
> +
> + be32_to_cpus(&packet->size);
> + if (packet->size > migrate_multifd_page_count()) {
> + error_setg(errp, "multifd: received packet "
> + "with size %d and expected maximum size %d",
> + packet->size, migrate_multifd_page_count()) ;
> + return -1;
> + }
> +
> + p->pages->used = be32_to_cpu(packet->used);
> + if (p->pages->used > packet->size) {
> + error_setg(errp, "multifd: received packet "
> + "with size %d and expected maximum size %d",
> + p->pages->used, packet->size) ;
> + return -1;
> + }
> +
> + p->seq = be32_to_cpu(packet->seq);
> +
> + if (p->pages->used) {
> + /* make sure that ramblock is 0 terminated */
> + packet->ramblock[255] = 0;
> + block = qemu_ram_block_by_name(packet->ramblock);
> + if (!block) {
> + error_setg(errp, "multifd: unknown ram block %s",
> + packet->ramblock);
> + return -1;
> + }
> + }
> +
> + for (i = 0; i < p->pages->used; i++) {
> + ram_addr_t offset = be64_to_cpu(packet->offset[i]);
> +
> + if (offset > (block->used_length - TARGET_PAGE_SIZE)) {
> + error_setg(errp, "multifd: offset too long %" PRId64
> + " (max %" PRId64 ")",
> + offset, block->max_length);
> + return -1;
> + }
> + p->pages->iov[i].iov_base = block->host + offset;
> + p->pages->iov[i].iov_len = TARGET_PAGE_SIZE;
> + }
> +
> + return 0;
> +}
> +
> struct {
> MultiFDSendParams *params;
> /* number of created threads */
> @@ -711,6 +831,9 @@ int multifd_save_cleanup(Error **errp)
> p->name = NULL;
> multifd_pages_clear(p->pages);
> p->pages = NULL;
> + p->packet_len = 0;
> + g_free(p->packet);
> + p->packet = NULL;
> }
> g_free(multifd_send_state->params);
> multifd_send_state->params = NULL;
> @@ -732,6 +855,7 @@ static void *multifd_send_thread(void *opaque)
>
> while (true) {
> qemu_mutex_lock(&p->mutex);
> + multifd_send_fill_packet(p);
> if (p->quit) {
> qemu_mutex_unlock(&p->mutex);
> break;
> @@ -796,6 +920,9 @@ int multifd_save_setup(void)
> p->quit = false;
> p->id = i;
> p->pages = multifd_pages_init(page_count);
> + p->packet_len = sizeof(MultiFDPacket_t)
> + + sizeof(ram_addr_t) * page_count;
> + p->packet = g_malloc0(p->packet_len);
> p->name = g_strdup_printf("multifdsend_%d", i);
> socket_send_channel_create(multifd_new_send_channel_async, p);
> }
> @@ -855,6 +982,9 @@ int multifd_load_cleanup(Error **errp)
> p->name = NULL;
> multifd_pages_clear(p->pages);
> p->pages = NULL;
> + p->packet_len = 0;
> + g_free(p->packet);
> + p->packet = NULL;
> }
> g_free(multifd_recv_state->params);
> multifd_recv_state->params = NULL;
> @@ -867,10 +997,20 @@ int multifd_load_cleanup(Error **errp)
> static void *multifd_recv_thread(void *opaque)
> {
> MultiFDRecvParams *p = opaque;
> + Error *local_err = NULL;
> + int ret;
>
> while (true) {
> qemu_mutex_lock(&p->mutex);
> - if (p->quit) {
> + if (false) {
> + /* ToDo: Packet reception goes here */
> +
> + ret = multifd_recv_unfill_packet(p, &local_err);
> + qemu_mutex_unlock(&p->mutex);
> + if (ret) {
> + break;
> + }
> + } else if (p->quit) {
> qemu_mutex_unlock(&p->mutex);
> break;
> }
> @@ -907,6 +1047,9 @@ int multifd_load_setup(void)
> p->quit = false;
> p->id = i;
> p->pages = multifd_pages_init(page_count);
> + p->packet_len = sizeof(MultiFDPacket_t)
> + + sizeof(ram_addr_t) * page_count;
> + p->packet = g_malloc0(p->packet_len);
> p->name = g_strdup_printf("multifdrecv_%d", i);
> }
> return 0;
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 07/12] migration: Synchronize multifd threads with main thread
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 07/12] migration: Synchronize multifd threads with main thread Juan Quintela
@ 2018-06-11 11:53 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-11 11:53 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> We synchronize all threads each RAM_SAVE_FLAG_EOS. Bitmap
> synchronizations don't happen inside a ram section, so we are safe
> about two channels trying to overwrite the same memory.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
> migration/ram.c | 117 +++++++++++++++++++++++++++++++++++++----
> migration/trace-events | 6 +++
> 2 files changed, 112 insertions(+), 11 deletions(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index c9a9bd79f3..3e99d48123 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -503,6 +503,8 @@ exit:
> #define MULTIFD_MAGIC 0x11223344U
> #define MULTIFD_VERSION 1
>
> +#define MULTIFD_FLAG_SYNC (1 << 0)
> +
> typedef struct {
> uint32_t magic;
> uint32_t version;
> @@ -570,6 +572,8 @@ typedef struct {
> uint32_t num_packets;
> /* pages sent through this channel */
> uint32_t num_pages;
> + /* syncs main thread and channels */
> + QemuSemaphore sem_sync;
> } MultiFDSendParams;
>
> typedef struct {
> @@ -607,6 +611,8 @@ typedef struct {
> uint32_t num_packets;
> /* pages sent through this channel */
> uint32_t num_pages;
> + /* syncs main thread and channels */
> + QemuSemaphore sem_sync;
> } MultiFDRecvParams;
>
> static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
> @@ -794,6 +800,10 @@ struct {
> int count;
> /* array of pages to sent */
> MultiFDPages_t *pages;
> + /* syncs main thread and channels */
> + QemuSemaphore sem_sync;
> + /* global number of generated multifd packets */
> + uint32_t seq;
> } *multifd_send_state;
>
> static void multifd_send_terminate_threads(Error *err)
> @@ -841,6 +851,7 @@ int multifd_save_cleanup(Error **errp)
> p->c = NULL;
> qemu_mutex_destroy(&p->mutex);
> qemu_sem_destroy(&p->sem);
> + qemu_sem_destroy(&p->sem_sync);
> g_free(p->name);
> p->name = NULL;
> multifd_pages_clear(p->pages);
> @@ -849,6 +860,7 @@ int multifd_save_cleanup(Error **errp)
> g_free(p->packet);
> p->packet = NULL;
> }
> + qemu_sem_destroy(&multifd_send_state->sem_sync);
> g_free(multifd_send_state->params);
> multifd_send_state->params = NULL;
> multifd_pages_clear(multifd_send_state->pages);
> @@ -858,6 +870,33 @@ int multifd_save_cleanup(Error **errp)
> return ret;
> }
>
> +static void multifd_send_sync_main(void)
> +{
> + int i;
> +
> + if (!migrate_use_multifd()) {
> + return;
> + }
> + for (i = 0; i < migrate_multifd_channels(); i++) {
> + MultiFDSendParams *p = &multifd_send_state->params[i];
> +
> + trace_multifd_send_sync_main_signal(p->id);
> +
> + qemu_mutex_lock(&p->mutex);
> + p->flags |= MULTIFD_FLAG_SYNC;
> + p->pending_job++;
> + qemu_mutex_unlock(&p->mutex);
> + qemu_sem_post(&p->sem);
> + }
> + for (i = 0; i < migrate_multifd_channels(); i++) {
> + MultiFDSendParams *p = &multifd_send_state->params[i];
> +
> + trace_multifd_send_sync_main_wait(p->id);
> + qemu_sem_wait(&multifd_send_state->sem_sync);
> + }
> + trace_multifd_send_sync_main(multifd_send_state->seq);
> +}
> +
> static void *multifd_send_thread(void *opaque)
> {
> MultiFDSendParams *p = opaque;
> @@ -894,15 +933,17 @@ static void *multifd_send_thread(void *opaque)
> qemu_mutex_lock(&p->mutex);
> p->pending_job--;
> qemu_mutex_unlock(&p->mutex);
> - continue;
> +
> + if (flags & MULTIFD_FLAG_SYNC) {
> + qemu_sem_post(&multifd_send_state->sem_sync);
> + }
> } else if (p->quit) {
> qemu_mutex_unlock(&p->mutex);
> break;
> + } else {
> + qemu_mutex_unlock(&p->mutex);
> + /* sometimes there are spurious wakeups */
> }
> - qemu_mutex_unlock(&p->mutex);
> - /* this is impossible */
> - error_setg(&local_err, "multifd_send_thread: Unknown command");
> - break;
> }
>
> out:
> @@ -954,12 +995,14 @@ int multifd_save_setup(void)
> multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
> atomic_set(&multifd_send_state->count, 0);
> multifd_send_state->pages = multifd_pages_init(page_count);
> + qemu_sem_init(&multifd_send_state->sem_sync, 0);
>
> for (i = 0; i < thread_count; i++) {
> MultiFDSendParams *p = &multifd_send_state->params[i];
>
> qemu_mutex_init(&p->mutex);
> qemu_sem_init(&p->sem, 0);
> + qemu_sem_init(&p->sem_sync, 0);
> p->quit = false;
> p->pending_job = 0;
> p->id = i;
> @@ -977,6 +1020,10 @@ struct {
> MultiFDRecvParams *params;
> /* number of created threads */
> int count;
> + /* syncs main thread and channels */
> + QemuSemaphore sem_sync;
> + /* global number of generated multifd packets */
> + uint32_t seq;
> } *multifd_recv_state;
>
> static void multifd_recv_terminate_threads(Error *err)
> @@ -1022,6 +1069,7 @@ int multifd_load_cleanup(Error **errp)
> p->c = NULL;
> qemu_mutex_destroy(&p->mutex);
> qemu_sem_destroy(&p->sem);
> + qemu_sem_destroy(&p->sem_sync);
> g_free(p->name);
> p->name = NULL;
> multifd_pages_clear(p->pages);
> @@ -1030,6 +1078,7 @@ int multifd_load_cleanup(Error **errp)
> g_free(p->packet);
> p->packet = NULL;
> }
> + qemu_sem_destroy(&multifd_recv_state->sem_sync);
> g_free(multifd_recv_state->params);
> multifd_recv_state->params = NULL;
> g_free(multifd_recv_state);
> @@ -1038,6 +1087,42 @@ int multifd_load_cleanup(Error **errp)
> return ret;
> }
>
> +static void multifd_recv_sync_main(void)
> +{
> + int i;
> +
> + if (!migrate_use_multifd()) {
> + return;
> + }
> + for (i = 0; i < migrate_multifd_channels(); i++) {
> + MultiFDRecvParams *p = &multifd_recv_state->params[i];
> +
> + trace_multifd_recv_sync_main_signal(p->id);
> + qemu_mutex_lock(&p->mutex);
> + p->pending_job = true;
> + qemu_mutex_unlock(&p->mutex);
> + }
> + for (i = 0; i < migrate_multifd_channels(); i++) {
> + MultiFDRecvParams *p = &multifd_recv_state->params[i];
> +
> + trace_multifd_recv_sync_main_wait(p->id);
> + qemu_sem_wait(&multifd_recv_state->sem_sync);
> + qemu_mutex_lock(&p->mutex);
> + if (multifd_recv_state->seq < p->seq) {
> + multifd_recv_state->seq = p->seq;
> + }
> + qemu_mutex_unlock(&p->mutex);
> + }
> + for (i = 0; i < migrate_multifd_channels(); i++) {
> + MultiFDRecvParams *p = &multifd_recv_state->params[i];
> +
> + trace_multifd_recv_sync_main_signal(p->id);
> +
> + qemu_sem_post(&p->sem_sync);
> + }
> + trace_multifd_recv_sync_main(multifd_recv_state->seq);
> +}
> +
> static void *multifd_recv_thread(void *opaque)
> {
> MultiFDRecvParams *p = opaque;
> @@ -1047,9 +1132,8 @@ static void *multifd_recv_thread(void *opaque)
> trace_multifd_recv_thread_start(p->id);
>
> while (true) {
> - qemu_sem_wait(&p->sem);
> qemu_mutex_lock(&p->mutex);
> - if (p->pending_job) {
> + if (true || p->pending_job) {
I think you said that one should have gone.
Dave
> uint32_t used;
> uint32_t flags;
> qemu_mutex_unlock(&p->mutex);
> @@ -1070,14 +1154,18 @@ static void *multifd_recv_thread(void *opaque)
> p->num_packets++;
> p->num_pages += used;
> qemu_mutex_unlock(&p->mutex);
> +
> + if (flags & MULTIFD_FLAG_SYNC) {
> + qemu_sem_post(&multifd_recv_state->sem_sync);
> + qemu_sem_wait(&p->sem_sync);
> + }
> } else if (p->quit) {
> qemu_mutex_unlock(&p->mutex);
> break;
> + } else {
> + qemu_mutex_unlock(&p->mutex);
> + /* sometimes there are spurious wakeups */
> }
> - qemu_mutex_unlock(&p->mutex);
> - /* this is impossible */
> - error_setg(&local_err, "multifd_recv_thread: Unknown command");
> - break;
> }
>
> if (local_err) {
> @@ -1105,12 +1193,14 @@ int multifd_load_setup(void)
> multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state));
> multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count);
> atomic_set(&multifd_recv_state->count, 0);
> + qemu_sem_init(&multifd_recv_state->sem_sync, 0);
>
> for (i = 0; i < thread_count; i++) {
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> qemu_mutex_init(&p->mutex);
> qemu_sem_init(&p->sem, 0);
> + qemu_sem_init(&p->sem_sync, 0);
> p->quit = false;
> p->pending_job = false;
> p->id = i;
> @@ -2847,6 +2937,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
> ram_control_before_iterate(f, RAM_CONTROL_SETUP);
> ram_control_after_iterate(f, RAM_CONTROL_SETUP);
>
> + multifd_send_sync_main();
> qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
>
> return 0;
> @@ -2922,6 +3013,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
> */
> ram_control_after_iterate(f, RAM_CONTROL_ROUND);
>
> + multifd_send_sync_main();
> out:
> qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
> ram_counters.transferred += 8;
> @@ -2975,6 +3067,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
>
> rcu_read_unlock();
>
> + multifd_send_sync_main();
> qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
>
> return 0;
> @@ -3459,6 +3552,7 @@ static int ram_load_postcopy(QEMUFile *f)
> break;
> case RAM_SAVE_FLAG_EOS:
> /* normal exit */
> + multifd_recv_sync_main();
> break;
> default:
> error_report("Unknown combination of migration flags: %#x"
> @@ -3644,6 +3738,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
> break;
> case RAM_SAVE_FLAG_EOS:
> /* normal exit */
> + multifd_recv_sync_main();
> break;
> default:
> if (flags & RAM_SAVE_FLAG_HOOK) {
> diff --git a/migration/trace-events b/migration/trace-events
> index 36e20b312d..b821041281 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -77,9 +77,15 @@ migration_bitmap_sync_start(void) ""
> migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
> migration_throttle(void) ""
> multifd_recv(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "channel %d seq number %d pages %d flags 0x%x"
> +multifd_recv_sync_main(uint32_t seq) "seq %d"
> +multifd_recv_sync_main_signal(uint8_t id) "channel %d"
> +multifd_recv_sync_main_wait(uint8_t id) "channel %d"
> multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
> multifd_recv_thread_start(uint8_t id) "%d"
> multifd_send(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "channel %d seq number %d pages %d flags 0x%x"
> +multifd_send_sync_main(uint32_t seq) "seq %d"
> +multifd_send_sync_main_signal(uint8_t id) "channel %d"
> +multifd_send_sync_main_wait(uint8_t id) "channel %d"
> multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "channel %d packets %d pages %d"
> multifd_send_thread_start(uint8_t id) "%d"
> ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: start: %" PRIx64 " %zx"
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 09/12] migration: Start sending messages
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 09/12] migration: Start sending messages Juan Quintela
@ 2018-06-11 15:49 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-11 15:49 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> migration/ram.c | 29 ++++++++++++++++++++++++-----
> 1 file changed, 24 insertions(+), 5 deletions(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index 28f5cea4d8..2584130c85 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -729,9 +729,6 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp)
> RAMBlock *block;
> int i;
>
> - /* ToDo: We can't use it until we haven't received a message */
> - return 0;
> -
> be32_to_cpus(&packet->magic);
> if (packet->magic != MULTIFD_MAGIC) {
> error_setg(errp, "multifd: received packet "
> @@ -965,6 +962,7 @@ static void *multifd_send_thread(void *opaque)
> {
> MultiFDSendParams *p = opaque;
> Error *local_err = NULL;
> + int ret;
>
> trace_multifd_send_thread_start(p->id);
>
> @@ -992,7 +990,16 @@ static void *multifd_send_thread(void *opaque)
>
> trace_multifd_send(p->id, seq, used, flags);
>
> - /* ToDo: send packet here */
> + ret = qio_channel_write_all(p->c, (void *)p->packet,
> + p->packet_len, &local_err);
> + if (ret != 0) {
> + break;
> + }
> +
> + ret = qio_channel_writev_all(p->c, p->pages->iov, used, &local_err);
> + if (ret != 0) {
> + break;
> + }
>
> qemu_mutex_lock(&p->mutex);
> p->pending_job--;
> @@ -1204,7 +1211,14 @@ static void *multifd_recv_thread(void *opaque)
> uint32_t flags;
> qemu_mutex_unlock(&p->mutex);
>
> - /* ToDo: recv packet here */
> + ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
> + p->packet_len, &local_err);
> + if (ret == 0) { /* EOF */
> + break;
> + }
> + if (ret == -1) { /* Error */
> + break;
> + }
>
> qemu_mutex_lock(&p->mutex);
> ret = multifd_recv_unfill_packet(p, &local_err);
> @@ -1221,6 +1235,11 @@ static void *multifd_recv_thread(void *opaque)
> p->num_pages += used;
> qemu_mutex_unlock(&p->mutex);
>
> + ret = qio_channel_readv_all(p->c, p->pages->iov, used, &local_err);
> + if (ret != 0) {
> + break;
> + }
> +
> if (flags & MULTIFD_FLAG_SYNC) {
> qemu_sem_post(&multifd_recv_state->sem_sync);
> qemu_sem_wait(&p->sem_sync);
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 10/12] migration: Wait for blocking IO
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 10/12] migration: Wait for blocking IO Juan Quintela
@ 2018-06-11 15:54 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-11 15:54 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> We have three conditions here:
> - channel fails -> error
> - we have to quit: we close the channel and reads fails
> - normal read that success, we are in bussiness
>
> So forget the complications of waiting in a semaphore.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
I'd still prefer this to be squashed into the earlier ones somehow, but
the result looks OK afterwards I think.
Dave
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> migration/ram.c | 81 ++++++++++++++++++-------------------------------
> 1 file changed, 29 insertions(+), 52 deletions(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index 2584130c85..a707d3ae80 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -595,8 +595,6 @@ typedef struct {
> bool running;
> /* should this thread finish */
> bool quit;
> - /* thread has work to do */
> - bool pending_job;
> /* array of pages to receive */
> MultiFDPages_t *pages;
> /* packet allocated len */
> @@ -1170,14 +1168,6 @@ static void multifd_recv_sync_main(void)
> for (i = 0; i < migrate_multifd_channels(); i++) {
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> - trace_multifd_recv_sync_main_signal(p->id);
> - qemu_mutex_lock(&p->mutex);
> - p->pending_job = true;
> - qemu_mutex_unlock(&p->mutex);
> - }
> - for (i = 0; i < migrate_multifd_channels(); i++) {
> - MultiFDRecvParams *p = &multifd_recv_state->params[i];
> -
> trace_multifd_recv_sync_main_wait(p->id);
> qemu_sem_wait(&multifd_recv_state->sem_sync);
> qemu_mutex_lock(&p->mutex);
> @@ -1190,7 +1180,6 @@ static void multifd_recv_sync_main(void)
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> trace_multifd_recv_sync_main_signal(p->id);
> -
> qemu_sem_post(&p->sem_sync);
> }
> trace_multifd_recv_sync_main(multifd_recv_state->seq);
> @@ -1205,51 +1194,40 @@ static void *multifd_recv_thread(void *opaque)
> trace_multifd_recv_thread_start(p->id);
>
> while (true) {
> + uint32_t used;
> + uint32_t flags;
> +
> + ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
> + p->packet_len, &local_err);
> + if (ret == 0) { /* EOF */
> + break;
> + }
> + if (ret == -1) { /* Error */
> + break;
> + }
> +
> qemu_mutex_lock(&p->mutex);
> - if (true || p->pending_job) {
> - uint32_t used;
> - uint32_t flags;
> - qemu_mutex_unlock(&p->mutex);
> -
> - ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
> - p->packet_len, &local_err);
> - if (ret == 0) { /* EOF */
> - break;
> - }
> - if (ret == -1) { /* Error */
> - break;
> - }
> -
> - qemu_mutex_lock(&p->mutex);
> - ret = multifd_recv_unfill_packet(p, &local_err);
> - if (ret) {
> - qemu_mutex_unlock(&p->mutex);
> - break;
> - }
> -
> - used = p->pages->used;
> - flags = p->flags;
> - trace_multifd_recv(p->id, p->seq, used, flags);
> - p->pending_job = false;
> - p->num_packets++;
> - p->num_pages += used;
> + ret = multifd_recv_unfill_packet(p, &local_err);
> + if (ret) {
> qemu_mutex_unlock(&p->mutex);
> + break;
> + }
>
> - ret = qio_channel_readv_all(p->c, p->pages->iov, used, &local_err);
> - if (ret != 0) {
> - break;
> - }
> + used = p->pages->used;
> + flags = p->flags;
> + trace_multifd_recv(p->id, p->seq, used, flags);
> + p->num_packets++;
> + p->num_pages += used;
> + qemu_mutex_unlock(&p->mutex);
>
> - if (flags & MULTIFD_FLAG_SYNC) {
> - qemu_sem_post(&multifd_recv_state->sem_sync);
> - qemu_sem_wait(&p->sem_sync);
> - }
> - } else if (p->quit) {
> - qemu_mutex_unlock(&p->mutex);
> + ret = qio_channel_readv_all(p->c, p->pages->iov, used, &local_err);
> + if (ret != 0) {
> break;
> - } else {
> - qemu_mutex_unlock(&p->mutex);
> - /* sometimes there are spurious wakeups */
> + }
> +
> + if (flags & MULTIFD_FLAG_SYNC) {
> + qemu_sem_post(&multifd_recv_state->sem_sync);
> + qemu_sem_wait(&p->sem_sync);
> }
> }
>
> @@ -1287,7 +1265,6 @@ int multifd_load_setup(void)
> qemu_sem_init(&p->sem, 0);
> qemu_sem_init(&p->sem_sync, 0);
> p->quit = false;
> - p->pending_job = false;
> p->id = i;
> p->pages = multifd_pages_init(page_count);
> p->packet_len = sizeof(MultiFDPacket_t)
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit Juan Quintela
@ 2018-06-11 16:45 ` Dr. David Alan Gilbert
2018-06-20 7:20 ` Juan Quintela
0 siblings, 1 reply; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-11 16:45 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> We know quit closing the QIO.
This patch does two different things; one of which I think I understand.
The 'quit' has been removed - I think that makes sense because the
multifd threads terminate when either they come to the end of a stream
or hit an error; there's no case of asking them to quit explicitly.
The 'sem' was basically for kicking the recv-thread to quit; which again
isn't needed.
Now, what about the object_unref on p->c ?
If I've got this right multifd_recv_terminate_threads is only called in
the error case; but doesn't multifd_load_cleanup also get called in that
case - it does the unref and p->c = NULL as well.
Dave
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
> migration/ram.c | 11 ++---------
> 1 file changed, 2 insertions(+), 9 deletions(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index a707d3ae80..62eea9e02f 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -587,14 +587,10 @@ typedef struct {
> QemuThread thread;
> /* communication channel */
> QIOChannel *c;
> - /* sem where to wait for more work */
> - QemuSemaphore sem;
> /* this mutex protects the following parameters */
> QemuMutex mutex;
> /* is this channel thread running */
> bool running;
> - /* should this thread finish */
> - bool quit;
> /* array of pages to receive */
> MultiFDPages_t *pages;
> /* packet allocated len */
> @@ -1115,8 +1111,8 @@ static void multifd_recv_terminate_threads(Error *err)
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> qemu_mutex_lock(&p->mutex);
> - p->quit = true;
> - qemu_sem_post(&p->sem);
> + object_unref(OBJECT(p->c));
> + p->c = NULL;
> qemu_mutex_unlock(&p->mutex);
> }
> }
> @@ -1139,7 +1135,6 @@ int multifd_load_cleanup(Error **errp)
> object_unref(OBJECT(p->c));
> p->c = NULL;
> qemu_mutex_destroy(&p->mutex);
> - qemu_sem_destroy(&p->sem);
> qemu_sem_destroy(&p->sem_sync);
> g_free(p->name);
> p->name = NULL;
> @@ -1262,9 +1257,7 @@ int multifd_load_setup(void)
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> qemu_mutex_init(&p->mutex);
> - qemu_sem_init(&p->sem, 0);
> qemu_sem_init(&p->sem_sync, 0);
> - p->quit = false;
> p->id = i;
> p->pages = multifd_pages_init(page_count);
> p->packet_len = sizeof(MultiFDPacket_t)
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 01/12] migration: Create multipage support
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 01/12] migration: Create multipage support Juan Quintela
@ 2018-06-11 16:58 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-11 16:58 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> We only create/destry the page list here. We will use it later.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> migration/ram.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 57 insertions(+)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index 5bcbf7a9f9..23cc5625eb 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -510,6 +510,20 @@ typedef struct {
> uint8_t id;
> } __attribute__((packed)) MultiFDInit_t;
>
> +typedef struct {
> + /* number of used pages */
> + uint32_t used;
> + /* number of allocated pages */
> + uint32_t allocated;
> + /* global number of generated multifd packets */
> + uint32_t seq;
> + /* offset of each page */
> + ram_addr_t *offset;
> + /* pointer to each page */
> + struct iovec *iov;
> + RAMBlock *block;
> +} MultiFDPages_t;
> +
> typedef struct {
> /* this fields are not changed once the thread is created */
> /* channel number */
> @@ -528,6 +542,8 @@ typedef struct {
> bool running;
> /* should this thread finish */
> bool quit;
> + /* array of pages to sent */
> + MultiFDPages_t *pages;
> } MultiFDSendParams;
>
> typedef struct {
> @@ -548,6 +564,8 @@ typedef struct {
> bool running;
> /* should this thread finish */
> bool quit;
> + /* array of pages to receive */
> + MultiFDPages_t *pages;
> } MultiFDRecvParams;
>
> static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
> @@ -612,10 +630,36 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp)
> return msg.id;
> }
>
> +static MultiFDPages_t *multifd_pages_init(size_t size)
> +{
> + MultiFDPages_t *pages = g_new0(MultiFDPages_t, 1);
> +
> + pages->allocated = size;
> + pages->iov = g_new0(struct iovec, size);
> + pages->offset = g_new0(ram_addr_t, size);
> +
> + return pages;
> +}
> +
> +static void multifd_pages_clear(MultiFDPages_t *pages)
> +{
> + pages->used = 0;
> + pages->allocated = 0;
> + pages->seq = 0;
> + pages->block = NULL;
> + g_free(pages->iov);
> + pages->iov = NULL;
> + g_free(pages->offset);
> + pages->offset = NULL;
> + g_free(pages);
> +}
> +
> struct {
> MultiFDSendParams *params;
> /* number of created threads */
> int count;
> + /* array of pages to sent */
> + MultiFDPages_t *pages;
> } *multifd_send_state;
>
> static void multifd_send_terminate_threads(Error *err)
> @@ -665,9 +709,13 @@ int multifd_save_cleanup(Error **errp)
> qemu_sem_destroy(&p->sem);
> g_free(p->name);
> p->name = NULL;
> + multifd_pages_clear(p->pages);
> + p->pages = NULL;
> }
> g_free(multifd_send_state->params);
> multifd_send_state->params = NULL;
> + multifd_pages_clear(multifd_send_state->pages);
> + multifd_send_state->pages = NULL;
> g_free(multifd_send_state);
> multifd_send_state = NULL;
> return ret;
> @@ -728,6 +776,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
> int multifd_save_setup(void)
> {
> int thread_count;
> + uint32_t page_count = migrate_multifd_page_count();
> uint8_t i;
>
> if (!migrate_use_multifd()) {
> @@ -737,6 +786,8 @@ int multifd_save_setup(void)
> multifd_send_state = g_malloc0(sizeof(*multifd_send_state));
> multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
> atomic_set(&multifd_send_state->count, 0);
> + multifd_send_state->pages = multifd_pages_init(page_count);
> +
> for (i = 0; i < thread_count; i++) {
> MultiFDSendParams *p = &multifd_send_state->params[i];
>
> @@ -744,6 +795,7 @@ int multifd_save_setup(void)
> qemu_sem_init(&p->sem, 0);
> p->quit = false;
> p->id = i;
> + p->pages = multifd_pages_init(page_count);
> p->name = g_strdup_printf("multifdsend_%d", i);
> socket_send_channel_create(multifd_new_send_channel_async, p);
> }
> @@ -801,6 +853,8 @@ int multifd_load_cleanup(Error **errp)
> qemu_sem_destroy(&p->sem);
> g_free(p->name);
> p->name = NULL;
> + multifd_pages_clear(p->pages);
> + p->pages = NULL;
> }
> g_free(multifd_recv_state->params);
> multifd_recv_state->params = NULL;
> @@ -834,6 +888,7 @@ static void *multifd_recv_thread(void *opaque)
> int multifd_load_setup(void)
> {
> int thread_count;
> + uint32_t page_count = migrate_multifd_page_count();
> uint8_t i;
>
> if (!migrate_use_multifd()) {
> @@ -843,6 +898,7 @@ int multifd_load_setup(void)
> multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state));
> multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count);
> atomic_set(&multifd_recv_state->count, 0);
> +
> for (i = 0; i < thread_count; i++) {
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> @@ -850,6 +906,7 @@ int multifd_load_setup(void)
> qemu_sem_init(&p->sem, 0);
> p->quit = false;
> p->id = i;
> + p->pages = multifd_pages_init(page_count);
> p->name = g_strdup_printf("multifdrecv_%d", i);
> }
> return 0;
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 05/12] migration: Multifd channels always wait on the sem
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 05/12] migration: Multifd channels always wait on the sem Juan Quintela
@ 2018-06-11 17:13 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-11 17:13 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> Either for quit, sync or packet, we first wake them.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
Again, I'd prefer if this was merged in with later stuff that
changes it heavily; but I realise it's difficult to split, and those
do clean it up, so:
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> migration/ram.c | 13 +++++++++++--
> 1 file changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index ab715cc148..90f62144e2 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -868,6 +868,7 @@ static void *multifd_send_thread(void *opaque)
> p->num_packets = 1;
>
> while (true) {
> + qemu_sem_wait(&p->sem);
> qemu_mutex_lock(&p->mutex);
> multifd_send_fill_packet(p);
> if (p->quit) {
> @@ -875,7 +876,9 @@ static void *multifd_send_thread(void *opaque)
> break;
> }
> qemu_mutex_unlock(&p->mutex);
> - qemu_sem_wait(&p->sem);
> + /* this is impossible */
> + error_setg(&local_err, "multifd_send_thread: Unknown command");
> + break;
> }
>
> out:
> @@ -1019,6 +1022,7 @@ static void *multifd_recv_thread(void *opaque)
> trace_multifd_recv_thread_start(p->id);
>
> while (true) {
> + qemu_sem_wait(&p->sem);
> qemu_mutex_lock(&p->mutex);
> if (false) {
> /* ToDo: Packet reception goes here */
> @@ -1033,9 +1037,14 @@ static void *multifd_recv_thread(void *opaque)
> break;
> }
> qemu_mutex_unlock(&p->mutex);
> - qemu_sem_wait(&p->sem);
> + /* this is impossible */
> + error_setg(&local_err, "multifd_recv_thread: Unknown command");
> + break;
> }
>
> + if (local_err) {
> + multifd_recv_terminate_threads(local_err);
> + }
> qemu_mutex_lock(&p->mutex);
> p->running = false;
> qemu_mutex_unlock(&p->mutex);
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 08/12] migration: Create ram_save_multifd_page
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 08/12] migration: Create ram_save_multifd_page Juan Quintela
@ 2018-06-11 17:33 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-11 17:33 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> The function still don't use multifd, but we have simplified
> ram_save_page, xbzrle and RDMA stuff is gone. We have added a new
> counter.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
>
> --
> Add last_page parameter
> Add commets for done and address
> Remove multifd field, it is the same than normal pages
> Merge next patch, now we send multiple pages at a time
> Remove counter for multifd pages, it is identical to normal pages
> Use iovec's instead of creating the equivalent.
> Clear memory used by pages (dave)
> Use g_new0(danp)
> define MULTIFD_CONTINUE
> now pages member is a pointer
> Fix off-by-one in number of pages in one packet
> Remove RAM_SAVE_FLAG_MULTIFD_PAGE
> s/multifd_pages_t/MultiFDPages_t/
> ---
> migration/ram.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 85 insertions(+)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index 3e99d48123..28f5cea4d8 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -55,6 +55,7 @@
> #include "sysemu/sysemu.h"
> #include "qemu/uuid.h"
> #include "savevm.h"
> +#include "qemu/iov.h"
>
> /***********************************************************/
> /* ram save/restore */
> @@ -804,8 +805,65 @@ struct {
> QemuSemaphore sem_sync;
> /* global number of generated multifd packets */
> uint32_t seq;
> + /* send channels ready */
> + QemuSemaphore channels_ready;
> } *multifd_send_state;
>
> +static void multifd_send_pages(void)
> +{
> + int i;
> + static int next_channel;
> + MultiFDSendParams *p = NULL; /* make happy gcc */
> + MultiFDPages_t *pages = multifd_send_state->pages;
> +
> + qemu_sem_wait(&multifd_send_state->channels_ready);
> + for (i = next_channel;; i = (i + 1) % migrate_multifd_channels()) {
> + p = &multifd_send_state->params[i];
> +
> + qemu_mutex_lock(&p->mutex);
> + if (!p->pending_job) {
> + p->pending_job++;
> + next_channel = (i + 1) % migrate_multifd_channels();
> + break;
> + }
> + qemu_mutex_unlock(&p->mutex);
> + }
> + p->pages->used = 0;
> + multifd_send_state->seq++;
> + p->seq = multifd_send_state->seq;
> + p->pages->block = NULL;
> + multifd_send_state->pages = p->pages;
> + p->pages = pages;
This function *really* needs a fat comment about the flow of ->pages
because it's pretty hard to follow.
Other than that;
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> + qemu_mutex_unlock(&p->mutex);
> + qemu_sem_post(&p->sem);
> +}
> +
> +static void multifd_queue_page(RAMBlock *block, ram_addr_t offset)
> +{
> + MultiFDPages_t *pages = multifd_send_state->pages;
> +
> + if (!pages->block) {
> + pages->block = block;
> + }
> +
> + if (pages->block == block) {
> + pages->offset[pages->used] = offset;
> + pages->iov[pages->used].iov_base = block->host + offset;
> + pages->iov[pages->used].iov_len = TARGET_PAGE_SIZE;
> + pages->used++;
> +
> + if (pages->used < pages->allocated) {
> + return;
> + }
> + }
> +
> + multifd_send_pages();
> +
> + if (pages->block != block) {
> + multifd_queue_page(block, offset);
> + }
> +}
> +
> static void multifd_send_terminate_threads(Error *err)
> {
> int i;
> @@ -860,6 +918,7 @@ int multifd_save_cleanup(Error **errp)
> g_free(p->packet);
> p->packet = NULL;
> }
> + qemu_sem_destroy(&multifd_send_state->channels_ready);
> qemu_sem_destroy(&multifd_send_state->sem_sync);
> g_free(multifd_send_state->params);
> multifd_send_state->params = NULL;
> @@ -877,12 +936,17 @@ static void multifd_send_sync_main(void)
> if (!migrate_use_multifd()) {
> return;
> }
> + if (multifd_send_state->pages->used) {
> + multifd_send_pages();
> + }
> for (i = 0; i < migrate_multifd_channels(); i++) {
> MultiFDSendParams *p = &multifd_send_state->params[i];
>
> trace_multifd_send_sync_main_signal(p->id);
>
> qemu_mutex_lock(&p->mutex);
> + multifd_send_state->seq++;
> + p->seq = multifd_send_state->seq;
> p->flags |= MULTIFD_FLAG_SYNC;
> p->pending_job++;
> qemu_mutex_unlock(&p->mutex);
> @@ -937,6 +1001,7 @@ static void *multifd_send_thread(void *opaque)
> if (flags & MULTIFD_FLAG_SYNC) {
> qemu_sem_post(&multifd_send_state->sem_sync);
> }
> + qemu_sem_post(&multifd_send_state->channels_ready);
> } else if (p->quit) {
> qemu_mutex_unlock(&p->mutex);
> break;
> @@ -996,6 +1061,7 @@ int multifd_save_setup(void)
> atomic_set(&multifd_send_state->count, 0);
> multifd_send_state->pages = multifd_pages_init(page_count);
> qemu_sem_init(&multifd_send_state->sem_sync, 0);
> + qemu_sem_init(&multifd_send_state->channels_ready, 0);
>
> for (i = 0; i < thread_count; i++) {
> MultiFDSendParams *p = &multifd_send_state->params[i];
> @@ -1713,6 +1779,23 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage)
> return pages;
> }
>
> +static int ram_save_multifd_page(RAMState *rs, RAMBlock *block,
> + ram_addr_t offset)
> +{
> + uint8_t *p;
> +
> + p = block->host + offset;
> +
> + ram_counters.transferred += save_page_header(rs, rs->f, block,
> + offset | RAM_SAVE_FLAG_PAGE);
> + multifd_queue_page(block, offset);
> + qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE);
> + ram_counters.transferred += TARGET_PAGE_SIZE;
> + ram_counters.normal++;
> +
> + return 1;
> +}
> +
> static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block,
> ram_addr_t offset, uint8_t *source_buf)
> {
> @@ -2116,6 +2199,8 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss,
> */
> if (block == rs->last_sent_block && save_page_use_compression(rs)) {
> return compress_page_with_multi_thread(rs, block, offset);
> + } else if (migrate_use_multifd()) {
> + return ram_save_multifd_page(rs, block, offset);
> }
>
> return ram_save_page(rs, pss, last_stage);
> --
> 2.17.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-06-11 16:45 ` Dr. David Alan Gilbert
@ 2018-06-20 7:20 ` Juan Quintela
2018-06-20 9:01 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-06-20 7:20 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: qemu-devel, lvivier, peterx
"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> We know quit closing the QIO.
>
> This patch does two different things; one of which I think I understand.
>
> The 'quit' has been removed - I think that makes sense because the
> multifd threads terminate when either they come to the end of a stream
> or hit an error; there's no case of asking them to quit explicitly.
> The 'sem' was basically for kicking the recv-thread to quit; which again
> isn't needed.
>
> Now, what about the object_unref on p->c ?
> If I've got this right multifd_recv_terminate_threads is only called in
> the error case; but doesn't multifd_load_cleanup also get called in that
> case - it does the unref and p->c = NULL as well.
Adding another comment O:-)
In normal exit case: we don't care.
In error case, we do a close of the p->c. Doing a close, means that the
channel threads that are waiting on qio_channel_read_all_eof() will stop
the waiting and return an error, so we are safe.
ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
p->packet_len, &local_err);
if (ret == 0) { /* EOF */
break;
}
if (ret == -1) { /* Error */
break;
}
We have to wait for three things:
- we receive a synchronization packet
- we receive EOF and finish
- we got somehow an error and have to quit.
Old versions have a semaphore, and we have three places where we set
that semaphore "we are ready", when we receive one error, when we have
data waiting to be read (it had a qio whatcher) or we just got a normal
exit.
Waiting on two things make code more complex that it used to be (and
whatchers are really lame, because we know there are data ready, but not
how much, dealing with non-whole packets/pages is a real mess). So we
moved to:
- quit is gone: we just close the channel, that makes the *_read_eof()
to end. Notice that we don't care if the close is due to one error
or because we have finished. It is just done.
- Now reception channel only have to wait on _read_eof(), the three
cases that we talked before are handled correctly.
I understand the confusion, this only makes sense when you came from the
previous version of the patchset.
Later, Juan.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-06-20 7:20 ` Juan Quintela
@ 2018-06-20 9:01 ` Dr. David Alan Gilbert
2018-06-20 9:42 ` Juan Quintela
0 siblings, 1 reply; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-20 9:01 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> > * Juan Quintela (quintela@redhat.com) wrote:
> >> We know quit closing the QIO.
> >
> > This patch does two different things; one of which I think I understand.
> >
> > The 'quit' has been removed - I think that makes sense because the
> > multifd threads terminate when either they come to the end of a stream
> > or hit an error; there's no case of asking them to quit explicitly.
> > The 'sem' was basically for kicking the recv-thread to quit; which again
> > isn't needed.
> >
> > Now, what about the object_unref on p->c ?
> > If I've got this right multifd_recv_terminate_threads is only called in
> > the error case; but doesn't multifd_load_cleanup also get called in that
> > case - it does the unref and p->c = NULL as well.
>
> Adding another comment O:-)
>
> In normal exit case: we don't care.
>
> In error case, we do a close of the p->c. Doing a close, means that the
> channel threads that are waiting on qio_channel_read_all_eof() will stop
> the waiting and return an error, so we are safe.
>
> ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
> p->packet_len, &local_err);
> if (ret == 0) { /* EOF */
> break;
> }
> if (ret == -1) { /* Error */
> break;
> }
>
> We have to wait for three things:
> - we receive a synchronization packet
> - we receive EOF and finish
> - we got somehow an error and have to quit.
>
> Old versions have a semaphore, and we have three places where we set
> that semaphore "we are ready", when we receive one error, when we have
> data waiting to be read (it had a qio whatcher) or we just got a normal
> exit.
>
> Waiting on two things make code more complex that it used to be (and
> whatchers are really lame, because we know there are data ready, but not
> how much, dealing with non-whole packets/pages is a real mess). So we
> moved to:
> - quit is gone: we just close the channel, that makes the *_read_eof()
> to end. Notice that we don't care if the close is due to one error
> or because we have finished. It is just done.
> - Now reception channel only have to wait on _read_eof(), the three
> cases that we talked before are handled correctly.
>
> I understand the confusion, this only makes sense when you came from the
> previous version of the patchset.
Two questions from that then:
a) Are you sure it's safe to close the qio_channel while another
thread is in qio_channel_read_all_eof? Is it really defined that it
causes the other thread to exit with an error; close() in some stuff
frees data structures that the other thread is still reading; that's
why I've used shutdown(2) in the past rather than close on fd's
b) I don't think your answer explains why it's an object_unref?
Dave
> Later, Juan.
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-06-20 9:01 ` Dr. David Alan Gilbert
@ 2018-06-20 9:42 ` Juan Quintela
2018-06-20 9:46 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-06-20 9:42 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: qemu-devel, lvivier, peterx
"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
>> > * Juan Quintela (quintela@redhat.com) wrote:
>> >> We know quit closing the QIO.
>> >
>> > This patch does two different things; one of which I think I understand.
>> >
>> > The 'quit' has been removed - I think that makes sense because the
>> > multifd threads terminate when either they come to the end of a stream
>> > or hit an error; there's no case of asking them to quit explicitly.
>> > The 'sem' was basically for kicking the recv-thread to quit; which again
>> > isn't needed.
>> >
>> > Now, what about the object_unref on p->c ?
>> > If I've got this right multifd_recv_terminate_threads is only called in
>> > the error case; but doesn't multifd_load_cleanup also get called in that
>> > case - it does the unref and p->c = NULL as well.
>>
>> Adding another comment O:-)
>>
>> In normal exit case: we don't care.
>>
>> In error case, we do a close of the p->c. Doing a close, means that the
>> channel threads that are waiting on qio_channel_read_all_eof() will stop
>> the waiting and return an error, so we are safe.
>>
>> ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
>> p->packet_len, &local_err);
>> if (ret == 0) { /* EOF */
>> break;
>> }
>> if (ret == -1) { /* Error */
>> break;
>> }
>>
>> We have to wait for three things:
>> - we receive a synchronization packet
>> - we receive EOF and finish
>> - we got somehow an error and have to quit.
>>
>> Old versions have a semaphore, and we have three places where we set
>> that semaphore "we are ready", when we receive one error, when we have
>> data waiting to be read (it had a qio whatcher) or we just got a normal
>> exit.
>>
>> Waiting on two things make code more complex that it used to be (and
>> whatchers are really lame, because we know there are data ready, but not
>> how much, dealing with non-whole packets/pages is a real mess). So we
>> moved to:
>> - quit is gone: we just close the channel, that makes the *_read_eof()
>> to end. Notice that we don't care if the close is due to one error
>> or because we have finished. It is just done.
>> - Now reception channel only have to wait on _read_eof(), the three
>> cases that we talked before are handled correctly.
>>
>> I understand the confusion, this only makes sense when you came from the
>> previous version of the patchset.
>
> Two questions from that then:
> a) Are you sure it's safe to close the qio_channel while another
> thread is in qio_channel_read_all_eof? Is it really defined that it
> causes the other thread to exit with an error; close() in some stuff
> frees data structures that the other thread is still reading; that's
> why I've used shutdown(2) in the past rather than close on fd's
Dunno if it is safe (I think it is), but I agree that shutdown will also
get what I need.
> b) I don't think your answer explains why it's an object_unref?
That is the standard way to closing qios to not have to take into
account who have it oppened. See previous paragraph, it is better to
use shutdown, done.
Thanks, Juan.
> Dave
>
>> Later, Juan.
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-06-20 9:42 ` Juan Quintela
@ 2018-06-20 9:46 ` Dr. David Alan Gilbert
2018-06-20 9:48 ` Juan Quintela
0 siblings, 1 reply; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-20 9:46 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> > * Juan Quintela (quintela@redhat.com) wrote:
> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> >> > * Juan Quintela (quintela@redhat.com) wrote:
> >> >> We know quit closing the QIO.
> >> >
> >> > This patch does two different things; one of which I think I understand.
> >> >
> >> > The 'quit' has been removed - I think that makes sense because the
> >> > multifd threads terminate when either they come to the end of a stream
> >> > or hit an error; there's no case of asking them to quit explicitly.
> >> > The 'sem' was basically for kicking the recv-thread to quit; which again
> >> > isn't needed.
> >> >
> >> > Now, what about the object_unref on p->c ?
> >> > If I've got this right multifd_recv_terminate_threads is only called in
> >> > the error case; but doesn't multifd_load_cleanup also get called in that
> >> > case - it does the unref and p->c = NULL as well.
> >>
> >> Adding another comment O:-)
> >>
> >> In normal exit case: we don't care.
> >>
> >> In error case, we do a close of the p->c. Doing a close, means that the
> >> channel threads that are waiting on qio_channel_read_all_eof() will stop
> >> the waiting and return an error, so we are safe.
> >>
> >> ret = qio_channel_read_all_eof(p->c, (void *)p->packet,
> >> p->packet_len, &local_err);
> >> if (ret == 0) { /* EOF */
> >> break;
> >> }
> >> if (ret == -1) { /* Error */
> >> break;
> >> }
> >>
> >> We have to wait for three things:
> >> - we receive a synchronization packet
> >> - we receive EOF and finish
> >> - we got somehow an error and have to quit.
> >>
> >> Old versions have a semaphore, and we have three places where we set
> >> that semaphore "we are ready", when we receive one error, when we have
> >> data waiting to be read (it had a qio whatcher) or we just got a normal
> >> exit.
> >>
> >> Waiting on two things make code more complex that it used to be (and
> >> whatchers are really lame, because we know there are data ready, but not
> >> how much, dealing with non-whole packets/pages is a real mess). So we
> >> moved to:
> >> - quit is gone: we just close the channel, that makes the *_read_eof()
> >> to end. Notice that we don't care if the close is due to one error
> >> or because we have finished. It is just done.
> >> - Now reception channel only have to wait on _read_eof(), the three
> >> cases that we talked before are handled correctly.
> >>
> >> I understand the confusion, this only makes sense when you came from the
> >> previous version of the patchset.
> >
> > Two questions from that then:
> > a) Are you sure it's safe to close the qio_channel while another
> > thread is in qio_channel_read_all_eof? Is it really defined that it
> > causes the other thread to exit with an error; close() in some stuff
> > frees data structures that the other thread is still reading; that's
> > why I've used shutdown(2) in the past rather than close on fd's
>
> Dunno if it is safe (I think it is), but I agree that shutdown will also
> get what I need.
>
> > b) I don't think your answer explains why it's an object_unref?
>
> That is the standard way to closing qios to not have to take into
> account who have it oppened. See previous paragraph, it is better to
> use shutdown, done.
OK, great; I suspect it's unsafe because as soon as you do the unref
it could free the object; actually you should have a ref from each of
the threads to sotp it being freed while they use it.
Dave
> Thanks, Juan.
>
> > Dave
> >
> >> Later, Juan.
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-06-20 9:46 ` Dr. David Alan Gilbert
@ 2018-06-20 9:48 ` Juan Quintela
2018-06-20 10:38 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-06-20 9:48 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: qemu-devel, lvivier, peterx
"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
>> > * Juan Quintela (quintela@redhat.com) wrote:
>> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
>> >> > * Juan Quintela (quintela@redhat.com) wrote:
>> >> >> We know quit closing the QIO.
...
>> > Two questions from that then:
>> > a) Are you sure it's safe to close the qio_channel while another
>> > thread is in qio_channel_read_all_eof? Is it really defined that it
>> > causes the other thread to exit with an error; close() in some stuff
>> > frees data structures that the other thread is still reading; that's
>> > why I've used shutdown(2) in the past rather than close on fd's
>>
>> Dunno if it is safe (I think it is), but I agree that shutdown will also
>> get what I need.
>>
>> > b) I don't think your answer explains why it's an object_unref?
>>
>> That is the standard way to closing qios to not have to take into
>> account who have it oppened. See previous paragraph, it is better to
>> use shutdown, done.
>
> OK, great; I suspect it's unsafe because as soon as you do the unref
> it could free the object; actually you should have a ref from each of
> the threads to sotp it being freed while they use it.
>
> Dave
>
>> Thanks, Juan.
>>
>> > Dave
>> >
>> >> Later, Juan.
>> > --
>> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
What do you think about this, to avoid me resend the whole series?
>From e03d77de1ca179fa0168cead7c23cfeae57f1787 Mon Sep 17 00:00:00 2001
From: Juan Quintela <quintela@redhat.com>
Date: Wed, 18 Apr 2018 00:49:19 +0200
Subject: [PATCH 17/18] migration: Remove not needed semaphore and quit
We know quit with shutdwon in the QIO.
Signed-off-by: Juan Quintela <quintela@redhat.com>
--
Add comment
Use shutdown() instead of unref()
---
migration/ram.c | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 2c3a452a7d..be5d26f4cb 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -594,14 +594,10 @@ typedef struct {
QemuThread thread;
/* communication channel */
QIOChannel *c;
- /* sem where to wait for more work */
- QemuSemaphore sem;
/* this mutex protects the following parameters */
QemuMutex mutex;
/* is this channel thread running */
bool running;
- /* should this thread finish */
- bool quit;
/* array of pages to receive */
MultiFDPages_t *pages;
/* packet allocated len */
@@ -1152,8 +1148,11 @@ static void multifd_recv_terminate_threads(Error *err)
MultiFDRecvParams *p = &multifd_recv_state->params[i];
qemu_mutex_lock(&p->mutex);
- p->quit = true;
- qemu_sem_post(&p->sem);
+ /* We could arrive here for two reasons:
+ - normal quit, i.e. everything went fine, just finished
+ - error quit: We close the channels so the channel threads
+ finish the qio_channel_read_all_eof() */
+ qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
qemu_mutex_unlock(&p->mutex);
}
}
@@ -1176,7 +1175,6 @@ int multifd_load_cleanup(Error **errp)
object_unref(OBJECT(p->c));
p->c = NULL;
qemu_mutex_destroy(&p->mutex);
- qemu_sem_destroy(&p->sem);
qemu_sem_destroy(&p->sem_sync);
g_free(p->name);
p->name = NULL;
@@ -1299,9 +1297,7 @@ int multifd_load_setup(void)
MultiFDRecvParams *p = &multifd_recv_state->params[i];
qemu_mutex_init(&p->mutex);
- qemu_sem_init(&p->sem, 0);
qemu_sem_init(&p->sem_sync, 0);
- p->quit = false;
p->id = i;
p->pages = multifd_pages_init(page_count);
p->packet_len = sizeof(MultiFDPacket_t)
--
2.17.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-06-20 9:48 ` Juan Quintela
@ 2018-06-20 10:38 ` Dr. David Alan Gilbert
2018-06-20 11:07 ` Juan Quintela
0 siblings, 1 reply; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-20 10:38 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> > * Juan Quintela (quintela@redhat.com) wrote:
> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> >> > * Juan Quintela (quintela@redhat.com) wrote:
> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> >> >> > * Juan Quintela (quintela@redhat.com) wrote:
> >> >> >> We know quit closing the QIO.
>
> ...
> >> > Two questions from that then:
> >> > a) Are you sure it's safe to close the qio_channel while another
> >> > thread is in qio_channel_read_all_eof? Is it really defined that it
> >> > causes the other thread to exit with an error; close() in some stuff
> >> > frees data structures that the other thread is still reading; that's
> >> > why I've used shutdown(2) in the past rather than close on fd's
> >>
> >> Dunno if it is safe (I think it is), but I agree that shutdown will also
> >> get what I need.
> >>
> >> > b) I don't think your answer explains why it's an object_unref?
> >>
> >> That is the standard way to closing qios to not have to take into
> >> account who have it oppened. See previous paragraph, it is better to
> >> use shutdown, done.
> >
> > OK, great; I suspect it's unsafe because as soon as you do the unref
> > it could free the object; actually you should have a ref from each of
> > the threads to sotp it being freed while they use it.
> >
> > Dave
> >
> >> Thanks, Juan.
> >>
> >> > Dave
> >> >
> >> >> Later, Juan.
> >> > --
> >> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>
> What do you think about this, to avoid me resend the whole series?
>
> From e03d77de1ca179fa0168cead7c23cfeae57f1787 Mon Sep 17 00:00:00 2001
> From: Juan Quintela <quintela@redhat.com>
> Date: Wed, 18 Apr 2018 00:49:19 +0200
> Subject: [PATCH 17/18] migration: Remove not needed semaphore and quit
>
> We know quit with shutdwon in the QIO.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> --
> Add comment
> Use shutdown() instead of unref()
> ---
> migration/ram.c | 14 +++++---------
> 1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index 2c3a452a7d..be5d26f4cb 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -594,14 +594,10 @@ typedef struct {
> QemuThread thread;
> /* communication channel */
> QIOChannel *c;
> - /* sem where to wait for more work */
> - QemuSemaphore sem;
> /* this mutex protects the following parameters */
> QemuMutex mutex;
> /* is this channel thread running */
> bool running;
> - /* should this thread finish */
> - bool quit;
> /* array of pages to receive */
> MultiFDPages_t *pages;
> /* packet allocated len */
> @@ -1152,8 +1148,11 @@ static void multifd_recv_terminate_threads(Error *err)
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> qemu_mutex_lock(&p->mutex);
> - p->quit = true;
> - qemu_sem_post(&p->sem);
> + /* We could arrive here for two reasons:
> + - normal quit, i.e. everything went fine, just finished
> + - error quit: We close the channels so the channel threads
> + finish the qio_channel_read_all_eof() */
> + qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
OK, so with any luck all the threads now exit; do we still have a
close/unref once we're sure they have all exited?
Dave
> qemu_mutex_unlock(&p->mutex);
> }
> }
> @@ -1176,7 +1175,6 @@ int multifd_load_cleanup(Error **errp)
> object_unref(OBJECT(p->c));
> p->c = NULL;
> qemu_mutex_destroy(&p->mutex);
> - qemu_sem_destroy(&p->sem);
> qemu_sem_destroy(&p->sem_sync);
> g_free(p->name);
> p->name = NULL;
> @@ -1299,9 +1297,7 @@ int multifd_load_setup(void)
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> qemu_mutex_init(&p->mutex);
> - qemu_sem_init(&p->sem, 0);
> qemu_sem_init(&p->sem_sync, 0);
> - p->quit = false;
> p->id = i;
> p->pages = multifd_pages_init(page_count);
> p->packet_len = sizeof(MultiFDPacket_t)
> --
> 2.17.1
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-06-20 10:38 ` Dr. David Alan Gilbert
@ 2018-06-20 11:07 ` Juan Quintela
2018-06-20 11:25 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 32+ messages in thread
From: Juan Quintela @ 2018-06-20 11:07 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: qemu-devel, lvivier, peterx
"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
>> > * Juan Quintela (quintela@redhat.com) wrote:
>> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
>> >> > * Juan Quintela (quintela@redhat.com) wrote:
>> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
>> >> >> > * Juan Quintela (quintela@redhat.com) wrote:
>> >> >> >> We know quit closing the QIO.
>>
>> ...
>> >> > Two questions from that then:
>> >> > a) Are you sure it's safe to close the qio_channel while another
>> >> > thread is in qio_channel_read_all_eof? Is it really defined that it
>> >> > causes the other thread to exit with an error; close() in some stuff
>> >> > frees data structures that the other thread is still reading; that's
>> >> > why I've used shutdown(2) in the past rather than close on fd's
>> >>
>> >> Dunno if it is safe (I think it is), but I agree that shutdown will also
>> >> get what I need.
>> >>
>> >> > b) I don't think your answer explains why it's an object_unref?
>> >>
>> >> That is the standard way to closing qios to not have to take into
>> >> account who have it oppened. See previous paragraph, it is better to
>> >> use shutdown, done.
>> >
>> > OK, great; I suspect it's unsafe because as soon as you do the unref
>> > it could free the object; actually you should have a ref from each of
>> > the threads to sotp it being freed while they use it.
>> >
>> > Dave
>> >
>> >> Thanks, Juan.
>> >>
>> >> > Dave
>> >> >
>> >> >> Later, Juan.
>> >> > --
>> >> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>> > --
>> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>
>> What do you think about this, to avoid me resend the whole series?
>>
>> From e03d77de1ca179fa0168cead7c23cfeae57f1787 Mon Sep 17 00:00:00 2001
>> From: Juan Quintela <quintela@redhat.com>
>> Date: Wed, 18 Apr 2018 00:49:19 +0200
>> Subject: [PATCH 17/18] migration: Remove not needed semaphore and quit
>>
>> We know quit with shutdwon in the QIO.
>>
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> --
>> Add comment
>> Use shutdown() instead of unref()
>> ---
>> migration/ram.c | 14 +++++---------
>> 1 file changed, 5 insertions(+), 9 deletions(-)
>>
>> diff --git a/migration/ram.c b/migration/ram.c
>> index 2c3a452a7d..be5d26f4cb 100644
>> --- a/migration/ram.c
>> +++ b/migration/ram.c
>> @@ -594,14 +594,10 @@ typedef struct {
>> QemuThread thread;
>> /* communication channel */
>> QIOChannel *c;
>> - /* sem where to wait for more work */
>> - QemuSemaphore sem;
>> /* this mutex protects the following parameters */
>> QemuMutex mutex;
>> /* is this channel thread running */
>> bool running;
>> - /* should this thread finish */
>> - bool quit;
>> /* array of pages to receive */
>> MultiFDPages_t *pages;
>> /* packet allocated len */
>> @@ -1152,8 +1148,11 @@ static void multifd_recv_terminate_threads(Error *err)
>> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>>
>> qemu_mutex_lock(&p->mutex);
>> - p->quit = true;
>> - qemu_sem_post(&p->sem);
>> + /* We could arrive here for two reasons:
>> + - normal quit, i.e. everything went fine, just finished
>> + - error quit: We close the channels so the channel threads
>> + finish the qio_channel_read_all_eof() */
>> + qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
>
> OK, so with any luck all the threads now exit; do we still have a
> close/unref once we're sure they have all exited?
Yeap.
static void multifd_recv_terminate_threads(Error *err)
{
[...]
for (i = 0; i < migrate_multifd_channels(); i++) {
MultiFDRecvParams *p = &multifd_recv_state->params[i];
qemu_mutex_lock(&p->mutex);
/* We could arrive here for two reasons:
- normal quit, i.e. everything went fine, just finished
- error quit: We close the channels so the channel threads
finish the qio_channel_read_all_eof() */
qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
qemu_mutex_unlock(&p->mutex);
}
}
int multifd_load_cleanup(Error **errp)
{
int i;
int ret = 0;
if (!migrate_use_multifd()) {
return 0;
}
multifd_recv_terminate_threads(NULL);
for (i = 0; i < migrate_multifd_channels(); i++) {
MultiFDRecvParams *p = &multifd_recv_state->params[i];
if (p->running) {
qemu_thread_join(&p->thread);
}
object_unref(OBJECT(p->c));
p->c = NULL;
[...]
}
[...]
}
Omited the things that we don't care about for this.
Thanks, Juan.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit
2018-06-20 11:07 ` Juan Quintela
@ 2018-06-20 11:25 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 32+ messages in thread
From: Dr. David Alan Gilbert @ 2018-06-20 11:25 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, lvivier, peterx
* Juan Quintela (quintela@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> > * Juan Quintela (quintela@redhat.com) wrote:
> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> >> > * Juan Quintela (quintela@redhat.com) wrote:
> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> >> >> > * Juan Quintela (quintela@redhat.com) wrote:
> >> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> >> >> >> > * Juan Quintela (quintela@redhat.com) wrote:
> >> >> >> >> We know quit closing the QIO.
> >>
> >> ...
> >> >> > Two questions from that then:
> >> >> > a) Are you sure it's safe to close the qio_channel while another
> >> >> > thread is in qio_channel_read_all_eof? Is it really defined that it
> >> >> > causes the other thread to exit with an error; close() in some stuff
> >> >> > frees data structures that the other thread is still reading; that's
> >> >> > why I've used shutdown(2) in the past rather than close on fd's
> >> >>
> >> >> Dunno if it is safe (I think it is), but I agree that shutdown will also
> >> >> get what I need.
> >> >>
> >> >> > b) I don't think your answer explains why it's an object_unref?
> >> >>
> >> >> That is the standard way to closing qios to not have to take into
> >> >> account who have it oppened. See previous paragraph, it is better to
> >> >> use shutdown, done.
> >> >
> >> > OK, great; I suspect it's unsafe because as soon as you do the unref
> >> > it could free the object; actually you should have a ref from each of
> >> > the threads to sotp it being freed while they use it.
> >> >
> >> > Dave
> >> >
> >> >> Thanks, Juan.
> >> >>
> >> >> > Dave
> >> >> >
> >> >> >> Later, Juan.
> >> >> > --
> >> >> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >> > --
> >> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >>
> >> What do you think about this, to avoid me resend the whole series?
> >>
> >> From e03d77de1ca179fa0168cead7c23cfeae57f1787 Mon Sep 17 00:00:00 2001
> >> From: Juan Quintela <quintela@redhat.com>
> >> Date: Wed, 18 Apr 2018 00:49:19 +0200
> >> Subject: [PATCH 17/18] migration: Remove not needed semaphore and quit
> >>
> >> We know quit with shutdwon in the QIO.
> >>
> >> Signed-off-by: Juan Quintela <quintela@redhat.com>
> >> --
> >> Add comment
> >> Use shutdown() instead of unref()
> >> ---
> >> migration/ram.c | 14 +++++---------
> >> 1 file changed, 5 insertions(+), 9 deletions(-)
> >>
> >> diff --git a/migration/ram.c b/migration/ram.c
> >> index 2c3a452a7d..be5d26f4cb 100644
> >> --- a/migration/ram.c
> >> +++ b/migration/ram.c
> >> @@ -594,14 +594,10 @@ typedef struct {
> >> QemuThread thread;
> >> /* communication channel */
> >> QIOChannel *c;
> >> - /* sem where to wait for more work */
> >> - QemuSemaphore sem;
> >> /* this mutex protects the following parameters */
> >> QemuMutex mutex;
> >> /* is this channel thread running */
> >> bool running;
> >> - /* should this thread finish */
> >> - bool quit;
> >> /* array of pages to receive */
> >> MultiFDPages_t *pages;
> >> /* packet allocated len */
> >> @@ -1152,8 +1148,11 @@ static void multifd_recv_terminate_threads(Error *err)
> >> MultiFDRecvParams *p = &multifd_recv_state->params[i];
> >>
> >> qemu_mutex_lock(&p->mutex);
> >> - p->quit = true;
> >> - qemu_sem_post(&p->sem);
> >> + /* We could arrive here for two reasons:
> >> + - normal quit, i.e. everything went fine, just finished
> >> + - error quit: We close the channels so the channel threads
> >> + finish the qio_channel_read_all_eof() */
> >> + qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
> >
> > OK, so with any luck all the threads now exit; do we still have a
> > close/unref once we're sure they have all exited?
>
> Yeap.
>
> static void multifd_recv_terminate_threads(Error *err)
> {
> [...]
> for (i = 0; i < migrate_multifd_channels(); i++) {
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> qemu_mutex_lock(&p->mutex);
> /* We could arrive here for two reasons:
> - normal quit, i.e. everything went fine, just finished
> - error quit: We close the channels so the channel threads
> finish the qio_channel_read_all_eof() */
> qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
> qemu_mutex_unlock(&p->mutex);
> }
> }
>
> int multifd_load_cleanup(Error **errp)
> {
> int i;
> int ret = 0;
>
> if (!migrate_use_multifd()) {
> return 0;
> }
> multifd_recv_terminate_threads(NULL);
> for (i = 0; i < migrate_multifd_channels(); i++) {
> MultiFDRecvParams *p = &multifd_recv_state->params[i];
>
> if (p->running) {
> qemu_thread_join(&p->thread);
> }
> object_unref(OBJECT(p->c));
> p->c = NULL;
> [...]
> }
> [...]
> }
>
> Omited the things that we don't care about for this.
I think that looks OK; you've got one unref at the end, and you do the
shutdown to actually kick them out.
Dave
> Thanks, Juan.
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 32+ messages in thread
end of thread, other threads:[~2018-06-20 11:25 UTC | newest]
Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-05-23 11:18 [Qemu-devel] [PATCH v13 00/12] Multifd Juan Quintela
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 01/12] migration: Create multipage support Juan Quintela
2018-06-11 16:58 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 02/12] migration: Create multifd packet Juan Quintela
2018-06-11 11:10 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 03/12] migration: Add multifd traces for start/end thread Juan Quintela
2018-05-31 16:11 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 04/12] migration: Calculate transferred ram correctly Juan Quintela
2018-05-31 17:14 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 05/12] migration: Multifd channels always wait on the sem Juan Quintela
2018-06-11 17:13 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 06/12] migration: Add block where to send/receive packets Juan Quintela
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 07/12] migration: Synchronize multifd threads with main thread Juan Quintela
2018-06-11 11:53 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 08/12] migration: Create ram_save_multifd_page Juan Quintela
2018-06-11 17:33 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 09/12] migration: Start sending messages Juan Quintela
2018-06-11 15:49 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 10/12] migration: Wait for blocking IO Juan Quintela
2018-06-11 15:54 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 11/12] migration: Remove not needed semaphore and quit Juan Quintela
2018-06-11 16:45 ` Dr. David Alan Gilbert
2018-06-20 7:20 ` Juan Quintela
2018-06-20 9:01 ` Dr. David Alan Gilbert
2018-06-20 9:42 ` Juan Quintela
2018-06-20 9:46 ` Dr. David Alan Gilbert
2018-06-20 9:48 ` Juan Quintela
2018-06-20 10:38 ` Dr. David Alan Gilbert
2018-06-20 11:07 ` Juan Quintela
2018-06-20 11:25 ` Dr. David Alan Gilbert
2018-05-23 11:18 ` [Qemu-devel] [PATCH v13 12/12] migration: Stop sending whole pages through main channel Juan Quintela
2018-05-23 11:41 ` [Qemu-devel] [PATCH v13 00/12] Multifd no-reply
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.