* [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
@ 2018-08-17 19:04 John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback John Snow
` (10 more replies)
0 siblings, 11 replies; 53+ messages in thread
From: John Snow @ 2018-08-17 19:04 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Max Reitz, John Snow
First, it's redundant to have each job manage this itself.
Second, doing so allows us to remove a tricky case where the completion
code is called under an aio_context lock, which then calls the
finalization code which is itself executed under a second aio_context
lock.
Removing this recursive lock acquisition is necessary for converting
mirror to only modify its graph post-finalization, but it's also just
safer and will bite us less in the future.
This series introduces a .job_exit callback, but after jobs are
fully transitioned to using the .commit/.abort callbacks, this new
completion callback will be removed.
John Snow (7):
jobs: change start callback to run callback
jobs: canonize Error object
jobs: add exit shim
block/commit: utilize job_exit shim
block/mirror: utilize job_exit shim
jobs: utilize job_exit shim
jobs: remove job_defer_to_main_loop
block/backup.c | 23 +++-------------
block/commit.c | 27 ++++++-------------
block/create.c | 19 +++++--------
block/mirror.c | 35 +++++++++++-------------
block/stream.c | 29 ++++++++------------
include/qemu/job.h | 36 +++++++++----------------
job-qmp.c | 5 ++--
job.c | 68 +++++++++++++++--------------------------------
tests/test-bdrv-drain.c | 13 +++------
tests/test-blockjob-txn.c | 25 +++++++----------
tests/test-blockjob.c | 17 ++++++------
11 files changed, 102 insertions(+), 195 deletions(-)
--
2.14.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
@ 2018-08-17 19:04 ` John Snow
2018-08-20 18:28 ` Eric Blake
2018-08-22 10:51 ` Max Reitz
2018-08-17 19:04 ` [Qemu-devel] [PATCH 2/7] jobs: canonize Error object John Snow
` (9 subsequent siblings)
10 siblings, 2 replies; 53+ messages in thread
From: John Snow @ 2018-08-17 19:04 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Max Reitz, John Snow
Presently we codify the entry point for a job as the "start" callback,
but a more apt name would be "run" to clarify the idea that when this
function returns we consider the job to have "finished," except for
any cleanup which occurs in separate callbacks later.
As part of this clarification, change the signature to include an error
object and a return code. The error ptr is not yet used, and the return
code while captured, will be overwritten by actions in the job_completed
function.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/backup.c | 7 ++++---
block/commit.c | 7 ++++---
block/create.c | 8 +++++---
block/mirror.c | 10 ++++++----
block/stream.c | 7 ++++---
include/qemu/job.h | 2 +-
job.c | 6 +++---
tests/test-bdrv-drain.c | 7 ++++---
tests/test-blockjob-txn.c | 16 ++++++++--------
tests/test-blockjob.c | 7 ++++---
10 files changed, 43 insertions(+), 34 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 8630d32926..5d47781840 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -480,9 +480,9 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
bdrv_dirty_iter_free(dbi);
}
-static void coroutine_fn backup_run(void *opaque)
+static int coroutine_fn backup_run(Job *opaque_job, Error **errp)
{
- BackupBlockJob *job = opaque;
+ BackupBlockJob *job = container_of(opaque_job, BackupBlockJob, common.job);
BackupCompleteData *data;
BlockDriverState *bs = blk_bs(job->common.blk);
int64_t offset, nb_clusters;
@@ -587,6 +587,7 @@ static void coroutine_fn backup_run(void *opaque)
data = g_malloc(sizeof(*data));
data->ret = ret;
job_defer_to_main_loop(&job->common.job, backup_complete, data);
+ return ret;
}
static const BlockJobDriver backup_job_driver = {
@@ -596,7 +597,7 @@ static const BlockJobDriver backup_job_driver = {
.free = block_job_free,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
- .start = backup_run,
+ .run = backup_run,
.commit = backup_commit,
.abort = backup_abort,
.clean = backup_clean,
diff --git a/block/commit.c b/block/commit.c
index eb414579bd..a0ea86ff64 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -134,9 +134,9 @@ static void commit_complete(Job *job, void *opaque)
bdrv_unref(top);
}
-static void coroutine_fn commit_run(void *opaque)
+static int coroutine_fn commit_run(Job *job, Error **errp)
{
- CommitBlockJob *s = opaque;
+ CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
CommitCompleteData *data;
int64_t offset;
uint64_t delay_ns = 0;
@@ -213,6 +213,7 @@ out:
data = g_malloc(sizeof(*data));
data->ret = ret;
job_defer_to_main_loop(&s->common.job, commit_complete, data);
+ return ret;
}
static const BlockJobDriver commit_job_driver = {
@@ -222,7 +223,7 @@ static const BlockJobDriver commit_job_driver = {
.free = block_job_free,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
- .start = commit_run,
+ .run = commit_run,
},
};
diff --git a/block/create.c b/block/create.c
index 915cd41bcc..04733c3618 100644
--- a/block/create.c
+++ b/block/create.c
@@ -45,9 +45,9 @@ static void blockdev_create_complete(Job *job, void *opaque)
job_completed(job, s->ret, s->err);
}
-static void coroutine_fn blockdev_create_run(void *opaque)
+static int coroutine_fn blockdev_create_run(Job *job, Error **errp)
{
- BlockdevCreateJob *s = opaque;
+ BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common);
job_progress_set_remaining(&s->common, 1);
s->ret = s->drv->bdrv_co_create(s->opts, &s->err);
@@ -55,12 +55,14 @@ static void coroutine_fn blockdev_create_run(void *opaque)
qapi_free_BlockdevCreateOptions(s->opts);
job_defer_to_main_loop(&s->common, blockdev_create_complete, NULL);
+
+ return s->ret;
}
static const JobDriver blockdev_create_job_driver = {
.instance_size = sizeof(BlockdevCreateJob),
.job_type = JOB_TYPE_CREATE,
- .start = blockdev_create_run,
+ .run = blockdev_create_run,
};
void qmp_blockdev_create(const char *job_id, BlockdevCreateOptions *options,
diff --git a/block/mirror.c b/block/mirror.c
index 6cc10df5c9..691763db41 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -812,9 +812,9 @@ static int mirror_flush(MirrorBlockJob *s)
return ret;
}
-static void coroutine_fn mirror_run(void *opaque)
+static int coroutine_fn mirror_run(Job *job, Error **errp)
{
- MirrorBlockJob *s = opaque;
+ MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job);
MirrorExitData *data;
BlockDriverState *bs = s->mirror_top_bs->backing->bs;
BlockDriverState *target_bs = blk_bs(s->target);
@@ -1041,7 +1041,9 @@ immediate_exit:
if (need_drain) {
bdrv_drained_begin(bs);
}
+
job_defer_to_main_loop(&s->common.job, mirror_exit, data);
+ return ret;
}
static void mirror_complete(Job *job, Error **errp)
@@ -1138,7 +1140,7 @@ static const BlockJobDriver mirror_job_driver = {
.free = block_job_free,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
- .start = mirror_run,
+ .run = mirror_run,
.pause = mirror_pause,
.complete = mirror_complete,
},
@@ -1154,7 +1156,7 @@ static const BlockJobDriver commit_active_job_driver = {
.free = block_job_free,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
- .start = mirror_run,
+ .run = mirror_run,
.pause = mirror_pause,
.complete = mirror_complete,
},
diff --git a/block/stream.c b/block/stream.c
index 9264b68a1e..b4b987df7e 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -97,9 +97,9 @@ out:
g_free(data);
}
-static void coroutine_fn stream_run(void *opaque)
+static int coroutine_fn stream_run(Job *job, Error **errp)
{
- StreamBlockJob *s = opaque;
+ StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
StreamCompleteData *data;
BlockBackend *blk = s->common.blk;
BlockDriverState *bs = blk_bs(blk);
@@ -206,6 +206,7 @@ out:
data = g_malloc(sizeof(*data));
data->ret = ret;
job_defer_to_main_loop(&s->common.job, stream_complete, data);
+ return ret;
}
static const BlockJobDriver stream_job_driver = {
@@ -213,7 +214,7 @@ static const BlockJobDriver stream_job_driver = {
.instance_size = sizeof(StreamBlockJob),
.job_type = JOB_TYPE_STREAM,
.free = block_job_free,
- .start = stream_run,
+ .run = stream_run,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
},
diff --git a/include/qemu/job.h b/include/qemu/job.h
index 18c9223e31..9cf463d228 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -169,7 +169,7 @@ struct JobDriver {
JobType job_type;
/** Mandatory: Entrypoint for the Coroutine. */
- CoroutineEntry *start;
+ int coroutine_fn (*run)(Job *job, Error **errp);
/**
* If the callback is not NULL, it will be invoked when the job transitions
diff --git a/job.c b/job.c
index fa671b431a..898260b2b3 100644
--- a/job.c
+++ b/job.c
@@ -544,16 +544,16 @@ static void coroutine_fn job_co_entry(void *opaque)
{
Job *job = opaque;
- assert(job && job->driver && job->driver->start);
+ assert(job && job->driver && job->driver->run);
job_pause_point(job);
- job->driver->start(job);
+ job->ret = job->driver->run(job, NULL);
}
void job_start(Job *job)
{
assert(job && !job_started(job) && job->paused &&
- job->driver && job->driver->start);
+ job->driver && job->driver->run);
job->co = qemu_coroutine_create(job_co_entry, job);
job->pause_count--;
job->busy = true;
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index 17bb8508ae..a7533861f6 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -757,9 +757,9 @@ static void test_job_completed(Job *job, void *opaque)
job_completed(job, 0, NULL);
}
-static void coroutine_fn test_job_start(void *opaque)
+static int coroutine_fn test_job_run(Job *job, Error **errp)
{
- TestBlockJob *s = opaque;
+ TestBlockJob *s = container_of(job, TestBlockJob, common.job);
job_transition_to_ready(&s->common.job);
while (!s->should_complete) {
@@ -771,6 +771,7 @@ static void coroutine_fn test_job_start(void *opaque)
}
job_defer_to_main_loop(&s->common.job, test_job_completed, NULL);
+ return 0;
}
static void test_job_complete(Job *job, Error **errp)
@@ -785,7 +786,7 @@ BlockJobDriver test_job_driver = {
.free = block_job_free,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
- .start = test_job_start,
+ .run = test_job_run,
.complete = test_job_complete,
},
};
diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c
index 58d9b87fb2..3194924071 100644
--- a/tests/test-blockjob-txn.c
+++ b/tests/test-blockjob-txn.c
@@ -38,25 +38,25 @@ static void test_block_job_complete(Job *job, void *opaque)
bdrv_unref(bs);
}
-static void coroutine_fn test_block_job_run(void *opaque)
+static int coroutine_fn test_block_job_run(Job *job, Error **errp)
{
- TestBlockJob *s = opaque;
- BlockJob *job = &s->common;
+ TestBlockJob *s = container_of(job, TestBlockJob, common.job);
while (s->iterations--) {
if (s->use_timer) {
- job_sleep_ns(&job->job, 0);
+ job_sleep_ns(job, 0);
} else {
- job_yield(&job->job);
+ job_yield(job);
}
- if (job_is_cancelled(&job->job)) {
+ if (job_is_cancelled(job)) {
break;
}
}
- job_defer_to_main_loop(&job->job, test_block_job_complete,
+ job_defer_to_main_loop(job, test_block_job_complete,
(void *)(intptr_t)s->rc);
+ return s->rc;
}
typedef struct {
@@ -80,7 +80,7 @@ static const BlockJobDriver test_block_job_driver = {
.free = block_job_free,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
- .start = test_block_job_run,
+ .run = test_block_job_run,
},
};
diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c
index cb42f06e61..b0462bfdec 100644
--- a/tests/test-blockjob.c
+++ b/tests/test-blockjob.c
@@ -176,9 +176,9 @@ static void cancel_job_complete(Job *job, Error **errp)
s->should_complete = true;
}
-static void coroutine_fn cancel_job_start(void *opaque)
+static int coroutine_fn cancel_job_run(Job *job, Error **errp)
{
- CancelJob *s = opaque;
+ CancelJob *s = container_of(job, CancelJob, common.job);
while (!s->should_complete) {
if (job_is_cancelled(&s->common.job)) {
@@ -194,6 +194,7 @@ static void coroutine_fn cancel_job_start(void *opaque)
defer:
job_defer_to_main_loop(&s->common.job, cancel_job_completed, s);
+ return 0;
}
static const BlockJobDriver test_cancel_driver = {
@@ -202,7 +203,7 @@ static const BlockJobDriver test_cancel_driver = {
.free = block_job_free,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
- .start = cancel_job_start,
+ .run = cancel_job_run,
.complete = cancel_job_complete,
},
};
--
2.14.4
^ permalink raw reply related [flat|nested] 53+ messages in thread
* [Qemu-devel] [PATCH 2/7] jobs: canonize Error object
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback John Snow
@ 2018-08-17 19:04 ` John Snow
2018-08-20 20:03 ` Eric Blake
` (3 more replies)
2018-08-17 19:04 ` [Qemu-devel] [PATCH 3/7] jobs: add exit shim John Snow
` (8 subsequent siblings)
10 siblings, 4 replies; 53+ messages in thread
From: John Snow @ 2018-08-17 19:04 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Max Reitz, John Snow
Jobs presently use both an Error object in the case of the create job,
and char strings in the case of generic errors elsewhere.
Unify the two paths as just j->err, and remove the extra argument from
job_completed. The integer error code for job_completed is kept for now
for use by pre-emptive cancellation.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/backup.c | 2 +-
block/commit.c | 2 +-
block/create.c | 5 ++---
block/mirror.c | 2 +-
block/stream.c | 2 +-
include/qemu/job.h | 10 ++++------
job-qmp.c | 5 +++--
job.c | 19 ++++++-------------
tests/test-bdrv-drain.c | 2 +-
tests/test-blockjob-txn.c | 2 +-
tests/test-blockjob.c | 2 +-
11 files changed, 22 insertions(+), 31 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 5d47781840..1e965d54e5 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -388,7 +388,7 @@ static void backup_complete(Job *job, void *opaque)
{
BackupCompleteData *data = opaque;
- job_completed(job, data->ret, NULL);
+ job_completed(job, data->ret);
g_free(data);
}
diff --git a/block/commit.c b/block/commit.c
index a0ea86ff64..4a17bb73ec 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -117,7 +117,7 @@ static void commit_complete(Job *job, void *opaque)
* bdrv_set_backing_hd() to fail. */
block_job_remove_all_bdrv(bjob);
- job_completed(job, ret, NULL);
+ job_completed(job, ret);
g_free(data);
/* If bdrv_drop_intermediate() didn't already do that, remove the commit
diff --git a/block/create.c b/block/create.c
index 04733c3618..26a385c6c7 100644
--- a/block/create.c
+++ b/block/create.c
@@ -35,14 +35,13 @@ typedef struct BlockdevCreateJob {
BlockDriver *drv;
BlockdevCreateOptions *opts;
int ret;
- Error *err;
} BlockdevCreateJob;
static void blockdev_create_complete(Job *job, void *opaque)
{
BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common);
- job_completed(job, s->ret, s->err);
+ job_completed(job, s->ret);
}
static int coroutine_fn blockdev_create_run(Job *job, Error **errp)
@@ -50,7 +49,7 @@ static int coroutine_fn blockdev_create_run(Job *job, Error **errp)
BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common);
job_progress_set_remaining(&s->common, 1);
- s->ret = s->drv->bdrv_co_create(s->opts, &s->err);
+ s->ret = s->drv->bdrv_co_create(s->opts, errp);
job_progress_update(&s->common, 1);
qapi_free_BlockdevCreateOptions(s->opts);
diff --git a/block/mirror.c b/block/mirror.c
index 691763db41..be5dc6b7b0 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -710,7 +710,7 @@ static void mirror_exit(Job *job, void *opaque)
blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort);
bs_opaque->job = NULL;
- job_completed(job, data->ret, NULL);
+ job_completed(job, data->ret);
g_free(data);
bdrv_drained_end(src);
diff --git a/block/stream.c b/block/stream.c
index b4b987df7e..26a775386b 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -93,7 +93,7 @@ out:
}
g_free(s->backing_file_str);
- job_completed(job, data->ret, NULL);
+ job_completed(job, data->ret);
g_free(data);
}
diff --git a/include/qemu/job.h b/include/qemu/job.h
index 9cf463d228..5c92c53ef0 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -124,12 +124,12 @@ typedef struct Job {
/** Estimated progress_current value at the completion of the job */
int64_t progress_total;
- /** Error string for a failed job (NULL if, and only if, job->ret == 0) */
- char *error;
-
/** ret code passed to job_completed. */
int ret;
+ /** Error object for a failed job **/
+ Error *err;
+
/** The completion function that will be called when the job completes. */
BlockCompletionFunc *cb;
@@ -484,15 +484,13 @@ void job_transition_to_ready(Job *job);
/**
* @job: The job being completed.
* @ret: The status code.
- * @error: The error message for a failing job (only with @ret < 0). If @ret is
- * negative, but NULL is given for @error, strerror() is used.
*
* Marks @job as completed. If @ret is non-zero, the job transaction it is part
* of is aborted. If @ret is zero, the job moves into the WAITING state. If it
* is the last job to complete in its transaction, all jobs in the transaction
* move from WAITING to PENDING.
*/
-void job_completed(Job *job, int ret, Error *error);
+void job_completed(Job *job, int ret);
/** Asynchronously complete the specified @job. */
void job_complete(Job *job, Error **errp);
diff --git a/job-qmp.c b/job-qmp.c
index 410775df61..a969b2bbf0 100644
--- a/job-qmp.c
+++ b/job-qmp.c
@@ -146,8 +146,9 @@ static JobInfo *job_query_single(Job *job, Error **errp)
.status = job->status,
.current_progress = job->progress_current,
.total_progress = job->progress_total,
- .has_error = !!job->error,
- .error = g_strdup(job->error),
+ .has_error = !!job->err,
+ .error = job->err ? \
+ g_strdup(error_get_pretty(job->err)) : NULL,
};
return info;
diff --git a/job.c b/job.c
index 898260b2b3..c9de1af556 100644
--- a/job.c
+++ b/job.c
@@ -369,7 +369,7 @@ void job_unref(Job *job)
QLIST_REMOVE(job, job_list);
- g_free(job->error);
+ error_free(job->err);
g_free(job->id);
g_free(job);
}
@@ -535,7 +535,6 @@ void job_drain(Job *job)
}
}
-
/**
* All jobs must allow a pause point before entering their job proper. This
* ensures that jobs can be paused prior to being started, then resumed later.
@@ -546,7 +545,7 @@ static void coroutine_fn job_co_entry(void *opaque)
assert(job && job->driver && job->driver->run);
job_pause_point(job);
- job->ret = job->driver->run(job, NULL);
+ job->ret = job->driver->run(job, &job->err);
}
@@ -666,8 +665,8 @@ static void job_update_rc(Job *job)
job->ret = -ECANCELED;
}
if (job->ret) {
- if (!job->error) {
- job->error = g_strdup(strerror(-job->ret));
+ if (!job->err) {
+ error_setg_errno(&job->err, -job->ret, "job failed");
}
job_state_transition(job, JOB_STATUS_ABORTING);
}
@@ -865,17 +864,11 @@ static void job_completed_txn_success(Job *job)
}
}
-void job_completed(Job *job, int ret, Error *error)
+void job_completed(Job *job, int ret)
{
assert(job && job->txn && !job_is_completed(job));
job->ret = ret;
- if (error) {
- assert(job->ret < 0);
- job->error = g_strdup(error_get_pretty(error));
- error_free(error);
- }
-
job_update_rc(job);
trace_job_completed(job, ret, job->ret);
if (job->ret) {
@@ -893,7 +886,7 @@ void job_cancel(Job *job, bool force)
}
job_cancel_async(job, force);
if (!job_started(job)) {
- job_completed(job, -ECANCELED, NULL);
+ job_completed(job, -ECANCELED);
} else if (job->deferred_to_main_loop) {
job_completed_txn_abort(job);
} else {
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index a7533861f6..00604dfc0c 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -754,7 +754,7 @@ typedef struct TestBlockJob {
static void test_job_completed(Job *job, void *opaque)
{
- job_completed(job, 0, NULL);
+ job_completed(job, 0);
}
static int coroutine_fn test_job_run(Job *job, Error **errp)
diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c
index 3194924071..82cedee78b 100644
--- a/tests/test-blockjob-txn.c
+++ b/tests/test-blockjob-txn.c
@@ -34,7 +34,7 @@ static void test_block_job_complete(Job *job, void *opaque)
rc = -ECANCELED;
}
- job_completed(job, rc, NULL);
+ job_completed(job, rc);
bdrv_unref(bs);
}
diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c
index b0462bfdec..408a226939 100644
--- a/tests/test-blockjob.c
+++ b/tests/test-blockjob.c
@@ -167,7 +167,7 @@ static void cancel_job_completed(Job *job, void *opaque)
{
CancelJob *s = opaque;
s->completed = true;
- job_completed(job, 0, NULL);
+ job_completed(job, 0);
}
static void cancel_job_complete(Job *job, Error **errp)
--
2.14.4
^ permalink raw reply related [flat|nested] 53+ messages in thread
* [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 2/7] jobs: canonize Error object John Snow
@ 2018-08-17 19:04 ` John Snow
2018-08-20 21:16 ` Eric Blake
2018-08-22 11:43 ` Max Reitz
2018-08-17 19:04 ` [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim John Snow
` (7 subsequent siblings)
10 siblings, 2 replies; 53+ messages in thread
From: John Snow @ 2018-08-17 19:04 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Max Reitz, John Snow
All jobs do the same thing when they leave their running loop:
- Store the return code in a structure
- wait to receive this structure in the main thread
- signal job completion via job_completed
Few jobs do anything beyond exactly this. Consolidate this exit
logic for a net reduction in SLOC.
More seriously, when we utilize job_defer_to_main_loop_bh to call
a function that calls job_completed, job_finalize_single will run
in a context where it has recursively taken the aio_context lock,
which can cause hangs if it puts down a reference that causes a flush.
You can observe this in practice by looking at mirror_exit's careful
placement of job_completed and bdrv_unref calls.
If we centralize job exiting, we can signal job completion from outside
of the aio_context, which should allow for job cleanup code to run with
only one lock, which makes cleanup callbacks less tricky to write.
Signed-off-by: John Snow <jsnow@redhat.com>
---
include/qemu/job.h | 7 +++++++
job.c | 19 +++++++++++++++++++
2 files changed, 26 insertions(+)
diff --git a/include/qemu/job.h b/include/qemu/job.h
index 5c92c53ef0..6f5b13751a 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -204,6 +204,13 @@ struct JobDriver {
*/
void (*drain)(Job *job);
+ /**
+ * If the callback is not NULL, exit will be invoked from the main thread
+ * when the job's coroutine has finished, but before transactional
+ * convergence; before @prepare or @abort.
+ */
+ void (*exit)(Job *job);
+
/**
* If the callback is not NULL, prepare will be invoked when all the jobs
* belonging to the same transaction complete; or upon this job's completion
diff --git a/job.c b/job.c
index c9de1af556..fae8e6047c 100644
--- a/job.c
+++ b/job.c
@@ -535,6 +535,19 @@ void job_drain(Job *job)
}
}
+static void job_exit(void *opaque)
+{
+ Job *job = (Job *)opaque;
+ AioContext *aio_context = job->aio_context;
+
+ if (job->driver->exit) {
+ aio_context_acquire(aio_context);
+ job->driver->exit(job);
+ aio_context_release(aio_context);
+ }
+ job_completed(job, job->ret);
+}
+
/**
* All jobs must allow a pause point before entering their job proper. This
* ensures that jobs can be paused prior to being started, then resumed later.
@@ -546,6 +559,12 @@ static void coroutine_fn job_co_entry(void *opaque)
assert(job && job->driver && job->driver->run);
job_pause_point(job);
job->ret = job->driver->run(job, &job->err);
+ if (!job->deferred_to_main_loop) {
+ job->deferred_to_main_loop = true;
+ aio_bh_schedule_oneshot(qemu_get_aio_context(),
+ job_exit,
+ job);
+ }
}
--
2.14.4
^ permalink raw reply related [flat|nested] 53+ messages in thread
* [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
` (2 preceding siblings ...)
2018-08-17 19:04 ` [Qemu-devel] [PATCH 3/7] jobs: add exit shim John Snow
@ 2018-08-17 19:04 ` John Snow
2018-08-17 19:18 ` John Snow
2018-08-22 11:55 ` Max Reitz
2018-08-17 19:04 ` [Qemu-devel] [PATCH 5/7] block/mirror: " John Snow
` (6 subsequent siblings)
10 siblings, 2 replies; 53+ messages in thread
From: John Snow @ 2018-08-17 19:04 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Max Reitz, John Snow
Change the manual deferment to commit_complete into the implicit
callback to job_exit, renaming commit_complete to commit_exit.
This conversion does change the timing of when job_completed is
called to after the bdrv_replace_node and bdrv_unref calls, which
could have implications for bjob->blk which will now be put down
after this cleanup.
Kevin highlights that we did not take any permissions for that backend
at job creation time, so it is safe to reorder these operations.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/commit.c | 20 ++++----------------
1 file changed, 4 insertions(+), 16 deletions(-)
diff --git a/block/commit.c b/block/commit.c
index 4a17bb73ec..93c3b6a39e 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -68,19 +68,13 @@ static int coroutine_fn commit_populate(BlockBackend *bs, BlockBackend *base,
return 0;
}
-typedef struct {
- int ret;
-} CommitCompleteData;
-
-static void commit_complete(Job *job, void *opaque)
+static void commit_exit(Job *job)
{
CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
BlockJob *bjob = &s->common;
- CommitCompleteData *data = opaque;
BlockDriverState *top = blk_bs(s->top);
BlockDriverState *base = blk_bs(s->base);
BlockDriverState *commit_top_bs = s->commit_top_bs;
- int ret = data->ret;
bool remove_commit_top_bs = false;
/* Make sure commit_top_bs and top stay around until bdrv_replace_node() */
@@ -93,8 +87,8 @@ static void commit_complete(Job *job, void *opaque)
if (!job_is_cancelled(job) && ret == 0) {
/* success */
- ret = bdrv_drop_intermediate(s->commit_top_bs, base,
- s->backing_file_str);
+ job->ret = bdrv_drop_intermediate(s->commit_top_bs, base,
+ s->backing_file_str);
} else {
/* XXX Can (or should) we somehow keep 'consistent read' blocked even
* after the failed/cancelled commit job is gone? If we already wrote
@@ -117,9 +111,6 @@ static void commit_complete(Job *job, void *opaque)
* bdrv_set_backing_hd() to fail. */
block_job_remove_all_bdrv(bjob);
- job_completed(job, ret);
- g_free(data);
-
/* If bdrv_drop_intermediate() didn't already do that, remove the commit
* filter driver from the backing chain. Do this as the final step so that
* the 'consistent read' permission can be granted. */
@@ -137,7 +128,6 @@ static void commit_complete(Job *job, void *opaque)
static int coroutine_fn commit_run(Job *job, Error **errp)
{
CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
- CommitCompleteData *data;
int64_t offset;
uint64_t delay_ns = 0;
int ret = 0;
@@ -210,9 +200,6 @@ static int coroutine_fn commit_run(Job *job, Error **errp)
out:
qemu_vfree(buf);
- data = g_malloc(sizeof(*data));
- data->ret = ret;
- job_defer_to_main_loop(&s->common.job, commit_complete, data);
return ret;
}
@@ -224,6 +211,7 @@ static const BlockJobDriver commit_job_driver = {
.user_resume = block_job_user_resume,
.drain = block_job_drain,
.run = commit_run,
+ .exit = commit_exit,
},
};
--
2.14.4
^ permalink raw reply related [flat|nested] 53+ messages in thread
* [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
` (3 preceding siblings ...)
2018-08-17 19:04 ` [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim John Snow
@ 2018-08-17 19:04 ` John Snow
2018-08-22 12:06 ` Max Reitz
2018-08-22 12:15 ` Max Reitz
2018-08-17 19:04 ` [Qemu-devel] [PATCH 6/7] jobs: " John Snow
` (5 subsequent siblings)
10 siblings, 2 replies; 53+ messages in thread
From: John Snow @ 2018-08-17 19:04 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Max Reitz, John Snow
Change the manual deferment to mirror_exit into the implicit
callback to job_exit and the mirror_exit callback.
This does change the order of some bdrv_unref calls and job_completed,
but thanks to the new context in which we call .job_exit, this is safe
to defer the possible flushing of any nodes to the job_finalize_single
cleanup stage.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/mirror.c | 25 +++++++++----------------
1 file changed, 9 insertions(+), 16 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index be5dc6b7b0..57b4ac97d8 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -607,21 +607,17 @@ static void mirror_wait_for_all_io(MirrorBlockJob *s)
}
}
-typedef struct {
- int ret;
-} MirrorExitData;
-
-static void mirror_exit(Job *job, void *opaque)
+static void mirror_exit(Job *job)
{
MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job);
BlockJob *bjob = &s->common;
- MirrorExitData *data = opaque;
MirrorBDSOpaque *bs_opaque = s->mirror_top_bs->opaque;
AioContext *replace_aio_context = NULL;
BlockDriverState *src = s->mirror_top_bs->backing->bs;
BlockDriverState *target_bs = blk_bs(s->target);
BlockDriverState *mirror_top_bs = s->mirror_top_bs;
Error *local_err = NULL;
+ int ret = job->ret;
bdrv_release_dirty_bitmap(src, s->dirty_bitmap);
@@ -652,7 +648,7 @@ static void mirror_exit(Job *job, void *opaque)
bdrv_set_backing_hd(target_bs, backing, &local_err);
if (local_err) {
error_report_err(local_err);
- data->ret = -EPERM;
+ ret = -EPERM;
}
}
}
@@ -662,7 +658,7 @@ static void mirror_exit(Job *job, void *opaque)
aio_context_acquire(replace_aio_context);
}
- if (s->should_complete && data->ret == 0) {
+ if (s->should_complete && ret == 0) {
BlockDriverState *to_replace = src;
if (s->to_replace) {
to_replace = s->to_replace;
@@ -679,7 +675,7 @@ static void mirror_exit(Job *job, void *opaque)
bdrv_drained_end(target_bs);
if (local_err) {
error_report_err(local_err);
- data->ret = -EPERM;
+ ret = -EPERM;
}
}
if (s->to_replace) {
@@ -710,12 +706,12 @@ static void mirror_exit(Job *job, void *opaque)
blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort);
bs_opaque->job = NULL;
- job_completed(job, data->ret);
- g_free(data);
bdrv_drained_end(src);
bdrv_unref(mirror_top_bs);
bdrv_unref(src);
+
+ job->ret = ret;
}
static void mirror_throttle(MirrorBlockJob *s)
@@ -815,7 +811,6 @@ static int mirror_flush(MirrorBlockJob *s)
static int coroutine_fn mirror_run(Job *job, Error **errp)
{
MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job);
- MirrorExitData *data;
BlockDriverState *bs = s->mirror_top_bs->backing->bs;
BlockDriverState *target_bs = blk_bs(s->target);
bool need_drain = true;
@@ -1035,14 +1030,10 @@ immediate_exit:
g_free(s->in_flight_bitmap);
bdrv_dirty_iter_free(s->dbi);
- data = g_malloc(sizeof(*data));
- data->ret = ret;
-
if (need_drain) {
bdrv_drained_begin(bs);
}
- job_defer_to_main_loop(&s->common.job, mirror_exit, data);
return ret;
}
@@ -1141,6 +1132,7 @@ static const BlockJobDriver mirror_job_driver = {
.user_resume = block_job_user_resume,
.drain = block_job_drain,
.run = mirror_run,
+ .exit = mirror_exit,
.pause = mirror_pause,
.complete = mirror_complete,
},
@@ -1157,6 +1149,7 @@ static const BlockJobDriver commit_active_job_driver = {
.user_resume = block_job_user_resume,
.drain = block_job_drain,
.run = mirror_run,
+ .exit = mirror_exit,
.pause = mirror_pause,
.complete = mirror_complete,
},
--
2.14.4
^ permalink raw reply related [flat|nested] 53+ messages in thread
* [Qemu-devel] [PATCH 6/7] jobs: utilize job_exit shim
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
` (4 preceding siblings ...)
2018-08-17 19:04 ` [Qemu-devel] [PATCH 5/7] block/mirror: " John Snow
@ 2018-08-17 19:04 ` John Snow
2018-08-22 12:20 ` Max Reitz
2018-08-17 19:04 ` [Qemu-devel] [PATCH 7/7] jobs: remove job_defer_to_main_loop John Snow
` (4 subsequent siblings)
10 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-17 19:04 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Max Reitz, John Snow
Utilize the job_exit shim by not calling job_defer_to_main_loop, and
where applicable, converting the deferred callback into the job_exit
callback.
This converts backup, stream, create, and the unit tests all at once.
None of these jobs undergo and order of operations changes, so it should
be a mechanical change.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/backup.c | 16 ----------------
block/create.c | 14 +++-----------
block/stream.c | 22 +++++++---------------
tests/test-bdrv-drain.c | 6 ------
tests/test-blockjob-txn.c | 11 ++---------
tests/test-blockjob.c | 10 ++++------
6 files changed, 16 insertions(+), 63 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 1e965d54e5..a67b7fa96b 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -380,18 +380,6 @@ static BlockErrorAction backup_error_action(BackupBlockJob *job,
}
}
-typedef struct {
- int ret;
-} BackupCompleteData;
-
-static void backup_complete(Job *job, void *opaque)
-{
- BackupCompleteData *data = opaque;
-
- job_completed(job, data->ret);
- g_free(data);
-}
-
static bool coroutine_fn yield_and_check(BackupBlockJob *job)
{
uint64_t delay_ns;
@@ -483,7 +471,6 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
static int coroutine_fn backup_run(Job *opaque_job, Error **errp)
{
BackupBlockJob *job = container_of(opaque_job, BackupBlockJob, common.job);
- BackupCompleteData *data;
BlockDriverState *bs = blk_bs(job->common.blk);
int64_t offset, nb_clusters;
int ret = 0;
@@ -584,9 +571,6 @@ static int coroutine_fn backup_run(Job *opaque_job, Error **errp)
qemu_co_rwlock_unlock(&job->flush_rwlock);
hbitmap_free(job->copy_bitmap);
- data = g_malloc(sizeof(*data));
- data->ret = ret;
- job_defer_to_main_loop(&job->common.job, backup_complete, data);
return ret;
}
diff --git a/block/create.c b/block/create.c
index 26a385c6c7..95341219ef 100644
--- a/block/create.c
+++ b/block/create.c
@@ -34,28 +34,20 @@ typedef struct BlockdevCreateJob {
Job common;
BlockDriver *drv;
BlockdevCreateOptions *opts;
- int ret;
} BlockdevCreateJob;
-static void blockdev_create_complete(Job *job, void *opaque)
-{
- BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common);
-
- job_completed(job, s->ret);
-}
-
static int coroutine_fn blockdev_create_run(Job *job, Error **errp)
{
BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common);
+ int ret;
job_progress_set_remaining(&s->common, 1);
- s->ret = s->drv->bdrv_co_create(s->opts, errp);
+ ret = s->drv->bdrv_co_create(s->opts, errp);
job_progress_update(&s->common, 1);
qapi_free_BlockdevCreateOptions(s->opts);
- job_defer_to_main_loop(&s->common, blockdev_create_complete, NULL);
- return s->ret;
+ return ret;
}
static const JobDriver blockdev_create_job_driver = {
diff --git a/block/stream.c b/block/stream.c
index 26a775386b..67e1e72e23 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -54,20 +54,16 @@ static int coroutine_fn stream_populate(BlockBackend *blk,
return blk_co_preadv(blk, offset, qiov.size, &qiov, BDRV_REQ_COPY_ON_READ);
}
-typedef struct {
- int ret;
-} StreamCompleteData;
-
-static void stream_complete(Job *job, void *opaque)
+static void stream_exit(Job *job)
{
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
BlockJob *bjob = &s->common;
- StreamCompleteData *data = opaque;
BlockDriverState *bs = blk_bs(bjob->blk);
BlockDriverState *base = s->base;
Error *local_err = NULL;
+ int ret = job->ret;
- if (!job_is_cancelled(job) && bs->backing && data->ret == 0) {
+ if (!job_is_cancelled(job) && bs->backing && ret == 0) {
const char *base_id = NULL, *base_fmt = NULL;
if (base) {
base_id = s->backing_file_str;
@@ -75,11 +71,11 @@ static void stream_complete(Job *job, void *opaque)
base_fmt = base->drv->format_name;
}
}
- data->ret = bdrv_change_backing_file(bs, base_id, base_fmt);
+ ret = bdrv_change_backing_file(bs, base_id, base_fmt);
bdrv_set_backing_hd(bs, base, &local_err);
if (local_err) {
error_report_err(local_err);
- data->ret = -EPERM;
+ ret = -EPERM;
goto out;
}
}
@@ -93,14 +89,12 @@ out:
}
g_free(s->backing_file_str);
- job_completed(job, data->ret);
- g_free(data);
+ job->ret = ret;
}
static int coroutine_fn stream_run(Job *job, Error **errp)
{
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
- StreamCompleteData *data;
BlockBackend *blk = s->common.blk;
BlockDriverState *bs = blk_bs(blk);
BlockDriverState *base = s->base;
@@ -203,9 +197,6 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
out:
/* Modify backing chain and close BDSes in main loop */
- data = g_malloc(sizeof(*data));
- data->ret = ret;
- job_defer_to_main_loop(&s->common.job, stream_complete, data);
return ret;
}
@@ -215,6 +206,7 @@ static const BlockJobDriver stream_job_driver = {
.job_type = JOB_TYPE_STREAM,
.free = block_job_free,
.run = stream_run,
+ .exit = stream_exit,
.user_resume = block_job_user_resume,
.drain = block_job_drain,
},
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index 00604dfc0c..9bcb3c72af 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -752,11 +752,6 @@ typedef struct TestBlockJob {
bool should_complete;
} TestBlockJob;
-static void test_job_completed(Job *job, void *opaque)
-{
- job_completed(job, 0);
-}
-
static int coroutine_fn test_job_run(Job *job, Error **errp)
{
TestBlockJob *s = container_of(job, TestBlockJob, common.job);
@@ -770,7 +765,6 @@ static int coroutine_fn test_job_run(Job *job, Error **errp)
job_pause_point(&s->common.job);
}
- job_defer_to_main_loop(&s->common.job, test_job_completed, NULL);
return 0;
}
diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c
index 82cedee78b..ef29f35e44 100644
--- a/tests/test-blockjob-txn.c
+++ b/tests/test-blockjob-txn.c
@@ -24,17 +24,11 @@ typedef struct {
int *result;
} TestBlockJob;
-static void test_block_job_complete(Job *job, void *opaque)
+static void test_block_job_exit(Job *job)
{
BlockJob *bjob = container_of(job, BlockJob, job);
BlockDriverState *bs = blk_bs(bjob->blk);
- int rc = (intptr_t)opaque;
- if (job_is_cancelled(job)) {
- rc = -ECANCELED;
- }
-
- job_completed(job, rc);
bdrv_unref(bs);
}
@@ -54,8 +48,6 @@ static int coroutine_fn test_block_job_run(Job *job, Error **errp)
}
}
- job_defer_to_main_loop(job, test_block_job_complete,
- (void *)(intptr_t)s->rc);
return s->rc;
}
@@ -81,6 +73,7 @@ static const BlockJobDriver test_block_job_driver = {
.user_resume = block_job_user_resume,
.drain = block_job_drain,
.run = test_block_job_run,
+ .exit = test_block_job_exit,
},
};
diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c
index 408a226939..ad4a65bc78 100644
--- a/tests/test-blockjob.c
+++ b/tests/test-blockjob.c
@@ -163,11 +163,10 @@ typedef struct CancelJob {
bool completed;
} CancelJob;
-static void cancel_job_completed(Job *job, void *opaque)
+static void cancel_job_exit(Job *job)
{
- CancelJob *s = opaque;
+ CancelJob *s = container_of(job, CancelJob, common.job);
s->completed = true;
- job_completed(job, 0);
}
static void cancel_job_complete(Job *job, Error **errp)
@@ -182,7 +181,7 @@ static int coroutine_fn cancel_job_run(Job *job, Error **errp)
while (!s->should_complete) {
if (job_is_cancelled(&s->common.job)) {
- goto defer;
+ return 0;
}
if (!job_is_ready(&s->common.job) && s->should_converge) {
@@ -192,8 +191,6 @@ static int coroutine_fn cancel_job_run(Job *job, Error **errp)
job_sleep_ns(&s->common.job, 100000);
}
- defer:
- job_defer_to_main_loop(&s->common.job, cancel_job_completed, s);
return 0;
}
@@ -204,6 +201,7 @@ static const BlockJobDriver test_cancel_driver = {
.user_resume = block_job_user_resume,
.drain = block_job_drain,
.run = cancel_job_run,
+ .exit = cancel_job_exit,
.complete = cancel_job_complete,
},
};
--
2.14.4
^ permalink raw reply related [flat|nested] 53+ messages in thread
* [Qemu-devel] [PATCH 7/7] jobs: remove job_defer_to_main_loop
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
` (5 preceding siblings ...)
2018-08-17 19:04 ` [Qemu-devel] [PATCH 6/7] jobs: " John Snow
@ 2018-08-17 19:04 ` John Snow
2018-08-22 12:21 ` Max Reitz
2018-08-18 16:27 ` [Qemu-devel] [PATCH 0/7] " no-reply
` (3 subsequent siblings)
10 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-17 19:04 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Max Reitz, John Snow
Now that the job infrastructure is handling the job_completed call for
all implemented jobs, we can remove the interface that allowed jobs to
schedule their own completion.
Signed-off-by: John Snow <jsnow@redhat.com>
---
include/qemu/job.h | 17 -----------------
job.c | 40 ++--------------------------------------
2 files changed, 2 insertions(+), 55 deletions(-)
diff --git a/include/qemu/job.h b/include/qemu/job.h
index 6f5b13751a..8349702e3e 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -558,23 +558,6 @@ void job_finalize(Job *job, Error **errp);
*/
void job_dismiss(Job **job, Error **errp);
-typedef void JobDeferToMainLoopFn(Job *job, void *opaque);
-
-/**
- * @job: The job
- * @fn: The function to run in the main loop
- * @opaque: The opaque value that is passed to @fn
- *
- * This function must be called by the main job coroutine just before it
- * returns. @fn is executed in the main loop with the job AioContext acquired.
- *
- * Block jobs must call bdrv_unref(), bdrv_close(), and anything that uses
- * bdrv_drain_all() in the main loop.
- *
- * The @job AioContext is held while @fn executes.
- */
-void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque);
-
/**
* Synchronously finishes the given @job. If @finish is given, it is called to
* trigger completion or cancellation of the job.
diff --git a/job.c b/job.c
index fae8e6047c..dbb225e35d 100644
--- a/job.c
+++ b/job.c
@@ -559,12 +559,8 @@ static void coroutine_fn job_co_entry(void *opaque)
assert(job && job->driver && job->driver->run);
job_pause_point(job);
job->ret = job->driver->run(job, &job->err);
- if (!job->deferred_to_main_loop) {
- job->deferred_to_main_loop = true;
- aio_bh_schedule_oneshot(qemu_get_aio_context(),
- job_exit,
- job);
- }
+ job->deferred_to_main_loop = true;
+ aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job);
}
@@ -968,38 +964,6 @@ void job_complete(Job *job, Error **errp)
job->driver->complete(job, errp);
}
-
-typedef struct {
- Job *job;
- JobDeferToMainLoopFn *fn;
- void *opaque;
-} JobDeferToMainLoopData;
-
-static void job_defer_to_main_loop_bh(void *opaque)
-{
- JobDeferToMainLoopData *data = opaque;
- Job *job = data->job;
- AioContext *aio_context = job->aio_context;
-
- aio_context_acquire(aio_context);
- data->fn(data->job, data->opaque);
- aio_context_release(aio_context);
-
- g_free(data);
-}
-
-void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque)
-{
- JobDeferToMainLoopData *data = g_malloc(sizeof(*data));
- data->job = job;
- data->fn = fn;
- data->opaque = opaque;
- job->deferred_to_main_loop = true;
-
- aio_bh_schedule_oneshot(qemu_get_aio_context(),
- job_defer_to_main_loop_bh, data);
-}
-
int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp)
{
Error *local_err = NULL;
--
2.14.4
^ permalink raw reply related [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim
2018-08-17 19:04 ` [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim John Snow
@ 2018-08-17 19:18 ` John Snow
2018-08-22 11:58 ` Max Reitz
2018-08-22 11:55 ` Max Reitz
1 sibling, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-17 19:18 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc, Max Reitz
On 08/17/2018 03:04 PM, John Snow wrote:
> Change the manual deferment to commit_complete into the implicit
> callback to job_exit, renaming commit_complete to commit_exit.
>
> This conversion does change the timing of when job_completed is
> called to after the bdrv_replace_node and bdrv_unref calls, which
> could have implications for bjob->blk which will now be put down
> after this cleanup.
>
> Kevin highlights that we did not take any permissions for that backend
> at job creation time, so it is safe to reorder these operations.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/commit.c | 20 ++++----------------
> 1 file changed, 4 insertions(+), 16 deletions(-)
>
> diff --git a/block/commit.c b/block/commit.c
> index 4a17bb73ec..93c3b6a39e 100644
> --- a/block/commit.c
> +++ b/block/commit.c
> @@ -68,19 +68,13 @@ static int coroutine_fn commit_populate(BlockBackend *bs, BlockBackend *base,
> return 0;
> }
>
> -typedef struct {
> - int ret;
> -} CommitCompleteData;
> -
> -static void commit_complete(Job *job, void *opaque)
> +static void commit_exit(Job *job)
> {
> CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
> BlockJob *bjob = &s->common;
> - CommitCompleteData *data = opaque;
> BlockDriverState *top = blk_bs(s->top);
> BlockDriverState *base = blk_bs(s->base);
> BlockDriverState *commit_top_bs = s->commit_top_bs;
> - int ret = data->ret;
> bool remove_commit_top_bs = false;
>
> /* Make sure commit_top_bs and top stay around until bdrv_replace_node() */
> @@ -93,8 +87,8 @@ static void commit_complete(Job *job, void *opaque)
>
> if (!job_is_cancelled(job) && ret == 0) {
forgot to actually squash the change in here that replaces `ret` with
`job->ret`.
> /* success */
> - ret = bdrv_drop_intermediate(s->commit_top_bs, base,
> - s->backing_file_str);
> + job->ret = bdrv_drop_intermediate(s->commit_top_bs, base,
> + s->backing_file_str);
> } else {
> /* XXX Can (or should) we somehow keep 'consistent read' blocked even
> * after the failed/cancelled commit job is gone? If we already wrote
> @@ -117,9 +111,6 @@ static void commit_complete(Job *job, void *opaque)
> * bdrv_set_backing_hd() to fail. */
> block_job_remove_all_bdrv(bjob);
>
> - job_completed(job, ret);
> - g_free(data);
> -
> /* If bdrv_drop_intermediate() didn't already do that, remove the commit
> * filter driver from the backing chain. Do this as the final step so that
> * the 'consistent read' permission can be granted. */
> @@ -137,7 +128,6 @@ static void commit_complete(Job *job, void *opaque)
> static int coroutine_fn commit_run(Job *job, Error **errp)
> {
> CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
> - CommitCompleteData *data;
> int64_t offset;
> uint64_t delay_ns = 0;
> int ret = 0;
> @@ -210,9 +200,6 @@ static int coroutine_fn commit_run(Job *job, Error **errp)
> out:
> qemu_vfree(buf);
>
> - data = g_malloc(sizeof(*data));
> - data->ret = ret;
> - job_defer_to_main_loop(&s->common.job, commit_complete, data);
> return ret;
> }
>
> @@ -224,6 +211,7 @@ static const BlockJobDriver commit_job_driver = {
> .user_resume = block_job_user_resume,
> .drain = block_job_drain,
> .run = commit_run,
> + .exit = commit_exit,
> },
> };
>
>
--
—js
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
` (6 preceding siblings ...)
2018-08-17 19:04 ` [Qemu-devel] [PATCH 7/7] jobs: remove job_defer_to_main_loop John Snow
@ 2018-08-18 16:27 ` no-reply
2018-08-18 16:31 ` no-reply
` (2 subsequent siblings)
10 siblings, 0 replies; 53+ messages in thread
From: no-reply @ 2018-08-18 16:27 UTC (permalink / raw)
To: jsnow; +Cc: famz, qemu-block, qemu-devel, kwolf, jcody, jtc, mreitz
Hi,
This series failed docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 20180817190457.8292-1-jsnow@redhat.com
Subject: [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-mingw@fedora SHOW_ENV=1 J=8
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
2d93111b33 jobs: remove job_defer_to_main_loop
082d48fa2c jobs: utilize job_exit shim
a2a302e5e1 block/mirror: utilize job_exit shim
7a113fae21 block/commit: utilize job_exit shim
df00a6221a jobs: add exit shim
16d80626ef jobs: canonize Error object
b2e0ebbc40 jobs: change start callback to run callback
=== OUTPUT BEGIN ===
BUILD fedora
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-a46s68_y/src'
GEN /var/tmp/patchew-tester-tmp-a46s68_y/src/docker-src.2018-08-18-12.25.38.11223/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-a46s68_y/src/docker-src.2018-08-18-12.25.38.11223/qemu.tar.vroot'...
done.
Checking out files: 49% (3137/6330)
Checking out files: 50% (3165/6330)
Checking out files: 51% (3229/6330)
Checking out files: 52% (3292/6330)
Checking out files: 53% (3355/6330)
Checking out files: 54% (3419/6330)
Checking out files: 55% (3482/6330)
Checking out files: 56% (3545/6330)
Checking out files: 57% (3609/6330)
Checking out files: 58% (3672/6330)
Checking out files: 59% (3735/6330)
Checking out files: 60% (3798/6330)
Checking out files: 61% (3862/6330)
Checking out files: 62% (3925/6330)
Checking out files: 63% (3988/6330)
Checking out files: 64% (4052/6330)
Checking out files: 65% (4115/6330)
Checking out files: 66% (4178/6330)
Checking out files: 67% (4242/6330)
Checking out files: 68% (4305/6330)
Checking out files: 69% (4368/6330)
Checking out files: 70% (4431/6330)
Checking out files: 71% (4495/6330)
Checking out files: 72% (4558/6330)
Checking out files: 73% (4621/6330)
Checking out files: 74% (4685/6330)
Checking out files: 75% (4748/6330)
Checking out files: 76% (4811/6330)
Checking out files: 77% (4875/6330)
Checking out files: 78% (4938/6330)
Checking out files: 79% (5001/6330)
Checking out files: 80% (5064/6330)
Checking out files: 81% (5128/6330)
Checking out files: 82% (5191/6330)
Checking out files: 83% (5254/6330)
Checking out files: 84% (5318/6330)
Checking out files: 85% (5381/6330)
Checking out files: 86% (5444/6330)
Checking out files: 87% (5508/6330)
Checking out files: 88% (5571/6330)
Checking out files: 89% (5634/6330)
Checking out files: 90% (5697/6330)
Checking out files: 91% (5761/6330)
Checking out files: 92% (5824/6330)
Checking out files: 93% (5887/6330)
Checking out files: 94% (5951/6330)
Checking out files: 95% (6014/6330)
Checking out files: 96% (6077/6330)
Checking out files: 97% (6141/6330)
Checking out files: 98% (6204/6330)
Checking out files: 99% (6267/6330)
Checking out files: 100% (6330/6330)
Checking out files: 100% (6330/6330), done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-a46s68_y/src/docker-src.2018-08-18-12.25.38.11223/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-a46s68_y/src/docker-src.2018-08-18-12.25.38.11223/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
COPY RUNNER
RUN test-mingw in qemu:fedora
Packages installed:
SDL2-devel-2.0.8-5.fc28.x86_64
bc-1.07.1-5.fc28.x86_64
bison-3.0.4-9.fc28.x86_64
bluez-libs-devel-5.49-3.fc28.x86_64
brlapi-devel-0.6.7-12.fc28.x86_64
bzip2-1.0.6-26.fc28.x86_64
bzip2-devel-1.0.6-26.fc28.x86_64
ccache-3.4.2-2.fc28.x86_64
clang-6.0.0-5.fc28.x86_64
device-mapper-multipath-devel-0.7.4-2.git07e7bd5.fc28.x86_64
findutils-4.6.0-19.fc28.x86_64
flex-2.6.1-7.fc28.x86_64
gcc-8.1.1-1.fc28.x86_64
gcc-c++-8.1.1-1.fc28.x86_64
gettext-0.19.8.1-14.fc28.x86_64
git-2.17.1-2.fc28.x86_64
glib2-devel-2.56.1-3.fc28.x86_64
glusterfs-api-devel-4.0.2-1.fc28.x86_64
gnutls-devel-3.6.2-1.fc28.x86_64
gtk3-devel-3.22.30-1.fc28.x86_64
hostname-3.20-3.fc28.x86_64
libaio-devel-0.3.110-11.fc28.x86_64
libasan-8.1.1-1.fc28.x86_64
libattr-devel-2.4.47-23.fc28.x86_64
libcap-devel-2.25-9.fc28.x86_64
libcap-ng-devel-0.7.9-1.fc28.x86_64
libcurl-devel-7.59.0-3.fc28.x86_64
libfdt-devel-1.4.6-4.fc28.x86_64
libpng-devel-1.6.34-3.fc28.x86_64
librbd-devel-12.2.5-1.fc28.x86_64
libssh2-devel-1.8.0-7.fc28.x86_64
libubsan-8.1.1-1.fc28.x86_64
libusbx-devel-1.0.21-6.fc28.x86_64
libxml2-devel-2.9.7-4.fc28.x86_64
llvm-6.0.0-11.fc28.x86_64
lzo-devel-2.08-12.fc28.x86_64
make-4.2.1-6.fc28.x86_64
mingw32-SDL2-2.0.5-3.fc27.noarch
mingw32-bzip2-1.0.6-9.fc27.noarch
mingw32-curl-7.57.0-1.fc28.noarch
mingw32-glib2-2.54.1-1.fc28.noarch
mingw32-gmp-6.1.2-2.fc27.noarch
mingw32-gnutls-3.5.13-2.fc27.noarch
mingw32-gtk3-3.22.16-1.fc27.noarch
mingw32-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw32-libpng-1.6.29-2.fc27.noarch
mingw32-libssh2-1.8.0-3.fc27.noarch
mingw32-libtasn1-4.13-1.fc28.noarch
mingw32-nettle-3.3-3.fc27.noarch
mingw32-pixman-0.34.0-3.fc27.noarch
mingw32-pkg-config-0.28-9.fc27.x86_64
mingw64-SDL2-2.0.5-3.fc27.noarch
mingw64-bzip2-1.0.6-9.fc27.noarch
mingw64-curl-7.57.0-1.fc28.noarch
mingw64-glib2-2.54.1-1.fc28.noarch
mingw64-gmp-6.1.2-2.fc27.noarch
mingw64-gnutls-3.5.13-2.fc27.noarch
mingw64-gtk3-3.22.16-1.fc27.noarch
mingw64-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw64-libpng-1.6.29-2.fc27.noarch
mingw64-libssh2-1.8.0-3.fc27.noarch
mingw64-libtasn1-4.13-1.fc28.noarch
mingw64-nettle-3.3-3.fc27.noarch
mingw64-pixman-0.34.0-3.fc27.noarch
mingw64-pkg-config-0.28-9.fc27.x86_64
ncurses-devel-6.1-5.20180224.fc28.x86_64
nettle-devel-3.4-2.fc28.x86_64
nss-devel-3.36.1-1.1.fc28.x86_64
numactl-devel-2.0.11-8.fc28.x86_64
package PyYAML is not installed
package libjpeg-devel is not installed
perl-5.26.2-411.fc28.x86_64
pixman-devel-0.34.0-8.fc28.x86_64
python3-3.6.5-1.fc28.x86_64
snappy-devel-1.1.7-5.fc28.x86_64
sparse-0.5.2-1.fc28.x86_64
spice-server-devel-0.14.0-4.fc28.x86_64
systemtap-sdt-devel-3.2-11.fc28.x86_64
tar-1.30-3.fc28.x86_64
usbredir-devel-0.7.1-7.fc28.x86_64
virglrenderer-devel-0.6.0-4.20170210git76b3da97b.fc28.x86_64
vte3-devel-0.36.5-6.fc28.x86_64
which-2.21-8.fc28.x86_64
xen-devel-4.10.1-3.fc28.x86_64
zlib-devel-1.2.11-8.fc28.x86_64
Environment variables:
TARGET_LIST=
PACKAGES=ccache gettext git tar PyYAML sparse flex bison python3 bzip2 hostname gcc gcc-c++ llvm clang make perl which bc findutils glib2-devel libaio-devel pixman-devel zlib-devel libfdt-devel libasan libubsan bluez-libs-devel brlapi-devel bzip2-devel device-mapper-multipath-devel glusterfs-api-devel gnutls-devel gtk3-devel libattr-devel libcap-devel libcap-ng-devel libcurl-devel libjpeg-devel libpng-devel librbd-devel libssh2-devel libusbx-devel libxml2-devel lzo-devel ncurses-devel nettle-devel nss-devel numactl-devel SDL2-devel snappy-devel spice-server-devel systemtap-sdt-devel usbredir-devel virglrenderer-devel vte3-devel xen-devel mingw32-pixman mingw32-glib2 mingw32-gmp mingw32-SDL2 mingw32-pkg-config mingw32-gtk3 mingw32-gnutls mingw32-nettle mingw32-libtasn1 mingw32-libjpeg-turbo mingw32-libpng mingw32-curl mingw32-libssh2 mingw32-bzip2 mingw64-pixman mingw64-glib2 mingw64-gmp mingw64-SDL2 mingw64-pkg-config mingw64-gtk3 mingw64-gnutls mingw64-nettle mingw64-libtasn1 mingw64-libjpeg-turbo mingw64-libpng mingw64-curl mingw64-libssh2 mingw64-bzip2
J=8
V=
HOSTNAME=169d75f31ea3
DEBUG=
SHOW_ENV=1
PWD=/
HOME=/
CCACHE_DIR=/var/tmp/ccache
DISTTAG=f28container
QEMU_CONFIGURE_OPTS=--python=/usr/bin/python3
FGC=f28
TEST_DIR=/tmp/qemu-test
SHLVL=1
FEATURES=mingw clang pyyaml asan dtc
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAKEFLAGS= -j8
EXTRA_CONFIGURE_OPTS=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=x86_64-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=2.0 --with-gtkabi=3.0
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install
local state directory queried at runtime
Windows SDK no
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules
C compiler x86_64-w64-mingw32-gcc
Host C compiler cc
C++ compiler x86_64-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -mms-bitfields -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/x86_64-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -m64 -mcx16 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/libpng16
LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g
QEMU_LDFLAGS -L$(BUILD_DIR)/dtc/libfdt
make make
install install
python /usr/bin/python3 -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (2.0.5)
GTK support yes (3.22.16)
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support yes
GNUTLS rnd yes
libgcrypt no
libgcrypt kdf no
nettle yes (3.3)
nettle kdf yes
libtasn1 yes
curses support no
virgl support no
curl support yes
mingw32 support yes
Audio drivers dsound
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support yes
VNC PNG support yes
xen support no
brlapi support no
bluez support no
Documentation no
PIE no
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support no
Install blobs yes
KVM support no
HAX support yes
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support no
RDMA support no
fdt support git
membarrier no
preadv support no
fdatasync no
madvise no
posix_madvise no
posix_memalign no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends simple
Trace output file trace-<pid>
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info yes
QGA MSI support no
seccomp support no
coroutine backend win32
coroutine pool yes
debug stack usage no
mutex debugging no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support yes
TPM passthrough no
TPM emulator no
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone no
docker no
NOTE: cross-compilers enabled: 'x86_64-w64-mingw32-gcc'
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GEN qemu-options.def
GEN qapi-gen
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN aarch64-softmmu/config-devices.mak
GEN x86_64-softmmu/config-devices.mak
GEN ui/input-keymap-atset1-to-qcode.c
GEN ui/input-keymap-linux-to-qcode.c
GEN ui/input-keymap-qcode-to-atset1.c
GEN ui/input-keymap-qcode-to-atset2.c
GEN ui/input-keymap-qcode-to-atset3.c
GEN ui/input-keymap-qcode-to-linux.c
GEN ui/input-keymap-qcode-to-qnum.c
GEN ui/input-keymap-qcode-to-sun.c
GEN ui/input-keymap-qnum-to-qcode.c
GEN ui/input-keymap-usb-to-qcode.c
GEN ui/input-keymap-win32-to-qcode.c
GEN ui/input-keymap-x11-to-qcode.c
GEN ui/input-keymap-xorgevdev-to-qcode.c
GEN ui/input-keymap-xorgkbd-to-qcode.c
GEN ui/input-keymap-xorgxquartz-to-qcode.c
GEN ui/input-keymap-xorgxwin-to-qcode.c
GEN ui/input-keymap-osx-to-qcode.c
GEN tests/test-qapi-gen
GEN trace-root.h
GEN accel/kvm/trace.h
GEN accel/tcg/trace.h
GEN audio/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN crypto/trace.h
GEN hw/9pfs/trace.h
GEN hw/acpi/trace.h
GEN hw/alpha/trace.h
GEN hw/arm/trace.h
GEN hw/audio/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/display/trace.h
GEN hw/dma/trace.h
GEN hw/hppa/trace.h
GEN hw/i2c/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/ide/trace.h
GEN hw/input/trace.h
GEN hw/intc/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/net/trace.h
GEN hw/nvram/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/ppc/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/s390x/trace.h
GEN hw/scsi/trace.h
GEN hw/sd/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/timer/trace.h
GEN hw/tpm/trace.h
GEN hw/usb/trace.h
GEN hw/vfio/trace.h
GEN hw/virtio/trace.h
GEN hw/xen/trace.h
GEN io/trace.h
GEN linux-user/trace.h
GEN migration/trace.h
GEN nbd/trace.h
GEN net/trace.h
GEN qapi/trace.h
GEN qom/trace.h
GEN scsi/trace.h
GEN target/arm/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/ppc/trace.h
GEN target/s390x/trace.h
GEN target/sparc/trace.h
GEN ui/trace.h
GEN util/trace.h
GEN trace-root.c
GEN accel/kvm/trace.c
GEN accel/tcg/trace.c
GEN audio/trace.c
GEN block/trace.c
GEN chardev/trace.c
GEN crypto/trace.c
GEN hw/9pfs/trace.c
GEN hw/acpi/trace.c
GEN hw/alpha/trace.c
GEN hw/arm/trace.c
GEN hw/audio/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/display/trace.c
GEN hw/dma/trace.c
GEN hw/hppa/trace.c
GEN hw/i2c/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/ide/trace.c
GEN hw/input/trace.c
GEN hw/intc/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/net/trace.c
GEN hw/nvram/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/ppc/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/s390x/trace.c
GEN hw/scsi/trace.c
GEN hw/sd/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/timer/trace.c
GEN hw/tpm/trace.c
GEN hw/usb/trace.c
GEN hw/vfio/trace.c
GEN hw/virtio/trace.c
GEN hw/xen/trace.c
GEN io/trace.c
GEN linux-user/trace.c
GEN migration/trace.c
GEN nbd/trace.c
GEN net/trace.c
GEN qapi/trace.c
GEN qom/trace.c
GEN scsi/trace.c
GEN target/arm/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/ppc/trace.c
GEN target/s390x/trace.c
GEN target/sparc/trace.c
GEN ui/trace.c
GEN util/trace.c
GEN config-all-devices.mak
DEP /tmp/qemu-test/src/dtc/tests/dumptrees.c
DEP /tmp/qemu-test/src/dtc/tests/trees.S
DEP /tmp/qemu-test/src/dtc/tests/testutils.c
DEP /tmp/qemu-test/src/dtc/tests/value-labels.c
DEP /tmp/qemu-test/src/dtc/tests/asm_tree_dump.c
DEP /tmp/qemu-test/src/dtc/tests/truncated_property.c
DEP /tmp/qemu-test/src/dtc/tests/check_path.c
DEP /tmp/qemu-test/src/dtc/tests/overlay_bad_fixup.c
DEP /tmp/qemu-test/src/dtc/tests/overlay.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/property_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/integer-expressions.c
DEP /tmp/qemu-test/src/dtc/tests/utilfdt_test.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset_aliases.c
DEP /tmp/qemu-test/src/dtc/tests/add_subnode_with_nops.c
DEP /tmp/qemu-test/src/dtc/tests/dtb_reverse.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_unordered.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_ordered.c
DEP /tmp/qemu-test/src/dtc/tests/extra-terminating-null.c
DEP /tmp/qemu-test/src/dtc/tests/incbin.c
DEP /tmp/qemu-test/src/dtc/tests/boot-cpuid.c
DEP /tmp/qemu-test/src/dtc/tests/phandle_format.c
DEP /tmp/qemu-test/src/dtc/tests/path-references.c
DEP /tmp/qemu-test/src/dtc/tests/references.c
DEP /tmp/qemu-test/src/dtc/tests/string_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/propname_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop2.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop1.c
DEP /tmp/qemu-test/src/dtc/tests/del_node.c
DEP /tmp/qemu-test/src/dtc/tests/del_property.c
DEP /tmp/qemu-test/src/dtc/tests/set_name.c
DEP /tmp/qemu-test/src/dtc/tests/setprop.c
DEP /tmp/qemu-test/src/dtc/tests/rw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/open_pack.c
DEP /tmp/qemu-test/src/dtc/tests/nopulate.c
DEP /tmp/qemu-test/src/dtc/tests/mangle-layout.c
DEP /tmp/qemu-test/src/dtc/tests/move_and_save.c
DEP /tmp/qemu-test/src/dtc/tests/sw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/nop_node.c
DEP /tmp/qemu-test/src/dtc/tests/nop_property.c
DEP /tmp/qemu-test/src/dtc/tests/setprop_inplace.c
DEP /tmp/qemu-test/src/dtc/tests/stringlist.c
DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells.c
DEP /tmp/qemu-test/src/dtc/tests/notfound.c
DEP /tmp/qemu-test/src/dtc/tests/sized_cells.c
DEP /tmp/qemu-test/src/dtc/tests/char_literal.c
DEP /tmp/qemu-test/src/dtc/tests/get_alias.c
DEP /tmp/qemu-test/src/dtc/tests/node_check_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_prop_value.c
DEP /tmp/qemu-test/src/dtc/tests/supernode_atdepth_offset.c
DEP /tmp/qemu-test/src/dtc/tests/parent_offset.c
DEP /tmp/qemu-test/src/dtc/tests/get_path.c
DEP /tmp/qemu-test/src/dtc/tests/get_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/getprop.c
DEP /tmp/qemu-test/src/dtc/tests/get_name.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_offset.c
DEP /tmp/qemu-test/src/dtc/tests/find_property.c
DEP /tmp/qemu-test/src/dtc/tests/get_mem_rsv.c
DEP /tmp/qemu-test/src/dtc/tests/root_node.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_overlay.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_addresses.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_empty_tree.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_strerror.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_rw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_sw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_wip.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_ro.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt.c
DEP /tmp/qemu-test/src/dtc/util.c
DEP /tmp/qemu-test/src/dtc/fdtoverlay.c
DEP /tmp/qemu-test/src/dtc/fdtput.c
DEP /tmp/qemu-test/src/dtc/fdtget.c
DEP /tmp/qemu-test/src/dtc/srcpos.c
DEP /tmp/qemu-test/src/dtc/fdtdump.c
LEX convert-dtsv0-lexer.lex.c
LEX dtc-lexer.lex.c
BISON dtc-parser.tab.c
DEP /tmp/qemu-test/src/dtc/treesource.c
DEP /tmp/qemu-test/src/dtc/livetree.c
DEP /tmp/qemu-test/src/dtc/fstree.c
DEP /tmp/qemu-test/src/dtc/flattree.c
DEP /tmp/qemu-test/src/dtc/dtc.c
DEP /tmp/qemu-test/src/dtc/data.c
DEP /tmp/qemu-test/src/dtc/checks.c
DEP convert-dtsv0-lexer.lex.c
DEP dtc-parser.tab.c
DEP dtc-lexer.lex.c
CHK version_gen.h
UPD version_gen.h
DEP /tmp/qemu-test/src/dtc/util.c
CC libfdt/fdt.o
CC libfdt/fdt_wip.o
CC libfdt/fdt_rw.o
CC libfdt/fdt_ro.o
CC libfdt/fdt_sw.o
CC libfdt/fdt_empty_tree.o
CC libfdt/fdt_strerror.o
CC libfdt/fdt_addresses.o
CC libfdt/fdt_overlay.o
AR libfdt/libfdt.a
x86_64-w64-mingw32-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
RC version.o
GEN qga/qapi-generated/qapi-gen
CC qapi/qapi-builtin-types.o
CC qapi/qapi-types.o
CC qapi/qapi-types-block.o
CC qapi/qapi-types-char.o
CC qapi/qapi-types-common.o
CC qapi/qapi-types-block-core.o
CC qapi/qapi-types-crypto.o
CC qapi/qapi-types-introspect.o
CC qapi/qapi-types-job.o
CC qapi/qapi-types-migration.o
CC qapi/qapi-types-misc.o
CC qapi/qapi-types-net.o
CC qapi/qapi-types-rocker.o
CC qapi/qapi-types-run-state.o
CC qapi/qapi-types-sockets.o
CC qapi/qapi-types-tpm.o
CC qapi/qapi-types-trace.o
CC qapi/qapi-types-transaction.o
CC qapi/qapi-types-ui.o
CC qapi/qapi-builtin-visit.o
CC qapi/qapi-visit.o
CC qapi/qapi-visit-block-core.o
CC qapi/qapi-visit-block.o
CC qapi/qapi-visit-char.o
CC qapi/qapi-visit-common.o
CC qapi/qapi-visit-crypto.o
CC qapi/qapi-visit-introspect.o
CC qapi/qapi-visit-job.o
CC qapi/qapi-visit-migration.o
CC qapi/qapi-visit-misc.o
CC qapi/qapi-visit-net.o
CC qapi/qapi-visit-rocker.o
CC qapi/qapi-visit-run-state.o
CC qapi/qapi-visit-sockets.o
CC qapi/qapi-visit-tpm.o
CC qapi/qapi-visit-trace.o
CC qapi/qapi-visit-transaction.o
CC qapi/qapi-visit-ui.o
CC qapi/qapi-events.o
CC qapi/qapi-events-block-core.o
CC qapi/qapi-events-block.o
CC qapi/qapi-events-char.o
CC qapi/qapi-events-crypto.o
CC qapi/qapi-events-common.o
CC qapi/qapi-events-introspect.o
CC qapi/qapi-events-job.o
CC qapi/qapi-events-migration.o
CC qapi/qapi-events-misc.o
CC qapi/qapi-events-net.o
CC qapi/qapi-events-rocker.o
CC qapi/qapi-events-run-state.o
CC qapi/qapi-events-sockets.o
CC qapi/qapi-events-tpm.o
CC qapi/qapi-events-trace.o
CC qapi/qapi-events-transaction.o
CC qapi/qapi-events-ui.o
CC qapi/qapi-introspect.o
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qobject-input-visitor.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qnum.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qbool.o
CC qobject/qlit.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
CC qobject/block-qdict.o
CC trace/simple.o
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/lockcnt.o
CC util/aiocb.o
CC util/async.o
CC util/aio-wait.o
CC util/thread-pool.o
CC util/qemu-timer.o
CC util/main-loop.o
CC util/iohandler.o
CC util/aio-win32.o
CC util/event_notifier-win32.o
CC util/oslib-win32.o
CC util/qemu-thread-win32.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/host-utils.o
CC util/bitmap.o
CC util/bitops.o
CC util/fifo8.o
CC util/hbitmap.o
CC util/acl.o
CC util/cacheinfo.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/keyval.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/readline.o
CC util/rcu.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-win32.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/pagesize.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC util/stats64.o
CC util/systemd.o
CC util/iova-tree.o
CC trace-root.o
CC accel/kvm/trace.o
CC accel/tcg/trace.o
CC audio/trace.o
CC block/trace.o
CC chardev/trace.o
CC crypto/trace.o
CC hw/9pfs/trace.o
CC hw/acpi/trace.o
CC hw/alpha/trace.o
CC hw/arm/trace.o
CC hw/audio/trace.o
CC hw/block/trace.o
CC hw/block/dataplane/trace.o
CC hw/char/trace.o
CC hw/display/trace.o
CC hw/dma/trace.o
CC hw/hppa/trace.o
CC hw/i2c/trace.o
CC hw/i386/trace.o
CC hw/i386/xen/trace.o
CC hw/ide/trace.o
CC hw/input/trace.o
CC hw/intc/trace.o
CC hw/isa/trace.o
CC hw/mem/trace.o
CC hw/misc/trace.o
CC hw/misc/macio/trace.o
CC hw/net/trace.o
CC hw/nvram/trace.o
CC hw/pci/trace.o
CC hw/pci-host/trace.o
CC hw/ppc/trace.o
CC hw/rdma/trace.o
CC hw/rdma/vmw/trace.o
CC hw/s390x/trace.o
CC hw/scsi/trace.o
CC hw/sd/trace.o
CC hw/sparc/trace.o
CC hw/sparc64/trace.o
CC hw/timer/trace.o
CC hw/tpm/trace.o
CC hw/usb/trace.o
CC hw/vfio/trace.o
CC hw/virtio/trace.o
CC hw/xen/trace.o
CC io/trace.o
CC linux-user/trace.o
CC migration/trace.o
CC nbd/trace.o
CC qapi/trace.o
CC net/trace.o
CC qom/trace.o
CC scsi/trace.o
CC target/arm/trace.o
CC target/i386/trace.o
CC target/mips/trace.o
CC target/ppc/trace.o
CC target/s390x/trace.o
CC target/sparc/trace.o
CC ui/trace.o
CC util/trace.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-clock.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/error-printf.o
CC stubs/fdset.o
CC stubs/gdbstub.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/iothread-lock.o
CC stubs/is-daemonized.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/change-state-handler.o
CC stubs/monitor.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/tpm.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/fd-register.o
CC stubs/qmp_memory_device.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/vmgenid.o
CC stubs/xen-common.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
CC stubs/ram-block.o
GEN qemu-img-cmds.h
CC block.o
CC blockjob.o
CC job.o
CC qemu-io-cmds.o
CC replication.o
CC block/raw-format.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2-refcount.o
CC block/qcow2.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qcow2-bitmap.o
CC block/qed.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/vhdx-log.o
CC block/quorum.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/blklogwrites.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/file-win32.o
CC block/win32-aio.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
CC block/io.o
CC block/create.o
CC block/throttle-groups.o
CC block/nbd.o
CC block/nbd-client.o
CC block/sheepdog.o
/tmp/qemu-test/src/block/commit.c: In function 'commit_exit':
/tmp/qemu-test/src/block/commit.c:88:35: error: 'ret' undeclared (first use in this function); did you mean 'recv'?
if (!job_is_cancelled(job) && ret == 0) {
^~~
recv
/tmp/qemu-test/src/block/commit.c:88:35: note: each undeclared identifier is reported only once for each function it appears in
make: *** [/tmp/qemu-test/src/rules.mak:69: block/commit.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 565, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 562, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 308, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 276, in run
quiet=quiet)
File "./tests/docker/docker.py", line 183, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=5ef95e22a30311e8a6b152540069c830', '-u', '1000', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-a46s68_y/src/docker-src.2018-08-18-12.25.38.11223:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:216: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-a46s68_y/src'
make: *** [tests/docker/Makefile.include:250: docker-run-test-mingw@fedora] Error 2
real 1m54.240s
user 0m5.373s
sys 0m3.709s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
` (7 preceding siblings ...)
2018-08-18 16:27 ` [Qemu-devel] [PATCH 0/7] " no-reply
@ 2018-08-18 16:31 ` no-reply
2018-09-04 2:06 ` no-reply
2018-09-04 2:09 ` no-reply
10 siblings, 0 replies; 53+ messages in thread
From: no-reply @ 2018-08-18 16:31 UTC (permalink / raw)
To: jsnow; +Cc: famz, qemu-block, qemu-devel, kwolf, jcody, jtc, mreitz
Hi,
This series failed docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 20180817190457.8292-1-jsnow@redhat.com
Subject: [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-quick@centos7 SHOW_ENV=1 J=8
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
2d93111b33 jobs: remove job_defer_to_main_loop
082d48fa2c jobs: utilize job_exit shim
a2a302e5e1 block/mirror: utilize job_exit shim
7a113fae21 block/commit: utilize job_exit shim
df00a6221a jobs: add exit shim
16d80626ef jobs: canonize Error object
b2e0ebbc40 jobs: change start callback to run callback
=== OUTPUT BEGIN ===
BUILD centos7
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-alplqnbt/src'
GEN /var/tmp/patchew-tester-tmp-alplqnbt/src/docker-src.2018-08-18-12.30.26.23421/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-alplqnbt/src/docker-src.2018-08-18-12.30.26.23421/qemu.tar.vroot'...
done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-alplqnbt/src/docker-src.2018-08-18-12.30.26.23421/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-alplqnbt/src/docker-src.2018-08-18-12.30.26.23421/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
COPY RUNNER
RUN test-quick in qemu:centos7
Packages installed:
SDL-devel-1.2.15-14.el7.x86_64
bison-3.0.4-1.el7.x86_64
bzip2-1.0.6-13.el7.x86_64
bzip2-devel-1.0.6-13.el7.x86_64
ccache-3.3.4-1.el7.x86_64
csnappy-devel-0-6.20150729gitd7bc683.el7.x86_64
flex-2.5.37-3.el7.x86_64
gcc-4.8.5-28.el7_5.1.x86_64
gettext-0.19.8.1-2.el7.x86_64
git-1.8.3.1-14.el7_5.x86_64
glib2-devel-2.54.2-2.el7.x86_64
libaio-devel-0.3.109-13.el7.x86_64
libepoxy-devel-1.3.1-2.el7_5.x86_64
libfdt-devel-1.4.6-1.el7.x86_64
lzo-devel-2.06-8.el7.x86_64
make-3.82-23.el7.x86_64
mesa-libEGL-devel-17.2.3-8.20171019.el7.x86_64
mesa-libgbm-devel-17.2.3-8.20171019.el7.x86_64
nettle-devel-2.7.1-8.el7.x86_64
package g++ is not installed
package librdmacm-devel is not installed
pixman-devel-0.34.0-1.el7.x86_64
spice-glib-devel-0.34-3.el7_5.1.x86_64
spice-server-devel-0.14.0-2.el7_5.4.x86_64
tar-1.26-34.el7.x86_64
vte-devel-0.28.2-10.el7.x86_64
xen-devel-4.6.6-12.el7.x86_64
zlib-devel-1.2.7-17.el7.x86_64
Environment variables:
PACKAGES=bison bzip2 bzip2-devel ccache csnappy-devel flex g++ gcc gettext git glib2-devel libaio-devel libepoxy-devel libfdt-devel librdmacm-devel lzo-devel nettle-devel make mesa-libEGL-devel mesa-libgbm-devel pixman-devel SDL-devel spice-glib-devel spice-server-devel tar vte-devel xen-devel zlib-devel
HOSTNAME=1aef66ac45f1
MAKEFLAGS= -j8
J=8
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
TARGET_LIST=
SHLVL=1
HOME=/home/patchew
TEST_DIR=/tmp/qemu-test
FEATURES= dtc
DEBUG=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install
No C++ compiler available; disabling C++ specific optional code
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install/share/qemu
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install/bin
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib/qemu
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install/etc
local state directory /tmp/qemu-test/install/var
Manual directory /tmp/qemu-test/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules
C compiler cc
Host C compiler cc
C++ compiler
Objective-C compiler cc
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/include/pixman-1 -Werror -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -Wno-missing-braces -I/usr/include/libpng15 -pthread -I/usr/include/spice-server -I/usr/include/cacard -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/nss3 -I/usr/include/nspr4 -I/usr/include/spice-1
LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
QEMU_LDFLAGS
make make
install install
python python -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (1.2.15)
GTK support yes (2.24.31)
GTK GL support no
VTE support yes (0.28.2)
TLS priority NORMAL
GNUTLS support no
GNUTLS rnd no
libgcrypt no
libgcrypt kdf no
nettle yes (2.7.1)
nettle kdf yes
libtasn1 no
curses support yes
virgl support no
curl support no
mingw32 support no
Audio drivers oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support no
VNC PNG support yes
xen support yes
xen ctrl version 40600
pv dom build no
brlapi support no
bluez support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support yes
ATTR/XATTR support yes
Install blobs yes
KVM support yes
HAX support no
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support yes
RDMA support yes
fdt support system
membarrier no
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
posix_memalign yes
libcap-ng support no
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends log
spice support yes (0.12.13/0.14.0)
rbd support no
xfsctl support no
smartcard support yes
libusb no
usb net redir no
OpenGL support yes
OpenGL dmabufs yes
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
debug stack usage no
mutex debugging no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support no
TPM passthrough yes
TPM emulator yes
QOM debugging yes
Live block migration yes
lzo support yes
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone no
docker no
WARNING: Use of GTK 2.0 is deprecated and will be removed in
WARNING: future releases. Please switch to using GTK 3.0
WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0
NOTE: cross-compilers enabled: 'cc'
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GEN qemu-options.def
GEN qapi-gen
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN x86_64-softmmu/config-devices.mak
GEN aarch64-softmmu/config-devices.mak
GEN ui/input-keymap-atset1-to-qcode.c
GEN ui/input-keymap-linux-to-qcode.c
GEN ui/input-keymap-qcode-to-atset1.c
GEN ui/input-keymap-qcode-to-atset3.c
GEN ui/input-keymap-qcode-to-atset2.c
GEN ui/input-keymap-qcode-to-linux.c
GEN ui/input-keymap-qcode-to-qnum.c
GEN ui/input-keymap-qcode-to-sun.c
GEN ui/input-keymap-qnum-to-qcode.c
GEN ui/input-keymap-usb-to-qcode.c
GEN ui/input-keymap-win32-to-qcode.c
GEN ui/input-keymap-x11-to-qcode.c
GEN ui/input-keymap-xorgevdev-to-qcode.c
GEN ui/input-keymap-xorgkbd-to-qcode.c
GEN ui/input-keymap-xorgxquartz-to-qcode.c
GEN ui/input-keymap-xorgxwin-to-qcode.c
GEN ui/input-keymap-osx-to-qcode.c
GEN tests/test-qapi-gen
GEN trace-root.h
GEN accel/kvm/trace.h
GEN accel/tcg/trace.h
GEN audio/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN crypto/trace.h
GEN hw/9pfs/trace.h
GEN hw/acpi/trace.h
GEN hw/alpha/trace.h
GEN hw/arm/trace.h
GEN hw/audio/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/display/trace.h
GEN hw/dma/trace.h
GEN hw/hppa/trace.h
GEN hw/i2c/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/ide/trace.h
GEN hw/input/trace.h
GEN hw/intc/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/net/trace.h
GEN hw/nvram/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/ppc/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/s390x/trace.h
GEN hw/scsi/trace.h
GEN hw/sd/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/timer/trace.h
GEN hw/tpm/trace.h
GEN hw/usb/trace.h
GEN hw/vfio/trace.h
GEN hw/virtio/trace.h
GEN hw/xen/trace.h
GEN io/trace.h
GEN linux-user/trace.h
GEN migration/trace.h
GEN nbd/trace.h
GEN net/trace.h
GEN qapi/trace.h
GEN qom/trace.h
GEN scsi/trace.h
GEN target/arm/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/ppc/trace.h
GEN target/s390x/trace.h
GEN target/sparc/trace.h
GEN ui/trace.h
GEN util/trace.h
GEN trace-root.c
GEN accel/kvm/trace.c
GEN accel/tcg/trace.c
GEN audio/trace.c
GEN block/trace.c
GEN chardev/trace.c
GEN crypto/trace.c
GEN hw/9pfs/trace.c
GEN hw/acpi/trace.c
GEN hw/alpha/trace.c
GEN hw/arm/trace.c
GEN hw/audio/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/display/trace.c
GEN hw/dma/trace.c
GEN hw/hppa/trace.c
GEN hw/i2c/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/ide/trace.c
GEN hw/input/trace.c
GEN hw/intc/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/net/trace.c
GEN hw/nvram/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/ppc/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/s390x/trace.c
GEN hw/scsi/trace.c
GEN hw/sd/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/timer/trace.c
GEN hw/tpm/trace.c
GEN hw/usb/trace.c
GEN hw/vfio/trace.c
GEN hw/virtio/trace.c
GEN hw/xen/trace.c
GEN io/trace.c
GEN linux-user/trace.c
GEN migration/trace.c
GEN nbd/trace.c
GEN net/trace.c
GEN qapi/trace.c
GEN qom/trace.c
GEN scsi/trace.c
GEN target/arm/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/ppc/trace.c
GEN target/s390x/trace.c
GEN target/sparc/trace.c
GEN ui/trace.c
GEN util/trace.c
GEN config-all-devices.mak
CC tests/qemu-iotests/socket_scm_helper.o
GEN qga/qapi-generated/qapi-gen
CC qapi/qapi-types-block-core.o
CC qapi/qapi-builtin-types.o
CC qapi/qapi-types-char.o
CC qapi/qapi-types.o
CC qapi/qapi-types-block.o
CC qapi/qapi-types-common.o
CC qapi/qapi-types-crypto.o
CC qapi/qapi-types-introspect.o
CC qapi/qapi-types-job.o
CC qapi/qapi-types-migration.o
CC qapi/qapi-types-misc.o
CC qapi/qapi-types-net.o
CC qapi/qapi-types-rocker.o
CC qapi/qapi-types-run-state.o
CC qapi/qapi-types-sockets.o
CC qapi/qapi-types-tpm.o
CC qapi/qapi-types-trace.o
CC qapi/qapi-types-transaction.o
CC qapi/qapi-types-ui.o
CC qapi/qapi-builtin-visit.o
CC qapi/qapi-visit.o
CC qapi/qapi-visit-block-core.o
CC qapi/qapi-visit-block.o
CC qapi/qapi-visit-char.o
CC qapi/qapi-visit-common.o
CC qapi/qapi-visit-crypto.o
CC qapi/qapi-visit-introspect.o
CC qapi/qapi-visit-job.o
CC qapi/qapi-visit-migration.o
CC qapi/qapi-visit-misc.o
CC qapi/qapi-visit-net.o
CC qapi/qapi-visit-rocker.o
CC qapi/qapi-visit-run-state.o
CC qapi/qapi-visit-sockets.o
CC qapi/qapi-visit-tpm.o
CC qapi/qapi-visit-trace.o
CC qapi/qapi-visit-transaction.o
CC qapi/qapi-visit-ui.o
CC qapi/qapi-events.o
CC qapi/qapi-events-block-core.o
CC qapi/qapi-events-block.o
CC qapi/qapi-events-char.o
CC qapi/qapi-events-common.o
CC qapi/qapi-events-crypto.o
CC qapi/qapi-events-introspect.o
CC qapi/qapi-events-job.o
CC qapi/qapi-events-migration.o
CC qapi/qapi-events-misc.o
CC qapi/qapi-events-net.o
CC qapi/qapi-events-rocker.o
CC qapi/qapi-events-run-state.o
CC qapi/qapi-events-sockets.o
CC qapi/qapi-events-tpm.o
CC qapi/qapi-events-trace.o
CC qapi/qapi-events-ui.o
CC qapi/qapi-events-transaction.o
CC qapi/qapi-introspect.o
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qnum.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qbool.o
CC qobject/qjson.o
CC qobject/qlit.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
CC qobject/block-qdict.o
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/lockcnt.o
CC util/bufferiszero.o
CC util/aiocb.o
CC util/async.o
CC util/aio-wait.o
CC util/thread-pool.o
CC util/qemu-timer.o
CC util/main-loop.o
CC util/iohandler.o
CC util/aio-posix.o
CC util/compatfd.o
CC util/event_notifier-posix.o
CC util/mmap-alloc.o
CC util/oslib-posix.o
CC util/qemu-openpty.o
CC util/qemu-thread-posix.o
CC util/memfd.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/host-utils.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/fifo8.o
CC util/acl.o
CC util/cacheinfo.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/keyval.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/rcu.o
CC util/readline.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-ucontext.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/pagesize.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC util/stats64.o
CC util/systemd.o
CC util/iova-tree.o
CC util/vfio-helpers.o
CC trace-root.o
CC accel/kvm/trace.o
CC accel/tcg/trace.o
CC audio/trace.o
CC block/trace.o
CC chardev/trace.o
CC crypto/trace.o
CC hw/9pfs/trace.o
CC hw/acpi/trace.o
CC hw/alpha/trace.o
CC hw/arm/trace.o
CC hw/audio/trace.o
CC hw/block/trace.o
CC hw/char/trace.o
CC hw/display/trace.o
CC hw/block/dataplane/trace.o
CC hw/dma/trace.o
CC hw/hppa/trace.o
CC hw/i2c/trace.o
CC hw/i386/trace.o
CC hw/i386/xen/trace.o
CC hw/ide/trace.o
CC hw/input/trace.o
CC hw/intc/trace.o
CC hw/isa/trace.o
CC hw/mem/trace.o
CC hw/misc/trace.o
CC hw/misc/macio/trace.o
CC hw/net/trace.o
CC hw/nvram/trace.o
CC hw/pci/trace.o
CC hw/pci-host/trace.o
CC hw/ppc/trace.o
CC hw/rdma/trace.o
CC hw/rdma/vmw/trace.o
CC hw/s390x/trace.o
CC hw/scsi/trace.o
CC hw/sd/trace.o
CC hw/sparc/trace.o
CC hw/sparc64/trace.o
CC hw/timer/trace.o
CC hw/tpm/trace.o
CC hw/usb/trace.o
CC hw/vfio/trace.o
CC hw/virtio/trace.o
CC hw/xen/trace.o
CC io/trace.o
CC linux-user/trace.o
CC migration/trace.o
CC nbd/trace.o
CC qapi/trace.o
CC net/trace.o
CC qom/trace.o
CC scsi/trace.o
CC target/arm/trace.o
CC target/i386/trace.o
CC target/mips/trace.o
CC target/ppc/trace.o
CC target/s390x/trace.o
CC target/sparc/trace.o
CC ui/trace.o
CC util/trace.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-clock.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/error-printf.o
CC stubs/fdset.o
CC stubs/gdbstub.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/iothread-lock.o
CC stubs/is-daemonized.o
CC stubs/linux-aio.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/change-state-handler.o
CC stubs/monitor.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/tpm.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/qmp_memory_device.o
CC stubs/target-get-monitor-def.o
CC stubs/target-monitor-defs.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/vmgenid.o
CC stubs/xen-common.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
CC stubs/ram-block.o
CC contrib/ivshmem-client/ivshmem-client.o
CC contrib/ivshmem-client/main.o
CC contrib/ivshmem-server/ivshmem-server.o
CC contrib/ivshmem-server/main.o
CC qemu-nbd.o
CC block.o
CC blockjob.o
CC job.o
CC qemu-io-cmds.o
CC replication.o
CC block/raw-format.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qcow2-bitmap.o
CC block/qed.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/vhdx-log.o
CC block/quorum.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/blklogwrites.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/file-posix.o
CC block/linux-aio.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
CC block/io.o
CC block/create.o
CC block/throttle-groups.o
CC block/nvme.o
/tmp/qemu-test/src/block/commit.c: In function 'commit_exit':
/tmp/qemu-test/src/block/commit.c:88:35: error: 'ret' undeclared (first use in this function)
if (!job_is_cancelled(job) && ret == 0) {
^
/tmp/qemu-test/src/block/commit.c:88:35: note: each undeclared identifier is reported only once for each function it appears in
make: *** [block/commit.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 565, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 562, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 308, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 276, in run
quiet=quiet)
File "./tests/docker/docker.py", line 183, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=091b848ea30411e8960852540069c830', '-u', '1000', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-alplqnbt/src/docker-src.2018-08-18-12.30.26.23421:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:216: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-alplqnbt/src'
make: *** [tests/docker/Makefile.include:250: docker-run-test-quick@centos7] Error 2
real 1m30.368s
user 0m4.914s
sys 0m3.565s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback
2018-08-17 19:04 ` [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback John Snow
@ 2018-08-20 18:28 ` Eric Blake
2018-08-20 19:04 ` John Snow
2018-08-22 10:51 ` Max Reitz
1 sibling, 1 reply; 53+ messages in thread
From: Eric Blake @ 2018-08-20 18:28 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc, Max Reitz
On 08/17/2018 02:04 PM, John Snow wrote:
> Presently we codify the entry point for a job as the "start" callback,
> but a more apt name would be "run" to clarify the idea that when this
> function returns we consider the job to have "finished," except for
> any cleanup which occurs in separate callbacks later.
>
> As part of this clarification, change the signature to include an error
> object and a return code. The error ptr is not yet used, and the return
> code while captured, will be overwritten by actions in the job_completed
> function.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> +++ b/block/backup.c
> @@ -480,9 +480,9 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
> bdrv_dirty_iter_free(dbi);
> }
>
> -static void coroutine_fn backup_run(void *opaque)
> +static int coroutine_fn backup_run(Job *opaque_job, Error **errp)
> {
> - BackupBlockJob *job = opaque;
> + BackupBlockJob *job = container_of(opaque_job, BackupBlockJob, common.job);
Hmm. Here, you used the naming pair 'opaque_job' vs. 'job',...
> +++ b/block/commit.c
> @@ -134,9 +134,9 @@ static void commit_complete(Job *job, void *opaque)
> bdrv_unref(top);
> }
>
> -static void coroutine_fn commit_run(void *opaque)
> +static int coroutine_fn commit_run(Job *job, Error **errp)
> {
> - CommitBlockJob *s = opaque;
> + CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
while in the majority of the other clients, it was 'job' vs. 's'. Is it
worth making these names consistent, or is that too much bikeshed paint?
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback
2018-08-20 18:28 ` Eric Blake
@ 2018-08-20 19:04 ` John Snow
0 siblings, 0 replies; 53+ messages in thread
From: John Snow @ 2018-08-20 19:04 UTC (permalink / raw)
To: Eric Blake, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc, Max Reitz
On 08/20/2018 02:28 PM, Eric Blake wrote:
> On 08/17/2018 02:04 PM, John Snow wrote:
>> Presently we codify the entry point for a job as the "start" callback,
>> but a more apt name would be "run" to clarify the idea that when this
>> function returns we consider the job to have "finished," except for
>> any cleanup which occurs in separate callbacks later.
>>
>> As part of this clarification, change the signature to include an error
>> object and a return code. The error ptr is not yet used, and the return
>> code while captured, will be overwritten by actions in the job_completed
>> function.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>
>> +++ b/block/backup.c
>> @@ -480,9 +480,9 @@ static void
>> backup_incremental_init_copy_bitmap(BackupBlockJob *job)
>> bdrv_dirty_iter_free(dbi);
>> }
>> -static void coroutine_fn backup_run(void *opaque)
>> +static int coroutine_fn backup_run(Job *opaque_job, Error **errp)
>> {
>> - BackupBlockJob *job = opaque;
>> + BackupBlockJob *job = container_of(opaque_job, BackupBlockJob,
>> common.job);
>
> Hmm. Here, you used the naming pair 'opaque_job' vs. 'job',...
>
>> +++ b/block/commit.c
>> @@ -134,9 +134,9 @@ static void commit_complete(Job *job, void *opaque)
>> bdrv_unref(top);
>> }
>> -static void coroutine_fn commit_run(void *opaque)
>> +static int coroutine_fn commit_run(Job *job, Error **errp)
>> {
>> - CommitBlockJob *s = opaque;
>> + CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
>
> while in the majority of the other clients, it was 'job' vs. 's'. Is it
> worth making these names consistent, or is that too much bikeshed paint?
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
>
:)
Was just trying to keep the static down, but it did annoy me that it was
different.
I can either change it for "v2" or send a follow-up, depending.
--js
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 2/7] jobs: canonize Error object
2018-08-17 19:04 ` [Qemu-devel] [PATCH 2/7] jobs: canonize Error object John Snow
@ 2018-08-20 20:03 ` Eric Blake
2018-08-21 0:10 ` John Snow
` (2 subsequent siblings)
3 siblings, 0 replies; 53+ messages in thread
From: Eric Blake @ 2018-08-20 20:03 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc, Max Reitz
On 08/17/2018 02:04 PM, John Snow wrote:
> Jobs presently use both an Error object in the case of the create job,
> and char strings in the case of generic errors elsewhere.
>
> Unify the two paths as just j->err, and remove the extra argument from
> job_completed. The integer error code for job_completed is kept for now
> for use by pre-emptive cancellation.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> @@ -535,7 +535,6 @@ void job_drain(Job *job)
> }
> }
>
> -
> /**
> * All jobs must allow a pause point before entering their job proper. This
> * ensures that jobs can be paused prior to being started, then resumed later.
Spurious hunk? I'm not opposed to consistent choice of how many newlines
between functions, although it looks a bit odd in the overall patch.
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-17 19:04 ` [Qemu-devel] [PATCH 3/7] jobs: add exit shim John Snow
@ 2018-08-20 21:16 ` Eric Blake
2018-08-22 11:43 ` Max Reitz
1 sibling, 0 replies; 53+ messages in thread
From: Eric Blake @ 2018-08-20 21:16 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc, Max Reitz
On 08/17/2018 02:04 PM, John Snow wrote:
> All jobs do the same thing when they leave their running loop:
> - Store the return code in a structure
> - wait to receive this structure in the main thread
> - signal job completion via job_completed
>
> Few jobs do anything beyond exactly this. Consolidate this exit
> logic for a net reduction in SLOC.
>
> More seriously, when we utilize job_defer_to_main_loop_bh to call
> a function that calls job_completed, job_finalize_single will run
> in a context where it has recursively taken the aio_context lock,
> which can cause hangs if it puts down a reference that causes a flush.
>
> You can observe this in practice by looking at mirror_exit's careful
> placement of job_completed and bdrv_unref calls.
>
> If we centralize job exiting, we can signal job completion from outside
> of the aio_context, which should allow for job cleanup code to run with
> only one lock, which makes cleanup callbacks less tricky to write.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> include/qemu/job.h | 7 +++++++
> job.c | 19 +++++++++++++++++++
> 2 files changed, 26 insertions(+)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 2/7] jobs: canonize Error object
2018-08-17 19:04 ` [Qemu-devel] [PATCH 2/7] jobs: canonize Error object John Snow
2018-08-20 20:03 ` Eric Blake
@ 2018-08-21 0:10 ` John Snow
2018-08-22 10:59 ` Max Reitz
2018-08-22 11:09 ` Max Reitz
2018-08-22 11:11 ` Max Reitz
3 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-21 0:10 UTC (permalink / raw)
To: qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc, Max Reitz
On 08/17/2018 03:04 PM, John Snow wrote:
> + error_setg_errno(&job->err, -job->ret, "job failed");
Kevin specifically asked for me to change this, and I lost it in the
shuffle. I'll send a v3 now, since there are enough nits to warrant it,
and I think I want to adjust a few things to set up the "part II"
portion of this changeset a little more nicely.
--js
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback
2018-08-17 19:04 ` [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback John Snow
2018-08-20 18:28 ` Eric Blake
@ 2018-08-22 10:51 ` Max Reitz
2018-08-22 23:01 ` John Snow
1 sibling, 1 reply; 53+ messages in thread
From: Max Reitz @ 2018-08-22 10:51 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 2015 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> Presently we codify the entry point for a job as the "start" callback,
> but a more apt name would be "run" to clarify the idea that when this
> function returns we consider the job to have "finished," except for
> any cleanup which occurs in separate callbacks later.
>
> As part of this clarification, change the signature to include an error
> object and a return code. The error ptr is not yet used, and the return
> code while captured, will be overwritten by actions in the job_completed
> function.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/backup.c | 7 ++++---
> block/commit.c | 7 ++++---
> block/create.c | 8 +++++---
> block/mirror.c | 10 ++++++----
> block/stream.c | 7 ++++---
> include/qemu/job.h | 2 +-
> job.c | 6 +++---
> tests/test-bdrv-drain.c | 7 ++++---
> tests/test-blockjob-txn.c | 16 ++++++++--------
> tests/test-blockjob.c | 7 ++++---
> 10 files changed, 43 insertions(+), 34 deletions(-)
[...]
> diff --git a/job.c b/job.c
> index fa671b431a..898260b2b3 100644
> --- a/job.c
> +++ b/job.c
> @@ -544,16 +544,16 @@ static void coroutine_fn job_co_entry(void *opaque)
> {
> Job *job = opaque;
>
> - assert(job && job->driver && job->driver->start);
> + assert(job && job->driver && job->driver->run);
> job_pause_point(job);
> - job->driver->start(job);
> + job->ret = job->driver->run(job, NULL);
> }
Hmmm, this breaks the iff relationship with job->error. We might call
job_update_rc() afterwards, but then job_completed() would need to free
it if it overwrites it with the error description from a potential error
object.
Also, I suspect the following patches might fix the relationship anyway?
(But then an "XXX: This does not hold right now, but will be fixed in a
future patch" in the documentation of Job.error might help.)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 2/7] jobs: canonize Error object
2018-08-21 0:10 ` John Snow
@ 2018-08-22 10:59 ` Max Reitz
2018-08-22 22:50 ` John Snow
0 siblings, 1 reply; 53+ messages in thread
From: Max Reitz @ 2018-08-22 10:59 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
[-- Attachment #1: Type: text/plain, Size: 843 bytes --]
On 2018-08-21 02:10, John Snow wrote:
>
>
> On 08/17/2018 03:04 PM, John Snow wrote:
>> + error_setg_errno(&job->err, -job->ret, "job failed");
>
> Kevin specifically asked for me to change this, and I lost it in the
> shuffle. I'll send a v3 now, since there are enough nits to warrant it,
> and I think I want to adjust a few things to set up the "part II"
> portion of this changeset a little more nicely.
But error_setg_errno() uses either strerror() or
g_win32_error_message(), depending on the host OS. I prefer that over a
blind strerror(), to be honest.
In general, it might make sense to introduce a qemu_strerror() (which
g_strdup()s strerror() on Linux, I presume, so it's compatible with
Win32); or to allow passing a NULL format to error_setg_errno() so you
only get the error string.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 2/7] jobs: canonize Error object
2018-08-17 19:04 ` [Qemu-devel] [PATCH 2/7] jobs: canonize Error object John Snow
2018-08-20 20:03 ` Eric Blake
2018-08-21 0:10 ` John Snow
@ 2018-08-22 11:09 ` Max Reitz
2018-08-22 11:11 ` Max Reitz
3 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-22 11:09 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 1847 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> Jobs presently use both an Error object in the case of the create job,
> and char strings in the case of generic errors elsewhere.
>
> Unify the two paths as just j->err, and remove the extra argument from
> job_completed. The integer error code for job_completed is kept for now
> for use by pre-emptive cancellation.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/backup.c | 2 +-
> block/commit.c | 2 +-
> block/create.c | 5 ++---
> block/mirror.c | 2 +-
> block/stream.c | 2 +-
> include/qemu/job.h | 10 ++++------
> job-qmp.c | 5 +++--
> job.c | 19 ++++++-------------
> tests/test-bdrv-drain.c | 2 +-
> tests/test-blockjob-txn.c | 2 +-
> tests/test-blockjob.c | 2 +-
> 11 files changed, 22 insertions(+), 31 deletions(-)
So... Why does this patch come before removing the @ret parameter from
job_completed()?
[...]
> diff --git a/include/qemu/job.h b/include/qemu/job.h
> index 9cf463d228..5c92c53ef0 100644
> --- a/include/qemu/job.h
> +++ b/include/qemu/job.h
> @@ -124,12 +124,12 @@ typedef struct Job {
> /** Estimated progress_current value at the completion of the job */
> int64_t progress_total;
>
> - /** Error string for a failed job (NULL if, and only if, job->ret == 0) */
> - char *error;
> -
> /** ret code passed to job_completed. */
> int ret;
>
> + /** Error object for a failed job **/
> + Error *err;
> +
Is there a reason why you remove the iff relationship?
(Maybe because job_completed() still receives @ret? :-))
Max
> /** The completion function that will be called when the job completes. */
> BlockCompletionFunc *cb;
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 2/7] jobs: canonize Error object
2018-08-17 19:04 ` [Qemu-devel] [PATCH 2/7] jobs: canonize Error object John Snow
` (2 preceding siblings ...)
2018-08-22 11:09 ` Max Reitz
@ 2018-08-22 11:11 ` Max Reitz
3 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-22 11:11 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 1426 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> Jobs presently use both an Error object in the case of the create job,
> and char strings in the case of generic errors elsewhere.
>
> Unify the two paths as just j->err, and remove the extra argument from
> job_completed. The integer error code for job_completed is kept for now
> for use by pre-emptive cancellation.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/backup.c | 2 +-
> block/commit.c | 2 +-
> block/create.c | 5 ++---
> block/mirror.c | 2 +-
> block/stream.c | 2 +-
> include/qemu/job.h | 10 ++++------
> job-qmp.c | 5 +++--
> job.c | 19 ++++++-------------
> tests/test-bdrv-drain.c | 2 +-
> tests/test-blockjob-txn.c | 2 +-
> tests/test-blockjob.c | 2 +-
> 11 files changed, 22 insertions(+), 31 deletions(-)
(Accidentally deleted this part from my first reply)
> diff --git a/job.c b/job.c
> index 898260b2b3..c9de1af556 100644
> --- a/job.c
> +++ b/job.c
[...]
> @@ -535,7 +535,6 @@ void job_drain(Job *job)
> }
> }
>
> -
> /**
> * All jobs must allow a pause point before entering their job proper. This
> * ensures that jobs can be paused prior to being started, then resumed later.
If anything, you should remove one empty line after this function. ;-)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-17 19:04 ` [Qemu-devel] [PATCH 3/7] jobs: add exit shim John Snow
2018-08-20 21:16 ` Eric Blake
@ 2018-08-22 11:43 ` Max Reitz
2018-08-22 11:52 ` Max Reitz
` (2 more replies)
1 sibling, 3 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-22 11:43 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 3815 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> All jobs do the same thing when they leave their running loop:
> - Store the return code in a structure
> - wait to receive this structure in the main thread
> - signal job completion via job_completed
>
> Few jobs do anything beyond exactly this. Consolidate this exit
> logic for a net reduction in SLOC.
>
> More seriously, when we utilize job_defer_to_main_loop_bh to call
> a function that calls job_completed, job_finalize_single will run
> in a context where it has recursively taken the aio_context lock,
> which can cause hangs if it puts down a reference that causes a flush.
>
> You can observe this in practice by looking at mirror_exit's careful
> placement of job_completed and bdrv_unref calls.
>
> If we centralize job exiting, we can signal job completion from outside
> of the aio_context, which should allow for job cleanup code to run with
> only one lock, which makes cleanup callbacks less tricky to write.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> include/qemu/job.h | 7 +++++++
> job.c | 19 +++++++++++++++++++
> 2 files changed, 26 insertions(+)
Currently all jobs do this, the question of course is why. The answer
is because they are block jobs that need to do some graph manipulation
in the main thread, right?
OK, that's reasonable enough, that sounds like even non-block jobs may
need this (i.e. modify some global qemu state that you can only do in
the main loop). Interestingly, the create job only calls
job_completed() of which it says nowhere that it needs to be executed in
the main loop.
...on second thought, do we really want to execute job_complete() in the
main loop? First of all, all of the transactional functions will run in
the main loop. Which makes sense, but it isn't noted anywhere.
Secondly, we may end up calling JobDriver.user_resume(), which is
probably not something we want to call in the main loop.
OTOH, job_finish_sync() is something that has to be run in the main loop
because it polls the main loop (and as far as my FUSE experiments have
told me, polling a foreign AioContext doesn't work).
So... I suppose it would be nice if we had a real distinction which
functions are run in which AioContext. It seems like we indeed want to
run job_completed() in the main loop, but what to do about the
user_resume() call in job_cancel_async()?
(And it should be noted for all of the transactional methods that they
are called in the main loop.)
OK, so that's that. Now that I know what it's for, I'd like to ask for
a different name. exit() means kill the process. JobDriver.exit() will
not mean kill the job. That's where I get a headache.
This function is for allowing the job to carry out global qemu state
modifications in the main loop. Neither is that exiting in the sense
that the job is destroyed (as this is done only later, and the job gets
to take part in it through the transactional callbacks, and .free()),
nor is it exiting in the sense that the job needs to do all
pre-transactional clean-ups here (they are supposed to do that in .run()
-- *unlees* something needs the main loop).
I'd like .main_loop_settle(). Or .main_loop_post_run(). I think it's
OK to have names that aren't as cool and tense as possible, when in
return they actually tell you what they're doing. (Sure,
.main_loop_post_run() sounds really stupid, but it tells you exactly
when the function is called and what it's for.)
(Maybe the problem of all your naming woes really is just that you
always try to find a single word that describes what's going on :-) -- I
don't want to go into ProblemSolveFactoryObserverFactorySingleton
either, but it's OK to use an underscore once in a while.)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-22 11:43 ` Max Reitz
@ 2018-08-22 11:52 ` Max Reitz
2018-08-22 21:45 ` John Snow
2018-08-22 21:52 ` John Snow
2018-08-22 22:01 ` Eric Blake
2 siblings, 1 reply; 53+ messages in thread
From: Max Reitz @ 2018-08-22 11:52 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 681 bytes --]
On 2018-08-22 13:43, Max Reitz wrote:
[...]
> I'd like .main_loop_settle(). Or .main_loop_post_run(). I think it's
> OK to have names that aren't as cool and tense as possible, when in
> return they actually tell you what they're doing. (Sure,
> .main_loop_post_run() sounds really stupid, but it tells you exactly
> when the function is called and what it's for.)
Looking at the next patch, I realized that .main_loop_complete() or
.complete_in_main_loop() would work just as well. (No, I don't see any
confusion with whether you need to call job_completed(), especially
since after this series you can probably make that function static to
job.c.)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim
2018-08-17 19:04 ` [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim John Snow
2018-08-17 19:18 ` John Snow
@ 2018-08-22 11:55 ` Max Reitz
1 sibling, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-22 11:55 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 1574 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> Change the manual deferment to commit_complete into the implicit
> callback to job_exit, renaming commit_complete to commit_exit.
>
> This conversion does change the timing of when job_completed is
> called to after the bdrv_replace_node and bdrv_unref calls, which
> could have implications for bjob->blk which will now be put down
> after this cleanup.
>
> Kevin highlights that we did not take any permissions for that backend
> at job creation time, so it is safe to reorder these operations.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/commit.c | 20 ++++----------------
> 1 file changed, 4 insertions(+), 16 deletions(-)
>
> diff --git a/block/commit.c b/block/commit.c
> index 4a17bb73ec..93c3b6a39e 100644
> --- a/block/commit.c
> +++ b/block/commit.c
[...]
> @@ -93,8 +87,8 @@ static void commit_complete(Job *job, void *opaque)
>
> if (!job_is_cancelled(job) && ret == 0) {
> /* success */
> - ret = bdrv_drop_intermediate(s->commit_top_bs, base,
> - s->backing_file_str);
> + job->ret = bdrv_drop_intermediate(s->commit_top_bs, base,
> + s->backing_file_str);
This makes me ask myself why .exit() doesn't just return an int, like
.run(). And takes an Error **.
Max
> } else {
> /* XXX Can (or should) we somehow keep 'consistent read' blocked even
> * after the failed/cancelled commit job is gone? If we already wrote
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim
2018-08-17 19:18 ` John Snow
@ 2018-08-22 11:58 ` Max Reitz
2018-08-22 21:55 ` John Snow
0 siblings, 1 reply; 53+ messages in thread
From: Max Reitz @ 2018-08-22 11:58 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
[-- Attachment #1: Type: text/plain, Size: 2294 bytes --]
On 2018-08-17 21:18, John Snow wrote:
>
>
> On 08/17/2018 03:04 PM, John Snow wrote:
>> Change the manual deferment to commit_complete into the implicit
>> callback to job_exit, renaming commit_complete to commit_exit.
>>
>> This conversion does change the timing of when job_completed is
>> called to after the bdrv_replace_node and bdrv_unref calls, which
>> could have implications for bjob->blk which will now be put down
>> after this cleanup.
>>
>> Kevin highlights that we did not take any permissions for that backend
>> at job creation time, so it is safe to reorder these operations.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> block/commit.c | 20 ++++----------------
>> 1 file changed, 4 insertions(+), 16 deletions(-)
>>
>> diff --git a/block/commit.c b/block/commit.c
>> index 4a17bb73ec..93c3b6a39e 100644
>> --- a/block/commit.c
>> +++ b/block/commit.c
>> @@ -68,19 +68,13 @@ static int coroutine_fn commit_populate(BlockBackend *bs, BlockBackend *base,
>> return 0;
>> }
>>
>> -typedef struct {
>> - int ret;
>> -} CommitCompleteData;
>> -
>> -static void commit_complete(Job *job, void *opaque)
>> +static void commit_exit(Job *job)
>> {
>> CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
>> BlockJob *bjob = &s->common;
>> - CommitCompleteData *data = opaque;
>> BlockDriverState *top = blk_bs(s->top);
>> BlockDriverState *base = blk_bs(s->base);
>> BlockDriverState *commit_top_bs = s->commit_top_bs;
>> - int ret = data->ret;
>> bool remove_commit_top_bs = false;
>>
>> /* Make sure commit_top_bs and top stay around until bdrv_replace_node() */
>> @@ -93,8 +87,8 @@ static void commit_complete(Job *job, void *opaque)
>>
>> if (!job_is_cancelled(job) && ret == 0) {
>
> forgot to actually squash the change in here that replaces `ret` with
> `job->ret`.
A better interface would be one function that is called when .run() was
successful, and one that is called when it was not. (They can still
resolve into a single function in the job that is just called with a
boolean that's either true or false, but accessing *job to find out
whether .run() was successful or not seems kind of convoluted to me.)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-17 19:04 ` [Qemu-devel] [PATCH 5/7] block/mirror: " John Snow
@ 2018-08-22 12:06 ` Max Reitz
2018-08-22 12:15 ` Max Reitz
1 sibling, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-22 12:06 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 841 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> Change the manual deferment to mirror_exit into the implicit
> callback to job_exit and the mirror_exit callback.
>
> This does change the order of some bdrv_unref calls and job_completed,
> but thanks to the new context in which we call .job_exit, this is safe
> to defer the possible flushing of any nodes to the job_finalize_single
> cleanup stage.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/mirror.c | 25 +++++++++----------------
> 1 file changed, 9 insertions(+), 16 deletions(-)
Looks OK, but the same comment from the previous patch applies (I
wouldn't throw 'ret' around and instead work with just a boolean that
signifies failure or success. Also, you might even want to remove
s->ret because the Error object should be enough, actually.)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-17 19:04 ` [Qemu-devel] [PATCH 5/7] block/mirror: " John Snow
2018-08-22 12:06 ` Max Reitz
@ 2018-08-22 12:15 ` Max Reitz
2018-08-22 22:05 ` John Snow
1 sibling, 1 reply; 53+ messages in thread
From: Max Reitz @ 2018-08-22 12:15 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 1139 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> Change the manual deferment to mirror_exit into the implicit
> callback to job_exit and the mirror_exit callback.
>
> This does change the order of some bdrv_unref calls and job_completed,
> but thanks to the new context in which we call .job_exit, this is safe
> to defer the possible flushing of any nodes to the job_finalize_single
> cleanup stage.
Ah, right, I forgot this. Hm, what exactly do you mean? This function
is executed in the main loop, so it can make 'src' go away. I don't see
any difference to before.
The only difference I see is that the BH-scheduled function is now
job_exit() instead of just mirror_complete() (which is now called as
part of job_exit()). But then again, it was mirror_complete() itself
that called job_completed(), like it is now job_exit(). So if
everything worked after this patch, I don't see why mirror_complete()
would bdrv_ref() 'src' around job_completed().
Max
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/mirror.c | 25 +++++++++----------------
> 1 file changed, 9 insertions(+), 16 deletions(-)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 6/7] jobs: utilize job_exit shim
2018-08-17 19:04 ` [Qemu-devel] [PATCH 6/7] jobs: " John Snow
@ 2018-08-22 12:20 ` Max Reitz
2018-08-22 23:40 ` John Snow
0 siblings, 1 reply; 53+ messages in thread
From: Max Reitz @ 2018-08-22 12:20 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> Utilize the job_exit shim by not calling job_defer_to_main_loop, and
> where applicable, converting the deferred callback into the job_exit
> callback.
>
> This converts backup, stream, create, and the unit tests all at once.
> None of these jobs undergo and order of operations changes, so it should
> be a mechanical change.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/backup.c | 16 ----------------
> block/create.c | 14 +++-----------
> block/stream.c | 22 +++++++---------------
> tests/test-bdrv-drain.c | 6 ------
> tests/test-blockjob-txn.c | 11 ++---------
> tests/test-blockjob.c | 10 ++++------
> 6 files changed, 16 insertions(+), 63 deletions(-)
[...]
> diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c
> index 82cedee78b..ef29f35e44 100644
> --- a/tests/test-blockjob-txn.c
> +++ b/tests/test-blockjob-txn.c
> @@ -24,17 +24,11 @@ typedef struct {
> int *result;
> } TestBlockJob;
>
> -static void test_block_job_complete(Job *job, void *opaque)
> +static void test_block_job_exit(Job *job)
> {
> BlockJob *bjob = container_of(job, BlockJob, job);
> BlockDriverState *bs = blk_bs(bjob->blk);
> - int rc = (intptr_t)opaque;
>
> - if (job_is_cancelled(job)) {
> - rc = -ECANCELED;
> - }
> -
> - job_completed(job, rc);
> bdrv_unref(bs);
> }
That is a change in the order of operations, actually. It's OK though
because the BDS is still owned by the block job's BlockBackend.
(So I'd give an R-b, but I'm still unsure about the .exit() interface.)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 7/7] jobs: remove job_defer_to_main_loop
2018-08-17 19:04 ` [Qemu-devel] [PATCH 7/7] jobs: remove job_defer_to_main_loop John Snow
@ 2018-08-22 12:21 ` Max Reitz
0 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-22 12:21 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 567 bytes --]
On 2018-08-17 21:04, John Snow wrote:
> Now that the job infrastructure is handling the job_completed call for
> all implemented jobs, we can remove the interface that allowed jobs to
> schedule their own completion.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> include/qemu/job.h | 17 -----------------
> job.c | 40 ++--------------------------------------
> 2 files changed, 2 insertions(+), 55 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
In addition, you could make job_completed() static to job.c.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-22 11:52 ` Max Reitz
@ 2018-08-22 21:45 ` John Snow
2018-08-25 12:54 ` Max Reitz
0 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-22 21:45 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
On 08/22/2018 07:52 AM, Max Reitz wrote:
> On 2018-08-22 13:43, Max Reitz wrote:
>
> [...]
>
>> I'd like .main_loop_settle(). Or .main_loop_post_run(). I think it's
>> OK to have names that aren't as cool and tense as possible, when in
>> return they actually tell you what they're doing. (Sure,
>> .main_loop_post_run() sounds really stupid, but it tells you exactly
>> when the function is called and what it's for.)
>
> Looking at the next patch, I realized that .main_loop_complete() or
> .complete_in_main_loop() would work just as well. (No, I don't see any
> confusion with whether you need to call job_completed(), especially
> since after this series you can probably make that function static to
> job.c.)
>
> Max
>
I'm sorry to announce that after the part two of this series, the
callback will be erased completely, so the name is maybe less... important.
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-22 11:43 ` Max Reitz
2018-08-22 11:52 ` Max Reitz
@ 2018-08-22 21:52 ` John Snow
2018-08-25 13:05 ` Max Reitz
2018-08-22 22:01 ` Eric Blake
2 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-22 21:52 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
On 08/22/2018 07:43 AM, Max Reitz wrote:
> On 2018-08-17 21:04, John Snow wrote:
>> All jobs do the same thing when they leave their running loop:
>> - Store the return code in a structure
>> - wait to receive this structure in the main thread
>> - signal job completion via job_completed
>>
>> Few jobs do anything beyond exactly this. Consolidate this exit
>> logic for a net reduction in SLOC.
>>
>> More seriously, when we utilize job_defer_to_main_loop_bh to call
>> a function that calls job_completed, job_finalize_single will run
>> in a context where it has recursively taken the aio_context lock,
>> which can cause hangs if it puts down a reference that causes a flush.
>>
>> You can observe this in practice by looking at mirror_exit's careful
>> placement of job_completed and bdrv_unref calls.
>>
>> If we centralize job exiting, we can signal job completion from outside
>> of the aio_context, which should allow for job cleanup code to run with
>> only one lock, which makes cleanup callbacks less tricky to write.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> include/qemu/job.h | 7 +++++++
>> job.c | 19 +++++++++++++++++++
>> 2 files changed, 26 insertions(+)
>
> Currently all jobs do this, the question of course is why. The answer
> is because they are block jobs that need to do some graph manipulation
> in the main thread, right?
>
Yep.
> OK, that's reasonable enough, that sounds like even non-block jobs may
> need this (i.e. modify some global qemu state that you can only do in
> the main loop). Interestingly, the create job only calls
> job_completed() of which it says nowhere that it needs to be executed in
> the main loop.
>
Yeah, not all jobs will have anything meaningful to do in the main loop
context. This is one of them.
> ...on second thought, do we really want to execute job_complete() in the
> main loop? First of all, all of the transactional functions will run in
> the main loop. Which makes sense, but it isn't noted anywhere.
> Secondly, we may end up calling JobDriver.user_resume(), which is
> probably not something we want to call in the main loop.
>
I think we need to execute job_complete in the main loop, or otherwise
restructure the code that can run between job_completed and
job_finalize_single so that .prepare/.commit/.abort/.clean run in the
main thread, which is something we want to preserve.
It's simpler just to say that complete will run from the main thread,
like it does presently.
Why would we not want to call user_resume from the main loop? That's
directly where it's called from, since it gets invoked directly from the
qmp thread.
> OTOH, job_finish_sync() is something that has to be run in the main loop
> because it polls the main loop (and as far as my FUSE experiments have
> told me, polling a foreign AioContext doesn't work).
>
> So... I suppose it would be nice if we had a real distinction which
> functions are run in which AioContext. It seems like we indeed want to
> run job_completed() in the main loop, but what to do about the
> user_resume() call in job_cancel_async()?
>
I don't think we need to do anything -- at least, these functions
*already* run from the main loop.
mirror_exit et al get scheduled from job_defer_to_main_loop and call
job_completed there, so it's already always done from the main loop; I'm
just cutting out the part where the jobs have to manually schedule this.
> (And it should be noted for all of the transactional methods that they
> are called in the main loop.)
>
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim
2018-08-22 11:58 ` Max Reitz
@ 2018-08-22 21:55 ` John Snow
2018-08-25 13:07 ` Max Reitz
0 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-22 21:55 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
On 08/22/2018 07:58 AM, Max Reitz wrote:
> On 2018-08-17 21:18, John Snow wrote:
>>
>>
>> On 08/17/2018 03:04 PM, John Snow wrote:
>>> Change the manual deferment to commit_complete into the implicit
>>> callback to job_exit, renaming commit_complete to commit_exit.
>>>
>>> This conversion does change the timing of when job_completed is
>>> called to after the bdrv_replace_node and bdrv_unref calls, which
>>> could have implications for bjob->blk which will now be put down
>>> after this cleanup.
>>>
>>> Kevin highlights that we did not take any permissions for that backend
>>> at job creation time, so it is safe to reorder these operations.
>>>
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>> ---
>>> block/commit.c | 20 ++++----------------
>>> 1 file changed, 4 insertions(+), 16 deletions(-)
>>>
>>> diff --git a/block/commit.c b/block/commit.c
>>> index 4a17bb73ec..93c3b6a39e 100644
>>> --- a/block/commit.c
>>> +++ b/block/commit.c
>>> @@ -68,19 +68,13 @@ static int coroutine_fn commit_populate(BlockBackend *bs, BlockBackend *base,
>>> return 0;
>>> }
>>>
>>> -typedef struct {
>>> - int ret;
>>> -} CommitCompleteData;
>>> -
>>> -static void commit_complete(Job *job, void *opaque)
>>> +static void commit_exit(Job *job)
>>> {
>>> CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
>>> BlockJob *bjob = &s->common;
>>> - CommitCompleteData *data = opaque;
>>> BlockDriverState *top = blk_bs(s->top);
>>> BlockDriverState *base = blk_bs(s->base);
>>> BlockDriverState *commit_top_bs = s->commit_top_bs;
>>> - int ret = data->ret;
>>> bool remove_commit_top_bs = false;
>>>
>>> /* Make sure commit_top_bs and top stay around until bdrv_replace_node() */
>>> @@ -93,8 +87,8 @@ static void commit_complete(Job *job, void *opaque)
>>>
>>> if (!job_is_cancelled(job) && ret == 0) {
>>
>> forgot to actually squash the change in here that replaces `ret` with
>> `job->ret`.
>
> A better interface would be one function that is called when .run() was
> successful, and one that is called when it was not. (They can still
> resolve into a single function in the job that is just called with a
> boolean that's either true or false, but accessing *job to find out
> whether .run() was successful or not seems kind of convoluted to me.)
>
> Max
>
Sorry, I need a better cover letter.
.exit() is going away, and either .prepare() or .abort() will be called
after .run(), so what you're asking for will be true, just not all at
once in this series.
--js
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-22 11:43 ` Max Reitz
2018-08-22 11:52 ` Max Reitz
2018-08-22 21:52 ` John Snow
@ 2018-08-22 22:01 ` Eric Blake
2018-08-22 22:04 ` John Snow
2 siblings, 1 reply; 53+ messages in thread
From: Eric Blake @ 2018-08-22 22:01 UTC (permalink / raw)
To: Max Reitz, John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
On 08/22/2018 06:43 AM, Max Reitz wrote:
> On 2018-08-17 21:04, John Snow wrote:
>> All jobs do the same thing when they leave their running loop:
>> - Store the return code in a structure
>> - wait to receive this structure in the main thread
>> - signal job completion via job_completed
>>
>> Few jobs do anything beyond exactly this. Consolidate this exit
>> logic for a net reduction in SLOC.
>>
> OK, so that's that. Now that I know what it's for, I'd like to ask for
> a different name. exit() means kill the process. JobDriver.exit() will
> not mean kill the job. That's where I get a headache.
>
> This function is for allowing the job to carry out global qemu state
> modifications in the main loop. Neither is that exiting in the sense
> that the job is destroyed (as this is done only later, and the job gets
> to take part in it through the transactional callbacks, and .free()),
> nor is it exiting in the sense that the job needs to do all
> pre-transactional clean-ups here (they are supposed to do that in .run()
> -- *unlees* something needs the main loop).
>
> I'd like .main_loop_settle(). Or .main_loop_post_run(). I think it's
> OK to have names that aren't as cool and tense as possible, when in
> return they actually tell you what they're doing. (Sure,
> .main_loop_post_run() sounds really stupid, but it tells you exactly
> when the function is called and what it's for.)
>
> (Maybe the problem of all your naming woes really is just that you
> always try to find a single word that describes what's going on :-) -- I
> don't want to go into ProblemSolveFactoryObserverFactorySingleton
> either, but it's OK to use an underscore once in a while.)
Does .wrapup or .conclude work any better than .exit for such a one-word
name that goes away in the next series? Actually, your suggestion of
.settle seems reasonable to me (if we want terser than
.main_loop_settle, because the name is going away, but still have a name
that is not as weird as '.exit' when there are more steps still to follow)
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-22 22:01 ` Eric Blake
@ 2018-08-22 22:04 ` John Snow
0 siblings, 0 replies; 53+ messages in thread
From: John Snow @ 2018-08-22 22:04 UTC (permalink / raw)
To: Eric Blake, Max Reitz, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
On 08/22/2018 06:01 PM, Eric Blake wrote:
> On 08/22/2018 06:43 AM, Max Reitz wrote:
>> On 2018-08-17 21:04, John Snow wrote:
>>> All jobs do the same thing when they leave their running loop:
>>> - Store the return code in a structure
>>> - wait to receive this structure in the main thread
>>> - signal job completion via job_completed
>>>
>>> Few jobs do anything beyond exactly this. Consolidate this exit
>>> logic for a net reduction in SLOC.
>>>
>
>> OK, so that's that. Now that I know what it's for, I'd like to ask for
>> a different name. exit() means kill the process. JobDriver.exit() will
>> not mean kill the job. That's where I get a headache.
>>
>> This function is for allowing the job to carry out global qemu state
>> modifications in the main loop. Neither is that exiting in the sense
>> that the job is destroyed (as this is done only later, and the job gets
>> to take part in it through the transactional callbacks, and .free()),
>> nor is it exiting in the sense that the job needs to do all
>> pre-transactional clean-ups here (they are supposed to do that in .run()
>> -- *unlees* something needs the main loop).
>>
>> I'd like .main_loop_settle(). Or .main_loop_post_run(). I think it's
>> OK to have names that aren't as cool and tense as possible, when in
>> return they actually tell you what they're doing. (Sure,
>> .main_loop_post_run() sounds really stupid, but it tells you exactly
>> when the function is called and what it's for.)
>>
>> (Maybe the problem of all your naming woes really is just that you
>> always try to find a single word that describes what's going on :-) -- I
>> don't want to go into ProblemSolveFactoryObserverFactorySingleton
>> either, but it's OK to use an underscore once in a while.)
>
> Does .wrapup or .conclude work any better than .exit for such a one-word
> name that goes away in the next series? Actually, your suggestion of
> .settle seems reasonable to me (if we want terser than
> .main_loop_settle, because the name is going away, but still have a name
> that is not as weird as '.exit' when there are more steps still to follow)
>
This is running away from me :)
.exit() goes away after part two of the series, once I refactor all of
these completion functions into their .prepare/.abort/.commit/.clean
components.
It's just important that I do this series ***FIRST***, to avoid
deadlocks in the component callbacks.
--js
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-22 12:15 ` Max Reitz
@ 2018-08-22 22:05 ` John Snow
2018-08-25 15:02 ` Max Reitz
0 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-22 22:05 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
On 08/22/2018 08:15 AM, Max Reitz wrote:
> On 2018-08-17 21:04, John Snow wrote:
>> Change the manual deferment to mirror_exit into the implicit
>> callback to job_exit and the mirror_exit callback.
>>
>> This does change the order of some bdrv_unref calls and job_completed,
>> but thanks to the new context in which we call .job_exit, this is safe
>> to defer the possible flushing of any nodes to the job_finalize_single
>> cleanup stage.
>
> Ah, right, I forgot this. Hm, what exactly do you mean? This function
> is executed in the main loop, so it can make 'src' go away. I don't see
> any difference to before.
>
This changes the order in which we unreference these objects; if you
look at this patch the job_completed call I delete is in the middle of
what becomes the .exit() callback, which means there is a subtle change
in the ordering of how references are put down.
Take a look at the weird ordering of mirror_exit as it exists right now;
we call job_completed first and *then* put down the last references. If
you re-order this upstream right now, you'll deadlock QEMU because this
means job_completed is responsible for putting down the last reference
to some of these block/bds objects.
However, job_completed takes an additional AIO context lock and calls
job_finalize_single under *two* locks, which will hang QEMU if we
attempt to flush any of these nodes when we put down the last reference.
Performing the reordering here is *safe* because by removing the call to
job_completed and utilizing the exit shim, the .exit() callback executes
only under one lock, and when the finalize code runs later it is also
executed under only one lock, making this re-ordering safe.
Clear as mud?
--js
> The only difference I see is that the BH-scheduled function is now
> job_exit() instead of just mirror_complete() (which is now called as
> part of job_exit()). But then again, it was mirror_complete() itself
> that called job_completed(), like it is now job_exit(). So if
> everything worked after this patch, I don't see why mirror_complete()
> would bdrv_ref() 'src' around job_completed().
>
> Max
>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> block/mirror.c | 25 +++++++++----------------
>> 1 file changed, 9 insertions(+), 16 deletions(-)
>
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 2/7] jobs: canonize Error object
2018-08-22 10:59 ` Max Reitz
@ 2018-08-22 22:50 ` John Snow
2018-08-25 13:15 ` Max Reitz
0 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-22 22:50 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
On 08/22/2018 06:59 AM, Max Reitz wrote:
> On 2018-08-21 02:10, John Snow wrote:
>>
>>
>> On 08/17/2018 03:04 PM, John Snow wrote:
>>> + error_setg_errno(&job->err, -job->ret, "job failed");
>>
>> Kevin specifically asked for me to change this, and I lost it in the
>> shuffle. I'll send a v3 now, since there are enough nits to warrant it,
>> and I think I want to adjust a few things to set up the "part II"
>> portion of this changeset a little more nicely.
>
> But error_setg_errno() uses either strerror() or
> g_win32_error_message(), depending on the host OS. I prefer that over a
> blind strerror(), to be honest.
>
uhh, does it...?
it looks like error_setg_errno is always error_setg_errno_internal,
which always uses strerror... am I misreading?
> In general, it might make sense to introduce a qemu_strerror() (which
> g_strdup()s strerror() on Linux, I presume, so it's compatible with
> Win32); or to allow passing a NULL format to error_setg_errno() so you
> only get the error string.
>
> Max
>
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback
2018-08-22 10:51 ` Max Reitz
@ 2018-08-22 23:01 ` John Snow
2018-08-25 13:33 ` Max Reitz
0 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-22 23:01 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
On 08/22/2018 06:51 AM, Max Reitz wrote:
> On 2018-08-17 21:04, John Snow wrote:
>> Presently we codify the entry point for a job as the "start" callback,
>> but a more apt name would be "run" to clarify the idea that when this
>> function returns we consider the job to have "finished," except for
>> any cleanup which occurs in separate callbacks later.
>>
>> As part of this clarification, change the signature to include an error
>> object and a return code. The error ptr is not yet used, and the return
>> code while captured, will be overwritten by actions in the job_completed
>> function.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> block/backup.c | 7 ++++---
>> block/commit.c | 7 ++++---
>> block/create.c | 8 +++++---
>> block/mirror.c | 10 ++++++----
>> block/stream.c | 7 ++++---
>> include/qemu/job.h | 2 +-
>> job.c | 6 +++---
>> tests/test-bdrv-drain.c | 7 ++++---
>> tests/test-blockjob-txn.c | 16 ++++++++--------
>> tests/test-blockjob.c | 7 ++++---
>> 10 files changed, 43 insertions(+), 34 deletions(-)
>
> [...]
>
>> diff --git a/job.c b/job.c
>> index fa671b431a..898260b2b3 100644
>> --- a/job.c
>> +++ b/job.c
>> @@ -544,16 +544,16 @@ static void coroutine_fn job_co_entry(void *opaque)
>> {
>> Job *job = opaque;
>>
>> - assert(job && job->driver && job->driver->start);
>> + assert(job && job->driver && job->driver->run);
>> job_pause_point(job);
>> - job->driver->start(job);
>> + job->ret = job->driver->run(job, NULL);
>> }
>
> Hmmm, this breaks the iff relationship with job->error. We might call
> job_update_rc() afterwards, but then job_completed() would need to free
> it if it overwrites it with the error description from a potential error
> object.
>
> Also, I suspect the following patches might fix the relationship anyway?
> (But then an "XXX: This does not hold right now, but will be fixed in a
> future patch" in the documentation of Job.error might help.)
>
> Max
>
Hmm... does it? ... I guess you mean that we are setting job->ret
earlier than we used to, which gives us a window where you can have ret
set, but error unset.
This will get settled out by the end of the series anyway:
- char *error gets replaced with Error *err,
- I remove the error object from job_completed
- v2 will remove the ret argument, too.
--js
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 6/7] jobs: utilize job_exit shim
2018-08-22 12:20 ` Max Reitz
@ 2018-08-22 23:40 ` John Snow
0 siblings, 0 replies; 53+ messages in thread
From: John Snow @ 2018-08-22 23:40 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
On 08/22/2018 08:20 AM, Max Reitz wrote:
> On 2018-08-17 21:04, John Snow wrote:
>> Utilize the job_exit shim by not calling job_defer_to_main_loop, and
>> where applicable, converting the deferred callback into the job_exit
>> callback.
>>
>> This converts backup, stream, create, and the unit tests all at once.
>> None of these jobs undergo and order of operations changes, so it should
>> be a mechanical change.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> block/backup.c | 16 ----------------
>> block/create.c | 14 +++-----------
>> block/stream.c | 22 +++++++---------------
>> tests/test-bdrv-drain.c | 6 ------
>> tests/test-blockjob-txn.c | 11 ++---------
>> tests/test-blockjob.c | 10 ++++------
>> 6 files changed, 16 insertions(+), 63 deletions(-)
>
> [...]
>
>> diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c
>> index 82cedee78b..ef29f35e44 100644
>> --- a/tests/test-blockjob-txn.c
>> +++ b/tests/test-blockjob-txn.c
>> @@ -24,17 +24,11 @@ typedef struct {
>> int *result;
>> } TestBlockJob;
>>
>> -static void test_block_job_complete(Job *job, void *opaque)
>> +static void test_block_job_exit(Job *job)
>> {
>> BlockJob *bjob = container_of(job, BlockJob, job);
>> BlockDriverState *bs = blk_bs(bjob->blk);
>> - int rc = (intptr_t)opaque;
>>
>> - if (job_is_cancelled(job)) {
>> - rc = -ECANCELED;
>> - }
>> -
>> - job_completed(job, rc);
>> bdrv_unref(bs);
>> }
>
> That is a change in the order of operations, actually. It's OK though
> because the BDS is still owned by the block job's BlockBackend.
>
> (So I'd give an R-b, but I'm still unsure about the .exit() interface.)
>
> Max
>
Good eye; it does change the order of operations here, but like the
preceding two patches it should be safe for the same reasons.
I'll just update the commit message to keep things easier.
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-22 21:45 ` John Snow
@ 2018-08-25 12:54 ` Max Reitz
0 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-25 12:54 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]
On 2018-08-22 23:45, John Snow wrote:
>
>
> On 08/22/2018 07:52 AM, Max Reitz wrote:
>> On 2018-08-22 13:43, Max Reitz wrote:
>>
>> [...]
>>
>>> I'd like .main_loop_settle(). Or .main_loop_post_run(). I think it's
>>> OK to have names that aren't as cool and tense as possible, when in
>>> return they actually tell you what they're doing. (Sure,
>>> .main_loop_post_run() sounds really stupid, but it tells you exactly
>>> when the function is called and what it's for.)
>>
>> Looking at the next patch, I realized that .main_loop_complete() or
>> .complete_in_main_loop() would work just as well. (No, I don't see any
>> confusion with whether you need to call job_completed(), especially
>> since after this series you can probably make that function static to
>> job.c.)
>>
>> Max
>>
>
> I'm sorry to announce that after the part two of this series, the
> callback will be erased completely, so the name is maybe less... important.
That's what I get for not reading the cover letter.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-22 21:52 ` John Snow
@ 2018-08-25 13:05 ` Max Reitz
2018-08-27 15:54 ` John Snow
0 siblings, 1 reply; 53+ messages in thread
From: Max Reitz @ 2018-08-25 13:05 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
[-- Attachment #1: Type: text/plain, Size: 4868 bytes --]
On 2018-08-22 23:52, John Snow wrote:
>
>
> On 08/22/2018 07:43 AM, Max Reitz wrote:
>> On 2018-08-17 21:04, John Snow wrote:
>>> All jobs do the same thing when they leave their running loop:
>>> - Store the return code in a structure
>>> - wait to receive this structure in the main thread
>>> - signal job completion via job_completed
>>>
>>> Few jobs do anything beyond exactly this. Consolidate this exit
>>> logic for a net reduction in SLOC.
>>>
>>> More seriously, when we utilize job_defer_to_main_loop_bh to call
>>> a function that calls job_completed, job_finalize_single will run
>>> in a context where it has recursively taken the aio_context lock,
>>> which can cause hangs if it puts down a reference that causes a flush.
>>>
>>> You can observe this in practice by looking at mirror_exit's careful
>>> placement of job_completed and bdrv_unref calls.
>>>
>>> If we centralize job exiting, we can signal job completion from outside
>>> of the aio_context, which should allow for job cleanup code to run with
>>> only one lock, which makes cleanup callbacks less tricky to write.
>>>
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>> ---
>>> include/qemu/job.h | 7 +++++++
>>> job.c | 19 +++++++++++++++++++
>>> 2 files changed, 26 insertions(+)
>>
>> Currently all jobs do this, the question of course is why. The answer
>> is because they are block jobs that need to do some graph manipulation
>> in the main thread, right?
>>
>
> Yep.
>
>> OK, that's reasonable enough, that sounds like even non-block jobs may
>> need this (i.e. modify some global qemu state that you can only do in
>> the main loop). Interestingly, the create job only calls
>> job_completed() of which it says nowhere that it needs to be executed in
>> the main loop.
>>
>
> Yeah, not all jobs will have anything meaningful to do in the main loop
> context. This is one of them.
>
>> ...on second thought, do we really want to execute job_complete() in the
>> main loop? First of all, all of the transactional functions will run in
>> the main loop. Which makes sense, but it isn't noted anywhere.
>> Secondly, we may end up calling JobDriver.user_resume(), which is
>> probably not something we want to call in the main loop.
>>
>
> I think we need to execute job_complete in the main loop, or otherwise
> restructure the code that can run between job_completed and
> job_finalize_single so that .prepare/.commit/.abort/.clean run in the
> main thread, which is something we want to preserve.
Sure.
> It's simpler just to say that complete will run from the main thread,
> like it does presently.
Yes, but we don't say that.
> Why would we not want to call user_resume from the main loop? That's
> directly where it's called from, since it gets invoked directly from the
> qmp thread.
Hmm! True indeed.
The reason why we might not want to do it is because the job may not run
in the main loop, so modifying the job (especially invoking a job
method) may be dangerous without taking precautions.
>> OTOH, job_finish_sync() is something that has to be run in the main loop
>> because it polls the main loop (and as far as my FUSE experiments have
>> told me, polling a foreign AioContext doesn't work).
>>
>> So... I suppose it would be nice if we had a real distinction which
>> functions are run in which AioContext. It seems like we indeed want to
>> run job_completed() in the main loop, but what to do about the
>> user_resume() call in job_cancel_async()?
>>
>
> I don't think we need to do anything -- at least, these functions
> *already* run from the main loop.
Yeah, but we don't mark that anywhere. I really don't like that. Jobs
need to know which of their functions are run in which AioContext.
> mirror_exit et al get scheduled from job_defer_to_main_loop and call
> job_completed there, so it's already always done from the main loop; I'm
> just cutting out the part where the jobs have to manually schedule this.
I'm not saying what you're doing is wrong, I'm just saying tracking
which things are running in which context is not easy because there are
no comments on how it's supposed to be run. (Apart from your new
.exit() method which does say that it's run in the main loop.)
No, I don't find it obvious which functions are run in which context
when first I have to think about in which context those functions are
used (e.g. user_resume is usually the result of a QMP command, so it's
run in the main loop; the transactional methods are part of completion,
which is done in the main loop, so they are also called in the main
loop; and so on).
But that's not part of this series. It just occurred to me when
tracking down which function belongs to which context when reviewing
this patch.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim
2018-08-22 21:55 ` John Snow
@ 2018-08-25 13:07 ` Max Reitz
0 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-25 13:07 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
[-- Attachment #1: Type: text/plain, Size: 2823 bytes --]
On 2018-08-22 23:55, John Snow wrote:
>
>
> On 08/22/2018 07:58 AM, Max Reitz wrote:
>> On 2018-08-17 21:18, John Snow wrote:
>>>
>>>
>>> On 08/17/2018 03:04 PM, John Snow wrote:
>>>> Change the manual deferment to commit_complete into the implicit
>>>> callback to job_exit, renaming commit_complete to commit_exit.
>>>>
>>>> This conversion does change the timing of when job_completed is
>>>> called to after the bdrv_replace_node and bdrv_unref calls, which
>>>> could have implications for bjob->blk which will now be put down
>>>> after this cleanup.
>>>>
>>>> Kevin highlights that we did not take any permissions for that backend
>>>> at job creation time, so it is safe to reorder these operations.
>>>>
>>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>>> ---
>>>> block/commit.c | 20 ++++----------------
>>>> 1 file changed, 4 insertions(+), 16 deletions(-)
>>>>
>>>> diff --git a/block/commit.c b/block/commit.c
>>>> index 4a17bb73ec..93c3b6a39e 100644
>>>> --- a/block/commit.c
>>>> +++ b/block/commit.c
>>>> @@ -68,19 +68,13 @@ static int coroutine_fn commit_populate(BlockBackend *bs, BlockBackend *base,
>>>> return 0;
>>>> }
>>>>
>>>> -typedef struct {
>>>> - int ret;
>>>> -} CommitCompleteData;
>>>> -
>>>> -static void commit_complete(Job *job, void *opaque)
>>>> +static void commit_exit(Job *job)
>>>> {
>>>> CommitBlockJob *s = container_of(job, CommitBlockJob, common.job);
>>>> BlockJob *bjob = &s->common;
>>>> - CommitCompleteData *data = opaque;
>>>> BlockDriverState *top = blk_bs(s->top);
>>>> BlockDriverState *base = blk_bs(s->base);
>>>> BlockDriverState *commit_top_bs = s->commit_top_bs;
>>>> - int ret = data->ret;
>>>> bool remove_commit_top_bs = false;
>>>>
>>>> /* Make sure commit_top_bs and top stay around until bdrv_replace_node() */
>>>> @@ -93,8 +87,8 @@ static void commit_complete(Job *job, void *opaque)
>>>>
>>>> if (!job_is_cancelled(job) && ret == 0) {
>>>
>>> forgot to actually squash the change in here that replaces `ret` with
>>> `job->ret`.
>>
>> A better interface would be one function that is called when .run() was
>> successful, and one that is called when it was not. (They can still
>> resolve into a single function in the job that is just called with a
>> boolean that's either true or false, but accessing *job to find out
>> whether .run() was successful or not seems kind of convoluted to me.)
>>
>> Max
>>
>
> Sorry, I need a better cover letter.
My mistake, I need to read more than the first few lines of a cover letter.
> .exit() is going away, and either .prepare() or .abort() will be called
> after .run(), so what you're asking for will be true, just not all at
> once in this series.
Yay!
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 2/7] jobs: canonize Error object
2018-08-22 22:50 ` John Snow
@ 2018-08-25 13:15 ` Max Reitz
0 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-25 13:15 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]
On 2018-08-23 00:50, John Snow wrote:
>
>
> On 08/22/2018 06:59 AM, Max Reitz wrote:
>> On 2018-08-21 02:10, John Snow wrote:
>>>
>>>
>>> On 08/17/2018 03:04 PM, John Snow wrote:
>>>> + error_setg_errno(&job->err, -job->ret, "job failed");
>>>
>>> Kevin specifically asked for me to change this, and I lost it in the
>>> shuffle. I'll send a v3 now, since there are enough nits to warrant it,
>>> and I think I want to adjust a few things to set up the "part II"
>>> portion of this changeset a little more nicely.
>>
>> But error_setg_errno() uses either strerror() or
>> g_win32_error_message(), depending on the host OS. I prefer that over a
>> blind strerror(), to be honest.
>>
>
> uhh, does it...?
>
> it looks like error_setg_errno is always error_setg_errno_internal,
> which always uses strerror... am I misreading?
Ah, right, I was... I thought the #ifdef below somehow was referring to
the same function, so it resolved to something else. Yeah, sure,
usually the errno values comes from something we set ourselves and not
even the real OS API, so we need to always use strerror().
Sorry.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback
2018-08-22 23:01 ` John Snow
@ 2018-08-25 13:33 ` Max Reitz
2018-08-25 14:15 ` Max Reitz
2018-08-27 16:01 ` John Snow
0 siblings, 2 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-25 13:33 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 3789 bytes --]
On 2018-08-23 01:01, John Snow wrote:
>
>
> On 08/22/2018 06:51 AM, Max Reitz wrote:
>> On 2018-08-17 21:04, John Snow wrote:
>>> Presently we codify the entry point for a job as the "start" callback,
>>> but a more apt name would be "run" to clarify the idea that when this
>>> function returns we consider the job to have "finished," except for
>>> any cleanup which occurs in separate callbacks later.
>>>
>>> As part of this clarification, change the signature to include an error
>>> object and a return code. The error ptr is not yet used, and the return
>>> code while captured, will be overwritten by actions in the job_completed
>>> function.
>>>
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>> ---
>>> block/backup.c | 7 ++++---
>>> block/commit.c | 7 ++++---
>>> block/create.c | 8 +++++---
>>> block/mirror.c | 10 ++++++----
>>> block/stream.c | 7 ++++---
>>> include/qemu/job.h | 2 +-
>>> job.c | 6 +++---
>>> tests/test-bdrv-drain.c | 7 ++++---
>>> tests/test-blockjob-txn.c | 16 ++++++++--------
>>> tests/test-blockjob.c | 7 ++++---
>>> 10 files changed, 43 insertions(+), 34 deletions(-)
>>
>> [...]
>>
>>> diff --git a/job.c b/job.c
>>> index fa671b431a..898260b2b3 100644
>>> --- a/job.c
>>> +++ b/job.c
>>> @@ -544,16 +544,16 @@ static void coroutine_fn job_co_entry(void *opaque)
>>> {
>>> Job *job = opaque;
>>>
>>> - assert(job && job->driver && job->driver->start);
>>> + assert(job && job->driver && job->driver->run);
>>> job_pause_point(job);
>>> - job->driver->start(job);
>>> + job->ret = job->driver->run(job, NULL);
>>> }
>>
>> Hmmm, this breaks the iff relationship with job->error. We might call
>> job_update_rc() afterwards, but then job_completed() would need to free
>> it if it overwrites it with the error description from a potential error
>> object.
>>
>> Also, I suspect the following patches might fix the relationship anyway?
>> (But then an "XXX: This does not hold right now, but will be fixed in a
>> future patch" in the documentation of Job.error might help.)
>>
>> Max
>>
>
> Hmm... does it? ... I guess you mean that we are setting job->ret
> earlier than we used to, which gives us a window where you can have ret
> set, but error unset.
>
> This will get settled out by the end of the series anyway:
Oh no, it appears I accidentally removed yet another chunk from my reply
to patch 2...
> - char *error gets replaced with Error *err,
Which is basically the same. I noted in the deleted chunk that patch 2
just removes the iff relationship from the describing comment, but, well...
> - I remove the error object from job_completed
> - v2 will remove the ret argument, too.
The most important bit of the chunk I removed was that I was complaining
about Job.ret still being there. I don't really see the point of this
patch here at this point.
Unfortunately I can't quite recall...
Having a central Error object doesn't really make sense. Whenever the
block job wants to return an error, it should probably do so as "return"
values of methods (like .run()). Same for ret, of course.
I understand that this is probably really only possible after v2 when
you've made more use of abort/commit. But still, I don't think this
patch improves anything, so I would leave this clean-up until later when
you can really do something.
I suppose the idea here is that you want to drop the errp parameter from
job_completed(), because it is not going to be called by .exit(). But
the obvious way around this would be to pass an errp to .exit() and then
pass the result on to job_completed().
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback
2018-08-25 13:33 ` Max Reitz
@ 2018-08-25 14:15 ` Max Reitz
2018-08-27 16:01 ` John Snow
1 sibling, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-25 14:15 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 2000 bytes --]
On 2018-08-25 15:33, Max Reitz wrote:
[...]
> Having a central Error object doesn't really make sense. Whenever the
> block job wants to return an error, it should probably do so as "return"
> values of methods (like .run()). Same for ret, of course.
>
> I understand that this is probably really only possible after v2 when
> you've made more use of abort/commit. But still, I don't think this
> patch improves anything, so I would leave this clean-up until later when
> you can really do something.
>
> I suppose the idea here is that you want to drop the errp parameter from
> job_completed(), because it is not going to be called by .exit(). But
> the obvious way around this would be to pass an errp to .exit() and then
> pass the result on to job_completed().
You know what, I wrote a really long reply and in the end I realized you
basically did exactly what I wanted. (Or, rather, I gave two
alternatives, and you did one of them.)
I noticed that having a central error object may still make sense;
.create() shows why, jobs usually fail in .run() and then you need to
keep the error around for a bit. You can either pass it around, or you
just keep it in Job (those are the two alternatives I gave).
(So I said, either rip out Job.error/Job.err and Job.ret completely, or
keep the iff relationship between Job.err and Job.ret, so together they
give you a meaningful state.)
Now by setting Job.ret and Job.err basically at the same time (both are
results of .run(), and whenever Job.ret is set somewhere else, it is
immediately followed by job_update_rc()), the iff relationship between
Job.err and Job.ret persists. So that's good!
(In that case I don't know why you removed the "iff" part from the
comment, though.)
The only remaining question I have is why you still pass job->ret to
job_completed(). That seems superfluous to me.
Max
(No, I don't know what's up with me and completely misunderstanding this
series.)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-22 22:05 ` John Snow
@ 2018-08-25 15:02 ` Max Reitz
2018-08-25 15:14 ` Max Reitz
` (2 more replies)
0 siblings, 3 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-25 15:02 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 3375 bytes --]
On 2018-08-23 00:05, John Snow wrote:
>
>
> On 08/22/2018 08:15 AM, Max Reitz wrote:
>> On 2018-08-17 21:04, John Snow wrote:
>>> Change the manual deferment to mirror_exit into the implicit
>>> callback to job_exit and the mirror_exit callback.
>>>
>>> This does change the order of some bdrv_unref calls and job_completed,
>>> but thanks to the new context in which we call .job_exit, this is safe
>>> to defer the possible flushing of any nodes to the job_finalize_single
>>> cleanup stage.
>>
>> Ah, right, I forgot this. Hm, what exactly do you mean? This function
>> is executed in the main loop, so it can make 'src' go away. I don't see
>> any difference to before.
>>
>
> This changes the order in which we unreference these objects; if you
> look at this patch the job_completed call I delete is in the middle of
> what becomes the .exit() callback, which means there is a subtle change
> in the ordering of how references are put down.
>
> Take a look at the weird ordering of mirror_exit as it exists right now;
> we call job_completed first and *then* put down the last references. If
> you re-order this upstream right now, you'll deadlock QEMU because this
> means job_completed is responsible for putting down the last reference
> to some of these block/bds objects.
>
> However, job_completed takes an additional AIO context lock and calls
> job_finalize_single under *two* locks, which will hang QEMU if we
> attempt to flush any of these nodes when we put down the last reference.
If you say so... I have to admit I don't really understand. The
comment doesn't explain why it's so important to keep src around until
job_completed(), so I don't know. I thought AioContexts are recursive
so it doesn't matter whether you take them recursively or not.
Anyway. So the difference now is that job_defer_to_main_loop() took the
lock around the whole exit function, whereas the new exit shim only
takes it around the .exit() method, but calls job_complete() without a
lock -- and then job_finalize_single() gets its lock again, so the job
methods are again called with locks. That sounds OK to me.
> Performing the reordering here is *safe* because by removing the call to
> job_completed and utilizing the exit shim, the .exit() callback executes
> only under one lock, and when the finalize code runs later it is also
> executed under only one lock, making this re-ordering safe.
>
> Clear as mud?
Well, I trust you that the drain issue was the reason that src had to
stay around until after job_completed(). It seems a bit
counter-intuitive, because the comment explaining that src needs to stay
around until job_completed() doesn't say much -- but it does imply that
without that bdrv_ref(), the BDS might be destroyed before
job_completed(). Which is different from simply having only one
reference left and then being deleted in job_completed().
Looking at 3f09bfbc7be, I'm inclined to believe the original reason may
be that src->job points to the job and that we shouldn't delete it as
long as it does (bdrv_delete() asserts that bs->job is NULL). Oh no, a
tangent appears.
...I would assume that when bdrv_replace_node() is called, BlockJob.blk
is updated to point to the new BDS. But nobody seems to update the
BDS.job field. Investigation is in order.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-25 15:02 ` Max Reitz
@ 2018-08-25 15:14 ` Max Reitz
2018-08-28 20:25 ` John Snow
2018-08-28 21:51 ` John Snow
2 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-25 15:14 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
[-- Attachment #1: Type: text/plain, Size: 3840 bytes --]
On 2018-08-25 17:02, Max Reitz wrote:
> On 2018-08-23 00:05, John Snow wrote:
>>
>>
>> On 08/22/2018 08:15 AM, Max Reitz wrote:
>>> On 2018-08-17 21:04, John Snow wrote:
>>>> Change the manual deferment to mirror_exit into the implicit
>>>> callback to job_exit and the mirror_exit callback.
>>>>
>>>> This does change the order of some bdrv_unref calls and job_completed,
>>>> but thanks to the new context in which we call .job_exit, this is safe
>>>> to defer the possible flushing of any nodes to the job_finalize_single
>>>> cleanup stage.
>>>
>>> Ah, right, I forgot this. Hm, what exactly do you mean? This function
>>> is executed in the main loop, so it can make 'src' go away. I don't see
>>> any difference to before.
>>>
>>
>> This changes the order in which we unreference these objects; if you
>> look at this patch the job_completed call I delete is in the middle of
>> what becomes the .exit() callback, which means there is a subtle change
>> in the ordering of how references are put down.
>>
>> Take a look at the weird ordering of mirror_exit as it exists right now;
>> we call job_completed first and *then* put down the last references. If
>> you re-order this upstream right now, you'll deadlock QEMU because this
>> means job_completed is responsible for putting down the last reference
>> to some of these block/bds objects.
>>
>> However, job_completed takes an additional AIO context lock and calls
>> job_finalize_single under *two* locks, which will hang QEMU if we
>> attempt to flush any of these nodes when we put down the last reference.
>
> If you say so... I have to admit I don't really understand. The
> comment doesn't explain why it's so important to keep src around until
> job_completed(), so I don't know. I thought AioContexts are recursive
> so it doesn't matter whether you take them recursively or not.
>
> Anyway. So the difference now is that job_defer_to_main_loop() took the
> lock around the whole exit function, whereas the new exit shim only
> takes it around the .exit() method, but calls job_complete() without a
> lock -- and then job_finalize_single() gets its lock again, so the job
> methods are again called with locks. That sounds OK to me.
>
>> Performing the reordering here is *safe* because by removing the call to
>> job_completed and utilizing the exit shim, the .exit() callback executes
>> only under one lock, and when the finalize code runs later it is also
>> executed under only one lock, making this re-ordering safe.
>>
>> Clear as mud?
>
> Well, I trust you that the drain issue was the reason that src had to
> stay around until after job_completed(). It seems a bit
> counter-intuitive, because the comment explaining that src needs to stay
> around until job_completed() doesn't say much -- but it does imply that
> without that bdrv_ref(), the BDS might be destroyed before
> job_completed(). Which is different from simply having only one
> reference left and then being deleted in job_completed().
>
> Looking at 3f09bfbc7be, I'm inclined to believe the original reason may
> be that src->job points to the job and that we shouldn't delete it as
> long as it does (bdrv_delete() asserts that bs->job is NULL). Oh no, a
> tangent appears.
>
> ...I would assume that when bdrv_replace_node() is called, BlockJob.blk
> is updated to point to the new BDS. But nobody seems to update the
> BDS.job field. Investigation is in order.
Aha! Mirror is run from the mirror filter (of course). So the node
where BDS.job is set is actually not replaced.
Well, it is, but then we specifically switch the job BB back to point to
the mirror filter. As long as it does not go away until then, all is
safe. (And that bdrv_ref() guard doesn't change with this patch.)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-25 13:05 ` Max Reitz
@ 2018-08-27 15:54 ` John Snow
2018-08-29 8:16 ` Max Reitz
0 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-27 15:54 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
On 08/25/2018 09:05 AM, Max Reitz wrote:
> On 2018-08-22 23:52, John Snow wrote:
>>
>>
>> On 08/22/2018 07:43 AM, Max Reitz wrote:
>>> On 2018-08-17 21:04, John Snow wrote:
>>>> All jobs do the same thing when they leave their running loop:
>>>> - Store the return code in a structure
>>>> - wait to receive this structure in the main thread
>>>> - signal job completion via job_completed
>>>>
>>>> Few jobs do anything beyond exactly this. Consolidate this exit
>>>> logic for a net reduction in SLOC.
>>>>
>>>> More seriously, when we utilize job_defer_to_main_loop_bh to call
>>>> a function that calls job_completed, job_finalize_single will run
>>>> in a context where it has recursively taken the aio_context lock,
>>>> which can cause hangs if it puts down a reference that causes a flush.
>>>>
>>>> You can observe this in practice by looking at mirror_exit's careful
>>>> placement of job_completed and bdrv_unref calls.
>>>>
>>>> If we centralize job exiting, we can signal job completion from outside
>>>> of the aio_context, which should allow for job cleanup code to run with
>>>> only one lock, which makes cleanup callbacks less tricky to write.
>>>>
>>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>>> ---
>>>> include/qemu/job.h | 7 +++++++
>>>> job.c | 19 +++++++++++++++++++
>>>> 2 files changed, 26 insertions(+)
>>>
>>> Currently all jobs do this, the question of course is why. The answer
>>> is because they are block jobs that need to do some graph manipulation
>>> in the main thread, right?
>>>
>>
>> Yep.
>>
>>> OK, that's reasonable enough, that sounds like even non-block jobs may
>>> need this (i.e. modify some global qemu state that you can only do in
>>> the main loop). Interestingly, the create job only calls
>>> job_completed() of which it says nowhere that it needs to be executed in
>>> the main loop.
>>>
>>
>> Yeah, not all jobs will have anything meaningful to do in the main loop
>> context. This is one of them.
>>
>>> ...on second thought, do we really want to execute job_complete() in the
>>> main loop? First of all, all of the transactional functions will run in
>>> the main loop. Which makes sense, but it isn't noted anywhere.
>>> Secondly, we may end up calling JobDriver.user_resume(), which is
>>> probably not something we want to call in the main loop.
>>>
>>
>> I think we need to execute job_complete in the main loop, or otherwise
>> restructure the code that can run between job_completed and
>> job_finalize_single so that .prepare/.commit/.abort/.clean run in the
>> main thread, which is something we want to preserve.
>
> Sure.
>
>> It's simpler just to say that complete will run from the main thread,
>> like it does presently.
>
> Yes, but we don't say that.
>
>> Why would we not want to call user_resume from the main loop? That's
>> directly where it's called from, since it gets invoked directly from the
>> qmp thread.
>
> Hmm! True indeed.
>
> The reason why we might not want to do it is because the job may not run
> in the main loop, so modifying the job (especially invoking a job
> method) may be dangerous without taking precautions.
>
>>> OTOH, job_finish_sync() is something that has to be run in the main loop
>>> because it polls the main loop (and as far as my FUSE experiments have
>>> told me, polling a foreign AioContext doesn't work).
>>>
>>> So... I suppose it would be nice if we had a real distinction which
>>> functions are run in which AioContext. It seems like we indeed want to
>>> run job_completed() in the main loop, but what to do about the
>>> user_resume() call in job_cancel_async()?
>>>
>>
>> I don't think we need to do anything -- at least, these functions
>> *already* run from the main loop.
>
> Yeah, but we don't mark that anywhere. I really don't like that. Jobs
> need to know which of their functions are run in which AioContext.
>
>> mirror_exit et al get scheduled from job_defer_to_main_loop and call
>> job_completed there, so it's already always done from the main loop; I'm
>> just cutting out the part where the jobs have to manually schedule this.
>
> I'm not saying what you're doing is wrong, I'm just saying tracking
> which things are running in which context is not easy because there are
> no comments on how it's supposed to be run. (Apart from your new
> .exit() method which does say that it's run in the main loop.)
>
> No, I don't find it obvious which functions are run in which context
> when first I have to think about in which context those functions are
> used (e.g. user_resume is usually the result of a QMP command, so it's
> run in the main loop; the transactional methods are part of completion,
> which is done in the main loop, so they are also called in the main
> loop; and so on).
>
> But that's not part of this series. It just occurred to me when
> tracking down which function belongs to which context when reviewing
> this patch.
>
> Max
>
Oh, I see. I can mark up the functions I/we expect to run in the main
thread with comments above the function implementation, would that help?
Probably also a top level document would also help... We're overdue for
one after all the changes recently.
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback
2018-08-25 13:33 ` Max Reitz
2018-08-25 14:15 ` Max Reitz
@ 2018-08-27 16:01 ` John Snow
1 sibling, 0 replies; 53+ messages in thread
From: John Snow @ 2018-08-27 16:01 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
On 08/25/2018 09:33 AM, Max Reitz wrote:
> On 2018-08-23 01:01, John Snow wrote:
>>
>>
>> On 08/22/2018 06:51 AM, Max Reitz wrote:
>>> On 2018-08-17 21:04, John Snow wrote:
>>>> Presently we codify the entry point for a job as the "start" callback,
>>>> but a more apt name would be "run" to clarify the idea that when this
>>>> function returns we consider the job to have "finished," except for
>>>> any cleanup which occurs in separate callbacks later.
>>>>
>>>> As part of this clarification, change the signature to include an error
>>>> object and a return code. The error ptr is not yet used, and the return
>>>> code while captured, will be overwritten by actions in the job_completed
>>>> function.
>>>>
>>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>>> ---
>>>> block/backup.c | 7 ++++---
>>>> block/commit.c | 7 ++++---
>>>> block/create.c | 8 +++++---
>>>> block/mirror.c | 10 ++++++----
>>>> block/stream.c | 7 ++++---
>>>> include/qemu/job.h | 2 +-
>>>> job.c | 6 +++---
>>>> tests/test-bdrv-drain.c | 7 ++++---
>>>> tests/test-blockjob-txn.c | 16 ++++++++--------
>>>> tests/test-blockjob.c | 7 ++++---
>>>> 10 files changed, 43 insertions(+), 34 deletions(-)
>>>
>>> [...]
>>>
>>>> diff --git a/job.c b/job.c
>>>> index fa671b431a..898260b2b3 100644
>>>> --- a/job.c
>>>> +++ b/job.c
>>>> @@ -544,16 +544,16 @@ static void coroutine_fn job_co_entry(void *opaque)
>>>> {
>>>> Job *job = opaque;
>>>>
>>>> - assert(job && job->driver && job->driver->start);
>>>> + assert(job && job->driver && job->driver->run);
>>>> job_pause_point(job);
>>>> - job->driver->start(job);
>>>> + job->ret = job->driver->run(job, NULL);
>>>> }
>>>
>>> Hmmm, this breaks the iff relationship with job->error. We might call
>>> job_update_rc() afterwards, but then job_completed() would need to free
>>> it if it overwrites it with the error description from a potential error
>>> object.
>>>
>>> Also, I suspect the following patches might fix the relationship anyway?
>>> (But then an "XXX: This does not hold right now, but will be fixed in a
>>> future patch" in the documentation of Job.error might help.)
>>>
>>> Max
>>>
>>
>> Hmm... does it? ... I guess you mean that we are setting job->ret
>> earlier than we used to, which gives us a window where you can have ret
>> set, but error unset.
>>
>> This will get settled out by the end of the series anyway:
>
> Oh no, it appears I accidentally removed yet another chunk from my reply
> to patch 2...
>
>> - char *error gets replaced with Error *err,
>
> Which is basically the same. I noted in the deleted chunk that patch 2
> just removes the iff relationship from the describing comment, but, well...
>
>> - I remove the error object from job_completed
>> - v2 will remove the ret argument, too.
>
> The most important bit of the chunk I removed was that I was complaining
> about Job.ret still being there. I don't really see the point of this
> patch here at this point.
>
> Unfortunately I can't quite recall...
>
> Having a central Error object doesn't really make sense. Whenever the
> block job wants to return an error, it should probably do so as "return"
> values of methods (like .run()). Same for ret, of course.
>
> I understand that this is probably really only possible after v2 when
> you've made more use of abort/commit. But still, I don't think this
> patch improves anything, so I would leave this clean-up until later when
> you can really do something.
>
> I suppose the idea here is that you want to drop the errp parameter from
> job_completed(), because it is not going to be called by .exit(). But
> the obvious way around this would be to pass an errp to .exit() and then
> pass the result on to job_completed().
>
> Max
>
That might be a good idea, but I need to look at the pathway for
actually showing that error to the user, since we need to pass it down a
few times anyway. It's certainly simpler to just stash it in the object.
I'll take a look.
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-25 15:02 ` Max Reitz
2018-08-25 15:14 ` Max Reitz
@ 2018-08-28 20:25 ` John Snow
2018-08-29 8:28 ` Max Reitz
2018-08-28 21:51 ` John Snow
2 siblings, 1 reply; 53+ messages in thread
From: John Snow @ 2018-08-28 20:25 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Fam Zheng
On 08/25/2018 11:02 AM, Max Reitz wrote:
> If you say so... I have to admit I don't really understand. The
> comment doesn't explain why it's so important to keep src around until
> job_completed(), so I don't know. I thought AioContexts are recursive
> so it doesn't matter whether you take them recursively or not.
bdrv_flush has troubles under a recursive lock if it is invoked from a
different thread. It tries to poll for flush completion but the
coroutine which gets scheduled (instead of entered) can't actually run
if we hold the lock twice from, say, the main thread -- which is where
we're doing the graph manipulation from.
> co = qemu_coroutine_create(bdrv_flush_co_entry, &flush_co);
> bdrv_coroutine_enter(bs, co);
> BDRV_POLL_WHILE(bs, flush_co.ret == NOT_DONE);
BDRV_POLL_WHILE there causes us the grief via AIO_WAIT_WHILE, which only
puts down one reference, so we deadlock in bdrv_flush in a recursive
context.
Kevin helped me figure it out; I CC'd him off-list on a mail that you
were also CC'd on ("hang in mirror_exit") that's probably pretty helpful:
> Took a little more than five minutes, but I think I've got it now. The
> important thing is that the test case is for dataplane, i.e. the job
> runs in an I/O thread AioContext. Job completion, however, happens in
> the main loop thread.
>
> Therefore, when bdrv_delete() wants to flush the node first, it needs to
> run bdrv_co_flush() in a foreign AioContext, so the coroutine is
> scheduled. The I/O thread backtrace shows that it's waiting for the
> AioContext lock before it can actually enter the bdrv_co_flush()
> coroutine, so we must have deadlocked:
--js
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-25 15:02 ` Max Reitz
2018-08-25 15:14 ` Max Reitz
2018-08-28 20:25 ` John Snow
@ 2018-08-28 21:51 ` John Snow
2 siblings, 0 replies; 53+ messages in thread
From: John Snow @ 2018-08-28 21:51 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc
On 08/25/2018 11:02 AM, Max Reitz wrote:
> On 2018-08-23 00:05, John Snow wrote:
>>
>>
>> On 08/22/2018 08:15 AM, Max Reitz wrote:
>>> On 2018-08-17 21:04, John Snow wrote:
>>>> Change the manual deferment to mirror_exit into the implicit
>>>> callback to job_exit and the mirror_exit callback.
>>>>
>>>> This does change the order of some bdrv_unref calls and job_completed,
>>>> but thanks to the new context in which we call .job_exit, this is safe
>>>> to defer the possible flushing of any nodes to the job_finalize_single
>>>> cleanup stage.
>>>
>>> Ah, right, I forgot this. Hm, what exactly do you mean? This function
>>> is executed in the main loop, so it can make 'src' go away. I don't see
>>> any difference to before.
>>>
>>
>> This changes the order in which we unreference these objects; if you
>> look at this patch the job_completed call I delete is in the middle of
>> what becomes the .exit() callback, which means there is a subtle change
>> in the ordering of how references are put down.
>>
>> Take a look at the weird ordering of mirror_exit as it exists right now;
>> we call job_completed first and *then* put down the last references. If
>> you re-order this upstream right now, you'll deadlock QEMU because this
>> means job_completed is responsible for putting down the last reference
>> to some of these block/bds objects.
>>
>> However, job_completed takes an additional AIO context lock and calls
>> job_finalize_single under *two* locks, which will hang QEMU if we
>> attempt to flush any of these nodes when we put down the last reference.
>
> If you say so... I have to admit I don't really understand. The
> comment doesn't explain why it's so important to keep src around until
> job_completed(), so I don't know. I thought AioContexts are recursive
> so it doesn't matter whether you take them recursively or not.
>
> Anyway. So the difference now is that job_defer_to_main_loop() took the
> lock around the whole exit function, whereas the new exit shim only
> takes it around the .exit() method, but calls job_complete() without a
> lock -- and then job_finalize_single() gets its lock again, so the job
> methods are again called with locks. That sounds OK to me.
>
>> Performing the reordering here is *safe* because by removing the call to
>> job_completed and utilizing the exit shim, the .exit() callback executes
>> only under one lock, and when the finalize code runs later it is also
>> executed under only one lock, making this re-ordering safe.
>>
>> Clear as mud?
>
> Well, I trust you that the drain issue was the reason that src had to
> stay around until after job_completed(). It seems a bit
> counter-intuitive, because the comment explaining that src needs to stay
> around until job_completed() doesn't say much -- but it does imply that
> without that bdrv_ref(), the BDS might be destroyed before
> job_completed(). Which is different from simply having only one
> reference left and then being deleted in job_completed().
>
> Looking at 3f09bfbc7be, I'm inclined to believe the original reason may
> be that src->job points to the job and that we shouldn't delete it as
> long as it does (bdrv_delete() asserts that bs->job is NULL). Oh no, a
> tangent appears.
>
> ...I would assume that when bdrv_replace_node() is called, BlockJob.blk
> is updated to point to the new BDS. But nobody seems to update the
> BDS.job field. Investigation is in order.
>
> Max
>
The real reason, presently, appears to just be that we want to call
bdrv_drained_end on src, but we may have put down the last reference
during bdrv_replace_node.
I'll update the comment accordingly. Anything beyond that is beyond the
scope of this series.
--js
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] jobs: add exit shim
2018-08-27 15:54 ` John Snow
@ 2018-08-29 8:16 ` Max Reitz
0 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-29 8:16 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: kwolf, Jeff Cody, jtc
[-- Attachment #1: Type: text/plain, Size: 5554 bytes --]
On 2018-08-27 17:54, John Snow wrote:
>
>
> On 08/25/2018 09:05 AM, Max Reitz wrote:
>> On 2018-08-22 23:52, John Snow wrote:
>>>
>>>
>>> On 08/22/2018 07:43 AM, Max Reitz wrote:
>>>> On 2018-08-17 21:04, John Snow wrote:
>>>>> All jobs do the same thing when they leave their running loop:
>>>>> - Store the return code in a structure
>>>>> - wait to receive this structure in the main thread
>>>>> - signal job completion via job_completed
>>>>>
>>>>> Few jobs do anything beyond exactly this. Consolidate this exit
>>>>> logic for a net reduction in SLOC.
>>>>>
>>>>> More seriously, when we utilize job_defer_to_main_loop_bh to call
>>>>> a function that calls job_completed, job_finalize_single will run
>>>>> in a context where it has recursively taken the aio_context lock,
>>>>> which can cause hangs if it puts down a reference that causes a flush.
>>>>>
>>>>> You can observe this in practice by looking at mirror_exit's careful
>>>>> placement of job_completed and bdrv_unref calls.
>>>>>
>>>>> If we centralize job exiting, we can signal job completion from outside
>>>>> of the aio_context, which should allow for job cleanup code to run with
>>>>> only one lock, which makes cleanup callbacks less tricky to write.
>>>>>
>>>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>>>> ---
>>>>> include/qemu/job.h | 7 +++++++
>>>>> job.c | 19 +++++++++++++++++++
>>>>> 2 files changed, 26 insertions(+)
>>>>
>>>> Currently all jobs do this, the question of course is why. The answer
>>>> is because they are block jobs that need to do some graph manipulation
>>>> in the main thread, right?
>>>>
>>>
>>> Yep.
>>>
>>>> OK, that's reasonable enough, that sounds like even non-block jobs may
>>>> need this (i.e. modify some global qemu state that you can only do in
>>>> the main loop). Interestingly, the create job only calls
>>>> job_completed() of which it says nowhere that it needs to be executed in
>>>> the main loop.
>>>>
>>>
>>> Yeah, not all jobs will have anything meaningful to do in the main loop
>>> context. This is one of them.
>>>
>>>> ...on second thought, do we really want to execute job_complete() in the
>>>> main loop? First of all, all of the transactional functions will run in
>>>> the main loop. Which makes sense, but it isn't noted anywhere.
>>>> Secondly, we may end up calling JobDriver.user_resume(), which is
>>>> probably not something we want to call in the main loop.
>>>>
>>>
>>> I think we need to execute job_complete in the main loop, or otherwise
>>> restructure the code that can run between job_completed and
>>> job_finalize_single so that .prepare/.commit/.abort/.clean run in the
>>> main thread, which is something we want to preserve.
>>
>> Sure.
>>
>>> It's simpler just to say that complete will run from the main thread,
>>> like it does presently.
>>
>> Yes, but we don't say that.
>>
>>> Why would we not want to call user_resume from the main loop? That's
>>> directly where it's called from, since it gets invoked directly from the
>>> qmp thread.
>>
>> Hmm! True indeed.
>>
>> The reason why we might not want to do it is because the job may not run
>> in the main loop, so modifying the job (especially invoking a job
>> method) may be dangerous without taking precautions.
>>
>>>> OTOH, job_finish_sync() is something that has to be run in the main loop
>>>> because it polls the main loop (and as far as my FUSE experiments have
>>>> told me, polling a foreign AioContext doesn't work).
>>>>
>>>> So... I suppose it would be nice if we had a real distinction which
>>>> functions are run in which AioContext. It seems like we indeed want to
>>>> run job_completed() in the main loop, but what to do about the
>>>> user_resume() call in job_cancel_async()?
>>>>
>>>
>>> I don't think we need to do anything -- at least, these functions
>>> *already* run from the main loop.
>>
>> Yeah, but we don't mark that anywhere. I really don't like that. Jobs
>> need to know which of their functions are run in which AioContext.
>>
>>> mirror_exit et al get scheduled from job_defer_to_main_loop and call
>>> job_completed there, so it's already always done from the main loop; I'm
>>> just cutting out the part where the jobs have to manually schedule this.
>>
>> I'm not saying what you're doing is wrong, I'm just saying tracking
>> which things are running in which context is not easy because there are
>> no comments on how it's supposed to be run. (Apart from your new
>> .exit() method which does say that it's run in the main loop.)
>>
>> No, I don't find it obvious which functions are run in which context
>> when first I have to think about in which context those functions are
>> used (e.g. user_resume is usually the result of a QMP command, so it's
>> run in the main loop; the transactional methods are part of completion,
>> which is done in the main loop, so they are also called in the main
>> loop; and so on).
>>
>> But that's not part of this series. It just occurred to me when
>> tracking down which function belongs to which context when reviewing
>> this patch.
>>
>> Max
>>
>
> Oh, I see. I can mark up the functions I/we expect to run in the main
> thread with comments above the function implementation, would that help?
Sure, that's exactly what I mean. :-)
> Probably also a top level document would also help... We're overdue for
> one after all the changes recently.
If you have the time, sure.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 5/7] block/mirror: utilize job_exit shim
2018-08-28 20:25 ` John Snow
@ 2018-08-29 8:28 ` Max Reitz
0 siblings, 0 replies; 53+ messages in thread
From: Max Reitz @ 2018-08-29 8:28 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Jeff Cody, kwolf, jtc, Fam Zheng
[-- Attachment #1: Type: text/plain, Size: 2321 bytes --]
On 2018-08-28 22:25, John Snow wrote:
>
>
> On 08/25/2018 11:02 AM, Max Reitz wrote:
>> If you say so... I have to admit I don't really understand. The
>> comment doesn't explain why it's so important to keep src around until
>> job_completed(), so I don't know. I thought AioContexts are recursive
>> so it doesn't matter whether you take them recursively or not.
>
> bdrv_flush has troubles under a recursive lock if it is invoked from a
> different thread. It tries to poll for flush completion but the
> coroutine which gets scheduled (instead of entered) can't actually run
> if we hold the lock twice from, say, the main thread -- which is where
> we're doing the graph manipulation from.
>
>> co = qemu_coroutine_create(bdrv_flush_co_entry, &flush_co);
>> bdrv_coroutine_enter(bs, co);
>> BDRV_POLL_WHILE(bs, flush_co.ret == NOT_DONE);
>
> BDRV_POLL_WHILE there causes us the grief via AIO_WAIT_WHILE, which only
> puts down one reference, so we deadlock in bdrv_flush in a recursive
> context.
>
> Kevin helped me figure it out; I CC'd him off-list on a mail that you
> were also CC'd on ("hang in mirror_exit") that's probably pretty helpful:
>
>> Took a little more than five minutes, but I think I've got it now. The
>> important thing is that the test case is for dataplane, i.e. the job
>> runs in an I/O thread AioContext. Job completion, however, happens in
>> the main loop thread.
>>
>> Therefore, when bdrv_delete() wants to flush the node first, it needs to
>> run bdrv_co_flush() in a foreign AioContext, so the coroutine is
>> scheduled. The I/O thread backtrace shows that it's waiting for the
>> AioContext lock before it can actually enter the bdrv_co_flush()
>> coroutine, so we must have deadlocked:
OK, that makes more sense. I still would have thought that you should
always be allowed to take an AioContext lock more than once in a single
other AioContext, but on my way through the commit log, I found
bd6458e410c1e7, d79df2a2ceb3cb07711, and maybe most importantly
17e2a4a47d4. So maybe not.
So at some point we decided that, yeah, you can take them multiple times
in a single context, and, yeah, that was how it was designed, but don't
do that if you expect a BDRV_POLL_WHILE().
OK. Got it now.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
` (8 preceding siblings ...)
2018-08-18 16:31 ` no-reply
@ 2018-09-04 2:06 ` no-reply
2018-09-04 2:09 ` no-reply
10 siblings, 0 replies; 53+ messages in thread
From: no-reply @ 2018-09-04 2:06 UTC (permalink / raw)
To: jsnow; +Cc: famz, qemu-block, qemu-devel, kwolf, jcody, jtc, mreitz
Hi,
This series failed docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 20180817190457.8292-1-jsnow@redhat.com
Subject: [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-mingw@fedora SHOW_ENV=1 J=8
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
8a7a269a6d jobs: remove job_defer_to_main_loop
f0b09a2775 jobs: utilize job_exit shim
c7e2df3361 block/mirror: utilize job_exit shim
d45d9fd099 block/commit: utilize job_exit shim
d5b28e614b jobs: add exit shim
6fb5dac7b5 jobs: canonize Error object
c6bee15d43 jobs: change start callback to run callback
=== OUTPUT BEGIN ===
BUILD fedora
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-n53ljno8/src'
GEN /var/tmp/patchew-tester-tmp-n53ljno8/src/docker-src.2018-09-03-22.04.33.30916/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-n53ljno8/src/docker-src.2018-09-03-22.04.33.30916/qemu.tar.vroot'...
done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-n53ljno8/src/docker-src.2018-09-03-22.04.33.30916/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-n53ljno8/src/docker-src.2018-09-03-22.04.33.30916/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
COPY RUNNER
RUN test-mingw in qemu:fedora
Packages installed:
SDL2-devel-2.0.8-5.fc28.x86_64
bc-1.07.1-5.fc28.x86_64
bison-3.0.4-9.fc28.x86_64
bluez-libs-devel-5.49-3.fc28.x86_64
brlapi-devel-0.6.7-12.fc28.x86_64
bzip2-1.0.6-26.fc28.x86_64
bzip2-devel-1.0.6-26.fc28.x86_64
ccache-3.4.2-2.fc28.x86_64
clang-6.0.0-5.fc28.x86_64
device-mapper-multipath-devel-0.7.4-2.git07e7bd5.fc28.x86_64
findutils-4.6.0-19.fc28.x86_64
flex-2.6.1-7.fc28.x86_64
gcc-8.1.1-1.fc28.x86_64
gcc-c++-8.1.1-1.fc28.x86_64
gettext-0.19.8.1-14.fc28.x86_64
git-2.17.1-2.fc28.x86_64
glib2-devel-2.56.1-3.fc28.x86_64
glusterfs-api-devel-4.0.2-1.fc28.x86_64
gnutls-devel-3.6.2-1.fc28.x86_64
gtk3-devel-3.22.30-1.fc28.x86_64
hostname-3.20-3.fc28.x86_64
libaio-devel-0.3.110-11.fc28.x86_64
libasan-8.1.1-1.fc28.x86_64
libattr-devel-2.4.47-23.fc28.x86_64
libcap-devel-2.25-9.fc28.x86_64
libcap-ng-devel-0.7.9-1.fc28.x86_64
libcurl-devel-7.59.0-3.fc28.x86_64
libfdt-devel-1.4.6-4.fc28.x86_64
libpng-devel-1.6.34-3.fc28.x86_64
librbd-devel-12.2.5-1.fc28.x86_64
libssh2-devel-1.8.0-7.fc28.x86_64
libubsan-8.1.1-1.fc28.x86_64
libusbx-devel-1.0.21-6.fc28.x86_64
libxml2-devel-2.9.7-4.fc28.x86_64
llvm-6.0.0-11.fc28.x86_64
lzo-devel-2.08-12.fc28.x86_64
make-4.2.1-6.fc28.x86_64
mingw32-SDL2-2.0.5-3.fc27.noarch
mingw32-bzip2-1.0.6-9.fc27.noarch
mingw32-curl-7.57.0-1.fc28.noarch
mingw32-glib2-2.54.1-1.fc28.noarch
mingw32-gmp-6.1.2-2.fc27.noarch
mingw32-gnutls-3.5.13-2.fc27.noarch
mingw32-gtk3-3.22.16-1.fc27.noarch
mingw32-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw32-libpng-1.6.29-2.fc27.noarch
mingw32-libssh2-1.8.0-3.fc27.noarch
mingw32-libtasn1-4.13-1.fc28.noarch
mingw32-nettle-3.3-3.fc27.noarch
mingw32-pixman-0.34.0-3.fc27.noarch
mingw32-pkg-config-0.28-9.fc27.x86_64
mingw64-SDL2-2.0.5-3.fc27.noarch
mingw64-bzip2-1.0.6-9.fc27.noarch
mingw64-curl-7.57.0-1.fc28.noarch
mingw64-glib2-2.54.1-1.fc28.noarch
mingw64-gmp-6.1.2-2.fc27.noarch
mingw64-gnutls-3.5.13-2.fc27.noarch
mingw64-gtk3-3.22.16-1.fc27.noarch
mingw64-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw64-libpng-1.6.29-2.fc27.noarch
mingw64-libssh2-1.8.0-3.fc27.noarch
mingw64-libtasn1-4.13-1.fc28.noarch
mingw64-nettle-3.3-3.fc27.noarch
mingw64-pixman-0.34.0-3.fc27.noarch
mingw64-pkg-config-0.28-9.fc27.x86_64
ncurses-devel-6.1-5.20180224.fc28.x86_64
nettle-devel-3.4-2.fc28.x86_64
nss-devel-3.36.1-1.1.fc28.x86_64
numactl-devel-2.0.11-8.fc28.x86_64
package PyYAML is not installed
package libjpeg-devel is not installed
perl-5.26.2-411.fc28.x86_64
pixman-devel-0.34.0-8.fc28.x86_64
python3-3.6.5-1.fc28.x86_64
snappy-devel-1.1.7-5.fc28.x86_64
sparse-0.5.2-1.fc28.x86_64
spice-server-devel-0.14.0-4.fc28.x86_64
systemtap-sdt-devel-3.2-11.fc28.x86_64
tar-1.30-3.fc28.x86_64
usbredir-devel-0.7.1-7.fc28.x86_64
virglrenderer-devel-0.6.0-4.20170210git76b3da97b.fc28.x86_64
vte3-devel-0.36.5-6.fc28.x86_64
which-2.21-8.fc28.x86_64
xen-devel-4.10.1-3.fc28.x86_64
zlib-devel-1.2.11-8.fc28.x86_64
Environment variables:
TARGET_LIST=
PACKAGES=ccache gettext git tar PyYAML sparse flex bison python3 bzip2 hostname gcc gcc-c++ llvm clang make perl which bc findutils glib2-devel libaio-devel pixman-devel zlib-devel libfdt-devel libasan libubsan bluez-libs-devel brlapi-devel bzip2-devel device-mapper-multipath-devel glusterfs-api-devel gnutls-devel gtk3-devel libattr-devel libcap-devel libcap-ng-devel libcurl-devel libjpeg-devel libpng-devel librbd-devel libssh2-devel libusbx-devel libxml2-devel lzo-devel ncurses-devel nettle-devel nss-devel numactl-devel SDL2-devel snappy-devel spice-server-devel systemtap-sdt-devel usbredir-devel virglrenderer-devel vte3-devel xen-devel mingw32-pixman mingw32-glib2 mingw32-gmp mingw32-SDL2 mingw32-pkg-config mingw32-gtk3 mingw32-gnutls mingw32-nettle mingw32-libtasn1 mingw32-libjpeg-turbo mingw32-libpng mingw32-curl mingw32-libssh2 mingw32-bzip2 mingw64-pixman mingw64-glib2 mingw64-gmp mingw64-SDL2 mingw64-pkg-config mingw64-gtk3 mingw64-gnutls mingw64-nettle mingw64-libtasn1 mingw64-libjpeg-turbo mingw64-libpng mingw64-curl mingw64-libssh2 mingw64-bzip2
J=8
V=
HOSTNAME=e9c32657ad4d
DEBUG=
SHOW_ENV=1
PWD=/
HOME=/home/patchew
CCACHE_DIR=/var/tmp/ccache
DISTTAG=f28container
QEMU_CONFIGURE_OPTS=--python=/usr/bin/python3
FGC=f28
TEST_DIR=/tmp/qemu-test
SHLVL=1
FEATURES=mingw clang pyyaml asan dtc
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAKEFLAGS= -j8
EXTRA_CONFIGURE_OPTS=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=x86_64-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=2.0 --with-gtkabi=3.0
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install
local state directory queried at runtime
Windows SDK no
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules
C compiler x86_64-w64-mingw32-gcc
Host C compiler cc
C++ compiler x86_64-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -mms-bitfields -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/x86_64-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -m64 -mcx16 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/libpng16
LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g
QEMU_LDFLAGS -L$(BUILD_DIR)/dtc/libfdt
make make
install install
python /usr/bin/python3 -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (2.0.5)
GTK support yes (3.22.16)
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support yes
GNUTLS rnd yes
libgcrypt no
libgcrypt kdf no
nettle yes (3.3)
nettle kdf yes
libtasn1 yes
curses support no
virgl support no
curl support yes
mingw32 support yes
Audio drivers dsound
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support yes
VNC PNG support yes
xen support no
brlapi support no
bluez support no
Documentation no
PIE no
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support no
Install blobs yes
KVM support no
HAX support yes
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support no
RDMA support no
PVRDMA support no
fdt support git
membarrier no
preadv support no
fdatasync no
madvise no
posix_madvise no
posix_memalign no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends simple
Trace output file trace-<pid>
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info yes
QGA MSI support no
seccomp support no
coroutine backend win32
coroutine pool yes
debug stack usage no
mutex debugging no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support yes
TPM passthrough no
TPM emulator no
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone no
docker no
NOTE: cross-compilers enabled: 'x86_64-w64-mingw32-gcc'
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GEN qemu-options.def
GEN qapi-gen
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN x86_64-softmmu/config-devices.mak
GEN aarch64-softmmu/config-devices.mak
GEN ui/input-keymap-atset1-to-qcode.c
GEN ui/input-keymap-linux-to-qcode.c
GEN ui/input-keymap-qcode-to-atset1.c
GEN ui/input-keymap-qcode-to-atset2.c
GEN ui/input-keymap-qcode-to-atset3.c
GEN ui/input-keymap-qcode-to-linux.c
GEN ui/input-keymap-qcode-to-qnum.c
GEN ui/input-keymap-qcode-to-sun.c
GEN ui/input-keymap-qnum-to-qcode.c
GEN ui/input-keymap-usb-to-qcode.c
GEN ui/input-keymap-x11-to-qcode.c
GEN ui/input-keymap-win32-to-qcode.c
GEN ui/input-keymap-xorgevdev-to-qcode.c
GEN ui/input-keymap-xorgkbd-to-qcode.c
GEN ui/input-keymap-xorgxquartz-to-qcode.c
GEN ui/input-keymap-xorgxwin-to-qcode.c
GEN ui/input-keymap-osx-to-qcode.c
GEN tests/test-qapi-gen
GEN trace-root.h
GEN accel/kvm/trace.h
GEN accel/tcg/trace.h
GEN audio/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN crypto/trace.h
GEN hw/9pfs/trace.h
GEN hw/acpi/trace.h
GEN hw/alpha/trace.h
GEN hw/arm/trace.h
GEN hw/audio/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/display/trace.h
GEN hw/dma/trace.h
GEN hw/hppa/trace.h
GEN hw/i2c/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/ide/trace.h
GEN hw/input/trace.h
GEN hw/intc/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/net/trace.h
GEN hw/nvram/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/ppc/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/s390x/trace.h
GEN hw/scsi/trace.h
GEN hw/sd/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/timer/trace.h
GEN hw/tpm/trace.h
GEN hw/usb/trace.h
GEN hw/vfio/trace.h
GEN hw/virtio/trace.h
GEN hw/watchdog/trace.h
GEN hw/xen/trace.h
GEN io/trace.h
GEN linux-user/trace.h
GEN migration/trace.h
GEN nbd/trace.h
GEN net/trace.h
GEN qapi/trace.h
GEN qom/trace.h
GEN scsi/trace.h
GEN target/arm/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/ppc/trace.h
GEN target/s390x/trace.h
GEN target/sparc/trace.h
GEN ui/trace.h
GEN util/trace.h
GEN trace-root.c
GEN accel/kvm/trace.c
GEN accel/tcg/trace.c
GEN audio/trace.c
GEN block/trace.c
GEN chardev/trace.c
GEN crypto/trace.c
GEN hw/9pfs/trace.c
GEN hw/acpi/trace.c
GEN hw/alpha/trace.c
GEN hw/arm/trace.c
GEN hw/audio/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/display/trace.c
GEN hw/dma/trace.c
GEN hw/hppa/trace.c
GEN hw/i2c/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/ide/trace.c
GEN hw/input/trace.c
GEN hw/intc/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/net/trace.c
GEN hw/nvram/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/ppc/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/s390x/trace.c
GEN hw/scsi/trace.c
GEN hw/sd/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/timer/trace.c
GEN hw/tpm/trace.c
GEN hw/usb/trace.c
GEN hw/vfio/trace.c
GEN hw/virtio/trace.c
GEN hw/watchdog/trace.c
GEN hw/xen/trace.c
GEN io/trace.c
GEN linux-user/trace.c
GEN migration/trace.c
GEN nbd/trace.c
GEN net/trace.c
GEN qapi/trace.c
GEN qom/trace.c
GEN scsi/trace.c
GEN target/arm/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/ppc/trace.c
GEN target/s390x/trace.c
GEN target/sparc/trace.c
GEN ui/trace.c
GEN util/trace.c
GEN config-all-devices.mak
DEP /tmp/qemu-test/src/dtc/tests/dumptrees.c
DEP /tmp/qemu-test/src/dtc/tests/trees.S
DEP /tmp/qemu-test/src/dtc/tests/testutils.c
DEP /tmp/qemu-test/src/dtc/tests/value-labels.c
DEP /tmp/qemu-test/src/dtc/tests/asm_tree_dump.c
DEP /tmp/qemu-test/src/dtc/tests/truncated_property.c
DEP /tmp/qemu-test/src/dtc/tests/check_path.c
DEP /tmp/qemu-test/src/dtc/tests/overlay_bad_fixup.c
DEP /tmp/qemu-test/src/dtc/tests/overlay.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/property_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/integer-expressions.c
DEP /tmp/qemu-test/src/dtc/tests/utilfdt_test.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset_aliases.c
DEP /tmp/qemu-test/src/dtc/tests/add_subnode_with_nops.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_unordered.c
DEP /tmp/qemu-test/src/dtc/tests/dtb_reverse.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_ordered.c
DEP /tmp/qemu-test/src/dtc/tests/extra-terminating-null.c
DEP /tmp/qemu-test/src/dtc/tests/incbin.c
DEP /tmp/qemu-test/src/dtc/tests/boot-cpuid.c
DEP /tmp/qemu-test/src/dtc/tests/phandle_format.c
DEP /tmp/qemu-test/src/dtc/tests/path-references.c
DEP /tmp/qemu-test/src/dtc/tests/references.c
DEP /tmp/qemu-test/src/dtc/tests/string_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/propname_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop2.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop1.c
DEP /tmp/qemu-test/src/dtc/tests/del_node.c
DEP /tmp/qemu-test/src/dtc/tests/del_property.c
DEP /tmp/qemu-test/src/dtc/tests/setprop.c
DEP /tmp/qemu-test/src/dtc/tests/set_name.c
DEP /tmp/qemu-test/src/dtc/tests/rw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/open_pack.c
DEP /tmp/qemu-test/src/dtc/tests/nopulate.c
DEP /tmp/qemu-test/src/dtc/tests/mangle-layout.c
DEP /tmp/qemu-test/src/dtc/tests/sw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/move_and_save.c
DEP /tmp/qemu-test/src/dtc/tests/nop_node.c
DEP /tmp/qemu-test/src/dtc/tests/nop_property.c
DEP /tmp/qemu-test/src/dtc/tests/setprop_inplace.c
DEP /tmp/qemu-test/src/dtc/tests/stringlist.c
DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells.c
DEP /tmp/qemu-test/src/dtc/tests/notfound.c
DEP /tmp/qemu-test/src/dtc/tests/sized_cells.c
DEP /tmp/qemu-test/src/dtc/tests/char_literal.c
DEP /tmp/qemu-test/src/dtc/tests/get_alias.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_check_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_prop_value.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/parent_offset.c
DEP /tmp/qemu-test/src/dtc/tests/get_path.c
DEP /tmp/qemu-test/src/dtc/tests/supernode_atdepth_offset.c
DEP /tmp/qemu-test/src/dtc/tests/get_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/getprop.c
DEP /tmp/qemu-test/src/dtc/tests/get_name.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset.c
DEP /tmp/qemu-test/src/dtc/tests/find_property.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_offset.c
DEP /tmp/qemu-test/src/dtc/tests/root_node.c
DEP /tmp/qemu-test/src/dtc/tests/get_mem_rsv.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_overlay.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_addresses.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_empty_tree.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_strerror.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_rw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_sw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_wip.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_ro.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt.c
DEP /tmp/qemu-test/src/dtc/util.c
DEP /tmp/qemu-test/src/dtc/fdtoverlay.c
DEP /tmp/qemu-test/src/dtc/fdtput.c
DEP /tmp/qemu-test/src/dtc/fdtget.c
DEP /tmp/qemu-test/src/dtc/fdtdump.c
LEX convert-dtsv0-lexer.lex.c
BISON dtc-parser.tab.c
DEP /tmp/qemu-test/src/dtc/srcpos.c
LEX dtc-lexer.lex.c
DEP /tmp/qemu-test/src/dtc/treesource.c
DEP /tmp/qemu-test/src/dtc/livetree.c
DEP /tmp/qemu-test/src/dtc/fstree.c
DEP /tmp/qemu-test/src/dtc/flattree.c
DEP /tmp/qemu-test/src/dtc/dtc.c
DEP /tmp/qemu-test/src/dtc/data.c
DEP /tmp/qemu-test/src/dtc/checks.c
DEP convert-dtsv0-lexer.lex.c
DEP dtc-parser.tab.c
DEP dtc-lexer.lex.c
CHK version_gen.h
UPD version_gen.h
DEP /tmp/qemu-test/src/dtc/util.c
CC libfdt/fdt.o
CC libfdt/fdt_ro.o
CC libfdt/fdt_rw.o
CC libfdt/fdt_wip.o
CC libfdt/fdt_strerror.o
CC libfdt/fdt_sw.o
CC libfdt/fdt_empty_tree.o
CC libfdt/fdt_addresses.o
CC libfdt/fdt_overlay.o
AR libfdt/libfdt.a
x86_64-w64-mingw32-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
RC version.o
GEN qga/qapi-generated/qapi-gen
CC qapi/qapi-types.o
CC qapi/qapi-types-block-core.o
CC qapi/qapi-types-char.o
CC qapi/qapi-types-common.o
CC qapi/qapi-types-block.o
CC qapi/qapi-builtin-types.o
CC qapi/qapi-types-crypto.o
CC qapi/qapi-types-introspect.o
CC qapi/qapi-types-job.o
CC qapi/qapi-types-migration.o
CC qapi/qapi-types-misc.o
CC qapi/qapi-types-net.o
CC qapi/qapi-types-rocker.o
CC qapi/qapi-types-run-state.o
CC qapi/qapi-types-sockets.o
CC qapi/qapi-types-tpm.o
CC qapi/qapi-types-trace.o
CC qapi/qapi-types-transaction.o
CC qapi/qapi-types-ui.o
CC qapi/qapi-builtin-visit.o
CC qapi/qapi-visit.o
CC qapi/qapi-visit-block-core.o
CC qapi/qapi-visit-block.o
CC qapi/qapi-visit-char.o
CC qapi/qapi-visit-common.o
CC qapi/qapi-visit-crypto.o
CC qapi/qapi-visit-introspect.o
CC qapi/qapi-visit-migration.o
CC qapi/qapi-visit-job.o
CC qapi/qapi-visit-misc.o
CC qapi/qapi-visit-net.o
CC qapi/qapi-visit-rocker.o
CC qapi/qapi-visit-run-state.o
CC qapi/qapi-visit-sockets.o
CC qapi/qapi-visit-tpm.o
CC qapi/qapi-visit-trace.o
CC qapi/qapi-visit-transaction.o
CC qapi/qapi-visit-ui.o
CC qapi/qapi-events.o
CC qapi/qapi-events-block-core.o
CC qapi/qapi-events-block.o
CC qapi/qapi-events-char.o
CC qapi/qapi-events-common.o
CC qapi/qapi-events-crypto.o
CC qapi/qapi-events-introspect.o
CC qapi/qapi-events-job.o
CC qapi/qapi-events-migration.o
CC qapi/qapi-events-misc.o
CC qapi/qapi-events-net.o
CC qapi/qapi-events-rocker.o
CC qapi/qapi-events-run-state.o
CC qapi/qapi-events-sockets.o
CC qapi/qapi-events-tpm.o
CC qapi/qapi-events-trace.o
CC qapi/qapi-events-transaction.o
CC qapi/qapi-events-ui.o
CC qapi/qapi-introspect.o
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qnum.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qbool.o
CC qobject/qlit.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
CC qobject/block-qdict.o
CC trace/simple.o
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/lockcnt.o
CC util/aiocb.o
CC util/async.o
CC util/aio-wait.o
CC util/thread-pool.o
CC util/qemu-timer.o
CC util/main-loop.o
CC util/iohandler.o
CC util/aio-win32.o
CC util/event_notifier-win32.o
CC util/oslib-win32.o
CC util/qemu-thread-win32.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/host-utils.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/fifo8.o
CC util/acl.o
CC util/cacheinfo.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/keyval.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/readline.o
CC util/rcu.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-win32.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/pagesize.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC util/stats64.o
CC util/systemd.o
CC util/iova-tree.o
CC trace-root.o
CC accel/kvm/trace.o
CC accel/tcg/trace.o
CC audio/trace.o
CC block/trace.o
CC chardev/trace.o
CC crypto/trace.o
CC hw/9pfs/trace.o
CC hw/acpi/trace.o
CC hw/arm/trace.o
CC hw/alpha/trace.o
CC hw/audio/trace.o
CC hw/block/trace.o
CC hw/block/dataplane/trace.o
CC hw/char/trace.o
CC hw/display/trace.o
CC hw/dma/trace.o
CC hw/hppa/trace.o
CC hw/i2c/trace.o
CC hw/i386/xen/trace.o
CC hw/i386/trace.o
CC hw/ide/trace.o
CC hw/input/trace.o
CC hw/intc/trace.o
CC hw/isa/trace.o
CC hw/mem/trace.o
CC hw/misc/trace.o
CC hw/misc/macio/trace.o
CC hw/net/trace.o
CC hw/nvram/trace.o
CC hw/pci/trace.o
CC hw/pci-host/trace.o
CC hw/ppc/trace.o
CC hw/rdma/trace.o
CC hw/rdma/vmw/trace.o
CC hw/s390x/trace.o
CC hw/scsi/trace.o
CC hw/sd/trace.o
CC hw/sparc/trace.o
CC hw/timer/trace.o
CC hw/sparc64/trace.o
CC hw/tpm/trace.o
CC hw/usb/trace.o
CC hw/vfio/trace.o
CC hw/virtio/trace.o
CC hw/watchdog/trace.o
CC hw/xen/trace.o
CC io/trace.o
CC linux-user/trace.o
CC migration/trace.o
CC nbd/trace.o
CC net/trace.o
CC qapi/trace.o
CC qom/trace.o
CC scsi/trace.o
CC target/arm/trace.o
CC target/i386/trace.o
CC target/mips/trace.o
CC target/ppc/trace.o
CC target/s390x/trace.o
CC target/sparc/trace.o
CC ui/trace.o
CC util/trace.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-clock.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/error-printf.o
CC stubs/fdset.o
CC stubs/gdbstub.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/iothread-lock.o
CC stubs/is-daemonized.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/change-state-handler.o
CC stubs/monitor.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/tpm.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/fd-register.o
CC stubs/qmp_memory_device.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/vmgenid.o
CC stubs/xen-common.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
CC stubs/ram-block.o
GEN qemu-img-cmds.h
CC block.o
CC blockjob.o
CC job.o
CC qemu-io-cmds.o
CC replication.o
CC block/raw-format.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qcow2-bitmap.o
CC block/qed.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/vhdx-log.o
CC block/quorum.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/blklogwrites.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/file-win32.o
CC block/win32-aio.o
CC block/mirror.o
CC block/null.o
CC block/commit.o
CC block/io.o
CC block/create.o
CC block/throttle-groups.o
CC block/nbd.o
CC block/nbd-client.o
/tmp/qemu-test/src/block/commit.c: In function 'commit_exit':
/tmp/qemu-test/src/block/commit.c:88:35: error: 'ret' undeclared (first use in this function); did you mean 'recv'?
if (!job_is_cancelled(job) && ret == 0) {
^~~
recv
/tmp/qemu-test/src/block/commit.c:88:35: note: each undeclared identifier is reported only once for each function it appears in
make: *** [/tmp/qemu-test/src/rules.mak:69: block/commit.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 565, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 562, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 308, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 276, in run
quiet=quiet)
File "./tests/docker/docker.py", line 183, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=e471cf0eafe611e8b5aa52540069c830', '-u', '1000', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-n53ljno8/src/docker-src.2018-09-03-22.04.33.30916:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:216: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-n53ljno8/src'
make: *** [tests/docker/Makefile.include:250: docker-run-test-mingw@fedora] Error 2
real 2m7.231s
user 0m4.692s
sys 0m3.681s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
` (9 preceding siblings ...)
2018-09-04 2:06 ` no-reply
@ 2018-09-04 2:09 ` no-reply
10 siblings, 0 replies; 53+ messages in thread
From: no-reply @ 2018-09-04 2:09 UTC (permalink / raw)
To: jsnow; +Cc: famz, qemu-block, qemu-devel, kwolf, jcody, jtc, mreitz
Hi,
This series failed docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 20180817190457.8292-1-jsnow@redhat.com
Subject: [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop
=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-quick@centos7 SHOW_ENV=1 J=8
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
8a7a269a6d jobs: remove job_defer_to_main_loop
f0b09a2775 jobs: utilize job_exit shim
c7e2df3361 block/mirror: utilize job_exit shim
d45d9fd099 block/commit: utilize job_exit shim
d5b28e614b jobs: add exit shim
6fb5dac7b5 jobs: canonize Error object
c6bee15d43 jobs: change start callback to run callback
=== OUTPUT BEGIN ===
BUILD centos7
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-jxjt0g36/src'
GEN /var/tmp/patchew-tester-tmp-jxjt0g36/src/docker-src.2018-09-03-22.07.23.2969/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-jxjt0g36/src/docker-src.2018-09-03-22.07.23.2969/qemu.tar.vroot'...
done.
Checking out files: 45% (2903/6332)
Checking out files: 46% (2913/6332)
Checking out files: 47% (2977/6332)
Checking out files: 48% (3040/6332)
Checking out files: 49% (3103/6332)
Checking out files: 50% (3166/6332)
Checking out files: 51% (3230/6332)
Checking out files: 52% (3293/6332)
Checking out files: 53% (3356/6332)
Checking out files: 54% (3420/6332)
Checking out files: 55% (3483/6332)
Checking out files: 56% (3546/6332)
Checking out files: 57% (3610/6332)
Checking out files: 58% (3673/6332)
Checking out files: 59% (3736/6332)
Checking out files: 60% (3800/6332)
Checking out files: 61% (3863/6332)
Checking out files: 62% (3926/6332)
Checking out files: 63% (3990/6332)
Checking out files: 64% (4053/6332)
Checking out files: 65% (4116/6332)
Checking out files: 66% (4180/6332)
Checking out files: 67% (4243/6332)
Checking out files: 68% (4306/6332)
Checking out files: 69% (4370/6332)
Checking out files: 70% (4433/6332)
Checking out files: 71% (4496/6332)
Checking out files: 72% (4560/6332)
Checking out files: 73% (4623/6332)
Checking out files: 74% (4686/6332)
Checking out files: 75% (4749/6332)
Checking out files: 76% (4813/6332)
Checking out files: 77% (4876/6332)
Checking out files: 78% (4939/6332)
Checking out files: 79% (5003/6332)
Checking out files: 80% (5066/6332)
Checking out files: 81% (5129/6332)
Checking out files: 82% (5193/6332)
Checking out files: 82% (5210/6332)
Checking out files: 83% (5256/6332)
Checking out files: 84% (5319/6332)
Checking out files: 85% (5383/6332)
Checking out files: 86% (5446/6332)
Checking out files: 87% (5509/6332)
Checking out files: 88% (5573/6332)
Checking out files: 89% (5636/6332)
Checking out files: 90% (5699/6332)
Checking out files: 91% (5763/6332)
Checking out files: 92% (5826/6332)
Checking out files: 93% (5889/6332)
Checking out files: 94% (5953/6332)
Checking out files: 95% (6016/6332)
Checking out files: 96% (6079/6332)
Checking out files: 97% (6143/6332)
Checking out files: 98% (6206/6332)
Checking out files: 99% (6269/6332)
Checking out files: 100% (6332/6332)
Checking out files: 100% (6332/6332), done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-jxjt0g36/src/docker-src.2018-09-03-22.07.23.2969/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-jxjt0g36/src/docker-src.2018-09-03-22.07.23.2969/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
COPY RUNNER
RUN test-quick in qemu:centos7
Packages installed:
SDL-devel-1.2.15-14.el7.x86_64
bison-3.0.4-1.el7.x86_64
bzip2-1.0.6-13.el7.x86_64
bzip2-devel-1.0.6-13.el7.x86_64
ccache-3.3.4-1.el7.x86_64
csnappy-devel-0-6.20150729gitd7bc683.el7.x86_64
flex-2.5.37-3.el7.x86_64
gcc-4.8.5-28.el7_5.1.x86_64
gettext-0.19.8.1-2.el7.x86_64
git-1.8.3.1-14.el7_5.x86_64
glib2-devel-2.54.2-2.el7.x86_64
libaio-devel-0.3.109-13.el7.x86_64
libepoxy-devel-1.3.1-2.el7_5.x86_64
libfdt-devel-1.4.6-1.el7.x86_64
lzo-devel-2.06-8.el7.x86_64
make-3.82-23.el7.x86_64
mesa-libEGL-devel-17.2.3-8.20171019.el7.x86_64
mesa-libgbm-devel-17.2.3-8.20171019.el7.x86_64
nettle-devel-2.7.1-8.el7.x86_64
package g++ is not installed
package librdmacm-devel is not installed
pixman-devel-0.34.0-1.el7.x86_64
spice-glib-devel-0.34-3.el7_5.1.x86_64
spice-server-devel-0.14.0-2.el7_5.4.x86_64
tar-1.26-34.el7.x86_64
vte-devel-0.28.2-10.el7.x86_64
xen-devel-4.6.6-12.el7.x86_64
zlib-devel-1.2.7-17.el7.x86_64
Environment variables:
PACKAGES=bison bzip2 bzip2-devel ccache csnappy-devel flex g++ gcc gettext git glib2-devel libaio-devel libepoxy-devel libfdt-devel librdmacm-devel lzo-devel nettle-devel make mesa-libEGL-devel mesa-libgbm-devel pixman-devel SDL-devel spice-glib-devel spice-server-devel tar vte-devel xen-devel zlib-devel
HOSTNAME=376af022be80
MAKEFLAGS= -j8
J=8
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
TARGET_LIST=
SHLVL=1
HOME=/home/patchew
TEST_DIR=/tmp/qemu-test
FEATURES= dtc
DEBUG=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install
No C++ compiler available; disabling C++ specific optional code
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install/share/qemu
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install/bin
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib/qemu
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install/etc
local state directory /tmp/qemu-test/install/var
Manual directory /tmp/qemu-test/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules
C compiler cc
Host C compiler cc
C++ compiler
Objective-C compiler cc
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/include/pixman-1 -Werror -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -Wno-missing-braces -I/usr/include/libpng15 -pthread -I/usr/include/spice-server -I/usr/include/cacard -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/nss3 -I/usr/include/nspr4 -I/usr/include/spice-1
LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
QEMU_LDFLAGS
make make
install install
python python -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (1.2.15)
GTK support yes (2.24.31)
GTK GL support no
VTE support yes (0.28.2)
TLS priority NORMAL
GNUTLS support no
GNUTLS rnd no
libgcrypt no
libgcrypt kdf no
nettle yes (2.7.1)
nettle kdf yes
libtasn1 no
curses support yes
virgl support no
curl support no
mingw32 support no
Audio drivers oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support no
VNC PNG support yes
xen support yes
xen ctrl version 40600
pv dom build no
brlapi support no
bluez support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support yes
ATTR/XATTR support yes
Install blobs yes
KVM support yes
HAX support no
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support yes
RDMA support yes
PVRDMA support yes
fdt support system
membarrier no
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
posix_memalign yes
libcap-ng support no
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends log
spice support yes (0.12.13/0.14.0)
rbd support no
xfsctl support no
smartcard support yes
libusb no
usb net redir no
OpenGL support yes
OpenGL dmabufs yes
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
debug stack usage no
mutex debugging no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support no
TPM passthrough yes
TPM emulator yes
QOM debugging yes
Live block migration yes
lzo support yes
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone no
docker no
WARNING: Use of GTK 2.0 is deprecated and will be removed in
WARNING: future releases. Please switch to using GTK 3.0
WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0
NOTE: cross-compilers enabled: 'cc'
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN qemu-options.def
GEN config-host.h
GEN qapi-gen
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN x86_64-softmmu/config-devices.mak
GEN aarch64-softmmu/config-devices.mak
GEN ui/input-keymap-atset1-to-qcode.c
GEN ui/input-keymap-linux-to-qcode.c
GEN ui/input-keymap-qcode-to-atset1.c
GEN ui/input-keymap-qcode-to-atset2.c
GEN ui/input-keymap-qcode-to-atset3.c
GEN ui/input-keymap-qcode-to-linux.c
GEN ui/input-keymap-qcode-to-qnum.c
GEN ui/input-keymap-qcode-to-sun.c
GEN ui/input-keymap-qnum-to-qcode.c
GEN ui/input-keymap-win32-to-qcode.c
GEN ui/input-keymap-usb-to-qcode.c
GEN ui/input-keymap-x11-to-qcode.c
GEN ui/input-keymap-xorgevdev-to-qcode.c
GEN ui/input-keymap-xorgkbd-to-qcode.c
GEN ui/input-keymap-xorgxquartz-to-qcode.c
GEN ui/input-keymap-xorgxwin-to-qcode.c
GEN ui/input-keymap-osx-to-qcode.c
GEN tests/test-qapi-gen
GEN trace-root.h
GEN accel/kvm/trace.h
GEN accel/tcg/trace.h
GEN audio/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN crypto/trace.h
GEN hw/9pfs/trace.h
GEN hw/acpi/trace.h
GEN hw/alpha/trace.h
GEN hw/arm/trace.h
GEN hw/audio/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/display/trace.h
GEN hw/dma/trace.h
GEN hw/hppa/trace.h
GEN hw/i2c/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/ide/trace.h
GEN hw/input/trace.h
GEN hw/intc/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/net/trace.h
GEN hw/nvram/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/ppc/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/s390x/trace.h
GEN hw/scsi/trace.h
GEN hw/sd/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/timer/trace.h
GEN hw/tpm/trace.h
GEN hw/usb/trace.h
GEN hw/vfio/trace.h
GEN hw/virtio/trace.h
GEN hw/watchdog/trace.h
GEN hw/xen/trace.h
GEN io/trace.h
GEN linux-user/trace.h
GEN migration/trace.h
GEN nbd/trace.h
GEN net/trace.h
GEN qapi/trace.h
GEN qom/trace.h
GEN scsi/trace.h
GEN target/arm/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/ppc/trace.h
GEN target/s390x/trace.h
GEN target/sparc/trace.h
GEN ui/trace.h
GEN util/trace.h
GEN trace-root.c
GEN accel/kvm/trace.c
GEN accel/tcg/trace.c
GEN audio/trace.c
GEN block/trace.c
GEN chardev/trace.c
GEN crypto/trace.c
GEN hw/9pfs/trace.c
GEN hw/acpi/trace.c
GEN hw/alpha/trace.c
GEN hw/arm/trace.c
GEN hw/audio/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/display/trace.c
GEN hw/dma/trace.c
GEN hw/hppa/trace.c
GEN hw/i2c/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/ide/trace.c
GEN hw/input/trace.c
GEN hw/intc/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/net/trace.c
GEN hw/nvram/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/ppc/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/s390x/trace.c
GEN hw/scsi/trace.c
GEN hw/sd/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/timer/trace.c
GEN hw/tpm/trace.c
GEN hw/usb/trace.c
GEN hw/vfio/trace.c
GEN hw/virtio/trace.c
GEN hw/watchdog/trace.c
GEN hw/xen/trace.c
GEN io/trace.c
GEN linux-user/trace.c
GEN migration/trace.c
GEN nbd/trace.c
GEN net/trace.c
GEN qapi/trace.c
GEN qom/trace.c
GEN scsi/trace.c
GEN target/arm/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/ppc/trace.c
GEN target/s390x/trace.c
GEN target/sparc/trace.c
GEN ui/trace.c
GEN util/trace.c
GEN config-all-devices.mak
CC tests/qemu-iotests/socket_scm_helper.o
GEN qga/qapi-generated/qapi-gen
CC qapi/qapi-types-block-core.o
CC qapi/qapi-builtin-types.o
CC qapi/qapi-types.o
CC qapi/qapi-types-char.o
CC qapi/qapi-types-block.o
CC qapi/qapi-types-common.o
CC qapi/qapi-types-crypto.o
CC qapi/qapi-types-introspect.o
CC qapi/qapi-types-job.o
CC qapi/qapi-types-misc.o
CC qapi/qapi-types-migration.o
CC qapi/qapi-types-net.o
CC qapi/qapi-types-rocker.o
CC qapi/qapi-types-run-state.o
CC qapi/qapi-types-sockets.o
CC qapi/qapi-types-tpm.o
CC qapi/qapi-types-trace.o
CC qapi/qapi-types-transaction.o
CC qapi/qapi-types-ui.o
CC qapi/qapi-builtin-visit.o
CC qapi/qapi-visit.o
CC qapi/qapi-visit-block-core.o
CC qapi/qapi-visit-block.o
CC qapi/qapi-visit-char.o
CC qapi/qapi-visit-common.o
CC qapi/qapi-visit-crypto.o
CC qapi/qapi-visit-introspect.o
CC qapi/qapi-visit-job.o
CC qapi/qapi-visit-migration.o
CC qapi/qapi-visit-misc.o
CC qapi/qapi-visit-net.o
CC qapi/qapi-visit-rocker.o
CC qapi/qapi-visit-run-state.o
CC qapi/qapi-visit-sockets.o
CC qapi/qapi-visit-tpm.o
CC qapi/qapi-visit-trace.o
CC qapi/qapi-visit-transaction.o
CC qapi/qapi-visit-ui.o
CC qapi/qapi-events.o
CC qapi/qapi-events-block-core.o
CC qapi/qapi-events-block.o
CC qapi/qapi-events-char.o
CC qapi/qapi-events-common.o
CC qapi/qapi-events-crypto.o
CC qapi/qapi-events-job.o
CC qapi/qapi-events-introspect.o
CC qapi/qapi-events-migration.o
CC qapi/qapi-events-misc.o
CC qapi/qapi-events-net.o
CC qapi/qapi-events-rocker.o
CC qapi/qapi-events-run-state.o
CC qapi/qapi-events-sockets.o
CC qapi/qapi-events-tpm.o
CC qapi/qapi-events-trace.o
CC qapi/qapi-events-transaction.o
CC qapi/qapi-events-ui.o
CC qapi/qapi-introspect.o
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qnum.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qbool.o
CC qobject/qlit.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
CC qobject/block-qdict.o
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/lockcnt.o
CC util/aiocb.o
CC util/async.o
CC util/aio-wait.o
CC util/thread-pool.o
CC util/qemu-timer.o
CC util/main-loop.o
CC util/iohandler.o
CC util/compatfd.o
CC util/aio-posix.o
CC util/event_notifier-posix.o
CC util/mmap-alloc.o
CC util/oslib-posix.o
CC util/qemu-openpty.o
CC util/qemu-thread-posix.o
CC util/memfd.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/host-utils.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/fifo8.o
CC util/acl.o
CC util/cacheinfo.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/keyval.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/readline.o
CC util/rcu.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-ucontext.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/pagesize.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC util/stats64.o
CC util/systemd.o
CC util/iova-tree.o
CC util/vfio-helpers.o
CC accel/kvm/trace.o
CC trace-root.o
CC accel/tcg/trace.o
CC audio/trace.o
CC block/trace.o
CC chardev/trace.o
CC crypto/trace.o
CC hw/9pfs/trace.o
CC hw/acpi/trace.o
CC hw/alpha/trace.o
CC hw/arm/trace.o
CC hw/audio/trace.o
CC hw/block/trace.o
CC hw/block/dataplane/trace.o
CC hw/char/trace.o
CC hw/display/trace.o
CC hw/dma/trace.o
CC hw/hppa/trace.o
CC hw/i2c/trace.o
CC hw/i386/trace.o
CC hw/i386/xen/trace.o
CC hw/input/trace.o
CC hw/ide/trace.o
CC hw/intc/trace.o
CC hw/isa/trace.o
CC hw/mem/trace.o
CC hw/misc/trace.o
CC hw/misc/macio/trace.o
CC hw/net/trace.o
CC hw/nvram/trace.o
CC hw/pci/trace.o
CC hw/pci-host/trace.o
CC hw/ppc/trace.o
CC hw/rdma/trace.o
CC hw/rdma/vmw/trace.o
CC hw/s390x/trace.o
CC hw/sd/trace.o
CC hw/scsi/trace.o
CC hw/sparc/trace.o
CC hw/sparc64/trace.o
CC hw/timer/trace.o
CC hw/tpm/trace.o
CC hw/usb/trace.o
CC hw/vfio/trace.o
CC hw/watchdog/trace.o
CC hw/virtio/trace.o
CC hw/xen/trace.o
CC io/trace.o
CC linux-user/trace.o
CC migration/trace.o
CC nbd/trace.o
CC net/trace.o
CC qapi/trace.o
CC qom/trace.o
CC scsi/trace.o
CC target/arm/trace.o
CC target/i386/trace.o
CC target/mips/trace.o
CC target/ppc/trace.o
CC target/s390x/trace.o
CC target/sparc/trace.o
CC ui/trace.o
CC util/trace.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-clock.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/error-printf.o
CC stubs/fdset.o
CC stubs/gdbstub.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/iothread-lock.o
CC stubs/is-daemonized.o
CC stubs/linux-aio.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/change-state-handler.o
CC stubs/monitor.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/tpm.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/qmp_memory_device.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/xen-common.o
CC stubs/vmgenid.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
CC stubs/ram-block.o
CC contrib/ivshmem-server/ivshmem-server.o
CC contrib/ivshmem-client/ivshmem-client.o
CC contrib/ivshmem-client/main.o
CC contrib/ivshmem-server/main.o
CC qemu-nbd.o
CC block.o
CC blockjob.o
CC job.o
CC qemu-io-cmds.o
CC replication.o
CC block/raw-format.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qcow2-bitmap.o
CC block/qed.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/vhdx-log.o
CC block/quorum.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/blklogwrites.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/file-posix.o
CC block/linux-aio.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
/tmp/qemu-test/src/block/commit.c: In function 'commit_exit':
/tmp/qemu-test/src/block/commit.c:88:35: error: 'ret' undeclared (first use in this function)
if (!job_is_cancelled(job) && ret == 0) {
^
/tmp/qemu-test/src/block/commit.c:88:35: note: each undeclared identifier is reported only once for each function it appears in
CC block/io.o
make: *** [block/commit.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 565, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 562, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 308, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 276, in run
quiet=quiet)
File "./tests/docker/docker.py", line 183, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=4a298a58afe711e8a9e352540069c830', '-u', '1000', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-jxjt0g36/src/docker-src.2018-09-03-22.07.23.2969:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:216: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-jxjt0g36/src'
make: *** [tests/docker/Makefile.include:250: docker-run-test-quick@centos7] Error 2
real 1m43.415s
user 0m4.776s
sys 0m4.531s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 53+ messages in thread
end of thread, other threads:[~2018-09-04 2:14 UTC | newest]
Thread overview: 53+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-17 19:04 [Qemu-devel] [PATCH 0/7] jobs: remove job_defer_to_main_loop John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 1/7] jobs: change start callback to run callback John Snow
2018-08-20 18:28 ` Eric Blake
2018-08-20 19:04 ` John Snow
2018-08-22 10:51 ` Max Reitz
2018-08-22 23:01 ` John Snow
2018-08-25 13:33 ` Max Reitz
2018-08-25 14:15 ` Max Reitz
2018-08-27 16:01 ` John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 2/7] jobs: canonize Error object John Snow
2018-08-20 20:03 ` Eric Blake
2018-08-21 0:10 ` John Snow
2018-08-22 10:59 ` Max Reitz
2018-08-22 22:50 ` John Snow
2018-08-25 13:15 ` Max Reitz
2018-08-22 11:09 ` Max Reitz
2018-08-22 11:11 ` Max Reitz
2018-08-17 19:04 ` [Qemu-devel] [PATCH 3/7] jobs: add exit shim John Snow
2018-08-20 21:16 ` Eric Blake
2018-08-22 11:43 ` Max Reitz
2018-08-22 11:52 ` Max Reitz
2018-08-22 21:45 ` John Snow
2018-08-25 12:54 ` Max Reitz
2018-08-22 21:52 ` John Snow
2018-08-25 13:05 ` Max Reitz
2018-08-27 15:54 ` John Snow
2018-08-29 8:16 ` Max Reitz
2018-08-22 22:01 ` Eric Blake
2018-08-22 22:04 ` John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 4/7] block/commit: utilize job_exit shim John Snow
2018-08-17 19:18 ` John Snow
2018-08-22 11:58 ` Max Reitz
2018-08-22 21:55 ` John Snow
2018-08-25 13:07 ` Max Reitz
2018-08-22 11:55 ` Max Reitz
2018-08-17 19:04 ` [Qemu-devel] [PATCH 5/7] block/mirror: " John Snow
2018-08-22 12:06 ` Max Reitz
2018-08-22 12:15 ` Max Reitz
2018-08-22 22:05 ` John Snow
2018-08-25 15:02 ` Max Reitz
2018-08-25 15:14 ` Max Reitz
2018-08-28 20:25 ` John Snow
2018-08-29 8:28 ` Max Reitz
2018-08-28 21:51 ` John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 6/7] jobs: " John Snow
2018-08-22 12:20 ` Max Reitz
2018-08-22 23:40 ` John Snow
2018-08-17 19:04 ` [Qemu-devel] [PATCH 7/7] jobs: remove job_defer_to_main_loop John Snow
2018-08-22 12:21 ` Max Reitz
2018-08-18 16:27 ` [Qemu-devel] [PATCH 0/7] " no-reply
2018-08-18 16:31 ` no-reply
2018-09-04 2:06 ` no-reply
2018-09-04 2:09 ` no-reply
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.