All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org, vsementsov@virtuozzo.com,
	eblake@redhat.com, mreitz@redhat.com, kwolf@redhat.com,
	rvkagan@yandex-team.ru, den@openvz.org,
	Stefan Hajnoczi <stefanha@redhat.com>, Fam Zheng <fam@euphon.net>
Subject: [PATCH v3 06/33] util/async: aio_co_schedule(): support reschedule in same ctx
Date: Fri, 16 Apr 2021 11:08:44 +0300	[thread overview]
Message-ID: <20210416080911.83197-7-vsementsov@virtuozzo.com> (raw)
In-Reply-To: <20210416080911.83197-1-vsementsov@virtuozzo.com>

With the following patch we want to call wake coroutine from thread.
And it doesn't work with aio_co_wake:
Assume we have no iothreads.
Assume we have a coroutine A, which waits in the yield point for
external aio_co_wake(), and no progress can be done until it happen.
Main thread is in blocking aio_poll() (for example, in blk_read()).

Now, in a separate thread we do aio_co_wake(). It calls  aio_co_enter(),
which goes through last "else" branch and do aio_context_acquire(ctx).

Now we have a deadlock, as aio_poll() will not release the context lock
until some progress is done, and progress can't be done until
aio_co_wake() wake the coroutine A. And it can't because it wait for
aio_context_acquire().

Still, aio_co_schedule() works well in parallel with blocking
aio_poll(). So we want use it. Let's add a possibility of rescheduling
coroutine in same ctx where it was yield'ed.

Fetch co->ctx in same way as it is done in aio_co_wake().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 include/block/aio.h | 2 +-
 util/async.c        | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/include/block/aio.h b/include/block/aio.h
index 5f342267d5..744b695525 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -643,7 +643,7 @@ static inline bool aio_node_check(AioContext *ctx, bool is_external)
 
 /**
  * aio_co_schedule:
- * @ctx: the aio context
+ * @ctx: the aio context, if NULL, the current ctx of @co will be used.
  * @co: the coroutine
  *
  * Start a coroutine on a remote AioContext.
diff --git a/util/async.c b/util/async.c
index 674dbefb7c..750be555c6 100644
--- a/util/async.c
+++ b/util/async.c
@@ -545,6 +545,14 @@ fail:
 
 void aio_co_schedule(AioContext *ctx, Coroutine *co)
 {
+    if (!ctx) {
+        /*
+         * Read coroutine before co->ctx.  Matches smp_wmb in
+         * qemu_coroutine_enter.
+         */
+        smp_read_barrier_depends();
+        ctx = qatomic_read(&co->ctx);
+    }
     trace_aio_co_schedule(ctx, co);
     const char *scheduled = qatomic_cmpxchg(&co->scheduled, NULL,
                                            __func__);
-- 
2.29.2



  parent reply	other threads:[~2021-04-16  8:12 UTC|newest]

Thread overview: 121+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-16  8:08 [PATCH v3 00/33] block/nbd: rework client connection Vladimir Sementsov-Ogievskiy
2021-04-16  8:08 ` [PATCH v3 01/33] block/nbd: fix channel object leak Vladimir Sementsov-Ogievskiy
2021-05-24 21:31   ` Eric Blake
2021-05-25  4:47     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:08 ` [PATCH v3 02/33] block/nbd: fix how state is cleared on nbd_open() failure paths Vladimir Sementsov-Ogievskiy
2021-04-21 14:00   ` Roman Kagan
2021-04-21 22:27     ` Vladimir Sementsov-Ogievskiy
2021-04-22  8:49       ` Roman Kagan
2021-06-01 21:39   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 03/33] block/nbd: ensure ->connection_thread is always valid Vladimir Sementsov-Ogievskiy
2021-06-01 21:41   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 04/33] block/nbd: nbd_client_handshake(): fix leak of s->ioc Vladimir Sementsov-Ogievskiy
2021-04-22  8:59   ` Roman Kagan
2021-04-16  8:08 ` [PATCH v3 05/33] block/nbd: BDRVNBDState: drop unused connect_err and connect_status Vladimir Sementsov-Ogievskiy
2021-06-01 21:43   ` Eric Blake
2021-04-16  8:08 ` Vladimir Sementsov-Ogievskiy [this message]
2021-04-23 10:09   ` [PATCH v3 06/33] util/async: aio_co_schedule(): support reschedule in same ctx Roman Kagan
2021-04-26  8:52     ` Vladimir Sementsov-Ogievskiy
2021-05-12  6:56   ` Paolo Bonzini
2021-05-12  7:15     ` Vladimir Sementsov-Ogievskiy
2021-05-13 21:04       ` Paolo Bonzini
2021-05-14 17:27         ` Roman Kagan
2021-05-14 21:19           ` Paolo Bonzini
2021-06-08 18:45         ` Vladimir Sementsov-Ogievskiy
2021-06-09  9:35           ` Paolo Bonzini
2021-06-09 10:24             ` Vladimir Sementsov-Ogievskiy
2021-06-09 12:17               ` Paolo Bonzini
2021-04-16  8:08 ` [PATCH v3 07/33] block/nbd: simplify waking of nbd_co_establish_connection() Vladimir Sementsov-Ogievskiy
2021-04-27 21:44   ` Roman Kagan
2021-06-02 19:05   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 08/33] block/nbd: drop thr->state Vladimir Sementsov-Ogievskiy
2021-04-27 22:23   ` Roman Kagan
2021-04-28  8:01     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:08 ` [PATCH v3 09/33] block/nbd: bs-independent interface for nbd_co_establish_connection() Vladimir Sementsov-Ogievskiy
2021-06-02 19:14   ` Eric Blake
2021-06-08 10:12     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:08 ` [PATCH v3 10/33] block/nbd: make nbd_co_establish_connection_cancel() bs-independent Vladimir Sementsov-Ogievskiy
2021-06-02 21:18   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 11/33] block/nbd: rename NBDConnectThread to NBDClientConnection Vladimir Sementsov-Ogievskiy
2021-04-27 22:28   ` Roman Kagan
2021-06-02 21:21   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 12/33] block/nbd: introduce nbd_client_connection_new() Vladimir Sementsov-Ogievskiy
2021-06-02 21:22   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 13/33] block/nbd: introduce nbd_client_connection_release() Vladimir Sementsov-Ogievskiy
2021-04-27 22:35   ` Roman Kagan
2021-04-28  8:06     ` Vladimir Sementsov-Ogievskiy
2021-06-02 21:27   ` Eric Blake
2021-06-03 11:59     ` Vladimir Sementsov-Ogievskiy
2021-06-08 10:00     ` Vladimir Sementsov-Ogievskiy
2021-06-08 14:18       ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 14/33] nbd: move connection code from block/nbd to nbd/client-connection Vladimir Sementsov-Ogievskiy
2021-04-27 22:45   ` Roman Kagan
2021-04-28  8:14     ` Vladimir Sementsov-Ogievskiy
2021-06-09 15:49       ` Vladimir Sementsov-Ogievskiy
2021-06-03 15:55   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 15/33] nbd/client-connection: use QEMU_LOCK_GUARD Vladimir Sementsov-Ogievskiy
2021-04-28  6:08   ` Roman Kagan
2021-04-28  8:17     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:08 ` [PATCH v3 16/33] nbd/client-connection: add possibility of negotiation Vladimir Sementsov-Ogievskiy
2021-05-11 10:45   ` Roman Kagan
2021-05-12  6:42     ` Vladimir Sementsov-Ogievskiy
2021-06-08 19:23       ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:08 ` [PATCH v3 17/33] nbd/client-connection: implement connection retry Vladimir Sementsov-Ogievskiy
2021-05-11 20:54   ` Roman Kagan
2021-06-08 10:24     ` Vladimir Sementsov-Ogievskiy
2021-06-03 16:17   ` Eric Blake
2021-06-03 17:49     ` Vladimir Sementsov-Ogievskiy
2021-06-08 10:38       ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:08 ` [PATCH v3 18/33] nbd/client-connection: shutdown connection on release Vladimir Sementsov-Ogievskiy
2021-05-11 21:08   ` Roman Kagan
2021-05-12  6:39     ` Vladimir Sementsov-Ogievskiy
2021-06-03 16:20   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 19/33] block/nbd: split nbd_handle_updated_info out of nbd_client_handshake() Vladimir Sementsov-Ogievskiy
2021-05-12  8:40   ` Roman Kagan
2021-06-03 16:29   ` Eric Blake
2021-06-09 17:23     ` Vladimir Sementsov-Ogievskiy
2021-06-09 18:28       ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 20/33] block/nbd: use negotiation of NBDClientConnection Vladimir Sementsov-Ogievskiy
2021-06-03 18:05   ` Eric Blake
2021-04-16  8:08 ` [PATCH v3 21/33] qemu-socket: pass monitor link to socket_get_fd directly Vladimir Sementsov-Ogievskiy
2021-04-19  9:34   ` Daniel P. Berrangé
2021-04-19 10:09     ` Vladimir Sementsov-Ogievskiy
2021-05-12  9:40     ` Roman Kagan
2021-05-12  9:59       ` Daniel P. Berrangé
2021-05-13 11:02         ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:09 ` [PATCH v3 22/33] block/nbd: pass monitor directly to connection thread Vladimir Sementsov-Ogievskiy
2021-06-03 18:16   ` Eric Blake
2021-06-03 18:31     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:09 ` [PATCH v3 23/33] block/nbd: nbd_teardown_connection() don't touch s->sioc Vladimir Sementsov-Ogievskiy
2021-06-03 19:04   ` Eric Blake
2021-04-16  8:09 ` [PATCH v3 24/33] block/nbd: drop BDRVNBDState::sioc Vladimir Sementsov-Ogievskiy
2021-06-03 19:12   ` Eric Blake
2021-04-16  8:09 ` [PATCH v3 25/33] nbd/client-connection: return only one io channel Vladimir Sementsov-Ogievskiy
2021-06-03 19:58   ` Eric Blake
2021-04-16  8:09 ` [PATCH v3 26/33] block-coroutine-wrapper: allow non bdrv_ prefix Vladimir Sementsov-Ogievskiy
2021-06-03 20:00   ` Eric Blake
2021-06-03 20:53   ` Eric Blake
2021-06-04  5:29     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:09 ` [PATCH v3 27/33] block/nbd: split nbd_co_do_establish_connection out of nbd_reconnect_attempt Vladimir Sementsov-Ogievskiy
2021-06-03 20:04   ` Eric Blake
2021-06-04  5:30     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:09 ` [PATCH v3 28/33] nbd/client-connection: do qio_channel_set_delay(false) Vladimir Sementsov-Ogievskiy
2021-06-03 20:48   ` Eric Blake
2021-06-04  5:32     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:09 ` [PATCH v3 29/33] nbd/client-connection: add option for non-blocking connection attempt Vladimir Sementsov-Ogievskiy
2021-06-03 20:51   ` Eric Blake
2021-04-16  8:09 ` [PATCH v3 30/33] block/nbd: reuse nbd_co_do_establish_connection() in nbd_open() Vladimir Sementsov-Ogievskiy
2021-06-03 20:57   ` Eric Blake
2021-04-16  8:09 ` [PATCH v3 31/33] block/nbd: add nbd_clinent_connected() helper Vladimir Sementsov-Ogievskiy
2021-05-12  7:06   ` Paolo Bonzini
2021-05-12  7:19     ` Vladimir Sementsov-Ogievskiy
2021-06-03 21:08   ` Eric Blake
2021-04-16  8:09 ` [PATCH v3 32/33] block/nbd: safer transition to receiving request Vladimir Sementsov-Ogievskiy
2021-06-03 21:11   ` Eric Blake
2021-06-04  5:36     ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:09 ` [PATCH v3 33/33] block/nbd: drop connection_co Vladimir Sementsov-Ogievskiy
2021-04-16  8:14   ` Vladimir Sementsov-Ogievskiy
2021-04-16  8:21     ` Vladimir Sementsov-Ogievskiy
2021-06-03 21:27   ` Eric Blake
2021-06-04  5:39     ` Vladimir Sementsov-Ogievskiy
2021-05-12  6:54 ` [PATCH v3 00/33] block/nbd: rework client connection Paolo Bonzini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210416080911.83197-7-vsementsov@virtuozzo.com \
    --to=vsementsov@virtuozzo.com \
    --cc=den@openvz.org \
    --cc=eblake@redhat.com \
    --cc=fam@euphon.net \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=rvkagan@yandex-team.ru \
    --cc=stefanha@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.