* [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect @ 2017-08-17 22:43 Annie Li 2017-08-18 9:14 ` Roger Pau Monné 0 siblings, 1 reply; 7+ messages in thread From: Annie Li @ 2017-08-17 22:43 UTC (permalink / raw) To: xen-devel, linux-kernel; +Cc: roger.pau, konrad.wilk, annie.li If there is inflight I/O in any non-last queue, blkback returns -EBUSY directly, and never stops thread of remaining queue and processs them. When removing vbd device with lots of disk I/O load, some queues with inflight I/O still have blkback thread running even though the corresponding vbd device or guest is gone. And this could cause some problems, for example, if the backend device type is file, some loop devices and blkback thread always lingers there forever after guest is destroyed, and this causes failure of umounting repositories unless rebooting the dom0. So stop all threads properly and return -EBUSY if any queue has inflight I/O. Signed-off-by: Annie Li <annie.li@oracle.com> Reviewed-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com> Reviewed-by: Bhavesh Davda <bhavesh.davda@oracle.com> Reviewed-by: Adnan Misherfi <adnan.misherfi@oracle.com> --- drivers/block/xen-blkback/xenbus.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c index 792da68..2adb859 100644 --- a/drivers/block/xen-blkback/xenbus.c +++ b/drivers/block/xen-blkback/xenbus.c @@ -244,6 +244,7 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) { struct pending_req *req, *n; unsigned int j, r; + bool busy = false; for (r = 0; r < blkif->nr_rings; r++) { struct xen_blkif_ring *ring = &blkif->rings[r]; @@ -261,8 +262,10 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) * don't have any discard_io or other_io requests. So, checking * for inflight IO is enough. */ - if (atomic_read(&ring->inflight) > 0) - return -EBUSY; + if (atomic_read(&ring->inflight) > 0) { + busy = true; + continue; + } if (ring->irq) { unbind_from_irqhandler(ring->irq, ring); @@ -300,6 +303,9 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) WARN_ON(i != (XEN_BLKIF_REQS_PER_PAGE * blkif->nr_ring_pages)); ring->active = false; } + if (busy) + return -EBUSY; + blkif->nr_ring_pages = 0; /* * blkif->rings was allocated in connect_ring, so we should free it in -- 1.9.3 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect 2017-08-17 22:43 [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect Annie Li @ 2017-08-18 9:14 ` Roger Pau Monné 2017-08-18 14:29 ` [Xen-devel] " annie li 0 siblings, 1 reply; 7+ messages in thread From: Roger Pau Monné @ 2017-08-18 9:14 UTC (permalink / raw) To: Annie Li; +Cc: xen-devel, linux-kernel, konrad.wilk On Thu, Aug 17, 2017 at 06:43:46PM -0400, Annie Li wrote: > If there is inflight I/O in any non-last queue, blkback returns -EBUSY > directly, and never stops thread of remaining queue and processs them. When > removing vbd device with lots of disk I/O load, some queues with inflight > I/O still have blkback thread running even though the corresponding vbd > device or guest is gone. > And this could cause some problems, for example, if the backend device type > is file, some loop devices and blkback thread always lingers there forever > after guest is destroyed, and this causes failure of umounting repositories > unless rebooting the dom0. So stop all threads properly and return -EBUSY > if any queue has inflight I/O. > > Signed-off-by: Annie Li <annie.li@oracle.com> > Reviewed-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com> > Reviewed-by: Bhavesh Davda <bhavesh.davda@oracle.com> > Reviewed-by: Adnan Misherfi <adnan.misherfi@oracle.com> > --- > drivers/block/xen-blkback/xenbus.c | 10 ++++++++-- > 1 file changed, 8 insertions(+), 2 deletions(-) > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c > index 792da68..2adb859 100644 > --- a/drivers/block/xen-blkback/xenbus.c > +++ b/drivers/block/xen-blkback/xenbus.c > @@ -244,6 +244,7 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) > { > struct pending_req *req, *n; > unsigned int j, r; > + bool busy = false; > > for (r = 0; r < blkif->nr_rings; r++) { > struct xen_blkif_ring *ring = &blkif->rings[r]; > @@ -261,8 +262,10 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) > * don't have any discard_io or other_io requests. So, checking > * for inflight IO is enough. > */ > - if (atomic_read(&ring->inflight) > 0) > - return -EBUSY; > + if (atomic_read(&ring->inflight) > 0) { > + busy = true; > + continue; > + } I guess I'm missing something, but I don't see how this is solving the problem described in the description. If the problem is that xen_blkif_disconnect returns without cleaning all the queues, this patch keeps the current behavior, just that it will try to remove more queues before returning, as opposed to returning when finding the first busy queue. Roger. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Xen-devel] [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect 2017-08-18 9:14 ` Roger Pau Monné @ 2017-08-18 14:29 ` annie li 2017-08-18 17:24 ` Roger Pau Monné 0 siblings, 1 reply; 7+ messages in thread From: annie li @ 2017-08-18 14:29 UTC (permalink / raw) To: Roger Pau Monné; +Cc: xen-devel, linux-kernel On 8/18/2017 5:14 AM, Roger Pau Monné wrote: > On Thu, Aug 17, 2017 at 06:43:46PM -0400, Annie Li wrote: >> If there is inflight I/O in any non-last queue, blkback returns -EBUSY >> directly, and never stops thread of remaining queue and processs them. When >> removing vbd device with lots of disk I/O load, some queues with inflight >> I/O still have blkback thread running even though the corresponding vbd >> device or guest is gone. >> And this could cause some problems, for example, if the backend device type >> is file, some loop devices and blkback thread always lingers there forever >> after guest is destroyed, and this causes failure of umounting repositories >> unless rebooting the dom0. So stop all threads properly and return -EBUSY >> if any queue has inflight I/O. >> >> Signed-off-by: Annie Li <annie.li@oracle.com> >> Reviewed-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com> >> Reviewed-by: Bhavesh Davda <bhavesh.davda@oracle.com> >> Reviewed-by: Adnan Misherfi <adnan.misherfi@oracle.com> >> --- >> drivers/block/xen-blkback/xenbus.c | 10 ++++++++-- >> 1 file changed, 8 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c >> index 792da68..2adb859 100644 >> --- a/drivers/block/xen-blkback/xenbus.c >> +++ b/drivers/block/xen-blkback/xenbus.c >> @@ -244,6 +244,7 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) >> { >> struct pending_req *req, *n; >> unsigned int j, r; >> + bool busy = false; >> >> for (r = 0; r < blkif->nr_rings; r++) { >> struct xen_blkif_ring *ring = &blkif->rings[r]; >> @@ -261,8 +262,10 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) >> * don't have any discard_io or other_io requests. So, checking >> * for inflight IO is enough. >> */ >> - if (atomic_read(&ring->inflight) > 0) >> - return -EBUSY; >> + if (atomic_read(&ring->inflight) > 0) { >> + busy = true; >> + continue; >> + } > I guess I'm missing something, but I don't see how this is solving the > problem described in the description. > > If the problem is that xen_blkif_disconnect returns without cleaning > all the queues, this patch keeps the current behavior, just that it > will try to remove more queues before returning, as opposed to > returning when finding the first busy queue. Before checking inflight, following code stops the blkback thread, if (ring->xenblkd) { kthread_stop(ring->xenblkd); wake_up(&ring->shutdown_wq); } This patch allows thread of every queue has the chance to get stopped. Otherwise, only thread of queue before(including) first busy one get stopped, threads of remaining queue will still run, and these blkthread and corresponding loop device will linger forever even after guest is destroyed. Thanks Annie ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Xen-devel] [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect 2017-08-18 14:29 ` [Xen-devel] " annie li @ 2017-08-18 17:24 ` Roger Pau Monné 2017-08-22 7:32 ` Roger Pau Monné 0 siblings, 1 reply; 7+ messages in thread From: Roger Pau Monné @ 2017-08-18 17:24 UTC (permalink / raw) To: annie li; +Cc: xen-devel, linux-kernel On Fri, Aug 18, 2017 at 10:29:15AM -0400, annie li wrote: > > On 8/18/2017 5:14 AM, Roger Pau Monné wrote: > > On Thu, Aug 17, 2017 at 06:43:46PM -0400, Annie Li wrote: > > > If there is inflight I/O in any non-last queue, blkback returns -EBUSY > > > directly, and never stops thread of remaining queue and processs them. When > > > removing vbd device with lots of disk I/O load, some queues with inflight > > > I/O still have blkback thread running even though the corresponding vbd > > > device or guest is gone. > > > And this could cause some problems, for example, if the backend device type > > > is file, some loop devices and blkback thread always lingers there forever > > > after guest is destroyed, and this causes failure of umounting repositories > > > unless rebooting the dom0. So stop all threads properly and return -EBUSY > > > if any queue has inflight I/O. > > > > > > Signed-off-by: Annie Li <annie.li@oracle.com> > > > Reviewed-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com> > > > Reviewed-by: Bhavesh Davda <bhavesh.davda@oracle.com> > > > Reviewed-by: Adnan Misherfi <adnan.misherfi@oracle.com> > > > --- > > > drivers/block/xen-blkback/xenbus.c | 10 ++++++++-- > > > 1 file changed, 8 insertions(+), 2 deletions(-) > > > > > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c > > > index 792da68..2adb859 100644 > > > --- a/drivers/block/xen-blkback/xenbus.c > > > +++ b/drivers/block/xen-blkback/xenbus.c > > > @@ -244,6 +244,7 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) > > > { > > > struct pending_req *req, *n; > > > unsigned int j, r; > > > + bool busy = false; > > > for (r = 0; r < blkif->nr_rings; r++) { > > > struct xen_blkif_ring *ring = &blkif->rings[r]; > > > @@ -261,8 +262,10 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) > > > * don't have any discard_io or other_io requests. So, checking > > > * for inflight IO is enough. > > > */ > > > - if (atomic_read(&ring->inflight) > 0) > > > - return -EBUSY; > > > + if (atomic_read(&ring->inflight) > 0) { > > > + busy = true; > > > + continue; > > > + } > > I guess I'm missing something, but I don't see how this is solving the > > problem described in the description. > > > > If the problem is that xen_blkif_disconnect returns without cleaning > > all the queues, this patch keeps the current behavior, just that it > > will try to remove more queues before returning, as opposed to > > returning when finding the first busy queue. > Before checking inflight, following code stops the blkback thread, > if (ring->xenblkd) { > kthread_stop(ring->xenblkd); > wake_up(&ring->shutdown_wq); > } > This patch allows thread of every queue has the chance to get stopped. > Otherwise, only thread of queue before(including) first busy one get > stopped, threads of remaining queue will still run, and these blkthread and > corresponding loop device will linger forever even after guest is destroyed. Thanks for the explanation: Acked-by: Roger Pau Monné <roger.pau@citrix.com> Roger. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Xen-devel] [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect 2017-08-18 17:24 ` Roger Pau Monné @ 2017-08-22 7:32 ` Roger Pau Monné 2017-08-23 21:20 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 7+ messages in thread From: Roger Pau Monné @ 2017-08-22 7:32 UTC (permalink / raw) To: annie li, xen-devel, linux-kernel On Fri, Aug 18, 2017 at 06:24:06PM +0100, Roger Pau Monné wrote: > On Fri, Aug 18, 2017 at 10:29:15AM -0400, annie li wrote: > > > > On 8/18/2017 5:14 AM, Roger Pau Monné wrote: > > > On Thu, Aug 17, 2017 at 06:43:46PM -0400, Annie Li wrote: > > > > If there is inflight I/O in any non-last queue, blkback returns -EBUSY > > > > directly, and never stops thread of remaining queue and processs them. When > > > > removing vbd device with lots of disk I/O load, some queues with inflight > > > > I/O still have blkback thread running even though the corresponding vbd > > > > device or guest is gone. > > > > And this could cause some problems, for example, if the backend device type > > > > is file, some loop devices and blkback thread always lingers there forever > > > > after guest is destroyed, and this causes failure of umounting repositories > > > > unless rebooting the dom0. So stop all threads properly and return -EBUSY > > > > if any queue has inflight I/O. > > > > > > > > Signed-off-by: Annie Li <annie.li@oracle.com> > > > > Reviewed-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com> > > > > Reviewed-by: Bhavesh Davda <bhavesh.davda@oracle.com> > > > > Reviewed-by: Adnan Misherfi <adnan.misherfi@oracle.com> > > > > --- > > > > drivers/block/xen-blkback/xenbus.c | 10 ++++++++-- > > > > 1 file changed, 8 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c > > > > index 792da68..2adb859 100644 > > > > --- a/drivers/block/xen-blkback/xenbus.c > > > > +++ b/drivers/block/xen-blkback/xenbus.c > > > > @@ -244,6 +244,7 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) > > > > { > > > > struct pending_req *req, *n; > > > > unsigned int j, r; > > > > + bool busy = false; > > > > for (r = 0; r < blkif->nr_rings; r++) { > > > > struct xen_blkif_ring *ring = &blkif->rings[r]; > > > > @@ -261,8 +262,10 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) > > > > * don't have any discard_io or other_io requests. So, checking > > > > * for inflight IO is enough. > > > > */ > > > > - if (atomic_read(&ring->inflight) > 0) > > > > - return -EBUSY; > > > > + if (atomic_read(&ring->inflight) > 0) { > > > > + busy = true; > > > > + continue; > > > > + } > > > I guess I'm missing something, but I don't see how this is solving the > > > problem described in the description. > > > > > > If the problem is that xen_blkif_disconnect returns without cleaning > > > all the queues, this patch keeps the current behavior, just that it > > > will try to remove more queues before returning, as opposed to > > > returning when finding the first busy queue. > > Before checking inflight, following code stops the blkback thread, > > if (ring->xenblkd) { > > kthread_stop(ring->xenblkd); > > wake_up(&ring->shutdown_wq); > > } > > This patch allows thread of every queue has the chance to get stopped. > > Otherwise, only thread of queue before(including) first busy one get > > stopped, threads of remaining queue will still run, and these blkthread and > > corresponding loop device will linger forever even after guest is destroyed. > > Thanks for the explanation: > > Acked-by: Roger Pau Monné <roger.pau@citrix.com> Forgot to add, this needs to be backported to stable branches, so: Cc: stable@vger.kernel.org Roger. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Xen-devel] [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect 2017-08-22 7:32 ` Roger Pau Monné @ 2017-08-23 21:20 ` Konrad Rzeszutek Wilk 2017-08-24 21:30 ` annie li 0 siblings, 1 reply; 7+ messages in thread From: Konrad Rzeszutek Wilk @ 2017-08-23 21:20 UTC (permalink / raw) To: Roger Pau Monné; +Cc: annie li, xen-devel, linux-kernel .snip.. > > > > Acked-by: Roger Pau Monné <roger.pau@citrix.com> > > Forgot to add, this needs to be backported to stable branches, so: Annie, could you resend the patch with the tags and an update to the description to me please? > > Cc: stable@vger.kernel.org > > Roger. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Xen-devel] [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect 2017-08-23 21:20 ` Konrad Rzeszutek Wilk @ 2017-08-24 21:30 ` annie li 0 siblings, 0 replies; 7+ messages in thread From: annie li @ 2017-08-24 21:30 UTC (permalink / raw) To: Konrad Rzeszutek Wilk, Roger Pau Monné; +Cc: xen-devel, linux-kernel On 8/23/2017 5:20 PM, Konrad Rzeszutek Wilk wrote: > ..snip.. >>> Acked-by: Roger Pau Monné <roger.pau@citrix.com> >> Forgot to add, this needs to be backported to stable branches, so: > Annie, could you resend the patch with the tags and an update > to the description to me please? Done Thanks Annie >> Cc: stable@vger.kernel.org >> >> Roger. >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-08-24 21:30 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-08-17 22:43 [PATCH 1/1] xen-blkback: stop blkback thread of every queue in xen_blkif_disconnect Annie Li 2017-08-18 9:14 ` Roger Pau Monné 2017-08-18 14:29 ` [Xen-devel] " annie li 2017-08-18 17:24 ` Roger Pau Monné 2017-08-22 7:32 ` Roger Pau Monné 2017-08-23 21:20 ` Konrad Rzeszutek Wilk 2017-08-24 21:30 ` annie li
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).