From: Christoph Hellwig <hch@lst.de>
To: Ming Lei <ming.lei@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>,
Thomas Gleixner <tglx@linutronix.de>,
linux-block@vger.kernel.org, John Garry <john.garry@huawei.com>,
Bart Van Assche <bvanassche@acm.org>,
Hannes Reinecke <hare@suse.com>
Subject: Re: [PATCH 5/9] blk-mq: don't set data->ctx and data->hctx in blk_mq_alloc_request_hctx
Date: Mon, 18 May 2020 18:56:19 +0200 [thread overview]
Message-ID: <20200518165619.GA17465@lst.de> (raw)
In-Reply-To: <20200518141107.GA50374@T590>
On Mon, May 18, 2020 at 10:11:07PM +0800, Ming Lei wrote:
> On Mon, May 18, 2020 at 03:16:34PM +0200, Christoph Hellwig wrote:
> > On Mon, May 18, 2020 at 07:54:54PM +0800, Ming Lei wrote:
> > >
> > > I guess I misunderstood your point, sorry for that.
> > >
> > > The requirement is just that the request needs to be allocated on one online
> > > CPU after INACTIVE is set, and we can use a workqueue to do that.
> >
> > I've looked over the code again, and I'm really not sure why we need that.
> > Presumable the CPU hotplug code ensures tasks don't get schedule on the
> > CPU running the shutdown state machine, so if we just do a retry of the
>
> percpu kthread still can be scheduled on the cpu to be online, see
> is_cpu_allowed(). And bound wq has been used widely in fs code.
s/to be online/to be offlined/ I guess.
Shouldn't all the per-cpu kthreads also stop as part of the offlining?
If they don't quiesce before the new blk-mq stop state I think we need
to make sure they do. It is rather pointless to quiesce the requests
if a thread that can submit I/O is still live.
> >
> > http://git.infradead.org/users/hch/block.git/shortlog/refs/heads/blk-mq-hotplug.2
> >
>
> After preempt_disable() is removed, the INACTIVE bit is set in the CPU
> to be offline, and the bit can be read on all other CPUs, so other CPUs
> may not get synced with the INACTIVE bit.
INACTIVE is set to the hctx, and it is set by the last CPU to be
offlined that is mapped to the hctx. once the bit is set the barrier
ensured it is seen everywhere before we start waiting for the requests
to finish. What is missing?:
>
>
> Thanks,
> Ming
---end quoted text---
next prev parent reply other threads:[~2020-05-18 16:56 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-18 6:39 blk-mq: improvement CPU hotplug (simplified version) v2 Christoph Hellwig
2020-05-18 6:39 ` [PATCH 1/9] blk-mq: split out a __blk_mq_get_driver_tag helper Christoph Hellwig
2020-05-18 6:39 ` [PATCH 2/9] blk-mq: remove the bio argument to ->prepare_request Christoph Hellwig
2020-05-18 6:39 ` [PATCH 3/9] blk-mq: simplify the blk_mq_get_request calling convention Christoph Hellwig
2020-05-18 6:39 ` [PATCH 4/9] blk-mq: merge blk_mq_rq_ctx_init into __blk_mq_alloc_request Christoph Hellwig
2020-05-18 6:39 ` [PATCH 5/9] blk-mq: don't set data->ctx and data->hctx in blk_mq_alloc_request_hctx Christoph Hellwig
2020-05-18 8:32 ` Thomas Gleixner
2020-05-18 9:31 ` Ming Lei
2020-05-18 10:42 ` Thomas Gleixner
2020-05-18 11:54 ` Ming Lei
2020-05-18 13:16 ` Christoph Hellwig
2020-05-18 14:11 ` Ming Lei
2020-05-18 16:56 ` Christoph Hellwig [this message]
2020-05-18 18:38 ` Thomas Gleixner
2020-05-18 18:45 ` Christoph Hellwig
2020-05-18 18:59 ` Thomas Gleixner
2020-05-19 1:54 ` Ming Lei
2020-05-19 15:30 ` Christoph Hellwig
2020-05-20 1:18 ` Ming Lei
2020-05-20 3:04 ` Ming Lei
2020-05-20 8:03 ` io_uring vs CPU hotplug, was " Christoph Hellwig
2020-05-20 14:45 ` Jens Axboe
2020-05-20 15:20 ` Jens Axboe
2020-05-20 15:31 ` Christoph Hellwig
2020-05-20 19:41 ` Thomas Gleixner
2020-05-20 20:18 ` Jens Axboe
2020-05-20 22:14 ` Thomas Gleixner
2020-05-20 22:40 ` Jens Axboe
2020-05-21 2:27 ` Ming Lei
2020-05-21 8:13 ` Thomas Gleixner
2020-05-21 9:23 ` Ming Lei
2020-05-21 18:39 ` Thomas Gleixner
2020-05-21 18:45 ` Jens Axboe
2020-05-21 20:00 ` Thomas Gleixner
2020-05-22 1:57 ` Ming Lei
2020-05-18 18:47 ` Thomas Gleixner
2020-05-18 13:18 ` Thomas Gleixner
2020-05-18 6:39 ` [PATCH 6/9] blk-mq: don't set data->ctx and data->hctx in __blk_mq_alloc_request Christoph Hellwig
2020-05-18 6:39 ` [PATCH 7/9] blk-mq: disable preemption during allocating request tag Christoph Hellwig
2020-05-18 6:39 ` [PATCH 8/9] blk-mq: add blk_mq_all_tag_iter Christoph Hellwig
2020-05-18 6:39 ` [PATCH 9/9] blk-mq: drain I/O when all CPUs in a hctx are offline Christoph Hellwig
2020-05-18 8:42 ` John Garry
2020-05-18 9:21 ` Ming Lei
2020-05-18 11:49 ` blk-mq: improvement CPU hotplug (simplified version) v2 John Garry
2020-05-19 15:30 ` Christoph Hellwig
2020-05-19 17:17 ` John Garry
2020-05-20 14:35 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200518165619.GA17465@lst.de \
--to=hch@lst.de \
--cc=bvanassche@acm.org \
--cc=hare@suse.com \
--cc=john.garry@huawei.com \
--cc=linux-block@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).