From: Sebastian Andrzej Siewior <bigeasy@linutronix.de> To: David Runge <dave@sleepmap.de> Cc: linux-rt-users@vger.kernel.org, Jens Axboe <axboe@kernel.dk>, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>, Thomas Gleixner <tglx@linutronix.de>, Daniel Wagner <dwagner@suse.de> Subject: [PATCH RFC] blk-mq: Don't IPI requests on PREEMPT_RT Date: Fri, 23 Oct 2020 13:04:00 +0200 Message-ID: <20201023110400.bx3uzsb7xy5jtsea@linutronix.de> (raw) In-Reply-To: <20201021175059.GA4989@hmbx> blk_mq_complete_request_remote() will dispatch request completion to another CPU via IPI if the CPU belongs to a different cache domain. This breaks on PREEMPT_RT because the IPI function will complete the request in IRQ context which includes acquiring spinlock_t typed locks. Completing the IPI request in softirq on the remote CPU is probably less efficient because it would require to wake ksoftirqd for this task (which runs at SCHED_OTHER). Ignoring the IPI request and completing the request locally is probably the best option. It be completed either in the IRQ-thread or at the end of the routine in softirq context. Let blk_mq_complete_need_ipi() return that there is no need for IPI on PREEMPT_RT. Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- block/blk-mq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index e37aa31332b70..99d2fb51e0e84 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -647,7 +647,7 @@ static inline bool blk_mq_complete_need_ipi(struct request *rq) { int cpu = raw_smp_processor_id(); - if (!IS_ENABLED(CONFIG_SMP) || + if (!IS_ENABLED(CONFIG_SMP) || IS_ENABLED(CONFIG_PREEMPT_RT) || !test_bit(QUEUE_FLAG_SAME_COMP, &rq->q->queue_flags)) return false; -- 2.28.0
next prev parent reply index Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-10-21 17:50 5.9.1-rt18: issues with Firewire card on AMD hardware David Runge 2020-10-23 11:04 ` Sebastian Andrzej Siewior [this message] 2020-10-23 11:21 ` [PATCH RFC] blk-mq: Don't IPI requests on PREEMPT_RT Christoph Hellwig 2020-10-23 13:52 ` Sebastian Andrzej Siewior 2020-10-27 9:26 ` Christoph Hellwig 2020-10-27 10:11 ` Sebastian Andrzej Siewior 2020-10-27 16:07 ` Christoph Hellwig 2020-10-27 17:05 ` Thomas Gleixner 2020-10-27 17:23 ` Christoph Hellwig 2020-10-27 17:59 ` Sebastian Andrzej Siewior 2020-10-27 20:58 ` Sebastian Andrzej Siewior 2020-10-28 6:56 ` Christoph Hellwig 2020-10-28 14:12 ` [PATCH 1/3] blk-mq: Don't complete on a remote CPU in force threaded mode Sebastian Andrzej Siewior 2020-10-28 14:12 ` [PATCH 2/3] blk-mq: Always complete remote completions requests in softirq Sebastian Andrzej Siewior 2020-10-28 14:12 ` [PATCH 3/3] blk-mq: Use llist_head for blk_cpu_done Sebastian Andrzej Siewior 2020-10-28 14:44 ` Christoph Hellwig 2020-10-28 14:47 ` Sebastian Andrzej Siewior 2020-10-29 13:12 ` Sebastian Andrzej Siewior 2020-10-29 14:05 ` Christoph Hellwig 2020-10-29 14:56 ` Sebastian Andrzej Siewior 2020-10-29 14:57 ` Christoph Hellwig 2020-10-29 20:03 ` Sagi Grimberg 2020-10-29 21:01 ` Sebastian Andrzej Siewior 2020-10-29 21:07 ` Sagi Grimberg 2020-10-31 10:41 ` Sebastian Andrzej Siewior 2020-10-31 15:00 ` Jens Axboe 2020-10-31 15:01 ` Jens Axboe 2020-10-31 18:09 ` Christoph Hellwig 2020-11-02 9:55 ` Sebastian Andrzej Siewior 2020-11-02 18:12 ` Christoph Hellwig 2020-11-04 19:15 ` Sagi Grimberg 2020-11-06 15:23 ` Sebastian Andrzej Siewior 2020-10-28 10:04 ` [PATCH RFC] blk-mq: Don't IPI requests on PREEMPT_RT Peter Zijlstra 2020-10-26 0:37 ` 5.9.1-rt18: issues with Firewire card on AMD hardware David Runge
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20201023110400.bx3uzsb7xy5jtsea@linutronix.de \ --to=bigeasy@linutronix.de \ --cc=axboe@kernel.dk \ --cc=dave@sleepmap.de \ --cc=dwagner@suse.de \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-rt-users@vger.kernel.org \ --cc=peterz@infradead.org \ --cc=tglx@linutronix.de \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-rt-users Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-rt-users/0 linux-rt-users/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-rt-users linux-rt-users/ https://lore.kernel.org/linux-rt-users \ linux-rt-users@vger.kernel.org public-inbox-index linux-rt-users Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rt-users AGPL code for this site: git clone https://public-inbox.org/public-inbox.git