* [PATCH] blk-stat: Optimise blk_stat_add()
@ 2019-10-07 21:16 Pavel Begunkov (Silence)
2019-10-08 3:17 ` Jens Axboe
0 siblings, 1 reply; 2+ messages in thread
From: Pavel Begunkov (Silence) @ 2019-10-07 21:16 UTC (permalink / raw)
To: Jens Axboe, linux-block, linux-kernel; +Cc: Pavel Begunkov
From: Pavel Begunkov <asml.silence@gmail.com>
blk_stat_add() calls {get,put}_cpu_ptr() in a loop, which entails
overhead of disabling/enabling preemption. The loop is under RCU
(i.e.short) anyway, so do get_cpu() in advance.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
block/blk-stat.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/block/blk-stat.c b/block/blk-stat.c
index d892ad2cb938..4239954b0bce 100644
--- a/block/blk-stat.c
+++ b/block/blk-stat.c
@@ -75,7 +75,7 @@ void blk_stat_add(struct request *rq, u64 now)
struct request_queue *q = rq->q;
struct blk_stat_callback *cb;
struct blk_rq_stat_staging *stat;
- int bucket;
+ int bucket, cpu;
u64 value;
value = (now >= rq->io_start_time_ns) ? now - rq->io_start_time_ns : 0;
@@ -83,6 +83,7 @@ void blk_stat_add(struct request *rq, u64 now)
blk_throtl_stat_add(rq, value);
rcu_read_lock();
+ cpu = get_cpu();
list_for_each_entry_rcu(cb, &q->stats->callbacks, list) {
if (!blk_stat_is_active(cb))
continue;
@@ -91,10 +92,10 @@ void blk_stat_add(struct request *rq, u64 now)
if (bucket < 0)
continue;
- stat = &get_cpu_ptr(cb->cpu_stat)[bucket];
+ stat = &per_cpu_ptr(cb->cpu_stat, cpu)[bucket];
blk_rq_stat_add(stat, value);
- put_cpu_ptr(cb->cpu_stat);
}
+ put_cpu();
rcu_read_unlock();
}
--
2.23.0
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH] blk-stat: Optimise blk_stat_add()
2019-10-07 21:16 [PATCH] blk-stat: Optimise blk_stat_add() Pavel Begunkov (Silence)
@ 2019-10-08 3:17 ` Jens Axboe
0 siblings, 0 replies; 2+ messages in thread
From: Jens Axboe @ 2019-10-08 3:17 UTC (permalink / raw)
To: Pavel Begunkov (Silence), linux-block, linux-kernel
On 10/7/19 3:16 PM, Pavel Begunkov (Silence) wrote:
> From: Pavel Begunkov <asml.silence@gmail.com>
>
> blk_stat_add() calls {get,put}_cpu_ptr() in a loop, which entails
> overhead of disabling/enabling preemption. The loop is under RCU
> (i.e.short) anyway, so do get_cpu() in advance.
Looks good, applied for 5.5.
--
Jens Axboe
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2019-10-08 3:18 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-07 21:16 [PATCH] blk-stat: Optimise blk_stat_add() Pavel Begunkov (Silence)
2019-10-08 3:17 ` Jens Axboe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).