From: Omar Sandoval <osandov@osandov.com>
To: Jens Axboe <axboe@fb.com>, linux-block@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com,
Alexei Starovoitov <ast@fb.com>
Subject: [PATCH v4 6/6] sbitmap: re-initialize allocation hints after resize
Date: Sat, 17 Sep 2016 01:28:26 -0700 [thread overview]
Message-ID: <bdfa41b6f22207f650ab743e87fec2cd7fd8b327.1474100040.git.osandov@fb.com> (raw)
In-Reply-To: <cover.1474100040.git.osandov@fb.com>
In-Reply-To: <cover.1474100040.git.osandov@fb.com>
From: Omar Sandoval <osandov@fb.com>
After a struct sbitmap_queue is resized smaller, the allocation hints
may still be set to bits beyond the new depth of the bitmap. This means
that, for example, if the number of blk-mq tags is reduced through
sysfs, more requests than the nominal queue depth may be in flight.
It's tempting to fix this at resize time by doing a one-time
reinitialization of the hints, but this can race with
__sbitmap_queue_get() updating the hint. Instead, check the hint before
we use it. This caused no measurable performance difference in my
synthetic benchmarks.
Signed-off-by: Omar Sandoval <osandov@fb.com>
---
lib/sbitmap.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/lib/sbitmap.c b/lib/sbitmap.c
index 928b82a..f736c52 100644
--- a/lib/sbitmap.c
+++ b/lib/sbitmap.c
@@ -246,10 +246,15 @@ EXPORT_SYMBOL_GPL(sbitmap_queue_resize);
int __sbitmap_queue_get(struct sbitmap_queue *sbq)
{
- unsigned int hint;
+ unsigned int hint, depth;
int nr;
hint = this_cpu_read(*sbq->alloc_hint);
+ depth = READ_ONCE(sbq->sb.depth);
+ if (unlikely(hint >= depth)) {
+ hint = depth ? prandom_u32() % depth : 0;
+ this_cpu_write(*sbq->alloc_hint, hint);
+ }
nr = sbitmap_get(&sbq->sb, hint, sbq->round_robin);
if (nr == -1) {
@@ -258,7 +263,7 @@ int __sbitmap_queue_get(struct sbitmap_queue *sbq)
} else if (nr == hint || unlikely(sbq->round_robin)) {
/* Only update the hint if we used it. */
hint = nr + 1;
- if (hint >= sbq->sb.depth - 1)
+ if (hint >= depth - 1)
hint = 0;
this_cpu_write(*sbq->alloc_hint, hint);
}
--
2.9.3
next prev parent reply other threads:[~2016-09-17 8:29 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-17 8:28 [PATCH v4 0/6] blk-mq: generalization and bug fixes for tag allocation Omar Sandoval
2016-09-17 8:28 ` [PATCH v4 1/6] blk-mq: abstract tag allocation out into sbitmap library Omar Sandoval
2016-09-17 8:28 ` [PATCH v4 2/6] sbitmap: allocate wait queues on a specific node Omar Sandoval
2016-09-17 8:28 ` [PATCH v4 3/6] sbitmap: push per-cpu last_tag into sbitmap_queue Omar Sandoval
2016-09-17 8:28 ` [PATCH v4 4/6] sbitmap: push alloc policy " Omar Sandoval
2016-09-17 8:28 ` [PATCH v4 5/6] sbitmap: randomize initial alloc_hint values Omar Sandoval
2016-09-17 8:28 ` Omar Sandoval [this message]
2016-09-17 14:42 ` [PATCH v4 0/6] blk-mq: generalization and bug fixes for tag allocation Jens Axboe
2016-09-17 19:20 ` [PATCH] sbitmap: don't update the allocation hint on clear after resize Omar Sandoval
2016-09-17 19:39 ` Jens Axboe
2016-09-17 19:49 ` Omar Sandoval
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bdfa41b6f22207f650ab743e87fec2cd7fd8b327.1474100040.git.osandov@fb.com \
--to=osandov@osandov.com \
--cc=ast@fb.com \
--cc=axboe@fb.com \
--cc=kernel-team@fb.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).