From: Jens Axboe <axboe@kernel.dk>
To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org
Cc: viro@ZenIV.linux.org.uk, Jens Axboe <axboe@kernel.dk>
Subject: [PATCH 6/7] io_uring: retry bulk slab allocs as single allocs
Date: Fri, 15 Mar 2019 08:54:41 -0600 [thread overview]
Message-ID: <20190315145442.21127-7-axboe@kernel.dk> (raw)
In-Reply-To: <20190315145442.21127-1-axboe@kernel.dk>
I've seen cases where bulk alloc fails, since the bulk alloc API
is all-or-nothing - either we get the number we ask for, or it
returns 0 as number of entries.
If we fail a batch bulk alloc, retry a "normal" kmem_cache_alloc()
and just use that instead of failing with -EAGAIN.
While in there, ensure we use GFP_KERNEL. That was an oversight in
the original code, when we switched away from GFP_ATOMIC.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
fs/io_uring.c | 22 ++++++++++++++++------
1 file changed, 16 insertions(+), 6 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 443ca5615554..5be6e4f99a9e 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -399,13 +399,14 @@ static void io_ring_drop_ctx_refs(struct io_ring_ctx *ctx, unsigned refs)
static struct io_kiocb *io_get_req(struct io_ring_ctx *ctx,
struct io_submit_state *state)
{
+ gfp_t gfp = GFP_KERNEL | __GFP_NOWARN;
struct io_kiocb *req;
if (!percpu_ref_tryget(&ctx->refs))
return NULL;
if (!state) {
- req = kmem_cache_alloc(req_cachep, __GFP_NOWARN);
+ req = kmem_cache_alloc(req_cachep, gfp);
if (unlikely(!req))
goto out;
} else if (!state->free_reqs) {
@@ -413,13 +414,22 @@ static struct io_kiocb *io_get_req(struct io_ring_ctx *ctx,
int ret;
sz = min_t(size_t, state->ios_left, ARRAY_SIZE(state->reqs));
- ret = kmem_cache_alloc_bulk(req_cachep, __GFP_NOWARN, sz,
- state->reqs);
- if (unlikely(ret <= 0))
- goto out;
+ ret = kmem_cache_alloc_bulk(req_cachep, gfp, sz, state->reqs);
+
+ /*
+ * Bulk alloc is all-or-nothing. If we fail to get a batch,
+ * retry single alloc to be on the safe side.
+ */
+ if (ret <= 0) {
+ req = kmem_cache_alloc(req_cachep, gfp);
+ if (unlikely(!req))
+ goto out;
+ ret = 1;
+ } else {
+ req = state->reqs[0];
+ }
state->free_reqs = ret - 1;
state->cur_req = 1;
- req = state->reqs[0];
} else {
req = state->reqs[state->cur_req];
state->free_reqs--;
--
2.17.1
next prev parent reply other threads:[~2019-03-15 14:55 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-15 14:54 [PATCHSET v2 0/7] io_uring fixes Jens Axboe
2019-03-15 14:54 ` [PATCH 1/7] io_uring: use regular request ref counts Jens Axboe
2019-03-15 14:54 ` [PATCH 2/7] io_uring: make io_read/write return an integer Jens Axboe
2019-03-15 14:54 ` [PATCH 3/7] io_uring: add prepped flag Jens Axboe
2019-03-15 14:54 ` [PATCH 4/7] io_uring: fix fget/fput handling Jens Axboe
2019-03-15 14:54 ` [PATCH 5/7] io_uring: fix poll races Jens Axboe
2019-03-15 14:54 ` Jens Axboe [this message]
2019-03-15 14:54 ` [PATCH 7/7] io_uring: mark me as the maintainer Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190315145442.21127-7-axboe@kernel.dk \
--to=axboe@kernel.dk \
--cc=linux-block@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=viro@ZenIV.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).