* [PATCH for-next 0/4] io_uring/zc fix and improvements
@ 2022-07-25 9:52 Pavel Begunkov
2022-07-25 9:52 ` [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types Pavel Begunkov
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw)
To: io-uring; +Cc: Jens Axboe, asml.silence
It mainly fixes net/zc memory accounting error handling and then makes
it more consistent with registered buffers mem accounting.
Pavel Begunkov (4):
io_uring/net: improve io_get_notif_slot types
io_uring/net: checks errors of zc mem accounting
io_uring/net: make page accounting more consistent
io_uring/net: use unsigned for flags
io_uring/net.c | 8 +++++---
io_uring/notif.c | 9 ++++-----
io_uring/notif.h | 21 ++++++++++++++++++++-
io_uring/rsrc.c | 12 ++++--------
io_uring/rsrc.h | 9 +++++++++
5 files changed, 42 insertions(+), 17 deletions(-)
--
2.37.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov @ 2022-07-25 9:52 ` Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 2/4] io_uring/net: checks errors of zc mem accounting Pavel Begunkov ` (3 subsequent siblings) 4 siblings, 0 replies; 6+ messages in thread From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw) To: io-uring; +Cc: Jens Axboe, asml.silence Don't use signed int for slot indexing. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- io_uring/notif.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/io_uring/notif.h b/io_uring/notif.h index 6cd73d7b965b..3e05d2cecb6f 100644 --- a/io_uring/notif.h +++ b/io_uring/notif.h @@ -67,7 +67,7 @@ static inline struct io_notif *io_get_notif(struct io_ring_ctx *ctx, } static inline struct io_notif_slot *io_get_notif_slot(struct io_ring_ctx *ctx, - int idx) + unsigned idx) __must_hold(&ctx->uring_lock) { if (idx >= ctx->nr_notif_slots) -- 2.37.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH for-next 2/4] io_uring/net: checks errors of zc mem accounting 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types Pavel Begunkov @ 2022-07-25 9:52 ` Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 3/4] io_uring/net: make page accounting more consistent Pavel Begunkov ` (2 subsequent siblings) 4 siblings, 0 replies; 6+ messages in thread From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw) To: io-uring; +Cc: Jens Axboe, asml.silence mm_account_pinned_pages() may fail, don't ignore the return value. Fixes: d3b8269075f67 ("io_uring: account locked pages for non-fixed zc") Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- io_uring/net.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/io_uring/net.c b/io_uring/net.c index 62be89837d82..8fb8469c3315 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -985,7 +985,9 @@ int io_sendzc(struct io_kiocb *req, unsigned int issue_flags) &msg.msg_iter); if (unlikely(ret)) return ret; - mm_account_pinned_pages(¬if->uarg.mmp, zc->len); + ret = mm_account_pinned_pages(¬if->uarg.mmp, zc->len); + if (unlikely(ret)) + return ret; } if (zc->addr) { -- 2.37.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH for-next 3/4] io_uring/net: make page accounting more consistent 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 2/4] io_uring/net: checks errors of zc mem accounting Pavel Begunkov @ 2022-07-25 9:52 ` Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 4/4] io_uring/net: use unsigned for flags Pavel Begunkov 2022-07-25 15:49 ` [PATCH for-next 0/4] io_uring/zc fix and improvements Jens Axboe 4 siblings, 0 replies; 6+ messages in thread From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw) To: io-uring; +Cc: Jens Axboe, asml.silence Make network page accounting more consistent with how buffer registration is working, i.e. account all memory to ctx->user. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- io_uring/net.c | 2 +- io_uring/notif.c | 9 ++++----- io_uring/notif.h | 19 +++++++++++++++++++ io_uring/rsrc.c | 12 ++++-------- io_uring/rsrc.h | 9 +++++++++ 5 files changed, 37 insertions(+), 14 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index 8fb8469c3315..c13d971c7826 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -985,7 +985,7 @@ int io_sendzc(struct io_kiocb *req, unsigned int issue_flags) &msg.msg_iter); if (unlikely(ret)) return ret; - ret = mm_account_pinned_pages(¬if->uarg.mmp, zc->len); + ret = io_notif_account_mem(notif, zc->len); if (unlikely(ret)) return ret; } diff --git a/io_uring/notif.c b/io_uring/notif.c index a93887451bbb..e986a0ed958c 100644 --- a/io_uring/notif.c +++ b/io_uring/notif.c @@ -14,12 +14,10 @@ static void __io_notif_complete_tw(struct callback_head *cb) struct io_notif *notif = container_of(cb, struct io_notif, task_work); struct io_rsrc_node *rsrc_node = notif->rsrc_node; struct io_ring_ctx *ctx = notif->ctx; - struct mmpin *mmp = ¬if->uarg.mmp; - if (mmp->user) { - atomic_long_sub(mmp->num_pg, &mmp->user->locked_vm); - free_uid(mmp->user); - mmp->user = NULL; + if (notif->account_pages && ctx->user) { + __io_unaccount_mem(ctx->user, notif->account_pages); + notif->account_pages = 0; } if (likely(notif->task)) { io_put_task(notif->task, 1); @@ -121,6 +119,7 @@ struct io_notif *io_alloc_notif(struct io_ring_ctx *ctx, notif->ctx = ctx; notif->uarg.flags = SKBFL_ZEROCOPY_FRAG | SKBFL_DONT_ORPHAN; notif->uarg.callback = io_uring_tx_zerocopy_callback; + notif->account_pages = 0; } notif->seq = slot->seq++; diff --git a/io_uring/notif.h b/io_uring/notif.h index 3e05d2cecb6f..d6f366b1518b 100644 --- a/io_uring/notif.h +++ b/io_uring/notif.h @@ -5,6 +5,8 @@ #include <net/sock.h> #include <linux/nospec.h> +#include "rsrc.h" + #define IO_NOTIF_SPLICE_BATCH 32 #define IORING_MAX_NOTIF_SLOTS (1U << 10) @@ -23,6 +25,8 @@ struct io_notif { /* hook into ctx->notif_list and ctx->notif_list_locked */ struct list_head cache_node; + unsigned long account_pages; + union { struct callback_head task_work; struct work_struct commit_work; @@ -85,3 +89,18 @@ static inline void io_notif_slot_flush_submit(struct io_notif_slot *slot, } io_notif_slot_flush(slot); } + +static inline int io_notif_account_mem(struct io_notif *notif, unsigned len) +{ + struct io_ring_ctx *ctx = notif->ctx; + unsigned nr_pages = (len >> PAGE_SHIFT) + 2; + int ret; + + if (ctx->user) { + ret = __io_account_mem(ctx->user, nr_pages); + if (ret) + return ret; + notif->account_pages += nr_pages; + } + return 0; +} diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 9165fdf64269..59704b9ac537 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -44,17 +44,13 @@ void io_rsrc_refs_drop(struct io_ring_ctx *ctx) } } -static inline void __io_unaccount_mem(struct user_struct *user, - unsigned long nr_pages) -{ - atomic_long_sub(nr_pages, &user->locked_vm); -} - -static inline int __io_account_mem(struct user_struct *user, - unsigned long nr_pages) +int __io_account_mem(struct user_struct *user, unsigned long nr_pages) { unsigned long page_limit, cur_pages, new_pages; + if (!nr_pages) + return 0; + /* Don't allow more pages than we can safely lock */ page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; diff --git a/io_uring/rsrc.h b/io_uring/rsrc.h index 21813a23215f..f3a9a177941f 100644 --- a/io_uring/rsrc.h +++ b/io_uring/rsrc.h @@ -169,4 +169,13 @@ static inline u64 *io_get_tag_slot(struct io_rsrc_data *data, unsigned int idx) int io_rsrc_update(struct io_kiocb *req, unsigned int issue_flags); int io_rsrc_update_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe); + +int __io_account_mem(struct user_struct *user, unsigned long nr_pages); + +static inline void __io_unaccount_mem(struct user_struct *user, + unsigned long nr_pages) +{ + atomic_long_sub(nr_pages, &user->locked_vm); +} + #endif -- 2.37.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH for-next 4/4] io_uring/net: use unsigned for flags 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov ` (2 preceding siblings ...) 2022-07-25 9:52 ` [PATCH for-next 3/4] io_uring/net: make page accounting more consistent Pavel Begunkov @ 2022-07-25 9:52 ` Pavel Begunkov 2022-07-25 15:49 ` [PATCH for-next 0/4] io_uring/zc fix and improvements Jens Axboe 4 siblings, 0 replies; 6+ messages in thread From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw) To: io-uring; +Cc: Jens Axboe, asml.silence Use unsigned int type for msg flags. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- io_uring/net.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index c13d971c7826..8276b9537194 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -55,10 +55,10 @@ struct io_sr_msg { struct user_msghdr __user *umsg; void __user *buf; }; - int msg_flags; + unsigned msg_flags; + unsigned flags; size_t len; size_t done_io; - unsigned int flags; }; struct io_sendzc { -- 2.37.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH for-next 0/4] io_uring/zc fix and improvements 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov ` (3 preceding siblings ...) 2022-07-25 9:52 ` [PATCH for-next 4/4] io_uring/net: use unsigned for flags Pavel Begunkov @ 2022-07-25 15:49 ` Jens Axboe 4 siblings, 0 replies; 6+ messages in thread From: Jens Axboe @ 2022-07-25 15:49 UTC (permalink / raw) To: asml.silence, io-uring On Mon, 25 Jul 2022 10:52:02 +0100, Pavel Begunkov wrote: > It mainly fixes net/zc memory accounting error handling and then makes > it more consistent with registered buffers mem accounting. > > Pavel Begunkov (4): > io_uring/net: improve io_get_notif_slot types > io_uring/net: checks errors of zc mem accounting > io_uring/net: make page accounting more consistent > io_uring/net: use unsigned for flags > > [...] Applied, thanks! [1/4] io_uring/net: improve io_get_notif_slot types commit: cb309ae49da7a7c28f0051deea13970291134fac [2/4] io_uring/net: checks errors of zc mem accounting commit: 2e32ba5607ee2b668baa8831dd74f7cc867a1f7e [3/4] io_uring/net: make page accounting more consistent commit: 6a9ce66f4d0872861e0bbc67eee6ce5dca5dd886 [4/4] io_uring/net: use unsigned for flags commit: 293402e564a7391f38541c7694e736f5fde20aea Best regards, -- Jens Axboe ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-07-25 15:49 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 2/4] io_uring/net: checks errors of zc mem accounting Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 3/4] io_uring/net: make page accounting more consistent Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 4/4] io_uring/net: use unsigned for flags Pavel Begunkov 2022-07-25 15:49 ` [PATCH for-next 0/4] io_uring/zc fix and improvements Jens Axboe
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).