From: Pavel Begunkov <asml.silence@gmail.com>
To: netdev@vger.kernel.org, "David S . Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>
Cc: Eric Dumazet <edumazet@google.com>, Wei Liu <wei.liu@kernel.org>,
Paul Durrant <paul@xen.org>,
Pavel Begunkov <asml.silence@gmail.com>
Subject: [PATCH net-next 08/27] skbuff: optimise alloc_skb_with_frags()
Date: Sun, 3 Apr 2022 14:06:20 +0100 [thread overview]
Message-ID: <6011c226ac8c8d3c178675dfc53ca1c5d84c1095.1648981571.git.asml.silence@gmail.com> (raw)
In-Reply-To: <cover.1648981570.git.asml.silence@gmail.com>
Some users of alloc_skb_with_frags() including UDP pass zero datalen,
Extract and inline the pure skb alloc part of it. We also save on
needlessly pre-setting errcode ptr and with other small refactorings.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
include/linux/skbuff.h | 41 ++++++++++++++++++++++++++++++++++++-----
net/core/skbuff.c | 31 ++++++++++++-------------------
2 files changed, 48 insertions(+), 24 deletions(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 410850832b6a..ebc4ad36c3a2 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -1300,11 +1300,42 @@ static inline struct sk_buff *alloc_skb(unsigned int size,
return __alloc_skb(size, priority, 0, NUMA_NO_NODE);
}
-struct sk_buff *alloc_skb_with_frags(unsigned long header_len,
- unsigned long data_len,
- int max_page_order,
- int *errcode,
- gfp_t gfp_mask);
+struct sk_buff *alloc_skb_frags(struct sk_buff *skb,
+ unsigned long data_len,
+ int max_page_order,
+ int *errcode,
+ gfp_t gfp_mask);
+
+/**
+ * alloc_skb_with_frags - allocate skb with page frags
+ *
+ * @header_len: size of linear part
+ * @data_len: needed length in frags
+ * @max_page_order: max page order desired.
+ * @errcode: pointer to error code if any
+ * @gfp_mask: allocation mask
+ *
+ * This can be used to allocate a paged skb, given a maximal order for frags.
+ */
+static inline struct sk_buff *alloc_skb_with_frags(unsigned long header_len,
+ unsigned long data_len,
+ int max_page_order,
+ int *errcode,
+ gfp_t gfp_mask)
+{
+ struct sk_buff *skb;
+
+ skb = alloc_skb(header_len, gfp_mask);
+ if (unlikely(!skb)) {
+ *errcode = -ENOBUFS;
+ return NULL;
+ }
+
+ if (!data_len)
+ return skb;
+ return alloc_skb_frags(skb, data_len, max_page_order, errcode, gfp_mask);
+}
+
struct sk_buff *alloc_skb_for_msg(struct sk_buff *first);
/* Layout of fast clones : [skb1][skb2][fclone_ref] */
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index f7842bfdd7ae..2c787d964a60 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -5955,40 +5955,32 @@ int skb_mpls_dec_ttl(struct sk_buff *skb)
EXPORT_SYMBOL_GPL(skb_mpls_dec_ttl);
/**
- * alloc_skb_with_frags - allocate skb with page frags
+ * alloc_skb_frags - allocate page frags for skb
*
- * @header_len: size of linear part
+ * @skb: buffer
* @data_len: needed length in frags
* @max_page_order: max page order desired.
* @errcode: pointer to error code if any
* @gfp_mask: allocation mask
*
- * This can be used to allocate a paged skb, given a maximal order for frags.
+ * This can be used to allocate pages for skb, given a maximal order for frags.
*/
-struct sk_buff *alloc_skb_with_frags(unsigned long header_len,
- unsigned long data_len,
- int max_page_order,
- int *errcode,
- gfp_t gfp_mask)
+struct sk_buff *alloc_skb_frags(struct sk_buff *skb,
+ unsigned long data_len,
+ int max_page_order,
+ int *errcode,
+ gfp_t gfp_mask)
{
int npages = (data_len + (PAGE_SIZE - 1)) >> PAGE_SHIFT;
unsigned long chunk;
- struct sk_buff *skb;
struct page *page;
int i;
- *errcode = -EMSGSIZE;
/* Note this test could be relaxed, if we succeed to allocate
* high order pages...
*/
- if (npages > MAX_SKB_FRAGS)
- return NULL;
-
- *errcode = -ENOBUFS;
- skb = alloc_skb(header_len, gfp_mask);
- if (!skb)
- return NULL;
-
+ if (unlikely(npages > MAX_SKB_FRAGS))
+ goto failure;
skb->truesize += npages << PAGE_SHIFT;
for (i = 0; npages > 0; i++) {
@@ -6022,9 +6014,10 @@ struct sk_buff *alloc_skb_with_frags(unsigned long header_len,
failure:
kfree_skb(skb);
+ *errcode = -EMSGSIZE;
return NULL;
}
-EXPORT_SYMBOL(alloc_skb_with_frags);
+EXPORT_SYMBOL(alloc_skb_frags);
/* carve out the first off bytes from skb when off < headlen */
static int pskb_carve_inside_header(struct sk_buff *skb, const u32 off,
--
2.35.1
next prev parent reply other threads:[~2022-04-03 13:08 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-03 13:06 [RFC net-next 00/27] net and/or udp optimisations Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 01/27] sock: deduplicate ->sk_wmem_alloc check Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 02/27] sock: optimise sock_def_write_space send refcounting Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 03/27] sock: optimise sock_def_write_space barriers Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 04/27] skbuff: drop zero check from skb_zcopy_set Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 05/27] skbuff: drop null check from skb_zcopy Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 06/27] net: xen: set zc flags only when there is ubuf Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 07/27] skbuff: introduce skb_is_zcopy() Pavel Begunkov
2022-04-03 13:06 ` Pavel Begunkov [this message]
2022-04-03 13:06 ` [PATCH net-next 09/27] net: inline sock_alloc_send_skb Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 10/27] net: inline part of skb_csum_hwoffload_help Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 11/27] net: inline skb_zerocopy_iter_dgram Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 12/27] ipv6: inline ip6_local_out() Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 13/27] ipv6: help __ip6_finish_output() inlining Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 14/27] ipv6: refactor ip6_finish_output2() Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 15/27] net: inline dev_queue_xmit() Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 16/27] ipv6: partially inline fl6_update_dst() Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 17/27] tcp: optimise skb_zerocopy_iter_stream() Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 18/27] net: optimise ipcm6 cookie init Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 19/27] udp/ipv6: refactor udpv6_sendmsg udplite checks Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 20/27] udp/ipv6: move pending section of udpv6_sendmsg Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 21/27] udp/ipv6: prioritise the ip6 path over ip4 checks Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 22/27] udp/ipv6: optimise udpv6_sendmsg() daddr checks Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 23/27] udp/ipv6: optimise out daddr reassignment Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 24/27] udp/ipv6: clean up udpv6_sendmsg's saddr init Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 25/27] ipv6: refactor opts push in __ip6_make_skb() Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 26/27] ipv6: improve opt-less __ip6_make_skb() Pavel Begunkov
2022-04-03 13:06 ` [PATCH net-next 27/27] ipv6: clean up ip6_setup_cork Pavel Begunkov
2022-04-06 9:44 ` [RFC net-next 00/27] net and/or udp optimisations Eric Dumazet
2022-04-11 12:04 ` Pavel Begunkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6011c226ac8c8d3c178675dfc53ca1c5d84c1095.1648981571.git.asml.silence@gmail.com \
--to=asml.silence@gmail.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=paul@xen.org \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).