From: "Matthew Wilcox (Oracle)" <willy@infradead.org>
To: Jesper Dangaard Brouer <hawk@kernel.org>,
Ilias Apalodimas <ilias.apalodimas@linaro.org>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>,
netdev@vger.kernel.org, linux-mm@kvack.org,
Shakeel Butt <shakeelb@google.com>
Subject: [PATCH v2 07/24] page_pool: Convert __page_pool_put_page() to __page_pool_put_netmem()
Date: Thu, 5 Jan 2023 21:46:14 +0000 [thread overview]
Message-ID: <20230105214631.3939268-8-willy@infradead.org> (raw)
In-Reply-To: <20230105214631.3939268-1-willy@infradead.org>
Removes the call to compound_head() hidden in put_page() which
saves 169 bytes of kernel text as __page_pool_put_page() is
inlined twice.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
net/core/page_pool.c | 29 +++++++++++++++++++----------
1 file changed, 19 insertions(+), 10 deletions(-)
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index b606952773a6..8f3f7cc5a2d5 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -558,8 +558,8 @@ static bool page_pool_recycle_in_cache(struct page *page,
* If the page refcnt != 1, then the page will be returned to memory
* subsystem.
*/
-static __always_inline struct page *
-__page_pool_put_page(struct page_pool *pool, struct page *page,
+static __always_inline struct netmem *
+__page_pool_put_netmem(struct page_pool *pool, struct netmem *nmem,
unsigned int dma_sync_size, bool allow_direct)
{
/* This allocator is optimized for the XDP mode that uses
@@ -571,19 +571,20 @@ __page_pool_put_page(struct page_pool *pool, struct page *page,
* page is NOT reusable when allocated when system is under
* some pressure. (page_is_pfmemalloc)
*/
- if (likely(page_ref_count(page) == 1 && !page_is_pfmemalloc(page))) {
- /* Read barrier done in page_ref_count / READ_ONCE */
+ if (likely(netmem_ref_count(nmem) == 1 &&
+ !netmem_is_pfmemalloc(nmem))) {
+ /* Read barrier done in netmem_ref_count / READ_ONCE */
if (pool->p.flags & PP_FLAG_DMA_SYNC_DEV)
- page_pool_dma_sync_for_device(pool, page,
+ page_pool_dma_sync_for_device(pool, netmem_page(nmem),
dma_sync_size);
if (allow_direct && in_serving_softirq() &&
- page_pool_recycle_in_cache(page, pool))
+ page_pool_recycle_in_cache(netmem_page(nmem), pool))
return NULL;
/* Page found as candidate for recycling */
- return page;
+ return nmem;
}
/* Fallback/non-XDP mode: API user have elevated refcnt.
*
@@ -599,13 +600,21 @@ __page_pool_put_page(struct page_pool *pool, struct page *page,
* will be invoking put_page.
*/
recycle_stat_inc(pool, released_refcnt);
- /* Do not replace this with page_pool_return_page() */
- page_pool_release_page(pool, page);
- put_page(page);
+ /* Do not replace this with page_pool_return_netmem() */
+ page_pool_release_netmem(pool, nmem);
+ netmem_put(nmem);
return NULL;
}
+static __always_inline struct page *
+__page_pool_put_page(struct page_pool *pool, struct page *page,
+ unsigned int dma_sync_size, bool allow_direct)
+{
+ return netmem_page(__page_pool_put_netmem(pool, page_netmem(page),
+ dma_sync_size, allow_direct));
+}
+
void page_pool_put_defragged_page(struct page_pool *pool, struct page *page,
unsigned int dma_sync_size, bool allow_direct)
{
--
2.35.1
next prev parent reply other threads:[~2023-01-05 21:49 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-05 21:46 [PATCH v2 00/24] Split netmem from struct page Matthew Wilcox (Oracle)
2023-01-05 21:46 ` [PATCH v2 01/24] netmem: Create new type Matthew Wilcox (Oracle)
2023-01-06 13:07 ` Jesper Dangaard Brouer
2023-01-09 17:20 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 02/24] netmem: Add utility functions Matthew Wilcox (Oracle)
2023-01-06 2:24 ` kernel test robot
2023-01-06 20:35 ` Matthew Wilcox
2023-01-06 13:35 ` Jesper Dangaard Brouer
2023-01-05 21:46 ` [PATCH v2 03/24] page_pool: Add netmem_set_dma_addr() and netmem_get_dma_addr() Matthew Wilcox (Oracle)
2023-01-06 13:43 ` Jesper Dangaard Brouer
2023-01-09 17:30 ` Ilias Apalodimas
2023-01-10 9:17 ` Ilias Apalodimas
2023-01-10 18:16 ` Matthew Wilcox
2023-01-10 18:15 ` Matthew Wilcox
2023-01-05 21:46 ` [PATCH v2 04/24] page_pool: Convert page_pool_release_page() to page_pool_release_netmem() Matthew Wilcox (Oracle)
2023-01-06 13:46 ` Jesper Dangaard Brouer
2023-01-10 9:28 ` Ilias Apalodimas
2023-01-10 18:47 ` Matthew Wilcox
2023-01-11 13:56 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 05/24] page_pool: Start using netmem in allocation path Matthew Wilcox (Oracle)
2023-01-06 2:34 ` kernel test robot
2023-01-06 13:59 ` Jesper Dangaard Brouer
2023-01-06 15:36 ` Matthew Wilcox
2023-01-10 9:30 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 06/24] page_pool: Convert page_pool_return_page() to page_pool_return_netmem() Matthew Wilcox (Oracle)
2023-01-06 14:10 ` Jesper Dangaard Brouer
2023-01-10 9:39 ` Ilias Apalodimas
2023-01-05 21:46 ` Matthew Wilcox (Oracle) [this message]
2023-01-06 14:14 ` [PATCH v2 07/24] page_pool: Convert __page_pool_put_page() to __page_pool_put_netmem() Jesper Dangaard Brouer
2023-01-10 9:47 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 08/24] page_pool: Convert pp_alloc_cache to contain netmem Matthew Wilcox (Oracle)
2023-01-06 14:18 ` Jesper Dangaard Brouer
2023-01-10 9:58 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 09/24] page_pool: Convert page_pool_defrag_page() to page_pool_defrag_netmem() Matthew Wilcox (Oracle)
2023-01-06 14:29 ` Jesper Dangaard Brouer
2023-01-10 10:27 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 10/24] page_pool: Convert page_pool_put_defragged_page() to netmem Matthew Wilcox (Oracle)
2023-01-06 14:32 ` Jesper Dangaard Brouer
2023-01-10 10:36 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 11/24] page_pool: Convert page_pool_empty_ring() to use netmem Matthew Wilcox (Oracle)
2023-01-06 15:22 ` Jesper Dangaard Brouer
2023-01-10 10:38 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 12/24] page_pool: Convert page_pool_alloc_pages() to page_pool_alloc_netmem() Matthew Wilcox (Oracle)
2023-01-06 15:27 ` Jesper Dangaard Brouer
2023-01-10 10:45 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 13/24] page_pool: Convert page_pool_dma_sync_for_device() to take a netmem Matthew Wilcox (Oracle)
2023-01-06 15:28 ` Jesper Dangaard Brouer
2023-01-10 10:47 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 14/24] page_pool: Convert page_pool_recycle_in_cache() to netmem Matthew Wilcox (Oracle)
2023-01-06 15:29 ` Jesper Dangaard Brouer
2023-01-10 10:48 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 15/24] page_pool: Remove page_pool_defrag_page() Matthew Wilcox (Oracle)
2023-01-06 15:29 ` Jesper Dangaard Brouer
2023-01-10 9:47 ` Ilias Apalodimas
2023-01-10 22:00 ` Matthew Wilcox
2023-01-11 13:58 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 16/24] page_pool: Use netmem in page_pool_drain_frag() Matthew Wilcox (Oracle)
2023-01-06 15:30 ` Jesper Dangaard Brouer
2023-01-10 11:00 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 17/24] page_pool: Convert page_pool_return_skb_page() to use netmem Matthew Wilcox (Oracle)
2023-01-06 15:49 ` Jesper Dangaard Brouer
2023-01-06 16:53 ` Matthew Wilcox
2023-01-06 20:16 ` Jesper Dangaard Brouer
2023-01-09 18:36 ` Matthew Wilcox
2023-01-10 10:04 ` Jesper Dangaard Brouer
2023-01-05 21:46 ` [PATCH v2 18/24] page_pool: Convert frag_page to frag_nmem Matthew Wilcox (Oracle)
2023-01-06 15:51 ` Jesper Dangaard Brouer
2023-01-10 11:36 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 19/24] xdp: Convert to netmem Matthew Wilcox (Oracle)
2023-01-06 15:53 ` Jesper Dangaard Brouer
2023-01-10 11:50 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 20/24] mm: Remove page pool members from struct page Matthew Wilcox (Oracle)
2023-01-06 15:56 ` Jesper Dangaard Brouer
2023-01-10 11:51 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 21/24] page_pool: Pass a netmem to init_callback() Matthew Wilcox (Oracle)
2023-01-06 16:02 ` Jesper Dangaard Brouer
2023-01-10 11:32 ` Ilias Apalodimas
2023-01-05 21:46 ` [PATCH v2 22/24] net: Add support for netmem in skb_frag Matthew Wilcox (Oracle)
2023-01-05 21:46 ` [PATCH v2 23/24] mvneta: Convert to netmem Matthew Wilcox (Oracle)
2023-01-05 21:46 ` [PATCH v2 24/24] mlx5: " Matthew Wilcox (Oracle)
2023-01-06 16:31 ` Jesper Dangaard Brouer
2023-01-09 11:46 ` Tariq Toukan
2023-01-09 12:27 ` Tariq Toukan
2023-01-06 1:20 ` [PATCH v2 00/24] Split netmem from struct page Jesse Brandeburg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230105214631.3939268-8-willy@infradead.org \
--to=willy@infradead.org \
--cc=hawk@kernel.org \
--cc=ilias.apalodimas@linaro.org \
--cc=linux-mm@kvack.org \
--cc=netdev@vger.kernel.org \
--cc=shakeelb@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).