From: Shay Agroskin <shayagr@amazon.com>
To: David Miller <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>, <netdev@vger.kernel.org>
Cc: Shay Agroskin <shayagr@amazon.com>,
"Woodhouse, David" <dwmw@amazon.com>,
"Machulsky, Zorik" <zorik@amazon.com>,
"Matushevsky, Alexander" <matua@amazon.com>,
Saeed Bshara <saeedb@amazon.com>, "Wilson, Matt" <msw@amazon.com>,
"Liguori, Anthony" <aliguori@amazon.com>,
"Bshara, Nafea" <nafea@amazon.com>,
"Tzalik, Guy" <gtzalik@amazon.com>,
"Belgazal, Netanel" <netanel@amazon.com>,
"Saidi, Ali" <alisaidi@amazon.com>,
"Herrenschmidt, Benjamin" <benh@amazon.com>,
"Kiyanovski, Arthur" <akiyano@amazon.com>,
"Jubran, Samih" <sameehj@amazon.com>,
"Dagan, Noam" <ndagan@amazon.com>
Subject: [RFC Patch v1 2/3] net: ena: update README file with a description about LPC
Date: Tue, 9 Mar 2021 19:10:13 +0200 [thread overview]
Message-ID: <20210309171014.2200020-3-shayagr@amazon.com> (raw)
In-Reply-To: <20210309171014.2200020-1-shayagr@amazon.com>
Add a description for local page cache system to the ENA driver readme
file.
Signed-off-by: Shay Agroskin <shayagr@amazon.com>
---
.../device_drivers/ethernet/amazon/ena.rst | 25 +++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst
index 3561a8a29fd2..d3423a2f472c 100644
--- a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst
+++ b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst
@@ -222,6 +222,31 @@ If the frame length is larger than rx_copybreak, napi_get_frags()
is used, otherwise netdev_alloc_skb_ip_align() is used, the buffer
content is copied (by CPU) to the SKB, and the buffer is recycled.
+Local Page Cache (LPC)
+======================
+ENA Linux driver allows to reduce lock contention and improve CPU usage by
+allocating RX buffers from a page cache rather than from Linux memory system
+(PCP or buddy allocator). The cache is created and binded per RX queue, and
+pages allocated for the queue are stored in the cache (up to cache maximum
+size).
+
+When enabled, LPC cache size is ENA_LPC_DEFAULT_MULTIPLIER * 1024 (2048 by
+default) pages.
+
+The cache usage for each queue can be monitored using ``ethtool -S`` counters. Where:
+
+- *rx_queue#_lpc_warm_up* - number of pages that were allocated and stored in
+ the cache
+- *rx_queue#_lpc_full* - number of pages that were allocated without using the
+ cache because it didn't have free pages
+- *rx_queue#_lpc_wrong_numa* - number of pages from the cache that belong to a
+ different NUMA node than the CPU which runs the NAPI routine. In this case,
+ the driver would try to allocate a new page from the same NUMA node instead
+
+LPC is disabled when using XDP or when using less than 16 queue pairs. Note that
+cache usage might increase the memory footprint of the driver (depending on the
+traffic).
+
Statistics
==========
The user can obtain ENA device and driver statistics using ethtool.
--
2.25.1
next prev parent reply other threads:[~2021-03-09 17:11 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-09 17:10 [RFC Patch v1 0/3] Introduce ENA local page cache Shay Agroskin
2021-03-09 17:10 ` [RFC Patch v1 1/3] net: ena: implement local page cache (LPC) system Shay Agroskin
2021-03-09 17:57 ` Eric Dumazet
2021-03-10 2:04 ` Andrew Lunn
2021-03-11 23:15 ` Saeed Mahameed
2021-03-16 8:23 ` Shay Agroskin
2021-03-16 8:26 ` Shay Agroskin
2021-03-09 17:10 ` Shay Agroskin [this message]
2021-03-09 17:10 ` [RFC Patch v1 3/3] net: ena: support ethtool priv-flags and LPC state change Shay Agroskin
-- strict thread matches above, loose matches on Subject: below --
2021-03-09 17:07 [RFC Patch v1 0/3] Introduce ENA local page cache Shay Agroskin
2021-03-09 17:07 ` [RFC Patch v1 2/3] net: ena: update README file with a description about LPC Shay Agroskin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210309171014.2200020-3-shayagr@amazon.com \
--to=shayagr@amazon.com \
--cc=akiyano@amazon.com \
--cc=aliguori@amazon.com \
--cc=alisaidi@amazon.com \
--cc=benh@amazon.com \
--cc=davem@davemloft.net \
--cc=dwmw@amazon.com \
--cc=gtzalik@amazon.com \
--cc=kuba@kernel.org \
--cc=matua@amazon.com \
--cc=msw@amazon.com \
--cc=nafea@amazon.com \
--cc=ndagan@amazon.com \
--cc=netanel@amazon.com \
--cc=netdev@vger.kernel.org \
--cc=saeedb@amazon.com \
--cc=sameehj@amazon.com \
--cc=zorik@amazon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).