* [PATCH] net: ena: Do not waste napi skb cache
@ 2022-01-23 11:56 Hyeonggon Yoo
2022-01-24 8:57 ` Julian Wiedmann
2022-01-26 13:16 ` [PATCH v2] " Hyeonggon Yoo
0 siblings, 2 replies; 12+ messages in thread
From: Hyeonggon Yoo @ 2022-01-23 11:56 UTC (permalink / raw)
To: netdev
Cc: Shay Agroskin, Arthur Kiyanovski, David Arinzon, Noam Dagan,
Saeed Bishara, David S . Miller, Jakub Kicinski, Daniel Borkmann,
Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi, linux-kernel,
Hyeonggon Yoo
By profiling, discovered that ena device driver allocates skb by
build_skb() and frees by napi_skb_cache_put(). Because the driver
does not use napi skb cache in allocation path, napi skb cache is
periodically filled and flushed. This is waste of napi skb cache.
As ena_alloc_skb() is called only in napi, Use napi_build_skb()
instead of build_skb() to when allocating skb.
This patch was tested on aws a1.metal instance.
Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
---
drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
index c72f0c7ff4aa..2c67fb1703c5 100644
--- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
+++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
@@ -1407,7 +1407,7 @@ static struct sk_buff *ena_alloc_skb(struct ena_ring *rx_ring, void *first_frag)
skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
rx_ring->rx_copybreak);
else
- skb = build_skb(first_frag, ENA_PAGE_SIZE);
+ skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
if (unlikely(!skb)) {
ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 1,
--
2.33.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH] net: ena: Do not waste napi skb cache
2022-01-23 11:56 [PATCH] net: ena: Do not waste napi skb cache Hyeonggon Yoo
@ 2022-01-24 8:57 ` Julian Wiedmann
2022-01-24 9:29 ` Julian Wiedmann
2022-01-26 13:16 ` [PATCH v2] " Hyeonggon Yoo
1 sibling, 1 reply; 12+ messages in thread
From: Julian Wiedmann @ 2022-01-24 8:57 UTC (permalink / raw)
To: Hyeonggon Yoo, netdev
Cc: Shay Agroskin, Arthur Kiyanovski, David Arinzon, Noam Dagan,
Saeed Bishara, David S . Miller, Jakub Kicinski, Daniel Borkmann,
Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi, linux-kernel
On 23.01.22 13:56, Hyeonggon Yoo wrote:
> By profiling, discovered that ena device driver allocates skb by
> build_skb() and frees by napi_skb_cache_put(). Because the driver
> does not use napi skb cache in allocation path, napi skb cache is
> periodically filled and flushed. This is waste of napi skb cache.
>
> As ena_alloc_skb() is called only in napi, Use napi_build_skb()
> instead of build_skb() to when allocating skb.
>
> This patch was tested on aws a1.metal instance.
>
> Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
> ---
> drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
> index c72f0c7ff4aa..2c67fb1703c5 100644
> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
> @@ -1407,7 +1407,7 @@ static struct sk_buff *ena_alloc_skb(struct ena_ring *rx_ring, void *first_frag)
> skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
> rx_ring->rx_copybreak);
To keep things consistent, this should then also be napi_alloc_skb().
> else
> - skb = build_skb(first_frag, ENA_PAGE_SIZE);
> + skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
>
> if (unlikely(!skb)) {
> ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 1,
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] net: ena: Do not waste napi skb cache
2022-01-24 8:57 ` Julian Wiedmann
@ 2022-01-24 9:29 ` Julian Wiedmann
2022-01-24 20:50 ` Shay Agroskin
0 siblings, 1 reply; 12+ messages in thread
From: Julian Wiedmann @ 2022-01-24 9:29 UTC (permalink / raw)
To: Hyeonggon Yoo, netdev
Cc: Shay Agroskin, Arthur Kiyanovski, David Arinzon, Noam Dagan,
Saeed Bishara, David S . Miller, Jakub Kicinski, Daniel Borkmann,
Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi, linux-kernel
On 24.01.22 10:57, Julian Wiedmann wrote:
> On 23.01.22 13:56, Hyeonggon Yoo wrote:
>> By profiling, discovered that ena device driver allocates skb by
>> build_skb() and frees by napi_skb_cache_put(). Because the driver
>> does not use napi skb cache in allocation path, napi skb cache is
>> periodically filled and flushed. This is waste of napi skb cache.
>>
>> As ena_alloc_skb() is called only in napi, Use napi_build_skb()
>> instead of build_skb() to when allocating skb.
>>
>> This patch was tested on aws a1.metal instance.
>>
>> Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> index c72f0c7ff4aa..2c67fb1703c5 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> @@ -1407,7 +1407,7 @@ static struct sk_buff *ena_alloc_skb(struct ena_ring *rx_ring, void *first_frag)
>> skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
>> rx_ring->rx_copybreak);
>
> To keep things consistent, this should then also be napi_alloc_skb().
>
And on closer look, this copybreak path also looks buggy. If rx_copybreak
gets reduced _while_ receiving a frame, the allocated skb can end up too
small to take all the data.
@ ena maintainers: can you please fix this?
>> else
>> - skb = build_skb(first_frag, ENA_PAGE_SIZE);
>> + skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
>>
>> if (unlikely(!skb)) {
>> ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 1,
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] net: ena: Do not waste napi skb cache
2022-01-24 9:29 ` Julian Wiedmann
@ 2022-01-24 20:50 ` Shay Agroskin
2022-01-25 9:34 ` Hyeonggon Yoo
2022-01-25 13:45 ` Julian Wiedmann
0 siblings, 2 replies; 12+ messages in thread
From: Shay Agroskin @ 2022-01-24 20:50 UTC (permalink / raw)
To: Julian Wiedmann
Cc: Hyeonggon Yoo, netdev, Arthur Kiyanovski, David Arinzon,
Noam Dagan, Saeed Bishara, David S . Miller, Jakub Kicinski,
Daniel Borkmann, Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi,
linux-kernel
Julian Wiedmann <jwiedmann.dev@gmail.com> writes:
> On 24.01.22 10:57, Julian Wiedmann wrote:
>> On 23.01.22 13:56, Hyeonggon Yoo wrote:
>>> By profiling, discovered that ena device driver allocates skb
>>> by
>>> build_skb() and frees by napi_skb_cache_put(). Because the
>>> driver
>>> does not use napi skb cache in allocation path, napi skb cache
>>> is
>>> periodically filled and flushed. This is waste of napi skb
>>> cache.
>>>
>>> As ena_alloc_skb() is called only in napi, Use
>>> napi_build_skb()
>>> instead of build_skb() to when allocating skb.
>>>
>>> This patch was tested on aws a1.metal instance.
>>>
>>> Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
>>> ---
>>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>> b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>> index c72f0c7ff4aa..2c67fb1703c5 100644
>>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>> @@ -1407,7 +1407,7 @@ static struct sk_buff
>>> *ena_alloc_skb(struct ena_ring *rx_ring, void *first_frag)
>>> skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
>>> rx_ring->rx_copybreak);
>>
>> To keep things consistent, this should then also be
>> napi_alloc_skb().
>>
>
> And on closer look, this copybreak path also looks buggy. If
> rx_copybreak
> gets reduced _while_ receiving a frame, the allocated skb can
> end up too
> small to take all the data.
>
> @ ena maintainers: can you please fix this?
>
Updating the copybreak value is done through ena_ethtool.c
(ena_set_tunable()) which updates `adapter->rx_copybreak`.
The adapter->rx_copybreak value is "propagated back" to the ring
local attributes (rx_ring->rx_copybreak) only after an interface
toggle which stops the napi routine first.
Unless I'm missing something here I don't think the bug you're
describing exists.
I agree that the netdev_alloc_skb_ip_align() can become
napi_alloc_skb(). Hyeonggon Yoo, can you please apply this change
as well to this patch?
Thanks,
Shay
>>> else
>>> - skb = build_skb(first_frag, ENA_PAGE_SIZE);
>>> + skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
>>>
>>> if (unlikely(!skb)) {
>>> ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail,
>>> 1,
>>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] net: ena: Do not waste napi skb cache
2022-01-24 20:50 ` Shay Agroskin
@ 2022-01-25 9:34 ` Hyeonggon Yoo
2022-01-25 13:50 ` Julian Wiedmann
2022-01-25 13:45 ` Julian Wiedmann
1 sibling, 1 reply; 12+ messages in thread
From: Hyeonggon Yoo @ 2022-01-25 9:34 UTC (permalink / raw)
To: Shay Agroskin
Cc: Julian Wiedmann, netdev, Arthur Kiyanovski, David Arinzon,
Noam Dagan, Saeed Bishara, David S . Miller, Jakub Kicinski,
Daniel Borkmann, Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi,
linux-kernel
On Mon, Jan 24, 2022 at 10:50:05PM +0200, Shay Agroskin wrote:
>
> Julian Wiedmann <jwiedmann.dev@gmail.com> writes:
>
> > On 24.01.22 10:57, Julian Wiedmann wrote:
> > > On 23.01.22 13:56, Hyeonggon Yoo wrote:
> > > > By profiling, discovered that ena device driver allocates skb by
> > > > build_skb() and frees by napi_skb_cache_put(). Because the
> > > > driver
> > > > does not use napi skb cache in allocation path, napi skb cache
> > > > is
> > > > periodically filled and flushed. This is waste of napi skb
> > > > cache.
> > > >
> > > > As ena_alloc_skb() is called only in napi, Use napi_build_skb()
> > > > instead of build_skb() to when allocating skb.
> > > >
> > > > This patch was tested on aws a1.metal instance.
> > > >
> > > > Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
> > > > ---
> > > > drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c
> > > > b/drivers/net/ethernet/amazon/ena/ena_netdev.c
> > > > index c72f0c7ff4aa..2c67fb1703c5 100644
> > > > --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
> > > > +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
> > > > @@ -1407,7 +1407,7 @@ static struct sk_buff
> > > > *ena_alloc_skb(struct ena_ring *rx_ring, void *first_frag)
> > > > skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
> > > > rx_ring->rx_copybreak);
> > >
> > > To keep things consistent, this should then also be
> > > napi_alloc_skb().
> > >
Right. I missed this. Thank you for pointing this!
> >
> > And on closer look, this copybreak path also looks buggy. If
> > rx_copybreak
> > gets reduced _while_ receiving a frame, the allocated skb can end up too
> > small to take all the data.
> >
> > @ ena maintainers: can you please fix this?
> >
>
Shay and Julian, Thank you for reviewing this.
> Updating the copybreak value is done through ena_ethtool.c
> (ena_set_tunable()) which updates `adapter->rx_copybreak`.
> The adapter->rx_copybreak value is "propagated back" to the ring local
> attributes (rx_ring->rx_copybreak) only after an interface toggle which
> stops the napi routine first.
>
> Unless I'm missing something here I don't think the bug you're describing
> exists.
>
> I agree that the netdev_alloc_skb_ip_align() can become napi_alloc_skb().
> Hyeonggon Yoo, can you please apply this change as well to this patch?
>
Okay. I'll update and test it again.
BTW, It seems netdev_alloc_skb_ip_align() is used to make some fields
be aligned. It's okay to just ignore this?
Thanks,
Hyeonggon.
> Thanks,
> Shay
>
>
> > > > else
> > > > - skb = build_skb(first_frag, ENA_PAGE_SIZE);
> > > > + skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
> > > > if (unlikely(!skb)) {
> > > > ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 1,
> > >
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] net: ena: Do not waste napi skb cache
2022-01-24 20:50 ` Shay Agroskin
2022-01-25 9:34 ` Hyeonggon Yoo
@ 2022-01-25 13:45 ` Julian Wiedmann
1 sibling, 0 replies; 12+ messages in thread
From: Julian Wiedmann @ 2022-01-25 13:45 UTC (permalink / raw)
To: Shay Agroskin
Cc: Hyeonggon Yoo, netdev, Arthur Kiyanovski, David Arinzon,
Noam Dagan, Saeed Bishara, David S . Miller, Jakub Kicinski,
Daniel Borkmann, Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi,
linux-kernel
On 24.01.22 22:50, Shay Agroskin wrote:
>
> Julian Wiedmann <jwiedmann.dev@gmail.com> writes:
>
>> On 24.01.22 10:57, Julian Wiedmann wrote:
>>> On 23.01.22 13:56, Hyeonggon Yoo wrote:
>>>> By profiling, discovered that ena device driver allocates skb by
>>>> build_skb() and frees by napi_skb_cache_put(). Because the driver
>>>> does not use napi skb cache in allocation path, napi skb cache is
>>>> periodically filled and flushed. This is waste of napi skb cache.
>>>>
>>>> As ena_alloc_skb() is called only in napi, Use napi_build_skb()
>>>> instead of build_skb() to when allocating skb.
>>>>
>>>> This patch was tested on aws a1.metal instance.
>>>>
>>>> Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
>>>> ---
>>>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>>> index c72f0c7ff4aa..2c67fb1703c5 100644
>>>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>>> @@ -1407,7 +1407,7 @@ static struct sk_buff *ena_alloc_skb(struct ena_ring *rx_ring, void *first_frag)
>>>> skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
>>>> rx_ring->rx_copybreak);
>>>
>>> To keep things consistent, this should then also be napi_alloc_skb().
>>>
>>
>> And on closer look, this copybreak path also looks buggy. If rx_copybreak
>> gets reduced _while_ receiving a frame, the allocated skb can end up too
>> small to take all the data.
>>
>> @ ena maintainers: can you please fix this?
>>
>
> Updating the copybreak value is done through ena_ethtool.c (ena_set_tunable()) which updates `adapter->rx_copybreak`.
> The adapter->rx_copybreak value is "propagated back" to the ring local attributes (rx_ring->rx_copybreak) only after an interface toggle which stops the napi routine first.
>
That's unfortunate. ena_get_tunable() returns the updated adapter->rx_copybreak, how would a user know that their update isn't actually live yet?
> Unless I'm missing something here I don't think the bug you're describing exists.
>
ack, thanks for double-checking!
> I agree that the netdev_alloc_skb_ip_align() can become napi_alloc_skb(). Hyeonggon Yoo, can you please apply this change as well to this patch?
>
> Thanks,
> Shay
>
>
>>>> else
>>>> - skb = build_skb(first_frag, ENA_PAGE_SIZE);
>>>> + skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
>>>>
>>>> if (unlikely(!skb)) {
>>>> ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 1,
>>>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] net: ena: Do not waste napi skb cache
2022-01-25 9:34 ` Hyeonggon Yoo
@ 2022-01-25 13:50 ` Julian Wiedmann
2022-01-29 8:56 ` Hyeonggon Yoo
0 siblings, 1 reply; 12+ messages in thread
From: Julian Wiedmann @ 2022-01-25 13:50 UTC (permalink / raw)
To: Hyeonggon Yoo, Shay Agroskin
Cc: netdev, Arthur Kiyanovski, David Arinzon, Noam Dagan,
Saeed Bishara, David S . Miller, Jakub Kicinski, Daniel Borkmann,
Wei Yongjun, Lorenzo Bianconi, linux-kernel
On 25.01.22 11:34, Hyeonggon Yoo wrote:
> On Mon, Jan 24, 2022 at 10:50:05PM +0200, Shay Agroskin wrote:
>>
[...]
>>
>> I agree that the netdev_alloc_skb_ip_align() can become napi_alloc_skb().
>> Hyeonggon Yoo, can you please apply this change as well to this patch?
>>
>
> Okay. I'll update and test it again.
>
> BTW, It seems netdev_alloc_skb_ip_align() is used to make some fields
> be aligned. It's okay to just ignore this?
>
napi_alloc_skb() adds NET_IP_ALIGN internally, so you end up with the same alignment as before.
> Thanks,
> Hyeonggon.
>
>> Thanks,
>> Shay
>>
>>
>>>>> else
>>>>> - skb = build_skb(first_frag, ENA_PAGE_SIZE);
>>>>> + skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
>>>>> if (unlikely(!skb)) {
>>>>> ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 1,
>>>>
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2] net: ena: Do not waste napi skb cache
2022-01-23 11:56 [PATCH] net: ena: Do not waste napi skb cache Hyeonggon Yoo
2022-01-24 8:57 ` Julian Wiedmann
@ 2022-01-26 13:16 ` Hyeonggon Yoo
2022-01-26 15:16 ` Shay Agroskin
2022-01-26 15:52 ` Jakub Kicinski
1 sibling, 2 replies; 12+ messages in thread
From: Hyeonggon Yoo @ 2022-01-26 13:16 UTC (permalink / raw)
To: jwiedmann.dev
Cc: Shay Agroskin, Arthur Kiyanovski, David Arinzon, Noam Dagan,
Saeed Bishara, David S . Miller, Jakub Kicinski, Daniel Borkmann,
Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi, linux-kernel
By profiling, discovered that ena device driver allocates skb by
build_skb() and frees by napi_skb_cache_put(). Because the driver
does not use napi skb cache in allocation path, napi skb cache is
periodically filled and flushed. This is waste of napi skb cache.
As ena_alloc_skb() is called only in napi, Use napi_build_skb()
and napi_alloc_skb() when allocating skb.
This patch was tested on aws a1.metal instance.
[ jwiedmann.dev@gmail.com: Use napi_alloc_skb() instead of
netdev_alloc_skb_ip_align() to keep things consistent. ]
Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
---
drivers/net/ethernet/amazon/ena/ena_netdev.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
index 53080fd143dc..07444aead3fd 100644
--- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
+++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
@@ -1400,10 +1400,9 @@ static struct sk_buff *ena_alloc_skb(struct ena_ring *rx_ring, void *first_frag)
struct sk_buff *skb;
if (!first_frag)
- skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
- rx_ring->rx_copybreak);
+ skb = napi_alloc_skb(rx_ring->napi, rx_ring->rx_copybreak);
else
- skb = build_skb(first_frag, ENA_PAGE_SIZE);
+ skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
if (unlikely(!skb)) {
ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 1,
--
2.33.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v2] net: ena: Do not waste napi skb cache
2022-01-26 13:16 ` [PATCH v2] " Hyeonggon Yoo
@ 2022-01-26 15:16 ` Shay Agroskin
2022-01-26 15:52 ` Jakub Kicinski
1 sibling, 0 replies; 12+ messages in thread
From: Shay Agroskin @ 2022-01-26 15:16 UTC (permalink / raw)
To: Hyeonggon Yoo
Cc: jwiedmann.dev, Arthur Kiyanovski, David Arinzon, Noam Dagan,
Saeed Bishara, David S . Miller, Jakub Kicinski, Daniel Borkmann,
Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi, linux-kernel
Hyeonggon Yoo <42.hyeyoo@gmail.com> writes:
> CAUTION: This email originated from outside of the
> organization. Do not click links or open attachments unless you
> can confirm the sender and know the content is safe.
>
>
>
> By profiling, discovered that ena device driver allocates skb by
> build_skb() and frees by napi_skb_cache_put(). Because the
> driver
> does not use napi skb cache in allocation path, napi skb cache
> is
> periodically filled and flushed. This is waste of napi skb
> cache.
>
> As ena_alloc_skb() is called only in napi, Use napi_build_skb()
> and napi_alloc_skb() when allocating skb.
>
> This patch was tested on aws a1.metal instance.
>
> [ jwiedmann.dev@gmail.com: Use napi_alloc_skb() instead of
> netdev_alloc_skb_ip_align() to keep things consistent. ]
>
> Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
> ---
> drivers/net/ethernet/amazon/ena/ena_netdev.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
Thank you for this work
Acked-by: Shay Agroskin <shayagr@amazon.com>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] net: ena: Do not waste napi skb cache
2022-01-26 13:16 ` [PATCH v2] " Hyeonggon Yoo
2022-01-26 15:16 ` Shay Agroskin
@ 2022-01-26 15:52 ` Jakub Kicinski
2022-01-29 8:55 ` Hyeonggon Yoo
1 sibling, 1 reply; 12+ messages in thread
From: Jakub Kicinski @ 2022-01-26 15:52 UTC (permalink / raw)
To: Hyeonggon Yoo
Cc: jwiedmann.dev, Shay Agroskin, Arthur Kiyanovski, David Arinzon,
Noam Dagan, Saeed Bishara, David S . Miller, Daniel Borkmann,
Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi, linux-kernel
On Wed, 26 Jan 2022 13:16:41 +0000 Hyeonggon Yoo wrote:
> By profiling, discovered that ena device driver allocates skb by
> build_skb() and frees by napi_skb_cache_put(). Because the driver
> does not use napi skb cache in allocation path, napi skb cache is
> periodically filled and flushed. This is waste of napi skb cache.
>
> As ena_alloc_skb() is called only in napi, Use napi_build_skb()
> and napi_alloc_skb() when allocating skb.
>
> This patch was tested on aws a1.metal instance.
>
> [ jwiedmann.dev@gmail.com: Use napi_alloc_skb() instead of
> netdev_alloc_skb_ip_align() to keep things consistent. ]
>
> Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
It seems you missed CCing netdev@, you'll need to repost.
You can keep the ack from Shay.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] net: ena: Do not waste napi skb cache
2022-01-26 15:52 ` Jakub Kicinski
@ 2022-01-29 8:55 ` Hyeonggon Yoo
0 siblings, 0 replies; 12+ messages in thread
From: Hyeonggon Yoo @ 2022-01-29 8:55 UTC (permalink / raw)
To: Jakub Kicinski
Cc: jwiedmann.dev, Shay Agroskin, Arthur Kiyanovski, David Arinzon,
Noam Dagan, Saeed Bishara, David S . Miller, Daniel Borkmann,
Sameeh Jubran, Wei Yongjun, Lorenzo Bianconi, linux-kernel
On Wed, Jan 26, 2022 at 07:52:20AM -0800, Jakub Kicinski wrote:
> On Wed, 26 Jan 2022 13:16:41 +0000 Hyeonggon Yoo wrote:
> > By profiling, discovered that ena device driver allocates skb by
> > build_skb() and frees by napi_skb_cache_put(). Because the driver
> > does not use napi skb cache in allocation path, napi skb cache is
> > periodically filled and flushed. This is waste of napi skb cache.
> >
> > As ena_alloc_skb() is called only in napi, Use napi_build_skb()
> > and napi_alloc_skb() when allocating skb.
> >
> > This patch was tested on aws a1.metal instance.
> >
> > [ jwiedmann.dev@gmail.com: Use napi_alloc_skb() instead of
> > netdev_alloc_skb_ip_align() to keep things consistent. ]
> >
> > Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
>
> It seems you missed CCing netdev@, you'll need to repost.
> You can keep the ack from Shay.
Oh my mistake. I did it. Thanks!
Hyeonggon
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] net: ena: Do not waste napi skb cache
2022-01-25 13:50 ` Julian Wiedmann
@ 2022-01-29 8:56 ` Hyeonggon Yoo
0 siblings, 0 replies; 12+ messages in thread
From: Hyeonggon Yoo @ 2022-01-29 8:56 UTC (permalink / raw)
To: Julian Wiedmann
Cc: Shay Agroskin, netdev, Arthur Kiyanovski, David Arinzon,
Noam Dagan, Saeed Bishara, David S . Miller, Jakub Kicinski,
Daniel Borkmann, Wei Yongjun, Lorenzo Bianconi, linux-kernel
On Tue, Jan 25, 2022 at 03:50:30PM +0200, Julian Wiedmann wrote:
> On 25.01.22 11:34, Hyeonggon Yoo wrote:
> > On Mon, Jan 24, 2022 at 10:50:05PM +0200, Shay Agroskin wrote:
> >>
>
> [...]
>
> >>
> >> I agree that the netdev_alloc_skb_ip_align() can become napi_alloc_skb().
> >> Hyeonggon Yoo, can you please apply this change as well to this patch?
> >>
> >
> > Okay. I'll update and test it again.
> >
> > BTW, It seems netdev_alloc_skb_ip_align() is used to make some fields
> > be aligned. It's okay to just ignore this?
> >
>
> napi_alloc_skb() adds NET_IP_ALIGN internally, so you end up with the same alignment as before.
>
Oh I was missing that. I updated the patch and tested again.
Thank you!
>
> > Thanks,
> > Hyeonggon.
> >
> >> Thanks,
> >> Shay
> >>
> >>
> >>>>> else
> >>>>> - skb = build_skb(first_frag, ENA_PAGE_SIZE);
> >>>>> + skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
> >>>>> if (unlikely(!skb)) {
> >>>>> ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 1,
> >>>>
>
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2022-01-29 8:56 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-23 11:56 [PATCH] net: ena: Do not waste napi skb cache Hyeonggon Yoo
2022-01-24 8:57 ` Julian Wiedmann
2022-01-24 9:29 ` Julian Wiedmann
2022-01-24 20:50 ` Shay Agroskin
2022-01-25 9:34 ` Hyeonggon Yoo
2022-01-25 13:50 ` Julian Wiedmann
2022-01-29 8:56 ` Hyeonggon Yoo
2022-01-25 13:45 ` Julian Wiedmann
2022-01-26 13:16 ` [PATCH v2] " Hyeonggon Yoo
2022-01-26 15:16 ` Shay Agroskin
2022-01-26 15:52 ` Jakub Kicinski
2022-01-29 8:55 ` Hyeonggon Yoo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).