From: Sukadev Bhattiprolu <sukadev@linux.ibm.com>
To: Lijun Pan <lijunp213@gmail.com>
Cc: netdev@vger.kernel.org
Subject: Re: [PATCH net-next] net: ibm: replenish rx pool and poll less frequently
Date: Wed, 2 Jun 2021 12:25:06 -0700 [thread overview]
Message-ID: <YLfbEjiu671HApgi@us.ibm.com> (raw)
In-Reply-To: <20210602170156.41643-1-lijunp213@gmail.com>
Lijun Pan [lijunp213@gmail.com] wrote:
> The old mechanism replenishes rx pool even only one frames is processed in
> the poll function, which causes lots of overheads. The old mechanism
The soft lockup is not seen when replenishing a small number of buffers at
a time. Its only under some conditions when replenishing a _large_ number
at once - appears to be because the netdev_alloc_skb() calls collectively
take a long time.
Replenishing a small number at a time is not a problem.
> restarts polling until processed frames reaches the budget, which can
> cause the poll function to loop into restart_poll 63 times at most and to
> call replenish_rx_poll 63 times at most. This will cause soft lockup very
> easily. So, don't replenish too often, and don't goto restart_poll in each
The 64 is from the budget the system gave us. And for us to hit the goto
restart_loop:
a. pending_scrq() in the while loop must not have a found a packet,
and
b. by the time we replenished the pool, completed napi etc we must
have found a packet
For this to happen 64 times, we must find
- exactly zero packets in a. and
- exactly one packet in b, and
- the tight sequence must occur 64 times.
IOW its more theoretical right?
Even if it did happen a handful of times, the only "overheads" in the
replenish are the netdev_alloc_skb() and the send-subcrq-indirect hcall.
The skb alloc cannot be avoided - we must do it now or in the future
anyway. The hcall is issued every 16 skbs. If we issue it for <16 skbs
it means the traffic is extremely low. No point optimizing for that.
Besides the hcalls are not very expensive.
There was a lot of testing done in Nov 2020 when the subcrq-indirect
hcall support was added. We would need to repeat that testing at the
least.
Thanks,
Sukadev
prev parent reply other threads:[~2021-06-02 19:25 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-02 17:01 [PATCH net-next] net: ibm: replenish rx pool and poll less frequently Lijun Pan
2021-06-02 17:47 ` Rick Lindsley
2021-06-02 17:58 ` Dany Madden
2021-06-02 18:23 ` Dany Madden
2021-06-02 19:25 ` Sukadev Bhattiprolu [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YLfbEjiu671HApgi@us.ibm.com \
--to=sukadev@linux.ibm.com \
--cc=lijunp213@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).