Re: [PATCH net] net: fix raising a softirq on the current cpu with rps enabled

From: Jason Xing <kerneljasonxing@gmail.com>
To: Eric Dumazet <edumazet@google.com>
Cc: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
	Jason Xing <kernelxing@tencent.com>
Subject: Re: [PATCH net] net: fix raising a softirq on the current cpu with rps enabled
Date: Sun, 26 Mar 2023 11:27:56 +0800	[thread overview]
Message-ID: <CAL+tcoBPOBs0qvWmV_0EJ-s1qF=FKz=bhcH5Px51cgZJ9NDrcw@mail.gmail.com> (raw)
In-Reply-To: <CANn89iJaVrObJNDC9TrnSUC3XQeo-zfmUXLVrNVcsbRDPuSNtA@mail.gmail.com>

On Sat, Mar 25, 2023 at 11:57 PM Eric Dumazet <edumazet@google.com> wrote:
>
> On Sat, Mar 25, 2023 at 8:26 AM Jason Xing <kerneljasonxing@gmail.com> wrote:
> >
> > From: Jason Xing <kernelxing@tencent.com>
> >
> > Since we decide to put the skb into a backlog queue of another
> > cpu, we should not raise the softirq for the current cpu. When
> > to raise a softirq is based on whether we have more data left to
> > process later. As to the current cpu, there is no indication of
> > more data enqueued, so we do not need this action. After enqueuing
> > to another cpu, net_rx_action() function will call ipi and then
> > another cpu will raise the softirq as expected.
> >
> > Also, raising more softirqs which set the corresponding bit field
> > can make the IRQ mechanism think we probably need to start ksoftirqd
> > on the current cpu. Actually it shouldn't happen.
> >
> > Fixes: 0a9627f2649a ("rps: Receive Packet Steering")
> > Signed-off-by: Jason Xing <kernelxing@tencent.com>
> > ---
> >  net/core/dev.c | 2 --
> >  1 file changed, 2 deletions(-)
> >
> > diff --git a/net/core/dev.c b/net/core/dev.c
> > index 1518a366783b..bfaaa652f50c 100644
> > --- a/net/core/dev.c
> > +++ b/net/core/dev.c
> > @@ -4594,8 +4594,6 @@ static int napi_schedule_rps(struct softnet_data *sd)
> >         if (sd != mysd) {
> >                 sd->rps_ipi_next = mysd->rps_ipi_list;
> >                 mysd->rps_ipi_list = sd;
> > -
> > -               __raise_softirq_irqoff(NET_RX_SOFTIRQ);
> >                 return 1;
> >         }
> >  #endif /* CONFIG_RPS */
> > --
> > 2.37.3
> >
>
> This is not going to work in some cases. Please take a deeper look.

I'll do it. I've already been struggling with this for a few days. I
still have no clue.

I found out in some cases that frequently starting ksoftirqd is not
that good because this thread may be blocked when waiting in the
runqueue.
So I tried to avoid raising softirqs to mitigate the issue and then I
noticed this one.

>
> I have to run, if you (or others) do not find the reason, I will give
> more details when I am done traveling.

Happy traveling :)

Thanks,
Jason