Re: net latency, long wait for softirq_entry

[Sebastian Andrzej Siewior <bigeasy@linutronix.de>]

On 2020-12-05 13:30:21 [-0500], Paul Thomas wrote:
> (this time with a proper subject)
> Hello,
Hi, 

> I've been testing out the 5.10 rt kernel[1], and now I'm trying to
…
> But then in the long latency case, instead of going straight to
> softirq_entry this is delayed and it does the softirq_entry from the
> ksoftirqd task instead of the irq/30-eth%d task. The problem is that
> before ksoftirqd runs, another lower priority task runs (in this case
> it's the macb tx interrupt thread at priority 80, but I've seen this
> be user threads in slightly different tests). It seems the paths
> diverge with a call to __kthread_should_park.
> 
> This is on an arm64 zynqmp platform.
> 
> Any thoughts on how this could be improved would be appreciated.

The usual flow is:
- wake IRQ thread
- IRQ thread raises NAPI (softirq NET_RX)
- NAPI is processed on the exit of the IRQ thread.

This flow was disturbed by the timer:
- stress-ng is running
- the HZ tick timer fired, raised TIMER softirq. This leads to waking
  the ksoftirqd thread to process it (this is normal).
- stress-ng continues.
- the networking IRQ fires. The thread is woken up, raises NET_RX
  softirq. Usually, the softirq would be invoked on return from the
  interrupt thread but since ksoftirqd is already woken nothing happens.
  It is delayed to ksoftirqd. The IRQ-thread goes idle.

Having a thread to run the tick-timer would avoid that scenario.

Side note: If the networking thread runs for a longer period of then it
will handover to ksoftirqd.

 -Paul

Sebastian