> Okay. Then it could be related to softirq rework which started in
> v5.0.19-rt11 and updated in v5.9-rc6-rt9.
v5.9 could also be significant as the R8125 driver support was added
in that version
I upgraded from Debian Buster to Bullseye (v5.10) on one PC for this reason.

> Are you able to recompile a kernel?
Yes,
I have attached a text file containing my 6.3 build steps.
Note we had to git pull the kernel source as it was not available via
http at that time
I have not tried with your 6.3-rc11 patch yet.

> However as per you Debian bug report v4.19-RT is fine, v5.10-RT isn't
> and v6.1-RT is better or is it fine?
I built v6.1-rc7-rt5 before Debian advanced to v6.1
It was reported to be an improvement and it solved the Debian latency
issue for some users.
v6.3 was reported as being as good as the 4.x kernes by our hardware vendor

> Where is the Intel ethernet coming from? You were saying it is the
> realtek nic.
Sorry, you mentioned coalesce but that's an Intel feature and does not
exist with Realtek.

> Is the vendor driver better than in the tree? This is not obvious from
> what you are writing.
I have been unable to compile the R8168 driver from source on the
Realtek website.
They are behind with supported kernels and the build process is
invariably broken for me.
On the Hardkernel Odroid forum, it has been reported that compilng the
R8125 from Realtek source offers 15-35% improvement over the in-tree
driver.
 I have one of their  PC's (H2+). It's currently running v5.10 and the
in-tree R8168 driver (which must be the Debian default).
 It is affected and I plan to upgrade to v6.3 with the R8125-dkms driver

> Two things you could do:
> - enabling tracing to see what is causing the delays/ latency spike. You
>   will need a trigger to notice the latency and then stop the trace at
>   this point. If so, I could provide additional steps unless you can do
>   it yourself.
Please advise additional steps to do this.
We might  be able to modify our hardware device driver to stop the
trace as it reports latency incursions if they occur.

> - You could try to isolate the realtek driver on one CPU and moving
>   everything else to another.
This sounds interesting. Please advise how.
On a 4 core machine we typically add isolcpus=2,3 for best RT performance.

Thanks for your support.

-end
-Rod Webster