Re: Performance regression on kernels 3.10 and newer

[Alexander Duyck <alexander.h.duyck@intel.com>]

On 08/18/2014 02:03 AM, David Laight wrote:
> From: Alexander Duyck
>> ...
>> Another test I tried was to hack the nettest_bsd.c file in netperf to
>> perform a poll() based receive.  That resolved the issue and had all the
>> performance of the tcp_low_latency case.  I may see if I can work with
>> Rick to push something like that into netperf as I really would prefer
>> to avoid having to advise everyone on how to setup the sysctl for
>> tcp_low_latency.
> 
> Doesn't that generate 2 system calls per receive?
> Unless it now returns more data per receive I'm surprised that
> it actually faster.

If you haven't been keeping up with the thread what I am gaining by
doing this is avoiding a significant cache thrash issue with the dst
entry as the prequeue path involves updating the reference count that is
shared by all of my CPUs.

By using poll to wait for it I don't load frames onto the TCP prequeue
and thereby avoid it.

> OTOH I've some code that runs a lot better when I run while :; do :; done
> for all but one of the cpus.
> I think that is because the processes spinning in userspace don't
> get pre-empted.
> 
> 	David
> 

What you are probably seeing is that the CPU doesn't go into a deep
sleep state so it likely runs better.  You might try the same thing with
a kernel booted with idle=poll and you would probably see the same result.

Thanks,

Alex