Re: [net] 19f92a030c: apachebench.requests_per_second -37.9% regression

[Eric Dumazet <edumazet@google.com>]

[-- Attachment #1: Type: text/plain, Size: 7535 bytes --]

On Wed, Nov 13, 2019 at 8:12 AM Feng Tang <feng.tang@intel.com> wrote:

> Hi Eric,
>
> On Wed, Nov 13, 2019 at 06:33:44AM -0800, Eric Dumazet wrote:
> > On Wed, Nov 13, 2019 at 2:35 AM Feng Tang <feng.tang@intel.com> wrote:
> > >
> > > Hi Eric,
> > >
> > > On Fri, Nov 08, 2019 at 04:35:13PM +0800, kernel test robot wrote:
> > > > Greeting,
> > > >
> > > > FYI, we noticed a -37.9% regression of
> apachebench.requests_per_second due to commit:
> > > >
> > > > commit: 19f92a030ca6d772ab44b22ee6a01378a8cb32d4 ("net: increase
> SOMAXCONN to 4096")
> > > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git
> master
> > >
> > > Any thought on this? The test is actually:
> > >
> > >         sysctl -w net.ipv4.tcp_syncookies=0
> >
> >
> > I have no plan trying to understand why anyone would disable syncookies .
> > This is a non starter really, since this makes a server vulnerable to
> > a trivial DOS attack.
> >
>
> Thanks for the prompt response.
>
> This sysctl change was added to work around one old error when
> running the apachebench. The error was "apr_socket_recv: Connection
> reset by peer (104)"
>
> I just removed the sysctl setting of "ipv4.tcp_synccookies" and
> re-run the same benchmark in 0day, the regression is still there.
>


This is fine. Please setup the sysctls if you need to make a benchmark
shiny.

The defaults should be sane, not based on some artificial workload.

We have been using 4096 at Google for about 5 years, and Google is still
alive.


>
> Thanks,
> Feng
>
> >
> > Since the test changes a sysctl, you also can change other sysctls if
> > you really need to show a ' number of transactions' for a particular
> > benchmark.
> >
> > The change on SOMAXCONN was driven by a security issue, and security
> > comes first.
> >
> >
> > >         enable_apache_mod auth_basic authn_core authn_file authz_core
> authz_host authz_user access_compat
> > >         systemctl restart apache2
> > >         ab -k -q -t 300 -n 1000000 -c 4000 127.0.0.1/
> > >
> > > And some info about apachebench result is:
> > >
> > > w/o patch:
> > >
> > >         Connection Times (ms)
> > >                       min  mean[+/-sd] median   max
> > >         Connect:        0    0  19.5      0    7145
> > >         Processing:     0    4 110.0      3   21647
> > >         Waiting:        0    2  92.4      1   21646
> > >         Total:          0    4 121.1      3   24762
> > >
> > > w/ patch:
> > >
> > >         Connection Times (ms)
> > >                       min  mean[+/-sd] median   max
> > >         Connect:        0    0  43.2      0    7143
> > >         Processing:     0   19 640.4      3   38708
> > >         Waiting:        0   24 796.5      1   38708
> > >         Total:          0   19 657.5      3   39725
> > >
> > >
> > > Thanks,
> > > Feng
> > >
> > > >
> > > > in testcase: apachebench
> > > > on test machine: 16 threads Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz
> with 48G memory
> > > > with following parameters:
> > > >
> > > >       runtime: 300s
> > > >       concurrency: 4000
> > > >       cluster: cs-localhost
> > > >       cpufreq_governor: performance
> > > >       ucode: 0x7000019
> > > >
> > > > test-description: apachebench is a tool for benchmarking your Apache
> Hypertext Transfer Protocol (HTTP) server.
> > > > test-url: https://httpd.apache.org/docs/2.4/programs/ab.html
> > > >
> > > > In addition to that, the commit also has significant impact on the
> following tests:
> > > >
> > > >
> +------------------+------------------------------------------------------------------+
> > > > | testcase: change | apachebench: apachebench.requests_per_second
> -37.5% regression   |
> > > > | test machine     | 16 threads Intel(R) Xeon(R) CPU D-1541 @
> 2.10GHz with 48G memory |
> > > > | test parameters  | cluster=cs-localhost
>                  |
> > > > |                  | concurrency=8000
>                  |
> > > > |                  | cpufreq_governor=performance
>                  |
> > > > |                  | runtime=300s
>                  |
> > > > |                  | ucode=0x7000019
>                   |
> > > >
> +------------------+------------------------------------------------------------------+
> > > >
> > > >
> > > > If you fix the issue, kindly add following tag
> > > > Reported-by: kernel test robot <rong.a.chen@intel.com>
> > > >
> > > >
> > > > Details are as below:
> > > >
> -------------------------------------------------------------------------------------------------->
> > > >
> > > >
> > > > To reproduce:
> > > >
> > > >         git clone https://github.com/intel/lkp-tests.git
> > > >         cd lkp-tests
> > > >         bin/lkp install job.yaml  # job file is attached in this
> email
> > > >         bin/lkp run     job.yaml
> > > >
> > > >
> =========================================================================================
> > > >
> cluster/compiler/concurrency/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/testcase/ucode:
> > > >
>  cs-localhost/gcc-7/4000/performance/x86_64-rhel-7.6/debian-x86_64-2019-09-23.cgz/300s/lkp-bdw-de1/apachebench/0x7000019
> > > >
> > > > commit:
> > > >   6d6f0383b6 ("netdevsim: Fix use-after-free during device
> dismantle")
> > > >   19f92a030c ("net: increase SOMAXCONN to 4096")
> > > >
> > > > 6d6f0383b697f004 19f92a030ca6d772ab44b22ee6a
> > > > ---------------- ---------------------------
> > > >          %stddev     %change         %stddev
> > > >              \          |                \
> > > >      22640 ±  4%     +71.1%      38734
> apachebench.connection_time.processing.max
> > > >      24701           +60.9%      39743
> apachebench.connection_time.total.max
> > > >      22639 ±  4%     +71.1%      38734
> apachebench.connection_time.waiting.max
> > > >      24701        +15042.0       39743
> apachebench.max_latency.100%
> > > >      40454           -37.9%      25128
> apachebench.requests_per_second
> > > >      25.69           +58.8%      40.79
> apachebench.time.elapsed_time
> > > >      25.69           +58.8%      40.79
> apachebench.time.elapsed_time.max
> > > >      79.00           -37.0%      49.75
> apachebench.time.percent_of_cpu_this_job_got
> > > >      98.88           +61.0%     159.18
> apachebench.time_per_request
> > > >     434631           -37.9%     269889
> apachebench.transfer_rate
> > > >    1.5e+08 ± 18%    +109.5%  3.141e+08 ± 27%  cpuidle.C3.time
> > > >     578957 ±  7%     +64.1%     949934 ± 12%  cpuidle.C3.usage
> > > >      79085 ±  4%     +24.8%      98720        meminfo.AnonHugePages
> > > >      41176           +14.2%      47013        meminfo.PageTables
> > > >      69429           -34.9%      45222        meminfo.max_used_kB
> > > >      63.48           +12.7       76.15        mpstat.cpu.all.idle%
> > > >       2.42 ±  2%      -0.9        1.56        mpstat.cpu.all.soft%
> > > >      15.30            -5.2       10.13        mpstat.cpu.all.sys%
> > > >      18.80            -6.6       12.16        mpstat.cpu.all.usr%
> > > >      65.00           +17.7%      76.50        vmstat.cpu.id
> > > >      17.00           -35.3%      11.00        vmstat.cpu.us
> > > >       7.00 ± 24%     -50.0%       3.50 ± 14%  vmstat.procs.r
> > > >      62957           -33.3%      42012        vmstat.system.cs
> > > >      33174            -1.4%      32693        vmstat.system.in
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 10907 bytes --]