Greeting, FYI, we noticed a -15.0% regression of stress-ng.icache.ops_per_sec due to commit: commit: ed93dfc6bc0084485ccad1ff6bd2ea81ab2c03cd ("rcu: Confine ->core_needs_qs accesses to the corresponding CPU") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: stress-ng on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory with following parameters: nr_threads: 100% disk: 1HDD testtime: 1s class: cpu-cache cpufreq_governor: performance ucode: 0x500002c If you fix the issue, kindly add following tag Reported-by: kernel test robot Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode: cpu-cache/gcc-7/performance/1HDD/x86_64-rhel-7.6/100%/debian-x86_64-2019-11-14.cgz/lkp-csl-2sp5/stress-ng/1s/0x500002c commit: 516e5ae0c9 ("rcu: Reset CPU hints when reporting a quiescent state") ed93dfc6bc ("rcu: Confine ->core_needs_qs accesses to the corresponding CPU") 516e5ae0c9401629 ed93dfc6bc0084485ccad1ff6bd ---------------- --------------------------- %stddev %change %stddev \ | \ 39049 -15.0% 33189 ± 14% stress-ng.icache.ops_per_sec 7784 -36.6% 4939 ± 9% stress-ng.membarrier.ops 7648 -37.3% 4793 ± 9% stress-ng.membarrier.ops_per_sec 41517989 -14.4% 35523501 ± 13% stress-ng.time.minor_page_faults 245.82 ± 3% -7.5% 227.44 ± 5% stress-ng.time.system_time 48037 -29.8% 33703 ± 7% stress-ng.time.voluntary_context_switches 2047 +1.6% 2081 boot-time.idle 41603210 -14.4% 35609851 ± 13% proc-vmstat.pgfault 5619 -10.3% 5039 ± 3% vmstat.system.cs 72138 ± 96% -96.6% 2469 ± 41% sched_debug.cpu.avg_idle.min 2.20 +15.1% 2.53 ± 10% sched_debug.cpu.clock.stddev 2.20 +14.9% 2.53 ± 10% sched_debug.cpu.clock_task.stddev 41320 ± 43% +57.0% 64886 ± 24% numa-meminfo.node0.Inactive 40882 ± 44% +57.0% 64182 ± 24% numa-meminfo.node0.Inactive(anon) 40672 ± 48% +59.0% 64687 ± 27% numa-meminfo.node0.Mapped 43343 ± 46% +53.9% 66694 ± 25% numa-meminfo.node0.Shmem 9465 ± 40% +66.8% 15790 ± 26% numa-vmstat.node0.nr_inactive_anon 9494 ± 44% +68.8% 16023 ± 28% numa-vmstat.node0.nr_mapped 10835 ± 46% +53.4% 16620 ± 25% numa-vmstat.node0.nr_shmem 9477 ± 40% +66.7% 15796 ± 26% numa-vmstat.node0.nr_zone_inactive_anon 120.00 ± 59% -55.8% 53.00 ± 64% interrupts.CPU4.RES:Rescheduling_interrupts 62.00 ± 56% -57.7% 26.25 ± 6% interrupts.CPU41.RES:Rescheduling_interrupts 51.50 ± 65% -55.8% 22.75 ± 18% interrupts.CPU52.RES:Rescheduling_interrupts 112.50 ± 66% -71.6% 32.00 ± 66% interrupts.CPU76.RES:Rescheduling_interrupts 74.50 ± 35% -72.8% 20.25 ± 41% interrupts.CPU92.RES:Rescheduling_interrupts 5.04 ±116% -5.0 0.00 perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput 5.04 ±116% -5.0 0.00 perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit 5.04 ±116% -5.0 0.00 perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit 5.04 ±116% -5.0 0.00 perf-profile.children.cycles-pp.release_pages 5.04 ±116% -5.0 0.00 perf-profile.children.cycles-pp.tlb_finish_mmu 5.04 ±116% -5.0 0.00 perf-profile.children.cycles-pp.tlb_flush_mmu 5994 ± 2% -15.7% 5054 ± 2% perf-stat.i.context-switches 1466 ± 8% -32.9% 983.03 ± 8% perf-stat.i.cpu-migrations 68.06 +1.4 69.47 perf-stat.i.iTLB-load-miss-rate% 0.03 ± 3% -0.0 0.03 ± 7% perf-stat.overall.dTLB-store-miss-rate% 5741 ± 2% -15.8% 4835 ± 2% perf-stat.ps.context-switches 1407 ± 8% -33.0% 942.99 ± 8% perf-stat.ps.cpu-migrations 5131 ± 40% +186.8% 14716 ± 5% softirqs.CPU0.RCU 6347 ± 30% +185.8% 18142 ± 8% softirqs.CPU1.RCU 4632 ± 13% +203.4% 14054 ± 3% softirqs.CPU10.RCU 3655 ± 3% +269.1% 13494 ± 5% softirqs.CPU11.RCU 4011 ± 12% +251.5% 14100 ± 8% softirqs.CPU12.RCU 4456 ± 28% +216.3% 14097 ± 12% softirqs.CPU13.RCU 3751 ± 4% +259.1% 13470 ± 3% softirqs.CPU14.RCU 3909 ± 9% +247.0% 13566 ± 2% softirqs.CPU15.RCU 3976 ± 6% +235.5% 13340 ± 4% softirqs.CPU16.RCU 4211 ± 20% +245.1% 14533 ± 14% softirqs.CPU17.RCU 6063 ± 63% +124.9% 13636 ± 8% softirqs.CPU18.RCU 4362 ± 32% +230.8% 14434 ± 3% softirqs.CPU19.RCU 3999 ± 3% +256.5% 14258 ± 7% softirqs.CPU2.RCU 5407 ± 29% +176.8% 14966 ± 5% softirqs.CPU20.RCU 4034 ± 14% +254.7% 14309 ± 7% softirqs.CPU21.RCU 3920 ± 8% +245.4% 13544 softirqs.CPU22.RCU 3829 ± 5% +260.7% 13811 ± 4% softirqs.CPU23.RCU 6202 ± 57% +133.9% 14509 ± 6% softirqs.CPU24.RCU 3782 ± 7% +250.6% 13263 ± 3% softirqs.CPU25.RCU 3600 ± 4% +274.9% 13499 ± 8% softirqs.CPU26.RCU 3486 ± 2% +268.7% 12855 ± 2% softirqs.CPU27.RCU 3990 ± 9% +244.4% 13744 ± 5% softirqs.CPU28.RCU 4249 ± 19% +229.1% 13984 ± 13% softirqs.CPU29.RCU 3661 ± 2% +270.9% 13578 ± 5% softirqs.CPU3.RCU 3580 +260.4% 12904 softirqs.CPU30.RCU 3960 ± 14% +220.8% 12703 ± 2% softirqs.CPU31.RCU 15192 ± 8% -9.2% 13790 softirqs.CPU31.TIMER 5046 ± 32% +167.6% 13504 ± 5% softirqs.CPU32.RCU 3461 ± 7% +264.3% 12607 ± 3% softirqs.CPU33.RCU 3569 ± 7% +254.4% 12652 ± 3% softirqs.CPU34.RCU 3719 ± 13% +275.8% 13978 ± 16% softirqs.CPU35.RCU 3433 ± 6% +275.9% 12906 ± 5% softirqs.CPU36.RCU 3749 ± 6% +256.2% 13353 ± 7% softirqs.CPU37.RCU 3596 ± 10% +258.4% 12889 ± 5% softirqs.CPU38.RCU 4498 ± 32% +178.7% 12535 ± 4% softirqs.CPU39.RCU 4117 ± 7% +229.3% 13558 ± 6% softirqs.CPU4.RCU 3547 ± 6% +310.3% 14554 ± 8% softirqs.CPU40.RCU 3652 ± 10% +265.1% 13336 ± 3% softirqs.CPU41.RCU 3388 ± 4% +287.0% 13114 ± 8% softirqs.CPU42.RCU 3314 ± 4% +278.3% 12538 ± 5% softirqs.CPU43.RCU 3386 ± 3% +278.1% 12803 ± 4% softirqs.CPU44.RCU 3497 ± 10% +270.3% 12948 ± 5% softirqs.CPU45.RCU 3592 ± 5% +252.5% 12660 ± 2% softirqs.CPU46.RCU 3511 ± 5% +248.9% 12249 ± 3% softirqs.CPU47.RCU 3579 ± 5% +291.3% 14006 ± 8% softirqs.CPU48.RCU 3496 ± 9% +280.6% 13307 ± 4% softirqs.CPU49.RCU 3772 ± 2% +256.2% 13436 softirqs.CPU5.RCU 3703 ± 4% +262.6% 13428 ± 2% softirqs.CPU50.RCU 3512 ± 6% +271.1% 13034 ± 5% softirqs.CPU51.RCU 4027 ± 21% +229.3% 13261 ± 6% softirqs.CPU52.RCU 3517 ± 6% +324.2% 14919 ± 20% softirqs.CPU53.RCU 3730 ± 5% +255.0% 13244 ± 3% softirqs.CPU54.RCU 3542 ± 4% +277.3% 13364 ± 2% softirqs.CPU55.RCU 3591 ± 8% +279.1% 13615 ± 5% softirqs.CPU56.RCU 4717 ± 46% +173.3% 12894 ± 2% softirqs.CPU57.RCU 3311 ± 3% +304.6% 13399 ± 7% softirqs.CPU58.RCU 3613 ± 7% +278.9% 13694 ± 2% softirqs.CPU59.RCU 4223 ± 12% +224.0% 13685 ± 2% softirqs.CPU6.RCU 3909 ± 25% +232.4% 12994 ± 5% softirqs.CPU60.RCU 4343 ± 28% +207.2% 13343 ± 3% softirqs.CPU61.RCU 3477 ± 7% +281.6% 13270 ± 4% softirqs.CPU62.RCU 3362 ± 4% +328.8% 14419 ± 9% softirqs.CPU63.RCU 4015 ± 16% +236.2% 13499 ± 5% softirqs.CPU64.RCU 4687 ± 44% +179.5% 13100 ± 4% softirqs.CPU65.RCU 3714 ± 11% +267.0% 13631 ± 11% softirqs.CPU66.RCU 3438 ± 5% +287.9% 13339 ± 3% softirqs.CPU67.RCU 3819 ± 13% +251.3% 13415 ± 6% softirqs.CPU68.RCU 3302 ± 5% +305.5% 13389 ± 4% softirqs.CPU69.RCU 3919 ± 9% +244.2% 13491 ± 6% softirqs.CPU7.RCU 3456 ± 3% +292.4% 13563 ± 7% softirqs.CPU70.RCU 3709 ± 13% +295.1% 14655 ± 5% softirqs.CPU71.RCU 3105 ± 6% +332.1% 13420 ± 6% softirqs.CPU72.RCU 3317 ± 8% +289.3% 12917 ± 3% softirqs.CPU73.RCU 3286 ± 3% +281.8% 12544 ± 3% softirqs.CPU74.RCU 3243 ± 4% +282.8% 12414 ± 4% softirqs.CPU75.RCU 3195 ± 3% +285.6% 12322 ± 3% softirqs.CPU76.RCU 3243 ± 6% +274.0% 12131 ± 3% softirqs.CPU77.RCU 3322 ± 6% +273.0% 12391 ± 5% softirqs.CPU78.RCU 3207 ± 3% +293.8% 12629 ± 5% softirqs.CPU79.RCU 4374 ± 13% +215.9% 13817 softirqs.CPU8.RCU 3409 ± 11% +271.5% 12666 softirqs.CPU80.RCU 3475 ± 14% +253.7% 12290 ± 3% softirqs.CPU81.RCU 3625 ± 22% +239.7% 12315 ± 2% softirqs.CPU82.RCU 3192 ± 4% +307.2% 13000 ± 11% softirqs.CPU83.RCU 3328 ± 13% +266.4% 12194 ± 3% softirqs.CPU84.RCU 3380 ± 10% +262.0% 12238 ± 5% softirqs.CPU85.RCU 3199 ± 7% +300.8% 12822 ± 10% softirqs.CPU86.RCU 3378 ± 11% +261.3% 12205 ± 5% softirqs.CPU87.RCU 3442 ± 7% +264.2% 12538 ± 6% softirqs.CPU88.RCU 3242 ± 7% +279.7% 12313 ± 6% softirqs.CPU89.RCU 4646 ± 25% +187.7% 13366 ± 5% softirqs.CPU9.RCU 3078 ± 5% +294.0% 12129 ± 5% softirqs.CPU90.RCU 3091 ± 5% +296.0% 12242 ± 5% softirqs.CPU91.RCU 3247 ± 6% +296.6% 12876 ± 11% softirqs.CPU92.RCU 3253 ± 4% +272.5% 12120 ± 5% softirqs.CPU93.RCU 3180 ± 4% +280.6% 12103 ± 3% softirqs.CPU94.RCU 3395 ± 9% +274.1% 12701 ± 6% softirqs.CPU95.RCU 364674 +250.4% 1277851 softirqs.RCU stress-ng.time.voluntary_context_switches 50000 +-+-----------------------------------------------------------------+ 45000 +-+ ++.++ ++ ++++++++++.++++++++ + +++++++++ +++ + +++.++ +| | : : :: : | 40000 +-+ : O O:: O : O O | 35000 +-+O: : O: O OOOOO O O OOOO O O | | :O :OO :OO : OOO OOO O OOOOO OO | 30000 +-+ : : :: : O O | 25000 +-+ : : :: : | 20000 +-+: : : : : | |: : : : : : | 15000 +-+: : : : : | 10000 OO+: : : : : | |: : : : : : | 5000 +-+: : : : : | 0 +-O-O--O------------------------------------------------------------+ stress-ng.membarrier.ops_per_sec 8000 +-+------------------------------------------------------------------+ | +.+++ ++ +++++++.++++++++.+++ + ++.+++++++ +.++ + + ++.+ ++| 7000 +-+ : : :: : + | 6000 +-+ : : :: : | | : O O:: OOO O O O O | 5000 +-+O: : O: :OOOO O O O O O O | | : O : :O : O O OO O O O | 4000 +-+ : :OO ::O : O O OOOO O O O | |: : : : : : | 3000 +-+: : : : : | 2000 +-+: : : : : | |: : : : : : | 1000 +-+: : : : : | OO : : : : : | 0 +-O-O--O-------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen