Greeting, FYI, we noticed a -12.9% regression of fsmark.files_per_sec due to commit: commit: 3bc6a407d1a090417218ecdb0dd4a92a5d75704b ("[PATCH RFC] SUNRPC: Use zero-copy to perform socket send operations") url: https://github.com/0day-ci/linux/commits/Chuck-Lever/SUNRPC-Use-zero-copy-to-perform-socket-send-operations/20201110-000351 base: git://git.linux-nfs.org/projects/trondmy/linux-nfs.git linux-next in testcase: fsmark on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: iterations: 1x nr_threads: 64t disk: 1BRD_48G fs: xfs fs2: nfsv4 filesize: 4M test_size: 40G sync_method: fsyncBeforeClose cpufreq_governor: performance ucode: 0x5003003 test-description: The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload. test-url: https://sourceforge.net/projects/fsmark/ In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------+ | testcase: change | fsmark: fsmark.files_per_sec -13.3% regression | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | disk=1BRD_48G | | | filesize=4M | | | fs2=nfsv4 | | | fs=xfs | | | iterations=1x | | | nr_threads=64t | | | sync_method=NoSync | | | test_size=40G | | | ucode=0x5003003 | +------------------+---------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/filesize/fs2/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode: gcc-9/performance/1BRD_48G/4M/nfsv4/xfs/1x/x86_64-rhel-8.3/64t/debian-10.4-x86_64-20200603.cgz/fsyncBeforeClose/lkp-csl-2ap2/40G/fsmark/0x5003003 commit: v5.10-rc2 3bc6a407d1 ("SUNRPC: Use zero-copy to perform socket send operations") v5.10-rc2 3bc6a407d1a090417218ecdb0dd ---------------- --------------------------- %stddev %change %stddev \ | \ 13020954 +12.4% 14633883 ± 3% fsmark.app_overhead 558.15 -12.9% 485.88 fsmark.files_per_sec 21.42 +12.5% 24.10 fsmark.time.elapsed_time 21.42 +12.5% 24.10 fsmark.time.elapsed_time.max 1788 ± 5% -13.3% 1551 ± 4% fsmark.time.involuntary_context_switches 245.25 -12.4% 214.75 fsmark.time.percent_of_cpu_this_job_got 1495010 -3.3% 1446006 fsmark.time.voluntary_context_switches 2.54 ± 3% +16.7% 2.97 iostat.cpu.iowait 3965 ± 7% +63.2% 6471 ± 44% softirqs.CPU45.RCU 12441 ± 3% +8.1% 13448 ± 4% meminfo.Active 12332 ± 3% +8.2% 13339 ± 4% meminfo.Active(anon) 27337 ± 7% +41.3% 38640 ± 18% sched_debug.cfs_rq:/.min_vruntime.avg 14415 ± 19% +52.1% 21925 ± 14% sched_debug.cfs_rq:/.min_vruntime.min 1637052 -8.0% 1505871 ± 2% vmstat.io.bo 195349 ± 2% -12.7% 170450 ± 2% vmstat.system.cs 2.74 ± 4% +0.4 3.18 mpstat.cpu.all.iowait% 1.09 ± 7% -0.1 0.94 ± 2% mpstat.cpu.all.irq% 0.14 ± 5% +0.3 0.41 ± 2% mpstat.cpu.all.soft% 3.45 ± 2% -0.5 2.99 ± 2% mpstat.cpu.all.sys% 6813 ± 40% -55.1% 3059 ± 57% numa-meminfo.node0.PageTables 119106 ± 4% -11.5% 105394 ± 11% numa-meminfo.node0.SUnreclaim 12837 ± 28% +30.5% 16748 ± 22% numa-meminfo.node3.Shmem 248191 +8.2% 268539 ± 4% numa-meminfo.node3.Unevictable 26140871 ± 2% -13.6% 22585817 ± 4% cpuidle.C1.time 432703 -24.4% 327005 ± 2% cpuidle.C1.usage 2.216e+09 ± 53% +108.7% 4.624e+09 ± 3% cpuidle.C1E.time 6704163 ± 23% +53.8% 10312626 ± 3% cpuidle.C1E.usage 2.113e+09 ± 57% -94.9% 1.081e+08 ± 94% cpuidle.C6.time 1700 ± 40% -55.1% 764.25 ± 57% numa-vmstat.node0.nr_page_table_pages 29779 ± 4% -11.5% 26356 ± 11% numa-vmstat.node0.nr_slab_unreclaimable 3176 ± 29% +32.8% 4219 ± 22% numa-vmstat.node3.nr_shmem 62047 +8.2% 67134 ± 4% numa-vmstat.node3.nr_unevictable 62047 +8.2% 67134 ± 4% numa-vmstat.node3.nr_zone_unevictable 3082 ± 3% +8.2% 3335 ± 4% proc-vmstat.nr_active_anon 3082 ± 3% +8.2% 3335 ± 4% proc-vmstat.nr_zone_active_anon 33095 ± 20% -30.7% 22929 ± 25% proc-vmstat.numa_pte_updates 76453330 -12.6% 66854033 proc-vmstat.pgalloc_normal 55754409 -17.7% 45900156 proc-vmstat.pgfree 10550 +7.1% 11301 ± 2% proc-vmstat.pgreuse 21843 -16.2% 18302 slabinfo.kmalloc-1k.active_objs 689.50 -15.6% 582.00 slabinfo.kmalloc-1k.active_slabs 22080 -15.6% 18635 slabinfo.kmalloc-1k.num_objs 689.50 -15.6% 582.00 slabinfo.kmalloc-1k.num_slabs 12057 ± 7% -12.6% 10544 ± 4% slabinfo.nfsd_file.active_objs 12250 ± 6% -12.0% 10779 ± 4% slabinfo.nfsd_file.num_objs 3.726e+09 -6.0% 3.502e+09 perf-stat.i.branch-instructions 3.236e+08 ± 2% -8.1% 2.974e+08 perf-stat.i.cache-references 223562 ± 3% -13.9% 192428 ± 2% perf-stat.i.context-switches 4.738e+09 -6.5% 4.428e+09 perf-stat.i.dTLB-loads 2.344e+09 ± 2% -6.8% 2.185e+09 perf-stat.i.dTLB-stores 1.885e+10 -5.8% 1.776e+10 perf-stat.i.instructions 58.38 -6.5% 54.61 perf-stat.i.metric.M/sec 9172 ± 3% -7.8% 8458 perf-stat.i.minor-faults 37419450 ± 3% -9.0% 34056149 ± 2% perf-stat.i.node-load-misses 10313099 ± 2% -11.4% 9134750 ± 5% perf-stat.i.node-loads 9181 ± 3% -7.8% 8467 perf-stat.i.page-faults 3.565e+09 -5.6% 3.365e+09 perf-stat.ps.branch-instructions 3.098e+08 ± 2% -7.7% 2.858e+08 perf-stat.ps.cache-references 214060 ± 3% -13.6% 184935 ± 2% perf-stat.ps.context-switches 4.534e+09 -6.2% 4.255e+09 perf-stat.ps.dTLB-loads 2.243e+09 -6.4% 2.099e+09 perf-stat.ps.dTLB-stores 1.803e+10 -5.4% 1.706e+10 perf-stat.ps.instructions 8793 ± 3% -7.2% 8164 ± 2% perf-stat.ps.minor-faults 35826191 ± 3% -8.7% 32716417 ± 2% perf-stat.ps.node-load-misses 9874765 ± 2% -11.0% 8786772 ± 5% perf-stat.ps.node-loads 8802 ± 3% -7.2% 8172 ± 2% perf-stat.ps.page-faults 4.219e+11 +3.9% 4.384e+11 perf-stat.total.instructions 1283 ± 44% -41.5% 750.50 ± 16% interrupts.CPU100.NMI:Non-maskable_interrupts 1283 ± 44% -41.5% 750.50 ± 16% interrupts.CPU100.PMI:Performance_monitoring_interrupts 603.75 ± 19% +82.7% 1103 ± 33% interrupts.CPU105.NMI:Non-maskable_interrupts 603.75 ± 19% +82.7% 1103 ± 33% interrupts.CPU105.PMI:Performance_monitoring_interrupts 83.00 ± 15% -43.1% 47.25 ± 29% interrupts.CPU115.RES:Rescheduling_interrupts 2540 ± 11% -24.4% 1919 ± 24% interrupts.CPU144.CAL:Function_call_interrupts 2703 ± 14% -40.1% 1619 ± 29% interrupts.CPU152.CAL:Function_call_interrupts 2791 ± 10% -31.7% 1905 ± 28% interrupts.CPU156.CAL:Function_call_interrupts 2660 ± 6% -37.7% 1657 ± 32% interrupts.CPU163.CAL:Function_call_interrupts 2.50 ± 72% +4810.0% 122.75 ±164% interrupts.CPU163.TLB:TLB_shootdowns 2499 ± 11% -34.5% 1636 ± 35% interrupts.CPU167.CAL:Function_call_interrupts 980.75 ± 18% -36.3% 624.50 ± 17% interrupts.CPU172.NMI:Non-maskable_interrupts 980.75 ± 18% -36.3% 624.50 ± 17% interrupts.CPU172.PMI:Performance_monitoring_interrupts 2671 ± 14% -24.8% 2009 ± 19% interrupts.CPU173.CAL:Function_call_interrupts 2.25 ± 57% +5244.4% 120.25 ±164% interrupts.CPU173.TLB:TLB_shootdowns 1048 ± 17% -40.2% 627.25 ± 35% interrupts.CPU175.NMI:Non-maskable_interrupts 1048 ± 17% -40.2% 627.25 ± 35% interrupts.CPU175.PMI:Performance_monitoring_interrupts 848.25 ± 24% -39.9% 509.75 ± 20% interrupts.CPU176.NMI:Non-maskable_interrupts 848.25 ± 24% -39.9% 509.75 ± 20% interrupts.CPU176.PMI:Performance_monitoring_interrupts 3.50 ± 58% +3421.4% 123.25 ±157% interrupts.CPU178.TLB:TLB_shootdowns 2913 ± 4% -29.5% 2055 ± 19% interrupts.CPU184.CAL:Function_call_interrupts 862.00 ± 20% -34.3% 566.75 ± 14% interrupts.CPU184.NMI:Non-maskable_interrupts 862.00 ± 20% -34.3% 566.75 ± 14% interrupts.CPU184.PMI:Performance_monitoring_interrupts 931.25 ± 34% -45.2% 510.25 ± 20% interrupts.CPU186.NMI:Non-maskable_interrupts 931.25 ± 34% -45.2% 510.25 ± 20% interrupts.CPU186.PMI:Performance_monitoring_interrupts 840.25 ± 17% -27.8% 606.75 ± 20% interrupts.CPU21.NMI:Non-maskable_interrupts 840.25 ± 17% -27.8% 606.75 ± 20% interrupts.CPU21.PMI:Performance_monitoring_interrupts 2851 ± 16% -25.1% 2135 ± 14% interrupts.CPU29.CAL:Function_call_interrupts 560.75 ± 13% +67.8% 941.00 ± 17% interrupts.CPU30.NMI:Non-maskable_interrupts 560.75 ± 13% +67.8% 941.00 ± 17% interrupts.CPU30.PMI:Performance_monitoring_interrupts 615.25 ± 16% +44.9% 891.75 ± 16% interrupts.CPU31.NMI:Non-maskable_interrupts 615.25 ± 16% +44.9% 891.75 ± 16% interrupts.CPU31.PMI:Performance_monitoring_interrupts 34.25 ± 28% +156.2% 87.75 ± 26% interrupts.CPU34.RES:Rescheduling_interrupts 1.50 ±137% +7700.0% 117.00 ±166% interrupts.CPU39.TLB:TLB_shootdowns 2394 ± 4% +18.6% 2840 ± 9% interrupts.CPU41.CAL:Function_call_interrupts 610.75 ± 21% -41.3% 358.75 ± 33% interrupts.CPU44.NMI:Non-maskable_interrupts 610.75 ± 21% -41.3% 358.75 ± 33% interrupts.CPU44.PMI:Performance_monitoring_interrupts 2608 ± 4% -36.5% 1655 ± 35% interrupts.CPU48.CAL:Function_call_interrupts 2820 ± 8% -34.7% 1841 ± 32% interrupts.CPU55.CAL:Function_call_interrupts 1249 ± 47% -49.6% 630.00 ± 44% interrupts.CPU56.NMI:Non-maskable_interrupts 1249 ± 47% -49.6% 630.00 ± 44% interrupts.CPU56.PMI:Performance_monitoring_interrupts 609.50 ± 29% +167.1% 1628 ± 78% interrupts.CPU67.NMI:Non-maskable_interrupts 609.50 ± 29% +167.1% 1628 ± 78% interrupts.CPU67.PMI:Performance_monitoring_interrupts 3176 ± 31% -35.7% 2043 ± 31% interrupts.CPU72.CAL:Function_call_interrupts 2671 ± 7% -40.8% 1582 ± 31% interrupts.CPU73.CAL:Function_call_interrupts 2626 ± 5% -32.6% 1770 ± 22% interrupts.CPU74.CAL:Function_call_interrupts 2674 ± 6% -35.1% 1735 ± 22% interrupts.CPU77.CAL:Function_call_interrupts 966.50 ± 17% -49.0% 492.75 ± 18% interrupts.CPU80.NMI:Non-maskable_interrupts 966.50 ± 17% -49.0% 492.75 ± 18% interrupts.CPU80.PMI:Performance_monitoring_interrupts 2659 ± 19% -29.4% 1878 ± 26% interrupts.CPU84.CAL:Function_call_interrupts 824.00 ± 11% -33.1% 551.25 ± 21% interrupts.CPU88.NMI:Non-maskable_interrupts 824.00 ± 11% -33.1% 551.25 ± 21% interrupts.CPU88.PMI:Performance_monitoring_interrupts 1012 ± 39% -41.9% 588.00 ± 37% interrupts.CPU98.NMI:Non-maskable_interrupts 1012 ± 39% -41.9% 588.00 ± 37% interrupts.CPU98.PMI:Performance_monitoring_interrupts 32.98 ± 3% -3.4 29.60 perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 32.98 ± 3% -3.4 29.61 perf-profile.calltrace.cycles-pp.ret_from_fork 21.71 ± 3% -2.5 19.25 ± 2% perf-profile.calltrace.cycles-pp.nfsd.kthread.ret_from_fork 16.07 ± 3% -2.2 13.84 perf-profile.calltrace.cycles-pp.svc_process.nfsd.kthread.ret_from_fork 16.07 ± 3% -2.2 13.84 perf-profile.calltrace.cycles-pp.svc_process_common.svc_process.nfsd.kthread.ret_from_fork 16.01 ± 3% -2.2 13.78 perf-profile.calltrace.cycles-pp.nfsd_dispatch.svc_process_common.svc_process.nfsd.kthread 15.96 ± 3% -2.2 13.74 perf-profile.calltrace.cycles-pp.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process.nfsd 16.78 ± 4% -1.6 15.17 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 16.68 ± 4% -1.6 15.08 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.34 ± 3% -1.3 8.09 ± 2% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.41 ± 3% -1.2 8.16 ± 2% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.39 ± 3% -1.2 8.14 ± 2% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.33 ± 3% -1.2 8.09 ± 2% perf-profile.calltrace.cycles-pp.nfs_file_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 9.01 ± 3% -1.2 7.80 ± 2% perf-profile.calltrace.cycles-pp.generic_perform_write.nfs_file_write.new_sync_write.vfs_write.ksys_write 9.75 ± 3% -1.1 8.66 perf-profile.calltrace.cycles-pp.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process 9.73 ± 3% -1.1 8.64 perf-profile.calltrace.cycles-pp.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch.svc_process_common 9.68 ± 3% -1.1 8.59 perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound 9.68 ± 3% -1.1 8.59 perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync.nfsd_commit 9.68 ± 3% -1.1 8.59 perf-profile.calltrace.cycles-pp.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync 9.69 ± 3% -1.1 8.61 perf-profile.calltrace.cycles-pp.file_write_and_wait_range.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch 5.84 ± 4% -1.1 4.78 ± 2% perf-profile.calltrace.cycles-pp.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process 5.81 ± 4% -1.1 4.75 ± 2% perf-profile.calltrace.cycles-pp.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch.svc_process_common 5.80 ± 4% -1.1 4.75 ± 2% perf-profile.calltrace.cycles-pp.do_iter_write.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch 5.79 ± 4% -1.0 4.74 ± 2% perf-profile.calltrace.cycles-pp.do_iter_readv_writev.do_iter_write.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound 5.79 ± 4% -1.0 4.74 ± 2% perf-profile.calltrace.cycles-pp.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write.nfsd_vfs_write.nfsd4_write 5.54 ± 4% -0.9 4.59 ± 2% perf-profile.calltrace.cycles-pp.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write.nfsd_vfs_write 5.54 ± 4% -0.9 4.59 ± 2% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write 5.51 ± 4% -0.9 4.57 ± 2% perf-profile.calltrace.cycles-pp.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev 11.19 ± 3% -0.9 10.29 ± 2% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 10.69 ± 3% -0.8 9.86 ± 2% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 1.78 ± 6% -0.7 1.09 ± 3% perf-profile.calltrace.cycles-pp.iomap_writepages.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range 1.77 ± 6% -0.7 1.08 ± 3% perf-profile.calltrace.cycles-pp.write_cache_pages.iomap_writepages.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range 5.28 ± 3% -0.6 4.64 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.28 ± 3% -0.6 4.64 ± 2% perf-profile.calltrace.cycles-pp.do_fsync.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.28 ± 3% -0.6 4.64 ± 2% perf-profile.calltrace.cycles-pp.nfs_file_fsync.do_fsync.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.23 ± 7% -0.6 0.62 ± 4% perf-profile.calltrace.cycles-pp.iomap_writepage_map.write_cache_pages.iomap_writepages.xfs_vm_writepages.do_writepages 4.30 ± 4% -0.6 3.69 ± 3% perf-profile.calltrace.cycles-pp.rpc_free_task.rpc_async_release.process_one_work.worker_thread.kthread 4.30 ± 4% -0.6 3.69 ± 3% perf-profile.calltrace.cycles-pp.rpc_async_release.process_one_work.worker_thread.kthread.ret_from_fork 4.37 ± 4% -0.6 3.75 ± 5% perf-profile.calltrace.cycles-pp.nfs_write_end.generic_perform_write.nfs_file_write.new_sync_write.vfs_write 4.75 ± 3% -0.6 4.19 ± 2% perf-profile.calltrace.cycles-pp.file_write_and_wait_range.nfs_file_fsync.do_fsync.__x64_sys_fsync.do_syscall_64 3.42 ± 5% -0.6 2.86 ± 3% perf-profile.calltrace.cycles-pp.nfs_write_completion.rpc_free_task.rpc_async_release.process_one_work.worker_thread 2.69 -0.5 2.21 ± 3% perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 2.65 -0.5 2.18 ± 3% perf-profile.calltrace.cycles-pp.memcpy_erms.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply.iomap_file_buffered_write 3.16 ± 5% -0.4 2.73 ± 7% perf-profile.calltrace.cycles-pp.nfs_updatepage.nfs_write_end.generic_perform_write.nfs_file_write.new_sync_write 2.34 ± 5% -0.4 1.92 ± 4% perf-profile.calltrace.cycles-pp.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release.process_one_work 3.59 ± 3% -0.4 3.19 ± 2% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.file_write_and_wait_range.nfs_file_fsync.do_fsync.__x64_sys_fsync 3.59 ± 3% -0.4 3.19 ± 2% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range.nfs_file_fsync.do_fsync 3.59 ± 3% -0.4 3.19 ± 2% perf-profile.calltrace.cycles-pp.nfs_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range.nfs_file_fsync 3.56 ± 3% -0.4 3.16 ± 2% perf-profile.calltrace.cycles-pp.write_cache_pages.nfs_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range 7.90 ± 4% -0.4 7.51 perf-profile.calltrace.cycles-pp.submit_bio.iomap_submit_ioend.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range 7.90 ± 4% -0.4 7.51 perf-profile.calltrace.cycles-pp.iomap_submit_ioend.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range 2.96 ± 4% -0.3 2.63 ± 2% perf-profile.calltrace.cycles-pp.nfs_writepages_callback.write_cache_pages.nfs_writepages.do_writepages.__filemap_fdatawrite_range 2.23 ± 3% -0.3 1.91 ± 2% perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.generic_perform_write.nfs_file_write.new_sync_write.vfs_write 0.58 ± 3% -0.3 0.27 ±100% perf-profile.calltrace.cycles-pp.mem_cgroup_charge.__add_to_page_cache_locked.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin 2.53 -0.3 2.21 ± 4% perf-profile.calltrace.cycles-pp.memcpy_to_page._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg 2.52 ± 2% -0.3 2.20 ± 4% perf-profile.calltrace.cycles-pp.memcpy_erms.memcpy_to_page._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter 2.92 ± 4% -0.3 2.61 ± 2% perf-profile.calltrace.cycles-pp.nfs_do_writepage.nfs_writepages_callback.write_cache_pages.nfs_writepages.do_writepages 2.18 ± 3% -0.3 1.88 ± 2% perf-profile.calltrace.cycles-pp.copyin.iov_iter_copy_from_user_atomic.generic_perform_write.nfs_file_write.new_sync_write 2.17 ± 3% -0.3 1.86 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.iov_iter_copy_from_user_atomic.generic_perform_write.nfs_file_write 0.55 ± 4% -0.3 0.25 ±100% perf-profile.calltrace.cycles-pp.__test_set_page_writeback.nfs_do_writepage.nfs_writepages_callback.write_cache_pages.nfs_writepages 2.55 ± 2% -0.3 2.25 ± 3% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet6_recvmsg 2.23 ± 4% -0.3 1.95 ± 7% perf-profile.calltrace.cycles-pp.nfs_writepage_setup.nfs_updatepage.nfs_write_end.generic_perform_write.nfs_file_write 2.07 ± 3% -0.3 1.82 ± 2% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write.new_sync_write 2.13 ± 3% -0.2 1.88 ± 2% perf-profile.calltrace.cycles-pp.nfs_write_begin.generic_perform_write.nfs_file_write.new_sync_write.vfs_write 2.05 ± 3% -0.2 1.80 ± 2% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write 0.66 ± 3% -0.2 0.42 ± 57% perf-profile.calltrace.cycles-pp.io_schedule.wait_on_page_bit.__filemap_fdatawait_range.file_write_and_wait_range.nfs_file_fsync 0.62 ± 4% -0.2 0.39 ± 57% perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_page_function.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback 1.25 ± 7% -0.2 1.02 ± 2% perf-profile.calltrace.cycles-pp.iomap_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 0.64 ± 3% -0.2 0.41 ± 57% perf-profile.calltrace.cycles-pp.schedule.io_schedule.wait_on_page_bit.__filemap_fdatawait_range.file_write_and_wait_range 0.62 ± 3% -0.2 0.40 ± 57% perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.wait_on_page_bit.__filemap_fdatawait_range 1.40 ± 6% -0.2 1.20 ± 3% perf-profile.calltrace.cycles-pp.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 1.33 ± 6% -0.2 1.13 ± 3% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write 1.32 ± 6% -0.2 1.12 ± 3% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply 1.17 ± 5% -0.2 1.00 perf-profile.calltrace.cycles-pp.__filemap_fdatawait_range.file_write_and_wait_range.nfs_file_fsync.do_fsync.__x64_sys_fsync 1.48 ± 3% -0.2 1.31 ± 2% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write 0.80 ± 4% -0.1 0.66 ± 3% perf-profile.calltrace.cycles-pp.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release 0.72 ± 5% -0.1 0.58 ± 4% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 0.89 ± 7% -0.1 0.76 ± 6% perf-profile.calltrace.cycles-pp.end_page_writeback.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release 1.04 ± 3% -0.1 0.91 ± 3% perf-profile.calltrace.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin 0.80 ± 7% -0.1 0.67 ± 3% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor 0.85 ± 8% -0.1 0.73 ± 6% perf-profile.calltrace.cycles-pp.test_clear_page_writeback.end_page_writeback.nfs_end_page_writeback.nfs_write_completion.rpc_free_task 0.74 ± 4% -0.1 0.62 ± 4% perf-profile.calltrace.cycles-pp.wait_on_page_bit.__filemap_fdatawait_range.file_write_and_wait_range.nfs_file_fsync.do_fsync 0.67 ± 4% -0.1 0.55 ± 4% perf-profile.calltrace.cycles-pp.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion.rpc_free_task 0.86 ± 4% -0.1 0.75 ± 5% perf-profile.calltrace.cycles-pp.__set_page_dirty_nobuffers.nfs_writepage_setup.nfs_updatepage.nfs_write_end.generic_perform_write 0.63 ± 4% -0.1 0.52 ± 3% perf-profile.calltrace.cycles-pp.wake_page_function.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion 0.66 ± 3% -0.1 0.56 ± 7% perf-profile.calltrace.cycles-pp.iomap_set_range_uptodate.iomap_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write 0.61 ± 5% -0.1 0.56 ± 5% perf-profile.calltrace.cycles-pp.xfs_end_io.process_one_work.worker_thread.kthread.ret_from_fork 0.61 ± 5% -0.1 0.56 ± 5% perf-profile.calltrace.cycles-pp.xfs_end_ioend.xfs_end_io.process_one_work.worker_thread.kthread 0.59 ± 6% -0.1 0.54 ± 5% perf-profile.calltrace.cycles-pp.iomap_finish_ioends.xfs_end_ioend.xfs_end_io.process_one_work.worker_thread 0.83 ± 2% -0.1 0.77 perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.brd_insert_page.brd_do_bvec.brd_submit_bio.submit_bio_noacct 0.77 ± 2% -0.0 0.72 perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.brd_insert_page.brd_do_bvec.brd_submit_bio 3.13 ± 2% +0.4 3.50 ± 4% perf-profile.calltrace.cycles-pp.inet6_recvmsg.svc_tcp_recvfrom.svc_handle_xprt.svc_recv.nfsd 3.12 ± 2% +0.4 3.50 ± 4% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet6_recvmsg.svc_tcp_recvfrom.svc_handle_xprt.svc_recv 0.00 +0.6 0.61 ± 6% perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_recvmsg.inet6_recvmsg.svc_tcp_recvfrom.svc_handle_xprt 0.00 +3.0 2.98 perf-profile.calltrace.cycles-pp.memcpy_erms.skb_copy_ubufs.__netif_receive_skb_core.__netif_receive_skb_one_core.process_backlog 0.00 +3.7 3.71 perf-profile.calltrace.cycles-pp.skb_copy_ubufs.__netif_receive_skb_core.__netif_receive_skb_one_core.process_backlog.net_rx_action 0.00 +3.7 3.72 perf-profile.calltrace.cycles-pp.__netif_receive_skb_core.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start 0.00 +3.9 3.88 ± 2% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack 0.00 +3.9 3.89 ± 2% perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack 0.00 +3.9 3.89 ± 2% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq 0.00 +3.9 3.90 ± 2% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip6_finish_output2 0.00 +3.9 3.90 ± 2% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip6_finish_output2.ip6_output 0.00 +3.9 3.90 ± 2% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq.__local_bh_enable_ip 0.00 +3.9 3.90 ± 2% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip6_finish_output2.ip6_output.ip6_xmit.inet6_csk_xmit 0.00 +3.9 3.90 ± 2% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip6_finish_output2.ip6_output.ip6_xmit 0.00 +3.9 3.94 ± 2% perf-profile.calltrace.cycles-pp.ip6_finish_output2.ip6_output.ip6_xmit.inet6_csk_xmit.__tcp_transmit_skb 0.00 +4.0 3.96 perf-profile.calltrace.cycles-pp.ip6_output.ip6_xmit.inet6_csk_xmit.__tcp_transmit_skb.tcp_write_xmit 0.00 +4.0 3.96 perf-profile.calltrace.cycles-pp.ip6_xmit.inet6_csk_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked 0.00 +4.0 3.99 perf-profile.calltrace.cycles-pp.inet6_csk_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg 0.00 +4.0 4.03 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg 0.00 +4.1 4.09 perf-profile.calltrace.cycles-pp.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.xprt_sock_sendmsg 42.90 ± 5% +4.1 47.02 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 43.95 ± 4% +4.2 48.13 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 49.22 ± 2% +4.9 54.12 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 49.23 ± 2% +4.9 54.14 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 49.22 ± 2% +4.9 54.13 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 49.50 ± 2% +5.0 54.45 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 32.98 ± 3% -3.4 29.60 perf-profile.children.cycles-pp.kthread 32.98 ± 3% -3.4 29.61 perf-profile.children.cycles-pp.ret_from_fork 21.71 ± 3% -2.5 19.25 ± 2% perf-profile.children.cycles-pp.nfsd 16.07 ± 3% -2.2 13.84 perf-profile.children.cycles-pp.svc_process 16.07 ± 3% -2.2 13.84 perf-profile.children.cycles-pp.svc_process_common 16.01 ± 3% -2.2 13.78 perf-profile.children.cycles-pp.nfsd_dispatch 15.96 ± 3% -2.2 13.74 perf-profile.children.cycles-pp.nfsd4_proc_compound 14.45 ± 2% -1.7 12.79 perf-profile.children.cycles-pp.file_write_and_wait_range 17.18 ± 4% -1.6 15.62 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 17.07 ± 4% -1.5 15.53 ± 2% perf-profile.children.cycles-pp.do_syscall_64 13.26 ± 2% -1.5 11.78 perf-profile.children.cycles-pp.__filemap_fdatawrite_range 13.26 ± 2% -1.5 11.78 perf-profile.children.cycles-pp.do_writepages 9.34 ± 3% -1.3 8.09 ± 2% perf-profile.children.cycles-pp.nfs_file_write 9.60 ± 3% -1.2 8.37 ± 2% perf-profile.children.cycles-pp.new_sync_write 9.66 ± 3% -1.2 8.44 ± 2% perf-profile.children.cycles-pp.ksys_write 9.64 ± 3% -1.2 8.43 ± 2% perf-profile.children.cycles-pp.vfs_write 9.01 ± 3% -1.2 7.81 ± 2% perf-profile.children.cycles-pp.generic_perform_write 9.75 ± 3% -1.1 8.66 perf-profile.children.cycles-pp.nfsd_commit 9.73 ± 3% -1.1 8.64 perf-profile.children.cycles-pp.xfs_file_fsync 5.34 ± 2% -1.1 4.25 ± 2% perf-profile.children.cycles-pp.write_cache_pages 9.68 ± 3% -1.1 8.59 perf-profile.children.cycles-pp.xfs_vm_writepages 5.84 ± 4% -1.1 4.78 ± 2% perf-profile.children.cycles-pp.nfsd4_write 5.81 ± 4% -1.1 4.75 ± 2% perf-profile.children.cycles-pp.nfsd_vfs_write 5.80 ± 4% -1.1 4.75 ± 2% perf-profile.children.cycles-pp.do_iter_write 5.79 ± 4% -1.0 4.74 ± 2% perf-profile.children.cycles-pp.do_iter_readv_writev 5.79 ± 4% -1.0 4.74 ± 2% perf-profile.children.cycles-pp.xfs_file_buffered_aio_write 5.54 ± 4% -0.9 4.59 ± 2% perf-profile.children.cycles-pp.iomap_file_buffered_write 5.54 ± 4% -0.9 4.59 ± 2% perf-profile.children.cycles-pp.iomap_apply 5.52 ± 3% -0.9 4.57 ± 2% perf-profile.children.cycles-pp.iomap_write_actor 8.46 ± 3% -0.9 7.56 perf-profile.children.cycles-pp.submit_bio_noacct 8.46 ± 3% -0.9 7.56 perf-profile.children.cycles-pp.brd_submit_bio 11.19 ± 3% -0.9 10.29 ± 2% perf-profile.children.cycles-pp.worker_thread 8.46 ± 3% -0.9 7.56 perf-profile.children.cycles-pp.submit_bio 8.42 ± 3% -0.9 7.53 perf-profile.children.cycles-pp.brd_do_bvec 8.40 ± 3% -0.9 7.51 perf-profile.children.cycles-pp.iomap_submit_ioend 10.69 ± 3% -0.8 9.86 ± 2% perf-profile.children.cycles-pp.process_one_work 4.91 ± 2% -0.8 4.12 ± 2% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic 1.78 ± 6% -0.7 1.09 ± 3% perf-profile.children.cycles-pp.iomap_writepages 4.50 ± 4% -0.6 3.85 ± 3% perf-profile.children.cycles-pp.rpc_free_task 5.28 ± 3% -0.6 4.64 ± 2% perf-profile.children.cycles-pp.__x64_sys_fsync 5.28 ± 3% -0.6 4.64 ± 2% perf-profile.children.cycles-pp.do_fsync 5.28 ± 3% -0.6 4.64 ± 2% perf-profile.children.cycles-pp.nfs_file_fsync 1.23 ± 7% -0.6 0.62 ± 4% perf-profile.children.cycles-pp.iomap_writepage_map 4.30 ± 4% -0.6 3.69 ± 3% perf-profile.children.cycles-pp.rpc_async_release 4.37 ± 4% -0.6 3.76 ± 5% perf-profile.children.cycles-pp.nfs_write_end 3.42 ± 5% -0.6 2.87 ± 3% perf-profile.children.cycles-pp.nfs_write_completion 5.67 ± 4% -0.6 5.12 ± 2% perf-profile.children.cycles-pp.brd_insert_page 5.11 ± 4% -0.5 4.66 ± 3% perf-profile.children.cycles-pp._raw_spin_lock 3.40 ± 4% -0.5 2.94 perf-profile.children.cycles-pp.grab_cache_page_write_begin 3.37 ± 4% -0.4 2.92 perf-profile.children.cycles-pp.pagecache_get_page 3.16 ± 4% -0.4 2.73 ± 7% perf-profile.children.cycles-pp.nfs_updatepage 2.34 ± 6% -0.4 1.92 ± 4% perf-profile.children.cycles-pp.nfs_end_page_writeback 3.59 ± 3% -0.4 3.19 ± 2% perf-profile.children.cycles-pp.nfs_writepages 1.84 ± 2% -0.3 1.52 ± 4% perf-profile.children.cycles-pp.__schedule 2.96 ± 4% -0.3 2.63 ± 2% perf-profile.children.cycles-pp.nfs_writepages_callback 2.53 -0.3 2.21 ± 4% perf-profile.children.cycles-pp.memcpy_to_page 2.93 ± 4% -0.3 2.62 ± 2% perf-profile.children.cycles-pp.nfs_do_writepage 2.18 ± 3% -0.3 1.88 ± 2% perf-profile.children.cycles-pp.copyin 2.29 ± 4% -0.3 1.98 perf-profile.children.cycles-pp.add_to_page_cache_lru 2.18 ± 3% -0.3 1.88 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 2.56 ± 2% -0.3 2.26 ± 3% perf-profile.children.cycles-pp._copy_to_iter 2.24 ± 4% -0.3 1.95 ± 6% perf-profile.children.cycles-pp.nfs_writepage_setup 2.14 ± 3% -0.3 1.89 ± 2% perf-profile.children.cycles-pp.nfs_write_begin 1.49 ± 2% -0.2 1.24 ± 5% perf-profile.children.cycles-pp.schedule 1.25 ± 7% -0.2 1.03 ± 2% perf-profile.children.cycles-pp.iomap_write_end 1.53 ± 5% -0.2 1.31 ± 2% perf-profile.children.cycles-pp.__add_to_page_cache_locked 1.41 ± 5% -0.2 1.20 ± 3% perf-profile.children.cycles-pp.iomap_write_begin 1.17 ± 3% -0.2 0.97 ± 2% perf-profile.children.cycles-pp.try_to_wake_up 1.07 -0.2 0.88 ± 5% perf-profile.children.cycles-pp.pick_next_task_fair 1.19 ± 5% -0.2 1.01 perf-profile.children.cycles-pp.__filemap_fdatawait_range 0.97 ± 4% -0.2 0.80 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair 0.98 ± 4% -0.2 0.81 ± 2% perf-profile.children.cycles-pp.ttwu_do_activate 1.43 ± 6% -0.2 1.26 ± 5% perf-profile.children.cycles-pp.end_page_writeback 1.41 ± 6% -0.2 1.24 ± 4% perf-profile.children.cycles-pp.test_clear_page_writeback 0.91 ± 3% -0.2 0.74 ± 3% perf-profile.children.cycles-pp.enqueue_entity 0.94 -0.2 0.78 ± 5% perf-profile.children.cycles-pp.newidle_balance 0.57 ± 3% -0.2 0.42 ± 6% perf-profile.children.cycles-pp.tcp_rcv_established 0.82 ± 4% -0.2 0.67 ± 3% perf-profile.children.cycles-pp.wake_up_page_bit 1.04 ± 5% -0.2 0.89 ± 3% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.58 ± 3% -0.1 0.43 ± 6% perf-profile.children.cycles-pp.tcp_v6_do_rcv 0.86 ± 3% -0.1 0.72 ± 3% perf-profile.children.cycles-pp.__wake_up_common 0.73 ± 8% -0.1 0.59 ± 6% perf-profile.children.cycles-pp.account_page_dirtied 1.05 ± 2% -0.1 0.91 ± 4% perf-profile.children.cycles-pp.load_balance 0.72 ± 5% -0.1 0.59 ± 3% perf-profile.children.cycles-pp.poll_idle 0.78 ± 6% -0.1 0.65 ± 6% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.61 ± 4% -0.1 0.47 ± 4% perf-profile.children.cycles-pp.sched_ttwu_pending 0.69 ± 3% -0.1 0.56 ± 3% perf-profile.children.cycles-pp.__account_scheduler_latency 0.44 ± 3% -0.1 0.31 ± 4% perf-profile.children.cycles-pp.__release_sock 0.55 ± 4% -0.1 0.42 ± 5% perf-profile.children.cycles-pp.release_sock 0.29 ± 3% -0.1 0.16 ± 6% perf-profile.children.cycles-pp.tcp_ack 0.27 ± 3% -0.1 0.15 ± 4% perf-profile.children.cycles-pp.tcp_clean_rtx_queue 0.56 ± 4% -0.1 0.43 ± 6% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle 0.97 ± 4% -0.1 0.85 ± 2% perf-profile.children.cycles-pp.__test_set_page_writeback 0.92 ± 4% -0.1 0.80 ± 5% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers 0.75 ± 4% -0.1 0.64 ± 5% perf-profile.children.cycles-pp.wait_on_page_bit 0.94 ± 2% -0.1 0.83 ± 5% perf-profile.children.cycles-pp.find_busiest_group 0.65 ± 3% -0.1 0.53 ± 2% perf-profile.children.cycles-pp.wake_page_function 0.73 ± 4% -0.1 0.62 ± 5% perf-profile.children.cycles-pp.mem_cgroup_charge 0.54 ± 7% -0.1 0.42 ± 6% perf-profile.children.cycles-pp.__mod_memcg_state 0.93 ± 2% -0.1 0.82 ± 5% perf-profile.children.cycles-pp.update_sd_lb_stats 0.66 ± 3% -0.1 0.55 ± 5% perf-profile.children.cycles-pp.io_schedule 0.89 ± 4% -0.1 0.79 ± 3% perf-profile.children.cycles-pp.__nfs_commit_inode 0.66 ± 3% -0.1 0.57 ± 7% perf-profile.children.cycles-pp.iomap_set_range_uptodate 0.50 ± 8% -0.1 0.40 ± 4% perf-profile.children.cycles-pp.nfs_ctx_key_to_expire 0.38 ± 15% -0.1 0.28 ± 13% perf-profile.children.cycles-pp.__set_page_dirty 0.28 ± 5% -0.1 0.18 ± 13% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.27 ± 17% -0.1 0.17 ± 13% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.20 ± 9% -0.1 0.11 ± 24% perf-profile.children.cycles-pp.xfs_ilock 0.27 ± 5% -0.1 0.17 ± 14% perf-profile.children.cycles-pp.rwsem_optimistic_spin 0.22 ± 7% -0.1 0.13 ± 19% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.53 ± 2% -0.1 0.44 ± 2% perf-profile.children.cycles-pp.stack_trace_save_tsk 1.06 ± 3% -0.1 0.98 perf-profile.children.cycles-pp.nfs_commit_release 1.06 ± 3% -0.1 0.98 perf-profile.children.cycles-pp.nfs_commit_release_pages 0.60 ± 3% -0.1 0.52 ± 3% perf-profile.children.cycles-pp.prep_new_page 0.72 ± 3% -0.1 0.64 perf-profile.children.cycles-pp.lru_cache_add 0.55 ± 5% -0.1 0.47 ± 2% perf-profile.children.cycles-pp.__radix_tree_lookup 0.41 ± 4% -0.1 0.33 ± 2% perf-profile.children.cycles-pp.schedule_idle 0.67 ± 3% -0.1 0.60 ± 2% perf-profile.children.cycles-pp.pagevec_lru_move_fn 0.39 ± 4% -0.1 0.32 ± 6% perf-profile.children.cycles-pp.ip6_protocol_deliver_rcu 0.39 ± 3% -0.1 0.32 ± 6% perf-profile.children.cycles-pp.ip6_input 0.38 ± 4% -0.1 0.31 ± 7% perf-profile.children.cycles-pp.tcp_v6_rcv 0.55 ± 3% -0.1 0.48 ± 2% perf-profile.children.cycles-pp.brd_lookup_page 0.51 ± 3% -0.1 0.44 ± 5% perf-profile.children.cycles-pp.clear_page_erms 0.46 ± 4% -0.1 0.39 ± 3% perf-profile.children.cycles-pp.arch_stack_walk 0.39 ± 4% -0.1 0.32 ± 6% perf-profile.children.cycles-pp.ip6_input_finish 0.40 ± 3% -0.1 0.34 ± 6% perf-profile.children.cycles-pp.ipv6_rcv 0.22 ± 5% -0.1 0.15 ± 7% perf-profile.children.cycles-pp.svc_xprt_do_enqueue 0.49 ± 3% -0.1 0.43 ± 2% perf-profile.children.cycles-pp.__pagevec_lru_add_fn 0.61 ± 5% -0.1 0.56 ± 5% perf-profile.children.cycles-pp.xfs_end_io 0.61 ± 5% -0.1 0.56 ± 5% perf-profile.children.cycles-pp.xfs_end_ioend 0.59 ± 6% -0.1 0.54 ± 5% perf-profile.children.cycles-pp.iomap_finish_ioends 0.59 ± 6% -0.1 0.54 ± 5% perf-profile.children.cycles-pp.iomap_finish_ioend 0.47 ± 4% -0.1 0.42 ± 4% perf-profile.children.cycles-pp.nfs_page_group_sync_on_bit 0.33 ± 4% -0.0 0.28 ± 3% perf-profile.children.cycles-pp.unwind_next_frame 0.18 ± 3% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.svc_xprt_received 0.18 ± 5% -0.0 0.14 ± 6% perf-profile.children.cycles-pp.do_nfsd_create 0.39 ± 3% -0.0 0.35 ± 8% perf-profile.children.cycles-pp.nfs_request_add_commit_list 0.37 ± 3% -0.0 0.32 ± 3% perf-profile.children.cycles-pp.xas_load 0.21 ± 4% -0.0 0.16 ± 6% perf-profile.children.cycles-pp.nfsd4_open 0.15 ± 5% -0.0 0.11 ± 10% perf-profile.children.cycles-pp.schedule_timeout 0.21 ± 5% -0.0 0.17 ± 6% perf-profile.children.cycles-pp.nfs_page_group_lock 0.21 ± 3% -0.0 0.17 ± 11% perf-profile.children.cycles-pp.dec_zone_page_state 0.14 ± 6% -0.0 0.10 ± 8% perf-profile.children.cycles-pp.clear_bdi_congested 0.32 ± 5% -0.0 0.28 ± 4% perf-profile.children.cycles-pp.pagevec_lookup_range_tag 0.32 ± 5% -0.0 0.28 ± 4% perf-profile.children.cycles-pp.find_get_pages_range_tag 0.17 ± 7% -0.0 0.13 ± 5% perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.34 ± 3% -0.0 0.30 ± 3% perf-profile.children.cycles-pp.__mod_lruvec_state 0.21 -0.0 0.17 ± 8% perf-profile.children.cycles-pp.try_charge 0.11 ± 9% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.xfs_generic_create 0.11 ± 9% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.vfs_create 0.27 ± 3% -0.0 0.24 ± 3% perf-profile.children.cycles-pp.__mod_node_page_state 0.18 ± 5% -0.0 0.15 ± 2% perf-profile.children.cycles-pp.alloc_pages_current 0.22 ± 5% -0.0 0.18 ± 8% perf-profile.children.cycles-pp.___might_sleep 0.15 ± 5% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.sysvec_call_function_single 0.14 ± 6% -0.0 0.10 ± 10% perf-profile.children.cycles-pp.__sysvec_call_function_single 0.20 ± 3% -0.0 0.17 ± 6% perf-profile.children.cycles-pp.nfs_initiate_commit 0.20 ± 5% -0.0 0.17 ± 6% perf-profile.children.cycles-pp.nfs_page_set_headlock 0.11 ± 7% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.xfs_create 0.37 ± 5% -0.0 0.34 ± 4% perf-profile.children.cycles-pp.update_rq_clock 0.14 ± 9% -0.0 0.11 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.18 ± 8% -0.0 0.15 ± 12% perf-profile.children.cycles-pp.dequeue_entity 0.14 ± 6% -0.0 0.11 ± 10% perf-profile.children.cycles-pp.find_get_entry 0.15 ± 2% -0.0 0.12 ± 8% perf-profile.children.cycles-pp.finish_task_switch 0.16 ± 5% -0.0 0.13 ± 10% perf-profile.children.cycles-pp.node_dirty_ok 0.14 ± 8% -0.0 0.11 ± 8% perf-profile.children.cycles-pp.ttwu_queue_wakelist 0.33 ± 3% -0.0 0.30 ± 3% perf-profile.children.cycles-pp.nfs_page_group_destroy 0.15 ± 10% -0.0 0.12 ± 4% perf-profile.children.cycles-pp.lock_sock_nested 0.10 ± 4% -0.0 0.07 ± 11% perf-profile.children.cycles-pp.rpc_wait_bit_killable 0.11 ± 7% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.xfs_map_blocks 0.12 ± 8% -0.0 0.09 ± 8% perf-profile.children.cycles-pp.update_curr 0.19 ± 8% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.do_mkdirat 0.19 ± 8% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.filename_create 0.18 ± 4% -0.0 0.15 ± 6% perf-profile.children.cycles-pp.xs_stream_data_receive_workfn 0.18 ± 4% -0.0 0.15 ± 6% perf-profile.children.cycles-pp.xs_stream_data_receive 0.15 ± 4% -0.0 0.12 ± 6% perf-profile.children.cycles-pp.lookup_dcache 0.18 ± 2% -0.0 0.15 ± 10% perf-profile.children.cycles-pp.mutex_lock 0.08 ± 5% -0.0 0.06 ± 11% perf-profile.children.cycles-pp._raw_spin_lock_bh 0.11 ± 6% -0.0 0.09 ± 10% perf-profile.children.cycles-pp.unlock_page 0.15 ± 5% -0.0 0.13 ± 5% perf-profile.children.cycles-pp.autoremove_wake_function 0.11 ± 7% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.xas_start 0.10 ± 7% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.__orc_find 0.11 ± 4% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.out_of_line_wait_on_bit 0.11 ± 4% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.__wait_on_bit 0.08 ± 11% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.__switch_to_asm 0.10 ± 10% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.nfs_key_timeout_notify 0.10 ± 9% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.set_next_entity 0.11 ± 14% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.__lock_sock 0.21 ± 3% -0.0 0.19 ± 4% perf-profile.children.cycles-pp.__queue_work 0.12 ± 4% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.nfs_unlock_and_release_request 0.10 -0.0 0.08 ± 8% perf-profile.children.cycles-pp.__pagevec_release 0.08 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.19 ± 4% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.queue_work_on 0.16 ± 2% -0.0 0.14 ± 5% perf-profile.children.cycles-pp.__wake_up_common_lock 0.09 ± 7% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.08 ± 5% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.__switch_to 0.11 ± 7% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.iov_iter_advance 0.08 ± 5% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.xas_set_mark 0.08 -0.0 0.07 ± 6% perf-profile.children.cycles-pp.nfs_request_add_commit_list_locked 0.01 ±173% +0.0 0.06 ± 11% perf-profile.children.cycles-pp.osq_lock 0.27 ± 4% +0.0 0.32 ± 5% perf-profile.children.cycles-pp.__tcp_push_pending_frames 0.01 ±173% +0.1 0.06 ± 6% perf-profile.children.cycles-pp.kfree 0.28 ± 7% +0.1 0.35 ± 4% perf-profile.children.cycles-pp.svc_tcp_sendto 0.30 ± 7% +0.1 0.38 ± 3% perf-profile.children.cycles-pp.svc_send 0.42 ± 10% +0.1 0.51 ± 4% perf-profile.children.cycles-pp.rebalance_domains 0.00 +0.1 0.11 ± 13% perf-profile.children.cycles-pp.pskb_expand_head 0.00 +0.1 0.13 perf-profile.children.cycles-pp.iov_iter_get_pages 1.22 ± 5% +0.2 1.37 ± 2% perf-profile.children.cycles-pp.rmqueue 0.09 ± 4% +0.2 0.24 ± 10% perf-profile.children.cycles-pp.__check_object_size 0.07 +0.2 0.24 ± 7% perf-profile.children.cycles-pp.simple_copy_to_iter 0.00 +0.2 0.18 ± 2% perf-profile.children.cycles-pp.__zerocopy_sg_from_iter 0.00 +0.2 0.18 ± 2% perf-profile.children.cycles-pp.skb_zerocopy_iter_stream 0.50 ± 4% +0.3 0.76 ± 3% perf-profile.children.cycles-pp.rmqueue_bulk 0.43 ± 5% +0.3 0.72 ± 4% perf-profile.children.cycles-pp.free_unref_page 0.24 ± 8% +0.3 0.53 ± 7% perf-profile.children.cycles-pp.free_pcppages_bulk 0.78 ± 5% +0.3 1.11 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid 0.23 ± 6% +0.4 0.59 ± 5% perf-profile.children.cycles-pp.skb_release_data 3.28 ± 2% +0.4 3.65 ± 4% perf-profile.children.cycles-pp.inet6_recvmsg 3.27 ± 2% +0.4 3.64 ± 4% perf-profile.children.cycles-pp.tcp_recvmsg 0.31 ± 4% +0.4 0.71 ± 7% perf-profile.children.cycles-pp.__kfree_skb 5.52 +2.3 7.82 perf-profile.children.cycles-pp.memcpy_erms 5.13 ± 10% +3.3 8.47 ± 3% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 0.73 ± 5% +3.7 4.43 perf-profile.children.cycles-pp.tcp_write_xmit 0.79 ± 4% +3.7 4.51 perf-profile.children.cycles-pp.__tcp_transmit_skb 0.69 ± 5% +3.7 4.44 perf-profile.children.cycles-pp.inet6_csk_xmit 0.62 ± 5% +3.8 4.40 perf-profile.children.cycles-pp.ip6_xmit 0.60 ± 4% +3.8 4.38 perf-profile.children.cycles-pp.ip6_output 0.56 ± 3% +3.8 4.34 ± 2% perf-profile.children.cycles-pp.ip6_finish_output2 0.48 ± 4% +3.8 4.26 ± 2% perf-profile.children.cycles-pp.__local_bh_enable_ip 0.47 ± 4% +3.8 4.26 ± 2% perf-profile.children.cycles-pp.do_softirq 0.42 ± 3% +3.8 4.21 ± 2% perf-profile.children.cycles-pp.__netif_receive_skb_one_core 0.44 ± 4% +3.8 4.23 ± 2% perf-profile.children.cycles-pp.net_rx_action 0.43 ± 4% +3.8 4.22 ± 2% perf-profile.children.cycles-pp.process_backlog 1.38 ± 8% +3.8 5.20 perf-profile.children.cycles-pp.do_softirq_own_stack 1.38 ± 8% +3.8 5.20 perf-profile.children.cycles-pp.__softirqentry_text_start 0.00 +3.9 3.86 ± 2% perf-profile.children.cycles-pp.skb_copy_ubufs 0.00 +3.9 3.87 ± 2% perf-profile.children.cycles-pp.__netif_receive_skb_core 44.19 ± 4% +4.2 48.41 perf-profile.children.cycles-pp.cpuidle_enter 44.18 ± 4% +4.2 48.40 perf-profile.children.cycles-pp.cpuidle_enter_state 49.23 ± 2% +4.9 54.14 perf-profile.children.cycles-pp.start_secondary 49.50 ± 2% +5.0 54.45 perf-profile.children.cycles-pp.secondary_startup_64_no_verify 49.50 ± 2% +5.0 54.45 perf-profile.children.cycles-pp.cpu_startup_entry 49.49 ± 2% +5.0 54.44 perf-profile.children.cycles-pp.do_idle 2.92 ± 3% -0.4 2.54 ± 3% perf-profile.self.cycles-pp._raw_spin_lock 2.16 ± 3% -0.3 1.86 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 2.19 ± 3% -0.3 1.92 ± 2% perf-profile.self.cycles-pp.brd_do_bvec 1.85 ± 6% -0.2 1.66 ± 4% perf-profile.self.cycles-pp.nfs_do_writepage 0.92 ± 5% -0.2 0.75 ± 5% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.92 ± 6% -0.1 0.79 ± 5% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.69 ± 5% -0.1 0.56 ± 4% perf-profile.self.cycles-pp.poll_idle 0.53 ± 8% -0.1 0.41 ± 6% perf-profile.self.cycles-pp.__mod_memcg_state 0.46 ± 10% -0.1 0.35 ± 9% perf-profile.self.cycles-pp.nfs_end_page_writeback 0.64 ± 2% -0.1 0.53 ± 4% perf-profile.self.cycles-pp.update_sd_lb_stats 0.82 ± 2% -0.1 0.72 ± 3% perf-profile.self.cycles-pp.nfs_write_end 0.24 ± 18% -0.1 0.14 ± 13% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.36 ± 6% -0.1 0.26 ± 4% perf-profile.self.cycles-pp.svc_tcp_recvfrom 0.22 ± 7% -0.1 0.12 ± 15% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.48 ± 8% -0.1 0.39 ± 4% perf-profile.self.cycles-pp.nfs_ctx_key_to_expire 0.46 ± 6% -0.1 0.37 ± 2% perf-profile.self.cycles-pp.rmqueue 0.65 ± 3% -0.1 0.56 ± 7% perf-profile.self.cycles-pp.iomap_set_range_uptodate 0.54 ± 4% -0.1 0.47 ± 2% perf-profile.self.cycles-pp.__radix_tree_lookup 0.50 ± 2% -0.1 0.44 ± 5% perf-profile.self.cycles-pp.clear_page_erms 0.30 ± 11% -0.1 0.24 ± 9% perf-profile.self.cycles-pp.clear_page_dirty_for_io 0.16 ± 5% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.try_to_wake_up 0.26 ± 5% -0.0 0.22 ± 5% perf-profile.self.cycles-pp.find_get_pages_range_tag 0.14 ± 6% -0.0 0.10 ± 8% perf-profile.self.cycles-pp.clear_bdi_congested 0.19 ± 5% -0.0 0.15 ± 12% perf-profile.self.cycles-pp.dec_zone_page_state 0.20 ± 5% -0.0 0.16 ± 5% perf-profile.self.cycles-pp.nfs_page_set_headlock 0.28 ± 4% -0.0 0.24 ± 2% perf-profile.self.cycles-pp.xas_load 0.26 ± 4% -0.0 0.23 ± 3% perf-profile.self.cycles-pp.__mod_node_page_state 0.21 ± 5% -0.0 0.18 ± 7% perf-profile.self.cycles-pp.___might_sleep 0.21 ± 6% -0.0 0.18 ± 4% perf-profile.self.cycles-pp.__nfs_create_request 0.22 -0.0 0.19 ± 5% perf-profile.self.cycles-pp.__pagevec_lru_add_fn 0.07 ± 12% -0.0 0.04 ± 57% perf-profile.self.cycles-pp.svc_recv 0.14 ± 9% -0.0 0.11 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.07 ± 7% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.iomap_writepage_map 0.11 ± 6% -0.0 0.09 ± 10% perf-profile.self.cycles-pp.unlock_page 0.10 ± 7% -0.0 0.08 ± 10% perf-profile.self.cycles-pp.__orc_find 0.08 ± 11% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.__switch_to_asm 0.15 ± 4% -0.0 0.13 ± 5% perf-profile.self.cycles-pp.do_idle 0.11 ± 4% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.iov_iter_advance 0.10 ± 10% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.alloc_pages_current 0.17 ± 2% -0.0 0.15 ± 5% perf-profile.self.cycles-pp.unwind_next_frame 0.15 ± 2% -0.0 0.13 ± 5% perf-profile.self.cycles-pp.release_pages 0.10 ± 5% -0.0 0.08 ± 14% perf-profile.self.cycles-pp.xas_start 0.07 ± 12% -0.0 0.05 perf-profile.self.cycles-pp.set_next_entity 0.08 -0.0 0.07 ± 7% perf-profile.self.cycles-pp.xas_set_mark 0.08 ± 6% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.enqueue_entity 0.14 ± 6% +0.0 0.16 ± 6% perf-profile.self.cycles-pp.skb_release_data 0.01 ±173% +0.0 0.06 ± 11% perf-profile.self.cycles-pp.osq_lock 0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.kfree 0.00 +0.1 0.09 ± 13% perf-profile.self.cycles-pp.pskb_expand_head 0.00 +0.1 0.12 ± 6% perf-profile.self.cycles-pp.skb_copy_ubufs 0.10 ± 12% +0.1 0.23 ± 8% perf-profile.self.cycles-pp.free_pcppages_bulk 0.00 +0.1 0.13 perf-profile.self.cycles-pp.iov_iter_get_pages 0.07 ± 7% +0.1 0.21 ± 10% perf-profile.self.cycles-pp.__check_object_size 0.77 ± 4% +0.3 1.10 ± 4% perf-profile.self.cycles-pp.__list_del_entry_valid 5.46 +2.3 7.73 perf-profile.self.cycles-pp.memcpy_erms fsmark.files_per_sec 580 +---------------------------------------------------------------------+ | .+ | 560 |..+. .+. + .+..+.. .+..+ | | +. +.. + + .+..+ +.+. | | +.. .. + .. + +. | 540 |-+ + + + + | | + | 520 |-+ | | | 500 |-+ | | O O O | | O O O O O O O O O O O O O O O | 480 |-+ O O O O O O | | O | 460 +---------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= compiler/cpufreq_governor/disk/filesize/fs2/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode: gcc-9/performance/1BRD_48G/4M/nfsv4/xfs/1x/x86_64-rhel-8.3/64t/debian-10.4-x86_64-20200603.cgz/NoSync/lkp-csl-2ap2/40G/fsmark/0x5003003 commit: v5.10-rc2 3bc6a407d1 ("SUNRPC: Use zero-copy to perform socket send operations") v5.10-rc2 3bc6a407d1a090417218ecdb0dd ---------------- --------------------------- %stddev %change %stddev \ | \ 12868393 +17.9% 15177801 ± 2% fsmark.app_overhead 559.38 -13.3% 485.20 fsmark.files_per_sec 21.35 +13.1% 24.16 fsmark.time.elapsed_time 21.35 +13.1% 24.16 fsmark.time.elapsed_time.max 1664 ± 4% -13.9% 1433 ± 3% fsmark.time.involuntary_context_switches 251.75 ± 2% -13.1% 218.75 fsmark.time.percent_of_cpu_this_job_got 25.96 -2.0% 25.44 boot-time.dhcp 2.47 ± 2% +19.0% 2.95 ± 2% iostat.cpu.iowait 671458 ± 2% +8.6% 728893 ± 3% softirqs.RCU 1669499 -8.2% 1532757 vmstat.io.bo 202132 -13.7% 174380 ± 2% vmstat.system.cs 937363 ± 62% -61.6% 359933 ± 9% cpuidle.C1.usage 2.258e+09 ± 51% +86.4% 4.209e+09 ± 9% cpuidle.C1E.time 6449059 ± 20% +51.7% 9781611 ± 5% cpuidle.C1E.usage 2.67 ± 2% +0.5 3.16 ± 2% mpstat.cpu.all.iowait% 0.14 ± 3% +0.3 0.42 mpstat.cpu.all.soft% 3.55 ± 4% -0.5 3.05 ± 2% mpstat.cpu.all.sys% 10140 ± 26% -31.3% 6970 ± 3% numa-meminfo.node2.Mapped 16956 ± 7% -17.1% 14052 ± 3% numa-meminfo.node2.Writeback 196488 ± 12% -17.5% 162160 ± 11% numa-meminfo.node3.Slab 3006 ± 4% +7.2% 3223 ± 4% proc-vmstat.nr_active_anon 3006 ± 4% +7.2% 3223 ± 4% proc-vmstat.nr_zone_active_anon 77623491 ± 3% -13.4% 67184071 proc-vmstat.pgalloc_normal 56668889 ± 4% -18.7% 46043533 proc-vmstat.pgfree 10476 +4.4% 10935 proc-vmstat.pgreuse 4779 ± 13% -27.6% 3460 ± 6% sched_debug.cfs_rq:/.min_vruntime.stddev 27500 ± 37% -78.2% 5982 ± 93% sched_debug.cfs_rq:/.spread0.max 4780 ± 13% -25.8% 3546 ± 7% sched_debug.cfs_rq:/.spread0.stddev 1730403 ± 10% +41.9% 2456125 ± 26% sched_debug.cpu.avg_idle.max 13907 ± 18% -28.8% 9897 ± 21% sched_debug.cpu.nr_switches.max 1151 ± 9% -15.7% 970.60 ± 8% sched_debug.cpu.nr_switches.stddev 21696 -15.9% 18236 ± 2% slabinfo.kmalloc-1k.active_objs 684.25 -15.1% 580.75 ± 2% slabinfo.kmalloc-1k.active_slabs 21907 -15.1% 18591 ± 2% slabinfo.kmalloc-1k.num_objs 684.25 -15.1% 580.75 ± 2% slabinfo.kmalloc-1k.num_slabs 11971 ± 3% -11.5% 10592 ± 7% slabinfo.nfsd_file.active_objs 3314 ± 5% -20.9% 2621 ± 13% slabinfo.xfs_efd_item.active_objs 3355 ± 4% -20.1% 2680 ± 12% slabinfo.xfs_efd_item.num_objs 3275 ± 5% -20.4% 2607 ± 13% slabinfo.xfs_efi_item.active_objs 3318 ± 5% -19.7% 2663 ± 13% slabinfo.xfs_efi_item.num_objs 3.817e+09 -7.6% 3.526e+09 perf-stat.i.branch-instructions 3.316e+08 ± 2% -9.4% 3.003e+08 ± 2% perf-stat.i.cache-references 234663 ± 2% -16.7% 195399 perf-stat.i.context-switches 3.805e+10 ± 2% -6.6% 3.552e+10 ± 2% perf-stat.i.cpu-cycles 1382 ± 3% -7.2% 1283 ± 3% perf-stat.i.cpu-migrations 4.805e+09 -7.6% 4.438e+09 perf-stat.i.dTLB-loads 2.407e+09 -8.7% 2.198e+09 perf-stat.i.dTLB-stores 7989114 ± 2% -4.2% 7652291 perf-stat.i.iTLB-load-misses 1.929e+10 -7.4% 1.787e+10 perf-stat.i.instructions 2469 ± 2% -4.9% 2348 perf-stat.i.instructions-per-iTLB-miss 0.20 ± 2% -6.6% 0.19 ± 2% perf-stat.i.metric.GHz 59.57 -7.9% 54.88 perf-stat.i.metric.M/sec 38311030 ± 5% -10.6% 34257932 ± 2% perf-stat.i.node-load-misses 10676843 ± 7% -13.7% 9213801 perf-stat.i.node-loads 10297341 ± 21% -21.5% 8080955 ± 3% perf-stat.i.node-stores 3.647e+09 -7.2% 3.386e+09 perf-stat.ps.branch-instructions 3.169e+08 ± 2% -9.0% 2.884e+08 perf-stat.ps.cache-references 224244 -16.3% 187638 perf-stat.ps.context-switches 3.636e+10 ± 2% -6.2% 3.411e+10 ± 2% perf-stat.ps.cpu-cycles 4.59e+09 -7.1% 4.262e+09 perf-stat.ps.dTLB-loads 2.299e+09 -8.2% 2.11e+09 perf-stat.ps.dTLB-stores 7634292 ± 2% -3.7% 7349222 perf-stat.ps.iTLB-load-misses 1.843e+10 -6.9% 1.716e+10 perf-stat.ps.instructions 36618483 ± 5% -10.1% 32904334 ± 2% perf-stat.ps.node-load-misses 10197765 ± 7% -13.3% 8839824 perf-stat.ps.node-loads 9840509 ± 21% -21.1% 7760538 ± 3% perf-stat.ps.node-stores 4.173e+11 +4.7% 4.368e+11 ± 2% perf-stat.total.instructions 27.50 ± 35% +1444.5% 424.75 ± 95% interrupts.33:PCI-MSI.524291-edge.eth0-TxRx-2 46517 ± 2% +12.0% 52079 interrupts.CPU0.LOC:Local_timer_interrupts 2.00 ± 35% +2775.0% 57.50 ±157% interrupts.CPU0.TLB:TLB_shootdowns 46340 ± 3% +12.3% 52034 ± 2% interrupts.CPU1.LOC:Local_timer_interrupts 46533 ± 3% +11.7% 51972 interrupts.CPU10.LOC:Local_timer_interrupts 46371 ± 3% +11.9% 51871 ± 2% interrupts.CPU100.LOC:Local_timer_interrupts 46334 ± 3% +12.1% 51963 interrupts.CPU101.LOC:Local_timer_interrupts 46349 ± 3% +11.7% 51764 interrupts.CPU102.LOC:Local_timer_interrupts 46306 ± 3% +12.0% 51860 interrupts.CPU103.LOC:Local_timer_interrupts 46351 ± 3% +11.7% 51795 ± 2% interrupts.CPU104.LOC:Local_timer_interrupts 46348 ± 3% +11.9% 51847 ± 2% interrupts.CPU105.LOC:Local_timer_interrupts 46414 ± 2% +11.8% 51900 interrupts.CPU106.LOC:Local_timer_interrupts 46379 ± 3% +11.9% 51902 interrupts.CPU107.LOC:Local_timer_interrupts 46358 ± 3% +11.9% 51883 ± 2% interrupts.CPU108.LOC:Local_timer_interrupts 46420 ± 2% +11.5% 51766 interrupts.CPU109.LOC:Local_timer_interrupts 46404 ± 3% +12.0% 51974 interrupts.CPU11.LOC:Local_timer_interrupts 46349 ± 3% +11.9% 51882 ± 2% interrupts.CPU110.LOC:Local_timer_interrupts 46346 ± 3% +11.9% 51882 ± 2% interrupts.CPU111.LOC:Local_timer_interrupts 46377 ± 3% +11.9% 51878 ± 2% interrupts.CPU112.LOC:Local_timer_interrupts 46357 ± 3% +11.8% 51827 interrupts.CPU113.LOC:Local_timer_interrupts 46221 ± 3% +12.2% 51862 interrupts.CPU114.LOC:Local_timer_interrupts 46360 ± 3% +11.9% 51876 ± 2% interrupts.CPU115.LOC:Local_timer_interrupts 46382 ± 3% +11.9% 51881 ± 2% interrupts.CPU116.LOC:Local_timer_interrupts 46363 ± 3% +12.0% 51921 ± 2% interrupts.CPU117.LOC:Local_timer_interrupts 46362 ± 3% +12.1% 51961 interrupts.CPU118.LOC:Local_timer_interrupts 46375 ± 3% +11.9% 51882 ± 2% interrupts.CPU119.LOC:Local_timer_interrupts 27.50 ± 35% +1444.5% 424.75 ± 95% interrupts.CPU12.33:PCI-MSI.524291-edge.eth0-TxRx-2 46330 ± 2% +12.4% 52054 interrupts.CPU12.LOC:Local_timer_interrupts 46146 ± 3% +12.5% 51905 ± 2% interrupts.CPU120.LOC:Local_timer_interrupts 46155 ± 3% +12.5% 51904 ± 2% interrupts.CPU121.LOC:Local_timer_interrupts 46153 ± 3% +12.5% 51922 ± 2% interrupts.CPU122.LOC:Local_timer_interrupts 46147 ± 3% +12.5% 51908 ± 2% interrupts.CPU123.LOC:Local_timer_interrupts 46163 ± 3% +12.5% 51942 interrupts.CPU124.LOC:Local_timer_interrupts 46238 ± 3% +12.3% 51921 interrupts.CPU125.LOC:Local_timer_interrupts 46340 ± 3% +12.0% 51886 ± 2% interrupts.CPU126.LOC:Local_timer_interrupts 46208 ± 3% +12.1% 51822 ± 2% interrupts.CPU127.LOC:Local_timer_interrupts 46153 ± 3% +12.5% 51910 ± 2% interrupts.CPU128.LOC:Local_timer_interrupts 46179 ± 3% +12.4% 51916 ± 2% interrupts.CPU129.LOC:Local_timer_interrupts 46554 ± 3% +11.4% 51855 interrupts.CPU13.LOC:Local_timer_interrupts 46246 ± 3% +12.3% 51947 ± 2% interrupts.CPU130.LOC:Local_timer_interrupts 45950 ± 3% +13.0% 51904 ± 2% interrupts.CPU131.LOC:Local_timer_interrupts 46146 ± 3% +12.4% 51845 ± 2% interrupts.CPU132.LOC:Local_timer_interrupts 46125 ± 3% +12.6% 51920 ± 2% interrupts.CPU133.LOC:Local_timer_interrupts 46236 ± 3% +12.3% 51912 ± 2% interrupts.CPU134.LOC:Local_timer_interrupts 46232 ± 3% +12.3% 51902 ± 2% interrupts.CPU135.LOC:Local_timer_interrupts 46196 ± 3% +12.5% 51954 interrupts.CPU136.LOC:Local_timer_interrupts 46239 ± 3% +11.9% 51745 ± 2% interrupts.CPU137.LOC:Local_timer_interrupts 46415 ± 3% +11.8% 51892 interrupts.CPU138.LOC:Local_timer_interrupts 46236 ± 3% +12.2% 51895 ± 2% interrupts.CPU139.LOC:Local_timer_interrupts 46508 ± 3% +11.7% 51938 interrupts.CPU14.LOC:Local_timer_interrupts 46279 ± 3% +12.1% 51896 interrupts.CPU140.LOC:Local_timer_interrupts 46142 ± 3% +12.5% 51917 ± 2% interrupts.CPU141.LOC:Local_timer_interrupts 482.50 ± 18% +28.0% 617.50 ± 13% interrupts.CPU141.NMI:Non-maskable_interrupts 482.50 ± 18% +28.0% 617.50 ± 13% interrupts.CPU141.PMI:Performance_monitoring_interrupts 46295 ± 3% +11.6% 51670 ± 2% interrupts.CPU142.LOC:Local_timer_interrupts 617.75 ± 8% +29.8% 801.75 ± 15% interrupts.CPU142.NMI:Non-maskable_interrupts 617.75 ± 8% +29.8% 801.75 ± 15% interrupts.CPU142.PMI:Performance_monitoring_interrupts 1.00 ± 70% +20750.0% 208.50 ±169% interrupts.CPU142.TLB:TLB_shootdowns 46152 ± 3% +12.4% 51862 ± 2% interrupts.CPU143.LOC:Local_timer_interrupts 46417 ± 3% +12.1% 52022 interrupts.CPU144.LOC:Local_timer_interrupts 46496 ± 3% +11.8% 51987 interrupts.CPU145.LOC:Local_timer_interrupts 46498 ± 2% +11.9% 52022 interrupts.CPU146.LOC:Local_timer_interrupts 46507 ± 2% +11.8% 51978 interrupts.CPU147.LOC:Local_timer_interrupts 46499 ± 2% +11.8% 51994 interrupts.CPU148.LOC:Local_timer_interrupts 46515 ± 2% +11.7% 51971 interrupts.CPU149.LOC:Local_timer_interrupts 46469 ± 2% +11.8% 51941 interrupts.CPU15.LOC:Local_timer_interrupts 46568 ± 3% +11.7% 52007 interrupts.CPU150.LOC:Local_timer_interrupts 46562 ± 3% +11.7% 52002 interrupts.CPU151.LOC:Local_timer_interrupts 46510 ± 2% +11.8% 51979 interrupts.CPU152.LOC:Local_timer_interrupts 46661 ± 2% +11.4% 51971 interrupts.CPU153.LOC:Local_timer_interrupts 46532 ± 2% +11.7% 51986 interrupts.CPU154.LOC:Local_timer_interrupts 46514 ± 2% +11.9% 52042 interrupts.CPU155.LOC:Local_timer_interrupts 46415 ± 2% +12.2% 52069 interrupts.CPU156.LOC:Local_timer_interrupts 2565 ± 12% -24.0% 1949 ± 17% interrupts.CPU157.CAL:Function_call_interrupts 46532 ± 2% +11.7% 51958 interrupts.CPU157.LOC:Local_timer_interrupts 46494 ± 2% +11.8% 51999 interrupts.CPU158.LOC:Local_timer_interrupts 46949 ± 3% +10.8% 52016 interrupts.CPU159.LOC:Local_timer_interrupts 46480 ± 2% +11.8% 51958 interrupts.CPU16.LOC:Local_timer_interrupts 46492 ± 2% +11.8% 51997 interrupts.CPU160.LOC:Local_timer_interrupts 46601 ± 3% +11.6% 51984 interrupts.CPU161.LOC:Local_timer_interrupts 524.75 ± 52% +105.4% 1078 ± 23% interrupts.CPU161.NMI:Non-maskable_interrupts 524.75 ± 52% +105.4% 1078 ± 23% interrupts.CPU161.PMI:Performance_monitoring_interrupts 46573 ± 3% +11.6% 51992 interrupts.CPU162.LOC:Local_timer_interrupts 46516 ± 3% +11.8% 51994 interrupts.CPU163.LOC:Local_timer_interrupts 46552 ± 3% +11.7% 52016 interrupts.CPU164.LOC:Local_timer_interrupts 46556 ± 2% +11.6% 51968 interrupts.CPU165.LOC:Local_timer_interrupts 46509 ± 2% +11.9% 52023 interrupts.CPU166.LOC:Local_timer_interrupts 46486 ± 2% +11.8% 51954 interrupts.CPU167.LOC:Local_timer_interrupts 2735 ± 17% -22.1% 2131 ± 7% interrupts.CPU168.CAL:Function_call_interrupts 46657 ± 3% +11.2% 51869 ± 2% interrupts.CPU168.LOC:Local_timer_interrupts 2577 ± 10% -31.5% 1765 ± 11% interrupts.CPU169.CAL:Function_call_interrupts 46474 ± 2% +11.8% 51975 interrupts.CPU169.LOC:Local_timer_interrupts 1882 ± 13% +29.5% 2438 ± 11% interrupts.CPU17.CAL:Function_call_interrupts 46241 ± 3% +12.3% 51932 interrupts.CPU17.LOC:Local_timer_interrupts 46536 ± 3% +11.6% 51918 interrupts.CPU170.LOC:Local_timer_interrupts 46614 ± 3% +11.4% 51912 interrupts.CPU171.LOC:Local_timer_interrupts 46464 ± 3% +11.8% 51967 interrupts.CPU172.LOC:Local_timer_interrupts 46584 ± 3% +11.5% 51924 interrupts.CPU173.LOC:Local_timer_interrupts 46526 ± 3% +11.5% 51868 ± 2% interrupts.CPU174.LOC:Local_timer_interrupts 46603 ± 3% +11.4% 51905 interrupts.CPU175.LOC:Local_timer_interrupts 46588 ± 3% +11.4% 51878 ± 2% interrupts.CPU176.LOC:Local_timer_interrupts 46476 ± 2% +11.7% 51930 ± 2% interrupts.CPU177.LOC:Local_timer_interrupts 46529 ± 2% +11.4% 51821 interrupts.CPU178.LOC:Local_timer_interrupts 46519 ± 3% +11.6% 51902 ± 2% interrupts.CPU179.LOC:Local_timer_interrupts 1137 ± 12% -41.7% 662.50 ± 40% interrupts.CPU179.NMI:Non-maskable_interrupts 1137 ± 12% -41.7% 662.50 ± 40% interrupts.CPU179.PMI:Performance_monitoring_interrupts 46228 ± 3% +12.3% 51928 interrupts.CPU18.LOC:Local_timer_interrupts 2552 ± 15% -19.6% 2052 ± 8% interrupts.CPU180.CAL:Function_call_interrupts 46592 ± 3% +11.4% 51889 interrupts.CPU180.LOC:Local_timer_interrupts 46437 ± 3% +11.4% 51749 ± 2% interrupts.CPU181.LOC:Local_timer_interrupts 46574 ± 2% +11.4% 51877 interrupts.CPU182.LOC:Local_timer_interrupts 877.50 ± 21% -52.0% 421.50 ± 28% interrupts.CPU182.NMI:Non-maskable_interrupts 877.50 ± 21% -52.0% 421.50 ± 28% interrupts.CPU182.PMI:Performance_monitoring_interrupts 46513 ± 3% +11.5% 51881 interrupts.CPU183.LOC:Local_timer_interrupts 803.00 ± 17% -35.1% 521.25 ± 11% interrupts.CPU183.NMI:Non-maskable_interrupts 803.00 ± 17% -35.1% 521.25 ± 11% interrupts.CPU183.PMI:Performance_monitoring_interrupts 46487 ± 3% +11.6% 51868 ± 2% interrupts.CPU184.LOC:Local_timer_interrupts 748.50 ± 16% -36.4% 476.00 ± 24% interrupts.CPU184.NMI:Non-maskable_interrupts 748.50 ± 16% -36.4% 476.00 ± 24% interrupts.CPU184.PMI:Performance_monitoring_interrupts 46583 ± 3% +11.3% 51870 ± 2% interrupts.CPU185.LOC:Local_timer_interrupts 46465 ± 3% +11.6% 51870 ± 2% interrupts.CPU186.LOC:Local_timer_interrupts 46465 ± 3% +11.8% 51937 ± 2% interrupts.CPU187.LOC:Local_timer_interrupts 46555 ± 3% +11.4% 51878 ± 2% interrupts.CPU188.LOC:Local_timer_interrupts 2884 ± 25% -29.4% 2037 ± 10% interrupts.CPU189.CAL:Function_call_interrupts 46581 ± 3% +11.4% 51886 interrupts.CPU189.LOC:Local_timer_interrupts 46447 ± 3% +11.8% 51919 interrupts.CPU19.LOC:Local_timer_interrupts 46576 ± 3% +11.4% 51870 ± 2% interrupts.CPU190.LOC:Local_timer_interrupts 46478 ± 2% +11.6% 51889 interrupts.CPU191.LOC:Local_timer_interrupts 46453 ± 3% +11.9% 51986 interrupts.CPU2.LOC:Local_timer_interrupts 46377 ± 3% +12.2% 52015 interrupts.CPU20.LOC:Local_timer_interrupts 1365 ± 85% -67.8% 439.50 ± 41% interrupts.CPU20.NMI:Non-maskable_interrupts 1365 ± 85% -67.8% 439.50 ± 41% interrupts.CPU20.PMI:Performance_monitoring_interrupts 46389 ± 3% +12.1% 52019 interrupts.CPU21.LOC:Local_timer_interrupts 46439 ± 3% +11.9% 51967 interrupts.CPU22.LOC:Local_timer_interrupts 750.00 ± 6% -43.7% 422.25 ± 27% interrupts.CPU22.NMI:Non-maskable_interrupts 750.00 ± 6% -43.7% 422.25 ± 27% interrupts.CPU22.PMI:Performance_monitoring_interrupts 46396 ± 3% +12.1% 52020 interrupts.CPU23.LOC:Local_timer_interrupts 46080 ± 3% +12.8% 51965 interrupts.CPU24.LOC:Local_timer_interrupts 46183 ± 3% +12.6% 51990 ± 2% interrupts.CPU25.LOC:Local_timer_interrupts 46226 ± 3% +12.4% 51979 ± 2% interrupts.CPU26.LOC:Local_timer_interrupts 46238 ± 3% +12.4% 51984 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts 46288 ± 3% +12.3% 51999 interrupts.CPU28.LOC:Local_timer_interrupts 46173 ± 3% +12.5% 51962 interrupts.CPU29.LOC:Local_timer_interrupts 46412 ± 3% +11.9% 51944 interrupts.CPU3.LOC:Local_timer_interrupts 46318 ± 3% +12.2% 51967 ± 2% interrupts.CPU30.LOC:Local_timer_interrupts 46311 ± 3% +12.2% 51981 interrupts.CPU31.LOC:Local_timer_interrupts 46262 ± 3% +12.4% 51981 ± 2% interrupts.CPU32.LOC:Local_timer_interrupts 46201 ± 3% +12.1% 51800 ± 2% interrupts.CPU33.LOC:Local_timer_interrupts 46263 ± 3% +12.3% 51957 interrupts.CPU34.LOC:Local_timer_interrupts 46287 ± 3% +12.1% 51870 ± 2% interrupts.CPU35.LOC:Local_timer_interrupts 46200 ± 3% +11.8% 51638 ± 2% interrupts.CPU36.LOC:Local_timer_interrupts 46224 ± 3% +12.5% 52006 interrupts.CPU37.LOC:Local_timer_interrupts 46163 ± 3% +12.5% 51950 ± 2% interrupts.CPU38.LOC:Local_timer_interrupts 46328 ± 3% +12.2% 51993 interrupts.CPU39.LOC:Local_timer_interrupts 46435 ± 3% +12.1% 52052 interrupts.CPU4.LOC:Local_timer_interrupts 46343 ± 3% +12.2% 52012 interrupts.CPU40.LOC:Local_timer_interrupts 484.00 ± 25% +52.1% 736.00 ± 23% interrupts.CPU40.NMI:Non-maskable_interrupts 484.00 ± 25% +52.1% 736.00 ± 23% interrupts.CPU40.PMI:Performance_monitoring_interrupts 46267 ± 3% +12.3% 51973 interrupts.CPU41.LOC:Local_timer_interrupts 46223 ± 3% +12.5% 51997 ± 2% interrupts.CPU42.LOC:Local_timer_interrupts 46251 ± 3% +12.5% 52055 interrupts.CPU43.LOC:Local_timer_interrupts 46293 ± 3% +12.2% 51951 interrupts.CPU44.LOC:Local_timer_interrupts 0.50 ±100% +22800.0% 114.50 ±169% interrupts.CPU44.TLB:TLB_shootdowns 46318 ± 3% +12.2% 51951 interrupts.CPU45.LOC:Local_timer_interrupts 46190 ± 3% +12.5% 51954 ± 2% interrupts.CPU46.LOC:Local_timer_interrupts 46195 ± 3% +12.4% 51912 ± 2% interrupts.CPU47.LOC:Local_timer_interrupts 46573 ± 3% +11.6% 51993 interrupts.CPU48.LOC:Local_timer_interrupts 46525 ± 2% +11.9% 52043 interrupts.CPU49.LOC:Local_timer_interrupts 46481 ± 2% +11.8% 51970 interrupts.CPU5.LOC:Local_timer_interrupts 46676 ± 2% +11.5% 52021 interrupts.CPU50.LOC:Local_timer_interrupts 46614 ± 3% +11.6% 52002 interrupts.CPU51.LOC:Local_timer_interrupts 46510 ± 3% +11.8% 52005 interrupts.CPU52.LOC:Local_timer_interrupts 46599 ± 3% +11.6% 52027 interrupts.CPU53.LOC:Local_timer_interrupts 46678 ± 2% +11.5% 52032 interrupts.CPU54.LOC:Local_timer_interrupts 46576 ± 3% +11.6% 51997 interrupts.CPU55.LOC:Local_timer_interrupts 46546 ± 3% +11.6% 51960 ± 2% interrupts.CPU56.LOC:Local_timer_interrupts 46574 ± 3% +11.7% 52019 interrupts.CPU57.LOC:Local_timer_interrupts 46620 ± 3% +11.6% 52046 interrupts.CPU58.LOC:Local_timer_interrupts 46589 ± 3% +11.6% 51978 interrupts.CPU59.LOC:Local_timer_interrupts 46399 ± 3% +12.1% 52033 interrupts.CPU6.LOC:Local_timer_interrupts 46580 ± 3% +11.6% 52004 interrupts.CPU60.LOC:Local_timer_interrupts 46567 ± 3% +11.8% 52042 interrupts.CPU61.LOC:Local_timer_interrupts 46574 ± 3% +11.7% 52011 interrupts.CPU62.LOC:Local_timer_interrupts 46504 ± 2% +11.9% 52021 interrupts.CPU63.LOC:Local_timer_interrupts 88.75 ± 20% -39.2% 54.00 ± 23% interrupts.CPU63.RES:Rescheduling_interrupts 46584 ± 3% +11.7% 52012 interrupts.CPU64.LOC:Local_timer_interrupts 46539 ± 2% +11.8% 52026 interrupts.CPU65.LOC:Local_timer_interrupts 492.00 ± 45% +123.2% 1098 ± 15% interrupts.CPU65.NMI:Non-maskable_interrupts 492.00 ± 45% +123.2% 1098 ± 15% interrupts.CPU65.PMI:Performance_monitoring_interrupts 46516 ± 2% +12.0% 52087 interrupts.CPU66.LOC:Local_timer_interrupts 46574 ± 2% +11.8% 52048 interrupts.CPU67.LOC:Local_timer_interrupts 46532 ± 2% +11.8% 52025 interrupts.CPU68.LOC:Local_timer_interrupts 46544 ± 3% +11.8% 52025 interrupts.CPU69.LOC:Local_timer_interrupts 46360 ± 3% +12.2% 52031 interrupts.CPU7.LOC:Local_timer_interrupts 46617 ± 3% +11.5% 51988 interrupts.CPU70.LOC:Local_timer_interrupts 46629 ± 3% +11.5% 52000 interrupts.CPU71.LOC:Local_timer_interrupts 46559 ± 2% +11.5% 51891 interrupts.CPU72.LOC:Local_timer_interrupts 46387 ± 3% +12.1% 51993 interrupts.CPU73.LOC:Local_timer_interrupts 46619 ± 3% +11.6% 52046 interrupts.CPU74.LOC:Local_timer_interrupts 46617 ± 3% +11.3% 51907 ± 2% interrupts.CPU75.LOC:Local_timer_interrupts 46621 ± 3% +11.3% 51904 ± 2% interrupts.CPU76.LOC:Local_timer_interrupts 942.75 ± 22% -47.9% 491.00 ± 27% interrupts.CPU76.NMI:Non-maskable_interrupts 942.75 ± 22% -47.9% 491.00 ± 27% interrupts.CPU76.PMI:Performance_monitoring_interrupts 46584 ± 3% +11.6% 51993 interrupts.CPU77.LOC:Local_timer_interrupts 85.75 ± 15% -28.3% 61.50 ± 20% interrupts.CPU77.RES:Rescheduling_interrupts 3195 ± 11% -38.2% 1974 ± 16% interrupts.CPU78.CAL:Function_call_interrupts 46556 ± 3% +11.5% 51905 ± 2% interrupts.CPU78.LOC:Local_timer_interrupts 1102 ± 51% -58.9% 453.00 ± 34% interrupts.CPU78.NMI:Non-maskable_interrupts 1102 ± 51% -58.9% 453.00 ± 34% interrupts.CPU78.PMI:Performance_monitoring_interrupts 130.75 ± 51% -62.1% 49.50 ± 22% interrupts.CPU78.RES:Rescheduling_interrupts 3086 ± 16% -28.7% 2202 ± 15% interrupts.CPU79.CAL:Function_call_interrupts 46568 ± 3% +11.4% 51897 ± 2% interrupts.CPU79.LOC:Local_timer_interrupts 934.25 ± 34% -40.8% 552.75 ± 33% interrupts.CPU79.NMI:Non-maskable_interrupts 934.25 ± 34% -40.8% 552.75 ± 33% interrupts.CPU79.PMI:Performance_monitoring_interrupts 95.75 ± 15% -47.8% 50.00 ± 18% interrupts.CPU79.RES:Rescheduling_interrupts 46388 ± 3% +12.1% 52009 interrupts.CPU8.LOC:Local_timer_interrupts 46580 ± 3% +11.4% 51896 interrupts.CPU80.LOC:Local_timer_interrupts 854.75 ± 29% -39.9% 513.50 ± 38% interrupts.CPU80.NMI:Non-maskable_interrupts 854.75 ± 29% -39.9% 513.50 ± 38% interrupts.CPU80.PMI:Performance_monitoring_interrupts 46569 ± 3% +11.5% 51906 ± 2% interrupts.CPU81.LOC:Local_timer_interrupts 947.75 ± 16% -37.4% 593.25 ± 34% interrupts.CPU81.NMI:Non-maskable_interrupts 947.75 ± 16% -37.4% 593.25 ± 34% interrupts.CPU81.PMI:Performance_monitoring_interrupts 80.75 ± 26% -36.8% 51.00 ± 24% interrupts.CPU81.RES:Rescheduling_interrupts 46580 ± 3% +11.7% 52016 interrupts.CPU82.LOC:Local_timer_interrupts 99.50 ± 49% -50.5% 49.25 ± 51% interrupts.CPU82.RES:Rescheduling_interrupts 46554 ± 3% +11.7% 51996 interrupts.CPU83.LOC:Local_timer_interrupts 1112 ± 16% -50.6% 549.50 ± 46% interrupts.CPU83.NMI:Non-maskable_interrupts 1112 ± 16% -50.6% 549.50 ± 46% interrupts.CPU83.PMI:Performance_monitoring_interrupts 46552 ± 3% +11.7% 51995 interrupts.CPU84.LOC:Local_timer_interrupts 46559 ± 3% +11.8% 52052 interrupts.CPU85.LOC:Local_timer_interrupts 985.00 ± 31% -48.5% 507.25 ± 5% interrupts.CPU85.NMI:Non-maskable_interrupts 985.00 ± 31% -48.5% 507.25 ± 5% interrupts.CPU85.PMI:Performance_monitoring_interrupts 46561 ± 3% +11.7% 51995 interrupts.CPU86.LOC:Local_timer_interrupts 807.25 ± 6% -45.9% 436.50 ± 28% interrupts.CPU86.NMI:Non-maskable_interrupts 807.25 ± 6% -45.9% 436.50 ± 28% interrupts.CPU86.PMI:Performance_monitoring_interrupts 46540 ± 3% +11.7% 51995 interrupts.CPU87.LOC:Local_timer_interrupts 834.50 ± 20% -45.2% 457.50 ± 57% interrupts.CPU87.NMI:Non-maskable_interrupts 834.50 ± 20% -45.2% 457.50 ± 57% interrupts.CPU87.PMI:Performance_monitoring_interrupts 46604 ± 3% +11.5% 51950 ± 2% interrupts.CPU88.LOC:Local_timer_interrupts 814.50 ± 15% -52.0% 391.25 ± 30% interrupts.CPU88.NMI:Non-maskable_interrupts 814.50 ± 15% -52.0% 391.25 ± 30% interrupts.CPU88.PMI:Performance_monitoring_interrupts 46563 ± 3% +11.5% 51929 ± 2% interrupts.CPU89.LOC:Local_timer_interrupts 881.75 ± 39% -51.2% 430.50 ± 22% interrupts.CPU89.NMI:Non-maskable_interrupts 881.75 ± 39% -51.2% 430.50 ± 22% interrupts.CPU89.PMI:Performance_monitoring_interrupts 46354 ± 3% +12.0% 51935 interrupts.CPU9.LOC:Local_timer_interrupts 46561 ± 3% +11.4% 51891 interrupts.CPU90.LOC:Local_timer_interrupts 1047 ± 41% -56.5% 455.75 ± 25% interrupts.CPU90.NMI:Non-maskable_interrupts 1047 ± 41% -56.5% 455.75 ± 25% interrupts.CPU90.PMI:Performance_monitoring_interrupts 46529 ± 3% +11.5% 51896 interrupts.CPU91.LOC:Local_timer_interrupts 856.00 ± 23% -47.2% 452.25 ± 42% interrupts.CPU91.NMI:Non-maskable_interrupts 856.00 ± 23% -47.2% 452.25 ± 42% interrupts.CPU91.PMI:Performance_monitoring_interrupts 46565 ± 3% +11.7% 51995 interrupts.CPU92.LOC:Local_timer_interrupts 889.75 ± 34% -45.3% 486.50 ± 50% interrupts.CPU92.NMI:Non-maskable_interrupts 889.75 ± 34% -45.3% 486.50 ± 50% interrupts.CPU92.PMI:Performance_monitoring_interrupts 46577 ± 3% +11.4% 51892 ± 2% interrupts.CPU93.LOC:Local_timer_interrupts 797.50 ± 8% -38.2% 492.75 ± 35% interrupts.CPU93.NMI:Non-maskable_interrupts 797.50 ± 8% -38.2% 492.75 ± 35% interrupts.CPU93.PMI:Performance_monitoring_interrupts 46564 ± 3% +11.5% 51903 interrupts.CPU94.LOC:Local_timer_interrupts 749.75 ± 23% -34.6% 490.00 ± 22% interrupts.CPU94.NMI:Non-maskable_interrupts 749.75 ± 23% -34.6% 490.00 ± 22% interrupts.CPU94.PMI:Performance_monitoring_interrupts 2895 ± 17% -25.5% 2158 ± 18% interrupts.CPU95.CAL:Function_call_interrupts 46572 ± 3% +11.4% 51890 interrupts.CPU95.LOC:Local_timer_interrupts 46376 ± 3% +11.9% 51882 ± 2% interrupts.CPU96.LOC:Local_timer_interrupts 46357 ± 3% +12.0% 51916 interrupts.CPU97.LOC:Local_timer_interrupts 46397 ± 3% +12.1% 51999 interrupts.CPU98.LOC:Local_timer_interrupts 46448 ± 2% +11.8% 51930 ± 2% interrupts.CPU99.LOC:Local_timer_interrupts 8914517 ± 3% +11.9% 9973298 interrupts.LOC:Local_timer_interrupts 33.74 -2.4 31.38 ± 4% perf-profile.calltrace.cycles-pp.ret_from_fork 33.74 -2.4 31.38 ± 4% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 17.77 ± 2% -2.2 15.55 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 17.53 ± 2% -2.1 15.47 ± 5% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 22.36 -2.0 20.38 ± 4% perf-profile.calltrace.cycles-pp.nfsd.kthread.ret_from_fork 16.52 ± 2% -1.9 14.63 ± 4% perf-profile.calltrace.cycles-pp.svc_process.nfsd.kthread.ret_from_fork 16.52 ± 2% -1.9 14.63 ± 4% perf-profile.calltrace.cycles-pp.svc_process_common.svc_process.nfsd.kthread.ret_from_fork 16.44 ± 2% -1.9 14.57 ± 3% perf-profile.calltrace.cycles-pp.nfsd_dispatch.svc_process_common.svc_process.nfsd.kthread 16.38 ± 3% -1.8 14.54 ± 4% perf-profile.calltrace.cycles-pp.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process.nfsd 9.78 ± 2% -1.2 8.60 ± 5% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.76 ± 2% -1.2 8.58 ± 5% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.69 ± 2% -1.2 8.53 ± 5% perf-profile.calltrace.cycles-pp.nfs_file_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 9.70 ± 2% -1.2 8.54 ± 5% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.34 ± 2% -1.1 8.23 ± 5% perf-profile.calltrace.cycles-pp.generic_perform_write.nfs_file_write.new_sync_write.vfs_write.ksys_write 10.04 ± 3% -1.0 9.04 ± 3% perf-profile.calltrace.cycles-pp.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process 10.02 ± 3% -1.0 9.03 ± 3% perf-profile.calltrace.cycles-pp.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch.svc_process_common 9.98 ± 3% -1.0 9.00 ± 3% perf-profile.calltrace.cycles-pp.file_write_and_wait_range.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch 9.97 ± 3% -1.0 8.98 ± 3% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound 9.96 ± 3% -1.0 8.98 ± 3% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync.nfsd_commit 9.96 ± 3% -1.0 8.98 ± 3% perf-profile.calltrace.cycles-pp.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync 5.98 ± 2% -0.8 5.17 ± 5% perf-profile.calltrace.cycles-pp.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process 5.95 ± 2% -0.8 5.14 ± 5% perf-profile.calltrace.cycles-pp.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch.svc_process_common 5.94 ± 2% -0.8 5.13 ± 5% perf-profile.calltrace.cycles-pp.do_iter_write.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch 5.92 ± 2% -0.8 5.12 ± 5% perf-profile.calltrace.cycles-pp.do_iter_readv_writev.do_iter_write.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound 5.92 ± 2% -0.8 5.12 ± 5% perf-profile.calltrace.cycles-pp.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write.nfsd_vfs_write.nfsd4_write 8.55 ± 3% -0.8 7.76 ± 3% perf-profile.calltrace.cycles-pp.brd_do_bvec.brd_submit_bio.submit_bio_noacct.submit_bio.iomap_submit_ioend 5.67 ± 2% -0.7 4.96 ± 5% perf-profile.calltrace.cycles-pp.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write.nfsd_vfs_write 5.67 ± 2% -0.7 4.96 ± 5% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write 5.63 ± 2% -0.7 4.93 ± 5% perf-profile.calltrace.cycles-pp.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev 1.86 ± 4% -0.7 1.19 ± 4% perf-profile.calltrace.cycles-pp.iomap_writepages.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range 1.85 ± 4% -0.7 1.19 ± 4% perf-profile.calltrace.cycles-pp.write_cache_pages.iomap_writepages.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range 1.30 ± 7% -0.6 0.66 ± 5% perf-profile.calltrace.cycles-pp.iomap_writepage_map.write_cache_pages.iomap_writepages.xfs_vm_writepages.do_writepages 4.55 ± 3% -0.6 3.99 ± 4% perf-profile.calltrace.cycles-pp.nfs_write_end.generic_perform_write.nfs_file_write.new_sync_write.vfs_write 5.47 ± 3% -0.5 4.93 ± 4% perf-profile.calltrace.cycles-pp.nfs4_file_flush.filp_close.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.47 ± 3% -0.5 4.94 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.47 ± 3% -0.5 4.94 ± 4% perf-profile.calltrace.cycles-pp.filp_close.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.47 ± 3% -0.5 4.93 ± 4% perf-profile.calltrace.cycles-pp.nfs_wb_all.nfs4_file_flush.filp_close.__x64_sys_close.do_syscall_64 4.44 ± 2% -0.5 3.95 ± 3% perf-profile.calltrace.cycles-pp.rpc_free_task.rpc_async_release.process_one_work.worker_thread.kthread 4.44 ± 2% -0.5 3.95 ± 3% perf-profile.calltrace.cycles-pp.rpc_async_release.process_one_work.worker_thread.kthread.ret_from_fork 3.58 ± 2% -0.4 3.13 ± 3% perf-profile.calltrace.cycles-pp.nfs_write_completion.rpc_free_task.rpc_async_release.process_one_work.worker_thread 3.35 ± 4% -0.4 2.90 ± 5% perf-profile.calltrace.cycles-pp.nfs_updatepage.nfs_write_end.generic_perform_write.nfs_file_write.new_sync_write 2.43 ± 3% -0.3 2.09 ± 3% perf-profile.calltrace.cycles-pp.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release.process_one_work 2.37 ± 3% -0.3 2.05 ± 4% perf-profile.calltrace.cycles-pp.nfs_writepage_setup.nfs_updatepage.nfs_write_end.generic_perform_write.nfs_file_write 0.57 ± 5% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.__test_set_page_writeback.nfs_do_writepage.nfs_writepages_callback.write_cache_pages.nfs_writepages 2.73 ± 4% -0.3 2.43 ± 5% perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 2.69 ± 4% -0.3 2.39 ± 5% perf-profile.calltrace.cycles-pp.memcpy_erms.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply.iomap_file_buffered_write 2.20 -0.3 1.93 ± 4% perf-profile.calltrace.cycles-pp.nfs_write_begin.generic_perform_write.nfs_file_write.new_sync_write.vfs_write 2.10 -0.2 1.85 ± 4% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write 2.11 -0.2 1.86 ± 4% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write.new_sync_write 0.66 ± 4% -0.2 0.42 ± 57% perf-profile.calltrace.cycles-pp.schedule.io_schedule.wait_on_page_bit.__filemap_fdatawait_range.filemap_write_and_wait_range 0.65 ± 4% -0.2 0.41 ± 57% perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.wait_on_page_bit.__filemap_fdatawait_range 1.48 -0.2 1.28 ± 5% perf-profile.calltrace.cycles-pp.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 0.60 ± 5% -0.2 0.40 ± 57% perf-profile.calltrace.cycles-pp.__nfs_commit_inode.nfs_wb_all.nfs4_file_flush.filp_close.__x64_sys_close 0.83 ± 13% -0.2 0.64 ± 2% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 1.40 -0.2 1.21 ± 5% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write 1.38 -0.2 1.20 ± 4% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply 1.19 ± 3% -0.2 1.02 ± 4% perf-profile.calltrace.cycles-pp.__filemap_fdatawait_range.filemap_write_and_wait_range.nfs_wb_all.nfs4_file_flush.filp_close 1.52 -0.2 1.35 ± 5% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write 0.78 ± 3% -0.1 0.65 ± 4% perf-profile.calltrace.cycles-pp.wait_on_page_bit.__filemap_fdatawait_range.filemap_write_and_wait_range.nfs_wb_all.nfs4_file_flush 0.85 ± 2% -0.1 0.72 ± 5% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor 0.67 ± 4% -0.1 0.54 ± 5% perf-profile.calltrace.cycles-pp.io_schedule.wait_on_page_bit.__filemap_fdatawait_range.filemap_write_and_wait_range.nfs_wb_all 0.91 ± 5% -0.1 0.79 ± 4% perf-profile.calltrace.cycles-pp.__set_page_dirty_nobuffers.nfs_writepage_setup.nfs_updatepage.nfs_write_end.generic_perform_write 0.83 ± 5% -0.1 0.71 ± 4% perf-profile.calltrace.cycles-pp.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release 1.05 ± 2% -0.1 0.94 ± 6% perf-profile.calltrace.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin 0.90 ± 4% -0.1 0.80 ± 3% perf-profile.calltrace.cycles-pp.end_page_writeback.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release 0.88 ± 4% -0.1 0.78 ± 3% perf-profile.calltrace.cycles-pp.test_clear_page_writeback.end_page_writeback.nfs_end_page_writeback.nfs_write_completion.rpc_free_task 0.70 ± 5% -0.1 0.61 ± 5% perf-profile.calltrace.cycles-pp.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion.rpc_free_task 0.66 ± 4% -0.1 0.58 ± 4% perf-profile.calltrace.cycles-pp.wake_page_function.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion 0.66 ± 4% -0.1 0.57 ± 5% perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_page_function.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback 0.87 ± 5% +0.1 0.97 ± 8% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.96 ± 5% +0.1 1.06 ± 7% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 0.83 ± 5% +0.1 0.94 ± 5% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.83 ± 5% +0.1 0.94 ± 5% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.83 ± 5% +0.1 0.94 ± 5% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt 1.02 ± 6% +0.1 1.15 ± 6% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 4.18 ± 2% +0.3 4.50 ± 5% perf-profile.calltrace.cycles-pp.svc_tcp_recvfrom.svc_handle_xprt.svc_recv.nfsd.kthread 0.00 +0.6 0.55 ± 6% perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_recvmsg.inet6_recvmsg.svc_tcp_recvfrom 3.21 +0.6 3.80 ± 4% perf-profile.calltrace.cycles-pp.inet6_recvmsg.svc_tcp_recvfrom.svc_handle_xprt.svc_recv.nfsd 3.20 ± 2% +0.6 3.79 ± 4% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet6_recvmsg.svc_tcp_recvfrom.svc_handle_xprt.svc_recv 0.00 +0.7 0.66 ± 5% perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_recvmsg.inet6_recvmsg.svc_tcp_recvfrom.svc_handle_xprt 4.16 ± 5% +1.7 5.85 ± 12% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.00 +3.1 3.13 ± 6% perf-profile.calltrace.cycles-pp.memcpy_erms.skb_copy_ubufs.__netif_receive_skb_core.__netif_receive_skb_one_core.process_backlog 41.73 +3.3 44.98 ± 5% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.00 +3.9 3.94 ± 6% perf-profile.calltrace.cycles-pp.skb_copy_ubufs.__netif_receive_skb_core.__netif_receive_skb_one_core.process_backlog.net_rx_action 0.00 +4.0 3.95 ± 6% perf-profile.calltrace.cycles-pp.__netif_receive_skb_core.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start 0.00 +4.1 4.13 ± 6% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack 0.00 +4.1 4.14 ± 6% perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack 0.00 +4.1 4.14 ± 6% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq 0.00 +4.2 4.16 ± 6% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip6_finish_output2 0.00 +4.2 4.16 ± 6% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip6_finish_output2.ip6_output 0.00 +4.2 4.16 ± 6% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq.__local_bh_enable_ip 0.00 +4.2 4.16 ± 6% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip6_finish_output2.ip6_output.ip6_xmit.inet6_csk_xmit 0.00 +4.2 4.16 ± 6% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip6_finish_output2.ip6_output.ip6_xmit 0.00 +4.2 4.20 ± 6% perf-profile.calltrace.cycles-pp.ip6_finish_output2.ip6_output.ip6_xmit.inet6_csk_xmit.__tcp_transmit_skb 0.00 +4.2 4.22 ± 6% perf-profile.calltrace.cycles-pp.ip6_output.ip6_xmit.inet6_csk_xmit.__tcp_transmit_skb.tcp_write_xmit 0.00 +4.2 4.22 ± 6% perf-profile.calltrace.cycles-pp.ip6_xmit.inet6_csk_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked 0.00 +4.2 4.25 ± 6% perf-profile.calltrace.cycles-pp.inet6_csk_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg 0.00 +4.3 4.30 ± 6% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg 0.00 +4.4 4.36 ± 6% perf-profile.calltrace.cycles-pp.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.xprt_sock_sendmsg 47.71 +4.6 52.32 ± 4% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 47.45 +4.7 52.16 ± 4% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 47.43 +4.7 52.15 ± 4% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 47.44 +4.7 52.16 ± 4% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 33.74 -2.4 31.38 ± 4% perf-profile.children.cycles-pp.ret_from_fork 33.74 -2.4 31.38 ± 4% perf-profile.children.cycles-pp.kthread 18.18 ± 2% -2.2 15.97 ± 5% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 17.93 ± 2% -2.0 15.89 ± 5% perf-profile.children.cycles-pp.do_syscall_64 22.36 -2.0 20.38 ± 4% perf-profile.children.cycles-pp.nfsd 16.52 ± 2% -1.9 14.63 ± 4% perf-profile.children.cycles-pp.svc_process 16.52 ± 2% -1.9 14.63 ± 4% perf-profile.children.cycles-pp.svc_process_common 16.44 ± 2% -1.9 14.57 ± 3% perf-profile.children.cycles-pp.nfsd_dispatch 16.38 ± 3% -1.8 14.54 ± 4% perf-profile.children.cycles-pp.nfsd4_proc_compound 13.64 ± 3% -1.3 12.38 ± 3% perf-profile.children.cycles-pp.__filemap_fdatawrite_range 13.64 ± 3% -1.3 12.38 ± 3% perf-profile.children.cycles-pp.do_writepages 10.04 ± 2% -1.2 8.87 ± 5% perf-profile.children.cycles-pp.ksys_write 10.02 ± 2% -1.2 8.85 ± 5% perf-profile.children.cycles-pp.vfs_write 9.70 ± 2% -1.2 8.53 ± 5% perf-profile.children.cycles-pp.nfs_file_write 9.96 ± 2% -1.2 8.81 ± 5% perf-profile.children.cycles-pp.new_sync_write 9.35 ± 2% -1.1 8.23 ± 5% perf-profile.children.cycles-pp.generic_perform_write 10.04 ± 3% -1.0 9.04 ± 3% perf-profile.children.cycles-pp.nfsd_commit 10.02 ± 3% -1.0 9.03 ± 3% perf-profile.children.cycles-pp.xfs_file_fsync 9.98 ± 3% -1.0 9.00 ± 3% perf-profile.children.cycles-pp.file_write_and_wait_range 9.96 ± 3% -1.0 8.98 ± 3% perf-profile.children.cycles-pp.xfs_vm_writepages 5.51 ± 3% -0.9 4.56 ± 4% perf-profile.children.cycles-pp.write_cache_pages 8.74 ± 3% -0.9 7.84 ± 3% perf-profile.children.cycles-pp.submit_bio 8.74 ± 3% -0.9 7.84 ± 3% perf-profile.children.cycles-pp.submit_bio_noacct 8.68 ± 3% -0.9 7.79 ± 3% perf-profile.children.cycles-pp.iomap_submit_ioend 8.73 ± 3% -0.9 7.84 ± 3% perf-profile.children.cycles-pp.brd_submit_bio 8.70 ± 3% -0.9 7.81 ± 3% perf-profile.children.cycles-pp.brd_do_bvec 5.98 ± 2% -0.8 5.17 ± 5% perf-profile.children.cycles-pp.nfsd4_write 5.95 ± 2% -0.8 5.14 ± 5% perf-profile.children.cycles-pp.nfsd_vfs_write 5.94 ± 2% -0.8 5.13 ± 5% perf-profile.children.cycles-pp.do_iter_write 5.92 ± 2% -0.8 5.12 ± 5% perf-profile.children.cycles-pp.do_iter_readv_writev 5.92 ± 2% -0.8 5.12 ± 5% perf-profile.children.cycles-pp.xfs_file_buffered_aio_write 5.67 ± 2% -0.7 4.96 ± 5% perf-profile.children.cycles-pp.iomap_file_buffered_write 5.67 ± 2% -0.7 4.96 ± 5% perf-profile.children.cycles-pp.iomap_apply 5.63 ± 2% -0.7 4.94 ± 5% perf-profile.children.cycles-pp.iomap_write_actor 1.86 ± 4% -0.7 1.19 ± 4% perf-profile.children.cycles-pp.iomap_writepages 1.31 ± 7% -0.6 0.67 ± 5% perf-profile.children.cycles-pp.iomap_writepage_map 5.90 ± 4% -0.6 5.30 ± 3% perf-profile.children.cycles-pp.brd_insert_page 5.00 ± 3% -0.6 4.44 ± 6% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic 4.56 ± 3% -0.6 4.00 ± 4% perf-profile.children.cycles-pp.nfs_write_end 5.47 ± 3% -0.5 4.94 ± 4% perf-profile.children.cycles-pp.filp_close 5.47 ± 3% -0.5 4.94 ± 4% perf-profile.children.cycles-pp.__x64_sys_close 5.47 ± 3% -0.5 4.93 ± 4% perf-profile.children.cycles-pp.nfs4_file_flush 5.47 ± 3% -0.5 4.93 ± 4% perf-profile.children.cycles-pp.nfs_wb_all 4.69 ± 2% -0.5 4.16 ± 3% perf-profile.children.cycles-pp.rpc_free_task 5.35 ± 5% -0.5 4.83 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 4.44 ± 2% -0.5 3.95 ± 3% perf-profile.children.cycles-pp.rpc_async_release 3.58 ± 2% -0.4 3.13 ± 3% perf-profile.children.cycles-pp.nfs_write_completion 3.35 ± 4% -0.4 2.91 ± 5% perf-profile.children.cycles-pp.nfs_updatepage 3.51 -0.4 3.08 ± 4% perf-profile.children.cycles-pp.grab_cache_page_write_begin 3.48 -0.4 3.06 ± 4% perf-profile.children.cycles-pp.pagecache_get_page 2.43 ± 3% -0.3 2.09 ± 3% perf-profile.children.cycles-pp.nfs_end_page_writeback 1.91 ± 2% -0.3 1.58 ± 3% perf-profile.children.cycles-pp.__schedule 2.38 ± 3% -0.3 2.06 ± 4% perf-profile.children.cycles-pp.nfs_writepage_setup 2.37 -0.3 2.07 ± 5% perf-profile.children.cycles-pp.add_to_page_cache_lru 1.54 -0.3 1.27 ± 3% perf-profile.children.cycles-pp.schedule 2.20 -0.3 1.93 ± 4% perf-profile.children.cycles-pp.nfs_write_begin 1.49 -0.2 1.29 ± 5% perf-profile.children.cycles-pp.iomap_write_begin 0.84 ± 13% -0.2 0.64 ± 2% perf-profile.children.cycles-pp.poll_idle 1.11 ± 2% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.pick_next_task_fair 1.56 ± 2% -0.2 1.38 ± 6% perf-profile.children.cycles-pp.__add_to_page_cache_locked 1.21 ± 2% -0.2 1.04 ± 5% perf-profile.children.cycles-pp.__filemap_fdatawait_range 0.98 ± 2% -0.2 0.81 ± 3% perf-profile.children.cycles-pp.newidle_balance 1.05 -0.2 0.88 ± 4% perf-profile.children.cycles-pp.ttwu_do_activate 0.23 ±102% -0.2 0.06 ± 6% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 1.04 ± 2% -0.2 0.88 ± 5% perf-profile.children.cycles-pp.enqueue_task_fair 1.08 ± 7% -0.2 0.93 ± 3% perf-profile.children.cycles-pp.percpu_counter_add_batch 1.21 ± 2% -0.2 1.05 ± 5% perf-profile.children.cycles-pp.try_to_wake_up 1.10 -0.2 0.94 ± 3% perf-profile.children.cycles-pp.load_balance 0.60 ± 6% -0.2 0.45 ± 7% perf-profile.children.cycles-pp.tcp_rcv_established 0.61 ± 6% -0.1 0.46 ± 7% perf-profile.children.cycles-pp.tcp_v6_do_rcv 1.00 -0.1 0.86 ± 3% perf-profile.children.cycles-pp.find_busiest_group 0.96 -0.1 0.82 ± 6% perf-profile.children.cycles-pp.enqueue_entity 0.79 ± 3% -0.1 0.65 ± 5% perf-profile.children.cycles-pp.wait_on_page_bit 0.99 -0.1 0.85 ± 2% perf-profile.children.cycles-pp.update_sd_lb_stats 0.30 ± 5% -0.1 0.16 ± 10% perf-profile.children.cycles-pp.tcp_ack 0.61 ± 3% -0.1 0.48 ± 3% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle 0.28 ± 5% -0.1 0.15 ± 12% perf-profile.children.cycles-pp.tcp_clean_rtx_queue 1.45 ± 3% -0.1 1.32 ± 3% perf-profile.children.cycles-pp.end_page_writeback 0.65 ± 2% -0.1 0.52 ± 3% perf-profile.children.cycles-pp.sched_ttwu_pending 0.76 ± 7% -0.1 0.64 ± 4% perf-profile.children.cycles-pp.account_page_dirtied 1.43 ± 3% -0.1 1.31 ± 3% perf-profile.children.cycles-pp.test_clear_page_writeback 0.97 ± 5% -0.1 0.84 ± 4% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers 0.68 ± 3% -0.1 0.55 ± 5% perf-profile.children.cycles-pp.io_schedule 0.74 -0.1 0.62 ± 6% perf-profile.children.cycles-pp.__account_scheduler_latency 0.45 ± 5% -0.1 0.33 ± 7% perf-profile.children.cycles-pp.__release_sock 0.85 ± 5% -0.1 0.73 ± 4% perf-profile.children.cycles-pp.wake_up_page_bit 0.56 ± 3% -0.1 0.44 ± 7% perf-profile.children.cycles-pp.release_sock 0.89 ± 3% -0.1 0.78 ± 4% perf-profile.children.cycles-pp.__wake_up_common 0.98 ± 2% -0.1 0.87 ± 4% perf-profile.children.cycles-pp.__test_set_page_writeback 0.77 -0.1 0.66 ± 5% perf-profile.children.cycles-pp.lru_cache_add 0.55 ± 8% -0.1 0.44 ± 4% perf-profile.children.cycles-pp.iomap_set_page_dirty 0.99 ± 4% -0.1 0.88 ± 5% perf-profile.children.cycles-pp.__nfs_commit_inode 0.23 ± 8% -0.1 0.14 ± 11% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.21 ± 9% -0.1 0.12 ± 17% perf-profile.children.cycles-pp.xfs_ilock 0.80 ± 3% -0.1 0.71 ± 5% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.71 ± 2% -0.1 0.62 ± 5% perf-profile.children.cycles-pp.pagevec_lru_move_fn 0.68 ± 5% -0.1 0.59 ± 4% perf-profile.children.cycles-pp.wake_page_function 0.58 -0.1 0.48 ± 6% perf-profile.children.cycles-pp.stack_trace_save_tsk 0.29 ± 11% -0.1 0.20 ± 13% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.52 ± 4% -0.1 0.43 ± 6% perf-profile.children.cycles-pp.arch_stack_walk 0.28 ± 11% -0.1 0.20 ± 15% perf-profile.children.cycles-pp.rwsem_optimistic_spin 0.54 ± 2% -0.1 0.46 ± 4% perf-profile.children.cycles-pp.__pagevec_lru_add_fn 0.55 ± 7% -0.1 0.48 ± 7% perf-profile.children.cycles-pp.__mod_memcg_state 0.72 ± 3% -0.1 0.65 ± 5% perf-profile.children.cycles-pp.nfs_scan_commit 0.75 -0.1 0.68 ± 5% perf-profile.children.cycles-pp.mem_cgroup_charge 0.38 ± 13% -0.1 0.31 ± 6% perf-profile.children.cycles-pp.__set_page_dirty 0.37 ± 2% -0.1 0.30 ± 3% perf-profile.children.cycles-pp.unwind_next_frame 0.21 ± 10% -0.1 0.15 ± 18% perf-profile.children.cycles-pp.svc_xprt_do_enqueue 0.41 ± 5% -0.1 0.35 ± 4% perf-profile.children.cycles-pp.schedule_idle 0.51 ± 2% -0.1 0.45 ± 9% perf-profile.children.cycles-pp.nfs_ctx_key_to_expire 0.49 ± 5% -0.1 0.43 ± 7% perf-profile.children.cycles-pp.nfs_create_request 0.18 ± 5% -0.1 0.12 ± 16% perf-profile.children.cycles-pp.svc_xprt_received 0.39 ± 4% -0.1 0.34 ± 6% perf-profile.children.cycles-pp.xas_load 0.23 ± 6% -0.1 0.18 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair 0.75 -0.1 0.70 perf-profile.children.cycles-pp.clear_page_dirty_for_io 0.21 ± 7% -0.0 0.16 ± 4% perf-profile.children.cycles-pp.dequeue_entity 0.33 -0.0 0.28 ± 2% perf-profile.children.cycles-pp.nfs_lock_and_join_requests 0.15 ± 3% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.clear_bdi_congested 0.16 ± 12% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.schedule_timeout 0.12 ± 11% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.iov_iter_advance 0.35 ± 3% -0.0 0.32 ± 2% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.16 ± 5% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.lock_sock_nested 0.06 -0.0 0.03 ±100% perf-profile.children.cycles-pp.__inc_numa_state 0.38 -0.0 0.35 ± 4% perf-profile.children.cycles-pp.nfs_scan_commit_list 0.25 ± 7% -0.0 0.22 ± 8% perf-profile.children.cycles-pp.nfs_initiate_commit 0.25 ± 3% -0.0 0.22 ± 8% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm 0.11 ± 4% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.__wait_on_bit 0.39 ± 2% -0.0 0.36 ± 3% perf-profile.children.cycles-pp.nfs_io_completion_release 0.33 ± 3% -0.0 0.30 ± 4% perf-profile.children.cycles-pp.find_get_pages_range_tag 0.13 ± 10% -0.0 0.10 ± 8% perf-profile.children.cycles-pp.update_curr 0.11 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.out_of_line_wait_on_bit 0.33 ± 3% -0.0 0.30 ± 2% perf-profile.children.cycles-pp.pagevec_lookup_range_tag 0.18 ± 3% -0.0 0.15 ± 5% perf-profile.children.cycles-pp.xs_stream_data_receive_workfn 0.18 ± 3% -0.0 0.15 ± 5% perf-profile.children.cycles-pp.xs_stream_data_receive 0.13 ± 8% -0.0 0.10 ± 7% perf-profile.children.cycles-pp.__x64_sys_gettimeofday 0.28 ± 2% -0.0 0.25 ± 5% perf-profile.children.cycles-pp.kmem_cache_alloc 0.15 ± 6% -0.0 0.12 ± 6% perf-profile.children.cycles-pp.nfs_page_find_private_request 0.10 -0.0 0.08 ± 6% perf-profile.children.cycles-pp.rpc_wait_bit_killable 0.08 ± 5% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.14 -0.0 0.12 ± 9% perf-profile.children.cycles-pp.sysvec_call_function_single 0.16 ± 2% -0.0 0.14 ± 8% perf-profile.children.cycles-pp.xas_store 0.15 ± 7% -0.0 0.12 ± 6% perf-profile.children.cycles-pp.find_get_entry 0.11 ± 4% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.xs_sock_recvmsg 0.11 ± 7% -0.0 0.09 ± 9% perf-profile.children.cycles-pp.__xa_set_mark 0.10 ± 7% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.set_next_entity 0.10 ± 7% -0.0 0.08 perf-profile.children.cycles-pp.__fput 0.08 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp._raw_spin_lock_bh 0.07 ± 11% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.task_work_run 0.09 ± 4% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.09 ± 9% -0.0 0.07 ± 10% perf-profile.children.cycles-pp.xas_set_mark 0.08 ± 5% -0.0 0.07 ± 12% perf-profile.children.cycles-pp.__switch_to 0.10 ± 5% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.xa_get_order 0.08 ± 10% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.refcount_dec_and_lock 0.11 ± 4% -0.0 0.10 ± 7% perf-profile.children.cycles-pp._cond_resched 0.07 -0.0 0.06 ± 7% perf-profile.children.cycles-pp.nfsd_set_fh_dentry 0.06 ± 11% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.sk_stream_alloc_skb 0.11 ± 8% +0.0 0.14 ± 7% perf-profile.children.cycles-pp.__slab_free 0.29 ± 5% +0.0 0.34 ± 6% perf-profile.children.cycles-pp.__tcp_push_pending_frames 0.44 ± 5% +0.1 0.49 ± 5% perf-profile.children.cycles-pp.rebalance_domains 0.26 ± 4% +0.1 0.31 ± 7% perf-profile.children.cycles-pp.native_irq_return_iret 0.01 ±173% +0.1 0.07 ± 7% perf-profile.children.cycles-pp.kfree 0.47 ± 5% +0.1 0.54 ± 7% perf-profile.children.cycles-pp.timekeeping_max_deferment 0.30 ± 6% +0.1 0.38 ± 10% perf-profile.children.cycles-pp.svc_tcp_sendto 0.33 ± 6% +0.1 0.41 ± 10% perf-profile.children.cycles-pp.svc_send 0.97 ± 5% +0.1 1.07 ± 7% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.00 +0.1 0.13 ± 6% perf-profile.children.cycles-pp.pskb_expand_head 0.00 +0.1 0.14 ± 16% perf-profile.children.cycles-pp.iov_iter_get_pages 0.10 ± 15% +0.2 0.26 ± 9% perf-profile.children.cycles-pp.__check_object_size 0.08 ± 13% +0.2 0.26 ± 10% perf-profile.children.cycles-pp.simple_copy_to_iter 0.00 +0.2 0.18 ± 12% perf-profile.children.cycles-pp.__zerocopy_sg_from_iter 0.00 +0.2 0.19 ± 12% perf-profile.children.cycles-pp.skb_zerocopy_iter_stream 0.28 ± 17% +0.3 0.55 ± 7% perf-profile.children.cycles-pp.free_pcppages_bulk 0.50 ± 6% +0.3 0.79 ± 9% perf-profile.children.cycles-pp.rmqueue_bulk 0.46 ± 5% +0.3 0.77 ± 7% perf-profile.children.cycles-pp.free_unref_page 4.19 ± 2% +0.3 4.50 ± 5% perf-profile.children.cycles-pp.svc_tcp_recvfrom 0.79 ± 2% +0.4 1.18 ± 6% perf-profile.children.cycles-pp.__list_del_entry_valid 0.24 ± 6% +0.4 0.63 ± 6% perf-profile.children.cycles-pp.skb_release_data 0.33 ± 6% +0.4 0.76 ± 5% perf-profile.children.cycles-pp.__kfree_skb 3.37 +0.6 3.95 ± 4% perf-profile.children.cycles-pp.inet6_recvmsg 3.36 ± 2% +0.6 3.94 ± 4% perf-profile.children.cycles-pp.tcp_recvmsg 4.19 ± 5% +1.7 5.87 ± 12% perf-profile.children.cycles-pp.menu_select 5.57 ± 2% +2.8 8.38 ± 5% perf-profile.children.cycles-pp.memcpy_erms 41.96 +3.1 45.11 ± 5% perf-profile.children.cycles-pp.cpuidle_enter 41.96 +3.1 45.11 ± 5% perf-profile.children.cycles-pp.cpuidle_enter_state 0.74 ± 5% +4.0 4.72 ± 5% perf-profile.children.cycles-pp.tcp_write_xmit 0.80 ± 6% +4.0 4.83 ± 5% perf-profile.children.cycles-pp.__tcp_transmit_skb 0.71 ± 6% +4.0 4.74 ± 5% perf-profile.children.cycles-pp.inet6_csk_xmit 0.63 ± 5% +4.0 4.68 ± 5% perf-profile.children.cycles-pp.ip6_output 0.59 ± 7% +4.0 4.64 ± 6% perf-profile.children.cycles-pp.ip6_finish_output2 0.65 ± 6% +4.1 4.70 ± 5% perf-profile.children.cycles-pp.ip6_xmit 0.43 ± 8% +4.1 4.49 ± 6% perf-profile.children.cycles-pp.__netif_receive_skb_one_core 0.50 ± 8% +4.1 4.56 ± 6% perf-profile.children.cycles-pp.__local_bh_enable_ip 0.45 ± 9% +4.1 4.51 ± 6% perf-profile.children.cycles-pp.process_backlog 0.46 ± 8% +4.1 4.53 ± 6% perf-profile.children.cycles-pp.net_rx_action 0.49 ± 8% +4.1 4.56 ± 6% perf-profile.children.cycles-pp.do_softirq 0.00 +4.1 4.10 ± 6% perf-profile.children.cycles-pp.skb_copy_ubufs 0.00 +4.1 4.12 ± 5% perf-profile.children.cycles-pp.__netif_receive_skb_core 1.40 ± 6% +4.2 5.56 ± 4% perf-profile.children.cycles-pp.__softirqentry_text_start 1.40 ± 6% +4.2 5.56 ± 4% perf-profile.children.cycles-pp.do_softirq_own_stack 5.04 ± 5% +4.2 9.28 ± 7% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 47.71 +4.6 52.32 ± 4% perf-profile.children.cycles-pp.secondary_startup_64_no_verify 47.71 +4.6 52.32 ± 4% perf-profile.children.cycles-pp.cpu_startup_entry 47.71 +4.6 52.31 ± 4% perf-profile.children.cycles-pp.do_idle 47.45 +4.7 52.16 ± 4% perf-profile.children.cycles-pp.start_secondary 1.83 ± 6% -0.3 1.53 ± 13% perf-profile.self.cycles-pp.ktime_get 2.21 ± 2% -0.2 1.97 ± 2% perf-profile.self.cycles-pp.brd_do_bvec 2.93 ± 2% -0.2 2.70 ± 3% perf-profile.self.cycles-pp._raw_spin_lock 0.81 ± 12% -0.2 0.62 ± 2% perf-profile.self.cycles-pp.poll_idle 0.97 ± 7% -0.1 0.82 ± 3% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.80 ± 7% -0.1 0.69 ± 7% perf-profile.self.cycles-pp.nfs_writepage_setup 0.67 -0.1 0.56 ± 2% perf-profile.self.cycles-pp.update_sd_lb_stats 0.50 ± 6% -0.1 0.39 ± 4% perf-profile.self.cycles-pp.nfs_end_page_writeback 0.92 ± 3% -0.1 0.81 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.47 ± 6% -0.1 0.37 ± 9% perf-profile.self.cycles-pp.rmqueue 0.36 ± 7% -0.1 0.27 ± 10% perf-profile.self.cycles-pp.svc_tcp_recvfrom 0.23 ± 8% -0.1 0.14 ± 11% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.83 ± 3% -0.1 0.75 ± 4% perf-profile.self.cycles-pp.nfs_write_end 0.54 ± 6% -0.1 0.46 ± 7% perf-profile.self.cycles-pp.__mod_memcg_state 0.25 ± 6% -0.1 0.18 ± 22% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.08 ± 5% -0.1 0.03 ±102% perf-profile.self.cycles-pp.svc_recv 0.30 ± 4% -0.0 0.26 ± 8% perf-profile.self.cycles-pp.xas_load 0.15 ± 5% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.clear_bdi_congested 0.19 ± 2% -0.0 0.15 ± 5% perf-profile.self.cycles-pp.unwind_next_frame 0.11 ± 14% -0.0 0.08 ± 6% perf-profile.self.cycles-pp.iov_iter_advance 0.25 ± 4% -0.0 0.22 ± 8% perf-profile.self.cycles-pp.__pagevec_lru_add_fn 0.08 ± 5% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.enqueue_task_fair 0.29 ± 2% -0.0 0.26 ± 4% perf-profile.self.cycles-pp.clear_page_dirty_for_io 0.24 ± 2% -0.0 0.22 ± 6% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm 0.21 ± 3% -0.0 0.18 ± 5% perf-profile.self.cycles-pp.get_page_from_freelist 0.19 ± 6% -0.0 0.17 ± 3% perf-profile.self.cycles-pp.__schedule 0.16 ± 4% -0.0 0.13 ± 8% perf-profile.self.cycles-pp.try_to_wake_up 0.11 ± 11% -0.0 0.09 ± 13% perf-profile.self.cycles-pp.alloc_pages_current 0.10 ± 8% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.brd_insert_page 0.26 ± 2% -0.0 0.24 ± 2% perf-profile.self.cycles-pp.find_get_pages_range_tag 0.08 ± 6% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.select_task_rq_fair 0.07 ± 5% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.iomap_writepage_map 0.08 ± 5% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.__switch_to 0.15 ± 5% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.release_pages 0.09 ± 4% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.nfs_scan_commit_list 0.08 ± 5% -0.0 0.07 ± 10% perf-profile.self.cycles-pp.xas_set_mark 0.14 ± 7% +0.0 0.17 ± 9% perf-profile.self.cycles-pp.skb_release_data 0.11 ± 8% +0.0 0.14 ± 7% perf-profile.self.cycles-pp.__slab_free 0.26 ± 4% +0.1 0.31 ± 7% perf-profile.self.cycles-pp.native_irq_return_iret 0.01 ±173% +0.1 0.07 ± 7% perf-profile.self.cycles-pp.kfree 0.47 ± 5% +0.1 0.54 ± 7% perf-profile.self.cycles-pp.timekeeping_max_deferment 0.00 +0.1 0.10 ± 5% perf-profile.self.cycles-pp.pskb_expand_head 0.12 ± 5% +0.1 0.24 ± 5% perf-profile.self.cycles-pp.free_pcppages_bulk 0.00 +0.1 0.14 ± 16% perf-profile.self.cycles-pp.iov_iter_get_pages 0.00 +0.1 0.14 ± 5% perf-profile.self.cycles-pp.skb_copy_ubufs 0.07 ± 15% +0.2 0.23 ± 6% perf-profile.self.cycles-pp.__check_object_size 0.78 ± 2% +0.4 1.16 ± 6% perf-profile.self.cycles-pp.__list_del_entry_valid 2.82 ± 7% +1.1 3.88 ± 18% perf-profile.self.cycles-pp.cpuidle_enter_state 3.14 ± 7% +1.6 4.71 ± 15% perf-profile.self.cycles-pp.menu_select 5.50 ± 2% +2.8 8.30 ± 5% perf-profile.self.cycles-pp.memcpy_erms Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen