All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Huang\, Ying" <ying.huang@intel.com>
To: Christoph Hellwig <hch@lst.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>, "Huang\,
	Ying" <ying.huang@intel.com>, Dave Chinner <david@fromorbit.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Bob Peterson <rpeterso@redhat.com>,
	Wu Fengguang <fengguang.wu@intel.com>, LKP <lkp@01.org>
Subject: Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression
Date: Thu, 11 Aug 2016 14:16:03 -0700	[thread overview]
Message-ID: <87ziojxazw.fsf@yhuang-mobile.sh.intel.com> (raw)
In-Reply-To: <20160811200018.GA28271@lst.de> (Christoph Hellwig's message of "Thu, 11 Aug 2016 22:00:18 +0200")

Christoph Hellwig <hch@lst.de> writes:

> On Thu, Aug 11, 2016 at 12:51:31PM -0700, Linus Torvalds wrote:
>> Ok. It does seem to also reset the active file page counts back, so
>> that part did seem to be related, but yeah, from a performance
>> standpoint that was clearly not a major issue.
>> 
>> Let's hope Dave can figure out something based on his numbers, because
>> I'm out of ideas. Or maybe it's the pagefault-atomic thing that
>> Christoph was looking at.
>
> I can't really think of any reason why the pagefaul_disable() would
> sіgnificantly change performance.  Anyway, the patch for the is below
> (on top of the previous mark_page_accessed() one), so feel free to
> re-run the test with it.  It would also be nice to see the profiles
> with the two patches applied.
>
> commit 43106eea246074acc4bb7d12fdb91f58002f52ed
> Author: Christoph Hellwig <hch@lst.de>
> Date:   Thu Aug 11 10:41:40 2016 -0700
>
>     fs: remove superflous pagefault_disable from iomap_write_actor
>     
>     No idea where this really came from, generic_perform_write only briefly
>     did a pagefaul_disable when trying a different prefault scheme.
>     
>     Signed-off-by: Christoph Hellwig <hch@lst.de>
>
> diff --git a/fs/iomap.c b/fs/iomap.c
> index f39c318..74712e2 100644
> --- a/fs/iomap.c
> +++ b/fs/iomap.c
> @@ -194,9 +194,7 @@ again:
>  		if (mapping_writably_mapped(inode->i_mapping))
>  			flush_dcache_page(page);
>  
> -		pagefault_disable();
>  		copied = iov_iter_copy_from_user_atomic(page, i, offset, bytes);
> -		pagefault_enable();
>  
>  		flush_dcache_page(page);
>  

Test result is as follow,

commit e129b86bfacc8bb517b843fca41d0d179de7a4ca
Author: Christoph Hellwig <hch@lst.de>
Date:   Thu Aug 11 10:41:40 2016 -0700

    fs: remove superflous pagefault_disable from iomap_write_actor
    
    No idea where this really came from, generic_perform_write only briefly
    did a pagefaul_disable when trying a different prefault scheme.
    
    Signed-off-by: Christoph Hellwig <hch@lst.de>

diff --git a/fs/iomap.c b/fs/iomap.c
index f39c318..74712e2 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -194,9 +194,7 @@ again:
 		if (mapping_writably_mapped(inode->i_mapping))
 			flush_dcache_page(page);
 
-		pagefault_disable();
 		copied = iov_iter_copy_from_user_atomic(page, i, offset, bytes);
-		pagefault_enable();
 
 		flush_dcache_page(page);
 

=========================================================================================
compiler/cpufreq_governor/debug-setup/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase:
  gcc-6/performance/profile/1BRD_48G/xfs/x86_64-rhel/3000/debian-x86_64-2015-02-07.cgz/ivb44/disk_wrt/aim7

commit: 
  f0c6bcba74ac51cb77aadb33ad35cb2dc1ad1506
  68a9f5e7007c1afa2cf6830b690a90d0187c0684
  e129b86bfacc8bb517b843fca41d0d179de7a4ca

f0c6bcba74ac51cb 68a9f5e7007c1afa2cf6830b69 e129b86bfacc8bb517b843fca4 
---------------- -------------------------- -------------------------- 
         %stddev     %change         %stddev     %change         %stddev
             \          |                \          |                \  
    484435 ±  0%     -13.3%     420004 ±  0%     -11.6%     428323 ±  0%  aim7.jobs-per-min
     37.37 ±  0%     +15.3%      43.09 ±  0%     +13.0%      42.24 ±  0%  aim7.time.elapsed_time
     37.37 ±  0%     +15.3%      43.09 ±  0%     +13.0%      42.24 ±  0%  aim7.time.elapsed_time.max
      6491 ±  3%     +30.8%       8491 ±  0%     +19.9%       7781 ±  4%  aim7.time.involuntary_context_switches
      2378 ±  0%      +1.1%       2404 ±  0%      +9.2%       2598 ±  0%  aim7.time.maximum_resident_set_size
    376.89 ±  0%     +28.4%     484.11 ±  0%     +22.8%     462.92 ±  0%  aim7.time.system_time
    430512 ±  0%     -20.1%     343838 ±  0%     -17.3%     356032 ±  0%  aim7.time.voluntary_context_switches
     26816 ±  8%     +10.2%      29542 ±  1%     +13.5%      30428 ±  0%  interrupts.CAL:Function_call_interrupts
      1016 ±  8%    +527.7%       6381 ± 59%    +483.5%       5932 ± 82%  latency_stats.sum.down.xfs_buf_lock._xfs_buf_find.xfs_buf_get_map.xfs_trans_get_buf_map.xfs_da_get_buf.xfs_dir3_data_init.xfs_dir2_sf_to_block.xfs_dir2_sf_addname.xfs_dir_createname.xfs_create.xfs_generic_create
    125122 ± 10%     -10.7%     111758 ± 12%      +8.2%     135440 ±  3%  softirqs.SCHED
    410707 ± 12%      +5.2%     432155 ± 11%     +24.9%     512838 ±  4%  softirqs.TIMER
     24772 ±  0%     -28.6%      17675 ±  0%     -24.1%      18813 ±  1%  vmstat.system.cs
     53477 ±  2%      +5.6%      56453 ±  0%      +6.1%      56716 ±  0%  vmstat.system.in
     43469 ±  0%      +5.3%      45792 ±  1%     +25.0%      54343 ± 16%  proc-vmstat.nr_active_anon
      3906 ±  0%     +28.8%       5032 ±  2%     -48.8%       2000 ± 96%  proc-vmstat.nr_active_file
    919.33 ±  5%     +14.8%       1055 ±  8%     +17.8%       1083 ± 10%  proc-vmstat.nr_dirty
      2270 ±  0%      +9.6%       2488 ±  0%   +2255.3%      53482 ± 95%  proc-vmstat.nr_inactive_anon
      3444 ±  5%     +41.8%       4884 ±  0%   +1809.2%      65752 ± 92%  proc-vmstat.nr_shmem
      4092 ± 14%     +61.2%       6595 ±  1%     +64.7%       6741 ±  9%  proc-vmstat.pgactivate
      1975 ± 15%     +63.2%       3224 ± 17%     +39.3%       2752 ± 15%  slabinfo.scsi_data_buffer.active_objs
      1975 ± 15%     +63.2%       3224 ± 17%     +39.3%       2752 ± 15%  slabinfo.scsi_data_buffer.num_objs
    464.33 ± 15%     +63.3%     758.33 ± 17%     +39.3%     647.00 ± 15%  slabinfo.xfs_efd_item.active_objs
    464.33 ± 15%     +63.3%     758.33 ± 17%     +39.3%     647.00 ± 15%  slabinfo.xfs_efd_item.num_objs
      1859 ±  0%      +9.3%       2032 ±  6%     +12.4%       2089 ±  7%  slabinfo.xfs_ili.active_objs
      1859 ±  0%      +9.3%       2032 ±  6%     +12.4%       2089 ±  7%  slabinfo.xfs_ili.num_objs
     37.37 ±  0%     +15.3%      43.09 ±  0%     +13.0%      42.24 ±  0%  time.elapsed_time
     37.37 ±  0%     +15.3%      43.09 ±  0%     +13.0%      42.24 ±  0%  time.elapsed_time.max
      6491 ±  3%     +30.8%       8491 ±  0%     +19.9%       7781 ±  4%  time.involuntary_context_switches
      1037 ±  0%     +10.8%       1148 ±  0%      +8.2%       1122 ±  0%  time.percent_of_cpu_this_job_got
    376.89 ±  0%     +28.4%     484.11 ±  0%     +22.8%     462.92 ±  0%  time.system_time
    430512 ±  0%     -20.1%     343838 ±  0%     -17.3%     356032 ±  0%  time.voluntary_context_switches
  52991525 ±  1%     -19.4%   42687208 ±  0%     -15.8%   44610884 ±  0%  cpuidle.C1-IVT.time
    319584 ±  1%     -26.5%     234868 ±  1%     -20.1%     255455 ±  1%  cpuidle.C1-IVT.usage
   3468808 ±  2%     -19.8%    2783341 ±  3%     -17.8%    2851560 ±  1%  cpuidle.C1E-IVT.time
     46760 ±  0%     -22.4%      36298 ±  0%     -19.3%      37738 ±  0%  cpuidle.C1E-IVT.usage
  12590471 ±  0%     -22.3%    9788585 ±  1%     -19.2%   10169486 ±  0%  cpuidle.C3-IVT.time
     79965 ±  0%     -19.0%      64749 ±  0%     -17.0%      66337 ±  0%  cpuidle.C3-IVT.usage
   1.3e+09 ±  0%     +13.3%  1.473e+09 ±  0%     +11.5%  1.449e+09 ±  0%  cpuidle.C6-IVT.time
   1645891 ±  1%      +6.2%    1747525 ±  0%     +10.3%    1814699 ±  1%  cpuidle.C6-IVT.usage
    352.33 ±  8%     -24.7%     265.33 ±  1%     -11.3%     312.50 ±  4%  cpuidle.POLL.usage
    189508 ±  0%      +6.3%     201410 ±  0%     +19.0%     225505 ± 12%  meminfo.Active
    173880 ±  0%      +4.4%     181454 ±  1%     +25.1%     217503 ± 16%  meminfo.Active(anon)
     15627 ±  0%     +27.7%      19956 ±  1%     -48.8%       8001 ± 96%  meminfo.Active(file)
     16103 ±  3%     +14.3%      18405 ±  8%     +15.3%      18575 ±  1%  meminfo.AnonHugePages
   2260771 ±  0%      -0.7%    2244069 ±  1%     +10.6%    2501050 ±  9%  meminfo.Committed_AS
   4330854 ± 11%      -8.5%    3960847 ±  0%     +16.2%    5034030 ±  0%  meminfo.DirectMap2M
    132898 ±  9%     +15.4%     153380 ±  1%      -3.1%     128773 ±  4%  meminfo.DirectMap4k
      9085 ±  0%      +9.4%       9940 ±  0%   +2254.7%     213930 ± 95%  meminfo.Inactive(anon)
     13777 ±  5%     +43.1%      19709 ±  0%   +1809.0%     263006 ± 92%  meminfo.Shmem
     24.18 ±  0%      +9.0%      26.35 ±  0%      +7.1%      25.91 ±  0%  turbostat.%Busy
    686.00 ±  0%      +9.5%     751.00 ±  0%      +7.5%     737.50 ±  0%  turbostat.Avg_MHz
      0.28 ±  0%     -25.0%       0.21 ±  0%     -17.9%       0.23 ±  0%  turbostat.CPU%c3
     93.33 ±  1%      +3.0%      96.15 ±  0%      +0.1%      93.44 ±  0%  turbostat.CorWatt
     79.00 ±  1%      -0.4%      78.67 ±  3%     -25.9%      58.50 ±  2%  turbostat.CoreTmp
      3.05 ± 25%     +24.9%       3.81 ± 44%     -54.3%       1.40 ±  3%  turbostat.Pkg%pc6
     78.67 ±  0%      +0.4%      79.00 ±  3%     -25.6%      58.50 ±  2%  turbostat.PkgTmp
    124.61 ±  0%      +2.1%     127.17 ±  0%      -1.0%     123.33 ±  0%  turbostat.PkgWatt
      4.74 ±  0%      -2.7%       4.61 ±  1%     -11.1%       4.21 ±  0%  turbostat.RAMWatt
   1724300 ± 27%     -40.5%    1025538 ±  1%     -39.2%    1048552 ±  0%  sched_debug.cfs_rq:/.load.max
    618.30 ±  4%      +0.2%     619.34 ±  2%     +12.0%     692.21 ±  3%  sched_debug.cfs_rq:/.min_vruntime.avg
     96.36 ±  3%     +18.6%     114.32 ± 15%     +19.1%     114.75 ± 17%  sched_debug.cfs_rq:/.util_avg.stddev
     15.54 ±  3%      +1.4%      15.76 ± 22%     -14.1%      13.35 ±  1%  sched_debug.cpu.cpu_load[4].avg
   1724300 ± 27%     -40.5%    1025538 ±  1%     -39.2%    1048552 ±  0%  sched_debug.cpu.load.max
      4751 ± 21%     -14.6%       4056 ± 25%     +25.1%       5944 ±  7%  sched_debug.cpu.nr_load_updates.avg
      7914 ±  1%     -14.1%       6797 ± 15%     +29.9%      10280 ± 18%  sched_debug.cpu.nr_load_updates.max
      2887 ± 30%     -28.2%       2073 ± 48%     +37.8%       3977 ±  9%  sched_debug.cpu.nr_load_updates.min
      1182 ±  2%      +5.2%       1244 ±  2%     +13.0%       1336 ± 11%  sched_debug.cpu.nr_load_updates.stddev
      1006 ±  3%      +3.7%       1044 ±  3%      +7.5%       1082 ±  5%  sched_debug.cpu.nr_switches.avg
      7.66 ± 20%     -24.9%       5.75 ± 15%     -20.7%       6.07 ±  4%  sched_debug.cpu.nr_uninterruptible.stddev
      7723 ±  0%     +32.6%      10238 ±  5%     -48.6%       3968 ± 92%  numa-meminfo.node0.Active(file)
      1589 ± 17%     +45.5%       2313 ± 24%     +17.4%       1866 ±  2%  numa-meminfo.node0.Dirty
     56052 ±  3%     +58.2%      88666 ± 17%     +99.1%     111572 ± 36%  numa-meminfo.node1.Active
     48142 ±  4%     +64.0%      78943 ± 19%    +123.4%     107536 ± 41%  numa-meminfo.node1.Active(anon)
      7908 ±  1%     +22.9%       9722 ±  3%     -49.0%       4035 ± 99%  numa-meminfo.node1.Active(file)
     46721 ±  3%     +55.9%      72837 ± 24%     +76.9%      82652 ± 34%  numa-meminfo.node1.AnonPages
      3283 ±122%      +4.7%       3436 ± 99%   +3034.3%     102920 ± 98%  numa-meminfo.node1.Inactive(anon)
      6005 ±  4%      -0.5%       5974 ±  1%    +340.4%      26444 ± 77%  numa-meminfo.node1.KernelStack
    545018 ±  2%      +9.2%     594908 ±  4%     +33.1%     725280 ± 19%  numa-meminfo.node1.MemUsed
     10518 ± 11%     +72.6%      18157 ± 33%    +330.3%      45256 ± 74%  numa-meminfo.node1.PageTables
     51114 ±  1%      +2.8%      52548 ±  8%     +78.0%      90996 ± 39%  numa-meminfo.node1.SUnreclaim
      4789 ± 69%    +102.3%       9687 ±  9%   +2571.5%     127936 ± 91%  numa-meminfo.node1.Shmem
     83631 ±  2%      -1.7%      82192 ±  9%     +47.0%     122949 ± 22%  numa-meminfo.node1.Slab
      1930 ±  0%     +33.9%       2585 ±  3%     -48.6%     992.00 ± 92%  numa-vmstat.node0.nr_active_file
      4468 ±  7%      -8.5%       4089 ±  5%      +9.7%       4902 ±  5%  numa-vmstat.node0.nr_alloc_batch
    466.67 ±  4%     +29.3%     603.33 ± 14%      +4.0%     485.50 ± 22%  numa-vmstat.node0.nr_dirty
     12026 ±  4%     +64.1%      19734 ± 20%    +123.3%      26852 ± 41%  numa-vmstat.node1.nr_active_anon
      1977 ±  1%     +23.6%       2444 ±  1%     -49.0%       1008 ± 99%  numa-vmstat.node1.nr_active_file
      3809 ±  6%     +16.1%       4422 ±  4%     +17.1%       4459 ± 17%  numa-vmstat.node1.nr_alloc_batch
     11671 ±  3%     +55.9%      18197 ± 24%     +76.8%      20633 ± 34%  numa-vmstat.node1.nr_anon_pages
  13239858 ±  0%      +2.7%   13602721 ±  4%      +9.4%   14489999 ±  2%  numa-vmstat.node1.nr_dirtied
    480.67 ±  4%      -5.2%     455.67 ± 24%      +7.8%     518.00 ±  6%  numa-vmstat.node1.nr_dirty
    820.33 ±122%      +4.7%     858.67 ± 99%   +3036.2%      25727 ± 98%  numa-vmstat.node1.nr_inactive_anon
    373.67 ±  4%      -0.5%     371.67 ±  1%    +340.5%       1646 ± 76%  numa-vmstat.node1.nr_kernel_stack
      2626 ± 11%     +72.6%       4533 ± 33%    +329.6%      11283 ± 74%  numa-vmstat.node1.nr_page_table_pages
      1197 ± 69%    +102.3%       2422 ±  9%   +2572.1%      31984 ± 91%  numa-vmstat.node1.nr_shmem
     12777 ±  1%      +2.8%      13134 ±  8%     +77.9%      22731 ± 39%  numa-vmstat.node1.nr_slab_unreclaimable
    456.33 ± 57%     -75.6%     111.33 ± 86%     -71.2%     131.50 ± 96%  numa-vmstat.node1.nr_written
  13421081 ±  0%      +2.9%   13803847 ±  4%      +9.8%   14735371 ±  2%  numa-vmstat.node1.numa_hit
  13421080 ±  0%      +2.9%   13803845 ±  4%      +9.8%   14735369 ±  2%  numa-vmstat.node1.numa_local
 2.658e+11 ±  4%     +24.7%  3.316e+11 ±  2%     +20.5%  3.204e+11 ±  0%  perf-stat.branch-instructions
      0.41 ±  1%      -9.1%       0.37 ±  1%     -22.9%       0.32 ±  0%  perf-stat.branch-miss-rate
  1.09e+09 ±  3%     +13.4%  1.237e+09 ±  1%      -7.0%  1.014e+09 ±  0%  perf-stat.branch-misses
    981138 ±  0%     -18.1%     803696 ±  0%     -14.7%     837107 ±  0%  perf-stat.context-switches
 1.511e+12 ±  5%     +23.4%  1.864e+12 ±  3%     +16.1%  1.754e+12 ±  0%  perf-stat.cpu-cycles
    102600 ±  1%      -7.3%      95075 ±  1%      -4.7%      97803 ±  0%  perf-stat.cpu-migrations
      0.26 ± 12%     -30.8%       0.18 ± 10%     +48.9%       0.39 ± 36%  perf-stat.dTLB-load-miss-rate
 8.332e+08 ± 13%      -3.8%  8.015e+08 ±  6%    +105.9%  1.716e+09 ± 34%  perf-stat.dTLB-load-misses
 3.164e+11 ±  1%     +39.9%  4.426e+11 ±  4%     +39.6%  4.417e+11 ±  2%  perf-stat.dTLB-loads
      0.03 ± 26%     -41.3%       0.02 ± 13%     +35.0%       0.04 ± 15%  perf-stat.dTLB-store-miss-rate
  60071678 ± 27%     -25.6%   44690199 ± 15%     +69.8%   1.02e+08 ± 17%  perf-stat.dTLB-store-misses
 2.247e+11 ±  6%     +26.4%  2.839e+11 ±  2%     +25.1%  2.812e+11 ±  2%  perf-stat.dTLB-stores
  1.49e+12 ±  4%     +30.1%  1.939e+12 ±  2%     +25.5%   1.87e+12 ±  0%  perf-stat.instructions
     43348 ±  2%     +34.2%      58161 ± 12%     +33.0%      57666 ±  8%  perf-stat.instructions-per-iTLB-miss
      0.99 ±  0%      +5.5%       1.04 ±  0%      +8.1%       1.07 ±  0%  perf-stat.ipc
    262799 ±  0%      +4.4%     274251 ±  1%      +4.6%     274993 ±  0%  perf-stat.minor-faults
     34.12 ±  1%      +2.1%      34.83 ±  0%      +5.5%      35.99 ±  0%  perf-stat.node-load-miss-rate
  46476754 ±  2%      +4.6%   48601269 ±  1%      +5.5%   49038648 ±  0%  perf-stat.node-load-misses
  89728871 ±  1%      +1.3%   90913257 ±  1%      -2.8%   87233401 ±  0%  perf-stat.node-loads
      9.96 ±  0%     +13.4%      11.30 ±  0%     +18.3%      11.79 ±  3%  perf-stat.node-store-miss-rate
  24460859 ±  1%     +14.4%   27971097 ±  1%      +5.3%   25747546 ±  3%  perf-stat.node-store-misses
 2.211e+08 ±  1%      -0.6%  2.197e+08 ±  1%     -12.6%  1.931e+08 ±  6%  perf-stat.node-stores
    262780 ±  0%      +4.4%     274227 ±  1%      +4.6%     274953 ±  0%  perf-stat.page-faults
     11.31 ±  1%     -18.1%       9.27 ±  0%     -17.3%       9.36 ±  0%  perf-profile.cycles-pp.____fput.task_work_run.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath
      0.00 ± -1%      +Inf%       1.68 ±  1%      +Inf%       1.74 ±  1%  perf-profile.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin
      1.80 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin
      2.55 ±  3%     -14.2%       2.19 ±  2%     -15.3%       2.16 ±  0%  perf-profile.cycles-pp.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin
      0.00 ± -1%      +Inf%       4.45 ±  1%      +Inf%       4.96 ±  1%  perf-profile.cycles-pp.__block_commit_write.isra.24.block_write_end.generic_write_end.iomap_write_actor.iomap_apply
      5.93 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__block_commit_write.isra.24.block_write_end.generic_write_end.xfs_vm_write_end.generic_perform_write
     13.71 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__block_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter
     10.36 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write
      0.00 ± -1%      +Inf%       3.64 ±  0%      +Inf%       3.79 ±  2%  perf-profile.cycles-pp.__block_write_begin_int.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      1.04 ±  2%     -18.9%       0.84 ±  1%     -15.4%       0.88 ±  0%  perf-profile.cycles-pp.__delete_from_page_cache.delete_from_page_cache.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final
     11.24 ±  2%     -18.1%       9.21 ±  0%     -17.3%       9.30 ±  0%  perf-profile.cycles-pp.__dentry_kill.dput.__fput.____fput.task_work_run
     11.31 ±  2%     -18.1%       9.26 ±  0%     -17.3%       9.36 ±  0%  perf-profile.cycles-pp.__fput.____fput.task_work_run.exit_to_usermode_loop.syscall_return_slowpath
      1.72 ±  2%     -10.1%       1.54 ±  1%     -17.6%       1.42 ±  0%  perf-profile.cycles-pp.__lru_cache_add.lru_cache_add.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin
      0.00 ± -1%      +Inf%       1.09 ±  2%      +Inf%       1.12 ±  1%  perf-profile.cycles-pp.__mark_inode_dirty.generic_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      1.32 ±  4%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__mark_inode_dirty.generic_write_end.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write
      0.00 ± -1%      +Inf%       2.68 ±  2%      +Inf%       2.65 ±  0%  perf-profile.cycles-pp.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor
      3.04 ±  3%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write
      2.50 ±  3%     -11.5%       2.21 ±  0%     -18.8%       2.03 ±  0%  perf-profile.cycles-pp.__pagevec_release.truncate_inode_pages_range.truncate_inode_pages_final.evict.iput
      1.00 ±  1%     -18.0%       0.82 ±  1%     -10.0%       0.90 ±  0%  perf-profile.cycles-pp.__radix_tree_lookup.radix_tree_lookup_slot.find_get_entry.pagecache_get_page.grab_cache_page_write_begin
      1.12 ±  2%     -17.6%       0.92 ±  4%     -13.8%       0.96 ±  0%  perf-profile.cycles-pp.__sb_start_write.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      1.38 ±  2%     -13.3%       1.19 ±  1%     -12.5%       1.21 ±  0%  perf-profile.cycles-pp.__set_page_dirty.mark_buffer_dirty.__block_commit_write.isra.24.block_write_end.generic_write_end
     54.10 ±  1%     +13.1%      61.20 ±  0%     +10.6%      59.86 ±  0%  perf-profile.cycles-pp.__vfs_write.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      6.34 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin
      0.00 ± -1%      +Inf%       3.69 ±  1%      +Inf%       3.62 ±  0%  perf-profile.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor
      4.02 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write
      0.98 ±  5%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.alloc_page_buffers.create_empty_buffers.create_page_buffers.__block_write_begin_int.__block_write_begin
      0.00 ± -1%      +Inf%       2.56 ±  2%      +Inf%       2.50 ±  0%  perf-profile.cycles-pp.alloc_pages_current.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin
      2.91 ±  3%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.alloc_pages_current.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin
      3.42 ±  0%     -20.9%       2.71 ±  2%     -15.7%       2.88 ±  0%  perf-profile.cycles-pp.block_invalidatepage.xfs_vm_invalidatepage.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final
      0.00 ± -1%      +Inf%       4.69 ±  0%      +Inf%       5.54 ±  1%  perf-profile.cycles-pp.block_write_end.generic_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      6.24 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.block_write_end.generic_write_end.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write
     19.18 ±  5%      -9.3%      17.40 ±  0%      -5.8%      18.06 ±  1%  perf-profile.cycles-pp.call_cpuidle.cpu_startup_entry.start_secondary
      0.94 ±  4%     -19.8%       0.76 ±  0%     -15.2%       0.80 ±  1%  perf-profile.cycles-pp.cancel_dirty_page.try_to_free_buffers.xfs_vm_releasepage.try_to_release_page.block_invalidatepage
      3.95 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.copy_user_enhanced_fast_string.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
      0.00 ± -1%      +Inf%       3.22 ±  0%      +Inf%       3.29 ±  1%  perf-profile.cycles-pp.copy_user_enhanced_fast_string.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
     19.75 ±  5%      -9.8%      17.81 ±  0%      -6.3%      18.50 ±  1%  perf-profile.cycles-pp.cpu_startup_entry.start_secondary
     19.18 ±  5%      -9.3%      17.40 ±  0%      -5.8%      18.05 ±  1%  perf-profile.cycles-pp.cpuidle_enter.call_cpuidle.cpu_startup_entry.start_secondary
     18.45 ±  5%      -9.2%      16.75 ±  0%      -5.6%      17.42 ±  1%  perf-profile.cycles-pp.cpuidle_enter_state.cpuidle_enter.call_cpuidle.cpu_startup_entry.start_secondary
      1.44 ±  3%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.create_empty_buffers.create_page_buffers.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin
      0.00 ± -1%      +Inf%       1.18 ±  1%      +Inf%       1.25 ±  2%  perf-profile.cycles-pp.create_empty_buffers.create_page_buffers.__block_write_begin_int.iomap_write_begin.iomap_write_actor
      1.86 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.create_page_buffers.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin.generic_perform_write
      0.00 ± -1%      +Inf%       1.53 ±  1%      +Inf%       1.61 ±  3%  perf-profile.cycles-pp.create_page_buffers.__block_write_begin_int.iomap_write_begin.iomap_write_actor.iomap_apply
      1.74 ±  2%     -19.9%       1.40 ±  3%     -16.8%       1.45 ±  0%  perf-profile.cycles-pp.delete_from_page_cache.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final.evict
      1.27 ±  0%     -22.5%       0.99 ±  4%     -22.3%       0.99 ±  0%  perf-profile.cycles-pp.destroy_inode.evict.iput.__dentry_kill.dput
      2.61 ±  1%     -24.3%       1.98 ±  1%     -20.7%       2.07 ±  0%  perf-profile.cycles-pp.do_filp_open.do_sys_open.sys_creat.entry_SYSCALL_64_fastpath
      2.66 ±  1%     -24.3%       2.01 ±  1%     -20.5%       2.12 ±  0%  perf-profile.cycles-pp.do_sys_open.sys_creat.entry_SYSCALL_64_fastpath
      1.79 ±  2%     -28.2%       1.28 ±  3%     -23.3%       1.37 ±  2%  perf-profile.cycles-pp.do_unlinkat.sys_unlink.entry_SYSCALL_64_fastpath
      1.07 ±  3%     -23.3%       0.82 ±  3%     -19.4%       0.86 ±  0%  perf-profile.cycles-pp.down_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write
      1.01 ±  3%     -17.9%       0.83 ±  2%     -13.6%       0.87 ±  1%  perf-profile.cycles-pp.down_write.xfs_ilock.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
     11.26 ±  2%     -18.1%       9.23 ±  0%     -17.2%       9.32 ±  0%  perf-profile.cycles-pp.dput.__fput.____fput.task_work_run.exit_to_usermode_loop
     11.21 ±  2%     -18.1%       9.18 ±  0%     -17.4%       9.26 ±  0%  perf-profile.cycles-pp.evict.iput.__dentry_kill.dput.__fput
     11.34 ±  2%     -18.1%       9.29 ±  0%     -17.3%       9.38 ±  0%  perf-profile.cycles-pp.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath
      0.00 ± -1%      +Inf%       1.55 ±  3%      +Inf%       1.65 ±  0%  perf-profile.cycles-pp.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor
      1.83 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write
     43.95 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write
      0.00 ± -1%      +Inf%       7.91 ±  1%      +Inf%       9.04 ±  0%  perf-profile.cycles-pp.generic_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
     10.68 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.generic_write_end.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter
      1.91 ±  3%     -16.4%       1.59 ±  1%     -17.7%       1.57 ±  0%  perf-profile.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc.pagecache_get_page
      0.00 ± -1%      +Inf%       9.85 ±  0%      +Inf%       9.91 ±  0%  perf-profile.cycles-pp.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write
     10.96 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter
      0.00 ± -1%      +Inf%      52.29 ±  0%      +Inf%      50.82 ±  0%  perf-profile.cycles-pp.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
      0.00 ± -1%      +Inf%      52.94 ±  0%      +Inf%      51.44 ±  0%  perf-profile.cycles-pp.iomap_file_buffered_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write
      0.00 ± -1%      +Inf%      34.35 ±  0%      +Inf%      32.27 ±  0%  perf-profile.cycles-pp.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.xfs_file_write_iter
      0.00 ± -1%      +Inf%      16.48 ±  0%      +Inf%      16.75 ±  1%  perf-profile.cycles-pp.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
     11.22 ±  2%     -18.1%       9.19 ±  0%     -17.3%       9.27 ±  0%  perf-profile.cycles-pp.iput.__dentry_kill.dput.__fput.____fput
      0.00 ± -1%      +Inf%       1.55 ±  1%      +Inf%       1.42 ±  0%  perf-profile.cycles-pp.lru_cache_add.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin
      1.72 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.lru_cache_add.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin
      0.00 ± -1%      +Inf%       2.78 ±  0%      +Inf%       2.88 ±  1%  perf-profile.cycles-pp.mark_buffer_dirty.__block_commit_write.isra.24.block_write_end.generic_write_end.iomap_write_actor
      3.39 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.mark_buffer_dirty.__block_commit_write.isra.24.block_write_end.generic_write_end.xfs_vm_write_end
      0.00 ± -1%      +Inf%       3.44 ±  1%      +NaN%       0.00 ± -1%  perf-profile.cycles-pp.mark_page_accessed.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
      3.03 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.memset_erms.__block_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write
      0.00 ± -1%      +Inf%       2.43 ±  0%      +Inf%       2.48 ±  3%  perf-profile.cycles-pp.memset_erms.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      0.00 ± -1%      +Inf%       9.25 ±  0%      +Inf%       9.25 ±  0%  perf-profile.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply
     10.37 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write
      1.52 ±  2%      -9.2%       1.38 ±  1%     -17.0%       1.27 ±  0%  perf-profile.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.lru_cache_add.add_to_page_cache_lru.pagecache_get_page
      2.58 ±  1%     -24.1%       1.96 ±  0%     -20.6%       2.05 ±  0%  perf-profile.cycles-pp.path_openat.do_filp_open.do_sys_open.sys_creat.entry_SYSCALL_64_fastpath
      0.00 ± -1%      +Inf%       0.95 ±  0%      +Inf%       1.04 ±  0%  perf-profile.cycles-pp.radix_tree_lookup_slot.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin
      1.17 ±  3%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.radix_tree_lookup_slot.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin
      2.39 ±  3%     -11.2%       2.12 ±  0%     -18.3%       1.95 ±  1%  perf-profile.cycles-pp.release_pages.__pagevec_release.truncate_inode_pages_range.truncate_inode_pages_final.evict
      2.06 ±  3%     -22.5%       1.60 ±  2%     -10.9%       1.83 ±  0%  perf-profile.cycles-pp.rw_verify_area.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      1.79 ±  3%     -22.2%       1.39 ±  0%      -9.8%       1.62 ±  0%  perf-profile.cycles-pp.security_file_permission.rw_verify_area.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      1.32 ±  4%     -21.4%       1.04 ±  0%      -7.4%       1.23 ±  1%  perf-profile.cycles-pp.selinux_file_permission.security_file_permission.rw_verify_area.vfs_write.sys_write
     19.79 ±  5%      -9.9%      17.84 ±  0%      -6.4%      18.54 ±  1%  perf-profile.cycles-pp.start_secondary
      2.67 ±  1%     -24.2%       2.02 ±  1%     -20.4%       2.12 ±  1%  perf-profile.cycles-pp.sys_creat.entry_SYSCALL_64_fastpath
      1.79 ±  3%     -27.9%       1.29 ±  3%     -23.0%       1.38 ±  2%  perf-profile.cycles-pp.sys_unlink.entry_SYSCALL_64_fastpath
     60.98 ±  1%      +9.5%      66.76 ±  0%      +7.8%      65.74 ±  0%  perf-profile.cycles-pp.sys_write.entry_SYSCALL_64_fastpath
     11.34 ±  1%     -18.1%       9.29 ±  0%     -17.2%       9.39 ±  0%  perf-profile.cycles-pp.syscall_return_slowpath.entry_SYSCALL_64_fastpath
     11.32 ±  1%     -18.0%       9.28 ±  0%     -17.3%       9.37 ±  0%  perf-profile.cycles-pp.task_work_run.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath
      5.96 ±  1%     -20.0%       4.77 ±  0%     -15.8%       5.02 ±  0%  perf-profile.cycles-pp.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final.evict.iput
      9.89 ±  2%     -17.4%       8.17 ±  0%     -16.7%       8.25 ±  0%  perf-profile.cycles-pp.truncate_inode_pages_final.evict.iput.__dentry_kill.dput
      9.87 ±  2%     -17.5%       8.15 ±  0%     -16.8%       8.21 ±  0%  perf-profile.cycles-pp.truncate_inode_pages_range.truncate_inode_pages_final.evict.iput.__dentry_kill
      2.07 ±  1%     -20.4%       1.65 ±  2%     -14.9%       1.77 ±  1%  perf-profile.cycles-pp.try_to_free_buffers.xfs_vm_releasepage.try_to_release_page.block_invalidatepage.xfs_vm_invalidatepage
      2.40 ±  1%     -21.0%       1.89 ±  2%     -15.3%       2.03 ±  1%  perf-profile.cycles-pp.try_to_release_page.block_invalidatepage.xfs_vm_invalidatepage.truncate_inode_page.truncate_inode_pages_range
      0.00 ± -1%      +Inf%       1.36 ±  1%      +Inf%       1.56 ±  3%  perf-profile.cycles-pp.unlock_page.generic_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      1.72 ±  4%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.unlock_page.generic_write_end.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write
     59.63 ±  1%     +10.2%      65.72 ±  0%      +8.5%      64.68 ±  0%  perf-profile.cycles-pp.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      0.00 ± -1%      +Inf%       1.52 ±  2%      +NaN%       0.00 ± -1%  perf-profile.cycles-pp.workingset_activation.mark_page_accessed.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      0.00 ± -1%      +Inf%       1.73 ±  1%      +Inf%       1.75 ±  2%  perf-profile.cycles-pp.xfs_bmap_search_extents.xfs_bmapi_delay.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply
      0.00 ± -1%      +Inf%       1.97 ±  2%      +Inf%       2.04 ±  0%  perf-profile.cycles-pp.xfs_bmap_search_extents.xfs_bmapi_read.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write
      0.00 ± -1%      +Inf%       1.61 ±  2%      +Inf%       1.65 ±  1%  perf-profile.cycles-pp.xfs_bmap_search_extents.xfs_bmapi_read.xfs_iomap_eof_want_preallocate.constprop.8.xfs_iomap_write_delay.xfs_file_iomap_begin
      0.00 ± -1%      +Inf%       1.24 ±  2%      +Inf%       1.21 ±  3%  perf-profile.cycles-pp.xfs_bmap_search_multi_extents.xfs_bmap_search_extents.xfs_bmapi_delay.xfs_iomap_write_delay.xfs_file_iomap_begin
      0.00 ± -1%      +Inf%       1.46 ±  1%      +Inf%       1.47 ±  1%  perf-profile.cycles-pp.xfs_bmap_search_multi_extents.xfs_bmap_search_extents.xfs_bmapi_read.xfs_file_iomap_begin.iomap_apply
      0.00 ± -1%      +Inf%       1.21 ±  2%      +Inf%       1.25 ±  0%  perf-profile.cycles-pp.xfs_bmap_search_multi_extents.xfs_bmap_search_extents.xfs_bmapi_read.xfs_iomap_eof_want_preallocate.constprop.8.xfs_iomap_write_delay
      1.25 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_bmapi_delay.xfs_iomap_write_delay.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int
      0.00 ± -1%      +Inf%       3.06 ±  1%      +Inf%       3.08 ±  1%  perf-profile.cycles-pp.xfs_bmapi_delay.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write
      1.04 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_bmapi_read.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int.__block_write_begin
      0.00 ± -1%      +Inf%       3.04 ±  1%      +Inf%       3.16 ±  1%  perf-profile.cycles-pp.xfs_bmapi_read.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
      0.00 ± -1%      +Inf%       3.05 ±  1%      +Inf%       3.09 ±  1%  perf-profile.cycles-pp.xfs_bmapi_read.xfs_iomap_eof_want_preallocate.constprop.8.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply
      1.32 ±  2%     -21.5%       1.04 ±  1%     -19.7%       1.06 ±  0%  perf-profile.cycles-pp.xfs_create.xfs_generic_create.xfs_vn_mknod.xfs_vn_create.path_openat
     51.83 ±  1%     +14.3%      59.25 ±  0%     +11.8%      57.95 ±  0%  perf-profile.cycles-pp.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write.sys_write
      0.00 ± -1%      +Inf%      16.05 ±  0%      +Inf%      16.68 ±  0%  perf-profile.cycles-pp.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.xfs_file_write_iter
     53.16 ±  1%     +13.6%      60.40 ±  0%     +11.1%      59.09 ±  0%  perf-profile.cycles-pp.xfs_file_write_iter.__vfs_write.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      1.24 ±  1%     -23.1%       0.95 ±  4%     -21.8%       0.97 ±  0%  perf-profile.cycles-pp.xfs_fs_destroy_inode.destroy_inode.evict.iput.__dentry_kill
      1.42 ±  2%     -21.2%       1.12 ±  1%     -20.6%       1.12 ±  0%  perf-profile.cycles-pp.xfs_generic_create.xfs_vn_mknod.xfs_vn_create.path_openat.do_filp_open
      6.46 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_get_blocks.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin.generic_perform_write
      1.29 ±  3%     -18.9%       1.04 ±  1%     -14.1%       1.10 ±  0%  perf-profile.cycles-pp.xfs_ilock.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write
      0.00 ± -1%      +Inf%       1.14 ±  3%      +Inf%       1.17 ±  1%  perf-profile.cycles-pp.xfs_ilock.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
      1.21 ±  1%     -23.4%       0.93 ±  4%     -22.5%       0.94 ±  0%  perf-profile.cycles-pp.xfs_inactive.xfs_fs_destroy_inode.destroy_inode.evict.iput
      1.23 ±  4%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_iomap_eof_want_preallocate.constprop.6.xfs_iomap_write_delay.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int
      0.00 ± -1%      +Inf%       4.14 ±  0%      +Inf%       4.15 ±  1%  perf-profile.cycles-pp.xfs_iomap_eof_want_preallocate.constprop.8.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write
      3.28 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_iomap_write_delay.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int.__block_write_begin
      0.00 ± -1%      +Inf%       9.08 ±  0%      +Inf%       9.19 ±  1%  perf-profile.cycles-pp.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
      3.54 ±  0%     -20.8%       2.81 ±  1%     -15.6%       2.99 ±  0%  perf-profile.cycles-pp.xfs_vm_invalidatepage.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final.evict
      2.35 ±  1%     -21.0%       1.86 ±  1%     -15.1%       2.00 ±  1%  perf-profile.cycles-pp.xfs_vm_releasepage.try_to_release_page.block_invalidatepage.xfs_vm_invalidatepage.truncate_inode_page
     25.10 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
     11.03 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
      1.42 ±  2%     -20.7%       1.13 ±  1%     -20.4%       1.13 ±  0%  perf-profile.cycles-pp.xfs_vn_create.path_openat.do_filp_open.do_sys_open.sys_creat
      1.42 ±  2%     -20.5%       1.13 ±  1%     -20.2%       1.13 ±  0%  perf-profile.cycles-pp.xfs_vn_mknod.xfs_vn_create.path_openat.do_filp_open.do_sys_open
      2.27 ±  1%     -10.6%       2.03 ±  0%      -6.7%       2.12 ±  1%  perf-profile.func.cycles-pp.___might_sleep
      2.49 ±  0%     -34.5%       1.63 ±  1%     -16.7%       2.08 ±  0%  perf-profile.func.cycles-pp.__block_commit_write.isra.24
      1.51 ±  2%     +15.4%       1.75 ±  1%     +18.0%       1.79 ±  2%  perf-profile.func.cycles-pp.__block_write_begin_int
      1.79 ±  4%     -16.8%       1.49 ±  1%     -14.5%       1.53 ±  0%  perf-profile.func.cycles-pp.__mark_inode_dirty
      1.32 ±  0%     -16.4%       1.10 ±  1%      -9.5%       1.19 ±  0%  perf-profile.func.cycles-pp.__radix_tree_lookup
      1.08 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.func.cycles-pp.__xfs_get_blocks
      1.16 ±  0%     -18.1%       0.95 ±  1%     -15.8%       0.98 ±  1%  perf-profile.func.cycles-pp._raw_spin_lock
      3.96 ±  2%     -18.4%       3.23 ±  0%     -16.9%       3.29 ±  1%  perf-profile.func.cycles-pp.copy_user_enhanced_fast_string
      1.41 ±  3%     -20.6%       1.12 ±  3%     -21.1%       1.11 ±  3%  perf-profile.func.cycles-pp.entry_SYSCALL_64_fastpath
      1.30 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.func.cycles-pp.generic_perform_write
      1.31 ±  2%     -46.7%       0.70 ±  0%     -43.8%       0.73 ±  0%  perf-profile.func.cycles-pp.generic_write_end
     18.43 ±  5%      -9.1%      16.76 ±  0%      -5.4%      17.44 ±  1%  perf-profile.func.cycles-pp.intel_idle
      0.00 ± -1%      +Inf%       1.12 ±  1%      +Inf%       0.90 ±  0%  perf-profile.func.cycles-pp.iomap_write_actor
      1.50 ±  1%     -20.9%       1.19 ±  1%     -17.0%       1.25 ±  2%  perf-profile.func.cycles-pp.mark_buffer_dirty
      0.00 ± -1%      +Inf%       1.91 ±  1%      +NaN%       0.00 ± -1%  perf-profile.func.cycles-pp.mark_page_accessed
      3.24 ±  0%     -19.8%       2.60 ±  0%     -18.1%       2.66 ±  3%  perf-profile.func.cycles-pp.memset_erms
      1.75 ±  2%     -18.9%       1.42 ±  1%      -7.3%       1.62 ±  4%  perf-profile.func.cycles-pp.unlock_page
      1.56 ±  2%      +6.0%       1.65 ±  3%     +11.8%       1.74 ±  1%  perf-profile.func.cycles-pp.up_write
      1.16 ±  1%     -21.6%       0.91 ±  1%     -17.7%       0.95 ±  1%  perf-profile.func.cycles-pp.vfs_write
      0.37 ±  2%    +243.6%       1.26 ±  2%    +272.3%       1.36 ±  2%  perf-profile.func.cycles-pp.xfs_bmap_search_extents
      0.41 ±  1%    +198.4%       1.22 ±  2%    +198.8%       1.23 ±  3%  perf-profile.func.cycles-pp.xfs_bmap_search_multi_extents
      0.70 ±  5%    +219.5%       2.24 ±  0%    +227.9%       2.29 ±  0%  perf-profile.func.cycles-pp.xfs_bmapi_read
      1.05 ±  2%     -15.6%       0.88 ±  3%     -18.8%       0.85 ±  1%  perf-profile.func.cycles-pp.xfs_file_write_iter
      0.64 ±  1%    +182.8%       1.81 ±  4%    +182.0%       1.81 ±  0%  perf-profile.func.cycles-pp.xfs_iext_bno_to_ext
      0.00 ± -1%      +Inf%       1.10 ±  3%      +Inf%       1.21 ±  2%  perf-profile.func.cycles-pp.xfs_iomap_eof_want_preallocate.constprop.8
      0.46 ±  4%    +161.6%       1.20 ±  1%    +171.7%       1.25 ±  1%  perf-profile.func.cycles-pp.xfs_iomap_write_delay


raw perf data:

  "perf-profile.func.cycles-pp.intel_idle": 17.66,
  "perf-profile.func.cycles-pp.copy_user_enhanced_fast_string": 3.25,
  "perf-profile.func.cycles-pp.memset_erms": 2.56,
  "perf-profile.func.cycles-pp.xfs_bmapi_read": 2.28,
  "perf-profile.func.cycles-pp.___might_sleep": 2.09,
  "perf-profile.func.cycles-pp.__block_commit_write.isra.24": 2.07,
  "perf-profile.func.cycles-pp.xfs_iext_bno_to_ext": 1.79,
  "perf-profile.func.cycles-pp.__block_write_begin_int": 1.74,
  "perf-profile.func.cycles-pp.up_write": 1.72,
  "perf-profile.func.cycles-pp.unlock_page": 1.69,
  "perf-profile.func.cycles-pp.down_write": 1.59,
  "perf-profile.func.cycles-pp.__mark_inode_dirty": 1.54,
  "perf-profile.func.cycles-pp.xfs_bmap_search_extents": 1.33,
  "perf-profile.func.cycles-pp.xfs_iomap_write_delay": 1.23,
  "perf-profile.func.cycles-pp.mark_buffer_dirty": 1.21,
  "perf-profile.func.cycles-pp.__radix_tree_lookup": 1.2,
  "perf-profile.func.cycles-pp.xfs_bmap_search_multi_extents": 1.18,
  "perf-profile.func.cycles-pp.xfs_iomap_eof_want_preallocate.constprop.8": 1.17,
  "perf-profile.func.cycles-pp.entry_SYSCALL_64_fastpath": 1.15,
  "perf-profile.func.cycles-pp.__might_sleep": 1.14,
  "perf-profile.func.cycles-pp._raw_spin_lock": 0.97,
  "perf-profile.func.cycles-pp.vfs_write": 0.94,
  "perf-profile.func.cycles-pp.xfs_bmapi_delay": 0.93,
  "perf-profile.func.cycles-pp.iomap_write_actor": 0.9,
  "perf-profile.func.cycles-pp.pagecache_get_page": 0.89,
  "perf-profile.func.cycles-pp.xfs_file_write_iter": 0.86,
  "perf-profile.func.cycles-pp.xfs_file_iomap_begin": 0.81,
  "perf-profile.func.cycles-pp.iov_iter_copy_from_user_atomic": 0.78,
  "perf-profile.func.cycles-pp.iomap_apply": 0.77,
  "perf-profile.func.cycles-pp.generic_write_end": 0.74,
  "perf-profile.func.cycles-pp.xfs_file_buffered_aio_write": 0.72,
  "perf-profile.func.cycles-pp.find_get_entry": 0.69,
  "perf-profile.func.cycles-pp.__vfs_write": 0.67,


Best Regards,
Huang, Ying

WARNING: multiple messages have this Message-ID (diff)
From: Huang, Ying <ying.huang@intel.com>
To: lkp@lists.01.org
Subject: Re: [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression
Date: Thu, 11 Aug 2016 14:16:03 -0700	[thread overview]
Message-ID: <87ziojxazw.fsf@yhuang-mobile.sh.intel.com> (raw)
In-Reply-To: <20160811200018.GA28271@lst.de>

[-- Attachment #1: Type: text/plain, Size: 45778 bytes --]

Christoph Hellwig <hch@lst.de> writes:

> On Thu, Aug 11, 2016 at 12:51:31PM -0700, Linus Torvalds wrote:
>> Ok. It does seem to also reset the active file page counts back, so
>> that part did seem to be related, but yeah, from a performance
>> standpoint that was clearly not a major issue.
>> 
>> Let's hope Dave can figure out something based on his numbers, because
>> I'm out of ideas. Or maybe it's the pagefault-atomic thing that
>> Christoph was looking at.
>
> I can't really think of any reason why the pagefaul_disable() would
> sіgnificantly change performance.  Anyway, the patch for the is below
> (on top of the previous mark_page_accessed() one), so feel free to
> re-run the test with it.  It would also be nice to see the profiles
> with the two patches applied.
>
> commit 43106eea246074acc4bb7d12fdb91f58002f52ed
> Author: Christoph Hellwig <hch@lst.de>
> Date:   Thu Aug 11 10:41:40 2016 -0700
>
>     fs: remove superflous pagefault_disable from iomap_write_actor
>     
>     No idea where this really came from, generic_perform_write only briefly
>     did a pagefaul_disable when trying a different prefault scheme.
>     
>     Signed-off-by: Christoph Hellwig <hch@lst.de>
>
> diff --git a/fs/iomap.c b/fs/iomap.c
> index f39c318..74712e2 100644
> --- a/fs/iomap.c
> +++ b/fs/iomap.c
> @@ -194,9 +194,7 @@ again:
>  		if (mapping_writably_mapped(inode->i_mapping))
>  			flush_dcache_page(page);
>  
> -		pagefault_disable();
>  		copied = iov_iter_copy_from_user_atomic(page, i, offset, bytes);
> -		pagefault_enable();
>  
>  		flush_dcache_page(page);
>  

Test result is as follow,

commit e129b86bfacc8bb517b843fca41d0d179de7a4ca
Author: Christoph Hellwig <hch@lst.de>
Date:   Thu Aug 11 10:41:40 2016 -0700

    fs: remove superflous pagefault_disable from iomap_write_actor
    
    No idea where this really came from, generic_perform_write only briefly
    did a pagefaul_disable when trying a different prefault scheme.
    
    Signed-off-by: Christoph Hellwig <hch@lst.de>

diff --git a/fs/iomap.c b/fs/iomap.c
index f39c318..74712e2 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -194,9 +194,7 @@ again:
 		if (mapping_writably_mapped(inode->i_mapping))
 			flush_dcache_page(page);
 
-		pagefault_disable();
 		copied = iov_iter_copy_from_user_atomic(page, i, offset, bytes);
-		pagefault_enable();
 
 		flush_dcache_page(page);
 

=========================================================================================
compiler/cpufreq_governor/debug-setup/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase:
  gcc-6/performance/profile/1BRD_48G/xfs/x86_64-rhel/3000/debian-x86_64-2015-02-07.cgz/ivb44/disk_wrt/aim7

commit: 
  f0c6bcba74ac51cb77aadb33ad35cb2dc1ad1506
  68a9f5e7007c1afa2cf6830b690a90d0187c0684
  e129b86bfacc8bb517b843fca41d0d179de7a4ca

f0c6bcba74ac51cb 68a9f5e7007c1afa2cf6830b69 e129b86bfacc8bb517b843fca4 
---------------- -------------------------- -------------------------- 
         %stddev     %change         %stddev     %change         %stddev
             \          |                \          |                \  
    484435 ±  0%     -13.3%     420004 ±  0%     -11.6%     428323 ±  0%  aim7.jobs-per-min
     37.37 ±  0%     +15.3%      43.09 ±  0%     +13.0%      42.24 ±  0%  aim7.time.elapsed_time
     37.37 ±  0%     +15.3%      43.09 ±  0%     +13.0%      42.24 ±  0%  aim7.time.elapsed_time.max
      6491 ±  3%     +30.8%       8491 ±  0%     +19.9%       7781 ±  4%  aim7.time.involuntary_context_switches
      2378 ±  0%      +1.1%       2404 ±  0%      +9.2%       2598 ±  0%  aim7.time.maximum_resident_set_size
    376.89 ±  0%     +28.4%     484.11 ±  0%     +22.8%     462.92 ±  0%  aim7.time.system_time
    430512 ±  0%     -20.1%     343838 ±  0%     -17.3%     356032 ±  0%  aim7.time.voluntary_context_switches
     26816 ±  8%     +10.2%      29542 ±  1%     +13.5%      30428 ±  0%  interrupts.CAL:Function_call_interrupts
      1016 ±  8%    +527.7%       6381 ± 59%    +483.5%       5932 ± 82%  latency_stats.sum.down.xfs_buf_lock._xfs_buf_find.xfs_buf_get_map.xfs_trans_get_buf_map.xfs_da_get_buf.xfs_dir3_data_init.xfs_dir2_sf_to_block.xfs_dir2_sf_addname.xfs_dir_createname.xfs_create.xfs_generic_create
    125122 ± 10%     -10.7%     111758 ± 12%      +8.2%     135440 ±  3%  softirqs.SCHED
    410707 ± 12%      +5.2%     432155 ± 11%     +24.9%     512838 ±  4%  softirqs.TIMER
     24772 ±  0%     -28.6%      17675 ±  0%     -24.1%      18813 ±  1%  vmstat.system.cs
     53477 ±  2%      +5.6%      56453 ±  0%      +6.1%      56716 ±  0%  vmstat.system.in
     43469 ±  0%      +5.3%      45792 ±  1%     +25.0%      54343 ± 16%  proc-vmstat.nr_active_anon
      3906 ±  0%     +28.8%       5032 ±  2%     -48.8%       2000 ± 96%  proc-vmstat.nr_active_file
    919.33 ±  5%     +14.8%       1055 ±  8%     +17.8%       1083 ± 10%  proc-vmstat.nr_dirty
      2270 ±  0%      +9.6%       2488 ±  0%   +2255.3%      53482 ± 95%  proc-vmstat.nr_inactive_anon
      3444 ±  5%     +41.8%       4884 ±  0%   +1809.2%      65752 ± 92%  proc-vmstat.nr_shmem
      4092 ± 14%     +61.2%       6595 ±  1%     +64.7%       6741 ±  9%  proc-vmstat.pgactivate
      1975 ± 15%     +63.2%       3224 ± 17%     +39.3%       2752 ± 15%  slabinfo.scsi_data_buffer.active_objs
      1975 ± 15%     +63.2%       3224 ± 17%     +39.3%       2752 ± 15%  slabinfo.scsi_data_buffer.num_objs
    464.33 ± 15%     +63.3%     758.33 ± 17%     +39.3%     647.00 ± 15%  slabinfo.xfs_efd_item.active_objs
    464.33 ± 15%     +63.3%     758.33 ± 17%     +39.3%     647.00 ± 15%  slabinfo.xfs_efd_item.num_objs
      1859 ±  0%      +9.3%       2032 ±  6%     +12.4%       2089 ±  7%  slabinfo.xfs_ili.active_objs
      1859 ±  0%      +9.3%       2032 ±  6%     +12.4%       2089 ±  7%  slabinfo.xfs_ili.num_objs
     37.37 ±  0%     +15.3%      43.09 ±  0%     +13.0%      42.24 ±  0%  time.elapsed_time
     37.37 ±  0%     +15.3%      43.09 ±  0%     +13.0%      42.24 ±  0%  time.elapsed_time.max
      6491 ±  3%     +30.8%       8491 ±  0%     +19.9%       7781 ±  4%  time.involuntary_context_switches
      1037 ±  0%     +10.8%       1148 ±  0%      +8.2%       1122 ±  0%  time.percent_of_cpu_this_job_got
    376.89 ±  0%     +28.4%     484.11 ±  0%     +22.8%     462.92 ±  0%  time.system_time
    430512 ±  0%     -20.1%     343838 ±  0%     -17.3%     356032 ±  0%  time.voluntary_context_switches
  52991525 ±  1%     -19.4%   42687208 ±  0%     -15.8%   44610884 ±  0%  cpuidle.C1-IVT.time
    319584 ±  1%     -26.5%     234868 ±  1%     -20.1%     255455 ±  1%  cpuidle.C1-IVT.usage
   3468808 ±  2%     -19.8%    2783341 ±  3%     -17.8%    2851560 ±  1%  cpuidle.C1E-IVT.time
     46760 ±  0%     -22.4%      36298 ±  0%     -19.3%      37738 ±  0%  cpuidle.C1E-IVT.usage
  12590471 ±  0%     -22.3%    9788585 ±  1%     -19.2%   10169486 ±  0%  cpuidle.C3-IVT.time
     79965 ±  0%     -19.0%      64749 ±  0%     -17.0%      66337 ±  0%  cpuidle.C3-IVT.usage
   1.3e+09 ±  0%     +13.3%  1.473e+09 ±  0%     +11.5%  1.449e+09 ±  0%  cpuidle.C6-IVT.time
   1645891 ±  1%      +6.2%    1747525 ±  0%     +10.3%    1814699 ±  1%  cpuidle.C6-IVT.usage
    352.33 ±  8%     -24.7%     265.33 ±  1%     -11.3%     312.50 ±  4%  cpuidle.POLL.usage
    189508 ±  0%      +6.3%     201410 ±  0%     +19.0%     225505 ± 12%  meminfo.Active
    173880 ±  0%      +4.4%     181454 ±  1%     +25.1%     217503 ± 16%  meminfo.Active(anon)
     15627 ±  0%     +27.7%      19956 ±  1%     -48.8%       8001 ± 96%  meminfo.Active(file)
     16103 ±  3%     +14.3%      18405 ±  8%     +15.3%      18575 ±  1%  meminfo.AnonHugePages
   2260771 ±  0%      -0.7%    2244069 ±  1%     +10.6%    2501050 ±  9%  meminfo.Committed_AS
   4330854 ± 11%      -8.5%    3960847 ±  0%     +16.2%    5034030 ±  0%  meminfo.DirectMap2M
    132898 ±  9%     +15.4%     153380 ±  1%      -3.1%     128773 ±  4%  meminfo.DirectMap4k
      9085 ±  0%      +9.4%       9940 ±  0%   +2254.7%     213930 ± 95%  meminfo.Inactive(anon)
     13777 ±  5%     +43.1%      19709 ±  0%   +1809.0%     263006 ± 92%  meminfo.Shmem
     24.18 ±  0%      +9.0%      26.35 ±  0%      +7.1%      25.91 ±  0%  turbostat.%Busy
    686.00 ±  0%      +9.5%     751.00 ±  0%      +7.5%     737.50 ±  0%  turbostat.Avg_MHz
      0.28 ±  0%     -25.0%       0.21 ±  0%     -17.9%       0.23 ±  0%  turbostat.CPU%c3
     93.33 ±  1%      +3.0%      96.15 ±  0%      +0.1%      93.44 ±  0%  turbostat.CorWatt
     79.00 ±  1%      -0.4%      78.67 ±  3%     -25.9%      58.50 ±  2%  turbostat.CoreTmp
      3.05 ± 25%     +24.9%       3.81 ± 44%     -54.3%       1.40 ±  3%  turbostat.Pkg%pc6
     78.67 ±  0%      +0.4%      79.00 ±  3%     -25.6%      58.50 ±  2%  turbostat.PkgTmp
    124.61 ±  0%      +2.1%     127.17 ±  0%      -1.0%     123.33 ±  0%  turbostat.PkgWatt
      4.74 ±  0%      -2.7%       4.61 ±  1%     -11.1%       4.21 ±  0%  turbostat.RAMWatt
   1724300 ± 27%     -40.5%    1025538 ±  1%     -39.2%    1048552 ±  0%  sched_debug.cfs_rq:/.load.max
    618.30 ±  4%      +0.2%     619.34 ±  2%     +12.0%     692.21 ±  3%  sched_debug.cfs_rq:/.min_vruntime.avg
     96.36 ±  3%     +18.6%     114.32 ± 15%     +19.1%     114.75 ± 17%  sched_debug.cfs_rq:/.util_avg.stddev
     15.54 ±  3%      +1.4%      15.76 ± 22%     -14.1%      13.35 ±  1%  sched_debug.cpu.cpu_load[4].avg
   1724300 ± 27%     -40.5%    1025538 ±  1%     -39.2%    1048552 ±  0%  sched_debug.cpu.load.max
      4751 ± 21%     -14.6%       4056 ± 25%     +25.1%       5944 ±  7%  sched_debug.cpu.nr_load_updates.avg
      7914 ±  1%     -14.1%       6797 ± 15%     +29.9%      10280 ± 18%  sched_debug.cpu.nr_load_updates.max
      2887 ± 30%     -28.2%       2073 ± 48%     +37.8%       3977 ±  9%  sched_debug.cpu.nr_load_updates.min
      1182 ±  2%      +5.2%       1244 ±  2%     +13.0%       1336 ± 11%  sched_debug.cpu.nr_load_updates.stddev
      1006 ±  3%      +3.7%       1044 ±  3%      +7.5%       1082 ±  5%  sched_debug.cpu.nr_switches.avg
      7.66 ± 20%     -24.9%       5.75 ± 15%     -20.7%       6.07 ±  4%  sched_debug.cpu.nr_uninterruptible.stddev
      7723 ±  0%     +32.6%      10238 ±  5%     -48.6%       3968 ± 92%  numa-meminfo.node0.Active(file)
      1589 ± 17%     +45.5%       2313 ± 24%     +17.4%       1866 ±  2%  numa-meminfo.node0.Dirty
     56052 ±  3%     +58.2%      88666 ± 17%     +99.1%     111572 ± 36%  numa-meminfo.node1.Active
     48142 ±  4%     +64.0%      78943 ± 19%    +123.4%     107536 ± 41%  numa-meminfo.node1.Active(anon)
      7908 ±  1%     +22.9%       9722 ±  3%     -49.0%       4035 ± 99%  numa-meminfo.node1.Active(file)
     46721 ±  3%     +55.9%      72837 ± 24%     +76.9%      82652 ± 34%  numa-meminfo.node1.AnonPages
      3283 ±122%      +4.7%       3436 ± 99%   +3034.3%     102920 ± 98%  numa-meminfo.node1.Inactive(anon)
      6005 ±  4%      -0.5%       5974 ±  1%    +340.4%      26444 ± 77%  numa-meminfo.node1.KernelStack
    545018 ±  2%      +9.2%     594908 ±  4%     +33.1%     725280 ± 19%  numa-meminfo.node1.MemUsed
     10518 ± 11%     +72.6%      18157 ± 33%    +330.3%      45256 ± 74%  numa-meminfo.node1.PageTables
     51114 ±  1%      +2.8%      52548 ±  8%     +78.0%      90996 ± 39%  numa-meminfo.node1.SUnreclaim
      4789 ± 69%    +102.3%       9687 ±  9%   +2571.5%     127936 ± 91%  numa-meminfo.node1.Shmem
     83631 ±  2%      -1.7%      82192 ±  9%     +47.0%     122949 ± 22%  numa-meminfo.node1.Slab
      1930 ±  0%     +33.9%       2585 ±  3%     -48.6%     992.00 ± 92%  numa-vmstat.node0.nr_active_file
      4468 ±  7%      -8.5%       4089 ±  5%      +9.7%       4902 ±  5%  numa-vmstat.node0.nr_alloc_batch
    466.67 ±  4%     +29.3%     603.33 ± 14%      +4.0%     485.50 ± 22%  numa-vmstat.node0.nr_dirty
     12026 ±  4%     +64.1%      19734 ± 20%    +123.3%      26852 ± 41%  numa-vmstat.node1.nr_active_anon
      1977 ±  1%     +23.6%       2444 ±  1%     -49.0%       1008 ± 99%  numa-vmstat.node1.nr_active_file
      3809 ±  6%     +16.1%       4422 ±  4%     +17.1%       4459 ± 17%  numa-vmstat.node1.nr_alloc_batch
     11671 ±  3%     +55.9%      18197 ± 24%     +76.8%      20633 ± 34%  numa-vmstat.node1.nr_anon_pages
  13239858 ±  0%      +2.7%   13602721 ±  4%      +9.4%   14489999 ±  2%  numa-vmstat.node1.nr_dirtied
    480.67 ±  4%      -5.2%     455.67 ± 24%      +7.8%     518.00 ±  6%  numa-vmstat.node1.nr_dirty
    820.33 ±122%      +4.7%     858.67 ± 99%   +3036.2%      25727 ± 98%  numa-vmstat.node1.nr_inactive_anon
    373.67 ±  4%      -0.5%     371.67 ±  1%    +340.5%       1646 ± 76%  numa-vmstat.node1.nr_kernel_stack
      2626 ± 11%     +72.6%       4533 ± 33%    +329.6%      11283 ± 74%  numa-vmstat.node1.nr_page_table_pages
      1197 ± 69%    +102.3%       2422 ±  9%   +2572.1%      31984 ± 91%  numa-vmstat.node1.nr_shmem
     12777 ±  1%      +2.8%      13134 ±  8%     +77.9%      22731 ± 39%  numa-vmstat.node1.nr_slab_unreclaimable
    456.33 ± 57%     -75.6%     111.33 ± 86%     -71.2%     131.50 ± 96%  numa-vmstat.node1.nr_written
  13421081 ±  0%      +2.9%   13803847 ±  4%      +9.8%   14735371 ±  2%  numa-vmstat.node1.numa_hit
  13421080 ±  0%      +2.9%   13803845 ±  4%      +9.8%   14735369 ±  2%  numa-vmstat.node1.numa_local
 2.658e+11 ±  4%     +24.7%  3.316e+11 ±  2%     +20.5%  3.204e+11 ±  0%  perf-stat.branch-instructions
      0.41 ±  1%      -9.1%       0.37 ±  1%     -22.9%       0.32 ±  0%  perf-stat.branch-miss-rate
  1.09e+09 ±  3%     +13.4%  1.237e+09 ±  1%      -7.0%  1.014e+09 ±  0%  perf-stat.branch-misses
    981138 ±  0%     -18.1%     803696 ±  0%     -14.7%     837107 ±  0%  perf-stat.context-switches
 1.511e+12 ±  5%     +23.4%  1.864e+12 ±  3%     +16.1%  1.754e+12 ±  0%  perf-stat.cpu-cycles
    102600 ±  1%      -7.3%      95075 ±  1%      -4.7%      97803 ±  0%  perf-stat.cpu-migrations
      0.26 ± 12%     -30.8%       0.18 ± 10%     +48.9%       0.39 ± 36%  perf-stat.dTLB-load-miss-rate
 8.332e+08 ± 13%      -3.8%  8.015e+08 ±  6%    +105.9%  1.716e+09 ± 34%  perf-stat.dTLB-load-misses
 3.164e+11 ±  1%     +39.9%  4.426e+11 ±  4%     +39.6%  4.417e+11 ±  2%  perf-stat.dTLB-loads
      0.03 ± 26%     -41.3%       0.02 ± 13%     +35.0%       0.04 ± 15%  perf-stat.dTLB-store-miss-rate
  60071678 ± 27%     -25.6%   44690199 ± 15%     +69.8%   1.02e+08 ± 17%  perf-stat.dTLB-store-misses
 2.247e+11 ±  6%     +26.4%  2.839e+11 ±  2%     +25.1%  2.812e+11 ±  2%  perf-stat.dTLB-stores
  1.49e+12 ±  4%     +30.1%  1.939e+12 ±  2%     +25.5%   1.87e+12 ±  0%  perf-stat.instructions
     43348 ±  2%     +34.2%      58161 ± 12%     +33.0%      57666 ±  8%  perf-stat.instructions-per-iTLB-miss
      0.99 ±  0%      +5.5%       1.04 ±  0%      +8.1%       1.07 ±  0%  perf-stat.ipc
    262799 ±  0%      +4.4%     274251 ±  1%      +4.6%     274993 ±  0%  perf-stat.minor-faults
     34.12 ±  1%      +2.1%      34.83 ±  0%      +5.5%      35.99 ±  0%  perf-stat.node-load-miss-rate
  46476754 ±  2%      +4.6%   48601269 ±  1%      +5.5%   49038648 ±  0%  perf-stat.node-load-misses
  89728871 ±  1%      +1.3%   90913257 ±  1%      -2.8%   87233401 ±  0%  perf-stat.node-loads
      9.96 ±  0%     +13.4%      11.30 ±  0%     +18.3%      11.79 ±  3%  perf-stat.node-store-miss-rate
  24460859 ±  1%     +14.4%   27971097 ±  1%      +5.3%   25747546 ±  3%  perf-stat.node-store-misses
 2.211e+08 ±  1%      -0.6%  2.197e+08 ±  1%     -12.6%  1.931e+08 ±  6%  perf-stat.node-stores
    262780 ±  0%      +4.4%     274227 ±  1%      +4.6%     274953 ±  0%  perf-stat.page-faults
     11.31 ±  1%     -18.1%       9.27 ±  0%     -17.3%       9.36 ±  0%  perf-profile.cycles-pp.____fput.task_work_run.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath
      0.00 ± -1%      +Inf%       1.68 ±  1%      +Inf%       1.74 ±  1%  perf-profile.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin
      1.80 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin
      2.55 ±  3%     -14.2%       2.19 ±  2%     -15.3%       2.16 ±  0%  perf-profile.cycles-pp.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin
      0.00 ± -1%      +Inf%       4.45 ±  1%      +Inf%       4.96 ±  1%  perf-profile.cycles-pp.__block_commit_write.isra.24.block_write_end.generic_write_end.iomap_write_actor.iomap_apply
      5.93 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__block_commit_write.isra.24.block_write_end.generic_write_end.xfs_vm_write_end.generic_perform_write
     13.71 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__block_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter
     10.36 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write
      0.00 ± -1%      +Inf%       3.64 ±  0%      +Inf%       3.79 ±  2%  perf-profile.cycles-pp.__block_write_begin_int.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      1.04 ±  2%     -18.9%       0.84 ±  1%     -15.4%       0.88 ±  0%  perf-profile.cycles-pp.__delete_from_page_cache.delete_from_page_cache.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final
     11.24 ±  2%     -18.1%       9.21 ±  0%     -17.3%       9.30 ±  0%  perf-profile.cycles-pp.__dentry_kill.dput.__fput.____fput.task_work_run
     11.31 ±  2%     -18.1%       9.26 ±  0%     -17.3%       9.36 ±  0%  perf-profile.cycles-pp.__fput.____fput.task_work_run.exit_to_usermode_loop.syscall_return_slowpath
      1.72 ±  2%     -10.1%       1.54 ±  1%     -17.6%       1.42 ±  0%  perf-profile.cycles-pp.__lru_cache_add.lru_cache_add.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin
      0.00 ± -1%      +Inf%       1.09 ±  2%      +Inf%       1.12 ±  1%  perf-profile.cycles-pp.__mark_inode_dirty.generic_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      1.32 ±  4%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__mark_inode_dirty.generic_write_end.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write
      0.00 ± -1%      +Inf%       2.68 ±  2%      +Inf%       2.65 ±  0%  perf-profile.cycles-pp.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor
      3.04 ±  3%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write
      2.50 ±  3%     -11.5%       2.21 ±  0%     -18.8%       2.03 ±  0%  perf-profile.cycles-pp.__pagevec_release.truncate_inode_pages_range.truncate_inode_pages_final.evict.iput
      1.00 ±  1%     -18.0%       0.82 ±  1%     -10.0%       0.90 ±  0%  perf-profile.cycles-pp.__radix_tree_lookup.radix_tree_lookup_slot.find_get_entry.pagecache_get_page.grab_cache_page_write_begin
      1.12 ±  2%     -17.6%       0.92 ±  4%     -13.8%       0.96 ±  0%  perf-profile.cycles-pp.__sb_start_write.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      1.38 ±  2%     -13.3%       1.19 ±  1%     -12.5%       1.21 ±  0%  perf-profile.cycles-pp.__set_page_dirty.mark_buffer_dirty.__block_commit_write.isra.24.block_write_end.generic_write_end
     54.10 ±  1%     +13.1%      61.20 ±  0%     +10.6%      59.86 ±  0%  perf-profile.cycles-pp.__vfs_write.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      6.34 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin
      0.00 ± -1%      +Inf%       3.69 ±  1%      +Inf%       3.62 ±  0%  perf-profile.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor
      4.02 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write
      0.98 ±  5%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.alloc_page_buffers.create_empty_buffers.create_page_buffers.__block_write_begin_int.__block_write_begin
      0.00 ± -1%      +Inf%       2.56 ±  2%      +Inf%       2.50 ±  0%  perf-profile.cycles-pp.alloc_pages_current.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin
      2.91 ±  3%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.alloc_pages_current.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin
      3.42 ±  0%     -20.9%       2.71 ±  2%     -15.7%       2.88 ±  0%  perf-profile.cycles-pp.block_invalidatepage.xfs_vm_invalidatepage.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final
      0.00 ± -1%      +Inf%       4.69 ±  0%      +Inf%       5.54 ±  1%  perf-profile.cycles-pp.block_write_end.generic_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      6.24 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.block_write_end.generic_write_end.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write
     19.18 ±  5%      -9.3%      17.40 ±  0%      -5.8%      18.06 ±  1%  perf-profile.cycles-pp.call_cpuidle.cpu_startup_entry.start_secondary
      0.94 ±  4%     -19.8%       0.76 ±  0%     -15.2%       0.80 ±  1%  perf-profile.cycles-pp.cancel_dirty_page.try_to_free_buffers.xfs_vm_releasepage.try_to_release_page.block_invalidatepage
      3.95 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.copy_user_enhanced_fast_string.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
      0.00 ± -1%      +Inf%       3.22 ±  0%      +Inf%       3.29 ±  1%  perf-profile.cycles-pp.copy_user_enhanced_fast_string.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
     19.75 ±  5%      -9.8%      17.81 ±  0%      -6.3%      18.50 ±  1%  perf-profile.cycles-pp.cpu_startup_entry.start_secondary
     19.18 ±  5%      -9.3%      17.40 ±  0%      -5.8%      18.05 ±  1%  perf-profile.cycles-pp.cpuidle_enter.call_cpuidle.cpu_startup_entry.start_secondary
     18.45 ±  5%      -9.2%      16.75 ±  0%      -5.6%      17.42 ±  1%  perf-profile.cycles-pp.cpuidle_enter_state.cpuidle_enter.call_cpuidle.cpu_startup_entry.start_secondary
      1.44 ±  3%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.create_empty_buffers.create_page_buffers.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin
      0.00 ± -1%      +Inf%       1.18 ±  1%      +Inf%       1.25 ±  2%  perf-profile.cycles-pp.create_empty_buffers.create_page_buffers.__block_write_begin_int.iomap_write_begin.iomap_write_actor
      1.86 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.create_page_buffers.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin.generic_perform_write
      0.00 ± -1%      +Inf%       1.53 ±  1%      +Inf%       1.61 ±  3%  perf-profile.cycles-pp.create_page_buffers.__block_write_begin_int.iomap_write_begin.iomap_write_actor.iomap_apply
      1.74 ±  2%     -19.9%       1.40 ±  3%     -16.8%       1.45 ±  0%  perf-profile.cycles-pp.delete_from_page_cache.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final.evict
      1.27 ±  0%     -22.5%       0.99 ±  4%     -22.3%       0.99 ±  0%  perf-profile.cycles-pp.destroy_inode.evict.iput.__dentry_kill.dput
      2.61 ±  1%     -24.3%       1.98 ±  1%     -20.7%       2.07 ±  0%  perf-profile.cycles-pp.do_filp_open.do_sys_open.sys_creat.entry_SYSCALL_64_fastpath
      2.66 ±  1%     -24.3%       2.01 ±  1%     -20.5%       2.12 ±  0%  perf-profile.cycles-pp.do_sys_open.sys_creat.entry_SYSCALL_64_fastpath
      1.79 ±  2%     -28.2%       1.28 ±  3%     -23.3%       1.37 ±  2%  perf-profile.cycles-pp.do_unlinkat.sys_unlink.entry_SYSCALL_64_fastpath
      1.07 ±  3%     -23.3%       0.82 ±  3%     -19.4%       0.86 ±  0%  perf-profile.cycles-pp.down_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write
      1.01 ±  3%     -17.9%       0.83 ±  2%     -13.6%       0.87 ±  1%  perf-profile.cycles-pp.down_write.xfs_ilock.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
     11.26 ±  2%     -18.1%       9.23 ±  0%     -17.2%       9.32 ±  0%  perf-profile.cycles-pp.dput.__fput.____fput.task_work_run.exit_to_usermode_loop
     11.21 ±  2%     -18.1%       9.18 ±  0%     -17.4%       9.26 ±  0%  perf-profile.cycles-pp.evict.iput.__dentry_kill.dput.__fput
     11.34 ±  2%     -18.1%       9.29 ±  0%     -17.3%       9.38 ±  0%  perf-profile.cycles-pp.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath
      0.00 ± -1%      +Inf%       1.55 ±  3%      +Inf%       1.65 ±  0%  perf-profile.cycles-pp.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor
      1.83 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write
     43.95 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write
      0.00 ± -1%      +Inf%       7.91 ±  1%      +Inf%       9.04 ±  0%  perf-profile.cycles-pp.generic_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
     10.68 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.generic_write_end.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter
      1.91 ±  3%     -16.4%       1.59 ±  1%     -17.7%       1.57 ±  0%  perf-profile.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc.pagecache_get_page
      0.00 ± -1%      +Inf%       9.85 ±  0%      +Inf%       9.91 ±  0%  perf-profile.cycles-pp.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write
     10.96 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter
      0.00 ± -1%      +Inf%      52.29 ±  0%      +Inf%      50.82 ±  0%  perf-profile.cycles-pp.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
      0.00 ± -1%      +Inf%      52.94 ±  0%      +Inf%      51.44 ±  0%  perf-profile.cycles-pp.iomap_file_buffered_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write
      0.00 ± -1%      +Inf%      34.35 ±  0%      +Inf%      32.27 ±  0%  perf-profile.cycles-pp.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.xfs_file_write_iter
      0.00 ± -1%      +Inf%      16.48 ±  0%      +Inf%      16.75 ±  1%  perf-profile.cycles-pp.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
     11.22 ±  2%     -18.1%       9.19 ±  0%     -17.3%       9.27 ±  0%  perf-profile.cycles-pp.iput.__dentry_kill.dput.__fput.____fput
      0.00 ± -1%      +Inf%       1.55 ±  1%      +Inf%       1.42 ±  0%  perf-profile.cycles-pp.lru_cache_add.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin
      1.72 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.lru_cache_add.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin
      0.00 ± -1%      +Inf%       2.78 ±  0%      +Inf%       2.88 ±  1%  perf-profile.cycles-pp.mark_buffer_dirty.__block_commit_write.isra.24.block_write_end.generic_write_end.iomap_write_actor
      3.39 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.mark_buffer_dirty.__block_commit_write.isra.24.block_write_end.generic_write_end.xfs_vm_write_end
      0.00 ± -1%      +Inf%       3.44 ±  1%      +NaN%       0.00 ± -1%  perf-profile.cycles-pp.mark_page_accessed.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
      3.03 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.memset_erms.__block_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write
      0.00 ± -1%      +Inf%       2.43 ±  0%      +Inf%       2.48 ±  3%  perf-profile.cycles-pp.memset_erms.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      0.00 ± -1%      +Inf%       9.25 ±  0%      +Inf%       9.25 ±  0%  perf-profile.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply
     10.37 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write
      1.52 ±  2%      -9.2%       1.38 ±  1%     -17.0%       1.27 ±  0%  perf-profile.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.lru_cache_add.add_to_page_cache_lru.pagecache_get_page
      2.58 ±  1%     -24.1%       1.96 ±  0%     -20.6%       2.05 ±  0%  perf-profile.cycles-pp.path_openat.do_filp_open.do_sys_open.sys_creat.entry_SYSCALL_64_fastpath
      0.00 ± -1%      +Inf%       0.95 ±  0%      +Inf%       1.04 ±  0%  perf-profile.cycles-pp.radix_tree_lookup_slot.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin
      1.17 ±  3%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.radix_tree_lookup_slot.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.xfs_vm_write_begin
      2.39 ±  3%     -11.2%       2.12 ±  0%     -18.3%       1.95 ±  1%  perf-profile.cycles-pp.release_pages.__pagevec_release.truncate_inode_pages_range.truncate_inode_pages_final.evict
      2.06 ±  3%     -22.5%       1.60 ±  2%     -10.9%       1.83 ±  0%  perf-profile.cycles-pp.rw_verify_area.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      1.79 ±  3%     -22.2%       1.39 ±  0%      -9.8%       1.62 ±  0%  perf-profile.cycles-pp.security_file_permission.rw_verify_area.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      1.32 ±  4%     -21.4%       1.04 ±  0%      -7.4%       1.23 ±  1%  perf-profile.cycles-pp.selinux_file_permission.security_file_permission.rw_verify_area.vfs_write.sys_write
     19.79 ±  5%      -9.9%      17.84 ±  0%      -6.4%      18.54 ±  1%  perf-profile.cycles-pp.start_secondary
      2.67 ±  1%     -24.2%       2.02 ±  1%     -20.4%       2.12 ±  1%  perf-profile.cycles-pp.sys_creat.entry_SYSCALL_64_fastpath
      1.79 ±  3%     -27.9%       1.29 ±  3%     -23.0%       1.38 ±  2%  perf-profile.cycles-pp.sys_unlink.entry_SYSCALL_64_fastpath
     60.98 ±  1%      +9.5%      66.76 ±  0%      +7.8%      65.74 ±  0%  perf-profile.cycles-pp.sys_write.entry_SYSCALL_64_fastpath
     11.34 ±  1%     -18.1%       9.29 ±  0%     -17.2%       9.39 ±  0%  perf-profile.cycles-pp.syscall_return_slowpath.entry_SYSCALL_64_fastpath
     11.32 ±  1%     -18.0%       9.28 ±  0%     -17.3%       9.37 ±  0%  perf-profile.cycles-pp.task_work_run.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath
      5.96 ±  1%     -20.0%       4.77 ±  0%     -15.8%       5.02 ±  0%  perf-profile.cycles-pp.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final.evict.iput
      9.89 ±  2%     -17.4%       8.17 ±  0%     -16.7%       8.25 ±  0%  perf-profile.cycles-pp.truncate_inode_pages_final.evict.iput.__dentry_kill.dput
      9.87 ±  2%     -17.5%       8.15 ±  0%     -16.8%       8.21 ±  0%  perf-profile.cycles-pp.truncate_inode_pages_range.truncate_inode_pages_final.evict.iput.__dentry_kill
      2.07 ±  1%     -20.4%       1.65 ±  2%     -14.9%       1.77 ±  1%  perf-profile.cycles-pp.try_to_free_buffers.xfs_vm_releasepage.try_to_release_page.block_invalidatepage.xfs_vm_invalidatepage
      2.40 ±  1%     -21.0%       1.89 ±  2%     -15.3%       2.03 ±  1%  perf-profile.cycles-pp.try_to_release_page.block_invalidatepage.xfs_vm_invalidatepage.truncate_inode_page.truncate_inode_pages_range
      0.00 ± -1%      +Inf%       1.36 ±  1%      +Inf%       1.56 ±  3%  perf-profile.cycles-pp.unlock_page.generic_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      1.72 ±  4%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.unlock_page.generic_write_end.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write
     59.63 ±  1%     +10.2%      65.72 ±  0%      +8.5%      64.68 ±  0%  perf-profile.cycles-pp.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      0.00 ± -1%      +Inf%       1.52 ±  2%      +NaN%       0.00 ± -1%  perf-profile.cycles-pp.workingset_activation.mark_page_accessed.iomap_write_actor.iomap_apply.iomap_file_buffered_write
      0.00 ± -1%      +Inf%       1.73 ±  1%      +Inf%       1.75 ±  2%  perf-profile.cycles-pp.xfs_bmap_search_extents.xfs_bmapi_delay.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply
      0.00 ± -1%      +Inf%       1.97 ±  2%      +Inf%       2.04 ±  0%  perf-profile.cycles-pp.xfs_bmap_search_extents.xfs_bmapi_read.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write
      0.00 ± -1%      +Inf%       1.61 ±  2%      +Inf%       1.65 ±  1%  perf-profile.cycles-pp.xfs_bmap_search_extents.xfs_bmapi_read.xfs_iomap_eof_want_preallocate.constprop.8.xfs_iomap_write_delay.xfs_file_iomap_begin
      0.00 ± -1%      +Inf%       1.24 ±  2%      +Inf%       1.21 ±  3%  perf-profile.cycles-pp.xfs_bmap_search_multi_extents.xfs_bmap_search_extents.xfs_bmapi_delay.xfs_iomap_write_delay.xfs_file_iomap_begin
      0.00 ± -1%      +Inf%       1.46 ±  1%      +Inf%       1.47 ±  1%  perf-profile.cycles-pp.xfs_bmap_search_multi_extents.xfs_bmap_search_extents.xfs_bmapi_read.xfs_file_iomap_begin.iomap_apply
      0.00 ± -1%      +Inf%       1.21 ±  2%      +Inf%       1.25 ±  0%  perf-profile.cycles-pp.xfs_bmap_search_multi_extents.xfs_bmap_search_extents.xfs_bmapi_read.xfs_iomap_eof_want_preallocate.constprop.8.xfs_iomap_write_delay
      1.25 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_bmapi_delay.xfs_iomap_write_delay.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int
      0.00 ± -1%      +Inf%       3.06 ±  1%      +Inf%       3.08 ±  1%  perf-profile.cycles-pp.xfs_bmapi_delay.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write
      1.04 ±  0%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_bmapi_read.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int.__block_write_begin
      0.00 ± -1%      +Inf%       3.04 ±  1%      +Inf%       3.16 ±  1%  perf-profile.cycles-pp.xfs_bmapi_read.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
      0.00 ± -1%      +Inf%       3.05 ±  1%      +Inf%       3.09 ±  1%  perf-profile.cycles-pp.xfs_bmapi_read.xfs_iomap_eof_want_preallocate.constprop.8.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply
      1.32 ±  2%     -21.5%       1.04 ±  1%     -19.7%       1.06 ±  0%  perf-profile.cycles-pp.xfs_create.xfs_generic_create.xfs_vn_mknod.xfs_vn_create.path_openat
     51.83 ±  1%     +14.3%      59.25 ±  0%     +11.8%      57.95 ±  0%  perf-profile.cycles-pp.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write.sys_write
      0.00 ± -1%      +Inf%      16.05 ±  0%      +Inf%      16.68 ±  0%  perf-profile.cycles-pp.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.xfs_file_write_iter
     53.16 ±  1%     +13.6%      60.40 ±  0%     +11.1%      59.09 ±  0%  perf-profile.cycles-pp.xfs_file_write_iter.__vfs_write.vfs_write.sys_write.entry_SYSCALL_64_fastpath
      1.24 ±  1%     -23.1%       0.95 ±  4%     -21.8%       0.97 ±  0%  perf-profile.cycles-pp.xfs_fs_destroy_inode.destroy_inode.evict.iput.__dentry_kill
      1.42 ±  2%     -21.2%       1.12 ±  1%     -20.6%       1.12 ±  0%  perf-profile.cycles-pp.xfs_generic_create.xfs_vn_mknod.xfs_vn_create.path_openat.do_filp_open
      6.46 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_get_blocks.__block_write_begin_int.__block_write_begin.xfs_vm_write_begin.generic_perform_write
      1.29 ±  3%     -18.9%       1.04 ±  1%     -14.1%       1.10 ±  0%  perf-profile.cycles-pp.xfs_ilock.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write.vfs_write
      0.00 ± -1%      +Inf%       1.14 ±  3%      +Inf%       1.17 ±  1%  perf-profile.cycles-pp.xfs_ilock.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
      1.21 ±  1%     -23.4%       0.93 ±  4%     -22.5%       0.94 ±  0%  perf-profile.cycles-pp.xfs_inactive.xfs_fs_destroy_inode.destroy_inode.evict.iput
      1.23 ±  4%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_iomap_eof_want_preallocate.constprop.6.xfs_iomap_write_delay.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int
      0.00 ± -1%      +Inf%       4.14 ±  0%      +Inf%       4.15 ±  1%  perf-profile.cycles-pp.xfs_iomap_eof_want_preallocate.constprop.8.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write
      3.28 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_iomap_write_delay.__xfs_get_blocks.xfs_get_blocks.__block_write_begin_int.__block_write_begin
      0.00 ± -1%      +Inf%       9.08 ±  0%      +Inf%       9.19 ±  1%  perf-profile.cycles-pp.xfs_iomap_write_delay.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
      3.54 ±  0%     -20.8%       2.81 ±  1%     -15.6%       2.99 ±  0%  perf-profile.cycles-pp.xfs_vm_invalidatepage.truncate_inode_page.truncate_inode_pages_range.truncate_inode_pages_final.evict
      2.35 ±  1%     -21.0%       1.86 ±  1%     -15.1%       2.00 ±  1%  perf-profile.cycles-pp.xfs_vm_releasepage.try_to_release_page.block_invalidatepage.xfs_vm_invalidatepage.truncate_inode_page
     25.10 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_vm_write_begin.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
     11.03 ±  1%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.cycles-pp.xfs_vm_write_end.generic_perform_write.xfs_file_buffered_aio_write.xfs_file_write_iter.__vfs_write
      1.42 ±  2%     -20.7%       1.13 ±  1%     -20.4%       1.13 ±  0%  perf-profile.cycles-pp.xfs_vn_create.path_openat.do_filp_open.do_sys_open.sys_creat
      1.42 ±  2%     -20.5%       1.13 ±  1%     -20.2%       1.13 ±  0%  perf-profile.cycles-pp.xfs_vn_mknod.xfs_vn_create.path_openat.do_filp_open.do_sys_open
      2.27 ±  1%     -10.6%       2.03 ±  0%      -6.7%       2.12 ±  1%  perf-profile.func.cycles-pp.___might_sleep
      2.49 ±  0%     -34.5%       1.63 ±  1%     -16.7%       2.08 ±  0%  perf-profile.func.cycles-pp.__block_commit_write.isra.24
      1.51 ±  2%     +15.4%       1.75 ±  1%     +18.0%       1.79 ±  2%  perf-profile.func.cycles-pp.__block_write_begin_int
      1.79 ±  4%     -16.8%       1.49 ±  1%     -14.5%       1.53 ±  0%  perf-profile.func.cycles-pp.__mark_inode_dirty
      1.32 ±  0%     -16.4%       1.10 ±  1%      -9.5%       1.19 ±  0%  perf-profile.func.cycles-pp.__radix_tree_lookup
      1.08 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.func.cycles-pp.__xfs_get_blocks
      1.16 ±  0%     -18.1%       0.95 ±  1%     -15.8%       0.98 ±  1%  perf-profile.func.cycles-pp._raw_spin_lock
      3.96 ±  2%     -18.4%       3.23 ±  0%     -16.9%       3.29 ±  1%  perf-profile.func.cycles-pp.copy_user_enhanced_fast_string
      1.41 ±  3%     -20.6%       1.12 ±  3%     -21.1%       1.11 ±  3%  perf-profile.func.cycles-pp.entry_SYSCALL_64_fastpath
      1.30 ±  2%    -100.0%       0.00 ± -1%    -100.0%       0.00 ± -1%  perf-profile.func.cycles-pp.generic_perform_write
      1.31 ±  2%     -46.7%       0.70 ±  0%     -43.8%       0.73 ±  0%  perf-profile.func.cycles-pp.generic_write_end
     18.43 ±  5%      -9.1%      16.76 ±  0%      -5.4%      17.44 ±  1%  perf-profile.func.cycles-pp.intel_idle
      0.00 ± -1%      +Inf%       1.12 ±  1%      +Inf%       0.90 ±  0%  perf-profile.func.cycles-pp.iomap_write_actor
      1.50 ±  1%     -20.9%       1.19 ±  1%     -17.0%       1.25 ±  2%  perf-profile.func.cycles-pp.mark_buffer_dirty
      0.00 ± -1%      +Inf%       1.91 ±  1%      +NaN%       0.00 ± -1%  perf-profile.func.cycles-pp.mark_page_accessed
      3.24 ±  0%     -19.8%       2.60 ±  0%     -18.1%       2.66 ±  3%  perf-profile.func.cycles-pp.memset_erms
      1.75 ±  2%     -18.9%       1.42 ±  1%      -7.3%       1.62 ±  4%  perf-profile.func.cycles-pp.unlock_page
      1.56 ±  2%      +6.0%       1.65 ±  3%     +11.8%       1.74 ±  1%  perf-profile.func.cycles-pp.up_write
      1.16 ±  1%     -21.6%       0.91 ±  1%     -17.7%       0.95 ±  1%  perf-profile.func.cycles-pp.vfs_write
      0.37 ±  2%    +243.6%       1.26 ±  2%    +272.3%       1.36 ±  2%  perf-profile.func.cycles-pp.xfs_bmap_search_extents
      0.41 ±  1%    +198.4%       1.22 ±  2%    +198.8%       1.23 ±  3%  perf-profile.func.cycles-pp.xfs_bmap_search_multi_extents
      0.70 ±  5%    +219.5%       2.24 ±  0%    +227.9%       2.29 ±  0%  perf-profile.func.cycles-pp.xfs_bmapi_read
      1.05 ±  2%     -15.6%       0.88 ±  3%     -18.8%       0.85 ±  1%  perf-profile.func.cycles-pp.xfs_file_write_iter
      0.64 ±  1%    +182.8%       1.81 ±  4%    +182.0%       1.81 ±  0%  perf-profile.func.cycles-pp.xfs_iext_bno_to_ext
      0.00 ± -1%      +Inf%       1.10 ±  3%      +Inf%       1.21 ±  2%  perf-profile.func.cycles-pp.xfs_iomap_eof_want_preallocate.constprop.8
      0.46 ±  4%    +161.6%       1.20 ±  1%    +171.7%       1.25 ±  1%  perf-profile.func.cycles-pp.xfs_iomap_write_delay


raw perf data:

  "perf-profile.func.cycles-pp.intel_idle": 17.66,
  "perf-profile.func.cycles-pp.copy_user_enhanced_fast_string": 3.25,
  "perf-profile.func.cycles-pp.memset_erms": 2.56,
  "perf-profile.func.cycles-pp.xfs_bmapi_read": 2.28,
  "perf-profile.func.cycles-pp.___might_sleep": 2.09,
  "perf-profile.func.cycles-pp.__block_commit_write.isra.24": 2.07,
  "perf-profile.func.cycles-pp.xfs_iext_bno_to_ext": 1.79,
  "perf-profile.func.cycles-pp.__block_write_begin_int": 1.74,
  "perf-profile.func.cycles-pp.up_write": 1.72,
  "perf-profile.func.cycles-pp.unlock_page": 1.69,
  "perf-profile.func.cycles-pp.down_write": 1.59,
  "perf-profile.func.cycles-pp.__mark_inode_dirty": 1.54,
  "perf-profile.func.cycles-pp.xfs_bmap_search_extents": 1.33,
  "perf-profile.func.cycles-pp.xfs_iomap_write_delay": 1.23,
  "perf-profile.func.cycles-pp.mark_buffer_dirty": 1.21,
  "perf-profile.func.cycles-pp.__radix_tree_lookup": 1.2,
  "perf-profile.func.cycles-pp.xfs_bmap_search_multi_extents": 1.18,
  "perf-profile.func.cycles-pp.xfs_iomap_eof_want_preallocate.constprop.8": 1.17,
  "perf-profile.func.cycles-pp.entry_SYSCALL_64_fastpath": 1.15,
  "perf-profile.func.cycles-pp.__might_sleep": 1.14,
  "perf-profile.func.cycles-pp._raw_spin_lock": 0.97,
  "perf-profile.func.cycles-pp.vfs_write": 0.94,
  "perf-profile.func.cycles-pp.xfs_bmapi_delay": 0.93,
  "perf-profile.func.cycles-pp.iomap_write_actor": 0.9,
  "perf-profile.func.cycles-pp.pagecache_get_page": 0.89,
  "perf-profile.func.cycles-pp.xfs_file_write_iter": 0.86,
  "perf-profile.func.cycles-pp.xfs_file_iomap_begin": 0.81,
  "perf-profile.func.cycles-pp.iov_iter_copy_from_user_atomic": 0.78,
  "perf-profile.func.cycles-pp.iomap_apply": 0.77,
  "perf-profile.func.cycles-pp.generic_write_end": 0.74,
  "perf-profile.func.cycles-pp.xfs_file_buffered_aio_write": 0.72,
  "perf-profile.func.cycles-pp.find_get_entry": 0.69,
  "perf-profile.func.cycles-pp.__vfs_write": 0.67,


Best Regards,
Huang, Ying

  parent reply	other threads:[~2016-08-11 21:16 UTC|newest]

Thread overview: 219+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-08-09 14:33 [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression kernel test robot
2016-08-09 14:33 ` kernel test robot
2016-08-10 18:24 ` [lkp] " Linus Torvalds
2016-08-10 18:24   ` Linus Torvalds
2016-08-10 23:08   ` [lkp] " Dave Chinner
2016-08-10 23:08     ` Dave Chinner
2016-08-10 23:51     ` [lkp] " Linus Torvalds
2016-08-10 23:51       ` Linus Torvalds
2016-08-10 23:58       ` [LKP] [lkp] " Huang, Ying
2016-08-10 23:58         ` Huang, Ying
2016-08-11  0:11         ` [LKP] [lkp] " Huang, Ying
2016-08-11  0:11           ` Huang, Ying
2016-08-11  0:23           ` [LKP] [lkp] " Linus Torvalds
2016-08-11  0:23             ` Linus Torvalds
2016-08-11  0:33             ` [LKP] [lkp] " Huang, Ying
2016-08-11  0:33               ` Huang, Ying
2016-08-11  1:00               ` [LKP] [lkp] " Linus Torvalds
2016-08-11  1:00                 ` Linus Torvalds
2016-08-11  4:46                 ` [LKP] [lkp] " Dave Chinner
2016-08-11  4:46                   ` Dave Chinner
2016-08-15 17:22                   ` [LKP] [lkp] " Huang, Ying
2016-08-15 17:22                     ` Huang, Ying
2016-08-16  0:08                     ` [LKP] [lkp] " Dave Chinner
2016-08-16  0:08                       ` Dave Chinner
2016-08-11 15:57                 ` [LKP] [lkp] " Christoph Hellwig
2016-08-11 15:57                   ` Christoph Hellwig
2016-08-11 16:55                   ` [LKP] [lkp] " Linus Torvalds
2016-08-11 16:55                     ` Linus Torvalds
2016-08-11 17:51                     ` [LKP] [lkp] " Huang, Ying
2016-08-11 17:51                       ` Huang, Ying
2016-08-11 19:51                       ` [LKP] [lkp] " Linus Torvalds
2016-08-11 19:51                         ` Linus Torvalds
2016-08-11 20:00                         ` [LKP] [lkp] " Christoph Hellwig
2016-08-11 20:00                           ` Christoph Hellwig
2016-08-11 20:35                           ` [LKP] [lkp] " Linus Torvalds
2016-08-11 20:35                             ` Linus Torvalds
2016-08-11 22:16                             ` [LKP] [lkp] " Al Viro
2016-08-11 22:16                               ` Al Viro
2016-08-11 22:30                               ` [LKP] [lkp] " Linus Torvalds
2016-08-11 22:30                                 ` Linus Torvalds
2016-08-11 21:16                           ` Huang, Ying [this message]
2016-08-11 21:16                             ` Huang, Ying
2016-08-11 21:40                             ` [LKP] [lkp] " Linus Torvalds
2016-08-11 21:40                               ` Linus Torvalds
2016-08-11 22:08                               ` [LKP] [lkp] " Christoph Hellwig
2016-08-11 22:08                                 ` Christoph Hellwig
2016-08-12  0:54                     ` [LKP] [lkp] " Dave Chinner
2016-08-12  0:54                       ` Dave Chinner
2016-08-12  2:23                       ` [LKP] [lkp] " Dave Chinner
2016-08-12  2:23                         ` Dave Chinner
2016-08-12  2:32                         ` [LKP] [lkp] " Linus Torvalds
2016-08-12  2:32                           ` Linus Torvalds
2016-08-12  2:52                         ` [LKP] [lkp] " Christoph Hellwig
2016-08-12  2:52                           ` Christoph Hellwig
2016-08-12  3:20                           ` [LKP] [lkp] " Linus Torvalds
2016-08-12  3:20                             ` Linus Torvalds
2016-08-12  4:16                             ` [LKP] [lkp] " Dave Chinner
2016-08-12  4:16                               ` Dave Chinner
2016-08-12  5:02                               ` [LKP] [lkp] " Linus Torvalds
2016-08-12  5:02                                 ` Linus Torvalds
2016-08-12  6:04                                 ` [LKP] [lkp] " Dave Chinner
2016-08-12  6:04                                   ` Dave Chinner
2016-08-12  6:29                                   ` [LKP] [lkp] " Ye Xiaolong
2016-08-12  6:29                                     ` Ye Xiaolong
2016-08-12  8:51                                     ` [LKP] [lkp] " Ye Xiaolong
2016-08-12  8:51                                       ` Ye Xiaolong
2016-08-12 10:02                                       ` [LKP] [lkp] " Dave Chinner
2016-08-12 10:02                                         ` Dave Chinner
2016-08-12 10:43                                         ` Fengguang Wu
2016-08-12 10:43                                           ` Fengguang Wu
2016-08-13  0:30                                         ` [LKP] [lkp] " Christoph Hellwig
2016-08-13  0:30                                           ` Christoph Hellwig
2016-08-13 21:48                                           ` [LKP] [lkp] " Christoph Hellwig
2016-08-13 21:48                                             ` Christoph Hellwig
2016-08-13 22:07                                             ` [LKP] [lkp] " Fengguang Wu
2016-08-13 22:07                                               ` Fengguang Wu
2016-08-13 22:15                                               ` [LKP] [lkp] " Christoph Hellwig
2016-08-13 22:15                                                 ` Christoph Hellwig
2016-08-13 22:51                                                 ` [LKP] [lkp] " Fengguang Wu
2016-08-13 22:51                                                   ` Fengguang Wu
2016-08-14 14:50                                                   ` [LKP] [lkp] " Fengguang Wu
2016-08-14 14:50                                                     ` Fengguang Wu
2016-08-14 16:17                                                     ` [LKP] [lkp] " Christoph Hellwig
2016-08-14 16:17                                                       ` Christoph Hellwig
2016-08-14 23:46                                                       ` [LKP] [lkp] " Dave Chinner
2016-08-14 23:46                                                         ` Dave Chinner
2016-08-14 23:57                                                       ` [LKP] [lkp] " Fengguang Wu
2016-08-14 23:57                                                         ` Fengguang Wu
2016-08-15 14:14                                                       ` [LKP] [lkp] " Fengguang Wu
2016-08-15 14:14                                                         ` Fengguang Wu
2016-08-15 21:22                                                         ` [LKP] [lkp] " Dave Chinner
2016-08-15 21:22                                                           ` Dave Chinner
2016-08-16 12:20                                                           ` [LKP] [lkp] " Fengguang Wu
2016-08-16 12:20                                                             ` Fengguang Wu
2016-08-15 20:30                                                       ` [LKP] [lkp] " Huang, Ying
2016-08-15 20:30                                                         ` Huang, Ying
2016-08-22 22:09                                                         ` [LKP] [lkp] " Huang, Ying
2016-08-22 22:09                                                           ` Huang, Ying
2016-09-26  6:25                                                           ` [LKP] [lkp] " Huang, Ying
2016-09-26  6:25                                                             ` Huang, Ying
2016-09-26 14:55                                                             ` [LKP] [lkp] " Christoph Hellwig
2016-09-26 14:55                                                               ` Christoph Hellwig
2016-09-27  0:52                                                               ` [LKP] [lkp] " Huang, Ying
2016-09-27  0:52                                                                 ` Huang, Ying
2016-08-16 13:25                                                       ` [LKP] [lkp] " Fengguang Wu
2016-08-16 13:25                                                         ` Fengguang Wu
2016-08-13 23:32                                           ` [LKP] [lkp] " Dave Chinner
2016-08-13 23:32                                             ` Dave Chinner
2016-08-12  2:27                       ` [LKP] [lkp] " Linus Torvalds
2016-08-12  2:27                         ` Linus Torvalds
2016-08-12  3:56                         ` [LKP] [lkp] " Dave Chinner
2016-08-12  3:56                           ` Dave Chinner
2016-08-12 18:03                           ` [LKP] [lkp] " Linus Torvalds
2016-08-12 18:03                             ` Linus Torvalds
2016-08-13 23:58                             ` [LKP] [lkp] " Fengguang Wu
2016-08-13 23:58                               ` Fengguang Wu
2016-08-15  0:48                             ` [LKP] [lkp] " Dave Chinner
2016-08-15  0:48                               ` Dave Chinner
2016-08-15  1:37                               ` [LKP] [lkp] " Linus Torvalds
2016-08-15  1:37                                 ` Linus Torvalds
2016-08-15  2:28                                 ` [LKP] [lkp] " Dave Chinner
2016-08-15  2:28                                   ` Dave Chinner
2016-08-15  2:53                                   ` [LKP] [lkp] " Linus Torvalds
2016-08-15  2:53                                     ` Linus Torvalds
2016-08-15  5:00                                     ` [LKP] [lkp] " Dave Chinner
2016-08-15  5:00                                       ` Dave Chinner
     [not found]                                       ` <CA+55aFwva2Xffai+Eqv1Jn_NGryk3YJ2i5JoHOQnbQv6qVPAsw@mail.gmail.com>
     [not found]                                         ` <CA+55aFy14nUnJQ_GdF=j8Fa9xiH70c6fY2G3q5HQ01+8z1z3qQ@mail.gmail.com>
2016-08-15  5:12                                           ` Linus Torvalds
2016-08-15 22:22                                             ` [LKP] [lkp] " Dave Chinner
2016-08-15 22:22                                               ` Dave Chinner
2016-08-15 22:42                                               ` [LKP] [lkp] " Dave Chinner
2016-08-15 22:42                                                 ` Dave Chinner
2016-08-15 23:20                                                 ` [LKP] [lkp] " Linus Torvalds
2016-08-15 23:20                                                   ` Linus Torvalds
2016-08-15 23:48                                                   ` [LKP] [lkp] " Linus Torvalds
2016-08-15 23:48                                                     ` Linus Torvalds
2016-08-16  0:44                                                     ` [LKP] [lkp] " Dave Chinner
2016-08-16  0:44                                                       ` Dave Chinner
2016-08-16 15:05                                                     ` [LKP] [lkp] " Mel Gorman
2016-08-16 15:05                                                       ` Mel Gorman
2016-08-16 17:47                                                       ` [LKP] [lkp] " Linus Torvalds
2016-08-16 17:47                                                         ` Linus Torvalds
2016-08-17 15:48                                                         ` [LKP] [lkp] " Michal Hocko
2016-08-17 15:48                                                           ` Michal Hocko
2016-08-17 16:42                                                           ` [LKP] [lkp] " Michal Hocko
2016-08-17 16:42                                                             ` Michal Hocko
2016-08-17 15:49                                                         ` [LKP] [lkp] " Mel Gorman
2016-08-17 15:49                                                           ` Mel Gorman
2016-08-18  0:45                                                           ` [LKP] [lkp] " Mel Gorman
2016-08-18  0:45                                                             ` Mel Gorman
2016-08-18  7:11                                                             ` [LKP] [lkp] " Dave Chinner
2016-08-18  7:11                                                               ` Dave Chinner
2016-08-18 13:24                                                               ` [LKP] [lkp] " Mel Gorman
2016-08-18 13:24                                                                 ` Mel Gorman
2016-08-18 17:55                                                                 ` [LKP] [lkp] " Linus Torvalds
2016-08-18 17:55                                                                   ` Linus Torvalds
2016-08-18 21:19                                                                   ` [LKP] [lkp] " Dave Chinner
2016-08-18 21:19                                                                     ` Dave Chinner
2016-08-18 22:25                                                                     ` [LKP] [lkp] " Linus Torvalds
2016-08-18 22:25                                                                       ` Linus Torvalds
2016-08-19  9:00                                                                       ` [LKP] [lkp] " Michal Hocko
2016-08-19  9:00                                                                         ` Michal Hocko
2016-08-19 10:49                                                                       ` [LKP] [lkp] " Mel Gorman
2016-08-19 10:49                                                                         ` Mel Gorman
2016-08-19 23:48                                                                         ` [LKP] [lkp] " Dave Chinner
2016-08-19 23:48                                                                           ` Dave Chinner
2016-08-20  1:08                                                                           ` [LKP] [lkp] " Linus Torvalds
2016-08-20  1:08                                                                             ` Linus Torvalds
2016-08-20 12:16                                                                           ` [LKP] [lkp] " Mel Gorman
2016-08-20 12:16                                                                             ` Mel Gorman
2016-08-19 15:08                                                               ` [LKP] [lkp] " Mel Gorman
2016-08-19 15:08                                                                 ` Mel Gorman
2016-09-01 23:32                                                                 ` [LKP] [lkp] " Dave Chinner
2016-09-01 23:32                                                                   ` Dave Chinner
2016-09-06 15:37                                                                   ` [LKP] [lkp] " Mel Gorman
2016-09-06 15:37                                                                     ` Mel Gorman
2016-09-06 15:52                                                                     ` [LKP] [lkp] " Huang, Ying
2016-09-06 15:52                                                                       ` Huang, Ying
2016-08-24 15:40                                                             ` [LKP] [lkp] " Huang, Ying
2016-08-24 15:40                                                               ` Huang, Ying
2016-08-25  9:37                                                               ` [LKP] [lkp] " Mel Gorman
2016-08-25  9:37                                                                 ` Mel Gorman
2016-08-18  2:44                                                           ` [LKP] [lkp] " Dave Chinner
2016-08-18  2:44                                                             ` Dave Chinner
2016-08-16  0:15                                                   ` [LKP] [lkp] " Linus Torvalds
2016-08-16  0:15                                                     ` Linus Torvalds
2016-08-16  0:38                                                     ` [LKP] [lkp] " Dave Chinner
2016-08-16  0:38                                                       ` Dave Chinner
2016-08-16  0:50                                                       ` [LKP] [lkp] " Linus Torvalds
2016-08-16  0:50                                                         ` Linus Torvalds
2016-08-16  0:19                                                   ` [LKP] [lkp] " Dave Chinner
2016-08-16  0:19                                                     ` Dave Chinner
2016-08-16  1:51                                                     ` [LKP] [lkp] " Linus Torvalds
2016-08-16  1:51                                                       ` Linus Torvalds
2016-08-16 22:02                                                       ` [LKP] [lkp] " Dave Chinner
2016-08-16 22:02                                                         ` Dave Chinner
2016-08-16 23:23                                                         ` [LKP] [lkp] " Linus Torvalds
2016-08-16 23:23                                                           ` Linus Torvalds
2016-08-15 23:01                                               ` [LKP] [lkp] " Linus Torvalds
2016-08-15 23:01                                                 ` Linus Torvalds
2016-08-16  0:17                                                 ` [LKP] [lkp] " Dave Chinner
2016-08-16  0:17                                                   ` Dave Chinner
2016-08-16  0:45                                                   ` [LKP] [lkp] " Linus Torvalds
2016-08-16  0:45                                                     ` Linus Torvalds
2016-08-15  5:03                                     ` [LKP] [lkp] " Ingo Molnar
2016-08-15  5:03                                       ` Ingo Molnar
2016-08-17 16:24                                       ` [LKP] [lkp] " Peter Zijlstra
2016-08-17 16:24                                         ` Peter Zijlstra
2016-08-15 12:58                             ` [LKP] [lkp] " Fengguang Wu
2016-08-15 12:58                               ` Fengguang Wu
2016-08-11  1:16               ` [LKP] [lkp] " Dave Chinner
2016-08-11  1:16                 ` Dave Chinner
2016-08-11  1:32                 ` [LKP] [lkp] " Dave Chinner
2016-08-11  1:32                   ` Dave Chinner
2016-08-11  2:36                   ` [LKP] [lkp] " Ye Xiaolong
2016-08-11  2:36                     ` Ye Xiaolong
2016-08-11  3:05                     ` [LKP] [lkp] " Dave Chinner
2016-08-11  3:05                       ` Dave Chinner
2016-08-12  1:26                 ` [LKP] [lkp] " Dave Chinner
2016-08-12  1:26                   ` Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87ziojxazw.fsf@yhuang-mobile.sh.intel.com \
    --to=ying.huang@intel.com \
    --cc=david@fromorbit.com \
    --cc=fengguang.wu@intel.com \
    --cc=hch@lst.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lkp@01.org \
    --cc=rpeterso@redhat.com \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.