All of lore.kernel.org
 help / color / mirror / Atom feed
* [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
@ 2014-08-19  4:32 ` Fengguang Wu
  0 siblings, 0 replies; 12+ messages in thread
From: Fengguang Wu @ 2014-08-19  4:32 UTC (permalink / raw)
  To: Mel Gorman; +Cc: LKML, lkp

[-- Attachment #1: Type: text/plain, Size: 67241 bytes --]

Hi Mel,

We noticed the below vm-scalability performance/power regressions on
commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
fields into read-only, page alloc, statistics and page reclaim lines").

24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
---------------  -------------------------  ---------------------------
     %stddev        %change               %stddev
            \          |                 /
      9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
     12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
  36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1138 ±46%     -94.7%         60 ±23%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1138 ±46%     -94.7%         60 ±23%  TOTAL proc-vmstat.compact_success

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      0.95 ±20%     -87.5%       0.12 ±12%  brickland3/vm-scalability/300s-lru-file-readonce
      0.95 ±20%     -87.5%       0.12 ±12%  TOTAL perf-profile.cpu-cycles.read_hpet.ktime_get.clockevents_program_event.tick_program_event.hrtimer_interrupt

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.40 ±17%     -96.1%       0.05 ±27%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2.70 ±36%     -95.0%       0.13 ± 8%  brickland3/vm-scalability/300s-lru-file-readonce
      4.10 ±29%     -95.4%       0.19 ±14%  TOTAL perf-profile.cpu-cycles._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.lru_cache_add.add_to_page_cache_lru

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.46 ±12%     -89.1%       0.16 ±18%  brickland3/vm-scalability/300s-lru-file-readonce
      1.46 ±12%     -89.1%       0.16 ±18%  TOTAL perf-profile.cpu-cycles.read_hpet.ktime_get.tick_sched_timer.__run_hrtimer.hrtimer_interrupt

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.18 ±23%     -82.5%       0.21 ±22%  brickland3/vm-scalability/300s-lru-file-readonce
      1.18 ±23%     -82.5%       0.21 ±22%  TOTAL perf-profile.cpu-cycles.read_hpet.ktime_get_update_offsets_now.hrtimer_interrupt.local_apic_timer_interrupt.smp_apic_timer_interrupt

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       763 ±31%     -84.5%        118 ±11%  brickland3/vm-scalability/300s-lru-file-mmap-read
       763 ±31%     -84.5%        118 ±11%  TOTAL proc-vmstat.kswapd_low_wmark_hit_quickly

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.59 ±12%     -85.1%       0.24 ±19%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1.59 ±12%     -85.1%       0.24 ±19%  TOTAL perf-profile.cpu-cycles.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc.__do_page_cache_readahead

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     19972 ±45%     -71.3%       5725 ±16%  brickland3/vm-scalability/300s-lru-file-mmap-read
     19972 ±45%     -71.3%       5725 ±16%  TOTAL proc-vmstat.pgmigrate_success

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       543 ±28%     -62.1%        205 ±14%  brickland3/vm-scalability/300s-lru-file-mmap-read
        47 ±16%    +828.5%        443 ± 7%  brickland3/vm-scalability/300s-lru-file-readonce
       590 ±27%     +10.0%        649 ± 9%  TOTAL proc-vmstat.kswapd_high_wmark_hit_quickly

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    518176 ± 5%    -100.0%          0 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
    687557 ± 0%     -53.8%     317522 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
   1205734 ± 2%     -73.7%     317522 ± 2%  TOTAL numa-vmstat.node1.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      2.61 ± 5%     -52.7%       1.24 ±25%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1.29 ±10%     -91.0%       0.12 ±15%  brickland3/vm-scalability/300s-lru-file-readonce
      3.91 ± 7%     -65.4%       1.35 ±24%  TOTAL perf-profile.cpu-cycles.read_hpet.ktime_get.sched_clock_tick.scheduler_tick.update_process_times

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.652e+08 ± 5%     -47.7%   86319616 ± 7%  brickland3/vm-scalability/300s-lru-file-mmap-read
  16732078 ± 7%    +894.1%  1.663e+08 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
 1.819e+08 ± 5%     +38.9%  2.527e+08 ± 5%  TOTAL proc-vmstat.pgscan_kswapd_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1565973 ± 2%    -100.0%          0 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
   2720970 ± 0%     -47.2%    1437269 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   4286943 ± 1%     -66.5%    1437269 ± 1%  TOTAL proc-vmstat.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  34449272 ± 5%     -48.1%   17892413 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  16725742 ± 7%    +894.4%  1.663e+08 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  51175014 ± 5%    +260.0%  1.842e+08 ± 3%  TOTAL proc-vmstat.pgsteal_kswapd_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    579964 ± 7%    -100.0%          0 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
    689893 ± 0%     -44.1%     385678 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
   1269857 ± 3%     -69.6%     385678 ± 2%  TOTAL numa-vmstat.node3.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     27.49 ± 3%     -45.8%      14.91 ±17%  brickland3/vm-scalability/300s-lru-file-readonce
     27.49 ± 3%     -45.8%      14.91 ±17%  TOTAL turbostat.%pc2

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    178797 ± 8%     -42.0%     103621 ± 6%  brickland3/vm-scalability/300s-lru-file-mmap-read
     47255 ±12%    +930.2%     486833 ±10%  brickland3/vm-scalability/300s-lru-file-readonce
    226052 ± 9%    +161.2%     590454 ± 9%  TOTAL proc-vmstat.pgsteal_kswapd_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    468136 ± 7%    -100.0%          0 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
    693085 ± 1%     -39.9%     416491 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
   1161222 ± 4%     -64.1%     416491 ± 2%  TOTAL numa-vmstat.node2.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   4318208 ± 1%     -99.7%      11627 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
   6822297 ± 0%     -40.1%    4084812 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  11140505 ± 1%     -63.2%    4096440 ± 1%  TOTAL proc-vmstat.slabs_scanned

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    395581 ±14%     -32.1%     268687 ±15%  brickland3/vm-scalability/300s-lru-file-mmap-read
     47338 ±12%    +928.6%     486919 ±10%  brickland3/vm-scalability/300s-lru-file-readonce
    442920 ±14%     +70.6%     755607 ±12%  TOTAL proc-vmstat.pgscan_kswapd_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      4.37 ± 7%     -61.2%       1.70 ±24%  brickland3/vm-scalability/300s-lru-file-mmap-read
      6.75 ± 5%     -64.9%       2.37 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
     11.13 ± 6%     -63.5%       4.06 ±14%  TOTAL perf-profile.cpu-cycles.memset.mpage_readpages.xfs_vm_readpages.__do_page_cache_readahead.ondemand_readahead

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.37 ± 8%     -62.2%       0.52 ±23%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1.59 ± 5%     -65.7%       0.55 ± 7%  brickland3/vm-scalability/300s-lru-file-readonce
      2.96 ± 6%     -64.1%       1.06 ±15%  TOTAL perf-profile.cpu-cycles.do_mpage_readpage.mpage_readpages.xfs_vm_readpages.__do_page_cache_readahead.ondemand_readahead

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1911 ±12%     -62.2%        722 ±25%  brickland3/vm-scalability/300s-lru-file-readonce
      1911 ±12%     -62.2%        722 ±25%  TOTAL proc-vmstat.pgscan_direct_throttle

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      3.37 ± 9%    +232.0%      11.18 ±25%  brickland3/vm-scalability/300s-lru-file-mmap-read
     26.57 ± 4%     +73.3%      46.04 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
     29.93 ± 5%     +91.2%      57.22 ± 6%  TOTAL perf-profile.cpu-cycles._raw_spin_lock.free_pcppages_bulk.free_hot_cold_page.free_hot_cold_page_list.shrink_page_list

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    292141 ± 4%    +165.3%     775139 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
    292141 ± 4%    +165.3%     775139 ± 4%  TOTAL softirqs.RCU

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  61899502 ± 1%     -53.7%   28663774 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11490237 ± 7%    +204.7%   35005345 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  73389740 ± 2%     -13.2%   63669120 ± 2%  TOTAL numa-numastat.node3.other_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  61894563 ± 1%     -53.7%   28660170 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11487385 ± 7%    +204.7%   35000524 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  73381949 ± 2%     -13.2%   63660694 ± 2%  TOTAL numa-numastat.node3.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  37543259 ± 2%     -54.3%   17139747 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7768241 ± 8%    +181.3%   21853872 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  45311501 ± 3%     -13.9%   38993620 ± 2%  TOTAL numa-vmstat.node3.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  37628483 ± 2%     -54.2%   17224017 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7851378 ± 8%    +179.4%   21938991 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  45479862 ± 3%     -13.9%   39163008 ± 2%  TOTAL numa-vmstat.node3.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.599e+08 ± 1%     -49.9%   80094895 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11422480 ± 2%    +194.8%   33675826 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
 1.713e+08 ± 1%     -33.6%  1.138e+08 ± 2%  TOTAL numa-numastat.node0.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96263262 ± 2%     -53.1%   45191652 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7622035 ± 2%    +178.4%   21217541 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
 103885297 ± 2%     -36.1%   66409193 ± 3%  TOTAL numa-vmstat.node0.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  85346466 ± 2%     -40.8%   50525376 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12056858 ± 3%    +215.0%   37973604 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
  97403324 ± 2%      -9.1%   88498980 ± 4%  TOTAL numa-numastat.node1.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  85350581 ± 2%     -40.8%   50532008 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12061005 ± 3%    +214.9%   37979653 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
  97411587 ± 2%      -9.1%   88511661 ± 4%  TOTAL numa-numastat.node1.other_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  53719356 ± 3%     -43.9%   30133382 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8183737 ± 3%    +192.7%   23954479 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
  61903093 ± 3%     -12.6%   54087861 ± 5%  TOTAL numa-vmstat.node1.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  53802537 ± 3%     -43.8%   30218405 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8266017 ± 3%    +190.8%   24038964 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
  62068554 ± 3%     -12.6%   54257370 ± 5%  TOTAL numa-vmstat.node1.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  14319949 ± 7%    +146.5%   35300490 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  14319949 ± 7%    +146.5%   35300490 ± 2%  TOTAL numa-numastat.node0.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  14327584 ± 7%    +146.4%   35303342 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  14327584 ± 7%    +146.4%   35303342 ± 2%  TOTAL numa-numastat.node0.other_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  41804307 ± 1%     -54.4%   19081241 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7633990 ± 8%    +151.7%   19214179 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  49438297 ± 2%     -22.5%   38295420 ± 3%  TOTAL numa-vmstat.node2.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.228e+08 ± 1%     -46.6%   1.19e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
  49195780 ± 4%    +182.4%  1.389e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  2.72e+08 ± 2%      -5.2%  2.579e+08 ± 1%  TOTAL proc-vmstat.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.228e+08 ± 1%     -46.6%   1.19e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
  49196215 ± 4%    +182.3%  1.389e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  2.72e+08 ± 2%      -5.2%  2.579e+08 ± 1%  TOTAL proc-vmstat.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.228e+08 ± 1%     -46.6%   1.19e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
  49217751 ± 4%    +182.3%  1.389e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
 2.721e+08 ± 2%      -5.2%   2.58e+08 ± 1%  TOTAL proc-vmstat.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  41884224 ± 1%     -54.2%   19165553 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7719489 ± 8%    +150.0%   19299287 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  49603713 ± 2%     -22.5%   38464840 ± 3%  TOTAL numa-vmstat.node2.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  69206605 ± 3%     -51.7%   33399613 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11331586 ± 8%    +170.3%   30629759 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  80538191 ± 4%     -20.5%   64029372 ± 3%  TOTAL numa-numastat.node2.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  69211755 ± 3%     -51.7%   33403393 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11338920 ± 8%    +170.2%   30634435 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  80550676 ± 4%     -20.5%   64037828 ± 3%  TOTAL numa-numastat.node2.other_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  26095298 ± 2%     -51.2%   12728144 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12262186 ± 6%    +153.2%   31042313 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  38357485 ± 4%     +14.1%   43770458 ± 3%  TOTAL numa-numastat.node1.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      5.57 ±11%    +142.5%      13.50 ±22%  brickland3/vm-scalability/300s-lru-file-mmap-read
     20.89 ± 2%     +85.3%      38.71 ± 7%  brickland3/vm-scalability/300s-lru-file-readonce
     26.45 ± 4%     +97.4%      52.21 ±11%  TOTAL perf-profile.cpu-cycles._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  22081863 ± 1%     -29.2%   15627383 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12674153 ± 5%    +205.0%   38656866 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
  34756017 ± 3%     +56.2%   54284249 ± 4%  TOTAL numa-numastat.node2.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   3835243 ± 1%     -54.0%    1766055 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
   3835243 ± 1%     -54.0%    1766055 ± 4%  TOTAL cpuidle.C1-IVT-4S.usage

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  10024520 ± 6%    +121.8%   22237848 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  10024520 ± 6%    +121.8%   22237848 ± 2%  TOTAL numa-vmstat.node0.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  10030708 ± 6%    +121.7%   22241976 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  10030708 ± 6%    +121.7%   22241976 ± 2%  TOTAL numa-vmstat.node0.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  16713245 ± 2%     -50.6%    8251655 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8357488 ± 5%    +133.0%   19470522 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  25070733 ± 3%     +10.6%   27722177 ± 2%  TOTAL numa-vmstat.node1.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  14762027 ± 3%     -28.5%   10559627 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12837395 ± 5%    +176.8%   35529663 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  27599422 ± 4%     +67.0%   46089291 ± 4%  TOTAL numa-numastat.node3.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  14597660 ± 4%     -28.9%   10375013 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8755827 ± 6%    +175.9%   24156475 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
  23353488 ± 5%     +47.9%   34531489 ± 4%  TOTAL numa-vmstat.node2.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.90 ± 2%     -45.2%       1.04 ±16%  brickland3/vm-scalability/300s-lru-file-readonce
      1.90 ± 2%     -45.2%       1.04 ±16%  TOTAL turbostat.%c1

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   9870153 ± 2%     -29.6%    6943700 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8877090 ± 6%    +152.5%   22416811 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  18747244 ± 4%     +56.6%   29360512 ± 4%  TOTAL numa-vmstat.node3.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      6004 ±26%     -36.2%       3829 ±14%  brickland3/vm-scalability/300s-lru-file-readonce
      6004 ±26%     -36.2%       3829 ±14%  TOTAL numa-meminfo.node3.Mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    678225 ± 1%     -48.8%     347400 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
    678225 ± 1%     -48.8%     347400 ± 5%  TOTAL numa-vmstat.node0.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   7744496 ± 2%     +85.2%   14345340 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
   7744496 ± 2%     +85.2%   14345340 ± 0%  TOTAL softirqs.TIMER

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     29388 ± 0%     -44.4%      16352 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
     29388 ± 0%     -44.4%      16352 ± 3%  TOTAL softirqs.HRTIMER

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1500 ±29%     -38.1%        928 ±12%  brickland3/vm-scalability/300s-lru-file-readonce
      1500 ±29%     -38.1%        928 ±12%  TOTAL numa-vmstat.node3.nr_mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     28488 ± 2%     -52.2%      13618 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
     28548 ± 0%     -28.1%      20530 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
     57037 ± 1%     -40.1%      34149 ± 1%  TOTAL proc-vmstat.allocstall

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      2916 ±12%     -54.4%       1331 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1664 ± 2%     +25.5%       2089 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
      4580 ± 9%     -25.3%       3420 ± 2%  TOTAL proc-vmstat.pageoutrun

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       267 ± 1%     +71.7%        458 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
       267 ± 1%     +71.7%        458 ± 0%  TOTAL uptime.boot

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1832198 ± 0%     -47.5%     962411 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   2535964 ± 0%     -31.9%    1726203 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   4368163 ± 0%     -38.4%    2688615 ± 1%  TOTAL proc-vmstat.pgsteal_direct_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   9.1e+08 ± 0%     -40.7%  5.398e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   9.1e+08 ± 0%     -40.7%  5.398e+08 ± 1%  TOTAL proc-vmstat.pgfree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 9.078e+08 ± 0%     -40.7%  5.386e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 9.078e+08 ± 0%     -40.7%  5.386e+08 ± 1%  TOTAL proc-vmstat.pgalloc_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   3787073 ± 2%     -43.4%    2143931 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   2536168 ± 0%     -31.9%    1726413 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   6323242 ± 1%     -38.8%    3870345 ± 2%  TOTAL proc-vmstat.pgscan_direct_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 7.411e+08 ± 0%     -47.4%  3.898e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 9.256e+08 ± 0%     -26.3%  6.822e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 1.667e+09 ± 0%     -35.7%  1.072e+09 ± 1%  TOTAL proc-vmstat.pgsteal_direct_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      2.94 ± 4%     +56.4%       4.60 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2.94 ± 4%     +56.4%       4.60 ± 9%  TOTAL perf-profile.cpu-cycles.mutex_spin_on_owner.__mutex_lock_slowpath.mutex_lock.rmap_walk.page_referenced

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       511 ± 4%     -35.2%        331 ± 9%  brickland3/vm-scalability/300s-lru-file-readonce
       511 ± 4%     -35.2%        331 ± 9%  TOTAL numa-vmstat.node2.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    773646 ± 1%     -35.7%     497235 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    773646 ± 1%     -35.7%     497235 ± 1%  TOTAL numa-meminfo.node1.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    193034 ± 1%     -35.3%     124819 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    193034 ± 1%     -35.3%     124819 ± 1%  TOTAL numa-vmstat.node1.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     24.83 ± 3%     -27.1%      18.11 ±10%  brickland3/vm-scalability/300s-lru-file-mmap-read
     42.78 ± 1%     -41.6%      25.00 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
     67.61 ± 2%     -36.2%      43.11 ± 5%  TOTAL turbostat.%c6

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.449e+09 ± 1%     -42.2%  8.376e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 9.257e+08 ± 0%     -26.3%  6.823e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 2.375e+09 ± 0%     -36.0%   1.52e+09 ± 1%  TOTAL proc-vmstat.pgscan_direct_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.625e+08 ± 1%     -41.8%   94564115 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.886e+08 ± 0%     -25.0%  1.415e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 3.511e+08 ± 1%     -32.8%   2.36e+08 ± 1%  TOTAL numa-vmstat.node0.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.625e+08 ± 1%     -41.8%   94575783 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.886e+08 ± 0%     -25.0%  1.415e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 3.511e+08 ± 1%     -32.8%  2.361e+08 ± 1%  TOTAL numa-vmstat.node0.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.129e+08 ± 1%     -39.9%   67808889 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.911e+08 ± 0%     -27.9%  1.378e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  3.04e+08 ± 1%     -32.4%  2.056e+08 ± 1%  TOTAL numa-vmstat.node1.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  1.13e+08 ± 1%     -39.9%   67893910 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.911e+08 ± 0%     -27.9%  1.379e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 3.041e+08 ± 1%     -32.3%  2.058e+08 ± 1%  TOTAL numa-vmstat.node1.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     16950 ± 5%     -34.4%      11111 ± 8%  brickland3/vm-scalability/300s-lru-file-mmap-read
     16950 ± 5%     -34.4%      11111 ± 8%  TOTAL cpuidle.C3-IVT-4S.usage

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    292183 ± 1%     -34.2%     192394 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    292183 ± 1%     -34.2%     192394 ± 1%  TOTAL numa-vmstat.node3.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  72128917 ± 0%     -40.4%   42986804 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1160192 ± 3%     +39.9%    1622852 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  73289109 ± 0%     -39.1%   44609656 ± 2%  TOTAL proc-vmstat.pgfault

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1170427 ± 1%     -34.5%     766302 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1170427 ± 1%     -34.5%     766302 ± 1%  TOTAL numa-meminfo.node3.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      5.15 ± 5%     +44.3%       7.43 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
      5.15 ± 5%     +44.3%       7.43 ± 9%  TOTAL perf-profile.cpu-cycles.generic_exec_single.smp_call_function_single.smp_call_function_many.native_flush_tlb_others.flush_tlb_page

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  21722028 ±14%     -27.4%   15765961 ±13%  brickland3/vm-scalability/300s-lru-file-mmap-read
  21722028 ±14%     -27.4%   15765961 ±13%  TOTAL cpuidle.C3-IVT-4S.time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    295382 ± 1%     -32.4%     199692 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
    295382 ± 1%     -32.4%     199692 ± 2%  TOTAL numa-vmstat.node1.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1202511 ± 1%     -33.4%     800394 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1202511 ± 1%     -33.4%     800394 ± 1%  TOTAL numa-meminfo.node3.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    135163 ± 0%     -31.6%      92402 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    135163 ± 0%     -31.6%      92402 ± 1%  TOTAL slabinfo.radix_tree_node.active_slabs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    135163 ± 0%     -31.6%      92402 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    135163 ± 0%     -31.6%      92402 ± 1%  TOTAL slabinfo.radix_tree_node.num_slabs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   7569136 ± 0%     -31.6%    5174561 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7569136 ± 0%     -31.6%    5174561 ± 1%  TOTAL slabinfo.radix_tree_node.num_objs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.412e+08 ± 1%     -40.4%   84248525 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.568e+08 ± 0%     -21.7%  2.011e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  3.98e+08 ± 0%     -28.3%  2.853e+08 ± 1%  TOTAL numa-numastat.node1.local_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.412e+08 ± 1%     -40.3%   84255157 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.568e+08 ± 0%     -21.7%  2.011e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  3.98e+08 ± 0%     -28.3%  2.853e+08 ± 1%  TOTAL numa-numastat.node1.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1183239 ± 1%     -32.7%     795729 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1183239 ± 1%     -32.7%     795729 ± 3%  TOTAL numa-meminfo.node1.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.289e+08 ± 1%     -37.7%   80278779 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.916e+08 ± 0%     -24.3%  1.451e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.206e+08 ± 0%     -29.7%  2.254e+08 ± 1%  TOTAL numa-vmstat.node3.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  1.29e+08 ± 1%     -37.7%   80363047 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.917e+08 ± 0%     -24.2%  1.452e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.207e+08 ± 0%     -29.7%  2.256e+08 ± 1%  TOTAL numa-vmstat.node3.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1144645 ± 2%     -32.8%     768993 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1144645 ± 2%     -32.8%     768993 ± 2%  TOTAL numa-meminfo.node2.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   5830125 ± 3%     -17.3%    4821835 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8557468 ± 2%     -41.0%    5045624 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  14387594 ± 2%     -31.4%    9867460 ± 1%  TOTAL numa-vmstat.node0.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1093892 ± 0%     -31.4%     750954 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1093892 ± 0%     -31.4%     750954 ± 1%  TOTAL proc-vmstat.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   5785378 ± 4%     -18.1%    4737611 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8191876 ± 2%     -39.0%    4994945 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  13977254 ± 3%     -30.4%    9732556 ± 2%  TOTAL numa-vmstat.node3.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.198e+08 ± 0%     -40.8%  1.302e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.537e+08 ± 0%     -18.4%  2.069e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 4.735e+08 ± 0%     -28.8%  3.371e+08 ± 1%  TOTAL numa-numastat.node0.local_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.199e+08 ± 0%     -40.8%  1.302e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.537e+08 ± 0%     -18.4%  2.069e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 4.736e+08 ± 0%     -28.8%  3.371e+08 ± 1%  TOTAL numa-numastat.node0.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    285717 ± 2%     -32.4%     193014 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    285717 ± 2%     -32.4%     193014 ± 1%  TOTAL numa-vmstat.node2.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  23302752 ± 3%     -17.5%   19233230 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  34168164 ± 2%     -41.0%   20160695 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  57470916 ± 2%     -31.5%   39393925 ± 2%  TOTAL numa-meminfo.node0.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  22464606 ± 4%     -15.9%   18899287 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  33409387 ± 1%     -39.7%   20144973 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  55873993 ± 2%     -30.1%   39044260 ± 2%  TOTAL numa-meminfo.node1.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  22596612 ± 2%     -17.3%   18690112 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  32888379 ± 1%     -40.5%   19557960 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  55484992 ± 1%     -31.1%   38248073 ± 1%  TOTAL proc-vmstat.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1219306 ± 1%     -31.9%     830492 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1219306 ± 1%     -31.9%     830492 ± 3%  TOTAL numa-meminfo.node1.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1176783 ± 2%     -31.8%     801979 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1176783 ± 2%     -31.8%     801979 ± 1%  TOTAL numa-meminfo.node2.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   7255253 ± 0%     -30.2%    5061846 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7255253 ± 0%     -30.2%    5061846 ± 1%  TOTAL slabinfo.radix_tree_node.active_objs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   4354231 ± 0%     -30.7%    3016317 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   4354231 ± 0%     -30.7%    3016317 ± 1%  TOTAL meminfo.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  89793802 ± 2%     -17.5%   74083099 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.314e+08 ± 1%     -40.3%   78477271 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 2.212e+08 ± 1%     -31.0%  1.526e+08 ± 2%  TOTAL meminfo.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.248e+08 ± 3%     -37.2%   78416804 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.918e+08 ± 0%     -21.7%  1.503e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.166e+08 ± 1%     -27.8%  2.287e+08 ± 1%  TOTAL numa-vmstat.node2.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.249e+08 ± 3%     -37.1%   78501114 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.919e+08 ± 0%     -21.6%  1.504e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.168e+08 ± 1%     -27.8%  2.289e+08 ± 1%  TOTAL numa-vmstat.node2.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  76888271 ± 2%     -14.1%   66010304 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.305e+08 ± 1%     -41.3%   76594029 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 2.074e+08 ± 1%     -31.2%  1.426e+08 ± 1%  TOTAL vmstat.memory.free

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       473 ± 7%     -23.5%        362 ±13%  brickland3/vm-scalability/300s-lru-file-readonce
       473 ± 7%     -23.5%        362 ±13%  TOTAL numa-vmstat.node3.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1559898 ± 0%     -30.0%    1092289 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    211875 ±20%     +56.4%     331443 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
   1771773 ± 2%     -19.6%    1423732 ± 1%  TOTAL softirqs.SCHED

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  23128109 ± 5%     -18.2%   18910364 ± 6%  brickland3/vm-scalability/300s-lru-file-mmap-read
  32678604 ± 2%     -38.9%   19957620 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  55806713 ± 3%     -30.4%   38867985 ± 3%  TOTAL numa-meminfo.node3.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   2363418 ± 0%     -40.7%    1402609 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   2920077 ± 0%     -12.7%    2547859 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   5283495 ± 0%     -25.2%    3950468 ± 1%  TOTAL proc-vmstat.pgalloc_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     21290 ± 5%     -29.3%      15054 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
     21290 ± 5%     -29.3%      15054 ± 4%  TOTAL meminfo.Mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    755935 ± 1%     -34.5%     495084 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1892 ± 8%     +29.4%       2447 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
    757827 ± 1%     -34.3%     497532 ± 1%  TOTAL numa-meminfo.node3.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   4496625 ± 0%     -29.7%    3159080 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   4496625 ± 0%     -29.7%    3159080 ± 1%  TOTAL meminfo.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  22503656 ± 5%     -15.3%   19052501 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  32274836 ± 3%     -37.7%   20119596 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  54778492 ± 4%     -28.5%   39172098 ± 2%  TOTAL numa-meminfo.node2.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 6.858e+08 ± 0%     -38.8%  4.199e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.026e+09 ± 0%     -17.9%  8.422e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 1.712e+09 ± 0%     -26.3%  1.262e+09 ± 1%  TOTAL proc-vmstat.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 6.858e+08 ± 0%     -38.8%  4.199e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.026e+09 ± 0%     -17.9%  8.422e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 1.712e+09 ± 0%     -26.3%  1.262e+09 ± 1%  TOTAL proc-vmstat.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   5619854 ± 4%     -15.7%    4738362 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8370497 ± 1%     -39.8%    5040037 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  13990351 ± 2%     -30.1%    9778399 ± 1%  TOTAL numa-vmstat.node1.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   8022752 ± 5%     +38.1%   11079276 ± 8%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8022752 ± 5%     +38.1%   11079276 ± 8%  TOTAL time.maximum_resident_set_size

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      7933 ± 6%     -30.8%       5487 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
      7933 ± 6%     -30.8%       5487 ± 9%  TOTAL cpuidle.C1E-IVT-4S.usage

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    188691 ± 1%     -34.1%     124329 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
       471 ± 7%     +29.9%        612 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
    189162 ± 1%     -34.0%     124941 ± 1%  TOTAL numa-vmstat.node3.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    138389 ± 9%     -32.5%      93430 ±18%  brickland3/vm-scalability/300s-lru-file-readonce
    138389 ± 9%     -32.5%      93430 ±18%  TOTAL proc-vmstat.pgactivate

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   5626193 ± 5%     -15.1%    4778409 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8083183 ± 3%     -37.7%    5035365 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  13709376 ± 4%     -28.4%    9813775 ± 2%  TOTAL numa-vmstat.node2.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.662e+08 ± 0%     -36.8%   1.05e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.575e+08 ± 0%     -17.4%  2.128e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 4.237e+08 ± 0%     -25.0%  3.178e+08 ± 1%  TOTAL numa-numastat.node3.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.662e+08 ± 0%     -36.8%   1.05e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.575e+08 ± 0%     -17.4%  2.128e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 4.237e+08 ± 0%     -25.0%  3.178e+08 ± 1%  TOTAL numa-numastat.node3.local_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
        66 ± 1%     +35.3%         89 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
        66 ± 1%     +35.3%         89 ± 0%  TOTAL vmstat.procs.r

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.585e+08 ± 1%     -36.6%  1.005e+08 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.578e+08 ± 0%     -14.1%  2.215e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 4.163e+08 ± 0%     -22.7%  3.219e+08 ± 1%  TOTAL numa-numastat.node2.local_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.585e+08 ± 1%     -36.6%  1.005e+08 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.578e+08 ± 0%     -14.1%  2.215e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 4.163e+08 ± 0%     -22.7%   3.22e+08 ± 1%  TOTAL numa-numastat.node2.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    759826 ± 1%     -34.8%     495074 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2057 ± 3%     +17.8%       2423 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
    761884 ± 1%     -34.7%     497497 ± 1%  TOTAL numa-meminfo.node2.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    759680 ± 1%     -34.5%     497889 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2045 ± 1%     +17.9%       2413 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
    761726 ± 1%     -34.3%     500302 ± 1%  TOTAL proc-vmstat.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    741326 ± 1%     -32.7%     499221 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2029 ±10%     +19.4%       2422 ± 8%  brickland3/vm-scalability/300s-lru-file-readonce
    743355 ± 1%     -32.5%     501644 ± 1%  TOTAL numa-meminfo.node0.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    184996 ± 1%     -32.3%     125326 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
       505 ±10%     +19.5%        603 ± 8%  brickland3/vm-scalability/300s-lru-file-readonce
    185501 ± 1%     -32.1%     125930 ± 1%  TOTAL numa-vmstat.node0.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    189703 ± 0%     -34.5%     124276 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
       516 ± 3%     +17.3%        605 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
    190219 ± 0%     -34.3%     124882 ± 1%  TOTAL numa-vmstat.node2.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   3022193 ± 1%     -33.6%    2007941 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      8169 ± 1%     +18.3%       9660 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   3030362 ± 1%     -33.4%    2017601 ± 1%  TOTAL meminfo.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      4969 ±12%     -23.7%       3792 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
      4969 ±12%     -23.7%       3792 ± 3%  TOTAL proc-vmstat.nr_mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    885165 ± 2%     -23.0%     681778 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
    885165 ± 2%     -23.0%     681778 ± 3%  TOTAL numa-meminfo.node0.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    221025 ± 2%     -22.6%     171160 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
    221025 ± 2%     -22.6%     171160 ± 3%  TOTAL numa-vmstat.node0.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1970 ± 3%     -22.7%       1522 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
      1970 ± 3%     -22.7%       1522 ± 3%  TOTAL proc-vmstat.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    927295 ± 2%     -22.1%     722698 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
    927295 ± 2%     -22.1%     722698 ± 3%  TOTAL numa-meminfo.node0.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1862 ± 9%     +26.1%       2349 ± 8%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1862 ± 9%     +26.1%       2349 ± 8%  TOTAL proc-vmstat.workingset_refault

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      9529 ± 3%     +29.1%      12300 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
      9529 ± 3%     +29.1%      12300 ± 4%  TOTAL proc-vmstat.numa_hint_faults_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    274122 ± 1%     -13.2%     237877 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
     10426 ± 3%     +36.4%      14223 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
    284549 ± 1%     -11.4%     252101 ± 2%  TOTAL proc-vmstat.numa_pte_updates

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1070 ± 3%     -19.7%        859 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
      1070 ± 3%     -19.7%        859 ± 5%  TOTAL numa-vmstat.node2.nr_mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     21.64 ± 2%     +26.3%      27.33 ±10%  brickland3/vm-scalability/300s-lru-file-mmap-read
     21.64 ± 2%     +26.3%      27.33 ±10%  TOTAL perf-profile.cpu-cycles.osq_lock.__mutex_lock_slowpath.mutex_lock.rmap_walk.page_referenced

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      4577 ± 7%     -18.7%       3723 ± 8%  brickland3/vm-scalability/300s-lru-file-readonce
      4577 ± 7%     -18.7%       3723 ± 8%  TOTAL numa-meminfo.node0.Mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    272410 ± 1%     -12.8%     237556 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      9802 ± 3%     +30.2%      12761 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
    282212 ± 1%     -11.3%     250317 ± 1%  TOTAL proc-vmstat.numa_hint_faults

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   2299504 ± 0%     -19.8%    1844020 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
    106698 ± 1%      +9.9%     117221 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
   2406202 ± 0%     -18.5%    1961242 ± 2%  TOTAL cpuidle.C6-IVT-4S.usage

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.697e+10 ± 2%     -15.3%  1.438e+10 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.697e+10 ± 2%     -15.3%  1.438e+10 ± 3%  TOTAL cpuidle.C6-IVT-4S.time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       467 ± 4%     -13.9%        402 ± 7%  brickland3/vm-scalability/300s-lru-file-readonce
       467 ± 4%     -13.9%        402 ± 7%  TOTAL numa-vmstat.node1.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  23927853 ± 0%     +14.7%   27436111 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  23927853 ± 0%     +14.7%   27436111 ± 0%  TOTAL numa-vmstat.node0.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  23897222 ± 0%     +14.5%   27361810 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  23897222 ± 0%     +14.5%   27361810 ± 0%  TOTAL numa-vmstat.node0.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  95772318 ± 0%     +14.6%  1.098e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  95772318 ± 0%     +14.6%  1.098e+08 ± 0%  TOTAL numa-meminfo.node0.FilePages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      6248 ±42%     -41.2%       3672 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
      6248 ±42%     -41.2%       3672 ± 2%  TOTAL numa-meminfo.node1.Mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  95649623 ± 0%     +14.4%  1.095e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  95649623 ± 0%     +14.4%  1.095e+08 ± 0%  TOTAL numa-meminfo.node0.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  95650669 ± 0%     +14.4%  1.095e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  95650669 ± 0%     +14.4%  1.095e+08 ± 0%  TOTAL numa-meminfo.node0.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97759674 ± 0%     +14.3%  1.118e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97759674 ± 0%     +14.3%  1.118e+08 ± 0%  TOTAL numa-meminfo.node0.MemUsed

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1637 ±46%     -43.7%        922 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
      1637 ±46%     -43.7%        922 ± 3%  TOTAL numa-vmstat.node1.nr_mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24013796 ± 0%     +14.1%   27392772 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24013796 ± 0%     +14.1%   27392772 ± 0%  TOTAL numa-vmstat.node1.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 3.899e+08 ± 0%     +13.8%  4.437e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.899e+08 ± 0%     +13.8%  4.437e+08 ± 0%  TOTAL vmstat.memory.cache

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96127542 ± 0%     +14.0%  1.096e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  96127542 ± 0%     +14.0%  1.096e+08 ± 0%  TOTAL numa-meminfo.node1.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96339092 ± 0%     +14.0%  1.098e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  96339092 ± 0%     +14.0%  1.098e+08 ± 0%  TOTAL numa-meminfo.node1.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96655792 ± 0%     +13.7%  1.099e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  96655792 ± 0%     +13.7%  1.099e+08 ± 0%  TOTAL numa-meminfo.node1.FilePages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 3.878e+08 ± 0%     +13.6%  4.406e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.878e+08 ± 0%     +13.6%  4.406e+08 ± 0%  TOTAL meminfo.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 3.881e+08 ± 0%     +13.6%  4.409e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.881e+08 ± 0%     +13.6%  4.409e+08 ± 0%  TOTAL meminfo.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  3.89e+08 ± 0%     +13.6%  4.418e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  3.89e+08 ± 0%     +13.6%  4.418e+08 ± 0%  TOTAL meminfo.Cached

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96912081 ± 0%     +13.7%  1.102e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  96912081 ± 0%     +13.7%  1.102e+08 ± 0%  TOTAL proc-vmstat.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97216175 ± 0%     +13.7%  1.105e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97216175 ± 0%     +13.7%  1.105e+08 ± 0%  TOTAL proc-vmstat.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24145923 ± 0%     +13.8%   27478127 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24145923 ± 0%     +13.8%   27478127 ± 0%  TOTAL numa-vmstat.node1.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     11086 ± 1%     +12.7%      12491 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
     11086 ± 1%     +12.7%      12491 ± 1%  TOTAL proc-vmstat.nr_anon_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     44374 ± 1%     +12.6%      49965 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
     44374 ± 1%     +12.6%      49965 ± 1%  TOTAL meminfo.AnonPages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  98706688 ± 0%     +13.4%   1.12e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  98706688 ± 0%     +13.4%   1.12e+08 ± 0%  TOTAL numa-meminfo.node1.MemUsed

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      9.40 ± 2%     +12.3%      10.56 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
      9.40 ± 2%     +12.3%      10.56 ± 9%  TOTAL perf-profile.cpu-cycles.osq_lock.__mutex_lock_slowpath.mutex_lock.rmap_walk.try_to_unmap

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24256827 ± 0%     +13.2%   27466497 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24256827 ± 0%     +13.2%   27466497 ± 0%  TOTAL numa-vmstat.node3.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     15468 ± 1%     +12.3%      17364 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
     15468 ± 1%     +12.3%      17364 ± 5%  TOTAL numa-meminfo.node2.AnonPages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24358756 ± 0%     +13.1%   27549191 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24358756 ± 0%     +13.1%   27549191 ± 0%  TOTAL numa-vmstat.node3.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      3869 ± 1%     +12.2%       4340 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
      3869 ± 1%     +12.2%       4340 ± 5%  TOTAL numa-vmstat.node2.nr_anon_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97523828 ± 0%     +13.0%  1.102e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97523828 ± 0%     +13.0%  1.102e+08 ± 0%  TOTAL numa-meminfo.node3.FilePages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97115503 ± 0%     +13.2%  1.099e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97115503 ± 0%     +13.2%  1.099e+08 ± 0%  TOTAL numa-meminfo.node3.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97126862 ± 0%     +13.1%  1.099e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97126862 ± 0%     +13.1%  1.099e+08 ± 0%  TOTAL numa-meminfo.node3.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  99437010 ± 0%     +12.8%  1.122e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  99437010 ± 0%     +12.8%  1.122e+08 ± 0%  TOTAL numa-meminfo.node3.MemUsed

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97930450 ± 1%     +12.4%    1.1e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97930450 ± 1%     +12.4%    1.1e+08 ± 0%  TOTAL numa-meminfo.node2.FilePages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  99841238 ± 1%     +12.2%   1.12e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  99841238 ± 1%     +12.2%   1.12e+08 ± 0%  TOTAL numa-meminfo.node2.MemUsed

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       480 ± 4%     -12.8%        418 ±11%  brickland3/vm-scalability/300s-lru-file-readonce
       480 ± 4%     -12.8%        418 ±11%  TOTAL numa-vmstat.node0.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24468312 ± 1%     +12.4%   27504476 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24468312 ± 1%     +12.4%   27504476 ± 0%  TOTAL numa-vmstat.node2.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97818315 ± 1%     +12.3%  1.098e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97818315 ± 1%     +12.3%  1.098e+08 ± 0%  TOTAL numa-meminfo.node2.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97767272 ± 1%     +12.3%  1.098e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97767272 ± 1%     +12.3%  1.098e+08 ± 0%  TOTAL numa-meminfo.node2.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24427538 ± 1%     +12.3%   27435973 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24427538 ± 1%     +12.3%   27435973 ± 0%  TOTAL numa-vmstat.node2.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     22711 ± 3%      -9.1%      20641 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
     13218 ± 3%      +8.2%      14303 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
     35929 ± 3%      -2.7%      34945 ± 2%  TOTAL uptime.idle

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     25195 ± 0%      +6.4%      26807 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
     14015 ± 1%    +151.8%      35296 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
     39211 ± 0%     +58.4%      62103 ± 0%  TOTAL time.system_time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24713087 ± 0%     -41.3%   14517140 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
      4189 ± 5%     -26.0%       3099 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
  24717276 ± 0%     -41.3%   14520239 ± 3%  TOTAL time.voluntary_context_switches

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    539861 ± 1%     -23.3%     413999 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    169833 ± 1%     +68.8%     286660 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
    709694 ± 1%      -1.3%     700660 ± 1%  TOTAL time.involuntary_context_switches

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       403 ± 0%      -4.7%        384 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
       213 ± 0%     +89.5%        404 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
       616 ± 0%     +27.9%        788 ± 0%  TOTAL time.elapsed_time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    126881 ± 0%     -37.6%      79236 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2749 ± 1%     -23.8%       2095 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
    129631 ± 0%     -37.3%      81331 ± 2%  TOTAL vmstat.system.cs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1243406 ± 1%     -37.0%     783419 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    114271 ± 0%     -26.0%      84537 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
   1357678 ± 1%     -36.1%     867957 ± 1%  TOTAL vmstat.system.in

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1620.48 ± 0%     -42.3%     935.48 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
     84.49 ± 0%     +15.3%      97.39 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
   1704.96 ± 0%     -39.4%    1032.87 ± 1%  TOTAL time.user_time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  70740442 ± 1%     -41.2%   41615768 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
    121780 ± 0%      +2.0%     124226 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  70862223 ± 1%     -41.1%   41739994 ± 2%  TOTAL time.minor_page_faults

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      6645 ± 0%      +8.5%       7212 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
      6607 ± 0%     +32.5%       8754 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
     13253 ± 0%     +20.5%      15967 ± 0%  TOTAL time.percent_of_cpu_this_job_got

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     58.13 ± 0%      +7.1%      62.24 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
     55.30 ± 0%     +33.7%      73.95 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
    113.43 ± 0%     +20.1%     136.19 ± 0%  TOTAL turbostat.%c0

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       334 ± 1%      +5.4%        353 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
       280 ± 0%     +19.6%        335 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
       615 ± 1%     +11.9%        688 ± 0%  TOTAL turbostat.Cor_W

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     89.53 ± 0%     -10.2%      80.43 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
     95.97 ± 0%     -11.2%      85.22 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
    185.50 ± 0%     -10.7%     165.64 ± 0%  TOTAL turbostat.RAM_W

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       407 ± 1%      +4.2%        424 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
       353 ± 0%     +15.3%        407 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
       760 ± 0%      +9.3%        831 ± 0%  TOTAL turbostat.Pkg_W


Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.

Thanks,
Fengguang

[-- Attachment #2: reproduce --]
[-- Type: text/plain, Size: 29759 bytes --]

echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu10/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu100/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu101/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu102/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu103/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu104/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu105/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu106/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu107/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu108/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu109/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu11/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu110/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu111/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu112/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu113/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu114/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu115/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu116/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu117/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu118/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu119/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu12/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu13/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu14/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu15/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu16/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu17/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu18/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu19/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu20/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu21/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu22/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu23/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu24/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu25/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu26/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu27/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu28/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu29/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu30/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu31/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu32/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu33/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu34/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu35/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu36/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu37/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu38/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu39/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu4/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu40/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu41/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu42/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu43/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu44/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu45/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu46/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu47/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu48/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu49/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu5/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu50/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu51/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu52/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu53/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu54/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu55/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu56/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu57/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu58/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu59/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu6/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu60/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu61/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu62/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu63/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu64/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu65/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu66/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu67/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu68/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu69/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu7/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu70/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu71/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu72/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu73/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu74/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu75/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu76/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu77/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu78/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu79/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu8/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu80/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu81/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu82/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu83/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu84/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu85/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu86/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu87/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu88/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu89/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu9/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu90/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu91/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu92/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu93/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu94/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu95/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu96/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu97/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu98/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu99/cpufreq/scaling_governor
mount -t tmpfs -o size=100% vm-scalability-tmp /tmp/vm-scalability-tmp
truncate -s 540952072192 /tmp/vm-scalability.img
mkfs.xfs -q /tmp/vm-scalability.img
mount -o loop /tmp/vm-scalability.img /tmp/vm-scalability
./case-lru-file-mmap-read
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-1 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-1 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-2 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-2 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-3 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-3 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-4 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-5 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-4 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-5 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-6 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-6 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-7 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-7 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-8 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-9 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-8 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-9 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-10 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-11 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-10 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-11 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-12 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-13 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-12 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-13 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-14 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-15 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-14 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-15 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-16 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-16 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-17 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-18 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-17 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-18 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-19 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-19 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-20 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-20 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-21 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-21 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-22 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-22 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-23 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-23 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-24 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-24 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-25 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-25 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-26 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-26 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-27 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-27 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-28 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-28 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-29 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-29 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-30 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-30 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-31 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-31 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-32 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-32 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-33 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-33 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-34 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-35 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-34 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-35 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-36 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-36 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-37 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-37 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-38 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-38 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-39 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-39 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-40 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-41 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-40 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-41 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-42 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-42 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-43 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-43 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-44 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-44 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-45 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-45 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-46 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-47 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-46 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-47 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-48 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-48 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-49 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-49 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-50 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-50 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-51 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-51 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-52 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-52 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-53 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-53 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-54 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-54 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-55 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-55 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-56 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-56 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-57 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-57 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-58 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-58 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-59 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-60 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-59 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-60 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-61 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-61 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-62 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-62 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-63 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-64 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-63 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-64 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-65 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-66 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-65 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-66 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-67 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-67 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-68 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-68 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-69 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-69 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-70 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-70 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-71 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-71 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-72 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-73 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-72 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-74 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-73 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-75 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-74 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-75 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-76 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-76 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-77 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-78 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-77 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-78 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-79 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-79 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-80 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-80 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-81 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-81 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-82 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-82 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-83 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-84 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-83 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-84 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-85 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-86 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-85 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-86 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-87 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-88 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-87 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-89 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-88 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-89 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-90 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-91 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-90 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-91 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-92 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-92 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-93 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-93 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-94 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-95 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-94 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-95 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-96 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-96 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-97 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-97 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-98 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-98 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-99 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-100 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-99 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-100 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-101 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-101 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-102 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-102 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-103 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-104 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-103 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-105 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-104 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-105 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-106 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-106 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-107 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-108 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-107 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-108 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-109 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-109 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-110 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-110 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-111 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-112 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-111 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-112 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-113 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-113 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-114 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-114 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-115 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-115 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-116 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-116 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-117 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-118 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-117 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-118 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-119 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-119 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-120 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-120 --readonly 36650387592
umount /tmp/vm-scalability-tmp
umount /tmp/vm-scalability
rm /tmp/vm-scalability.img

[-- Attachment #3: Type: text/plain, Size: 85 bytes --]

_______________________________________________
LKP mailing list
LKP@linux.intel.com

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
@ 2014-08-19  4:32 ` Fengguang Wu
  0 siblings, 0 replies; 12+ messages in thread
From: Fengguang Wu @ 2014-08-19  4:32 UTC (permalink / raw)
  To: lkp

[-- Attachment #1: Type: text/plain, Size: 68494 bytes --]

Hi Mel,

We noticed the below vm-scalability performance/power regressions on
commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
fields into read-only, page alloc, statistics and page reclaim lines").

24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
---------------  -------------------------  ---------------------------
     %stddev        %change               %stddev
            \          |                 /
      9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
     12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
  36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1138 ±46%     -94.7%         60 ±23%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1138 ±46%     -94.7%         60 ±23%  TOTAL proc-vmstat.compact_success

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      0.95 ±20%     -87.5%       0.12 ±12%  brickland3/vm-scalability/300s-lru-file-readonce
      0.95 ±20%     -87.5%       0.12 ±12%  TOTAL perf-profile.cpu-cycles.read_hpet.ktime_get.clockevents_program_event.tick_program_event.hrtimer_interrupt

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.40 ±17%     -96.1%       0.05 ±27%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2.70 ±36%     -95.0%       0.13 ± 8%  brickland3/vm-scalability/300s-lru-file-readonce
      4.10 ±29%     -95.4%       0.19 ±14%  TOTAL perf-profile.cpu-cycles._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.lru_cache_add.add_to_page_cache_lru

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.46 ±12%     -89.1%       0.16 ±18%  brickland3/vm-scalability/300s-lru-file-readonce
      1.46 ±12%     -89.1%       0.16 ±18%  TOTAL perf-profile.cpu-cycles.read_hpet.ktime_get.tick_sched_timer.__run_hrtimer.hrtimer_interrupt

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.18 ±23%     -82.5%       0.21 ±22%  brickland3/vm-scalability/300s-lru-file-readonce
      1.18 ±23%     -82.5%       0.21 ±22%  TOTAL perf-profile.cpu-cycles.read_hpet.ktime_get_update_offsets_now.hrtimer_interrupt.local_apic_timer_interrupt.smp_apic_timer_interrupt

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       763 ±31%     -84.5%        118 ±11%  brickland3/vm-scalability/300s-lru-file-mmap-read
       763 ±31%     -84.5%        118 ±11%  TOTAL proc-vmstat.kswapd_low_wmark_hit_quickly

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.59 ±12%     -85.1%       0.24 ±19%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1.59 ±12%     -85.1%       0.24 ±19%  TOTAL perf-profile.cpu-cycles.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc.__do_page_cache_readahead

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     19972 ±45%     -71.3%       5725 ±16%  brickland3/vm-scalability/300s-lru-file-mmap-read
     19972 ±45%     -71.3%       5725 ±16%  TOTAL proc-vmstat.pgmigrate_success

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       543 ±28%     -62.1%        205 ±14%  brickland3/vm-scalability/300s-lru-file-mmap-read
        47 ±16%    +828.5%        443 ± 7%  brickland3/vm-scalability/300s-lru-file-readonce
       590 ±27%     +10.0%        649 ± 9%  TOTAL proc-vmstat.kswapd_high_wmark_hit_quickly

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    518176 ± 5%    -100.0%          0 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
    687557 ± 0%     -53.8%     317522 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
   1205734 ± 2%     -73.7%     317522 ± 2%  TOTAL numa-vmstat.node1.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      2.61 ± 5%     -52.7%       1.24 ±25%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1.29 ±10%     -91.0%       0.12 ±15%  brickland3/vm-scalability/300s-lru-file-readonce
      3.91 ± 7%     -65.4%       1.35 ±24%  TOTAL perf-profile.cpu-cycles.read_hpet.ktime_get.sched_clock_tick.scheduler_tick.update_process_times

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.652e+08 ± 5%     -47.7%   86319616 ± 7%  brickland3/vm-scalability/300s-lru-file-mmap-read
  16732078 ± 7%    +894.1%  1.663e+08 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
 1.819e+08 ± 5%     +38.9%  2.527e+08 ± 5%  TOTAL proc-vmstat.pgscan_kswapd_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1565973 ± 2%    -100.0%          0 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
   2720970 ± 0%     -47.2%    1437269 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   4286943 ± 1%     -66.5%    1437269 ± 1%  TOTAL proc-vmstat.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  34449272 ± 5%     -48.1%   17892413 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  16725742 ± 7%    +894.4%  1.663e+08 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  51175014 ± 5%    +260.0%  1.842e+08 ± 3%  TOTAL proc-vmstat.pgsteal_kswapd_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    579964 ± 7%    -100.0%          0 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
    689893 ± 0%     -44.1%     385678 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
   1269857 ± 3%     -69.6%     385678 ± 2%  TOTAL numa-vmstat.node3.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     27.49 ± 3%     -45.8%      14.91 ±17%  brickland3/vm-scalability/300s-lru-file-readonce
     27.49 ± 3%     -45.8%      14.91 ±17%  TOTAL turbostat.%pc2

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    178797 ± 8%     -42.0%     103621 ± 6%  brickland3/vm-scalability/300s-lru-file-mmap-read
     47255 ±12%    +930.2%     486833 ±10%  brickland3/vm-scalability/300s-lru-file-readonce
    226052 ± 9%    +161.2%     590454 ± 9%  TOTAL proc-vmstat.pgsteal_kswapd_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    468136 ± 7%    -100.0%          0 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
    693085 ± 1%     -39.9%     416491 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
   1161222 ± 4%     -64.1%     416491 ± 2%  TOTAL numa-vmstat.node2.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   4318208 ± 1%     -99.7%      11627 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
   6822297 ± 0%     -40.1%    4084812 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  11140505 ± 1%     -63.2%    4096440 ± 1%  TOTAL proc-vmstat.slabs_scanned

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    395581 ±14%     -32.1%     268687 ±15%  brickland3/vm-scalability/300s-lru-file-mmap-read
     47338 ±12%    +928.6%     486919 ±10%  brickland3/vm-scalability/300s-lru-file-readonce
    442920 ±14%     +70.6%     755607 ±12%  TOTAL proc-vmstat.pgscan_kswapd_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      4.37 ± 7%     -61.2%       1.70 ±24%  brickland3/vm-scalability/300s-lru-file-mmap-read
      6.75 ± 5%     -64.9%       2.37 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
     11.13 ± 6%     -63.5%       4.06 ±14%  TOTAL perf-profile.cpu-cycles.memset.mpage_readpages.xfs_vm_readpages.__do_page_cache_readahead.ondemand_readahead

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.37 ± 8%     -62.2%       0.52 ±23%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1.59 ± 5%     -65.7%       0.55 ± 7%  brickland3/vm-scalability/300s-lru-file-readonce
      2.96 ± 6%     -64.1%       1.06 ±15%  TOTAL perf-profile.cpu-cycles.do_mpage_readpage.mpage_readpages.xfs_vm_readpages.__do_page_cache_readahead.ondemand_readahead

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1911 ±12%     -62.2%        722 ±25%  brickland3/vm-scalability/300s-lru-file-readonce
      1911 ±12%     -62.2%        722 ±25%  TOTAL proc-vmstat.pgscan_direct_throttle

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      3.37 ± 9%    +232.0%      11.18 ±25%  brickland3/vm-scalability/300s-lru-file-mmap-read
     26.57 ± 4%     +73.3%      46.04 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
     29.93 ± 5%     +91.2%      57.22 ± 6%  TOTAL perf-profile.cpu-cycles._raw_spin_lock.free_pcppages_bulk.free_hot_cold_page.free_hot_cold_page_list.shrink_page_list

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    292141 ± 4%    +165.3%     775139 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
    292141 ± 4%    +165.3%     775139 ± 4%  TOTAL softirqs.RCU

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  61899502 ± 1%     -53.7%   28663774 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11490237 ± 7%    +204.7%   35005345 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  73389740 ± 2%     -13.2%   63669120 ± 2%  TOTAL numa-numastat.node3.other_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  61894563 ± 1%     -53.7%   28660170 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11487385 ± 7%    +204.7%   35000524 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  73381949 ± 2%     -13.2%   63660694 ± 2%  TOTAL numa-numastat.node3.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  37543259 ± 2%     -54.3%   17139747 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7768241 ± 8%    +181.3%   21853872 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  45311501 ± 3%     -13.9%   38993620 ± 2%  TOTAL numa-vmstat.node3.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  37628483 ± 2%     -54.2%   17224017 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7851378 ± 8%    +179.4%   21938991 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  45479862 ± 3%     -13.9%   39163008 ± 2%  TOTAL numa-vmstat.node3.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.599e+08 ± 1%     -49.9%   80094895 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11422480 ± 2%    +194.8%   33675826 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
 1.713e+08 ± 1%     -33.6%  1.138e+08 ± 2%  TOTAL numa-numastat.node0.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96263262 ± 2%     -53.1%   45191652 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7622035 ± 2%    +178.4%   21217541 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
 103885297 ± 2%     -36.1%   66409193 ± 3%  TOTAL numa-vmstat.node0.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  85346466 ± 2%     -40.8%   50525376 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12056858 ± 3%    +215.0%   37973604 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
  97403324 ± 2%      -9.1%   88498980 ± 4%  TOTAL numa-numastat.node1.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  85350581 ± 2%     -40.8%   50532008 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12061005 ± 3%    +214.9%   37979653 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
  97411587 ± 2%      -9.1%   88511661 ± 4%  TOTAL numa-numastat.node1.other_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  53719356 ± 3%     -43.9%   30133382 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8183737 ± 3%    +192.7%   23954479 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
  61903093 ± 3%     -12.6%   54087861 ± 5%  TOTAL numa-vmstat.node1.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  53802537 ± 3%     -43.8%   30218405 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8266017 ± 3%    +190.8%   24038964 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
  62068554 ± 3%     -12.6%   54257370 ± 5%  TOTAL numa-vmstat.node1.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  14319949 ± 7%    +146.5%   35300490 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  14319949 ± 7%    +146.5%   35300490 ± 2%  TOTAL numa-numastat.node0.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  14327584 ± 7%    +146.4%   35303342 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  14327584 ± 7%    +146.4%   35303342 ± 2%  TOTAL numa-numastat.node0.other_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  41804307 ± 1%     -54.4%   19081241 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7633990 ± 8%    +151.7%   19214179 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  49438297 ± 2%     -22.5%   38295420 ± 3%  TOTAL numa-vmstat.node2.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.228e+08 ± 1%     -46.6%   1.19e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
  49195780 ± 4%    +182.4%  1.389e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  2.72e+08 ± 2%      -5.2%  2.579e+08 ± 1%  TOTAL proc-vmstat.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.228e+08 ± 1%     -46.6%   1.19e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
  49196215 ± 4%    +182.3%  1.389e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  2.72e+08 ± 2%      -5.2%  2.579e+08 ± 1%  TOTAL proc-vmstat.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.228e+08 ± 1%     -46.6%   1.19e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
  49217751 ± 4%    +182.3%  1.389e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
 2.721e+08 ± 2%      -5.2%   2.58e+08 ± 1%  TOTAL proc-vmstat.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  41884224 ± 1%     -54.2%   19165553 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7719489 ± 8%    +150.0%   19299287 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  49603713 ± 2%     -22.5%   38464840 ± 3%  TOTAL numa-vmstat.node2.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  69206605 ± 3%     -51.7%   33399613 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11331586 ± 8%    +170.3%   30629759 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  80538191 ± 4%     -20.5%   64029372 ± 3%  TOTAL numa-numastat.node2.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  69211755 ± 3%     -51.7%   33403393 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
  11338920 ± 8%    +170.2%   30634435 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  80550676 ± 4%     -20.5%   64037828 ± 3%  TOTAL numa-numastat.node2.other_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  26095298 ± 2%     -51.2%   12728144 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12262186 ± 6%    +153.2%   31042313 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  38357485 ± 4%     +14.1%   43770458 ± 3%  TOTAL numa-numastat.node1.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      5.57 ±11%    +142.5%      13.50 ±22%  brickland3/vm-scalability/300s-lru-file-mmap-read
     20.89 ± 2%     +85.3%      38.71 ± 7%  brickland3/vm-scalability/300s-lru-file-readonce
     26.45 ± 4%     +97.4%      52.21 ±11%  TOTAL perf-profile.cpu-cycles._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  22081863 ± 1%     -29.2%   15627383 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12674153 ± 5%    +205.0%   38656866 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
  34756017 ± 3%     +56.2%   54284249 ± 4%  TOTAL numa-numastat.node2.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   3835243 ± 1%     -54.0%    1766055 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
   3835243 ± 1%     -54.0%    1766055 ± 4%  TOTAL cpuidle.C1-IVT-4S.usage

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  10024520 ± 6%    +121.8%   22237848 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  10024520 ± 6%    +121.8%   22237848 ± 2%  TOTAL numa-vmstat.node0.numa_miss

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  10030708 ± 6%    +121.7%   22241976 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  10030708 ± 6%    +121.7%   22241976 ± 2%  TOTAL numa-vmstat.node0.numa_other

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  16713245 ± 2%     -50.6%    8251655 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8357488 ± 5%    +133.0%   19470522 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  25070733 ± 3%     +10.6%   27722177 ± 2%  TOTAL numa-vmstat.node1.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  14762027 ± 3%     -28.5%   10559627 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
  12837395 ± 5%    +176.8%   35529663 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  27599422 ± 4%     +67.0%   46089291 ± 4%  TOTAL numa-numastat.node3.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  14597660 ± 4%     -28.9%   10375013 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8755827 ± 6%    +175.9%   24156475 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
  23353488 ± 5%     +47.9%   34531489 ± 4%  TOTAL numa-vmstat.node2.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1.90 ± 2%     -45.2%       1.04 ±16%  brickland3/vm-scalability/300s-lru-file-readonce
      1.90 ± 2%     -45.2%       1.04 ±16%  TOTAL turbostat.%c1

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   9870153 ± 2%     -29.6%    6943700 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8877090 ± 6%    +152.5%   22416811 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  18747244 ± 4%     +56.6%   29360512 ± 4%  TOTAL numa-vmstat.node3.numa_foreign

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      6004 ±26%     -36.2%       3829 ±14%  brickland3/vm-scalability/300s-lru-file-readonce
      6004 ±26%     -36.2%       3829 ±14%  TOTAL numa-meminfo.node3.Mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    678225 ± 1%     -48.8%     347400 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
    678225 ± 1%     -48.8%     347400 ± 5%  TOTAL numa-vmstat.node0.workingset_nodereclaim

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   7744496 ± 2%     +85.2%   14345340 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
   7744496 ± 2%     +85.2%   14345340 ± 0%  TOTAL softirqs.TIMER

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     29388 ± 0%     -44.4%      16352 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
     29388 ± 0%     -44.4%      16352 ± 3%  TOTAL softirqs.HRTIMER

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1500 ±29%     -38.1%        928 ±12%  brickland3/vm-scalability/300s-lru-file-readonce
      1500 ±29%     -38.1%        928 ±12%  TOTAL numa-vmstat.node3.nr_mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     28488 ± 2%     -52.2%      13618 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
     28548 ± 0%     -28.1%      20530 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
     57037 ± 1%     -40.1%      34149 ± 1%  TOTAL proc-vmstat.allocstall

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      2916 ±12%     -54.4%       1331 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1664 ± 2%     +25.5%       2089 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
      4580 ± 9%     -25.3%       3420 ± 2%  TOTAL proc-vmstat.pageoutrun

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       267 ± 1%     +71.7%        458 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
       267 ± 1%     +71.7%        458 ± 0%  TOTAL uptime.boot

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1832198 ± 0%     -47.5%     962411 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   2535964 ± 0%     -31.9%    1726203 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   4368163 ± 0%     -38.4%    2688615 ± 1%  TOTAL proc-vmstat.pgsteal_direct_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   9.1e+08 ± 0%     -40.7%  5.398e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   9.1e+08 ± 0%     -40.7%  5.398e+08 ± 1%  TOTAL proc-vmstat.pgfree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 9.078e+08 ± 0%     -40.7%  5.386e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 9.078e+08 ± 0%     -40.7%  5.386e+08 ± 1%  TOTAL proc-vmstat.pgalloc_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   3787073 ± 2%     -43.4%    2143931 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   2536168 ± 0%     -31.9%    1726413 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   6323242 ± 1%     -38.8%    3870345 ± 2%  TOTAL proc-vmstat.pgscan_direct_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 7.411e+08 ± 0%     -47.4%  3.898e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 9.256e+08 ± 0%     -26.3%  6.822e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 1.667e+09 ± 0%     -35.7%  1.072e+09 ± 1%  TOTAL proc-vmstat.pgsteal_direct_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      2.94 ± 4%     +56.4%       4.60 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2.94 ± 4%     +56.4%       4.60 ± 9%  TOTAL perf-profile.cpu-cycles.mutex_spin_on_owner.__mutex_lock_slowpath.mutex_lock.rmap_walk.page_referenced

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       511 ± 4%     -35.2%        331 ± 9%  brickland3/vm-scalability/300s-lru-file-readonce
       511 ± 4%     -35.2%        331 ± 9%  TOTAL numa-vmstat.node2.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    773646 ± 1%     -35.7%     497235 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    773646 ± 1%     -35.7%     497235 ± 1%  TOTAL numa-meminfo.node1.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    193034 ± 1%     -35.3%     124819 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    193034 ± 1%     -35.3%     124819 ± 1%  TOTAL numa-vmstat.node1.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     24.83 ± 3%     -27.1%      18.11 ±10%  brickland3/vm-scalability/300s-lru-file-mmap-read
     42.78 ± 1%     -41.6%      25.00 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
     67.61 ± 2%     -36.2%      43.11 ± 5%  TOTAL turbostat.%c6

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.449e+09 ± 1%     -42.2%  8.376e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 9.257e+08 ± 0%     -26.3%  6.823e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 2.375e+09 ± 0%     -36.0%   1.52e+09 ± 1%  TOTAL proc-vmstat.pgscan_direct_normal

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.625e+08 ± 1%     -41.8%   94564115 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.886e+08 ± 0%     -25.0%  1.415e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 3.511e+08 ± 1%     -32.8%   2.36e+08 ± 1%  TOTAL numa-vmstat.node0.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.625e+08 ± 1%     -41.8%   94575783 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.886e+08 ± 0%     -25.0%  1.415e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 3.511e+08 ± 1%     -32.8%  2.361e+08 ± 1%  TOTAL numa-vmstat.node0.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.129e+08 ± 1%     -39.9%   67808889 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.911e+08 ± 0%     -27.9%  1.378e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  3.04e+08 ± 1%     -32.4%  2.056e+08 ± 1%  TOTAL numa-vmstat.node1.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  1.13e+08 ± 1%     -39.9%   67893910 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.911e+08 ± 0%     -27.9%  1.379e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 3.041e+08 ± 1%     -32.3%  2.058e+08 ± 1%  TOTAL numa-vmstat.node1.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     16950 ± 5%     -34.4%      11111 ± 8%  brickland3/vm-scalability/300s-lru-file-mmap-read
     16950 ± 5%     -34.4%      11111 ± 8%  TOTAL cpuidle.C3-IVT-4S.usage

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    292183 ± 1%     -34.2%     192394 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    292183 ± 1%     -34.2%     192394 ± 1%  TOTAL numa-vmstat.node3.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  72128917 ± 0%     -40.4%   42986804 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1160192 ± 3%     +39.9%    1622852 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  73289109 ± 0%     -39.1%   44609656 ± 2%  TOTAL proc-vmstat.pgfault

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1170427 ± 1%     -34.5%     766302 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1170427 ± 1%     -34.5%     766302 ± 1%  TOTAL numa-meminfo.node3.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      5.15 ± 5%     +44.3%       7.43 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
      5.15 ± 5%     +44.3%       7.43 ± 9%  TOTAL perf-profile.cpu-cycles.generic_exec_single.smp_call_function_single.smp_call_function_many.native_flush_tlb_others.flush_tlb_page

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  21722028 ±14%     -27.4%   15765961 ±13%  brickland3/vm-scalability/300s-lru-file-mmap-read
  21722028 ±14%     -27.4%   15765961 ±13%  TOTAL cpuidle.C3-IVT-4S.time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    295382 ± 1%     -32.4%     199692 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
    295382 ± 1%     -32.4%     199692 ± 2%  TOTAL numa-vmstat.node1.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1202511 ± 1%     -33.4%     800394 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1202511 ± 1%     -33.4%     800394 ± 1%  TOTAL numa-meminfo.node3.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    135163 ± 0%     -31.6%      92402 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    135163 ± 0%     -31.6%      92402 ± 1%  TOTAL slabinfo.radix_tree_node.active_slabs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    135163 ± 0%     -31.6%      92402 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    135163 ± 0%     -31.6%      92402 ± 1%  TOTAL slabinfo.radix_tree_node.num_slabs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   7569136 ± 0%     -31.6%    5174561 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7569136 ± 0%     -31.6%    5174561 ± 1%  TOTAL slabinfo.radix_tree_node.num_objs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.412e+08 ± 1%     -40.4%   84248525 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.568e+08 ± 0%     -21.7%  2.011e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  3.98e+08 ± 0%     -28.3%  2.853e+08 ± 1%  TOTAL numa-numastat.node1.local_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.412e+08 ± 1%     -40.3%   84255157 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.568e+08 ± 0%     -21.7%  2.011e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  3.98e+08 ± 0%     -28.3%  2.853e+08 ± 1%  TOTAL numa-numastat.node1.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1183239 ± 1%     -32.7%     795729 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1183239 ± 1%     -32.7%     795729 ± 3%  TOTAL numa-meminfo.node1.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.289e+08 ± 1%     -37.7%   80278779 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.916e+08 ± 0%     -24.3%  1.451e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.206e+08 ± 0%     -29.7%  2.254e+08 ± 1%  TOTAL numa-vmstat.node3.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  1.29e+08 ± 1%     -37.7%   80363047 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.917e+08 ± 0%     -24.2%  1.452e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.207e+08 ± 0%     -29.7%  2.256e+08 ± 1%  TOTAL numa-vmstat.node3.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1144645 ± 2%     -32.8%     768993 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1144645 ± 2%     -32.8%     768993 ± 2%  TOTAL numa-meminfo.node2.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   5830125 ± 3%     -17.3%    4821835 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8557468 ± 2%     -41.0%    5045624 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  14387594 ± 2%     -31.4%    9867460 ± 1%  TOTAL numa-vmstat.node0.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1093892 ± 0%     -31.4%     750954 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1093892 ± 0%     -31.4%     750954 ± 1%  TOTAL proc-vmstat.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   5785378 ± 4%     -18.1%    4737611 ± 4%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8191876 ± 2%     -39.0%    4994945 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  13977254 ± 3%     -30.4%    9732556 ± 2%  TOTAL numa-vmstat.node3.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.198e+08 ± 0%     -40.8%  1.302e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.537e+08 ± 0%     -18.4%  2.069e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 4.735e+08 ± 0%     -28.8%  3.371e+08 ± 1%  TOTAL numa-numastat.node0.local_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 2.199e+08 ± 0%     -40.8%  1.302e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.537e+08 ± 0%     -18.4%  2.069e+08 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 4.736e+08 ± 0%     -28.8%  3.371e+08 ± 1%  TOTAL numa-numastat.node0.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    285717 ± 2%     -32.4%     193014 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    285717 ± 2%     -32.4%     193014 ± 1%  TOTAL numa-vmstat.node2.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  23302752 ± 3%     -17.5%   19233230 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  34168164 ± 2%     -41.0%   20160695 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  57470916 ± 2%     -31.5%   39393925 ± 2%  TOTAL numa-meminfo.node0.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  22464606 ± 4%     -15.9%   18899287 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  33409387 ± 1%     -39.7%   20144973 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  55873993 ± 2%     -30.1%   39044260 ± 2%  TOTAL numa-meminfo.node1.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  22596612 ± 2%     -17.3%   18690112 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  32888379 ± 1%     -40.5%   19557960 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  55484992 ± 1%     -31.1%   38248073 ± 1%  TOTAL proc-vmstat.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1219306 ± 1%     -31.9%     830492 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1219306 ± 1%     -31.9%     830492 ± 3%  TOTAL numa-meminfo.node1.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1176783 ± 2%     -31.8%     801979 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   1176783 ± 2%     -31.8%     801979 ± 1%  TOTAL numa-meminfo.node2.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   7255253 ± 0%     -30.2%    5061846 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   7255253 ± 0%     -30.2%    5061846 ± 1%  TOTAL slabinfo.radix_tree_node.active_objs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   4354231 ± 0%     -30.7%    3016317 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   4354231 ± 0%     -30.7%    3016317 ± 1%  TOTAL meminfo.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  89793802 ± 2%     -17.5%   74083099 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.314e+08 ± 1%     -40.3%   78477271 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 2.212e+08 ± 1%     -31.0%  1.526e+08 ± 2%  TOTAL meminfo.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.248e+08 ± 3%     -37.2%   78416804 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.918e+08 ± 0%     -21.7%  1.503e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.166e+08 ± 1%     -27.8%  2.287e+08 ± 1%  TOTAL numa-vmstat.node2.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.249e+08 ± 3%     -37.1%   78501114 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.919e+08 ± 0%     -21.6%  1.504e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.168e+08 ± 1%     -27.8%  2.289e+08 ± 1%  TOTAL numa-vmstat.node2.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  76888271 ± 2%     -14.1%   66010304 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.305e+08 ± 1%     -41.3%   76594029 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
 2.074e+08 ± 1%     -31.2%  1.426e+08 ± 1%  TOTAL vmstat.memory.free

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       473 ± 7%     -23.5%        362 ±13%  brickland3/vm-scalability/300s-lru-file-readonce
       473 ± 7%     -23.5%        362 ±13%  TOTAL numa-vmstat.node3.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1559898 ± 0%     -30.0%    1092289 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    211875 ±20%     +56.4%     331443 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
   1771773 ± 2%     -19.6%    1423732 ± 1%  TOTAL softirqs.SCHED

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  23128109 ± 5%     -18.2%   18910364 ± 6%  brickland3/vm-scalability/300s-lru-file-mmap-read
  32678604 ± 2%     -38.9%   19957620 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
  55806713 ± 3%     -30.4%   38867985 ± 3%  TOTAL numa-meminfo.node3.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   2363418 ± 0%     -40.7%    1402609 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   2920077 ± 0%     -12.7%    2547859 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   5283495 ± 0%     -25.2%    3950468 ± 1%  TOTAL proc-vmstat.pgalloc_dma32

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     21290 ± 5%     -29.3%      15054 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
     21290 ± 5%     -29.3%      15054 ± 4%  TOTAL meminfo.Mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    755935 ± 1%     -34.5%     495084 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1892 ± 8%     +29.4%       2447 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
    757827 ± 1%     -34.3%     497532 ± 1%  TOTAL numa-meminfo.node3.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   4496625 ± 0%     -29.7%    3159080 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   4496625 ± 0%     -29.7%    3159080 ± 1%  TOTAL meminfo.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  22503656 ± 5%     -15.3%   19052501 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
  32274836 ± 3%     -37.7%   20119596 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  54778492 ± 4%     -28.5%   39172098 ± 2%  TOTAL numa-meminfo.node2.MemFree

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 6.858e+08 ± 0%     -38.8%  4.199e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.026e+09 ± 0%     -17.9%  8.422e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 1.712e+09 ± 0%     -26.3%  1.262e+09 ± 1%  TOTAL proc-vmstat.numa_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 6.858e+08 ± 0%     -38.8%  4.199e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.026e+09 ± 0%     -17.9%  8.422e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 1.712e+09 ± 0%     -26.3%  1.262e+09 ± 1%  TOTAL proc-vmstat.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   5619854 ± 4%     -15.7%    4738362 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8370497 ± 1%     -39.8%    5040037 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
  13990351 ± 2%     -30.1%    9778399 ± 1%  TOTAL numa-vmstat.node1.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   8022752 ± 5%     +38.1%   11079276 ± 8%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8022752 ± 5%     +38.1%   11079276 ± 8%  TOTAL time.maximum_resident_set_size

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      7933 ± 6%     -30.8%       5487 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
      7933 ± 6%     -30.8%       5487 ± 9%  TOTAL cpuidle.C1E-IVT-4S.usage

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    188691 ± 1%     -34.1%     124329 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
       471 ± 7%     +29.9%        612 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
    189162 ± 1%     -34.0%     124941 ± 1%  TOTAL numa-vmstat.node3.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    138389 ± 9%     -32.5%      93430 ±18%  brickland3/vm-scalability/300s-lru-file-readonce
    138389 ± 9%     -32.5%      93430 ±18%  TOTAL proc-vmstat.pgactivate

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   5626193 ± 5%     -15.1%    4778409 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
   8083183 ± 3%     -37.7%    5035365 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
  13709376 ± 4%     -28.4%    9813775 ± 2%  TOTAL numa-vmstat.node2.nr_free_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.662e+08 ± 0%     -36.8%   1.05e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.575e+08 ± 0%     -17.4%  2.128e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 4.237e+08 ± 0%     -25.0%  3.178e+08 ± 1%  TOTAL numa-numastat.node3.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.662e+08 ± 0%     -36.8%   1.05e+08 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.575e+08 ± 0%     -17.4%  2.128e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 4.237e+08 ± 0%     -25.0%  3.178e+08 ± 1%  TOTAL numa-numastat.node3.local_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
        66 ± 1%     +35.3%         89 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
        66 ± 1%     +35.3%         89 ± 0%  TOTAL vmstat.procs.r

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.585e+08 ± 1%     -36.6%  1.005e+08 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.578e+08 ± 0%     -14.1%  2.215e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 4.163e+08 ± 0%     -22.7%  3.219e+08 ± 1%  TOTAL numa-numastat.node2.local_node

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.585e+08 ± 1%     -36.6%  1.005e+08 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
 2.578e+08 ± 0%     -14.1%  2.215e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 4.163e+08 ± 0%     -22.7%   3.22e+08 ± 1%  TOTAL numa-numastat.node2.numa_hit

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    759826 ± 1%     -34.8%     495074 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2057 ± 3%     +17.8%       2423 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
    761884 ± 1%     -34.7%     497497 ± 1%  TOTAL numa-meminfo.node2.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    759680 ± 1%     -34.5%     497889 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2045 ± 1%     +17.9%       2413 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
    761726 ± 1%     -34.3%     500302 ± 1%  TOTAL proc-vmstat.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    741326 ± 1%     -32.7%     499221 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2029 ±10%     +19.4%       2422 ± 8%  brickland3/vm-scalability/300s-lru-file-readonce
    743355 ± 1%     -32.5%     501644 ± 1%  TOTAL numa-meminfo.node0.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    184996 ± 1%     -32.3%     125326 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
       505 ±10%     +19.5%        603 ± 8%  brickland3/vm-scalability/300s-lru-file-readonce
    185501 ± 1%     -32.1%     125930 ± 1%  TOTAL numa-vmstat.node0.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    189703 ± 0%     -34.5%     124276 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
       516 ± 3%     +17.3%        605 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
    190219 ± 0%     -34.3%     124882 ± 1%  TOTAL numa-vmstat.node2.nr_page_table_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   3022193 ± 1%     -33.6%    2007941 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      8169 ± 1%     +18.3%       9660 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
   3030362 ± 1%     -33.4%    2017601 ± 1%  TOTAL meminfo.PageTables

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      4969 ±12%     -23.7%       3792 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
      4969 ±12%     -23.7%       3792 ± 3%  TOTAL proc-vmstat.nr_mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    885165 ± 2%     -23.0%     681778 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
    885165 ± 2%     -23.0%     681778 ± 3%  TOTAL numa-meminfo.node0.SReclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    221025 ± 2%     -22.6%     171160 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
    221025 ± 2%     -22.6%     171160 ± 3%  TOTAL numa-vmstat.node0.nr_slab_reclaimable

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1970 ± 3%     -22.7%       1522 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
      1970 ± 3%     -22.7%       1522 ± 3%  TOTAL proc-vmstat.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    927295 ± 2%     -22.1%     722698 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
    927295 ± 2%     -22.1%     722698 ± 3%  TOTAL numa-meminfo.node0.Slab

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1862 ± 9%     +26.1%       2349 ± 8%  brickland3/vm-scalability/300s-lru-file-mmap-read
      1862 ± 9%     +26.1%       2349 ± 8%  TOTAL proc-vmstat.workingset_refault

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      9529 ± 3%     +29.1%      12300 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
      9529 ± 3%     +29.1%      12300 ± 4%  TOTAL proc-vmstat.numa_hint_faults_local

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    274122 ± 1%     -13.2%     237877 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
     10426 ± 3%     +36.4%      14223 ± 6%  brickland3/vm-scalability/300s-lru-file-readonce
    284549 ± 1%     -11.4%     252101 ± 2%  TOTAL proc-vmstat.numa_pte_updates

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1070 ± 3%     -19.7%        859 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
      1070 ± 3%     -19.7%        859 ± 5%  TOTAL numa-vmstat.node2.nr_mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     21.64 ± 2%     +26.3%      27.33 ±10%  brickland3/vm-scalability/300s-lru-file-mmap-read
     21.64 ± 2%     +26.3%      27.33 ±10%  TOTAL perf-profile.cpu-cycles.osq_lock.__mutex_lock_slowpath.mutex_lock.rmap_walk.page_referenced

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      4577 ± 7%     -18.7%       3723 ± 8%  brickland3/vm-scalability/300s-lru-file-readonce
      4577 ± 7%     -18.7%       3723 ± 8%  TOTAL numa-meminfo.node0.Mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    272410 ± 1%     -12.8%     237556 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
      9802 ± 3%     +30.2%      12761 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
    282212 ± 1%     -11.3%     250317 ± 1%  TOTAL proc-vmstat.numa_hint_faults

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   2299504 ± 0%     -19.8%    1844020 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
    106698 ± 1%      +9.9%     117221 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
   2406202 ± 0%     -18.5%    1961242 ± 2%  TOTAL cpuidle.C6-IVT-4S.usage

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 1.697e+10 ± 2%     -15.3%  1.438e+10 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
 1.697e+10 ± 2%     -15.3%  1.438e+10 ± 3%  TOTAL cpuidle.C6-IVT-4S.time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       467 ± 4%     -13.9%        402 ± 7%  brickland3/vm-scalability/300s-lru-file-readonce
       467 ± 4%     -13.9%        402 ± 7%  TOTAL numa-vmstat.node1.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  23927853 ± 0%     +14.7%   27436111 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  23927853 ± 0%     +14.7%   27436111 ± 0%  TOTAL numa-vmstat.node0.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  23897222 ± 0%     +14.5%   27361810 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  23897222 ± 0%     +14.5%   27361810 ± 0%  TOTAL numa-vmstat.node0.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  95772318 ± 0%     +14.6%  1.098e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  95772318 ± 0%     +14.6%  1.098e+08 ± 0%  TOTAL numa-meminfo.node0.FilePages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      6248 ±42%     -41.2%       3672 ± 2%  brickland3/vm-scalability/300s-lru-file-readonce
      6248 ±42%     -41.2%       3672 ± 2%  TOTAL numa-meminfo.node1.Mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  95649623 ± 0%     +14.4%  1.095e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  95649623 ± 0%     +14.4%  1.095e+08 ± 0%  TOTAL numa-meminfo.node0.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  95650669 ± 0%     +14.4%  1.095e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  95650669 ± 0%     +14.4%  1.095e+08 ± 0%  TOTAL numa-meminfo.node0.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97759674 ± 0%     +14.3%  1.118e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97759674 ± 0%     +14.3%  1.118e+08 ± 0%  TOTAL numa-meminfo.node0.MemUsed

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      1637 ±46%     -43.7%        922 ± 3%  brickland3/vm-scalability/300s-lru-file-readonce
      1637 ±46%     -43.7%        922 ± 3%  TOTAL numa-vmstat.node1.nr_mapped

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24013796 ± 0%     +14.1%   27392772 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24013796 ± 0%     +14.1%   27392772 ± 0%  TOTAL numa-vmstat.node1.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 3.899e+08 ± 0%     +13.8%  4.437e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.899e+08 ± 0%     +13.8%  4.437e+08 ± 0%  TOTAL vmstat.memory.cache

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96127542 ± 0%     +14.0%  1.096e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  96127542 ± 0%     +14.0%  1.096e+08 ± 0%  TOTAL numa-meminfo.node1.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96339092 ± 0%     +14.0%  1.098e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  96339092 ± 0%     +14.0%  1.098e+08 ± 0%  TOTAL numa-meminfo.node1.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96655792 ± 0%     +13.7%  1.099e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  96655792 ± 0%     +13.7%  1.099e+08 ± 0%  TOTAL numa-meminfo.node1.FilePages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 3.878e+08 ± 0%     +13.6%  4.406e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.878e+08 ± 0%     +13.6%  4.406e+08 ± 0%  TOTAL meminfo.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
 3.881e+08 ± 0%     +13.6%  4.409e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
 3.881e+08 ± 0%     +13.6%  4.409e+08 ± 0%  TOTAL meminfo.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  3.89e+08 ± 0%     +13.6%  4.418e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  3.89e+08 ± 0%     +13.6%  4.418e+08 ± 0%  TOTAL meminfo.Cached

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  96912081 ± 0%     +13.7%  1.102e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  96912081 ± 0%     +13.7%  1.102e+08 ± 0%  TOTAL proc-vmstat.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97216175 ± 0%     +13.7%  1.105e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97216175 ± 0%     +13.7%  1.105e+08 ± 0%  TOTAL proc-vmstat.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24145923 ± 0%     +13.8%   27478127 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24145923 ± 0%     +13.8%   27478127 ± 0%  TOTAL numa-vmstat.node1.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     11086 ± 1%     +12.7%      12491 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
     11086 ± 1%     +12.7%      12491 ± 1%  TOTAL proc-vmstat.nr_anon_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     44374 ± 1%     +12.6%      49965 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
     44374 ± 1%     +12.6%      49965 ± 1%  TOTAL meminfo.AnonPages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  98706688 ± 0%     +13.4%   1.12e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  98706688 ± 0%     +13.4%   1.12e+08 ± 0%  TOTAL numa-meminfo.node1.MemUsed

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      9.40 ± 2%     +12.3%      10.56 ± 9%  brickland3/vm-scalability/300s-lru-file-mmap-read
      9.40 ± 2%     +12.3%      10.56 ± 9%  TOTAL perf-profile.cpu-cycles.osq_lock.__mutex_lock_slowpath.mutex_lock.rmap_walk.try_to_unmap

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24256827 ± 0%     +13.2%   27466497 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24256827 ± 0%     +13.2%   27466497 ± 0%  TOTAL numa-vmstat.node3.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     15468 ± 1%     +12.3%      17364 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
     15468 ± 1%     +12.3%      17364 ± 5%  TOTAL numa-meminfo.node2.AnonPages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24358756 ± 0%     +13.1%   27549191 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24358756 ± 0%     +13.1%   27549191 ± 0%  TOTAL numa-vmstat.node3.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      3869 ± 1%     +12.2%       4340 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
      3869 ± 1%     +12.2%       4340 ± 5%  TOTAL numa-vmstat.node2.nr_anon_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97523828 ± 0%     +13.0%  1.102e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97523828 ± 0%     +13.0%  1.102e+08 ± 0%  TOTAL numa-meminfo.node3.FilePages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97115503 ± 0%     +13.2%  1.099e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97115503 ± 0%     +13.2%  1.099e+08 ± 0%  TOTAL numa-meminfo.node3.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97126862 ± 0%     +13.1%  1.099e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97126862 ± 0%     +13.1%  1.099e+08 ± 0%  TOTAL numa-meminfo.node3.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  99437010 ± 0%     +12.8%  1.122e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  99437010 ± 0%     +12.8%  1.122e+08 ± 0%  TOTAL numa-meminfo.node3.MemUsed

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97930450 ± 1%     +12.4%    1.1e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97930450 ± 1%     +12.4%    1.1e+08 ± 0%  TOTAL numa-meminfo.node2.FilePages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  99841238 ± 1%     +12.2%   1.12e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  99841238 ± 1%     +12.2%   1.12e+08 ± 0%  TOTAL numa-meminfo.node2.MemUsed

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       480 ± 4%     -12.8%        418 ±11%  brickland3/vm-scalability/300s-lru-file-readonce
       480 ± 4%     -12.8%        418 ±11%  TOTAL numa-vmstat.node0.nr_isolated_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24468312 ± 1%     +12.4%   27504476 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24468312 ± 1%     +12.4%   27504476 ± 0%  TOTAL numa-vmstat.node2.nr_file_pages

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97818315 ± 1%     +12.3%  1.098e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97818315 ± 1%     +12.3%  1.098e+08 ± 0%  TOTAL numa-meminfo.node2.Inactive

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  97767272 ± 1%     +12.3%  1.098e+08 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  97767272 ± 1%     +12.3%  1.098e+08 ± 0%  TOTAL numa-meminfo.node2.Inactive(file)

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24427538 ± 1%     +12.3%   27435973 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  24427538 ± 1%     +12.3%   27435973 ± 0%  TOTAL numa-vmstat.node2.nr_inactive_file

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     22711 ± 3%      -9.1%      20641 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
     13218 ± 3%      +8.2%      14303 ± 1%  brickland3/vm-scalability/300s-lru-file-readonce
     35929 ± 3%      -2.7%      34945 ± 2%  TOTAL uptime.idle

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     25195 ± 0%      +6.4%      26807 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
     14015 ± 1%    +151.8%      35296 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
     39211 ± 0%     +58.4%      62103 ± 0%  TOTAL time.system_time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  24713087 ± 0%     -41.3%   14517140 ± 3%  brickland3/vm-scalability/300s-lru-file-mmap-read
      4189 ± 5%     -26.0%       3099 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
  24717276 ± 0%     -41.3%   14520239 ± 3%  TOTAL time.voluntary_context_switches

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    539861 ± 1%     -23.3%     413999 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    169833 ± 1%     +68.8%     286660 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
    709694 ± 1%      -1.3%     700660 ± 1%  TOTAL time.involuntary_context_switches

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       403 ± 0%      -4.7%        384 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
       213 ± 0%     +89.5%        404 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
       616 ± 0%     +27.9%        788 ± 0%  TOTAL time.elapsed_time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
    126881 ± 0%     -37.6%      79236 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
      2749 ± 1%     -23.8%       2095 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
    129631 ± 0%     -37.3%      81331 ± 2%  TOTAL vmstat.system.cs

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1243406 ± 1%     -37.0%     783419 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
    114271 ± 0%     -26.0%      84537 ± 4%  brickland3/vm-scalability/300s-lru-file-readonce
   1357678 ± 1%     -36.1%     867957 ± 1%  TOTAL vmstat.system.in

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
   1620.48 ± 0%     -42.3%     935.48 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
     84.49 ± 0%     +15.3%      97.39 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
   1704.96 ± 0%     -39.4%    1032.87 ± 1%  TOTAL time.user_time

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
  70740442 ± 1%     -41.2%   41615768 ± 2%  brickland3/vm-scalability/300s-lru-file-mmap-read
    121780 ± 0%      +2.0%     124226 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
  70862223 ± 1%     -41.1%   41739994 ± 2%  TOTAL time.minor_page_faults

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
      6645 ± 0%      +8.5%       7212 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
      6607 ± 0%     +32.5%       8754 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
     13253 ± 0%     +20.5%      15967 ± 0%  TOTAL time.percent_of_cpu_this_job_got

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     58.13 ± 0%      +7.1%      62.24 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
     55.30 ± 0%     +33.7%      73.95 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
    113.43 ± 0%     +20.1%     136.19 ± 0%  TOTAL turbostat.%c0

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       334 ± 1%      +5.4%        353 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
       280 ± 0%     +19.6%        335 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
       615 ± 1%     +11.9%        688 ± 0%  TOTAL turbostat.Cor_W

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
     89.53 ± 0%     -10.2%      80.43 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
     95.97 ± 0%     -11.2%      85.22 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
    185.50 ± 0%     -10.7%     165.64 ± 0%  TOTAL turbostat.RAM_W

24b7e5819ad5cbe  3484b2de9499df23c4604a513  
---------------  -------------------------  
       407 ± 1%      +4.2%        424 ± 0%  brickland3/vm-scalability/300s-lru-file-mmap-read
       353 ± 0%     +15.3%        407 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
       760 ± 0%      +9.3%        831 ± 0%  TOTAL turbostat.Pkg_W


Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.

Thanks,
Fengguang

_______________________________________________
LKP mailing list
LKP(a)linux.intel.com

[-- Attachment #2: reproduce.ksh --]
[-- Type: text/plain, Size: 29759 bytes --]

echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu10/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu100/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu101/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu102/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu103/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu104/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu105/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu106/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu107/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu108/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu109/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu11/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu110/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu111/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu112/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu113/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu114/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu115/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu116/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu117/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu118/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu119/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu12/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu13/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu14/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu15/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu16/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu17/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu18/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu19/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu20/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu21/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu22/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu23/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu24/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu25/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu26/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu27/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu28/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu29/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu30/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu31/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu32/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu33/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu34/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu35/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu36/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu37/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu38/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu39/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu4/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu40/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu41/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu42/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu43/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu44/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu45/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu46/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu47/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu48/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu49/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu5/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu50/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu51/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu52/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu53/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu54/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu55/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu56/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu57/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu58/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu59/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu6/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu60/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu61/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu62/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu63/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu64/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu65/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu66/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu67/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu68/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu69/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu7/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu70/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu71/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu72/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu73/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu74/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu75/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu76/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu77/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu78/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu79/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu8/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu80/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu81/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu82/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu83/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu84/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu85/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu86/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu87/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu88/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu89/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu9/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu90/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu91/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu92/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu93/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu94/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu95/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu96/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu97/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu98/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu99/cpufreq/scaling_governor
mount -t tmpfs -o size=100% vm-scalability-tmp /tmp/vm-scalability-tmp
truncate -s 540952072192 /tmp/vm-scalability.img
mkfs.xfs -q /tmp/vm-scalability.img
mount -o loop /tmp/vm-scalability.img /tmp/vm-scalability
./case-lru-file-mmap-read
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-1 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-1 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-2 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-2 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-3 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-3 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-4 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-5 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-4 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-5 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-6 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-6 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-7 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-7 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-8 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-9 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-8 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-9 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-10 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-11 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-10 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-11 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-12 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-13 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-12 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-13 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-14 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-15 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-14 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-15 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-16 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-16 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-17 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-18 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-17 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-18 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-19 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-19 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-20 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-20 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-21 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-21 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-22 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-22 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-23 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-23 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-24 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-24 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-25 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-25 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-26 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-26 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-27 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-27 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-28 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-28 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-29 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-29 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-30 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-30 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-31 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-31 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-32 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-32 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-33 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-33 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-34 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-35 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-34 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-35 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-36 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-36 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-37 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-37 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-38 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-38 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-39 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-39 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-40 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-41 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-40 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-41 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-42 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-42 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-43 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-43 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-44 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-44 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-45 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-45 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-46 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-47 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-46 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-47 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-48 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-48 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-49 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-49 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-50 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-50 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-51 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-51 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-52 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-52 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-53 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-53 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-54 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-54 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-55 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-55 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-56 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-56 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-57 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-57 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-58 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-58 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-59 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-60 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-59 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-60 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-61 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-61 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-62 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-62 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-63 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-64 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-63 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-64 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-65 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-66 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-65 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-66 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-67 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-67 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-68 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-68 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-69 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-69 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-70 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-70 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-71 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-71 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-72 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-73 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-72 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-74 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-73 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-75 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-74 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-75 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-76 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-76 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-77 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-78 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-77 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-78 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-79 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-79 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-80 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-80 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-81 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-81 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-82 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-82 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-83 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-84 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-83 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-84 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-85 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-86 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-85 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-86 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-87 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-88 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-87 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-89 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-88 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-89 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-90 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-91 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-90 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-91 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-92 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-92 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-93 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-93 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-94 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-95 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-94 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-95 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-96 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-96 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-97 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-97 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-98 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-98 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-99 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-100 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-99 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-100 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-101 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-101 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-102 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-102 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-103 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-104 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-103 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-105 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-104 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-105 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-106 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-106 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-107 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-108 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-107 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-108 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-109 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-109 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-110 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-110 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-111 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-112 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-111 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-112 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-113 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-113 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-114 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-114 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-115 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-115 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-116 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-116 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-117 -s 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-118 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-117 --readonly 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-118 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-119 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-119 --readonly 36650387592
truncate /tmp/vm-scalability/sparse-lru-file-mmap-read-120 -s 36650387592
./usemem --runtime 300 -f /tmp/vm-scalability/sparse-lru-file-mmap-read-120 --readonly 36650387592
umount /tmp/vm-scalability-tmp
umount /tmp/vm-scalability
rm /tmp/vm-scalability.img

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
  2014-08-19  4:32 ` Fengguang Wu
@ 2014-08-19 14:29   ` Mel Gorman
  -1 siblings, 0 replies; 12+ messages in thread
From: Mel Gorman @ 2014-08-19 14:29 UTC (permalink / raw)
  To: Fengguang Wu; +Cc: LKML, lkp

On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> Hi Mel,
> 
> We noticed the below vm-scalability performance/power regressions on
> commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> fields into read-only, page alloc, statistics and page reclaim lines").
> 
> 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> ---------------  -------------------------  ---------------------------
>      %stddev        %change               %stddev
>             \          |                 /
>       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
>       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
>      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> 
> 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> ---------------  -------------------------  
>   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
>   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
>   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> 


What units are these? It's completely unclear what is good and bad from the
figures. 300s-lru-file-mmap-read appears multiple times in this report,
each with different numbers beside them but little clue as to what they
mean or what I'm meant to be looking for :(

This is the same patch that was reported as having a performance gain in
another set of tests from lkp so am a little confused.

More importantly, as this patch is primary abougt cache misses it should
be very unlikely that it makes a noticable difference to IO as the
relative cost of a cache miss is so low. Similarly any difference it
makes to reclaim activity is likely to be a coincidence or due to test
variance.

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
@ 2014-08-19 14:29   ` Mel Gorman
  0 siblings, 0 replies; 12+ messages in thread
From: Mel Gorman @ 2014-08-19 14:29 UTC (permalink / raw)
  To: lkp

[-- Attachment #1: Type: text/plain, Size: 1937 bytes --]

On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> Hi Mel,
> 
> We noticed the below vm-scalability performance/power regressions on
> commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> fields into read-only, page alloc, statistics and page reclaim lines").
> 
> 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> ---------------  -------------------------  ---------------------------
>      %stddev        %change               %stddev
>             \          |                 /
>       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
>       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
>      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> 
> 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> ---------------  -------------------------  
>   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
>   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
>   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> 


What units are these? It's completely unclear what is good and bad from the
figures. 300s-lru-file-mmap-read appears multiple times in this report,
each with different numbers beside them but little clue as to what they
mean or what I'm meant to be looking for :(

This is the same patch that was reported as having a performance gain in
another set of tests from lkp so am a little confused.

More importantly, as this patch is primary abougt cache misses it should
be very unlikely that it makes a noticable difference to IO as the
relative cost of a cache miss is so low. Similarly any difference it
makes to reclaim activity is likely to be a coincidence or due to test
variance.

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
  2014-08-19 14:29   ` Mel Gorman
@ 2014-08-19 15:51     ` Fengguang Wu
  -1 siblings, 0 replies; 12+ messages in thread
From: Fengguang Wu @ 2014-08-19 15:51 UTC (permalink / raw)
  To: Mel Gorman; +Cc: LKML, lkp

On Tue, Aug 19, 2014 at 03:29:25PM +0100, Mel Gorman wrote:
> On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> > Hi Mel,
> > 
> > We noticed the below vm-scalability performance/power regressions on
> > commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> > fields into read-only, page alloc, statistics and page reclaim lines").
> > 
> > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> > ---------------  -------------------------  ---------------------------
> >      %stddev        %change               %stddev
> >             \          |                 /
> >       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
> >       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
> >      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> > 
> > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> > ---------------  -------------------------  
> >   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
> >   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
> >   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> > 
> 
> 
> What units are these? It's completely unclear what is good and bad from the
> figures. 300s-lru-file-mmap-read appears multiple times in this report,
> each with different numbers beside them but little clue as to what they
> mean or what I'm meant to be looking for :(

Sorry the output format is a bit obscure: the stats names are after
the "TOTAL" word, such as TOTAL vm-scalability.stddev, TOTAL
vm-scalability.throughput, ...

> This is the same patch that was reported as having a performance gain in
> another set of tests from lkp so am a little confused.

That can happen some times -- we've seen some commits to benefit some
workloads while hurting some others. I'm now running more complete
test set for the commit, hopefully can get some results tomorrow.

> More importantly, as this patch is primary abougt cache misses it should
> be very unlikely that it makes a noticable difference to IO as the
> relative cost of a cache miss is so low. Similarly any difference it
> makes to reclaim activity is likely to be a coincidence or due to test
> variance.

The test case creates tmpfs files and read them fast to exercise the LRU.
So it's VM test and do not involve disk IO.

Thanks,
Fengguang

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
@ 2014-08-19 15:51     ` Fengguang Wu
  0 siblings, 0 replies; 12+ messages in thread
From: Fengguang Wu @ 2014-08-19 15:51 UTC (permalink / raw)
  To: lkp

[-- Attachment #1: Type: text/plain, Size: 2553 bytes --]

On Tue, Aug 19, 2014 at 03:29:25PM +0100, Mel Gorman wrote:
> On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> > Hi Mel,
> > 
> > We noticed the below vm-scalability performance/power regressions on
> > commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> > fields into read-only, page alloc, statistics and page reclaim lines").
> > 
> > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> > ---------------  -------------------------  ---------------------------
> >      %stddev        %change               %stddev
> >             \          |                 /
> >       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
> >       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
> >      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> > 
> > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> > ---------------  -------------------------  
> >   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
> >   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
> >   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> > 
> 
> 
> What units are these? It's completely unclear what is good and bad from the
> figures. 300s-lru-file-mmap-read appears multiple times in this report,
> each with different numbers beside them but little clue as to what they
> mean or what I'm meant to be looking for :(

Sorry the output format is a bit obscure: the stats names are after
the "TOTAL" word, such as TOTAL vm-scalability.stddev, TOTAL
vm-scalability.throughput, ...

> This is the same patch that was reported as having a performance gain in
> another set of tests from lkp so am a little confused.

That can happen some times -- we've seen some commits to benefit some
workloads while hurting some others. I'm now running more complete
test set for the commit, hopefully can get some results tomorrow.

> More importantly, as this patch is primary abougt cache misses it should
> be very unlikely that it makes a noticable difference to IO as the
> relative cost of a cache miss is so low. Similarly any difference it
> makes to reclaim activity is likely to be a coincidence or due to test
> variance.

The test case creates tmpfs files and read them fast to exercise the LRU.
So it's VM test and do not involve disk IO.

Thanks,
Fengguang

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
  2014-08-19 15:51     ` Fengguang Wu
@ 2014-08-19 17:13       ` Mel Gorman
  -1 siblings, 0 replies; 12+ messages in thread
From: Mel Gorman @ 2014-08-19 17:13 UTC (permalink / raw)
  To: Fengguang Wu; +Cc: LKML, lkp

On Tue, Aug 19, 2014 at 11:51:25PM +0800, Fengguang Wu wrote:
> On Tue, Aug 19, 2014 at 03:29:25PM +0100, Mel Gorman wrote:
> > On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> > > Hi Mel,
> > > 
> > > We noticed the below vm-scalability performance/power regressions on
> > > commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> > > fields into read-only, page alloc, statistics and page reclaim lines").
> > > 
> > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> > > ---------------  -------------------------  ---------------------------
> > >      %stddev        %change               %stddev
> > >             \          |                 /
> > >       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > >       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
> > >      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> > > 
> > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> > > ---------------  -------------------------  
> > >   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > >   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
> > >   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> > > 
> > 
> > 
> > What units are these? It's completely unclear what is good and bad from the
> > figures. 300s-lru-file-mmap-read appears multiple times in this report,
> > each with different numbers beside them but little clue as to what they
> > mean or what I'm meant to be looking for :(
> 
> Sorry the output format is a bit obscure: the stats names are after
> the "TOTAL" word, such as TOTAL vm-scalability.stddev, TOTAL
> vm-scalability.throughput, ...
> 
> > This is the same patch that was reported as having a performance gain in
> > another set of tests from lkp so am a little confused.
> 
> That can happen some times -- we've seen some commits to benefit some
> workloads while hurting some others. I'm now running more complete
> test set for the commit, hopefully can get some results tomorrow.
> 
> > More importantly, as this patch is primary abougt cache misses it should
> > be very unlikely that it makes a noticable difference to IO as the
> > relative cost of a cache miss is so low. Similarly any difference it
> > makes to reclaim activity is likely to be a coincidence or due to test
> > variance.
> 
> The test case creates tmpfs files and read them fast to exercise the LRU.
> So it's VM test and do not involve disk IO.
> 

Ok. How reliable is this? I was reading down through some of the figures and
some of them are extremely different in areas that this patch should have
no impact on -- it's just rearranging a struct after all. I note for example
that automatic NUMA balancing was enabled and there was activity from it so
pages are getting migrated.  There is a large amount of reclaim activity on
different nodes so where the test case gets scheduled and how nodes it runs
on during the test matters. What I'm wondering is if the performance of the
test tends to vary quite a lot naturally. Can you check if the performance
is consistently lower after this patch or if it tends to vary please?

Thanks

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
@ 2014-08-19 17:13       ` Mel Gorman
  0 siblings, 0 replies; 12+ messages in thread
From: Mel Gorman @ 2014-08-19 17:13 UTC (permalink / raw)
  To: lkp

[-- Attachment #1: Type: text/plain, Size: 3422 bytes --]

On Tue, Aug 19, 2014 at 11:51:25PM +0800, Fengguang Wu wrote:
> On Tue, Aug 19, 2014 at 03:29:25PM +0100, Mel Gorman wrote:
> > On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> > > Hi Mel,
> > > 
> > > We noticed the below vm-scalability performance/power regressions on
> > > commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> > > fields into read-only, page alloc, statistics and page reclaim lines").
> > > 
> > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> > > ---------------  -------------------------  ---------------------------
> > >      %stddev        %change               %stddev
> > >             \          |                 /
> > >       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > >       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
> > >      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> > > 
> > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> > > ---------------  -------------------------  
> > >   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > >   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
> > >   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> > > 
> > 
> > 
> > What units are these? It's completely unclear what is good and bad from the
> > figures. 300s-lru-file-mmap-read appears multiple times in this report,
> > each with different numbers beside them but little clue as to what they
> > mean or what I'm meant to be looking for :(
> 
> Sorry the output format is a bit obscure: the stats names are after
> the "TOTAL" word, such as TOTAL vm-scalability.stddev, TOTAL
> vm-scalability.throughput, ...
> 
> > This is the same patch that was reported as having a performance gain in
> > another set of tests from lkp so am a little confused.
> 
> That can happen some times -- we've seen some commits to benefit some
> workloads while hurting some others. I'm now running more complete
> test set for the commit, hopefully can get some results tomorrow.
> 
> > More importantly, as this patch is primary abougt cache misses it should
> > be very unlikely that it makes a noticable difference to IO as the
> > relative cost of a cache miss is so low. Similarly any difference it
> > makes to reclaim activity is likely to be a coincidence or due to test
> > variance.
> 
> The test case creates tmpfs files and read them fast to exercise the LRU.
> So it's VM test and do not involve disk IO.
> 

Ok. How reliable is this? I was reading down through some of the figures and
some of them are extremely different in areas that this patch should have
no impact on -- it's just rearranging a struct after all. I note for example
that automatic NUMA balancing was enabled and there was activity from it so
pages are getting migrated.  There is a large amount of reclaim activity on
different nodes so where the test case gets scheduled and how nodes it runs
on during the test matters. What I'm wondering is if the performance of the
test tends to vary quite a lot naturally. Can you check if the performance
is consistently lower after this patch or if it tends to vary please?

Thanks

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
  2014-08-19 17:13       ` Mel Gorman
@ 2014-08-19 17:52         ` Fengguang Wu
  -1 siblings, 0 replies; 12+ messages in thread
From: Fengguang Wu @ 2014-08-19 17:52 UTC (permalink / raw)
  To: Mel Gorman; +Cc: LKML, lkp

On Tue, Aug 19, 2014 at 06:13:58PM +0100, Mel Gorman wrote:
> On Tue, Aug 19, 2014 at 11:51:25PM +0800, Fengguang Wu wrote:
> > On Tue, Aug 19, 2014 at 03:29:25PM +0100, Mel Gorman wrote:
> > > On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> > > > Hi Mel,
> > > > 
> > > > We noticed the below vm-scalability performance/power regressions on
> > > > commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> > > > fields into read-only, page alloc, statistics and page reclaim lines").
> > > > 
> > > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> > > > ---------------  -------------------------  ---------------------------
> > > >      %stddev        %change               %stddev
> > > >             \          |                 /
> > > >       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > > >       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
> > > >      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> > > > 
> > > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> > > > ---------------  -------------------------  
> > > >   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > > >   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
> > > >   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> > > > 
> > > 
> > > 
> > > What units are these? It's completely unclear what is good and bad from the
> > > figures. 300s-lru-file-mmap-read appears multiple times in this report,
> > > each with different numbers beside them but little clue as to what they
> > > mean or what I'm meant to be looking for :(
> > 
> > Sorry the output format is a bit obscure: the stats names are after
> > the "TOTAL" word, such as TOTAL vm-scalability.stddev, TOTAL
> > vm-scalability.throughput, ...
> > 
> > > This is the same patch that was reported as having a performance gain in
> > > another set of tests from lkp so am a little confused.
> > 
> > That can happen some times -- we've seen some commits to benefit some
> > workloads while hurting some others. I'm now running more complete
> > test set for the commit, hopefully can get some results tomorrow.
> > 
> > > More importantly, as this patch is primary abougt cache misses it should
> > > be very unlikely that it makes a noticable difference to IO as the
> > > relative cost of a cache miss is so low. Similarly any difference it
> > > makes to reclaim activity is likely to be a coincidence or due to test
> > > variance.
> > 
> > The test case creates tmpfs files and read them fast to exercise the LRU.
> > So it's VM test and do not involve disk IO.
> > 
> 
> Ok. How reliable is this? I was reading down through some of the figures and
> some of them are extremely different in areas that this patch should have
> no impact on -- it's just rearranging a struct after all.

Yup. One difficulty with performance testing is that we sometimes
bisect into commit whose change set is obviously not related to the
changed performance.. and the bisect itself looks good.

> I note for example that automatic NUMA balancing was enabled and
> there was activity from it so pages are getting migrated.  There is
> a large amount of reclaim activity on different nodes so where the
> test case gets scheduled and how nodes it runs on during the test
> matters. What I'm wondering is if the performance of the test tends
> to vary quite a lot naturally. Can you check if the performance is
> consistently lower after this patch or if it tends to vary please?

At least for vm-scalability.throughput, it's pretty stable over *all*
the kernels tested during the bisect. Which is a strong indication
that the bisect process itself is fine.

mmap-read:

                               vm-scalability.throughput

  1.5e+07 ++----------------------------------------------------------------+
          |                                                                 |
  1.4e+07 *+*..*.*.*.*..*.*.*.*..*.*.*..*.*. .*..*.*.*..*.*                 |
          |                                 *                               |
  1.3e+07 ++                                                                |
          |                                                                 |
  1.2e+07 ++                                                                |
          |                                                                 |
  1.1e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
          |                                                                 |
    9e+06 O+O  O O O O  O O O O  O O O    O   O  O O O  O O O O    O O    O O
          |                             O   O                    O     O    |
    8e+06 ++----------------------------------------------------------------+

        [*] bisect-good sample
        [O] bisect-bad  sample

readonce:

                               vm-scalability.throughput

    4e+07 ++----*-----*----**-----------------------------------------------+
          *****  ***** *.**  *****.* * ** *. *   *                          |
  3.5e+07 ++                        * *  *  * *** *                         |
          |                                                                 |
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
  1.5e+07 OOOOO OOO OOOO OOOOOOOOO OOO OOOO  OOOOOOO OOOOOOOOO OOO          |
          |        O                  O     O                     OOOOO OOOOO
    1e+07 ++----------------------------------------------------------------+

Here are more graphs for other stats in the readonce case, which can
give us a visual impression which stats are stable and have clear
separable values before/after the first bad commit.

                                  time.user_time

  105 ++--------------------------------------------------------------------+
      |                              O O   O O                              |
  100 ++                            O O O O O O O O  O OOOO                 |
      | O OO                                     O OO                     O O
      OO                                                         O O   OO  O|
   95 ++    OOOO OOO   OOO    OO                             O OO   OOO     |
      |       * .* :OO    OO O  OOO                        O  O             |
   90 ++      :*   *      * .***                                            |
      ***.***:      :*.*  :*   :                                            |
   85 ++     *      *   **      :  .*        * .**                          |
      |                         * *  * **. ** *                             |
      |                          ::   *   *                                 |
   80 ++                         *                                          |
      |                                                                     |
   75 ++--------------------------------------------------------------------+


                                  time.system_time

  40000 ++------------------------------------------------------------------+
        |                                                                   |
  35000 OOOO OOOOOO OOOOOOO OOOOOO OOOOOOO OOOOOOO OOOOOO OOOOOOO OOOOOO OOOO
        |                                                                   |
        |                                                                   |
  30000 ++                                                                  |
        |                                                                   |
  25000 ++                                                                  |
        |                                                                   |
  20000 ++                                                                  |
        |                                                                   |
        |                                                                   |
  15000 ****.* ****.* ***  .**** *.*******.*******                          |
        |     *      *   **     *                                           |
  10000 ++------------------------------------------------------------------+


                         time.percent_of_cpu_this_job_got

  9000 +O--OO---------------------------------------------------------------+
       O O   O OO O OO    OOOOO O    O  O  O  O  O     OO O      O OOOOOO OOO
  8500 ++     O    O  OO O       OOOO  O OO O  OO OO OO  O  OOOOO           |
       |                                                                    |
       |                                                                    |
  8000 ++                                                                   |
       |                                                                    |
  7500 ++                                                                   |
       |                                                                    |
  7000 ++                                                                   |
       |                                                                    |
       | *.     *.            *. ****     * *.** *                          |
  6500 **  *****  ******.*****  *    *.*** *    *                           |
       |                                                                    |
  6000 ++-------------------------------------------------------------------+


                                 time.elapsed_time

  450 ++--------------------------------------------------------------------+
      |                                                                     |
      |      O   OO  O OOOO   O OOO O OOO O OOO OO OO    O O OOO       OO   |
  400 OOO OOO OO   OO      O O O     O     O      O  O OO O     OO OOOO   OOO
      |                                                                     |
      |                                                                     |
  350 ++                                                                    |
      |                                                                     |
  300 ++                                                                    |
      |                                                                     |
      |                                                                     |
  250 ++                                                                    |
      |                                                                     |
      *      * *.   * .*        ***.*****.*****.**                          |
  200 +**-***-*--***-*--****-***--------------------------------------------+


                               time.minor_page_faults

  126000 ++-----------------------------------------------------------------+
         |                                                                  O
  125000 ++                                                        OO OO  O |
  124000 ++ O O   O    O               O  O O   O O     O        OO        O|
         |     OOO  OO   O  O    O O O  OO O OO  O O     O OO O O    O      |
  123000 OO        O      OO O OO O           *.*   OOO      O              |
  122000 ++O       *       *    *  *O*   * *  : :         O              O  |
         |       ** * .*O* : *. :: :* : * ::*:   :*                         |
  121000 ++      :   *  * * :: * **   : : *  *   *                          |
  120000 ++      :          *          *                                    |
         |       :                                                          |
  119000 *+* .* :                                                           |
  118000 +* *  ::                                                           |
         |     **                                                           |
  117000 ++-----------------------------------------------------------------+


                          time.voluntary_context_switches

  5000 ++-------------------------------------------------------------------+
       |                                                                    |
       |                                                                    |
  4500 ++                                      *                            |
       |        *.    *      *          * *  .*:                            |
       |        : * ** :  *  :*.* *     :*: *   :*                          |
  4000 +**.*****   *   *.* :*    :::*  :   ::   *                           |
       *                   *     * * *.*   :                                |
  3500 ++                                  *                                |
       |                                                                    O
       |                                                           O  O     |
  3000 ++  O                                                     O  OO O  OO|
       |      O O      O   OOOO   O OO  OOOO  OOOO     O    OO          O   |
       OOO  OO O  OOOOO  OO      O O   O    O     OO OO OOO   O O           |
  2500 ++-----------------------O------------------------------O------------+


                          time.involuntary_context_switches

  300000 ++-----------------------------------------------------------------+
         |OO  OOOOOOOO O  OOOO O   OOO  O OO O  OOOOOOO  OOO OO OOOOOO O OO O
  280000 O+ O           OO      OOO    O O  O O         O   O         O    O|
         |                                                                  |
  260000 ++                                                                 |
         |                                                                  |
  240000 ****.***                                                           |
         |      :                                                           |
  220000 ++     :                                                           |
         |      :                                                           |
  200000 ++      :                                                          |
         |       :                                                          |
  180000 ++      :                                                          |
         |       :*   .* *      * ****.********.***                         |
  160000 ++------*-***--*-****-*-*------------------------------------------+


                                    uptime.boot

  500 ++--------------------------------------------------------------------+
      |                                                                     |
      |     OOOO OO OO OOOOO OOOOOO OOOOO OOOOO OOOOOO OOOOO OOOOO OOO OO OO|
  450 OOO OO       O                                                  O     O
      |                                                                     |
      |                                                                     |
  400 ++                                                                    |
      |                                                                     |
  350 ++                                                                    |
      |                                                                     |
      |                                                                     |
  300 ++                                                                    |
      |                                                                     |
      ***.******.** **.*  **.** ***.*****.*****.**                          |
  250 ++-----------*----**-----*--------------------------------------------+


                               vm-scalability.throughput

    4e+07 ++----*-----*----**-----------------------------------------------+
          *****  ***** *.**  *****.* * ** *. *   *                          |
  3.5e+07 ++                        * *  *  * *** *                         |
          |                                                                 |
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
  1.5e+07 OOOOO OOO OOOO OOOOOOOOO OOO OOOO  OOOOOOO OOOOOOOOO OOO          |
          |        O                  O     O                     OOOOO OOOOO
    1e+07 ++----------------------------------------------------------------+


                              vm-scalability.stddev

  10 ++---------------------------------------------------------------------+
     |     O         O         O  O O     O           O                     |
   9 ++     O  O      O  O O  O                O  OO       OO               |
   8 +OO O         O   O     O   O O   OOO   OO O    O    O   OOO     OO   O|
     O    O  O  OOO     O            O      O       O   OO        O O   O O O
   7 ++                     O                                    O   O      |
   6 ++                                                                     |
     |                                                                      |
   5 ++                                                                     |
   4 ++                                                                     |
     |                                                                      |
   3 +**. *   .*** *.* *                                                    |
   2 *+  * :**    *   * **.*****.*****.****.*****                           |
     |     *                                                                |
   1 ++---------------------------------------------------------------------+




  50 ++-------------------------------------------------------O-------------+
     |          O    O            O                            OO    O      |
  45 OO   OOOO    O            O    O     O    OO   O O O OOO    O          |
  40 ++O       O O O  OOOO    O                    O O            O       OOO
     |                     OO    O O O O O  OO    O                 O   O   |
  35 ++  O                              O     O          O            O     |
     |                       O                                         O    |
  30 ++                                                                     |
     |                                                                      |
  25 ++                                                                     |
  20 ** .*  * .*   *. *      *            *.*****                           |
     | *  ** * : **  * ***.** **.*****. **                                  |
  15 ++         ::                     *                                    |
     |          *                                                           |
  10 ++---------------------------------------------------------------------+




  50 ++---------------------------------------------------------------------+
     |                       O   O   O O    OO                      O O O   |
  45 ++O O       O O    OO    O                   OO              O    O   OO
     OO   OOO  O      OO       O    O   O O   OOO    O  O OO              O |
     |       O    O         O     OO     O          O O  O  O    O   O      |
  40 ++         O          O                                  OOO           |
     |               O                                                      |
  35 ++                                                                     |
     |          *                                                           |
  30 ++      *. :**      *.   **                                            |
     ***.   *  *   *.*  *  ***  : ***     *.**                              |
     |   ***          **        : :  :.***    **                            |
  25 ++                          :   *          *                           |
     |                           *                                          |
  20 ++---------------------------------------------------------------------+




  11 ++---------*-----------------------------------------------------------+
     |          :                                                           |
  10 ++         :                                                           |
   9 ++         :                                                           |
     |         : :                                                          |
   8 ++        : :     **              *                                    |
   7 ++*. ****.* :* .**  *.*****.**** + *  .** *                            |
     **  *       * *                 *   **   * :                           |
   6 ++                                         *                           |
   5 ++                                                                     |
     |                                                                      |
   4 ++                                                                     |
   3 ++                    O                                                |
     |OO  O OO OO OO OOOO   O    OOO    OO    O   OOOOO  O     OOOO O  O  O |
   2 O+--O-O-----O-------O---OOO----OO-O--O-OO-OO-------O-OOO-O------OO-O--OO




    3 ++--------------------------------------------------------------------+
      |          *                                                          |
  2.5 ++         :                                                          |
      |         ::                                                          |
      |         : :                                                         |
    2 ++        : :     **                                                  |
      | *.*****:  :* *.*  **.******.** **.  **                              |
  1.5 **       *  * *                 *   **  *.**                          |
      |                                                                     |
    1 ++                                                                    |
      |                    O                                                |
      |OO     O   O  O OOO   OO     O   O O   O O OOOO   O     O O  O  OO O |
  0.5 O+  OOOO O O OO     O    OOOO  OOO   OOO   O     OO OO OO O  O OO    OO
      |                                                                     |
    0 ++--------------------------------------------------------------------+




  1.8 ++-----------------------------------*--------------------------------+
      |                                    :                                |
  1.6 ++                                   :                                |
  1.4 ++       *.                          ::                               |
      |        : *        *               : :                               |
  1.2 ++       : :       * :       .*     : :                               |
      |     ** :  : * .**  *  *   * :  *  : **  **                          |
    1 *+*.** ::   * :*     O:O:   : O: :O.O   O+                       O    |
      |*      :    :        ::O: :   ** *     *          O                  |
  0.8 ++  O   *    *         * ***                                          |
  0.6 ++                                                                    |
      |                                                                     |
  0.4 ++                                                                    |
      |    OOOOO O O O OOO     OOOO             O OOOO    OO OOOOO  OO  O OOO
  0.2 OOO---------O-O-----O----------OOO---OOO---O-----OO----------O--O-----+




  7 ++--------------------------*-------------------------------------------+
    |      *       *            :            *. *                           |
  6 ++     :+   * : :      *    :    * *.*   : *                            |
    |   ** : *  ::: **   * :*  ::    ::: :   :                              |
  5 +* : ::   *: *    :  ::  + *:  ** *   : :                               |
    * ::  :    *      *  :*   *  : :      * *                               |
  4 ++*   *            +:        ::        ::                               |
    |                   *        ::        *                                |
  3 ++                           :                                    O     |
    |                            *                      O                   |
  2 ++                                                                      |
    |                    O                      O                      O    |
  1 ++        O                                       O                     |
    |                   O       O   O     O                     OO O        |
  0 OOO-OOOO-O-OOO-OOOO---OOO-OO-O-O-OOO-O-OOO-O-OO-OO-O--OOOO-O--O--O--O-OOO




  2.5 ++--------------------------------------------------------------------+
      |                                                                     |
      |                                                                     |
    2 ++    * **  ** *     *.* *   .*                                       |
      ***.**: ::  : ::    *   * * *  ** *.**                                |
      |      :::  : * :   :     : :    ::  :  *.*                           |
  1.5 ++     :  : :   :  :       :     *    : :  *                          |
      |      *  : :    :**       *          **                              |
    1 ++        : :    *                                                    |
      |         ::                                                          |
      |         ::                                                          |
  0.5 ++        ::                                                          |
      |          :                                       O                  |
      OOO OOOOOO OOOOO OOOOO OOOOOO OOOOO OOOOO OOOOOO OO OO OOOOO OOOOOO OOO
    0 ++---------*----------------------------------------------------------+




    5 ++--------------------------------------------------------------------+
  4.5 ++   **         .*                         *                          |
      |    ::       ** :                         :                          |
    4 *+  * :       :   :             *          :                          |
  3.5 ++  : :   .*  :   *             :         :                           |
      |: :   : * : :     : *      *  : :*       :                           |
    3 +* :   : :  ::     * :+ * * :+ * ::+      *                           |
  2.5 ++*    ::   **      *  * * *  *  *  ***  :                            |
    2 ++     **                              : :                            |
      |                                      **                             |
  1.5 ++                                                                    |
    1 ++                                                                    |
      |                                                                     |
  0.5 ++  O                O O          O     O          O           O      |
    0 OOO--OOOOO-OOOOO-OOOO---OOOOO-OOOO--OOOO--OOOOOO-OO-OO-OOOOO-OO-OOO-OOO




  1.8 ++--------------------------------------------------------------------+
      |                                *                                    |
  1.6 ++     *           *       *     ::                                   |
  1.4 ++*. ** **   * *. * **.* * :*.**: *.  **                              |
      **  *    :  * :: *      * *     *   **  :.*                           |
  1.2 ++       :  : *                         *  *                          |
    1 ++        : :                                                         |
      |         : :                                                         |
  0.8 ++        : :                                                         |
  0.6 ++        : :                                                         |
      |         ::                                                          |
  0.4 ++        ::                                                          |
  0.2 ++         :           O                           O                  |
      OOO OOOOOO OOOOO OOOOO  OOOOO OOOOO OOOOO OOOOOO OO OO OOOOO OOOOOO OOO
    0 ++---------*----------------------------------------------------------+




  1.4 ++-------------------------------*------------------------------------+
      |                  *             :                                    |
  1.2 ++     *          *:       *     :    **                              |
      |      :         * :       :     ::   ::                              |
    1 ++     ::        :  :      ::   : :   : :                             |
      |     : :       :   :     : :   : :  :  :                             |
  0.8 ***.*** **  *   :   **. *** *. ** *.*:  *.**                          |
      |        :  :***       *      *      *                                |
  0.6 ++        : :                                                         |
      |         ::                                                          |
  0.4 ++         :                                                          |
      |          *           O                           O                  |
  0.2 ++  O               OO  O     O     O         O              O        |
      OOO  OOOOO OOOOO OOO     OOOO  OOOO  OOOO OOOO O OO OO OOOOO  OOOOO OOO
    0 ++--------------------------------------------------------------------+




  8 ++----------------*-----------------------------------------------------+
    |    **           ::                                                    |
  7 ++ .* :          : :                                                    |
  6 *+*    :     *.***  * *     *    *          *                           |
    |*     :.* * :      : :**.  :*.* :*         :                           |
  5 ++     *  ::*        :    **    *  *.      :                            |
    |         *          *               * *  .*                            |
  4 ++                                    * **                              |
    |                                                                       |
  3 ++                                                                      |
  2 ++                                                                      |
    |                                                                       |
  1 ++                                                                      |
    |                                                                       |
  0 OOO-OOOO-OOOOO-OOOO-OOOOO-OOOO-OOOOO-OOOOO-OOOO-OOOOO-OOOO-OOOOO-OOOO-OOO


                                cpuidle.C6-IVT-4S.time

   1.4e+10 ++---------------------------------------------------------------+
           |                                          O                     |
  1.35e+10 ++                                                               |
           |                             O    O                             |
           |      O   O             OO O    O     O       O   OOO           |
   1.3e+10 ++        O   OO OOO  OOO      OO   OO  OOO O     O   O      OO  |
           |     O OO   O      OO       O    O          OO O                |
  1.25e+10 ++          O                                          OOO       |
           |                                                          O   OOO
   1.2e+10 ++                            *                           O      |
           O O          *                :*                                 |
           *O OO.       ::*.        **  *  * **   **                        |
  1.15e+10 +* :  ** *  * *  ****   *  + :  : : : +                          |
           | *     * **         ***    *    :  **                           |
   1.1e+10 ++-------------------------------*-------------------------------+


                               cpuidle.C6-IVT-4S.usage

  150000 ++-----------------------------------------------------------------+
         |               O                                                  |
  140000 ++                                                                 |
         |          O               O  O    O    O    O O     O             |
         | O   O O                O      O    O    O O           O          |
  130000 ++     O OO    O O OO  O       O O  O  O   O      O O              |
         |           O O         O O              O      OO                 |
  120000 OO O              O         O     O                O   O      O  O O
         |    O                O                                  OO     O O|
  110000 ++                                                          OO     |
         |         *  .** * * .**  **   **   **.* *                 O       |
         |        * **   * * *   **  *.*  ***    *                          |
  100000 ++  .*  *                                                          |
         ****  :*                                                           |
   90000 ++----*------------------------------------------------------------+


                                  turbostat.%c0

  80 ++---------------------------------------------------------------------+
     |                                                                      |
  75 OOO OO                                                          O      |
     |     OOO OOOOO OOOOO OOOOO OOOOO OOOO OOOOO OO OO OOOOO OOOOO O OOO OOO
     |                                              O                       |
  70 ++                                                                     |
     |                                                                      |
  65 ++                                                                     |
     |                                                                      |
  60 ++                                                                     |
     |                                                                      |
     |                             *     *   **                             |
  55 ***.*****.*****.*****.*****.** **.** *.*  **                           |
     |                                                                      |
  50 ++---------------------------------------------------------------------+


                                   turbostat.%c1

  2.8 ++--------------------------------------------------------------------+
  2.6 ++         *     *                                                    |
      |          :     :                                                    |
  2.4 ++        ::    : :                                                   |
  2.2 ++        : :   : :                                                   |
      |        *  : **  *       *   *                                       |
    2 ++       :  * :    ***. * :* + :* *. ** *. *                          |
  1.8 +**.**O**    *         * *  *  O *  *O *  *                           |
  1.6 *+    *                                                               |
      | O              O   O            O     O   O  O                      |
  1.4 ++      O                 O O    O     O  O  O   O     OO O          O|
  1.2 ++     O O OO     O      O      O     O             O                 |
      OO   O       OOO   OO   O  O                  O   OO O   O   O  O     |
    1 ++  O                  O      O     O      O               O   O OO O O
  0.8 ++------------------------------------------------------------O-------+


                                  turbostat.%c6

  50 ++---------------------------------------------------------------------+
     |                                                                      |
  45 ++   **                                                                |
     ***.*  **.*****.*****.*****.** **.** *.** **                           |
     |                             *     *    *                             |
  40 ++                                                                     |
     |                                                                      |
  35 ++                                                                     |
     |                                                                      |
  30 ++                                                                     |
     |                                                                      |
     |      O    O   O OOO   O O OOO O   O  O  OO   O    O OO OO       OO   |
  25 OOO OOO O OO OO  O    OO O     O  OO O  OO   OO OO O O     OOO OOO   OOO
     |                                                                      |
  20 ++---------------------------------------------------------------------+


                                 turbostat.%pc2

  35 ++---------------------------------------------------------------------+
     |                                                                      |
  30 *+    *                                                                |
     |**.** :*   **     **. **    * * .**  .** **                           |
     |      * + *  *.  *   *  * .* * *   **   *                             |
  25 ++        *     * :       *                                            |
     |                :                                                     |
  20 ++               *                                                     |
     |      O    O   O       O    O  O      O       O   OO                  |
  15 +O  OO  O OO OO  OOOO OO OO O O   OOO   OOOO OO OO   OOO OOOOO OO OO O O
     O O                                                              O     |
     |                              O     O                                 |
  10 ++    O                                                               O|
     |                                                                      |
   5 ++---------------------------------------------------------------------+


                                  turbostat.Pkg_W

  420 ++--------------------------------------------------------------------+
      OOO OOO      O                 O     O                               O|
  410 ++     OOO OO OO OOOOO OOOOOO O OOO O OOO OOOO      OO    OO OOOO   O O
  400 ++                                             O OOO   OOO       OO   |
      |                                             O                       |
  390 ++                                                                    |
      |                                                                     |
  380 ++                                                                    |
      |                                                                     |
  370 ++                                                                    |
  360 ++                                                                    |
      |      * *.*   *.*        ***.*    .*   *.                            |
  350 +**. ** *   ***   ****.***     * **  ***  **                          |
      *   *                           *                                     |
  340 ++--------------------------------------------------------------------+


                                  turbostat.Cor_W

  350 ++--------------------------------------------------------------------+
      OOO OOO                        O     O                                |
  340 ++      OO  OOO  OO  O O O       O     O  O O                  OO    OO
  330 ++     O   O   O   OO   O OOO O O O O O O  O O O OOOOO OOOOO OO  OO O |
      |                                             O                       |
  320 ++                                                                    |
      |                                                                     |
  310 ++                                                                    |
      |                                                                     |
  300 ++                                                                    |
  290 ++                                                                    |
      |               .*        *  .*         *                             |
  280 ++     ***.*****  ****. ** **  :   .* ** + *                          |
      ***. **                *       ****  *    *                           |
  270 ++--*-----------------------------------------------------------------+


                                 turbostat.RAM_W

  98 ++---------------------------------------------------------------------+
     | *.    *.        **   **                                              |
  96 **  ****  *****.**  *.*  **. ***   ***.*****                           |
     |                           *   *.*                                    |
  94 ++                                                                     |
     |                                                                      |
  92 ++                                                                     |
     |                                                                      |
  90 ++                                                                     |
     |                                                                      |
  88 ++                                                                     |
     |                                                                      |
  86 OOO OOOOO  OOOO OOOOO OOOOO  O OO OO O OOOOO OO OO OOOOO OOO           |
     |         O                 O O     O          O            OO OOOOO OOO
  84 ++---------------------------------------------------------------------+


                             numa-numastat.node0.numa_hit

  2.6e+08 ++----------------------------------------------------------------+
          |****.*   * * .* ***** *. ** ****.* *****                         |
  2.5e+08 *+     *** * *  *     *  *  *      *                              |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          |                                                                 |
  2.1e+08 O+O    OO OO   O    OOO    O OOO   OOO      OO OOOO  O   OO     O |
          |O OO O  O  OO   OOO   O  O     O     OOOO O  O    O  OOO  OO OO OO
    2e+08 ++              O        O  O     O                               |
          |                                                                 |
  1.9e+08 ++----------------------------------------------------------------+


                             numa-numastat.node0.numa_miss

  4.5e+07 ++-------------------------------------O-----------O--O-----------+
          |O    OOO  OO   OO       OOOOO  O OO  O  O O OO OO     O          |
    4e+07 O+OOO    OO  O O  OOOOOO      OO    OO  O   O  O  O  O            |
          |                                                         OO  O   |
  3.5e+07 ++                                                      O      OOOO
          |                                                        O  O     |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          *       *    *.  *      .*  *      *                              |
  1.5e+07 +****. * *** : ** * ****  :* : **. :* ***                         |
          |     *     *      *      *  **   *  *                            |
    1e+07 ++----------------------------------------------------------------+


                           numa-numastat.node0.numa_foreign

    4e+07 ++----O-O--------O--O-----------O-----O---------------------------+
          |OO    O OOOOO OO O  OOO OOOOOOO  OOOO OOO  O   OOOO OOO          |
  3.5e+07 O+ OO              O                       O OOO        OO  O  O O|
          |                                                         OO  O O O
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++           *. *                                                 |
          *  *    **** : * ** ****.** * ***. * ****                         |
    1e+07 +** *.**    *      *       * *    * *                             |
          |                                                                 |
    5e+06 ++----------------------------------------------------------------+


                            numa-numastat.node0.local_node

  2.6e+08 ++----------------------------------------------------------------+
          |****.*   * * .* ***** *. ** ****.* *****                         |
  2.5e+08 *+     *** * *  *     *  *  *      *                              |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          |                                                                 |
  2.1e+08 O+O    OO OO   O    OOO    O OOO   OOO      OO OOOO  O   OO     O |
          |O OO O  O  OO   OOO   O  O     O     OOOO O  O    O  OOO  OO OO OO
    2e+08 ++              O        O  O     O                               |
          |                                                                 |
  1.9e+08 ++----------------------------------------------------------------+


                            numa-numastat.node0.other_node

  4.5e+07 ++-------------------------------------O-----------O--O-----------+
          |O    OOO  OO   OO       OOOOO  O OO  O  O O OO OO     O          |
    4e+07 O+OOO    OO  O O  OOOOOO      OO    OO  O   O  O  O  O            |
          |                                                         OO  O   |
  3.5e+07 ++                                                      O      OOOO
          |                                                        O  O     |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          *       *    *.  *      .*  *      *                              |
  1.5e+07 +****. * *** : ** * ****  :* : **. :* ***                         |
          |     *     *      *      *  **   *  *                            |
    1e+07 ++----------------------------------------------------------------+


                             numa-numastat.node1.numa_hit

  2.6e+08 ++----*-----*---------------------*-------------------------------+
          *****  ***** *.*********.********  ******                         |
  2.5e+08 ++                                                                |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          |                                                                 |
  2.1e+08 ++                                                   O            O
          O O O   O OOO        O        OO    OO O O  OO O O         O      |
    2e+08 +O O  O  O   O  OOOO  OO OOO O  O  O  O    O  O O OO  OOOOO O OOO |
          |      O       O    O       O     O     O                        O|
  1.9e+08 ++----------------------------------------------------------------+


                             numa-numastat.node1.numa_miss

  4.5e+07 ++------------------O---------------------------------------------+
          |OO   OO  O O  OOOO   O    OOOO O OOO O O    O  O  O  OO         O|
    4e+07 O+ O    OO O O     O O O OO    O     O O O OO  O OO     OO    O   |
          |   O                                         O      O    OOO  OO |
  3.5e+07 ++                                                                O
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++       *   *.**                                                 |
          *  **. ** ** :   *******.********. ******                         |
    1e+07 +**---*-----*---------------------*-------------------------------+


                           numa-numastat.node1.numa_foreign

    4e+07 ++----------------------------------------------------------------+
          |       O            O          O     O     O   O                 |
  3.5e+07 OOOOO O  OOOOO OOOOOO OO O OOOOO  OOOO OOO O O O OOO OOO          |
          |      O                  O                   O         O O   O O |
    3e+07 ++                                                       O OO  O OO
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                    *                                           |
          ****   *******.*** *** :.***** **.*** ***                         |
    1e+07 ++  *.*           *    *      *      *                            |
          |                                                                 |
    5e+06 ++----------------------------------------------------------------+


                            numa-numastat.node1.local_node

  2.6e+08 ++----*-----*---------------------*-------------------------------+
          *****  ***** *.*********.********  ******                         |
  2.5e+08 ++                                                                |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          |                                                                 |
  2.1e+08 ++                                                   O            O
          O O O   O OOO        O        OO    OO O O  OO O O         O      |
    2e+08 +O O  O  O   O  OOOO  OO OOO O  O  O  O    O  O O OO  OOOOO O OOO |
          |      O       O    O       O     O     O                        O|
  1.9e+08 ++----------------------------------------------------------------+


                            numa-numastat.node1.other_node

  4.5e+07 ++------------------O---------------------------------------------+
          |OO   OO  O O  OOOO   O    OOOO O OOO O O    O  O  O  OO         O|
    4e+07 O+ O    OO O O     O O O OO    O     O O O OO  O OO     OO    O   |
          |   O                                         O      O    OOO  OO |
  3.5e+07 ++                                                                O
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++       *   *.**                                                 |
          *  **. ** ** :   *******.********. ******                         |
    1e+07 +**---*-----*---------------------*-------------------------------+


                             numa-numastat.node2.numa_hit

  2.65e+08 ++---------------------------------------------------------------+
   2.6e+08 +*  *.*                *          *                              |
           * **   ******** .****** ***.****** ***.**                        |
  2.55e+08 ++             *                                                 |
   2.5e+08 ++                                                               |
           |                                                                |
  2.45e+08 ++                                                               |
   2.4e+08 ++                                                               |
  2.35e+08 ++                                                               |
           | O                                         O                    |
   2.3e+08 ++  O O O      O  O O  O     OO   OO   O  O                      |
  2.25e+08 ++     O  OO  O  O O  O OOO    O    O   OO   OO O  OOOO  O       |
           OO       O  O        O      O   OO   O     O   O        O OO O  OO
   2.2e+08 ++ O         O                                    O    O      OO |
  2.15e+08 ++---------------------------------------------------------------+


                             numa-numastat.node2.numa_miss

    4e+07 ++----------------------------------------------------------------+
          |                    O          O     O     O                     |
  3.5e+07 ++  O O O  O O O       O      O     O   OO      O O  OOO          |
          OOOO     OO O   OOOOO O  OOOOO O  OO O O   O O O O O      O     O |
    3e+07 ++     O                                      O         OO OO OO OO
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++              *                                                 |
          |           **. :*  * * .** **      * * *                         |
    1e+07 *****.******   *  ** * *   *  ***.** * *                          |
          |                                                                 |
    5e+06 ++----------------------------------------------------------------+


                           numa-numastat.node2.numa_foreign

    5e+07 ++-------------------------O--------------------------------------+
          |     OO  O O  O O OO OO O   O     O         OO O      O          |
  4.5e+07 +O OO   OO O O  O O  O    O O OOO O OOOOOO OO  O OOO  O           |
    4e+07 O+O                                                  O  O     O  O|
          |                                                         OOO   O |
  3.5e+07 ++                                                       O     O  O
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 *+      *  * *.*    * *                                           |
          |****.** ** *   **** * *.********.*******                         |
    1e+07 ++----------------------------------------------------------------+


                            numa-numastat.node2.local_node

  2.65e+08 ++---------------------------------------------------------------+
   2.6e+08 +*  *.*                *          *                              |
           * **   ******** .****** ***.****** ***.**                        |
  2.55e+08 ++             *                                                 |
   2.5e+08 ++                                                               |
           |                                                                |
  2.45e+08 ++                                                               |
   2.4e+08 ++                                                               |
  2.35e+08 ++                                                               |
           | O                                         O                    |
   2.3e+08 ++  O O O      O  O O  O     OO   OO   O  O                      |
  2.25e+08 ++     O  OO  O  O O  O OOO    O    O   OO   OO O  OOOO  O       |
           OO       O  O        O      O   OO   O     O   O        O OO O  OO
   2.2e+08 ++ O         O                                    O    O      OO |
  2.15e+08 ++---------------------------------------------------------------+


                            numa-numastat.node2.other_node

    4e+07 ++----------------------------------------------------------------+
          |                    O          O     O     O                     |
  3.5e+07 ++  O O O  O O O       O      O     O   OO      O O  OOO          |
          OOOO     OO O   OOOOO O  OOOOO O  OO O O   O O O O O      O     O |
    3e+07 ++     O                                      O         OO OO OO OO
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++              *                                                 |
          |           **. :*  * * .** **      * * *                         |
    1e+07 *****.******   *  ** * *   *  ***.** * *                          |
          |                                                                 |
    5e+06 ++----------------------------------------------------------------+


                             numa-numastat.node3.numa_hit

  2.6e+08 ++----**----------------------------**----------------------------+
          *****   ******.*********.********.**  ***                         |
  2.5e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          | O             O              O     O   O               O        O
          OO OO OOO  OOO   OO O O   O   O O   O OOO  O O OOO O OOOO OO  OOO |
  2.1e+08 ++       OO        O O   O OOO    OO        O O             O    O|
          |              O       O                          O               |
    2e+08 ++----------------------------------------------------------------+


                             numa-numastat.node3.numa_miss

  4.5e+07 ++-------------O-----------O-O-----O------------------------------+
          |     OO  O O    OOOO OO O      O     O  O  OOOOOOO    O          |
    4e+07 OOOOO   OO O O  O    O      O OO  O OO OO  O       O OO           |
          |                         O                                       |
  3.5e+07 ++                                                      O O O O OOO
          |                                                        O O   O  |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                 *  *                                           |
          *****.  ******.**** ** :.********.*   ***                         |
    1e+07 ++----**---------------*-----------***----------------------------+


                           numa-numastat.node3.numa_foreign

  4.5e+07 ++----------------------------------------------------------------+
          |     OO   O O             O           O           O  OO          |
    4e+07 OOOOO   OOO O  OOOO OOOO OO OOO O OOO O OO OOOO OOO  O            |
          |                  O           O     O         O        O OO  OO  |
  3.5e+07 ++                                                       O  O   OO|
          |                                                                 O
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 *+      ** * *. *       .*  *                                     |
          |****. *  * *  * *******  ** ****.*******                         |
    1e+07 ++----*-----------------------------------------------------------+


                            numa-numastat.node3.local_node

  2.6e+08 ++----**----------------------------**----------------------------+
          *****   ******.*********.********.**  ***                         |
  2.5e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          | O             O              O     O   O               O        O
          OO OO OOO  OOO   OO O O   O   O O   O OOO  O O OOO O OOOO OO  OOO |
  2.1e+08 ++       OO        O O   O OOO    OO        O O             O    O|
          |              O       O                          O               |
    2e+08 ++----------------------------------------------------------------+


                            numa-numastat.node3.other_node

  4.5e+07 ++-------------O-----------O-O-----O------------------------------+
          |     OO  O O    OOOO OO O     OO    OO  O  OOOOOOO    O          |
    4e+07 OOOOO   OO O O  O    O      O O   O O  OO  O       O OO           |
          |                         O                                       |
  3.5e+07 ++                                                      O O O O OOO
          |                                                        O O   O  |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                 *  *                                           |
          *****.  ******.**** ** :.********.*   ***                         |
    1e+07 ++----**---------------*-----------***----------------------------+


                                 vmstat.procs.r

  95 ++---------------------------------------------------------------------+
     |                                                                      |
  90 OOO OOO O OOOOO OOOOO OOOOO O OO  OOOO  OOOO OO OO OOOO  OOOOO OOOOO OOO
     |      O                     O  O      O       O       O               |
  85 ++                                                                     |
     |                                                                      |
  80 ++                                                                     |
     |                                                                      |
  75 ++                                                                     |
     |                                                                      |
  70 ++                                                                     |
     |                                   *   **                             |
  65 ***.*****.*****.*****.*****.*****.** *.*  **                           |
     |                                                                      |
  60 ++---------------------------------------------------------------------+


                                  vmstat.memory.free

  1.4e+08 ++--------------*-------------------------------------------------+
          |   *.*     *   ::            *        *                          |
  1.3e+08 ** *   ***** *.* *******.*** * **.***** *                         |
          | *                         *                                     |
  1.2e+08 ++                                                                |
          |                                                                 |
  1.1e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    9e+07 ++                                                                |
          |                                           O                     |
    8e+07 ++     O          O  OOO  OOO O O O O OOO       O OO  O       OO  |
          OOOOO O OOOOOO OOO OO    O   O O   O O   O O OOO O   O OOOOOO   OOO
    7e+07 ++----------------------------------------------------------------+


                                  vmstat.memory.cache

  4.5e+08 +O-O--------------------------------------------------------------+
          O O O O OOOOOO OOO OOOO  O   O OO  O OO  O O OOO O   O OOOOOO  OOOO
  4.4e+08 ++     O          O    O  OOO O   O O  OO       O OO  O       O   |
          |                                           O                     |
  4.3e+08 ++                                                                |
          |                                                                 |
  4.2e+08 ++                                                                |
          |                                                                 |
  4.1e+08 ++                                                                |
          |                                                                 |
    4e+08 ++                                                                |
          | *                         *                                     |
  3.9e+08 ** *   *******.* *******.*** * **.*******                         |
          |   *.*         *             *                                   |
  3.8e+08 ++----------------------------------------------------------------+


                                  vmstat.system.in

  120000 ++-----------------------------------------------------------------+
         |                                 *  *.                            |
  115000 ****.  *** *    *  **.*******.* ** **  ***                         |
  110000 ++   **  : :*.** **            *                                   |
         |        : :                                                       |
  105000 ++       : :                                                       |
  100000 ++        :                                                        |
         |         :                                                        |
   95000 ++        :                                                        |
   90000 ++   O    *                                                        |
         OOO   OOO OOO OOOOOO    OOOO  OOO  OOO   O O O OO OOOO OO OOO      O
   85000 ++       O                              O O O            O   O  OOO|
   80000 ++ O                  O                                            |
         |                   O  O    O    OO    O         O            O    |
   75000 ++-----------------------------------------------------------------+


                                 vmstat.system.cs

  3600 ++-------------------------------------------------------------------+
       |                                                                    |
  3400 ***.****                                                             |
  3200 ++     :                                                             |
       |      :                                                             |
  3000 ++      :                                                            |
       |       :                                                            |
  2800 ++      **.* *   .******.* *          .**                            |
       |           * ***         * ***.******   **                          |
  2600 ++                                                                   |
  2400 ++                                                                   |
       |                                                                    |
  2200 ++                                                                   |
       OOO OOOOOO OOOOOO OOOOOO OOOOOO OOOOOO OOOOOO OOOOOO  OOOOO OOOOOO OOO
  2000 ++---------------------------------------------------O---------------+


                                    softirqs.TIMER

  1.5e+07 ++-------------------O--------O-----O----------------O------------+
          OOOOO OOOOOOOO OOOOOO OO OOOOO OO OO OOOOO OOOOOOOOO  OOOOOOO OOOOO
  1.4e+07 ++                                                                |
  1.3e+07 ++                                                                |
          |                                                                 |
  1.2e+07 ++                                                                |
  1.1e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
    9e+06 ++                                                                |
          |                                                                 |
    8e+06 ++       *              .*****  *.***** *                         |
    7e+06 *****.*** ****.**  *****      **       *                          |
          |                **                                               |
    6e+06 ++----------------------------------------------------------------+


                                   softirqs.SCHED

  350000 ++-----------------------------------------------------------------+
         OOOO OOOOO O   OOO OO OOOOOOO OOO OOOO  OOOOO  OOOOOOO OOOOOOOO OOOO
         |                                                                  |
  300000 ++                                                                 |
         |                                                                  |
         |                                                                  |
  250000 ++                        *         * .*                           |
         ****.*****O*O.O** O   **** **.*  O** * O *   O                     |
  200000 ++              :     :       :  :     : :                         |
         |               :    :        :  :     : :                         |
         |                :   :         ::       ::                         |
  150000 ++               :   :         ::       :                          |
         |                :  :          :*       :                          |
         |                ****          *        *                          |
  100000 ++-----------------------------------------------------------------+


                                    softirqs.RCU

  900000 ++-----------------------------------------------------------------+
         |                          OO     O                              O |
  800000 OO               O                        O        OO             O|
         | O  OOOO  O   OO OOO OOOOO   O    O        OO OOO   O OO O     O  O
  700000 ++ O     OO O O                OOO  OO OOO O      O      O OOOO    |
         |                                                                  |
  600000 ++                                                                 |
         |                                                                  |
  500000 ++                                                                 |
         |                                                                  |
  400000 +**                                                                |
         *  :   * **  .*        *  **                                       |
  300000 ++ *.** *  **  *     .* **  *.* ******.* *                         |
         |               *****          *        *                          |
  200000 ++-----------------------------------------------------------------+


                               proc-vmstat.nr_free_pages

  3.6e+07 ++----------------------------------------------------------------+
          |               *             *                                   |
  3.4e+07 ** **.**  * **.* ** ** *.*** * **. *   **                         |
  3.2e+07 ++*     ** *       *  *     *     * ***                           |
          |                                                                 |
    3e+07 ++                                                                |
  2.8e+07 ++                                                                |
          |                                                                 |
  2.6e+07 ++                                                                |
  2.4e+07 ++                                                                |
          |                                                                 |
  2.2e+07 ++                                          O                     |
    2e+07 ++     O         O     O  O O O   O O   O         O               |
          OOO O O OOOOOO OO OOOOO  O O O OO  O OOO O O OOOOO O OOOOOOOO OOOOO
  1.8e+07 ++-O--------------------------------------------------------------+


                             proc-vmstat.nr_inactive_file

  1.12e+08 ++---------------------------------------------------------------+
           OOOOO O OO  OO O   OO   O    O O  O O    O  OOO O     O O OO   OOO
   1.1e+08 ++     O  OO  O  OO  OOO OO O O OO O O OO O    O  OOOO O O   OO  |
  1.08e+08 ++                                         O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 ++*      * *       *  *    .*      * *.                          |
   9.6e+07 ** **. ** * *** .** ** ****  * **** *  **                        |
           |     *        *              *                                  |
   9.4e+07 ++---------------------------------------------------------------+


                              proc-vmstat.nr_anon_pages

  12800 ++------------------------------------------------------------O-----+
  12600 ++   O       O                             O    O          OO    O  |
        |     O       O  O  O O O  OO  O     O      OO    OO O            O O
  12400 ++OO   O    O   O O  O   O    O  O  O  O O    OO    O OO     O O   O|
  12200 OO      OO     O       O                O               O           |
        |         O                  O  O  O  O                   O         |
  12000 ++                                                                  |
  11800 ++                                                                  |
  11600 ++                                                                  |
        |                                                                   |
  11400 +*                                    *                             |
  11200 ++:*  * *  .*         *  *.*     *.*  ::                            |
        * * + :* :*  :****   * : :  **  *   :* *                            |
  11000 ++   *   *   *    *.*  **     **    *   **                          |
  10800 ++------------------------------------------------------------------+


                               proc-vmstat.nr_mapped

  6000 *+-----------------------*-------------------------------------------+
       |                       ::  *      *      *                          |
  5500 ++*   *        *        : : :   *  :**    :                          |
       |::+  : *      :*.*    *  ** :  : *  :    :                          |
       |*  ** ::*.   *   :  **      * : ::   : * :                          |
  5000 ++     :   ***     :*         :: *    : : :                          |
       |      *           ::         *        :::                           |
  4500 ++                 *                   * :                           |
       |                                        :                           |
  4000 +O            O O         O              *       O   O      O        |
       |    O OOO  OO O  O        OO O OO  OO O    O O       OOO O   O  O OOO
       | O O O    O       OO    O   O     O     OOO   OO        O   O  O    |
  3500 O+                   OOO          O     O         OO           O     |
       |                                                                    |
  3000 ++-------------------------------------------------------------------+


                               proc-vmstat.nr_file_pages

  1.12e+08 +O-O-------------------------------------------------------------+
           O O O O OOOOOOOO OOOOOOOO O OO OOOO OO O OO OOOOO  OOOOOOOOO OOOOO
   1.1e+08 ++     O                 O    O    O    O         O              |
  1.08e+08 ++                                         O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 +**     ** *     * ****    .**   * ***.                          |
   9.6e+07 *+ **.**  * *** + *    ****   :** *    **                        |
           |              *              *                                  |
   9.4e+07 ++---------------------------------------------------------------+


                          proc-vmstat.nr_page_table_pages

  2500 ++-------------------------------------------------------------------+
       |                               O    O                      O      O O
  2400 +O    O               O    O               O  O OO O   OO    O O O   |
       O O OO OO   OOOOO OOOO   O  O    O     O          O  OO       O O   O|
       |        O O           O  O  OO   OOO   OOO O  O         OO          |
  2300 ++                                                                   |
       |                                                                    |
  2200 ++                                                                   |
       |                                                                    |
  2100 ++                                                                   |
       |*   ****   *        *      *        *.**                            |
       |:*.*    *. :** *.   :**   * :*.** * :   *                           |
  2000 *+         *   *  **:   + *  *    * *     *                          |
       |                   *    *                                           |
  1900 ++-------------------------------------------------------------------+


                           proc-vmstat.nr_isolated_file

  2300 ++-------------------------------------------------------------------+
  2200 ++                    *            *                                 |
       |   *   *             :*.          :                                 |
  2100 +* : * * *.**** *  * *   *         : *                               |
  2000 ++::  ::       * + :::    : * *   : ::+ *                            |
       | *   *           * *     * :* +  : :  * *                           |
  1900 *+                         *    * * *     *                          |
  1800 ++   O                           *                           O       |
  1700 ++            O                            O                         |
       O                    O      O                 O             O        |
  1600 ++           O      O      O                    O       O      O   O O
  1500 ++                O   O                     O    O O            O   O|
       | O O  OO      OO        O                     O  O  OO   O      O   |
  1400 +O       O O       O   O  O   O OOO OO OOOO            O O    O      |
  1300 ++----O-----O----------------O-----O---------------------------------+


                                 proc-vmstat.numa_hit

  1.05e+09 ++---------------------------------------------------------------+
           |****.** **** *   ** * * **. *********.**                        |
           *       *    * *.*  * * *   *                                    |
     1e+09 ++                                                               |
           |                                                                |
           |                                                                |
   9.5e+08 ++                                                               |
           |                                                                |
     9e+08 ++                                                               |
           |                                                                |
           |                                                                |
   8.5e+08 ++O O  OO OO   O OO OOO  O   OOO  OOO  O OOOO O O   OOO OOO      O
           OO O  O  O  OOO    O   OO O O   OO   O  O    O O  OO   O   O OOOO|
           |                                                                |
     8e+08 ++---------------------------------------------------------------+


                                 proc-vmstat.numa_miss

  1.8e+08 ++----------------------------------------------------------------+
          |                                                                 |
  1.6e+08 +OO   OOOOOOOO OOOO OOOO O OOOO O OOO OOOO OOO  OOOO  OO          |
          O  OO              O      O    O     O        OO     O            |
  1.4e+08 ++                                                      OOOOO OOOO|
          |                                                                 O
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    8e+07 ++                                                                |
          |                                                                 |
    6e+07 ++           *. *                                                 |
          *****. ******  * *******.********.*******                         |
    4e+07 ++----*-----------------------------------------------------------+


                               proc-vmstat.numa_foreign

  1.8e+08 ++----------------------------------------------------------------+
          |                                                                 |
  1.6e+08 +OO   OOOOOOOO OOOO OOOO O OOOO O OOO OOOO OOO  OOOO  OO          |
          O  OO              O      O    O     O        OO     O            |
  1.4e+08 ++                                                      OOOOO OOOO|
          |                                                                 O
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    8e+07 ++                                                                |
          |                                                                 |
    6e+07 ++           *. *                                                 |
          *****. ******  * *******.********.*******                         |
    4e+07 ++----*-----------------------------------------------------------+


                                proc-vmstat.numa_local

  1.05e+09 ++---------------------------------------------------------------+
           |****.** ** * *   ** * * **. *********.**                        |
           *       *  * * *.*  * * *   *                                    |
     1e+09 ++                                                               |
           |                                                                |
           |                                                                |
   9.5e+08 ++                                                               |
           |                                                                |
     9e+08 ++                                                               |
           |                                                                |
           |                                                                |
   8.5e+08 ++O O  OO OO   O OO OOO  O   OOO  OOO  O OOOO O O   OOO OOO      O
           OO O  O  O  OOO    O   OO O O   OO   O  O    O O  OO   O   O OOOO|
           |                                                                |
     8e+08 ++---------------------------------------------------------------+


                                proc-vmstat.numa_other

  1.8e+08 ++----------------------------------------------------------------+
          |                                                                 |
  1.6e+08 +OO   OOOOOOOO OOOO OOOO O OOOO O OOO OOOO OOO  OOOO  OO          |
          O  OO              O      O    O     O        OO     O            |
  1.4e+08 ++                                                      OOOOO OOOO|
          |                                                                 O
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    8e+07 ++                                                                |
          |                                                                 |
    6e+07 ++           *. *                                                 |
          *****. ******  * *******.********.*******                         |
    4e+07 ++----*-----------------------------------------------------------+


                          proc-vmstat.workingset_nodereclaim

    3e+06 ++----------------------------------------------------------------+
          *  **.*     **.  **           *                                   |
  2.8e+06 +**    *****   **  *****.***** **.*******                         |
  2.6e+06 ++                                                                |
          |                                                                 |
  2.4e+06 ++                                                                |
  2.2e+06 ++                                                                |
          |                                                                 |
    2e+06 ++                                                                |
  1.8e+06 ++                                                                |
          | O     O  O                  O     O    O  OO       OO           |
  1.6e+06 ++  O OO  O OO OOOOOOOOO  OO O OO  O OOOO  O  OOOOOO   O          |
  1.4e+06 OO O     O               O  O     O                     OOOO  OOO O
          |                                                           O    O|
  1.2e+06 ++----------------------------------------------------------------+


                               proc-vmstat.pgalloc_dma32

    3e+06 ++----------------------------------------------------------------+
          **** .***  ***.* ** ** *.********. ******                         |
  2.9e+06 ++  *    **     *  *  *           *                               |
          |                                                                 |
  2.8e+06 ++                                                                |
  2.7e+06 ++                                                                |
          |                                                                 |
  2.6e+06 ++                                                       OO     O |
          |                                                       O  O  OO  O
  2.5e+06 ++                                                          O    O|
  2.4e+06 ++                                                                |
          |                         O                          O            |
  2.3e+06 OOOOO OOO OO   O  OOOO     O OOOO  OOOOOOO OOOOOOOO   OO          |
          |        O  OO  OO    OO O  O     O                O              |
  2.2e+06 ++----------------------------------------------------------------+


                                  proc-vmstat.pgfault

  1.8e+06 ++----------------------------------------------------------------+
          |      O   O     OO       OO  O O   O O O  OO     OO              |
  1.7e+06 OOOO     O  O  O    OOO                O      O OO   O OO O O    OO
  1.6e+06 ++    O O                O                   O                O O |
          |         O  O  O  O   O    OO O  OO O   O     O      O  O O   O  |
  1.5e+06 ++  O                                                             |
  1.4e+06 ++                                                                |
          |                                                                 |
  1.3e+06 ++                                                                |
  1.2e+06 ++                      .**     *.  * * *                         |
          *  *   * **     * *    *   *  * : * :* ::                         |
  1.1e+06 +** *. :* : **.* * :  *     :* *   *   *                          |
    1e+06 ++    *    ::      * *      *                                     |
          |          *        *                                             |
   900000 ++----------------------------------------------------------------+


                          proc-vmstat.pgsteal_kswapd_dma32

  600000 ++-----------------------------------------------------------------+
         |                                                                O |
  500000 ++                                                       O OO O O  |
         |                                                         O       O|
         |                                                            O     |
  400000 ++                                                                 O
         |                                                                  |
  300000 ++                                                                 |
         |     O   O       O            O    O       OO     O O O           |
  200000 OOOO O OOO O  OOOO OO OOOOOOO O OOOO O OOOOO   O OO O   O          |
         |           O                                   O                  |
         |                                                                  |
  100000 ++           .*  *       *     *   *                               |
         ****.********  ** ***.*** ***.* *** **.***                         |
       0 ++-----------------------------------------------------------------+


                           proc-vmstat.pgsteal_kswapd_normal

  2.2e+08 ++----------------------------------------------------------------+
    2e+08 ++O   O O       OO  O OO   O    O     O  O      OO O  OO          |
          |O  O  O  OOOO O  OO O   O   OOO   OOO O   OOO       O            |
  1.8e+08 O+ O     O                O O     O     O     OO  O     O OO  O OO|
  1.6e+08 ++                                                       O  O  O  O
  1.4e+08 ++                                                                |
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
    8e+07 ++                                                                |
    6e+07 ++                                                                |
    4e+07 ++                                                                |
          |                                                                 |
    2e+07 *****.********.*********.********.*******                         |
        0 ++----------------------------------------------------------------+


                           proc-vmstat.pgsteal_direct_dma32

  2.6e+06 ++----------------------------------------------------------------+
  2.5e+06 *****.********.*********.********.*******                         |
          |                                                                 |
  2.4e+06 ++                                                                |
  2.3e+06 ++                                                                |
          |                                                                 |
  2.2e+06 ++                                                                |
  2.1e+06 ++                                                                |
    2e+06 ++                                                                |
          |                                                                 |
  1.9e+06 ++                                                                |
  1.8e+06 ++                        O    O     O                   O        |
          OOOOO OOOOOOOO OOOOOOOOO O OOOO O OOO OOOO OOOOOOOO  OOO  OOO OOO O
  1.7e+06 ++                                                 O    O        O|
  1.6e+06 ++----------------------------------------------------------------+


                           proc-vmstat.pgsteal_direct_normal

  9.5e+08 ++----------------------------------------------------------------+
          *****.********.*********.********.*******                         |
    9e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
  8.5e+08 ++                                                                |
          |                                                                 |
    8e+08 ++                                                                |
          |                                                                 |
  7.5e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
    7e+08 O+O O      O      O       O   OO    OO OOO OOOOO     O   OO O     O
          |O O  OOOOO OO OOO OOOOO O OOO  O OO  O         OOOO  OOO  O  OOOO|
  6.5e+08 ++----------------------------------------------------------------+


                           proc-vmstat.pgscan_kswapd_dma32

  600000 ++-----------------------------------------------------------------+
         |                                                                O |
  500000 ++                                                       O OO O O  |
         |                                                         O       O|
         |                                                            O     |
  400000 ++                                                                 O
         |                                                                  |
  300000 ++                                                                 |
         |     O   O       O            O    O       OO     O O O           |
  200000 OOOO O OOO O  OOOO OO OOOOOOO O OOOO O OOOOO   O OO O   O          |
         |           O                                   O                  |
         |                                                                  |
  100000 ++           .*  *       *     *   *                               |
         ****.********  ** ***.*** ***.* *** **.***                         |
       0 ++-----------------------------------------------------------------+


                           proc-vmstat.pgscan_kswapd_normal

  2.2e+08 ++----------------------------------------------------------------+
    2e+08 ++O   O O       OO  O OO   O    O     O  O      OO O  OO          |
          |O  O  O  OOOO O  OO O   O   OOO   OOO O   OOO       O            |
  1.8e+08 O+ O     O                O O     O     O     OO  O     O OO  O OO|
  1.6e+08 ++                                                       O  O  O  O
  1.4e+08 ++                                                                |
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
    8e+07 ++                                                                |
    6e+07 ++                                                                |
    4e+07 ++                                                                |
          |                                                                 |
    2e+07 *****.********.*********.********.*******                         |
        0 ++----------------------------------------------------------------+


                            proc-vmstat.pgscan_direct_dma32

  2.6e+06 ++----------------------------------------------------------------+
  2.5e+06 *****.********.*********.********.*******                         |
          |                                                                 |
  2.4e+06 ++                                                                |
  2.3e+06 ++                                                                |
          |                                                                 |
  2.2e+06 ++                                                                |
  2.1e+06 ++                                                                |
    2e+06 ++                                                                |
          |                                                                 |
  1.9e+06 ++                                                                |
  1.8e+06 ++                        O    O     O                   O        |
          OOOOO OOOOOOOO OOOOOOOOO O OOOO O OOO OOOO OOOOOOOO  OOO  OOO OOO O
  1.7e+06 ++                                                 O    O        O|
  1.6e+06 ++----------------------------------------------------------------+


                           proc-vmstat.pgscan_direct_normal

  9.5e+08 ++----------------------------------------------------------------+
          *****.********.*********.********.*******                         |
    9e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
  8.5e+08 ++                                                                |
          |                                                                 |
    8e+08 ++                                                                |
          |                                                                 |
  7.5e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
    7e+08 O+O O      O      O       O   OO    OO OOO OOOOO     O   OO O     O
          |O O  OOOOO OO OOO OOOOO O OOO  O OO  O         OOOO  OOO  O  OOOO|
  6.5e+08 ++----------------------------------------------------------------+


                        proc-vmstat.pgscan_direct_throttle

  2500 ++-------------------------------------------------------------------+
       |                                       *                            |
       |                                      *:                            |
  2000 +* .** * *.    *      * .*       * *  +  :                           |
       * *   * :: * ** *.**  :* : *     :*: *   :*                          |
       |       *   *       **    :::**.*   ::   *                           |
  1500 ++                        * *       :                                |
       |                                   *                                |
  1000 ++                                                                   O
       |                                                           O        |
       |                                                         O  OOOO  OO|
   500 ++  O    O      O   OOOO   O OO  O OO  O  O          O           O   |
       OOO  OOOO  OOOOO  OO      O     O O  O  OO OO OOOOOO  OO O           |
       |                        O  O                           O            |
     0 ++-------------------------------------------------------------------+


                               proc-vmstat.slabs_scanned

    7e+06 +*--------------------------*------*------------------------------+
          * ***.********.*********.*** ****.* *****                         |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
          |                                                                 |
  5.5e+06 ++                                                                |
          |                                                                 |
    5e+06 ++                                                                |
          | O     O  O                                 O                    |
  4.5e+06 +O  O OO  O O   OOO OOOO   O OOOO  OOOOO O OO  OOO O OOO          |
          O  O     O   O O   O     OO O     O     O     O   O      OOO      O
    4e+06 ++                                                      O   O OOOO|
          |                                                                 |
  3.5e+06 ++----------------------------------------------------------------+


                     proc-vmstat.kswapd_high_wmark_hit_quickly

  550 ++--------------------------------------------------------------------+
  500 ++            O  O                                     O          O   |
      |OO  OOOO      O  OOO      O  OOO   OOO    O  OO   O     O O  O  O   O|
  450 O+  O      OOO       O OOOO O    OO    OO O OO   OO OO  O O  O OO   O |
  400 ++       O                                                            O
  350 ++                                                                    |
  300 ++                                                                    |
      |                                                                     |
  250 ++                                                                    |
  200 ++                                                                    |
  150 ++                                                                    |
  100 ++                                                                    |
      |      *      * .*   *.   *                                           |
   50 ***.*** **.*** *  ***  *** **.*****.*****.**                          |
    0 ++--------------------------------------------------------------------+


                              proc-vmstat.pageoutrun

  2200 ++----------------------------------------------------O--------------+
       O O            O                              O           O      O   |
  2100 +O     O   OO O     O O   O     O    O            O     O    O  O   O|
       |   OOO O    O       O     O  O   O O   OO             O O    O    O O
  2000 ++              O O    O    OO     O      O O  O   O O         O     |
       |                  O     O       O     O   O    OO          O        |
  1900 ++       O                                                           |
       |                                                                    |
  1800 ++                                                                   |
       |           *                        *                               |
  1700 *+          : *  .*  *    *  **  *  * +  *                           |
       |**.***    : :::*  ** :*.* **  + :* :  * :*                          |
  1600 ++     ***.* * *      *         *  *    *                            |
       |                                                                    |
  1500 ++-------------------------------------------------------------------+


                               proc-vmstat.allocstall

  29000 ++------------------------------------------------------------------+
        ****.******.*******.******.*******.*******                          |
  28000 ++                                                                  |
  27000 ++                                                                  |
        |                                                                   |
  26000 ++                                                                  |
  25000 ++                                                                  |
        |                                                                   |
  24000 ++                                                                  |
  23000 ++                                                                  |
        |                                                                   |
  22000 ++                                                                  |
  21000 ++                                                         O        O
        O OO O  O O OOOOO O  OO    O  OOO   OOO OO OOOOOO  OO O     OOOO OO |
  20000 +O----OO-O-------O--O--OOO--OO---O-O---O----------O--O-OO-O--------O+


                            proc-vmstat.numa_pte_updates

  16000 ++------------------------------------------------------------------+
  15000 ++      O           O                                             O O
        |  O     O    O        O   O   O     O   O                 OO O     |
  14000 O+                O           O     O      OO       O O   O  O O   O|
  13000 +O   OO   O OO  OO      OO  OO  O  O  O O    O O   O   O         O  |
        |      O       O     O           O     O        O O  O  O           |
  12000 ++O           *       O                       O                     |
  11000 ++            ::                      *  *                          |
  10000 ++        *. : ** *. * * *.*** ** .* * **                           |
        |       **  **   *  * * *     *  *  *                               |
   9000 ++      :                                                           |
   8000 ++      :                                                           |
        |** .* :                                                            |
   7000 *+ *  **                                                            |
   6000 ++------------------------------------------------------------------+


                            proc-vmstat.numa_hint_faults

  18000 ++------------------------------------------------------------------+
        |                          O                                        |
  16000 ++                                                                  |
        |                                                                   |
        |                   O                                               |
  14000 ++       O    O   O                                               O O
        |       O                                                 OOO O     |
  12000 ++ O  O      O  O       O    O OOO O OOOOO OOO      O O      O O O O|
        OOO  O O  O O  O O   OOO O  O O     O         OOO OO O OO           |
  10000 ++                                    ** *                          |
        |        **.*******.******.*******.***  *                           |
        |       *                                                           |
   8000 ++      :                                                           |
        |** .* *                                                            |
   6000 *+-*--*-------------------------------------------------------------+


                         proc-vmstat.numa_hint_faults_local

  15000 ++------------------------------------------------------------------+
  14000 ++                         O                                        |
        |                   O                                               |
  13000 ++       O    O   O                                           O   O O
  12000 ++ O    O                                O            O   OOO  O    |
        O     O     OO  O      OOO   OOOOO OOOOOO  OOOO O O O   O    O   O O|
  11000 +OO  O O  O    O O   OO     O                  O   O O O            |
  10000 ++                                    ** *                          |
   9000 ++       **. **** *. * * *.*** ** .***  *                           |
        |       *   *    *  * * *     *  *                                  |
   8000 ++      :                                                           |
   7000 ++      :                                                           |
        |      :                                                            |
   6000 ****.***                                                            |
   5000 ++------------------------------------------------------------------+


                                    meminfo.MemFree

  1.4e+08 ++--------------*-------------------------------------------------+
          *  **.**    **. ::*  *       ***       **                         |
  1.3e+08 +**     ****   * * ** **.****   *.*****                           |
          |                                                                 |
  1.2e+08 ++                                                                |
          |                                                                 |
  1.1e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    9e+07 ++                                                                |
          |                                           O                     |
    8e+07 ++    OOOOOO O OOOOO OOO OOOOOOOO OOOOOOOO O O OOOOO OOOOOO   OOOO|
          OOOOO       O       O                         O            OO     O
    7e+07 ++----------------------------------------------------------------+


                                    meminfo.Cached

  4.5e+08 ++----------------------------------------------------------------+
          OOOOO       O       O                         O            OO     O
  4.4e+08 ++    OOOOOO O OOOOO OOO OOOOOOOO OOOOOOOO O O OOOOO OOOOOO   OOOO|
          |                                           O                     |
  4.3e+08 ++                                                                |
          |                                                                 |
  4.2e+08 ++                                                                |
          |                                                                 |
  4.1e+08 ++                                                                |
          |                                                                 |
    4e+08 ++                                                                |
          | *                                                               |
  3.9e+08 +* :    ****  .* **** **.*****  *.******                          |
          *  **.**    **  ::   *        **        *                         |
  3.8e+08 ++--------------*-------------------------------------------------+


                                   meminfo.Inactive

  4.5e+08 ++----------------------------------------------------------------+
          OOOOO       O                                 O            OO     |
  4.4e+08 ++    OOOOOO O OOOOOOOOO OOOOOOOO OOOOOOOO O O OOOOO OOOOOO   OOOOO
          |                                           O                     |
  4.3e+08 ++                                                                |
          |                                                                 |
  4.2e+08 ++                                                                |
          |                                                                 |
  4.1e+08 ++                                                                |
          |                                                                 |
    4e+08 ++                                                                |
          |                                                                 |
  3.9e+08 +**     ** *     * ** **.****    .* ***                           |
          *  **. *  * **.* :*  *       ****  *   **                         |
  3.8e+08 ++----*---------*-------------------------------------------------+


                                meminfo.Inactive(file)

  4.5e+08 ++----------------------------------------------------------------+
          OOOOO       O                                 O            O      |
  4.4e+08 ++    OOOOOO O OOOOOOOOO OOOOOOOO OOOOOOOO O O OOOOO OOOOOO O OOOOO
          |                                           O                     |
  4.3e+08 ++                                                                |
          |                                                                 |
  4.2e+08 ++                                                                |
          |                                                                 |
  4.1e+08 ++                                                                |
          |                                                                 |
    4e+08 ++                                                                |
          |                                                                 |
  3.9e+08 ++*     ** *       ** **.** *    .* ***                           |
          ** **. *  * **.* **  *     * ****  *   **                         |
  3.8e+08 ++----*---------*-------------------------------------------------+


                                  meminfo.AnonPages

  52000 ++------------------------------------------------------------------+
        |                                                             O     |
  51000 ++   O       O                             O    O          OO    O  |
  50000 ++    O     O O  O  O O O  OO  O     O      OO    OO O            O O
        OOOO   O        O O  O   O    O  O  O  OOO    OO    O OO     O O   O|
  49000 ++      OOO    O       O        O     O                 O           |
  48000 ++                           O     O                      O         |
        |                                                                   |
  47000 ++                                                                  |
  46000 ++                                                                  |
        |*                                    *                             |
  45000 *+:*.   *  .*         *  *.*    **.*  ::                            |
  44000 ++*  *** :*  :****   * * :  **  :   ** **                           |
        |        *   *    *.*   *     **         *                          |
  43000 ++------------------------------------------------------------------+


                                   meminfo.Mapped

  24000 ++-------------------*----------------------------------------------+
        |                    : *                 *                          |
  22000 ++     **     *    .*: :       *    *    :                          |
        |*     : :  ** *  *  : ::*  * *: *  :: * :                          |
        |:**.* : *  :   : :  : :* + :*  ::+: ** :                           |
  20000 *+    :  : :    *:    ::   *    :  *    *                           |
        |     *   ::     *    :         *                                   |
  18000 ++        *           :                                             |
        |                     :                                             |
  16000 ++O    O    O     O   *         O     O  O          O  O   OO       O
        OO    O O    O O         O OO O     O      OOOOO     OO      OO     |
        |  O     OO   O OO  O        O     O    O         O     O O    O OOO|
  14000 ++   O               OO O      O O   O O        O  O                |
        |                      O                                            |
  12000 ++------------------------------------------------------------------+


                                 meminfo.PageTables

  10000 ++------------------------------------------------------------------+
        |                              O     O       O             O      O O
        |OO   O      O    O   O O  O               O   OO  O  OO    OOO  O  |
   9500 O+ O O OO   O OOOO  OO      O   OO    OOO   O O   O OO         O   O|
        |        OO            O O   OO    OO    O              O O         |
        |                                                                   |
   9000 ++                                                                  |
        |                                                                   |
   8500 ++                                                                  |
        |                    *                *                             |
        |***.****  .*   *    ::*  .**   *    * *                            |
   8000 *+       **  ** :*  : * :*   :** *.* :  *                           |
        |              *  *.*   *    *      *    *                          |
        |                                                                   |
   7500 ++------------------------------------------------------------------+


                              numa-meminfo.node0.MemFree

  3.8e+07 ++----------------------------------------------------------------+
  3.6e+07 *+          *    **           *    *                              |
          |* **.**    :   *  : *  .*** * :*.* : **                          |
  3.4e+07 ++*     ***: :.*   * : *    :: *    * : *                         |
  3.2e+07 ++         * *      * ::    *        *                            |
          |                     *                                           |
    3e+07 ++                                                                |
  2.8e+07 ++                                                                |
  2.6e+07 ++                                                                |
          |                                                                 |
  2.4e+07 ++                                                                |
  2.2e+07 ++             O                            O   O                 |
          |     OOOOOOOO  O OOO OO OO OOO O OOO OO O O OO   OO OOOOOO    OO |
    2e+07 OOOOO            O   O     O   O     O  O      O O         OO O  OO
  1.8e+07 ++----------------------------------------------------------------+


                              numa-meminfo.node0.MemUsed

  1.14e+08 ++---------------------------------------------------------------+
  1.12e+08 +OOOO            O        O             O     O O         OO O  OO
           O     OOOOOOOO O  OOOOOOOO  OOOOOOOOOO O OO OO    OOO OOOO    OO |
   1.1e+08 ++            O                            O   O     O           |
  1.08e+08 ++                                                               |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
           |                     *                                          |
     1e+08 ++         * *      * ::    *       *                            |
   9.8e+07 ++*     * *: :*    * : *   + : *   * :  *                        |
           |* :*.** *  :  *.  : *  ***  * :** : *.*                         |
   9.6e+07 *+ *        *    **           *   *                              |
   9.4e+07 ++---------------------------------------------------------------+


                              numa-meminfo.node0.Inactive

  1.12e+08 ++---------------------------------------------------------------+
   1.1e+08 +OOOO            O        O                   O O               O|
           O     O OOOOOO O  OOOOOOOO  OO OOOO OO OOOO OO    OOO OOOOOO OOO O
  1.08e+08 ++     O      O               O    O       O   O     O           |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 ++           *      * *     *                                    |
   9.6e+07 ++*     * ** :*    * ::*   + :     **   *                        |
           |* :*.** *  :  *.  : *  ***  * *** : *.*                         |
   9.4e+07 *+ *        *    **           *   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                           numa-meminfo.node0.Inactive(file)

  1.12e+08 ++---------------------------------------------------------------+
   1.1e+08 +OOOO            O        O                   O O               O|
           O     O OOOOOO O  OO OOOOO  OO OOOO OO OOOO OO    OOO OOOOOO OOO O
  1.08e+08 ++     O      O     O         O    O       O   O     O           |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 ++           *      * *     *                                    |
   9.6e+07 ++*     * ** :*    * ::*   + :     **   *                        |
           |* :*.** *  :  *.  : *  ***  * *** : *.*                         |
   9.4e+07 *+ *        *    **           *   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                             numa-meminfo.node0.FilePages

  1.12e+08 ++---------------------------------------------------------------+
   1.1e+08 +OOOO            O        O             O     O O         OO O  OO
           O     OOOOOOOO O  OOOOOOOO  OOOOOOOOOO O OO OO    OOO OOOO    OO |
  1.08e+08 ++            O                            O   O     O           |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 ++         * *      * *     *       *                            |
   9.6e+07 ++*     ***: :*    * ::*   + : *   * :  *                        |
           |* :*.**    :  *.  : *  ***  * :** : *.*                         |
   9.4e+07 *+ *        *    **           *   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                           numa-meminfo.node0.PageTables

  3000 ++-----------------------------------------------------------O-------+
       |                                                           O        |
  2800 ++            O                                    O     O           |
  2600 ++       O   O     O                     O       O     O            O|
       |   OO          O     O   O                   O O                    O
  2400 ++O   OO   O  *O    O  O   OO O    *O    *     O      O   O    O O O |
       O           O :   O              O :*  O :OO      O                  |
  2200 ++   *      : :          O   O  O*OO:O  O:: O           O     O      |
       *O   :*    : : :  *  O*    * *   :   :   :*          O          O    |
  2000 ++  :  *O  * * : + * * *   :::  :    *. :                            |
  1800 +**.*   :  :   **   ::  + * * :.*      *:                            |
       |        ::         *    *    *         *                            |
  1600 ++       ::                                                          |
       |        *                                                           |
  1400 ++-------------------------------------------------------------------+


                              numa-meminfo.node1.MemFree

  3.8e+07 ++----------------------------------------------------------------+
  3.6e+07 ++     *      .*                   *                              |
          |      :*    *  *          *       ::                             |
  3.4e+07 ++*   * : * *   : ***     * :  *   :*  **                         |
  3.2e+07 +* * +   ::*     ::  ***.*  * * *.:  **                           |
          *   *    *       *           *    *                               |
    3e+07 ++                                                                |
  2.8e+07 ++                                                                |
  2.6e+07 ++                                                                |
          |                                                                 |
  2.4e+07 ++                                                                |
  2.2e+07 ++                                          O                     |
          |      O OO      OO  OOO O O  O O   O OOOO O O OO OO OO   O    O  O
    2e+07 OOOOO O O  OOO OO  OO     O OO O  OO O        O        OOO OO O OO|
  1.8e+07 ++-----------------------------------------------O----------------+


                              numa-meminfo.node1.MemUsed

  1.14e+08 O+-O--------------------------------------------O----------------+
  1.12e+08 +OO O O O  OOOOO   OO    O  OOOO OOOO     O  O        OOOOOO O OO|
           |      O OO      OO  OOOO O     O    O OOO  O OO  OOOO        O  O
   1.1e+08 ++                                         O                     |
  1.08e+08 ++                                                               |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
           *   *    *       *      *    *   *                               |
     1e+08 +* * +   ::*    : : **** :  * * *:  **.                          |
   9.8e+07 ++*   * : * *   : **     * +   *  :*   **                        |
           |      :*    * *          *       *                              |
   9.6e+07 ++     *      *                                                  |
   9.4e+07 ++---------------------------------------------------------------+


                              numa-meminfo.node1.Inactive

  1.12e+08 ++----------------------------------------------O----------------+
   1.1e+08 OOOOO   O  OOOOO   OO       OO   OO          O        OOO OO   O |
           |     OO OO      OO  OOOOOO   OOO  OOO OOOO O OO  OOOO   O   OO OO
  1.08e+08 ++                                         O                     |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |        *       *                                               |
   9.8e+07 ** **.   ::*    : :  ****    ** **  *                            |
   9.6e+07 ++::  * : * *   : ***    : .*  * : * *.**                        |
           | *    :*    * *         **       ::                             |
   9.4e+07 ++     *      *                   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                           numa-meminfo.node1.Inactive(file)

  1.12e+08 ++----------------------------------------------O----------------+
   1.1e+08 OOOOO   O  O O O   OO        O    O                   OO   O   O |
           |     OO OO O O  OO  OOOOOO O OOOO OOO OOOO OOOO  OOOO  OOO  OO OO
  1.08e+08 ++                                         O                     |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |        *       *                                               |
   9.8e+07 *+  *.   ::*    : :  * **    ** **  *                            |
   9.6e+07 +* *  * : * :   : * * *  : .*  * :  :*.**                        |
           | *    :*   ** *   *     **       :*                             |
   9.4e+07 ++     *      *                   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                             numa-meminfo.node1.FilePages

  1.12e+08 O+----------------------------------------------O----------------+
   1.1e+08 +OOOO O O  OOOOO   OO    O  OO O OO O     O  O        OOOOOO O OO|
           |      O OO      OO  OOOO O   O O  O O OOO  O OO  OOOO        O  O
  1.08e+08 ++                                         O                     |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           *   *    *       *           *   *                               |
   9.8e+07 +* * +   ::*    : : *****   * * *:  **.                          |
   9.6e+07 ++*   * : * *   : **     * +   *  :*   **                        |
           |      :*    * *          *       *                              |
   9.4e+07 ++     *      *                                                  |
   9.2e+07 ++---------------------------------------------------------------+


                              numa-meminfo.node2.MemFree

  3.6e+07 ++--------------------*-------*-----------------------------------+
          |               *   * ::      :      *                            |
  3.4e+07 *+  *.*   *** .* *  : :*.*   : :     :                            |
  3.2e+07 +*  :  :**   *    ** :    * ** **.  : :**                         |
          | :*   *             *     *      *** *                           |
    3e+07 ++*                                                               |
  2.8e+07 ++                                                                |
          |                                                                 |
  2.6e+07 ++                                                                |
  2.4e+07 ++                                                                |
          |                                                                 |
  2.2e+07 ++                                          O                     |
    2e+07 ++    OOO    O       O    OOO OO  O OO O O       OO    O  O   OO  |
          OOO O    OOOO  O OOO  OO O   O  O  O  O O  O O OO  O OO OO OO   OOO
  1.8e+07 ++-O------------O---O-------------------------O-------------------+


                              numa-meminfo.node2.MemUsed

  1.16e+08 ++---------------------------------------------------------------+
  1.14e+08 ++ O           O    O                                            |
           OOO O     OOO O  O    O O    O    O     O O OO O   OO  O  O      O
  1.12e+08 ++    O OO   O    OO O O OO O OOOO OOO O O    O O O  OO OO O OOOO|
   1.1e+08 ++     O                                   O                     |
           |                                                                |
  1.08e+08 ++                                                               |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           | *                                                              |
  1.02e+08 ++::                 *    *      *   *.                          |
     1e+08 +* *   ***   *     * :   * + * ** ** : *                         |
           *   :.*   *** * .** : :**   *: :    :   *                        |
   9.8e+07 ++  *          *    * ::      :     *                            |
   9.6e+07 ++--------------------*-------*----------------------------------+


                              numa-meminfo.node2.Inactive

  1.12e+08 ++-O-----------O----O--------------------------------------------+
           OOO O    OOOO      O  OOO    O  O O  O  O O  OOO   OO  OO OO   O O
   1.1e+08 ++    OOO    OO  OO  O   OO O OO O OO  O O  O   O O  OO  O   OO O|
  1.08e+08 ++                                         O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++*                                                              |
           | ::                 *    *.     *** *.                          |
   9.8e+07 +* *   ***    *   ** :   *  ** **  : : **                        |
   9.6e+07 *+  *.*   **** :.*  * :**    : :    :                            |
           |              *      ::      :     *                            |
   9.4e+07 ++--------------------*-------*----------------------------------+


                           numa-meminfo.node2.Inactive(file)

  1.12e+08 ++-O-----------O----O--------------------------------------------+
           OOO O    OOOO      O  O O       O    O  O O  OOO   OO  OO O    O O
   1.1e+08 ++    O O    OO  OO  O O OO OOOO OOOO  O O  O   O O  OO  O O OO O|
  1.08e+08 ++     O                                   O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++*                                                              |
           | ::                 *    *      *** *.                          |
   9.8e+07 +* *   ***        ** :   * + * **  : : **                        |
   9.6e+07 *+  *.*   * *** .*  * :**   *: :    :                            |
           |          *   *      ::      :     *                            |
   9.4e+07 ++--------------------*-------*----------------------------------+


                             numa-meminfo.node2.FilePages

  1.14e+08 ++---------------------------------------------------------------+
  1.12e+08 ++ O           O    O                                            |
           OOO O     OOO O  O    O O    O    O     O O OO O   OO  O  OO    OO
   1.1e+08 ++    O OO   O    OO O O OO O OOOO OOO O O    O O O  OO OO   OOO |
  1.08e+08 ++     O                                   O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
           | *                                                              |
     1e+08 ++::                 *    *      *   *.                          |
   9.8e+07 +* *   ***   *    ** :   * + * ** ** : *                         |
           *   *.*   *** * .*  : :**   *: :    :   *                        |
   9.6e+07 ++             *    * ::      :     *                            |
   9.4e+07 ++--------------------*-------*----------------------------------+


                              numa-meminfo.node3.MemFree

  3.8e+07 ++----------------------------------------------------------------+
  3.6e+07 ++  *.*                        *                                  |
          |  *  :                       *:                                  |
  3.4e+07 *+ :   :     *.** *  * *.    *  :.*  * *                          |
  3.2e+07 +**    ******    * ** :: ** *   *  * :* *                         |
          |                     *    *        *                             |
    3e+07 ++                                                                |
  2.8e+07 ++                                                                |
  2.6e+07 ++                                                                |
          |                                                                 |
  2.4e+07 ++                                                                |
  2.2e+07 ++                                          O                     |
          |          O    OOOO  OO O  O OO  O OO  O  O     OO   O       O O |
    2e+07 OO OO OOOOO OO O    OO    OO O  O  O  OO O   OOOO  O O OOOOOO  O OO
  1.8e+07 ++O---------------------------------------------------------------+


                              numa-meminfo.node3.MemUsed

  1.14e+08 ++O-O------------------------------------------------------------+
  1.12e+08 OO O  OOOOO OOO    OOO   OO  O  O O  O O OO OOOO   OO OOOOOO OOOOO
           |          O   O OO   OOO   O OO O OO   O       O O  O           |
   1.1e+08 ++                                         O                     |
  1.08e+08 ++                                                               |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
           |                     *   *.       *                             |
     1e+08 +**    ******   .* ** ::**  *   * * **. *                        |
   9.8e+07 *+ :   :     ***  *  * *     *  :*     *                         |
           |  **.:                       *:                                 |
   9.6e+07 ++    *                        *                                 |
   9.4e+07 ++---------------------------------------------------------------+


                              numa-meminfo.node3.Inactive

  1.12e+08 ++O--------------------------------------------------------------+
   1.1e+08 OO OO OOOOO OOO     OO   OO  O  O O  O O O  OO O    O OOOOOO O OOO
           |          O   O OOO  OOO   O OO O OO   O O   O   OO O        O  |
  1.08e+08 ++                                         O    O                |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                     *   *.                                     |
   9.8e+07 +**    ******   .* *  ::**  *     ** *. *                        |
   9.6e+07 *+ :   :     ***  * ** *     *  **  *  *                         |
           |  **.:                       * :                                |
   9.4e+07 ++    *                        *                                 |
   9.2e+07 ++---------------------------------------------------------------+


                           numa-meminfo.node3.Inactive(file)

  1.12e+08 ++O--------------------------------------------------------------+
   1.1e+08 OO OO OOOOO OOO     OO   OO  O  O O  O O O  OO O    O OOOOOO O OOO
           |          O   O OOO  OOO   O OO O OO   O O   O   OO O        O  |
  1.08e+08 ++                                         O    O                |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                     *   *.                                     |
   9.8e+07 +**    ******   .* *  ::**  *     ** *. *                        |
   9.6e+07 *+ :   :     ***  * ** *     *  **  *  *                         |
           |  **.:                       * :                                |
   9.4e+07 ++    *                        *                                 |
   9.2e+07 ++---------------------------------------------------------------+


                             numa-meminfo.node3.FilePages

  1.14e+08 ++---------------------------------------------------------------+
  1.12e+08 ++O                                                              |
           OO OO OOOOO OOO     OO   OO  O  O O  O O O  OOOO   OO OOOOOO  O OO
   1.1e+08 ++         O   O OOO  OOO   O OO O OO   O O     O O  O       O O |
  1.08e+08 ++                                         O                     |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++                    *                                          |
   9.8e+07 ++*    ** ***      *  ::* *.*     ** *. *                        |
   9.6e+07 ** :   : *   ***.** ** * *   *  **  *  *                         |
           |  *  :                       * :                                |
   9.4e+07 ++  *.*                        *                                 |
   9.2e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node0.nr_free_pages

  9.5e+06 ++----------------------------------------------------------------+
    9e+06 *+          *    *         *  *                                   |
          |* **.** *  :   * *  *  .**: * :*.**  ***                         |
  8.5e+06 ++*     * *: :  :  * ::*    :: *    * :                           |
    8e+06 ++         * *.*    * *     *        *                            |
          |                                                                 |
  7.5e+06 ++                                                                |
    7e+06 ++                                                                |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
  5.5e+06 ++             O                            O   O                 |
          |     OOOOOOOO  O OOOOOO OO OOO O OOO OO O O OO   OO OOOOOO    O  |
    5e+06 OOOOO            O         O   O     O  O      O O         OO O OOO
  4.5e+06 ++----------------------------------------------------------------+


                          numa-vmstat.node0.nr_inactive_file

   2.8e+07 ++---------------------------------------------------------------+
  2.75e+07 +OOOO            O                            O O                O
           O     OOOOOOOO O  OO OOOOOO OO OOOO OO OOOO OO    OOOOOOOOOO OOOO|
   2.7e+07 ++            O     O         O    O       O   O                 |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           |                                                                |
  2.45e+07 ++           *      * *     *       *                            |
   2.4e+07 ++*     * ** :*    *: :*   : :     * :                           |
           |* **.** *  :  *.  : :  ** : * * **  *.**                        |
  2.35e+07 *+          *    **  *    *   * *                                |
   2.3e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node0.nr_file_pages

   2.8e+07 ++---------------------------------------------------------------+
  2.75e+07 +OOOO            O                      O     O O         OO O  OO
           O     OOOOOOOO O  OOOOOOOOO OOOOOOOOOO O OO OO    OOOOOOOO    OO |
   2.7e+07 ++            O                            O   O                 |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           |                                                                |
  2.45e+07 ++         * *      * *     *       *                            |
   2.4e+07 ++*     * *: :*    * ::*   : : *   * :                           |
           |* **.** *  :  *. *  *  ** : * :***  *.**                        |
  2.35e+07 *+          *    *        *   *                                  |
   2.3e+07 ++---------------------------------------------------------------+


                       numa-vmstat.node0.nr_page_table_pages

  750 ++------------------------------------------------------------O-------+
      |                                                            O        |
  700 ++            O                                     O     O           |
  650 ++       O         O                      O       O     O            O|
      |   OO       O   O        O                   O  O                    O
  600 ++O   OO   O  *O    O  OO  OO  O    *O    *    O       O   O    O O   |
      O           O :   O              O : * O  :O       O                O |
  550 ++   *      : :          O    O O*O:O:O O: :OO           O     O      |
      *O   :*    : : :  *  O     *  *  :*   :  : *         O           O    |
  500 ++  :  :O  * * :  :* *.**  ::+: :     *  :                            |
  450 +**.*  *:  :   *.*  ::   :* *  :*      *:                             |
      |        ::         *    *     *        *                             |
  400 ++       ::                                                           |
      |        *                                                            |
  350 ++--------------------------------------------------------------------+


                              numa-vmstat.node0.numa_hit

    2e+08 ++----------------------------------------------------------------+
          |    .*     *                                                     |
  1.9e+08 *****  ***** *.*********.*** ****.**** **                         |
          |                           *         *                           |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++                                                                |
          |      OO  O   O    O O    O  OO    OO      OO  OOO      OO     O |
  1.4e+08 OOOOO O  OO OO  OOOO O O  O  O  O  O  OOOO O  OO   O OOOO  OO OO OO
          |                        O  O     O                               |
  1.3e+08 ++----------------------------------------------------------------+


                              numa-vmstat.node0.numa_miss

    3e+07 ++----------------------------------------------------------------+
  2.8e+07 ++         O                                       O  O           |
          |O    OOO O O   OO       OOO    O     OO O OOOO OO     O          |
  2.6e+07 ++OOO    O   O O    OOOO    OO O  OO O  O      O  O  O            |
  2.4e+07 O+                OO          O     O                     OO      |
  2.2e+07 ++                                                      O     OOOOO
    2e+07 ++                                                       O  O     |
          |                                                                 |
  1.8e+07 ++                                                                |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
  1.2e+07 *+      *    *.  *                 *                              |
          |***   * * * : ** *  ***.*  *  *   :*  *                          |
    1e+07 ++  *. :  * *      **     ** ** *.*  ** *                         |
    8e+06 ++----*-----------------------------------------------------------+


                            numa-vmstat.node0.numa_foreign

  2.6e+07 ++----------------------------------------------------------------+
  2.4e+07 ++    O O      O OO O OO   O  O O   O O          O O  O           |
          |OO    O OOOOO  O  O O   OO OO O  OO O OOO OOO  O O  O O          |
  2.2e+07 O+ OO                                         OO        OO  O  OOO|
    2e+07 ++                                                        OO  O   O
          |                                                                 |
  1.8e+07 ++                                                                |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
          |                                                                 |
  1.2e+07 ++                                                                |
    1e+07 ++              *                                                 |
          |        **  *.* *  *** .*     *                                  |
    8e+06 *****. **  **     **   *  ***** *.*******                         |
    6e+06 ++----*-----------------------------------------------------------+


                             numa-vmstat.node0.numa_local

    2e+08 ++----------------------------------------------------------------+
          |    .*     *                                                     |
  1.9e+08 *****  ***** *.*********.*** ****.**** **                         |
          |                           *         *                           |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++                                                                |
          |      OO  O   O    O O    O  OO    OO      OO  OOO      OO     O |
  1.4e+08 OOOOO O  OO OO  OOOO O O  O  O  O  O  OOOO O  OO   O OOOO  OO OO OO
          |                        O  O     O                               |
  1.3e+08 ++----------------------------------------------------------------+


                             numa-vmstat.node0.numa_other

    3e+07 ++----------------------------------------------------------------+
  2.8e+07 ++         O                                       O  O           |
          |O    OOO O O   OO       OOO    O     OO O OOOO OO     O          |
  2.6e+07 ++OOO    O   O O    OOOO    OO O  OO O  O      O  O  O            |
  2.4e+07 O+                OO          O     O                     OO      |
  2.2e+07 ++                                                      O     OOOOO
    2e+07 ++                                                       O  O     |
          |                                                                 |
  1.8e+07 ++                                                                |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
  1.2e+07 *+      *    *.  *                 *                              |
          |***   * * * : ** *  ***.*  *  *   :*  *                          |
    1e+07 ++  *. :  * *      **     ** ** *.*  ** *                         |
    8e+06 ++----*-----------------------------------------------------------+


                      numa-vmstat.node0.workingset_nodereclaim

  750000 ++-----------------------------------------------------------------+
         |           *.** **  .*        **                                  |
  700000 *+**.*******    *  **  ******.*  *****. **                         |
  650000 +*                                     *                           |
         |                                                                  |
  600000 ++                                                                 |
  550000 ++                                                                 |
         |                                                                  |
  500000 ++                                                                 |
  450000 ++         O                                OO                     |
         |       O      O    O          OO   OO           OOOO  O           |
  400000 OO    OO  O   O OO O  O    O            O OO   OO       O          |
  350000 ++O  O   O  O     O    OO O   O  O O   O O           O    OO  O OO |
         |  O                     O  O     O                         OO    OO
  300000 ++-------------------------------------------------------O---------+


                            numa-vmstat.node1.nr_free_pages

  9.5e+06 ++----------------------------------------------------------------+
    9e+06 ++     *                                                          |
          |      :*    *.**          *       *                              |
  8.5e+06 ++*   * : * *   :  **     * :  **  :* ***                         |
    8e+06 +* * +   ::*     :*  ***.*  * *  +:  *                            |
          *   *    *       *           *    *                               |
  7.5e+06 ++                                                                |
    7e+06 ++                                                                |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
  5.5e+06 ++                                          O                     |
          |      O OO      OO OOOO O O  O O   O OOOO O O OO OO OO   O    O OO
    5e+06 OOOOO O O  OOO OO  O      O OO O  OO O        O        OOO OO O O |
  4.5e+06 ++-----------------------------------------------O----------------+


                          numa-vmstat.node1.nr_inactive_file

   2.8e+07 ++----------------------------------------------O----------------+
  2.75e+07 OOOOO O O  O O O   O         O    O                   OO   O   O |
           |      O OO O O  OO OOOOOOO O OOOO OOO OOOO OOOO  OOOO  OOO  OO OO
   2.7e+07 ++                                         O                     |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           |                *                                               |
  2.45e+07 *+  *.   * *    : *    **    **  *  *                            |
   2.4e+07 +* *  *  ::::   :  ****  : .*  ** : :*.*                         |
           | *    :* * ****         **       **    *                        |
  2.35e+07 ++     *                                                         |
   2.3e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node1.nr_file_pages

   2.8e+07 ++----------------------------------------------O----------------+
  2.75e+07 OOOOO OOO  OOOOO   OO  O O  OO O OO O     O  O       OOOOOOO O O |
           |        OO      OO  OO O O   O O  O O OOO  O OO  OOO         O OO
   2.7e+07 ++                                         O                     |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           *   *    *       *           *   *                               |
  2.45e+07 +* * +   ::*    : *  ****   * *  :  *                            |
   2.4e+07 ++*   * : * *   :  **    * +   ** :* *.**                        |
           |      :*    ***          *       *                              |
  2.35e+07 ++     *                                                         |
   2.3e+07 ++---------------------------------------------------------------+


                              numa-vmstat.node1.numa_hit

    2e+08 ++----------------------------------------------------------------+
          *****.***  ***. ****** *.     * *. *                              |
  1.9e+08 ++       **    *      *  ***** *  * *****                         |
          |                                                                 |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++                                                                |
          |                                           O        O            |
  1.4e+08 ++OOO   O OOO   O O  O O  O  OOOO  OOOOO O O OOOOO      OOOO  OO  O
          OO    OO O   O O O OO O  O OO     O     O         OO  OO    O   OO|
  1.3e+08 ++----------------------------------------------------------------+


                              numa-vmstat.node1.numa_miss

    3e+07 ++----------------------------------------------------------------+
          |                   O      O                       O              |
          |OO   OOOOOOOO OOOO   OO O  OOO O OOO OOOO  OO  OOO   OOO        O|
  2.5e+07 O+ O               O O    O    O     O     O  OO     O   O  O OO  |
          |   O                                                     OO    O |
          |                                                                 O
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    1e+07 ++       **  *.**                                                 |
          * ***. **  * :   *******.*** ****. ******                         |
          |*    *     *               *     *                               |
    5e+06 ++----------------------------------------------------------------+


                            numa-vmstat.node1.numa_foreign

  2.6e+07 ++----------------------------------------------------------------+
  2.4e+07 ++                              O     O         O                 |
          |       O    O       O O                 O  O        OO           |
  2.2e+07 OOOOO O   OOO  OOOOOO O  OOOOOOO  OOOO OO  O O O OOO   O          |
    2e+07 ++     O O                                    O         O OO  O O |
          |                                                        O  O  O OO
  1.8e+07 ++                                                                |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
          |                                                                 |
  1.2e+07 ++                                                                |
    1e+07 ++                                                                |
          |**    *     *. **  * * .** *   *.* * * *                         |
    8e+06 *+ **. :*****  *  ** * *   * ***   * * *                          |
    6e+06 ++----*-----------------------------------------------------------+


                             numa-vmstat.node1.numa_local

    2e+08 ++----------------------------------------------------------------+
          *****.***  ***. ****** *.     * *. *                              |
  1.9e+08 ++       **    *      *  ***** *  * *****                         |
          |                                                                 |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++                                                                |
          |                                           O        O            |
  1.4e+08 ++OOO   O OOO   O O  O O  O  OOOO  OOOOO O O OOOOO      OOOO  O   O
          OO    OO O   O O O OO O  O OO     O     O         OO  OO    O  OOO|
  1.3e+08 ++----------------------------------------------------------------+


                             numa-vmstat.node1.numa_other

    3e+07 ++----------------------------------------------------------------+
          |                   O      O                       O              |
          |OO   OOOOOOOO OOOO   OO O  OOO O OOO OOOO  OO  OOO   OOO        O|
  2.5e+07 O+ OO              O O    O    O     O     O  OO     O   O  O OO  |
          |                                                         OO    O |
          |                                                                 O
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    1e+07 ++       **  *.**                                                 |
          * ***. **  * :   *******.*** ****. ******                         |
          |*    *     *               *     *                               |
    5e+06 ++----------------------------------------------------------------+


                      numa-vmstat.node1.workingset_nodereclaim

  750000 ++-----------------------------------------------------------------+
  700000 ** *.** ** **.******    *      *                                   |
         | *    *  *         *.** ****.* ******.***                         |
  650000 ++                                                                 |
  600000 ++                                                                 |
         |                                                                  |
  550000 ++                                                                 |
  500000 ++                                                                 |
  450000 ++                                                                 |
         |                                                                  |
  400000 ++         O      O                         O                      |
  350000 ++OO  OOO O O    O     OOO    OOOO OOO OO O  O  OOO  O O O         |
         OO   O   O      O  OO O   OOO     O      O O   O   OO   O OOO O OOOO
  300000 ++            OO                                             O     |
  250000 ++-----------------------------------------------------------------+


                            numa-vmstat.node2.nr_free_pages

  9.5e+06 ++----------------------------------------------------------------+
    9e+06 ++                    *                                           |
          |               *     ::      *                                   |
  8.5e+06 *+  *.*   ***   :*  * :*.*    ::     *  *                         |
    8e+06 +*  :  :**   *.*  ** :    : ** **.   ::*                          |
          | :*   *             *    **      *** *                           |
  7.5e+06 ++*                                                               |
    7e+06 ++                                                                |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
  5.5e+06 ++                                                                |
          |      OO    O       O    OOO O   O O       O    O     O      OO O|
    5e+06 O+O O O  OOOO  O OOO  OO O   O OO  O OOOOO O O OO OO OO OOOOO   O O
  4.5e+06 +O-O------------O---O-------------------------O-------------------+


                          numa-vmstat.node2.nr_inactive_file

   2.8e+07 +O-O-----------O----O--------------------------------------------+
  2.75e+07 O+O O    OO O O  O O  OOO      O    O   OOO OOOO   OOO OO OO   O O
           |     OOO  O O    O  O   OO OOO OOOO O O   O    O O   O  O   OO O|
   2.7e+07 ++                                                               |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++*                                                              |
           | ::                 *   *       *** *.                          |
  2.45e+07 +* *   ***    *   ** :   :*. * **   :: **                        |
   2.4e+07 *+  *.*   **** :.*  * :**   * ::    *                            |
           |              *      ::      *                                  |
  2.35e+07 ++                    *                                          |
   2.3e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node2.nr_file_pages

  2.85e+07 ++---------------------------------------------------------------+
   2.8e+07 ++ O           O    O                                            |
           OOO O     O O O  O    OOO    O    O     O O OO     OOO    OO   O O
  2.75e+07 ++    OOOO O O    OO O   OO O OOOO OOO O O    OOO O   OOOO   OO O|
   2.7e+07 ++                                         O                     |
           |                                                                |
  2.65e+07 ++                                                               |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
           | *                                                              |
   2.5e+07 ++::                 *    *      **  *.                          |
  2.45e+07 +* *   **    **   ** :   * + * **  * : *                         |
           *   :.*  ****  :.*  : :**   *: :    *   *                        |
   2.4e+07 ++  *          *    * ::      :                                  |
  2.35e+07 ++--------------------*-------*----------------------------------+


                        numa-vmstat.node2.nr_isolated_file

  600 ++--------------------------------------------------------------------+
      |    *                                                                |
  550 ++ .* :      *    *    * *            * *.                            |
  500 ***   *  *  * O*  : * :: :*   **  *  *: : **                          |
      |      * :+ : * +: :::: :: : :  ** : : :                        O     |
  450 ++      :  *     * * *  :  * :     : : *                 O   O        |
      |       *         O     *   *       :                                 |
  400 ++                  O       O       *                               O |
      |              O O   O           O     O    O                         |
  350 ++O                        O                 OOO     O O  O      O   O|
  300 +O     OO              OO     O     O             O O   O  O  OO  O   O
      O   OOO  O OO      O     OO    OO O  OO O  O     O                    |
  250 ++           O                            O        O                  |
      |                                                                     |
  200 ++--------------------------------------------------------------------+


                              numa-vmstat.node2.numa_hit

     2e+08 ++---------------------------------------------------------------+
  1.95e+08 *****.**********.*** * *      *   *                              |
           |                   * * ***.** *** ***.**                        |
   1.9e+08 ++                                                               |
  1.85e+08 ++                                                               |
   1.8e+08 ++                                                               |
  1.75e+08 ++                                                               |
           |                                                                |
   1.7e+08 ++                                                               |
  1.65e+08 ++                                                               |
   1.6e+08 ++                                                               |
  1.55e+08 ++O     O  O   O    O  O  O  OO   OO   O  O OO       O           |
           |   O OO  O O O  OOO OO OO  O  OOO  OO  OO O  OOO OOO O OO O    OO
   1.5e+08 OO O     O   O                                         O  O  OOO |
  1.45e+08 ++---------------------------------------------------------------+


                              numa-vmstat.node2.numa_miss

  2.4e+07 ++------------------------------O-----O---------------------------+
  2.2e+07 ++      O            O        O     O       O   O                 |
          OOO O O   OO O O  OOO OO O OO  O  O  O OOO O     OO  OOO          |
    2e+07 ++ O   O O  O   OO        O  O     O         O O   O    O OO   OOO|
  1.8e+07 ++                                            O          O  O O   O
          |                                                                 |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
  1.2e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
    8e+06 ++          **. **  * * .*  *    .* * *                           |
          ***  .******   *  ** * *  ** ****  ::* **                         |
    6e+06 ++ **                              *                              |
    4e+06 ++----------------------------------------------------------------+


                            numa-vmstat.node2.numa_foreign

  3.5e+07 ++----------------------------------------------------------------+
          |                                                                 |
    3e+07 ++     O                   O O     O                              |
          |O  O O OOOOO  O OOOO OO O     OO    OOOOO OOOOOOOOO  OO          |
          O OO         O  O    O    O O O   O O                O  O        O|
  2.5e+07 ++                                                        OOO O O O
          |                                                        O     O  |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          *       *     .*    * *                                           |
    1e+07 +****. * *****  **** * *.********.*** ***                         |
          |     *                              *                            |
    5e+06 ++----------------------------------------------------------------+


                             numa-vmstat.node2.numa_local

     2e+08 ++---------------------------------------------------------------+
  1.95e+08 *****.**********.*** * *      *   *                              |
           |                   * * ***.** *** ***.**                        |
   1.9e+08 ++                                                               |
  1.85e+08 ++                                                               |
   1.8e+08 ++                                                               |
  1.75e+08 ++                                                               |
           |                                                                |
   1.7e+08 ++                                                               |
  1.65e+08 ++                                                               |
   1.6e+08 ++                                                               |
  1.55e+08 ++O     O  O   O    O  O  O  OO   OO   O  O OO                   |
           |   O OO  O O O  OOO OO OO  O  OOO  OO  OO O  OOO OOOOO OO O    OO
   1.5e+08 OO O     O   O                                         O  O  OOO |
  1.45e+08 ++---------------------------------------------------------------+


                             numa-vmstat.node2.numa_other

  2.4e+07 ++------------------------------O-----O---------------------------+
          |       O            O                      O   O                 |
  2.2e+07 +O  O O   OO O O  O O OO O OO O   O O    O O     OO  OOO          |
    2e+07 O+OO     O  O   OO O      O  O O   O O OO    O O   O      OO   OO |
          |      O                                      O         O   O O  OO
  1.8e+07 ++                                                       O        |
  1.6e+07 ++                                                                |
          |                                                                 |
  1.4e+07 ++                                                                |
  1.2e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
    8e+06 ++           *. *   * *  *  *       * *                           |
          * *    **** *  * *  :* :+ ** * **.* :* **                         |
    6e+06 +*-**-*----*------**---*------*----*------------------------------+


                      numa-vmstat.node2.workingset_nodereclaim

  750000 ++-----------------------------------------------------------------+
         ** *.**  ****.  **  *.* *      **  *                               |
  700000 ++*    **     **  **   * ****. : ** * .***                         |
         |                             *      *                             |
  650000 ++                                                                 |
  600000 ++                                                                 |
         |                                                                  |
  550000 ++                                                                 |
         |       O                                                          |
  500000 ++                            OO   OO      O         O             |
  450000 ++O  O    OO    O O O OOOO O    OO   O OOOO OO OOO  O  OO          |
         |     OO    O  O   O      O                       OO       OO     OO
  400000 ++ O     O    O             O     O                      OO   O OO |
         |O               O                                           O     |
  350000 O+-----------------------------------------------------------------+


                            numa-vmstat.node3.nr_free_pages

  9.5e+06 ++----------------------------------------------------------------+
    9e+06 ++  *.*                        *                                  |
          |  *  :                       *:                                  |
  8.5e+06 ** :   : *   *.**   ** *.    *  :.*    **                         |
    8e+06 ++*    ** ***    ***  :: ****   *  * **                           |
          |                     *             *                             |
  7.5e+06 ++                                                                |
    7e+06 ++                                                                |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
  5.5e+06 ++                                          O                     |
          |          O    OOOO  OO O O  OOO   OOO O  O   O OO  OO O        O|
    5e+06 OO OO OOOOO OO O    OO    O OO    OO   O O   OO O  O   O OOOO OOO O
  4.5e+06 ++O---------------------------------------------------------------+


                          numa-vmstat.node3.nr_inactive_file

   2.8e+07 ++O--------------------------------------------------------------+
  2.75e+07 OO OO OOOOO OOO     OO   O  OO   OO    O O  OO O    O O OOOO O O O
           |          O   O OOO  OOO O   OOO  OOO  O O   O   OO O O      O O|
   2.7e+07 ++                                         O    O                |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           |                     *                                          |
  2.45e+07 ++*    ** ***   .* *  ::***.*     ** *.                          |
   2.4e+07 ** :   : *   ***  * ** *     *  **  *  **                        |
           |  *  :                       * :                                |
  2.35e+07 ++  *.*                        *                                 |
   2.3e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node3.nr_file_pages

  2.85e+07 ++---------------------------------------------------------------+
   2.8e+07 ++O                                                              |
           OO OO OOOOO OOO     OO   O   O    O    O O  OO O   O  O OOOO  OO O
  2.75e+07 ++         O   O OOO  OOO O O OOOO OOO  O O   O O O OO O     O  O|
   2.7e+07 ++                                         O                     |
  2.65e+07 ++                                                               |
   2.6e+07 ++                                                               |
           |                                                                |
  2.55e+07 ++                                                               |
   2.5e+07 ++                    *                                          |
  2.45e+07 ++*    ** ***   .* *  ::***.      **                             |
   2.4e+07 ** :   : *   :**  * ** *    **  **  **.**                        |
           |  *  :      *                * :                                |
  2.35e+07 ++  *.*                        *                                 |
   2.3e+07 ++---------------------------------------------------------------+


                        numa-vmstat.node3.nr_isolated_file

  600 ++--------------------------------------------------------------------+
      |                    *                                                |
  550 ++            *      :   *          *                                 |
      | *           :    *: : *:         : * *                      O       |
  500 +* +  **   *  ::   :* : : :**      :  ::*                             |
      |:  **  * + *O *. *    :  *  + ** *   *  +                            |
  450 O+       *   :   *     *      *  ::       *                         O |
      |            *              O    *         *     O           O        |
  400 ++   O            O                                                   |
      |              O                              O    O                  |
  350 ++    OOO   O    O   O           O     O     O    O       O          O|
      | O O    O O        O   O  O  OO  O OO  O  O   O    OO OOO       OO   O
  300 +O            O    O   O OO     O     O   O O                   O     |
      |                                                          O   O      |
  250 ++--------------------------------------------------------------------+


                              numa-vmstat.node3.numa_hit

    2e+08 ++----------------------------------------------------------------+
          *****.********.**** ** *.     *    *   *                          |
  1.9e+08 ++                 *  *  ***** **.* *** *                         |
          |                                                                 |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++              O                                                 |
          OOO O OOO OOOO   OO OOO   OO  OOO   OOOOOO OOOOOOO O OOOOOOOO OOOOO
  1.4e+08 ++ O     O     O   O   O O  OO    OO              O               |
          |                                                                 |
  1.3e+08 ++----------------------------------------------------------------+


                              numa-vmstat.node3.numa_miss

    3e+07 ++----------------------------------------------------------------+
          |      O       O   O       O O     O                              |
          |     O OOO O    OO OOOO O     OO    OOOOO  OOOOOOO  OOO          |
  2.5e+07 OOOOO      O O  O         O O O   O O      O       O              |
          |                                                       O     O  OO
          |                                                        OOOO  OO |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    1e+07 ++                 *  *                                           |
          *****.  ******.**** ** *.********.*   ***                         |
          |     **                           ***                            |
    5e+06 ++----------------------------------------------------------------+


                            numa-vmstat.node3.numa_foreign

  2.8e+07 ++----------------------------------------------------------------+
  2.6e+07 +O    OOO  O O  OO       O OO   O O   OO O O     O O  OO          |
          | O O    OO O  O  O OOOO  O  OO    OO   O   OOOOO O  O            |
  2.4e+07 O+ O               O           O     O                  O OO  OO O|
  2.2e+07 ++                                                       O  O   O O
    2e+07 ++                                                                |
  1.8e+07 ++                                                                |
          |                                                                 |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
  1.2e+07 ++                                                                |
    1e+07 *+      **   *. *                                                 |
          |****. *  ***  * *******.* ******. ** ***                         |
    8e+06 ++    *                   *       *  *                            |
    6e+06 ++----------------------------------------------------------------+


                             numa-vmstat.node3.numa_local

    2e+08 ++----------------------------------------------------------------+
          *****.********.**** ** *.     *    *                              |
  1.9e+08 ++                 *  *  ***** **.* *****                         |
          |                                                                 |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++              O                                                 |
          OOO O OOO OOOO   OO OOO   OO  OOO   OOOOOO OOOOOOO O OOOOOOOO OOOOO
  1.4e+08 ++ O     O     O   O   O O  OO    OO              O               |
          |                                                                 |
  1.3e+08 ++----------------------------------------------------------------+


                             numa-vmstat.node3.numa_other

    3e+07 ++----------------------------------------------------------------+
          |      O       O   O       O O     O                              |
          |     O OOO O    OO OOOO O    OOO   OOOOOO  OOOOOOO  OOO          |
  2.5e+07 OOOOO      O O  O         O O     O        O       O              |
          |                                                       O   O O  OO
          |                                                        OOO   OO |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    1e+07 ++                 *  *                                           |
          *****.  ******.**** ** *.********.**  ***                         |
          |     **                            **                            |
    5e+06 ++----------------------------------------------------------------+


                      numa-vmstat.node3.workingset_nodereclaim

  750000 ++----*------------------------------------------------------------+
         ****.* ** * *.  *   *. *   * .*                                    |
  700000 ++       * *  ** ***  * *** *  *******.***                         |
         |                                                                  |
  650000 ++                                                                 |
  600000 ++                                                                 |
         |                                                                  |
  550000 ++                                                                 |
         |                O                                                 |
  500000 ++                                                                 |
  450000 OOO           O O      O  O               O       O                |
         |    OOO  OO      O O   O  O  OOOO OOO O O O         O OO          |
  400000 ++          O  O   O  O  O  O     O     O   OO  OO  O    OOO       O
         |  O    OO                                     O            OOO OOO|
  350000 ++-------------------------------------------------O---------------+

Thanks,
Fengguang

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
@ 2014-08-19 17:52         ` Fengguang Wu
  0 siblings, 0 replies; 12+ messages in thread
From: Fengguang Wu @ 2014-08-19 17:52 UTC (permalink / raw)
  To: lkp

[-- Attachment #1: Type: text/plain, Size: 200965 bytes --]

On Tue, Aug 19, 2014 at 06:13:58PM +0100, Mel Gorman wrote:
> On Tue, Aug 19, 2014 at 11:51:25PM +0800, Fengguang Wu wrote:
> > On Tue, Aug 19, 2014 at 03:29:25PM +0100, Mel Gorman wrote:
> > > On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> > > > Hi Mel,
> > > > 
> > > > We noticed the below vm-scalability performance/power regressions on
> > > > commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> > > > fields into read-only, page alloc, statistics and page reclaim lines").
> > > > 
> > > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> > > > ---------------  -------------------------  ---------------------------
> > > >      %stddev        %change               %stddev
> > > >             \          |                 /
> > > >       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > > >       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
> > > >      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> > > > 
> > > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> > > > ---------------  -------------------------  
> > > >   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > > >   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
> > > >   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> > > > 
> > > 
> > > 
> > > What units are these? It's completely unclear what is good and bad from the
> > > figures. 300s-lru-file-mmap-read appears multiple times in this report,
> > > each with different numbers beside them but little clue as to what they
> > > mean or what I'm meant to be looking for :(
> > 
> > Sorry the output format is a bit obscure: the stats names are after
> > the "TOTAL" word, such as TOTAL vm-scalability.stddev, TOTAL
> > vm-scalability.throughput, ...
> > 
> > > This is the same patch that was reported as having a performance gain in
> > > another set of tests from lkp so am a little confused.
> > 
> > That can happen some times -- we've seen some commits to benefit some
> > workloads while hurting some others. I'm now running more complete
> > test set for the commit, hopefully can get some results tomorrow.
> > 
> > > More importantly, as this patch is primary abougt cache misses it should
> > > be very unlikely that it makes a noticable difference to IO as the
> > > relative cost of a cache miss is so low. Similarly any difference it
> > > makes to reclaim activity is likely to be a coincidence or due to test
> > > variance.
> > 
> > The test case creates tmpfs files and read them fast to exercise the LRU.
> > So it's VM test and do not involve disk IO.
> > 
> 
> Ok. How reliable is this? I was reading down through some of the figures and
> some of them are extremely different in areas that this patch should have
> no impact on -- it's just rearranging a struct after all.

Yup. One difficulty with performance testing is that we sometimes
bisect into commit whose change set is obviously not related to the
changed performance.. and the bisect itself looks good.

> I note for example that automatic NUMA balancing was enabled and
> there was activity from it so pages are getting migrated.  There is
> a large amount of reclaim activity on different nodes so where the
> test case gets scheduled and how nodes it runs on during the test
> matters. What I'm wondering is if the performance of the test tends
> to vary quite a lot naturally. Can you check if the performance is
> consistently lower after this patch or if it tends to vary please?

At least for vm-scalability.throughput, it's pretty stable over *all*
the kernels tested during the bisect. Which is a strong indication
that the bisect process itself is fine.

mmap-read:

                               vm-scalability.throughput

  1.5e+07 ++----------------------------------------------------------------+
          |                                                                 |
  1.4e+07 *+*..*.*.*.*..*.*.*.*..*.*.*..*.*. .*..*.*.*..*.*                 |
          |                                 *                               |
  1.3e+07 ++                                                                |
          |                                                                 |
  1.2e+07 ++                                                                |
          |                                                                 |
  1.1e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
          |                                                                 |
    9e+06 O+O  O O O O  O O O O  O O O    O   O  O O O  O O O O    O O    O O
          |                             O   O                    O     O    |
    8e+06 ++----------------------------------------------------------------+

        [*] bisect-good sample
        [O] bisect-bad  sample

readonce:

                               vm-scalability.throughput

    4e+07 ++----*-----*----**-----------------------------------------------+
          *****  ***** *.**  *****.* * ** *. *   *                          |
  3.5e+07 ++                        * *  *  * *** *                         |
          |                                                                 |
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
  1.5e+07 OOOOO OOO OOOO OOOOOOOOO OOO OOOO  OOOOOOO OOOOOOOOO OOO          |
          |        O                  O     O                     OOOOO OOOOO
    1e+07 ++----------------------------------------------------------------+

Here are more graphs for other stats in the readonce case, which can
give us a visual impression which stats are stable and have clear
separable values before/after the first bad commit.

                                  time.user_time

  105 ++--------------------------------------------------------------------+
      |                              O O   O O                              |
  100 ++                            O O O O O O O O  O OOOO                 |
      | O OO                                     O OO                     O O
      OO                                                         O O   OO  O|
   95 ++    OOOO OOO   OOO    OO                             O OO   OOO     |
      |       * .* :OO    OO O  OOO                        O  O             |
   90 ++      :*   *      * .***                                            |
      ***.***:      :*.*  :*   :                                            |
   85 ++     *      *   **      :  .*        * .**                          |
      |                         * *  * **. ** *                             |
      |                          ::   *   *                                 |
   80 ++                         *                                          |
      |                                                                     |
   75 ++--------------------------------------------------------------------+


                                  time.system_time

  40000 ++------------------------------------------------------------------+
        |                                                                   |
  35000 OOOO OOOOOO OOOOOOO OOOOOO OOOOOOO OOOOOOO OOOOOO OOOOOOO OOOOOO OOOO
        |                                                                   |
        |                                                                   |
  30000 ++                                                                  |
        |                                                                   |
  25000 ++                                                                  |
        |                                                                   |
  20000 ++                                                                  |
        |                                                                   |
        |                                                                   |
  15000 ****.* ****.* ***  .**** *.*******.*******                          |
        |     *      *   **     *                                           |
  10000 ++------------------------------------------------------------------+


                         time.percent_of_cpu_this_job_got

  9000 +O--OO---------------------------------------------------------------+
       O O   O OO O OO    OOOOO O    O  O  O  O  O     OO O      O OOOOOO OOO
  8500 ++     O    O  OO O       OOOO  O OO O  OO OO OO  O  OOOOO           |
       |                                                                    |
       |                                                                    |
  8000 ++                                                                   |
       |                                                                    |
  7500 ++                                                                   |
       |                                                                    |
  7000 ++                                                                   |
       |                                                                    |
       | *.     *.            *. ****     * *.** *                          |
  6500 **  *****  ******.*****  *    *.*** *    *                           |
       |                                                                    |
  6000 ++-------------------------------------------------------------------+


                                 time.elapsed_time

  450 ++--------------------------------------------------------------------+
      |                                                                     |
      |      O   OO  O OOOO   O OOO O OOO O OOO OO OO    O O OOO       OO   |
  400 OOO OOO OO   OO      O O O     O     O      O  O OO O     OO OOOO   OOO
      |                                                                     |
      |                                                                     |
  350 ++                                                                    |
      |                                                                     |
  300 ++                                                                    |
      |                                                                     |
      |                                                                     |
  250 ++                                                                    |
      |                                                                     |
      *      * *.   * .*        ***.*****.*****.**                          |
  200 +**-***-*--***-*--****-***--------------------------------------------+


                               time.minor_page_faults

  126000 ++-----------------------------------------------------------------+
         |                                                                  O
  125000 ++                                                        OO OO  O |
  124000 ++ O O   O    O               O  O O   O O     O        OO        O|
         |     OOO  OO   O  O    O O O  OO O OO  O O     O OO O O    O      |
  123000 OO        O      OO O OO O           *.*   OOO      O              |
  122000 ++O       *       *    *  *O*   * *  : :         O              O  |
         |       ** * .*O* : *. :: :* : * ::*:   :*                         |
  121000 ++      :   *  * * :: * **   : : *  *   *                          |
  120000 ++      :          *          *                                    |
         |       :                                                          |
  119000 *+* .* :                                                           |
  118000 +* *  ::                                                           |
         |     **                                                           |
  117000 ++-----------------------------------------------------------------+


                          time.voluntary_context_switches

  5000 ++-------------------------------------------------------------------+
       |                                                                    |
       |                                                                    |
  4500 ++                                      *                            |
       |        *.    *      *          * *  .*:                            |
       |        : * ** :  *  :*.* *     :*: *   :*                          |
  4000 +**.*****   *   *.* :*    :::*  :   ::   *                           |
       *                   *     * * *.*   :                                |
  3500 ++                                  *                                |
       |                                                                    O
       |                                                           O  O     |
  3000 ++  O                                                     O  OO O  OO|
       |      O O      O   OOOO   O OO  OOOO  OOOO     O    OO          O   |
       OOO  OO O  OOOOO  OO      O O   O    O     OO OO OOO   O O           |
  2500 ++-----------------------O------------------------------O------------+


                          time.involuntary_context_switches

  300000 ++-----------------------------------------------------------------+
         |OO  OOOOOOOO O  OOOO O   OOO  O OO O  OOOOOOO  OOO OO OOOOOO O OO O
  280000 O+ O           OO      OOO    O O  O O         O   O         O    O|
         |                                                                  |
  260000 ++                                                                 |
         |                                                                  |
  240000 ****.***                                                           |
         |      :                                                           |
  220000 ++     :                                                           |
         |      :                                                           |
  200000 ++      :                                                          |
         |       :                                                          |
  180000 ++      :                                                          |
         |       :*   .* *      * ****.********.***                         |
  160000 ++------*-***--*-****-*-*------------------------------------------+


                                    uptime.boot

  500 ++--------------------------------------------------------------------+
      |                                                                     |
      |     OOOO OO OO OOOOO OOOOOO OOOOO OOOOO OOOOOO OOOOO OOOOO OOO OO OO|
  450 OOO OO       O                                                  O     O
      |                                                                     |
      |                                                                     |
  400 ++                                                                    |
      |                                                                     |
  350 ++                                                                    |
      |                                                                     |
      |                                                                     |
  300 ++                                                                    |
      |                                                                     |
      ***.******.** **.*  **.** ***.*****.*****.**                          |
  250 ++-----------*----**-----*--------------------------------------------+


                               vm-scalability.throughput

    4e+07 ++----*-----*----**-----------------------------------------------+
          *****  ***** *.**  *****.* * ** *. *   *                          |
  3.5e+07 ++                        * *  *  * *** *                         |
          |                                                                 |
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
  1.5e+07 OOOOO OOO OOOO OOOOOOOOO OOO OOOO  OOOOOOO OOOOOOOOO OOO          |
          |        O                  O     O                     OOOOO OOOOO
    1e+07 ++----------------------------------------------------------------+


                              vm-scalability.stddev

  10 ++---------------------------------------------------------------------+
     |     O         O         O  O O     O           O                     |
   9 ++     O  O      O  O O  O                O  OO       OO               |
   8 +OO O         O   O     O   O O   OOO   OO O    O    O   OOO     OO   O|
     O    O  O  OOO     O            O      O       O   OO        O O   O O O
   7 ++                     O                                    O   O      |
   6 ++                                                                     |
     |                                                                      |
   5 ++                                                                     |
   4 ++                                                                     |
     |                                                                      |
   3 +**. *   .*** *.* *                                                    |
   2 *+  * :**    *   * **.*****.*****.****.*****                           |
     |     *                                                                |
   1 ++---------------------------------------------------------------------+




  50 ++-------------------------------------------------------O-------------+
     |          O    O            O                            OO    O      |
  45 OO   OOOO    O            O    O     O    OO   O O O OOO    O          |
  40 ++O       O O O  OOOO    O                    O O            O       OOO
     |                     OO    O O O O O  OO    O                 O   O   |
  35 ++  O                              O     O          O            O     |
     |                       O                                         O    |
  30 ++                                                                     |
     |                                                                      |
  25 ++                                                                     |
  20 ** .*  * .*   *. *      *            *.*****                           |
     | *  ** * : **  * ***.** **.*****. **                                  |
  15 ++         ::                     *                                    |
     |          *                                                           |
  10 ++---------------------------------------------------------------------+




  50 ++---------------------------------------------------------------------+
     |                       O   O   O O    OO                      O O O   |
  45 ++O O       O O    OO    O                   OO              O    O   OO
     OO   OOO  O      OO       O    O   O O   OOO    O  O OO              O |
     |       O    O         O     OO     O          O O  O  O    O   O      |
  40 ++         O          O                                  OOO           |
     |               O                                                      |
  35 ++                                                                     |
     |          *                                                           |
  30 ++      *. :**      *.   **                                            |
     ***.   *  *   *.*  *  ***  : ***     *.**                              |
     |   ***          **        : :  :.***    **                            |
  25 ++                          :   *          *                           |
     |                           *                                          |
  20 ++---------------------------------------------------------------------+




  11 ++---------*-----------------------------------------------------------+
     |          :                                                           |
  10 ++         :                                                           |
   9 ++         :                                                           |
     |         : :                                                          |
   8 ++        : :     **              *                                    |
   7 ++*. ****.* :* .**  *.*****.**** + *  .** *                            |
     **  *       * *                 *   **   * :                           |
   6 ++                                         *                           |
   5 ++                                                                     |
     |                                                                      |
   4 ++                                                                     |
   3 ++                    O                                                |
     |OO  O OO OO OO OOOO   O    OOO    OO    O   OOOOO  O     OOOO O  O  O |
   2 O+--O-O-----O-------O---OOO----OO-O--O-OO-OO-------O-OOO-O------OO-O--OO




    3 ++--------------------------------------------------------------------+
      |          *                                                          |
  2.5 ++         :                                                          |
      |         ::                                                          |
      |         : :                                                         |
    2 ++        : :     **                                                  |
      | *.*****:  :* *.*  **.******.** **.  **                              |
  1.5 **       *  * *                 *   **  *.**                          |
      |                                                                     |
    1 ++                                                                    |
      |                    O                                                |
      |OO     O   O  O OOO   OO     O   O O   O O OOOO   O     O O  O  OO O |
  0.5 O+  OOOO O O OO     O    OOOO  OOO   OOO   O     OO OO OO O  O OO    OO
      |                                                                     |
    0 ++--------------------------------------------------------------------+




  1.8 ++-----------------------------------*--------------------------------+
      |                                    :                                |
  1.6 ++                                   :                                |
  1.4 ++       *.                          ::                               |
      |        : *        *               : :                               |
  1.2 ++       : :       * :       .*     : :                               |
      |     ** :  : * .**  *  *   * :  *  : **  **                          |
    1 *+*.** ::   * :*     O:O:   : O: :O.O   O+                       O    |
      |*      :    :        ::O: :   ** *     *          O                  |
  0.8 ++  O   *    *         * ***                                          |
  0.6 ++                                                                    |
      |                                                                     |
  0.4 ++                                                                    |
      |    OOOOO O O O OOO     OOOO             O OOOO    OO OOOOO  OO  O OOO
  0.2 OOO---------O-O-----O----------OOO---OOO---O-----OO----------O--O-----+




  7 ++--------------------------*-------------------------------------------+
    |      *       *            :            *. *                           |
  6 ++     :+   * : :      *    :    * *.*   : *                            |
    |   ** : *  ::: **   * :*  ::    ::: :   :                              |
  5 +* : ::   *: *    :  ::  + *:  ** *   : :                               |
    * ::  :    *      *  :*   *  : :      * *                               |
  4 ++*   *            +:        ::        ::                               |
    |                   *        ::        *                                |
  3 ++                           :                                    O     |
    |                            *                      O                   |
  2 ++                                                                      |
    |                    O                      O                      O    |
  1 ++        O                                       O                     |
    |                   O       O   O     O                     OO O        |
  0 OOO-OOOO-O-OOO-OOOO---OOO-OO-O-O-OOO-O-OOO-O-OO-OO-O--OOOO-O--O--O--O-OOO




  2.5 ++--------------------------------------------------------------------+
      |                                                                     |
      |                                                                     |
    2 ++    * **  ** *     *.* *   .*                                       |
      ***.**: ::  : ::    *   * * *  ** *.**                                |
      |      :::  : * :   :     : :    ::  :  *.*                           |
  1.5 ++     :  : :   :  :       :     *    : :  *                          |
      |      *  : :    :**       *          **                              |
    1 ++        : :    *                                                    |
      |         ::                                                          |
      |         ::                                                          |
  0.5 ++        ::                                                          |
      |          :                                       O                  |
      OOO OOOOOO OOOOO OOOOO OOOOOO OOOOO OOOOO OOOOOO OO OO OOOOO OOOOOO OOO
    0 ++---------*----------------------------------------------------------+




    5 ++--------------------------------------------------------------------+
  4.5 ++   **         .*                         *                          |
      |    ::       ** :                         :                          |
    4 *+  * :       :   :             *          :                          |
  3.5 ++  : :   .*  :   *             :         :                           |
      |: :   : * : :     : *      *  : :*       :                           |
    3 +* :   : :  ::     * :+ * * :+ * ::+      *                           |
  2.5 ++*    ::   **      *  * * *  *  *  ***  :                            |
    2 ++     **                              : :                            |
      |                                      **                             |
  1.5 ++                                                                    |
    1 ++                                                                    |
      |                                                                     |
  0.5 ++  O                O O          O     O          O           O      |
    0 OOO--OOOOO-OOOOO-OOOO---OOOOO-OOOO--OOOO--OOOOOO-OO-OO-OOOOO-OO-OOO-OOO




  1.8 ++--------------------------------------------------------------------+
      |                                *                                    |
  1.6 ++     *           *       *     ::                                   |
  1.4 ++*. ** **   * *. * **.* * :*.**: *.  **                              |
      **  *    :  * :: *      * *     *   **  :.*                           |
  1.2 ++       :  : *                         *  *                          |
    1 ++        : :                                                         |
      |         : :                                                         |
  0.8 ++        : :                                                         |
  0.6 ++        : :                                                         |
      |         ::                                                          |
  0.4 ++        ::                                                          |
  0.2 ++         :           O                           O                  |
      OOO OOOOOO OOOOO OOOOO  OOOOO OOOOO OOOOO OOOOOO OO OO OOOOO OOOOOO OOO
    0 ++---------*----------------------------------------------------------+




  1.4 ++-------------------------------*------------------------------------+
      |                  *             :                                    |
  1.2 ++     *          *:       *     :    **                              |
      |      :         * :       :     ::   ::                              |
    1 ++     ::        :  :      ::   : :   : :                             |
      |     : :       :   :     : :   : :  :  :                             |
  0.8 ***.*** **  *   :   **. *** *. ** *.*:  *.**                          |
      |        :  :***       *      *      *                                |
  0.6 ++        : :                                                         |
      |         ::                                                          |
  0.4 ++         :                                                          |
      |          *           O                           O                  |
  0.2 ++  O               OO  O     O     O         O              O        |
      OOO  OOOOO OOOOO OOO     OOOO  OOOO  OOOO OOOO O OO OO OOOOO  OOOOO OOO
    0 ++--------------------------------------------------------------------+




  8 ++----------------*-----------------------------------------------------+
    |    **           ::                                                    |
  7 ++ .* :          : :                                                    |
  6 *+*    :     *.***  * *     *    *          *                           |
    |*     :.* * :      : :**.  :*.* :*         :                           |
  5 ++     *  ::*        :    **    *  *.      :                            |
    |         *          *               * *  .*                            |
  4 ++                                    * **                              |
    |                                                                       |
  3 ++                                                                      |
  2 ++                                                                      |
    |                                                                       |
  1 ++                                                                      |
    |                                                                       |
  0 OOO-OOOO-OOOOO-OOOO-OOOOO-OOOO-OOOOO-OOOOO-OOOO-OOOOO-OOOO-OOOOO-OOOO-OOO


                                cpuidle.C6-IVT-4S.time

   1.4e+10 ++---------------------------------------------------------------+
           |                                          O                     |
  1.35e+10 ++                                                               |
           |                             O    O                             |
           |      O   O             OO O    O     O       O   OOO           |
   1.3e+10 ++        O   OO OOO  OOO      OO   OO  OOO O     O   O      OO  |
           |     O OO   O      OO       O    O          OO O                |
  1.25e+10 ++          O                                          OOO       |
           |                                                          O   OOO
   1.2e+10 ++                            *                           O      |
           O O          *                :*                                 |
           *O OO.       ::*.        **  *  * **   **                        |
  1.15e+10 +* :  ** *  * *  ****   *  + :  : : : +                          |
           | *     * **         ***    *    :  **                           |
   1.1e+10 ++-------------------------------*-------------------------------+


                               cpuidle.C6-IVT-4S.usage

  150000 ++-----------------------------------------------------------------+
         |               O                                                  |
  140000 ++                                                                 |
         |          O               O  O    O    O    O O     O             |
         | O   O O                O      O    O    O O           O          |
  130000 ++     O OO    O O OO  O       O O  O  O   O      O O              |
         |           O O         O O              O      OO                 |
  120000 OO O              O         O     O                O   O      O  O O
         |    O                O                                  OO     O O|
  110000 ++                                                          OO     |
         |         *  .** * * .**  **   **   **.* *                 O       |
         |        * **   * * *   **  *.*  ***    *                          |
  100000 ++  .*  *                                                          |
         ****  :*                                                           |
   90000 ++----*------------------------------------------------------------+


                                  turbostat.%c0

  80 ++---------------------------------------------------------------------+
     |                                                                      |
  75 OOO OO                                                          O      |
     |     OOO OOOOO OOOOO OOOOO OOOOO OOOO OOOOO OO OO OOOOO OOOOO O OOO OOO
     |                                              O                       |
  70 ++                                                                     |
     |                                                                      |
  65 ++                                                                     |
     |                                                                      |
  60 ++                                                                     |
     |                                                                      |
     |                             *     *   **                             |
  55 ***.*****.*****.*****.*****.** **.** *.*  **                           |
     |                                                                      |
  50 ++---------------------------------------------------------------------+


                                   turbostat.%c1

  2.8 ++--------------------------------------------------------------------+
  2.6 ++         *     *                                                    |
      |          :     :                                                    |
  2.4 ++        ::    : :                                                   |
  2.2 ++        : :   : :                                                   |
      |        *  : **  *       *   *                                       |
    2 ++       :  * :    ***. * :* + :* *. ** *. *                          |
  1.8 +**.**O**    *         * *  *  O *  *O *  *                           |
  1.6 *+    *                                                               |
      | O              O   O            O     O   O  O                      |
  1.4 ++      O                 O O    O     O  O  O   O     OO O          O|
  1.2 ++     O O OO     O      O      O     O             O                 |
      OO   O       OOO   OO   O  O                  O   OO O   O   O  O     |
    1 ++  O                  O      O     O      O               O   O OO O O
  0.8 ++------------------------------------------------------------O-------+


                                  turbostat.%c6

  50 ++---------------------------------------------------------------------+
     |                                                                      |
  45 ++   **                                                                |
     ***.*  **.*****.*****.*****.** **.** *.** **                           |
     |                             *     *    *                             |
  40 ++                                                                     |
     |                                                                      |
  35 ++                                                                     |
     |                                                                      |
  30 ++                                                                     |
     |                                                                      |
     |      O    O   O OOO   O O OOO O   O  O  OO   O    O OO OO       OO   |
  25 OOO OOO O OO OO  O    OO O     O  OO O  OO   OO OO O O     OOO OOO   OOO
     |                                                                      |
  20 ++---------------------------------------------------------------------+


                                 turbostat.%pc2

  35 ++---------------------------------------------------------------------+
     |                                                                      |
  30 *+    *                                                                |
     |**.** :*   **     **. **    * * .**  .** **                           |
     |      * + *  *.  *   *  * .* * *   **   *                             |
  25 ++        *     * :       *                                            |
     |                :                                                     |
  20 ++               *                                                     |
     |      O    O   O       O    O  O      O       O   OO                  |
  15 +O  OO  O OO OO  OOOO OO OO O O   OOO   OOOO OO OO   OOO OOOOO OO OO O O
     O O                                                              O     |
     |                              O     O                                 |
  10 ++    O                                                               O|
     |                                                                      |
   5 ++---------------------------------------------------------------------+


                                  turbostat.Pkg_W

  420 ++--------------------------------------------------------------------+
      OOO OOO      O                 O     O                               O|
  410 ++     OOO OO OO OOOOO OOOOOO O OOO O OOO OOOO      OO    OO OOOO   O O
  400 ++                                             O OOO   OOO       OO   |
      |                                             O                       |
  390 ++                                                                    |
      |                                                                     |
  380 ++                                                                    |
      |                                                                     |
  370 ++                                                                    |
  360 ++                                                                    |
      |      * *.*   *.*        ***.*    .*   *.                            |
  350 +**. ** *   ***   ****.***     * **  ***  **                          |
      *   *                           *                                     |
  340 ++--------------------------------------------------------------------+


                                  turbostat.Cor_W

  350 ++--------------------------------------------------------------------+
      OOO OOO                        O     O                                |
  340 ++      OO  OOO  OO  O O O       O     O  O O                  OO    OO
  330 ++     O   O   O   OO   O OOO O O O O O O  O O O OOOOO OOOOO OO  OO O |
      |                                             O                       |
  320 ++                                                                    |
      |                                                                     |
  310 ++                                                                    |
      |                                                                     |
  300 ++                                                                    |
  290 ++                                                                    |
      |               .*        *  .*         *                             |
  280 ++     ***.*****  ****. ** **  :   .* ** + *                          |
      ***. **                *       ****  *    *                           |
  270 ++--*-----------------------------------------------------------------+


                                 turbostat.RAM_W

  98 ++---------------------------------------------------------------------+
     | *.    *.        **   **                                              |
  96 **  ****  *****.**  *.*  **. ***   ***.*****                           |
     |                           *   *.*                                    |
  94 ++                                                                     |
     |                                                                      |
  92 ++                                                                     |
     |                                                                      |
  90 ++                                                                     |
     |                                                                      |
  88 ++                                                                     |
     |                                                                      |
  86 OOO OOOOO  OOOO OOOOO OOOOO  O OO OO O OOOOO OO OO OOOOO OOO           |
     |         O                 O O     O          O            OO OOOOO OOO
  84 ++---------------------------------------------------------------------+


                             numa-numastat.node0.numa_hit

  2.6e+08 ++----------------------------------------------------------------+
          |****.*   * * .* ***** *. ** ****.* *****                         |
  2.5e+08 *+     *** * *  *     *  *  *      *                              |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          |                                                                 |
  2.1e+08 O+O    OO OO   O    OOO    O OOO   OOO      OO OOOO  O   OO     O |
          |O OO O  O  OO   OOO   O  O     O     OOOO O  O    O  OOO  OO OO OO
    2e+08 ++              O        O  O     O                               |
          |                                                                 |
  1.9e+08 ++----------------------------------------------------------------+


                             numa-numastat.node0.numa_miss

  4.5e+07 ++-------------------------------------O-----------O--O-----------+
          |O    OOO  OO   OO       OOOOO  O OO  O  O O OO OO     O          |
    4e+07 O+OOO    OO  O O  OOOOOO      OO    OO  O   O  O  O  O            |
          |                                                         OO  O   |
  3.5e+07 ++                                                      O      OOOO
          |                                                        O  O     |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          *       *    *.  *      .*  *      *                              |
  1.5e+07 +****. * *** : ** * ****  :* : **. :* ***                         |
          |     *     *      *      *  **   *  *                            |
    1e+07 ++----------------------------------------------------------------+


                           numa-numastat.node0.numa_foreign

    4e+07 ++----O-O--------O--O-----------O-----O---------------------------+
          |OO    O OOOOO OO O  OOO OOOOOOO  OOOO OOO  O   OOOO OOO          |
  3.5e+07 O+ OO              O                       O OOO        OO  O  O O|
          |                                                         OO  O O O
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++           *. *                                                 |
          *  *    **** : * ** ****.** * ***. * ****                         |
    1e+07 +** *.**    *      *       * *    * *                             |
          |                                                                 |
    5e+06 ++----------------------------------------------------------------+


                            numa-numastat.node0.local_node

  2.6e+08 ++----------------------------------------------------------------+
          |****.*   * * .* ***** *. ** ****.* *****                         |
  2.5e+08 *+     *** * *  *     *  *  *      *                              |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          |                                                                 |
  2.1e+08 O+O    OO OO   O    OOO    O OOO   OOO      OO OOOO  O   OO     O |
          |O OO O  O  OO   OOO   O  O     O     OOOO O  O    O  OOO  OO OO OO
    2e+08 ++              O        O  O     O                               |
          |                                                                 |
  1.9e+08 ++----------------------------------------------------------------+


                            numa-numastat.node0.other_node

  4.5e+07 ++-------------------------------------O-----------O--O-----------+
          |O    OOO  OO   OO       OOOOO  O OO  O  O O OO OO     O          |
    4e+07 O+OOO    OO  O O  OOOOOO      OO    OO  O   O  O  O  O            |
          |                                                         OO  O   |
  3.5e+07 ++                                                      O      OOOO
          |                                                        O  O     |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          *       *    *.  *      .*  *      *                              |
  1.5e+07 +****. * *** : ** * ****  :* : **. :* ***                         |
          |     *     *      *      *  **   *  *                            |
    1e+07 ++----------------------------------------------------------------+


                             numa-numastat.node1.numa_hit

  2.6e+08 ++----*-----*---------------------*-------------------------------+
          *****  ***** *.*********.********  ******                         |
  2.5e+08 ++                                                                |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          |                                                                 |
  2.1e+08 ++                                                   O            O
          O O O   O OOO        O        OO    OO O O  OO O O         O      |
    2e+08 +O O  O  O   O  OOOO  OO OOO O  O  O  O    O  O O OO  OOOOO O OOO |
          |      O       O    O       O     O     O                        O|
  1.9e+08 ++----------------------------------------------------------------+


                             numa-numastat.node1.numa_miss

  4.5e+07 ++------------------O---------------------------------------------+
          |OO   OO  O O  OOOO   O    OOOO O OOO O O    O  O  O  OO         O|
    4e+07 O+ O    OO O O     O O O OO    O     O O O OO  O OO     OO    O   |
          |   O                                         O      O    OOO  OO |
  3.5e+07 ++                                                                O
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++       *   *.**                                                 |
          *  **. ** ** :   *******.********. ******                         |
    1e+07 +**---*-----*---------------------*-------------------------------+


                           numa-numastat.node1.numa_foreign

    4e+07 ++----------------------------------------------------------------+
          |       O            O          O     O     O   O                 |
  3.5e+07 OOOOO O  OOOOO OOOOOO OO O OOOOO  OOOO OOO O O O OOO OOO          |
          |      O                  O                   O         O O   O O |
    3e+07 ++                                                       O OO  O OO
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                    *                                           |
          ****   *******.*** *** :.***** **.*** ***                         |
    1e+07 ++  *.*           *    *      *      *                            |
          |                                                                 |
    5e+06 ++----------------------------------------------------------------+


                            numa-numastat.node1.local_node

  2.6e+08 ++----*-----*---------------------*-------------------------------+
          *****  ***** *.*********.********  ******                         |
  2.5e+08 ++                                                                |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          |                                                                 |
  2.1e+08 ++                                                   O            O
          O O O   O OOO        O        OO    OO O O  OO O O         O      |
    2e+08 +O O  O  O   O  OOOO  OO OOO O  O  O  O    O  O O OO  OOOOO O OOO |
          |      O       O    O       O     O     O                        O|
  1.9e+08 ++----------------------------------------------------------------+


                            numa-numastat.node1.other_node

  4.5e+07 ++------------------O---------------------------------------------+
          |OO   OO  O O  OOOO   O    OOOO O OOO O O    O  O  O  OO         O|
    4e+07 O+ O    OO O O     O O O OO    O     O O O OO  O OO     OO    O   |
          |   O                                         O      O    OOO  OO |
  3.5e+07 ++                                                                O
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++       *   *.**                                                 |
          *  **. ** ** :   *******.********. ******                         |
    1e+07 +**---*-----*---------------------*-------------------------------+


                             numa-numastat.node2.numa_hit

  2.65e+08 ++---------------------------------------------------------------+
   2.6e+08 +*  *.*                *          *                              |
           * **   ******** .****** ***.****** ***.**                        |
  2.55e+08 ++             *                                                 |
   2.5e+08 ++                                                               |
           |                                                                |
  2.45e+08 ++                                                               |
   2.4e+08 ++                                                               |
  2.35e+08 ++                                                               |
           | O                                         O                    |
   2.3e+08 ++  O O O      O  O O  O     OO   OO   O  O                      |
  2.25e+08 ++     O  OO  O  O O  O OOO    O    O   OO   OO O  OOOO  O       |
           OO       O  O        O      O   OO   O     O   O        O OO O  OO
   2.2e+08 ++ O         O                                    O    O      OO |
  2.15e+08 ++---------------------------------------------------------------+


                             numa-numastat.node2.numa_miss

    4e+07 ++----------------------------------------------------------------+
          |                    O          O     O     O                     |
  3.5e+07 ++  O O O  O O O       O      O     O   OO      O O  OOO          |
          OOOO     OO O   OOOOO O  OOOOO O  OO O O   O O O O O      O     O |
    3e+07 ++     O                                      O         OO OO OO OO
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++              *                                                 |
          |           **. :*  * * .** **      * * *                         |
    1e+07 *****.******   *  ** * *   *  ***.** * *                          |
          |                                                                 |
    5e+06 ++----------------------------------------------------------------+


                           numa-numastat.node2.numa_foreign

    5e+07 ++-------------------------O--------------------------------------+
          |     OO  O O  O O OO OO O   O     O         OO O      O          |
  4.5e+07 +O OO   OO O O  O O  O    O O OOO O OOOOOO OO  O OOO  O           |
    4e+07 O+O                                                  O  O     O  O|
          |                                                         OOO   O |
  3.5e+07 ++                                                       O     O  O
          |                                                                 |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 *+      *  * *.*    * *                                           |
          |****.** ** *   **** * *.********.*******                         |
    1e+07 ++----------------------------------------------------------------+


                            numa-numastat.node2.local_node

  2.65e+08 ++---------------------------------------------------------------+
   2.6e+08 +*  *.*                *          *                              |
           * **   ******** .****** ***.****** ***.**                        |
  2.55e+08 ++             *                                                 |
   2.5e+08 ++                                                               |
           |                                                                |
  2.45e+08 ++                                                               |
   2.4e+08 ++                                                               |
  2.35e+08 ++                                                               |
           | O                                         O                    |
   2.3e+08 ++  O O O      O  O O  O     OO   OO   O  O                      |
  2.25e+08 ++     O  OO  O  O O  O OOO    O    O   OO   OO O  OOOO  O       |
           OO       O  O        O      O   OO   O     O   O        O OO O  OO
   2.2e+08 ++ O         O                                    O    O      OO |
  2.15e+08 ++---------------------------------------------------------------+


                            numa-numastat.node2.other_node

    4e+07 ++----------------------------------------------------------------+
          |                    O          O     O     O                     |
  3.5e+07 ++  O O O  O O O       O      O     O   OO      O O  OOO          |
          OOOO     OO O   OOOOO O  OOOOO O  OO O O   O O O O O      O     O |
    3e+07 ++     O                                      O         OO OO OO OO
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++              *                                                 |
          |           **. :*  * * .** **      * * *                         |
    1e+07 *****.******   *  ** * *   *  ***.** * *                          |
          |                                                                 |
    5e+06 ++----------------------------------------------------------------+


                             numa-numastat.node3.numa_hit

  2.6e+08 ++----**----------------------------**----------------------------+
          *****   ******.*********.********.**  ***                         |
  2.5e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          | O             O              O     O   O               O        O
          OO OO OOO  OOO   OO O O   O   O O   O OOO  O O OOO O OOOO OO  OOO |
  2.1e+08 ++       OO        O O   O OOO    OO        O O             O    O|
          |              O       O                          O               |
    2e+08 ++----------------------------------------------------------------+


                             numa-numastat.node3.numa_miss

  4.5e+07 ++-------------O-----------O-O-----O------------------------------+
          |     OO  O O    OOOO OO O      O     O  O  OOOOOOO    O          |
    4e+07 OOOOO   OO O O  O    O      O OO  O OO OO  O       O OO           |
          |                         O                                       |
  3.5e+07 ++                                                      O O O O OOO
          |                                                        O O   O  |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                 *  *                                           |
          *****.  ******.**** ** :.********.*   ***                         |
    1e+07 ++----**---------------*-----------***----------------------------+


                           numa-numastat.node3.numa_foreign

  4.5e+07 ++----------------------------------------------------------------+
          |     OO   O O             O           O           O  OO          |
    4e+07 OOOOO   OOO O  OOOO OOOO OO OOO O OOO O OO OOOO OOO  O            |
          |                  O           O     O         O        O OO  OO  |
  3.5e+07 ++                                                       O  O   OO|
          |                                                                 O
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 *+      ** * *. *       .*  *                                     |
          |****. *  * *  * *******  ** ****.*******                         |
    1e+07 ++----*-----------------------------------------------------------+


                            numa-numastat.node3.local_node

  2.6e+08 ++----**----------------------------**----------------------------+
          *****   ******.*********.********.**  ***                         |
  2.5e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
  2.4e+08 ++                                                                |
          |                                                                 |
  2.3e+08 ++                                                                |
          |                                                                 |
  2.2e+08 ++                                                                |
          | O             O              O     O   O               O        O
          OO OO OOO  OOO   OO O O   O   O O   O OOO  O O OOO O OOOO OO  OOO |
  2.1e+08 ++       OO        O O   O OOO    OO        O O             O    O|
          |              O       O                          O               |
    2e+08 ++----------------------------------------------------------------+


                            numa-numastat.node3.other_node

  4.5e+07 ++-------------O-----------O-O-----O------------------------------+
          |     OO  O O    OOOO OO O     OO    OO  O  OOOOOOO    O          |
    4e+07 OOOOO   OO O O  O    O      O O   O O  OO  O       O OO           |
          |                         O                                       |
  3.5e+07 ++                                                      O O O O OOO
          |                                                        O O   O  |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                 *  *                                           |
          *****.  ******.**** ** :.********.*   ***                         |
    1e+07 ++----**---------------*-----------***----------------------------+


                                 vmstat.procs.r

  95 ++---------------------------------------------------------------------+
     |                                                                      |
  90 OOO OOO O OOOOO OOOOO OOOOO O OO  OOOO  OOOO OO OO OOOO  OOOOO OOOOO OOO
     |      O                     O  O      O       O       O               |
  85 ++                                                                     |
     |                                                                      |
  80 ++                                                                     |
     |                                                                      |
  75 ++                                                                     |
     |                                                                      |
  70 ++                                                                     |
     |                                   *   **                             |
  65 ***.*****.*****.*****.*****.*****.** *.*  **                           |
     |                                                                      |
  60 ++---------------------------------------------------------------------+


                                  vmstat.memory.free

  1.4e+08 ++--------------*-------------------------------------------------+
          |   *.*     *   ::            *        *                          |
  1.3e+08 ** *   ***** *.* *******.*** * **.***** *                         |
          | *                         *                                     |
  1.2e+08 ++                                                                |
          |                                                                 |
  1.1e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    9e+07 ++                                                                |
          |                                           O                     |
    8e+07 ++     O          O  OOO  OOO O O O O OOO       O OO  O       OO  |
          OOOOO O OOOOOO OOO OO    O   O O   O O   O O OOO O   O OOOOOO   OOO
    7e+07 ++----------------------------------------------------------------+


                                  vmstat.memory.cache

  4.5e+08 +O-O--------------------------------------------------------------+
          O O O O OOOOOO OOO OOOO  O   O OO  O OO  O O OOO O   O OOOOOO  OOOO
  4.4e+08 ++     O          O    O  OOO O   O O  OO       O OO  O       O   |
          |                                           O                     |
  4.3e+08 ++                                                                |
          |                                                                 |
  4.2e+08 ++                                                                |
          |                                                                 |
  4.1e+08 ++                                                                |
          |                                                                 |
    4e+08 ++                                                                |
          | *                         *                                     |
  3.9e+08 ** *   *******.* *******.*** * **.*******                         |
          |   *.*         *             *                                   |
  3.8e+08 ++----------------------------------------------------------------+


                                  vmstat.system.in

  120000 ++-----------------------------------------------------------------+
         |                                 *  *.                            |
  115000 ****.  *** *    *  **.*******.* ** **  ***                         |
  110000 ++   **  : :*.** **            *                                   |
         |        : :                                                       |
  105000 ++       : :                                                       |
  100000 ++        :                                                        |
         |         :                                                        |
   95000 ++        :                                                        |
   90000 ++   O    *                                                        |
         OOO   OOO OOO OOOOOO    OOOO  OOO  OOO   O O O OO OOOO OO OOO      O
   85000 ++       O                              O O O            O   O  OOO|
   80000 ++ O                  O                                            |
         |                   O  O    O    OO    O         O            O    |
   75000 ++-----------------------------------------------------------------+


                                 vmstat.system.cs

  3600 ++-------------------------------------------------------------------+
       |                                                                    |
  3400 ***.****                                                             |
  3200 ++     :                                                             |
       |      :                                                             |
  3000 ++      :                                                            |
       |       :                                                            |
  2800 ++      **.* *   .******.* *          .**                            |
       |           * ***         * ***.******   **                          |
  2600 ++                                                                   |
  2400 ++                                                                   |
       |                                                                    |
  2200 ++                                                                   |
       OOO OOOOOO OOOOOO OOOOOO OOOOOO OOOOOO OOOOOO OOOOOO  OOOOO OOOOOO OOO
  2000 ++---------------------------------------------------O---------------+


                                    softirqs.TIMER

  1.5e+07 ++-------------------O--------O-----O----------------O------------+
          OOOOO OOOOOOOO OOOOOO OO OOOOO OO OO OOOOO OOOOOOOOO  OOOOOOO OOOOO
  1.4e+07 ++                                                                |
  1.3e+07 ++                                                                |
          |                                                                 |
  1.2e+07 ++                                                                |
  1.1e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
    9e+06 ++                                                                |
          |                                                                 |
    8e+06 ++       *              .*****  *.***** *                         |
    7e+06 *****.*** ****.**  *****      **       *                          |
          |                **                                               |
    6e+06 ++----------------------------------------------------------------+


                                   softirqs.SCHED

  350000 ++-----------------------------------------------------------------+
         OOOO OOOOO O   OOO OO OOOOOOO OOO OOOO  OOOOO  OOOOOOO OOOOOOOO OOOO
         |                                                                  |
  300000 ++                                                                 |
         |                                                                  |
         |                                                                  |
  250000 ++                        *         * .*                           |
         ****.*****O*O.O** O   **** **.*  O** * O *   O                     |
  200000 ++              :     :       :  :     : :                         |
         |               :    :        :  :     : :                         |
         |                :   :         ::       ::                         |
  150000 ++               :   :         ::       :                          |
         |                :  :          :*       :                          |
         |                ****          *        *                          |
  100000 ++-----------------------------------------------------------------+


                                    softirqs.RCU

  900000 ++-----------------------------------------------------------------+
         |                          OO     O                              O |
  800000 OO               O                        O        OO             O|
         | O  OOOO  O   OO OOO OOOOO   O    O        OO OOO   O OO O     O  O
  700000 ++ O     OO O O                OOO  OO OOO O      O      O OOOO    |
         |                                                                  |
  600000 ++                                                                 |
         |                                                                  |
  500000 ++                                                                 |
         |                                                                  |
  400000 +**                                                                |
         *  :   * **  .*        *  **                                       |
  300000 ++ *.** *  **  *     .* **  *.* ******.* *                         |
         |               *****          *        *                          |
  200000 ++-----------------------------------------------------------------+


                               proc-vmstat.nr_free_pages

  3.6e+07 ++----------------------------------------------------------------+
          |               *             *                                   |
  3.4e+07 ** **.**  * **.* ** ** *.*** * **. *   **                         |
  3.2e+07 ++*     ** *       *  *     *     * ***                           |
          |                                                                 |
    3e+07 ++                                                                |
  2.8e+07 ++                                                                |
          |                                                                 |
  2.6e+07 ++                                                                |
  2.4e+07 ++                                                                |
          |                                                                 |
  2.2e+07 ++                                          O                     |
    2e+07 ++     O         O     O  O O O   O O   O         O               |
          OOO O O OOOOOO OO OOOOO  O O O OO  O OOO O O OOOOO O OOOOOOOO OOOOO
  1.8e+07 ++-O--------------------------------------------------------------+


                             proc-vmstat.nr_inactive_file

  1.12e+08 ++---------------------------------------------------------------+
           OOOOO O OO  OO O   OO   O    O O  O O    O  OOO O     O O OO   OOO
   1.1e+08 ++     O  OO  O  OO  OOO OO O O OO O O OO O    O  OOOO O O   OO  |
  1.08e+08 ++                                         O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 ++*      * *       *  *    .*      * *.                          |
   9.6e+07 ** **. ** * *** .** ** ****  * **** *  **                        |
           |     *        *              *                                  |
   9.4e+07 ++---------------------------------------------------------------+


                              proc-vmstat.nr_anon_pages

  12800 ++------------------------------------------------------------O-----+
  12600 ++   O       O                             O    O          OO    O  |
        |     O       O  O  O O O  OO  O     O      OO    OO O            O O
  12400 ++OO   O    O   O O  O   O    O  O  O  O O    OO    O OO     O O   O|
  12200 OO      OO     O       O                O               O           |
        |         O                  O  O  O  O                   O         |
  12000 ++                                                                  |
  11800 ++                                                                  |
  11600 ++                                                                  |
        |                                                                   |
  11400 +*                                    *                             |
  11200 ++:*  * *  .*         *  *.*     *.*  ::                            |
        * * + :* :*  :****   * : :  **  *   :* *                            |
  11000 ++   *   *   *    *.*  **     **    *   **                          |
  10800 ++------------------------------------------------------------------+


                               proc-vmstat.nr_mapped

  6000 *+-----------------------*-------------------------------------------+
       |                       ::  *      *      *                          |
  5500 ++*   *        *        : : :   *  :**    :                          |
       |::+  : *      :*.*    *  ** :  : *  :    :                          |
       |*  ** ::*.   *   :  **      * : ::   : * :                          |
  5000 ++     :   ***     :*         :: *    : : :                          |
       |      *           ::         *        :::                           |
  4500 ++                 *                   * :                           |
       |                                        :                           |
  4000 +O            O O         O              *       O   O      O        |
       |    O OOO  OO O  O        OO O OO  OO O    O O       OOO O   O  O OOO
       | O O O    O       OO    O   O     O     OOO   OO        O   O  O    |
  3500 O+                   OOO          O     O         OO           O     |
       |                                                                    |
  3000 ++-------------------------------------------------------------------+


                               proc-vmstat.nr_file_pages

  1.12e+08 +O-O-------------------------------------------------------------+
           O O O O OOOOOOOO OOOOOOOO O OO OOOO OO O OO OOOOO  OOOOOOOOO OOOOO
   1.1e+08 ++     O                 O    O    O    O         O              |
  1.08e+08 ++                                         O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 +**     ** *     * ****    .**   * ***.                          |
   9.6e+07 *+ **.**  * *** + *    ****   :** *    **                        |
           |              *              *                                  |
   9.4e+07 ++---------------------------------------------------------------+


                          proc-vmstat.nr_page_table_pages

  2500 ++-------------------------------------------------------------------+
       |                               O    O                      O      O O
  2400 +O    O               O    O               O  O OO O   OO    O O O   |
       O O OO OO   OOOOO OOOO   O  O    O     O          O  OO       O O   O|
       |        O O           O  O  OO   OOO   OOO O  O         OO          |
  2300 ++                                                                   |
       |                                                                    |
  2200 ++                                                                   |
       |                                                                    |
  2100 ++                                                                   |
       |*   ****   *        *      *        *.**                            |
       |:*.*    *. :** *.   :**   * :*.** * :   *                           |
  2000 *+         *   *  **:   + *  *    * *     *                          |
       |                   *    *                                           |
  1900 ++-------------------------------------------------------------------+


                           proc-vmstat.nr_isolated_file

  2300 ++-------------------------------------------------------------------+
  2200 ++                    *            *                                 |
       |   *   *             :*.          :                                 |
  2100 +* : * * *.**** *  * *   *         : *                               |
  2000 ++::  ::       * + :::    : * *   : ::+ *                            |
       | *   *           * *     * :* +  : :  * *                           |
  1900 *+                         *    * * *     *                          |
  1800 ++   O                           *                           O       |
  1700 ++            O                            O                         |
       O                    O      O                 O             O        |
  1600 ++           O      O      O                    O       O      O   O O
  1500 ++                O   O                     O    O O            O   O|
       | O O  OO      OO        O                     O  O  OO   O      O   |
  1400 +O       O O       O   O  O   O OOO OO OOOO            O O    O      |
  1300 ++----O-----O----------------O-----O---------------------------------+


                                 proc-vmstat.numa_hit

  1.05e+09 ++---------------------------------------------------------------+
           |****.** **** *   ** * * **. *********.**                        |
           *       *    * *.*  * * *   *                                    |
     1e+09 ++                                                               |
           |                                                                |
           |                                                                |
   9.5e+08 ++                                                               |
           |                                                                |
     9e+08 ++                                                               |
           |                                                                |
           |                                                                |
   8.5e+08 ++O O  OO OO   O OO OOO  O   OOO  OOO  O OOOO O O   OOO OOO      O
           OO O  O  O  OOO    O   OO O O   OO   O  O    O O  OO   O   O OOOO|
           |                                                                |
     8e+08 ++---------------------------------------------------------------+


                                 proc-vmstat.numa_miss

  1.8e+08 ++----------------------------------------------------------------+
          |                                                                 |
  1.6e+08 +OO   OOOOOOOO OOOO OOOO O OOOO O OOO OOOO OOO  OOOO  OO          |
          O  OO              O      O    O     O        OO     O            |
  1.4e+08 ++                                                      OOOOO OOOO|
          |                                                                 O
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    8e+07 ++                                                                |
          |                                                                 |
    6e+07 ++           *. *                                                 |
          *****. ******  * *******.********.*******                         |
    4e+07 ++----*-----------------------------------------------------------+


                               proc-vmstat.numa_foreign

  1.8e+08 ++----------------------------------------------------------------+
          |                                                                 |
  1.6e+08 +OO   OOOOOOOO OOOO OOOO O OOOO O OOO OOOO OOO  OOOO  OO          |
          O  OO              O      O    O     O        OO     O            |
  1.4e+08 ++                                                      OOOOO OOOO|
          |                                                                 O
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    8e+07 ++                                                                |
          |                                                                 |
    6e+07 ++           *. *                                                 |
          *****. ******  * *******.********.*******                         |
    4e+07 ++----*-----------------------------------------------------------+


                                proc-vmstat.numa_local

  1.05e+09 ++---------------------------------------------------------------+
           |****.** ** * *   ** * * **. *********.**                        |
           *       *  * * *.*  * * *   *                                    |
     1e+09 ++                                                               |
           |                                                                |
           |                                                                |
   9.5e+08 ++                                                               |
           |                                                                |
     9e+08 ++                                                               |
           |                                                                |
           |                                                                |
   8.5e+08 ++O O  OO OO   O OO OOO  O   OOO  OOO  O OOOO O O   OOO OOO      O
           OO O  O  O  OOO    O   OO O O   OO   O  O    O O  OO   O   O OOOO|
           |                                                                |
     8e+08 ++---------------------------------------------------------------+


                                proc-vmstat.numa_other

  1.8e+08 ++----------------------------------------------------------------+
          |                                                                 |
  1.6e+08 +OO   OOOOOOOO OOOO OOOO O OOOO O OOO OOOO OOO  OOOO  OO          |
          O  OO              O      O    O     O        OO     O            |
  1.4e+08 ++                                                      OOOOO OOOO|
          |                                                                 O
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    8e+07 ++                                                                |
          |                                                                 |
    6e+07 ++           *. *                                                 |
          *****. ******  * *******.********.*******                         |
    4e+07 ++----*-----------------------------------------------------------+


                          proc-vmstat.workingset_nodereclaim

    3e+06 ++----------------------------------------------------------------+
          *  **.*     **.  **           *                                   |
  2.8e+06 +**    *****   **  *****.***** **.*******                         |
  2.6e+06 ++                                                                |
          |                                                                 |
  2.4e+06 ++                                                                |
  2.2e+06 ++                                                                |
          |                                                                 |
    2e+06 ++                                                                |
  1.8e+06 ++                                                                |
          | O     O  O                  O     O    O  OO       OO           |
  1.6e+06 ++  O OO  O OO OOOOOOOOO  OO O OO  O OOOO  O  OOOOOO   O          |
  1.4e+06 OO O     O               O  O     O                     OOOO  OOO O
          |                                                           O    O|
  1.2e+06 ++----------------------------------------------------------------+


                               proc-vmstat.pgalloc_dma32

    3e+06 ++----------------------------------------------------------------+
          **** .***  ***.* ** ** *.********. ******                         |
  2.9e+06 ++  *    **     *  *  *           *                               |
          |                                                                 |
  2.8e+06 ++                                                                |
  2.7e+06 ++                                                                |
          |                                                                 |
  2.6e+06 ++                                                       OO     O |
          |                                                       O  O  OO  O
  2.5e+06 ++                                                          O    O|
  2.4e+06 ++                                                                |
          |                         O                          O            |
  2.3e+06 OOOOO OOO OO   O  OOOO     O OOOO  OOOOOOO OOOOOOOO   OO          |
          |        O  OO  OO    OO O  O     O                O              |
  2.2e+06 ++----------------------------------------------------------------+


                                  proc-vmstat.pgfault

  1.8e+06 ++----------------------------------------------------------------+
          |      O   O     OO       OO  O O   O O O  OO     OO              |
  1.7e+06 OOOO     O  O  O    OOO                O      O OO   O OO O O    OO
  1.6e+06 ++    O O                O                   O                O O |
          |         O  O  O  O   O    OO O  OO O   O     O      O  O O   O  |
  1.5e+06 ++  O                                                             |
  1.4e+06 ++                                                                |
          |                                                                 |
  1.3e+06 ++                                                                |
  1.2e+06 ++                      .**     *.  * * *                         |
          *  *   * **     * *    *   *  * : * :* ::                         |
  1.1e+06 +** *. :* : **.* * :  *     :* *   *   *                          |
    1e+06 ++    *    ::      * *      *                                     |
          |          *        *                                             |
   900000 ++----------------------------------------------------------------+


                          proc-vmstat.pgsteal_kswapd_dma32

  600000 ++-----------------------------------------------------------------+
         |                                                                O |
  500000 ++                                                       O OO O O  |
         |                                                         O       O|
         |                                                            O     |
  400000 ++                                                                 O
         |                                                                  |
  300000 ++                                                                 |
         |     O   O       O            O    O       OO     O O O           |
  200000 OOOO O OOO O  OOOO OO OOOOOOO O OOOO O OOOOO   O OO O   O          |
         |           O                                   O                  |
         |                                                                  |
  100000 ++           .*  *       *     *   *                               |
         ****.********  ** ***.*** ***.* *** **.***                         |
       0 ++-----------------------------------------------------------------+


                           proc-vmstat.pgsteal_kswapd_normal

  2.2e+08 ++----------------------------------------------------------------+
    2e+08 ++O   O O       OO  O OO   O    O     O  O      OO O  OO          |
          |O  O  O  OOOO O  OO O   O   OOO   OOO O   OOO       O            |
  1.8e+08 O+ O     O                O O     O     O     OO  O     O OO  O OO|
  1.6e+08 ++                                                       O  O  O  O
  1.4e+08 ++                                                                |
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
    8e+07 ++                                                                |
    6e+07 ++                                                                |
    4e+07 ++                                                                |
          |                                                                 |
    2e+07 *****.********.*********.********.*******                         |
        0 ++----------------------------------------------------------------+


                           proc-vmstat.pgsteal_direct_dma32

  2.6e+06 ++----------------------------------------------------------------+
  2.5e+06 *****.********.*********.********.*******                         |
          |                                                                 |
  2.4e+06 ++                                                                |
  2.3e+06 ++                                                                |
          |                                                                 |
  2.2e+06 ++                                                                |
  2.1e+06 ++                                                                |
    2e+06 ++                                                                |
          |                                                                 |
  1.9e+06 ++                                                                |
  1.8e+06 ++                        O    O     O                   O        |
          OOOOO OOOOOOOO OOOOOOOOO O OOOO O OOO OOOO OOOOOOOO  OOO  OOO OOO O
  1.7e+06 ++                                                 O    O        O|
  1.6e+06 ++----------------------------------------------------------------+


                           proc-vmstat.pgsteal_direct_normal

  9.5e+08 ++----------------------------------------------------------------+
          *****.********.*********.********.*******                         |
    9e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
  8.5e+08 ++                                                                |
          |                                                                 |
    8e+08 ++                                                                |
          |                                                                 |
  7.5e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
    7e+08 O+O O      O      O       O   OO    OO OOO OOOOO     O   OO O     O
          |O O  OOOOO OO OOO OOOOO O OOO  O OO  O         OOOO  OOO  O  OOOO|
  6.5e+08 ++----------------------------------------------------------------+


                           proc-vmstat.pgscan_kswapd_dma32

  600000 ++-----------------------------------------------------------------+
         |                                                                O |
  500000 ++                                                       O OO O O  |
         |                                                         O       O|
         |                                                            O     |
  400000 ++                                                                 O
         |                                                                  |
  300000 ++                                                                 |
         |     O   O       O            O    O       OO     O O O           |
  200000 OOOO O OOO O  OOOO OO OOOOOOO O OOOO O OOOOO   O OO O   O          |
         |           O                                   O                  |
         |                                                                  |
  100000 ++           .*  *       *     *   *                               |
         ****.********  ** ***.*** ***.* *** **.***                         |
       0 ++-----------------------------------------------------------------+


                           proc-vmstat.pgscan_kswapd_normal

  2.2e+08 ++----------------------------------------------------------------+
    2e+08 ++O   O O       OO  O OO   O    O     O  O      OO O  OO          |
          |O  O  O  OOOO O  OO O   O   OOO   OOO O   OOO       O            |
  1.8e+08 O+ O     O                O O     O     O     OO  O     O OO  O OO|
  1.6e+08 ++                                                       O  O  O  O
  1.4e+08 ++                                                                |
  1.2e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
    8e+07 ++                                                                |
    6e+07 ++                                                                |
    4e+07 ++                                                                |
          |                                                                 |
    2e+07 *****.********.*********.********.*******                         |
        0 ++----------------------------------------------------------------+


                            proc-vmstat.pgscan_direct_dma32

  2.6e+06 ++----------------------------------------------------------------+
  2.5e+06 *****.********.*********.********.*******                         |
          |                                                                 |
  2.4e+06 ++                                                                |
  2.3e+06 ++                                                                |
          |                                                                 |
  2.2e+06 ++                                                                |
  2.1e+06 ++                                                                |
    2e+06 ++                                                                |
          |                                                                 |
  1.9e+06 ++                                                                |
  1.8e+06 ++                        O    O     O                   O        |
          OOOOO OOOOOOOO OOOOOOOOO O OOOO O OOO OOOO OOOOOOOO  OOO  OOO OOO O
  1.7e+06 ++                                                 O    O        O|
  1.6e+06 ++----------------------------------------------------------------+


                           proc-vmstat.pgscan_direct_normal

  9.5e+08 ++----------------------------------------------------------------+
          *****.********.*********.********.*******                         |
    9e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
  8.5e+08 ++                                                                |
          |                                                                 |
    8e+08 ++                                                                |
          |                                                                 |
  7.5e+08 ++                                                                |
          |                                                                 |
          |                                                                 |
    7e+08 O+O O      O      O       O   OO    OO OOO OOOOO     O   OO O     O
          |O O  OOOOO OO OOO OOOOO O OOO  O OO  O         OOOO  OOO  O  OOOO|
  6.5e+08 ++----------------------------------------------------------------+


                        proc-vmstat.pgscan_direct_throttle

  2500 ++-------------------------------------------------------------------+
       |                                       *                            |
       |                                      *:                            |
  2000 +* .** * *.    *      * .*       * *  +  :                           |
       * *   * :: * ** *.**  :* : *     :*: *   :*                          |
       |       *   *       **    :::**.*   ::   *                           |
  1500 ++                        * *       :                                |
       |                                   *                                |
  1000 ++                                                                   O
       |                                                           O        |
       |                                                         O  OOOO  OO|
   500 ++  O    O      O   OOOO   O OO  O OO  O  O          O           O   |
       OOO  OOOO  OOOOO  OO      O     O O  O  OO OO OOOOOO  OO O           |
       |                        O  O                           O            |
     0 ++-------------------------------------------------------------------+


                               proc-vmstat.slabs_scanned

    7e+06 +*--------------------------*------*------------------------------+
          * ***.********.*********.*** ****.* *****                         |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
          |                                                                 |
  5.5e+06 ++                                                                |
          |                                                                 |
    5e+06 ++                                                                |
          | O     O  O                                 O                    |
  4.5e+06 +O  O OO  O O   OOO OOOO   O OOOO  OOOOO O OO  OOO O OOO          |
          O  O     O   O O   O     OO O     O     O     O   O      OOO      O
    4e+06 ++                                                      O   O OOOO|
          |                                                                 |
  3.5e+06 ++----------------------------------------------------------------+


                     proc-vmstat.kswapd_high_wmark_hit_quickly

  550 ++--------------------------------------------------------------------+
  500 ++            O  O                                     O          O   |
      |OO  OOOO      O  OOO      O  OOO   OOO    O  OO   O     O O  O  O   O|
  450 O+  O      OOO       O OOOO O    OO    OO O OO   OO OO  O O  O OO   O |
  400 ++       O                                                            O
  350 ++                                                                    |
  300 ++                                                                    |
      |                                                                     |
  250 ++                                                                    |
  200 ++                                                                    |
  150 ++                                                                    |
  100 ++                                                                    |
      |      *      * .*   *.   *                                           |
   50 ***.*** **.*** *  ***  *** **.*****.*****.**                          |
    0 ++--------------------------------------------------------------------+


                              proc-vmstat.pageoutrun

  2200 ++----------------------------------------------------O--------------+
       O O            O                              O           O      O   |
  2100 +O     O   OO O     O O   O     O    O            O     O    O  O   O|
       |   OOO O    O       O     O  O   O O   OO             O O    O    O O
  2000 ++              O O    O    OO     O      O O  O   O O         O     |
       |                  O     O       O     O   O    OO          O        |
  1900 ++       O                                                           |
       |                                                                    |
  1800 ++                                                                   |
       |           *                        *                               |
  1700 *+          : *  .*  *    *  **  *  * +  *                           |
       |**.***    : :::*  ** :*.* **  + :* :  * :*                          |
  1600 ++     ***.* * *      *         *  *    *                            |
       |                                                                    |
  1500 ++-------------------------------------------------------------------+


                               proc-vmstat.allocstall

  29000 ++------------------------------------------------------------------+
        ****.******.*******.******.*******.*******                          |
  28000 ++                                                                  |
  27000 ++                                                                  |
        |                                                                   |
  26000 ++                                                                  |
  25000 ++                                                                  |
        |                                                                   |
  24000 ++                                                                  |
  23000 ++                                                                  |
        |                                                                   |
  22000 ++                                                                  |
  21000 ++                                                         O        O
        O OO O  O O OOOOO O  OO    O  OOO   OOO OO OOOOOO  OO O     OOOO OO |
  20000 +O----OO-O-------O--O--OOO--OO---O-O---O----------O--O-OO-O--------O+


                            proc-vmstat.numa_pte_updates

  16000 ++------------------------------------------------------------------+
  15000 ++      O           O                                             O O
        |  O     O    O        O   O   O     O   O                 OO O     |
  14000 O+                O           O     O      OO       O O   O  O O   O|
  13000 +O   OO   O OO  OO      OO  OO  O  O  O O    O O   O   O         O  |
        |      O       O     O           O     O        O O  O  O           |
  12000 ++O           *       O                       O                     |
  11000 ++            ::                      *  *                          |
  10000 ++        *. : ** *. * * *.*** ** .* * **                           |
        |       **  **   *  * * *     *  *  *                               |
   9000 ++      :                                                           |
   8000 ++      :                                                           |
        |** .* :                                                            |
   7000 *+ *  **                                                            |
   6000 ++------------------------------------------------------------------+


                            proc-vmstat.numa_hint_faults

  18000 ++------------------------------------------------------------------+
        |                          O                                        |
  16000 ++                                                                  |
        |                                                                   |
        |                   O                                               |
  14000 ++       O    O   O                                               O O
        |       O                                                 OOO O     |
  12000 ++ O  O      O  O       O    O OOO O OOOOO OOO      O O      O O O O|
        OOO  O O  O O  O O   OOO O  O O     O         OOO OO O OO           |
  10000 ++                                    ** *                          |
        |        **.*******.******.*******.***  *                           |
        |       *                                                           |
   8000 ++      :                                                           |
        |** .* *                                                            |
   6000 *+-*--*-------------------------------------------------------------+


                         proc-vmstat.numa_hint_faults_local

  15000 ++------------------------------------------------------------------+
  14000 ++                         O                                        |
        |                   O                                               |
  13000 ++       O    O   O                                           O   O O
  12000 ++ O    O                                O            O   OOO  O    |
        O     O     OO  O      OOO   OOOOO OOOOOO  OOOO O O O   O    O   O O|
  11000 +OO  O O  O    O O   OO     O                  O   O O O            |
  10000 ++                                    ** *                          |
   9000 ++       **. **** *. * * *.*** ** .***  *                           |
        |       *   *    *  * * *     *  *                                  |
   8000 ++      :                                                           |
   7000 ++      :                                                           |
        |      :                                                            |
   6000 ****.***                                                            |
   5000 ++------------------------------------------------------------------+


                                    meminfo.MemFree

  1.4e+08 ++--------------*-------------------------------------------------+
          *  **.**    **. ::*  *       ***       **                         |
  1.3e+08 +**     ****   * * ** **.****   *.*****                           |
          |                                                                 |
  1.2e+08 ++                                                                |
          |                                                                 |
  1.1e+08 ++                                                                |
          |                                                                 |
    1e+08 ++                                                                |
          |                                                                 |
    9e+07 ++                                                                |
          |                                           O                     |
    8e+07 ++    OOOOOO O OOOOO OOO OOOOOOOO OOOOOOOO O O OOOOO OOOOOO   OOOO|
          OOOOO       O       O                         O            OO     O
    7e+07 ++----------------------------------------------------------------+


                                    meminfo.Cached

  4.5e+08 ++----------------------------------------------------------------+
          OOOOO       O       O                         O            OO     O
  4.4e+08 ++    OOOOOO O OOOOO OOO OOOOOOOO OOOOOOOO O O OOOOO OOOOOO   OOOO|
          |                                           O                     |
  4.3e+08 ++                                                                |
          |                                                                 |
  4.2e+08 ++                                                                |
          |                                                                 |
  4.1e+08 ++                                                                |
          |                                                                 |
    4e+08 ++                                                                |
          | *                                                               |
  3.9e+08 +* :    ****  .* **** **.*****  *.******                          |
          *  **.**    **  ::   *        **        *                         |
  3.8e+08 ++--------------*-------------------------------------------------+


                                   meminfo.Inactive

  4.5e+08 ++----------------------------------------------------------------+
          OOOOO       O                                 O            OO     |
  4.4e+08 ++    OOOOOO O OOOOOOOOO OOOOOOOO OOOOOOOO O O OOOOO OOOOOO   OOOOO
          |                                           O                     |
  4.3e+08 ++                                                                |
          |                                                                 |
  4.2e+08 ++                                                                |
          |                                                                 |
  4.1e+08 ++                                                                |
          |                                                                 |
    4e+08 ++                                                                |
          |                                                                 |
  3.9e+08 +**     ** *     * ** **.****    .* ***                           |
          *  **. *  * **.* :*  *       ****  *   **                         |
  3.8e+08 ++----*---------*-------------------------------------------------+


                                meminfo.Inactive(file)

  4.5e+08 ++----------------------------------------------------------------+
          OOOOO       O                                 O            O      |
  4.4e+08 ++    OOOOOO O OOOOOOOOO OOOOOOOO OOOOOOOO O O OOOOO OOOOOO O OOOOO
          |                                           O                     |
  4.3e+08 ++                                                                |
          |                                                                 |
  4.2e+08 ++                                                                |
          |                                                                 |
  4.1e+08 ++                                                                |
          |                                                                 |
    4e+08 ++                                                                |
          |                                                                 |
  3.9e+08 ++*     ** *       ** **.** *    .* ***                           |
          ** **. *  * **.* **  *     * ****  *   **                         |
  3.8e+08 ++----*---------*-------------------------------------------------+


                                  meminfo.AnonPages

  52000 ++------------------------------------------------------------------+
        |                                                             O     |
  51000 ++   O       O                             O    O          OO    O  |
  50000 ++    O     O O  O  O O O  OO  O     O      OO    OO O            O O
        OOOO   O        O O  O   O    O  O  O  OOO    OO    O OO     O O   O|
  49000 ++      OOO    O       O        O     O                 O           |
  48000 ++                           O     O                      O         |
        |                                                                   |
  47000 ++                                                                  |
  46000 ++                                                                  |
        |*                                    *                             |
  45000 *+:*.   *  .*         *  *.*    **.*  ::                            |
  44000 ++*  *** :*  :****   * * :  **  :   ** **                           |
        |        *   *    *.*   *     **         *                          |
  43000 ++------------------------------------------------------------------+


                                   meminfo.Mapped

  24000 ++-------------------*----------------------------------------------+
        |                    : *                 *                          |
  22000 ++     **     *    .*: :       *    *    :                          |
        |*     : :  ** *  *  : ::*  * *: *  :: * :                          |
        |:**.* : *  :   : :  : :* + :*  ::+: ** :                           |
  20000 *+    :  : :    *:    ::   *    :  *    *                           |
        |     *   ::     *    :         *                                   |
  18000 ++        *           :                                             |
        |                     :                                             |
  16000 ++O    O    O     O   *         O     O  O          O  O   OO       O
        OO    O O    O O         O OO O     O      OOOOO     OO      OO     |
        |  O     OO   O OO  O        O     O    O         O     O O    O OOO|
  14000 ++   O               OO O      O O   O O        O  O                |
        |                      O                                            |
  12000 ++------------------------------------------------------------------+


                                 meminfo.PageTables

  10000 ++------------------------------------------------------------------+
        |                              O     O       O             O      O O
        |OO   O      O    O   O O  O               O   OO  O  OO    OOO  O  |
   9500 O+ O O OO   O OOOO  OO      O   OO    OOO   O O   O OO         O   O|
        |        OO            O O   OO    OO    O              O O         |
        |                                                                   |
   9000 ++                                                                  |
        |                                                                   |
   8500 ++                                                                  |
        |                    *                *                             |
        |***.****  .*   *    ::*  .**   *    * *                            |
   8000 *+       **  ** :*  : * :*   :** *.* :  *                           |
        |              *  *.*   *    *      *    *                          |
        |                                                                   |
   7500 ++------------------------------------------------------------------+


                              numa-meminfo.node0.MemFree

  3.8e+07 ++----------------------------------------------------------------+
  3.6e+07 *+          *    **           *    *                              |
          |* **.**    :   *  : *  .*** * :*.* : **                          |
  3.4e+07 ++*     ***: :.*   * : *    :: *    * : *                         |
  3.2e+07 ++         * *      * ::    *        *                            |
          |                     *                                           |
    3e+07 ++                                                                |
  2.8e+07 ++                                                                |
  2.6e+07 ++                                                                |
          |                                                                 |
  2.4e+07 ++                                                                |
  2.2e+07 ++             O                            O   O                 |
          |     OOOOOOOO  O OOO OO OO OOO O OOO OO O O OO   OO OOOOOO    OO |
    2e+07 OOOOO            O   O     O   O     O  O      O O         OO O  OO
  1.8e+07 ++----------------------------------------------------------------+


                              numa-meminfo.node0.MemUsed

  1.14e+08 ++---------------------------------------------------------------+
  1.12e+08 +OOOO            O        O             O     O O         OO O  OO
           O     OOOOOOOO O  OOOOOOOO  OOOOOOOOOO O OO OO    OOO OOOO    OO |
   1.1e+08 ++            O                            O   O     O           |
  1.08e+08 ++                                                               |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
           |                     *                                          |
     1e+08 ++         * *      * ::    *       *                            |
   9.8e+07 ++*     * *: :*    * : *   + : *   * :  *                        |
           |* :*.** *  :  *.  : *  ***  * :** : *.*                         |
   9.6e+07 *+ *        *    **           *   *                              |
   9.4e+07 ++---------------------------------------------------------------+


                              numa-meminfo.node0.Inactive

  1.12e+08 ++---------------------------------------------------------------+
   1.1e+08 +OOOO            O        O                   O O               O|
           O     O OOOOOO O  OOOOOOOO  OO OOOO OO OOOO OO    OOO OOOOOO OOO O
  1.08e+08 ++     O      O               O    O       O   O     O           |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 ++           *      * *     *                                    |
   9.6e+07 ++*     * ** :*    * ::*   + :     **   *                        |
           |* :*.** *  :  *.  : *  ***  * *** : *.*                         |
   9.4e+07 *+ *        *    **           *   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                           numa-meminfo.node0.Inactive(file)

  1.12e+08 ++---------------------------------------------------------------+
   1.1e+08 +OOOO            O        O                   O O               O|
           O     O OOOOOO O  OO OOOOO  OO OOOO OO OOOO OO    OOO OOOOOO OOO O
  1.08e+08 ++     O      O     O         O    O       O   O     O           |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 ++           *      * *     *                                    |
   9.6e+07 ++*     * ** :*    * ::*   + :     **   *                        |
           |* :*.** *  :  *.  : *  ***  * *** : *.*                         |
   9.4e+07 *+ *        *    **           *   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                             numa-meminfo.node0.FilePages

  1.12e+08 ++---------------------------------------------------------------+
   1.1e+08 +OOOO            O        O             O     O O         OO O  OO
           O     OOOOOOOO O  OOOOOOOO  OOOOOOOOOO O OO OO    OOO OOOO    OO |
  1.08e+08 ++            O                            O   O     O           |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                                                                |
   9.8e+07 ++         * *      * *     *       *                            |
   9.6e+07 ++*     ***: :*    * ::*   + : *   * :  *                        |
           |* :*.**    :  *.  : *  ***  * :** : *.*                         |
   9.4e+07 *+ *        *    **           *   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                           numa-meminfo.node0.PageTables

  3000 ++-----------------------------------------------------------O-------+
       |                                                           O        |
  2800 ++            O                                    O     O           |
  2600 ++       O   O     O                     O       O     O            O|
       |   OO          O     O   O                   O O                    O
  2400 ++O   OO   O  *O    O  O   OO O    *O    *     O      O   O    O O O |
       O           O :   O              O :*  O :OO      O                  |
  2200 ++   *      : :          O   O  O*OO:O  O:: O           O     O      |
       *O   :*    : : :  *  O*    * *   :   :   :*          O          O    |
  2000 ++  :  *O  * * : + * * *   :::  :    *. :                            |
  1800 +**.*   :  :   **   ::  + * * :.*      *:                            |
       |        ::         *    *    *         *                            |
  1600 ++       ::                                                          |
       |        *                                                           |
  1400 ++-------------------------------------------------------------------+


                              numa-meminfo.node1.MemFree

  3.8e+07 ++----------------------------------------------------------------+
  3.6e+07 ++     *      .*                   *                              |
          |      :*    *  *          *       ::                             |
  3.4e+07 ++*   * : * *   : ***     * :  *   :*  **                         |
  3.2e+07 +* * +   ::*     ::  ***.*  * * *.:  **                           |
          *   *    *       *           *    *                               |
    3e+07 ++                                                                |
  2.8e+07 ++                                                                |
  2.6e+07 ++                                                                |
          |                                                                 |
  2.4e+07 ++                                                                |
  2.2e+07 ++                                          O                     |
          |      O OO      OO  OOO O O  O O   O OOOO O O OO OO OO   O    O  O
    2e+07 OOOOO O O  OOO OO  OO     O OO O  OO O        O        OOO OO O OO|
  1.8e+07 ++-----------------------------------------------O----------------+


                              numa-meminfo.node1.MemUsed

  1.14e+08 O+-O--------------------------------------------O----------------+
  1.12e+08 +OO O O O  OOOOO   OO    O  OOOO OOOO     O  O        OOOOOO O OO|
           |      O OO      OO  OOOO O     O    O OOO  O OO  OOOO        O  O
   1.1e+08 ++                                         O                     |
  1.08e+08 ++                                                               |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
           *   *    *       *      *    *   *                               |
     1e+08 +* * +   ::*    : : **** :  * * *:  **.                          |
   9.8e+07 ++*   * : * *   : **     * +   *  :*   **                        |
           |      :*    * *          *       *                              |
   9.6e+07 ++     *      *                                                  |
   9.4e+07 ++---------------------------------------------------------------+


                              numa-meminfo.node1.Inactive

  1.12e+08 ++----------------------------------------------O----------------+
   1.1e+08 OOOOO   O  OOOOO   OO       OO   OO          O        OOO OO   O |
           |     OO OO      OO  OOOOOO   OOO  OOO OOOO O OO  OOOO   O   OO OO
  1.08e+08 ++                                         O                     |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |        *       *                                               |
   9.8e+07 ** **.   ::*    : :  ****    ** **  *                            |
   9.6e+07 ++::  * : * *   : ***    : .*  * : * *.**                        |
           | *    :*    * *         **       ::                             |
   9.4e+07 ++     *      *                   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                           numa-meminfo.node1.Inactive(file)

  1.12e+08 ++----------------------------------------------O----------------+
   1.1e+08 OOOOO   O  O O O   OO        O    O                   OO   O   O |
           |     OO OO O O  OO  OOOOOO O OOOO OOO OOOO OOOO  OOOO  OOO  OO OO
  1.08e+08 ++                                         O                     |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |        *       *                                               |
   9.8e+07 *+  *.   ::*    : :  * **    ** **  *                            |
   9.6e+07 +* *  * : * :   : * * *  : .*  * :  :*.**                        |
           | *    :*   ** *   *     **       :*                             |
   9.4e+07 ++     *      *                   *                              |
   9.2e+07 ++---------------------------------------------------------------+


                             numa-meminfo.node1.FilePages

  1.12e+08 O+----------------------------------------------O----------------+
   1.1e+08 +OOOO O O  OOOOO   OO    O  OO O OO O     O  O        OOOOOO O OO|
           |      O OO      OO  OOOO O   O O  O O OOO  O OO  OOOO        O  O
  1.08e+08 ++                                         O                     |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           *   *    *       *           *   *                               |
   9.8e+07 +* * +   ::*    : : *****   * * *:  **.                          |
   9.6e+07 ++*   * : * *   : **     * +   *  :*   **                        |
           |      :*    * *          *       *                              |
   9.4e+07 ++     *      *                                                  |
   9.2e+07 ++---------------------------------------------------------------+


                              numa-meminfo.node2.MemFree

  3.6e+07 ++--------------------*-------*-----------------------------------+
          |               *   * ::      :      *                            |
  3.4e+07 *+  *.*   *** .* *  : :*.*   : :     :                            |
  3.2e+07 +*  :  :**   *    ** :    * ** **.  : :**                         |
          | :*   *             *     *      *** *                           |
    3e+07 ++*                                                               |
  2.8e+07 ++                                                                |
          |                                                                 |
  2.6e+07 ++                                                                |
  2.4e+07 ++                                                                |
          |                                                                 |
  2.2e+07 ++                                          O                     |
    2e+07 ++    OOO    O       O    OOO OO  O OO O O       OO    O  O   OO  |
          OOO O    OOOO  O OOO  OO O   O  O  O  O O  O O OO  O OO OO OO   OOO
  1.8e+07 ++-O------------O---O-------------------------O-------------------+


                              numa-meminfo.node2.MemUsed

  1.16e+08 ++---------------------------------------------------------------+
  1.14e+08 ++ O           O    O                                            |
           OOO O     OOO O  O    O O    O    O     O O OO O   OO  O  O      O
  1.12e+08 ++    O OO   O    OO O O OO O OOOO OOO O O    O O O  OO OO O OOOO|
   1.1e+08 ++     O                                   O                     |
           |                                                                |
  1.08e+08 ++                                                               |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           | *                                                              |
  1.02e+08 ++::                 *    *      *   *.                          |
     1e+08 +* *   ***   *     * :   * + * ** ** : *                         |
           *   :.*   *** * .** : :**   *: :    :   *                        |
   9.8e+07 ++  *          *    * ::      :     *                            |
   9.6e+07 ++--------------------*-------*----------------------------------+


                              numa-meminfo.node2.Inactive

  1.12e+08 ++-O-----------O----O--------------------------------------------+
           OOO O    OOOO      O  OOO    O  O O  O  O O  OOO   OO  OO OO   O O
   1.1e+08 ++    OOO    OO  OO  O   OO O OO O OO  O O  O   O O  OO  O   OO O|
  1.08e+08 ++                                         O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++*                                                              |
           | ::                 *    *.     *** *.                          |
   9.8e+07 +* *   ***    *   ** :   *  ** **  : : **                        |
   9.6e+07 *+  *.*   **** :.*  * :**    : :    :                            |
           |              *      ::      :     *                            |
   9.4e+07 ++--------------------*-------*----------------------------------+


                           numa-meminfo.node2.Inactive(file)

  1.12e+08 ++-O-----------O----O--------------------------------------------+
           OOO O    OOOO      O  O O       O    O  O O  OOO   OO  OO O    O O
   1.1e+08 ++    O O    OO  OO  O O OO OOOO OOOO  O O  O   O O  OO  O O OO O|
  1.08e+08 ++     O                                   O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++*                                                              |
           | ::                 *    *      *** *.                          |
   9.8e+07 +* *   ***        ** :   * + * **  : : **                        |
   9.6e+07 *+  *.*   * *** .*  * :**   *: :    :                            |
           |          *   *      ::      :     *                            |
   9.4e+07 ++--------------------*-------*----------------------------------+


                             numa-meminfo.node2.FilePages

  1.14e+08 ++---------------------------------------------------------------+
  1.12e+08 ++ O           O    O                                            |
           OOO O     OOO O  O    O O    O    O     O O OO O   OO  O  OO    OO
   1.1e+08 ++    O OO   O    OO O O OO O OOOO OOO O O    O O O  OO OO   OOO |
  1.08e+08 ++     O                                   O                     |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
           | *                                                              |
     1e+08 ++::                 *    *      *   *.                          |
   9.8e+07 +* *   ***   *    ** :   * + * ** ** : *                         |
           *   *.*   *** * .*  : :**   *: :    :   *                        |
   9.6e+07 ++             *    * ::      :     *                            |
   9.4e+07 ++--------------------*-------*----------------------------------+


                              numa-meminfo.node3.MemFree

  3.8e+07 ++----------------------------------------------------------------+
  3.6e+07 ++  *.*                        *                                  |
          |  *  :                       *:                                  |
  3.4e+07 *+ :   :     *.** *  * *.    *  :.*  * *                          |
  3.2e+07 +**    ******    * ** :: ** *   *  * :* *                         |
          |                     *    *        *                             |
    3e+07 ++                                                                |
  2.8e+07 ++                                                                |
  2.6e+07 ++                                                                |
          |                                                                 |
  2.4e+07 ++                                                                |
  2.2e+07 ++                                          O                     |
          |          O    OOOO  OO O  O OO  O OO  O  O     OO   O       O O |
    2e+07 OO OO OOOOO OO O    OO    OO O  O  O  OO O   OOOO  O O OOOOOO  O OO
  1.8e+07 ++O---------------------------------------------------------------+


                              numa-meminfo.node3.MemUsed

  1.14e+08 ++O-O------------------------------------------------------------+
  1.12e+08 OO O  OOOOO OOO    OOO   OO  O  O O  O O OO OOOO   OO OOOOOO OOOOO
           |          O   O OO   OOO   O OO O OO   O       O O  O           |
   1.1e+08 ++                                         O                     |
  1.08e+08 ++                                                               |
           |                                                                |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
           |                     *   *.       *                             |
     1e+08 +**    ******   .* ** ::**  *   * * **. *                        |
   9.8e+07 *+ :   :     ***  *  * *     *  :*     *                         |
           |  **.:                       *:                                 |
   9.6e+07 ++    *                        *                                 |
   9.4e+07 ++---------------------------------------------------------------+


                              numa-meminfo.node3.Inactive

  1.12e+08 ++O--------------------------------------------------------------+
   1.1e+08 OO OO OOOOO OOO     OO   OO  O  O O  O O O  OO O    O OOOOOO O OOO
           |          O   O OOO  OOO   O OO O OO   O O   O   OO O        O  |
  1.08e+08 ++                                         O    O                |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                     *   *.                                     |
   9.8e+07 +**    ******   .* *  ::**  *     ** *. *                        |
   9.6e+07 *+ :   :     ***  * ** *     *  **  *  *                         |
           |  **.:                       * :                                |
   9.4e+07 ++    *                        *                                 |
   9.2e+07 ++---------------------------------------------------------------+


                           numa-meminfo.node3.Inactive(file)

  1.12e+08 ++O--------------------------------------------------------------+
   1.1e+08 OO OO OOOOO OOO     OO   OO  O  O O  O O O  OO O    O OOOOOO O OOO
           |          O   O OOO  OOO   O OO O OO   O O   O   OO O        O  |
  1.08e+08 ++                                         O    O                |
  1.06e+08 ++                                                               |
           |                                                                |
  1.04e+08 ++                                                               |
  1.02e+08 ++                                                               |
     1e+08 ++                                                               |
           |                     *   *.                                     |
   9.8e+07 +**    ******   .* *  ::**  *     ** *. *                        |
   9.6e+07 *+ :   :     ***  * ** *     *  **  *  *                         |
           |  **.:                       * :                                |
   9.4e+07 ++    *                        *                                 |
   9.2e+07 ++---------------------------------------------------------------+


                             numa-meminfo.node3.FilePages

  1.14e+08 ++---------------------------------------------------------------+
  1.12e+08 ++O                                                              |
           OO OO OOOOO OOO     OO   OO  O  O O  O O O  OOOO   OO OOOOOO  O OO
   1.1e+08 ++         O   O OOO  OOO   O OO O OO   O O     O O  O       O O |
  1.08e+08 ++                                         O                     |
  1.06e+08 ++                                                               |
  1.04e+08 ++                                                               |
           |                                                                |
  1.02e+08 ++                                                               |
     1e+08 ++                    *                                          |
   9.8e+07 ++*    ** ***      *  ::* *.*     ** *. *                        |
   9.6e+07 ** :   : *   ***.** ** * *   *  **  *  *                         |
           |  *  :                       * :                                |
   9.4e+07 ++  *.*                        *                                 |
   9.2e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node0.nr_free_pages

  9.5e+06 ++----------------------------------------------------------------+
    9e+06 *+          *    *         *  *                                   |
          |* **.** *  :   * *  *  .**: * :*.**  ***                         |
  8.5e+06 ++*     * *: :  :  * ::*    :: *    * :                           |
    8e+06 ++         * *.*    * *     *        *                            |
          |                                                                 |
  7.5e+06 ++                                                                |
    7e+06 ++                                                                |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
  5.5e+06 ++             O                            O   O                 |
          |     OOOOOOOO  O OOOOOO OO OOO O OOO OO O O OO   OO OOOOOO    O  |
    5e+06 OOOOO            O         O   O     O  O      O O         OO O OOO
  4.5e+06 ++----------------------------------------------------------------+


                          numa-vmstat.node0.nr_inactive_file

   2.8e+07 ++---------------------------------------------------------------+
  2.75e+07 +OOOO            O                            O O                O
           O     OOOOOOOO O  OO OOOOOO OO OOOO OO OOOO OO    OOOOOOOOOO OOOO|
   2.7e+07 ++            O     O         O    O       O   O                 |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           |                                                                |
  2.45e+07 ++           *      * *     *       *                            |
   2.4e+07 ++*     * ** :*    *: :*   : :     * :                           |
           |* **.** *  :  *.  : :  ** : * * **  *.**                        |
  2.35e+07 *+          *    **  *    *   * *                                |
   2.3e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node0.nr_file_pages

   2.8e+07 ++---------------------------------------------------------------+
  2.75e+07 +OOOO            O                      O     O O         OO O  OO
           O     OOOOOOOO O  OOOOOOOOO OOOOOOOOOO O OO OO    OOOOOOOO    OO |
   2.7e+07 ++            O                            O   O                 |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           |                                                                |
  2.45e+07 ++         * *      * *     *       *                            |
   2.4e+07 ++*     * *: :*    * ::*   : : *   * :                           |
           |* **.** *  :  *. *  *  ** : * :***  *.**                        |
  2.35e+07 *+          *    *        *   *                                  |
   2.3e+07 ++---------------------------------------------------------------+


                       numa-vmstat.node0.nr_page_table_pages

  750 ++------------------------------------------------------------O-------+
      |                                                            O        |
  700 ++            O                                     O     O           |
  650 ++       O         O                      O       O     O            O|
      |   OO       O   O        O                   O  O                    O
  600 ++O   OO   O  *O    O  OO  OO  O    *O    *    O       O   O    O O   |
      O           O :   O              O : * O  :O       O                O |
  550 ++   *      : :          O    O O*O:O:O O: :OO           O     O      |
      *O   :*    : : :  *  O     *  *  :*   :  : *         O           O    |
  500 ++  :  :O  * * :  :* *.**  ::+: :     *  :                            |
  450 +**.*  *:  :   *.*  ::   :* *  :*      *:                             |
      |        ::         *    *     *        *                             |
  400 ++       ::                                                           |
      |        *                                                            |
  350 ++--------------------------------------------------------------------+


                              numa-vmstat.node0.numa_hit

    2e+08 ++----------------------------------------------------------------+
          |    .*     *                                                     |
  1.9e+08 *****  ***** *.*********.*** ****.**** **                         |
          |                           *         *                           |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++                                                                |
          |      OO  O   O    O O    O  OO    OO      OO  OOO      OO     O |
  1.4e+08 OOOOO O  OO OO  OOOO O O  O  O  O  O  OOOO O  OO   O OOOO  OO OO OO
          |                        O  O     O                               |
  1.3e+08 ++----------------------------------------------------------------+


                              numa-vmstat.node0.numa_miss

    3e+07 ++----------------------------------------------------------------+
  2.8e+07 ++         O                                       O  O           |
          |O    OOO O O   OO       OOO    O     OO O OOOO OO     O          |
  2.6e+07 ++OOO    O   O O    OOOO    OO O  OO O  O      O  O  O            |
  2.4e+07 O+                OO          O     O                     OO      |
  2.2e+07 ++                                                      O     OOOOO
    2e+07 ++                                                       O  O     |
          |                                                                 |
  1.8e+07 ++                                                                |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
  1.2e+07 *+      *    *.  *                 *                              |
          |***   * * * : ** *  ***.*  *  *   :*  *                          |
    1e+07 ++  *. :  * *      **     ** ** *.*  ** *                         |
    8e+06 ++----*-----------------------------------------------------------+


                            numa-vmstat.node0.numa_foreign

  2.6e+07 ++----------------------------------------------------------------+
  2.4e+07 ++    O O      O OO O OO   O  O O   O O          O O  O           |
          |OO    O OOOOO  O  O O   OO OO O  OO O OOO OOO  O O  O O          |
  2.2e+07 O+ OO                                         OO        OO  O  OOO|
    2e+07 ++                                                        OO  O   O
          |                                                                 |
  1.8e+07 ++                                                                |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
          |                                                                 |
  1.2e+07 ++                                                                |
    1e+07 ++              *                                                 |
          |        **  *.* *  *** .*     *                                  |
    8e+06 *****. **  **     **   *  ***** *.*******                         |
    6e+06 ++----*-----------------------------------------------------------+


                             numa-vmstat.node0.numa_local

    2e+08 ++----------------------------------------------------------------+
          |    .*     *                                                     |
  1.9e+08 *****  ***** *.*********.*** ****.**** **                         |
          |                           *         *                           |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++                                                                |
          |      OO  O   O    O O    O  OO    OO      OO  OOO      OO     O |
  1.4e+08 OOOOO O  OO OO  OOOO O O  O  O  O  O  OOOO O  OO   O OOOO  OO OO OO
          |                        O  O     O                               |
  1.3e+08 ++----------------------------------------------------------------+


                             numa-vmstat.node0.numa_other

    3e+07 ++----------------------------------------------------------------+
  2.8e+07 ++         O                                       O  O           |
          |O    OOO O O   OO       OOO    O     OO O OOOO OO     O          |
  2.6e+07 ++OOO    O   O O    OOOO    OO O  OO O  O      O  O  O            |
  2.4e+07 O+                OO          O     O                     OO      |
  2.2e+07 ++                                                      O     OOOOO
    2e+07 ++                                                       O  O     |
          |                                                                 |
  1.8e+07 ++                                                                |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
  1.2e+07 *+      *    *.  *                 *                              |
          |***   * * * : ** *  ***.*  *  *   :*  *                          |
    1e+07 ++  *. :  * *      **     ** ** *.*  ** *                         |
    8e+06 ++----*-----------------------------------------------------------+


                      numa-vmstat.node0.workingset_nodereclaim

  750000 ++-----------------------------------------------------------------+
         |           *.** **  .*        **                                  |
  700000 *+**.*******    *  **  ******.*  *****. **                         |
  650000 +*                                     *                           |
         |                                                                  |
  600000 ++                                                                 |
  550000 ++                                                                 |
         |                                                                  |
  500000 ++                                                                 |
  450000 ++         O                                OO                     |
         |       O      O    O          OO   OO           OOOO  O           |
  400000 OO    OO  O   O OO O  O    O            O OO   OO       O          |
  350000 ++O  O   O  O     O    OO O   O  O O   O O           O    OO  O OO |
         |  O                     O  O     O                         OO    OO
  300000 ++-------------------------------------------------------O---------+


                            numa-vmstat.node1.nr_free_pages

  9.5e+06 ++----------------------------------------------------------------+
    9e+06 ++     *                                                          |
          |      :*    *.**          *       *                              |
  8.5e+06 ++*   * : * *   :  **     * :  **  :* ***                         |
    8e+06 +* * +   ::*     :*  ***.*  * *  +:  *                            |
          *   *    *       *           *    *                               |
  7.5e+06 ++                                                                |
    7e+06 ++                                                                |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
  5.5e+06 ++                                          O                     |
          |      O OO      OO OOOO O O  O O   O OOOO O O OO OO OO   O    O OO
    5e+06 OOOOO O O  OOO OO  O      O OO O  OO O        O        OOO OO O O |
  4.5e+06 ++-----------------------------------------------O----------------+


                          numa-vmstat.node1.nr_inactive_file

   2.8e+07 ++----------------------------------------------O----------------+
  2.75e+07 OOOOO O O  O O O   O         O    O                   OO   O   O |
           |      O OO O O  OO OOOOOOO O OOOO OOO OOOO OOOO  OOOO  OOO  OO OO
   2.7e+07 ++                                         O                     |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           |                *                                               |
  2.45e+07 *+  *.   * *    : *    **    **  *  *                            |
   2.4e+07 +* *  *  ::::   :  ****  : .*  ** : :*.*                         |
           | *    :* * ****         **       **    *                        |
  2.35e+07 ++     *                                                         |
   2.3e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node1.nr_file_pages

   2.8e+07 ++----------------------------------------------O----------------+
  2.75e+07 OOOOO OOO  OOOOO   OO  O O  OO O OO O     O  O       OOOOOOO O O |
           |        OO      OO  OO O O   O O  O O OOO  O OO  OOO         O OO
   2.7e+07 ++                                         O                     |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           *   *    *       *           *   *                               |
  2.45e+07 +* * +   ::*    : *  ****   * *  :  *                            |
   2.4e+07 ++*   * : * *   :  **    * +   ** :* *.**                        |
           |      :*    ***          *       *                              |
  2.35e+07 ++     *                                                         |
   2.3e+07 ++---------------------------------------------------------------+


                              numa-vmstat.node1.numa_hit

    2e+08 ++----------------------------------------------------------------+
          *****.***  ***. ****** *.     * *. *                              |
  1.9e+08 ++       **    *      *  ***** *  * *****                         |
          |                                                                 |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++                                                                |
          |                                           O        O            |
  1.4e+08 ++OOO   O OOO   O O  O O  O  OOOO  OOOOO O O OOOOO      OOOO  OO  O
          OO    OO O   O O O OO O  O OO     O     O         OO  OO    O   OO|
  1.3e+08 ++----------------------------------------------------------------+


                              numa-vmstat.node1.numa_miss

    3e+07 ++----------------------------------------------------------------+
          |                   O      O                       O              |
          |OO   OOOOOOOO OOOO   OO O  OOO O OOO OOOO  OO  OOO   OOO        O|
  2.5e+07 O+ O               O O    O    O     O     O  OO     O   O  O OO  |
          |   O                                                     OO    O |
          |                                                                 O
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    1e+07 ++       **  *.**                                                 |
          * ***. **  * :   *******.*** ****. ******                         |
          |*    *     *               *     *                               |
    5e+06 ++----------------------------------------------------------------+


                            numa-vmstat.node1.numa_foreign

  2.6e+07 ++----------------------------------------------------------------+
  2.4e+07 ++                              O     O         O                 |
          |       O    O       O O                 O  O        OO           |
  2.2e+07 OOOOO O   OOO  OOOOOO O  OOOOOOO  OOOO OO  O O O OOO   O          |
    2e+07 ++     O O                                    O         O OO  O O |
          |                                                        O  O  O OO
  1.8e+07 ++                                                                |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
          |                                                                 |
  1.2e+07 ++                                                                |
    1e+07 ++                                                                |
          |**    *     *. **  * * .** *   *.* * * *                         |
    8e+06 *+ **. :*****  *  ** * *   * ***   * * *                          |
    6e+06 ++----*-----------------------------------------------------------+


                             numa-vmstat.node1.numa_local

    2e+08 ++----------------------------------------------------------------+
          *****.***  ***. ****** *.     * *. *                              |
  1.9e+08 ++       **    *      *  ***** *  * *****                         |
          |                                                                 |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++                                                                |
          |                                           O        O            |
  1.4e+08 ++OOO   O OOO   O O  O O  O  OOOO  OOOOO O O OOOOO      OOOO  O   O
          OO    OO O   O O O OO O  O OO     O     O         OO  OO    O  OOO|
  1.3e+08 ++----------------------------------------------------------------+


                             numa-vmstat.node1.numa_other

    3e+07 ++----------------------------------------------------------------+
          |                   O      O                       O              |
          |OO   OOOOOOOO OOOO   OO O  OOO O OOO OOOO  OO  OOO   OOO        O|
  2.5e+07 O+ OO              O O    O    O     O     O  OO     O   O  O OO  |
          |                                                         OO    O |
          |                                                                 O
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    1e+07 ++       **  *.**                                                 |
          * ***. **  * :   *******.*** ****. ******                         |
          |*    *     *               *     *                               |
    5e+06 ++----------------------------------------------------------------+


                      numa-vmstat.node1.workingset_nodereclaim

  750000 ++-----------------------------------------------------------------+
  700000 ** *.** ** **.******    *      *                                   |
         | *    *  *         *.** ****.* ******.***                         |
  650000 ++                                                                 |
  600000 ++                                                                 |
         |                                                                  |
  550000 ++                                                                 |
  500000 ++                                                                 |
  450000 ++                                                                 |
         |                                                                  |
  400000 ++         O      O                         O                      |
  350000 ++OO  OOO O O    O     OOO    OOOO OOO OO O  O  OOO  O O O         |
         OO   O   O      O  OO O   OOO     O      O O   O   OO   O OOO O OOOO
  300000 ++            OO                                             O     |
  250000 ++-----------------------------------------------------------------+


                            numa-vmstat.node2.nr_free_pages

  9.5e+06 ++----------------------------------------------------------------+
    9e+06 ++                    *                                           |
          |               *     ::      *                                   |
  8.5e+06 *+  *.*   ***   :*  * :*.*    ::     *  *                         |
    8e+06 +*  :  :**   *.*  ** :    : ** **.   ::*                          |
          | :*   *             *    **      *** *                           |
  7.5e+06 ++*                                                               |
    7e+06 ++                                                                |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
  5.5e+06 ++                                                                |
          |      OO    O       O    OOO O   O O       O    O     O      OO O|
    5e+06 O+O O O  OOOO  O OOO  OO O   O OO  O OOOOO O O OO OO OO OOOOO   O O
  4.5e+06 +O-O------------O---O-------------------------O-------------------+


                          numa-vmstat.node2.nr_inactive_file

   2.8e+07 +O-O-----------O----O--------------------------------------------+
  2.75e+07 O+O O    OO O O  O O  OOO      O    O   OOO OOOO   OOO OO OO   O O
           |     OOO  O O    O  O   OO OOO OOOO O O   O    O O   O  O   OO O|
   2.7e+07 ++                                                               |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++*                                                              |
           | ::                 *   *       *** *.                          |
  2.45e+07 +* *   ***    *   ** :   :*. * **   :: **                        |
   2.4e+07 *+  *.*   **** :.*  * :**   * ::    *                            |
           |              *      ::      *                                  |
  2.35e+07 ++                    *                                          |
   2.3e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node2.nr_file_pages

  2.85e+07 ++---------------------------------------------------------------+
   2.8e+07 ++ O           O    O                                            |
           OOO O     O O O  O    OOO    O    O     O O OO     OOO    OO   O O
  2.75e+07 ++    OOOO O O    OO O   OO O OOOO OOO O O    OOO O   OOOO   OO O|
   2.7e+07 ++                                         O                     |
           |                                                                |
  2.65e+07 ++                                                               |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
           | *                                                              |
   2.5e+07 ++::                 *    *      **  *.                          |
  2.45e+07 +* *   **    **   ** :   * + * **  * : *                         |
           *   :.*  ****  :.*  : :**   *: :    *   *                        |
   2.4e+07 ++  *          *    * ::      :                                  |
  2.35e+07 ++--------------------*-------*----------------------------------+


                        numa-vmstat.node2.nr_isolated_file

  600 ++--------------------------------------------------------------------+
      |    *                                                                |
  550 ++ .* :      *    *    * *            * *.                            |
  500 ***   *  *  * O*  : * :: :*   **  *  *: : **                          |
      |      * :+ : * +: :::: :: : :  ** : : :                        O     |
  450 ++      :  *     * * *  :  * :     : : *                 O   O        |
      |       *         O     *   *       :                                 |
  400 ++                  O       O       *                               O |
      |              O O   O           O     O    O                         |
  350 ++O                        O                 OOO     O O  O      O   O|
  300 +O     OO              OO     O     O             O O   O  O  OO  O   O
      O   OOO  O OO      O     OO    OO O  OO O  O     O                    |
  250 ++           O                            O        O                  |
      |                                                                     |
  200 ++--------------------------------------------------------------------+


                              numa-vmstat.node2.numa_hit

     2e+08 ++---------------------------------------------------------------+
  1.95e+08 *****.**********.*** * *      *   *                              |
           |                   * * ***.** *** ***.**                        |
   1.9e+08 ++                                                               |
  1.85e+08 ++                                                               |
   1.8e+08 ++                                                               |
  1.75e+08 ++                                                               |
           |                                                                |
   1.7e+08 ++                                                               |
  1.65e+08 ++                                                               |
   1.6e+08 ++                                                               |
  1.55e+08 ++O     O  O   O    O  O  O  OO   OO   O  O OO       O           |
           |   O OO  O O O  OOO OO OO  O  OOO  OO  OO O  OOO OOO O OO O    OO
   1.5e+08 OO O     O   O                                         O  O  OOO |
  1.45e+08 ++---------------------------------------------------------------+


                              numa-vmstat.node2.numa_miss

  2.4e+07 ++------------------------------O-----O---------------------------+
  2.2e+07 ++      O            O        O     O       O   O                 |
          OOO O O   OO O O  OOO OO O OO  O  O  O OOO O     OO  OOO          |
    2e+07 ++ O   O O  O   OO        O  O     O         O O   O    O OO   OOO|
  1.8e+07 ++                                            O          O  O O   O
          |                                                                 |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
  1.2e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
    8e+06 ++          **. **  * * .*  *    .* * *                           |
          ***  .******   *  ** * *  ** ****  ::* **                         |
    6e+06 ++ **                              *                              |
    4e+06 ++----------------------------------------------------------------+


                            numa-vmstat.node2.numa_foreign

  3.5e+07 ++----------------------------------------------------------------+
          |                                                                 |
    3e+07 ++     O                   O O     O                              |
          |O  O O OOOOO  O OOOO OO O     OO    OOOOO OOOOOOOOO  OO          |
          O OO         O  O    O    O O O   O O                O  O        O|
  2.5e+07 ++                                                        OOO O O O
          |                                                        O     O  |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          *       *     .*    * *                                           |
    1e+07 +****. * *****  **** * *.********.*** ***                         |
          |     *                              *                            |
    5e+06 ++----------------------------------------------------------------+


                             numa-vmstat.node2.numa_local

     2e+08 ++---------------------------------------------------------------+
  1.95e+08 *****.**********.*** * *      *   *                              |
           |                   * * ***.** *** ***.**                        |
   1.9e+08 ++                                                               |
  1.85e+08 ++                                                               |
   1.8e+08 ++                                                               |
  1.75e+08 ++                                                               |
           |                                                                |
   1.7e+08 ++                                                               |
  1.65e+08 ++                                                               |
   1.6e+08 ++                                                               |
  1.55e+08 ++O     O  O   O    O  O  O  OO   OO   O  O OO                   |
           |   O OO  O O O  OOO OO OO  O  OOO  OO  OO O  OOO OOOOO OO O    OO
   1.5e+08 OO O     O   O                                         O  O  OOO |
  1.45e+08 ++---------------------------------------------------------------+


                             numa-vmstat.node2.numa_other

  2.4e+07 ++------------------------------O-----O---------------------------+
          |       O            O                      O   O                 |
  2.2e+07 +O  O O   OO O O  O O OO O OO O   O O    O O     OO  OOO          |
    2e+07 O+OO     O  O   OO O      O  O O   O O OO    O O   O      OO   OO |
          |      O                                      O         O   O O  OO
  1.8e+07 ++                                                       O        |
  1.6e+07 ++                                                                |
          |                                                                 |
  1.4e+07 ++                                                                |
  1.2e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
    8e+06 ++           *. *   * *  *  *       * *                           |
          * *    **** *  * *  :* :+ ** * **.* :* **                         |
    6e+06 +*-**-*----*------**---*------*----*------------------------------+


                      numa-vmstat.node2.workingset_nodereclaim

  750000 ++-----------------------------------------------------------------+
         ** *.**  ****.  **  *.* *      **  *                               |
  700000 ++*    **     **  **   * ****. : ** * .***                         |
         |                             *      *                             |
  650000 ++                                                                 |
  600000 ++                                                                 |
         |                                                                  |
  550000 ++                                                                 |
         |       O                                                          |
  500000 ++                            OO   OO      O         O             |
  450000 ++O  O    OO    O O O OOOO O    OO   O OOOO OO OOO  O  OO          |
         |     OO    O  O   O      O                       OO       OO     OO
  400000 ++ O     O    O             O     O                      OO   O OO |
         |O               O                                           O     |
  350000 O+-----------------------------------------------------------------+


                            numa-vmstat.node3.nr_free_pages

  9.5e+06 ++----------------------------------------------------------------+
    9e+06 ++  *.*                        *                                  |
          |  *  :                       *:                                  |
  8.5e+06 ** :   : *   *.**   ** *.    *  :.*    **                         |
    8e+06 ++*    ** ***    ***  :: ****   *  * **                           |
          |                     *             *                             |
  7.5e+06 ++                                                                |
    7e+06 ++                                                                |
  6.5e+06 ++                                                                |
          |                                                                 |
    6e+06 ++                                                                |
  5.5e+06 ++                                          O                     |
          |          O    OOOO  OO O O  OOO   OOO O  O   O OO  OO O        O|
    5e+06 OO OO OOOOO OO O    OO    O OO    OO   O O   OO O  O   O OOOO OOO O
  4.5e+06 ++O---------------------------------------------------------------+


                          numa-vmstat.node3.nr_inactive_file

   2.8e+07 ++O--------------------------------------------------------------+
  2.75e+07 OO OO OOOOO OOO     OO   O  OO   OO    O O  OO O    O O OOOO O O O
           |          O   O OOO  OOO O   OOO  OOO  O O   O   OO O O      O O|
   2.7e+07 ++                                         O    O                |
  2.65e+07 ++                                                               |
           |                                                                |
   2.6e+07 ++                                                               |
  2.55e+07 ++                                                               |
   2.5e+07 ++                                                               |
           |                     *                                          |
  2.45e+07 ++*    ** ***   .* *  ::***.*     ** *.                          |
   2.4e+07 ** :   : *   ***  * ** *     *  **  *  **                        |
           |  *  :                       * :                                |
  2.35e+07 ++  *.*                        *                                 |
   2.3e+07 ++---------------------------------------------------------------+


                            numa-vmstat.node3.nr_file_pages

  2.85e+07 ++---------------------------------------------------------------+
   2.8e+07 ++O                                                              |
           OO OO OOOOO OOO     OO   O   O    O    O O  OO O   O  O OOOO  OO O
  2.75e+07 ++         O   O OOO  OOO O O OOOO OOO  O O   O O O OO O     O  O|
   2.7e+07 ++                                         O                     |
  2.65e+07 ++                                                               |
   2.6e+07 ++                                                               |
           |                                                                |
  2.55e+07 ++                                                               |
   2.5e+07 ++                    *                                          |
  2.45e+07 ++*    ** ***   .* *  ::***.      **                             |
   2.4e+07 ** :   : *   :**  * ** *    **  **  **.**                        |
           |  *  :      *                * :                                |
  2.35e+07 ++  *.*                        *                                 |
   2.3e+07 ++---------------------------------------------------------------+


                        numa-vmstat.node3.nr_isolated_file

  600 ++--------------------------------------------------------------------+
      |                    *                                                |
  550 ++            *      :   *          *                                 |
      | *           :    *: : *:         : * *                      O       |
  500 +* +  **   *  ::   :* : : :**      :  ::*                             |
      |:  **  * + *O *. *    :  *  + ** *   *  +                            |
  450 O+       *   :   *     *      *  ::       *                         O |
      |            *              O    *         *     O           O        |
  400 ++   O            O                                                   |
      |              O                              O    O                  |
  350 ++    OOO   O    O   O           O     O     O    O       O          O|
      | O O    O O        O   O  O  OO  O OO  O  O   O    OO OOO       OO   O
  300 +O            O    O   O OO     O     O   O O                   O     |
      |                                                          O   O      |
  250 ++--------------------------------------------------------------------+


                              numa-vmstat.node3.numa_hit

    2e+08 ++----------------------------------------------------------------+
          *****.********.**** ** *.     *    *   *                          |
  1.9e+08 ++                 *  *  ***** **.* *** *                         |
          |                                                                 |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++              O                                                 |
          OOO O OOO OOOO   OO OOO   OO  OOO   OOOOOO OOOOOOO O OOOOOOOO OOOOO
  1.4e+08 ++ O     O     O   O   O O  OO    OO              O               |
          |                                                                 |
  1.3e+08 ++----------------------------------------------------------------+


                              numa-vmstat.node3.numa_miss

    3e+07 ++----------------------------------------------------------------+
          |      O       O   O       O O     O                              |
          |     O OOO O    OO OOOO O     OO    OOOOO  OOOOOOO  OOO          |
  2.5e+07 OOOOO      O O  O         O O O   O O      O       O              |
          |                                                       O     O  OO
          |                                                        OOOO  OO |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    1e+07 ++                 *  *                                           |
          *****.  ******.**** ** *.********.*   ***                         |
          |     **                           ***                            |
    5e+06 ++----------------------------------------------------------------+


                            numa-vmstat.node3.numa_foreign

  2.8e+07 ++----------------------------------------------------------------+
  2.6e+07 +O    OOO  O O  OO       O OO   O O   OO O O     O O  OO          |
          | O O    OO O  O  O OOOO  O  OO    OO   O   OOOOO O  O            |
  2.4e+07 O+ O               O           O     O                  O OO  OO O|
  2.2e+07 ++                                                       O  O   O O
    2e+07 ++                                                                |
  1.8e+07 ++                                                                |
          |                                                                 |
  1.6e+07 ++                                                                |
  1.4e+07 ++                                                                |
  1.2e+07 ++                                                                |
    1e+07 *+      **   *. *                                                 |
          |****. *  ***  * *******.* ******. ** ***                         |
    8e+06 ++    *                   *       *  *                            |
    6e+06 ++----------------------------------------------------------------+


                             numa-vmstat.node3.numa_local

    2e+08 ++----------------------------------------------------------------+
          *****.********.**** ** *.     *    *                              |
  1.9e+08 ++                 *  *  ***** **.* *****                         |
          |                                                                 |
  1.8e+08 ++                                                                |
          |                                                                 |
  1.7e+08 ++                                                                |
          |                                                                 |
  1.6e+08 ++                                                                |
          |                                                                 |
  1.5e+08 ++              O                                                 |
          OOO O OOO OOOO   OO OOO   OO  OOO   OOOOOO OOOOOOO O OOOOOOOO OOOOO
  1.4e+08 ++ O     O     O   O   O O  OO    OO              O               |
          |                                                                 |
  1.3e+08 ++----------------------------------------------------------------+


                             numa-vmstat.node3.numa_other

    3e+07 ++----------------------------------------------------------------+
          |      O       O   O       O O     O                              |
          |     O OOO O    OO OOOO O    OOO   OOOOOO  OOOOOOO  OOO          |
  2.5e+07 OOOOO      O O  O         O O     O        O       O              |
          |                                                       O   O O  OO
          |                                                        OOO   OO |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    1e+07 ++                 *  *                                           |
          *****.  ******.**** ** *.********.**  ***                         |
          |     **                            **                            |
    5e+06 ++----------------------------------------------------------------+


                      numa-vmstat.node3.workingset_nodereclaim

  750000 ++----*------------------------------------------------------------+
         ****.* ** * *.  *   *. *   * .*                                    |
  700000 ++       * *  ** ***  * *** *  *******.***                         |
         |                                                                  |
  650000 ++                                                                 |
  600000 ++                                                                 |
         |                                                                  |
  550000 ++                                                                 |
         |                O                                                 |
  500000 ++                                                                 |
  450000 OOO           O O      O  O               O       O                |
         |    OOO  OO      O O   O  O  OOOO OOO O O O         O OO          |
  400000 ++          O  O   O  O  O  O     O     O   OO  OO  O    OOO       O
         |  O    OO                                     O            OOO OOO|
  350000 ++-------------------------------------------------O---------------+

Thanks,
Fengguang

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
  2014-08-19 17:52         ` Fengguang Wu
@ 2014-08-19 19:18           ` Mel Gorman
  -1 siblings, 0 replies; 12+ messages in thread
From: Mel Gorman @ 2014-08-19 19:18 UTC (permalink / raw)
  To: Fengguang Wu; +Cc: LKML, lkp

On Wed, Aug 20, 2014 at 01:52:39AM +0800, Fengguang Wu wrote:
> On Tue, Aug 19, 2014 at 06:13:58PM +0100, Mel Gorman wrote:
> > On Tue, Aug 19, 2014 at 11:51:25PM +0800, Fengguang Wu wrote:
> > > On Tue, Aug 19, 2014 at 03:29:25PM +0100, Mel Gorman wrote:
> > > > On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> > > > > Hi Mel,
> > > > > 
> > > > > We noticed the below vm-scalability performance/power regressions on
> > > > > commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> > > > > fields into read-only, page alloc, statistics and page reclaim lines").
> > > > > 
> > > > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> > > > > ---------------  -------------------------  ---------------------------
> > > > >      %stddev        %change               %stddev
> > > > >             \          |                 /
> > > > >       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > > > >       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
> > > > >      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> > > > > 
> > > > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> > > > > ---------------  -------------------------  
> > > > >   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > > > >   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
> > > > >   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> > > > > 
> > > > 
> > > > 
> > > > What units are these? It's completely unclear what is good and bad from the
> > > > figures. 300s-lru-file-mmap-read appears multiple times in this report,
> > > > each with different numbers beside them but little clue as to what they
> > > > mean or what I'm meant to be looking for :(
> > > 
> > > Sorry the output format is a bit obscure: the stats names are after
> > > the "TOTAL" word, such as TOTAL vm-scalability.stddev, TOTAL
> > > vm-scalability.throughput, ...
> > > 
> > > > This is the same patch that was reported as having a performance gain in
> > > > another set of tests from lkp so am a little confused.
> > > 
> > > That can happen some times -- we've seen some commits to benefit some
> > > workloads while hurting some others. I'm now running more complete
> > > test set for the commit, hopefully can get some results tomorrow.
> > > 
> > > > More importantly, as this patch is primary abougt cache misses it should
> > > > be very unlikely that it makes a noticable difference to IO as the
> > > > relative cost of a cache miss is so low. Similarly any difference it
> > > > makes to reclaim activity is likely to be a coincidence or due to test
> > > > variance.
> > > 
> > > The test case creates tmpfs files and read them fast to exercise the LRU.
> > > So it's VM test and do not involve disk IO.
> > > 
> > 
> > Ok. How reliable is this? I was reading down through some of the figures and
> > some of them are extremely different in areas that this patch should have
> > no impact on -- it's just rearranging a struct after all.
> 
> Yup. One difficulty with performance testing is that we sometimes
> bisect into commit whose change set is obviously not related to the
> changed performance.. and the bisect itself looks good.
> 

I still worry that the result of this depends on what node the testcase
happens to run on due to differences in the allocation stats between
tests. What happens if you bind the test case to use CPUs from only one
node? e.g. numactl --cpunodebind=nodes testcase

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W
@ 2014-08-19 19:18           ` Mel Gorman
  0 siblings, 0 replies; 12+ messages in thread
From: Mel Gorman @ 2014-08-19 19:18 UTC (permalink / raw)
  To: lkp

[-- Attachment #1: Type: text/plain, Size: 3754 bytes --]

On Wed, Aug 20, 2014 at 01:52:39AM +0800, Fengguang Wu wrote:
> On Tue, Aug 19, 2014 at 06:13:58PM +0100, Mel Gorman wrote:
> > On Tue, Aug 19, 2014 at 11:51:25PM +0800, Fengguang Wu wrote:
> > > On Tue, Aug 19, 2014 at 03:29:25PM +0100, Mel Gorman wrote:
> > > > On Tue, Aug 19, 2014 at 12:32:52PM +0800, Fengguang Wu wrote:
> > > > > Hi Mel,
> > > > > 
> > > > > We noticed the below vm-scalability performance/power regressions on
> > > > > commit 3484b2de9499df23c4604a513b36f96326ae81ad ("mm: rearrange zone
> > > > > fields into read-only, page alloc, statistics and page reclaim lines").
> > > > > 
> > > > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  testbox/testcase/testparams
> > > > > ---------------  -------------------------  ---------------------------
> > > > >      %stddev        %change               %stddev
> > > > >             \          |                 /
> > > > >       9.95 ± 2%     +69.1%      16.83 ± 5%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > > > >       2.32 ± 6%    +229.4%       7.63 ± 5%  brickland3/vm-scalability/300s-lru-file-readonce
> > > > >      12.27 ± 3%     +99.4%      24.46 ± 5%  TOTAL vm-scalability.stddev
> > > > > 
> > > > > 24b7e5819ad5cbe  3484b2de9499df23c4604a513  
> > > > > ---------------  -------------------------  
> > > > >   13882598 ± 0%     -35.8%    8915310 ± 1%  brickland3/vm-scalability/300s-lru-file-mmap-read
> > > > >   36379953 ± 1%     -64.0%   13093373 ± 0%  brickland3/vm-scalability/300s-lru-file-readonce
> > > > >   50262551 ± 0%     -56.2%   22008683 ± 0%  TOTAL vm-scalability.throughput
> > > > > 
> > > > 
> > > > 
> > > > What units are these? It's completely unclear what is good and bad from the
> > > > figures. 300s-lru-file-mmap-read appears multiple times in this report,
> > > > each with different numbers beside them but little clue as to what they
> > > > mean or what I'm meant to be looking for :(
> > > 
> > > Sorry the output format is a bit obscure: the stats names are after
> > > the "TOTAL" word, such as TOTAL vm-scalability.stddev, TOTAL
> > > vm-scalability.throughput, ...
> > > 
> > > > This is the same patch that was reported as having a performance gain in
> > > > another set of tests from lkp so am a little confused.
> > > 
> > > That can happen some times -- we've seen some commits to benefit some
> > > workloads while hurting some others. I'm now running more complete
> > > test set for the commit, hopefully can get some results tomorrow.
> > > 
> > > > More importantly, as this patch is primary abougt cache misses it should
> > > > be very unlikely that it makes a noticable difference to IO as the
> > > > relative cost of a cache miss is so low. Similarly any difference it
> > > > makes to reclaim activity is likely to be a coincidence or due to test
> > > > variance.
> > > 
> > > The test case creates tmpfs files and read them fast to exercise the LRU.
> > > So it's VM test and do not involve disk IO.
> > > 
> > 
> > Ok. How reliable is this? I was reading down through some of the figures and
> > some of them are extremely different in areas that this patch should have
> > no impact on -- it's just rearranging a struct after all.
> 
> Yup. One difficulty with performance testing is that we sometimes
> bisect into commit whose change set is obviously not related to the
> changed performance.. and the bisect itself looks good.
> 

I still worry that the result of this depends on what node the testcase
happens to run on due to differences in the allocation stats between
tests. What happens if you bind the test case to use CPUs from only one
node? e.g. numactl --cpunodebind=nodes testcase

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2014-08-19 19:20 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-19  4:32 [mm] 3484b2de949: -56.2% vm-scalability.throughput, +9.3% turbostat.Pkg_W Fengguang Wu
2014-08-19  4:32 ` Fengguang Wu
2014-08-19 14:29 ` Mel Gorman
2014-08-19 14:29   ` Mel Gorman
2014-08-19 15:51   ` Fengguang Wu
2014-08-19 15:51     ` Fengguang Wu
2014-08-19 17:13     ` Mel Gorman
2014-08-19 17:13       ` Mel Gorman
2014-08-19 17:52       ` Fengguang Wu
2014-08-19 17:52         ` Fengguang Wu
2014-08-19 19:18         ` Mel Gorman
2014-08-19 19:18           ` Mel Gorman

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.