* [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-09-28 6:49 ` kernel test robot
0 siblings, 0 replies; 17+ messages in thread
From: kernel test robot @ 2015-09-28 6:49 UTC (permalink / raw)
To: Jeff Layton; +Cc: lkp, LKML
[-- Attachment #1: Type: text/plain, Size: 43199 bytes --]
FYI, we noticed the below changes on
=========================================================================================
tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
commit:
cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
4aac1bf05b053a201a4b392dd9a684fb2b7e6103
cd2d35ff27c4fda9 4aac1bf05b053a201a4b392dd9
---------------- --------------------------
%stddev %change %stddev
\ | \
14415356 ± 0% +2.6% 14788625 ± 1% fsmark.app_overhead
441.60 ± 0% -2.9% 428.80 ± 0% fsmark.files_per_sec
185.78 ± 0% +2.9% 191.26 ± 0% fsmark.time.elapsed_time
185.78 ± 0% +2.9% 191.26 ± 0% fsmark.time.elapsed_time.max
97472 ± 0% -2.8% 94713 ± 0% fsmark.time.involuntary_context_switches
3077117 ± 95% +251.2% 10805440 ±112% latency_stats.sum.nfs_wait_on_request.nfs_updatepage.nfs_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.nfs_file_write.__vfs_write.vfs_write.SyS_write.entry_SYSCALL_64_fastpath
12999 ± 0% +32.9% 17276 ± 0% proc-vmstat.nr_slab_unreclaimable
64568 ± 4% -14.8% 55032 ± 0% softirqs.RCU
51999 ± 0% +32.9% 69111 ± 0% meminfo.SUnreclaim
159615 ± 0% +13.5% 181115 ± 0% meminfo.Slab
3.75 ± 0% +3.3% 3.88 ± 1% turbostat.%Busy
77.25 ± 0% +6.5% 82.25 ± 0% turbostat.Avg_MHz
30813025 ± 2% -14.5% 26338527 ± 9% cpuidle.C1E-NHM.time
164180 ± 0% -28.9% 116758 ± 7% cpuidle.C1E-NHM.usage
1738 ± 2% -81.2% 326.75 ± 4% cpuidle.POLL.usage
29979 ± 2% +44.3% 43273 ± 4% numa-meminfo.node0.SUnreclaim
94889 ± 0% +19.8% 113668 ± 2% numa-meminfo.node0.Slab
22033 ± 3% +17.3% 25835 ± 7% numa-meminfo.node1.SUnreclaim
7404 ± 1% -2.7% 7206 ± 0% vmstat.io.bo
27121 ± 0% -4.8% 25817 ± 0% vmstat.system.cs
3025 ± 0% -13.5% 2615 ± 0% vmstat.system.in
50126 ± 1% +11.5% 55893 ± 1% numa-vmstat.node0.nr_dirtied
7494 ± 2% +44.3% 10818 ± 4% numa-vmstat.node0.nr_slab_unreclaimable
50088 ± 1% +11.6% 55900 ± 1% numa-vmstat.node0.nr_written
5507 ± 3% +17.3% 6458 ± 7% numa-vmstat.node1.nr_slab_unreclaimable
7164 ± 2% +275.2% 26885 ± 0% slabinfo.kmalloc-16.active_objs
7164 ± 2% +275.3% 26885 ± 0% slabinfo.kmalloc-16.num_objs
7367 ± 1% +787.7% 65401 ± 0% slabinfo.kmalloc-192.active_objs
179.00 ± 1% +771.8% 1560 ± 0% slabinfo.kmalloc-192.active_slabs
7537 ± 1% +770.0% 65572 ± 0% slabinfo.kmalloc-192.num_objs
179.00 ± 1% +771.8% 1560 ± 0% slabinfo.kmalloc-192.num_slabs
3631 ± 7% +522.3% 22600 ± 0% slabinfo.kmalloc-256.active_objs
145.50 ± 4% +398.1% 724.75 ± 0% slabinfo.kmalloc-256.active_slabs
4667 ± 4% +397.3% 23210 ± 0% slabinfo.kmalloc-256.num_objs
145.50 ± 4% +398.1% 724.75 ± 0% slabinfo.kmalloc-256.num_slabs
17448 ± 2% +75.6% 30643 ± 0% slabinfo.kmalloc-32.active_objs
137.50 ± 2% +76.5% 242.75 ± 0% slabinfo.kmalloc-32.active_slabs
17651 ± 2% +76.4% 31139 ± 0% slabinfo.kmalloc-32.num_objs
137.50 ± 2% +76.5% 242.75 ± 0% slabinfo.kmalloc-32.num_slabs
2387 ± 3% -10.7% 2132 ± 8% slabinfo.kmalloc-512.active_objs
491.25 ± 3% +33.9% 658.00 ± 11% slabinfo.numa_policy.active_objs
491.25 ± 3% +33.9% 658.00 ± 11% slabinfo.numa_policy.num_objs
2128 ± 9% +59.3% 3391 ± 34% sched_debug.cfs_rq[10]:/.exec_clock
18088 ± 17% +47.0% 26582 ± 29% sched_debug.cfs_rq[10]:/.min_vruntime
4326 ± 11% -22.2% 3368 ± 18% sched_debug.cfs_rq[5]:/.exec_clock
1459 ± 1% -10.8% 1302 ± 3% sched_debug.cpu#0.nr_uninterruptible
122217 ± 7% -18.6% 99447 ± 2% sched_debug.cpu#1.nr_switches
122732 ± 8% -18.5% 99972 ± 2% sched_debug.cpu#1.sched_count
45603 ± 10% -20.1% 36442 ± 2% sched_debug.cpu#1.sched_goidle
27004 ± 3% -18.9% 21895 ± 5% sched_debug.cpu#1.ttwu_local
15469 ± 5% +17.2% 18132 ± 6% sched_debug.cpu#10.nr_load_updates
78564 ± 8% +26.6% 99492 ± 5% sched_debug.cpu#10.nr_switches
78605 ± 8% +26.7% 99557 ± 4% sched_debug.cpu#10.sched_count
27470 ± 9% +24.7% 34268 ± 7% sched_debug.cpu#10.sched_goidle
38215 ± 1% +37.4% 52499 ± 13% sched_debug.cpu#10.ttwu_count
14816 ± 5% +22.8% 18196 ± 2% sched_debug.cpu#10.ttwu_local
19690 ± 21% -29.9% 13802 ± 15% sched_debug.cpu#11.nr_switches
54.25 ± 2% -47.5% 28.50 ± 25% sched_debug.cpu#11.nr_uninterruptible
19721 ± 21% -29.9% 13828 ± 15% sched_debug.cpu#11.sched_count
14545 ± 2% +15.4% 16779 ± 4% sched_debug.cpu#12.nr_load_updates
72087 ± 11% +27.9% 92204 ± 7% sched_debug.cpu#12.nr_switches
72126 ± 11% +28.1% 92422 ± 7% sched_debug.cpu#12.sched_count
25418 ± 13% +24.4% 31626 ± 7% sched_debug.cpu#12.sched_goidle
33399 ± 15% +38.5% 46255 ± 13% sched_debug.cpu#12.ttwu_count
51.25 ± 10% -39.0% 31.25 ± 21% sched_debug.cpu#13.nr_uninterruptible
2593 ± 11% -21.8% 2028 ± 10% sched_debug.cpu#13.ttwu_local
71266 ± 3% +20.1% 85620 ± 5% sched_debug.cpu#14.nr_switches
71306 ± 3% +20.4% 85827 ± 5% sched_debug.cpu#14.sched_count
24634 ± 3% +18.8% 29259 ± 4% sched_debug.cpu#14.sched_goidle
34625 ± 11% +19.9% 41506 ± 11% sched_debug.cpu#14.ttwu_count
13866 ± 3% +20.6% 16726 ± 5% sched_debug.cpu#14.ttwu_local
12683 ± 4% -14.7% 10817 ± 2% sched_debug.cpu#15.nr_load_updates
49.75 ± 6% -46.2% 26.75 ± 28% sched_debug.cpu#15.nr_uninterruptible
3374 ± 12% -28.1% 2427 ± 18% sched_debug.cpu#15.ttwu_local
186563 ± 5% -12.1% 163975 ± 4% sched_debug.cpu#2.nr_switches
-1324 ± -2% -16.0% -1111 ± -1% sched_debug.cpu#2.nr_uninterruptible
187499 ± 5% -11.2% 166447 ± 4% sched_debug.cpu#2.sched_count
67465 ± 7% -13.6% 58308 ± 6% sched_debug.cpu#2.sched_goidle
36525 ± 4% -14.6% 31193 ± 1% sched_debug.cpu#2.ttwu_local
23697 ± 5% -13.2% 20572 ± 9% sched_debug.cpu#3.nr_load_updates
128070 ± 1% -22.9% 98687 ± 5% sched_debug.cpu#3.nr_switches
129859 ± 2% -23.5% 99357 ± 4% sched_debug.cpu#3.sched_count
48833 ± 1% -23.7% 37243 ± 6% sched_debug.cpu#3.sched_goidle
61622 ± 3% -24.2% 46694 ± 5% sched_debug.cpu#3.ttwu_count
27510 ± 7% -20.6% 21840 ± 8% sched_debug.cpu#3.ttwu_local
81675 ± 7% -13.6% 70536 ± 1% sched_debug.cpu#4.ttwu_count
34076 ± 3% -12.9% 29683 ± 1% sched_debug.cpu#4.ttwu_local
124470 ± 4% -14.1% 106865 ± 8% sched_debug.cpu#5.sched_count
62502 ± 3% -20.8% 49519 ± 9% sched_debug.cpu#5.ttwu_count
26562 ± 0% -17.7% 21853 ± 10% sched_debug.cpu#5.ttwu_local
181661 ± 10% -15.1% 154229 ± 6% sched_debug.cpu#6.nr_switches
181937 ± 10% -13.5% 157379 ± 6% sched_debug.cpu#6.sched_count
66672 ± 14% -16.6% 55632 ± 9% sched_debug.cpu#6.sched_goidle
78296 ± 2% -10.2% 70346 ± 6% sched_debug.cpu#6.ttwu_count
33536 ± 1% -14.4% 28696 ± 1% sched_debug.cpu#6.ttwu_local
131463 ± 6% -17.0% 109140 ± 4% sched_debug.cpu#7.nr_switches
-32.25 ±-58% -100.8% 0.25 ±9467% sched_debug.cpu#7.nr_uninterruptible
133606 ± 7% -17.2% 110671 ± 4% sched_debug.cpu#7.sched_count
50986 ± 7% -16.6% 42525 ± 6% sched_debug.cpu#7.sched_goidle
61388 ± 2% -19.8% 49213 ± 5% sched_debug.cpu#7.ttwu_count
26637 ± 2% -21.8% 20837 ± 3% sched_debug.cpu#7.ttwu_local
12312 ± 3% +9.4% 13474 ± 4% sched_debug.cpu#8.nr_load_updates
53.50 ± 6% -44.9% 29.50 ± 27% sched_debug.cpu#9.nr_uninterruptible
2724 ± 15% -23.7% 2078 ± 26% sched_debug.cpu#9.ttwu_local
lkp-ne04: Nehalem-EP
Memory: 12G
cpuidle.POLL.usage
1800 ++----------*-----------------------------*-----*-----*--------*--*--+
*..*..*..*. .*..*..*..*..*..*..*..*. *. *. *..*. *
1600 ++ *. |
1400 ++ |
| |
1200 ++ |
| |
1000 ++ |
| |
800 ++ |
600 ++ |
| |
400 O+ |
| O O O O O O O O O O O O O O O O O O O O O O |
200 ++-------------------------------------------------------------------+
cpuidle.C1E-NHM.usage
190000 ++-----------------------------------------------------*-----------+
| : : |
180000 ++ : : |
170000 ++ .*. : : |
| .*.. .*.. .*. .*.. .*..*..*..*. * *..*..*..*
160000 *+ *. *. *..*..*. *..*..*. |
150000 ++ |
| |
140000 ++ |
130000 ++ O |
| |
120000 ++ O O O O |
110000 ++ O O O O O O O O O |
O O O O O O O O O |
100000 ++-----------------------------------------------------------------+
fsmark.files_per_sec
446 ++--------------------------------------------------------------------+
444 ++ *.. |
| .. |
442 *+.*..*..*..*..*..*.. *..*..*..*...*..*..*..*..*..*..* *..*..*..*
440 ++ .. |
438 ++ * |
436 ++ |
| |
434 ++ |
432 ++ |
430 ++ |
428 ++ O O O O O O O O O O O O O O O O O O |
| |
426 O+ O O O O |
424 ++--------------------------------------------------------------------+
fsmark.time.elapsed_time
193 ++--------------------------------------------------------------------+
O O O |
192 ++ O O O O O O O O O O |
191 ++ O O O O O O O O |
| O O |
190 ++ |
189 ++ |
| |
188 ++ |
187 ++ |
| .*.. |
186 *+. .*..*..*..*..*. *.. .*.. ..*.. .*.. .*.. .*.. *..*..*..*
185 ++ *. *. *. *. *. *. .. |
| * |
184 ++--------------------------------------------------------------------+
fsmark.time.elapsed_time.max
193 ++--------------------------------------------------------------------+
O O O |
192 ++ O O O O O O O O O O |
191 ++ O O O O O O O O |
| O O |
190 ++ |
189 ++ |
| |
188 ++ |
187 ++ |
| .*.. |
186 *+. .*..*..*..*..*. *.. .*.. ..*.. .*.. .*.. .*.. *..*..*..*
185 ++ *. *. *. *. *. *. .. |
| * |
184 ++--------------------------------------------------------------------+
fsmark.time.involuntary_context_switches
98500 ++------------------------------------------------------------------+
98000 ++ .* |
| *.. .*. + *.. .*..*..* *
97500 ++. *..*..*. + .. *..*.*..*..*..*..*..*..*. + +|
97000 *+ *..* + + |
| * |
96500 ++ |
96000 ++ |
95500 ++ |
| |
95000 ++ O O O O O O O |
94500 ++ O O O O O O O O O |
O O O O O |
94000 ++ O O |
93500 ++------------------------------------------------------------------+
vmstat.system.in
3100 ++-------------------------------------------------------------------+
3050 *+. .*.. .*.. .*..*.. .*..*.. |
| .*..*.. .*..*. *. *. *.. .*..*. .*..*..*..*
3000 ++ *. *. *. *. |
2950 ++ |
2900 ++ |
2850 ++ |
| |
2800 ++ |
2750 ++ |
2700 ++ |
2650 ++ O O |
| O O O O O O O O O O O |
2600 O+ O O O O O O O O O |
2550 ++-------------------------------------------------------------------+
numa-vmstat.node0.nr_slab_unreclaimable
12000 ++------------------------------------------------------------------+
| O |
11000 ++ O O O O O O |
O O O O O |
| O O O O O O O O |
10000 ++ O O O |
| |
9000 ++ |
| |
8000 ++ *.. |
| .*..*.. + .*.. .*
| .*.. .*.. .*.*. + *..*..*. *..*. |
7000 *+.*. .*. .*..*. * |
| *..*. *..*. |
6000 ++------------------------------------------------------------------+
numa-vmstat.node0.nr_dirtied
58000 ++------------------------------------------------------------------+
57000 ++ O |
O O O O |
56000 ++ O O O O O O |
55000 ++ O O O O O O O O O O |
| O O |
54000 ++ |
53000 ++ |
52000 ++ |
| *.. * |
51000 +++ .*.. .*.. .. + .*.. *..|
50000 ++ *. *..*..*.. .* * + .*.. *. *.. .. *
* *.. .*..*. *. .. * |
49000 ++ *. * |
48000 ++------------------------------------------------------------------+
numa-vmstat.node0.nr_written
58000 ++------------------------------------------------------------------+
57000 ++ O |
O O O O O |
56000 ++ O O O O O O |
55000 ++ O O O O O O O O O |
| O O |
54000 ++ |
53000 ++ |
52000 ++ |
| * |
51000 ++ *.. .*.. .*.. .. + * .*.. *..|
50000 ++. *. *..*..*.. .* * + .. + *. *.. .. *
* *.. .*..*. * + .. * |
49000 ++ *. * |
48000 ++------------------------------------------------------------------+
numa-meminfo.node0.SUnreclaim
50000 ++------------------------------------------------------------------+
| |
| O |
45000 ++ O O O O O O |
O O O O O |
| O O O O O O O O |
40000 ++ O O O |
| |
35000 ++ |
| |
| *.. *.. |
30000 ++ .. *.. .. *.. .*.. .*
*..*..*.. .*.. .*..*.* * *..*. *..*. |
| *..*..*. .*. |
25000 ++-------------------*--*-------------------------------------------+
proc-vmstat.nr_slab_unreclaimable
17500 ++-O--O-----O--O-------------------O--O--------O-----O-----O-----O--+
17000 O+ O O O O O O O O O O O O |
| |
16500 ++ |
16000 ++ |
| |
15500 ++ |
15000 ++ |
14500 ++ |
| |
14000 ++ |
13500 ++ |
| |
13000 *+.*..*..*..*..*..*..*..*..*..*..*.*..*..*..*..*..*..*..*..*..*..*..*
12500 ++------------------------------------------------------------------+
meminfo.Slab
185000 ++-----------------------------------------------------------------+
| O O O O O |
180000 O+ O O O O O O O O O O O O O O O O O |
| |
| |
175000 ++ |
| |
170000 ++ |
| |
165000 ++ |
| |
| |
160000 *+.*..*..*..*..*.*..*.. .*..*..*..*..*.. .*.. .*..*..*..*..*..*
| *..*. *. * |
155000 ++-----------------------------------------------------------------+
meminfo.SUnreclaim
70000 ++-O--O-----O--O-------------------O--O--------O-----O-----O-----O--+
68000 O+ O O O O O O O O O O O O |
| |
66000 ++ |
64000 ++ |
| |
62000 ++ |
60000 ++ |
58000 ++ |
| |
56000 ++ |
54000 ++ |
| |
52000 *+.*..*..*..*..*..*..*..*..*..*..*.*..*..*..*..*..*..*..*..*..*..*..*
50000 ++------------------------------------------------------------------+
slabinfo.kmalloc-256.active_objs
25000 ++------------------------------------------------------------------+
| O O O O O O O O O O O O O O O O O O O O O |
O O |
20000 ++ |
| |
| |
15000 ++ |
| |
10000 ++ |
| |
| |
5000 ++ |
*..*..*..*..*..*..*..*..*..*..*..*.*..*..*..*..*..*..*..*..*..*..*..*
| |
0 ++------------------------------------------------------------------+
slabinfo.kmalloc-256.num_objs
24000 ++-O--O-----O--O--------O-----O--O-O--O--O--O--O--O--O-----O-----O--+
22000 O+ O O O O O O |
| |
20000 ++ |
18000 ++ |
| |
16000 ++ |
14000 ++ |
12000 ++ |
| |
10000 ++ |
8000 ++ |
| |
6000 ++ .*.. .*..*..*.. .*.. .*..|
4000 *+-*--*--*--*--*--*--*--*--*--*--*-*-----*-----------*--*-----*-----*
slabinfo.kmalloc-256.active_slabs
800 ++--------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O |
700 ++ O O O O |
| |
600 ++ |
| |
500 ++ |
| |
400 ++ |
| |
300 ++ |
| |
200 ++ |
*..*..*..*..*..*..*..*..*..*..*..*...*..*..*..*..*..*..*..*..*..*..*..*
100 ++--------------------------------------------------------------------+
slabinfo.kmalloc-256.num_slabs
800 ++--------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O |
700 ++ O O O O |
| |
600 ++ |
| |
500 ++ |
| |
400 ++ |
| |
300 ++ |
| |
200 ++ |
*..*..*..*..*..*..*..*..*..*..*..*...*..*..*..*..*..*..*..*..*..*..*..*
100 ++--------------------------------------------------------------------+
slabinfo.kmalloc-192.active_objs
70000 ++------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O O O O O |
60000 ++ |
| |
50000 ++ |
| |
40000 ++ |
| |
30000 ++ |
| |
20000 ++ |
| |
10000 *+. .*.. .*..|
| *..*..*..*..*..*..*..*..*..*..*.*..*..*..*..*. *..*..*..*. *
0 ++------------------------------------------------------------------+
slabinfo.kmalloc-192.num_objs
70000 ++------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O O O O O |
60000 ++ |
| |
50000 ++ |
| |
40000 ++ |
| |
30000 ++ |
| |
20000 ++ |
| |
10000 *+. .*.. .*..*..*. .*.. .*.. .*.. .*.. .*..*
| *. *..*..*..*..*..*. *. *. *. *..*. *. |
0 ++------------------------------------------------------------------+
slabinfo.kmalloc-192.active_slabs
1600 O+-O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--+
| |
1400 ++ |
1200 ++ |
| |
1000 ++ |
| |
800 ++ |
| |
600 ++ |
400 ++ |
| |
200 *+.*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*
| |
0 ++-------------------------------------------------------------------+
slabinfo.kmalloc-192.num_slabs
1600 O+-O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--+
| |
1400 ++ |
1200 ++ |
| |
1000 ++ |
| |
800 ++ |
| |
600 ++ |
400 ++ |
| |
200 *+.*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*
| |
0 ++-------------------------------------------------------------------+
slabinfo.kmalloc-32.active_objs
55000 ++------------------------------------------------------------------+
| O |
50000 O+ O O O |
45000 ++ |
| |
40000 ++ |
| |
35000 ++ |
| O O O O O O O O O |
30000 ++ O O O O O O O O O |
25000 ++ |
| |
20000 ++ |
*..*..*..*.. .*..*..*..*..*..*..*.*..*..*..*.. .*..*..*..*..*..*..*
15000 ++----------*----------------------------------*--------------------+
slabinfo.kmalloc-32.num_objs
55000 ++------------------------------------------------------------------+
O O O O |
50000 ++ O |
45000 ++ |
| |
40000 ++ |
| |
35000 ++ |
| O O O O O O O O O O O O O O O O |
30000 ++ O O |
25000 ++ |
| |
20000 ++ |
*..*..*..*.. .*..*..*..*..*..*..*.*..*..*..*.. .*..*..*..*..*..*..*
15000 ++----------*----------------------------------*--------------------+
slabinfo.kmalloc-32.active_slabs
400 O+-O--O--O--O---------------------------------------------------------+
| |
350 ++ |
| |
| |
300 ++ |
| |
250 ++ O O O O O O O O O O O O O O O O |
| O O |
200 ++ |
| |
| |
150 *+.*..*..*.. .*..*..*.. .*..*..*...*..*.. .*.. .*..*.. .*..*..*..*
| *. *. *. *. *. |
100 ++--------------------------------------------------------------------+
slabinfo.kmalloc-32.num_slabs
400 O+-O--O--O--O---------------------------------------------------------+
| |
350 ++ |
| |
| |
300 ++ |
| |
250 ++ O O O O O O O O O O O O O O O O |
| O O |
200 ++ |
| |
| |
150 *+.*..*..*.. .*..*..*.. .*..*..*...*..*.. .*.. .*..*.. .*..*..*..*
| *. *. *. *. *. |
100 ++--------------------------------------------------------------------+
kmsg.usb_usb7:can_t_set_config___error
1 ++--------------------------------------------------------------------*
| |
| :|
0.8 ++ :|
| :|
| :|
0.6 ++ : |
| : |
0.4 ++ : |
| : |
| : |
0.2 ++ : |
| : |
| : |
0 *+--*---*---*---*----*---*---*---*---*---*---*---*----*---*---*---*---+
[*] bisect-good sample
[O] bisect-bad sample
To reproduce:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Ying Huang
[-- Attachment #2: job.yaml --]
[-- Type: text/plain, Size: 3753 bytes --]
---
LKP_SERVER: inn
LKP_CGI_PORT: 80
LKP_CIFS_PORT: 139
testcase: fsmark
default-monitors:
wait: activate-monitor
kmsg:
uptime:
iostat:
vmstat:
numa-numastat:
numa-vmstat:
numa-meminfo:
proc-vmstat:
proc-stat:
interval: 10
meminfo:
slabinfo:
interrupts:
lock_stat:
latency_stats:
softirqs:
bdi_dev_mapping:
diskstats:
nfsstat:
cpuidle:
cpufreq-stats:
turbostat:
pmeter:
sched_debug:
interval: 60
cpufreq_governor: performance
default-watchdogs:
oom-killer:
watchdog:
commit: 31e1a90a1e09e8e600f1b0b5dcb871c399448f81
model: Nehalem-EP
memory: 12G
hdd_partitions: "/dev/disk/by-id/ata-ST3500514NS_9WJ03EBA-part3"
swap_partitions: "/dev/disk/by-id/ata-ST3120026AS_5MS07HA2-part2"
rootfs_partition: "/dev/disk/by-id/ata-ST3500514NS_9WJ03EBA-part1"
category: benchmark
iterations: 1x
nr_threads: 32t
disk: 1HDD
fs: xfs
fs2: nfsv4
fsmark:
filesize: 5K
test_size: 400M
sync_method: fsyncBeforeClose
nr_directories: 16d
nr_files_per_directory: 256fpd
queue: cyclic
testbox: lkp-ne04
tbox_group: lkp-ne04
kconfig: x86_64-rhel
enqueue_time: 2015-09-16 01:53:44.450007854 +08:00
id: fa22bba34d95d9194b42a923ea8e320612db9c82
user: lkp
compiler: gcc-4.9
head_commit: 31e1a90a1e09e8e600f1b0b5dcb871c399448f81
base_commit: 6ff33f3902c3b1c5d0db6b1e2c70b6d76fba357f
branch: linux-devel/devel-hourly-2015091604
kernel: "/pkg/linux/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/vmlinuz-4.3.0-rc1-wl-ath-00944-g31e1a90"
rootfs: debian-x86_64-2015-02-07.cgz
result_root: "/result/fsmark/performance-1x-32t-1HDD-xfs-nfsv4-5K-400M-fsyncBeforeClose-16d-256fpd/lkp-ne04/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/0"
job_file: "/lkp/scheduled/lkp-ne04/cyclic_fsmark-performance-1x-32t-1HDD-xfs-nfsv4-5K-400M-fsyncBeforeClose-16d-256fpd-x86_64-rhel-CYCLIC_HEAD-31e1a90a1e09e8e600f1b0b5dcb871c399448f81-20150916-88681-jka6un-0.yaml"
dequeue_time: 2015-09-16 05:10:38.959773945 +08:00
nr_cpu: "$(nproc)"
max_uptime: 1073.42
initrd: "/osimage/debian/debian-x86_64-2015-02-07.cgz"
bootloader_append:
- root=/dev/ram0
- user=lkp
- job=/lkp/scheduled/lkp-ne04/cyclic_fsmark-performance-1x-32t-1HDD-xfs-nfsv4-5K-400M-fsyncBeforeClose-16d-256fpd-x86_64-rhel-CYCLIC_HEAD-31e1a90a1e09e8e600f1b0b5dcb871c399448f81-20150916-88681-jka6un-0.yaml
- ARCH=x86_64
- kconfig=x86_64-rhel
- branch=linux-devel/devel-hourly-2015091604
- commit=31e1a90a1e09e8e600f1b0b5dcb871c399448f81
- BOOT_IMAGE=/pkg/linux/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/vmlinuz-4.3.0-rc1-wl-ath-00944-g31e1a90
- max_uptime=1073
- RESULT_ROOT=/result/fsmark/performance-1x-32t-1HDD-xfs-nfsv4-5K-400M-fsyncBeforeClose-16d-256fpd/lkp-ne04/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/0
- LKP_SERVER=inn
- |2-
earlyprintk=ttyS0,115200 systemd.log_level=err
debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100
panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0
console=ttyS0,115200 console=tty0 vga=normal
rw
lkp_initrd: "/lkp/lkp/lkp-x86_64.cgz"
modules_initrd: "/pkg/linux/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/modules.cgz"
bm_initrd: "/osimage/deps/debian-x86_64-2015-02-07.cgz/lkp.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/run-ipconfig.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/turbostat.cgz,/lkp/benchmarks/turbostat.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/fs.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/fs2.cgz,/lkp/benchmarks/fsmark.cgz"
job_state: finished
loadavg: 32.41 16.01 6.32 1/269 5937
start_time: '1442351486'
end_time: '1442351678'
version: "/lkp/lkp/.src-20150915-235159"
[-- Attachment #3: reproduce --]
[-- Type: text/plain, Size: 1944 bytes --]
echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu10/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu11/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu12/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu13/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu14/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu15/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu4/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu5/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu6/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu7/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu8/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu9/cpufreq/scaling_governor
mkfs -t xfs /dev/sda3
mount -t xfs -o nobarrier,inode64 /dev/sda3 /fs/sda3
/etc/init.d/rpcbind start
/etc/init.d/nfs-common start
/etc/init.d/nfs-kernel-server start
mount -t nfs -o vers=4 localhost:/fs/sda3 /nfs/sda3
./fs_mark -d /nfs/sda3/1 -d /nfs/sda3/2 -d /nfs/sda3/3 -d /nfs/sda3/4 -d /nfs/sda3/5 -d /nfs/sda3/6 -d /nfs/sda3/7 -d /nfs/sda3/8 -d /nfs/sda3/9 -d /nfs/sda3/10 -d /nfs/sda3/11 -d /nfs/sda3/12 -d /nfs/sda3/13 -d /nfs/sda3/14 -d /nfs/sda3/15 -d /nfs/sda3/16 -d /nfs/sda3/17 -d /nfs/sda3/18 -d /nfs/sda3/19 -d /nfs/sda3/20 -d /nfs/sda3/21 -d /nfs/sda3/22 -d /nfs/sda3/23 -d /nfs/sda3/24 -d /nfs/sda3/25 -d /nfs/sda3/26 -d /nfs/sda3/27 -d /nfs/sda3/28 -d /nfs/sda3/29 -d /nfs/sda3/30 -d /nfs/sda3/31 -d /nfs/sda3/32 -D 16 -N 256 -n 2560 -L 1 -S 1 -s 5120
^ permalink raw reply [flat|nested] 17+ messages in thread
* [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-09-28 6:49 ` kernel test robot
0 siblings, 0 replies; 17+ messages in thread
From: kernel test robot @ 2015-09-28 6:49 UTC (permalink / raw)
To: lkp
[-- Attachment #1: Type: text/plain, Size: 43407 bytes --]
FYI, we noticed the below changes on
=========================================================================================
tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
commit:
cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
4aac1bf05b053a201a4b392dd9a684fb2b7e6103
cd2d35ff27c4fda9 4aac1bf05b053a201a4b392dd9
---------------- --------------------------
%stddev %change %stddev
\ | \
14415356 ± 0% +2.6% 14788625 ± 1% fsmark.app_overhead
441.60 ± 0% -2.9% 428.80 ± 0% fsmark.files_per_sec
185.78 ± 0% +2.9% 191.26 ± 0% fsmark.time.elapsed_time
185.78 ± 0% +2.9% 191.26 ± 0% fsmark.time.elapsed_time.max
97472 ± 0% -2.8% 94713 ± 0% fsmark.time.involuntary_context_switches
3077117 ± 95% +251.2% 10805440 ±112% latency_stats.sum.nfs_wait_on_request.nfs_updatepage.nfs_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.nfs_file_write.__vfs_write.vfs_write.SyS_write.entry_SYSCALL_64_fastpath
12999 ± 0% +32.9% 17276 ± 0% proc-vmstat.nr_slab_unreclaimable
64568 ± 4% -14.8% 55032 ± 0% softirqs.RCU
51999 ± 0% +32.9% 69111 ± 0% meminfo.SUnreclaim
159615 ± 0% +13.5% 181115 ± 0% meminfo.Slab
3.75 ± 0% +3.3% 3.88 ± 1% turbostat.%Busy
77.25 ± 0% +6.5% 82.25 ± 0% turbostat.Avg_MHz
30813025 ± 2% -14.5% 26338527 ± 9% cpuidle.C1E-NHM.time
164180 ± 0% -28.9% 116758 ± 7% cpuidle.C1E-NHM.usage
1738 ± 2% -81.2% 326.75 ± 4% cpuidle.POLL.usage
29979 ± 2% +44.3% 43273 ± 4% numa-meminfo.node0.SUnreclaim
94889 ± 0% +19.8% 113668 ± 2% numa-meminfo.node0.Slab
22033 ± 3% +17.3% 25835 ± 7% numa-meminfo.node1.SUnreclaim
7404 ± 1% -2.7% 7206 ± 0% vmstat.io.bo
27121 ± 0% -4.8% 25817 ± 0% vmstat.system.cs
3025 ± 0% -13.5% 2615 ± 0% vmstat.system.in
50126 ± 1% +11.5% 55893 ± 1% numa-vmstat.node0.nr_dirtied
7494 ± 2% +44.3% 10818 ± 4% numa-vmstat.node0.nr_slab_unreclaimable
50088 ± 1% +11.6% 55900 ± 1% numa-vmstat.node0.nr_written
5507 ± 3% +17.3% 6458 ± 7% numa-vmstat.node1.nr_slab_unreclaimable
7164 ± 2% +275.2% 26885 ± 0% slabinfo.kmalloc-16.active_objs
7164 ± 2% +275.3% 26885 ± 0% slabinfo.kmalloc-16.num_objs
7367 ± 1% +787.7% 65401 ± 0% slabinfo.kmalloc-192.active_objs
179.00 ± 1% +771.8% 1560 ± 0% slabinfo.kmalloc-192.active_slabs
7537 ± 1% +770.0% 65572 ± 0% slabinfo.kmalloc-192.num_objs
179.00 ± 1% +771.8% 1560 ± 0% slabinfo.kmalloc-192.num_slabs
3631 ± 7% +522.3% 22600 ± 0% slabinfo.kmalloc-256.active_objs
145.50 ± 4% +398.1% 724.75 ± 0% slabinfo.kmalloc-256.active_slabs
4667 ± 4% +397.3% 23210 ± 0% slabinfo.kmalloc-256.num_objs
145.50 ± 4% +398.1% 724.75 ± 0% slabinfo.kmalloc-256.num_slabs
17448 ± 2% +75.6% 30643 ± 0% slabinfo.kmalloc-32.active_objs
137.50 ± 2% +76.5% 242.75 ± 0% slabinfo.kmalloc-32.active_slabs
17651 ± 2% +76.4% 31139 ± 0% slabinfo.kmalloc-32.num_objs
137.50 ± 2% +76.5% 242.75 ± 0% slabinfo.kmalloc-32.num_slabs
2387 ± 3% -10.7% 2132 ± 8% slabinfo.kmalloc-512.active_objs
491.25 ± 3% +33.9% 658.00 ± 11% slabinfo.numa_policy.active_objs
491.25 ± 3% +33.9% 658.00 ± 11% slabinfo.numa_policy.num_objs
2128 ± 9% +59.3% 3391 ± 34% sched_debug.cfs_rq[10]:/.exec_clock
18088 ± 17% +47.0% 26582 ± 29% sched_debug.cfs_rq[10]:/.min_vruntime
4326 ± 11% -22.2% 3368 ± 18% sched_debug.cfs_rq[5]:/.exec_clock
1459 ± 1% -10.8% 1302 ± 3% sched_debug.cpu#0.nr_uninterruptible
122217 ± 7% -18.6% 99447 ± 2% sched_debug.cpu#1.nr_switches
122732 ± 8% -18.5% 99972 ± 2% sched_debug.cpu#1.sched_count
45603 ± 10% -20.1% 36442 ± 2% sched_debug.cpu#1.sched_goidle
27004 ± 3% -18.9% 21895 ± 5% sched_debug.cpu#1.ttwu_local
15469 ± 5% +17.2% 18132 ± 6% sched_debug.cpu#10.nr_load_updates
78564 ± 8% +26.6% 99492 ± 5% sched_debug.cpu#10.nr_switches
78605 ± 8% +26.7% 99557 ± 4% sched_debug.cpu#10.sched_count
27470 ± 9% +24.7% 34268 ± 7% sched_debug.cpu#10.sched_goidle
38215 ± 1% +37.4% 52499 ± 13% sched_debug.cpu#10.ttwu_count
14816 ± 5% +22.8% 18196 ± 2% sched_debug.cpu#10.ttwu_local
19690 ± 21% -29.9% 13802 ± 15% sched_debug.cpu#11.nr_switches
54.25 ± 2% -47.5% 28.50 ± 25% sched_debug.cpu#11.nr_uninterruptible
19721 ± 21% -29.9% 13828 ± 15% sched_debug.cpu#11.sched_count
14545 ± 2% +15.4% 16779 ± 4% sched_debug.cpu#12.nr_load_updates
72087 ± 11% +27.9% 92204 ± 7% sched_debug.cpu#12.nr_switches
72126 ± 11% +28.1% 92422 ± 7% sched_debug.cpu#12.sched_count
25418 ± 13% +24.4% 31626 ± 7% sched_debug.cpu#12.sched_goidle
33399 ± 15% +38.5% 46255 ± 13% sched_debug.cpu#12.ttwu_count
51.25 ± 10% -39.0% 31.25 ± 21% sched_debug.cpu#13.nr_uninterruptible
2593 ± 11% -21.8% 2028 ± 10% sched_debug.cpu#13.ttwu_local
71266 ± 3% +20.1% 85620 ± 5% sched_debug.cpu#14.nr_switches
71306 ± 3% +20.4% 85827 ± 5% sched_debug.cpu#14.sched_count
24634 ± 3% +18.8% 29259 ± 4% sched_debug.cpu#14.sched_goidle
34625 ± 11% +19.9% 41506 ± 11% sched_debug.cpu#14.ttwu_count
13866 ± 3% +20.6% 16726 ± 5% sched_debug.cpu#14.ttwu_local
12683 ± 4% -14.7% 10817 ± 2% sched_debug.cpu#15.nr_load_updates
49.75 ± 6% -46.2% 26.75 ± 28% sched_debug.cpu#15.nr_uninterruptible
3374 ± 12% -28.1% 2427 ± 18% sched_debug.cpu#15.ttwu_local
186563 ± 5% -12.1% 163975 ± 4% sched_debug.cpu#2.nr_switches
-1324 ± -2% -16.0% -1111 ± -1% sched_debug.cpu#2.nr_uninterruptible
187499 ± 5% -11.2% 166447 ± 4% sched_debug.cpu#2.sched_count
67465 ± 7% -13.6% 58308 ± 6% sched_debug.cpu#2.sched_goidle
36525 ± 4% -14.6% 31193 ± 1% sched_debug.cpu#2.ttwu_local
23697 ± 5% -13.2% 20572 ± 9% sched_debug.cpu#3.nr_load_updates
128070 ± 1% -22.9% 98687 ± 5% sched_debug.cpu#3.nr_switches
129859 ± 2% -23.5% 99357 ± 4% sched_debug.cpu#3.sched_count
48833 ± 1% -23.7% 37243 ± 6% sched_debug.cpu#3.sched_goidle
61622 ± 3% -24.2% 46694 ± 5% sched_debug.cpu#3.ttwu_count
27510 ± 7% -20.6% 21840 ± 8% sched_debug.cpu#3.ttwu_local
81675 ± 7% -13.6% 70536 ± 1% sched_debug.cpu#4.ttwu_count
34076 ± 3% -12.9% 29683 ± 1% sched_debug.cpu#4.ttwu_local
124470 ± 4% -14.1% 106865 ± 8% sched_debug.cpu#5.sched_count
62502 ± 3% -20.8% 49519 ± 9% sched_debug.cpu#5.ttwu_count
26562 ± 0% -17.7% 21853 ± 10% sched_debug.cpu#5.ttwu_local
181661 ± 10% -15.1% 154229 ± 6% sched_debug.cpu#6.nr_switches
181937 ± 10% -13.5% 157379 ± 6% sched_debug.cpu#6.sched_count
66672 ± 14% -16.6% 55632 ± 9% sched_debug.cpu#6.sched_goidle
78296 ± 2% -10.2% 70346 ± 6% sched_debug.cpu#6.ttwu_count
33536 ± 1% -14.4% 28696 ± 1% sched_debug.cpu#6.ttwu_local
131463 ± 6% -17.0% 109140 ± 4% sched_debug.cpu#7.nr_switches
-32.25 ±-58% -100.8% 0.25 ±9467% sched_debug.cpu#7.nr_uninterruptible
133606 ± 7% -17.2% 110671 ± 4% sched_debug.cpu#7.sched_count
50986 ± 7% -16.6% 42525 ± 6% sched_debug.cpu#7.sched_goidle
61388 ± 2% -19.8% 49213 ± 5% sched_debug.cpu#7.ttwu_count
26637 ± 2% -21.8% 20837 ± 3% sched_debug.cpu#7.ttwu_local
12312 ± 3% +9.4% 13474 ± 4% sched_debug.cpu#8.nr_load_updates
53.50 ± 6% -44.9% 29.50 ± 27% sched_debug.cpu#9.nr_uninterruptible
2724 ± 15% -23.7% 2078 ± 26% sched_debug.cpu#9.ttwu_local
lkp-ne04: Nehalem-EP
Memory: 12G
cpuidle.POLL.usage
1800 ++----------*-----------------------------*-----*-----*--------*--*--+
*..*..*..*. .*..*..*..*..*..*..*..*. *. *. *..*. *
1600 ++ *. |
1400 ++ |
| |
1200 ++ |
| |
1000 ++ |
| |
800 ++ |
600 ++ |
| |
400 O+ |
| O O O O O O O O O O O O O O O O O O O O O O |
200 ++-------------------------------------------------------------------+
cpuidle.C1E-NHM.usage
190000 ++-----------------------------------------------------*-----------+
| : : |
180000 ++ : : |
170000 ++ .*. : : |
| .*.. .*.. .*. .*.. .*..*..*..*. * *..*..*..*
160000 *+ *. *. *..*..*. *..*..*. |
150000 ++ |
| |
140000 ++ |
130000 ++ O |
| |
120000 ++ O O O O |
110000 ++ O O O O O O O O O |
O O O O O O O O O |
100000 ++-----------------------------------------------------------------+
fsmark.files_per_sec
446 ++--------------------------------------------------------------------+
444 ++ *.. |
| .. |
442 *+.*..*..*..*..*..*.. *..*..*..*...*..*..*..*..*..*..* *..*..*..*
440 ++ .. |
438 ++ * |
436 ++ |
| |
434 ++ |
432 ++ |
430 ++ |
428 ++ O O O O O O O O O O O O O O O O O O |
| |
426 O+ O O O O |
424 ++--------------------------------------------------------------------+
fsmark.time.elapsed_time
193 ++--------------------------------------------------------------------+
O O O |
192 ++ O O O O O O O O O O |
191 ++ O O O O O O O O |
| O O |
190 ++ |
189 ++ |
| |
188 ++ |
187 ++ |
| .*.. |
186 *+. .*..*..*..*..*. *.. .*.. ..*.. .*.. .*.. .*.. *..*..*..*
185 ++ *. *. *. *. *. *. .. |
| * |
184 ++--------------------------------------------------------------------+
fsmark.time.elapsed_time.max
193 ++--------------------------------------------------------------------+
O O O |
192 ++ O O O O O O O O O O |
191 ++ O O O O O O O O |
| O O |
190 ++ |
189 ++ |
| |
188 ++ |
187 ++ |
| .*.. |
186 *+. .*..*..*..*..*. *.. .*.. ..*.. .*.. .*.. .*.. *..*..*..*
185 ++ *. *. *. *. *. *. .. |
| * |
184 ++--------------------------------------------------------------------+
fsmark.time.involuntary_context_switches
98500 ++------------------------------------------------------------------+
98000 ++ .* |
| *.. .*. + *.. .*..*..* *
97500 ++. *..*..*. + .. *..*.*..*..*..*..*..*..*. + +|
97000 *+ *..* + + |
| * |
96500 ++ |
96000 ++ |
95500 ++ |
| |
95000 ++ O O O O O O O |
94500 ++ O O O O O O O O O |
O O O O O |
94000 ++ O O |
93500 ++------------------------------------------------------------------+
vmstat.system.in
3100 ++-------------------------------------------------------------------+
3050 *+. .*.. .*.. .*..*.. .*..*.. |
| .*..*.. .*..*. *. *. *.. .*..*. .*..*..*..*
3000 ++ *. *. *. *. |
2950 ++ |
2900 ++ |
2850 ++ |
| |
2800 ++ |
2750 ++ |
2700 ++ |
2650 ++ O O |
| O O O O O O O O O O O |
2600 O+ O O O O O O O O O |
2550 ++-------------------------------------------------------------------+
numa-vmstat.node0.nr_slab_unreclaimable
12000 ++------------------------------------------------------------------+
| O |
11000 ++ O O O O O O |
O O O O O |
| O O O O O O O O |
10000 ++ O O O |
| |
9000 ++ |
| |
8000 ++ *.. |
| .*..*.. + .*.. .*
| .*.. .*.. .*.*. + *..*..*. *..*. |
7000 *+.*. .*. .*..*. * |
| *..*. *..*. |
6000 ++------------------------------------------------------------------+
numa-vmstat.node0.nr_dirtied
58000 ++------------------------------------------------------------------+
57000 ++ O |
O O O O |
56000 ++ O O O O O O |
55000 ++ O O O O O O O O O O |
| O O |
54000 ++ |
53000 ++ |
52000 ++ |
| *.. * |
51000 +++ .*.. .*.. .. + .*.. *..|
50000 ++ *. *..*..*.. .* * + .*.. *. *.. .. *
* *.. .*..*. *. .. * |
49000 ++ *. * |
48000 ++------------------------------------------------------------------+
numa-vmstat.node0.nr_written
58000 ++------------------------------------------------------------------+
57000 ++ O |
O O O O O |
56000 ++ O O O O O O |
55000 ++ O O O O O O O O O |
| O O |
54000 ++ |
53000 ++ |
52000 ++ |
| * |
51000 ++ *.. .*.. .*.. .. + * .*.. *..|
50000 ++. *. *..*..*.. .* * + .. + *. *.. .. *
* *.. .*..*. * + .. * |
49000 ++ *. * |
48000 ++------------------------------------------------------------------+
numa-meminfo.node0.SUnreclaim
50000 ++------------------------------------------------------------------+
| |
| O |
45000 ++ O O O O O O |
O O O O O |
| O O O O O O O O |
40000 ++ O O O |
| |
35000 ++ |
| |
| *.. *.. |
30000 ++ .. *.. .. *.. .*.. .*
*..*..*.. .*.. .*..*.* * *..*. *..*. |
| *..*..*. .*. |
25000 ++-------------------*--*-------------------------------------------+
proc-vmstat.nr_slab_unreclaimable
17500 ++-O--O-----O--O-------------------O--O--------O-----O-----O-----O--+
17000 O+ O O O O O O O O O O O O |
| |
16500 ++ |
16000 ++ |
| |
15500 ++ |
15000 ++ |
14500 ++ |
| |
14000 ++ |
13500 ++ |
| |
13000 *+.*..*..*..*..*..*..*..*..*..*..*.*..*..*..*..*..*..*..*..*..*..*..*
12500 ++------------------------------------------------------------------+
meminfo.Slab
185000 ++-----------------------------------------------------------------+
| O O O O O |
180000 O+ O O O O O O O O O O O O O O O O O |
| |
| |
175000 ++ |
| |
170000 ++ |
| |
165000 ++ |
| |
| |
160000 *+.*..*..*..*..*.*..*.. .*..*..*..*..*.. .*.. .*..*..*..*..*..*
| *..*. *. * |
155000 ++-----------------------------------------------------------------+
meminfo.SUnreclaim
70000 ++-O--O-----O--O-------------------O--O--------O-----O-----O-----O--+
68000 O+ O O O O O O O O O O O O |
| |
66000 ++ |
64000 ++ |
| |
62000 ++ |
60000 ++ |
58000 ++ |
| |
56000 ++ |
54000 ++ |
| |
52000 *+.*..*..*..*..*..*..*..*..*..*..*.*..*..*..*..*..*..*..*..*..*..*..*
50000 ++------------------------------------------------------------------+
slabinfo.kmalloc-256.active_objs
25000 ++------------------------------------------------------------------+
| O O O O O O O O O O O O O O O O O O O O O |
O O |
20000 ++ |
| |
| |
15000 ++ |
| |
10000 ++ |
| |
| |
5000 ++ |
*..*..*..*..*..*..*..*..*..*..*..*.*..*..*..*..*..*..*..*..*..*..*..*
| |
0 ++------------------------------------------------------------------+
slabinfo.kmalloc-256.num_objs
24000 ++-O--O-----O--O--------O-----O--O-O--O--O--O--O--O--O-----O-----O--+
22000 O+ O O O O O O |
| |
20000 ++ |
18000 ++ |
| |
16000 ++ |
14000 ++ |
12000 ++ |
| |
10000 ++ |
8000 ++ |
| |
6000 ++ .*.. .*..*..*.. .*.. .*..|
4000 *+-*--*--*--*--*--*--*--*--*--*--*-*-----*-----------*--*-----*-----*
slabinfo.kmalloc-256.active_slabs
800 ++--------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O |
700 ++ O O O O |
| |
600 ++ |
| |
500 ++ |
| |
400 ++ |
| |
300 ++ |
| |
200 ++ |
*..*..*..*..*..*..*..*..*..*..*..*...*..*..*..*..*..*..*..*..*..*..*..*
100 ++--------------------------------------------------------------------+
slabinfo.kmalloc-256.num_slabs
800 ++--------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O |
700 ++ O O O O |
| |
600 ++ |
| |
500 ++ |
| |
400 ++ |
| |
300 ++ |
| |
200 ++ |
*..*..*..*..*..*..*..*..*..*..*..*...*..*..*..*..*..*..*..*..*..*..*..*
100 ++--------------------------------------------------------------------+
slabinfo.kmalloc-192.active_objs
70000 ++------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O O O O O |
60000 ++ |
| |
50000 ++ |
| |
40000 ++ |
| |
30000 ++ |
| |
20000 ++ |
| |
10000 *+. .*.. .*..|
| *..*..*..*..*..*..*..*..*..*..*.*..*..*..*..*. *..*..*..*. *
0 ++------------------------------------------------------------------+
slabinfo.kmalloc-192.num_objs
70000 ++------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O O O O O |
60000 ++ |
| |
50000 ++ |
| |
40000 ++ |
| |
30000 ++ |
| |
20000 ++ |
| |
10000 *+. .*.. .*..*..*. .*.. .*.. .*.. .*.. .*..*
| *. *..*..*..*..*..*. *. *. *. *..*. *. |
0 ++------------------------------------------------------------------+
slabinfo.kmalloc-192.active_slabs
1600 O+-O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--+
| |
1400 ++ |
1200 ++ |
| |
1000 ++ |
| |
800 ++ |
| |
600 ++ |
400 ++ |
| |
200 *+.*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*
| |
0 ++-------------------------------------------------------------------+
slabinfo.kmalloc-192.num_slabs
1600 O+-O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--O--+
| |
1400 ++ |
1200 ++ |
| |
1000 ++ |
| |
800 ++ |
| |
600 ++ |
400 ++ |
| |
200 *+.*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*..*
| |
0 ++-------------------------------------------------------------------+
slabinfo.kmalloc-32.active_objs
55000 ++------------------------------------------------------------------+
| O |
50000 O+ O O O |
45000 ++ |
| |
40000 ++ |
| |
35000 ++ |
| O O O O O O O O O |
30000 ++ O O O O O O O O O |
25000 ++ |
| |
20000 ++ |
*..*..*..*.. .*..*..*..*..*..*..*.*..*..*..*.. .*..*..*..*..*..*..*
15000 ++----------*----------------------------------*--------------------+
slabinfo.kmalloc-32.num_objs
55000 ++------------------------------------------------------------------+
O O O O |
50000 ++ O |
45000 ++ |
| |
40000 ++ |
| |
35000 ++ |
| O O O O O O O O O O O O O O O O |
30000 ++ O O |
25000 ++ |
| |
20000 ++ |
*..*..*..*.. .*..*..*..*..*..*..*.*..*..*..*.. .*..*..*..*..*..*..*
15000 ++----------*----------------------------------*--------------------+
slabinfo.kmalloc-32.active_slabs
400 O+-O--O--O--O---------------------------------------------------------+
| |
350 ++ |
| |
| |
300 ++ |
| |
250 ++ O O O O O O O O O O O O O O O O |
| O O |
200 ++ |
| |
| |
150 *+.*..*..*.. .*..*..*.. .*..*..*...*..*.. .*.. .*..*.. .*..*..*..*
| *. *. *. *. *. |
100 ++--------------------------------------------------------------------+
slabinfo.kmalloc-32.num_slabs
400 O+-O--O--O--O---------------------------------------------------------+
| |
350 ++ |
| |
| |
300 ++ |
| |
250 ++ O O O O O O O O O O O O O O O O |
| O O |
200 ++ |
| |
| |
150 *+.*..*..*.. .*..*..*.. .*..*..*...*..*.. .*.. .*..*.. .*..*..*..*
| *. *. *. *. *. |
100 ++--------------------------------------------------------------------+
kmsg.usb_usb7:can_t_set_config___error
1 ++--------------------------------------------------------------------*
| |
| :|
0.8 ++ :|
| :|
| :|
0.6 ++ : |
| : |
0.4 ++ : |
| : |
| : |
0.2 ++ : |
| : |
| : |
0 *+--*---*---*---*----*---*---*---*---*---*---*---*----*---*---*---*---+
[*] bisect-good sample
[O] bisect-bad sample
To reproduce:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Ying Huang
[-- Attachment #2: job.yaml --]
[-- Type: text/plain, Size: 3753 bytes --]
---
LKP_SERVER: inn
LKP_CGI_PORT: 80
LKP_CIFS_PORT: 139
testcase: fsmark
default-monitors:
wait: activate-monitor
kmsg:
uptime:
iostat:
vmstat:
numa-numastat:
numa-vmstat:
numa-meminfo:
proc-vmstat:
proc-stat:
interval: 10
meminfo:
slabinfo:
interrupts:
lock_stat:
latency_stats:
softirqs:
bdi_dev_mapping:
diskstats:
nfsstat:
cpuidle:
cpufreq-stats:
turbostat:
pmeter:
sched_debug:
interval: 60
cpufreq_governor: performance
default-watchdogs:
oom-killer:
watchdog:
commit: 31e1a90a1e09e8e600f1b0b5dcb871c399448f81
model: Nehalem-EP
memory: 12G
hdd_partitions: "/dev/disk/by-id/ata-ST3500514NS_9WJ03EBA-part3"
swap_partitions: "/dev/disk/by-id/ata-ST3120026AS_5MS07HA2-part2"
rootfs_partition: "/dev/disk/by-id/ata-ST3500514NS_9WJ03EBA-part1"
category: benchmark
iterations: 1x
nr_threads: 32t
disk: 1HDD
fs: xfs
fs2: nfsv4
fsmark:
filesize: 5K
test_size: 400M
sync_method: fsyncBeforeClose
nr_directories: 16d
nr_files_per_directory: 256fpd
queue: cyclic
testbox: lkp-ne04
tbox_group: lkp-ne04
kconfig: x86_64-rhel
enqueue_time: 2015-09-16 01:53:44.450007854 +08:00
id: fa22bba34d95d9194b42a923ea8e320612db9c82
user: lkp
compiler: gcc-4.9
head_commit: 31e1a90a1e09e8e600f1b0b5dcb871c399448f81
base_commit: 6ff33f3902c3b1c5d0db6b1e2c70b6d76fba357f
branch: linux-devel/devel-hourly-2015091604
kernel: "/pkg/linux/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/vmlinuz-4.3.0-rc1-wl-ath-00944-g31e1a90"
rootfs: debian-x86_64-2015-02-07.cgz
result_root: "/result/fsmark/performance-1x-32t-1HDD-xfs-nfsv4-5K-400M-fsyncBeforeClose-16d-256fpd/lkp-ne04/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/0"
job_file: "/lkp/scheduled/lkp-ne04/cyclic_fsmark-performance-1x-32t-1HDD-xfs-nfsv4-5K-400M-fsyncBeforeClose-16d-256fpd-x86_64-rhel-CYCLIC_HEAD-31e1a90a1e09e8e600f1b0b5dcb871c399448f81-20150916-88681-jka6un-0.yaml"
dequeue_time: 2015-09-16 05:10:38.959773945 +08:00
nr_cpu: "$(nproc)"
max_uptime: 1073.42
initrd: "/osimage/debian/debian-x86_64-2015-02-07.cgz"
bootloader_append:
- root=/dev/ram0
- user=lkp
- job=/lkp/scheduled/lkp-ne04/cyclic_fsmark-performance-1x-32t-1HDD-xfs-nfsv4-5K-400M-fsyncBeforeClose-16d-256fpd-x86_64-rhel-CYCLIC_HEAD-31e1a90a1e09e8e600f1b0b5dcb871c399448f81-20150916-88681-jka6un-0.yaml
- ARCH=x86_64
- kconfig=x86_64-rhel
- branch=linux-devel/devel-hourly-2015091604
- commit=31e1a90a1e09e8e600f1b0b5dcb871c399448f81
- BOOT_IMAGE=/pkg/linux/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/vmlinuz-4.3.0-rc1-wl-ath-00944-g31e1a90
- max_uptime=1073
- RESULT_ROOT=/result/fsmark/performance-1x-32t-1HDD-xfs-nfsv4-5K-400M-fsyncBeforeClose-16d-256fpd/lkp-ne04/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/0
- LKP_SERVER=inn
- |2-
earlyprintk=ttyS0,115200 systemd.log_level=err
debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100
panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0
console=ttyS0,115200 console=tty0 vga=normal
rw
lkp_initrd: "/lkp/lkp/lkp-x86_64.cgz"
modules_initrd: "/pkg/linux/x86_64-rhel/gcc-4.9/31e1a90a1e09e8e600f1b0b5dcb871c399448f81/modules.cgz"
bm_initrd: "/osimage/deps/debian-x86_64-2015-02-07.cgz/lkp.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/run-ipconfig.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/turbostat.cgz,/lkp/benchmarks/turbostat.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/fs.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/fs2.cgz,/lkp/benchmarks/fsmark.cgz"
job_state: finished
loadavg: 32.41 16.01 6.32 1/269 5937
start_time: '1442351486'
end_time: '1442351678'
version: "/lkp/lkp/.src-20150915-235159"
[-- Attachment #3: reproduce.ksh --]
[-- Type: text/plain, Size: 1944 bytes --]
echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu10/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu11/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu12/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu13/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu14/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu15/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu4/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu5/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu6/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu7/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu8/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu9/cpufreq/scaling_governor
mkfs -t xfs /dev/sda3
mount -t xfs -o nobarrier,inode64 /dev/sda3 /fs/sda3
/etc/init.d/rpcbind start
/etc/init.d/nfs-common start
/etc/init.d/nfs-kernel-server start
mount -t nfs -o vers=4 localhost:/fs/sda3 /nfs/sda3
./fs_mark -d /nfs/sda3/1 -d /nfs/sda3/2 -d /nfs/sda3/3 -d /nfs/sda3/4 -d /nfs/sda3/5 -d /nfs/sda3/6 -d /nfs/sda3/7 -d /nfs/sda3/8 -d /nfs/sda3/9 -d /nfs/sda3/10 -d /nfs/sda3/11 -d /nfs/sda3/12 -d /nfs/sda3/13 -d /nfs/sda3/14 -d /nfs/sda3/15 -d /nfs/sda3/16 -d /nfs/sda3/17 -d /nfs/sda3/18 -d /nfs/sda3/19 -d /nfs/sda3/20 -d /nfs/sda3/21 -d /nfs/sda3/22 -d /nfs/sda3/23 -d /nfs/sda3/24 -d /nfs/sda3/25 -d /nfs/sda3/26 -d /nfs/sda3/27 -d /nfs/sda3/28 -d /nfs/sda3/29 -d /nfs/sda3/30 -d /nfs/sda3/31 -d /nfs/sda3/32 -D 16 -N 256 -n 2560 -L 1 -S 1 -s 5120
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
2015-09-28 6:49 ` kernel test robot
@ 2015-09-29 11:41 ` Jeff Layton
-1 siblings, 0 replies; 17+ messages in thread
From: Jeff Layton @ 2015-09-29 11:41 UTC (permalink / raw)
To: kernel test robot; +Cc: lkp, LKML
On Mon, 28 Sep 2015 14:49:32 +0800
kernel test robot <ying.huang@intel.com> wrote:
> FYI, we noticed the below changes on
>
> =========================================================================================
> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
>
> commit:
> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
>
A question...
I think my tree should now contain a fix for this, but with a
performance regression like this it's difficult to know for sure.
Is there some (automated) way to request that the KTR redo this test?
If not, will I get a note saying "problem seems to now be fixed" or do
I just take a lack of further emails from the KTR about this as a sign
that it's resolved?
Thanks!
--
Jeff Layton <jeff.layton@primarydata.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-09-29 11:41 ` Jeff Layton
0 siblings, 0 replies; 17+ messages in thread
From: Jeff Layton @ 2015-09-29 11:41 UTC (permalink / raw)
To: lkp
[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]
On Mon, 28 Sep 2015 14:49:32 +0800
kernel test robot <ying.huang@intel.com> wrote:
> FYI, we noticed the below changes on
>
> =========================================================================================
> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
>
> commit:
> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
>
A question...
I think my tree should now contain a fix for this, but with a
performance regression like this it's difficult to know for sure.
Is there some (automated) way to request that the KTR redo this test?
If not, will I get a note saying "problem seems to now be fixed" or do
I just take a lack of further emails from the KTR about this as a sign
that it's resolved?
Thanks!
--
Jeff Layton <jeff.layton@primarydata.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
2015-09-29 11:41 ` Jeff Layton
@ 2015-09-29 23:27 ` Huang, Ying
-1 siblings, 0 replies; 17+ messages in thread
From: Huang, Ying @ 2015-09-29 23:27 UTC (permalink / raw)
To: Jeff Layton; +Cc: kernel test robot, lkp, LKML
Jeff Layton <jeff.layton@primarydata.com> writes:
> On Mon, 28 Sep 2015 14:49:32 +0800
> kernel test robot <ying.huang@intel.com> wrote:
>
>> FYI, we noticed the below changes on
>>
>> =========================================================================================
>> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
>> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
>>
>> commit:
>> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
>> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
>>
>
> A question...
>
> I think my tree should now contain a fix for this, but with a
> performance regression like this it's difficult to know for sure.
>
> Is there some (automated) way to request that the KTR redo this test?
> If not, will I get a note saying "problem seems to now be fixed" or do
> I just take a lack of further emails from the KTR about this as a sign
> that it's resolved?
Can you provide the branch name and commit ID for your tree with fix? I
can confirm whether it is fixed for you.
Best Regards,
Huang, Ying
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-09-29 23:27 ` Huang, Ying
0 siblings, 0 replies; 17+ messages in thread
From: Huang, Ying @ 2015-09-29 23:27 UTC (permalink / raw)
To: lkp
[-- Attachment #1: Type: text/plain, Size: 1264 bytes --]
Jeff Layton <jeff.layton@primarydata.com> writes:
> On Mon, 28 Sep 2015 14:49:32 +0800
> kernel test robot <ying.huang@intel.com> wrote:
>
>> FYI, we noticed the below changes on
>>
>> =========================================================================================
>> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
>> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
>>
>> commit:
>> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
>> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
>>
>
> A question...
>
> I think my tree should now contain a fix for this, but with a
> performance regression like this it's difficult to know for sure.
>
> Is there some (automated) way to request that the KTR redo this test?
> If not, will I get a note saying "problem seems to now be fixed" or do
> I just take a lack of further emails from the KTR about this as a sign
> that it's resolved?
Can you provide the branch name and commit ID for your tree with fix? I
can confirm whether it is fixed for you.
Best Regards,
Huang, Ying
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
2015-09-29 23:27 ` Huang, Ying
@ 2015-09-30 0:06 ` Jeff Layton
-1 siblings, 0 replies; 17+ messages in thread
From: Jeff Layton @ 2015-09-30 0:06 UTC (permalink / raw)
To: Huang, Ying; +Cc: kernel test robot, lkp, LKML
On Wed, 30 Sep 2015 07:27:54 +0800
"Huang\, Ying" <ying.huang@linux.intel.com> wrote:
> Jeff Layton <jeff.layton@primarydata.com> writes:
>
> > On Mon, 28 Sep 2015 14:49:32 +0800
> > kernel test robot <ying.huang@intel.com> wrote:
> >
> >> FYI, we noticed the below changes on
> >>
> >> =========================================================================================
> >> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
> >> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
> >>
> >> commit:
> >> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
> >> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
> >>
> >
> > A question...
> >
> > I think my tree should now contain a fix for this, but with a
> > performance regression like this it's difficult to know for sure.
> >
> > Is there some (automated) way to request that the KTR redo this test?
> > If not, will I get a note saying "problem seems to now be fixed" or do
> > I just take a lack of further emails from the KTR about this as a sign
> > that it's resolved?
>
> Can you provide the branch name and commit ID for your tree with fix? I
> can confirm whether it is fixed for you.
>
Sure:
git://git.samba.org/jlayton/linux.git nfsd-4.4
The tip commit is ed3d7c1e01a76f5ecc7444067704a82af4c2f76e.
Many thanks!
--
Jeff Layton <jeff.layton@primarydata.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-09-30 0:06 ` Jeff Layton
0 siblings, 0 replies; 17+ messages in thread
From: Jeff Layton @ 2015-09-30 0:06 UTC (permalink / raw)
To: lkp
[-- Attachment #1: Type: text/plain, Size: 1568 bytes --]
On Wed, 30 Sep 2015 07:27:54 +0800
"Huang\, Ying" <ying.huang@linux.intel.com> wrote:
> Jeff Layton <jeff.layton@primarydata.com> writes:
>
> > On Mon, 28 Sep 2015 14:49:32 +0800
> > kernel test robot <ying.huang@intel.com> wrote:
> >
> >> FYI, we noticed the below changes on
> >>
> >> =========================================================================================
> >> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
> >> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
> >>
> >> commit:
> >> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
> >> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
> >>
> >
> > A question...
> >
> > I think my tree should now contain a fix for this, but with a
> > performance regression like this it's difficult to know for sure.
> >
> > Is there some (automated) way to request that the KTR redo this test?
> > If not, will I get a note saying "problem seems to now be fixed" or do
> > I just take a lack of further emails from the KTR about this as a sign
> > that it's resolved?
>
> Can you provide the branch name and commit ID for your tree with fix? I
> can confirm whether it is fixed for you.
>
Sure:
git://git.samba.org/jlayton/linux.git nfsd-4.4
The tip commit is ed3d7c1e01a76f5ecc7444067704a82af4c2f76e.
Many thanks!
--
Jeff Layton <jeff.layton@primarydata.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
2015-09-30 0:06 ` Jeff Layton
@ 2015-09-30 8:35 ` Huang, Ying
-1 siblings, 0 replies; 17+ messages in thread
From: Huang, Ying @ 2015-09-30 8:35 UTC (permalink / raw)
To: Jeff Layton; +Cc: kernel test robot, lkp, LKML
Jeff Layton <jeff.layton@primarydata.com> writes:
> On Wed, 30 Sep 2015 07:27:54 +0800
> "Huang\, Ying" <ying.huang@linux.intel.com> wrote:
>
>> Jeff Layton <jeff.layton@primarydata.com> writes:
>>
>> > On Mon, 28 Sep 2015 14:49:32 +0800
>> > kernel test robot <ying.huang@intel.com> wrote:
>> >
>> >> FYI, we noticed the below changes on
>> >>
>> >> =========================================================================================
>> >> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
>> >> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
>> >>
>> >> commit:
>> >> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
>> >> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
>> >>
>> >
>> > A question...
>> >
>> > I think my tree should now contain a fix for this, but with a
>> > performance regression like this it's difficult to know for sure.
>> >
>> > Is there some (automated) way to request that the KTR redo this test?
>> > If not, will I get a note saying "problem seems to now be fixed" or do
>> > I just take a lack of further emails from the KTR about this as a sign
>> > that it's resolved?
>>
>> Can you provide the branch name and commit ID for your tree with fix? I
>> can confirm whether it is fixed for you.
>>
> Sure:
>
> git://git.samba.org/jlayton/linux.git nfsd-4.4
>
> The tip commit is ed3d7c1e01a76f5ecc7444067704a82af4c2f76e.
>
It seems that the regression is fixed at that commit. Thanks!
=========================================================================================
tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
commit:
cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
4aac1bf05b053a201a4b392dd9a684fb2b7e6103
ed3d7c1e01a76f5ecc7444067704a82af4c2f76e
cd2d35ff27c4fda9 4aac1bf05b053a201a4b392dd9 ed3d7c1e01a76f5ecc74440677
---------------- -------------------------- --------------------------
%stddev %change %stddev %change %stddev
\ | \ | \
14415356 ± 0% +2.6% 14788625 ± 1% +4.1% 15008301 ± 0% fsmark.app_overhead
441.60 ± 0% -2.9% 428.80 ± 0% -0.4% 439.68 ± 0% fsmark.files_per_sec
185.78 ± 0% +2.9% 191.26 ± 0% +0.3% 186.37 ± 0% fsmark.time.elapsed_time
185.78 ± 0% +2.9% 191.26 ± 0% +0.3% 186.37 ± 0% fsmark.time.elapsed_time.max
97472 ± 0% -2.8% 94713 ± 0% -0.8% 96657 ± 0% fsmark.time.involuntary_context_switches
Best Regards,
Huang, Ying
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-09-30 8:35 ` Huang, Ying
0 siblings, 0 replies; 17+ messages in thread
From: Huang, Ying @ 2015-09-30 8:35 UTC (permalink / raw)
To: lkp
[-- Attachment #1: Type: text/plain, Size: 3061 bytes --]
Jeff Layton <jeff.layton@primarydata.com> writes:
> On Wed, 30 Sep 2015 07:27:54 +0800
> "Huang\, Ying" <ying.huang@linux.intel.com> wrote:
>
>> Jeff Layton <jeff.layton@primarydata.com> writes:
>>
>> > On Mon, 28 Sep 2015 14:49:32 +0800
>> > kernel test robot <ying.huang@intel.com> wrote:
>> >
>> >> FYI, we noticed the below changes on
>> >>
>> >> =========================================================================================
>> >> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
>> >> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
>> >>
>> >> commit:
>> >> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
>> >> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
>> >>
>> >
>> > A question...
>> >
>> > I think my tree should now contain a fix for this, but with a
>> > performance regression like this it's difficult to know for sure.
>> >
>> > Is there some (automated) way to request that the KTR redo this test?
>> > If not, will I get a note saying "problem seems to now be fixed" or do
>> > I just take a lack of further emails from the KTR about this as a sign
>> > that it's resolved?
>>
>> Can you provide the branch name and commit ID for your tree with fix? I
>> can confirm whether it is fixed for you.
>>
> Sure:
>
> git://git.samba.org/jlayton/linux.git nfsd-4.4
>
> The tip commit is ed3d7c1e01a76f5ecc7444067704a82af4c2f76e.
>
It seems that the regression is fixed at that commit. Thanks!
=========================================================================================
tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
commit:
cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
4aac1bf05b053a201a4b392dd9a684fb2b7e6103
ed3d7c1e01a76f5ecc7444067704a82af4c2f76e
cd2d35ff27c4fda9 4aac1bf05b053a201a4b392dd9 ed3d7c1e01a76f5ecc74440677
---------------- -------------------------- --------------------------
%stddev %change %stddev %change %stddev
\ | \ | \
14415356 ± 0% +2.6% 14788625 ± 1% +4.1% 15008301 ± 0% fsmark.app_overhead
441.60 ± 0% -2.9% 428.80 ± 0% -0.4% 439.68 ± 0% fsmark.files_per_sec
185.78 ± 0% +2.9% 191.26 ± 0% +0.3% 186.37 ± 0% fsmark.time.elapsed_time
185.78 ± 0% +2.9% 191.26 ± 0% +0.3% 186.37 ± 0% fsmark.time.elapsed_time.max
97472 ± 0% -2.8% 94713 ± 0% -0.8% 96657 ± 0% fsmark.time.involuntary_context_switches
Best Regards,
Huang, Ying
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
2015-09-30 8:35 ` Huang, Ying
@ 2015-09-30 10:03 ` Jeff Layton
-1 siblings, 0 replies; 17+ messages in thread
From: Jeff Layton @ 2015-09-30 10:03 UTC (permalink / raw)
To: Huang, Ying; +Cc: kernel test robot, lkp, LKML, bfields
On Wed, 30 Sep 2015 16:35:58 +0800
"Huang\, Ying" <ying.huang@linux.intel.com> wrote:
> Jeff Layton <jeff.layton@primarydata.com> writes:
>
> > On Wed, 30 Sep 2015 07:27:54 +0800
> > "Huang\, Ying" <ying.huang@linux.intel.com> wrote:
> >
> >> Jeff Layton <jeff.layton@primarydata.com> writes:
> >>
> >> > On Mon, 28 Sep 2015 14:49:32 +0800
> >> > kernel test robot <ying.huang@intel.com> wrote:
> >> >
> >> >> FYI, we noticed the below changes on
> >> >>
> >> >> =========================================================================================
> >> >> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
> >> >> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
> >> >>
> >> >> commit:
> >> >> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
> >> >> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
> >> >>
> >> >
> >> > A question...
> >> >
> >> > I think my tree should now contain a fix for this, but with a
> >> > performance regression like this it's difficult to know for sure.
> >> >
> >> > Is there some (automated) way to request that the KTR redo this test?
> >> > If not, will I get a note saying "problem seems to now be fixed" or do
> >> > I just take a lack of further emails from the KTR about this as a sign
> >> > that it's resolved?
> >>
> >> Can you provide the branch name and commit ID for your tree with fix? I
> >> can confirm whether it is fixed for you.
> >>
> > Sure:
> >
> > git://git.samba.org/jlayton/linux.git nfsd-4.4
> >
> > The tip commit is ed3d7c1e01a76f5ecc7444067704a82af4c2f76e.
> >
>
> It seems that the regression is fixed at that commit. Thanks!
>
> =========================================================================================
> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
>
> commit:
> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
> ed3d7c1e01a76f5ecc7444067704a82af4c2f76e
>
> cd2d35ff27c4fda9 4aac1bf05b053a201a4b392dd9 ed3d7c1e01a76f5ecc74440677
> ---------------- -------------------------- --------------------------
> %stddev %change %stddev %change %stddev
> \ | \ | \
> 14415356 ± 0% +2.6% 14788625 ± 1% +4.1% 15008301 ± 0% fsmark.app_overhead
> 441.60 ± 0% -2.9% 428.80 ± 0% -0.4% 439.68 ± 0% fsmark.files_per_sec
> 185.78 ± 0% +2.9% 191.26 ± 0% +0.3% 186.37 ± 0% fsmark.time.elapsed_time
> 185.78 ± 0% +2.9% 191.26 ± 0% +0.3% 186.37 ± 0% fsmark.time.elapsed_time.max
> 97472 ± 0% -2.8% 94713 ± 0% -0.8% 96657 ± 0% fsmark.time.involuntary_context_switches
>
> Best Regards,
> Huang, Ying
Thanks for testing it and catching the problem in the first place!
FWIW, the problem seems to have been bad hash distribution generated by
hash_ptr on struct inode pointers. When the cache had ~10000 entries in
it total, one of the hash chains had almost 2000 entries. When I
switched to hashing on inode->i_ino, the distribution was much better.
I'm not sure if it was just rotten luck or there is something about
inode pointers that makes hash_ptr generate a lot of duplicates. That
really could use more investigation...
--
Jeff Layton <jeff.layton@primarydata.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-09-30 10:03 ` Jeff Layton
0 siblings, 0 replies; 17+ messages in thread
From: Jeff Layton @ 2015-09-30 10:03 UTC (permalink / raw)
To: lkp
[-- Attachment #1: Type: text/plain, Size: 3867 bytes --]
On Wed, 30 Sep 2015 16:35:58 +0800
"Huang\, Ying" <ying.huang@linux.intel.com> wrote:
> Jeff Layton <jeff.layton@primarydata.com> writes:
>
> > On Wed, 30 Sep 2015 07:27:54 +0800
> > "Huang\, Ying" <ying.huang@linux.intel.com> wrote:
> >
> >> Jeff Layton <jeff.layton@primarydata.com> writes:
> >>
> >> > On Mon, 28 Sep 2015 14:49:32 +0800
> >> > kernel test robot <ying.huang@intel.com> wrote:
> >> >
> >> >> FYI, we noticed the below changes on
> >> >>
> >> >> =========================================================================================
> >> >> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
> >> >> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
> >> >>
> >> >> commit:
> >> >> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
> >> >> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
> >> >>
> >> >
> >> > A question...
> >> >
> >> > I think my tree should now contain a fix for this, but with a
> >> > performance regression like this it's difficult to know for sure.
> >> >
> >> > Is there some (automated) way to request that the KTR redo this test?
> >> > If not, will I get a note saying "problem seems to now be fixed" or do
> >> > I just take a lack of further emails from the KTR about this as a sign
> >> > that it's resolved?
> >>
> >> Can you provide the branch name and commit ID for your tree with fix? I
> >> can confirm whether it is fixed for you.
> >>
> > Sure:
> >
> > git://git.samba.org/jlayton/linux.git nfsd-4.4
> >
> > The tip commit is ed3d7c1e01a76f5ecc7444067704a82af4c2f76e.
> >
>
> It seems that the regression is fixed at that commit. Thanks!
>
> =========================================================================================
> tbox_group/testcase/rootfs/kconfig/compiler/cpufreq_governor/iterations/nr_threads/disk/fs/fs2/filesize/test_size/sync_method/nr_directories/nr_files_per_directory:
> lkp-ne04/fsmark/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/performance/1x/32t/1HDD/xfs/nfsv4/5K/400M/fsyncBeforeClose/16d/256fpd
>
> commit:
> cd2d35ff27c4fda9ba73b0aa84313e8e20ce4d2c
> 4aac1bf05b053a201a4b392dd9a684fb2b7e6103
> ed3d7c1e01a76f5ecc7444067704a82af4c2f76e
>
> cd2d35ff27c4fda9 4aac1bf05b053a201a4b392dd9 ed3d7c1e01a76f5ecc74440677
> ---------------- -------------------------- --------------------------
> %stddev %change %stddev %change %stddev
> \ | \ | \
> 14415356 ± 0% +2.6% 14788625 ± 1% +4.1% 15008301 ± 0% fsmark.app_overhead
> 441.60 ± 0% -2.9% 428.80 ± 0% -0.4% 439.68 ± 0% fsmark.files_per_sec
> 185.78 ± 0% +2.9% 191.26 ± 0% +0.3% 186.37 ± 0% fsmark.time.elapsed_time
> 185.78 ± 0% +2.9% 191.26 ± 0% +0.3% 186.37 ± 0% fsmark.time.elapsed_time.max
> 97472 ± 0% -2.8% 94713 ± 0% -0.8% 96657 ± 0% fsmark.time.involuntary_context_switches
>
> Best Regards,
> Huang, Ying
Thanks for testing it and catching the problem in the first place!
FWIW, the problem seems to have been bad hash distribution generated by
hash_ptr on struct inode pointers. When the cache had ~10000 entries in
it total, one of the hash chains had almost 2000 entries. When I
switched to hashing on inode->i_ino, the distribution was much better.
I'm not sure if it was just rotten luck or there is something about
inode pointers that makes hash_ptr generate a lot of duplicates. That
really could use more investigation...
--
Jeff Layton <jeff.layton@primarydata.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
2015-09-30 10:03 ` Jeff Layton
@ 2015-09-30 23:17 ` Dave Chinner
-1 siblings, 0 replies; 17+ messages in thread
From: Dave Chinner @ 2015-09-30 23:17 UTC (permalink / raw)
To: Jeff Layton; +Cc: Huang, Ying, kernel test robot, lkp, LKML, bfields
On Wed, Sep 30, 2015 at 06:03:59AM -0400, Jeff Layton wrote:
> Thanks for testing it and catching the problem in the first place!
>
> FWIW, the problem seems to have been bad hash distribution generated by
> hash_ptr on struct inode pointers. When the cache had ~10000 entries in
> it total, one of the hash chains had almost 2000 entries. When I
> switched to hashing on inode->i_ino, the distribution was much better.
>
> I'm not sure if it was just rotten luck or there is something about
> inode pointers that makes hash_ptr generate a lot of duplicates. That
> really could use more investigation...
Inode pointers have no entropy in the lower 9-10 bits because of
their size, and being allocated from a slab they are all going to
have the same set of values in the next 3-4 bits (i.e. offset into
the slab page which is defined by sizeof(inode)). Pointers also
have very similar upper bits, too, because they are all in kernel
memory.
hash_64 trys to fold all the entropy from the lower bits into into
the upper bits and then takes the result from the upper bits. Hence
if there is no entropy in either the lower or upper bits to start
with, then the hash may not end up with much entropy in it at all...
FWIW, see fs/inode.c::hash() to see how the fs code hashes inode
numbers (called from insert_inode_hash()). It's very different
because because inode numbers have the majority of their entropy in
the lower bits and (usually) none in the upper bits...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-09-30 23:17 ` Dave Chinner
0 siblings, 0 replies; 17+ messages in thread
From: Dave Chinner @ 2015-09-30 23:17 UTC (permalink / raw)
To: lkp
[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]
On Wed, Sep 30, 2015 at 06:03:59AM -0400, Jeff Layton wrote:
> Thanks for testing it and catching the problem in the first place!
>
> FWIW, the problem seems to have been bad hash distribution generated by
> hash_ptr on struct inode pointers. When the cache had ~10000 entries in
> it total, one of the hash chains had almost 2000 entries. When I
> switched to hashing on inode->i_ino, the distribution was much better.
>
> I'm not sure if it was just rotten luck or there is something about
> inode pointers that makes hash_ptr generate a lot of duplicates. That
> really could use more investigation...
Inode pointers have no entropy in the lower 9-10 bits because of
their size, and being allocated from a slab they are all going to
have the same set of values in the next 3-4 bits (i.e. offset into
the slab page which is defined by sizeof(inode)). Pointers also
have very similar upper bits, too, because they are all in kernel
memory.
hash_64 trys to fold all the entropy from the lower bits into into
the upper bits and then takes the result from the upper bits. Hence
if there is no entropy in either the lower or upper bits to start
with, then the hash may not end up with much entropy in it at all...
FWIW, see fs/inode.c::hash() to see how the fs code hashes inode
numbers (called from insert_inode_hash()). It's very different
because because inode numbers have the majority of their entropy in
the lower bits and (usually) none in the upper bits...
Cheers,
Dave.
--
Dave Chinner
david(a)fromorbit.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
2015-09-30 23:17 ` Dave Chinner
(?)
@ 2015-10-01 0:56 ` J. Bruce Fields
-1 siblings, 0 replies; 17+ messages in thread
From: J. Bruce Fields @ 2015-10-01 0:56 UTC (permalink / raw)
To: Dave Chinner; +Cc: Jeff Layton, Huang, Ying, kernel test robot, lkp, LKML
On Thu, Oct 01, 2015 at 09:17:42AM +1000, Dave Chinner wrote:
> Inode pointers have no entropy in the lower 9-10 bits because of
> their size, and being allocated from a slab they are all going to
> have the same set of values in the next 3-4 bits (i.e. offset into
> the slab page which is defined by sizeof(inode)). Pointers also
> have very similar upper bits, too, because they are all in kernel
> memory.
>
> hash_64 trys to fold all the entropy from the lower bits into into
> the upper bits and then takes the result from the upper bits. Hence
> if there is no entropy in either the lower or upper bits to start
> with, then the hash may not end up with much entropy in it at all...
So we have something hash_ptr() that turns out to be terrible at hashing
pointers? Argh.
(I understand you're saying this isn't necessarily the case for all
pointers, but inode pointers on their own seem likely to be a common
case, and there must be many more that are similar.)
--b.
>
> FWIW, see fs/inode.c::hash() to see how the fs code hashes inode
> numbers (called from insert_inode_hash()). It's very different
> because because inode numbers have the majority of their entropy in
> the lower bits and (usually) none in the upper bits...
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
2015-09-30 23:17 ` Dave Chinner
@ 2015-10-01 11:35 ` Jeff Layton
-1 siblings, 0 replies; 17+ messages in thread
From: Jeff Layton @ 2015-10-01 11:35 UTC (permalink / raw)
To: Dave Chinner; +Cc: Huang, Ying, kernel test robot, lkp, LKML, bfields
On Thu, 1 Oct 2015 09:17:42 +1000
Dave Chinner <david@fromorbit.com> wrote:
> On Wed, Sep 30, 2015 at 06:03:59AM -0400, Jeff Layton wrote:
> > Thanks for testing it and catching the problem in the first place!
> >
> > FWIW, the problem seems to have been bad hash distribution generated by
> > hash_ptr on struct inode pointers. When the cache had ~10000 entries in
> > it total, one of the hash chains had almost 2000 entries. When I
> > switched to hashing on inode->i_ino, the distribution was much better.
> >
> > I'm not sure if it was just rotten luck or there is something about
> > inode pointers that makes hash_ptr generate a lot of duplicates. That
> > really could use more investigation...
>
> Inode pointers have no entropy in the lower 9-10 bits because of
> their size, and being allocated from a slab they are all going to
> have the same set of values in the next 3-4 bits (i.e. offset into
> the slab page which is defined by sizeof(inode)). Pointers also
> have very similar upper bits, too, because they are all in kernel
> memory.
>
> hash_64 trys to fold all the entropy from the lower bits into into
> the upper bits and then takes the result from the upper bits. Hence
> if there is no entropy in either the lower or upper bits to start
> with, then the hash may not end up with much entropy in it at all...
>
> FWIW, see fs/inode.c::hash() to see how the fs code hashes inode
> numbers (called from insert_inode_hash()). It's very different
> because because inode numbers have the majority of their entropy in
> the lower bits and (usually) none in the upper bits...
>
Thanks for the explanation, Dave. That makes sense.
In hindsight I should have looked at how the vfs code hashes inodes in
its hashtable. Given that we're basically creating "shadow" inode
structures here that would probably work fairly well.
--
Jeff Layton <jeff.layton@primarydata.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec
@ 2015-10-01 11:35 ` Jeff Layton
0 siblings, 0 replies; 17+ messages in thread
From: Jeff Layton @ 2015-10-01 11:35 UTC (permalink / raw)
To: lkp
[-- Attachment #1: Type: text/plain, Size: 1935 bytes --]
On Thu, 1 Oct 2015 09:17:42 +1000
Dave Chinner <david@fromorbit.com> wrote:
> On Wed, Sep 30, 2015 at 06:03:59AM -0400, Jeff Layton wrote:
> > Thanks for testing it and catching the problem in the first place!
> >
> > FWIW, the problem seems to have been bad hash distribution generated by
> > hash_ptr on struct inode pointers. When the cache had ~10000 entries in
> > it total, one of the hash chains had almost 2000 entries. When I
> > switched to hashing on inode->i_ino, the distribution was much better.
> >
> > I'm not sure if it was just rotten luck or there is something about
> > inode pointers that makes hash_ptr generate a lot of duplicates. That
> > really could use more investigation...
>
> Inode pointers have no entropy in the lower 9-10 bits because of
> their size, and being allocated from a slab they are all going to
> have the same set of values in the next 3-4 bits (i.e. offset into
> the slab page which is defined by sizeof(inode)). Pointers also
> have very similar upper bits, too, because they are all in kernel
> memory.
>
> hash_64 trys to fold all the entropy from the lower bits into into
> the upper bits and then takes the result from the upper bits. Hence
> if there is no entropy in either the lower or upper bits to start
> with, then the hash may not end up with much entropy in it at all...
>
> FWIW, see fs/inode.c::hash() to see how the fs code hashes inode
> numbers (called from insert_inode_hash()). It's very different
> because because inode numbers have the majority of their entropy in
> the lower bits and (usually) none in the upper bits...
>
Thanks for the explanation, Dave. That makes sense.
In hindsight I should have looked at how the vfs code hashes inodes in
its hashtable. Given that we're basically creating "shadow" inode
structures here that would probably work fairly well.
--
Jeff Layton <jeff.layton@primarydata.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2015-10-01 11:35 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-09-28 6:49 [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec kernel test robot
2015-09-28 6:49 ` kernel test robot
2015-09-29 11:41 ` [lkp] " Jeff Layton
2015-09-29 11:41 ` Jeff Layton
2015-09-29 23:27 ` [lkp] " Huang, Ying
2015-09-29 23:27 ` Huang, Ying
2015-09-30 0:06 ` [lkp] " Jeff Layton
2015-09-30 0:06 ` Jeff Layton
2015-09-30 8:35 ` [lkp] " Huang, Ying
2015-09-30 8:35 ` Huang, Ying
2015-09-30 10:03 ` [lkp] " Jeff Layton
2015-09-30 10:03 ` Jeff Layton
2015-09-30 23:17 ` [lkp] " Dave Chinner
2015-09-30 23:17 ` Dave Chinner
2015-10-01 0:56 ` [lkp] " J. Bruce Fields
2015-10-01 11:35 ` Jeff Layton
2015-10-01 11:35 ` Jeff Layton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.