linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Re: [BENCHMARK] AIM7 fserver regressed in 2.5.70*
@ 2003-06-05 10:47 rwhron
  0 siblings, 0 replies; 4+ messages in thread
From: rwhron @ 2003-06-05 10:47 UTC (permalink / raw)
  To: akpm, piggin; +Cc: linux-kernel

On Wed, 4 Jun 2003 20:35:09 -0700 Andrew Morton wrote;
> I'd assume that the improvements would be wholly due to the
> IO controller changes?  Are you saying that there is something
> else involved?

I don't know.  The IO controller changes could be for redundancy,
or for performance.  It may be the kernel too.  I mention 
the controller changes because it's a real hardware difference.

On Thu, 05 Jun 2003 15:00:22 +1000 Nick Piggin wrote:
> I don't know what sort of disk IO fserver does, but it
> could be the same problem.

AIM7 dbase (improved in 2.5.70*) is about 11% synchronous IO, 
AIM7 fserver is about 2.5% synchronous IO.  
AIM7 dbase is about 15% regular IO, fserver is 30% regular IO.
AIM7 fserver has the additional creat-clo, dir_rtns_1 loads.
AIM7 fserver may be more seeky than the dbase load.

dbase workload:
load                    percentage of workload
add_int                 3.636
add_long                3.636
add_short               3.636
disk_rd                 7.273
disk_rr                 7.273
div_int                 1.818
div_long                1.818
div_short               1.818
jmp_test                1.818
mem_rtns_1              7.273
mem_rtns_2              7.273
mul_int                 1.818
mul_long                1.818
mul_short               1.818
page_test               7.273
ram_copy                3.636
shared_memory           7.273
sieve                   5.455
sort_rtns_1             5.455
stream_pipe             1.818
string_rtns             5.455
sync_disk_rw            5.455
sync_disk_update        5.455

fserver workload
load                    percentage of workload
add_int                 3.361
add_long                3.361
add_short               3.361
creat-clo               3.361
dir_rtns_1              3.361
disk_cp                 5.042
disk_rd                 5.042
disk_rr                 5.042
disk_rw                 5.042
disk_src                5.042
disk_wrt                5.042
div_int                 1.681
div_long                1.681
div_short               1.681
jmp_test                1.681
link_test               3.361
mem_rtns_1              6.723
mem_rtns_2              1.681
misc_rtns_1             3.361
mul_int                 1.681
mul_long                1.681
mul_short               1.681
ram_copy                3.361
signal_test             1.681
sort_rtns_1             5.042
string_rtns             5.042
sync_disk_cp            0.840
sync_disk_rw            0.840
sync_disk_wrt           0.840
tcp_test                1.681
udp_test                6.723




> If you could share the jobfile and means-to-reproduce I can
> take a look, thanks.

The underlying filesystem was ext2.


This is the AIM7 fserver workload:
# @(#) workfile.fserver:1.3 1/22/96 00:00:00
# Fileserver Mix
FILESIZE: 10M
POOLSIZE: 20M
20  add_int
20  add_long
20  add_short
20  creat-clo
20  dir_rtns_1
30  disk_cp
30  disk_rd
30  disk_rr
30  disk_rw
30  disk_src
30  disk_wrt
10  div_int
10  div_long
10  div_short
10  jmp_test
20  link_test
40  mem_rtns_1
10  mem_rtns_2
20  misc_rtns_1
10  mul_int
10  mul_long
10  mul_short
20  ram_copy
10  signal_test
30  sort_rtns_1
30  string_rtns
5   sync_disk_cp
5   sync_disk_rw
5   sync_disk_wrt
10  tcp_test
40  udp_test


The script to run AIM7 looks like this:

#!/bin/bash
# Run fserver once with load 4-32 increment by 4
export input=input.fserver
cat >$input<<EOF
$(hostname)
$(uname -r)-4xP3/3.75GB/SCSI-raid5
1
4
2
32
4
EOF



for n in fserver
do
        echo "AIM7 $w workload"
        cp -p workfile.${w} workfile
        # suite7.ss is appended to by default.
        # zero it out before run
        >suite7.ss
        # -N newtonian adapter for crossover
        # -t normal timing adapter
        # -nl no log
        # multitask -N -nl < $input
        SECONDS=0
        multitask -t < input.${w}
        echo "AIM7 $w completed in $SECONDS seconds"
        cp -p suite7.ss aimresults/suite7-${w}-$(uname -r).ss
done


iozone writes for a 64 k file has a small regression in 2.5.70.
rewrite, read, reread are improved in 2.5.70.

              KB  reclen   write rewrite    read    reread
2.5.70        64       4  227773  504003   695900   876083
2.5.70        64       8  309248  556588   810228   914439
2.5.70        64      16  321593  587073   790141   864828
2.5.70        64      32  304694  604016   820049   876206
2.5.70        64      64  239686  537800   842702   941127

              KB  reclen   write rewrite    read    reread
2.5.69-bk1    64       4  292251  488642   659659   820706
2.5.69-bk1    64       8  316922  547154   810303   913709
2.5.69-bk1    64      16  328129  586978   770969   877370
2.5.69-bk1    64      32  316906  621315   809981   941515
2.5.69-bk1    64      64  287051  639902   852688   970454


For a 5 MB file, iozone write regression is smaller.

                KB    reclen   write rewrite    read    reread
2.5.70        524288       4  153271  236749   311157   320316
2.5.70        524288       8  174955  241503   326604   336004
2.5.70        524288      16  180028  242359   342079   352546
2.5.70        524288      32  176625  235474   350006   360833
2.5.70        524288      64  165770  215484   354577   365812
2.5.70        524288     128  152281  191098   357021   368434
2.5.70        524288     256  141154  173406   353768   365583
2.5.70        524288     512  137654  168346   294428   302423
2.5.70        524288    1024  136510  167359   229127   233676
2.5.70        524288    2048  136923  167848   187121   190085
2.5.70        524288    4096  136880  167633   175851   178419
2.5.70        524288    8192  136938  167755   174985   177523
2.5.70        524288   16384  136868  167449   174699   177208


                KB    reclen   write rewrite    read    reread
2.5.69-bk1    524288       4  162156  229090   301785   310414
2.5.69-bk1    524288       8  172918  236818   321115   330003
2.5.69-bk1    524288      16  175961  238366   337755   348203
2.5.69-bk1    524288      32  172919  229970   346889   357655
2.5.69-bk1    524288      64  163741  211673   351416   362329
2.5.69-bk1    524288     128  150270  188487   354342   365391
2.5.69-bk1    524288     256  138487  170444   354947   366019
2.5.69-bk1    524288     512  136009  166210   288743   296204
2.5.69-bk1    524288    1024  134791  164132   222318   226508
2.5.69-bk1    524288    2048  134851  163984   184880   187753
2.5.69-bk1    524288    4096  134929  164422   173763   176235
2.5.69-bk1    524288    8192  134810  164287   173071   175598
2.5.69-bk1    524288   16384  134854  164087   173027   175449



-- 
Randy Hron
http://home.earthlink.net/~rwhron/kernel/bigbox.html


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [BENCHMARK] AIM7 fserver regressed in 2.5.70*
  2003-06-05  2:49 rwhron
  2003-06-05  3:35 ` Andrew Morton
@ 2003-06-05  5:00 ` Nick Piggin
  1 sibling, 0 replies; 4+ messages in thread
From: Nick Piggin @ 2003-06-05  5:00 UTC (permalink / raw)
  To: rwhron; +Cc: linux-kernel, Andrew Morton, Joel Becker



rwhron@earthlink.net wrote:

>Summary:
>AIM7 fileserver workload behaviour changed with 2.5.70.
>At low task counts (load average), 2.5.70* takes 40% 
>longer than 2.5.69.  As task count increases, regression
>disappears.
>
>Hardware has (4) 700 mhz P3 Xeons.
>3.75 GB RAM
>RAID 0 LUN (hardware raid)
>
>Background:
>AIM7 fserver is the only regressed workload.  In general, 
>2.5.70* has better numbers than 2.5.69* for a variety of
>benchmarks.
>
>
[snip]

>AIM7 fserver workload
>kernel             Tasks  Jobs/Min      Real       CPU
>2.5.69               4	 120.9	      200.5	  32.8
>2.5.69-bk1           4	 122.3	      198.2	  33.8
>2.5.69-mm3           4	 122.3	      198.3	  37.9
>2.5.69-mm5           4	 124.0	      195.5	  38.0
>
      ^^^^^^
I think this was the last kernel Joel tested before a
similar magnitude dropoff in WimMark performance.

>
>2.5.70               4	  79.0	      306.9	  34.2
>2.5.70-mjb1          4	  83.4	      290.8	  33.6
>2.5.70-mm3           4	  71.7	      338.0	  34.9
>2.5.70-mm4	     4    73.9        328.0       33.9
> 
>

I don't know what sort of disk IO fserver does, but it
could be the same problem.


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [BENCHMARK] AIM7 fserver regressed in 2.5.70*
  2003-06-05  2:49 rwhron
@ 2003-06-05  3:35 ` Andrew Morton
  2003-06-05  5:00 ` Nick Piggin
  1 sibling, 0 replies; 4+ messages in thread
From: Andrew Morton @ 2003-06-05  3:35 UTC (permalink / raw)
  To: rwhron; +Cc: linux-kernel

rwhron@earthlink.net wrote:
>
> Summary:
>  AIM7 fileserver workload behaviour changed with 2.5.70.
>  At low task counts (load average), 2.5.70* takes 40% 
>  longer than 2.5.69.  As task count increases, regression
>  disappears.
> 
>  Hardware has (4) 700 mhz P3 Xeons.
>  3.75 GB RAM
>  RAID 0 LUN (hardware raid)
> 
>  Background:
>  AIM7 fserver is the only regressed workload.  In general, 
>  2.5.70* has better numbers than 2.5.69* for a variety of
>  benchmarks.
> 
>  Part of the improvement in 2.5.70 I/O benchmarks is
>  from a fiber channel configuration change.  2.5.70* has
>  two online fiber channels.  Earlier kernels had only one
>  fiber channel online.  
> 
>  Tiobench and bonnie++ show about 10% improvement.  
>  LMbench microbenchmarks are generally improving or stable
>  in recent 2.5.x.  

I'd assume that the improvements would be wholly due to the
IO controller changes?  Are you saying that there is something
else involved?

If you could share the jobfile and means-to-reproduce I can
take a look, thanks.


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [BENCHMARK] AIM7 fserver regressed in 2.5.70*
@ 2003-06-05  2:49 rwhron
  2003-06-05  3:35 ` Andrew Morton
  2003-06-05  5:00 ` Nick Piggin
  0 siblings, 2 replies; 4+ messages in thread
From: rwhron @ 2003-06-05  2:49 UTC (permalink / raw)
  To: linux-kernel


Summary:
AIM7 fileserver workload behaviour changed with 2.5.70.
At low task counts (load average), 2.5.70* takes 40% 
longer than 2.5.69.  As task count increases, regression
disappears.

Hardware has (4) 700 mhz P3 Xeons.
3.75 GB RAM
RAID 0 LUN (hardware raid)

Background:
AIM7 fserver is the only regressed workload.  In general, 
2.5.70* has better numbers than 2.5.69* for a variety of
benchmarks.

Part of the improvement in 2.5.70 I/O benchmarks is
from a fiber channel configuration change.  2.5.70* has
two online fiber channels.  Earlier kernels had only one
fiber channel online.  

Tiobench and bonnie++ show about 10% improvement.  
LMbench microbenchmarks are generally improving or stable
in recent 2.5.x.  

So, it's strange that AIM7 fserver is regressed.  

The kernels below are listed in chronological order.

AIM7 database workloads show a nice 27-30% improvement
with 2.5.70*.

Real and CPU are time in seconds.

AIM7 dbase workload
kernel             Tasks  Jobs/Min      Real       CPU
2.5.69              32	 477.6	      398.0	 149.3
2.5.69-bk1          32	 476.3	      399.0	 143.1
2.5.69-mm3          32	 560.5	      339.1	 159.3
2.5.69-mm5          32	 560.9	      338.9	 164.0
2.5.70              32	 611.1	      311.0	 153.0
2.5.70-mjb1         32	 606.4	      313.5	 159.2
2.5.70-mm3          32	 685.8	      277.2	 161.8


2.5.69             256	 769.9	     1975.2	 982.2
2.5.69-bk1         256	 768.0	     1979.9	 977.9
2.5.69-mm3         256	 906.5	     1677.5	1132.8
2.5.69-mm5         256	 909.5	     1671.9	1088.4
2.5.70             256	1042.7	     1458.4	1036.3
2.5.70-mjb1        256	1030.0	     1476.4	1049.0
2.5.70-mm3         256	1186.4	     1281.8	1066.8


AIM7 fileserver is regressed about 40% at 4 tasks.
As the task load increases, the regression becomes less.
At 32 tasks, 2.5.70* is even or ahead of 2.5.69*.

AIM7 fserver workload
kernel             Tasks  Jobs/Min      Real       CPU
2.5.69               4	 120.9	      200.5	  32.8
2.5.69-bk1           4	 122.3	      198.2	  33.8
2.5.69-mm3           4	 122.3	      198.3	  37.9
2.5.69-mm5           4	 124.0	      195.5	  38.0
2.5.70               4	  79.0	      306.9	  34.2
2.5.70-mjb1          4	  83.4	      290.8	  33.6
2.5.70-mm3           4	  71.7	      338.0	  34.9
2.5.70-mm4	     4    73.9        328.0       33.9
 
 
2.5.69               8	 174.7	      277.5  	  61.1
2.5.69-bk1           8	 175.8	      275.8	  64.1
2.5.69-mm3           8	 179.4	      270.2	  65.7
2.5.69-mm5           8	 184.3	      263.0	  66.3
2.5.70               8	 136.6	      354.9	  58.8
2.5.70-mjb1          8	 137.3	      353.0	  57.0
2.5.70-mm3           8	 123.9	      391.3	  58.7
2.5.70-mm4	     8   118.4        409.4       57.5
 

2.5.69              32	 234.3	      827.6	 221.8
2.5.69-bk1          32	 236.0	      821.8	 220.8
2.5.69-mm3          32	 253.8	      764.0	 246.6
2.5.69-mm5          32	 254.8	      761.0	 248.3
2.5.70              32	 239.7	      809.1	 219.4
2.5.70-mjb1         32	 248.9	      779.2	 226.3
2.5.70-mm3          32	 231.3	      838.4	 224.9

AIM7 shared has a similar behavior.  At 64 tasks, 2.5.69*
is close to or a little ahead of 2.5.70*.

AIM7 shared workload
kernel             Tasks  Jobs/Min      Real       CPU
2.5.69              64	2121.2	      175.6	 167.5
2.5.69-bk1          64	2096.7	      177.7	 168.6
2.5.69-mm3          64	2422.3	      153.8	 179.2
2.5.69-mm5          64	2429.0	      153.3	 178.3
2.5.70              64	2123.1	      175.4	 170.0
2.5.70-mjb1         64	2163.9	      172.1	 175.5
2.5.70-mm3          64	2186.9	      170.3	 175.7

2.5.69             128	2257.8	      329.9	 333.9
2.5.69-bk1         128	2269.9	      328.2	 333.1
2.5.69-mm3         128	2700.6	      275.9	 352.9
2.5.69-mm5         128	2697.8	      276.1	 352.8
2.5.70             128	2410.4	      309.1	 338.2
2.5.70-mjb1        128	2580.1	      288.7	 354.7
2.5.70-mm3         128	2705.2	      275.4	 350.6

By 512 tasks, 2.5.70* is AIM7 shared is ahead of 2.5.69*
by 14-20%.

2.5.69             512	2314.7	     1287.3	1369.2
2.5.69-bk1         512	2319.4	     1284.7	1370.5
2.5.69-mm3         512	2574.1	     1157.6	1457.7
2.5.69-mm5         512	2698.3	     1104.3	1481.0
2.5.70             512	2788.0	     1068.8	1399.1
2.5.70-mjb1        512	2607.6	     1142.8	1670.8
2.5.70-mm3         512	3075.9	      968.8	1462.8


-- 
Randy Hron
http://home.earthlink.net/~rwhron/kernel/bigbox.html


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2003-06-05 10:32 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2003-06-05 10:47 [BENCHMARK] AIM7 fserver regressed in 2.5.70* rwhron
  -- strict thread matches above, loose matches on Subject: below --
2003-06-05  2:49 rwhron
2003-06-05  3:35 ` Andrew Morton
2003-06-05  5:00 ` Nick Piggin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).