latency performance of 2.4 and 2.5...

latency performance of 2.4 and 2.5...

[Joern Nettingsmeier]

hi andrew, hi everyone !

as per your request, i fearlessly built and booted 2.5.41 with
preemption enabled.
although it was my first take at the dev kernel, it went very smoothly
:-D

anyway, it seems to perform worse than 2.4.20-pre1-lowlatency. it feels
very snappy on startup, but under load it is a lot less responsive to
user interaction.

i have run benno senoner's latencytest on both kernels. you can see the
results at 

	http://spunk.dnsalias.org/latencytest .

for me, the biggest problem seems to be soft-raid. i have my /var
partition striped over 2 scsi disks, and accessing them causes huge
latency peaks.
is there any chance to get this under control ?
i don't have a spare disk around, so i can't easily compare to non-raid,
but i'm quite sure it's the source of the problem, because people with
similar hardware are doing fine.

other than that, there were a few peaks under /proc stress, but they are
not easily reproducible.

i have not yet tested your mm-patches due to a compile failure (log is
attached below). i used
http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.41/2.5.41-mm3/2.5.41-mm3.gz.


if there's any patches to try or more benchmarks to run, let me know.

best,

jörn


***
Hardware: Dual-P3 @ 600Mhz, 512MB PC100 SDRAM, ASUS P2B-DS
one 4G IBM DCAS SCSI disk (/), 2 9G COMPAQ SCSI disks (/var, soft-RAID
0),
SCSI CDROM and CDR, no IDE devices.

Soundcards: SB Live (used during the tests) and Ensoniq 1371 (idle)
The audio file that was played back during the tests came over NFS.

***
make -f init/Makefile
  Generating include/linux/compile.h (updated)
  gcc -Wp,-MD,init/.version.o.d -D__KERNEL__ -Iinclude -Wall
-Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing -fno-common
-pipe -mpreferred-stack-boundary=2 -march=i686 -Iarch/i386/mach-generic
-fomit-frame-pointer -nostdinc -iwithprefix include   
-DKBUILD_BASENAME=version   -c -o init/version.o init/version.c
   ld -m elf_i386  -r -o init/built-in.o init/main.o init/version.o
init/do_mounts.o
        ld -m elf_i386 -e stext -T arch/i386/vmlinux.lds.s
arch/i386/kernel/head.o arch/i386/kernel/init_task.o  init/built-in.o
--start-group  arch/i386/kernel/built-in.o  arch/i386/mm/built-in.o 
arch/i386/mach-generic/built-in.o  kernel/built-in.o  mm/built-in.o 
fs/built-in.o  ipc/built-in.o  security/built-in.o  lib/lib.a 
arch/i386/lib/lib.a  drivers/built-in.o  sound/built-in.o 
arch/i386/pci/built-in.o  net/built-in.o --end-group  -o .tmp_vmlinux1
fs/built-in.o: In function `isofs_put_super':
fs/built-in.o(.text+0x390e5): undefined reference to `unload_nls'
fs/built-in.o: In function `isofs_fill_super':
fs/built-in.o(.text+0x39f0b): undefined reference to `load_nls'
fs/built-in.o(.text+0x39f25): undefined reference to `load_nls_default'
fs/built-in.o(.text+0x3a0c6): undefined reference to `unload_nls'
fs/built-in.o: In function `wcsntombs_be':
fs/built-in.o(.text+0x3c60b): undefined reference to `utf8_wctomb'
make: *** [.tmp_vmlinux1] Error 1




-- 
Jörn Nettingsmeier     
Kurfürstenstr 49, 45138 Essen, Germany      
http://spunk.dnsalias.org (my server)
http://www.linuxdj.com/audio/lad/ (Linux Audio Developers)

Re: [linux-audio-dev] latency performance of 2.4 and 2.5...

[Joern Nettingsmeier]

some new interesting results with 2.5.42:

http://spunk.dnsalias.org/latencytest/2.5.42/2x256.html

overall much worse, *but* greatly reduced latency peaks (max. 6 ms) as
compared to 2.5.41:

http://spunk.dnsalias.org/latencytest/2.5.41/2x256.html

here the peaks easily reach 13 ms.
i'm not really sure what to make of this....
can someone explain ?

andrew, it seems part of your mm patch was merged - is there an updated
patch around that will add the missing hunks ?

best,

jörn

-- 
Jörn Nettingsmeier     
Kurfürstenstr 49, 45138 Essen, Germany      
http://spunk.dnsalias.org (my server)
http://www.linuxdj.com/audio/lad/ (Linux Audio Developers)

Re: [linux-audio-dev] latency performance of 2.4 and 2.5...

[Andrew Morton]

Joern Nettingsmeier wrote:
> 
> some new interesting results with 2.5.42:
> 
> http://spunk.dnsalias.org/latencytest/2.5.42/2x256.html
> 
> overall much worse, *but* greatly reduced latency peaks (max. 6 ms) as
> compared to 2.5.41:
> 
> http://spunk.dnsalias.org/latencytest/2.5.41/2x256.html
> 
> here the peaks easily reach 13 ms.

Rather depends on the filesystem.  ext3 does its own write scheduling,
and does stuff inside lock_kernel().  It needs a couple of scheduling
points I guess.

I'd expect ext2 to work OK with preemption, but nobody has really
looked yet.  Unless you're using ftruncate() (grr.)

> i'm not really sure what to make of this....
> can someone explain ?
> 
> andrew, it seems part of your mm patch was merged - is there an updated
> patch around that will add the missing hunks ?

Well 2.5.41 had the big change to the truncate code, which was
a rather important part of this whole effort.  I can't immediately
think of a 2.5.42 change which would do this...

But nobody has really got down and dug out the toolkit on
2.5 scheduling latency.  I guess I should do that soon.

Re: [linux-audio-dev] latency performance of 2.4 and 2.5...

[Joern Nettingsmeier]

Andrew Morton wrote:
> 
> Joern Nettingsmeier wrote:
> >
> > some new interesting results with 2.5.42:
> >
> > http://spunk.dnsalias.org/latencytest/2.5.42/2x256.html
> >
> > overall much worse, *but* greatly reduced latency peaks (max. 6 ms) as
> > compared to 2.5.41:
> >
> > http://spunk.dnsalias.org/latencytest/2.5.41/2x256.html
> >
> > here the peaks easily reach 13 ms.
> 
> Rather depends on the filesystem.  ext3 does its own write scheduling,
> and does stuff inside lock_kernel().  It needs a couple of scheduling
> points I guess.
> 
> I'd expect ext2 to work OK with preemption, but nobody has really
> looked yet.  Unless you're using ftruncate() (grr.)

oh, i should have stated i'm using reiserfs on /, /usr and /var (var
being a softraid-0).


-- 
Jörn Nettingsmeier     
Kurfürstenstr 49, 45138 Essen, Germany      
http://spunk.dnsalias.org (my server)
http://www.linuxdj.com/audio/lad/ (Linux Audio Developers)

RE: [linux-audio-dev] latency performance of 2.4 and 2.5...

[Mark Knecht]

Jörn,
   I did a little work on this file system choice a couple of weeks ago. My
results said stay away from ext3. I'm now using reiserfs.

   There is a directory with some results on the same drive (different
partitions) using ext2, ext3 and reiserfs. ext3 and reiserfs also have data
on how ext2 performed on that partition, but it didn't matter. Each
partition was mostly the same.

http://www.controlnet.com/~makeMusic/disklatency/fstest/

   FYI - I had to drop Gnome as it had too many things going on in the
background. These results were done using fluxbox.

Mark

-----Original Message-----
From: linux-audio-dev-admin@music.columbia.edu
[mailto:linux-audio-dev-admin@music.columbia.edu]On Behalf Of Joern
Nettingsmeier
Sent: Monday, October 14, 2002 1:53 AM
To: Andrew Morton
Cc: linux-audio-dev@music.columbia.edu; linux-kernel@vger.kernel.org
Subject: Re: [linux-audio-dev] latency performance of 2.4 and 2.5...


Andrew Morton wrote:
>
> Joern Nettingsmeier wrote:
> >
> > some new interesting results with 2.5.42:
> >
> > http://spunk.dnsalias.org/latencytest/2.5.42/2x256.html
> >
> > overall much worse, *but* greatly reduced latency peaks (max. 6 ms) as
> > compared to 2.5.41:
> >
> > http://spunk.dnsalias.org/latencytest/2.5.41/2x256.html
> >
> > here the peaks easily reach 13 ms.
>
> Rather depends on the filesystem.  ext3 does its own write scheduling,
> and does stuff inside lock_kernel().  It needs a couple of scheduling
> points I guess.
>
> I'd expect ext2 to work OK with preemption, but nobody has really
> looked yet.  Unless you're using ftruncate() (grr.)

oh, i should have stated i'm using reiserfs on /, /usr and /var (var
being a softraid-0).


--
Jörn Nettingsmeier
Kurfürstenstr 49, 45138 Essen, Germany
http://spunk.dnsalias.org (my server)
http://www.linuxdj.com/audio/lad/ (Linux Audio Developers)