2.6.17-...: looong writeouts

2.6.17-...: looong writeouts

[Alexey Dobriyan]

Immediate problem: from time to time post 2.6.17 kernel [1] decides that it
really really needs disk.

	[kernel compilation goes as usual]
	  CC [M]  fs/xfs/xfs_inode.o
	[compilation blocks for ~10 seconds, disk LED is red]
	[then it continues]

Again, from time to time saving 2k file makes vi inoperational for same
period.

Scheduler is CFQ, fs is reiserfs mounted with noatime, notail. 2.6.17-rc
and 2.6.17 kernels were OK.

It occured only several times in 4 hours.

[1] 2.6.17-95eaa5fa8eb2c345244acd5f65b200b115ae8c65 to be precise


Probably related problem below:

During 2.6.17-rc cycle CFQ subjectively became less F.

	[   unpacking  ]
	[kernel tarball]
		.
		.
		.
		.
		.
		.
		.		[:wq on little file]
		.
		.
		.
		.
		.
		.
	[              ]

IIRC, on 2.6.16 that :wq took say 0.5 sec, on late 2.6.17-rc it was
several times slower. I don't have numbers but it was psychologically
noticeable, but not BFD.

Re: 2.6.17-...: looong writeouts

[Andrew Morton]

On Sat, 24 Jun 2006 19:15:31 +0400
Alexey Dobriyan <adobriyan@gmail.com> wrote:

> Immediate problem: from time to time post 2.6.17 kernel [1] decides that it
> really really needs disk.
> 
> 	[kernel compilation goes as usual]
> 	  CC [M]  fs/xfs/xfs_inode.o
> 	[compilation blocks for ~10 seconds, disk LED is red]
> 	[then it continues]
> 
> Again, from time to time saving 2k file makes vi inoperational for same
> period.
> 
> Scheduler is CFQ, fs is reiserfs mounted with noatime, notail. 2.6.17-rc
> and 2.6.17 kernels were OK.
> 
> It occured only several times in 4 hours.
> 
> [1] 2.6.17-95eaa5fa8eb2c345244acd5f65b200b115ae8c65 to be precise
> 
> 
> Probably related problem below:
> 
> During 2.6.17-rc cycle CFQ subjectively became less F.
> 
> 	[   unpacking  ]
> 	[kernel tarball]
> 		.
> 		.
> 		.
> 		.
> 		.
> 		.
> 		.		[:wq on little file]
> 		.
> 		.
> 		.
> 		.
> 		.
> 		.
> 	[              ]
> 
> IIRC, on 2.6.16 that :wq took say 0.5 sec, on late 2.6.17-rc it was
> several times slower. I don't have numbers but it was psychologically
> noticeable, but not BFD.
> 

There have been quite a few CFQ changes.

It'd help if you can come up with a simple test case which others can use
to reproduce this.  Say,

while true
do
	dd if=/dev/zero of=foo bs-1M count=1000 conv=notrunc
done &

versus

time dd if=/dev/zero of=bar bs=16k count=1 conv=fsync

or something like that.

Thanks.

Re: 2.6.17-...: looong writeouts

[Donald Parsons]

On Sat, 2006-06-24 at 20:11 -0400, Donald Parsons wrote:
> On Sat, 2006-06-24 at 21:36 +0400, Alexey Dobriyan wrote:
> > On Sat, Jun 24, 2006 at 06:50:52PM +0200, Arjan van de Ven wrote:
> > > just a random question to rule things out: can you check if laptop mode
> > > is enabled? (see the /proc/sys/vm/laptop_mode file). Laptop mode will
> > > have the effect of grouping writes together, so if that got enabled
> > > accidentally for some reason, that could explain the behavior you are
> > > seeing. (and it would narrow down the "what broke" search problem to
> > > something that is a lot easier to work on)
> > 
> > Mine is not laptop and I've never enabled it.
> > 
> 
> Mine is a laptop T43 running FC3 and the mode is not enabled:
> 
> # cat /proc/sys/vm/laptop_mode
> 0
> 
> I have rebooted to 2.6.17 and the problem is there too.  It
> happened when I tried to paste the above path from the mail.
> 
> Don

Another data point (now 2.6.17):

I started up Streamtuner to see if it was affected.  It is not.
While I was trying to get a window to respond for many seconds
I noticed sound continued on with no interference at all.

Don

Re: 2.6.17-...: looong writeouts

[Donald Parsons]

On Sat, 2006-06-24 at 21:36 +0400, Alexey Dobriyan wrote:
> On Sat, Jun 24, 2006 at 06:50:52PM +0200, Arjan van de Ven wrote:
> > just a random question to rule things out: can you check if laptop mode
> > is enabled? (see the /proc/sys/vm/laptop_mode file). Laptop mode will
> > have the effect of grouping writes together, so if that got enabled
> > accidentally for some reason, that could explain the behavior you are
> > seeing. (and it would narrow down the "what broke" search problem to
> > something that is a lot easier to work on)
> 
> Mine is not laptop and I've never enabled it.
> 

Mine is a laptop T43 running FC3 and the mode is not enabled:

# cat /proc/sys/vm/laptop_mode
0

I have rebooted to 2.6.17 and the problem is there too.  It
happened when I tried to paste the above path from the mail.

Don

Re: 2.6.17-...: looong writeouts

[Arjan van de Ven]

On Sat, 2006-06-24 at 21:36 +0400, Alexey Dobriyan wrote:
> On Sat, Jun 24, 2006 at 06:50:52PM +0200, Arjan van de Ven wrote:
> > just a random question to rule things out: can you check if laptop mode
> > is enabled? (see the /proc/sys/vm/laptop_mode file). Laptop mode will
> > have the effect of grouping writes together, so if that got enabled
> > accidentally for some reason, that could explain the behavior you are
> > seeing. (and it would narrow down the "what broke" search problem to
> > something that is a lot easier to work on)
> 
> Mine is not laptop and I've never enabled it.

but a script in your distro might have...

Re: 2.6.17-...: looong writeouts

[Alexey Dobriyan]

On Sat, Jun 24, 2006 at 06:50:52PM +0200, Arjan van de Ven wrote:
> just a random question to rule things out: can you check if laptop mode
> is enabled? (see the /proc/sys/vm/laptop_mode file). Laptop mode will
> have the effect of grouping writes together, so if that got enabled
> accidentally for some reason, that could explain the behavior you are
> seeing. (and it would narrow down the "what broke" search problem to
> something that is a lot easier to work on)

Mine is not laptop and I've never enabled it.

Re: 2.6.17-...: looong writeouts

[Arjan van de Ven]

On Sat, 2006-06-24 at 12:30 -0400, Donald Parsons wrote:
> I am seeing the same problem.  I had only noticed it
> since I installed Opera-9.0 and thought it was Opera's
> problem.  It even happened when I clicked to read this
> message at www.uwsg.iu.edu/hypermail/linux/kernel/0606.3/0091.html
> to which I am replying.  I had to wait for the disk
> access to end before the message link would display.


Hi,

just a random question to rule things out: can you check if laptop mode
is enabled? (see the /proc/sys/vm/laptop_mode file). Laptop mode will
have the effect of grouping writes together, so if that got enabled
accidentally for some reason, that could explain the behavior you are
seeing. (and it would narrow down the "what broke" search problem to
something that is a lot easier to work on)

Greetings,
   Arjan van de Ven

Re: 2.6.17-...: looong writeouts

[Donald Parsons]

I am seeing the same problem.  I had only noticed it
since I installed Opera-9.0 and thought it was Opera's
problem.  It even happened when I clicked to read this
message at www.uwsg.iu.edu/hypermail/linux/kernel/0606.3/0091.html
to which I am replying.  I had to wait for the disk
access to end before the message link would display.

Something serious has happened to Linux 2.6.17.1  It
appears I installed both  2.6.17.1 and Opera-9.0 on
June 20 and that is when the disk timeout was first
noticed.  (I compile and use most every git/rc kernel
so if the problem is seen by people without Opera then
it must have entered the kernel very close to 2.6.17.1)

Next time I boot I will go back to 2.6.17
(Too bad I deleted earlier rc[567]-gits).

Don