* reiserfs: still problems with tail conversion
@ 2001-02-23 21:18 Erik Mouw
2001-02-23 22:10 ` Chris Mason
` (2 more replies)
0 siblings, 3 replies; 24+ messages in thread
From: Erik Mouw @ 2001-02-23 21:18 UTC (permalink / raw)
To: Linux kernel mailing list; +Cc: mason
Hi all,
I am running linux-2.4.2-pre4 with Chris Mason's tailconversion bug fix
applied, but I still have problems with null bytes in files. I wrote a
little test program that clearly shows the problem:
/* reisertest.c: test for tailconversion bug in reiserfs
*
* Compile with: gcc -O2 -o reisertest reisertest.c
*/
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#define MAXBYTES 8192
int main(int argc, char *argv[])
{
int fd;
int i;
char name[32];
char buf[MAXBYTES];
char check[MAXBYTES];
memset(buf, 0x55, MAXBYTES);
fprintf(stderr, "Creating %d files ... ", MAXBYTES);
for(i = 0; i < MAXBYTES; i++) {
sprintf(name, "reiser-%05d.test", i);
fd = open(name, O_WRONLY | O_CREAT, 0644);
write(fd, buf, i);
close(fd);
}
fprintf(stderr, "done\n");
fprintf(stderr, "Appending to the files ... ");
for(i = 0; i < MAXBYTES; i++) {
sprintf(name, "reiser-%05d.test", i);
fd = open(name, O_WRONLY | O_APPEND);
write(fd, buf, MAXBYTES - i);
close(fd);
}
fprintf(stderr, "done\n");
fprintf(stderr, "Checking files for null bytes ...\n");
for(i = 0; i < MAXBYTES; i++) {
sprintf(name, "reiser-%05d.test", i);
fd = open(name, O_RDONLY);
read(fd, check, MAXBYTES);
if(memcmp(buf, check, MAXBYTES) != 0)
fprintf(stderr, " %s contains null bytes\n", name);
}
fprintf(stderr, "Checking done\n");
return 0;
}
When I run this on a reiserfs partition, I get output like this:
erik@arthur:~/reisertest/foo> ../reisertest
Creating 8192 files ... done
Appending to the files ... done
Checking files for null bytes ...
reiser-00193.test contains null bytes
reiser-00220.test contains null bytes
reiser-00256.test contains null bytes
reiser-00289.test contains null bytes
reiser-00329.test contains null bytes
reiser-00338.test contains null bytes
reiser-00374.test contains null bytes
reiser-00407.test contains null bytes
reiser-00415.test contains null bytes
reiser-00430.test contains null bytes
reiser-00438.test contains null bytes
reiser-00445.test contains null bytes
reiser-00459.test contains null bytes
reiser-00481.test contains null bytes
reiser-00501.test contains null bytes
reiser-00508.test contains null bytes
reiser-00521.test contains null bytes
reiser-00534.test contains null bytes
reiser-00558.test contains null bytes
reiser-00577.test contains null bytes
reiser-00583.test contains null bytes
reiser-00600.test contains null bytes
reiser-00606.test contains null bytes
reiser-00612.test contains null bytes
reiser-00623.test contains null bytes
reiser-00634.test contains null bytes
reiser-00645.test contains null bytes
reiser-00665.test contains null bytes
reiser-00685.test contains null bytes
reiser-00730.test contains null bytes
reiser-00735.test contains null bytes
reiser-00740.test contains null bytes
reiser-00745.test contains null bytes
reiser-00750.test contains null bytes
reiser-00759.test contains null bytes
reiser-00764.test contains null bytes
reiser-00773.test contains null bytes
reiser-00778.test contains null bytes
reiser-00787.test contains null bytes
reiser-00796.test contains null bytes
reiser-00805.test contains null bytes
reiser-00814.test contains null bytes
reiser-00866.test contains null bytes
reiser-00915.test contains null bytes
reiser-00930.test contains null bytes
reiser-00934.test contains null bytes
reiser-00938.test contains null bytes
reiser-00942.test contains null bytes
reiser-00946.test contains null bytes
reiser-00950.test contains null bytes
reiser-00954.test contains null bytes
reiser-00958.test contains null bytes
reiser-00965.test contains null bytes
reiser-00969.test contains null bytes
reiser-00973.test contains null bytes
reiser-00977.test contains null bytes
reiser-00984.test contains null bytes
reiser-00988.test contains null bytes
reiser-00995.test contains null bytes
reiser-00999.test contains null bytes
reiser-01006.test contains null bytes
reiser-01010.test contains null bytes
reiser-01017.test contains null bytes
Checking done
Running the test a couple of times doesn't really show a pattern,
sometimes the same files contains null bytes, sometimes others do. The
files with null bytes seem to be with index < 1024.
I did the same test with an ext2 filesystem, but didn't see any error.
System is SuSE 7.0, compiler gcc-2.95.2.
Erik
--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031, 2600 GA Delft, The Netherlands
Phone: +31-15-2783635 Fax: +31-15-2781843 Email: J.A.K.Mouw@its.tudelft.nl
WWW: http://www-ict.its.tudelft.nl/~erik/
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: reiserfs: still problems with tail conversion
2001-02-23 21:18 reiserfs: still problems with tail conversion Erik Mouw
@ 2001-02-23 22:10 ` Chris Mason
2001-02-23 22:19 ` Erik Mouw
2001-02-24 0:34 ` Oliver Teuber
[not found] ` <01022318321302.01755@flash>
2001-02-24 15:45 ` Arjan Filius
2 siblings, 2 replies; 24+ messages in thread
From: Chris Mason @ 2001-02-23 22:10 UTC (permalink / raw)
To: Erik Mouw, Linux kernel mailing list
On Friday, February 23, 2001 10:18:56 PM +0100 Erik Mouw
<J.A.K.Mouw@ITS.TUDelft.NL> wrote:
> Hi all,
>
> I am running linux-2.4.2-pre4 with Chris Mason's tailconversion bug fix
> applied, but I still have problems with null bytes in files. I wrote a
> little test program that clearly shows the problem:
>
Many thanks for sending along a test program for reproducing. But, it
doesn't seem to reproduce the problem here, how many times did you have to
run it to see the null bytes? Do you remove the files between runs?
-chris
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: reiserfs: still problems with tail conversion
2001-02-23 22:10 ` Chris Mason
@ 2001-02-23 22:19 ` Erik Mouw
2001-02-24 2:52 ` Michal Gornisiewicz
` (4 more replies)
2001-02-24 0:34 ` Oliver Teuber
1 sibling, 5 replies; 24+ messages in thread
From: Erik Mouw @ 2001-02-23 22:19 UTC (permalink / raw)
To: Chris Mason; +Cc: Erik Mouw, Linux kernel mailing list
On Fri, Feb 23, 2001 at 05:10:46PM -0500, Chris Mason wrote:
> Many thanks for sending along a test program for reproducing. But, it
> doesn't seem to reproduce the problem here, how many times did you have to
> run it to see the null bytes? Do you remove the files between runs?
I got them immediately at the first run, which more or less was what I
expected because reiserfs ate one of my mailfolders that way (only a
CVS log folder, so nothing special was lost). You have to remove the
files between runs, otherwise the same blocks seem to be allocated to
the files.
I'll upgrade to linux-2.4.2 to see if it solves the problem. (was
running 2.4.2-pre4 + your patch)
Erik
--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031, 2600 GA Delft, The Netherlands
Phone: +31-15-2783635 Fax: +31-15-2781843 Email: J.A.K.Mouw@its.tudelft.nl
WWW: http://www-ict.its.tudelft.nl/~erik/
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: reiserfs: still problems with tail conversion
2001-02-23 22:19 ` Erik Mouw
@ 2001-02-24 2:52 ` Michal Gornisiewicz
2001-02-24 4:13 ` Meino Christian Cramer
` (3 subsequent siblings)
4 siblings, 0 replies; 24+ messages in thread
From: Michal Gornisiewicz @ 2001-02-24 2:52 UTC (permalink / raw)
To: mason, linux-kernel
On Saturday 24 February 2001 06:19, Erik Mouw wrote:
> I'll upgrade to linux-2.4.2 to see if it solves the problem. (was
> running 2.4.2-pre4 + your patch)
>
>
> Erik
I'm running 2.4.2 and I get similar results using your test program.
This is on an IBM 390 Laptop. P2-233, 96mb RAM, 3.2gb HDD.
See program results below. I can supply more info
(.config etc, & do testing) if needed.
MG
Creating 8192 files ... done
Appending to the files ... done
Checking files for null bytes ...
reiser-00107.test contains null bytes
reiser-00127.test contains null bytes
reiser-00208.test contains null bytes
reiser-00234.test contains null bytes
reiser-00259.test contains null bytes
reiser-00303.test contains null bytes
reiser-00324.test contains null bytes
reiser-00343.test contains null bytes
reiser-00371.test contains null bytes
reiser-00388.test contains null bytes
reiser-00444.test contains null bytes
reiser-00452.test contains null bytes
reiser-00459.test contains null bytes
reiser-00466.test contains null bytes
reiser-00495.test contains null bytes
reiser-00502.test contains null bytes
reiser-00515.test contains null bytes
reiser-00522.test contains null bytes
reiser-00535.test contains null bytes
reiser-00548.test contains null bytes
reiser-00608.test contains null bytes
reiser-00614.test contains null bytes
reiser-00620.test contains null bytes
reiser-00626.test contains null bytes
reiser-00637.test contains null bytes
reiser-00648.test contains null bytes
reiser-00659.test contains null bytes
reiser-00669.test contains null bytes
reiser-00674.test contains null bytes
reiser-00704.test contains null bytes
reiser-00744.test contains null bytes
reiser-00749.test contains null bytes
reiser-00754.test contains null bytes
reiser-00759.test contains null bytes
reiser-00764.test contains null bytes
reiser-00773.test contains null bytes
reiser-00778.test contains null bytes
reiser-00787.test contains null bytes
reiser-00792.test contains null bytes
reiser-00801.test contains null bytes
reiser-00810.test contains null bytes
reiser-00819.test contains null bytes
reiser-00847.test contains null bytes
reiser-00855.test contains null bytes
reiser-00899.test contains null bytes
reiser-00947.test contains null bytes
reiser-00951.test contains null bytes
reiser-00955.test contains null bytes
reiser-00959.test contains null bytes
reiser-00963.test contains null bytes
reiser-00967.test contains null bytes
reiser-00971.test contains null bytes
reiser-00978.test contains null bytes
reiser-00982.test contains null bytes
reiser-00986.test contains null bytes
reiser-00990.test contains null bytes
reiser-00997.test contains null bytes
reiser-01001.test contains null bytes
reiser-01005.test contains null bytes
reiser-01012.test contains null bytes
reiser-01016.test contains null bytes
Checking done
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: reiserfs: still problems with tail conversion
2001-02-23 22:19 ` Erik Mouw
2001-02-24 2:52 ` Michal Gornisiewicz
@ 2001-02-24 4:13 ` Meino Christian Cramer
2001-02-24 8:18 ` thunder7
` (2 subsequent siblings)
4 siblings, 0 replies; 24+ messages in thread
From: Meino Christian Cramer @ 2001-02-24 4:13 UTC (permalink / raw)
To: jedin; +Cc: mason, linux-kernel
From: Michal Gornisiewicz <jedin@iname.com>
Subject: Re: reiserfs: still problems with tail conversion
Date: Sat, 24 Feb 2001 10:52:07 +0800
Message-ID: <01022410520700.01447@Defiance.Hell>
I'm running 2.4.2ac3 and tried also the reisertest program.
No problems here...
The created files are all of 8192 bytes. Partition size was 1Gig.
keep hacking the right site of life ! :-)
Meino
> I'm running 2.4.2 and I get similar results using your test program.
> This is on an IBM 390 Laptop. P2-233, 96mb RAM, 3.2gb HDD.
>
> See program results below. I can supply more info
> (.config etc, & do testing) if needed.
>
> MG
>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: reiserfs: still problems with tail conversion
2001-02-23 22:19 ` Erik Mouw
2001-02-24 2:52 ` Michal Gornisiewicz
2001-02-24 4:13 ` Meino Christian Cramer
@ 2001-02-24 8:18 ` thunder7
2001-02-24 19:27 ` Ken Moffat
2001-02-25 16:37 ` Erik Mouw
4 siblings, 0 replies; 24+ messages in thread
From: thunder7 @ 2001-02-24 8:18 UTC (permalink / raw)
To: linux-kernel; +Cc: mason
On Fri, Feb 23, 2001 at 11:19:49PM +0100, Erik Mouw wrote:
> I got them immediately at the first run, which more or less was what I
> expected because reiserfs ate one of my mailfolders that way (only a
> CVS log folder, so nothing special was lost). You have to remove the
> files between runs, otherwise the same blocks seem to be allocated to
> the files.
>
> I'll upgrade to linux-2.4.2 to see if it solves the problem. (was
> running 2.4.2-pre4 + your patch)
>
kernel 2.4.2-ac3, gcc-2.95.2, libc-2.1.3, SuSE-7.0 base system.
Using ReiserFS 3.5.x disk format, binutils-2.10, x86(P3/866)-SMP system.
The program from Erik did the following:
Creating 8192 files ... done
Appending to the files ... done
Checking files for null bytes ...
reiser-00000.test contains null bytes
reiser-00001.test contains null bytes
reiser-00002.test contains null bytes
reiser-00003.test contains null bytes
reiser-00004.test contains null bytes
<snip a lot of lines>
reiser-08189.test contains null bytes
reiser-08190.test contains null bytes
reiser-08191.test contains null bytes
Checking done
Good luck,
Jurriaan
--
Backup Not Found (A)ssasinate Bill Gates (R)etry (K)eep trying until 6 am?
GNU/Linux 2.4.2-ac3 SMP/ReiserFS 2x1730 bogomips load av: 0.08 0.02 0.01
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-23 22:19 ` Erik Mouw
` (2 preceding siblings ...)
2001-02-24 8:18 ` thunder7
@ 2001-02-24 19:27 ` Ken Moffat
2001-02-24 20:53 ` Alan Cox
2001-02-25 16:37 ` Erik Mouw
4 siblings, 1 reply; 24+ messages in thread
From: Ken Moffat @ 2001-02-24 19:27 UTC (permalink / raw)
To: Erik Mouw; +Cc: Chris Mason, Linux kernel mailing list
(reisertest)
I get the same problems with straight 2.4.2, machine is a k5 with
32Mb. The test results vary depending on what else is on the partition,
but in each case the last file affected is 01017 and there are sequences
of previous_number+4, for up to 8 files (but next file after this might be
previous+7 or previous +15, or sporadic). From other problems I've seen on
the list, maybe I need more memory to run reiserfs ?
This happens whether I compile the kernel (and/or the test program) with
Red Hat's revised gcc-2.96 or with egcs. First testing was with a
partition created from the rpm version of mkreiserfs, while running a
2.96-built-kernel. I've now recreated the partition while running a kernel
compiled with egcs ('kgcc'), the only difference is some of the numbers
for the affected files differ.
Partition approx 1.7Gb, built with defaults, block size is 4096.
More details of config or whatever available if required.
Ken
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: reiserfs: still problems with tail conversion
2001-02-24 19:27 ` Ken Moffat
@ 2001-02-24 20:53 ` Alan Cox
2001-02-25 0:41 ` Chris Mason
0 siblings, 1 reply; 24+ messages in thread
From: Alan Cox @ 2001-02-24 20:53 UTC (permalink / raw)
To: Ken Moffat; +Cc: Erik Mouw, Chris Mason, Linux kernel mailing list
> 32Mb. The test results vary depending on what else is on the partition,
> but in each case the last file affected is 01017 and there are sequences
> of previous_number+4, for up to 8 files (but next file after this might be
> previous+7 or previous +15, or sporadic). From other problems I've seen on
> the list, maybe I need more memory to run reiserfs ?
No. Reiserfs cannot go around corrupting files regardless of the amount of
memory you have. What is however quite possible is that there is a race
condition on reiserfs (or in the VFS) that is triggered when you are paging
and programs are thus sleeping on buffer and memory allocations
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-24 20:53 ` Alan Cox
@ 2001-02-25 0:41 ` Chris Mason
2001-02-25 4:21 ` Bernd Eckenfels
0 siblings, 1 reply; 24+ messages in thread
From: Chris Mason @ 2001-02-25 0:41 UTC (permalink / raw)
To: Alan Cox, Ken Moffat; +Cc: Erik Mouw, Linux kernel mailing list
On Saturday, February 24, 2001 08:53:15 PM +0000 Alan Cox
<alan@lxorguk.ukuu.org.uk> wrote:
>> 32Mb. The test results vary depending on what else is on the partition,
>> but in each case the last file affected is 01017 and there are sequences
>> of previous_number+4, for up to 8 files (but next file after this might
>> be previous+7 or previous +15, or sporadic). From other problems I've
>> seen on the list, maybe I need more memory to run reiserfs ?
>
> No. Reiserfs cannot go around corrupting files regardless of the amount of
> memory you have. What is however quite possible is that there is a race
> condition on reiserfs (or in the VFS) that is triggered when you are
> paging and programs are thus sleeping on buffer and memory allocations
>
>
Exactly. The tail conversion code depends heavily on the page up to date
bit being set right. It is more than possible that I've screwed up
something there, and the code thinks a page is valid when it really isn't.
-chris
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-25 0:41 ` Chris Mason
@ 2001-02-25 4:21 ` Bernd Eckenfels
2001-02-25 5:41 ` Mike Galbraith
2001-02-25 13:53 ` Alan Cox
0 siblings, 2 replies; 24+ messages in thread
From: Bernd Eckenfels @ 2001-02-25 4:21 UTC (permalink / raw)
To: linux-kernel
In article <878610000.983061717@tiny> you wrote:
> Exactly. The tail conversion code depends heavily on the page up to date
> bit being set right. It is more than possible that I've screwed up
> something there, and the code thinks a page is valid when it really isn't.
I have seen null byte corruptions in syslog files with ext2 in various
kernels. So perhaps it is a general VFS problem?
Greetings
Bernd
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-25 4:21 ` Bernd Eckenfels
@ 2001-02-25 5:41 ` Mike Galbraith
2001-02-25 13:53 ` Alan Cox
1 sibling, 0 replies; 24+ messages in thread
From: Mike Galbraith @ 2001-02-25 5:41 UTC (permalink / raw)
To: Bernd Eckenfels; +Cc: linux-kernel
On Sun, 25 Feb 2001, Bernd Eckenfels wrote:
> In article <878610000.983061717@tiny> you wrote:
> > Exactly. The tail conversion code depends heavily on the page up to date
> > bit being set right. It is more than possible that I've screwed up
> > something there, and the code thinks a page is valid when it really isn't.
>
> I have seen null byte corruptions in syslog files with ext2 in various
> kernels. So perhaps it is a general VFS problem?
aol.. none since 2.4.0 release here though.
-Mike
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-25 4:21 ` Bernd Eckenfels
2001-02-25 5:41 ` Mike Galbraith
@ 2001-02-25 13:53 ` Alan Cox
1 sibling, 0 replies; 24+ messages in thread
From: Alan Cox @ 2001-02-25 13:53 UTC (permalink / raw)
To: Bernd Eckenfels; +Cc: linux-kernel
> I have seen null byte corruptions in syslog files with ext2 in various
> kernels. So perhaps it is a general VFS problem?
Im very dubious. The post fsck nulls in syslog case is a well understood one
with the box crashing as it extends the file and never gets to commit the data
bytes.
Different thing I suspect.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-23 22:19 ` Erik Mouw
` (3 preceding siblings ...)
2001-02-24 19:27 ` Ken Moffat
@ 2001-02-25 16:37 ` Erik Mouw
2001-02-25 17:32 ` Erik Mouw
2001-02-25 20:32 ` Chris Mason
4 siblings, 2 replies; 24+ messages in thread
From: Erik Mouw @ 2001-02-25 16:37 UTC (permalink / raw)
To: mason; +Cc: Linux kernel mailing list
On Fri, Feb 23, 2001 at 11:19:49PM +0100, Erik Mouw wrote:
> On Fri, Feb 23, 2001 at 05:10:46PM -0500, Chris Mason wrote:
> > Many thanks for sending along a test program for reproducing. But, it
> > doesn't seem to reproduce the problem here, how many times did you have to
> > run it to see the null bytes? Do you remove the files between runs?
>
> I got them immediately at the first run, which more or less was what I
> expected because reiserfs ate one of my mailfolders that way (only a
> CVS log folder, so nothing special was lost). You have to remove the
> files between runs, otherwise the same blocks seem to be allocated to
> the files.
>
> I'll upgrade to linux-2.4.2 to see if it solves the problem. (was
> running 2.4.2-pre4 + your patch)
I upgraded to 2.4.2, and initially I couldn't reproduce the problem.
Besides the kernel version difference, another difference was the fact
that I did the 2.4.2. test on a freshly booted system, while the
2.4.2-pre4 test was done on a system with quite some VM pressure:
uptime a couple of days, running acroread, netscape, xemacs, couple of
gnome-terminals with large scroll back buffers (10000 lines).
John Adams told me that the data didn't hit the disk on his system and
that he had to add O_SYNC to the open()s. After I did that, I could
reproduce the problem on linux-2.4.2, with the strange results that the
bug is in *every* file with initial size >=1024 bytes.
John also told that his machine doesn't have swap, but I fail to see
why that could influence the reiserfs subsystem. Anyway, the bug seems
to appear when the data hits the disk, either by high VM pressure, or
by using O_SYNC.
Erik
--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031, 2600 GA Delft, The Netherlands
Phone: +31-15-2783635 Fax: +31-15-2781843 Email: J.A.K.Mouw@its.tudelft.nl
WWW: http://www-ict.its.tudelft.nl/~erik/
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: reiserfs: still problems with tail conversion
2001-02-25 16:37 ` Erik Mouw
@ 2001-02-25 17:32 ` Erik Mouw
2001-02-26 2:40 ` [PATCH] " Chris Mason
2001-02-25 20:32 ` Chris Mason
1 sibling, 1 reply; 24+ messages in thread
From: Erik Mouw @ 2001-02-25 17:32 UTC (permalink / raw)
To: mason; +Cc: Linux kernel mailing list
[-- Attachment #1: Type: text/plain, Size: 1669 bytes --]
On Sun, Feb 25, 2001 at 05:37:52PM +0100, Erik Mouw wrote:
> I upgraded to 2.4.2, and initially I couldn't reproduce the problem.
> Besides the kernel version difference, another difference was the fact
> that I did the 2.4.2. test on a freshly booted system, while the
> 2.4.2-pre4 test was done on a system with quite some VM pressure:
> uptime a couple of days, running acroread, netscape, xemacs, couple of
> gnome-terminals with large scroll back buffers (10000 lines).
>
> John Adams told me that the data didn't hit the disk on his system and
> that he had to add O_SYNC to the open()s. After I did that, I could
> reproduce the problem on linux-2.4.2, with the strange results that the
> bug is in *every* file with initial size >=1024 bytes.
Hohum, this is probably because I forgot to close() the file in the
read() part of the test... *blush*
I just reran the test on my laptop (arthur: Celeron (Coppermine) 500,
128MB, SuSE 7.0 + updates, linux-2.4.2, gcc-2.95.2) and got "only" 963
errors. I also ran the test om my desktop (zaphod: AMD K6-3D/333,
160MB, Debian 2.2 + updates, linux-2.4.2-pre4+reiserfs patch,
gcc-2.95.2) and got 888 errors.
An interesting observation is that the two systems have the bugs in
almost the same places. Attached is a diff between the output files of
both systems.
Erik
--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031, 2600 GA Delft, The Netherlands
Phone: +31-15-2783635 Fax: +31-15-2781843 Email: J.A.K.Mouw@its.tudelft.nl
WWW: http://www-ict.its.tudelft.nl/~erik/
[-- Attachment #2: reiserdiff.txt --]
[-- Type: text/plain, Size: 6392 bytes --]
--- arthur-test3.txt Sun Feb 25 18:17:14 2001
+++ zaphod-test1.txt Sun Feb 25 18:26:40 2001
@@ -1,7 +1,6 @@
Creating 8192 files ... done
Appending to the files ... done
Checking files for null bytes ...
- reiser-00039.test contains null bytes
reiser-00129.test contains null bytes
reiser-00178.test contains null bytes
reiser-00219.test contains null bytes
@@ -583,7 +582,7 @@
reiser-05080.test contains null bytes
reiser-05087.test contains null bytes
reiser-05094.test contains null bytes
- reiser-05100.test contains null bytes
+ reiser-05101.test contains null bytes
reiser-05108.test contains null bytes
reiser-05115.test contains null bytes
reiser-05122.test contains null bytes
@@ -676,98 +675,13 @@
reiser-05722.test contains null bytes
reiser-05724.test contains null bytes
reiser-05726.test contains null bytes
- reiser-05784.test contains null bytes
- reiser-05786.test contains null bytes
- reiser-05824.test contains null bytes
- reiser-05826.test contains null bytes
- reiser-05864.test contains null bytes
- reiser-05906.test contains null bytes
- reiser-06014.test contains null bytes
- reiser-06016.test contains null bytes
- reiser-06018.test contains null bytes
- reiser-06020.test contains null bytes
- reiser-06022.test contains null bytes
- reiser-06024.test contains null bytes
- reiser-06026.test contains null bytes
- reiser-06028.test contains null bytes
- reiser-06030.test contains null bytes
- reiser-06032.test contains null bytes
- reiser-06034.test contains null bytes
- reiser-06036.test contains null bytes
- reiser-06038.test contains null bytes
- reiser-06040.test contains null bytes
- reiser-06042.test contains null bytes
- reiser-06044.test contains null bytes
- reiser-06046.test contains null bytes
- reiser-06048.test contains null bytes
- reiser-06050.test contains null bytes
- reiser-06052.test contains null bytes
- reiser-06054.test contains null bytes
- reiser-06056.test contains null bytes
- reiser-06058.test contains null bytes
- reiser-06060.test contains null bytes
- reiser-06063.test contains null bytes
- reiser-06065.test contains null bytes
- reiser-06067.test contains null bytes
- reiser-06069.test contains null bytes
- reiser-06071.test contains null bytes
- reiser-06073.test contains null bytes
- reiser-06075.test contains null bytes
- reiser-06077.test contains null bytes
- reiser-06079.test contains null bytes
- reiser-06081.test contains null bytes
- reiser-06083.test contains null bytes
- reiser-06085.test contains null bytes
- reiser-06088.test contains null bytes
- reiser-06090.test contains null bytes
- reiser-06092.test contains null bytes
- reiser-06094.test contains null bytes
- reiser-06096.test contains null bytes
- reiser-06098.test contains null bytes
- reiser-06100.test contains null bytes
- reiser-06102.test contains null bytes
- reiser-06104.test contains null bytes
- reiser-06106.test contains null bytes
- reiser-06109.test contains null bytes
- reiser-06111.test contains null bytes
- reiser-06113.test contains null bytes
- reiser-06115.test contains null bytes
- reiser-06117.test contains null bytes
- reiser-06119.test contains null bytes
- reiser-06121.test contains null bytes
- reiser-06123.test contains null bytes
+ reiser-05764.test contains null bytes
+ reiser-05766.test contains null bytes
+ reiser-05804.test contains null bytes
+ reiser-05844.test contains null bytes
+ reiser-05866.test contains null bytes
reiser-06126.test contains null bytes
reiser-06128.test contains null bytes
- reiser-06130.test contains null bytes
- reiser-06132.test contains null bytes
- reiser-06134.test contains null bytes
- reiser-06136.test contains null bytes
- reiser-06138.test contains null bytes
- reiser-06141.test contains null bytes
- reiser-06143.test contains null bytes
- reiser-06145.test contains null bytes
- reiser-06147.test contains null bytes
- reiser-06149.test contains null bytes
- reiser-06151.test contains null bytes
- reiser-06154.test contains null bytes
- reiser-06156.test contains null bytes
- reiser-06158.test contains null bytes
- reiser-06160.test contains null bytes
- reiser-06162.test contains null bytes
- reiser-06164.test contains null bytes
- reiser-06167.test contains null bytes
- reiser-06169.test contains null bytes
- reiser-06171.test contains null bytes
- reiser-06173.test contains null bytes
- reiser-06175.test contains null bytes
- reiser-06178.test contains null bytes
- reiser-06180.test contains null bytes
- reiser-06182.test contains null bytes
- reiser-06184.test contains null bytes
- reiser-06186.test contains null bytes
- reiser-06189.test contains null bytes
- reiser-06191.test contains null bytes
- reiser-06193.test contains null bytes
reiser-06195.test contains null bytes
reiser-06198.test contains null bytes
reiser-06200.test contains null bytes
@@ -832,7 +746,6 @@
reiser-06339.test contains null bytes
reiser-06342.test contains null bytes
reiser-06344.test contains null bytes
- reiser-06347.test contains null bytes
reiser-06349.test contains null bytes
reiser-06352.test contains null bytes
reiser-06354.test contains null bytes
@@ -947,7 +860,6 @@
reiser-06668.test contains null bytes
reiser-06671.test contains null bytes
reiser-06674.test contains null bytes
- reiser-06677.test contains null bytes
reiser-06680.test contains null bytes
reiser-06683.test contains null bytes
reiser-06684.test contains null bytes
@@ -961,7 +873,20 @@
reiser-06702.test contains null bytes
reiser-06703.test contains null bytes
reiser-06706.test contains null bytes
+ reiser-06707.test contains null bytes
reiser-06710.test contains null bytes
reiser-06711.test contains null bytes
+ reiser-06717.test contains null bytes
+ reiser-06718.test contains null bytes
+ reiser-06719.test contains null bytes
+ reiser-06720.test contains null bytes
+ reiser-06721.test contains null bytes
+ reiser-06722.test contains null bytes
+ reiser-06723.test contains null bytes
+ reiser-06724.test contains null bytes
+ reiser-06725.test contains null bytes
+ reiser-06726.test contains null bytes
+ reiser-06727.test contains null bytes
+ reiser-06728.test contains null bytes
reiser-07077.test contains null bytes
Checking done
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH] Re: reiserfs: still problems with tail conversion
2001-02-25 17:32 ` Erik Mouw
@ 2001-02-26 2:40 ` Chris Mason
2001-02-26 11:07 ` Erik Mouw
2001-02-26 22:46 ` Ken Moffat
0 siblings, 2 replies; 24+ messages in thread
From: Chris Mason @ 2001-02-26 2:40 UTC (permalink / raw)
To: Erik Mouw; +Cc: Linux kernel mailing list, Nick Pasich, reiserfs-list
Hi guys,
This patch should take care of the other cause for null bytes
in small files. It has been through a few hours of testing,
with some of the usual load programs + Erik's code concurrently.
I'll let things run overnight to try and find more bugs. The
patch is against 2.4.2, and does a few things:
don't dirty the direct->indirect target until all direct items
have been copied in. Before it was dirtied for each direct item.
make the target up to date before dirtying (it was done after).
don't try to zero the unused part of the target until all bytes
have been copied. This was the big bug, it was zeroing previously
copied bytes.
Any testing on non-production machines would be appreciated,
I'll forward to Linus/Alan once I've gotten more feedback.
-chris
diff -ur diff/linux/fs/reiserfs/inode.c linux/fs/reiserfs/inode.c
--- diff/linux/fs/reiserfs/inode.c Tue Jan 16 14:14:22 2001
+++ linux/fs/reiserfs/inode.c Sun Feb 25 16:25:31 2001
@@ -771,6 +771,7 @@
** flush unbh before the transaction commits
*/
reiserfs_add_page_to_flush_list(&th, inode, unbh) ;
+ mark_buffer_dirty(unbh) ;
//inode->i_blocks += inode->i_sb->s_blocksize / 512;
//mark_tail_converted (inode);
diff -ur diff/linux/fs/reiserfs/stree.c linux/fs/reiserfs/stree.c
--- diff/linux/fs/reiserfs/stree.c Mon Jan 15 18:31:19 2001
+++ linux/fs/reiserfs/stree.c Sun Feb 25 16:25:31 2001
@@ -1438,7 +1438,6 @@
if ( p_s_un_bh ) {
int off;
- int block_off ;
char *data ;
/* We are in direct2indirect conversion, so move tail contents
@@ -1452,7 +1451,8 @@
** the unformatted node, which might schedule, meaning we'd have to
** loop all the way back up to the start of the while loop.
**
- ** The unformatted node is prepared and logged after the do_balance.
+ ** The unformatted node must be dirtied later on. We can't be
+ ** sure here if the entire tail has been deleted yet.
**
** p_s_un_bh is from the page cache (all unformatted nodes are
** from the page cache) and might be a highmem page. So, we
@@ -1463,24 +1463,12 @@
data = page_address(p_s_un_bh->b_page) ;
off = ((le_ih_k_offset (&s_ih) - 1) & (PAGE_CACHE_SIZE - 1));
- block_off = off & (p_s_un_bh->b_size - 1) ;
memcpy(data + off,
B_I_PITEM(PATH_PLAST_BUFFER(p_s_path), &s_ih), n_ret_value);
-
- /* clear out the rest of the block past the end of the file. */
- if (block_off + n_ret_value < p_s_un_bh->b_size) {
- memset(data + off + n_ret_value, 0,
- p_s_un_bh->b_size - block_off - n_ret_value) ;
- }
}
/* Perform balancing after all resources have been collected at once. */
do_balance(&s_del_balance, NULL, NULL, M_DELETE);
-
- /* see comment above for why this is after the do_balance */
- if (p_s_un_bh) {
- mark_buffer_dirty(p_s_un_bh) ;
- }
/* Return deleted body length */
return n_ret_value;
diff -ur diff/linux/fs/reiserfs/tail_conversion.c linux/fs/reiserfs/tail_conversion.c
--- diff/linux/fs/reiserfs/tail_conversion.c Mon Feb 19 13:07:32 2001
+++ linux/fs/reiserfs/tail_conversion.c Sun Feb 25 19:42:54 2001
@@ -32,6 +32,7 @@
struct super_block * sb = inode->i_sb;
struct buffer_head *up_to_date_bh ;
struct item_head * p_le_ih = PATH_PITEM_HEAD (path);
+ unsigned long total_tail = 0 ;
struct cpu_key end_key; /* Key to search for the last byte of the
converted item. */
struct item_head ind_ih; /* new indirect item to be inserted or
@@ -121,10 +122,19 @@
n_retval = reiserfs_delete_item (th, path, &end_key, inode,
up_to_date_bh) ;
+ total_tail += n_retval ;
if (tail_size == n_retval)
// done: file does not have direct items anymore
break;
+ }
+ /* if we've copied bytes from disk into the page, we need to zero
+ ** out the unused part of the block (it was not up to date before)
+ ** the page is still kmapped (by whoever called reiserfs_get_block)
+ */
+ if (up_to_date_bh) {
+ unsigned pgoff = (tail_offset + total_tail - 1) & (PAGE_CACHE_SIZE - 1);
+ memset(page_address(unbh->b_page) + pgoff, 0, n_blk_size - total_tail) ;
}
inode->u.reiserfs_i.i_first_direct_byte = U32_MAX;
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH] Re: reiserfs: still problems with tail conversion
2001-02-26 2:40 ` [PATCH] " Chris Mason
@ 2001-02-26 11:07 ` Erik Mouw
2001-02-26 15:37 ` Erik Mouw
2001-02-26 22:46 ` Ken Moffat
1 sibling, 1 reply; 24+ messages in thread
From: Erik Mouw @ 2001-02-26 11:07 UTC (permalink / raw)
To: Chris Mason; +Cc: Linux kernel mailing list, Nick Pasich, reiserfs-list
On Sun, Feb 25, 2001 at 09:40:44PM -0500, Chris Mason wrote:
> This patch should take care of the other cause for null bytes
> in small files. It has been through a few hours of testing,
> with some of the usual load programs + Erik's code concurrently.
>
> I'll let things run overnight to try and find more bugs. The
> patch is against 2.4.2, and does a few things:
>
> don't dirty the direct->indirect target until all direct items
> have been copied in. Before it was dirtied for each direct item.
>
> make the target up to date before dirtying (it was done after).
>
> don't try to zero the unused part of the target until all bytes
> have been copied. This was the big bug, it was zeroing previously
> copied bytes.
>
> Any testing on non-production machines would be appreciated,
> I'll forward to Linus/Alan once I've gotten more feedback.
Yes, this did the trick, I can't repeat it anymore after a first run.
I'll let my code run for a couple of times to stress the system, but at
first glance the bug seems to be fixed.
Thanks for your efforts,
Erik
--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031, 2600 GA Delft, The Netherlands
Phone: +31-15-2783635 Fax: +31-15-2781843 Email: J.A.K.Mouw@its.tudelft.nl
WWW: http://www-ict.its.tudelft.nl/~erik/
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Re: reiserfs: still problems with tail conversion
2001-02-26 11:07 ` Erik Mouw
@ 2001-02-26 15:37 ` Erik Mouw
0 siblings, 0 replies; 24+ messages in thread
From: Erik Mouw @ 2001-02-26 15:37 UTC (permalink / raw)
To: Chris Mason; +Cc: Linux kernel mailing list, Nick Pasich, reiserfs-list
On Mon, Feb 26, 2001 at 12:07:04PM +0100, Erik Mouw wrote:
> On Sun, Feb 25, 2001 at 09:40:44PM -0500, Chris Mason wrote:
> > Any testing on non-production machines would be appreciated,
> > I'll forward to Linus/Alan once I've gotten more feedback.
>
> Yes, this did the trick, I can't repeat it anymore after a first run.
> I'll let my code run for a couple of times to stress the system, but at
> first glance the bug seems to be fixed.
It has been running for quite some time in the background, and I can't
reproduce the bug.
Erik
--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031, 2600 GA Delft, The Netherlands
Phone: +31-15-2783635 Fax: +31-15-2781843 Email: J.A.K.Mouw@its.tudelft.nl
WWW: http://www-ict.its.tudelft.nl/~erik/
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Re: reiserfs: still problems with tail conversion
2001-02-26 2:40 ` [PATCH] " Chris Mason
2001-02-26 11:07 ` Erik Mouw
@ 2001-02-26 22:46 ` Ken Moffat
1 sibling, 0 replies; 24+ messages in thread
From: Ken Moffat @ 2001-02-26 22:46 UTC (permalink / raw)
To: Chris Mason; +Cc: Linux kernel mailing list, reiserfs-list
Chris,
it works nicely here (i586, 32Mb, Red Hat's gcc-2.96-69). Thanks.
Ken
On Sun, 25 Feb 2001, Chris Mason wrote:
>
> Hi guys,
>
> This patch should take care of the other cause for null bytes
> in small files. It has been through a few hours of testing,
> with some of the usual load programs + Erik's code concurrently.
>
> I'll let things run overnight to try and find more bugs. The
> patch is against 2.4.2, and does a few things:
>
> don't dirty the direct->indirect target until all direct items
> have been copied in. Before it was dirtied for each direct item.
>
> make the target up to date before dirtying (it was done after).
>
> don't try to zero the unused part of the target until all bytes
> have been copied. This was the big bug, it was zeroing previously
> copied bytes.
>
> Any testing on non-production machines would be appreciated,
> I'll forward to Linus/Alan once I've gotten more feedback.
>
> -chris
>
> diff -ur diff/linux/fs/reiserfs/inode.c linux/fs/reiserfs/inode.c
> --- diff/linux/fs/reiserfs/inode.c Tue Jan 16 14:14:22 2001
> +++ linux/fs/reiserfs/inode.c Sun Feb 25 16:25:31 2001
> @@ -771,6 +771,7 @@
> ** flush unbh before the transaction commits
> */
> reiserfs_add_page_to_flush_list(&th, inode, unbh) ;
> + mark_buffer_dirty(unbh) ;
>
> //inode->i_blocks += inode->i_sb->s_blocksize / 512;
> //mark_tail_converted (inode);
> diff -ur diff/linux/fs/reiserfs/stree.c linux/fs/reiserfs/stree.c
> --- diff/linux/fs/reiserfs/stree.c Mon Jan 15 18:31:19 2001
> +++ linux/fs/reiserfs/stree.c Sun Feb 25 16:25:31 2001
> @@ -1438,7 +1438,6 @@
>
> if ( p_s_un_bh ) {
> int off;
> - int block_off ;
> char *data ;
>
> /* We are in direct2indirect conversion, so move tail contents
> @@ -1452,7 +1451,8 @@
> ** the unformatted node, which might schedule, meaning we'd have to
> ** loop all the way back up to the start of the while loop.
> **
> - ** The unformatted node is prepared and logged after the do_balance.
> + ** The unformatted node must be dirtied later on. We can't be
> + ** sure here if the entire tail has been deleted yet.
> **
> ** p_s_un_bh is from the page cache (all unformatted nodes are
> ** from the page cache) and might be a highmem page. So, we
> @@ -1463,24 +1463,12 @@
>
> data = page_address(p_s_un_bh->b_page) ;
> off = ((le_ih_k_offset (&s_ih) - 1) & (PAGE_CACHE_SIZE - 1));
> - block_off = off & (p_s_un_bh->b_size - 1) ;
> memcpy(data + off,
> B_I_PITEM(PATH_PLAST_BUFFER(p_s_path), &s_ih), n_ret_value);
> -
> - /* clear out the rest of the block past the end of the file. */
> - if (block_off + n_ret_value < p_s_un_bh->b_size) {
> - memset(data + off + n_ret_value, 0,
> - p_s_un_bh->b_size - block_off - n_ret_value) ;
> - }
> }
>
> /* Perform balancing after all resources have been collected at once. */
> do_balance(&s_del_balance, NULL, NULL, M_DELETE);
> -
> - /* see comment above for why this is after the do_balance */
> - if (p_s_un_bh) {
> - mark_buffer_dirty(p_s_un_bh) ;
> - }
>
> /* Return deleted body length */
> return n_ret_value;
> diff -ur diff/linux/fs/reiserfs/tail_conversion.c linux/fs/reiserfs/tail_conversion.c
> --- diff/linux/fs/reiserfs/tail_conversion.c Mon Feb 19 13:07:32 2001
> +++ linux/fs/reiserfs/tail_conversion.c Sun Feb 25 19:42:54 2001
> @@ -32,6 +32,7 @@
> struct super_block * sb = inode->i_sb;
> struct buffer_head *up_to_date_bh ;
> struct item_head * p_le_ih = PATH_PITEM_HEAD (path);
> + unsigned long total_tail = 0 ;
> struct cpu_key end_key; /* Key to search for the last byte of the
> converted item. */
> struct item_head ind_ih; /* new indirect item to be inserted or
> @@ -121,10 +122,19 @@
> n_retval = reiserfs_delete_item (th, path, &end_key, inode,
> up_to_date_bh) ;
>
> + total_tail += n_retval ;
> if (tail_size == n_retval)
> // done: file does not have direct items anymore
> break;
>
> + }
> + /* if we've copied bytes from disk into the page, we need to zero
> + ** out the unused part of the block (it was not up to date before)
> + ** the page is still kmapped (by whoever called reiserfs_get_block)
> + */
> + if (up_to_date_bh) {
> + unsigned pgoff = (tail_offset + total_tail - 1) & (PAGE_CACHE_SIZE - 1);
> + memset(page_address(unbh->b_page) + pgoff, 0, n_blk_size - total_tail) ;
> }
>
> inode->u.reiserfs_i.i_first_direct_byte = U32_MAX;
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-25 16:37 ` Erik Mouw
2001-02-25 17:32 ` Erik Mouw
@ 2001-02-25 20:32 ` Chris Mason
1 sibling, 0 replies; 24+ messages in thread
From: Chris Mason @ 2001-02-25 20:32 UTC (permalink / raw)
To: Erik Mouw; +Cc: Linux kernel mailing list
Ok, found it. It is related to the last null byte problem in that it also
only happens when the direct item is split between two blocks. This is
more likely as the tail increases in size, which is why you saw it on
larger small files.
The bug is in the code that zeros the unused part of the unformatted node
after a direct->indirect conversion. This code only gets called when the
page/buffer wasn't already up to date, which is why you see it more when
there is less ram.
Fix will be out shortly....
-chris
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-23 22:10 ` Chris Mason
2001-02-23 22:19 ` Erik Mouw
@ 2001-02-24 0:34 ` Oliver Teuber
1 sibling, 0 replies; 24+ messages in thread
From: Oliver Teuber @ 2001-02-24 0:34 UTC (permalink / raw)
To: Chris Mason; +Cc: linux-kernel, J.A.K.Mouw
On Fri, Feb 23, 2001 at 05:10:46PM -0500, Chris Mason wrote:
> On Friday, February 23, 2001 10:18:56 PM +0100 Erik Mouw
> <J.A.K.Mouw@ITS.TUDelft.NL> wrote:
> > I am running linux-2.4.2-pre4 with Chris Mason's tailconversion bug fix
> > applied, but I still have problems with null bytes in files. I wrote a
> > little test program that clearly shows the problem:
> Many thanks for sending along a test program for reproducing. But, it
> doesn't seem to reproduce the problem here, how many times did you have to
> run it to see the null bytes? Do you remove the files between runs?
i cant reproduce the problem on my system runnig 2.4.2, too.
no problem so far ;)
reiserfs on an 46gb ibm ide hdd, 750mhz p3, 256mb ram
yours, oliver teuber
^ permalink raw reply [flat|nested] 24+ messages in thread
[parent not found: <01022318321302.01755@flash>]
* Re: reiserfs: still problems with tail conversion
[not found] ` <01022318321302.01755@flash>
@ 2001-02-24 8:40 ` John E. Adams
0 siblings, 0 replies; 24+ messages in thread
From: John E. Adams @ 2001-02-24 8:40 UTC (permalink / raw)
To: linux-kernel%vger.kernel.org
On Friday 23 February 2001 16:18, Erik Mouw wrote:
> Hi all,
>
> I am running linux-2.4.2-pre4 with Chris Mason's tailconversion bug fix
> applied, but I still have problems with null bytes in files. I wrote a
> little test program that clearly shows the problem:
I tried the test pgm.
First run didn't even hit the disk, so I added O_SYNC to the opens.
No errors were detected. Then i split out the checking code from the
building code after noticing the checks were run against the buffer cache.
Still no errors.
This is a SuSE 7.0 machine w/512M memory, plenty of disk space and no swap,
running 2.4.2
johna@onevistacom
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: reiserfs: still problems with tail conversion
2001-02-23 21:18 reiserfs: still problems with tail conversion Erik Mouw
2001-02-23 22:10 ` Chris Mason
[not found] ` <01022318321302.01755@flash>
@ 2001-02-24 15:45 ` Arjan Filius
2001-02-24 16:24 ` [lkml]Re: " thunder7
2001-02-24 17:28 ` Chris Mason
2 siblings, 2 replies; 24+ messages in thread
From: Arjan Filius @ 2001-02-24 15:45 UTC (permalink / raw)
To: Erik Mouw; +Cc: Linux kernel mailing list, mason
Hello,
I tried Erik's trigger-program.
After some test i thing it's memory related, and it seems to match the
other reports i saw on lkm.
With my 384M ram i was not able te reproduce it.
With "mem=32M" linux hang while starting a test oracle-db.
However i tried (not repeated tests, and after a fresh reboot):
ram=128M ; Triggered
ram=138M ; Triggered
ram=180M ; Triggered
ram=192M ; NOT Triggered
ram=250M ; NOT Triggered
ram=256M ; NOT Triggered
These results say that it memory dependent, and perhaps memory use
dependent.
With the mem=180M i did some additional tests:
reisertest ; triggered
free ; shows only 60M on cached data and 8192 files*8192
bytes=64M
/sbin/swapout 100M ; make sure enough cache to hold 64M data
reisertest ; NOT Triggered !!!!
While leaving the data, and executing reisertest in a new dir i'm
triggring it again!
So i think i can say, it's triggerable when the cache has no space to hold
all the data (64M), but i didn't extensive tests.
Running suse7.0+updates
kernel 2.4.2
Greatings,
On Fri, 23 Feb 2001, Erik Mouw wrote:
> Hi all,
>
> I am running linux-2.4.2-pre4 with Chris Mason's tailconversion bug fix
> applied, but I still have problems with null bytes in files. I wrote a
> little test program that clearly shows the problem:
>
>
> /* reisertest.c: test for tailconversion bug in reiserfs
> *
> * Compile with: gcc -O2 -o reisertest reisertest.c
> */
> #include <stdio.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <unistd.h>
>
> #define MAXBYTES 8192
>
> int main(int argc, char *argv[])
> {
> int fd;
> int i;
> char name[32];
> char buf[MAXBYTES];
> char check[MAXBYTES];
>
> memset(buf, 0x55, MAXBYTES);
>
> fprintf(stderr, "Creating %d files ... ", MAXBYTES);
>
> for(i = 0; i < MAXBYTES; i++) {
> sprintf(name, "reiser-%05d.test", i);
> fd = open(name, O_WRONLY | O_CREAT, 0644);
> write(fd, buf, i);
> close(fd);
> }
>
> fprintf(stderr, "done\n");
> fprintf(stderr, "Appending to the files ... ");
>
> for(i = 0; i < MAXBYTES; i++) {
> sprintf(name, "reiser-%05d.test", i);
> fd = open(name, O_WRONLY | O_APPEND);
> write(fd, buf, MAXBYTES - i);
> close(fd);
> }
>
> fprintf(stderr, "done\n");
> fprintf(stderr, "Checking files for null bytes ...\n");
>
> for(i = 0; i < MAXBYTES; i++) {
> sprintf(name, "reiser-%05d.test", i);
> fd = open(name, O_RDONLY);
> read(fd, check, MAXBYTES);
> if(memcmp(buf, check, MAXBYTES) != 0)
> fprintf(stderr, " %s contains null bytes\n", name);
> }
>
> fprintf(stderr, "Checking done\n");
>
> return 0;
> }
>
>
> When I run this on a reiserfs partition, I get output like this:
>
>
> erik@arthur:~/reisertest/foo> ../reisertest
> Creating 8192 files ... done
> Appending to the files ... done
> Checking files for null bytes ...
> reiser-00193.test contains null bytes
> reiser-00220.test contains null bytes
> reiser-00256.test contains null bytes
> reiser-00289.test contains null bytes
> reiser-00329.test contains null bytes
> reiser-00338.test contains null bytes
> reiser-00374.test contains null bytes
> reiser-00407.test contains null bytes
> reiser-00415.test contains null bytes
> reiser-00430.test contains null bytes
> reiser-00438.test contains null bytes
> reiser-00445.test contains null bytes
> reiser-00459.test contains null bytes
> reiser-00481.test contains null bytes
> reiser-00501.test contains null bytes
> reiser-00508.test contains null bytes
> reiser-00521.test contains null bytes
> reiser-00534.test contains null bytes
> reiser-00558.test contains null bytes
> reiser-00577.test contains null bytes
> reiser-00583.test contains null bytes
> reiser-00600.test contains null bytes
> reiser-00606.test contains null bytes
> reiser-00612.test contains null bytes
> reiser-00623.test contains null bytes
> reiser-00634.test contains null bytes
> reiser-00645.test contains null bytes
> reiser-00665.test contains null bytes
> reiser-00685.test contains null bytes
> reiser-00730.test contains null bytes
> reiser-00735.test contains null bytes
> reiser-00740.test contains null bytes
> reiser-00745.test contains null bytes
> reiser-00750.test contains null bytes
> reiser-00759.test contains null bytes
> reiser-00764.test contains null bytes
> reiser-00773.test contains null bytes
> reiser-00778.test contains null bytes
> reiser-00787.test contains null bytes
> reiser-00796.test contains null bytes
> reiser-00805.test contains null bytes
> reiser-00814.test contains null bytes
> reiser-00866.test contains null bytes
> reiser-00915.test contains null bytes
> reiser-00930.test contains null bytes
> reiser-00934.test contains null bytes
> reiser-00938.test contains null bytes
> reiser-00942.test contains null bytes
> reiser-00946.test contains null bytes
> reiser-00950.test contains null bytes
> reiser-00954.test contains null bytes
> reiser-00958.test contains null bytes
> reiser-00965.test contains null bytes
> reiser-00969.test contains null bytes
> reiser-00973.test contains null bytes
> reiser-00977.test contains null bytes
> reiser-00984.test contains null bytes
> reiser-00988.test contains null bytes
> reiser-00995.test contains null bytes
> reiser-00999.test contains null bytes
> reiser-01006.test contains null bytes
> reiser-01010.test contains null bytes
> reiser-01017.test contains null bytes
> Checking done
>
>
> Running the test a couple of times doesn't really show a pattern,
> sometimes the same files contains null bytes, sometimes others do. The
> files with null bytes seem to be with index < 1024.
>
> I did the same test with an ext2 filesystem, but didn't see any error.
> System is SuSE 7.0, compiler gcc-2.95.2.
>
>
> Erik
>
>
--
Arjan Filius
mailto:iafilius@xs4all.nl
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [lkml]Re: reiserfs: still problems with tail conversion
2001-02-24 15:45 ` Arjan Filius
@ 2001-02-24 16:24 ` thunder7
2001-02-24 17:28 ` Chris Mason
1 sibling, 0 replies; 24+ messages in thread
From: thunder7 @ 2001-02-24 16:24 UTC (permalink / raw)
To: linux-kernel
On Sat, Feb 24, 2001 at 04:45:04PM +0100, Arjan Filius wrote:
> Hello,
>
> I tried Erik's trigger-program.
>
> After some test i thing it's memory related, and it seems to match the
> other reports i saw on lkm.
> With my 384M ram i was not able te reproduce it.
> With "mem=32M" linux hang while starting a test oracle-db.
> However i tried (not repeated tests, and after a fresh reboot):
> ram=128M ; Triggered
> ram=138M ; Triggered
> ram=180M ; Triggered
> ram=192M ; NOT Triggered
> ram=250M ; NOT Triggered
> ram=256M ; NOT Triggered
>
> These results say that it memory dependent, and perhaps memory use
> dependent.
> With the mem=180M i did some additional tests:
>
> reisertest ; triggered
> free ; shows only 60M on cached data and 8192 files*8192
> bytes=64M
> /sbin/swapout 100M ; make sure enough cache to hold 64M data
> reisertest ; NOT Triggered !!!!
> While leaving the data, and executing reisertest in a new dir i'm
> triggring it again!
>
> So i think i can say, it's triggerable when the cache has no space to hold
> all the data (64M), but i didn't extensive tests.
>
I can't confirm that. This machine has 512 Mb memory:
free
total used free shared buffers cached
Mem: 512940 144916 368024 0 12052 106552
-/+ buffers/cache: 26312 486628
Swap: 1992052 0 1992052
<after the failing test>
total used free shared buffers cached
Mem: 512940 144924 368016 0 12052 106552
-/+ buffers/cache: 26320 486620
Swap: 1992052 0 1992052
Jurriaan
--
BOFH excuse #167:
excessive collisions & not enough packet ambulances
GNU/Linux 2.4.2-ac3 SMP/ReiserFS 2x1730 bogomips load av: 0.26 0.06 0.02
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: reiserfs: still problems with tail conversion
2001-02-24 15:45 ` Arjan Filius
2001-02-24 16:24 ` [lkml]Re: " thunder7
@ 2001-02-24 17:28 ` Chris Mason
1 sibling, 0 replies; 24+ messages in thread
From: Chris Mason @ 2001-02-24 17:28 UTC (permalink / raw)
To: Arjan Filius, Erik Mouw; +Cc: Linux kernel mailing list
On Saturday, February 24, 2001 04:45:04 PM +0100 Arjan Filius
<iafilius@xs4all.nl> wrote:
> Hello,
>
> I tried Erik's trigger-program.
>
> After some test i thing it's memory related, and it seems to match the
> other reports i saw on lkm.
> With my 384M ram i was not able te reproduce it.
> With "mem=32M" linux hang while starting a test oracle-db.
> However i tried (not repeated tests, and after a fresh reboot):
> ram=128M ; Triggered
Ah, I did not get it at 128M, but did get the messages at 32MB. The read
stage of the test program does not close the fd by the way, so some of the
errors were from that (but not all).
So, there must be somewhere else that we are screwing up the tail
conversion, I'll see what I can find.
-chris
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2001-02-26 22:46 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-02-23 21:18 reiserfs: still problems with tail conversion Erik Mouw
2001-02-23 22:10 ` Chris Mason
2001-02-23 22:19 ` Erik Mouw
2001-02-24 2:52 ` Michal Gornisiewicz
2001-02-24 4:13 ` Meino Christian Cramer
2001-02-24 8:18 ` thunder7
2001-02-24 19:27 ` Ken Moffat
2001-02-24 20:53 ` Alan Cox
2001-02-25 0:41 ` Chris Mason
2001-02-25 4:21 ` Bernd Eckenfels
2001-02-25 5:41 ` Mike Galbraith
2001-02-25 13:53 ` Alan Cox
2001-02-25 16:37 ` Erik Mouw
2001-02-25 17:32 ` Erik Mouw
2001-02-26 2:40 ` [PATCH] " Chris Mason
2001-02-26 11:07 ` Erik Mouw
2001-02-26 15:37 ` Erik Mouw
2001-02-26 22:46 ` Ken Moffat
2001-02-25 20:32 ` Chris Mason
2001-02-24 0:34 ` Oliver Teuber
[not found] ` <01022318321302.01755@flash>
2001-02-24 8:40 ` John E. Adams
2001-02-24 15:45 ` Arjan Filius
2001-02-24 16:24 ` [lkml]Re: " thunder7
2001-02-24 17:28 ` Chris Mason
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).