* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 14:18 ` Joshua Schmidlkofer
@ 2001-07-27 14:55 ` Hans Reiser
2001-07-27 15:02 ` Chris Wedgwood
2001-07-28 13:45 ` Matthew Gardiner
2001-07-27 15:06 ` Daniel Phillips
` (2 subsequent siblings)
3 siblings, 2 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-27 14:55 UTC (permalink / raw)
To: Joshua Schmidlkofer; +Cc: kernel
Joshua Schmidlkofer wrote:
>
> I've almost quit using reiser, because everytime I have a power outage, the
> last 2 or three files that I've editted, even ones that I haven't touched in
> a while, will usually be hopelessly corrupted. The '<file>~' that Emacs
> makes is usually fine though. It seems to be that any open file is
> in danger. I don't know if this is normal, or not, but I switched to XFS on
> several machines. I have nothing against reiser. I assumed that these
> problems were due to immaturity....
>
> One more thing - All my computers with Reiser as '/' on them had a
> disturbingly long boot time. From the time when the Redhat startup scripts
> began, it was.... hideously slow. I thought nothing of it, blaming bash,
Don't use RedHat with ReiserFS, they screw things up so many ways.....
For instance, they compile it with the wrong options set, their boot scripts are wrong, they just
shovel software onto the CD.
Use SuSE, and trust me, ReiserFS will boot faster than ext2.
Actually, I am curious as to exactly how they manage to make ReiserFS boot longer than ext2. Do
they run fsck or what?
Hans
> etc, Until I switched to ext2 on all those. Now the boot time is... SUPER
> fast. [3 Computers, 1 K6-2, a Pentium III, and a Pentium II, all 128+meg,
> and IDE] I currently have 3 computers running reiserfs left, all are using
> MySQL databases.
> Once, I lost power in on my SQL box, [it was blessedly during a
> period of no use.] I had to rebuild all the indexes. Not only THAT, but
> what happens to that box if I lose power whilst in the middle of operations?
> I am working on the migration plan to move that to XFS because of these
> concerns. [However, I am doing a better job of testing with XFS first.]
>
> I think that Reiser is cool, and has neat ideology, but I am un-nerved by
> this behaviour.
>
> js
>
> >
> > Yup. I know ext2 can do it. I expect a filesystem to not foul up my data
> > when something happens. Especially not shuffle around sectors in several
> > files. I can understand that the changes I made are not on disc, I can
> > even understand it if my files are gone, but not when it corrupts my data.
> > That just plain sucks.
> >
> > A friend of mine has had crashes as well (not reiser related btw), where
> > files he was using at the time suddenly contained different pieces of
> > different files. It's just plain annoying. The reason why *I* use(d)
> > reiserfs was the fact that I thought that it would protect my data when
> > something does crash. From my experience, it doesn't, and I'd rather wait
> > a couple of minutes for ext2 to fsck than use reiserfs and be sure I can
> > start all over again.
> >
> > Regards,
> >
> > Bas Vermeulen
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 14:55 ` Hans Reiser
@ 2001-07-27 15:02 ` Chris Wedgwood
2001-07-27 16:06 ` Henning P. Schmiedehausen
2001-07-27 22:02 ` Luigi Genoni
2001-07-28 13:45 ` Matthew Gardiner
1 sibling, 2 replies; 94+ messages in thread
From: Chris Wedgwood @ 2001-07-27 15:02 UTC (permalink / raw)
To: Hans Reiser; +Cc: Joshua Schmidlkofer, kernel
On Fri, Jul 27, 2001 at 06:55:09PM +0400, Hans Reiser wrote:
Don't use RedHat with ReiserFS, they screw things up so many
ways.....
For instance, they compile it with the wrong options set, their
boot scripts are wrong, they just shovel software onto the CD.
Use SuSE, and trust me, ReiserFS will boot faster than ext2.
Actually, I am curious as to exactly how they manage to make
ReiserFS boot longer than ext2. Do they run fsck or what?
FWIW, Debian although it doesn't support reiserfs "out of the box" at
present, works flawlessly for a large number of people I know. I also
hear Mandrake 7.2 and 8.0 work pretty nice if you want a pointy-clicky
experience :)
Since so many people seem to run RedHat, perhaps it's worth someone
determining exactly what is busted with their init scripts or whatever
that makes reiserfs barf more often that with other distributions.
--cw
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 15:02 ` Chris Wedgwood
@ 2001-07-27 16:06 ` Henning P. Schmiedehausen
2001-07-27 22:02 ` Luigi Genoni
1 sibling, 0 replies; 94+ messages in thread
From: Henning P. Schmiedehausen @ 2001-07-27 16:06 UTC (permalink / raw)
To: linux-kernel
Chris Wedgwood <cw@f00f.org> writes:
>FWIW, Debian although it doesn't support reiserfs "out of the box" at
>present, works flawlessly for a large number of people I know. I also
>hear Mandrake 7.2 and 8.0 work pretty nice if you want a pointy-clicky
>experience :)
>Since so many people seem to run RedHat, perhaps it's worth someone
>determining exactly what is busted with their init scripts or whatever
>that makes reiserfs barf more often that with other distributions.
There is nothing wrong with RedHat init scripts.
I run RH 6.2 on my self-rolled 2.2.x kernels and they boot ReiserFS
fine and neither faster nor slower than ext2. Nothing wrong with
RedHat here.
I consider a vendor that does not always ship "latest and greatest"
but tries to stress test its software superior to one that crams out
one new release every three months. And if they enable paranoia mode
in the experimental sections of the kernel: Fine! Goes well with my
philosophy of server running. Leads to 500+ days uptime.
I dropped out of ReiserFS again, however, because of unexplained
problems with various user space applications (PostgreSQL 6.5.3 and
7.x or Highwind oops bCandid oops Software.Com oops Highwind-again
Cyclone and Typhoon) that are heavy thread and mmap() users.
I use ReiserFS for my ftp-data-caroussels and as spool and staging
disks. Not for system disks that contain binaries or performance
critical application disks. That works fine.
Regards
Henning
--
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH hps@intermeta.de
Am Schwabachgrund 22 Fon.: 09131 / 50654-0 info@intermeta.de
D-91054 Buckenhof Fax.: 09131 / 50654-20
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 15:02 ` Chris Wedgwood
2001-07-27 16:06 ` Henning P. Schmiedehausen
@ 2001-07-27 22:02 ` Luigi Genoni
1 sibling, 0 replies; 94+ messages in thread
From: Luigi Genoni @ 2001-07-27 22:02 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: Hans Reiser, Joshua Schmidlkofer, kernel
On Sat, 28 Jul 2001, Chris Wedgwood wrote:
> On Fri, Jul 27, 2001 at 06:55:09PM +0400, Hans Reiser wrote:
>
> Don't use RedHat with ReiserFS, they screw things up so many
> ways.....
>
> For instance, they compile it with the wrong options set, their
> boot scripts are wrong, they just shovel software onto the CD.
>
> Use SuSE, and trust me, ReiserFS will boot faster than ext2.
>
> Actually, I am curious as to exactly how they manage to make
> ReiserFS boot longer than ext2. Do they run fsck or what?
>
> FWIW, Debian although it doesn't support reiserfs "out of the box" at
> present, works flawlessly for a large number of people I know. I also
> hear Mandrake 7.2 and 8.0 work pretty nice if you want a pointy-clicky
> experience :)
>
I could add that also slackware is just faster with / with reiserFS
than with ext2.
But i saw that some of RH init script are, how can I say, redundant....
Luigi
> Since so many people seem to run RedHat, perhaps it's worth someone
> determining exactly what is busted with their init scripts or whatever
> that makes reiserfs barf more often that with other distributions.
>
>
>
> --cw
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 14:55 ` Hans Reiser
2001-07-27 15:02 ` Chris Wedgwood
@ 2001-07-28 13:45 ` Matthew Gardiner
2001-07-28 16:15 ` Hans Reiser
` (2 more replies)
1 sibling, 3 replies; 94+ messages in thread
From: Matthew Gardiner @ 2001-07-28 13:45 UTC (permalink / raw)
To: Hans Reiser, Joshua Schmidlkofer; +Cc: kernel
On Saturday 28 July 2001 02:55, Hans Reiser wrote:
> Joshua Schmidlkofer wrote:
> > I've almost quit using reiser, because everytime I have a power outage,
> > the last 2 or three files that I've editted, even ones that I haven't
> > touched in a while, will usually be hopelessly corrupted. The '<file>~'
> > that Emacs makes is usually fine though. It seems to be that any open
> > file is in danger. I don't know if this is normal, or not, but I
> > switched to XFS on several machines. I have nothing against reiser. I
> > assumed that these problems were due to immaturity....
> >
> > One more thing - All my computers with Reiser as '/' on them had a
> > disturbingly long boot time. From the time when the Redhat startup
> > scripts began, it was.... hideously slow. I thought nothing of it,
> > blaming bash,
>
> Don't use RedHat with ReiserFS, they screw things up so many ways.....
>
> For instance, they compile it with the wrong options set, their boot
> scripts are wrong, they just shovel software onto the CD.
>
> Use SuSE, and trust me, ReiserFS will boot faster than ext2.
>
> Actually, I am curious as to exactly how they manage to make ReiserFS boot
> longer than ext2. Do they run fsck or what?
>
> Hans
Regards to the ReiserFS. Something more spookie, OpenLinux (no boos and
hisses please ;) ), they have ReiserFS as a module, yet, when I have the root
partition as reiser I have no problems, voo doo magic perhaps? because when I
compiled 2.4.7 w/ ReiserFS as a module, the boot forks up.
Regarding the last comment, I think Redhat and Caldera have debugging enable
(God knows why?), well, Caldera definately dones, after having a look at
their default kernel configuration, hence, when I recompiled my kernel to
2.4.7, threw the reiserFS into the guts of the kernel with debugging turned
off, there was a speed increase.
Also, to speed it up, I have heard a urban myth (I am not too sure whether it
is true), you add the tag notail. A little more disk space is used, however,
apparently, it is meant to speed up access.
Matthew Gardiner
--
WARNING:
This email was written on an OS using the viral 'GPL' as its license.
Please check with Bill Gates before continuing to read this email/posting.
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-28 13:45 ` Matthew Gardiner
@ 2001-07-28 16:15 ` Hans Reiser
2001-07-28 16:45 ` Marcus Meissner
2001-07-28 16:45 ` Christoph Hellwig
2 siblings, 0 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-28 16:15 UTC (permalink / raw)
To: Matthew Gardiner; +Cc: Joshua Schmidlkofer, kernel
Matthew Gardiner wrote:
>
> On Saturday 28 July 2001 02:55, Hans Reiser wrote:
> > Joshua Schmidlkofer wrote:
> > > I've almost quit using reiser, because everytime I have a power outage,
> > > the last 2 or three files that I've editted, even ones that I haven't
> > > touched in a while, will usually be hopelessly corrupted. The '<file>~'
> > > that Emacs makes is usually fine though. It seems to be that any open
> > > file is in danger. I don't know if this is normal, or not, but I
> > > switched to XFS on several machines. I have nothing against reiser. I
> > > assumed that these problems were due to immaturity....
> > >
> > > One more thing - All my computers with Reiser as '/' on them had a
> > > disturbingly long boot time. From the time when the Redhat startup
> > > scripts began, it was.... hideously slow. I thought nothing of it,
> > > blaming bash,
> >
> > Don't use RedHat with ReiserFS, they screw things up so many ways.....
> >
> > For instance, they compile it with the wrong options set, their boot
> > scripts are wrong, they just shovel software onto the CD.
> >
> > Use SuSE, and trust me, ReiserFS will boot faster than ext2.
> >
> > Actually, I am curious as to exactly how they manage to make ReiserFS boot
> > longer than ext2. Do they run fsck or what?
> >
> > Hans
>
> Regards to the ReiserFS. Something more spookie, OpenLinux (no boos and
> hisses please ;) ), they have ReiserFS as a module, yet, when I have the root
> partition as reiser I have no problems, voo doo magic perhaps? because when I
> compiled 2.4.7 w/ ReiserFS as a module, the boot forks up.
Perhaps there is a problem in which the reiserfs module does not get loaded
before you need to read the root partition?
If you isolate the problem to where you think it is a reiserfs bug, please let
me know. It sounds like not.
>
> Also, to speed it up, I have heard a urban myth (I am not too sure whether it
> is true), you add the tag notail. A little more disk space is used, however,
> apparently, it is meant to speed up access.
This is entirely correct. Moving tails around costs performance, ReiserFS
cannot give you something for nothing in this respect.
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-28 13:45 ` Matthew Gardiner
2001-07-28 16:15 ` Hans Reiser
@ 2001-07-28 16:45 ` Marcus Meissner
2001-07-28 16:45 ` Christoph Hellwig
2 siblings, 0 replies; 94+ messages in thread
From: Marcus Meissner @ 2001-07-28 16:45 UTC (permalink / raw)
To: Matthew Gardiner, linux-kernel
In article <01072901450000.02683@kiwiunixman.nodomain.nowhere> you wrote:
> Regards to the ReiserFS. Something more spookie, OpenLinux (no boos and
> hisses please ;) ), they have ReiserFS as a module, yet, when I have the root
> partition as reiser I have no problems, voo doo magic perhaps? because when I
> compiled 2.4.7 w/ ReiserFS as a module, the boot forks up.
We have the reiserfs module in the initial ramdisk in such setups.
You need to recreate the initrd in those cases.
(Run "/usr/libexec/modules/mkinitrd.sh 2.4.7" in the /boot directory, this
will create /boot/initrd-2.4.7.gz.)
> Regarding the last comment, I think Redhat and Caldera have debugging enable
> (God knows why?), well, Caldera definately dones, after having a look at
> their default kernel configuration, hence, when I recompiled my kernel to
> 2.4.7, threw the reiserFS into the guts of the kernel with debugging turned
> off, there was a speed increase.
ReiserFS is experimental in the 2.4 series, thats why we ship with a big
disclaimer and with checking enabled.
(And before you argue again, we ship 2.4.2-ac26. Since then several major
bugs have been found in reiserfs, including the knfsd lossage.)
Ciao, Marcus
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-28 13:45 ` Matthew Gardiner
2001-07-28 16:15 ` Hans Reiser
2001-07-28 16:45 ` Marcus Meissner
@ 2001-07-28 16:45 ` Christoph Hellwig
2001-07-29 10:19 ` Matthew Gardiner
2001-07-30 10:08 ` Hans Reiser
2 siblings, 2 replies; 94+ messages in thread
From: Christoph Hellwig @ 2001-07-28 16:45 UTC (permalink / raw)
To: Matthew Gardiner; +Cc: kernel, Hans Reiser, Joshua Schmidlkofer
Hi Matthew,
In article <01072901450000.02683@kiwiunixman.nodomain.nowhere> you wrote:
> Regards to the ReiserFS. Something more spookie, OpenLinux (no boos and
> hisses please ;) ), they have ReiserFS as a module, yet, when I have the root
> partition as reiser I have no problems, voo doo magic perhaps? because when I
> compiled 2.4.7 w/ ReiserFS as a module, the boot forks up.
Well, as reiserfs is a module it needs to be on the initrd. The install
of the kernel kernel RPM automatically creates a new initrd which includes
the modules in /etc/modules/rootfs. If you don't create a new initrd your
modular reiserfs setup will of course fail.
> Regarding the last comment, I think Redhat and Caldera have debugging enable
> (God knows why?), well, Caldera definately dones, after having a look at
> their default kernel configuration, hence, when I recompiled my kernel to
> 2.4.7, threw the reiserFS into the guts of the kernel with debugging turned
> off, there was a speed increase.
Reiserfs as implemented in the 2.4.2-based kernel of OpenLinux 3.1 is
everything but stable and has a lot of issues (e.g. NFS-exporting doesn't
work). That is the reason why it is a) marked experimental and is completly
unsupported (and that is written _big_ _fat_ in manuals and similar stuff)
and b) has debugging enabled to have the additional sanity checks that are
under this option and give addtional hints if reiserfs fails again.
Christoph
--
Of course it doesn't work. We've performed a software upgrade.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-28 16:45 ` Christoph Hellwig
@ 2001-07-29 10:19 ` Matthew Gardiner
2001-07-29 11:04 ` Chris Wedgwood
2001-07-30 10:08 ` Hans Reiser
1 sibling, 1 reply; 94+ messages in thread
From: Matthew Gardiner @ 2001-07-29 10:19 UTC (permalink / raw)
To: Christoph Hellwig, Matthew Gardiner
Cc: kernel, Hans Reiser, Joshua Schmidlkofer
On Sunday 29 July 2001 04:45, Christoph Hellwig wrote:
> Hi Matthew,
>
> In article <01072901450000.02683@kiwiunixman.nodomain.nowhere> you wrote:
> > Regards to the ReiserFS. Something more spookie, OpenLinux (no boos and
> > hisses please ;) ), they have ReiserFS as a module, yet, when I have the
> > root partition as reiser I have no problems, voo doo magic perhaps?
> > because when I compiled 2.4.7 w/ ReiserFS as a module, the boot forks up.
>
> Well, as reiserfs is a module it needs to be on the initrd. The install
> of the kernel kernel RPM automatically creates a new initrd which includes
> the modules in /etc/modules/rootfs. If you don't create a new initrd your
> modular reiserfs setup will of course fail.
>
> > Regarding the last comment, I think Redhat and Caldera have debugging
> > enable (God knows why?), well, Caldera definately dones, after having a
> > look at their default kernel configuration, hence, when I recompiled my
> > kernel to 2.4.7, threw the reiserFS into the guts of the kernel with
> > debugging turned off, there was a speed increase.
>
> Reiserfs as implemented in the 2.4.2-based kernel of OpenLinux 3.1 is
> everything but stable and has a lot of issues (e.g. NFS-exporting doesn't
> work). That is the reason why it is a) marked experimental and is
> completly unsupported (and that is written _big_ _fat_ in manuals and
> similar stuff) and b) has debugging enabled to have the additional sanity
> checks that are under this option and give addtional hints if reiserfs
> fails again.
I've upgraded to 2.4.7 without any problems.
Regard to the above, that is, moduler ReiserFS, its not really an issue, as
compiling into the kernel hasn't caused any problems.
Just a little suggestion. Is it possible to offer "kernel binary upgrades"
every other major release, for example, it shipped with 2.4.2, hence, the
next upgrade to be release, 2.4.4 then 2.4.6 then 2.4.8. I can compile
everthing however, I know a couple of people too scared to get down into the
nitty gritty of Linux and compile their own kernel.
Matthew Gardiner
--
WARNING:
This email was written on an OS using the viral 'GPL' as its license.
Please check with Bill Gates before continuing to read this email/posting.
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-29 10:19 ` Matthew Gardiner
@ 2001-07-29 11:04 ` Chris Wedgwood
0 siblings, 0 replies; 94+ messages in thread
From: Chris Wedgwood @ 2001-07-29 11:04 UTC (permalink / raw)
To: Matthew Gardiner
Cc: Christoph Hellwig, kernel, Hans Reiser, Joshua Schmidlkofer
On Sun, Jul 29, 2001 at 10:19:32PM +1200, Matthew Gardiner wrote:
Just a little suggestion. Is it possible to offer "kernel binary
upgrades" every other major release, for example, it shipped with
2.4.2, hence, the next upgrade to be release, 2.4.4 then 2.4.6
then 2.4.8. I can compile everthing however, I know a couple of
people too scared to get down into the nitty gritty of Linux and
compile their own kernel.
Umm.. most (if not all) vendors provide kernel binary upgrades. If
these are not frequent enough for your needs, you need to complain to
them.
--cw
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-28 16:45 ` Christoph Hellwig
2001-07-29 10:19 ` Matthew Gardiner
@ 2001-07-30 10:08 ` Hans Reiser
2001-07-30 19:06 ` Christoph Hellwig
1 sibling, 1 reply; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 10:08 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Matthew Gardiner, kernel, Joshua Schmidlkofer
Christoph Hellwig wrote:
>
> Reiserfs as implemented in the 2.4.2-based kernel of OpenLinux 3.1 is
> everything but stable and has a lot of issues (e.g. NFS-exporting doesn't
> work). That is the reason why it is a) marked experimental and is completly
> unsupported (and that is written _big_ _fat_ in manuals and similar stuff)
> and b) has debugging enabled to have the additional sanity checks that are
> under this option and give addtional hints if reiserfs fails again.
The debugging won't prevent a single crash, it will only print a diagnostic that
might help to understand why it crashed. It makes zero sense for a distro to
have it on, and I think we make that pretty clear in the help button. It would
be nice if distros read the help buttons before selecting options when
configuring their kernels.:-/
I make no claims that users should use ReiserFS as it is in a 2.4.2 kernel....
>
> Christoph
>
> --
> Of course it doesn't work. We've performed a software upgrade.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 10:08 ` Hans Reiser
@ 2001-07-30 19:06 ` Christoph Hellwig
2001-07-30 20:30 ` Hans Reiser
0 siblings, 1 reply; 94+ messages in thread
From: Christoph Hellwig @ 2001-07-30 19:06 UTC (permalink / raw)
To: Hans Reiser
Cc: Christoph Hellwig, Matthew Gardiner, kernel, Joshua Schmidlkofer
On Mon, Jul 30, 2001 at 02:08:17PM +0400, Hans Reiser wrote:
> Christoph Hellwig wrote:
> >
>
> > Reiserfs as implemented in the 2.4.2-based kernel of OpenLinux 3.1 is
> > everything but stable and has a lot of issues (e.g. NFS-exporting doesn't
> > work). That is the reason why it is a) marked experimental and is completly
> > unsupported (and that is written _big_ _fat_ in manuals and similar stuff)
> > and b) has debugging enabled to have the additional sanity checks that are
> > under this option and give addtional hints if reiserfs fails again.
>
> The debugging won't prevent a single crash, it will only print a diagnostic that
> might help to understand why it crashed.
I don't know when you took a look at you code the last time, but when
I did some time before the COL 3.1 release, there were lots of places
in the reiserfs code where functions assumed that they have valid
arguments when compiled without debugging and did the check explicitly
when compiled with. Given the state the reiserfs code is in I really
prefer to see this option turned on.
> It makes zero sense for a distro to
> have it on, and I think we make that pretty clear in the help button. It would
> be nice if distros read the help buttons before selecting options when
> configuring their kernels.:-/
Well sometimes it's even better to take a look at the code..
Christoph
> I make no claims that users should use ReiserFS as it is in a 2.4.2 kernel....
No one said that (and even if I wouldn't believe him).
Christoph
--
Of course it doesn't work. We've performed a software upgrade.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 19:06 ` Christoph Hellwig
@ 2001-07-30 20:30 ` Hans Reiser
2001-07-30 20:49 ` Christoph Hellwig
2001-08-02 13:44 ` Pavel Machek
0 siblings, 2 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 20:30 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Matthew Gardiner, kernel, Joshua Schmidlkofer
Christoph Hellwig wrote:
>
> On Mon, Jul 30, 2001 at 02:08:17PM +0400, Hans Reiser wrote:
> > Christoph Hellwig wrote:
> > >
> >
> > > Reiserfs as implemented in the 2.4.2-based kernel of OpenLinux 3.1 is
> > > everything but stable and has a lot of issues (e.g. NFS-exporting doesn't
> > > work). That is the reason why it is a) marked experimental and is completly
> > > unsupported (and that is written _big_ _fat_ in manuals and similar stuff)
> > > and b) has debugging enabled to have the additional sanity checks that are
> > > under this option and give addtional hints if reiserfs fails again.
> >
> > The debugging won't prevent a single crash, it will only print a diagnostic that
> > might help to understand why it crashed.
>
> I don't know when you took a look at you code the last time, but when
> I did some time before the COL 3.1 release, there were lots of places
> in the reiserfs code where functions assumed that they have valid
> arguments when compiled without debugging and did the check explicitly
> when compiled with. Given the state the reiserfs code is in I really
> prefer to see this option turned on.
But there is not one where they recover from invalid arguments without a panic
(unless I failed to notice something), so it gets you nothing except a message
that we the developers will find more informative when trying to find what made
it crash. We check invalid arguments at entry to reiserfs, and for those we
error gracefully. (We also have some checks for garbage blocks having been
read, and we have made some efforts to error gracefully from those.)
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 20:30 ` Hans Reiser
@ 2001-07-30 20:49 ` Christoph Hellwig
2001-07-30 21:05 ` Hans Reiser
` (2 more replies)
2001-08-02 13:44 ` Pavel Machek
1 sibling, 3 replies; 94+ messages in thread
From: Christoph Hellwig @ 2001-07-30 20:49 UTC (permalink / raw)
To: Hans Reiser; +Cc: linux-kernel
On Tue, Jul 31, 2001 at 12:30:12AM +0400, Hans Reiser wrote:
> But there is not one where they recover from invalid arguments without a panic
> (unless I failed to notice something),
Right.
> so it gets you nothing except a message
> that we the developers will find more informative when trying to find what made
> it crash.
Nope. It does a reiserfs_panic instead of letting the wrong arguments
slipping into lower layers and possibly on disk and thus corrupting data.
And in my opinion correct data is much more worth than one crash more or
less (especially with a journaling filesystem).
Christoph
--
Whip me. Beat me. Make me maintain AIX.
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 20:49 ` Christoph Hellwig
@ 2001-07-30 21:05 ` Hans Reiser
2001-07-30 21:29 ` Christoph Hellwig
2001-07-30 21:59 ` Rik van Riel
2001-07-30 21:13 ` Hans Reiser
2001-07-30 21:21 ` Hans Reiser
2 siblings, 2 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 21:05 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-kernel, Vitaly Fertman
Christoph Hellwig wrote:
>
> On Tue, Jul 31, 2001 at 12:30:12AM +0400, Hans Reiser wrote:
> > But there is not one where they recover from invalid arguments without a panic
> > (unless I failed to notice something),
>
> Right.
>
> > so it gets you nothing except a message
> > that we the developers will find more informative when trying to find what made
> > it crash.
>
> Nope. It does a reiserfs_panic instead of letting the wrong arguments
> slipping into lower layers and possibly on disk and thus corrupting data.
>
> And in my opinion correct data is much more worth than one crash more or
> less (especially with a journaling filesystem).
>
> Christoph
>
> --
> Whip me. Beat me. Make me maintain AIX.
There is nothing like a distro maintainer overriding the design decisions made
by the lead architect of a package, not believing that said architect knows what
the fuck he is doing.
We will make this unusable by you from this point onwards. Vitaly, I told you
what to do weeks ago in this regard, do it today.
Does it get worse than shovelware? I suppose it does....
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:05 ` Hans Reiser
@ 2001-07-30 21:29 ` Christoph Hellwig
2001-07-30 21:44 ` Hans Reiser
2001-07-30 21:48 ` Hans Reiser
2001-07-30 21:59 ` Rik van Riel
1 sibling, 2 replies; 94+ messages in thread
From: Christoph Hellwig @ 2001-07-30 21:29 UTC (permalink / raw)
To: Hans Reiser; +Cc: linux-kernel, Vitaly Fertman
On Tue, Jul 31, 2001 at 01:05:11AM +0400, Hans Reiser wrote:
> There is nothing like a distro maintainer
[NOTE: I do not maintain the Caldera kernel RPM, but I was
involved in the decision to turn reiserfs debugging on]
> overriding the design decisions made
> by the lead architect of a package, not believing that said architect knows what
> the fuck he is doing.
Reiserfs lately had a lot of stability issues, reports of data corruption
and as you said before you don't considere the reiserfs version in 2.4.2-ac
stable yourself.
The averange user will not blame you if he loses data through a problem
in reiserfs but the distribtuion, even if this filesystem is clearly
marked unsupported.
>
> We will make this unusable by you from this point onwards.
>
I do not see the debug kernel removed from the official kernel tree
before reiserfs has proven known stable.
Of course there is still the option of CONFIG_REISERFS_FS=n if you
intentionally want to make your filesystem less acceptable..
Christoph
--
Of course it doesn't work. We've performed a software upgrade.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:29 ` Christoph Hellwig
@ 2001-07-30 21:44 ` Hans Reiser
2001-07-30 21:48 ` Hans Reiser
1 sibling, 0 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 21:44 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-kernel, Vitaly Fertman
Christoph Hellwig wrote:
>
> On Tue, Jul 31, 2001 at 01:05:11AM +0400, Hans Reiser wrote:
> > There is nothing like a distro maintainer
>
> [NOTE: I do not maintain the Caldera kernel RPM, but I was
> involved in the decision to turn reiserfs debugging on]
>
> > overriding the design decisions made
> > by the lead architect of a package, not believing that said architect knows what
> > the fuck he is doing.
>
> Reiserfs lately had a lot of stability issues, reports of data corruption
> and as you said before you don't considere the reiserfs version in 2.4.2-ac
> stable yourself.
>
> The averange user will not blame you if he loses data through a problem
> in reiserfs but the distribtuion, even if this filesystem is clearly
> marked unsupported.
>
> >
> > We will make this unusable by you from this point onwards.
> >
>
> I do not see the debug kernel removed from the official kernel tree
> before reiserfs has proven known stable.
>
> Of course there is still the option of CONFIG_REISERFS_FS=n if you
> intentionally want to make your filesystem less acceptable..
>
> Christoph
>
> --
> Of course it doesn't work. We've performed a software upgrade.
We'd rather be off, than have debug on. Debug is not designed to be on, unless
you are debugging. Most users don't know that you have turned debug on. They
just think ReiserFS isn't as fast as their SuSE using friends say it is. We
will make sure users know that their distro has turned debug on.
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:29 ` Christoph Hellwig
2001-07-30 21:44 ` Hans Reiser
@ 2001-07-30 21:48 ` Hans Reiser
2001-07-30 21:57 ` Chris Wedgwood
2001-07-31 7:45 ` Henning P. Schmiedehausen
1 sibling, 2 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 21:48 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-kernel, Vitaly Fertman
Christoph Hellwig wrote:
>
> On Tue, Jul 31, 2001 at 01:05:11AM +0400, Hans Reiser wrote:
> > There is nothing like a distro maintainer
>
> [NOTE: I do not maintain the Caldera kernel RPM, but I was
> involved in the decision to turn reiserfs debugging on]
>
> > overriding the design decisions made
> > by the lead architect of a package, not believing that said architect knows what
> > the fuck he is doing.
>
> Reiserfs lately had a lot of stability issues, reports of data corruption
> and as you said before you don't considere the reiserfs version in 2.4.2-ac
> stable yourself.
I also don't consider any 2.4 prior to 2.4.4 to be stable, and I don't consider
2.4.4 to be especially stable but it is usable.
Shipping 2.4.2 is something you and RedHat did for understandable marketing
reasons. SuSE waited for 2.4.4.
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:48 ` Hans Reiser
@ 2001-07-30 21:57 ` Chris Wedgwood
2001-07-30 21:58 ` Christoph Hellwig
2001-07-31 7:45 ` Henning P. Schmiedehausen
1 sibling, 1 reply; 94+ messages in thread
From: Chris Wedgwood @ 2001-07-30 21:57 UTC (permalink / raw)
To: Hans Reiser; +Cc: Christoph Hellwig, linux-kernel, Vitaly Fertman
On Tue, Jul 31, 2001 at 01:48:03AM +0400, Hans Reiser wrote:
I also don't consider any 2.4 prior to 2.4.4 to be stable, and I
don't consider 2.4.4 to be especially stable but it is usable.
Shipping 2.4.2 is something you and RedHat did for understandable
marketing reasons. SuSE waited for 2.4.4.
This is a myth.
As has been explained --- RedHat does _NOT_ ship a stock
kernel. Redhat 2.4.2 is not the same as stock/linux 2.4.2 so you are
comparing apples with oranges.
--cw
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:57 ` Chris Wedgwood
@ 2001-07-30 21:58 ` Christoph Hellwig
0 siblings, 0 replies; 94+ messages in thread
From: Christoph Hellwig @ 2001-07-30 21:58 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: Hans Reiser, linux-kernel, Vitaly Fertman
On Tue, Jul 31, 2001 at 09:57:21AM +1200, Chris Wedgwood wrote:
> On Tue, Jul 31, 2001 at 01:48:03AM +0400, Hans Reiser wrote:
>
> I also don't consider any 2.4 prior to 2.4.4 to be stable, and I
> don't consider 2.4.4 to be especially stable but it is usable.
>
> Shipping 2.4.2 is something you and RedHat did for understandable
> marketing reasons. SuSE waited for 2.4.4.
>
> This is a myth.
>
> As has been explained --- RedHat does _NOT_ ship a stock
> kernel. Redhat 2.4.2 is not the same as stock/linux 2.4.2 so you are
> comparing apples with oranges.
The same is true for Caldera, btw.
Christoph
--
Of course it doesn't work. We've performed a software upgrade.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:48 ` Hans Reiser
2001-07-30 21:57 ` Chris Wedgwood
@ 2001-07-31 7:45 ` Henning P. Schmiedehausen
2001-07-31 9:55 ` Hans Reiser
1 sibling, 1 reply; 94+ messages in thread
From: Henning P. Schmiedehausen @ 2001-07-31 7:45 UTC (permalink / raw)
To: linux-kernel
Hans Reiser <reiser@namesys.com> writes:
>I also don't consider any 2.4 prior to 2.4.4 to be stable, and I don't consider
>2.4.4 to be especially stable but it is usable.
>Shipping 2.4.2 is something you and RedHat did for understandable marketing
>reasons. SuSE waited for 2.4.4.
Well, SuSE shipped 2.4.2 on their 7.1 release and I didn't see you
jumping up and down in anger for "shipping an unstable release":
ftp://ftp.suse.com/pub/suse/i386/7.1/full-names/i386/k_i386_24-2.4.2-12.i386.rpm
Ah, but then again, you got money from them... He who pays the piper,
calls the tune. And you're a fine piper.
Sorry, but I can't take you seriously. Especially as you're _so_
_obviously_ vendor biased, that it stinks.
Regards
Henning
--
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH hps@intermeta.de
Am Schwabachgrund 22 Fon.: 09131 / 50654-0 info@intermeta.de
D-91054 Buckenhof Fax.: 09131 / 50654-20
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 7:45 ` Henning P. Schmiedehausen
@ 2001-07-31 9:55 ` Hans Reiser
2001-07-31 10:24 ` Arjan van de Ven
2001-07-31 10:24 ` Anders Eriksson
0 siblings, 2 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-31 9:55 UTC (permalink / raw)
To: hps; +Cc: linux-kernel
"Henning P. Schmiedehausen" wrote:
>
> Hans Reiser <reiser@namesys.com> writes:
>
> >I also don't consider any 2.4 prior to 2.4.4 to be stable, and I don't consider
> >2.4.4 to be especially stable but it is usable.
>
> >Shipping 2.4.2 is something you and RedHat did for understandable marketing
> >reasons. SuSE waited for 2.4.4.
>
> Well, SuSE shipped 2.4.2 on their 7.1 release and I didn't see you
> jumping up and down in anger for "shipping an unstable release":
>
> ftp://ftp.suse.com/pub/suse/i386/7.1/full-names/i386/k_i386_24-2.4.2-12.i386.rpm
>
> Ah, but then again, you got money from them... He who pays the piper,
> calls the tune. And you're a fine piper.
>
> Sorry, but I can't take you seriously. Especially as you're _so_
> _obviously_ vendor biased, that it stinks.
>
> Regards
> Henning
>
> --
> Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer
> INTERMETA - Gesellschaft fuer Mehrwertdienste mbH hps@intermeta.de
>
> Am Schwabachgrund 22 Fon.: 09131 / 50654-0 info@intermeta.de
> D-91054 Buckenhof Fax.: 09131 / 50654-20
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
I stand corrected, this means that all of the distros shipped 2.4 before they
should have. (I use SuSE 7.2. I thought that 7.1 had a 2.2 kernel as the
default, but I guess I was wrong.)
SuSE has its flaws also. I have complained to them about the yast license, for
instance. (I think the best single thing they could do for SuSE sales is change
that license.) All the distros I know of except debian like to put kernel
patches into their distros first. You would think they would want them in the
kernel first so that they could know they are stable, but that would give them
no "advantage". Sigh. I suppose there are much worse things they could do.
Hans
PS
I don't get money from SuSE anymore, I get it from DARPA. I do run SuSE on my
computer though, which is probably enough to bias me.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 9:55 ` Hans Reiser
@ 2001-07-31 10:24 ` Arjan van de Ven
2001-07-31 10:24 ` Anders Eriksson
1 sibling, 0 replies; 94+ messages in thread
From: Arjan van de Ven @ 2001-07-31 10:24 UTC (permalink / raw)
To: Hans Reiser, linux-kernel
Hans Reiser wrote:
> All the distros I know of except debian like to put kernel
> patches into their distros first. You would think they would want them in the
> kernel first so that they could know they are stable, but that would give them
> no "advantage". Sigh. I suppose there are much worse things they could do.
In the future, please check your facts more thoroughly. Almost all of
the patches in the Red Hat
2.4.2-2 kernel were bugfixes from later upstream kernel releases.
INCLUDING reiserfs corruption fixes.
Caldera, Suse, Conectiva and Mandrake all do the same. Ok so we all
differ slightly in which bugfixes
each distro picks, and which base version we start with. That's a matter
of taste. And fwiw,
the 2.4.2-2 Red Hat shipped was closer to 2.4.3-acX than the actual
2.4.2, due to the dozens and
dozens of bugfixes applied from these newer kernels. (and yes we do test
our kernels. hard. That's
why we can't recommend reiserfs on anything non Little Endian or 64 bit
right now)
Please take your false conspiracy theories to some place where they are
more appropriate.
Greetings,
Arjan van de Ven
Red Hat Linux kernel maintainer
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 9:55 ` Hans Reiser
2001-07-31 10:24 ` Arjan van de Ven
@ 2001-07-31 10:24 ` Anders Eriksson
2001-07-31 10:32 ` Chris Wedgwood
2001-07-31 17:01 ` [OT] " J Sloan
1 sibling, 2 replies; 94+ messages in thread
From: Anders Eriksson @ 2001-07-31 10:24 UTC (permalink / raw)
To: linux-kernel
Side note. I vaguely recall a distribution who's name has escaped me since.
Thier selling point was "It's harder to install" and they claimed not to patch
any source. "If it's good enough for the author, it's good enough for us".
Might be worth checking out. If someone has a disro name for it, please...
/A
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:05 ` Hans Reiser
2001-07-30 21:29 ` Christoph Hellwig
@ 2001-07-30 21:59 ` Rik van Riel
2001-07-30 22:34 ` Hans Reiser
2001-07-30 22:41 ` ReiserFS / 2.4.6 / Data Corruption Kip Macy
1 sibling, 2 replies; 94+ messages in thread
From: Rik van Riel @ 2001-07-30 21:59 UTC (permalink / raw)
To: Hans Reiser; +Cc: Christoph Hellwig, linux-kernel, Vitaly Fertman
On Tue, 31 Jul 2001, Hans Reiser wrote:
> Christoph Hellwig wrote:
> >
> > Nope. It does a reiserfs_panic instead of letting the wrong arguments
> > slipping into lower layers and possibly on disk and thus corrupting data.
> >
> > And in my opinion correct data is much more worth than one crash more or
> > less (especially with a journaling filesystem).
>
> There is nothing like a distro maintainer overriding the design
> decisions made by the lead architect of a package, not believing
> that said architect knows what the fuck he is doing.
Are you actually saying you don't care about user's data,
or is it just my imagination ?
(I hope it's my imagination ...)
cheers,
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com/
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:59 ` Rik van Riel
@ 2001-07-30 22:34 ` Hans Reiser
2001-07-31 11:34 ` David Weinehall
2001-07-30 22:41 ` ReiserFS / 2.4.6 / Data Corruption Kip Macy
1 sibling, 1 reply; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 22:34 UTC (permalink / raw)
To: Rik van Riel; +Cc: Christoph Hellwig, linux-kernel, Vitaly Fertman
Rik van Riel wrote:
>
> On Tue, 31 Jul 2001, Hans Reiser wrote:
> > Christoph Hellwig wrote:
> > >
> > > Nope. It does a reiserfs_panic instead of letting the wrong arguments
> > > slipping into lower layers and possibly on disk and thus corrupting data.
> > >
> > > And in my opinion correct data is much more worth than one crash more or
> > > less (especially with a journaling filesystem).
> >
> > There is nothing like a distro maintainer overriding the design
> > decisions made by the lead architect of a package, not believing
> > that said architect knows what the fuck he is doing.
>
> Are you actually saying you don't care about user's data,
> or is it just my imagination ?
>
> (I hope it's my imagination ...)
>
> cheers,
>
> Rik
> --
> Executive summary of a recent Microsoft press release:
> "we are concerned about the GNU General Public License (GPL)"
>
> http://www.surriel.com/
> http://www.conectiva.com/ http://distro.conectiva.com/
I am saying that you can put so many internal checks into a filesytem that it is
unusable for any real usage. Guess what? ReiserFS does that! But we surround
the checks with a #define. The only limit we have on the checks, is that after
the relevant bug disappears we cut out the ones that make things so slow that it
noticeably inconveniences our debugging. It has to slow things down quite a lot
that we can't stand to wait for it while debugging, but there are some kinds of
checks that you can do that are that slow.
ReiserFS checks more things than the rest of the kernel does. We can do this
because we use the #define, and pay no price for it. You should do this also in
your code....
Every major kernel component should have a #define which if on checks every
imaginable thing the developer can think of to check regardless of how slow it
makes the code go to check it. Then, when users (or at least as usefully,
developers adding a new feature) have bugs in that component, they can turn it
on.
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 22:34 ` Hans Reiser
@ 2001-07-31 11:34 ` David Weinehall
2001-07-31 12:22 ` ReiserFS / 2.4.6 / Data Corruption (patch to cause redhat to unmount reiserfs on halt included) Hans Reiser
0 siblings, 1 reply; 94+ messages in thread
From: David Weinehall @ 2001-07-31 11:34 UTC (permalink / raw)
To: Hans Reiser
Cc: Rik van Riel, Christoph Hellwig, linux-kernel, Vitaly Fertman,
Linus Torvalds
On Tue, Jul 31, 2001 at 02:34:38AM +0400, Hans Reiser wrote:
[snipping earlier discussion]
> I am saying that you can put so many internal checks into a filesytem
> that it is unusable for any real usage. Guess what? ReiserFS does
> that! But we surround the checks with a #define. The only limit we
> have on the checks, is that after the relevant bug disappears we cut
> out the ones that make things so slow that it noticeably
> inconveniences our debugging. It has to slow things down quite a lot
> that we can't stand to wait for it while debugging, but there are some
> kinds of checks that you can do that are that slow.
>
> ReiserFS checks more things than the rest of the kernel does. We can
> do this because we use the #define, and pay no price for it. You
> should do this also in your code....
>
> Every major kernel component should have a #define which if on checks
> every imaginable thing the developer can think of to check regardless
> of how slow it makes the code go to check it. Then, when users (or at
> least as usefully, developers adding a new feature) have bugs in that
> component, they can turn it on.
Ugh! I think you need to have a little chat with Linus about this
opinion of yours on how to use #ifdef / #endif in code... I'm not all
that sure he'll agree with you.
/David
_ _
// David Weinehall <tao@acc.umu.se> /> Northern lights wander \\
// Project MCA Linux hacker // Dance across the winter sky //
\> http://www.acc.umu.se/~tao/ </ Full colour fire </
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption (patch to cause redhat to unmount reiserfs on halt included)
2001-07-31 11:34 ` David Weinehall
@ 2001-07-31 12:22 ` Hans Reiser
2001-07-31 12:37 ` Christoph Hellwig
0 siblings, 1 reply; 94+ messages in thread
From: Hans Reiser @ 2001-07-31 12:22 UTC (permalink / raw)
To: David Weinehall
Cc: Rik van Riel, Christoph Hellwig, linux-kernel, Vitaly Fertman,
Linus Torvalds
David Weinehall wrote:
> > Every major kernel component should have a #define which if on checks
> > every imaginable thing the developer can think of to check regardless
> > of how slow it makes the code go to check it. Then, when users (or at
> > least as usefully, developers adding a new feature) have bugs in that
> > component, they can turn it on.
>
> Ugh! I think you need to have a little chat with Linus about this
> opinion of yours on how to use #ifdef / #endif in code... I'm not all
> that sure he'll agree with you.
I didn't say he would agree with me, in fact I am sure he doesn't alike
assertions in the code. I merely said it should be done.:-) As a final little
quibble, let me mention that nikita has created macros that neatly hide the
#ifdefs, and sent them out for testing.
We will consider pulling all but the essential assertions out of ReiserFS.
Sigh. This is the difference between engineering, and marketing. As an
engineer, I said overengineer the checks so that our testing process will catch
more things, and then #define them out so that there is no performance cost.
Perfectly logical. Then along come the distros, and they turn on debugging,
they don't tell the users that debugging is on, and users think we are slower
than other filesystems when we are just configured exactly as we tell the users
not to configure us, sigh.
I'll try simply ensuring that users are warned that debugging is on first. Of
course, with the way syslog is usually misconfigured on most distros we'll have
to be careful to ensure that they ever see the messages.... Should I ask
whether, with ReiserFS debugging on, and the default syslog.conf, the assertions
being checked for on these particular distros ever reach the users? Better I
not ask....?
If Chris wants to run ReiserFS with the checks on, fine, he is a user, and he at
least knows he is doing it, but when a distro does it without warning users the
FS is crippled it is really foul.
Well, if any of you users out there are interested in knowing practical details
of how to overcome the shovelware, even more important than recompiling your
kernel, these patches will help. Note the cute patch that causes reiserfs to
get unmounted rather than unpowered by these folks so concerned about corruption
of data.:-O I am merely passing these patches onwards, I have not verified that
they are correct (because I lack a redhat machine to test on). If RedHat could
confirm that the patch is correct it would be nice, and mindboggling as well.
Vitaly, make sure these are on our website.
>From Dustin Byford:
--- rc.sysinit.orig Mon Jul 30 22:58:45 2001
+++ rc.sysinit Mon Jul 30 22:57:16 2001
@@ -211,7 +211,8 @@
_RUN_QUOTACHECK=0
ROOTFSTYPE=`grep " / " /proc/mounts | awk '{ print $3 }'`
-if [ -z "$fastboot" -a "$ROOTFSTYPE" != "nfs" ]; then
+if [ -z "$fastboot" -a "$ROOTFSTYPE" != "nfs" \
+ -a "$ROOTFSTYPE" != "reiserfs" ]; then
STRING=$"Checking root filesystem"
echo $STRING
>From David Rees:
--- halt.orig Mon Jul 30 17:26:24 2001
+++ halt Mon Jul 30 17:26:36 2001
@@ -165,7 +165,7 @@
# Remount read only anything that's left mounted.
#echo $"Remounting remaining filesystems (if any) readonly"
-mount | awk '/ext2/ { print $3 }' | while read line; do
+mount | awk '/ext2|reiserfs/ { print $3 }' | while read line; do
mount -n -o ro,remount $line
done
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption (patch to cause redhat to unmount reiserfs on halt included)
2001-07-31 12:22 ` ReiserFS / 2.4.6 / Data Corruption (patch to cause redhat to unmount reiserfs on halt included) Hans Reiser
@ 2001-07-31 12:37 ` Christoph Hellwig
2001-07-31 13:12 ` Hans Reiser
0 siblings, 1 reply; 94+ messages in thread
From: Christoph Hellwig @ 2001-07-31 12:37 UTC (permalink / raw)
To: Hans Reiser
Cc: David Weinehall, Rik van Riel, linux-kernel, Vitaly Fertman,
Linus Torvalds
On Tue, Jul 31, 2001 at 04:22:29PM +0400, Hans Reiser wrote:
> I'll try simply ensuring that users are warned that debugging is on first.
Shouldn't the user be warned when mounting a reiserfs filesystem without
checking instead?
> Of
> course, with the way syslog is usually misconfigured on most distros we'll have
> to be careful to ensure that they ever see the messages.... Should I ask
> whether, with ReiserFS debugging on, and the default syslog.conf, the assertions
> being checked for on these particular distros ever reach the users? Better I
> not ask....?
I think you got quite a few facts wrong:
o when a kernel with non-modular reiserfs is booted, reitherfs is
loaded before syslogd even starts
o wether iy hits some logfile, the console or not usually depends on
the KERN_ prefix you give to reiserfs
o on Caldera the user won't see any kernel messages unless something
unexpected happens or he explicitly wants it.
In either case one could rip that message out if there is any gain from it..
> If Chris wants to run ReiserFS with the checks on, fine, he is a user,
I am _not_ a reiserfs user.
> and he at
> least knows he is doing it, but when a distro does it without warning users the
> FS is crippled it is really foul.
I think you got that wrong. It's really foul to not have checks in that
can prevent silent corruption on a filesystem that is not know for being
very stable.
> Well, if any of you users out there are interested in knowing practical details
> of how to overcome the shovelware,
There is no reason why you can't put a reiserfs_nocheck.o module on
your website. If you want to I can send you a Caldera OpenLinux 3.1
package as reference.
Christoph
--
Of course it doesn't work. We've performed a software upgrade.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:59 ` Rik van Riel
2001-07-30 22:34 ` Hans Reiser
@ 2001-07-30 22:41 ` Kip Macy
2001-07-30 22:50 ` Christoph Hellwig
1 sibling, 1 reply; 94+ messages in thread
From: Kip Macy @ 2001-07-30 22:41 UTC (permalink / raw)
To: Rik van Riel; +Cc: Hans Reiser, Christoph Hellwig, linux-kernel, Vitaly Fertman
How does compiling in debug infrastructure protect the user's data? By
making the file system so slow that he won't use it? :-)
-Kip
> Are you actually saying you don't care about user's data,
> or is it just my imagination ?
>
> (I hope it's my imagination ...)
>
> cheers,
>
> Rik
> --
> Executive summary of a recent Microsoft press release:
> "we are concerned about the GNU General Public License (GPL)"
>
>
> http://www.surriel.com/
> http://www.conectiva.com/ http://distro.conectiva.com/
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 22:41 ` ReiserFS / 2.4.6 / Data Corruption Kip Macy
@ 2001-07-30 22:50 ` Christoph Hellwig
0 siblings, 0 replies; 94+ messages in thread
From: Christoph Hellwig @ 2001-07-30 22:50 UTC (permalink / raw)
To: Kip Macy
Cc: Rik van Riel, Hans Reiser, Christoph Hellwig, linux-kernel,
Vitaly Fertman
On Mon, Jul 30, 2001 at 03:41:16PM -0700, Kip Macy wrote:
> How does compiling in debug infrastructure protect the user's data? By
> making the file system so slow that he won't use it? :-)
The <<reiserfs debugging code>> isn't debugging code in a strict sense.
It mostly it consists of sequences in the form of:
(sometimes there is also code that the documentation states as deadlock-
avoidance, why it is not enabled without _CHECK defined is left as
exercise to the reader)
#ifdef CONFIG_REISERFS_CHECK
if (condition_that_should_not_happen)
reiserfs_panic (sb, "some_obscure_error_code");
#endif
This way the system stops with a indication of the failing component
instead of silently corrupting disk contents. As reiserfs maintains
a log the recovery from that panic shouldn't take that long either.
(On the other hand I've seen some reiserfs systems that destroyed their
disk contents while trying to recover. That's a reason why I still
can't recomend using reiserfs for anything but /tmp, test machines
or proxy caches).
Christoph
--
Of course it doesn't work. We've performed a software upgrade.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 20:49 ` Christoph Hellwig
2001-07-30 21:05 ` Hans Reiser
@ 2001-07-30 21:13 ` Hans Reiser
2001-07-30 21:21 ` Hans Reiser
2 siblings, 0 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 21:13 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-kernel
The debugging tests in reiserfs were deliberately encouraged to be excessive and
performance unconcerned. That is part of how we get programmers to write
excessively paranoid bug finding code, we tell them not to worry about the
effect on performance, it will only be used when looking for bugs.
People like you destroy my ability to get lots of tests put into the code by the
coders.
Hans
Christoph Hellwig wrote:
>
> On Tue, Jul 31, 2001 at 12:30:12AM +0400, Hans Reiser wrote:
> > But there is not one where they recover from invalid arguments without a panic
> > (unless I failed to notice something),
>
> Right.
>
> > so it gets you nothing except a message
> > that we the developers will find more informative when trying to find what made
> > it crash.
>
> Nope. It does a reiserfs_panic instead of letting the wrong arguments
> slipping into lower layers and possibly on disk and thus corrupting data.
>
> And in my opinion correct data is much more worth than one crash more or
> less (especially with a journaling filesystem).
>
> Christoph
>
> --
> Whip me. Beat me. Make me maintain AIX.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 20:49 ` Christoph Hellwig
2001-07-30 21:05 ` Hans Reiser
2001-07-30 21:13 ` Hans Reiser
@ 2001-07-30 21:21 ` Hans Reiser
2001-07-30 21:49 ` Christoph Hellwig
2001-07-30 22:04 ` Rik van Riel
2 siblings, 2 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 21:21 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-kernel
Christoph Hellwig wrote:
>
> On Tue, Jul 31, 2001 at 12:30:12AM +0400, Hans Reiser wrote:
> > But there is not one where they recover from invalid arguments without a panic
> > (unless I failed to notice something),
>
> Right.
>
> > so it gets you nothing except a message
> > that we the developers will find more informative when trying to find what made
> > it crash.
>
> Nope. It does a reiserfs_panic instead of letting the wrong arguments
> slipping into lower layers and possibly on disk and thus corrupting data.
>
> And in my opinion correct data is much more worth than one crash more or
> less (especially with a journaling filesystem).
The cost is not a crash, the cost is performance sucks.
>
> Christoph
>
> --
> Whip me. Beat me. Make me maintain AIX.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
Are you going to leave it on for future versions of ReiserFS, or just for Linux
2.4.2?
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:21 ` Hans Reiser
@ 2001-07-30 21:49 ` Christoph Hellwig
2001-07-31 2:34 ` Andrew Morton
2001-07-30 22:04 ` Rik van Riel
1 sibling, 1 reply; 94+ messages in thread
From: Christoph Hellwig @ 2001-07-30 21:49 UTC (permalink / raw)
To: Hans Reiser; +Cc: linux-kernel
On Tue, Jul 31, 2001 at 01:21:09AM +0400, Hans Reiser wrote:
>
> The cost is not a crash, the cost is performance sucks.
>
I give a damn for the performance if my filesystem doesn't prove stable.
And I think you can't deny that all reiserfs versions for 2.4 had issues
in that area. _IF_ reiserfs proves stable in the next time I don't see
any reason why this checks should stay in.
For example I've just turned of the debugging on my ext3-using boxens.
Not only ext3 has proven stable, but I also know that if it fails there
is still e2fsck which has proven absolutly reliable in the last years.
Another example is the write support I currently add to my freevxfs driver
(and no, I'm neither working for RedHat nor is it the VxFS module that
played a central role in your 3/2000 conspiration theories): until it has
proven stable for a long time I will not even add a option to turn off
all the consistency checks I've added. I'll give a damn if ext2, reiserfs
or VERITAS will beat me until it is stable.
>
> Are you going to leave it on for future versions of ReiserFS, or just for Linux
> 2.4.2?
I'm not in a position to decide it. But if I'm asked for my opion (again)
the answer will depend on wether reiserfs will be more stable than now
at that point.
Christoph
--
Whip me. Beat me. Make me maintain AIX.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:49 ` Christoph Hellwig
@ 2001-07-31 2:34 ` Andrew Morton
0 siblings, 0 replies; 94+ messages in thread
From: Andrew Morton @ 2001-07-31 2:34 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Hans Reiser, linux-kernel
Christoph Hellwig wrote:
>
> For example I've just turned of the debugging on my ext3-using boxens.
FYI... CONFIG_JBD_DEBUG is really just that - debug stuff. Mainly,
it enables the printks which are controlled by /proc/sys/fs/jbd-debug.
Early on, sct made the decision that the assertion checks in ext3:
akpm-1:/usr/src/ext3> grep -r ASSERT . | wc -l
187
cannot be disabled. Each and every one of these will nicely
crash the machine. The idea being, as you stated earlier,
that data integrity is golden - if we detect an inconsistency
we take the machine out and let recovery fix it up.
Turns out that at present we're over-aggressive on this. A modest
filesytem inconsistency (bit already free in bitmap, whatever)
or an IO error could force a panic. Stephen is working on changing
the fs to be more selective in its handling of errors - less severe
errors will turn the fs readonly.
I would support your decision to enable reiserfs checking. It's
a valuable feature. It can save your data from hardware failures
as well as software failures. Perhaps Hans' team should look into
moving the expensive checks into a different ifdef.
-
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 21:21 ` Hans Reiser
2001-07-30 21:49 ` Christoph Hellwig
@ 2001-07-30 22:04 ` Rik van Riel
2001-07-30 22:36 ` Hans Reiser
2001-07-31 22:08 ` Jussi Laako
1 sibling, 2 replies; 94+ messages in thread
From: Rik van Riel @ 2001-07-30 22:04 UTC (permalink / raw)
To: Hans Reiser; +Cc: Christoph Hellwig, linux-kernel
On Tue, 31 Jul 2001, Hans Reiser wrote:
> Christoph Hellwig wrote:
> > Nope. It does a reiserfs_panic instead of letting the wrong arguments
> > slipping into lower layers and possibly on disk and thus corrupting data.
> >
> > And in my opinion correct data is much more worth than one crash more or
> > less (especially with a journaling filesystem).
>
> The cost is not a crash, the cost is performance sucks.
If you can chose between sucky performance or a chance
at silent data corruption ... which would you chose ?
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com/
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 22:04 ` Rik van Riel
@ 2001-07-30 22:36 ` Hans Reiser
2001-07-30 22:53 ` Rik van Riel
2001-07-31 10:32 ` Chris Wedgwood
2001-07-31 22:08 ` Jussi Laako
1 sibling, 2 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 22:36 UTC (permalink / raw)
To: Rik van Riel; +Cc: Christoph Hellwig, linux-kernel
Rik van Riel wrote:
>
> On Tue, 31 Jul 2001, Hans Reiser wrote:
> > Christoph Hellwig wrote:
>
> > > Nope. It does a reiserfs_panic instead of letting the wrong arguments
> > > slipping into lower layers and possibly on disk and thus corrupting data.
> > >
> > > And in my opinion correct data is much more worth than one crash more or
> > > less (especially with a journaling filesystem).
> >
> > The cost is not a crash, the cost is performance sucks.
>
> If you can chose between sucky performance or a chance
> at silent data corruption ... which would you chose ?
>
> Rik
> --
> Executive summary of a recent Microsoft press release:
> "we are concerned about the GNU General Public License (GPL)"
>
> http://www.surriel.com/
> http://www.conectiva.com/ http://distro.conectiva.com/
If you could halve linux memory manager performance and check as many things as
reiserfs checks, would you do it. I think not, or else you would have. You
made the right choice. Now, if you add a #define, you can check as many things
as ReiserFS checks, and still go just as fast....
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 22:36 ` Hans Reiser
@ 2001-07-30 22:53 ` Rik van Riel
2001-07-30 23:12 ` Hans Reiser
2001-07-31 10:32 ` Chris Wedgwood
1 sibling, 1 reply; 94+ messages in thread
From: Rik van Riel @ 2001-07-30 22:53 UTC (permalink / raw)
To: Hans Reiser; +Cc: Christoph Hellwig, linux-kernel
On Tue, 31 Jul 2001, Hans Reiser wrote:
> Rik van Riel wrote:
> > On Tue, 31 Jul 2001, Hans Reiser wrote:
> > > The cost is not a crash, the cost is performance sucks.
> >
> > If you can chose between sucky performance or a chance
> > at silent data corruption ... which would you chose ?
>
> If you could halve linux memory manager performance and check as
> many things as reiserfs checks, would you do it.
I haven't removed a single debugging check from the
2.4 VM. Performance is MUCH more reliant on things
like evicting the right page from RAM or reading in
the right page at the right time.
CPU usage is only secondary.
> .. You made the right choice.
Thanks ;) [yeah, yeah ... flame me about out-of-context]
> Now, if you add a #define, you can check as many things as
> ReiserFS checks, and still go just as fast....
I'm sure these checks make reiserfs a tad more CPU hungry,
but isn't the real win in reiserfs supposed to come from
superior disk layout, readahead across files, etc... ?
Or is that all just a myth ?
regards,
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com/
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 22:53 ` Rik van Riel
@ 2001-07-30 23:12 ` Hans Reiser
0 siblings, 0 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-30 23:12 UTC (permalink / raw)
To: Rik van Riel; +Cc: Christoph Hellwig, linux-kernel
Rik van Riel wrote:
> > If you could halve linux memory manager performance and check as
> > many things as reiserfs checks, would you do it.
>
> I haven't removed a single debugging check from the
> 2.4 VM. Performance is MUCH more reliant on things
> like evicting the right page from RAM or reading in
> the right page at the right time.
>
> CPU usage is only secondary.
>
> > .. You made the right choice.
>
> Thanks ;) [yeah, yeah ... flame me about out-of-context]
>
> > Now, if you add a #define, you can check as many things as
> > ReiserFS checks, and still go just as fast....
>
> I'm sure these checks make reiserfs a tad more CPU hungry,
> but isn't the real win in reiserfs supposed to come from
> superior disk layout, readahead across files, etc... ?
>
> Or is that all just a myth ?
>
> regards,
>
> Rik
> --
> Executive summary of a recent Microsoft press release:
> "we are concerned about the GNU General Public License (GPL)"
>
> http://www.surriel.com/
> http://www.conectiva.com/ http://distro.conectiva.com/
A tree is a complex structure. You can check it, and the temporary structures
involved in balancing it, quite a lot of ways while balancing it. I believe you
that the checks you need for your code have no significant performance impact.
Ours sometimes do. Consistency checks can be quite a bit more than a tad
consumptive of CPU. Like I said, there were a few checks we removed after the
bug was gone because we got tired waiting for our debugging iterations taking so
long because of them.
Using the #define means we don't have to think about the effect on performance
of a check, we just leave it in. Some checks belong outside the #define
(checking to see if garbage came back from disk is left outside the define
nowadays.) Distros should trust the developers in these tradeoff decisions.
Otherwise, things just get stupid.
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 22:36 ` Hans Reiser
2001-07-30 22:53 ` Rik van Riel
@ 2001-07-31 10:32 ` Chris Wedgwood
2001-07-31 10:59 ` Hans Reiser
1 sibling, 1 reply; 94+ messages in thread
From: Chris Wedgwood @ 2001-07-31 10:32 UTC (permalink / raw)
To: Hans Reiser; +Cc: Rik van Riel, Christoph Hellwig, linux-kernel
On Tue, Jul 31, 2001 at 02:36:39AM +0400, Hans Reiser wrote:
If you could halve linux memory manager performance and check as
many things as reiserfs checks, would you do it. I think not, or
else you would have. You made the right choice. Now, if you add
a #define, you can check as many things as ReiserFS checks, and
still go just as fast....
The memory manager is stress much more often that reiserfs, EVERYBODY
has it.
The MM system does have various sanity checks, things might be
slightly faster without them, but having the sanity checks is still
very important.
If the memory manager does something bad, chances are your system will
go boom --- upon reboot all is happy. If as fs goes bad, that
corruption might still be there when you reboot, even if to another
kernel! This is a major difference.
Anyhow, I use resierfs with debugging/checking on in lots of places.
The speed difference is negligible, so I think this whole thread is
pointless.
FWIW, if the mainline kernels remove the debugging option, I will hack
it back in --- I for one am happy with the performance and am pleased
there is additional sanity checking.
--cw
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 10:32 ` Chris Wedgwood
@ 2001-07-31 10:59 ` Hans Reiser
2001-07-31 11:42 ` Chris Wedgwood
2001-07-31 13:41 ` Chris Mason
0 siblings, 2 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-31 10:59 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: Rik van Riel, Christoph Hellwig, linux-kernel
Chris Wedgwood wrote:
>
> On Tue, Jul 31, 2001 at 02:36:39AM +0400, Hans Reiser wrote:
>
> If you could halve linux memory manager performance and check as
> many things as reiserfs checks, would you do it. I think not, or
> else you would have. You made the right choice. Now, if you add
> a #define, you can check as many things as ReiserFS checks, and
> still go just as fast....
>
> The memory manager is stress much more often that reiserfs, EVERYBODY
> has it.
>
> The MM system does have various sanity checks, things might be
> slightly faster without them, but having the sanity checks is still
> very important.
>
> If the memory manager does something bad, chances are your system will
> go boom --- upon reboot all is happy. If as fs goes bad, that
> corruption might still be there when you reboot, even if to another
> kernel! This is a major difference.
>
> Anyhow, I use resierfs with debugging/checking on in lots of places.
> The speed difference is negligible, so I think this whole thread is
> pointless.
>
> FWIW, if the mainline kernels remove the debugging option, I will hack
> it back in --- I for one am happy with the performance and am pleased
> there is additional sanity checking.
>
> --cw
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
Last I ran benchmarks the performance cost was 30-40%, but this was some time
ago. I think that the coders have been quietly culling some checks out of the
FS, and so it does not cost as much anymore. I would prefer that the "excesive"
checks had stayed in.
Sigh, I see I cannot persuade in this argument. It seems Linus is right, and
debugging checks don't belong in debugged code even if they would make it easier
for persons hacking on the code to debug their latest hacks.
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 10:59 ` Hans Reiser
@ 2001-07-31 11:42 ` Chris Wedgwood
2001-07-31 13:41 ` Chris Mason
1 sibling, 0 replies; 94+ messages in thread
From: Chris Wedgwood @ 2001-07-31 11:42 UTC (permalink / raw)
To: Hans Reiser; +Cc: Rik van Riel, Christoph Hellwig, linux-kernel
On Tue, Jul 31, 2001 at 02:59:46PM +0400, Hans Reiser wrote:
Sigh, I see I cannot persuade in this argument. It seems Linus is
right, and debugging checks don't belong in debugged code even if
they would make it easier for persons hacking on the code to debug
their latest hacks.
In six months time, or whenever people feel more confident about
resierfs stability (there are still many bigs to be found) then these
checks can be relaxed.
Right now, reiserfs is still relatively new --- and its much more
complex and ext2, so having additional sanity checks is a good idea.
--cw
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 10:59 ` Hans Reiser
2001-07-31 11:42 ` Chris Wedgwood
@ 2001-07-31 13:41 ` Chris Mason
2001-07-31 15:15 ` Chris Wedgwood
2001-07-31 15:22 ` Hans Reiser
1 sibling, 2 replies; 94+ messages in thread
From: Chris Mason @ 2001-07-31 13:41 UTC (permalink / raw)
To: Hans Reiser, Chris Wedgwood; +Cc: Rik van Riel, Christoph Hellwig, linux-kernel
On Tuesday, July 31, 2001 02:59:46 PM +0400 Hans Reiser <reiser@namesys.com>
wrote:
[ CONFIG_REISERFS_CHECK ]
> Last I ran benchmarks the performance cost was 30-40%, but this was some
> time ago. I think that the coders have been quietly culling some checks
> out of the FS, and so it does not cost as much anymore. I would prefer
> that the "excesive" checks had stayed in.
>
> Sigh, I see I cannot persuade in this argument. It seems Linus is right,
> and debugging checks don't belong in debugged code even if they would make
> it easier for persons hacking on the code to debug their latest hacks.
>
In the end, the distributions are responsible for their own quality control,
and they are free to turn on whatever debugging features they like. You can
yell, scream, call them names, and in general piss them off however you like
and they will still be absolutely correct in turning on whatever debugging
check they feel is important.
The right way to deal with this is ask why they think it's important to turn
on the checks. The goal behind code under CONFIG_REISERFS_CHECK is to add
extra runtime consistency checks, but without CONFIG_REISERFS_CHECK on, the
code should still make sure it isn't hosing the disk. In other words, the
goal is like this:
if (some_error) {
#ifdef CONFIG_REISERFS_CHECK
panic("some_error") ;
#else
gracefully_recover
#endif
There are places CONFIG_REISERFS_CHECK does extra scanning of the metadata
and such, but all of these are supposed to be things that can be recovered
from with the debugging off. Anything else is a bug.
-chris
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 13:41 ` Chris Mason
@ 2001-07-31 15:15 ` Chris Wedgwood
2001-07-31 15:58 ` Chris Mason
2001-07-31 15:22 ` Hans Reiser
1 sibling, 1 reply; 94+ messages in thread
From: Chris Wedgwood @ 2001-07-31 15:15 UTC (permalink / raw)
To: Chris Mason; +Cc: Hans Reiser, Rik van Riel, Christoph Hellwig, linux-kernel
On Tue, Jul 31, 2001 at 09:41:25AM -0400, Chris Mason wrote:
if (some_error) {
#ifdef CONFIG_REISERFS_CHECK
panic("some_error") ;
#else
gracefully_recover
#endif
What a terrible construct... if would be much more elegant as:
if(some_error) {
_namesys_internal_foo("some_error");
recover_bar();
}
where _namesys_internal_foo is compiled differently and may not return
depending on CONFIG_REISERFS_CHECK and maybe also the error type.
That way we don't end up with even more #ifdef BLAH / #endif cruft
which obfuscates what is already hard to read code in places!
Flames welcome :)
--cw
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 15:15 ` Chris Wedgwood
@ 2001-07-31 15:58 ` Chris Mason
0 siblings, 0 replies; 94+ messages in thread
From: Chris Mason @ 2001-07-31 15:58 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: Hans Reiser, Rik van Riel, Christoph Hellwig, linux-kernel
On Wednesday, August 01, 2001 03:15:54 AM +1200 Chris Wedgwood <cw@f00f.org>
wrote:
> On Tue, Jul 31, 2001 at 09:41:25AM -0400, Chris Mason wrote:
>
> if (some_error) {
> #ifdef CONFIG_REISERFS_CHECK
> panic("some_error") ;
> #else
> gracefully_recover
> #endif
>
> What a terrible construct... if would be much more elegant as:
>
> if(some_error) {
> _namesys_internal_foo("some_error");
> recover_bar();
> }
>
> where _namesys_internal_foo is compiled differently and may not return
> depending on CONFIG_REISERFS_CHECK and maybe also the error type.
Two part answer...
1) almost none of the CONFIG_REISERFS_CHECKs look like that, it was an
oversimplified example ;-)
2) Even still, the #ifdefs look nasty, and make the code hard to read. Take
a look at the latest ac release, which has a patch from Nikita that is
similar to what you describe.
-chris
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 13:41 ` Chris Mason
2001-07-31 15:15 ` Chris Wedgwood
@ 2001-07-31 15:22 ` Hans Reiser
2001-07-31 15:49 ` Chris Mason
1 sibling, 1 reply; 94+ messages in thread
From: Hans Reiser @ 2001-07-31 15:22 UTC (permalink / raw)
To: Chris Mason; +Cc: Chris Wedgwood, Rik van Riel, Christoph Hellwig, linux-kernel
Chris Mason wrote:
>
> On Tuesday, July 31, 2001 02:59:46 PM +0400 Hans Reiser <reiser@namesys.com>
> wrote:
>
> [ CONFIG_REISERFS_CHECK ]
>
> > Last I ran benchmarks the performance cost was 30-40%, but this was some
> > time ago. I think that the coders have been quietly culling some checks
> > out of the FS, and so it does not cost as much anymore. I would prefer
> > that the "excesive" checks had stayed in.
> >
> > Sigh, I see I cannot persuade in this argument. It seems Linus is right,
> > and debugging checks don't belong in debugged code even if they would make
> > it easier for persons hacking on the code to debug their latest hacks.
> >
>
> In the end, the distributions are responsible for their own quality control,
> and they are free to turn on whatever debugging features they like. You can
> yell, scream, call them names, and in general piss them off however you like
> and they will still be absolutely correct in turning on whatever debugging
> check they feel is important.
If they tell the user that the debugging is on and the FS is slowed. I think
this is my solution, we will just make sure that the user knows with every mount
and every boot that debug is on and things are going to be slow.
>
> The right way to deal with this is ask why they think it's important to turn
> on the checks. The goal behind code under CONFIG_REISERFS_CHECK is to add
> extra runtime consistency checks, but without CONFIG_REISERFS_CHECK on, the
> code should still make sure it isn't hosing the disk. In other words, the
> goal is like this:
>
> if (some_error) {
> #ifdef CONFIG_REISERFS_CHECK
> panic("some_error") ;
> #else
> gracefully_recover
> #endif
>
> There are places CONFIG_REISERFS_CHECK does extra scanning of the metadata
> and such, but all of these are supposed to be things that can be recovered
> from with the debugging off. Anything else is a bug.
>
> -chris
I am sorry Chris, but I cannot see the sense in what you say.
CONFIG_REISERFS_CHECK is not a flag that indicates whether the user desires
graceful recovery, it is a flag that indicates whether every imaginable check
should be in the code, performance be damned, because there is a bug in the code
somewhere, and we are desperately trying to get a clue about what its source is
earlier in its life prior to the machine hanging. (Bugs where there is a time
lag between data structure corrupting and FS crashing are harder than others to
debug, and checking the data structures excessively is one way to try to fing
those bugs.) Making graceful recovery a selectable option is a separate topic.
There are lots of arguments that naturally arise in a development team about
what checks are debug only and what ones belong outside. I lose many of these
arguments to the betterment of ReiserFS. If persons not on the development
team, and not involved in those discussions, and not end users, turn debug on
and let users think it is normal slow motion reiserfs that they run, it screws
our whole methodology.
It may help readers if they understand that Chris does not like the big heavy
checks that one would not want to run all the time being inside
CONFIG_REISERFS_CHECK.
Having levels of CONFIG_REISERFS_CHECK, in which one level is something to the
effect of CHECK_EVERYTHING_YOU_CAN_WITHOUT_MY_NOTICING_THINGS_GOING_SLOWER,
would be reasonable. We have what we have though.
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 15:22 ` Hans Reiser
@ 2001-07-31 15:49 ` Chris Mason
0 siblings, 0 replies; 94+ messages in thread
From: Chris Mason @ 2001-07-31 15:49 UTC (permalink / raw)
To: Hans Reiser; +Cc: Chris Wedgwood, Rik van Riel, Christoph Hellwig, linux-kernel
On Tuesday, July 31, 2001 07:22:01 PM +0400 Hans Reiser <reiser@namesys.com>
wrote:
> Chris Mason wrote:
>>
>> On Tuesday, July 31, 2001 02:59:46 PM +0400 Hans Reiser
>> <reiser@namesys.com> wrote:
>>
>> [ CONFIG_REISERFS_CHECK ]
>>
>> > Last I ran benchmarks the performance cost was 30-40%, but this was some
>> > time ago. I think that the coders have been quietly culling some checks
>> > out of the FS, and so it does not cost as much anymore. I would prefer
>> > that the "excesive" checks had stayed in.
>> >
>> > Sigh, I see I cannot persuade in this argument. It seems Linus is right,
>> > and debugging checks don't belong in debugged code even if they would
>> > make it easier for persons hacking on the code to debug their latest
>> > hacks.
>> >
>>
>> In the end, the distributions are responsible for their own quality
>> control, and they are free to turn on whatever debugging features they
>> like. You can yell, scream, call them names, and in general piss them off
>> however you like and they will still be absolutely correct in turning on
>> whatever debugging check they feel is important.
>
> If they tell the user that the debugging is on and the FS is slowed. I
> think this is my solution, we will just make sure that the user knows with
> every mount and every boot that debug is on and things are going to be slow.
It already does. Read the mount output ;-)
>>
>> The right way to deal with this is ask why they think it's important to
>> turn on the checks. The goal behind code under CONFIG_REISERFS_CHECK is
>> to add extra runtime consistency checks, but without CONFIG_REISERFS_CHECK
>> on, the code should still make sure it isn't hosing the disk. In other
>> words, the goal is like this:
>>
>> if (some_error) {
>> #ifdef CONFIG_REISERFS_CHECK
>> panic("some_error") ;
>> #else
>> gracefully_recover
>> #endif
>>
>> There are places CONFIG_REISERFS_CHECK does extra scanning of the metadata
>> and such, but all of these are supposed to be things that can be recovered
>> from with the debugging off. Anything else is a bug.
>>
>> -chris
>
>
> I am sorry Chris, but I cannot see the sense in what you say.
> CONFIG_REISERFS_CHECK is not a flag that indicates whether the user desires
> graceful recovery, it is a flag that indicates whether every imaginable
> check should be in the code, performance be damned, because there is a bug
> in the code somewhere, and we are desperately trying to get a clue about
> what its source is earlier in its life prior to the machine hanging.
If graceful recovery is not possible with CONFIG_REISERFS_CHECK off, the FS
is supposed to panic (or remount readonly). Anything less is a bug.
CONFIG_REISERFS_CHECK might put the panic in a different place, for example,
it might notice when a block is read in that one of the items is hosed. That
doesn't mean we can completely ignore hosed items with CONFIG_REISERFS_CHECK
off though.
> (Bugs where there is a time lag between data structure corrupting and FS
> crashing are harder than others to debug, and checking the data structures
> excessively is one way to try to fing those bugs.) Making graceful
> recovery a selectable option is a separate topic.
>
> There are lots of arguments that naturally arise in a development team about
> what checks are debug only and what ones belong outside. I lose many of
> these arguments to the betterment of ReiserFS. If persons not on the
> development team, and not involved in those discussions, and not end users,
> turn debug on and let users think it is normal slow motion reiserfs that
> they run, it screws our whole methodology.
Distributions have methodogies too, its their job to apply it to the products
they ship. They aren't adding code or breaking existing code, they are
simply enabling one of the options we provide. If you really don't want
anyone to enable it, it shouldn't be there at all.
>
> It may help readers if they understand that Chris does not like the big
> heavy checks that one would not want to run all the time being inside
> CONFIG_REISERFS_CHECK.
As per the rules above, this is somewhat true. I want the FS to be fast as
much as anyone else, but reasonable safety checks are much more important.
>
> Having levels of CONFIG_REISERFS_CHECK, in which one level is something to
> the effect of
> CHECK_EVERYTHING_YOU_CAN_WITHOUT_MY_NOTICING_THINGS_GOING_SLOWER, would be
> reasonable. We have what we have though.
>
If you can do the check without being slower, why would anyone ever turn it
off? Speed is not the determining factor in which checks are done, safety is.
-chris
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 22:04 ` Rik van Riel
2001-07-30 22:36 ` Hans Reiser
@ 2001-07-31 22:08 ` Jussi Laako
2001-07-31 22:32 ` Dan Hollis
2001-08-01 16:23 ` ReiserFS / 2.4.6 / Data Corruption Andreas Dilger
1 sibling, 2 replies; 94+ messages in thread
From: Jussi Laako @ 2001-07-31 22:08 UTC (permalink / raw)
To: linux-kernel
Rik van Riel wrote:
>
> If you can chose between sucky performance or a chance
> at silent data corruption ... which would you chose ?
Just a side note to this discussion.
I'd be very happy with full data journalling even with 50% performance
penalty... There are applications that require extreme data integrity all
times no matter what happens.
- Jussi Laako
--
PGP key fingerprint: 161D 6FED 6A92 39E2 EB5B 39DD A4DE 63EB C216 1E4B
Available at PGP keyservers
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 22:08 ` Jussi Laako
@ 2001-07-31 22:32 ` Dan Hollis
2001-07-31 23:45 ` Chris Wedgwood
2001-08-05 22:19 ` CRC loop method (was Re: ReiserFS / 2.4.6 / Data Corruption) Pavel Machek
2001-08-01 16:23 ` ReiserFS / 2.4.6 / Data Corruption Andreas Dilger
1 sibling, 2 replies; 94+ messages in thread
From: Dan Hollis @ 2001-07-31 22:32 UTC (permalink / raw)
To: Jussi Laako; +Cc: linux-kernel
On Wed, 1 Aug 2001, Jussi Laako wrote:
> I'd be very happy with full data journalling even with 50% performance
> penalty... There are applications that require extreme data integrity all
> times no matter what happens.
How about an idea I proposed a while back, 'integrity loopback'?
A loopback device which writes a CRC with each block and checks the CRC
when read back.
So if you have a flaky DMA controller, bad cables, etc you will know
instantly. It would at least help catch the 'silent corruption' cases.
-Dan
--
[-] Omae no subete no kichi wa ore no mono da. [-]
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 22:32 ` Dan Hollis
@ 2001-07-31 23:45 ` Chris Wedgwood
2001-08-05 22:19 ` CRC loop method (was Re: ReiserFS / 2.4.6 / Data Corruption) Pavel Machek
1 sibling, 0 replies; 94+ messages in thread
From: Chris Wedgwood @ 2001-07-31 23:45 UTC (permalink / raw)
To: Dan Hollis; +Cc: Jussi Laako, linux-kernel
On Tue, Jul 31, 2001 at 03:32:39PM -0700, Dan Hollis wrote:
How about an idea I proposed a while back, 'integrity loopback'?
A loopback device which writes a CRC with each block and checks
the CRC when read back.
So if you have a flaky DMA controller, bad cables, etc you will
know instantly. It would at least help catch the 'silent
corruption' cases.
It still doesn't help with block-reordering, the fs needs some way to
communication write-barriers or relative block write ordering to the
lower-levels.
To implement the device, I would hack loopback to take no only the
loopback file, but also another 'checksum' file of 160-bits or
whatever for each 4096 (or whatever) block. This file might initially
be of zero-length, in which case the bind is responsible for
checksumming the blocks and writing the checksums out on attach.
I say 160-bits (or whatever) so you can use something like SHA1 for
the checksums, this way you can use a small application to resync the
entire fs at the block level over a network without having to read
every block (ie. you compared checksums and then xmit the blocks).
The latter is something I needed a while ago.
--cw
^ permalink raw reply [flat|nested] 94+ messages in thread* CRC loop method (was Re: ReiserFS / 2.4.6 / Data Corruption)
2001-07-31 22:32 ` Dan Hollis
2001-07-31 23:45 ` Chris Wedgwood
@ 2001-08-05 22:19 ` Pavel Machek
1 sibling, 0 replies; 94+ messages in thread
From: Pavel Machek @ 2001-08-05 22:19 UTC (permalink / raw)
To: Dan Hollis; +Cc: Jussi Laako, linux-kernel
Hi!
> > I'd be very happy with full data journalling even with 50% performance
> > penalty... There are applications that require extreme data integrity all
> > times no matter what happens.
>
> How about an idea I proposed a while back, 'integrity loopback'?
>
> A loopback device which writes a CRC with each block and checks the CRC
> when read back.
I have written that. Do you want a copy?
--
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-31 22:08 ` Jussi Laako
2001-07-31 22:32 ` Dan Hollis
@ 2001-08-01 16:23 ` Andreas Dilger
1 sibling, 0 replies; 94+ messages in thread
From: Andreas Dilger @ 2001-08-01 16:23 UTC (permalink / raw)
To: Jussi Laako; +Cc: linux-kernel
Jussi Laako writes:
> Just a side note to this discussion.
>
> I'd be very happy with full data journalling even with 50% performance
> penalty... There are applications that require extreme data integrity all
> times no matter what happens.
Use ext3 then. It allows full data journaling. It will even support data
journaling on some files and metadata-only journaling on the rest of the
filesystem (this is currently broken on SMP machines, because not many
people have had a need to use it yet, but it will get fixed).
Cheers, Andreas
--
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-30 20:30 ` Hans Reiser
2001-07-30 20:49 ` Christoph Hellwig
@ 2001-08-02 13:44 ` Pavel Machek
1 sibling, 0 replies; 94+ messages in thread
From: Pavel Machek @ 2001-08-02 13:44 UTC (permalink / raw)
To: Hans Reiser
Cc: Christoph Hellwig, Matthew Gardiner, kernel, Joshua Schmidlkofer
Hi!
> > > > Reiserfs as implemented in the 2.4.2-based kernel of OpenLinux 3.1 is
> > > > everything but stable and has a lot of issues (e.g. NFS-exporting doesn't
> > > > work). That is the reason why it is a) marked experimental and is completly
> > > > unsupported (and that is written _big_ _fat_ in manuals and similar stuff)
> > > > and b) has debugging enabled to have the additional sanity checks that are
> > > > under this option and give addtional hints if reiserfs fails again.
> > >
> > > The debugging won't prevent a single crash, it will only print a diagnostic that
> > > might help to understand why it crashed.
> >
> > I don't know when you took a look at you code the last time, but when
> > I did some time before the COL 3.1 release, there were lots of places
> > in the reiserfs code where functions assumed that they have valid
> > arguments when compiled without debugging and did the check explicitly
> > when compiled with. Given the state the reiserfs code is in I really
> > prefer to see this option turned on.
>
> But there is not one where they recover from invalid arguments without a panic
> (unless I failed to notice something), so it gets you nothing except a message
> that we the developers will find more informative when trying to find what made
> it crash. We check invalid arguments at entry to reiserfs, and for those we
It is better to panic() immediately than "maybe crash" later.
Unless reiserfs is totaly screwed up, CONFIG_CHECKING can not make it *less*
stable -- that's why I understand people turning it on for distributions. Its
*their* choice, not yours. Pavel
--
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 14:18 ` Joshua Schmidlkofer
2001-07-27 14:55 ` Hans Reiser
@ 2001-07-27 15:06 ` Daniel Phillips
2001-07-27 15:33 ` Hans Reiser
2001-07-27 15:07 ` ReiserFS / 2.4.6 / Data Corruption Chris Wedgwood
2001-07-27 16:39 ` Andrew Morton
3 siblings, 1 reply; 94+ messages in thread
From: Daniel Phillips @ 2001-07-27 15:06 UTC (permalink / raw)
To: Joshua Schmidlkofer, Hans Reiser; +Cc: kernel
On Friday 27 July 2001 16:18, Joshua Schmidlkofer wrote:
> I've almost quit using reiser, because everytime I have a power
> outage, the last 2 or three files that I've editted, even ones that I
> haven't touched in a while, will usually be hopelessly corrupted.
My early flush patch will fix this, or at least it will if I get
together with the ReiserFS guys and figure out how to integrate their
flushing mechanism with the standard bdflush. Or they could
incorporate the ideas from my early flush in their own flush daemon,
though generalizing the standard flush would have more value in the
long run.
> The '<file>~' that Emacs makes is usually fine though.
Because it's "created" by a rename.
[...]
> Once, I lost power in on my SQL box, [it was blessedly during a
> period of no use.] I had to rebuild all the indexes. Not only
> THAT, but what happens to that box if I lose power whilst in the
> middle of operations? I am working on the migration plan to move that
> to XFS because of these concerns. [However, I am doing a better job
> of testing with XFS first.]
Help is on its way. You can try full-data journalling with the journal
on NVRAM or on a separate disk. You can also wait for me to get a
usable version of Tux2 working. It's progressed a little slowly
because of frequent side trips ;-) But hopefully I'll be able to do
something about that soon.
Which flavor of SQL are you using? Are the indices in separate files?
(Sounds like they are.)
> I think that Reiser is cool, and has neat ideology, but I am
> un-nerved by this behaviour.
I think it's not hard to fix.
--
Daniel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 15:06 ` Daniel Phillips
@ 2001-07-27 15:33 ` Hans Reiser
2001-07-27 16:30 ` Daniel Phillips
0 siblings, 1 reply; 94+ messages in thread
From: Hans Reiser @ 2001-07-27 15:33 UTC (permalink / raw)
To: Daniel Phillips; +Cc: Joshua Schmidlkofer, kernel, Chris Mason
Daniel Phillips wrote:
>
> On Friday 27 July 2001 16:18, Joshua Schmidlkofer wrote:
> > I've almost quit using reiser, because everytime I have a power
> > outage, the last 2 or three files that I've editted, even ones that I
> > haven't touched in a while, will usually be hopelessly corrupted.
>
> My early flush patch will fix this, or at least it will if I get
> together with the ReiserFS guys and figure out how to integrate their
> flushing mechanism with the standard bdflush. Or they could
> incorporate the ideas from my early flush in their own flush daemon,
> though generalizing the standard flush would have more value in the
> long run.
Can you describe early flush?
>
> > The '<file>~' that Emacs makes is usually fine though.
>
> Because it's "created" by a rename.
>
> [...]
> > Once, I lost power in on my SQL box, [it was blessedly during a
> > period of no use.] I had to rebuild all the indexes. Not only
> > THAT, but what happens to that box if I lose power whilst in the
> > middle of operations? I am working on the migration plan to move that
> > to XFS because of these concerns. [However, I am doing a better job
> > of testing with XFS first.]
>
> Help is on its way. You can try full-data journalling with the journal
> on NVRAM or on a separate disk. You can also wait for me to get a
Well, if you have NVRAM, you might try using our new journal relocation patch to put the journal on
NVRAM.
> I think it's not hard to fix.
>
> --
> Daniel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 15:33 ` Hans Reiser
@ 2001-07-27 16:30 ` Daniel Phillips
2001-07-27 16:49 ` Early Flush Hans Reiser
0 siblings, 1 reply; 94+ messages in thread
From: Daniel Phillips @ 2001-07-27 16:30 UTC (permalink / raw)
To: Hans Reiser; +Cc: Joshua Schmidlkofer, kernel, Chris Mason
On Friday 27 July 2001 17:33, Hans Reiser wrote:
> Daniel Phillips wrote:
> > On Friday 27 July 2001 16:18, Joshua Schmidlkofer wrote:
> > > I've almost quit using reiser, because everytime I have a power
> > > outage, the last 2 or three files that I've editted, even ones
> > > that I haven't touched in a while, will usually be hopelessly
> > > corrupted.
> >
> > My early flush patch will fix this, or at least it will if I get
> > together with the ReiserFS guys and figure out how to integrate
> > their flushing mechanism with the standard bdflush. Or they could
> > incorporate the ideas from my early flush in their own flush
> > daemon, though generalizing the standard flush would have more
> > value in the long run.
>
> Can you describe early flush?
The idea is to do what amounts to a sync within a tenth of a second of
disk bandwidth usage falling below a certain threshhold.
The original posts/patches are here:
[RFC] Early flush (was: spindown)
[RFC] Early flush: new, improved (updated)
and there are long threads attached to each of them. The clearest
explanation is probably Jonathan Corbet's writeup on lwn:
http://lwn.net/2001/0628/kernel.php3
(Thanks, Jonathan, I often get the feeling I understand what I actually
did only *after* reading your writeups:-)
The second of the two patches needs more work - I think I goofed on
some needed "volatile" handling, see the current flam^H^H^H^H thread
about that.
--
Daniel
^ permalink raw reply [flat|nested] 94+ messages in thread* Early Flush
2001-07-27 16:30 ` Daniel Phillips
@ 2001-07-27 16:49 ` Hans Reiser
0 siblings, 0 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-27 16:49 UTC (permalink / raw)
To: Daniel Phillips; +Cc: Joshua Schmidlkofer, kernel, Chris Mason
Daniel Phillips wrote:
>
> On Friday 27 July 2001 17:33, Hans Reiser wrote:
> > Daniel Phillips wrote:
> > > On Friday 27 July 2001 16:18, Joshua Schmidlkofer wrote:
> > > > I've almost quit using reiser, because everytime I have a power
> > > > outage, the last 2 or three files that I've editted, even ones
> > > > that I haven't touched in a while, will usually be hopelessly
> > > > corrupted.
> > >
> > > My early flush patch will fix this, or at least it will if I get
> > > together with the ReiserFS guys and figure out how to integrate
> > > their flushing mechanism with the standard bdflush. Or they could
> > > incorporate the ideas from my early flush in their own flush
> > > daemon, though generalizing the standard flush would have more
> > > value in the long run.
> >
> > Can you describe early flush?
>
> The idea is to do what amounts to a sync within a tenth of a second of
> disk bandwidth usage falling below a certain threshhold.
>
> The original posts/patches are here:
>
> [RFC] Early flush (was: spindown)
> [RFC] Early flush: new, improved (updated)
>
> and there are long threads attached to each of them. The clearest
> explanation is probably Jonathan Corbet's writeup on lwn:
>
> http://lwn.net/2001/0628/kernel.php3
>
> (Thanks, Jonathan, I often get the feeling I understand what I actually
> did only *after* reading your writeups:-)
>
> The second of the two patches needs more work - I think I goofed on
> some needed "volatile" handling, see the current flam^H^H^H^H thread
> about that.
>
> --
> Daniel
Daniel, what you have done is something that I have wanted and believed in for a long time.
Spell out what you need from us and we will support you.
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 14:18 ` Joshua Schmidlkofer
2001-07-27 14:55 ` Hans Reiser
2001-07-27 15:06 ` Daniel Phillips
@ 2001-07-27 15:07 ` Chris Wedgwood
2001-07-27 16:39 ` Andrew Morton
3 siblings, 0 replies; 94+ messages in thread
From: Chris Wedgwood @ 2001-07-27 15:07 UTC (permalink / raw)
To: Joshua Schmidlkofer; +Cc: Hans Reiser, kernel
On Fri, Jul 27, 2001 at 08:18:12AM -0600, Joshua Schmidlkofer wrote:
Once, I lost power in on my SQL box, [it was blessedly during
a period of no use.] I had to rebuild all the indexes. Not only
THAT, but what happens to that box if I lose power whilst in the
middle of operations? I am working on the migration plan to move
that to XFS because of these concerns. [However, I am doing a
better job of testing with XFS first.]
Sounds like a MySQL bug (assuming data is on disk when perhaps it's
not). Using either Oracle or Sybase you seem to be able to yank the
power at pretty much any time even under load and things will recovery
gracefully upon restart.
--cw
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 14:18 ` Joshua Schmidlkofer
` (2 preceding siblings ...)
2001-07-27 15:07 ` ReiserFS / 2.4.6 / Data Corruption Chris Wedgwood
@ 2001-07-27 16:39 ` Andrew Morton
2001-07-27 16:57 ` Hans Reiser
2001-07-27 17:10 ` Steve Lord
3 siblings, 2 replies; 94+ messages in thread
From: Andrew Morton @ 2001-07-27 16:39 UTC (permalink / raw)
To: Joshua Schmidlkofer; +Cc: Hans Reiser, kernel
Joshua Schmidlkofer wrote:
>
> I've almost quit using reiser, because everytime I have a power outage, the
> last 2 or three files that I've editted, even ones that I haven't touched in
> a while, will usually be hopelessly corrupted. The '<file>~' that Emacs
> makes is usually fine though.
It's a matter of timing. There is a lengthy window where the metadata
is written, but its data is not. If you crash in this window, the files
contain old data.
You can narrow the window of exposure by fiddling with the
parameters in /proc/sys/vm/bdflush - force a full flush every
five seconds, say.
> It seems to be that any open file is
> in danger. I don't know if this is normal, or not, but I switched to XFS on
> several machines. I have nothing against reiser. I assumed that these
> problems were due to immaturity....
I'm under the impression that XFS also leaves data in the hands
of the kernel's normal writeback mechanisms and will thus be
exposed to the same problem. I may be wrong about this.
Here's a ten-minute hack which gives reiserfs a simple `ordered data'
mode. It simply pushes all the dirty buffers and pages out to disk
before running a commit. Performance is still OK.
I hit reset partway through a massive file tree copy and every
file which had been copied came up peachy - which is very different
from the behaviour without the patch. Interestingly, there were
zero truncated files as well. hmmm...
--- linux-2.4.7/include/linux/fs.h Sat Jul 21 12:37:14 2001
+++ lk-ext3/include/linux/fs.h Sat Jul 28 02:37:43 2001
@@ -1061,6 +1061,7 @@ extern int fs_may_remount_ro(struct supe
extern int try_to_free_buffers(struct page *, unsigned int);
extern void refile_buffer(struct buffer_head * buf);
extern void end_buffer_io_sync(struct buffer_head *bh, int uptodate);
+extern int flush_all_but_supers(kdev_t dev);
/* reiserfs_writepage needs this */
extern void set_buffer_async_io(struct buffer_head *bh) ;
--- linux-2.4.7/include/linux/reiserfs_fs_sb.h Sat Jul 21 12:37:14 2001
+++ lk-ext3/include/linux/reiserfs_fs_sb.h Sat Jul 28 02:37:43 2001
@@ -289,6 +289,8 @@ struct reiserfs_sb_info
/* To be obsoleted soon by per buffer seals.. -Hans */
atomic_t s_generation_counter; // increased by one every time the
// tree gets re-balanced
+
+ int no_sync;
/* session statistics */
int s_kmallocs;
--- linux-2.4.7/fs/reiserfs/journal.c Sat Jul 21 12:37:14 2001
+++ lk-ext3/fs/reiserfs/journal.c Sat Jul 28 02:37:43 2001
@@ -2719,6 +2719,9 @@ static int do_journal_end(struct reiserf
reiserfs_discard_all_prealloc(th); /* it should not involve new blocks into
* the transaction */
#endif
+
+ if (!p_s_sb->u.reiserfs_sb.no_sync)
+ flush_all_but_supers(p_s_sb->s_dev);
rs = SB_DISK_SUPER_BLOCK(p_s_sb) ;
/* setup description block */
--- linux-2.4.7/fs/reiserfs/super.c Wed Jul 4 18:21:31 2001
+++ lk-ext3/fs/reiserfs/super.c Sat Jul 28 02:37:43 2001
@@ -116,7 +116,9 @@ void reiserfs_put_super (struct super_bl
/* note, journal_release checks for readonly mount, and can decide not
** to do a journal_end
*/
+ s->u.reiserfs_sb.no_sync = 1;
journal_release(&th, s) ;
+ s->u.reiserfs_sb.no_sync = 0;
for (i = 0; i < SB_BMAP_NR (s); i ++)
brelse (SB_AP_BITMAP (s)[i]);
--- linux-2.4.7/fs/buffer.c Sat Jul 21 12:37:14 2001
+++ lk-ext3/fs/buffer.c Sat Jul 28 02:37:43 2001
@@ -333,6 +333,18 @@ int fsync_dev(kdev_t dev)
return sync_buffers(dev, 1);
}
+int flush_all_but_supers(kdev_t dev)
+{
+ sync_buffers(dev, 0);
+
+ lock_kernel();
+ sync_inodes(dev);
+ DQUOT_SYNC(dev);
+ unlock_kernel();
+
+ return sync_buffers(dev, 1);
+}
+
/*
* There's no real reason to pretend we should
* ever do anything differently
--- linux-2.4.7/kernel/ksyms.c Sat Jul 21 12:37:14 2001
+++ lk-ext3/kernel/ksyms.c Sat Jul 28 02:37:43 2001
@@ -161,6 +161,7 @@ EXPORT_SYMBOL(d_lookup);
EXPORT_SYMBOL(__d_path);
EXPORT_SYMBOL(mark_buffer_dirty);
EXPORT_SYMBOL(set_buffer_async_io); /* for reiserfs_writepage */
+EXPORT_SYMBOL(flush_all_but_supers);
EXPORT_SYMBOL(__mark_buffer_dirty);
EXPORT_SYMBOL(__mark_inode_dirty);
EXPORT_SYMBOL(get_empty_filp);
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 16:39 ` Andrew Morton
@ 2001-07-27 16:57 ` Hans Reiser
2001-07-27 17:28 ` Andrew Morton
2001-07-27 17:10 ` Steve Lord
1 sibling, 1 reply; 94+ messages in thread
From: Hans Reiser @ 2001-07-27 16:57 UTC (permalink / raw)
To: Andrew Morton
Cc: Joshua Schmidlkofer, kernel, Chris Mason, Gryaznova E.,
Vladimir V. Saveliev
Andrew, can you do this such that there is no disruption of our disk format, and make a mount option
out of it, and probably we should use this patch....
After you make a mount option out of it, grev will benchmark it for us using the usual suite of
benchmarks.
Comments Chris?
Thanks,
Hans
Andrew Morton wrote:
>
> Joshua Schmidlkofer wrote:
> >
> > I've almost quit using reiser, because everytime I have a power outage, the
> > last 2 or three files that I've editted, even ones that I haven't touched in
> > a while, will usually be hopelessly corrupted. The '<file>~' that Emacs
> > makes is usually fine though.
>
> It's a matter of timing. There is a lengthy window where the metadata
> is written, but its data is not. If you crash in this window, the files
> contain old data.
>
> You can narrow the window of exposure by fiddling with the
> parameters in /proc/sys/vm/bdflush - force a full flush every
> five seconds, say.
>
> > It seems to be that any open file is
> > in danger. I don't know if this is normal, or not, but I switched to XFS on
> > several machines. I have nothing against reiser. I assumed that these
> > problems were due to immaturity....
>
> I'm under the impression that XFS also leaves data in the hands
> of the kernel's normal writeback mechanisms and will thus be
> exposed to the same problem. I may be wrong about this.
>
> Here's a ten-minute hack which gives reiserfs a simple `ordered data'
> mode. It simply pushes all the dirty buffers and pages out to disk
> before running a commit. Performance is still OK.
>
> I hit reset partway through a massive file tree copy and every
> file which had been copied came up peachy - which is very different
> from the behaviour without the patch. Interestingly, there were
> zero truncated files as well. hmmm...
>
> --- linux-2.4.7/include/linux/fs.h Sat Jul 21 12:37:14 2001
> +++ lk-ext3/include/linux/fs.h Sat Jul 28 02:37:43 2001
> @@ -1061,6 +1061,7 @@ extern int fs_may_remount_ro(struct supe
> extern int try_to_free_buffers(struct page *, unsigned int);
> extern void refile_buffer(struct buffer_head * buf);
> extern void end_buffer_io_sync(struct buffer_head *bh, int uptodate);
> +extern int flush_all_but_supers(kdev_t dev);
>
> /* reiserfs_writepage needs this */
> extern void set_buffer_async_io(struct buffer_head *bh) ;
> --- linux-2.4.7/include/linux/reiserfs_fs_sb.h Sat Jul 21 12:37:14 2001
> +++ lk-ext3/include/linux/reiserfs_fs_sb.h Sat Jul 28 02:37:43 2001
> @@ -289,6 +289,8 @@ struct reiserfs_sb_info
> /* To be obsoleted soon by per buffer seals.. -Hans */
> atomic_t s_generation_counter; // increased by one every time the
> // tree gets re-balanced
> +
> + int no_sync;
>
> /* session statistics */
> int s_kmallocs;
> --- linux-2.4.7/fs/reiserfs/journal.c Sat Jul 21 12:37:14 2001
> +++ lk-ext3/fs/reiserfs/journal.c Sat Jul 28 02:37:43 2001
> @@ -2719,6 +2719,9 @@ static int do_journal_end(struct reiserf
> reiserfs_discard_all_prealloc(th); /* it should not involve new blocks into
> * the transaction */
> #endif
> +
> + if (!p_s_sb->u.reiserfs_sb.no_sync)
> + flush_all_but_supers(p_s_sb->s_dev);
>
> rs = SB_DISK_SUPER_BLOCK(p_s_sb) ;
> /* setup description block */
> --- linux-2.4.7/fs/reiserfs/super.c Wed Jul 4 18:21:31 2001
> +++ lk-ext3/fs/reiserfs/super.c Sat Jul 28 02:37:43 2001
> @@ -116,7 +116,9 @@ void reiserfs_put_super (struct super_bl
> /* note, journal_release checks for readonly mount, and can decide not
> ** to do a journal_end
> */
> + s->u.reiserfs_sb.no_sync = 1;
> journal_release(&th, s) ;
> + s->u.reiserfs_sb.no_sync = 0;
>
> for (i = 0; i < SB_BMAP_NR (s); i ++)
> brelse (SB_AP_BITMAP (s)[i]);
> --- linux-2.4.7/fs/buffer.c Sat Jul 21 12:37:14 2001
> +++ lk-ext3/fs/buffer.c Sat Jul 28 02:37:43 2001
> @@ -333,6 +333,18 @@ int fsync_dev(kdev_t dev)
> return sync_buffers(dev, 1);
> }
>
> +int flush_all_but_supers(kdev_t dev)
> +{
> + sync_buffers(dev, 0);
> +
> + lock_kernel();
> + sync_inodes(dev);
> + DQUOT_SYNC(dev);
> + unlock_kernel();
> +
> + return sync_buffers(dev, 1);
> +}
> +
> /*
> * There's no real reason to pretend we should
> * ever do anything differently
> --- linux-2.4.7/kernel/ksyms.c Sat Jul 21 12:37:14 2001
> +++ lk-ext3/kernel/ksyms.c Sat Jul 28 02:37:43 2001
> @@ -161,6 +161,7 @@ EXPORT_SYMBOL(d_lookup);
> EXPORT_SYMBOL(__d_path);
> EXPORT_SYMBOL(mark_buffer_dirty);
> EXPORT_SYMBOL(set_buffer_async_io); /* for reiserfs_writepage */
> +EXPORT_SYMBOL(flush_all_but_supers);
> EXPORT_SYMBOL(__mark_buffer_dirty);
> EXPORT_SYMBOL(__mark_inode_dirty);
> EXPORT_SYMBOL(get_empty_filp);
^ permalink raw reply [flat|nested] 94+ messages in thread* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 16:57 ` Hans Reiser
@ 2001-07-27 17:28 ` Andrew Morton
2001-07-27 17:45 ` Hans Reiser
0 siblings, 1 reply; 94+ messages in thread
From: Andrew Morton @ 2001-07-27 17:28 UTC (permalink / raw)
To: Hans Reiser
Cc: Joshua Schmidlkofer, kernel, Chris Mason, Gryaznova E.,
Vladimir V. Saveliev
Hans Reiser wrote:
>
> Andrew, can you do this such that there is no disruption of our
> disk format, and make a mount option
> out of it, and probably we should use this patch....
I'll defer to Chris :)
There's no disruption to disk format - it just simulates
the user typing `sync' at the right time. I think the
concept is sound, and I'm sure Chris can find a more efficient
way...
> After you make a mount option out of it, grev will benchmark
> it for us using the usual suite of benchmarks.
>
Ordered-data is a funny thing. Under heavy loads it tends
to make a significant throughput difference - on ext3 it
almost halves throughput wrt writeback mode.
But this by no means indicates that writes are half as slow;
what happens is that metadata-intensive workloads fill the
journal up quickly, so the `sync' happens more frequently.
Under normal workloads, or less metadata-intense workloads
the difference is very small.
During testing of that little patch I noted that the
disk went crunch every thirty seconds or so, which is good.
Presumably the reiserfs journal is larger, or more space-efficient.
-
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 17:28 ` Andrew Morton
@ 2001-07-27 17:45 ` Hans Reiser
0 siblings, 0 replies; 94+ messages in thread
From: Hans Reiser @ 2001-07-27 17:45 UTC (permalink / raw)
To: Andrew Morton
Cc: Joshua Schmidlkofer, kernel, Chris Mason, Gryaznova E.,
Vladimir V. Saveliev
Andrew Morton wrote:
>
> Hans Reiser wrote:
> >
> > Andrew, can you do this such that there is no disruption of our
> > disk format, and make a mount option
> > out of it, and probably we should use this patch....
>
> I'll defer to Chris :)
Yes, I'll let him think carefully through the details of how it affects ordering of the writes.
>
> There's no disruption to disk format - it just simulates
> the user typing `sync' at the right time. I think the
> concept is sound, and I'm sure Chris can find a more efficient
> way...
Oops, sorry, you changed the in-ram not the on-disk sb....
>
> > After you make a mount option out of it, grev will benchmark
> > it for us using the usual suite of benchmarks.
> >
>
> Ordered-data is a funny thing. Under heavy loads it tends
> to make a significant throughput difference - on ext3 it
> almost halves throughput wrt writeback mode.
>
> But this by no means indicates that writes are half as slow;
> what happens is that metadata-intensive workloads fill the
> journal up quickly, so the `sync' happens more frequently.
> Under normal workloads, or less metadata-intense workloads
> the difference is very small.
>
> During testing of that little patch I noted that the
> disk went crunch every thirty seconds or so, which is good.
> Presumably the reiserfs journal is larger, or more space-efficient.
>
> -
Thanks Andrew
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: ReiserFS / 2.4.6 / Data Corruption
2001-07-27 16:39 ` Andrew Morton
2001-07-27 16:57 ` Hans Reiser
@ 2001-07-27 17:10 ` Steve Lord
1 sibling, 0 replies; 94+ messages in thread
From: Steve Lord @ 2001-07-27 17:10 UTC (permalink / raw)
To: Andrew Morton; +Cc: Joshua Schmidlkofer, Hans Reiser, kernel
>
> You can narrow the window of exposure by fiddling with the
> parameters in /proc/sys/vm/bdflush - force a full flush every
> five seconds, say.
>
> > It seems to be that any open file is
> > in danger. I don't know if this is normal, or not, but I switched to XFS o
> n
> > several machines. I have nothing against reiser. I assumed that these
> > problems were due to immaturity....
>
> I'm under the impression that XFS also leaves data in the hands
> of the kernel's normal writeback mechanisms and will thus be
> exposed to the same problem. I may be wrong about this.
>
Yes, XFS does leave writing the data to the normal writeback mechanisms,
however, what happens with XFS is usually:
o a file with no extents - the size made it out to disk but the data did not.
since on writes to new space we do not allocate the space until we flush
you tend not to see old data. The only way out of something like this is
to prevent the inode size update from hitting disk until the file data
is on disk. The performance consequences of doing that are probably
large.
This situation is somewhat helped by the fact that if one page gets
flushed by bdflush and it calls back into xfs to allocate space, we
will allocate space for, and flush all surrounding data in the file,
so this may be causing earler flushing than might otherwise happen.
Since xfs usually operates with a much smaller in memory log than other
filesystems (64K default) and we have some synchronous transactions which
cause a flush of the in memory log, the amount that time can go backwards
by in a crash is a lot smaller.
Steve
^ permalink raw reply [flat|nested] 94+ messages in thread