How errorproof is ext2 fs?

How errorproof is ext2 fs?

[Otto Wyss]

While reading the thread about "HFS Plus on Linux?" at
"debian-powerpc@list.debian.org" I had the following experience:

Within an hour I had to hard reset both of my computers, first my Linux-i386 due
to a complete lockup of the system while using el3diag, second my MacOS-powermac
due to an not responding USB-keyboard/-mouse (what a nice coincident). Now while
the Mac restarted without any fuse I had to fix the ext2-fs manually for about
15 min. Luckily it seems I haven't lost anything on both system. 

This leaves me a bad taste of Linux in my mouth. Does ext2 fs really behave so
worse in case of a crash? Okay Linux does not crash that often as MacOS does, so
it does not need a good  error proof fs. Still can't ext2 be made a little more
error proof?

Okay, there are other fs for Linux which cope better with such a situation, but
are they really more errorproof or are they just better in fixing up the mess
afterwards? Could there be more attention in not creating errors instead of
fixing them afterwards?

O. Wyss

Re: How errorproof is ext2 fs?

[Joel Jaeggli]

I hope this will help a bit...

On Thu, 13 Sep 2001, Otto Wyss wrote:

> While reading the thread about "HFS Plus on Linux?" at
> "debian-powerpc@list.debian.org" I had the following experience:
>
> Within an hour I had to hard reset both of my computers, first my Linux-i386 due
> to a complete lockup of the system while using el3diag, second my MacOS-powermac
> due to an not responding USB-keyboard/-mouse (what a nice coincident). Now while
> the Mac restarted without any fuse I had to fix the ext2-fs manually for about
> 15 min. Luckily it seems I haven't lost anything on both system.
>
> This leaves me a bad taste of Linux in my mouth. Does ext2 fs really behave so
> worse in case of a crash? Okay Linux does not crash that often as MacOS does, so
> it does not need a good  error proof fs. Still can't ext2 be made a little more
> error proof?

If you mount you ext2 filesytem with synchronous i/o rather than async
they'll be signficantly less brittle at the expense of some speed. I do
this occasionaly when testing new hardware or radical changes
something like:

mount -o remount sync /fsname

> Okay, there are other fs for Linux which cope better with such a
> situation, but are they really more errorproof or are they just better
> in fixing up the mess afterwards?

both actually if your're refering to reiserfs/ext3/jfs

> Could there be more attention in not
> creating errors instead of fixing them afterwards?

The errors are the result of omitted data not incorrect data... the files
were open the data that was gonna be written was sitting in the buffer, it
just hadn't been comitted yet...

> O. Wyss
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

-- 
--------------------------------------------------------------------------
Joel Jaeggli				       joelja@darkwing.uoregon.edu
Academic User Services			     consult@gladstone.uoregon.edu
     PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E
--------------------------------------------------------------------------
It is clear that the arm of criticism cannot replace the criticism of
arms.  Karl Marx -- Introduction to the critique of Hegel's Philosophy of
the right, 1843.

Re: How errorproof is ext2 fs?

[Alan Cox]

> due to an not responding USB-keyboard/-mouse (what a nice coincident). Now while
> the Mac restarted without any fuse I had to fix the ext2-fs manually for about
> 15 min. Luckily it seems I haven't lost anything on both system. 
> 
> This leaves me a bad taste of Linux in my mouth. Does ext2 fs really behave so
> worse in case of a crash? Okay Linux does not crash that often as MacOS does, so

That sounds like it behaved well. fsck didnt have enough info to safely
do all the fixup without asking you. Its not a reliability issue as such.

> it does not need a good  error proof fs. Still can't ext2 be made a little more
> error proof?

Ext3 is a journalled ext2. Its in the 2.4-ac kernel trees. Reiserfs in the
-ac tree also supports big endian boxes.

Alan

Re: How errorproof is ext2 fs?

[Otto Wyss]

> > This leaves me a bad taste of Linux in my mouth. Does ext2 fs really behave so
> > worse in case of a crash? Okay Linux does not crash that often as MacOS does, so
> 
> That sounds like it behaved well. fsck didnt have enough info to safely
> do all the fixup without asking you. Its not a reliability issue as such.
> 
Well it could also be the fact that almost no activity was going on on both
systems. 

> > it does not need a good  error proof fs. Still can't ext2 be made a little more
> > error proof?
> 
> Ext3 is a journalled ext2. Its in the 2.4-ac kernel trees. Reiserfs in the
> -ac tree also supports big endian boxes.
> 
At least ext2 and probably all the journalling fs lacks a feature the HFS+ from
the Mac has (bad tongues might say "needs"), to keep open files without activity
in a state where a crash has no effect. I don't know how it is done since I'm no
fs expert but my experience with my Mac (resetting about once a month without
loosing anything) shows that it's possible.

I'd rather like to see this feature appear in one fs for Linux (preferable
ext2). I think it's always better to not have error instead of fixing them afterwards.

O. Wyss

Re: How errorproof is ext2 fs?

[Mike Fedyk]

On Fri, Sep 14, 2001 at 09:16:53PM +0200, Otto Wyss wrote:
> > Ext3 is a journalled ext2. Its in the 2.4-ac kernel trees. Reiserfs in the
> > -ac tree also supports big endian boxes.
> > 
> At least ext2 and probably all the journalling fs lacks a feature the HFS+ from
> the Mac has (bad tongues might say "needs"), to keep open files without activity
> in a state where a crash has no effect. I don't know how it is done since I'm no
> fs expert but my experience with my Mac (resetting about once a month without
> loosing anything) shows that it's possible.
> 
> I'd rather like to see this feature appear in one fs for Linux (preferable
> ext2). I think it's always better to not have error instead of fixing them afterwards.
> 

You can.  Make sure that all mounts use the "sync" flag in /etc/fstab.

You need to remember that unix systems are designed to be in a stable
environment, and enable optimizations that rely on those conditions.  That
way, it can do more work in RAM which is faster than the disks.

If you expect to be in unstable conditions, you should take the speed hit,
and enable the sync flag.

Also remember, XFS, JFS, ReiserFS, and especially EXT3 take steps to do the
work for you without requiring the sync flag.

Mike

Re: How errorproof is ext2 fs?

[Rogier Wolff]

Alan Cox wrote:
> > due to an not responding USB-keyboard/-mouse (what a nice coincident). Now while
> > the Mac restarted without any fuse I had to fix the ext2-fs manually for about
> > 15 min. Luckily it seems I haven't lost anything on both system. 
> > 
> > This leaves me a bad taste of Linux in my mouth. Does ext2 fs really behave so
> > worse in case of a crash? Okay Linux does not crash that often as MacOS does, so
 
> That sounds like it behaved well. fsck didnt have enough info to safely
> do all the fixup without asking you. Its not a reliability issue as such.

Well, fsck wants to ask 

	"Found an unattached inode, connect to lost+found?"

to the user and will interrupt an automatic reboot for that.

This is bad: The safe choice is safe: It won't cause data-loss. 

Maybe it should report it (say by Email), but interrupting a reboot
just for connecting a couple of files to lost+found, that's
rediculous.

If it would give me enough information when I do this manually, I'd
make an informed decision. However, what are the chances of me knowing
that inode 123456 is a staroffice bak-file? So the only way to safely
operate is to link them into lost+found, and then to look at the files
manually.

			Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 

Re: How errorproof is ext2 fs?

[Frank Schneider]

Rogier Wolff schrieb:
> 
> Alan Cox wrote:
> > > due to an not responding USB-keyboard/-mouse (what a nice coincident). Now while
> > > the Mac restarted without any fuse I had to fix the ext2-fs manually for about
> > > 15 min. Luckily it seems I haven't lost anything on both system.
> > >
> > > This leaves me a bad taste of Linux in my mouth. Does ext2 fs really behave so
> > > worse in case of a crash? Okay Linux does not crash that often as MacOS does, so
> 
> > That sounds like it behaved well. fsck didnt have enough info to safely
> > do all the fixup without asking you. Its not a reliability issue as such.
> 
> Well, fsck wants to ask
> 
>         "Found an unattached inode, connect to lost+found?"
> 
> to the user and will interrupt an automatic reboot for that.
> 
> This is bad: The safe choice is safe: It won't cause data-loss.
> 
> Maybe it should report it (say by Email), but interrupting a reboot
> just for connecting a couple of files to lost+found, that's
> rediculous.
> 
> If it would give me enough information when I do this manually, I'd
> make an informed decision. However, what are the chances of me knowing
> that inode 123456 is a staroffice bak-file? So the only way to safely
> operate is to link them into lost+found, and then to look at the files
> manually.

Hello...

This is true, most distros are relatively rigid in dropping you to a
shell, because they call fsck with very weak options and do not care
about the fact that most servers are not standing under a table of
someone with easy access to the console.

If i have such a problem and get dropped to a shell, i normaly do a
simple "e2fsck /dev/XXX -p" or "-y" and this runs through and fixes the
filesystem without any questions.

I had only one time in the recent history where this did not work, i had
to repeat the steps a second and third time, but that was due to an
extreme error, i did e2fsck on a mounted filesystem during writing
.tar-backups there (error in crontab)...no good idea..:-).

So if you want to come around this "dropping-you-to-a-shell" problem you
could easily patch the file "/etc/rc.d/rc.sysinit" (RH) and call fsck
with the option "-p" or "-y", and you could easily change this script
that in cases of really bad trouble the system mounts / (or a
reserve-partition, even a CD would do AFAIK) readonly but starts up
normaly, so you can log in via net and do the repair by hand.

Solong..
Frank.

--
Frank Schneider, <SPATZ1@T-ONLINE.DE>.                           
... -.-

Re: How errorproof is ext2 fs?

[Luigi Genoni]

On Sun, 16 Sep 2001, Rogier Wolff wrote:

> Alan Cox wrote:
> > > due to an not responding USB-keyboard/-mouse (what a nice coincident). Now while
> > > the Mac restarted without any fuse I had to fix the ext2-fs manually for about
> > > 15 min. Luckily it seems I haven't lost anything on both system.
> > >
> > > This leaves me a bad taste of Linux in my mouth. Does ext2 fs really behave so
> > > worse in case of a crash? Okay Linux does not crash that often as MacOS does, so
>
> > That sounds like it behaved well. fsck didnt have enough info to safely
> > do all the fixup without asking you. Its not a reliability issue as such.
>
> Well, fsck wants to ask
>
> 	"Found an unattached inode, connect to lost+found?"
>
> to the user and will interrupt an automatic reboot for that.
>
> This is bad: The safe choice is safe: It won't cause data-loss.
>
> Maybe it should report it (say by Email), but interrupting a reboot
> just for connecting a couple of files to lost+found, that's
> rediculous.
>
> If it would give me enough information when I do this manually, I'd
> make an informed decision. However, what are the chances of me knowing
> that inode 123456 is a staroffice bak-file? So the only way to safely
> operate is to link them into lost+found, and then to look at the files
> manually.
>
That is the right way a block based FS non journaled should act in a Unix
system. Basically fsck can see inode numbers, not file names.
That is also the way UFS acts, and belive me, older UFS versions were
mutch more prone to corruptions and crashes (Solaris 8 UFS seems t make
journaling of part of meta-data, but I am not sure). In front of UFS ext2
is more than rock solid. Unattached i-nodes are just something that can
happen for this kind of filesystems.

If you want a different behaviour, like for sure many others suggested, go
to a journaled filesystem. It seems to me that you would need a filesystem
that would make journaling of both meta-data and data like ext3 or jfs,
(actually reiserFS journals just metadata)

Luigi

Re: How errorproof is ext2 fs?

[Otto Wyss]

> > it does not need a good  error proof fs. Still can't ext2 be made a little more
> > error proof?
> 
> First, you seem to be making quite a few assumptions which are not
> necessarily correct:
> 
> 1) Just because an OS does not _tell_ you there was a problem does not
> necessarily mean there wasn't one.  In particular, MacOS is notorious
> for not "bothering" users with detailed (or in many cases any)
> information about problems.  I don't know about HFS fragility in
> 
Well I can assure you I used DiskFirstAid and Norton Utilities on the Mac.
Neither found any problems. Since I use MacOS9, my Mac crashes at least once a
month but I never ever lost anything. Most of the time my USB-keyboard/-mouse
doesn't react anymore after switching back from my Linux system. Usually I
simply press the reset switch after a few minutes. 

Besides Linux also does not react occasionally when switching my
USB-keyboard/-mouse but since I also have an AT-keyboard handy I don't have to
reset it.

> 3) You're basing your linux experience on (apparently) only one
> incident.  As a professional sysadmin, I've experienced many many

I'm not bashing linux, I'm only stating that MacOS9 (or HFS+) has a nice feature
Linux (or its fs's) lacks. My experience shows clearly that I had at least 10
craches on the Mac durning the last year but I never lost anything. During the
same time I had 2 crashes on my linux, one due to the USB-problem without a
handy AT-keyboard, the second now due to a malfunction el3diag.

O. Wyss

Re: How errorproof is ext2 fs?

[Alan Cox]

> > 3) You're basing your linux experience on (apparently) only one
> > incident.  As a professional sysadmin, I've experienced many many
> 
> I'm not bashing linux, I'm only stating that MacOS9 (or HFS+) has a nice feature
> Linux (or its fs's) lacks. My experience shows clearly that I had at least 10

Its a case of "the fs you are using". Ext3 is a big improvement - getting
stuff wrong in my prototype voodoo2 dri module is now a 30 second problem 8)

Re: How errorproof is ext2 fs?

[Jesse Pollard]

Otto Wyss <otto.wyss@bluewin.ch>:
> While reading the thread about "HFS Plus on Linux?" at
> "debian-powerpc@list.debian.org" I had the following experience:
> 
> Within an hour I had to hard reset both of my computers, first my Linux-i386 due
> to a complete lockup of the system while using el3diag, second my MacOS-powermac
> due to an not responding USB-keyboard/-mouse (what a nice coincident). Now while
> the Mac restarted without any fuse I had to fix the ext2-fs manually for about
> 15 min. Luckily it seems I haven't lost anything on both system. 
> 
> This leaves me a bad taste of Linux in my mouth. Does ext2 fs really behave so
> worse in case of a crash? Okay Linux does not crash that often as MacOS does, so
> it does not need a good  error proof fs. Still can't ext2 be made a little more
> error proof?
> 
> Okay, there are other fs for Linux which cope better with such a situation, but
> are they really more errorproof or are they just better in fixing up the mess
> afterwards? Could there be more attention in not creating errors instead of
> fixing them afterwards?

I've used linux for about 8 years now. The only time I've had a catastrophic
failure was with a disk drive went south.

About the only times I've seen ext2fs require manual repair is a crash/power
failure during fsck on boot. It doesn't happen very often. Even then, it
may not be a serious falure, just the type of error that requires a choice
in fix - missing inode/partially written inode in the root file system will
usually require the choice of deleting, or putting in lost+found.

No file system is immune to that level of failure. Some are better at
hiding the damage (xfs will lose free data blocks like mad - 3 in a row lost
6GB out of 12, though no used data was (visibly) lost.

15 minutes isn't that bad - wait until you have to spend 30 minutes to
3 hours on an NTFS or FAT32 rebuild, only to find you have to reinstall.

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

Re: How errorproof is ext2 fs?

[Otto Wyss]

> there's no reason you can't configure your boot scripts to automatically
> fix such problems.  and by "manually", I hope you meant that "fsck -y"
> took 15 minutes to run.
> ...
What's this "-y" meaning, "man fsck" does not show it. Could this mean answer
any question with "yes"? The 15 minutes I needed to answer about the first
hundreds of questions with "y", afterwards I simply pressed "y" until the fsck
was finished.

O. Wyss

Re: How errorproof is ext2 fs?

[Andreas Dilger]

On Sep 14, 2001  22:47 +0200, Otto Wyss wrote:
> > there's no reason you can't configure your boot scripts to automatically
> > fix such problems.  and by "manually", I hope you meant that "fsck -y"
> > took 15 minutes to run.
> > ...
> What's this "-y" meaning, "man fsck" does not show it. Could this mean answer
> any question with "yes"? The 15 minutes I needed to answer about the first
> hundreds of questions with "y", afterwards I simply pressed "y" until the fsck
> was finished.

Well, the "best" way of running e2fsck is with the "-p" option, so that it
will pick y/n as appropriate.  In almost all cases "-p" and "-y" behave
the same, but in a few rare cases they do not.

In general, if you are manually fsck'ing a filesystem, it is better to
run fsck.<fstype> directly (and read its man page) instead of the "fsck"
wrapper program.

In some rare cases, fsck cannot decide what is the right thing to do, so
you need to run it in manual mode, which appears to be what happened to you.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: How errorproof is ext2 fs?

[Timothy A. Seufert]

Otto Wyss <otto.wyss@bluewin.ch> wrote:

>At least ext2 and probably all the journalling fs lacks a feature the HFS+
>from the Mac has (bad tongues might say "needs"), to keep open files
>without activity in a state where a crash has no effect. I don't know how
>it is done since I'm no fs expert but my experience with my Mac (resetting
>about once a month without loosing anything) shows that it's possible.

HFS+ (the file system, as opposed to implementations of it) has no 
such feature.

In fact, HFS+ is probably more vulnerable to file system damage than 
a FS like ext2, simply because it uses a B-Tree structure.  B-Trees 
have their purposes (such as searching the whole FS for a file 
quickly, a capability very important to classic MacOS), but are 
generally not as robust as a simple inode FS like ext2.

The only thing which prevents damage from being common on MacOS 9 is 
the slow and unsophisticated MacOS 9 *implementation* of HFS+.  I'm 
pretty sure it is synchronous and single-threaded.  And the MacOS 
cache doesn't keep dirty buffers for any significant amount of time 
(no more than ~0.5s I think).  These things mean that at any given 
point in time, the state of the HFS+ metadata on disk is probably 
coherent or close to it.

As others have mentioned -- if you want FS crash resistance somewhat 
more like MacOS 9, mount your ext2 filesystems sync.  Be prepared for 
a huge performance loss.  But the correct thing to do is to figure 
out why Linux is crashing and fix it -- Linux can and should stay up 
for months or years.

Also, under Darwin (aka the MacOS X BSD layer), the likelihood of 
damage to a HFS+ volume after a crash/reboot is significantly higher. 
Darwin has a high performance async HFS+ implementation with a real 
buffer cache behind it.

-- 
Tim Seufert