System partially unusable after power loss...

System partially unusable after power loss...

[Manuel Reimer]

Hello,

Kernel is 2.6.27.7.

My Problem started as I got called, that login from KDM is not longer 
possible. In fact, the KDE login is possible, but soon after entering 
the password, a "kbuildsycoca" hangs and takes 100% CPU. Shortly before 
this happened, power was abruptly lost.

I rebooted to a live CD. As, for some reason, xfs_check was broken on 
that live CD (something with "db" in it was missing), I directly ran 
xfs_repair on the broken hard drive and it found some zero byte files 
and two files with ELF header, which may be (a part of) library files, I 
don't know, as the name wasn't restored.

Is it possible to find out what exactly happened to cause this system to 
be unusable? For me it seems like those restored ELF files are 
something, needed by KDE to start up properly. Why did library files get 
destroyed, the user, using this PC, isn't able to write to? Doesn't this 
mean, they don't get into a write cache and even a power loss can't 
destroy them?

Yours

Manuel

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: System partially unusable after power loss...

[Eric Sandeen]

Manuel Reimer wrote:
> Hello,
> 
> Kernel is 2.6.27.7.
> 
> My Problem started as I got called, that login from KDM is not longer
> possible. In fact, the KDE login is possible, but soon after entering
> the password, a "kbuildsycoca" hangs and takes 100% CPU. Shortly before
> this happened, power was abruptly lost.
> 
> I rebooted to a live CD. As, for some reason, xfs_check was broken on
> that live CD (something with "db" in it was missing), I directly ran

xfs_check is a shell script that invokes xfs_db

> xfs_repair on the broken hard drive and it found some zero byte files

better to run xfs_repair -n rather than xfs_check anyway ...

> and two files with ELF header, which may be (a part of) library files, I
> don't know, as the name wasn't restored.

Having the xfs_repair output would be helpful.

> Is it possible to find out what exactly happened to cause this system to
> be unusable? For me it seems like those restored ELF files are
> something, needed by KDE to start up properly. Why did library files get
> destroyed, the user, using this PC, isn't able to write to? Doesn't this
> mean, they don't get into a write cache and even a power loss can't
> destroy them?

Was there a system software update just prior to the power loss?

Did your storage support IO barriers (i.e. was it lvm or md)?

Buffered data is always lost on a power loss, but I'm not sure
why you should see problems with system files unless they had
just been written out (and not synced).

-Eric

> Yours
> 
> Manuel

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: System partially unusable after power loss...

[Michael Monnerie]

[-- Attachment #1.1: Type: Text/Plain, Size: 914 bytes --]

On Donnerstag, 16. September 2010 Manuel Reimer wrote:
> it found some zero byte files 
 
I've had the same issues, that's why I now run my KDE directory on an 
ext4 partition again. I really love XFS, but KDE it seems doesn't sync 
enough, or always changes some files, as I've had crashes at a time 
where nobody worked still some KDE config files were 0 afterwards. 
Luckily, I do a nightly backup of all those files too.

Sorry this is no help for you, I know. But a nightly rsync of the KDE 
dir would be good to do, just in case...

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services
http://proteger.at [gesprochen: Prot-e-schee]
Tel: 0660 / 415 65 31

****** Aktuelles Radiointerview! ******
http://www.it-podcast.at/aktuelle-sendung.html

// Wir haben im Moment zwei Häuser zu verkaufen:
// http://zmi.at/langegg/
// http://zmi.at/haus2009/

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: System partially unusable after power loss...

[Peter Grandi]

> I've had the same issues, that's why I now run my KDE directory on an 
> ext4 partition again. I really love XFS, but KDE it seems doesn't sync 
> enough, or always changes some files, as I've had crashes at a time 
> where nobody worked still some KDE config files were 0 afterwards. 

Why would anyone still be caught out by the 'O_PONIES' issue?

It is very well documented, and even minimally aware people
should have read at least the XFS FAQ, and/or:

http://sandeen.net/wordpress/?p=42
http://sandeen.net/wordpress/?p=34
http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: System partially unusable after power loss...

[Michael Monnerie]

[-- Attachment #1.1: Type: Text/Plain, Size: 553 bytes --]

On Freitag, 17. September 2010 Peter Grandi wrote:
> Why would anyone still be caught out by the 'O_PONIES' issue?
 
Because the problem is not solved. Documented yes, but not solved.

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services
http://proteger.at [gesprochen: Prot-e-schee]
Tel: 0660 / 415 65 31

****** Aktuelles Radiointerview! ******
http://www.it-podcast.at/aktuelle-sendung.html

// Wir haben im Moment zwei Häuser zu verkaufen:
// http://zmi.at/langegg/
// http://zmi.at/haus2009/

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: System partially unusable after power loss...

[Manuel Reimer]

Eric Sandeen wrote:
>> and two files with ELF header, which may be (a part of) library files, I
>> don't know, as the name wasn't restored.
>
> Having the xfs_repair output would be helpful.

I didn't save this...

> Was there a system software update just prior to the power loss?

The affected files have been updated some days/weeks before.

> Did your storage support IO barriers (i.e. was it lvm or md)?

No lvm, no md. A simple IDE drive.

> Buffered data is always lost on a power loss, but I'm not sure
> why you should see problems with system files unless they had
> just been written out (and not synced).

I fixed this system. The reason was this silly "kbuildsycoca" or better 
a config file, accessible by the user, using this PC. Seems like this 
"non system file" got corrupted in some way. A run of "kbuildsycoca 
--noincremental" fixed the config and the system was up and running again.

But why did xfs_repair restore files... For me it seems like xfs_repair 
restored an *old* version of openssl, which has been replaced by the 
system update process. This would mean: xfs_repair made a deleted file 
visible. Is this possible?

Yours

Manuel

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: System partially unusable after power loss...

[Eric Sandeen]

Manuel Reimer wrote:

> But why did xfs_repair restore files... For me it seems like xfs_repair
> restored an *old* version of openssl, which has been replaced by the
> system update process. This would mean: xfs_repair made a deleted file
> visible. Is this possible?

I don't know what to make of that; if you mean the openssl files got replaced
by older versions, it sounds very unlikely.

Sadly without repair output I can't say what repair did, but resurrecting
overwritten files from the past would be unexpected to say the least...

-Eric
 
> Yours
> 
> Manuel

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: System partially unusable after power loss...

[Manuel Reimer]

Eric Sandeen wrote:
> I don't know what to make of that; if you mean the openssl files got replaced
> by older versions, it sounds very unlikely.

No. What I mean is, that xfs_repair seems to have restored an old 
version to "/lost+found" without restoring the file name for it.

Yours

Manuel

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs