[PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Matt]

Hi guys,

before diving into testing out 2.6.37-rc* kernels I wanted to make
sure that the patch:

[dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3

is safe to use with e.g. >2.6.37-rc1 kernels

I know that it's not a "fix all" patch but it significantly seems to
speed up my backup jobs (by factor 2-3)
and 2.6.37* has evolved that much that interactivity isn't hurt too much

Thanks !

Matt

Re: [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Milan Broz]

On 11/06/2010 11:16 PM, Matt wrote:
> before diving into testing out 2.6.37-rc* kernels I wanted to make
> sure that the patch:
> 
> [dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3
> 
> is safe to use with e.g. >2.6.37-rc1 kernels
> 
> I know that it's not a "fix all" patch but it significantly seems to
> speed up my backup jobs (by factor 2-3)
> and 2.6.37* has evolved that much that interactivity isn't hurt too much

yes, it should work for the simple mappings without problems.

I hope we will fix the patch soon to be ready for upstream.

Milan

Re: [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Matt]

On Sun, Nov 7, 2010 at 3:30 PM, Milan Broz <mbroz@redhat.com> wrote:
>
> On 11/06/2010 11:16 PM, Matt wrote:
>> before diving into testing out 2.6.37-rc* kernels I wanted to make
>> sure that the patch:
>>
>> [dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3
>>
>> is safe to use with e.g. >2.6.37-rc1 kernels
>>
>> I know that it's not a "fix all" patch but it significantly seems to
>> speed up my backup jobs (by factor 2-3)
>> and 2.6.37* has evolved that much that interactivity isn't hurt too much
>
> yes, it should work for the simple mappings without problems.
>
> I hope we will fix the patch soon to be ready for upstream.
>
> Milan
>

Hi Milan,

thanks for your answer !

Unfortunately I have to post a "Warning" that it's currently not safe
(at least for me) to use it

a few hours ago before 2.6.37-rc1 was tagged I already had shortly
tested it with the dm-crypt multi-cpu patch and massive "silent" data
corruption or loss occured:

fortunately I don't/didn't see any data-corruption on my /home
partition (yet) but everytime I boot into my system things are screwed
up on the root-partition, e.g.:

where
eselect opengl list would show
>Available OpenGL implementations:
>  [1]   ati *
>  [2]   xorg-x11

normally it's
>cat /etc/env.d/03opengl
># Configuration file for eselect
># This file has been automatically generated.
>LDPATH="/usr/lib32/opengl/ati/lib:/usr/lib64/opengl/ati/lib"
>OPENGL_PROFILE="ati"


it currently says:

>eselect opengl list
>Available OpenGL implementations:
>  [1]   ati
>  [2]   xorg-x11


>cat /etc/env.d/03opengl
># Configuration file for eselect

and another example was a corrupted /etc/init.d/killprocs

so since this (a corrupted killprocs) already had happened in the past
(last time due to a hardlock with fglrx/amd's catalyst driver) I
thought it was some kind of system problem which could be fixed:
I fired up a rebuild-job (emerge -e system) for my system and (surely)
some other stuff disappeared - after 2-3 reboots I wanted to continue
finishing the rebuild and gcc was gone (!)

I don't have the time to re-test everything since this is my testing &
production machine (I'll play back a system-backup tarball) but this
didn't happen (yet) with 2.6.36 and
the following patches related to multi-cpu dm-crypt:

* [dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3
* [PATCH] Use generic private pointer in per-cpu struct

so it seems to be safe.

It has to be changes which got introduced with 2.6.37* which broke
stuff. 2.6.36 seems to work perfectly fine with those 2 patches since
several days already

I'll stick with 2.6.36 for some time now

Thanks !

Matt

Re: [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Matt]

On Sun, Nov 7, 2010 at 6:49 PM, Matt <jackdachef@gmail.com> wrote:
> On Sun, Nov 7, 2010 at 3:30 PM, Milan Broz <mbroz@redhat.com> wrote:
>>
>> On 11/06/2010 11:16 PM, Matt wrote:
>>> before diving into testing out 2.6.37-rc* kernels I wanted to make
>>> sure that the patch:
>>>
>>> [dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3
>>>
>>> is safe to use with e.g. >2.6.37-rc1 kernels
>>>
>>> I know that it's not a "fix all" patch but it significantly seems to
>>> speed up my backup jobs (by factor 2-3)
>>> and 2.6.37* has evolved that much that interactivity isn't hurt too much
>>
>> yes, it should work for the simple mappings without problems.
>>
>> I hope we will fix the patch soon to be ready for upstream.
>>
>> Milan
>>
>
> Hi Milan,
>
> thanks for your answer !
>
> Unfortunately I have to post a "Warning" that it's currently not safe
> (at least for me) to use it
>
> a few hours ago before 2.6.37-rc1 was tagged I already had shortly
> tested it with the dm-crypt multi-cpu patch and massive "silent" data
> corruption or loss occured:
>
> fortunately I don't/didn't see any data-corruption on my /home
> partition (yet) but everytime I boot into my system things are screwed
> up on the root-partition, e.g.:
>
> where
> eselect opengl list would show
>>Available OpenGL implementations:
>>  [1]   ati *
>>  [2]   xorg-x11
>
> normally it's
>>cat /etc/env.d/03opengl
>># Configuration file for eselect
>># This file has been automatically generated.
>>LDPATH="/usr/lib32/opengl/ati/lib:/usr/lib64/opengl/ati/lib"
>>OPENGL_PROFILE="ati"
>
>
> it currently says:
>
>>eselect opengl list
>>Available OpenGL implementations:
>>  [1]   ati
>>  [2]   xorg-x11
>
>
>>cat /etc/env.d/03opengl
>># Configuration file for eselect
>
> and another example was a corrupted /etc/init.d/killprocs
>
> so since this (a corrupted killprocs) already had happened in the past
> (last time due to a hardlock with fglrx/amd's catalyst driver) I
> thought it was some kind of system problem which could be fixed:
> I fired up a rebuild-job (emerge -e system) for my system and (surely)
> some other stuff disappeared - after 2-3 reboots I wanted to continue
> finishing the rebuild and gcc was gone (!)
>
> I don't have the time to re-test everything since this is my testing &
> production machine (I'll play back a system-backup tarball) but this
> didn't happen (yet) with 2.6.36 and
> the following patches related to multi-cpu dm-crypt:
>
> * [dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3
> * [PATCH] Use generic private pointer in per-cpu struct
>
> so it seems to be safe.
>
> It has to be changes which got introduced with 2.6.37* which broke
> stuff. 2.6.36 seems to work perfectly fine with those 2 patches since
> several days already
>
> I'll stick with 2.6.36 for some time now
>
> Thanks !
>
> Matt
>

sorry - I forgot to include the most important part:

the system-partition is on an LVM/Volume Group that sits on an
cryptsetup partition so:

cryptsetup (luks) -> LVM/Volume Group (2 partitions, one of them
system - the other swap) -> system (ext4)

[cryptsetup -> LVM -> ext4-partition]

the mount-options were/are:

noatime,nodiratime,barrier=1

sometimes also

noatime,nodiratime,barrier=1,commit=600
(when the system runs for several hours to make it consume less
energy/write less)

the other settings are:

echo "3000" > /proc/sys/vm/dirty_expire_centisecs
echo "1500"  > /proc/sys/vm/dirty_writeback_centisecs
echo "15" > /proc/sys/vm/dirty_background_ratio
echo "50"   > /proc/sys/vm/dirty_ratio
echo "50" > /proc/sys/vm/vfs_cache_pressure

i/o scheduler: cfq

as already mentioned - this problem didn't appear or wasn't noticable
(yet) with or until 2.6.36 - my system-memory should also be
error-free (tested via memtest86), the harddisk too (previously tested
several times via badblocks without errors)

during every shutdown, reboot, hibernate, etc.

I do a manual:

sync && sdparm -C sync /dev/foo

to make sure data gets to the partition

I read about barrier-problems and data getting to the partition when
using dm-crypt and several layers so I don't know if that could be
related

Regards

Matt

Re: [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Andi Kleen]

> I read about barrier-problems and data getting to the partition when
> using dm-crypt and several layers so I don't know if that could be
> related

Barriers seem to be totally broken on dm-crypt currently.

But that's probably not your problem. I use the scalability patch
on 2.6.36 and it's very stable. Most likely some mistake 
in the forward port.

-Andi

Re: [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Milan Broz]

On 11/07/2010 08:45 PM, Andi Kleen wrote:
>> I read about barrier-problems and data getting to the partition when
>> using dm-crypt and several layers so I don't know if that could be
>> related
> 
> Barriers seem to be totally broken on dm-crypt currently.

Can you explain it?

Barriers/flush change should work, if it is broken, it is not only dm-crypt.
(dm-crypt simply relies on dm-core implementation, when barrier/flush
request come to dmcrypt, all previous IO must be already finished).

Milan

Re: [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Andi Kleen]

On Sun, Nov 07, 2010 at 10:39:23PM +0100, Milan Broz wrote:
> On 11/07/2010 08:45 PM, Andi Kleen wrote:
> >> I read about barrier-problems and data getting to the partition when
> >> using dm-crypt and several layers so I don't know if that could be
> >> related
> > 
> > Barriers seem to be totally broken on dm-crypt currently.
> 
> Can you explain it?

e.g. the btrfs mailing list is full of corruption reports
on dm-crypt and most of the symptoms point to broken barriers.

> Barriers/flush change should work, if it is broken, it is not only dm-crypt.
> (dm-crypt simply relies on dm-core implementation, when barrier/flush
> request come to dmcrypt, all previous IO must be already finished).

Possibly, at least it doesn't seem to work.

-Andi
-- 
ak@linux.intel.com -- Speaking for myself only.

Re: [dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Alasdair G Kergon]

On Mon, Nov 08, 2010 at 12:05:09AM +0100, Andi Kleen wrote:
> e.g. the btrfs mailing list is full of corruption reports
> on dm-crypt and most of the symptoms point to broken barriers.

linux-btrfs?  I'm not subscribed, but I the searches I've tried
don't show it to be "full of corruption reports".

Could you post links to the threads concerned so we can investigate?

Are we just talking -rc1 or earlier too?

Thanks,
Alasdair

Re: DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Mike Snitzer]

On Sun, Nov 07 2010 at  6:05pm -0500,
Andi Kleen <andi@firstfloor.org> wrote:

> On Sun, Nov 07, 2010 at 10:39:23PM +0100, Milan Broz wrote:
> > On 11/07/2010 08:45 PM, Andi Kleen wrote:
> > >> I read about barrier-problems and data getting to the partition when
> > >> using dm-crypt and several layers so I don't know if that could be
> > >> related
> > > 
> > > Barriers seem to be totally broken on dm-crypt currently.
> > 
> > Can you explain it?
> 
> e.g. the btrfs mailing list is full of corruption reports
> on dm-crypt and most of the symptoms point to broken barriers.

[cc'ing linux-btrfs, hopefully in the future dm-devel will get cc'd when
concerns about DM come up on linux-btrfs (or other lists)]

I spoke with Josef Bacik and these corruption reports are apparently
against older kernels (e.g. <= 2.6.33).  I say <= 2.6.33 because:

https://btrfs.wiki.kernel.org/index.php/Gotchas states:
"btrfs volumes on top of dm-crypt block devices (and possibly LVM)
require write-caching to be turned off on the underlying HDD. Failing to
do so, in the event of a power failure, may result in corruption not yet
handled by btrfs code. (2.6.33)"

But Josef was not aware of any reports with kernels newer than 2.6.32
(F12).

Josef also noted that until last week btrfs wouldn't retry another
mirror in the face of some corruption, the fix is here:
http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-unstable.git;a=commit;h=cb44921a09221

This obviously doesn't fix any source of corruption but it makes btrfs
more resilient when it encounters the corruption.

> > Barriers/flush change should work, if it is broken, it is not only dm-crypt.
> > (dm-crypt simply relies on dm-core implementation, when barrier/flush
> > request come to dmcrypt, all previous IO must be already finished).
> 
> Possibly, at least it doesn't seem to work.

Can you please be more specific?  What test(s)?  What kernel(s)?

Any pointers to previous (and preferably: recent) reports would be
appreciated.

The DM barrier code has seen considerable change recently (via flush+fua
changes in 2.6.37).  Those changes have been tested quite a bit
(including ext4 consistency after a crash).

But even prior to those flush+fua changes DM's support for barriers
(Linux >= 2.6.31) was held to be robust.  No known (at least no
reported) issues with DM's barrier support.

Mike

Re: DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Chris Mason]

Excerpts from Mike Snitzer's message of 2010-11-08 09:58:09 -0500:
> On Sun, Nov 07 2010 at  6:05pm -0500,
> Andi Kleen <andi@firstfloor.org> wrote:
> 
> > On Sun, Nov 07, 2010 at 10:39:23PM +0100, Milan Broz wrote:
> > > On 11/07/2010 08:45 PM, Andi Kleen wrote:
> > > >> I read about barrier-problems and data getting to the partition when
> > > >> using dm-crypt and several layers so I don't know if that could be
> > > >> related
> > > > 
> > > > Barriers seem to be totally broken on dm-crypt currently.
> > > 
> > > Can you explain it?
> > 
> > e.g. the btrfs mailing list is full of corruption reports
> > on dm-crypt and most of the symptoms point to broken barriers.
> 
> [cc'ing linux-btrfs, hopefully in the future dm-devel will get cc'd when
> concerns about DM come up on linux-btrfs (or other lists)]
> 
> I spoke with Josef Bacik and these corruption reports are apparently
> against older kernels (e.g. <= 2.6.33).  I say <= 2.6.33 because:

We've consistently seen reports about corruptions on power hits with
dm-crypt.  The logs didn't have any messages about barriers failing, but
the corruptions were still there.  The most likely cause is that
barriers just aren't getting through somehow.

> 
> https://btrfs.wiki.kernel.org/index.php/Gotchas states:
> "btrfs volumes on top of dm-crypt block devices (and possibly LVM)
> require write-caching to be turned off on the underlying HDD. Failing to
> do so, in the event of a power failure, may result in corruption not yet
> handled by btrfs code. (2.6.33)"
> 
> But Josef was not aware of any reports with kernels newer than 2.6.32
> (F12).
> 
> Josef also noted that until last week btrfs wouldn't retry another
> mirror in the face of some corruption, the fix is here:
> http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-unstable.git;a=commit;h=cb44921a09221
> 
> This obviously doesn't fix any source of corruption but it makes btrfs
> more resilient when it encounters the corruption.

Right.

> 
> > > Barriers/flush change should work, if it is broken, it is not only dm-crypt.
> > > (dm-crypt simply relies on dm-core implementation, when barrier/flush
> > > request come to dmcrypt, all previous IO must be already finished).
> > 
> > Possibly, at least it doesn't seem to work.
> 
> Can you please be more specific?  What test(s)?  What kernel(s)?
> 
> Any pointers to previous (and preferably: recent) reports would be
> appreciated.
> 
> The DM barrier code has seen considerable change recently (via flush+fua
> changes in 2.6.37).  Those changes have been tested quite a bit
> (including ext4 consistency after a crash).
> 
> But even prior to those flush+fua changes DM's support for barriers
> (Linux >= 2.6.31) was held to be robust.  No known (at least no
> reported) issues with DM's barrier support.

I think it would be best to move forward with just hammering on the
dm-crypt barriers:

http://oss.oracle.com/~mason/barrier-test

This script is the best I've found so far to reliably trigger
corruptions with barriers off.  I'd start with ext3 + barriers off just
to prove it corrupts things, then move to ext3 + barriers on.

-chris

dm-crypt barrier support is effective (was: Re: DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?)

[Mike Snitzer]

On Mon, Nov 08 2010 at 12:59pm -0500,
Chris Mason <chris.mason@oracle.com> wrote:

> Excerpts from Mike Snitzer's message of 2010-11-08 09:58:09 -0500:
> > On Sun, Nov 07 2010 at  6:05pm -0500,
> > Andi Kleen <andi@firstfloor.org> wrote:
> > 
> > > On Sun, Nov 07, 2010 at 10:39:23PM +0100, Milan Broz wrote:
> > > > On 11/07/2010 08:45 PM, Andi Kleen wrote:
> > > > >> I read about barrier-problems and data getting to the partition when
> > > > >> using dm-crypt and several layers so I don't know if that could be
> > > > >> related
> > > > > 
> > > > > Barriers seem to be totally broken on dm-crypt currently.
> > > > 
> > > > Can you explain it?
> > > 
> > > e.g. the btrfs mailing list is full of corruption reports
> > > on dm-crypt and most of the symptoms point to broken barriers.
> > 
> > [cc'ing linux-btrfs, hopefully in the future dm-devel will get cc'd when
> > concerns about DM come up on linux-btrfs (or other lists)]
> > 
> > I spoke with Josef Bacik and these corruption reports are apparently
> > against older kernels (e.g. <= 2.6.33).  I say <= 2.6.33 because:
> 
> We've consistently seen reports about corruptions on power hits with
> dm-crypt.  The logs didn't have any messages about barriers failing, but
> the corruptions were still there.  The most likely cause is that
> barriers just aren't getting through somehow.

Can't blame anyone for assuming as much (although it does create FUD)
but in practice (testing dm-crypt with ext4 using your barrier-test
script) I have not been able to see any evidence that dm-crypt's barrier
support is ineffective.

Could be that the barrier-test script isn't able to reproduce the unique
failure case that btrfs does (on power failure)?

> > > > Barriers/flush change should work, if it is broken, it is not only dm-crypt.
> > > > (dm-crypt simply relies on dm-core implementation, when barrier/flush
> > > > request come to dmcrypt, all previous IO must be already finished).
> > > 
> > > Possibly, at least it doesn't seem to work.
> > 
> > Can you please be more specific?  What test(s)?  What kernel(s)?
> > 
> > Any pointers to previous (and preferably: recent) reports would be
> > appreciated.

I still think we need specific bug reports that detail workloads and if
possible reproducers.

> > The DM barrier code has seen considerable change recently (via flush+fua
> > changes in 2.6.37).  Those changes have been tested quite a bit
> > (including ext4 consistency after a crash).
> > 
> > But even prior to those flush+fua changes DM's support for barriers
> > (Linux >= 2.6.31) was held to be robust.  No known (at least no
> > reported) issues with DM's barrier support.
> 
> I think it would be best to move forward with just hammering on the
> dm-crypt barriers:
> 
> http://oss.oracle.com/~mason/barrier-test
> 
> This script is the best I've found so far to reliably trigger
> corruptions with barriers off.  I'd start with ext3 + barriers off just
> to prove it corrupts things, then move to ext3 + barriers on.

I started with ext4 + barrier=0,journal_async_commit and could reliably
cause directory corruption (~75% of the time).  I then switched to
barrier=1 and could not cause corruption.

I then added dm-crypt and got the same results: with barrier=1 I could
not cause directory corruption.  barrier=0 resulted in directory
corruption (again ~75% of the time), no corruption occurred with
barrier=1.

Both 2.6.36 (original barrier code) and latest 2.6.37-rc1+ (new
flush+fua code) were tested.  6 iterations of barrier=0 and 10
iterations of barrier=1.

So my hope is we can now put this general dm-crypt barrier doubt to one
side and work together on identifying the cause of corruption when
dm-crypt is paired with btrfs.

Thanks,
Mike

Re: dm-crypt barrier support is effective (was: Re: DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?)

[Matt]

On Sun, Nov 14, 2010 at 9:59 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Mon, Nov 08 2010 at 12:59pm -0500,
> Chris Mason <chris.mason@oracle.com> wrote:
>
>> Excerpts from Mike Snitzer's message of 2010-11-08 09:58:09 -0500:
>> > On Sun, Nov 07 2010 at  6:05pm -0500,
>> > Andi Kleen <andi@firstfloor.org> wrote:
>> >
>> > > On Sun, Nov 07, 2010 at 10:39:23PM +0100, Milan Broz wrote:
>> > > > On 11/07/2010 08:45 PM, Andi Kleen wrote:
>> > > > >> I read about barrier-problems and data getting to the partition when
>> > > > >> using dm-crypt and several layers so I don't know if that could be
>> > > > >> related
>> > > > >
>> > > > > Barriers seem to be totally broken on dm-crypt currently.
>> > > >
>> > > > Can you explain it?
>> > >
>> > > e.g. the btrfs mailing list is full of corruption reports
>> > > on dm-crypt and most of the symptoms point to broken barriers.
>> >
>> > [cc'ing linux-btrfs, hopefully in the future dm-devel will get cc'd when
>> > concerns about DM come up on linux-btrfs (or other lists)]
>> >
>> > I spoke with Josef Bacik and these corruption reports are apparently
>> > against older kernels (e.g. <= 2.6.33).  I say <= 2.6.33 because:
>>
>> We've consistently seen reports about corruptions on power hits with
>> dm-crypt.  The logs didn't have any messages about barriers failing, but
>> the corruptions were still there.  The most likely cause is that
>> barriers just aren't getting through somehow.
>
> Can't blame anyone for assuming as much (although it does create FUD)
> but in practice (testing dm-crypt with ext4 using your barrier-test
> script) I have not been able to see any evidence that dm-crypt's barrier
> support is ineffective.
>
> Could be that the barrier-test script isn't able to reproduce the unique
> failure case that btrfs does (on power failure)?
>
>> > > > Barriers/flush change should work, if it is broken, it is not only dm-crypt.
>> > > > (dm-crypt simply relies on dm-core implementation, when barrier/flush
>> > > > request come to dmcrypt, all previous IO must be already finished).
>> > >
>> > > Possibly, at least it doesn't seem to work.
>> >
>> > Can you please be more specific?  What test(s)?  What kernel(s)?
>> >
>> > Any pointers to previous (and preferably: recent) reports would be
>> > appreciated.
>
> I still think we need specific bug reports that detail workloads and if
> possible reproducers.
>
>> > The DM barrier code has seen considerable change recently (via flush+fua
>> > changes in 2.6.37).  Those changes have been tested quite a bit
>> > (including ext4 consistency after a crash).
>> >
>> > But even prior to those flush+fua changes DM's support for barriers
>> > (Linux >= 2.6.31) was held to be robust.  No known (at least no
>> > reported) issues with DM's barrier support.
>>
>> I think it would be best to move forward with just hammering on the
>> dm-crypt barriers:
>>
>> http://oss.oracle.com/~mason/barrier-test
>>
>> This script is the best I've found so far to reliably trigger
>> corruptions with barriers off.  I'd start with ext3 + barriers off just
>> to prove it corrupts things, then move to ext3 + barriers on.
>
> I started with ext4 + barrier=0,journal_async_commit and could reliably
> cause directory corruption (~75% of the time).  I then switched to
> barrier=1 and could not cause corruption.
>
> I then added dm-crypt and got the same results: with barrier=1 I could
> not cause directory corruption.  barrier=0 resulted in directory
> corruption (again ~75% of the time), no corruption occurred with
> barrier=1.
>
> Both 2.6.36 (original barrier code) and latest 2.6.37-rc1+ (new
> flush+fua code) were tested.  6 iterations of barrier=0 and 10
> iterations of barrier=1.
>
> So my hope is we can now put this general dm-crypt barrier doubt to one
> side and work together on identifying the cause of corruption when
> dm-crypt is paired with btrfs.
>
> Thanks,
> Mike
>

Hi Mike,

I'm pretty sure that dm-crypt is rockstable :)

My report wasn't meant to be / cause FUD sorry if it got picked up that way:

with the vanilla dm-crypt implementation I saw *NO* corruption at all
in the small testing amount of time I ran it
however as soon as I applied the

[dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3
and
[PATCH] Fix double free and use generic private pointer in per-cpu

patches, recompiled the kernel and rebooted into that new environment

it seemingly caused corruptions right from the start (the mentioned
corruption /etc/env.d/02opengl to be the most obvious candidate and
probably even more)
with those corruptions being anticipated over longer uptime and heavy
use-patterns (such as re-compiling the whole system).

I don't know if the new multi-cpu scaling patch for dm-crypt makes a
change (since I can't test it right now due to a busy schedule)

[PATCH v5] dm crypt: scale to multiple CPUs

I however have a request:

could you guys please take this patch through a "battery of heavy
tests" until it's included in mainline ?

so that you can spot any issues (races, BUGs, etc.) which might be
inherent/triggered in current dm-crypt so that my reported corruptions
might be prevented in the future ?

Again:

the vanilla kernel and dm-crypt are perfectly stable !

only with the dm-crypt scaling patch I could observe the data-corruption


Thanks !

Matt

Re: dm-crypt barrier support is effective

[Milan Broz]

On 11/14/2010 10:49 PM, Matt wrote:
> only with the dm-crypt scaling patch I could observe the data-corruption

even with v5 I sent on Friday?

Are you sure that it is not related to some fs problem in 2.6.37-rc1?

If it works on 2.6.36 without problems, it is probably problems somewhere
else (flush/fua conversion was trivial here - DM is still doing full flush
and there are no other changes in code IMHO.)

Milan

Re: dm-crypt barrier support is effective

[Matt]

On Sun, Nov 14, 2010 at 10:54 PM, Milan Broz <mbroz@redhat.com> wrote:
> On 11/14/2010 10:49 PM, Matt wrote:
>> only with the dm-crypt scaling patch I could observe the data-corruption
>
> even with v5 I sent on Friday?
>
> Are you sure that it is not related to some fs problem in 2.6.37-rc1?
>
> If it works on 2.6.36 without problems, it is probably problems somewhere
> else (flush/fua conversion was trivial here - DM is still doing full flush
> and there are no other changes in code IMHO.)
>
> Milan
>

Hi Milan,

I'm aware of your new v5 patch (which should include several
improvements (or potential fixes in my case) over the v3 patch)

as I already wrote my schedule unfortunately currently doesn't allow
me to test it

* in the case of no corruption it would be nice to have 2.6.37-rc* running :)

* in the case of data corruption that would mean restoring my system -
since it's my production box and right now I don't have a fallback at
reach
at earliest I could give it a shot at the beginning of December. Then
I could also test reiserfs and ext4 as a system partition to rule out
that it's
a ext4-specific thing (currently I'm running reiserfs on my system-partition).

Thanks !

Matt

Re: dm-crypt barrier support is effective

[Matt]

On Mon, Nov 15, 2010 at 12:24 AM, Matt <jackdachef@gmail.com> wrote:
> On Sun, Nov 14, 2010 at 10:54 PM, Milan Broz <mbroz@redhat.com> wrote:
>> On 11/14/2010 10:49 PM, Matt wrote:
>>> only with the dm-crypt scaling patch I could observe the data-corruption
>>
>> even with v5 I sent on Friday?
>>
>> Are you sure that it is not related to some fs problem in 2.6.37-rc1?
>>
>> If it works on 2.6.36 without problems, it is probably problems somewhere
>> else (flush/fua conversion was trivial here - DM is still doing full flush
>> and there are no other changes in code IMHO.)
>>
>> Milan
>>
>
> Hi Milan,
>
> I'm aware of your new v5 patch (which should include several
> improvements (or potential fixes in my case) over the v3 patch)
>
> as I already wrote my schedule unfortunately currently doesn't allow
> me to test it
>
> * in the case of no corruption it would be nice to have 2.6.37-rc* running :)
>
> * in the case of data corruption that would mean restoring my system -
> since it's my production box and right now I don't have a fallback at
> reach
> at earliest I could give it a shot at the beginning of December. Then
> I could also test reiserfs and ext4 as a system partition to rule out
> that it's
> a ext4-specific thing (currently I'm running reiserfs on my system-partition).
>
> Thanks !
>
> Matt
>


OK guys,

I've updated my system to latest glibc 2.12.1-r3 (on gentoo) and gcc
hardened 4.5.1-r1 with 1.4 patchset which also uses pie (that one
should fix problems with graphite)

not much system changes besides that,

with those it worked fine with 2.6.36 and I couldn't observe any
filesystem corruption



the bad news is: I'm again seeing corruption (!) [on ext4, on the /
(root) partition]:

I was re-emerging/re-installing stuff - pretty trivial stuff actually
(which worked fine in the past): emerging gnome-base programs (gconf,
librsvg, nautilus, gnome-mount, gnome-vfs, gvfs, imagemagick,
xine-lib) and some others: terminal (from xfce), vtwm, rman, vala
(library), xclock, xload, atk, gtk+, vte

during that I noticed some corruption and programs kept failing to
configure/compile, saying that g++ was missing; I re-extracted gcc
(which I previously had made an backup-tarball), that seemed to help
for some time until programs again failed with some corrupted files
from gcc

so I re-emerged gcc (compiling it) and after it had finished the same
error occured I already had written about in an previous email:
the content of /etc/env.d/03opengl got corrupted - but NOT the whole file:

normally it's
# Configuration file for eselect
# This file has been automatically generated.
LDPATH=
OPENGL_PROFILE=
<-- where the path to the graphics-drivers and the opengl profile is written;

in this case of the corruption it only where @@@@@@@@@@@@
symbols


I have no clue how this file could be connected with gcc


===> so the No.1 trigger of this kind of corruption where files are
empty, missing or the content gets corrupted (at least for me) is
compiling software which is part of the system (e.g. emerge -e
system);

the system is Gentoo ~amd64; with binutils 2.20.51.0.12 (afaik this
one has changed from 2.20.51.0.10 to 2.20.51.0.12 from my last
report); gcc 4.5.1 (Gentoo Hardened 4.5.1-r1 p1.4, pie-0.4.5) <--
works fine with 2.6.36 and 2.6.36.1

I'm not sure whether benchmarks would have the same "impact"



the kernel currently running is 2.6.37-rc4 with the [PATCH v5] dm
crypt: scale to multiple CPUs

besides that additional patchsets are applied (I apologize that it's
not only plain vanilla with the dm-crypt patch):
* Prevent kswapd dumping excessive amounts of memory in response to
high-order allocation
* ext4: coordinate data-only flush requests sent by fsync
* vmscan: protect executable page from inactive list scan
* writeback livelock fixes v2

I originally had hoped that the mentioned patch in "ext4: coordinate
data-only flush requests sent by fsync", namely: "md: Call
blk_queue_flush() to establish flush/fua" and additional changes &
fixes to 2.6.37-rc4 would once and for all fix problems but it didn't

I'm also using the the writeback livelock fixes and the dm-crypt scale
to multiple CPUs with 2.6.36 so those generally work fine

so it has be something that changed from 2.6.36->2.6.37 within
dm-crypt or other parts that gets stressed and breaks during usage of
the "[PATCH v5] dm crypt: scale to multiple CPUs" patch

the other included patches surely won't be the cause for that (100%).

Filesystem corruption only seems to occur on the / (root) where the
system resides -

Fortunately I haven't encountered any corruption on my /home partition
which also uses ext4 and during rsync'ing from /home to other data
partitions with ext4 and xfs (I don't want to try to seriously corrupt
any of my data so I played it safe from the beginning and didn't use
anything heavy such as virtualmachines, etc.) - browsing the web,
using firefox & chromium, amarok, etc. worked fine so far

the system is in a pretty "new" state - which means I extracted it
from a tarball out of an liveCD environment with 2.6.35 kernel to the
harddrive - 1st boot was to and 2.6.36 kernel where the 2.6.37-rc4*
kernel was compiled
2nd boot -> current uptime 4 hours

harddrive: Samsung HD203WI (no bad blocks reported by smartmontools,
also no corruptions reported by a run of badblocks (the tool) itself)

harddrive -> cryptsetup -> LVM (volume group: system and swap) -> on
system: ext4

lvm-version is 2.02.74; cryptsetup 1.1.3;
mount options:
noatime,commit=60,barrier=1

currently the system is still running

@Tejun, Milan, Mike:
is there something like the following from reiser4 but for ext4 that
you could use to identify the problem:
--> debugfs.reiser4 -P <device> | bzip2 -c > <device>.bz2

I read about debugfs and catastrophic mode but I have no clue how that
should help

If you need any more info please tell, otherwise I'll wipe that system
and revert back to 2.6.36

I really hope that someone with the big boxes can reproduce this

unfortunately bisecting under these consequences would be impossible
for me (I need to study; waiting hours until the first corruption
occurs ...)

to make things easier:

the first kernel of the 2.6.37-line I compiled was before 2.6.37-rc1
got tagged and was shortly after btrfs got merged:

which should be around:
http://git.eu.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=67577927e8d7a1f4b09b4992df640eadc6aacb36

that should help cut time to narrow possible causes ...

Thanks ! && Regards

Matt

Re: dm-crypt barrier support is effective

[Mike Snitzer]

On Wed, Dec 01 2010 at 11:05am -0500,
Matt <jackdachef@gmail.com> wrote:

> On Mon, Nov 15, 2010 at 12:24 AM, Matt <jackdachef@gmail.com> wrote:
> > On Sun, Nov 14, 2010 at 10:54 PM, Milan Broz <mbroz@redhat.com> wrote:
> >> On 11/14/2010 10:49 PM, Matt wrote:
> >>> only with the dm-crypt scaling patch I could observe the data-corruption
> >>
> >> even with v5 I sent on Friday?
> >>
> >> Are you sure that it is not related to some fs problem in 2.6.37-rc1?
> >>
> >> If it works on 2.6.36 without problems, it is probably problems somewhere
> >> else (flush/fua conversion was trivial here - DM is still doing full flush
> >> and there are no other changes in code IMHO.)
> >>
> >> Milan
> >>
> >
> > Hi Milan,
> >
> > I'm aware of your new v5 patch (which should include several
> > improvements (or potential fixes in my case) over the v3 patch)
> >
> > as I already wrote my schedule unfortunately currently doesn't allow
> > me to test it
> >
> > * in the case of no corruption it would be nice to have 2.6.37-rc* running :)
> >
> > * in the case of data corruption that would mean restoring my system -
> > since it's my production box and right now I don't have a fallback at
> > reach
> > at earliest I could give it a shot at the beginning of December. Then
> > I could also test reiserfs and ext4 as a system partition to rule out
> > that it's
> > a ext4-specific thing (currently I'm running reiserfs on my system-partition).
> >
> > Thanks !
> >
> > Matt
> >
> 
> 
> OK guys,
> 
> I've updated my system to latest glibc 2.12.1-r3 (on gentoo) and gcc
> hardened 4.5.1-r1 with 1.4 patchset which also uses pie (that one
> should fix problems with graphite)
> 
> not much system changes besides that,
> 
> with those it worked fine with 2.6.36 and I couldn't observe any
> filesystem corruption

So dm-crypt cpu scalability v5 with 2.6.36 worked fine.

> the bad news is: I'm again seeing corruption (!) [on ext4, on the /
> (root) partition]:

...

> ===> so the No.1 trigger of this kind of corruption where files are
> empty, missing or the content gets corrupted (at least for me) is
> compiling software which is part of the system (e.g. emerge -e
> system);
> 
> the system is Gentoo ~amd64; with binutils 2.20.51.0.12 (afaik this
> one has changed from 2.20.51.0.10 to 2.20.51.0.12 from my last
> report); gcc 4.5.1 (Gentoo Hardened 4.5.1-r1 p1.4, pie-0.4.5) <--
> works fine with 2.6.36 and 2.6.36.1
> 
> I'm not sure whether benchmarks would have the same "impact"

Seems this emerge is a good test if it reliably enduces the corruption.

> the kernel currently running is 2.6.37-rc4 with the [PATCH v5] dm
> crypt: scale to multiple CPUs
> 
> besides that additional patchsets are applied (I apologize that it's
> not only plain vanilla with the dm-crypt patch):
> * Prevent kswapd dumping excessive amounts of memory in response to
> high-order allocation
> * ext4: coordinate data-only flush requests sent by fsync
> * vmscan: protect executable page from inactive list scan
> * writeback livelock fixes v2

Have you actually experienced any of the issues the above patches are
meant to address?  Seems you're applying patches guessing/hoping
that they'll fix the dm-crypt corruption.

> I originally had hoped that the mentioned patch in "ext4: coordinate
> data-only flush requests sent by fsync", namely: "md: Call
> blk_queue_flush() to establish flush/fua" and additional changes &
> fixes to 2.6.37-rc4 would once and for all fix problems but it didn't

That md patch doesn't help DM at all.  And the ext4 coordination patch
is completely bleeding and actually broken (especially as it relates to
DM -- but that breakage is ony a concern for request-based DM,
e.g. DM-mapth), anyway see: 
https://www.redhat.com/archives/dm-devel/2010-November/msg00185.html

I'm not sure which patches you're using for the ext4 fsync changes but
please don't use them at all.  It is purely an optimization for
extremely heavy fsync workloads and is only getting in the way at this
point.

> I'm also using the the writeback livelock fixes and the dm-crypt scale
> to multiple CPUs with 2.6.36 so those generally work fine
> 
> so it has be something that changed from 2.6.36->2.6.37 within
> dm-crypt or other parts that gets stressed and breaks during usage of
> the "[PATCH v5] dm crypt: scale to multiple CPUs" patch
> 
> the other included patches surely won't be the cause for that (100%).
> 
> Filesystem corruption only seems to occur on the / (root) where the
> system resides -

We need better fault isolation; you've introduced enough change that it
isn't helping zero in on what your particular problem is.  Milan has
tested he latest version of the dm-crypt cpu scalability patch quite a
bit and hasn't seen any corruption -- but clearly the corruption you're
seeing is a real concern and we need to get to the bottom of it.

I'd really appreciate it if you could just use Linus' latest linux-2.6
tree plus Milan's latest patch (technically v6 even though it wasn't
labeled as such): https://patchwork.kernel.org/patch/365542/

Porting that same v6 patch to 2.6.36 would also be nice (to verify you
still don't see any corruption there).

Mike

Re: dm-crypt barrier support is effective

[Matt]

On Wed, Dec 1, 2010 at 5:52 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Wed, Dec 01 2010 at 11:05am -0500,
> Matt <jackdachef@gmail.com> wrote:
>
>> On Mon, Nov 15, 2010 at 12:24 AM, Matt <jackdachef@gmail.com> wrote:
>> > On Sun, Nov 14, 2010 at 10:54 PM, Milan Broz <mbroz@redhat.com> wrote:
>> >> On 11/14/2010 10:49 PM, Matt wrote:
>> >>> only with the dm-crypt scaling patch I could observe the data-corruption
>> >>
>> >> even with v5 I sent on Friday?
>> >>
>> >> Are you sure that it is not related to some fs problem in 2.6.37-rc1?
>> >>
>> >> If it works on 2.6.36 without problems, it is probably problems somewhere
>> >> else (flush/fua conversion was trivial here - DM is still doing full flush
>> >> and there are no other changes in code IMHO.)
>> >>
>> >> Milan
>> >>
>> >
>> > Hi Milan,
>> >
>> > I'm aware of your new v5 patch (which should include several
>> > improvements (or potential fixes in my case) over the v3 patch)
>> >
>> > as I already wrote my schedule unfortunately currently doesn't allow
>> > me to test it
>> >
>> > * in the case of no corruption it would be nice to have 2.6.37-rc* running :)
>> >
>> > * in the case of data corruption that would mean restoring my system -
>> > since it's my production box and right now I don't have a fallback at
>> > reach
>> > at earliest I could give it a shot at the beginning of December. Then
>> > I could also test reiserfs and ext4 as a system partition to rule out
>> > that it's
>> > a ext4-specific thing (currently I'm running reiserfs on my system-partition).
>> >
>> > Thanks !
>> >
>> > Matt
>> >
>>
>>
>> OK guys,
>>
>> I've updated my system to latest glibc 2.12.1-r3 (on gentoo) and gcc
>> hardened 4.5.1-r1 with 1.4 patchset which also uses pie (that one
>> should fix problems with graphite)
>>
>> not much system changes besides that,
>>
>> with those it worked fine with 2.6.36 and I couldn't observe any
>> filesystem corruption
>
> So dm-crypt cpu scalability v5 with 2.6.36 worked fine.
>
>> the bad news is: I'm again seeing corruption (!) [on ext4, on the /
>> (root) partition]:
>
> ...
>
>> ===> so the No.1 trigger of this kind of corruption where files are
>> empty, missing or the content gets corrupted (at least for me) is
>> compiling software which is part of the system (e.g. emerge -e
>> system);
>>
>> the system is Gentoo ~amd64; with binutils 2.20.51.0.12 (afaik this
>> one has changed from 2.20.51.0.10 to 2.20.51.0.12 from my last
>> report); gcc 4.5.1 (Gentoo Hardened 4.5.1-r1 p1.4, pie-0.4.5) <--
>> works fine with 2.6.36 and 2.6.36.1
>>
>> I'm not sure whether benchmarks would have the same "impact"
>
> Seems this emerge is a good test if it reliably enduces the corruption.
>
>> the kernel currently running is 2.6.37-rc4 with the [PATCH v5] dm
>> crypt: scale to multiple CPUs
>>
>> besides that additional patchsets are applied (I apologize that it's
>> not only plain vanilla with the dm-crypt patch):
>> * Prevent kswapd dumping excessive amounts of memory in response to
>> high-order allocation
>> * ext4: coordinate data-only flush requests sent by fsync
>> * vmscan: protect executable page from inactive list scan
>> * writeback livelock fixes v2
>
> Have you actually experienced any of the issues the above patches are
> meant to address?  Seems you're applying patches guessing/hoping
> that they'll fix the dm-crypt corruption.
>
>> I originally had hoped that the mentioned patch in "ext4: coordinate
>> data-only flush requests sent by fsync", namely: "md: Call
>> blk_queue_flush() to establish flush/fua" and additional changes &
>> fixes to 2.6.37-rc4 would once and for all fix problems but it didn't
>
> That md patch doesn't help DM at all.  And the ext4 coordination patch
> is completely bleeding and actually broken (especially as it relates to
> DM -- but that breakage is ony a concern for request-based DM,
> e.g. DM-mapth), anyway see:
> https://www.redhat.com/archives/dm-devel/2010-November/msg00185.html
>
> I'm not sure which patches you're using for the ext4 fsync changes but
> please don't use them at all.  It is purely an optimization for
> extremely heavy fsync workloads and is only getting in the way at this
> point.
>
>> I'm also using the the writeback livelock fixes and the dm-crypt scale
>> to multiple CPUs with 2.6.36 so those generally work fine
>>
>> so it has be something that changed from 2.6.36->2.6.37 within
>> dm-crypt or other parts that gets stressed and breaks during usage of
>> the "[PATCH v5] dm crypt: scale to multiple CPUs" patch
>>
>> the other included patches surely won't be the cause for that (100%).
>>
>> Filesystem corruption only seems to occur on the / (root) where the
>> system resides -
>
> We need better fault isolation; you've introduced enough change that it
> isn't helping zero in on what your particular problem is.  Milan has
> tested he latest version of the dm-crypt cpu scalability patch quite a
> bit and hasn't seen any corruption -- but clearly the corruption you're
> seeing is a real concern and we need to get to the bottom of it.
>
> I'd really appreciate it if you could just use Linus' latest linux-2.6
> tree plus Milan's latest patch (technically v6 even though it wasn't
> labeled as such): https://patchwork.kernel.org/patch/365542/
>
> Porting that same v6 patch to 2.6.36 would also be nice (to verify you
> still don't see any corruption there).
>
> Mike
>

Hi Mike,

those other patches were for other problems I was seeing: e.g.
interactivity/latency problems I was experiencing during heavy
flushing, etc. and some more - so I speculated that those would
improve it


OK, enough of that additional stuff which distracts from this issue -
I'll leave them out for now ...

Thanks for the info !

To console you: I was using v5 from Milan's patch up until now and I
haven't noticed any corruption with it in conjunction with 2.6.36

I modified it according to Milan's mail:

>On 11/15/2010 08:25 AM, Heinz Diehl wrote:
>> On 15.11.2010, Milan Broz wrote:
>
>> drivers/md/dm-crypt.c: In function crypt_ctr':
>> drivers/md/dm-crypt.c:1408: error: WQ_MEM_RECLAIM' undeclared (first use in this function)
>> drivers/md/dm-crypt.c:1408: error: (Each undeclared identifier is reported only once
>> drivers/md/dm-crypt.c:1408: error: for each function it appears in.)

>It should be enough to just replace WQ_MEM_RECLAIM to WQ_RESCUER for 2.6.36.
>(that define is new in 2.6.37)

>Milan

http://www.redhat.com/archives/dm-devel/2010-November/msg00099.html

and that worked fine

Thanks for pointing to v6 ! I hadn't noticed that there was a new one :)

Well, so I'll restore my box to a working/productive state and will
try out v6 (I'm pretty confident that it'll work without problems).

After that I'll see what info Tejun and the others need when the next
corruption might occur with vanilla 2.6.37-rc* and v6 so that there's
something to investigate

Thanks & Regards

Matt

Re: dm-crypt barrier support is effective

[Milan Broz]

On 12/01/2010 06:35 PM, Matt wrote:
> Thanks for pointing to v6 ! I hadn't noticed that there was a new one :)
> 
> Well, so I'll restore my box to a working/productive state and will
> try out v6 (I'm pretty confident that it'll work without problems).

It's the same as previous, just with fixed header (to track it properly
in patchwork) , second patch adds some read optimisation, nothing what
should help here.

Anyway, I run several tests on 2.6.37-rc3+ and see no integrity
problems (using xfs,ext3 and ext4 over dmcrypt).

So please try to check which change causes these problems for you,
it can be something completely unrelated to these patches.

(If if anyone know how to trigger some corruption with btrfs/dmcrypt,
let me know I am not able to reproduce it either.)

Milan

Re: dm-crypt barrier support is effective

[Jon Nelson]

On Wed, Dec 1, 2010 at 12:24 PM, Milan Broz <mbroz@redhat.com> wrote:
> On 12/01/2010 06:35 PM, Matt wrote:
>> Thanks for pointing to v6 ! I hadn't noticed that there was a new one :)
>>
>> Well, so I'll restore my box to a working/productive state and will
>> try out v6 (I'm pretty confident that it'll work without problems).
>
> It's the same as previous, just with fixed header (to track it properly
> in patchwork) , second patch adds some read optimisation, nothing what
> should help here.
>
> Anyway, I run several tests on 2.6.37-rc3+ and see no integrity
> problems (using xfs,ext3 and ext4 over dmcrypt).
>
> So please try to check which change causes these problems for you,
> it can be something completely unrelated to these patches.
>
> (If if anyone know how to trigger some corruption with btrfs/dmcrypt,
> let me know I am not able to reproduce it either.)

Perhaps this is useful: for myself, I found that when I started using
2.6.37rc3 that postgresql starting having a *lot* of problems with
corruption. Specifically, I noted zeroed pages, corruption in headers,
all sorts of stuff on /newly created/ tables, especially during index
creation. I had a fairly high hit rate of failure. I backed off to
2.6.34.7 and have *zero* problems (in fact, prior to 2.6.37rc3, I had
never had a corruption issue with postgresql). I ran on 2.6.36 for a
few weeks as well, without issue.

I am using kcrypt with lvm on top of that, and ext4 on top of that.

-- 
Jon

Re: dm-crypt barrier support is effective

[Milan Broz]

On 12/01/2010 08:34 PM, Jon Nelson wrote:
> Perhaps this is useful: for myself, I found that when I started using
> 2.6.37rc3 that postgresql starting having a *lot* of problems with
> corruption. Specifically, I noted zeroed pages, corruption in headers,
> all sorts of stuff on /newly created/ tables, especially during index
> creation. I had a fairly high hit rate of failure. I backed off to
> 2.6.34.7 and have *zero* problems (in fact, prior to 2.6.37rc3, I had
> never had a corruption issue with postgresql). I ran on 2.6.36 for a
> few weeks as well, without issue.
> 
> I am using kcrypt with lvm on top of that, and ext4 on top of that.

With unpatched dmcrypt (IOW with Linus' git)? Then it must be ext4 or
dm-core problem because there were no patches for dm-crypt...

Anyway, thanks for hint how to reproduce it.

Milan

hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Mike Snitzer]

On Wed, Dec 01 2010 at  3:45pm -0500,
Milan Broz <mbroz@redhat.com> wrote:

> 
> On 12/01/2010 08:34 PM, Jon Nelson wrote:
> > Perhaps this is useful: for myself, I found that when I started using
> > 2.6.37rc3 that postgresql starting having a *lot* of problems with
> > corruption. Specifically, I noted zeroed pages, corruption in headers,
> > all sorts of stuff on /newly created/ tables, especially during index
> > creation. I had a fairly high hit rate of failure. I backed off to
> > 2.6.34.7 and have *zero* problems (in fact, prior to 2.6.37rc3, I had
> > never had a corruption issue with postgresql). I ran on 2.6.36 for a
> > few weeks as well, without issue.
> > 
> > I am using kcrypt with lvm on top of that, and ext4 on top of that.
> 
> With unpatched dmcrypt (IOW with Linus' git)? Then it must be ext4 or
> dm-core problem because there were no patches for dm-crypt...

Matt and Jon,

If you'd be up to it: could you try testing your dm-crypt+ext4
corruption reproducers against the following two 2.6.37-rc commits:

1) 1de3e3df917459422cb2aecac440febc8879d410
then
2) bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc

Then, depending on results of no corruption for those commits, bonus
points for testing the same commits but with Andi and Milan's latest
dm-crypt cpu scalability patch applied too:
https://patchwork.kernel.org/patch/365542/

Thanks!
Mike

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Wed, Dec 1, 2010 at 10:23 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Wed, Dec 01 2010 at  3:45pm -0500,
> Milan Broz <mbroz@redhat.com> wrote:
>
>>
>> On 12/01/2010 08:34 PM, Jon Nelson wrote:
>> > Perhaps this is useful: for myself, I found that when I started using
>> > 2.6.37rc3 that postgresql starting having a *lot* of problems with
>> > corruption. Specifically, I noted zeroed pages, corruption in headers,
>> > all sorts of stuff on /newly created/ tables, especially during index
>> > creation. I had a fairly high hit rate of failure. I backed off to
>> > 2.6.34.7 and have *zero* problems (in fact, prior to 2.6.37rc3, I had
>> > never had a corruption issue with postgresql). I ran on 2.6.36 for a
>> > few weeks as well, without issue.
>> >
>> > I am using kcrypt with lvm on top of that, and ext4 on top of that.
>>
>> With unpatched dmcrypt (IOW with Linus' git)? Then it must be ext4 or
>> dm-core problem because there were no patches for dm-crypt...
>
> Matt and Jon,
>
> If you'd be up to it: could you try testing your dm-crypt+ext4
> corruption reproducers against the following two 2.6.37-rc commits:
>
> 1) 1de3e3df917459422cb2aecac440febc8879d410
> then
> 2) bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
>
> Then, depending on results of no corruption for those commits, bonus
> points for testing the same commits but with Andi and Milan's latest
> dm-crypt cpu scalability patch applied too:
> https://patchwork.kernel.org/patch/365542/
>
> Thanks!
> Mike
>

Yeah sure,

I'll have to set up another testing system (on a separate partition /
volume group) for its own so that will take some time,
first tests will be run probably in the weekend,

thanks for those pointers !

I took a look at git-web - you think
5a87b7a5da250c9be6d757758425dfeaf8ed3179 might be relevant, too ?

the others seem rather minor compared to those you posted

Afaik last time I run vanilla 2.6.37-rc* (which was probably around
rc1) I saw no corruption at all but I'll give it a test-run without
the dm-crypt patch anyway

Thanks & Regards

Matt

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Wed, Dec 1, 2010 at 10:23 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Wed, Dec 01 2010 at  3:45pm -0500,
> Milan Broz <mbroz@redhat.com> wrote:
>
>>
>> On 12/01/2010 08:34 PM, Jon Nelson wrote:
>> > Perhaps this is useful: for myself, I found that when I started using
>> > 2.6.37rc3 that postgresql starting having a *lot* of problems with
>> > corruption. Specifically, I noted zeroed pages, corruption in headers,
>> > all sorts of stuff on /newly created/ tables, especially during index
>> > creation. I had a fairly high hit rate of failure. I backed off to
>> > 2.6.34.7 and have *zero* problems (in fact, prior to 2.6.37rc3, I had
>> > never had a corruption issue with postgresql). I ran on 2.6.36 for a
>> > few weeks as well, without issue.
>> >
>> > I am using kcrypt with lvm on top of that, and ext4 on top of that.
>>
>> With unpatched dmcrypt (IOW with Linus' git)? Then it must be ext4 or
>> dm-core problem because there were no patches for dm-crypt...
>
> Matt and Jon,
>
> If you'd be up to it: could you try testing your dm-crypt+ext4
> corruption reproducers against the following two 2.6.37-rc commits:
>
> 1) 1de3e3df917459422cb2aecac440febc8879d410
> then
> 2) bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
>
> Then, depending on results of no corruption for those commits, bonus
> points for testing the same commits but with Andi and Milan's latest
> dm-crypt cpu scalability patch applied too:
> https://patchwork.kernel.org/patch/365542/
>
> Thanks!
> Mike
>

Hi Mike,

it seems like there isn't even much testing to do:

I tested all 3 commits / checkouts by re-compiling gcc which was/is
the 2nd easy way to trigger this "corruption", compiling google's
chromium (v9) and looking at the output/existance of gcc, g++ and
eselect opengl list

so far everything went fine

After that I used the new patch (v6 or pre-v6), before that I had to

replace WQ_MEM_RECLAIM with WQ_RESCUER

and, re-compiled the kernels

shortly after I had booted up the system with the first kernel
(http://git.eu.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5a87b7a5da250c9be6d757758425dfeaf8ed3179)
the output of 'eselect opengl list' did show no opengl backend
selected

so it seems to manifest itself even earlier (ext4: call
mpage_da_submit_io() from mpage_da_map_blocks()) even if only subtly
and over time -
I'm still currently running that kernel and posting from it & having tests run

I'm not sure if it's even a problem with ext4 - I haven't had the time
to test with XFS yet - maybe it's also happening with that so it more
likely would be dm-core, like Milan suspected
(http://marc.info/?l=linux-kernel&m=129123636223477&w=2) :(

@Jon,

you had time to do some tests meanwhile ? what did you find out ?

even though most of the time it's compiling I don't need to do much -
I need the box for work so if my time allows next tests would be next
weekend and I'm back to my other partition

I really do hope that this bugger can be nailed down ASAP - I like the
improvements made in 2.6.37 but without the dm-crypt multi-cpu patch
it's only half the "fun" ;)

Thanks & Regards

Matt

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Mike Snitzer]

On Sat, Dec 04 2010 at  2:18pm -0500,
Matt <jackdachef@gmail.com> wrote:

> On Wed, Dec 1, 2010 at 10:23 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> > Matt and Jon,
> >
> > If you'd be up to it: could you try testing your dm-crypt+ext4
> > corruption reproducers against the following two 2.6.37-rc commits:
> >
> > 1) 1de3e3df917459422cb2aecac440febc8879d410
> > then
> > 2) bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
> >
> > Then, depending on results of no corruption for those commits, bonus
> > points for testing the same commits but with Andi and Milan's latest
> > dm-crypt cpu scalability patch applied too:
> > https://patchwork.kernel.org/patch/365542/
> >
> > Thanks!
> > Mike
> >
> 
> Hi Mike,
> 
> it seems like there isn't even much testing to do:
> 
> I tested all 3 commits / checkouts by re-compiling gcc which was/is
> the 2nd easy way to trigger this "corruption", compiling google's
> chromium (v9) and looking at the output/existance of gcc, g++ and
> eselect opengl list

Can you be a bit more precise about what you're doing to reproduce?
What sequence?  What (if any) builds are going in parallel?  Etc.

> so far everything went fine
> 
> After that I used the new patch (v6 or pre-v6), before that I had to
> 
> replace WQ_MEM_RECLAIM with WQ_RESCUER
> 
> and, re-compiled the kernels
> 
> shortly after I had booted up the system with the first kernel
> (http://git.eu.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5a87b7a5da250c9be6d757758425dfeaf8ed3179)
> the output of 'eselect opengl list' did show no opengl backend
> selected
> 
> so it seems to manifest itself even earlier (ext4: call
> mpage_da_submit_io() from mpage_da_map_blocks()) even if only subtly
> and over time -
> I'm still currently running that kernel and posting from it & having tests run

OK.
 
> I'm not sure if it's even a problem with ext4 - I haven't had the time
> to test with XFS yet - maybe it's also happening with that so it more
> likely would be dm-core, like Milan suspected
> (http://marc.info/?l=linux-kernel&m=129123636223477&w=2) :(

It'd be interesting to try to reproduce with that same kernel but using
XFS.  I'll check with Milan on what he thinks would be the best next
steps.  Ideally we'll be able to reproduce your results to aid in
pinpointing the issue.  I think Milan will be trying to do so shortly
(if he hasn't started already -- using gentoo emerge, etc).

> even though most of the time it's compiling I don't need to do much -
> I need the box for work so if my time allows next tests would be next
> weekend and I'm back to my other partition
> 
> I really do hope that this bugger can be nailed down ASAP - I like the
> improvements made in 2.6.37 but without the dm-crypt multi-cpu patch
> it's only half the "fun" ;)

Sure, we'll need to get to the bottom of this before we can have
confidence sending the dm-crypt cpu scalability patch upstream.

Thanks for your testing,
Mike

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Sat, Dec 4, 2010 at 8:38 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Sat, Dec 04 2010 at  2:18pm -0500,
> Matt <jackdachef@gmail.com> wrote:
>
>> On Wed, Dec 1, 2010 at 10:23 PM, Mike Snitzer <snitzer@redhat.com> wrote:
>> > Matt and Jon,
>> >
>> > If you'd be up to it: could you try testing your dm-crypt+ext4
>> > corruption reproducers against the following two 2.6.37-rc commits:
>> >
>> > 1) 1de3e3df917459422cb2aecac440febc8879d410
>> > then
>> > 2) bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
>> >
>> > Then, depending on results of no corruption for those commits, bonus
>> > points for testing the same commits but with Andi and Milan's latest
>> > dm-crypt cpu scalability patch applied too:
>> > https://patchwork.kernel.org/patch/365542/
>> >
>> > Thanks!
>> > Mike
>> >
>>
>> Hi Mike,
>>
>> it seems like there isn't even much testing to do:
>>
>> I tested all 3 commits / checkouts by re-compiling gcc which was/is
>> the 2nd easy way to trigger this "corruption", compiling google's
>> chromium (v9) and looking at the output/existance of gcc, g++ and
>> eselect opengl list
>
> Can you be a bit more precise about what you're doing to reproduce?
> What sequence?  What (if any) builds are going in parallel?  Etc.
>

sure:

running on a core i7 860,
~amd64, with MAKEOPTS="-j9", -march=native -pipe

parallel-build or other features are not enabled so it's always only
one program serial running/compiling

emerge -1 sys-devel/gcc -v
in the past after that the output of
eselect opengl list

with a very high probability would be corrupted (the descriptions at
the top NOT but the (dynamic) paths to the opengl files) - I'm running
fglrx/catalyst from AMD so (unfortunately) it's a tainted kernel with
a Radeon HD 5850 (I'm currently having problems to get the evergreen
xf86-video-ati driver to run so I use fglrx)
so it would display

eselect opengl list
Available OpenGL implementations:
  [1]   ati
  [2]   xorg-x11

instead of

eselect opengl list
Available OpenGL implementations:
  [1]   ati *
  [2]   xorg-x11

(mark the *)

and

instead of

cat /etc/env.d/03opengl
# Configuration file for eselect
# This file has been automatically generated.
LDPATH="//usr/lib32/opengl/ati/lib://usr/lib64/opengl/ati/lib"
OPENGL_PROFILE="ati"

it would be

cat /etc/env.d/03opengl
# Configuration file for eselect
# This file has been automatically generated.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@

or something similar nonsensical


other things I did/tested for after every 10-50 packages got emerged:

gcc-config -l
<-- check whether output is correct and (still) properly set

eselect opengl list
<-- check whether output is correct and (still) properly set

gcc -v

ldd

g++

(those files or files having to do with them got corrupted with a
higher probability in the past)

also programs such as firefox and chromium, gedit, etc. would have
missing libraries and/or didn't run anymore or would hang (having a Dl
or D in htop - I don't know what Dl means)

the environment I was in was vtwm, twm or sometimes even gnome running
from the root-account (to test stuff)

>> so far everything went fine
>>
>> After that I used the new patch (v6 or pre-v6), before that I had to
>>
>> replace WQ_MEM_RECLAIM with WQ_RESCUER
>>
>> and, re-compiled the kernels
>>
>> shortly after I had booted up the system with the first kernel
>> (http://git.eu.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5a87b7a5da250c9be6d757758425dfeaf8ed3179)
>> the output of 'eselect opengl list' did show no opengl backend
>> selected
>>
>> so it seems to manifest itself even earlier (ext4: call
>> mpage_da_submit_io() from mpage_da_map_blocks()) even if only subtly
>> and over time -
>> I'm still currently running that kernel and posting from it & having tests run
>
> OK.

meanwhile I think I got some interesting news:

after some time of running (around 1 to 1.5 hours) I noticed the
following BUG with ext4:

[ 4421.503477] ------------[ cut here ]------------
[ 4421.503482] kernel BUG at fs/ext4/inode.c:2714!
[ 4421.503484] invalid opcode: 0000 [#1] PREEMPT SMP
[ 4421.503487] last sysfs file:
/sys/devices/pci0000:00/0000:00:1e.0/0000:07:06.0/local_cpus
[ 4421.503489] CPU 5
[ 4421.503490] Modules linked in: iptable_filter ipt_addrtype
xt_iprange xt_conntrack xt_hashlimit xt_string xt_DSCP xt_NFQUEUE
xt_connmark nf_conntrack xt_mark xt_multiport xt_dscp xt_owner
ip_tables x_tables it87 hwmon_vid coretemp hwmon fglrx(P) i2c_i801 wmi
shpchp e1000e libphy e1000 scsi_wait_scan sl811_hcd ohci_hcd ssb
usb_storage ehci_hcd [last unloaded: tg3]
[ 4421.503513]
[ 4421.503516] Pid: 4541, comm: jbd2/dm-2-8 Tainted: P
2.6.36-rc6_1de3e3df917459422cb2aecac440febc8879d410+ #2 FMP55/ipower
G3710
[ 4421.503519] RIP: 0010:[<ffffffff8119cef3>]  [<ffffffff8119cef3>]
ext4_writepage+0x243/0x250
[ 4421.503526] RSP: 0018:ffff8801bc3dfb50  EFLAGS: 00010246
[ 4421.503528] RAX: 800000000002002d RBX: ffffea00047e9190 RCX: 0000000000000030
[ 4421.503530] RDX: 0000000000000040 RSI: ffff8801bc3dfcc0 RDI: ffffea00047e9190
[ 4421.503531] RBP: ffff88015d52c120 R08: ffff88012bffede8 R09: 0000000000000000
[ 4421.503533] R10: 0000000000000001 R11: 0000000000000008 R12: 0000000000001000
[ 4421.503535] R13: ffffea00047e9190 R14: ffff8801bc3dfcc0 R15: ffff8801bc3dfc30
[ 4421.503538] FS:  0000000000000000(0000) GS:ffff880002140000(0000)
knlGS:0000000000000000
[ 4421.503540] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 4421.503542] CR2: 00007f824cff07e0 CR3: 0000000001c04000 CR4: 00000000000006e0
[ 4421.503544] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4421.503546] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 4421.503548] Process jbd2/dm-2-8 (pid: 4541, threadinfo
ffff8801bc3de000, task ffff8801bfe3d040)
[ 4421.503549] Stack:
[ 4421.503551]  ffff88015d52c288 ffff88015d52c288 ffff88015d52c288
0000000000000040
[ 4421.503554] <0> ffffea00047e9190 0000000000000007 ffff8801bc3dfc30
ffffffff810a261d
[ 4421.503558] <0> 000000000000000a ffffffff810a2ab1 0000000000000019
ffff8801bc3dfcc0
[ 4421.503562] Call Trace:
[ 4421.503568]  [<ffffffff810a261d>] ? __writepage+0xd/0x40
[ 4421.503571]  [<ffffffff810a2ab1>] ? write_cache_pages+0x1b1/0x3d0
[ 4421.503574]  [<ffffffff810a2610>] ? __writepage+0x0/0x40
[ 4421.503579]  [<ffffffff811d1129>] ? journal_submit_data_buffers+0x99/0x100
[ 4421.503583]  [<ffffffff811d1671>] ?
jbd2_journal_commit_transaction+0x331/0x1330
[ 4421.503588]  [<ffffffff8172064b>] ? schedule+0x37b/0x8d0
[ 4421.503591]  [<ffffffff817234c8>] ? _raw_spin_lock_irqsave+0x18/0x20
[ 4421.503596]  [<ffffffff810538c3>] ? lock_timer_base.clone.23+0x33/0x70
[ 4421.503599]  [<ffffffff8105396e>] ? try_to_del_timer_sync+0x6e/0x90
[ 4421.503602]  [<ffffffff811d5651>] ? kjournald2+0xb1/0x210
[ 4421.503606]  [<ffffffff81062b80>] ? autoremove_wake_function+0x0/0x30
[ 4421.503609]  [<ffffffff811d55a0>] ? kjournald2+0x0/0x210
[ 4421.503611]  [<ffffffff811d55a0>] ? kjournald2+0x0/0x210
[ 4421.503614]  [<ffffffff81062706>] ? kthread+0x96/0xa0
[ 4421.503619]  [<ffffffff81003394>] ? kernel_thread_helper+0x4/0x10
[ 4421.503622]  [<ffffffff81062670>] ? kthread+0x0/0xa0
[ 4421.503625]  [<ffffffff81003390>] ? kernel_thread_helper+0x0/0x10
[ 4421.503626] Code: ff eb 85 0f 1f 44 00 00 8b 42 70 25 00 0c 00 00
3d 00 04 00 00 74 a4 48 8b 85 78 ff ff ff f6 c4 40 0f 84 6e fe ff ff
eb 92 0f 0b <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 48 83 ec 08 ba 01
00 00
[ 4421.503654] RIP  [<ffffffff8119cef3>] ext4_writepage+0x243/0x250
[ 4421.503658]  RSP <ffff8801bc3dfb50>
[ 4421.503660] ---[ end trace e4015dccdd3e00bb ]---

kernel compiled was from sources checked out at
1de3e3df917459422cb2aecac440febc8879d410

after that I counter-checked logs and couldn't find anything similar
for 5a87b7a5da250c9be6d757758425dfeaf8ed3179 (that checkout/commit
seemingly worked fine besides the
little "anomaly" of the empty/not set /etc/env.d/03opengl - I didn't
observe anything else broken on that one)

seems 1de3e3df917459422cb2aecac440febc8879d410 might be a possible
candidate ... (strangely I didn't observe any corruption with that -
everything seemed to still be there except the BUG entry in dmesg and
the not anymore working system [apps not wanting to close or open,
hardlocks -> pointing to a filesystem issue which I already had
experienced in a similar way with other filesystems in the past] -
maybe bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc leads to corruption, I
don't know ?)

I ASAP saved the output to my /boot partition (reiserfs) and wanted to
send you a mail but firefox didn't work anymore correctly at that time
- so I wanted to close and restart it (at that time I was running
vtwm),
firefox (via gksu) like above mentioned it showed D or Dl from htop
and couldn't be closed - also not via killall -9

chromium and other apps also stopped to work (I couldn't launch them
anymore, also not via root-account/xterm),

as a last resort I wanted to close the (launched via gksu) running
terminal and open up another one via gksu -> terminal (selecting a
user-account)

shortly after that the box hardlocked (even magic SYSRQ key didn't
work anymore, numlock and caps lock also didn't work - I'm using a
usb-keyboard, perhaps a PS/2 keyboard would have helped but the PS/2
port of my system doesn't work that well/at all under linux and
windows [a bios problem]

summary/reference:
activities: regular stuff - such as browsing the web, hearing to mp3
files, streaming webradio, etc.

system activies: emerge sys-devel/gcc -1 && emerge -e system

testing/checking: gcc -v; ldd; g++; gcc-config -l; eselect opengl
list; cat /etc/env.d/03opengl

mount-options: noatime,commit=600 [now]; during previous tests/usage:
noatime,nodiratime,barrier=1,commit=600

i/o scheduler: CFQ (I have deadline set at boot because that one works
more reliable in case something's broken with CFS; after boot has
completed I manually switch to CFQ)

kernel: using the mentioned kernels compiled with genkernel (for
support with lvm and luks/cryptsetup)

compile via: genkernel --makeopts="-j16" --luks --dmraid --lvm all
--kernname=37_rc-checkout-name --no-clean

(sys-kernel/genkernel version is: genkernel-3.4.10.907-r3.ebuild [from
sabayon-overlay]; there shouldn't be much difference to the one in the
portage-tree: 3.4.10.907-r1)

partitioning: cryptsetup (around 30 GiB partition) [e.g. MAIN] ->
pvcreate /dev/mapper/MAIN
vgcreate GENTOO /dev/mapper/MAIN
lvcreate -L9200M -nSWAP GENTOO
lvcreate -l100%FREE -nROOT GENTOO

so cryptsetup -> LVM -> ROOT (20 GiB) and SWAP (around 9200 MiB) -> on
ROOT (ext4)

filesystem: mkfs.ext4 (default settings; other settings didn't make a
change if the corruption/problems occured or not)

system characteristics:

eselect profile list
Available profile symlink targets:
  [1]   default/linux/amd64/10.0
  [2]   default/linux/amd64/10.0/desktop *
  [3]   default/linux/amd64/10.0/desktop/gnome
  [4]   default/linux/amd64/10.0/desktop/kde

gcc version 4.5.1 (Gentoo Hardened 4.5.1-r1 p1.4, pie-0.4.5)

Portage 2.2.0_alpha6 (default/linux/amd64/10.0/desktop, gcc-4.5.1,
glibc-2.12.1-r3, 2.6.36.1_plus_v4_TOI x86_64)
=================================================================
System uname: Linux-2.6.36.1_plus_v4_TOI-x86_64-Intel-R-_Core-TM-_i7_CPU_860_@_2.80GHz-with-gentoo-2.0.1
Timestamp of tree: Thu, 02 Dec 2010 07:15:02 +0000
app-shells/bash:     4.1_p7
dev-java/java-config: 2.1.11-r1
dev-lang/python:     2.6.6-r1, 3.1.2-r4
dev-util/cmake:      2.8.1-r2
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.6.3
sys-apps/sandbox:    2.3-r1
sys-devel/autoconf:  2.13, 2.68
sys-devel/automake:  1.4_p6-r1, 1.5-r1, 1.6.3-r1, 1.7.9-r2, 1.8.5-r4,
1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1, 2.20.51.0.10, 2.20.51.0.11, 2.20.51.0.12,
2.21.51.0.1
sys-devel/gcc:       4.3.5, 4.4.4-r1, 4.5.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.10
sys-devel/make:      3.82
virtual/os-headers:  2.6.35 (sys-kernel/linux-headers)
Repositories: gentoo kist piczu jasiu portage kde kde-sunset
qting-edge desktop-effects java-overlay java-experimental sunrise voip
jokey-overlay virtualbox-overlay vmware suka-overlay hardened-dev
tallica-overlay mozilla geki-overlay
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* @EULA skype-eula PUEL dlj-1.1 sun-bcla-java-vm
googleearth AdobeFlash-10"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe -fno-ident -mno-push-args
-maccumulate-outgoing-args -mno-align-stringops
-minline-stringops-dynamically"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/X11/xorg.conf /etc/ca-certificates.conf
/etc/env.d /etc/env.d/java/ /etc/eselect/postgresql
/etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release
/etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/
/etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d
/etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d
/etc/texmf/updmap.d /etc/texmf/web2c /usr/X11R6/bin/startx"
CXXFLAGS="-O2 -march=native -pipe -fno-ident -mno-push-args
-maccumulate-outgoing-args -mno-align-stringops
-minline-stringops-dynamically"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going"
FEATURES="assume-digests binpkg-logs candy distlocks fixlafiles
fixpackages metadata-transfer news parallel-fetch preserve-libs
protect-owned sandbox sfperms strict unknown-features-warn
unmerge-logs unmerge-orphans userfetch usersandbox"
GENTOO_MIRRORS="ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/
ftp://de-mirror.org/distro/gentoo/
ftp://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/
http://mirror.jamit.de/gentoo/ ftp://mirror.netcologne.de/gentoo/
ftp://mirror.ovh.net/gentoo-distfiles/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,-z,combreloc -Wl,-z,now -Wl,-z,relro
-Wl,--as-needed -Wl,--hash-style=both -Wl,--sort-common
-Wl,--enable-new-dtags -Wl,--relax"
# most of those are covered via the hardened toolchain and gentoo
defaults, setting them explicitly anyway
LINGUAS="de en"
MAKEOPTS="-j9"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times
--compress --force --whole-file --delete --stats --timeout=180
--exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/gentoo/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/kist-overlay
/usr/local/portage/layman/piczu /usr/local/portage/layman/jasiu
/usr/gentoo/overlays /usr/gentoo/overlays/kde-testing
/usr/gentoo/overlays/kde-sunset /usr/gentoo/overlays/qting-edge
/usr/gentoo/overlays/desktop-effects /usr/gentoo/overlays/java
/usr/gentoo/overlays/java-experimental /usr/gentoo/overlays/sunrise
/usr/gentoo/overlays/voip /usr/gentoo/overlays/jokey
/usr/gentoo/overlays/virtualbox /usr/gentoo/overlays/vmware
/usr/gentoo/overlays/suka /usr/gentoo/overlays/hardened-dev
/usr/gentoo/overlays/tallica /usr/gentoo/overlays/mozilla
/usr/gentoo/overlays/gekis-playground"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="X a52 aac aalib acl acpi akonadi alsa amd64 aspell audio avahi
berkdb bluetooth branding bsf bzip2 cairo cdda cddb cdr cjk cleartype
cli consolekit contrast cracklib crypt cups curl cxx dbus
device-mapper djvu dmraid dri dts dvd dvdr dvi ebook eds emboss encode
exif extensions extras faac faad fam fat fax fbcon ffmpeg fftw firefox
flac fontconfig fortran fuse gd gdbm gdu gif gimp glade gmp gnome
gnome-vfs gnutls gpm gsf gstreamer gtk guile hal handbook hardened
hashstyle hfs html iconv id3tag idn imagemagick imlib inotify ipv6
ithreads java javascript jfs jpeg jpeg2k kde kdehiddenvisibility kipi
lame latex lcms ldap libnotify lm_sensors logrotate lzma lzo mad mdadm
mikmod mjpeg mmap mmx mng mode-paranoid modplug modules mp3 mp4 mpeg
mudflap multilib multislot musepack musicbrainz mysql mysqli ncurses
network networking networkmanager nis nls nptl nptlonly nss ntfs ogg
openexr opengl openmp pam pango pcre pdf perl pg-intdatetime phonon
pic pie pkcs11 plasma pmu png policykit postproc postscript ppds pppd
pulseaudio python qimageblitz qscintilla qt3support qt4 quicktime qwt
raster readline reiser4 reiserfs resolvconf samba sasl scanner sdl
semantic-desktop session shout slang smp sndfile sound soup spell
sqlite sse sse2 sse3 sse4 ssl ssse3 startup-notification svg sysfs
t1lib tcl tcpd theora threads thumbnail tiff tk toolbar truetype
twolame unicode urandom usb utils v4l v4l2 vcd video vorbis wav
wavpack webkit wmf x264 xcb xcomposite xfs xft xinerama xml xmp xorg
xulrunner xv xvid xvmc zip zlib" ALSA_CARDS="hda-intel"
ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty
extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul
mulaw multi null plug rate route share shm softvol"
APACHE2_MODULES="alias autoindex auth_basic authn_alias authn_anon
authn_dbm authn_default authn_file authz_default authz_groupfile
authz_host authz_owner authz_user authz_dbm cache dav dav_fs dav_lock
deflate dir disk_cache env expires ext_filter file_cache filter
headers include info mime mime_magic negotiation setenvif speling
status unique_id vhost_alias" APACHE2_MPMS="worker" CAMERAS="canon
casio kodak konica minolta mustek panasonic polaroid ricoh samsung
sonix sonydscf1 sonydscf55 soundvision toshiba" COLLECTD_PLUGINS="df
interface irq load memory rrdtool swap syslog" ELIBC="glibc"
GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt
gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore
rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx"
INPUT_DEVICES="keyboard mouse linuxinput ps2mouse serialmouse evdev"
KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216
lcdm001 mtxorb ncurses text" LINGUAS="de en" LIRC_DEVICES="asusdh"
PHP_TARGETS="php5-2" RUBY_TARGETS="ruby18" USERLAND="GNU"
VIDEO_CARDS="ati vesa vga radeon r600" XTABLES_ADDONS="quota2 psd
pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy
condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude
chaos account"
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL,
PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS,
PORTAGE_RSYNC_EXTRA_OPTS

hardened toolchain and use-flags are unmasked via:

cat /etc/portage/profile/package.use.mask
sys-devel/gcc -hardened
sys-libs/glibc -hardened

and
USE="hardened pic pie multislot"
(besides that other flags [non-hardened related] are set in USE but
those should be already covered by emerge --info above)

>
>> I'm not sure if it's even a problem with ext4 - I haven't had the time
>> to test with XFS yet - maybe it's also happening with that so it more
>> likely would be dm-core, like Milan suspected
>> (http://marc.info/?l=linux-kernel&m=129123636223477&w=2) :(
>
> It'd be interesting to try to reproduce with that same kernel but using
> XFS.  I'll check with Milan on what he thinks would be the best next
> steps.  Ideally we'll be able to reproduce your results to aid in
> pinpointing the issue.  I think Milan will be trying to do so shortly
> (if he hasn't started already -- using gentoo emerge, etc).
>
>> even though most of the time it's compiling I don't need to do much -
>> I need the box for work so if my time allows next tests would be next
>> weekend and I'm back to my other partition
>>
>> I really do hope that this bugger can be nailed down ASAP - I like the
>> improvements made in 2.6.37 but without the dm-crypt multi-cpu patch
>> it's only half the "fun" ;)
>
> Sure, we'll need to get to the bottom of this before we can have
> confidence sending the dm-crypt cpu scalability patch upstream.
>
> Thanks for your testing,
> Mike
>

Sure, you're welcome - I'm glad that I can help

Below attached/posted you'll find the full output of dmesg for that
session and my kernel-config (attaching file strangely doesn't seem to
work right now)

So that should be enough information to (hopefully) reproduce it or at
least fix underlying causes to prevent it in the future

Thanks & Regards

Matt









[    0.000000] Linux version
2.6.36-rc6_1de3e3df917459422cb2aecac440febc8879d410+ (root@lupus) (gcc
version 4.5.1 (Gentoo Hardened 4.5.1-r1 p1.4, pie-0.4.5) ) #2 SMP
PREEMPT Sat Dec 4 18:35:31 CET 2010
[    0.000000] Command line: dolvm root=/dev/ram0 init=/linuxrc
ramdisk=8192 crypt_root=/dev/sdd6 real_root=/dev/mapper/XPS-ROOT
noresume noresume2 udev ro elevator=deadline
snd-hda-intel.enable_msi=1 fbcon=scrollback:256K pax_softmode=1
clocksource=tsc usbcore.autosuspend=1 raid=noautodetect
video=vesafb:mtrr:3,ywrap vga=794 nomodeset
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
[    0.000000]  BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000bf780000 (usable)
[    0.000000]  BIOS-e820: 00000000bf780000 - 00000000bf78e000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bf78e000 - 00000000bf7d0000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf7ed000 - 00000000c0000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
[    0.000000]  BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 00000001c0000000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI present.
[    0.000000] AMI BIOS detected: BIOS may corrupt low RAM, working around it.
[    0.000000] e820 update range: 0000000000000000 - 0000000000010000
(usable) ==> (reserved)
[    0.000000] e820 update range: 0000000000000000 - 0000000000001000
(usable) ==> (reserved)
[    0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x1c0000 max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: uncachable
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-BFFFF uncachable
[    0.000000]   C0000-D3FFF write-protect
[    0.000000]   D4000-DFFFF uncachable
[    0.000000]   E0000-E3FFF write-protect
[    0.000000]   E4000-E7FFF write-through
[    0.000000]   E8000-EBFFF write-protect
[    0.000000]   EC000-EFFFF write-through
[    0.000000]   F0000-FFFFF write-protect
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 1C0000000 mask FC0000000 uncachable
[    0.000000]   1 base 000000000 mask E00000000 write-back
[    0.000000]   2 base 0C0000000 mask FC0000000 uncachable
[    0.000000]   3 disabled
[    0.000000]   4 disabled
[    0.000000]   5 disabled
[    0.000000]   6 disabled
[    0.000000]   7 disabled
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] original variable MTRRs
[    0.000000] reg 0, base: 7GB, range: 1GB, type UC
[    0.000000] reg 1, base: 0GB, range: 8GB, type WB
[    0.000000] reg 2, base: 3GB, range: 1GB, type UC
[    0.000000] total RAM covered: 6144M
[    0.000000] Found optimal setting for mtrr clean up
[    0.000000]  gran_size: 64K 	chunk_size: 64K 	num_reg: 4  	lose cover RAM: 0G
[    0.000000] New variable MTRRs
[    0.000000] reg 0, base: 0GB, range: 2GB, type WB
[    0.000000] reg 1, base: 2GB, range: 1GB, type WB
[    0.000000] reg 2, base: 4GB, range: 2GB, type WB
[    0.000000] reg 3, base: 6GB, range: 1GB, type WB
[    0.000000] e820 update range: 00000000c0000000 - 0000000100000000
(usable) ==> (reserved)
[    0.000000] last_pfn = 0xbf780 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] modified physical RAM map:
[    0.000000]  modified: 0000000000000000 - 0000000000010000 (reserved)
[    0.000000]  modified: 0000000000010000 - 000000000009dc00 (usable)
[    0.000000]  modified: 000000000009dc00 - 00000000000a0000 (reserved)
[    0.000000]  modified: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  modified: 0000000000100000 - 00000000bf780000 (usable)
[    0.000000]  modified: 00000000bf780000 - 00000000bf78e000 (ACPI data)
[    0.000000]  modified: 00000000bf78e000 - 00000000bf7d0000 (ACPI NVS)
[    0.000000]  modified: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
[    0.000000]  modified: 00000000bf7ed000 - 00000000c0000000 (reserved)
[    0.000000]  modified: 00000000fee00000 - 00000000fee01000 (reserved)
[    0.000000]  modified: 00000000ffb00000 - 0000000100000000 (reserved)
[    0.000000]  modified: 0000000100000000 - 00000001c0000000 (usable)
[    0.000000] initial memory mapped : 0 - 20000000
[    0.000000] found SMP MP-table at [ffff8800000ff780] ff780
[    0.000000] init_memory_mapping: 0000000000000000-00000000bf780000
[    0.000000]  0000000000 - 00bf600000 page 2M
[    0.000000]  00bf600000 - 00bf780000 page 4k
[    0.000000] kernel direct mapping tables up to bf780000 @ 16000-1b000
[    0.000000] init_memory_mapping: 0000000100000000-00000001c0000000
[    0.000000]  0100000000 - 01c0000000 page 2M
[    0.000000] kernel direct mapping tables up to 1c0000000 @ 19000-21000
[    0.000000] RAMDISK: 37cd4000 - 37ff0000
[    0.000000] ACPI: RSDP 00000000000f9cf0 00024 (v02 ACPIAM)
[    0.000000] ACPI: XSDT 00000000bf780100 0006C (v01 ACRSYS ACRPRDCT
20100329 MSFT 00000097)
[    0.000000] ACPI: FACP 00000000bf780290 000F4 (v04 ACRSYS FACP1137
20100329 MSFT 00000097)
[    0.000000] ACPI: DSDT 00000000bf7805e0 07E42 (v02  926A1 926A1P15
00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf78e000 00040
[    0.000000] ACPI: APIC 00000000bf780390 0008C (v02 ACRSYS APIC1137
20100329 MSFT 00000097)
[    0.000000] ACPI: MCFG 00000000bf780420 0003C (v01 ACRSYS OEMMCFG
20100329 MSFT 00000097)
[    0.000000] ACPI: SLIC 00000000bf780460 00176 (v01 ACRSYS ACRPRDCT
20100329 MSFT 00000097)
[    0.000000] ACPI: OEMB 00000000bf78e040 00072 (v01 ACRSYS OEMB1137
20100329 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bf78a5e0 00038 (v01 ACRSYS OEMHPET
20100329 MSFT 00000097)
[    0.000000] ACPI: GSCI 00000000bf78e0c0 02024 (v01 ACRSYS GMCHSCI
20100329 MSFT 00000097)
[    0.000000] ACPI: AWMI 00000000bf7900f0 0004E (v01 ACRSYS OEMB1137
20100329 MSFT 00000097)
[    0.000000] ACPI: SSDT 00000000bf792c10 00363 (v01 DpgPmm    CpuPm
00000012 INTL 20051117)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000]  [ffffea0000000000-ffffea00061fffff] PMD ->
[ffff880002600000-ffff8800079fffff] on node 0
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x001c0000
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[3] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x000bf780
[    0.000000]     0: 0x00100000 -> 0x001c0000
[    0.000000] On node 0 totalpages: 1570573
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 3925 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 14280 pages used for memmap
[    0.000000]   DMA32 zone: 765880 pages, LIFO batch:31
[    0.000000]   Normal zone: 10752 pages used for memmap
[    0.000000]   Normal zone: 775680 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x808
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
[    0.000000] ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0xffffffff base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] nr_irqs_gsi: 40
[    0.000000] PM: Registered nosave memory: 000000000009d000 - 000000000009e000
[    0.000000] PM: Registered nosave memory: 000000000009e000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
[    0.000000] PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
[    0.000000] PM: Registered nosave memory: 00000000bf780000 - 00000000bf78e000
[    0.000000] PM: Registered nosave memory: 00000000bf78e000 - 00000000bf7d0000
[    0.000000] PM: Registered nosave memory: 00000000bf7d0000 - 00000000bf7e0000
[    0.000000] PM: Registered nosave memory: 00000000bf7e0000 - 00000000bf7ed000
[    0.000000] PM: Registered nosave memory: 00000000bf7ed000 - 00000000c0000000
[    0.000000] PM: Registered nosave memory: 00000000c0000000 - 00000000fee00000
[    0.000000] PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
[    0.000000] PM: Registered nosave memory: 00000000fee01000 - 00000000ffb00000
[    0.000000] PM: Registered nosave memory: 00000000ffb00000 - 0000000100000000
[    0.000000] Allocating PCI resources starting at c0000000 (gap:
c0000000:3ee00000)
[    0.000000] early_res array is doubled to 64 at [1c000 - 1c7ff]
[    0.000000] setup_percpu: NR_CPUS:16 nr_cpumask_bits:16
nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 27 pages/cpu @ffff880002000000 s81088
r8192 d21312 u262144
[    0.000000] pcpu-alloc: s81088 r8192 d21312 u262144 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.
Total pages: 1545485
[    0.000000] Kernel command line: dolvm root=/dev/ram0 init=/linuxrc
ramdisk=8192 crypt_root=/dev/sdd6 real_root=/dev/mapper/XPS-ROOT
noresume noresume2 udev ro elevator=deadline
snd-hda-intel.enable_msi=1 fbcon=scrollback:256K pax_softmode=1
clocksource=tsc usbcore.autosuspend=1 raid=noautodetect
video=vesafb:mtrr:3,ywrap vga=794 nomodeset
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11,
8388608 bytes)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Calgary: detecting Calgary via BIOS EBDA area
[    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
[    0.000000] Subtract (51 early reservations)
[    0.000000]   #1 [0001000000 - 0001ded3cc]   TEXT DATA BSS
[    0.000000]   #2 [0037cd4000 - 0037ff0000]         RAMDISK
[    0.000000]   #3 [0001dee000 - 0001dee243]             BRK
[    0.000000]   #4 [00000ff790 - 0000100000]   BIOS reserved
[    0.000000]   #5 [00000ff780 - 00000ff790]    MP-table mpf
[    0.000000]   #6 [000009dc00 - 00000fced0]   BIOS reserved
[    0.000000]   #7 [00000fd074 - 00000ff780]   BIOS reserved
[    0.000000]   #8 [00000fced0 - 00000fd074]    MP-table mpc
[    0.000000]   #9 [0000010000 - 0000012000]      TRAMPOLINE
[    0.000000]   #10 [0000012000 - 0000016000]     ACPI WAKEUP
[    0.000000]   #11 [0000016000 - 0000019000]         PGTABLE
[    0.000000]   #12 [0000019000 - 000001c000]         PGTABLE
[    0.000000]   #13 [0001dee280 - 0001def280]         BOOTMEM
[    0.000000]   #14 [0001ded400 - 0001ded880]         BOOTMEM
[    0.000000]   #15 [00025f0000 - 00025f1000]         BOOTMEM
[    0.000000]   #16 [00025f1000 - 00025f2000]         BOOTMEM
[    0.000000]   #17 [0002600000 - 0007a00000]        MEMMAP 0
[    0.000000]   #18 [0001ded880 - 0001dedb00]         BOOTMEM
[    0.000000]   #19 [0001def280 - 0001e17280]         BOOTMEM
[    0.000000]   #20 [0001e17280 - 0001e3f280]         BOOTMEM
[    0.000000]   #21 [0001e40000 - 0001e41000]         BOOTMEM
[    0.000000]   #22 [0001dedb00 - 0001dedb41]         BOOTMEM
[    0.000000]   #23 [0001dedb80 - 0001dedbc3]         BOOTMEM
[    0.000000]   #24 [0001dedc00 - 0001dedea0]         BOOTMEM
[    0.000000]   #25 [0001dedec0 - 0001dedee0]         BOOTMEM
[    0.000000]   #26 [0001dedf00 - 0001dedf20]         BOOTMEM
[    0.000000]   #27 [0001e3f280 - 0001e3f3b5]         BOOTMEM
[    0.000000]   #28 [0001e3f3c0 - 0001e3f4f5]         BOOTMEM
[    0.000000]   #29 [0002000000 - 000201b000]         BOOTMEM
[    0.000000]   #30 [0002040000 - 000205b000]         BOOTMEM
[    0.000000]   #31 [0002080000 - 000209b000]         BOOTMEM
[    0.000000]   #32 [00020c0000 - 00020db000]         BOOTMEM
[    0.000000]   #33 [0002100000 - 000211b000]         BOOTMEM
[    0.000000]   #34 [0002140000 - 000215b000]         BOOTMEM
[    0.000000]   #35 [0002180000 - 000219b000]         BOOTMEM
[    0.000000]   #36 [00021c0000 - 00021db000]         BOOTMEM
[    0.000000]   #37 [0001dedf40 - 0001dedf48]         BOOTMEM
[    0.000000]   #38 [0001dedf80 - 0001dedf88]         BOOTMEM
[    0.000000]   #39 [0001dedfc0 - 0001dedfe0]         BOOTMEM
[    0.000000]   #40 [0001e3f500 - 0001e3f540]         BOOTMEM
[    0.000000]   #41 [0001e3f540 - 0001e3f660]         BOOTMEM
[    0.000000]   #42 [0001e3f680 - 0001e3f6c8]         BOOTMEM
[    0.000000]   #43 [0001e3f700 - 0001e3f748]         BOOTMEM
[    0.000000]   #44 [0001e41000 - 0001e49000]         BOOTMEM
[    0.000000]   #45 [0007a00000 - 0008200000]         BOOTMEM
[    0.000000]   #46 [00021db000 - 00025db000]         BOOTMEM
[    0.000000]   #47 [0008200000 - 000c200000]         BOOTMEM
[    0.000000]   #48 [0001e49000 - 0001e69000]         BOOTMEM
[    0.000000]   #49 [0001e69000 - 0001ea9000]         BOOTMEM
[    0.000000]   #50 [000001c800 - 0000024800]         BOOTMEM
[    0.000000] Memory: 6099300k/7340032k available (7320k kernel code,
1057740k absent, 182992k reserved, 5819k data, 548k init)
[    0.000000] Preemptable hierarchical RCU implementation.
[    0.000000] 	RCU-based detection of stalled CPUs is disabled.
[    0.000000] 	Verbose stalled-CPUs detection is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:744
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] ------------------------
[    0.000000] | Locking API testsuite:
[    0.000000] ----------------------------------------------------------------------------
[    0.000000]                                  | spin |wlock |rlock
|mutex | wsem | rsem |
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]                      A-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]                  A-B-B-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]              A-B-B-C-C-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]              A-B-C-A-B-C deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]          A-B-B-C-C-D-D-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]          A-B-C-D-B-D-D-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]          A-B-C-D-B-C-D-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]                     double unlock:  ok  |  ok  |failed|
 ok  |failed|failed|
[    0.000000]                   initialize
held:failed|failed|failed|failed|failed|failed|
[    0.000000]                  bad unlock order:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]               recursive read-lock:             |  ok  |
            |failed|
[    0.000000]            recursive read-lock #2:             |  ok  |
            |failed|
[    0.000000]             mixed read-write-lock:             |failed|
            |failed|
[    0.000000]             mixed write-read-lock:             |failed|
            |failed|
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]      hard-irqs-on + irq-safe-A/12:failed|failed|  ok  |
[    0.000000]      soft-irqs-on + irq-safe-A/12:failed|failed|  ok  |
[    0.000000]      hard-irqs-on + irq-safe-A/21:failed|failed|  ok  |
[    0.000000]      soft-irqs-on + irq-safe-A/21:failed|failed|  ok  |
[    0.000000]        sirq-safe-A => hirqs-on/12:failed|failed|  ok  |
[    0.000000]        sirq-safe-A => hirqs-on/21:failed|failed|  ok  |
[    0.000000]          hard-safe-A + irqs-on/12:failed|failed|  ok  |
[    0.000000]          soft-safe-A + irqs-on/12:failed|failed|  ok  |
[    0.000000]          hard-safe-A + irqs-on/21:failed|failed|  ok  |
[    0.000000]          soft-safe-A + irqs-on/21:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/123:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/123:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/132:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/132:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/213:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/213:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/231:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/231:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/312:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/312:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/321:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/321:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/123:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/123:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/132:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/132:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/213:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/213:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/231:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/231:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/312:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/312:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/321:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/321:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/123:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/123:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/132:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/132:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/213:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/213:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/231:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/231:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/312:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/312:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/321:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/321:failed|failed|  ok  |
[    0.000000]       hard-irq read-recursion/123:  ok  |
[    0.000000]       soft-irq read-recursion/123:  ok  |
[    0.000000]       hard-irq read-recursion/132:  ok  |
[    0.000000]       soft-irq read-recursion/132:  ok  |
[    0.000000]       hard-irq read-recursion/213:  ok  |
[    0.000000]       soft-irq read-recursion/213:  ok  |
[    0.000000]       hard-irq read-recursion/231:  ok  |
[    0.000000]       soft-irq read-recursion/231:  ok  |
[    0.000000]       hard-irq read-recursion/312:  ok  |
[    0.000000]       soft-irq read-recursion/312:  ok  |
[    0.000000]       hard-irq read-recursion/321:  ok  |
[    0.000000]       soft-irq read-recursion/321:  ok  |
[    0.000000] --------------------------------------------------------
[    0.000000] 142 out of 218 testcases failed, as expected. |
[    0.000000] ----------------------------------------------------
[    0.000000] hpet clockevent registered
[    0.000000] Fast TSC calibration using PIT
[    0.001000] Detected 2793.165 MHz processor.
[    0.000003] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5586.33 BogoMIPS (lpj=2793165)
[    0.000011] pid_max: default: 32768 minimum: 301
[    0.000064] Mount-cache hash table entries: 256
[    0.000160] CPU: Physical Processor ID: 0
[    0.000163] CPU: Processor Core ID: 0
[    0.000169] mce: CPU supports 9 MCE banks
[    0.000182] CPU0: Thermal monitoring enabled (TM1)
[    0.000190] using mwait in idle threads.
[    0.000193] Performance Events: PEBS fmt1+, Nehalem events, Intel PMU driver.
[    0.000202] ... version:                3
[    0.000205] ... bit width:              48
[    0.000207] ... generic registers:      4
[    0.000210] ... value mask:             0000ffffffffffff
[    0.000214] ... max period:             000000007fffffff
[    0.000217] ... fixed-purpose events:   3
[    0.000220] ... event mask:             000000070000000f
[    0.000273] ACPI: Core revision 20100702
[    0.028494] Setting APIC routing to flat
[    0.028833] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.038821] CPU0: Intel(R) Core(TM) i7 CPU         860  @ 2.80GHz stepping 05
[    0.142448] NMI watchdog enabled, takes one hw-pmu counter.
[    0.145357] Booting Node   0, Processors  #1
[    0.236270] NMI watchdog enabled, takes one hw-pmu counter.
[    0.238186]  #2
[    0.329091] NMI watchdog enabled, takes one hw-pmu counter.
[    0.331006]  #3
[    0.421916] NMI watchdog enabled, takes one hw-pmu counter.
[    0.423830]  #4
[    0.514762] NMI watchdog enabled, takes one hw-pmu counter.
[    0.516651]  #5
[    0.607543] NMI watchdog enabled, takes one hw-pmu counter.
[    0.609475]  #6
[    0.700368] NMI watchdog enabled, takes one hw-pmu counter.
[    0.702296]  #7 Ok.
[    0.793193] NMI watchdog enabled, takes one hw-pmu counter.
[    0.794119] Brought up 8 CPUs
[    0.794125] Total of 8 processors activated (44681.50 BogoMIPS).
[    0.797214] xor: automatically using best checksumming function: generic_sse
[    0.802099]    generic_sse: 10440.000 MB/sec
[    0.802102] xor: using function: generic_sse (10440.000 MB/sec)
[    0.802142] NET: Registered protocol family 16
[    0.802311] ACPI FADT declares the system doesn't support PCIe
ASPM, so disable it
[    0.802316] ACPI: bus type pci registered
[    0.802389] dca service started, version 1.12.1
[    0.802429] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem
0xe0000000-0xefffffff] (base 0xe0000000)
[    0.802434] PCI: not using MMCONFIG
[    0.802436] PCI: Using configuration type 1 for base access
[    0.806692] bio: create slab <bio-0> at 0
[    0.823077] raid6: int64x1   2539 MB/s
[    0.840047] raid6: int64x2   3042 MB/s
[    0.857013] raid6: int64x4   2148 MB/s
[    0.873970] raid6: int64x8   2183 MB/s
[    0.890931] raid6: sse2x1    6937 MB/s
[    0.907897] raid6: sse2x2    8089 MB/s
[    0.924870] raid6: sse2x4    9019 MB/s
[    0.924873] raid6: using algorithm sse2x4 (9019 MB/s)
[    0.925950] ACPI: EC: Look up EC in DSDT
[    0.927603] ACPI: Executed 1 blocks of module-level executable AML code
[    0.931520] ACPI: SSDT 00000000bf790140 0244C (v01 DpgPmm  P001Ist
00000011 INTL 20051117)
[    0.931840] ACPI: Dynamic OEM Table Load:
[    0.931844] ACPI: SSDT (null) 0244C (v01 DpgPmm  P001Ist 00000011
INTL 20051117)
[    0.932024] ACPI: SSDT 00000000bf792590 00678 (v01  PmRef  P001Cst
00003001 INTL 20051117)
[    0.932297] ACPI: Dynamic OEM Table Load:
[    0.932301] ACPI: SSDT (null) 00678 (v01  PmRef  P001Cst 00003001
INTL 20051117)
[    0.932369] ACPI: Interpreter enabled
[    0.932372] ACPI: (supports S0 S3 S4 S5)
[    0.932387] ACPI: Using IOAPIC for interrupt routing
[    0.932411] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem
0xe0000000-0xefffffff] (base 0xe0000000)
[    0.934599] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved
in ACPI motherboard resources
[    0.969259] ACPI: No dock devices found.
[    0.969263] PCI: Using host bridge windows from ACPI; if necessary,
use "pci=nocrs" and report a bug
[    0.969460] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.969649] pci_root PNP0A08:00: host bridge window [io  0x0000-0x0cf7]
[    0.969654] pci_root PNP0A08:00: host bridge window [io  0x0d00-0xffff]
[    0.969657] pci_root PNP0A08:00: host bridge window [mem
0x000a0000-0x000bffff]
[    0.969661] pci_root PNP0A08:00: host bridge window [mem
0x000d0000-0x000dffff]
[    0.969665] pci_root PNP0A08:00: host bridge window [mem
0xc0000000-0xdfffffff]
[    0.969669] pci_root PNP0A08:00: host bridge window [mem
0xf0000000-0xfed8ffff]
[    0.969753] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
[    0.969755] pci 0000:00:03.0: PME# disabled
[    0.969974] pci 0000:00:19.0: reg 10: [mem 0xfbcc0000-0xfbcdffff]
[    0.969981] pci 0000:00:19.0: reg 14: [mem 0xfbcfc000-0xfbcfcfff]
[    0.969989] pci 0000:00:19.0: reg 18: [io  0xbc00-0xbc1f]
[    0.970032] pci 0000:00:19.0: PME# supported from D0 D3hot D3cold
[    0.970035] pci 0000:00:19.0: PME# disabled
[    0.970072] pci 0000:00:1a.0: reg 10: [mem 0xfbcfa000-0xfbcfa3ff]
[    0.970138] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold
[    0.970141] pci 0000:00:1a.0: PME# disabled
[    0.970172] pci 0000:00:1b.0: reg 10: [mem 0xfbcf4000-0xfbcf7fff 64bit]
[    0.970220] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    0.970223] pci 0000:00:1b.0: PME# disabled
[    0.970284] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    0.970287] pci 0000:00:1c.0: PME# disabled
[    0.970350] pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
[    0.970353] pci 0000:00:1c.1: PME# disabled
[    0.970415] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold
[    0.970418] pci 0000:00:1c.2: PME# disabled
[    0.970481] pci 0000:00:1c.3: PME# supported from D0 D3hot D3cold
[    0.970484] pci 0000:00:1c.3: PME# disabled
[    0.970547] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold
[    0.970549] pci 0000:00:1c.4: PME# disabled
[    0.970591] pci 0000:00:1d.0: reg 10: [mem 0xfbcf8000-0xfbcf83ff]
[    0.970656] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[    0.970659] pci 0000:00:1d.0: PME# disabled
[    0.970842] pci 0000:00:1f.2: reg 10: [io  0xb880-0xb887]
[    0.970848] pci 0000:00:1f.2: reg 14: [io  0xb800-0xb803]
[    0.970855] pci 0000:00:1f.2: reg 18: [io  0xb480-0xb487]
[    0.970862] pci 0000:00:1f.2: reg 1c: [io  0xb400-0xb403]
[    0.970869] pci 0000:00:1f.2: reg 20: [io  0xb080-0xb09f]
[    0.970875] pci 0000:00:1f.2: reg 24: [mem 0xfbcf2000-0xfbcf27ff]
[    0.970905] pci 0000:00:1f.2: PME# supported from D3hot
[    0.970908] pci 0000:00:1f.2: PME# disabled
[    0.970933] pci 0000:00:1f.3: reg 10: [mem 0xfbcff800-0xfbcff8ff 64bit]
[    0.970952] pci 0000:00:1f.3: reg 20: [io  0x0400-0x041f]
[    0.971017] pci 0000:01:00.0: reg 10: [mem 0xd0000000-0xdfffffff 64bit pref]
[    0.971026] pci 0000:01:00.0: reg 18: [mem 0xfbde0000-0xfbdfffff 64bit]
[    0.971033] pci 0000:01:00.0: reg 20: [io  0xc000-0xc0ff]
[    0.971044] pci 0000:01:00.0: reg 30: [mem 0xfbdc0000-0xfbddffff pref]
[    0.971060] pci 0000:01:00.0: supports D1 D2
[    0.971087] pci 0000:01:00.1: reg 10: [mem 0xfbdbc000-0xfbdbffff 64bit]
[    0.971128] pci 0000:01:00.1: supports D1 D2
[    0.971142] pci 0000:00:03.0: PCI bridge to [bus 01-01]
[    0.971146] pci 0000:00:03.0:   bridge window [io  0xc000-0xcfff]
[    0.971149] pci 0000:00:03.0:   bridge window [mem 0xfbd00000-0xfbdfffff]
[    0.971153] pci 0000:00:03.0:   bridge window [mem
0xd0000000-0xdfffffff 64bit pref]
[    0.971188] pci 0000:00:1c.0: PCI bridge to [bus 02-02]
[    0.971192] pci 0000:00:1c.0:   bridge window [io  0xf000-0x0000] (disabled)
[    0.971196] pci 0000:00:1c.0:   bridge window [mem
0xfff00000-0x000fffff] (disabled)
[    0.971200] pci 0000:00:1c.0:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971332] pci 0000:03:00.0: reg 24: [mem 0xfbefe000-0xfbefffff]
[    0.971373] pci 0000:03:00.0: PME# supported from D3hot
[    0.971377] pci 0000:03:00.0: PME# disabled
[    0.971423] pci 0000:03:00.1: reg 10: [io  0xdc00-0xdc07]
[    0.971436] pci 0000:03:00.1: reg 14: [io  0xd880-0xd883]
[    0.971449] pci 0000:03:00.1: reg 18: [io  0xd800-0xd807]
[    0.971461] pci 0000:03:00.1: reg 1c: [io  0xd480-0xd483]
[    0.971474] pci 0000:03:00.1: reg 20: [io  0xd400-0xd40f]
[    0.971540] pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe
device.  You can enable it with 'pcie_aspm=force'
[    0.971545] pci 0000:00:1c.1: PCI bridge to [bus 03-03]
[    0.971549] pci 0000:00:1c.1:   bridge window [io  0xd000-0xdfff]
[    0.971552] pci 0000:00:1c.1:   bridge window [mem 0xfbe00000-0xfbefffff]
[    0.971557] pci 0000:00:1c.1:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971592] pci 0000:00:1c.2: PCI bridge to [bus 04-04]
[    0.971597] pci 0000:00:1c.2:   bridge window [io  0xf000-0x0000] (disabled)
[    0.971600] pci 0000:00:1c.2:   bridge window [mem
0xfff00000-0x000fffff] (disabled)
[    0.971605] pci 0000:00:1c.2:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971640] pci 0000:00:1c.3: PCI bridge to [bus 05-05]
[    0.971644] pci 0000:00:1c.3:   bridge window [io  0xf000-0x0000] (disabled)
[    0.971647] pci 0000:00:1c.3:   bridge window [mem
0xfff00000-0x000fffff] (disabled)
[    0.971652] pci 0000:00:1c.3:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971687] pci 0000:00:1c.4: PCI bridge to [bus 06-06]
[    0.971692] pci 0000:00:1c.4:   bridge window [io  0xf000-0x0000] (disabled)
[    0.971695] pci 0000:00:1c.4:   bridge window [mem
0xfff00000-0x000fffff] (disabled)
[    0.971700] pci 0000:00:1c.4:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971748] pci 0000:07:06.0: reg 10: [mem 0xfbfff800-0xfbffffff]
[    0.971757] pci 0000:07:06.0: reg 14: [io  0xec00-0xec7f]
[    0.971814] pci 0000:07:06.0: supports D2
[    0.971815] pci 0000:07:06.0: PME# supported from D2 D3hot D3cold
[    0.971819] pci 0000:07:06.0: PME# disabled
[    0.971852] pci 0000:00:1e.0: PCI bridge to [bus 07-07] (subtractive decode)
[    0.971857] pci 0000:00:1e.0:   bridge window [io  0xe000-0xefff]
[    0.971860] pci 0000:00:1e.0:   bridge window [mem 0xfbf00000-0xfbffffff]
[    0.971865] pci 0000:00:1e.0:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971866] pci 0000:00:1e.0:   bridge window [io  0x0000-0x0cf7]
(subtractive decode)
[    0.971868] pci 0000:00:1e.0:   bridge window [io  0x0d00-0xffff]
(subtractive decode)
[    0.971870] pci 0000:00:1e.0:   bridge window [mem
0x000a0000-0x000bffff] (subtractive decode)
[    0.971872] pci 0000:00:1e.0:   bridge window [mem
0x000d0000-0x000dffff] (subtractive decode)
[    0.971873] pci 0000:00:1e.0:   bridge window [mem
0xc0000000-0xdfffffff] (subtractive decode)
[    0.971875] pci 0000:00:1e.0:   bridge window [mem
0xf0000000-0xfed8ffff] (subtractive decode)
[    0.971900] pci_bus 0000:00: on NUMA node 0
[    0.971902] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[    0.972009] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR1E._PRT]
[    0.972041] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR20._PRT]
[    0.972062] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR21._PRT]
[    0.972089] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR22._PRT]
[    0.972110] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR23._PRT]
[    0.972130] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR24._PRT]
[    0.979958] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15)
[    0.980000] ACPI: PCI Interrupt Link [LNKB] (IRQs *5)
[    0.980039] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 *11 12 14 15)
[    0.980080] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 11 12 14 *15)
[    0.980120] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 *14 15)
[    0.980161] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 7 10 11 12
14 15) *0, disabled.
[    0.980203] ACPI: PCI Interrupt Link [LNKG] (IRQs *3 4 6 7 10 11 12 14 15)
[    0.980243] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 6 *7 10 11 12 14 15)
[    0.980281] HEST: Table is not found!
[    0.980427] SCSI subsystem initialized
[    0.980440] libata version 3.00 loaded.
[    0.980485] usbcore: registered new interface driver usbfs
[    0.980524] usbcore: registered new interface driver hub
[    0.980552] usbcore: registered new device driver usb
[    0.980669] Advanced Linux Sound Architecture Driver Version 1.0.23.
[    0.980673] PCI: Using ACPI for IRQ routing
[    0.980676] PCI: pci_cache_line_size set to 64 bytes
[    0.980738] reserve RAM buffer: 000000000009dc00 - 000000000009ffff
[    0.980740] reserve RAM buffer: 00000000bf780000 - 00000000bfffffff
[    0.980843]   alloc irq_desc for 40 on node 0
[    0.980845]   alloc kstat_irqs on node 0
[    0.980850]   alloc irq_desc for 41 on node 0
[    0.980851]   alloc kstat_irqs on node 0
[    0.980854]   alloc irq_desc for 42 on node 0
[    0.980855]   alloc kstat_irqs on node 0
[    0.980858]   alloc irq_desc for 43 on node 0
[    0.980859]   alloc kstat_irqs on node 0
[    0.980861]   alloc irq_desc for 44 on node 0
[    0.980862]   alloc kstat_irqs on node 0
[    0.980865] HPET: 8 timers in total, 5 timers will be used for per-cpu timer
[    0.980874] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 40, 41, 42, 43, 44, 0
[    0.980881] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[    0.983779] hpet: hpet2 irq 40 for MSI
[    0.983880] hpet: hpet3 irq 41 for MSI
[    0.984869] hpet: hpet4 irq 42 for MSI
[    0.985877] hpet: hpet5 irq 43 for MSI
[    0.986910] hpet: hpet6 irq 44 for MSI
[    0.990871] Switching to clocksource tsc
[    0.990938] pnp: PnP ACPI init
[    0.990947] ACPI: bus type pnp registered
[    0.993484] pnp: PnP ACPI: found 16 devices
[    0.993488] ACPI: ACPI bus type pnp unregistered
[    0.993497] system 00:01: [mem 0xfc000000-0xfcffffff] has been reserved
[    0.993501] system 00:01: [mem 0xfd000000-0xfdffffff] has been reserved
[    0.993504] system 00:01: [mem 0xfe000000-0xfebfffff] has been reserved
[    0.993508] system 00:01: [mem 0xfed14000-0xfed19fff] has been reserved
[    0.993515] system 00:08: [io  0x0a00-0x0a0f] has been reserved
[    0.993518] system 00:08: [io  0x0a10-0x0a1f] has been reserved
[    0.993521] system 00:08: [io  0x0a20-0x0a2f] has been reserved
[    0.993525] system 00:08: [io  0x0a30-0x0a3f] has been reserved
[    0.993530] system 00:09: [io  0x04d0-0x04d1] has been reserved
[    0.993534] system 00:09: [io  0x0800-0x087f] has been reserved
[    0.993537] system 00:09: [io  0x0500-0x057f] has been reserved
[    0.993541] system 00:09: [mem 0xfed1c000-0xfed1ffff] has been reserved
[    0.993544] system 00:09: [mem 0xfed20000-0xfed3ffff] has been reserved
[    0.993548] system 00:09: [mem 0xfed40000-0xfed8ffff] has been reserved
[    0.993554] system 00:0c: [mem 0xffc00000-0xffefffff] has been reserved
[    0.993559] system 00:0d: [mem 0xfec00000-0xfec00fff] could not be reserved
[    0.993563] system 00:0d: [mem 0xfee00000-0xfee00fff] has been reserved
[    0.993568] system 00:0e: [mem 0xe0000000-0xefffffff] has been reserved
[    0.993575] system 00:0f: [mem 0x00000000-0x0009ffff] could not be reserved
[    0.993579] system 00:0f: [mem 0x000c0000-0x000cffff] has been reserved
[    0.993582] system 00:0f: [mem 0x000e0000-0x000fffff] could not be reserved
[    0.993586] system 00:0f: [mem 0x00100000-0xbfffffff] could not be reserved
[    0.993590] system 00:0f: [mem 0xfed90000-0xffffffff] could not be reserved
[    1.000673] pci 0000:00:1c.0: BAR 8: assigned [mem 0xc0000000-0xc01fffff]
[    1.000679] pci 0000:00:1c.0: BAR 9: assigned [mem
0xc0200000-0xc03fffff 64bit pref]
[    1.000684] pci 0000:00:1c.1: BAR 9: assigned [mem
0xc0400000-0xc05fffff 64bit pref]
[    1.000688] pci 0000:00:1c.2: BAR 8: assigned [mem 0xc0600000-0xc07fffff]
[    1.000692] pci 0000:00:1c.2: BAR 9: assigned [mem
0xc0800000-0xc09fffff 64bit pref]
[    1.000696] pci 0000:00:1c.3: BAR 8: assigned [mem 0xc0a00000-0xc0bfffff]
[    1.000700] pci 0000:00:1c.3: BAR 9: assigned [mem
0xc0c00000-0xc0dfffff 64bit pref]
[    1.000705] pci 0000:00:1c.4: BAR 8: assigned [mem 0xc0e00000-0xc0ffffff]
[    1.000709] pci 0000:00:1c.4: BAR 9: assigned [mem
0xc1000000-0xc11fffff 64bit pref]
[    1.000713] pci 0000:00:1c.0: BAR 7: assigned [io  0x1000-0x1fff]
[    1.000717] pci 0000:00:1c.2: BAR 7: assigned [io  0x2000-0x2fff]
[    1.000720] pci 0000:00:1c.3: BAR 7: assigned [io  0x3000-0x3fff]
[    1.000724] pci 0000:00:1c.4: BAR 7: assigned [io  0x4000-0x4fff]
[    1.000727] pci 0000:00:03.0: PCI bridge to [bus 01-01]
[    1.000731] pci 0000:00:03.0:   bridge window [io  0xc000-0xcfff]
[    1.000736] pci 0000:00:03.0:   bridge window [mem 0xfbd00000-0xfbdfffff]
[    1.000740] pci 0000:00:03.0:   bridge window [mem
0xd0000000-0xdfffffff 64bit pref]
[    1.000746] pci 0000:00:1c.0: PCI bridge to [bus 02-02]
[    1.000750] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
[    1.000756] pci 0000:00:1c.0:   bridge window [mem 0xc0000000-0xc01fffff]
[    1.000761] pci 0000:00:1c.0:   bridge window [mem
0xc0200000-0xc03fffff 64bit pref]
[    1.000768] pci 0000:00:1c.1: PCI bridge to [bus 03-03]
[    1.000771] pci 0000:00:1c.1:   bridge window [io  0xd000-0xdfff]
[    1.000777] pci 0000:00:1c.1:   bridge window [mem 0xfbe00000-0xfbefffff]
[    1.000782] pci 0000:00:1c.1:   bridge window [mem
0xc0400000-0xc05fffff 64bit pref]
[    1.000789] pci 0000:00:1c.2: PCI bridge to [bus 04-04]
[    1.000793] pci 0000:00:1c.2:   bridge window [io  0x2000-0x2fff]
[    1.000798] pci 0000:00:1c.2:   bridge window [mem 0xc0600000-0xc07fffff]
[    1.000803] pci 0000:00:1c.2:   bridge window [mem
0xc0800000-0xc09fffff 64bit pref]
[    1.000810] pci 0000:00:1c.3: PCI bridge to [bus 05-05]
[    1.000814] pci 0000:00:1c.3:   bridge window [io  0x3000-0x3fff]
[    1.000819] pci 0000:00:1c.3:   bridge window [mem 0xc0a00000-0xc0bfffff]
[    1.000824] pci 0000:00:1c.3:   bridge window [mem
0xc0c00000-0xc0dfffff 64bit pref]
[    1.000831] pci 0000:00:1c.4: PCI bridge to [bus 06-06]
[    1.000835] pci 0000:00:1c.4:   bridge window [io  0x4000-0x4fff]
[    1.000840] pci 0000:00:1c.4:   bridge window [mem 0xc0e00000-0xc0ffffff]
[    1.000845] pci 0000:00:1c.4:   bridge window [mem
0xc1000000-0xc11fffff 64bit pref]
[    1.000852] pci 0000:00:1e.0: PCI bridge to [bus 07-07]
[    1.000856] pci 0000:00:1e.0:   bridge window [io  0xe000-0xefff]
[    1.000861] pci 0000:00:1e.0:   bridge window [mem 0xfbf00000-0xfbffffff]
[    1.000866] pci 0000:00:1e.0:   bridge window [mem pref disabled]
[    1.000877]   alloc irq_desc for 16 on node -1
[    1.000878]   alloc kstat_irqs on node -1
[    1.000881] pci 0000:00:03.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.000886] pci 0000:00:03.0: setting latency timer to 64
[    1.000895] pci 0000:00:1c.0: enabling device (0104 -> 0107)
[    1.000900]   alloc irq_desc for 17 on node -1
[    1.000901]   alloc kstat_irqs on node -1
[    1.000904] pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.000909] pci 0000:00:1c.0: setting latency timer to 64
[    1.000915] pci 0000:00:1c.1: PCI INT B -> GSI 16 (level, low) -> IRQ 16
[    1.000920] pci 0000:00:1c.1: setting latency timer to 64
[    1.000925] pci 0000:00:1c.2: enabling device (0104 -> 0107)
[    1.000929]   alloc irq_desc for 18 on node -1
[    1.000930]   alloc kstat_irqs on node -1
[    1.000933] pci 0000:00:1c.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    1.000937] pci 0000:00:1c.2: setting latency timer to 64
[    1.000943] pci 0000:00:1c.3: enabling device (0104 -> 0107)
[    1.000947]   alloc irq_desc for 19 on node -1
[    1.000948]   alloc kstat_irqs on node -1
[    1.000951] pci 0000:00:1c.3: PCI INT D -> GSI 19 (level, low) -> IRQ 19
[    1.000955] pci 0000:00:1c.3: setting latency timer to 64
[    1.000961] pci 0000:00:1c.4: enabling device (0104 -> 0107)
[    1.000965] pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.000970] pci 0000:00:1c.4: setting latency timer to 64
[    1.000975] pci 0000:00:1e.0: setting latency timer to 64
[    1.000978] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
[    1.000979] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
[    1.000981] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
[    1.000982] pci_bus 0000:00: resource 7 [mem 0x000d0000-0x000dffff]
[    1.000984] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xdfffffff]
[    1.000985] pci_bus 0000:00: resource 9 [mem 0xf0000000-0xfed8ffff]
[    1.000987] pci_bus 0000:01: resource 0 [io  0xc000-0xcfff]
[    1.000988] pci_bus 0000:01: resource 1 [mem 0xfbd00000-0xfbdfffff]
[    1.000990] pci_bus 0000:01: resource 2 [mem 0xd0000000-0xdfffffff
64bit pref]
[    1.000992] pci_bus 0000:02: resource 0 [io  0x1000-0x1fff]
[    1.000993] pci_bus 0000:02: resource 1 [mem 0xc0000000-0xc01fffff]
[    1.000995] pci_bus 0000:02: resource 2 [mem 0xc0200000-0xc03fffff
64bit pref]
[    1.000996] pci_bus 0000:03: resource 0 [io  0xd000-0xdfff]
[    1.000998] pci_bus 0000:03: resource 1 [mem 0xfbe00000-0xfbefffff]
[    1.000999] pci_bus 0000:03: resource 2 [mem 0xc0400000-0xc05fffff
64bit pref]
[    1.001001] pci_bus 0000:04: resource 0 [io  0x2000-0x2fff]
[    1.001002] pci_bus 0000:04: resource 1 [mem 0xc0600000-0xc07fffff]
[    1.001004] pci_bus 0000:04: resource 2 [mem 0xc0800000-0xc09fffff
64bit pref]
[    1.001006] pci_bus 0000:05: resource 0 [io  0x3000-0x3fff]
[    1.001007] pci_bus 0000:05: resource 1 [mem 0xc0a00000-0xc0bfffff]
[    1.001009] pci_bus 0000:05: resource 2 [mem 0xc0c00000-0xc0dfffff
64bit pref]
[    1.001010] pci_bus 0000:06: resource 0 [io  0x4000-0x4fff]
[    1.001012] pci_bus 0000:06: resource 1 [mem 0xc0e00000-0xc0ffffff]
[    1.001013] pci_bus 0000:06: resource 2 [mem 0xc1000000-0xc11fffff
64bit pref]
[    1.001015] pci_bus 0000:07: resource 0 [io  0xe000-0xefff]
[    1.001016] pci_bus 0000:07: resource 1 [mem 0xfbf00000-0xfbffffff]
[    1.001018] pci_bus 0000:07: resource 4 [io  0x0000-0x0cf7]
[    1.001019] pci_bus 0000:07: resource 5 [io  0x0d00-0xffff]
[    1.001021] pci_bus 0000:07: resource 6 [mem 0x000a0000-0x000bffff]
[    1.001022] pci_bus 0000:07: resource 7 [mem 0x000d0000-0x000dffff]
[    1.001024] pci_bus 0000:07: resource 8 [mem 0xc0000000-0xdfffffff]
[    1.001026] pci_bus 0000:07: resource 9 [mem 0xf0000000-0xfed8ffff]
[    1.001066] NET: Registered protocol family 2
[    1.001118] IP route cache hash table entries: 262144 (order: 9,
2097152 bytes)
[    1.001453] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    1.002537] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes)
[    1.002973] TCP: Hash tables configured (established 262144 bind 65536)
[    1.002977] TCP reno registered
[    1.002984] UDP hash table entries: 4096 (order: 6, 393216 bytes)
[    1.003068] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes)
[    1.003227] NET: Registered protocol family 1
[    1.003310] RPC: Registered udp transport module.
[    1.003313] RPC: Registered tcp transport module.
[    1.003315] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.003387] pci 0000:01:00.0: Boot video device
[    1.003398] PCI: CLS 32 bytes, default 64
[    1.013299] Trying to unpack rootfs image as initramfs...
[    1.064930] Freeing initrd memory: 3184k freed
[    1.065267] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    1.065271] Placing 64MB software IO TLB between ffff880008200000 -
ffff88000c200000
[    1.065275] software IO TLB at phys 0x8200000 - 0xc200000
[    1.067231] microcode: CPU0 sig=0x106e5, pf=0x2, revision=0x3
[    1.067238] microcode: CPU1 sig=0x106e5, pf=0x2, revision=0x3
[    1.067246] microcode: CPU2 sig=0x106e5, pf=0x2, revision=0x3
[    1.067253] microcode: CPU3 sig=0x106e5, pf=0x2, revision=0x3
[    1.067261] microcode: CPU4 sig=0x106e5, pf=0x2, revision=0x3
[    1.067269] microcode: CPU5 sig=0x106e5, pf=0x2, revision=0x3
[    1.067275] microcode: CPU6 sig=0x106e5, pf=0x2, revision=0x3
[    1.067282] microcode: CPU7 sig=0x106e5, pf=0x2, revision=0x3
[    1.067316] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    1.067322] Scanning for low memory corruption every 60 seconds
[    1.067414] Intel AES-NI instructions are not detected.
[    1.067417] Intel PCLMULQDQ-NI instructions are not detected.
[    1.067801] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    1.067983] VFS: Disk quotas dquot_6.5.2
[    1.068012] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.068177] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    1.068290] fuse init (API version 7.15)
[    1.068418] JFS: nTxBlock = 8192, nTxLock = 65536
[    1.069560] SGI XFS with ACLs, security attributes, realtime, large
block/inode numbers, no debug enabled
[    1.069758] SGI XFS Quota Management subsystem
[    1.069866] Btrfs loaded
[    1.069871] msgmni has been set to 11918
[    1.070586] alg: No test for fcrypt (fcrypt-generic)
[    1.071793] alg: No test for stdrng (krng)
[    1.071837] async_tx: api initialized (async)
[    1.071878] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[    1.071883] io scheduler noop registered
[    1.071885] io scheduler deadline registered (default)
[    1.071903] io scheduler cfq registered
[    1.073361] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    1.074040] vesafb: framebuffer at 0xd0000000, mapped to
0xffffc90010780000, using 5120k, total 16384k
[    1.074046] vesafb: mode is 1280x1024x16, linelength=2560, pages=5
[    1.074049] vesafb: scrolling: redraw
[    1.074052] vesafb: Truecolor: size=0:5:6:5, shift=0:11:5:0
[    1.077657] Console: switching to colour frame buffer device 160x64
[    1.079024] fb0: VESA VGA frame buffer device
[    1.079074] intel_idle: MWAIT substates: 0x1120
[    1.079076] intel_idle: v0.4 model 0x1E
[    1.079077] intel_idle: lapic_timer_reliable_states 0x2
[    1.079404] input: Power Button as
/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
[    1.079427] ACPI: Power Button [PWRB]
[    1.079485] input: Power Button as
/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
[    1.079504] ACPI: Power Button [PWRF]
[    1.079689] ACPI: acpi_idle yielding to intel_idle
[    1.082403] ERST: Table is not found!
[    1.082660] Linux agpgart interface v0.103
[    1.082779] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180
seconds, margin is 60 seconds).
[    1.082799] Hangcheck: Using getrawmonotonic().
[    1.082812] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    1.084389] brd: module loaded
[    1.084562] Loading iSCSI transport class v2.0-870.
[    1.084815] SCSI Media Changer driver v0.25
[    1.084885] ahci 0000:00:1f.2: version 3.0
[    1.084895] ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
[    1.084934]   alloc irq_desc for 45 on node -1
[    1.084936]   alloc kstat_irqs on node -1
[    1.084943] ahci 0000:00:1f.2: irq 45 for MSI/MSI-X
[    1.084964] ahci: SSS flag set, parallel bus scan disabled
[    1.095774] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 3
Gbps 0x3f impl SATA mode
[    1.095816] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led
clo pio slum part ems sxs apst
[    1.095844] ahci 0000:00:1f.2: setting latency timer to 64
[    1.105801] scsi0 : ahci
[    1.105921] scsi1 : ahci
[    1.106010] scsi2 : ahci
[    1.106098] scsi3 : ahci
[    1.106186] scsi4 : ahci
[    1.106276] scsi5 : ahci
[    1.106411] ata1: SATA max UDMA/133 abar m2048@0xfbcf2000 port
0xfbcf2100 irq 45
[    1.106429] ata2: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.106447] ata3: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.106466] ata4: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.106484] ata5: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.106502] ata6: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.106541] ahci 0000:03:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.116744] ahci 0000:03:00.0: AHCI 0001.0000 32 slots 2 ports 3
Gbps 0x3 impl SATA mode
[    1.116786] ahci 0000:03:00.0: flags: 64bit ncq led clo pmp pio
[    1.116808] ahci 0000:03:00.0: setting latency timer to 64
[    1.116897] scsi6 : ahci
[    1.117005] scsi7 : ahci
[    1.118326] ata7: SATA max UDMA/133 abar m8192@0xfbefe000 port
0xfbefe100 irq 17
[    1.119571] ata8: SATA max UDMA/133 abar m8192@0xfbefe000 port
0xfbefe180 irq 17
[    1.121721] pata_jmicron 0000:03:00.1: enabling device (0000 -> 0001)
[    1.122948] pata_jmicron 0000:03:00.1: PCI INT B -> GSI 18 (level,
low) -> IRQ 18
[    1.124195] pata_jmicron 0000:03:00.1: setting latency timer to 64
[    1.124246] scsi8 : pata_jmicron
[    1.125580] scsi9 : pata_jmicron
[    1.127154] ata9: PATA max UDMA/100 cmd 0xdc00 ctl 0xd880 bmdma 0xd400 irq 18
[    1.128415] ata10: PATA max UDMA/100 cmd 0xd800 ctl 0xd480 bmdma
0xd408 irq 18
[    1.130412] tun: Universal TUN/TAP device driver, 1.6
[    1.131678] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    1.133073] uhci_hcd: USB Universal Host Controller Interface driver
[    1.134435] usbcore: registered new interface driver wusb-cbaf
[    1.135739] usbcore: registered new interface driver libusual
[    1.137101] PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at
0x60,0x64 irq 1,12
[    1.138737] serio: i8042 KBD port at 0x60,0x64 irq 1
[    1.139992] serio: i8042 AUX port at 0x60,0x64 irq 12
[    1.141343] mice: PS/2 mouse device common for all mice
[    1.142820] rtc_cmos 00:03: RTC can wake from S4
[    1.144111] rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
[    1.145389] rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
[    1.146731] lirc_dev: IR Remote Control driver registered, major 251
[    1.147969] Linux video capture interface: v2.00
[    1.149289] md: linear personality registered for level -1
[    1.150550] md: raid0 personality registered for level 0
[    1.151805] md: raid1 personality registered for level 1
[    1.153046] md: raid10 personality registered for level 10
[    1.154291] md: raid6 personality registered for level 6
[    1.155527] md: raid5 personality registered for level 5
[    1.156746] md: raid4 personality registered for level 4
[    1.157965] md: multipath personality registered for level -4
[    1.159214] device-mapper: uevent: version 1.0.3
[    1.160492] device-mapper: ioctl: 4.18.0-ioctl (2010-06-29)
initialised: dm-devel@redhat.com
[    1.161727] device-mapper: multipath: version 1.1.1 loaded
[    1.162898] device-mapper: multipath round-robin: version 1.0.0 loaded
[    1.164101] EDAC MC: Ver: 2.1.0 Dec  4 2010
[    1.166261] cpuidle: using governor ladder
[    1.169101] cpuidle: using governor menu
[    1.170238] ioatdma: Intel(R) QuickData Technology Driver 4.00
[    1.172437] usbcore: registered new interface driver hiddev
[    1.173580] usbcore: registered new interface driver usbhid
[    1.174680] usbhid: USB HID core driver
[    1.178355]   alloc irq_desc for 22 on node -1
[    1.178356]   alloc kstat_irqs on node -1
[    1.178361] HDA Intel 0000:00:1b.0: PCI INT A -> GSI 22 (level,
low) -> IRQ 22
[    1.179462]   alloc irq_desc for 46 on node -1
[    1.179463]   alloc kstat_irqs on node -1
[    1.179470] HDA Intel 0000:00:1b.0: irq 46 for MSI/MSI-X
[    1.179487] HDA Intel 0000:00:1b.0: setting latency timer to 64
[    1.193885] hda_codec: ALC888: BIOS auto-probing.
[    1.200893] HDA Intel 0000:01:00.1: PCI INT B -> GSI 17 (level,
low) -> IRQ 17
[    1.202105]   alloc irq_desc for 47 on node -1
[    1.202107]   alloc kstat_irqs on node -1
[    1.202113] HDA Intel 0000:01:00.1: irq 47 for MSI/MSI-X
[    1.202128] HDA Intel 0000:01:00.1: setting latency timer to 64
[    1.206666] ALSA device list:
[    1.207848]   #0: HDA Intel at 0xfbcf4000 irq 46
[    1.209044]   #1: HD-Audio Generic at 0xfbdbc000 irq 47
[    1.210481] TCP cubic registered
[    1.211681] Initializing XFRM netlink socket
[    1.213232] NET: Registered protocol family 10
[    1.214809] lo: Disabled Privacy Extensions
[    1.216238] IPv6 over IPv4 tunneling driver
[    1.217729] sit0: Disabled Privacy Extensions
[    1.219244] NET: Registered protocol family 17
[    1.220445] NET: Registered protocol family 15
[    1.221630] lib80211: common routines for IEEE802.11 drivers
[    1.222801] lib80211_crypt: registered algorithm 'NULL'
[    1.222803] Registering the dns_resolver key type
[    1.226063] registered taskstats version 1
[    1.227575] rtc_cmos 00:03: setting system clock to 2010-12-04
21:56:40 UTC (1291499800)
[    1.410376] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    1.413187] ata1.00: ATA-8: WDC WD1001FALS-00J7B0, 05.00K05, max UDMA/133
[    1.414865] ata1.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    1.417886] ata1.00: configured for UDMA/133
[    1.425361] ata8: SATA link down (SStatus 0 SControl 300)
[    1.427131] ata7: SATA link down (SStatus 0 SControl 300)
[    1.430440] scsi 0:0:0:0: Direct-Access     ATA      WDC
WD1001FALS-0 05.0 PQ: 0 ANSI: 5
[    1.433231] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks:
(1.00 TB/931 GiB)
[    1.433669] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    1.436592] sd 0:0:0:0: [sda] Write Protect is off
[    1.438220] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    1.438248] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    1.458941]  sda: sda1 sda2 < sda5 >
[    1.461929] sd 0:0:0:0: [sda] Attached SCSI disk
[    2.155092] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    2.158494] ata2.00: ATAPI: ATAPI   iHAS324   Y, BL1Y, max UDMA/100
[    2.162223] ata2.00: configured for UDMA/100
[    2.176329] scsi 1:0:0:0: CD-ROM            ATAPI    iHAS324   Y
  BL1Y PQ: 0 ANSI: 5
[    2.183221] sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw
xa/form2 cdda tray
[    2.184982] cdrom: Uniform CD-ROM driver Revision: 3.20
[    2.187139] sr 1:0:0:0: Attached scsi CD-ROM sr0
[    2.187396] sr 1:0:0:0: Attached scsi generic sg1 type 5
[    2.910824] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    2.914641] ata3.00: ATA-8: WDC WD10EADS-22M2B0, 01.00A01, max UDMA/133
[    2.916402] ata3.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    2.920428] ata3.00: configured for UDMA/133
[    2.932909] scsi 2:0:0:0: Direct-Access     ATA      WDC
WD10EADS-22M 01.0 PQ: 0 ANSI: 5
[    2.935428] sd 2:0:0:0: [sdb] 1953525168 512-byte logical blocks:
(1.00 TB/931 GiB)
[    2.935738] sd 2:0:0:0: Attached scsi generic sg2 type 0
[    2.939296] sd 2:0:0:0: [sdb] Write Protect is off
[    2.941169] sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    2.941197] sd 2:0:0:0: [sdb] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    2.978580]  sdb: sdb1 sdb2 < sdb5 >
[    2.981312] sd 2:0:0:0: [sdb] Attached SCSI disk
[    3.657635] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    3.662585] ata4.00: ATAPI: HL-DT-ST DVDRAM GH41N, MN01, max UDMA/100
[    3.668021] ata4.00: configured for UDMA/100
[    3.687830] scsi 3:0:0:0: CD-ROM            HL-DT-ST DVDRAM GH41N
  MN01 PQ: 0 ANSI: 5
[    3.712257] sr1: scsi3-mmc drive: 40x/40x writer dvd-ram cd/rw
xa/form2 cdda tray
[    3.714638] sr 3:0:0:0: Attached scsi CD-ROM sr1
[    3.714995] sr 3:0:0:0: Attached scsi generic sg3 type 5
[    4.438337] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    4.446952] ata5.00: ATA-7: SAMSUNG HD154UI, 1AG01118, max UDMA7
[    4.448960] ata5.00: 2930277168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    4.457703] ata5.00: configured for UDMA/133
[    4.470275] scsi 4:0:0:0: Direct-Access     ATA      SAMSUNG
HD154UI  1AG0 PQ: 0 ANSI: 5
[    4.472883] sd 4:0:0:0: [sdc] 2930277168 512-byte logical blocks:
(1.50 TB/1.36 TiB)
[    4.473014] sd 4:0:0:0: Attached scsi generic sg4 type 0
[    4.476923] sd 4:0:0:0: [sdc] Write Protect is off
[    4.478852] sd 4:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[    4.478888] sd 4:0:0:0: [sdc] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    4.547327]  sdc: sdc1 sdc2 sdc3 sdc4 < sdc5 sdc6 sdc7 sdc8 sdc9 >
[    4.550459] sd 4:0:0:0: [sdc] Attached SCSI disk
[    5.195012] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    5.202976] ata6.00: ATA-8: SAMSUNG HD203WI, 1AN10003, max UDMA/133
[    5.204987] ata6.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    5.213178] ata6.00: configured for UDMA/133
[    5.225054] scsi 5:0:0:0: Direct-Access     ATA      SAMSUNG
HD203WI  1AN1 PQ: 0 ANSI: 5
[    5.227692] sd 5:0:0:0: [sdd] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[    5.227820] sd 5:0:0:0: Attached scsi generic sg5 type 0
[    5.232018] sd 5:0:0:0: [sdd] Write Protect is off
[    5.234096] sd 5:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[    5.234124] sd 5:0:0:0: [sdd] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    5.314664]  sdd: sdd1 sdd2 sdd3 < sdd5 sdd6 sdd7 sdd8 sdd9 sdd10 sdd11 >
[    5.317923] sd 5:0:0:0: [sdd] Attached SCSI disk
[    5.390527] Freeing unused kernel memory: 548k freed
[    5.392723] Write protecting the kernel read-only data: 12288k
[    5.395303] Freeing unused kernel memory: 852k freed
[    5.397766] Freeing unused kernel memory: 1472k freed
[    6.144288] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    6.144293] Warning! ehci_hcd should always be loaded before
uhci_hcd and ohci_hcd, not after
[    6.144297] ehci_hcd: block sizes: qh 104 qtd 96 itd 192 sitd 96
[    6.144415] ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    6.144494] ehci_hcd 0000:00:1a.0: setting latency timer to 64
[    6.144499] ehci_hcd 0000:00:1a.0: EHCI Host Controller
[    6.144521] drivers/usb/core/inode.c: creating file 'devices'
[    6.144526] drivers/usb/core/inode.c: creating file '001'
[    6.144745] ehci_hcd 0000:00:1a.0: new USB bus registered, assigned
bus number 1
[    6.144756] ehci_hcd 0000:00:1a.0: reset hcs_params 0x200002 dbg=2
cc=0 pcc=0 ordered !ppc ports=2
[    6.144762] ehci_hcd 0000:00:1a.0: reset hcc_params 36881 caching
frame 1024 64 bit addr
[    6.144781] ehci_hcd 0000:00:1a.0: support lpm
[    6.144794] ehci_hcd 0000:00:1a.0: debug port 2
[    6.144801] ehci_hcd 0000:00:1a.0: reset command 0080002 (park)=0
ithresh=8 period=1024 Reset HALT
[    6.148700] ehci_hcd 0000:00:1a.0: cache line size of 32 is not supported
[    6.148703] ehci_hcd 0000:00:1a.0: supports USB remote wakeup
[    6.148729] ehci_hcd 0000:00:1a.0: irq 16, io mem 0xfbcfa000
[    6.148735] ehci_hcd 0000:00:1a.0: reset command 0080002 (park)=0
ithresh=8 period=1024 Reset HALT
[    6.152614] ehci_hcd 0000:00:1a.0: init command 0010001 (park)=0
ithresh=1 period=1024 RUN
[    6.155435] ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[    6.155465] usb usb1: default language 0x0409
[    6.155476] usb usb1: udev 1, busnum 1, minor = 0
[    6.155479] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    6.155483] usb usb1: New USB device strings: Mfr=3, Product=2,
SerialNumber=1
[    6.155486] usb usb1: Product: EHCI Host Controller
[    6.155490] usb usb1: Manufacturer: Linux
2.6.36-rc6_1de3e3df917459422cb2aecac440febc8879d410+ ehci_hcd
[    6.155493] usb usb1: SerialNumber: 0000:00:1a.0
[    6.155714] usb usb1: usb_probe_device
[    6.155719] usb usb1: configuration #1 chosen from 1 choice
[    6.155732] usb usb1: adding 1-0:1.0 (config #1, interface 0)
[    6.155935] hub 1-0:1.0: usb_probe_interface
[    6.155939] hub 1-0:1.0: usb_probe_interface - got id
[    6.155943] hub 1-0:1.0: USB hub found
[    6.155950] hub 1-0:1.0: 2 ports detected
[    6.155953] hub 1-0:1.0: standalone hub
[    6.155955] hub 1-0:1.0: no power switching (usb 1.0)
[    6.155958] hub 1-0:1.0: individual port over-current protection
[    6.155961] hub 1-0:1.0: power on to power good time: 20ms
[    6.155968] hub 1-0:1.0: local power source is good
[    6.155971] hub 1-0:1.0: trying to enable port power on non-switchable hub
[    6.156180] drivers/usb/core/inode.c: creating file '001'
[    6.156250]   alloc irq_desc for 23 on node -1
[    6.156253]   alloc kstat_irqs on node -1
[    6.156261] ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[    6.156282] ehci_hcd 0000:00:1d.0: setting latency timer to 64
[    6.156287] ehci_hcd 0000:00:1d.0: EHCI Host Controller
[    6.156299] drivers/usb/core/inode.c: creating file '002'
[    6.156565] ehci_hcd 0000:00:1d.0: new USB bus registered, assigned
bus number 2
[    6.156576] ehci_hcd 0000:00:1d.0: reset hcs_params 0x200002 dbg=2
cc=0 pcc=0 ordered !ppc ports=2
[    6.156581] ehci_hcd 0000:00:1d.0: reset hcc_params 36881 caching
frame 1024 64 bit addr
[    6.156600] ehci_hcd 0000:00:1d.0: support lpm
[    6.156613] ehci_hcd 0000:00:1d.0: debug port 2
[    6.156619] ehci_hcd 0000:00:1d.0: reset command 0080002 (park)=0
ithresh=8 period=1024 Reset HALT
[    6.160517] ehci_hcd 0000:00:1d.0: cache line size of 32 is not supported
[    6.160521] ehci_hcd 0000:00:1d.0: supports USB remote wakeup
[    6.160541] ehci_hcd 0000:00:1d.0: irq 23, io mem 0xfbcf8000
[    6.160548] ehci_hcd 0000:00:1d.0: reset command 0080002 (park)=0
ithresh=8 period=1024 Reset HALT
[    6.164433] ehci_hcd 0000:00:1d.0: init command 0010001 (park)=0
ithresh=1 period=1024 RUN
[    6.167411] ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[    6.167436] usb usb2: default language 0x0409
[    6.167446] usb usb2: udev 1, busnum 2, minor = 128
[    6.167450] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
[    6.167453] usb usb2: New USB device strings: Mfr=3, Product=2,
SerialNumber=1
[    6.167457] usb usb2: Product: EHCI Host Controller
[    6.167460] usb usb2: Manufacturer: Linux
2.6.36-rc6_1de3e3df917459422cb2aecac440febc8879d410+ ehci_hcd
[    6.167464] usb usb2: SerialNumber: 0000:00:1d.0
[    6.167754] usb usb2: usb_probe_device
[    6.167759] usb usb2: configuration #1 chosen from 1 choice
[    6.167770] usb usb2: adding 2-0:1.0 (config #1, interface 0)
[    6.167958] hub 2-0:1.0: usb_probe_interface
[    6.167962] hub 2-0:1.0: usb_probe_interface - got id
[    6.167966] hub 2-0:1.0: USB hub found
[    6.167973] hub 2-0:1.0: 2 ports detected
[    6.167976] hub 2-0:1.0: standalone hub
[    6.167978] hub 2-0:1.0: no power switching (usb 1.0)
[    6.167981] hub 2-0:1.0: individual port over-current protection
[    6.167984] hub 2-0:1.0: power on to power good time: 20ms
[    6.167990] hub 2-0:1.0: local power source is good
[    6.167993] hub 2-0:1.0: trying to enable port power on non-switchable hub
[    6.168195] drivers/usb/core/inode.c: creating file '001'
[    6.200453] Initializing USB Mass Storage driver...
[    6.200652] usbcore: registered new interface driver usb-storage
[    6.200656] USB Mass Storage support registered.
[    6.240820] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    6.240825] ohci_hcd: block sizes: ed 80 td 96
[    6.255195] ehci_hcd 0000:00:1a.0: GetStatus port:1 status 001803 0
 ACK POWER sig=j CSC CONNECT
[    6.255203] hub 1-0:1.0: port 1: status 0501 change 0001
[    6.265495] sl811: driver sl811-hcd, 19 May 2005
[    6.267247] ehci_hcd 0000:00:1d.0: GetStatus port:1 status 001803 0
 ACK POWER sig=j CSC CONNECT
[    6.267254] hub 2-0:1.0: port 1: status 0501 change 0001
[    6.356074] hub 1-0:1.0: state 7 ports 2 chg 0002 evt 0000
[    6.356087] hub 1-0:1.0: port 1, status 0501, change 0000, 480 Mb/s
[    6.367194] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k6-NAPI
[    6.367199] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    6.407201] ehci_hcd 0000:00:1a.0: port 1 high speed
[    6.407210] ehci_hcd 0000:00:1a.0: GetStatus port:1 status 001005 0
 ACK POWER sig=se0 PE CONNECT
[    6.457962] usb 1-1: new high speed USB device using ehci_hcd and address 2
[    6.509149] ehci_hcd 0000:00:1a.0: port 1 high speed
[    6.509158] ehci_hcd 0000:00:1a.0: GetStatus port:1 status 001005 0
 ACK POWER sig=se0 PE CONNECT
[    6.571806] ehci_hcd 0000:00:1a.0: set dev address 2 for port 1
[    6.571813] ehci_hcd 0000:00:1a.0: LPM: no device attached
[    6.572107] usb 1-1: udev 2, busnum 1, minor = 1
[    6.572112] usb 1-1: New USB device found, idVendor=8087, idProduct=0020
[    6.572117] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    6.572558] usb 1-1: usb_probe_device
[    6.572564] usb 1-1: configuration #1 chosen from 1 choice
[    6.572753] usb 1-1: adding 1-1:1.0 (config #1, interface 0)
[    6.573049] hub 1-1:1.0: usb_probe_interface
[    6.573054] hub 1-1:1.0: usb_probe_interface - got id
[    6.573059] hub 1-1:1.0: USB hub found
[    6.573242] hub 1-1:1.0: 6 ports detected
[    6.573246] hub 1-1:1.0: standalone hub
[    6.573249] hub 1-1:1.0: individual port power switching
[    6.573252] hub 1-1:1.0: individual port over-current protection
[    6.573255] hub 1-1:1.0: Single TT
[    6.573258] hub 1-1:1.0: TT requires at most 8 FS bit times (666 ns)
[    6.573261] hub 1-1:1.0: Port indicators are supported
[    6.573264] hub 1-1:1.0: power on to power good time: 100ms
[    6.573548] hub 1-1:1.0: local power source is good
[    6.573554] hub 1-1:1.0: enabling power on all ports
[    6.574729] drivers/usb/core/inode.c: creating file '002'
[    6.574766] hub 2-0:1.0: state 7 ports 2 chg 0002 evt 0000
[    6.574777] hub 2-0:1.0: port 1, status 0501, change 0000, 480 Mb/s
[    6.625949] ehci_hcd 0000:00:1d.0: port 1 high speed
[    6.625957] ehci_hcd 0000:00:1d.0: GetStatus port:1 status 001005 0
 ACK POWER sig=se0 PE CONNECT
[    6.673666] hub 1-1:1.0: port 1: status 0101 change 0001
[    6.673945] hub 1-1:1.0: port 2: status 0101 change 0001
[    6.674306] hub 1-1:1.0: port 4: status 0101 change 0001
[    6.676541] usb 2-1: new high speed USB device using ehci_hcd and address 2
[    6.727648] ehci_hcd 0000:00:1d.0: port 1 high speed
[    6.727657] ehci_hcd 0000:00:1d.0: GetStatus port:1 status 001005 0
 ACK POWER sig=se0 PE CONNECT
[    6.774447] usb 1-1: link qh256-0001/ffff8801bc3ce840 start 1 [1/0 us]
[    6.790467] ehci_hcd 0000:00:1d.0: set dev address 2 for port 1
[    6.790474] ehci_hcd 0000:00:1d.0: LPM: no device attached
[    6.790710] usb 2-1: udev 2, busnum 2, minor = 129
[    6.790714] usb 2-1: New USB device found, idVendor=8087, idProduct=0020
[    6.790717] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    6.791121] usb 2-1: usb_probe_device
[    6.791128] usb 2-1: configuration #1 chosen from 1 choice
[    6.791317] usb 2-1: adding 2-1:1.0 (config #1, interface 0)
[    6.791656] hub 2-1:1.0: usb_probe_interface
[    6.791661] hub 2-1:1.0: usb_probe_interface - got id
[    6.791667] hub 2-1:1.0: USB hub found
[    6.791808] hub 2-1:1.0: 8 ports detected
[    6.791813] hub 2-1:1.0: standalone hub
[    6.791817] hub 2-1:1.0: individual port power switching
[    6.791821] hub 2-1:1.0: individual port over-current protection
[    6.791825] hub 2-1:1.0: Single TT
[    6.791830] hub 2-1:1.0: TT requires at most 8 FS bit times (666 ns)
[    6.791834] hub 2-1:1.0: Port indicators are supported
[    6.791839] hub 2-1:1.0: power on to power good time: 100ms
[    6.792078] hub 2-1:1.0: local power source is good
[    6.792082] hub 2-1:1.0: enabling power on all ports
[    6.793468] drivers/usb/core/inode.c: creating file '002'
[    6.793514] hub 1-1:1.0: state 7 ports 6 chg 0016 evt 0000
[    6.793679] hub 1-1:1.0: port 1, status 0101, change 0000, 12 Mb/s
[    6.804466] hub 1-1:1.0: port 1 not reset yet, waiting 10ms
[    6.866510] usb 1-1.1: new low speed USB device using ehci_hcd and address 3
[    6.878438] hub 1-1:1.0: port 1 not reset yet, waiting 10ms
[    6.893031] hub 2-1:1.0: port 7: status 0101 change 0001
[    6.956053] usb 1-1.1: skipped 1 descriptor after interface
[    6.956059] usb 1-1.1: skipped 1 descriptor after interface
[    6.956673] usb 1-1.1: default language 0x0409
[    6.958673] usb 1-1.1: udev 3, busnum 1, minor = 2
[    6.958677] usb 1-1.1: New USB device found, idVendor=046d, idProduct=c521
[    6.958680] usb 1-1.1: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[    6.958684] usb 1-1.1: Product: USB Receiver
[    6.958687] usb 1-1.1: Manufacturer: Logitech
[    6.959104] usb 1-1.1: usb_probe_device
[    6.959110] usb 1-1.1: configuration #1 chosen from 1 choice
[    6.961093] usb 1-1.1: adding 1-1.1:1.0 (config #1, interface 0)
[    6.961447] usbhid 1-1.1:1.0: usb_probe_interface
[    6.961452] usbhid 1-1.1:1.0: usb_probe_interface - got id
[    6.965526] input: Logitech USB Receiver as
/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.1/1-1.1:1.0/input/input2
[    6.966455] generic-usb 0003:046D:C521.0001: input,hidraw0: USB HID
v1.11 Mouse [Logitech USB Receiver] on usb-0000:00:1a.0-1.1/input0
[    6.966487] usb 1-1.1: adding 1-1.1:1.1 (config #1, interface 1)
[    6.966784] usbhid 1-1.1:1.1: usb_probe_interface
[    6.966789] usbhid 1-1.1:1.1: usb_probe_interface - got id
[    6.973941] input: Logitech USB Receiver as
/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.1/1-1.1:1.1/input/input3
[    6.974013] usb 1-1.1: link qh8-0e01/ffff8801bc3cef40 start 2 [1/2 us]
[    6.974841] usbhid 1-1.1:1.1: looking for a minor, starting at 0
[    6.975665] generic-usb 0003:046D:C521.0002: input,hiddev0,hidraw1:
USB HID v1.11 Device [Logitech USB Receiver] on
usb-0000:00:1a.0-1.1/input1
[    6.976135] drivers/usb/core/inode.c: creating file '003'
[    6.976254] hub 1-1:1.0: port 2, status 0101, change 0000, 12 Mb/s
[    6.987129] hub 1-1:1.0: port 2 not reset yet, waiting 10ms
[    6.993013] usb 2-1: link qh256-0001/ffff8801bf5bdc40 start 1 [1/0 us]
[    7.049198] usb 1-1.2: new high speed USB device using ehci_hcd and address 4
[    7.060126] hub 1-1:1.0: port 2 not reset yet, waiting 10ms
[    7.135040] usb 1-1.2: default language 0x0409
[    7.135800] usb 1-1.2: udev 4, busnum 1, minor = 3
[    7.135805] usb 1-1.2: New USB device found, idVendor=04f9, idProduct=002a
[    7.135809] usb 1-1.2: New USB device strings: Mfr=1, Product=2,
SerialNumber=3
[    7.135813] usb 1-1.2: Product: HL-5240
[    7.135816] usb 1-1.2: Manufacturer: Brother
[    7.135818] usb 1-1.2: SerialNumber: H7J241026
[    7.136251] usb 1-1.2: usb_probe_device
[    7.136257] usb 1-1.2: configuration #1 chosen from 1 choice
[    7.136401] usb 1-1.2: adding 1-1.2:1.0 (config #1, interface 0)
[    7.137100] drivers/usb/core/inode.c: creating file '004'
[    7.137272] hub 1-1:1.0: port 4, status 0101, change 0000, 12 Mb/s
[    7.147977] hub 1-1:1.0: port 4 not reset yet, waiting 10ms
[    7.209924] usb 1-1.4: new low speed USB device using ehci_hcd and address 5
[    7.222846] hub 1-1:1.0: port 4 not reset yet, waiting 10ms
[    7.307325] usb 1-1.4: skipped 1 descriptor after interface
[    7.307331] usb 1-1.4: skipped 1 descriptor after interface
[    7.308100] usb 1-1.4: default language 0x0409
[    7.317605] usb 1-1.4: udev 5, busnum 1, minor = 4
[    7.317610] usb 1-1.4: New USB device found, idVendor=045e, idProduct=00db
[    7.317615] usb 1-1.4: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[    7.317618] usb 1-1.4: Product: Natural® Ergonomic Keyboard 4000
[    7.317622] usb 1-1.4: Manufacturer: Microsoft
[    7.318051] usb 1-1.4: usb_probe_device
[    7.318057] usb 1-1.4: configuration #1 chosen from 1 choice
[    7.318819] usb 1-1.4: adding 1-1.4:1.0 (config #1, interface 0)
[    7.319174] usbhid 1-1.4:1.0: usb_probe_interface
[    7.319179] usbhid 1-1.4:1.0: usb_probe_interface - got id
[    7.327300] input: Microsoft Natural® Ergonomic Keyboard 4000 as
/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.4/1-1.4:1.0/input/input4
[    7.327333] usb 1-1.4: link qh8-0e01/ffff8801bc24b7c0 start 3 [1/2 us]
[    7.327994] microsoft 0003:045E:00DB.0003: input,hidraw2: USB HID
v1.11 Keyboard [Microsoft Natural® Ergonomic Keyboard 4000] on
usb-0000:00:1a.0-1.4/input0
[    7.328028] usb 1-1.4: adding 1-1.4:1.1 (config #1, interface 1)
[    7.328318] usbhid 1-1.4:1.1: usb_probe_interface
[    7.328323] usbhid 1-1.4:1.1: usb_probe_interface - got id
[    7.339687] input: Microsoft Natural® Ergonomic Keyboard 4000 as
/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.4/1-1.4:1.1/input/input5
[    7.339719] usb 1-1.4: link qh8-0e01/ffff8801bc24ba40 start 4 [1/2 us]
[    7.340332] microsoft 0003:045E:00DB.0004: input,hidraw3: USB HID
v1.11 Device [Microsoft Natural® Ergonomic Keyboard 4000] on
usb-0000:00:1a.0-1.4/input1
[    7.340615] drivers/usb/core/inode.c: creating file '005'
[    7.340648] hub 1-1:1.0: state 7 ports 6 chg 0000 evt 0010
[    7.340765] hub 2-1:1.0: state 7 ports 8 chg 0080 evt 0000
[    7.340917] hub 2-1:1.0: port 7, status 0101, change 0000, 12 Mb/s
[    7.351506] hub 2-1:1.0: port 7 not reset yet, waiting 10ms
[    7.413561] usb 2-1.7: new high speed USB device using ehci_hcd and address 3
[    7.427493] hub 2-1:1.0: port 7 not reset yet, waiting 10ms
[    7.513356] usb 2-1.7: skipped 1 descriptor after interface
[    7.514189] usb 2-1.7: default language 0x0409
[    7.519623] usb 2-1.7: udev 3, busnum 2, minor = 130
[    7.519628] usb 2-1.7: New USB device found, idVendor=0bda, idProduct=0182
[    7.519633] usb 2-1.7: New USB device strings: Mfr=1, Product=2,
SerialNumber=3
[    7.519637] usb 2-1.7: Product: USB2.0-CRW
[    7.519639] usb 2-1.7: Manufacturer: Generic
[    7.519642] usb 2-1.7: SerialNumber: 20060413092100000
[    7.520026] usb 2-1.7: usb_probe_device
[    7.520033] usb 2-1.7: configuration #1 chosen from 1 choice
[    7.522459] usb 2-1.7: adding 2-1.7:1.0 (config #1, interface 0)
[    7.526585] libusual 2-1.7:1.0: usb_probe_interface
[    7.526597] libusual 2-1.7:1.0: usb_probe_interface - got id
[    7.526620] usb-storage 2-1.7:1.0: usb_probe_interface
[    7.526630] usb-storage 2-1.7:1.0: usb_probe_interface - got id
[    7.526949] scsi10 : usb-storage 2-1.7:1.0
[    7.527738] usb 2-1.7: adding 2-1.7:1.1 (config #1, interface 1)
[    7.528014] usbhid 2-1.7:1.1: usb_probe_interface
[    7.528020] usbhid 2-1.7:1.1: usb_probe_interface - got id
[    7.534756] generic-usb: probe of 0003:0BDA:0182.0005 failed with error -22
[    7.535052] drivers/usb/core/inode.c: creating file '003'
[    7.535284] hub 2-1:1.0: state 7 ports 8 chg 0000 evt fe80
[    8.529494] scsi 10:0:0:0: Direct-Access     Generic- Compact Flash
   1.00 PQ: 0 ANSI: 0 CCS
[    8.532663] scsi 10:0:0:1: Direct-Access     Generic- xD-Picture
   1.00 PQ: 0 ANSI: 0 CCS
[    8.535795] scsi 10:0:0:2: Direct-Access     Generic- SD/MMC
   1.00 PQ: 0 ANSI: 0 CCS
[    8.539166] scsi 10:0:0:3: Direct-Access     Generic- MS/MS-Pro/HG
   1.00 PQ: 0 ANSI: 0 CCS
[    8.542286] scsi 10:0:0:4: Direct-Access     Generic- MicroSD
   1.00 PQ: 0 ANSI: 0 CCS
[    8.543971] sd 10:0:0:0: Attached scsi generic sg6 type 0
[    8.544781] sd 10:0:0:1: Attached scsi generic sg7 type 0
[    8.545644] sd 10:0:0:2: Attached scsi generic sg8 type 0
[    8.546625] sd 10:0:0:3: Attached scsi generic sg9 type 0
[    8.547454] sd 10:0:0:4: Attached scsi generic sg10 type 0
[    8.556428] sd 10:0:0:3: [sdh] Attached SCSI removable disk
[    8.559077] sd 10:0:0:1: [sdf] Attached SCSI removable disk
[    8.561561] sd 10:0:0:0: [sde] Attached SCSI removable disk
[    8.566573] sd 10:0:0:2: [sdg] Attached SCSI removable disk
[    8.569374] sd 10:0:0:4: [sdi] Attached SCSI removable disk
[   61.717812] alg: No test for xts(twofish) (xts(twofish-asm))
[   61.740032] CE: hpet increased min_delta_ns to 7500 nsec
[   64.151414] EXT3-fs (dm-2): error: couldn't mount because of
unsupported optional features (240)
[   64.154045] EXT2-fs (dm-2): error: couldn't mount because of
unsupported optional features (240)
[   64.176266] EXT4-fs (dm-2): mounted filesystem with ordered data
mode. Opts: (null)
[   66.032986] udev[4646]: starting version 164
[   66.221244] e1000e: Intel(R) PRO/1000 Network Driver - 1.2.7-k2
[   66.221246] e1000e: Copyright (c) 1999 - 2010 Intel Corporation.
[   66.221275]   alloc irq_desc for 20 on node -1
[   66.221277]   alloc kstat_irqs on node -1
[   66.221281] e1000e 0000:00:19.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
[   66.221287] e1000e 0000:00:19.0: setting latency timer to 64
[   66.221437]   alloc irq_desc for 48 on node -1
[   66.221438]   alloc kstat_irqs on node -1
[   66.221445] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[   66.243447] usb 1-1.1: link qh8-0e01/ffff8801bc2456c0 start 5 [1/2 us]
[   66.253281] ACPI: WMI: Mapper loaded
[   66.278254] usb 1-1.1: unlink qh8-0e01/ffff8801bc2456c0 start 5 [1/2 us]
[   66.301578] firewire_ohci 0000:07:06.0: PCI INT A -> GSI 19 (level,
low) -> IRQ 19
[   66.354258] firewire_ohci: Added fw-ohci device 0000:07:06.0, OHCI
v1.10, 4 IR + 8 IT contexts, quirks 0x1
[   66.449171] e1000e 0000:00:19.0: eth0: (PCI Express:2.5GB/s:Width
x1) 90:fb:a6:46:2d:ec
[   66.449174] e1000e 0000:00:19.0: eth0: Intel(R) PRO/1000 Network Connection
[   66.449208] e1000e 0000:00:19.0: eth0: MAC: 9, PHY: 9, PBA No: ffffff-0ff
[   66.449249] i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level,
low) -> IRQ 18
[   66.449429] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[   66.675286] fglrx: module license 'Proprietary. (C) 2002 - ATI
Technologies, Starnberg, GERMANY' taints kernel.
[   66.675291] Disabling lock debugging due to kernel taint
[   66.698309] [fglrx] Maximum main memory to use for locked dma
buffers: 5765 MBytes.
[   66.698356] [fglrx]   vendor: 1002 device: 6899 count: 1
[   66.698613] [fglrx] ioport: bar 4, base 0xc000, size: 0x100
[   66.698625] pci 0000:01:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[   66.698628] pci 0000:01:00.0: setting latency timer to 64
[   66.698847] [fglrx] Kernel PAT support is enabled
[   66.698859] [fglrx] module loaded - fglrx 8.79.4 [Oct 26 2010] with 1 minors
[   66.854507] firewire_core: created device fw0: GUID 00016c20013727d2, S400
[   69.559903] EXT4-fs (dm-2): re-mounted. Opts: commit=600
[   69.711466] REISERFS (device sdd1): found reiserfs format "3.6"
with standard journal
[   69.711483] REISERFS (device sdd1): using ordered data mode
[   69.722619] REISERFS (device sdd1): journal params: device sdd1,
size 8192, journal first block 18, max trans len 1024, max batch 900,
max commit age 600, max trans age 600
[   69.722965] REISERFS (device sdd1): checking transaction log (sdd1)
[   69.736959] REISERFS (device sdd1): Using r5 hash to sort names
[   71.122482] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[   71.173254] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[   71.174534] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   72.658979] e1000e: eth0 NIC Link is Up 100 Mbps Full Duplex, Flow
Control: RX
[   72.658986] e1000e 0000:00:19.0: eth0: 10/100 speed: disabling TSO
[   72.660164] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   76.012185] Adding 9420796k swap on /dev/mapper/XPS-SWAP.
Priority:-1 extents:1 across:9420796k
[   82.851744] eth0: no IPv6 routers present
[   93.047830] ehci_hcd 0000:00:1a.0: reused qh ffff8801bc2456c0 schedule
[   93.047838] usb 1-1.1: link qh8-0e01/ffff8801bc2456c0 start 5 [1/2 us]
[   93.973379] coretemp coretemp.0: TjMax is 99 C.
[   93.973668] coretemp coretemp.1: TjMax is 99 C.
[   93.973802] coretemp coretemp.2: TjMax is 99 C.
[   93.974013] coretemp coretemp.3: TjMax is 99 C.
[   94.000505] it87: Found IT8720F chip at 0xa10, revision 8
[   94.000513] it87: VID is disabled (pins used for GPIO)
[   94.000523] it87: Beeping is supported
[  112.781656] ip_tables: (C) 2000-2006 Netfilter Core Team
[  112.828569] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[  114.449277] CPUFREQ: Per core conservative sysfs interface is
deprecated - up_threshold
[  114.466801] CPUFREQ: Per core conservative sysfs interface is
deprecated - down_threshold
[  114.538954] firewire_ohci 0000:07:06.0: PCI INT A disabled
[  114.538959] firewire_ohci: Removed fw-ohci device.
[  114.555531] usb 2-1.7: usb_disable_device nuking non-ep0 URBs
[  114.555540] usb 2-1.7: unregistering interface 2-1.7:1.0
[  114.600307] usb 2-1.7: unregistering interface 2-1.7:1.1
[  114.600867] hub 2-1:1.0: logical disconnect on port 7
[  114.601105] hub 2-1:1.0: state 7 ports 8 chg 0080 evt fe00
[  114.601221] hub 2-1:1.0: port 7, status 0501, change 0000, 480 Mb/s
[  114.601353] usb 2-1.7: USB disconnect, address 3
[  114.601358] usb 2-1.7: unregistering device
[  114.601362] usb 2-1.7: usb_disable_device nuking all URBs
[  114.604484] Enabling EXPERIMENTAL delayed logging feature - use at
your own risk.
[  114.641240] XFS mounting filesystem dm-4
[  114.899543] Ending clean XFS mount for filesystem: dm-4
[  115.719119] hub 2-1:1.0: hub_suspend
[  115.719130] usb 2-1: unlink qh256-0001/ffff8801bf5bdc40 start 1 [1/0 us]
[  115.719368] usb 2-1: usb auto-suspend
[  117.723726] hub 2-0:1.0: hub_suspend
[  117.723737] usb usb2: bus auto-suspend
[  117.723743] ehci_hcd 0000:00:1d.0: suspend root hub
[  124.414736]   alloc irq_desc for 49 on node -1
[  124.414741]   alloc kstat_irqs on node -1
[  124.414754] fglrx_pci 0000:01:00.0: irq 49 for MSI/MSI-X
[  124.415803] [fglrx] Firegl kernel thread PID: 6526
[  124.416124] [fglrx] IRQ 49 Enabled
[  124.746047] [fglrx] Gart USWC size:1280 M.
[  124.746052] [fglrx] Gart cacheable size:508 M.
[  124.746059] [fglrx] Reserved FB block: Shared offset:0, size:1000000
[  124.746063] [fglrx] Reserved FB block: Unshared offset:f8fd000, size:403000
[  124.746067] [fglrx] Reserved FB block: Unshared offset:3fff4000, size:c000
[  188.989504] EXT4-fs (dm-3): mounted filesystem with ordered data
mode. Opts: commit=600
[ 1402.175961] conftest[3904]: segfault at ff845f5c ip
00000000f779fff7 sp 00000000ff845f60 error 6 in
ld-2.12.1.so[f779e000+1c000]
[ 1702.266982] hda-intel: IRQ timing workaround is activated for card
#1. Suggest a bigger bdl_pos_adj.
[ 4308.683872] chrome_sandbox (3170): /proc/3168/oom_adj is
deprecated, please use /proc/3168/oom_score_adj instead.
[ 4421.503477] ------------[ cut here ]------------
[ 4421.503482] kernel BUG at fs/ext4/inode.c:2714!
[ 4421.503484] invalid opcode: 0000 [#1] PREEMPT SMP
[ 4421.503487] last sysfs file:
/sys/devices/pci0000:00/0000:00:1e.0/0000:07:06.0/local_cpus
[ 4421.503489] CPU 5
[ 4421.503490] Modules linked in: iptable_filter ipt_addrtype
xt_iprange xt_conntrack xt_hashlimit xt_string xt_DSCP xt_NFQUEUE
xt_connmark nf_conntrack xt_mark xt_multiport xt_dscp xt_owner
ip_tables x_tables it87 hwmon_vid coretemp hwmon fglrx(P) i2c_i801 wmi
shpchp e1000e libphy e1000 scsi_wait_scan sl811_hcd ohci_hcd ssb
usb_storage ehci_hcd [last unloaded: tg3]
[ 4421.503513]
[ 4421.503516] Pid: 4541, comm: jbd2/dm-2-8 Tainted: P
2.6.36-rc6_1de3e3df917459422cb2aecac440febc8879d410+ #2 FMP55/ipower
G3710
[ 4421.503519] RIP: 0010:[<ffffffff8119cef3>]  [<ffffffff8119cef3>]
ext4_writepage+0x243/0x250
[ 4421.503526] RSP: 0018:ffff8801bc3dfb50  EFLAGS: 00010246
[ 4421.503528] RAX: 800000000002002d RBX: ffffea00047e9190 RCX: 0000000000000030
[ 4421.503530] RDX: 0000000000000040 RSI: ffff8801bc3dfcc0 RDI: ffffea00047e9190
[ 4421.503531] RBP: ffff88015d52c120 R08: ffff88012bffede8 R09: 0000000000000000
[ 4421.503533] R10: 0000000000000001 R11: 0000000000000008 R12: 0000000000001000
[ 4421.503535] R13: ffffea00047e9190 R14: ffff8801bc3dfcc0 R15: ffff8801bc3dfc30
[ 4421.503538] FS:  0000000000000000(0000) GS:ffff880002140000(0000)
knlGS:0000000000000000
[ 4421.503540] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 4421.503542] CR2: 00007f824cff07e0 CR3: 0000000001c04000 CR4: 00000000000006e0
[ 4421.503544] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4421.503546] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 4421.503548] Process jbd2/dm-2-8 (pid: 4541, threadinfo
ffff8801bc3de000, task ffff8801bfe3d040)
[ 4421.503549] Stack:
[ 4421.503551]  ffff88015d52c288 ffff88015d52c288 ffff88015d52c288
0000000000000040
[ 4421.503554] <0> ffffea00047e9190 0000000000000007 ffff8801bc3dfc30
ffffffff810a261d
[ 4421.503558] <0> 000000000000000a ffffffff810a2ab1 0000000000000019
ffff8801bc3dfcc0
[ 4421.503562] Call Trace:
[ 4421.503568]  [<ffffffff810a261d>] ? __writepage+0xd/0x40
[ 4421.503571]  [<ffffffff810a2ab1>] ? write_cache_pages+0x1b1/0x3d0
[ 4421.503574]  [<ffffffff810a2610>] ? __writepage+0x0/0x40
[ 4421.503579]  [<ffffffff811d1129>] ? journal_submit_data_buffers+0x99/0x100
[ 4421.503583]  [<ffffffff811d1671>] ?
jbd2_journal_commit_transaction+0x331/0x1330
[ 4421.503588]  [<ffffffff8172064b>] ? schedule+0x37b/0x8d0
[ 4421.503591]  [<ffffffff817234c8>] ? _raw_spin_lock_irqsave+0x18/0x20
[ 4421.503596]  [<ffffffff810538c3>] ? lock_timer_base.clone.23+0x33/0x70
[ 4421.503599]  [<ffffffff8105396e>] ? try_to_del_timer_sync+0x6e/0x90
[ 4421.503602]  [<ffffffff811d5651>] ? kjournald2+0xb1/0x210
[ 4421.503606]  [<ffffffff81062b80>] ? autoremove_wake_function+0x0/0x30
[ 4421.503609]  [<ffffffff811d55a0>] ? kjournald2+0x0/0x210
[ 4421.503611]  [<ffffffff811d55a0>] ? kjournald2+0x0/0x210
[ 4421.503614]  [<ffffffff81062706>] ? kthread+0x96/0xa0
[ 4421.503619]  [<ffffffff81003394>] ? kernel_thread_helper+0x4/0x10
[ 4421.503622]  [<ffffffff81062670>] ? kthread+0x0/0xa0
[ 4421.503625]  [<ffffffff81003390>] ? kernel_thread_helper+0x0/0x10
[ 4421.503626] Code: ff eb 85 0f 1f 44 00 00 8b 42 70 25 00 0c 00 00
3d 00 04 00 00 74 a4 48 8b 85 78 ff ff ff f6 c4 40 0f 84 6e fe ff ff
eb 92 0f 0b <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 48 83 ec 08 ba 01
00 00
[ 4421.503654] RIP  [<ffffffff8119cef3>] ext4_writepage+0x243/0x250
[ 4421.503658]  RSP <ffff8801bc3dfb50>
[ 4421.503660] ---[ end trace e4015dccdd3e00bb ]---



output after mounting the system-partition from my production-system read-only:

[  297.916518] EXT4-fs (dm-7): INFO: recovery required on readonly filesystem
[  297.916524] EXT4-fs (dm-7): write access will be enabled during recovery
[  299.163548] EXT4-fs (dm-7): orphan cleanup on readonly fs
[  299.163560] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2135728
[  299.163590] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2135724
[  299.163602] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2135723
[  299.163613] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2130123
[  299.163625] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2130118
[  299.163637] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2130112
[  299.163649] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2124353
[  299.163663] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2124352
[  299.163676] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231602
[  299.163719] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231855
[  299.163742] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239521
[  299.163763] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239522
[  299.163784] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239543
[  299.163805] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256039
[  299.163909] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2229219
[  299.163943] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231604
[  299.163968] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239061
[  299.163993] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239066
[  299.164019] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239076
[  299.164043] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255614
[  299.164065] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231130
[  299.164086] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231418
[  299.164107] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239494
[  299.164128] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239501
[  299.164147] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239502
[  299.164168] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255995
[  299.164190] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2124230
[  299.164219] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2124229
[  299.164240] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239530
[  299.164260] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239675
[  299.164281] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239698
[  299.164303] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256311
[  299.164331] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2121629
[  299.164352] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231128
[  299.164373] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256642
[  299.164393] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396379
[  299.164425] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396382
[  299.164444] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2394210
[  299.164504] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2394225
[  299.164558] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2394213
[  299.164578] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231127
[  299.164598] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231129
[  299.164618] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255759
[  299.164638] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255765
[  299.164659] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255766
[  299.164679] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256260
[  299.164757] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 1310848
[  299.164789] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 1310828
[  299.164833] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2119751
[  299.164855] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2119749
[  299.164876] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2121630
[  299.164896] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239661
[  299.164918] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239662
[  299.164938] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256248
[  299.164970] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118252
[  299.164991] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118116
[  299.165011] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239162
[  299.165032] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239210
[  299.165054] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255869
[  299.165075] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118051
[  299.165095] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118050
[  299.165115] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2124234
[  299.165134] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239552
[  299.165153] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239553
[  299.165174] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256120
[  299.165202] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118049
[  299.165221] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118048
[  299.165241] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239340
[  299.165263] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239341
[  299.165288] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239342
[  299.165579] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255886
[  299.165609] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118047
[  299.165640] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118046
[  299.165663] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2141211
[  299.165689] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239013
[  299.165714] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255754
[  299.165741] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2115691
[  299.165767] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2111912
[  299.165792] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239350
[  299.165828] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239382
[  299.165859] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255884
[  299.165890] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2111911
[  299.165910] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2111910
[  299.176933] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144426
[  299.176975] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144430
[  299.177024] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144438
[  299.177055] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144443
[  299.177092] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144446
[  299.177120] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144448
[  299.177150] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 1310823
[  299.192467] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 1318750
[  299.200600] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144930
[  299.200669] EXT4-fs (dm-7): 92 orphan inodes deleted
[  299.200674] EXT4-fs (dm-7): recovery complete
[  300.069586] EXT4-fs (dm-7): mounted filesystem with ordered data
mode. Opts: commit=600,barrier=1


addr2line -e /mnt/gentoo/usr/src/linux-2.6.37-rc_1de3e3df917459422cb2aecac440febc8879d410/vmlinux
-i ffffffff8119cef3
inode.c:0






















#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.36-rc6_1de3e3df917459422cb2aecac440febc8879d410
# Sat Dec  4 18:32:40 2010
#
CONFIG_64BIT=y
# CONFIG_X86_32 is not set
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_HAVE_CPUMASK_OF_CPU_MAP=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ZONE_DMA32=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_HAVE_EARLY_RES=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_HARDIRQS_NO__DO_IRQ=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_X86_64_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_TRAMPOLINE=y
CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-rdi -fcall-saved-rsi
-fcall-saved-rdx -fcall-saved-rcx -fcall-saved-r8 -fcall-saved-r9
-fcall-saved-r10 -fcall-saved-r11"
# CONFIG_KTIME_SCALAR is not set
CONFIG_ARCH_CPU_PROBE_RELEASE=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_CONSTRUCTORS=y

#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_LZO=y
# CONFIG_KERNEL_GZIP is not set
# CONFIG_KERNEL_BZIP2 is not set
CONFIG_KERNEL_LZMA=y
# CONFIG_KERNEL_LZO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
CONFIG_TASKSTATS=y
CONFIG_TASK_DELAY_ACCT=y
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
# CONFIG_AUDIT is not set

#
# RCU Subsystem
#
# CONFIG_TREE_RCU is not set
CONFIG_TREE_PREEMPT_RCU=y
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_FANOUT=64
# CONFIG_RCU_FANOUT_EXACT is not set
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=17
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
# CONFIG_CGROUPS is not set
# CONFIG_SYSFS_DEPRECATED_V2 is not set
CONFIG_RELAY=y
# CONFIG_NAMESPACES is not set
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="/usr/share/v86d/initramfs"
CONFIG_INITRAMFS_ROOT_UID=0
CONFIG_INITRAMFS_ROOT_GID=0
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_LZO=y
# CONFIG_INITRAMFS_COMPRESSION_NONE is not set
# CONFIG_INITRAMFS_COMPRESSION_GZIP is not set
# CONFIG_INITRAMFS_COMPRESSION_BZIP2 is not set
CONFIG_INITRAMFS_COMPRESSION_LZMA=y
# CONFIG_INITRAMFS_COMPRESSION_LZO is not set
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
CONFIG_EMBEDDED=y
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_HAVE_PERF_EVENTS=y

#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# CONFIG_PERF_COUNTERS is not set
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
# CONFIG_COMPAT_BRK is not set
CONFIG_SLAB=y
# CONFIG_SLUB is not set
# CONFIG_SLOB is not set
# CONFIG_PROFILING is not set
CONFIG_HAVE_OPROFILE=y
# CONFIG_KPROBES is not set
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y

#
# GCOV-based kernel profiling
#
# CONFIG_GCOV_KERNEL is not set
# CONFIG_HAVE_GENERIC_DMA_COHERENT is not set
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_BLK_DEV_BSG=y
# CONFIG_BLK_DEV_INTEGRITY is not set
CONFIG_BLOCK_COMPAT=y

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
CONFIG_DEFAULT_DEADLINE=y
# CONFIG_DEFAULT_CFQ is not set
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="deadline"
CONFIG_PADATA=y
# CONFIG_INLINE_SPIN_TRYLOCK is not set
# CONFIG_INLINE_SPIN_TRYLOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK is not set
# CONFIG_INLINE_SPIN_LOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK_IRQ is not set
# CONFIG_INLINE_SPIN_LOCK_IRQSAVE is not set
# CONFIG_INLINE_SPIN_UNLOCK is not set
# CONFIG_INLINE_SPIN_UNLOCK_BH is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQ is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_READ_TRYLOCK is not set
# CONFIG_INLINE_READ_LOCK is not set
# CONFIG_INLINE_READ_LOCK_BH is not set
# CONFIG_INLINE_READ_LOCK_IRQ is not set
# CONFIG_INLINE_READ_LOCK_IRQSAVE is not set
# CONFIG_INLINE_READ_UNLOCK is not set
# CONFIG_INLINE_READ_UNLOCK_BH is not set
# CONFIG_INLINE_READ_UNLOCK_IRQ is not set
# CONFIG_INLINE_READ_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_WRITE_TRYLOCK is not set
# CONFIG_INLINE_WRITE_LOCK is not set
# CONFIG_INLINE_WRITE_LOCK_BH is not set
# CONFIG_INLINE_WRITE_LOCK_IRQ is not set
# CONFIG_INLINE_WRITE_LOCK_IRQSAVE is not set
# CONFIG_INLINE_WRITE_UNLOCK is not set
# CONFIG_INLINE_WRITE_UNLOCK_BH is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQ is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQRESTORE is not set
# CONFIG_MUTEX_SPIN_ON_OWNER is not set
CONFIG_FREEZER=y

#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
CONFIG_SPARSE_IRQ=y
CONFIG_X86_MPPARSE=y
# CONFIG_X86_EXTENDED_PLATFORM is not set
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
# CONFIG_PARAVIRT_GUEST is not set
CONFIG_NO_BOOTMEM=y
# CONFIG_MEMTEST is not set
# CONFIG_MK8 is not set
# CONFIG_MPSC is not set
CONFIG_MCORE2=y
# CONFIG_MATOM is not set
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_CPU=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_CMPXCHG=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_XADD=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_P6_NOP=y
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_PROCESSOR_SELECT=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_AMD=y
# CONFIG_CPU_SUP_CENTAUR is not set
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
CONFIG_GART_IOMMU=y
CONFIG_CALGARY_IOMMU=y
CONFIG_CALGARY_IOMMU_ENABLED_BY_DEFAULT=y
# CONFIG_AMD_IOMMU is not set
CONFIG_SWIOTLB=y
CONFIG_IOMMU_HELPER=y
# CONFIG_IOMMU_API is not set
# CONFIG_MAXSMP is not set
CONFIG_NR_CPUS=16
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_INTEL=y
# CONFIG_X86_MCE_AMD is not set
CONFIG_X86_MCE_THRESHOLD=y
# CONFIG_X86_MCE_INJECT is not set
CONFIG_X86_THERMAL_VECTOR=y
CONFIG_I8K=m
CONFIG_MICROCODE=y
CONFIG_MICROCODE_INTEL=y
# CONFIG_MICROCODE_AMD is not set
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
CONFIG_DIRECT_GBPAGES=y
# CONFIG_NUMA is not set
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_HAVE_MEMORY_PRESENT=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER=y
CONFIG_SPARSEMEM_VMEMMAP=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=999999
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_KSM=y
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
# CONFIG_MEMORY_FAILURE is not set
CONFIG_X86_CHECK_BIOS_CORRUPTION=y
CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK=y
CONFIG_X86_RESERVE_LOW_64K=y
CONFIG_MTRR=y
CONFIG_MTRR_SANITIZER=y
CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=1
CONFIG_MTRR_SANITIZER_SPARE_REG_NR_DEFAULT=1
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
# CONFIG_EFI is not set
CONFIG_SECCOMP=y
CONFIG_CC_STACKPROTECTOR=y
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_SCHED_HRTICK=y
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x200000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x1000000
CONFIG_HOTPLUG_CPU=y
# CONFIG_COMPAT_VDSO is not set
# CONFIG_CMDLINE_BOOL is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y

#
# Power management and ACPI options
#
CONFIG_ARCH_HIBERNATION_HEADER=y
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_SLEEP=y
CONFIG_SUSPEND_NVS=y
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
CONFIG_HIBERNATION=y
CONFIG_PM_STD_PARTITION=""
CONFIG_PM_RUNTIME=y
CONFIG_PM_OPS=y
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_PROCFS=y
CONFIG_ACPI_PROCFS_POWER=y
CONFIG_ACPI_POWER_METER=m
CONFIG_ACPI_SYSFS_POWER=y
# CONFIG_ACPI_EC_DEBUGFS is not set
CONFIG_ACPI_PROC_EVENT=y
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_PROCESSOR_AGGREGATOR=m
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_PCI_SLOT=y
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=y
# CONFIG_ACPI_SBS is not set
CONFIG_ACPI_HED=m
CONFIG_ACPI_APEI=y
CONFIG_ACPI_APEI_GHES=m
# CONFIG_ACPI_APEI_EINJ is not set
# CONFIG_ACPI_APEI_ERST_DEBUG is not set
# CONFIG_SFI is not set

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
CONFIG_CPU_FREQ_DEBUG=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y

#
# CPUFreq processor drivers
#
# CONFIG_X86_PCC_CPUFREQ is not set
CONFIG_X86_ACPI_CPUFREQ=y
# CONFIG_X86_POWERNOW_K8 is not set
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
# CONFIG_X86_P4_CLOCKMOD is not set

#
# shared options
#
# CONFIG_X86_SPEEDSTEP_LIB is not set
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y
CONFIG_INTEL_IDLE=y

#
# Memory power savings
#
CONFIG_I7300_IDLE_IOAT_CHANNEL=y
CONFIG_I7300_IDLE=y

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCI_CNB20LE_QUIRK=y
# CONFIG_DMAR is not set
# CONFIG_INTR_REMAP is not set
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=m
CONFIG_PCIEAER=y
# CONFIG_PCIE_ECRC is not set
# CONFIG_PCIEAER_INJECT is not set
CONFIG_PCIEASPM=y
CONFIG_PCIEASPM_DEBUG=y
CONFIG_PCIE_PME=y
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_STUB is not set
CONFIG_HT_IRQ=y
# CONFIG_PCI_IOV is not set
CONFIG_PCI_IOAPIC=y
CONFIG_ISA_DMA_API=y
CONFIG_K8_NB=y
# CONFIG_PCCARD is not set
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_HOTPLUG_PCI_ACPI=m
CONFIG_HOTPLUG_PCI_ACPI_IBM=m
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m
CONFIG_HOTPLUG_PCI_SHPC=m

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
# CONFIG_HAVE_AOUT is not set
CONFIG_BINFMT_MISC=y
CONFIG_IA32_EMULATION=y
# CONFIG_IA32_AOUT is not set
CONFIG_COMPAT=y
CONFIG_COMPAT_FOR_U64_ALIGNMENT=y
CONFIG_SYSVIPC_COMPAT=y
CONFIG_NET=y
CONFIG_COMPAT_NETLINK_MESSAGES=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_XFRM=y
CONFIG_XFRM_USER=y
# CONFIG_XFRM_SUB_POLICY is not set
CONFIG_XFRM_MIGRATE=y
# CONFIG_XFRM_STATISTICS is not set
CONFIG_XFRM_IPCOMP=y
CONFIG_NET_KEY=y
CONFIG_NET_KEY_MIGRATE=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_ASK_IP_FIB_HASH=y
# CONFIG_IP_FIB_TRIE is not set
CONFIG_IP_FIB_HASH=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
CONFIG_IP_PNP_BOOTP=y
CONFIG_IP_PNP_RARP=y
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE=y
# CONFIG_IP_MROUTE_MULTIPLE_TABLES is not set
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=y
CONFIG_INET_ESP=y
CONFIG_INET_IPCOMP=y
CONFIG_INET_XFRM_TUNNEL=y
CONFIG_INET_TUNNEL=y
CONFIG_INET_XFRM_MODE_TRANSPORT=y
CONFIG_INET_XFRM_MODE_TUNNEL=y
CONFIG_INET_XFRM_MODE_BEET=y
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
CONFIG_IPV6_PRIVACY=y
# CONFIG_IPV6_ROUTER_PREF is not set
# CONFIG_IPV6_OPTIMISTIC_DAD is not set
CONFIG_INET6_AH=y
CONFIG_INET6_ESP=y
CONFIG_INET6_IPCOMP=y
CONFIG_IPV6_MIP6=m
CONFIG_INET6_XFRM_TUNNEL=y
CONFIG_INET6_TUNNEL=y
CONFIG_INET6_XFRM_MODE_TRANSPORT=y
CONFIG_INET6_XFRM_MODE_TUNNEL=y
CONFIG_INET6_XFRM_MODE_BEET=y
CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION=m
CONFIG_IPV6_SIT=y
# CONFIG_IPV6_SIT_6RD is not set
CONFIG_IPV6_NDISC_NODETYPE=y
CONFIG_IPV6_TUNNEL=m
# CONFIG_IPV6_MULTIPLE_TABLES is not set
# CONFIG_IPV6_MROUTE is not set
CONFIG_NETWORK_SECMARK=y
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=y

#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_MARK=y
CONFIG_NF_CONNTRACK_SECMARK=y
CONFIG_NF_CONNTRACK_EVENTS=y
CONFIG_NF_CT_PROTO_DCCP=m
CONFIG_NF_CT_PROTO_GRE=m
CONFIG_NF_CT_PROTO_SCTP=m
CONFIG_NF_CT_PROTO_UDPLITE=m
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NF_CONNTRACK_IRC=m
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NF_CONNTRACK_PPTP=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
CONFIG_NF_CT_NETLINK=m
CONFIG_NETFILTER_TPROXY=m
CONFIG_NETFILTER_XTABLES=m

#
# Xtables combined modules
#
CONFIG_NETFILTER_XT_MARK=m
CONFIG_NETFILTER_XT_CONNMARK=m

#
# Xtables targets
#
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_CONNSECMARK=m
# CONFIG_NETFILTER_XT_TARGET_CT is not set
CONFIG_NETFILTER_XT_TARGET_DSCP=m
CONFIG_NETFILTER_XT_TARGET_HL=m
CONFIG_NETFILTER_XT_TARGET_IDLETIMER=m
CONFIG_NETFILTER_XT_TARGET_LED=m
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_TARGET_NFLOG=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_RATEEST=m
# CONFIG_NETFILTER_XT_TARGET_TEE is not set
CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_SECMARK=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m

#
# Xtables matches
#
CONFIG_NETFILTER_XT_MATCH_CLUSTER=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNLIMIT=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_CPU=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_DSCP=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_HL=m
CONFIG_NETFILTER_XT_MATCH_IPRANGE=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
CONFIG_NETFILTER_XT_MATCH_OSF=m
CONFIG_NETFILTER_XT_MATCH_OWNER=m
CONFIG_NETFILTER_XT_MATCH_POLICY=m
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
CONFIG_NETFILTER_XT_MATCH_QUOTA=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_NETFILTER_XT_MATCH_TIME=m
CONFIG_NETFILTER_XT_MATCH_U32=m
# CONFIG_IP_VS is not set

#
# IP: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV4=m
CONFIG_NF_CONNTRACK_IPV4=m
CONFIG_NF_CONNTRACK_PROC_COMPAT=y
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT=m
CONFIG_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_NF_NAT_SNMP_BASIC=m
CONFIG_NF_NAT_PROTO_DCCP=m
CONFIG_NF_NAT_PROTO_GRE=m
CONFIG_NF_NAT_PROTO_UDPLITE=m
CONFIG_NF_NAT_PROTO_SCTP=m
CONFIG_NF_NAT_FTP=m
CONFIG_NF_NAT_IRC=m
CONFIG_NF_NAT_TFTP=m
CONFIG_NF_NAT_AMANDA=m
CONFIG_NF_NAT_PPTP=m
CONFIG_NF_NAT_H323=m
CONFIG_NF_NAT_SIP=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_CLUSTERIP=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m

#
# IPv6: Netfilter Configuration
#
CONFIG_NF_CONNTRACK_IPV6=m
CONFIG_IP6_NF_QUEUE=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_MATCH_FRAG=m
CONFIG_IP6_NF_MATCH_OPTS=m
CONFIG_IP6_NF_MATCH_HL=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m
CONFIG_IP6_NF_MATCH_MH=m
CONFIG_IP6_NF_MATCH_RT=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_TARGET_LOG=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_IP6=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
CONFIG_BRIDGE_EBT_ULOG=m
CONFIG_BRIDGE_EBT_NFLOG=m
# CONFIG_IP_DCCP is not set
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_HMAC_NONE is not set
# CONFIG_SCTP_HMAC_SHA1 is not set
CONFIG_SCTP_HMAC_MD5=y
CONFIG_RDS=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
# CONFIG_TIPC is not set
CONFIG_ATM=m
CONFIG_ATM_CLIP=m
# CONFIG_ATM_CLIP_NO_ICMP is not set
CONFIG_ATM_LANE=m
CONFIG_ATM_MPOA=m
# CONFIG_ATM_BR2684 is not set
# CONFIG_L2TP is not set
CONFIG_STP=m
CONFIG_BRIDGE=m
CONFIG_BRIDGE_IGMP_SNOOPING=y
# CONFIG_NET_DSA is not set
CONFIG_VLAN_8021Q=m
# CONFIG_VLAN_8021Q_GVRP is not set
# CONFIG_DECNET is not set
CONFIG_LLC=m
# CONFIG_LLC2 is not set
CONFIG_IPX=m
# CONFIG_IPX_INTERN is not set
CONFIG_ATALK=m
CONFIG_DEV_APPLETALK=m
CONFIG_IPDDP=m
CONFIG_IPDDP_ENCAP=y
CONFIG_IPDDP_DECAP=y
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
CONFIG_PHONET=m
CONFIG_IEEE802154=m
# CONFIG_NET_SCHED is not set
CONFIG_NET_CLS_ROUTE=y
# CONFIG_DCB is not set
CONFIG_DNS_RESOLVER=y
CONFIG_RPS=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
CONFIG_CAN=m
CONFIG_CAN_RAW=m
CONFIG_CAN_BCM=m

#
# CAN Device Drivers
#
CONFIG_CAN_VCAN=m
CONFIG_CAN_DEV=m
CONFIG_CAN_CALC_BITTIMING=y
CONFIG_CAN_SJA1000=m
CONFIG_CAN_SJA1000_PLATFORM=m
CONFIG_CAN_EMS_PCI=m
CONFIG_CAN_KVASER_PCI=m
# CONFIG_CAN_PLX_PCI is not set

#
# CAN USB interfaces
#
# CONFIG_CAN_EMS_USB is not set
# CONFIG_CAN_ESD_USB2 is not set
# CONFIG_CAN_DEBUG_DEVICES is not set
# CONFIG_IRDA is not set
CONFIG_BT=m
CONFIG_BT_L2CAP=m
CONFIG_BT_SCO=m
CONFIG_BT_RFCOMM=m
CONFIG_BT_RFCOMM_TTY=y
CONFIG_BT_BNEP=m
CONFIG_BT_BNEP_MC_FILTER=y
CONFIG_BT_BNEP_PROTO_FILTER=y
CONFIG_BT_CMTP=m
CONFIG_BT_HIDP=m

#
# Bluetooth device drivers
#
CONFIG_BT_HCIBTUSB=m
CONFIG_BT_HCIBTSDIO=m
CONFIG_BT_HCIUART=m
CONFIG_BT_HCIUART_H4=y
CONFIG_BT_HCIUART_BCSP=y
CONFIG_BT_HCIUART_ATH3K=y
CONFIG_BT_HCIUART_LL=y
CONFIG_BT_HCIBCM203X=m
CONFIG_BT_HCIBPA10X=m
CONFIG_BT_HCIBFUSB=m
CONFIG_BT_HCIVHCI=m
CONFIG_BT_MRVL=m
CONFIG_BT_MRVL_SDIO=m
# CONFIG_BT_ATH3K is not set
# CONFIG_AF_RXRPC is not set
CONFIG_FIB_RULES=y
CONFIG_WIRELESS=y
CONFIG_WIRELESS_EXT=y
CONFIG_WEXT_CORE=y
CONFIG_WEXT_PROC=y
CONFIG_WEXT_SPY=y
CONFIG_WEXT_PRIV=y
CONFIG_CFG80211=m
# CONFIG_NL80211_TESTMODE is not set
# CONFIG_CFG80211_DEVELOPER_WARNINGS is not set
# CONFIG_CFG80211_REG_DEBUG is not set
CONFIG_CFG80211_DEFAULT_PS=y
# CONFIG_CFG80211_DEBUGFS is not set
# CONFIG_CFG80211_INTERNAL_REGDB is not set
CONFIG_CFG80211_WEXT=y
CONFIG_WIRELESS_EXT_SYSFS=y
CONFIG_LIB80211=y
CONFIG_LIB80211_CRYPT_WEP=m
CONFIG_LIB80211_CRYPT_CCMP=m
CONFIG_LIB80211_CRYPT_TKIP=m
# CONFIG_LIB80211_DEBUG is not set
CONFIG_MAC80211=m
CONFIG_MAC80211_HAS_RC=y
# CONFIG_MAC80211_RC_PID is not set
CONFIG_MAC80211_RC_MINSTREL=y
CONFIG_MAC80211_RC_MINSTREL_HT=y
CONFIG_MAC80211_RC_DEFAULT_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT="minstrel_ht"
# CONFIG_MAC80211_MESH is not set
CONFIG_MAC80211_LEDS=y
# CONFIG_MAC80211_DEBUGFS is not set
# CONFIG_MAC80211_DEBUG_MENU is not set
CONFIG_WIMAX=m
CONFIG_WIMAX_DEBUG_LEVEL=8
CONFIG_RFKILL=m
CONFIG_RFKILL_LEDS=y
CONFIG_RFKILL_INPUT=y
# CONFIG_NET_9P is not set
# CONFIG_CAIF is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_DEVTMPFS is not set
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE=""
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
CONFIG_CONNECTOR=y
CONFIG_PROC_EVENTS=y
# CONFIG_MTD is not set
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_SERIAL=m
CONFIG_PARPORT_PC_FIFO=y
CONFIG_PARPORT_PC_SUPERIO=y
# CONFIG_PARPORT_GSC is not set
CONFIG_PARPORT_AX88796=m
CONFIG_PARPORT_1284=y
CONFIG_PARPORT_NOT_PC=y
CONFIG_PNP=y
CONFIG_PNP_DEBUG_MESSAGES=y

#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_FD=m
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_DRBD is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=16384
CONFIG_BLK_DEV_XIP=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
# CONFIG_ATA_OVER_ETH is not set
# CONFIG_BLK_DEV_HD is not set
CONFIG_MISC_DEVICES=y
CONFIG_AD525X_DPOT=m
# CONFIG_AD525X_DPOT_I2C is not set
# CONFIG_IBM_ASM is not set
# CONFIG_PHANTOM is not set
# CONFIG_SGI_IOC4 is not set
CONFIG_TIFM_CORE=m
# CONFIG_TIFM_7XX1 is not set
# CONFIG_ICS932S401 is not set
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_CS5535_MFGPT is not set
# CONFIG_HP_ILO is not set
# CONFIG_ISL29003 is not set
CONFIG_SENSORS_TSL2550=m
CONFIG_SENSORS_BH1780=m
CONFIG_HMC6352=m
CONFIG_DS1682=m
# CONFIG_VMWARE_BALLOON is not set
CONFIG_BMP085=m
# CONFIG_C2PORT is not set

#
# EEPROM support
#
# CONFIG_EEPROM_AT24 is not set
# CONFIG_EEPROM_LEGACY is not set
CONFIG_EEPROM_MAX6875=m
CONFIG_EEPROM_93CX6=m
CONFIG_CB710_CORE=m
# CONFIG_CB710_DEBUG is not set
CONFIG_CB710_DEBUG_ASSUMPTIONS=y
CONFIG_IWMC3200TOP=m
# CONFIG_IWMC3200TOP_DEBUG is not set
# CONFIG_IWMC3200TOP_DEBUGFS is not set
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set

#
# SCSI device support
#
CONFIG_SCSI_MOD=y
CONFIG_RAID_ATTRS=y
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_TGT=y
# CONFIG_SCSI_NETLINK is not set
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=y
CONFIG_CHR_DEV_SCH=y
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
# CONFIG_SCSI_LOGGING is not set
CONFIG_SCSI_SCAN_ASYNC=y
CONFIG_SCSI_WAIT_SCAN=m

#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=y
# CONFIG_SCSI_FC_ATTRS is not set
CONFIG_SCSI_ISCSI_ATTRS=y
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# CONFIG_SCSI_LOWLEVEL is not set
# CONFIG_SCSI_DH is not set
# CONFIG_SCSI_OSD_INITIATOR is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_ACPI=y
CONFIG_SATA_PMP=y

#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=y
# CONFIG_SATA_AHCI_PLATFORM is not set
CONFIG_SATA_INIC162X=y
CONFIG_SATA_SIL24=y
CONFIG_ATA_SFF=y

#
# SFF controllers with custom DMA interface
#
CONFIG_PDC_ADMA=y
CONFIG_SATA_QSTOR=y
CONFIG_SATA_SX4=y
CONFIG_ATA_BMDMA=y

#
# SATA SFF controllers with BMDMA
#
CONFIG_ATA_PIIX=y
CONFIG_SATA_MV=y
CONFIG_SATA_NV=y
CONFIG_SATA_PROMISE=y
CONFIG_SATA_SIL=y
CONFIG_SATA_SIS=y
CONFIG_SATA_SVW=y
CONFIG_SATA_ULI=y
CONFIG_SATA_VIA=y
CONFIG_SATA_VITESSE=y

#
# PATA SFF controllers with BMDMA
#
CONFIG_PATA_ALI=y
CONFIG_PATA_AMD=y
CONFIG_PATA_ARTOP=y
CONFIG_PATA_ATIIXP=y
CONFIG_PATA_ATP867X=y
CONFIG_PATA_CMD64X=y
CONFIG_PATA_CS5520=y
CONFIG_PATA_CS5530=y
CONFIG_PATA_CYPRESS=y
CONFIG_PATA_EFAR=y
CONFIG_PATA_HPT366=y
CONFIG_PATA_HPT37X=y
CONFIG_PATA_HPT3X2N=y
CONFIG_PATA_HPT3X3=y
# CONFIG_PATA_HPT3X3_DMA is not set
CONFIG_PATA_IT8213=y
CONFIG_PATA_IT821X=y
CONFIG_PATA_JMICRON=y
CONFIG_PATA_MARVELL=y
CONFIG_PATA_NETCELL=y
CONFIG_PATA_NINJA32=y
CONFIG_PATA_NS87415=y
CONFIG_PATA_OLDPIIX=y
CONFIG_PATA_OPTIDMA=y
CONFIG_PATA_PDC2027X=y
CONFIG_PATA_PDC_OLD=y
CONFIG_PATA_RADISYS=y
CONFIG_PATA_RDC=y
CONFIG_PATA_SC1200=y
CONFIG_PATA_SCH=y
CONFIG_PATA_SERVERWORKS=y
CONFIG_PATA_SIL680=y
CONFIG_PATA_SIS=y
CONFIG_PATA_TOSHIBA=y
CONFIG_PATA_TRIFLEX=y
CONFIG_PATA_VIA=y
CONFIG_PATA_WINBOND=y

#
# PIO-only SFF controllers
#
CONFIG_PATA_CMD640_PCI=y
CONFIG_PATA_MPIIX=y
CONFIG_PATA_NS87410=y
CONFIG_PATA_OPTI=y
# CONFIG_PATA_PLATFORM is not set
CONFIG_PATA_RZ1000=y

#
# Generic fallback / legacy drivers
#
CONFIG_PATA_ACPI=y
# CONFIG_ATA_GENERIC is not set
# CONFIG_PATA_LEGACY is not set
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_AUTODETECT=y
CONFIG_MD_LINEAR=y
CONFIG_MD_RAID0=y
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=y
CONFIG_MD_RAID456=y
CONFIG_MULTICORE_RAID456=y
CONFIG_MD_MULTIPATH=y
CONFIG_MD_FAULTY=m
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_DEBUG is not set
CONFIG_DM_CRYPT=y
CONFIG_DM_SNAPSHOT=y
CONFIG_DM_MIRROR=y
# CONFIG_DM_LOG_USERSPACE is not set
CONFIG_DM_ZERO=y
CONFIG_DM_MULTIPATH=y
# CONFIG_DM_MULTIPATH_QL is not set
# CONFIG_DM_MULTIPATH_ST is not set
# CONFIG_DM_DELAY is not set
CONFIG_DM_UEVENT=y
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#

#
# You can enable one or both FireWire driver stacks.
#

#
# The newer stack is recommended.
#
CONFIG_FIREWIRE=m
CONFIG_FIREWIRE_OHCI=m
CONFIG_FIREWIRE_OHCI_DEBUG=y
CONFIG_FIREWIRE_SBP2=m
# CONFIG_FIREWIRE_NET is not set
# CONFIG_IEEE1394 is not set
# CONFIG_FIREWIRE_NOSY is not set
CONFIG_I2O=m
CONFIG_I2O_LCT_NOTIFY_ON_CHANGES=y
CONFIG_I2O_EXT_ADAPTEC=y
CONFIG_I2O_EXT_ADAPTEC_DMA64=y
CONFIG_I2O_CONFIG=m
CONFIG_I2O_CONFIG_OLD_IOCTL=y
CONFIG_I2O_BUS=m
CONFIG_I2O_BLOCK=m
CONFIG_I2O_SCSI=m
CONFIG_I2O_PROC=m
CONFIG_MACINTOSH_DRIVERS=y
CONFIG_MAC_EMUMOUSEBTN=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_BONDING=m
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=y
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_ARCNET is not set
CONFIG_PHYLIB=m

#
# MII PHY device drivers
#
CONFIG_MARVELL_PHY=m
CONFIG_DAVICOM_PHY=m
CONFIG_QSEMI_PHY=m
CONFIG_LXT_PHY=m
CONFIG_CICADA_PHY=m
CONFIG_VITESSE_PHY=m
CONFIG_SMSC_PHY=m
CONFIG_BROADCOM_PHY=m
CONFIG_ICPLUS_PHY=m
CONFIG_REALTEK_PHY=m
CONFIG_NATIONAL_PHY=m
CONFIG_STE10XP=m
CONFIG_LSI_ET1011C_PHY=m
CONFIG_MICREL_PHY=m
# CONFIG_MDIO_BITBANG is not set
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
CONFIG_HAPPYMEAL=m
CONFIG_SUNGEM=m
CONFIG_CASSINI=m
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=m
CONFIG_TYPHOON=m
CONFIG_ETHOC=m
CONFIG_DNET=m
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_DE2104X_DSL=0
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_TULIP_NAPI is not set
CONFIG_DE4X5=m
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
CONFIG_ULI526X=m
CONFIG_HP100=m
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
# CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set
# CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set
# CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set
CONFIG_NET_PCI=y
CONFIG_PCNET32=m
CONFIG_AMD8111_ETH=m
CONFIG_ADAPTEC_STARFIRE=m
# CONFIG_KSZ884X_PCI is not set
CONFIG_B44=m
CONFIG_B44_PCI_AUTOSELECT=y
CONFIG_B44_PCICORE_AUTOSELECT=y
CONFIG_B44_PCI=y
CONFIG_FORCEDETH=m
CONFIG_E100=m
CONFIG_FEALNX=m
CONFIG_NATSEMI=m
CONFIG_NE2K_PCI=m
CONFIG_8139CP=m
CONFIG_8139TOO=m
# CONFIG_8139TOO_PIO is not set
CONFIG_8139TOO_TUNE_TWISTER=y
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_R6040=m
CONFIG_SIS900=m
CONFIG_EPIC100=m
CONFIG_SMSC9420=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
CONFIG_TLAN=m
CONFIG_KS8842=m
CONFIG_KS8851_MLL=m
CONFIG_VIA_RHINE=m
CONFIG_VIA_RHINE_MMIO=y
CONFIG_SC92031=m
CONFIG_NET_POCKET=y
CONFIG_ATP=m
CONFIG_DE600=m
CONFIG_DE620=m
CONFIG_ATL2=m
CONFIG_NETDEV_1000=y
CONFIG_ACENIC=m
# CONFIG_ACENIC_OMIT_TIGON_I is not set
CONFIG_DL2K=m
CONFIG_E1000=m
CONFIG_E1000E=m
CONFIG_IP1000=m
CONFIG_IGB=m
CONFIG_IGB_DCA=y
CONFIG_IGBVF=m
CONFIG_NS83820=m
CONFIG_HAMACHI=m
CONFIG_YELLOWFIN=m
CONFIG_R8169=m
CONFIG_R8169_VLAN=y
CONFIG_SIS190=m
CONFIG_SKGE=m
# CONFIG_SKGE_DEBUG is not set
CONFIG_SKY2=m
# CONFIG_SKY2_DEBUG is not set
CONFIG_VIA_VELOCITY=m
CONFIG_TIGON3=m
CONFIG_BNX2=m
CONFIG_CNIC=m
CONFIG_QLA3XXX=m
CONFIG_ATL1=m
CONFIG_ATL1E=m
CONFIG_ATL1C=m
CONFIG_JME=m
CONFIG_NETDEV_10000=y
CONFIG_MDIO=m
CONFIG_CHELSIO_T1=m
CONFIG_CHELSIO_T1_1G=y
CONFIG_CHELSIO_T3_DEPENDS=y
CONFIG_CHELSIO_T3=m
CONFIG_CHELSIO_T4_DEPENDS=y
# CONFIG_CHELSIO_T4 is not set
CONFIG_CHELSIO_T4VF_DEPENDS=y
CONFIG_CHELSIO_T4VF=m
CONFIG_ENIC=m
# CONFIG_IXGBE is not set
# CONFIG_IXGBEVF is not set
CONFIG_IXGB=m
CONFIG_S2IO=m
CONFIG_VXGE=m
# CONFIG_VXGE_DEBUG_TRACE_ALL is not set
CONFIG_MYRI10GE=m
CONFIG_MYRI10GE_DCA=y
CONFIG_NETXEN_NIC=m
CONFIG_NIU=m
CONFIG_MLX4_EN=m
CONFIG_MLX4_CORE=m
# CONFIG_MLX4_DEBUG is not set
CONFIG_TEHUTI=m
CONFIG_BNX2X=m
# CONFIG_QLCNIC is not set
CONFIG_QLGE=m
CONFIG_SFC=m
CONFIG_BE2NET=m
# CONFIG_TR is not set
CONFIG_WLAN=y
CONFIG_LIBERTAS_THINFIRM=m
# CONFIG_LIBERTAS_THINFIRM_DEBUG is not set
CONFIG_LIBERTAS_THINFIRM_USB=m
CONFIG_AIRO=m
CONFIG_ATMEL=m
CONFIG_PCI_ATMEL=m
CONFIG_AT76C50X_USB=m
CONFIG_PRISM54=m
CONFIG_USB_ZD1201=m
CONFIG_USB_NET_RNDIS_WLAN=m
CONFIG_RTL8180=m
CONFIG_RTL8187=m
CONFIG_RTL8187_LEDS=y
CONFIG_ADM8211=m
CONFIG_MAC80211_HWSIM=m
CONFIG_MWL8K=m
CONFIG_ATH_COMMON=m
# CONFIG_ATH_DEBUG is not set
CONFIG_ATH5K=m
# CONFIG_ATH5K_DEBUG is not set
# CONFIG_ATH9K is not set
# CONFIG_ATH9K_HTC is not set
CONFIG_AR9170_USB=m
CONFIG_AR9170_LEDS=y
CONFIG_B43=m
CONFIG_B43_PCI_AUTOSELECT=y
CONFIG_B43_PCICORE_AUTOSELECT=y
# CONFIG_B43_SDIO is not set
CONFIG_B43_PIO=y
CONFIG_B43_PHY_LP=y
CONFIG_B43_LEDS=y
CONFIG_B43_HWRNG=y
# CONFIG_B43_DEBUG is not set
CONFIG_B43LEGACY=m
CONFIG_B43LEGACY_PCI_AUTOSELECT=y
CONFIG_B43LEGACY_PCICORE_AUTOSELECT=y
CONFIG_B43LEGACY_LEDS=y
CONFIG_B43LEGACY_HWRNG=y
CONFIG_B43LEGACY_DEBUG=y
CONFIG_B43LEGACY_DMA=y
CONFIG_B43LEGACY_PIO=y
CONFIG_B43LEGACY_DMA_AND_PIO_MODE=y
# CONFIG_B43LEGACY_DMA_MODE is not set
# CONFIG_B43LEGACY_PIO_MODE is not set
CONFIG_HOSTAP=m
CONFIG_HOSTAP_FIRMWARE=y
# CONFIG_HOSTAP_FIRMWARE_NVRAM is not set
CONFIG_HOSTAP_PLX=m
CONFIG_HOSTAP_PCI=m
# CONFIG_IPW2100 is not set
# CONFIG_IPW2200 is not set
CONFIG_IWLWIFI=m
# CONFIG_IWLWIFI_DEBUG is not set
# CONFIG_IWLAGN is not set
CONFIG_IWL3945=m
CONFIG_IWM=m
# CONFIG_IWM_DEBUG is not set
CONFIG_LIBERTAS=m
CONFIG_LIBERTAS_USB=m
CONFIG_LIBERTAS_SDIO=m
# CONFIG_LIBERTAS_DEBUG is not set
# CONFIG_LIBERTAS_MESH is not set
CONFIG_HERMES=m
# CONFIG_HERMES_PRISM is not set
CONFIG_HERMES_CACHE_FW_ON_INIT=y
CONFIG_PLX_HERMES=m
CONFIG_TMD_HERMES=m
CONFIG_NORTEL_HERMES=m
# CONFIG_ORINOCO_USB is not set
CONFIG_P54_COMMON=m
CONFIG_P54_USB=m
CONFIG_P54_PCI=m
CONFIG_P54_LEDS=y
CONFIG_RT2X00=m
CONFIG_RT2400PCI=m
CONFIG_RT2500PCI=m
CONFIG_RT61PCI=m
CONFIG_RT2800PCI_PCI=y
CONFIG_RT2800PCI=m
# CONFIG_RT2800PCI_RT30XX is not set
# CONFIG_RT2800PCI_RT35XX is not set
CONFIG_RT2500USB=m
CONFIG_RT73USB=m
CONFIG_RT2800USB=m
# CONFIG_RT2800USB_RT30XX is not set
# CONFIG_RT2800USB_RT35XX is not set
# CONFIG_RT2800USB_UNKNOWN is not set
CONFIG_RT2800_LIB=m
CONFIG_RT2X00_LIB_PCI=m
CONFIG_RT2X00_LIB_USB=m
CONFIG_RT2X00_LIB=m
CONFIG_RT2X00_LIB_HT=y
CONFIG_RT2X00_LIB_FIRMWARE=y
CONFIG_RT2X00_LIB_CRYPTO=y
CONFIG_RT2X00_LIB_LEDS=y
# CONFIG_RT2X00_DEBUG is not set
# CONFIG_WL12XX is not set
CONFIG_ZD1211RW=m
# CONFIG_ZD1211RW_DEBUG is not set

#
# WiMAX Wireless Broadband devices
#
# CONFIG_WIMAX_I2400M_USB is not set
# CONFIG_WIMAX_I2400M_SDIO is not set

#
# USB Network Adapters
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_CDCETHER=m
# CONFIG_USB_NET_CDC_EEM is not set
CONFIG_USB_NET_DM9601=m
# CONFIG_USB_NET_SMSC75XX is not set
CONFIG_USB_NET_SMSC95XX=m
CONFIG_USB_NET_GL620A=m
CONFIG_USB_NET_NET1080=m
CONFIG_USB_NET_PLUSB=m
CONFIG_USB_NET_MCS7830=m
CONFIG_USB_NET_RNDIS_HOST=m
CONFIG_USB_NET_CDC_SUBSET=m
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_KC2190=y
CONFIG_USB_NET_ZAURUS=m
CONFIG_USB_HSO=m
CONFIG_USB_NET_INT51X1=m
CONFIG_USB_CDC_PHONET=m
# CONFIG_USB_IPHETH is not set
# CONFIG_USB_SIERRA_NET is not set
# CONFIG_WAN is not set
CONFIG_ATM_DRIVERS=y
# CONFIG_ATM_DUMMY is not set
CONFIG_ATM_TCP=m
CONFIG_ATM_LANAI=m
CONFIG_ATM_ENI=m
# CONFIG_ATM_ENI_DEBUG is not set
# CONFIG_ATM_ENI_TUNE_BURST is not set
CONFIG_ATM_FIRESTREAM=m
CONFIG_ATM_ZATM=m
# CONFIG_ATM_ZATM_DEBUG is not set
# CONFIG_ATM_NICSTAR is not set
CONFIG_ATM_IDT77252=m
# CONFIG_ATM_IDT77252_DEBUG is not set
# CONFIG_ATM_IDT77252_RCV_ALL is not set
CONFIG_ATM_IDT77252_USE_SUNI=y
CONFIG_ATM_AMBASSADOR=m
# CONFIG_ATM_AMBASSADOR_DEBUG is not set
CONFIG_ATM_HORIZON=m
# CONFIG_ATM_HORIZON_DEBUG is not set
CONFIG_ATM_IA=m
# CONFIG_ATM_IA_DEBUG is not set
CONFIG_ATM_FORE200E=m
# CONFIG_ATM_FORE200E_USE_TASKLET is not set
CONFIG_ATM_FORE200E_TX_RETRY=16
CONFIG_ATM_FORE200E_DEBUG=0
CONFIG_ATM_HE=m
# CONFIG_ATM_HE_USE_SUNI is not set
CONFIG_ATM_SOLOS=m
CONFIG_IEEE802154_DRIVERS=m
# CONFIG_IEEE802154_FAKEHARD is not set

#
# CAIF transport drivers
#
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
CONFIG_PPP_MPPE=m
CONFIG_PPPOE=m
CONFIG_PPPOATM=m
# CONFIG_SLIP is not set
CONFIG_SLHC=m
# CONFIG_NET_FC is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
CONFIG_VMXNET3=m
CONFIG_ISDN=y
CONFIG_ISDN_I4L=m
# CONFIG_ISDN_PPP is not set
# CONFIG_ISDN_AUDIO is not set

#
# ISDN feature submodules
#
# CONFIG_ISDN_DIVERSION is not set

#
# ISDN4Linux hardware drivers
#

#
# Passive cards
#
# CONFIG_ISDN_DRV_HISAX is not set

#
# Active cards
#
CONFIG_ISDN_CAPI=m
CONFIG_ISDN_DRV_AVMB1_VERBOSE_REASON=y
CONFIG_CAPI_TRACE=y
CONFIG_ISDN_CAPI_MIDDLEWARE=y
CONFIG_ISDN_CAPI_CAPI20=m
CONFIG_ISDN_CAPI_CAPIFS_BOOL=y
CONFIG_ISDN_CAPI_CAPIFS=m
# CONFIG_ISDN_CAPI_CAPIDRV is not set

#
# CAPI hardware drivers
#
CONFIG_CAPI_AVM=y
CONFIG_ISDN_DRV_AVMB1_B1PCI=m
CONFIG_ISDN_DRV_AVMB1_B1PCIV4=y
CONFIG_ISDN_DRV_AVMB1_T1PCI=m
CONFIG_ISDN_DRV_AVMB1_C4=m
CONFIG_CAPI_EICON=y
CONFIG_ISDN_DIVAS=m
CONFIG_ISDN_DIVAS_BRIPCI=y
CONFIG_ISDN_DIVAS_PRIPCI=y
CONFIG_ISDN_DIVAS_DIVACAPI=m
CONFIG_ISDN_DIVAS_USERIDI=m
CONFIG_ISDN_DIVAS_MAINT=m
# CONFIG_ISDN_DRV_GIGASET is not set
# CONFIG_HYSDN is not set
CONFIG_MISDN=m
CONFIG_MISDN_DSP=m
CONFIG_MISDN_L1OIP=m

#
# mISDN hardware drivers
#
CONFIG_MISDN_HFCPCI=m
CONFIG_MISDN_HFCMULTI=m
CONFIG_MISDN_HFCUSB=m
CONFIG_MISDN_AVMFRITZ=m
CONFIG_MISDN_SPEEDFAX=m
CONFIG_MISDN_INFINEON=m
CONFIG_MISDN_W6692=m
CONFIG_MISDN_NETJET=m
CONFIG_MISDN_IPAC=m
CONFIG_MISDN_ISAR=m
CONFIG_ISDN_HDLC=m
CONFIG_PHONE=m
CONFIG_PHONE_IXJ=m

#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_FF_MEMLESS=y
CONFIG_INPUT_POLLDEV=y
# CONFIG_INPUT_SPARSEKMAP is not set

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=y
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ADP5588=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT2160 is not set
CONFIG_KEYBOARD_LKKBD=m
# CONFIG_KEYBOARD_TCA6416 is not set
CONFIG_KEYBOARD_LM8323=m
CONFIG_KEYBOARD_MAX7359=m
CONFIG_KEYBOARD_MCS=m
CONFIG_KEYBOARD_NEWTON=m
CONFIG_KEYBOARD_OPENCORES=m
CONFIG_KEYBOARD_STOWAWAY=m
CONFIG_KEYBOARD_SUNKBD=m
CONFIG_KEYBOARD_XTKBD=m
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
CONFIG_MOUSE_PS2_ELANTECH=y
CONFIG_MOUSE_PS2_SENTELIC=y
CONFIG_MOUSE_PS2_TOUCHKIT=y
CONFIG_MOUSE_SERIAL=m
CONFIG_MOUSE_APPLETOUCH=m
CONFIG_MOUSE_BCM5974=m
CONFIG_MOUSE_VSXXXAA=m
CONFIG_MOUSE_SYNAPTICS_I2C=y
# CONFIG_INPUT_JOYSTICK is not set
CONFIG_INPUT_TABLET=y
CONFIG_TABLET_USB_ACECAD=m
CONFIG_TABLET_USB_AIPTEK=m
CONFIG_TABLET_USB_GTCO=m
CONFIG_TABLET_USB_KBTAB=m
CONFIG_TABLET_USB_WACOM=m
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_AD7879=m
CONFIG_TOUCHSCREEN_AD7879_I2C=m
CONFIG_TOUCHSCREEN_DYNAPRO=m
# CONFIG_TOUCHSCREEN_HAMPSHIRE is not set
CONFIG_TOUCHSCREEN_EETI=m
CONFIG_TOUCHSCREEN_FUJITSU=m
CONFIG_TOUCHSCREEN_GUNZE=m
CONFIG_TOUCHSCREEN_ELO=m
CONFIG_TOUCHSCREEN_WACOM_W8001=m
CONFIG_TOUCHSCREEN_MCS5000=m
CONFIG_TOUCHSCREEN_MTOUCH=m
CONFIG_TOUCHSCREEN_INEXIO=m
CONFIG_TOUCHSCREEN_MK712=m
CONFIG_TOUCHSCREEN_PENMOUNT=m
CONFIG_TOUCHSCREEN_QT602240=m
CONFIG_TOUCHSCREEN_TOUCHRIGHT=m
CONFIG_TOUCHSCREEN_TOUCHWIN=m
CONFIG_TOUCHSCREEN_WM97XX=m
CONFIG_TOUCHSCREEN_WM9705=y
CONFIG_TOUCHSCREEN_WM9712=y
CONFIG_TOUCHSCREEN_WM9713=y
CONFIG_TOUCHSCREEN_USB_COMPOSITE=m
CONFIG_TOUCHSCREEN_USB_EGALAX=y
CONFIG_TOUCHSCREEN_USB_PANJIT=y
CONFIG_TOUCHSCREEN_USB_3M=y
CONFIG_TOUCHSCREEN_USB_ITM=y
CONFIG_TOUCHSCREEN_USB_ETURBO=y
CONFIG_TOUCHSCREEN_USB_GUNZE=y
CONFIG_TOUCHSCREEN_USB_DMC_TSC10=y
CONFIG_TOUCHSCREEN_USB_IRTOUCH=y
CONFIG_TOUCHSCREEN_USB_IDEALTEK=y
CONFIG_TOUCHSCREEN_USB_GENERAL_TOUCH=y
CONFIG_TOUCHSCREEN_USB_GOTOP=y
CONFIG_TOUCHSCREEN_USB_JASTEC=y
CONFIG_TOUCHSCREEN_USB_E2I=y
CONFIG_TOUCHSCREEN_USB_ZYTRONIC=y
CONFIG_TOUCHSCREEN_USB_ETT_TC45USB=y
CONFIG_TOUCHSCREEN_USB_NEXIO=y
CONFIG_TOUCHSCREEN_TOUCHIT213=m
CONFIG_TOUCHSCREEN_TSC2007=m
# CONFIG_TOUCHSCREEN_TPS6507X is not set
CONFIG_INPUT_MISC=y
# CONFIG_INPUT_AD714X is not set
# CONFIG_INPUT_PCSPKR is not set
# CONFIG_INPUT_APANEL is not set
# CONFIG_INPUT_ATLAS_BTNS is not set
# CONFIG_INPUT_ATI_REMOTE is not set
# CONFIG_INPUT_ATI_REMOTE2 is not set
# CONFIG_INPUT_KEYSPAN_REMOTE is not set
# CONFIG_INPUT_POWERMATE is not set
# CONFIG_INPUT_YEALINK is not set
# CONFIG_INPUT_CM109 is not set
CONFIG_INPUT_UINPUT=m
CONFIG_INPUT_WINBOND_CIR=m
# CONFIG_INPUT_PCF8574 is not set
# CONFIG_INPUT_ADXL34X is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
CONFIG_SERIO_CT82C710=m
CONFIG_SERIO_PARKBD=m
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
CONFIG_SERIO_ALTERA_PS2=m
CONFIG_GAMEPORT=m
CONFIG_GAMEPORT_NS558=m
CONFIG_GAMEPORT_L4=m
CONFIG_GAMEPORT_EMU10K1=m
CONFIG_GAMEPORT_FM801=m

#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
# CONFIG_DEVKMEM is not set
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_N_GSM is not set
CONFIG_NOZOMI=m

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_NR_UARTS=32
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
CONFIG_SERIAL_8250_RSA=y

#
# Non-8250 serial port support
#
CONFIG_SERIAL_MFD_HSU=m
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_SERIAL_JSM=m
# CONFIG_SERIAL_TIMBERDALE is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
# CONFIG_LEGACY_PTYS is not set
CONFIG_PRINTER=m
CONFIG_LP_CONSOLE=y
CONFIG_PPDEV=m
# CONFIG_IPMI_HANDLER is not set
CONFIG_HW_RANDOM=y
CONFIG_HW_RANDOM_TIMERIOMEM=y
CONFIG_HW_RANDOM_INTEL=y
CONFIG_HW_RANDOM_AMD=m
CONFIG_HW_RANDOM_VIA=m
# CONFIG_NVRAM is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
CONFIG_MWAVE=m
# CONFIG_RAW_DRIVER is not set
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
CONFIG_HANGCHECK_TIMER=y
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
# CONFIG_RAMOOPS is not set
CONFIG_I2C=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=m
CONFIG_I2C_MUX=m

#
# Multiplexer I2C Chip support
#
CONFIG_I2C_MUX_PCA954x=m
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_SMBUS=m
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCA=m

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD756_S4882=m
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_ISCH=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_NFORCE2_S4985=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m

#
# ACPI drivers
#
CONFIG_I2C_SCMI=m

#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
CONFIG_I2C_OCORES=m
CONFIG_I2C_PCA_PLATFORM=m
CONFIG_I2C_SIMTEC=m
# CONFIG_I2C_XILINX is not set

#
# External I2C/SMBus adapter drivers
#
# CONFIG_I2C_PARPORT is not set
CONFIG_I2C_PARPORT_LIGHT=m
CONFIG_I2C_TAOS_EVM=m
CONFIG_I2C_TINY_USB=m

#
# Other I2C/SMBus bus drivers
#
CONFIG_I2C_STUB=m
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_SPI is not set

#
# PPS support
#
CONFIG_PPS=m
# CONFIG_PPS_DEBUG is not set

#
# PPS clients support
#
# CONFIG_PPS_CLIENT_KTIMER is not set
# CONFIG_PPS_CLIENT_LDISC is not set
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
# CONFIG_GPIOLIB is not set
CONFIG_W1=m
CONFIG_W1_CON=y

#
# 1-wire Bus Masters
#
# CONFIG_W1_MASTER_MATROX is not set
# CONFIG_W1_MASTER_DS2490 is not set
# CONFIG_W1_MASTER_DS2482 is not set

#
# 1-wire Slaves
#
# CONFIG_W1_SLAVE_THERM is not set
# CONFIG_W1_SLAVE_SMEM is not set
# CONFIG_W1_SLAVE_DS2431 is not set
# CONFIG_W1_SLAVE_DS2433 is not set
CONFIG_W1_SLAVE_DS2760=m
# CONFIG_W1_SLAVE_BQ27000 is not set
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
CONFIG_PDA_POWER=m
# CONFIG_TEST_POWER is not set
CONFIG_BATTERY_DS2760=m
CONFIG_BATTERY_DS2782=m
CONFIG_BATTERY_BQ27x00=m
CONFIG_BATTERY_MAX17040=m
CONFIG_HWMON=m
CONFIG_HWMON_VID=m
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Native drivers
#
CONFIG_SENSORS_ABITUGURU=m
CONFIG_SENSORS_ABITUGURU3=m
CONFIG_SENSORS_AD7414=m
CONFIG_SENSORS_AD7418=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1026=m
CONFIG_SENSORS_ADM1029=m
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ADM9240=m
CONFIG_SENSORS_ADT7411=m
CONFIG_SENSORS_ADT7462=m
CONFIG_SENSORS_ADT7470=m
CONFIG_SENSORS_ADT7475=m
CONFIG_SENSORS_ASC7621=m
CONFIG_SENSORS_K8TEMP=m
CONFIG_SENSORS_K10TEMP=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_ATXP1=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_I5K_AMB=m
CONFIG_SENSORS_F71805F=m
CONFIG_SENSORS_F71882FG=m
CONFIG_SENSORS_F75375S=m
CONFIG_SENSORS_FSCHMD=m
CONFIG_SENSORS_G760A=m
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_GL520SM=m
CONFIG_SENSORS_CORETEMP=m
CONFIG_SENSORS_PKGTEMP=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_JC42=m
CONFIG_SENSORS_LM63=m
CONFIG_SENSORS_LM73=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM87=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_LM92=m
CONFIG_SENSORS_LM93=m
CONFIG_SENSORS_LTC4215=m
CONFIG_SENSORS_LTC4245=m
CONFIG_SENSORS_LM95241=m
CONFIG_SENSORS_MAX1619=m
CONFIG_SENSORS_MAX6650=m
CONFIG_SENSORS_PC87360=m
CONFIG_SENSORS_PC87427=m
CONFIG_SENSORS_PCF8591=m
CONFIG_SENSORS_SIS5595=m
CONFIG_SENSORS_SMM665=m
CONFIG_SENSORS_DME1737=m
CONFIG_SENSORS_EMC1403=m
CONFIG_SENSORS_EMC2103=m
CONFIG_SENSORS_SMSC47M1=m
CONFIG_SENSORS_SMSC47M192=m
CONFIG_SENSORS_SMSC47B397=m
CONFIG_SENSORS_ADS7828=m
CONFIG_SENSORS_AMC6821=m
CONFIG_SENSORS_THMC50=m
CONFIG_SENSORS_TMP102=m
CONFIG_SENSORS_TMP401=m
CONFIG_SENSORS_TMP421=m
CONFIG_SENSORS_VIA_CPUTEMP=m
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_VT1211=m
CONFIG_SENSORS_VT8231=m
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83791D=m
CONFIG_SENSORS_W83792D=m
CONFIG_SENSORS_W83793=m
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83L786NG=m
CONFIG_SENSORS_W83627HF=m
CONFIG_SENSORS_W83627EHF=m
CONFIG_SENSORS_HDAPS=m
CONFIG_SENSORS_LIS3_I2C=m
CONFIG_SENSORS_APPLESMC=m

#
# ACPI drivers
#
CONFIG_SENSORS_ATK0110=m
CONFIG_SENSORS_LIS3LV02D=m
CONFIG_THERMAL=y
# CONFIG_WATCHDOG is not set
CONFIG_SSB_POSSIBLE=y

#
# Sonics Silicon Backplane
#
CONFIG_SSB=m
CONFIG_SSB_SPROM=y
CONFIG_SSB_BLOCKIO=y
CONFIG_SSB_PCIHOST_POSSIBLE=y
CONFIG_SSB_PCIHOST=y
CONFIG_SSB_B43_PCI_BRIDGE=y
CONFIG_SSB_SDIOHOST_POSSIBLE=y
# CONFIG_SSB_SDIOHOST is not set
# CONFIG_SSB_SILENT is not set
# CONFIG_SSB_DEBUG is not set
CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
CONFIG_SSB_DRIVER_PCICORE=y
# CONFIG_MFD_SUPPORT is not set
CONFIG_MFD_CORE=m
CONFIG_LPC_SCH=m
# CONFIG_REGULATOR is not set
CONFIG_MEDIA_SUPPORT=y

#
# Multimedia core support
#
CONFIG_VIDEO_DEV=y
CONFIG_VIDEO_V4L2_COMMON=y
# CONFIG_VIDEO_ALLOW_V4L1 is not set
CONFIG_VIDEO_V4L1_COMPAT=y
# CONFIG_DVB_CORE is not set
CONFIG_VIDEO_MEDIA=y

#
# Multimedia drivers
#
CONFIG_VIDEO_SAA7146=m
CONFIG_VIDEO_SAA7146_VV=m
CONFIG_IR_CORE=y
CONFIG_VIDEO_IR=y
CONFIG_LIRC=y
# CONFIG_RC_MAP is not set
# CONFIG_IR_NEC_DECODER is not set
# CONFIG_IR_RC5_DECODER is not set
# CONFIG_IR_RC6_DECODER is not set
# CONFIG_IR_JVC_DECODER is not set
# CONFIG_IR_SONY_DECODER is not set
# CONFIG_IR_LIRC_CODEC is not set
# CONFIG_IR_IMON is not set
CONFIG_IR_MCEUSB=m
# CONFIG_IR_ENE is not set
# CONFIG_IR_STREAMZAP is not set
CONFIG_MEDIA_ATTACH=y
CONFIG_MEDIA_TUNER=y
# CONFIG_MEDIA_TUNER_CUSTOMISE is not set
CONFIG_MEDIA_TUNER_SIMPLE=y
CONFIG_MEDIA_TUNER_TDA8290=y
CONFIG_MEDIA_TUNER_TDA9887=y
CONFIG_MEDIA_TUNER_TEA5761=y
CONFIG_MEDIA_TUNER_TEA5767=y
CONFIG_MEDIA_TUNER_MT20XX=y
CONFIG_MEDIA_TUNER_XC2028=y
CONFIG_MEDIA_TUNER_XC5000=y
CONFIG_MEDIA_TUNER_MC44S803=y
CONFIG_VIDEO_V4L2=y
CONFIG_VIDEOBUF_GEN=m
CONFIG_VIDEOBUF_DMA_SG=m
CONFIG_VIDEOBUF_VMALLOC=m
CONFIG_VIDEO_BTCX=m
CONFIG_VIDEO_TVEEPROM=m
CONFIG_VIDEO_TUNER=m
CONFIG_VIDEO_CAPTURE_DRIVERS=y
# CONFIG_VIDEO_ADV_DEBUG is not set
# CONFIG_VIDEO_FIXED_MINOR_RANGES is not set
CONFIG_VIDEO_HELPER_CHIPS_AUTO=y
CONFIG_VIDEO_IR_I2C=y
CONFIG_VIDEO_TVAUDIO=m
CONFIG_VIDEO_TDA7432=m
CONFIG_VIDEO_MSP3400=m
CONFIG_VIDEO_CS53L32A=m
CONFIG_VIDEO_M52790=m
CONFIG_VIDEO_WM8775=m
CONFIG_VIDEO_WM8739=m
CONFIG_VIDEO_VP27SMPX=m
CONFIG_VIDEO_SAA6588=m
CONFIG_VIDEO_BT819=m
CONFIG_VIDEO_BT856=m
CONFIG_VIDEO_BT866=m
CONFIG_VIDEO_KS0127=m
CONFIG_VIDEO_OV7670=m
CONFIG_VIDEO_MT9V011=m
CONFIG_VIDEO_SAA7110=m
CONFIG_VIDEO_SAA711X=m
CONFIG_VIDEO_SAA717X=m
CONFIG_VIDEO_TVP5150=m
CONFIG_VIDEO_VPX3220=m
CONFIG_VIDEO_CX25840=m
CONFIG_VIDEO_CX2341X=m
CONFIG_VIDEO_SAA7127=m
CONFIG_VIDEO_SAA7185=m
CONFIG_VIDEO_ADV7170=m
CONFIG_VIDEO_ADV7175=m
CONFIG_VIDEO_UPD64031A=m
CONFIG_VIDEO_UPD64083=m
# CONFIG_VIDEO_VIVI is not set
CONFIG_VIDEO_BT848=m
# CONFIG_VIDEO_BWQCAM is not set
# CONFIG_VIDEO_CQCAM is not set
# CONFIG_VIDEO_W9966 is not set
CONFIG_VIDEO_SAA5246A=m
CONFIG_VIDEO_SAA5249=m
CONFIG_VIDEO_ZORAN=m
CONFIG_VIDEO_ZORAN_DC30=m
CONFIG_VIDEO_ZORAN_ZR36060=m
CONFIG_VIDEO_ZORAN_BUZ=m
CONFIG_VIDEO_ZORAN_DC10=m
CONFIG_VIDEO_ZORAN_LML33=m
CONFIG_VIDEO_ZORAN_LML33R10=m
CONFIG_VIDEO_ZORAN_AVS6EYES=m
CONFIG_VIDEO_SAA7134=m
CONFIG_VIDEO_SAA7134_ALSA=m
# CONFIG_VIDEO_MXB is not set
CONFIG_VIDEO_HEXIUM_ORION=m
CONFIG_VIDEO_HEXIUM_GEMINI=m
CONFIG_VIDEO_CX88=m
CONFIG_VIDEO_CX88_ALSA=m
CONFIG_VIDEO_CX88_BLACKBIRD=m
CONFIG_VIDEO_CX88_MPEG=m
CONFIG_VIDEO_IVTV=m
CONFIG_VIDEO_FB_IVTV=m
CONFIG_VIDEO_CAFE_CCIC=m
CONFIG_SOC_CAMERA=m
CONFIG_SOC_CAMERA_MT9M001=m
CONFIG_SOC_CAMERA_MT9M111=m
CONFIG_SOC_CAMERA_MT9T031=m
CONFIG_SOC_CAMERA_MT9T112=m
CONFIG_SOC_CAMERA_MT9V022=m
CONFIG_SOC_CAMERA_RJ54N1=m
CONFIG_SOC_CAMERA_TW9910=m
CONFIG_SOC_CAMERA_PLATFORM=m
CONFIG_SOC_CAMERA_OV772X=m
CONFIG_SOC_CAMERA_OV9640=m
CONFIG_V4L_USB_DRIVERS=y
CONFIG_USB_VIDEO_CLASS=m
CONFIG_USB_VIDEO_CLASS_INPUT_EVDEV=y
CONFIG_USB_GSPCA=m
CONFIG_USB_M5602=m
CONFIG_USB_STV06XX=m
CONFIG_USB_GL860=m
# CONFIG_USB_GSPCA_BENQ is not set
CONFIG_USB_GSPCA_CONEX=m
# CONFIG_USB_GSPCA_CPIA1 is not set
CONFIG_USB_GSPCA_ETOMS=m
CONFIG_USB_GSPCA_FINEPIX=m
CONFIG_USB_GSPCA_JEILINJ=m
CONFIG_USB_GSPCA_MARS=m
CONFIG_USB_GSPCA_MR97310A=m
CONFIG_USB_GSPCA_OV519=m
CONFIG_USB_GSPCA_OV534=m
# CONFIG_USB_GSPCA_OV534_9 is not set
CONFIG_USB_GSPCA_PAC207=m
CONFIG_USB_GSPCA_PAC7302=m
CONFIG_USB_GSPCA_PAC7311=m
# CONFIG_USB_GSPCA_SN9C2028 is not set
CONFIG_USB_GSPCA_SN9C20X=m
CONFIG_USB_GSPCA_SONIXB=m
CONFIG_USB_GSPCA_SONIXJ=m
CONFIG_USB_GSPCA_SPCA500=m
CONFIG_USB_GSPCA_SPCA501=m
CONFIG_USB_GSPCA_SPCA505=m
CONFIG_USB_GSPCA_SPCA506=m
CONFIG_USB_GSPCA_SPCA508=m
CONFIG_USB_GSPCA_SPCA561=m
CONFIG_USB_GSPCA_SPCA1528=m
CONFIG_USB_GSPCA_SQ905=m
CONFIG_USB_GSPCA_SQ905C=m
CONFIG_USB_GSPCA_SQ930X=m
CONFIG_USB_GSPCA_STK014=m
CONFIG_USB_GSPCA_STV0680=m
CONFIG_USB_GSPCA_SUNPLUS=m
CONFIG_USB_GSPCA_T613=m
CONFIG_USB_GSPCA_TV8532=m
CONFIG_USB_GSPCA_VC032X=m
CONFIG_USB_GSPCA_ZC3XX=m
CONFIG_VIDEO_PVRUSB2=m
CONFIG_VIDEO_PVRUSB2_SYSFS=y
# CONFIG_VIDEO_PVRUSB2_DEBUGIFC is not set
CONFIG_VIDEO_HDPVR=m
CONFIG_VIDEO_EM28XX=m
CONFIG_VIDEO_EM28XX_ALSA=m
CONFIG_VIDEO_CX231XX=m
CONFIG_VIDEO_CX231XX_ALSA=m
CONFIG_VIDEO_USBVISION=m
CONFIG_USB_ET61X251=m
CONFIG_USB_SN9C102=m
CONFIG_USB_ZR364XX=m
CONFIG_USB_STKWEBCAM=m
CONFIG_USB_S2255=m
# CONFIG_V4L_MEM2MEM_DRIVERS is not set
# CONFIG_RADIO_ADAPTERS is not set
# CONFIG_DAB is not set

#
# Graphics support
#
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
CONFIG_AGP_INTEL=y
CONFIG_AGP_SIS=y
CONFIG_AGP_VIA=y
# CONFIG_VGA_ARB is not set
# CONFIG_VGA_SWITCHEROO is not set
# CONFIG_DRM is not set
# CONFIG_VGASTATE is not set
# CONFIG_VIDEO_OUTPUT_CONTROL is not set
CONFIG_FB=y
CONFIG_FIRMWARE_EDID=y
# CONFIG_FB_DDC is not set
CONFIG_FB_BOOT_VESA_SUPPORT=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
# CONFIG_FB_SYS_FILLRECT is not set
# CONFIG_FB_SYS_COPYAREA is not set
# CONFIG_FB_SYS_IMAGEBLIT is not set
# CONFIG_FB_FOREIGN_ENDIAN is not set
# CONFIG_FB_SYS_FOPS is not set
# CONFIG_FB_SVGALIB is not set
# CONFIG_FB_MACMODES is not set
# CONFIG_FB_BACKLIGHT is not set
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y

#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_UVESA is not set
CONFIG_FB_VESA=y
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_INTEL is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_VIA is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_CARMINE is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_TMIO is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_MB862XX is not set
# CONFIG_FB_BROADSHEET is not set
CONFIG_BACKLIGHT_LCD_SUPPORT=y
# CONFIG_LCD_CLASS_DEVICE is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=m
CONFIG_BACKLIGHT_GENERIC=m
CONFIG_BACKLIGHT_PROGEAR=m
CONFIG_BACKLIGHT_MBP_NVIDIA=m
CONFIG_BACKLIGHT_SAHARA=m
# CONFIG_BACKLIGHT_ADP8860 is not set

#
# Display device support
#
CONFIG_DISPLAY_SUPPORT=y

#
# Display hardware drivers
#

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=256
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
CONFIG_FONTS=y
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
CONFIG_FONT_6x11=y
CONFIG_FONT_7x14=y
CONFIG_FONT_PEARL_8x8=y
CONFIG_FONT_ACORN_8x8=y
CONFIG_FONT_MINI_4x6=y
CONFIG_FONT_SUN8x16=y
CONFIG_FONT_SUN12x22=y
CONFIG_FONT_10x18=y
CONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
# CONFIG_LOGO_LINUX_VGA16 is not set
CONFIG_LOGO_LINUX_CLUT224=y
CONFIG_SOUND=y
CONFIG_SOUND_OSS_CORE=y
CONFIG_SOUND_OSS_CORE_PRECLAIM=y
CONFIG_SND=y
CONFIG_SND_TIMER=y
CONFIG_SND_PCM=y
CONFIG_SND_HWDEP=y
CONFIG_SND_RAWMIDI=m
CONFIG_SND_JACK=y
CONFIG_SND_SEQUENCER=y
CONFIG_SND_SEQ_DUMMY=y
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=y
CONFIG_SND_PCM_OSS=y
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
# CONFIG_SND_HRTIMER is not set
CONFIG_SND_DYNAMIC_MINORS=y
CONFIG_SND_SUPPORT_OLD_API=y
# CONFIG_SND_VERBOSE_PROCFS is not set
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
CONFIG_SND_VMASTER=y
CONFIG_SND_DMA_SGBUF=y
CONFIG_SND_RAWMIDI_SEQ=m
# CONFIG_SND_OPL3_LIB_SEQ is not set
# CONFIG_SND_OPL4_LIB_SEQ is not set
# CONFIG_SND_SBAWE_SEQ is not set
# CONFIG_SND_EMU10K1_SEQ is not set
CONFIG_SND_MPU401_UART=m
CONFIG_SND_AC97_CODEC=m
CONFIG_SND_DRIVERS=y
# CONFIG_SND_PCSP is not set
# CONFIG_SND_DUMMY is not set
CONFIG_SND_VIRMIDI=m
CONFIG_SND_MTPAV=m
CONFIG_SND_MTS64=m
CONFIG_SND_SERIAL_U16550=m
CONFIG_SND_MPU401=m
CONFIG_SND_PORTMAN2X4=m
CONFIG_SND_AC97_POWER_SAVE=y
CONFIG_SND_AC97_POWER_SAVE_DEFAULT=30
CONFIG_SND_PCI=y
# CONFIG_SND_AD1889 is not set
# CONFIG_SND_ALS300 is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ASIHPI is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AW2 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CA0106 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_OXYGEN is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS5530 is not set
CONFIG_SND_CS5535AUDIO=m
# CONFIG_SND_CTXFI is not set
# CONFIG_SND_DARLA20 is not set
# CONFIG_SND_GINA20 is not set
# CONFIG_SND_LAYLA20 is not set
# CONFIG_SND_DARLA24 is not set
# CONFIG_SND_GINA24 is not set
# CONFIG_SND_LAYLA24 is not set
# CONFIG_SND_MONA is not set
# CONFIG_SND_MIA is not set
# CONFIG_SND_ECHO3G is not set
# CONFIG_SND_INDIGO is not set
# CONFIG_SND_INDIGOIO is not set
# CONFIG_SND_INDIGODJ is not set
# CONFIG_SND_INDIGOIOX is not set
# CONFIG_SND_INDIGODJX is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_EMU10K1X is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_FM801 is not set
CONFIG_SND_HDA_INTEL=y
CONFIG_SND_HDA_HWDEP=y
# CONFIG_SND_HDA_RECONFIG is not set
# CONFIG_SND_HDA_INPUT_BEEP is not set
CONFIG_SND_HDA_INPUT_JACK=y
# CONFIG_SND_HDA_PATCH_LOADER is not set
CONFIG_SND_HDA_CODEC_REALTEK=y
CONFIG_SND_HDA_CODEC_ANALOG=y
CONFIG_SND_HDA_CODEC_SIGMATEL=y
CONFIG_SND_HDA_CODEC_VIA=y
CONFIG_SND_HDA_CODEC_ATIHDMI=y
CONFIG_SND_HDA_CODEC_NVHDMI=y
CONFIG_SND_HDA_CODEC_INTELHDMI=y
CONFIG_SND_HDA_ELD=y
CONFIG_SND_HDA_CODEC_CIRRUS=y
CONFIG_SND_HDA_CODEC_CONEXANT=y
CONFIG_SND_HDA_CODEC_CA0110=y
CONFIG_SND_HDA_CODEC_CMEDIA=y
CONFIG_SND_HDA_CODEC_SI3054=y
CONFIG_SND_HDA_GENERIC=y
CONFIG_SND_HDA_POWER_SAVE=y
CONFIG_SND_HDA_POWER_SAVE_DEFAULT=30
# CONFIG_SND_HDSP is not set
# CONFIG_SND_HDSPM is not set
# CONFIG_SND_HIFIER is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
# CONFIG_SND_INTEL8X0 is not set
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_LX6464ES is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_PCXHR is not set
# CONFIG_SND_RIPTIDE is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VIA82XX_MODEM is not set
# CONFIG_SND_VIRTUOSO is not set
# CONFIG_SND_VX222 is not set
# CONFIG_SND_YMFPCI is not set
# CONFIG_SND_USB is not set
# CONFIG_SND_SOC is not set
# CONFIG_SOUND_PRIME is not set
CONFIG_AC97_BUS=m
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
CONFIG_HIDRAW=y

#
# USB Input Devices
#
CONFIG_USB_HID=y
CONFIG_HID_PID=y
CONFIG_USB_HIDDEV=y

#
# Special HID drivers
#
# CONFIG_HID_3M_PCT is not set
CONFIG_HID_A4TECH=y
# CONFIG_HID_ACRUX_FF is not set
CONFIG_HID_APPLE=y
CONFIG_HID_BELKIN=y
# CONFIG_HID_CANDO is not set
CONFIG_HID_CHERRY=y
CONFIG_HID_CHICONY=y
# CONFIG_HID_PRODIKEYS is not set
CONFIG_HID_CYPRESS=y
CONFIG_HID_DRAGONRISE=y
CONFIG_DRAGONRISE_FF=y
# CONFIG_HID_EGALAX is not set
CONFIG_HID_ELECOM=m
CONFIG_HID_EZKEY=y
CONFIG_HID_KYE=y
CONFIG_HID_GYRATION=y
CONFIG_HID_TWINHAN=y
CONFIG_HID_KENSINGTON=y
CONFIG_HID_LOGITECH=y
CONFIG_LOGITECH_FF=y
CONFIG_LOGIRUMBLEPAD2_FF=y
# CONFIG_LOGIG940_FF is not set
# CONFIG_HID_MAGICMOUSE is not set
CONFIG_HID_MICROSOFT=y
# CONFIG_HID_MOSART is not set
CONFIG_HID_MONTEREY=y
CONFIG_HID_NTRIG=y
# CONFIG_HID_ORTEK is not set
CONFIG_HID_PANTHERLORD=y
CONFIG_PANTHERLORD_FF=y
CONFIG_HID_PETALYNX=y
# CONFIG_HID_PICOLCD is not set
# CONFIG_HID_QUANTA is not set
# CONFIG_HID_ROCCAT is not set
# CONFIG_HID_ROCCAT_KONE is not set
CONFIG_HID_SAMSUNG=y
CONFIG_HID_SONY=y
# CONFIG_HID_STANTUM is not set
CONFIG_HID_SUNPLUS=y
CONFIG_HID_GREENASIA=y
CONFIG_GREENASIA_FF=y
CONFIG_HID_SMARTJOYPLUS=y
CONFIG_SMARTJOYPLUS_FF=y
CONFIG_HID_TOPSEED=y
CONFIG_HID_THRUSTMASTER=y
CONFIG_THRUSTMASTER_FF=y
CONFIG_HID_WACOM=m
# CONFIG_HID_WACOM_POWER_SUPPLY is not set
CONFIG_HID_ZEROPLUS=y
CONFIG_ZEROPLUS_FF=y
# CONFIG_HID_ZYDACRON is not set
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
CONFIG_USB_DEBUG=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_DEVICE_CLASS=y
CONFIG_USB_DYNAMIC_MINORS=y
CONFIG_USB_SUSPEND=y
# CONFIG_USB_OTG is not set
# CONFIG_USB_OTG_WHITELIST is not set
# CONFIG_USB_OTG_BLACKLIST_HUB is not set
CONFIG_USB_MON=y
CONFIG_USB_WUSB=y
CONFIG_USB_WUSB_CBAF=y
# CONFIG_USB_WUSB_CBAF_DEBUG is not set

#
# USB Host Controller Drivers
#
CONFIG_USB_C67X00_HCD=m
CONFIG_USB_XHCI_HCD=m
# CONFIG_USB_XHCI_HCD_DEBUGGING is not set
CONFIG_USB_EHCI_HCD=m
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_OXU210HP_HCD=m
CONFIG_USB_ISP116X_HCD=m
CONFIG_USB_ISP1760_HCD=m
CONFIG_USB_ISP1362_HCD=m
CONFIG_USB_OHCI_HCD=m
CONFIG_USB_OHCI_HCD_SSB=y
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_U132_HCD=m
CONFIG_USB_SL811_HCD=m
CONFIG_USB_R8A66597_HCD=m
CONFIG_USB_WHCI_HCD=m
CONFIG_USB_HWA_HCD=m

#
# USB Device Class drivers
#
CONFIG_USB_ACM=m
# CONFIG_USB_PRINTER is not set
CONFIG_USB_WDM=m
# CONFIG_USB_TMC is not set

#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#

#
# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_DATAFAB=m
CONFIG_USB_STORAGE_FREECOM=m
CONFIG_USB_STORAGE_ISD200=m
CONFIG_USB_STORAGE_USBAT=m
CONFIG_USB_STORAGE_SDDR09=m
CONFIG_USB_STORAGE_SDDR55=m
CONFIG_USB_STORAGE_JUMPSHOT=m
CONFIG_USB_STORAGE_ALAUDA=m
CONFIG_USB_STORAGE_ONETOUCH=m
CONFIG_USB_STORAGE_KARMA=m
CONFIG_USB_STORAGE_CYPRESS_ATACB=m
CONFIG_USB_LIBUSUAL=y

#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m

#
# USB port drivers
#
CONFIG_USB_USS720=m
CONFIG_USB_SERIAL=m
CONFIG_USB_EZUSB=y
# CONFIG_USB_SERIAL_GENERIC is not set
CONFIG_USB_SERIAL_AIRCABLE=m
CONFIG_USB_SERIAL_ARK3116=m
CONFIG_USB_SERIAL_BELKIN=m
CONFIG_USB_SERIAL_CH341=m
CONFIG_USB_SERIAL_WHITEHEAT=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
CONFIG_USB_SERIAL_CP210X=m
CONFIG_USB_SERIAL_CYPRESS_M8=m
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_FUNSOFT=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
CONFIG_USB_SERIAL_GARMIN=m
CONFIG_USB_SERIAL_IPW=m
CONFIG_USB_SERIAL_IUU=m
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
CONFIG_USB_SERIAL_KEYSPAN_MPR=y
CONFIG_USB_SERIAL_KEYSPAN_USA28=y
CONFIG_USB_SERIAL_KEYSPAN_USA28X=y
CONFIG_USB_SERIAL_KEYSPAN_USA28XA=y
CONFIG_USB_SERIAL_KEYSPAN_USA28XB=y
CONFIG_USB_SERIAL_KEYSPAN_USA19=y
CONFIG_USB_SERIAL_KEYSPAN_USA18X=y
CONFIG_USB_SERIAL_KEYSPAN_USA19W=y
CONFIG_USB_SERIAL_KEYSPAN_USA19QW=y
CONFIG_USB_SERIAL_KEYSPAN_USA19QI=y
CONFIG_USB_SERIAL_KEYSPAN_USA49W=y
CONFIG_USB_SERIAL_KEYSPAN_USA49WLC=y
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
CONFIG_USB_SERIAL_MOS7720=m
# CONFIG_USB_SERIAL_MOS7715_PARPORT is not set
CONFIG_USB_SERIAL_MOS7840=m
CONFIG_USB_SERIAL_MOTOROLA=m
CONFIG_USB_SERIAL_NAVMAN=m
CONFIG_USB_SERIAL_PL2303=m
CONFIG_USB_SERIAL_OTI6858=m
# CONFIG_USB_SERIAL_QCAUX is not set
CONFIG_USB_SERIAL_QUALCOMM=m
CONFIG_USB_SERIAL_SPCP8X5=m
CONFIG_USB_SERIAL_HP4X=m
CONFIG_USB_SERIAL_SAFE=m
CONFIG_USB_SERIAL_SAFE_PADDED=y
CONFIG_USB_SERIAL_SIEMENS_MPI=m
CONFIG_USB_SERIAL_SIERRAWIRELESS=m
CONFIG_USB_SERIAL_SYMBOL=m
CONFIG_USB_SERIAL_TI=m
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_XIRCOM=m
CONFIG_USB_SERIAL_WWAN=m
CONFIG_USB_SERIAL_OPTION=m
CONFIG_USB_SERIAL_OMNINET=m
CONFIG_USB_SERIAL_OPTICON=m
# CONFIG_USB_SERIAL_VIVOPAY_SERIAL is not set
# CONFIG_USB_SERIAL_ZIO is not set
CONFIG_USB_SERIAL_SSU100=m
# CONFIG_USB_SERIAL_DEBUG is not set

#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
CONFIG_USB_ADUTUX=m
CONFIG_USB_SEVSEG=m
CONFIG_USB_RIO500=m
# CONFIG_USB_LEGOTOWER is not set
CONFIG_USB_LCD=m
CONFIG_USB_LED=m
CONFIG_USB_CYPRESS_CY7C63=m
CONFIG_USB_CYTHERM=m
CONFIG_USB_IDMOUSE=m
CONFIG_USB_FTDI_ELAN=m
CONFIG_USB_APPLEDISPLAY=m
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
CONFIG_USB_ISIGHTFW=m
CONFIG_USB_ATM=m
CONFIG_USB_SPEEDTOUCH=m
CONFIG_USB_CXACRU=m
CONFIG_USB_UEAGLEATM=m
CONFIG_USB_XUSBATM=m
CONFIG_USB_GADGET=y
# CONFIG_USB_GADGET_DEBUG is not set
# CONFIG_USB_GADGET_DEBUG_FILES is not set
# CONFIG_USB_GADGET_DEBUG_FS is not set
CONFIG_USB_GADGET_VBUS_DRAW=2
CONFIG_USB_GADGET_SELECTED=y
# CONFIG_USB_GADGET_R8A66597 is not set
# CONFIG_USB_GADGET_M66592 is not set
# CONFIG_USB_GADGET_AMD5536UDC is not set
# CONFIG_USB_GADGET_CI13XXX is not set
# CONFIG_USB_GADGET_NET2280 is not set
# CONFIG_USB_GADGET_GOKU is not set
CONFIG_USB_GADGET_LANGWELL=y
CONFIG_USB_LANGWELL=y
# CONFIG_USB_GADGET_DUMMY_HCD is not set
CONFIG_USB_GADGET_DUALSPEED=y
CONFIG_USB_ZERO=m
CONFIG_USB_AUDIO=m
CONFIG_USB_ETH=m
CONFIG_USB_ETH_RNDIS=y
# CONFIG_USB_ETH_EEM is not set
CONFIG_USB_GADGETFS=m
# CONFIG_USB_FUNCTIONFS is not set
CONFIG_USB_FILE_STORAGE=m
# CONFIG_USB_FILE_STORAGE_TEST is not set
CONFIG_USB_MASS_STORAGE=m
CONFIG_USB_G_SERIAL=m
CONFIG_USB_MIDI_GADGET=m
CONFIG_USB_G_PRINTER=m
CONFIG_USB_CDC_COMPOSITE=m
# CONFIG_USB_G_NOKIA is not set
CONFIG_USB_G_MULTI=m
CONFIG_USB_G_MULTI_RNDIS=y
CONFIG_USB_G_MULTI_CDC=y
# CONFIG_USB_G_HID is not set
# CONFIG_USB_G_DBGP is not set
# CONFIG_USB_G_WEBCAM is not set

#
# OTG and related infrastructure
#
CONFIG_USB_OTG_UTILS=y
CONFIG_NOP_USB_XCEIV=m
CONFIG_UWB=y
CONFIG_UWB_HWA=m
CONFIG_UWB_WHCI=m
CONFIG_UWB_WLP=m
CONFIG_UWB_I1480U=m
CONFIG_UWB_I1480U_WLP=m
CONFIG_MMC=m
# CONFIG_MMC_DEBUG is not set
# CONFIG_MMC_UNSAFE_RESUME is not set

#
# MMC/SD/SDIO Card Drivers
#
CONFIG_MMC_BLOCK=m
CONFIG_MMC_BLOCK_BOUNCE=y
CONFIG_SDIO_UART=m
# CONFIG_MMC_TEST is not set

#
# MMC/SD/SDIO Host Controller Drivers
#
CONFIG_MMC_SDHCI=m
CONFIG_MMC_SDHCI_PCI=m
# CONFIG_MMC_RICOH_MMC is not set
CONFIG_MMC_SDHCI_PLTFM=m
CONFIG_MMC_WBSD=m
CONFIG_MMC_TIFM_SD=m
CONFIG_MMC_CB710=m
CONFIG_MMC_VIA_SDMMC=m
CONFIG_MEMSTICK=m
# CONFIG_MEMSTICK_DEBUG is not set

#
# MemoryStick drivers
#
# CONFIG_MEMSTICK_UNSAFE_RESUME is not set
CONFIG_MSPRO_BLOCK=m

#
# MemoryStick Host Controller Drivers
#
CONFIG_MEMSTICK_TIFM_MS=m
CONFIG_MEMSTICK_JMICRON_38X=m
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=m

#
# LED drivers
#
CONFIG_LEDS_ALIX2=m
CONFIG_LEDS_PCA9532=m
CONFIG_LEDS_LP3944=m
# CONFIG_LEDS_CLEVO_MAIL is not set
CONFIG_LEDS_PCA955X=m
CONFIG_LEDS_BD2802=m
CONFIG_LEDS_INTEL_SS4200=m
# CONFIG_LEDS_DELL_NETBOOKS is not set
CONFIG_LEDS_TRIGGERS=y

#
# LED Triggers
#
CONFIG_LEDS_TRIGGER_TIMER=m
CONFIG_LEDS_TRIGGER_HEARTBEAT=m
# CONFIG_LEDS_TRIGGER_BACKLIGHT is not set
CONFIG_LEDS_TRIGGER_DEFAULT_ON=m

#
# iptables trigger is under Netfilter config (LED target)
#
# CONFIG_ACCESSIBILITY is not set
# CONFIG_INFINIBAND is not set
CONFIG_EDAC=y

#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_DECODE_MCE=y
CONFIG_EDAC_MM_EDAC=y
CONFIG_EDAC_MCE=y
CONFIG_EDAC_AMD64=m
# CONFIG_EDAC_AMD64_ERROR_INJECTION is not set
CONFIG_EDAC_E752X=m
CONFIG_EDAC_I82975X=m
CONFIG_EDAC_I3000=m
CONFIG_EDAC_I3200=m
CONFIG_EDAC_X38=m
CONFIG_EDAC_I5400=m
CONFIG_EDAC_I7CORE=m
CONFIG_EDAC_I5000=m
CONFIG_EDAC_I5100=m
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
# CONFIG_RTC_DEBUG is not set

#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
# CONFIG_RTC_DRV_TEST is not set

#
# I2C RTC drivers
#
CONFIG_RTC_DRV_DS1307=m
CONFIG_RTC_DRV_DS1374=m
CONFIG_RTC_DRV_DS1672=m
CONFIG_RTC_DRV_DS3232=m
CONFIG_RTC_DRV_MAX6900=m
CONFIG_RTC_DRV_RS5C372=m
CONFIG_RTC_DRV_ISL1208=m
CONFIG_RTC_DRV_ISL12022=m
CONFIG_RTC_DRV_X1205=m
CONFIG_RTC_DRV_PCF8563=m
CONFIG_RTC_DRV_PCF8583=m
CONFIG_RTC_DRV_M41T80=m
CONFIG_RTC_DRV_M41T80_WDT=y
CONFIG_RTC_DRV_BQ32K=m
CONFIG_RTC_DRV_S35390A=m
CONFIG_RTC_DRV_FM3130=m
CONFIG_RTC_DRV_RX8581=m
CONFIG_RTC_DRV_RX8025=m

#
# SPI RTC drivers
#

#
# Platform RTC drivers
#
CONFIG_RTC_DRV_CMOS=y
CONFIG_RTC_DRV_DS1286=m
CONFIG_RTC_DRV_DS1511=m
CONFIG_RTC_DRV_DS1553=m
CONFIG_RTC_DRV_DS1742=m
CONFIG_RTC_DRV_STK17TA8=m
CONFIG_RTC_DRV_M48T86=m
CONFIG_RTC_DRV_M48T35=m
CONFIG_RTC_DRV_M48T59=m
CONFIG_RTC_DRV_MSM6242=m
CONFIG_RTC_DRV_BQ4802=m
CONFIG_RTC_DRV_RP5C01=m
CONFIG_RTC_DRV_V3020=m

#
# on-CPU RTC drivers
#
CONFIG_DMADEVICES=y
# CONFIG_DMADEVICES_DEBUG is not set

#
# DMA Devices
#
CONFIG_INTEL_MID_DMAC=m
CONFIG_ASYNC_TX_DISABLE_CHANNEL_SWITCH=y
CONFIG_INTEL_IOATDMA=y
CONFIG_TIMB_DMA=m
CONFIG_PCH_DMA=m
CONFIG_DMA_ENGINE=y

#
# DMA Clients
#
CONFIG_NET_DMA=y
# CONFIG_ASYNC_TX_DMA is not set
# CONFIG_DMATEST is not set
CONFIG_DCA=y
# CONFIG_AUXDISPLAY is not set
CONFIG_UIO=m
# CONFIG_UIO_CIF is not set
CONFIG_UIO_PDRV=m
CONFIG_UIO_PDRV_GENIRQ=m
CONFIG_UIO_AEC=m
CONFIG_UIO_SERCOS3=m
CONFIG_UIO_PCI_GENERIC=m
# CONFIG_UIO_NETX is not set
CONFIG_STAGING=y
# CONFIG_STAGING_EXCLUDE_BUILD is not set
# CONFIG_ET131X is not set
# CONFIG_SLICOSS is not set
# CONFIG_VIDEO_GO7007 is not set
# CONFIG_VIDEO_TM6000 is not set
# CONFIG_USB_IP_COMMON is not set
# CONFIG_W35UND is not set
# CONFIG_PRISM2_USB is not set
# CONFIG_ECHO is not set
# CONFIG_OTUS is not set
# CONFIG_RT2860 is not set
# CONFIG_RT2870 is not set
# CONFIG_COMEDI is not set
# CONFIG_ASUS_OLED is not set
# CONFIG_PANEL is not set
# CONFIG_R8187SE is not set
# CONFIG_RTL8192SU is not set
# CONFIG_RTL8192U is not set
# CONFIG_RTL8192E is not set
# CONFIG_TRANZPORT is not set
# CONFIG_POHMELFS is not set
# CONFIG_IDE_PHISON is not set
# CONFIG_LINE6_USB is not set
# CONFIG_USB_SERIAL_QUATECH2 is not set
# CONFIG_USB_SERIAL_QUATECH_USB2 is not set
# CONFIG_VT6655 is not set
# CONFIG_VT6656 is not set
# CONFIG_FB_UDL is not set
# CONFIG_HYPERV is not set
# CONFIG_VME_BUS is not set
# CONFIG_IIO is not set
CONFIG_ZRAM=m
CONFIG_ZRAM_STATS=y
# CONFIG_BATMAN_ADV is not set
# CONFIG_SAMSUNG_LAPTOP is not set
# CONFIG_FB_SM7XX is not set
# CONFIG_VIDEO_DT3155 is not set
# CONFIG_CRYSTALHD is not set

#
# Texas Instruments shared transport line discipline
#
# CONFIG_TI_ST is not set
# CONFIG_ST_BT is not set
# CONFIG_FB_XGI is not set
# CONFIG_LIRC_STAGING is not set
# CONFIG_EASYCAP is not set
# CONFIG_SOLO6X10 is not set
# CONFIG_ACPI_QUICKSTART is not set
CONFIG_X86_PLATFORM_DEVICES=y
# CONFIG_ACER_WMI is not set
# CONFIG_ASUS_LAPTOP is not set
CONFIG_DELL_WMI=m
# CONFIG_FUJITSU_LAPTOP is not set
# CONFIG_HP_WMI is not set
# CONFIG_MSI_LAPTOP is not set
# CONFIG_PANASONIC_LAPTOP is not set
# CONFIG_COMPAL_LAPTOP is not set
# CONFIG_SONY_LAPTOP is not set
# CONFIG_IDEAPAD_ACPI is not set
# CONFIG_THINKPAD_ACPI is not set
CONFIG_INTEL_MENLOW=m
# CONFIG_EEEPC_LAPTOP is not set
# CONFIG_EEEPC_WMI is not set
CONFIG_ACPI_WMI=m
# CONFIG_MSI_WMI is not set
# CONFIG_ACPI_ASUS is not set
# CONFIG_TOPSTAR_LAPTOP is not set
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_TOSHIBA_BT_RFKILL=m
CONFIG_ACPI_CMPC=m
CONFIG_INTEL_IPS=m

#
# Firmware Drivers
#
# CONFIG_EDD is not set
# CONFIG_FIRMWARE_MEMMAP is not set
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
CONFIG_DMIID=y
# CONFIG_ISCSI_IBFT_FIND is not set

#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
# CONFIG_EXT2_FS_XIP is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_XATTR=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_DEBUG is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_JBD2=y
# CONFIG_JBD2_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=y
# CONFIG_REISERFS_CHECK is not set
CONFIG_REISERFS_PROC_INFO=y
CONFIG_REISERFS_FS_XATTR=y
CONFIG_REISERFS_FS_POSIX_ACL=y
CONFIG_REISERFS_FS_SECURITY=y
CONFIG_JFS_FS=y
CONFIG_JFS_POSIX_ACL=y
CONFIG_JFS_SECURITY=y
# CONFIG_JFS_DEBUG is not set
CONFIG_JFS_STATISTICS=y
CONFIG_FS_POSIX_ACL=y
CONFIG_XFS_FS=y
CONFIG_XFS_QUOTA=y
CONFIG_XFS_POSIX_ACL=y
CONFIG_XFS_RT=y
# CONFIG_XFS_DEBUG is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
CONFIG_BTRFS_FS=y
# CONFIG_BTRFS_FS_POSIX_ACL is not set
CONFIG_NILFS2_FS=m
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
# CONFIG_FANOTIFY is not set
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
# CONFIG_PRINT_QUOTA_WARNING is not set
# CONFIG_QUOTA_DEBUG is not set
CONFIG_QUOTA_TREE=y
# CONFIG_QFMT_V1 is not set
CONFIG_QFMT_V2=y
CONFIG_QUOTACTL=y
CONFIG_QUOTACTL_COMPAT=y
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
CONFIG_FUSE_FS=y
CONFIG_CUSE=y
CONFIG_GENERIC_ACL=y

#
# Caches
#
# CONFIG_FSCACHE is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-15"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_PROC_KCORE is not set
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_CONFIGFS_FS=y
CONFIG_MISC_FILESYSTEMS=y
CONFIG_ADFS_FS=m
# CONFIG_ADFS_FS_RW is not set
CONFIG_AFFS_FS=m
# CONFIG_ECRYPT_FS is not set
CONFIG_HFS_FS=m
CONFIG_HFSPLUS_FS=m
CONFIG_BEFS_FS=m
# CONFIG_BEFS_DEBUG is not set
CONFIG_BFS_FS=m
CONFIG_EFS_FS=m
# CONFIG_LOGFS is not set
CONFIG_CRAMFS=m
CONFIG_SQUASHFS=y
# CONFIG_SQUASHFS_XATTR is not set
# CONFIG_SQUASHFS_LZO is not set
# CONFIG_SQUASHFS_EMBEDDED is not set
CONFIG_SQUASHFS_FRAGMENT_CACHE_SIZE=3
CONFIG_VXFS_FS=m
CONFIG_MINIX_FS=m
CONFIG_OMFS_FS=m
CONFIG_HPFS_FS=m
CONFIG_QNX4FS_FS=m
CONFIG_ROMFS_FS=m
CONFIG_ROMFS_BACKED_BY_BLOCK=y
CONFIG_ROMFS_ON_BLOCK=y
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
# CONFIG_UFS_FS_WRITE is not set
# CONFIG_UFS_DEBUG is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
# CONFIG_NFS_V4_1 is not set
CONFIG_ROOT_NFS=y
# CONFIG_NFS_USE_LEGACY_DNS is not set
CONFIG_NFS_USE_KERNEL_DNS=y
# CONFIG_NFSD is not set
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=y
CONFIG_NFS_ACL_SUPPORT=y
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=y
CONFIG_RPCSEC_GSS_KRB5=y
CONFIG_RPCSEC_GSS_SPKM3=m
# CONFIG_SMB_FS is not set
# CONFIG_CEPH_FS is not set
CONFIG_CIFS=m
CONFIG_CIFS_STATS=y
CONFIG_CIFS_STATS2=y
# CONFIG_CIFS_WEAK_PW_HASH is not set
# CONFIG_CIFS_UPCALL is not set
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_DFS_UPCALL is not set
CONFIG_CIFS_EXPERIMENTAL=y
CONFIG_NCP_FS=m
CONFIG_NCPFS_PACKET_SIGNING=y
# CONFIG_NCPFS_IOCTL_LOCKING is not set
CONFIG_NCPFS_STRONG=y
CONFIG_NCPFS_NFS_NS=y
CONFIG_NCPFS_OS2_NS=y
# CONFIG_NCPFS_SMALLDOS is not set
CONFIG_NCPFS_NLS=y
CONFIG_NCPFS_EXTRAS=y
CONFIG_CODA_FS=m
# CONFIG_AFS_FS is not set

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
CONFIG_ACORN_PARTITION=y
CONFIG_ACORN_PARTITION_CUMANA=y
CONFIG_ACORN_PARTITION_EESOX=y
CONFIG_ACORN_PARTITION_ICS=y
CONFIG_ACORN_PARTITION_ADFS=y
CONFIG_ACORN_PARTITION_POWERTEC=y
CONFIG_ACORN_PARTITION_RISCIX=y
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
CONFIG_ATARI_PARTITION=y
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_LDM_PARTITION=y
# CONFIG_LDM_DEBUG is not set
CONFIG_SGI_PARTITION=y
CONFIG_ULTRIX_PARTITION=y
CONFIG_SUN_PARTITION=y
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
CONFIG_SYSV68_PARTITION=y
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="utf8"
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=y
CONFIG_NLS_CODEPAGE_950=y
CONFIG_NLS_CODEPAGE_932=y
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=y
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=y
CONFIG_DLM=m
# CONFIG_DLM_DEBUG is not set

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_PRINTK_TIME=y
CONFIG_ENABLE_WARN_DEPRECATED=y
CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_WARN=2048
CONFIG_MAGIC_SYSRQ=y
CONFIG_STRIP_ASM_SYMS=y
CONFIG_UNUSED_SYMBOLS=y
CONFIG_DEBUG_FS=y
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_SHIRQ=y
CONFIG_LOCKUP_DETECTOR=y
CONFIG_HARDLOCKUP_DETECTOR=y
# CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
CONFIG_DETECT_HUNG_TASK=y
# CONFIG_BOOTPARAM_HUNG_TASK_PANIC is not set
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_KMEMLEAK is not set
# CONFIG_DEBUG_PREEMPT is not set
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_PI_LIST=y
# CONFIG_RT_MUTEX_TESTER is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
# CONFIG_DEBUG_LOCK_ALLOC is not set
# CONFIG_PROVE_LOCKING is not set
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_VIRTUAL is not set
# CONFIG_DEBUG_WRITECOUNT is not set
CONFIG_DEBUG_MEMORY_INIT=y
# CONFIG_DEBUG_LIST is not set
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
# CONFIG_DEBUG_CREDENTIALS is not set
CONFIG_ARCH_WANT_FRAME_POINTERS=y
# CONFIG_FRAME_POINTER is not set
# CONFIG_BOOT_PRINTK_DELAY is not set
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_RCU_CPU_STALL_DETECTOR is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
# CONFIG_LKDTM is not set
# CONFIG_CPU_NOTIFIER_ERROR_INJECT is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_LATENCYTOP is not set
CONFIG_SYSCTL_SYSCALL_CHECK=y
# CONFIG_DEBUG_PAGEALLOC is not set
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_TRACING_SUPPORT=y
# CONFIG_FTRACE is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_FIREWIRE_OHCI_REMOTE_DMA is not set
# CONFIG_DYNAMIC_DEBUG is not set
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_ATOMIC64_SELFTEST is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
CONFIG_HAVE_ARCH_KMEMCHECK=y
# CONFIG_KMEMCHECK is not set
CONFIG_STRICT_DEVMEM=y
CONFIG_X86_VERBOSE_BOOTUP=y
# CONFIG_EARLY_PRINTK is not set
CONFIG_DEBUG_STACKOVERFLOW=y
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PER_CPU_MAPS is not set
# CONFIG_X86_PTDUMP is not set
CONFIG_DEBUG_RODATA=y
# CONFIG_DEBUG_RODATA_TEST is not set
# CONFIG_DEBUG_NX_TEST is not set
# CONFIG_IOMMU_DEBUG is not set
# CONFIG_IOMMU_STRESS is not set
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=0
# CONFIG_DEBUG_BOOT_PARAMS is not set
# CONFIG_CPA_DEBUG is not set
CONFIG_OPTIMIZE_INLINING=y
CONFIG_DEBUG_STRICT_USER_COPY_CHECKS=y

#
# Security options
#
CONFIG_KEYS=y
CONFIG_KEYS_DEBUG_PROC_KEYS=y
# CONFIG_SECURITY is not set
# CONFIG_SECURITYFS is not set
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_DEFAULT_SECURITY=""
CONFIG_XOR_BLOCKS=y
CONFIG_ASYNC_CORE=y
CONFIG_ASYNC_MEMCPY=y
CONFIG_ASYNC_XOR=y
CONFIG_ASYNC_PQ=y
CONFIG_ASYNC_RAID6_RECOV=y
# CONFIG_ASYNC_RAID6_TEST is not set
CONFIG_ASYNC_TX_DISABLE_PQ_VAL_DMA=y
CONFIG_ASYNC_TX_DISABLE_XOR_VAL_DMA=y
CONFIG_CRYPTO=y

#
# Crypto core or helper
#
CONFIG_CRYPTO_FIPS=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD=y
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=y
CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG=y
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_PCOMP=y
CONFIG_CRYPTO_PCOMP2=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
CONFIG_CRYPTO_GF128MUL=y
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_PCRYPT=y
CONFIG_CRYPTO_WORKQUEUE=y
CONFIG_CRYPTO_CRYPTD=y
CONFIG_CRYPTO_AUTHENC=y
CONFIG_CRYPTO_TEST=m

#
# Authenticated Encryption with Associated Data
#
CONFIG_CRYPTO_CCM=y
CONFIG_CRYPTO_GCM=y
CONFIG_CRYPTO_SEQIV=y

#
# Block modes
#
CONFIG_CRYPTO_CBC=y
CONFIG_CRYPTO_CTR=y
CONFIG_CRYPTO_CTS=y
CONFIG_CRYPTO_ECB=y
CONFIG_CRYPTO_LRW=y
CONFIG_CRYPTO_PCBC=y
CONFIG_CRYPTO_XTS=y
CONFIG_CRYPTO_FPU=y

#
# Hash modes
#
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_XCBC=y
CONFIG_CRYPTO_VMAC=y

#
# Digest
#
CONFIG_CRYPTO_CRC32C=y
CONFIG_CRYPTO_CRC32C_INTEL=y
CONFIG_CRYPTO_GHASH=y
CONFIG_CRYPTO_MD4=y
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=y
CONFIG_CRYPTO_RMD128=y
CONFIG_CRYPTO_RMD160=y
CONFIG_CRYPTO_RMD256=y
CONFIG_CRYPTO_RMD320=y
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=y
CONFIG_CRYPTO_TGR192=y
CONFIG_CRYPTO_WP512=y
CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL=y

#
# Ciphers
#
CONFIG_CRYPTO_AES=y
CONFIG_CRYPTO_AES_X86_64=y
CONFIG_CRYPTO_AES_NI_INTEL=y
CONFIG_CRYPTO_ANUBIS=y
CONFIG_CRYPTO_ARC4=y
CONFIG_CRYPTO_BLOWFISH=y
CONFIG_CRYPTO_CAMELLIA=y
CONFIG_CRYPTO_CAST5=y
CONFIG_CRYPTO_CAST6=y
CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_FCRYPT=y
CONFIG_CRYPTO_KHAZAD=y
CONFIG_CRYPTO_SALSA20=y
CONFIG_CRYPTO_SALSA20_X86_64=y
CONFIG_CRYPTO_SEED=y
CONFIG_CRYPTO_SERPENT=y
CONFIG_CRYPTO_TEA=y
CONFIG_CRYPTO_TWOFISH=y
CONFIG_CRYPTO_TWOFISH_COMMON=y
CONFIG_CRYPTO_TWOFISH_X86_64=y

#
# Compression
#
CONFIG_CRYPTO_DEFLATE=y
CONFIG_CRYPTO_ZLIB=y
CONFIG_CRYPTO_LZO=y

#
# Random Number Generation
#
CONFIG_CRYPTO_ANSI_CPRNG=m
CONFIG_CRYPTO_HW=y
CONFIG_CRYPTO_DEV_PADLOCK=m
CONFIG_CRYPTO_DEV_PADLOCK_AES=m
CONFIG_CRYPTO_DEV_PADLOCK_SHA=m
CONFIG_CRYPTO_DEV_HIFN_795X=m
CONFIG_CRYPTO_DEV_HIFN_795X_RNG=y
CONFIG_HAVE_KVM=y
CONFIG_VIRTUALIZATION=y
# CONFIG_KVM is not set
# CONFIG_VHOST_NET is not set
# CONFIG_VIRTIO_PCI is not set
# CONFIG_VIRTIO_BALLOON is not set
# CONFIG_BINARY_PRINTF is not set

#
# Library routines
#
CONFIG_RAID6_PQ=y
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_GENERIC_FIND_LAST_BIT=y
CONFIG_CRC_CCITT=y
CONFIG_CRC16=y
CONFIG_CRC_T10DIF=y
CONFIG_CRC_ITU_T=y
CONFIG_CRC32=y
CONFIG_CRC7=y
CONFIG_LIBCRC32C=y
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_LZO=y
CONFIG_TEXTSEARCH=y
CONFIG_TEXTSEARCH_KMP=m
CONFIG_TEXTSEARCH_BM=m
CONFIG_TEXTSEARCH_FSM=m
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_NLATTR=y

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

On Sun, Dec 05, 2010 at 12:47:11AM +0100, Matt wrote:
> > OK.
> 
> meanwhile I think I got some interesting news:
> 
> after some time of running (around 1 to 1.5 hours) I noticed the
> following BUG with ext4:
> 
> [ 4421.503477] ------------[ cut here ]------------
> [ 4421.503482] kernel BUG at fs/ext4/inode.c:2714!
>
> kernel compiled was from sources checked out at
> 1de3e3df917459422cb2aecac440febc8879d410

Looking at 1de3e3df917459422cb2aecac440febc8879d410:

Line 2714 in fs/ext4/inode.c is this:

       /*
         * If the page does not have buffers (for whatever reason),
         * try to create them using block_prepare_write.  If this
         * fails, redirty the page and move on.
         */
        if (!page_buffers(page)) {
	^^^^^^^^^^^^^^^^^^^^^^^^^^^
                if (block_prepare_write(page, 0, len,
                                        noalloc_get_block_write)) {
                redirty_page:
                        redirty_page_for_writepage(wbc, page);
                        unlock_page(page);
                        return 0;
                }
                commit_write = 1;
        }

Which means we're really hitting this:

/* If we *know* page->private refers to buffer_heads */
#define page_buffers(page)                                      \
        ({                                                      \
                BUG_ON(!PagePrivate(page));                     \
		^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                ((struct buffer_head *)page_private(page));     \
        })
#define page_has_buffers(page)  PagePrivate(page)

Looks like Ted fixed it here:

commit b1142e8fec6a594723e5054055a7b53379b90490
Author: Theodore Ts'o <tytso@mit.edu>
Date:   Thu Oct 28 17:33:57 2010 -0400

    ext4: BUG_ON fix: check if page has buffers before calling page_buffers()


Basically, once you hit this oops, ext4 is done.  No files you created
after the oops will be there when you reboot, and the rest of your
lockups etc are because the jbd process had some locks held when it
crashed.

Was there also a report of corruption w/dm-crypt and XFS?

Last night I ran dm-crypt + the cpu scalability patch + ext4 +
2.6.37-rc3 in a long stress, and it passed without any problems.  If
dm-crypt were not doing the IO properly, this test probably would have
found it (+/- strange block sizes, races with O_DIRECT and other exotic
fun).

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

I finally found some time to test this out. With 2.6.37-rc4 (openSUSE
KOTD kernel) I easily encounter the issue.

Using a virtual machine, I created a stock, minimal openSUSE 11.3 x86_64
install, installed all updates, installed postgresql and the 'KOTD'
(Kernel of the Day)
kernel, and ran the following tests (as postgres user because I'm
lazy).

1. create a database (from bash):

createdb test

2. place the following contents in a file (I used 't.sql'):

begin;
create temporary table foo as select x as a, ARRAY[x] as b FROM
generate_series(1, 10000000 ) AS x;
create index foo_a_idx on foo (a);
create index foo_b_idx on foo USING GIN (b);
rollback;

3. execute that sql:

psql -f t.sql --echo-all test


With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
without issue.

With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
pretty frequently.

Then I tested with the 'vanilla' kernel available here:

http://download.opensuse.org/repositories/Kernel:/vanilla/standard/

The 'vanilla' kernel exhibited the same problems.
The version I tested:  2.6.37-rc4-219-g771f8bc-vanilla.

Incidentally, quick tests of jfs, xfs, and ext3 do _not_ show the same
problems, although I will note that I usually saw failure at least 1
in 3, but sometimes had to re-run the sql test 4 or 5 times before I
saw failure.

I will continue to do some testing, but I will hold off on testing the
commits above until I receive further testing suggestions.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Jon Nelson's message of 2010-12-07 13:10:49 -0500:
> I finally found some time to test this out. With 2.6.37-rc4 (openSUSE
> KOTD kernel) I easily encounter the issue.
> 
> Using a virtual machine, I created a stock, minimal openSUSE 11.3 x86_64
> install, installed all updates, installed postgresql and the 'KOTD'
> (Kernel of the Day)
> kernel, and ran the following tests (as postgres user because I'm
> lazy).
> 
> 1. create a database (from bash):
> 
> createdb test
> 
> 2. place the following contents in a file (I used 't.sql'):
> 
> begin;
> create temporary table foo as select x as a, ARRAY[x] as b FROM
> generate_series(1, 10000000 ) AS x;
> create index foo_a_idx on foo (a);
> create index foo_b_idx on foo USING GIN (b);
> rollback;
> 
> 3. execute that sql:
> 
> psql -f t.sql --echo-all test
> 
> 
> With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
> without issue.
> 
> With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
> pretty frequently.
> 
> Then I tested with the 'vanilla' kernel available here:
> 
> http://download.opensuse.org/repositories/Kernel:/vanilla/standard/
> 
> The 'vanilla' kernel exhibited the same problems.
> The version I tested:  2.6.37-rc4-219-g771f8bc-vanilla.
> 
> Incidentally, quick tests of jfs, xfs, and ext3 do _not_ show the same
> problems, although I will note that I usually saw failure at least 1
> in 3, but sometimes had to re-run the sql test 4 or 5 times before I
> saw failure.
> 
> I will continue to do some testing, but I will hold off on testing the
> commits above until I receive further testing suggestions.
> 

Sorry if you already detailed this in another message, could you please
give information about how it fails?  Please include any kernel messages
(or point me at the past messages where you had them).

thanks
Chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Mike Snitzer]

On Tue, Dec 07 2010 at  1:10pm -0500,
Jon Nelson <jnelson@jamponi.net> wrote:

> I finally found some time to test this out. With 2.6.37-rc4 (openSUSE
> KOTD kernel) I easily encounter the issue.
> 
> Using a virtual machine, I created a stock, minimal openSUSE 11.3 x86_64
> install, installed all updates, installed postgresql and the 'KOTD'
> (Kernel of the Day)
> kernel, and ran the following tests (as postgres user because I'm
> lazy).
> 
> 1. create a database (from bash):
> 
> createdb test
> 
> 2. place the following contents in a file (I used 't.sql'):
> 
> begin;
> create temporary table foo as select x as a, ARRAY[x] as b FROM
> generate_series(1, 10000000 ) AS x;
> create index foo_a_idx on foo (a);
> create index foo_b_idx on foo USING GIN (b);
> rollback;
> 
> 3. execute that sql:
> 
> psql -f t.sql --echo-all test
> 
> 
> With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
> without issue.
> 
> With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
> pretty frequently.

How does it fail?  postgres errors?  kernel errors?
 
> Then I tested with the 'vanilla' kernel available here:
> 
> http://download.opensuse.org/repositories/Kernel:/vanilla/standard/
> 
> The 'vanilla' kernel exhibited the same problems.
> The version I tested:  2.6.37-rc4-219-g771f8bc-vanilla.
>
> Incidentally, quick tests of jfs, xfs, and ext3 do _not_ show the same
> problems, although I will note that I usually saw failure at least 1
> in 3, but sometimes had to re-run the sql test 4 or 5 times before I
> saw failure.
> 
> I will continue to do some testing, but I will hold off on testing the
> commits above until I receive further testing suggestions.

OK, so to be clear: your testing is on dm-crypt + ext4?

And you're testing upstream based kernels (meaning the dm-crypt
scalability patch that has been in question is _not_ in the mix)?

That is a very different report than the other 2 reports of corruption
(those corruption instances were only seen once the dm-crypt scalability
patch was introduced).

Mike

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 12:22 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Tue, Dec 07 2010 at  1:10pm -0500,
> Jon Nelson <jnelson@jamponi.net> wrote:
>
>> I finally found some time to test this out. With 2.6.37-rc4 (openSUSE
>> KOTD kernel) I easily encounter the issue.
>>
>> Using a virtual machine, I created a stock, minimal openSUSE 11.3 x86_64
>> install, installed all updates, installed postgresql and the 'KOTD'
>> (Kernel of the Day)
>> kernel, and ran the following tests (as postgres user because I'm
>> lazy).
>>
>> 1. create a database (from bash):
>>
>> createdb test
>>
>> 2. place the following contents in a file (I used 't.sql'):
>>
>> begin;
>> create temporary table foo as select x as a, ARRAY[x] as b FROM
>> generate_series(1, 10000000 ) AS x;
>> create index foo_a_idx on foo (a);
>> create index foo_b_idx on foo USING GIN (b);
>> rollback;
>>
>> 3. execute that sql:
>>
>> psql -f t.sql --echo-all test
>>
>>
>> With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
>> without issue.
>>
>> With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
>> pretty frequently.
>
> How does it fail?  postgres errors?  kernel errors?

postgresql errors. Typically, header corruption but from the limited
visibility I've had into this via strace, what I see is zeroed pages
where there shouldn't be.

I just ran a test and got:

ERROR:  invalid page header in block 37483 of relation base/16384/16417

but that is not the only error one might get.

>> Then I tested with the 'vanilla' kernel available here:
>>
>> http://download.opensuse.org/repositories/Kernel:/vanilla/standard/
>>
>> The 'vanilla' kernel exhibited the same problems.
>> The version I tested:  2.6.37-rc4-219-g771f8bc-vanilla.
>>
>> Incidentally, quick tests of jfs, xfs, and ext3 do _not_ show the same
>> problems, although I will note that I usually saw failure at least 1
>> in 3, but sometimes had to re-run the sql test 4 or 5 times before I
>> saw failure.
>>
>> I will continue to do some testing, but I will hold off on testing the
>> commits above until I receive further testing suggestions.
>
> OK, so to be clear: your testing is on dm-crypt + ext4?

Yes. I took a virtual hard disk which shows up as /dev/sdb, used
cryptsetup to format it as a LUKS volume, mounted the LUKS volume,
formatted as ext4 (or whatever), mounted that, rsync'd over a blank
postgresql 'data' directory, started postgresql, became the postgres
user, and proceeded to create the db and run the sql.

> And you're testing upstream based kernels (meaning the dm-crypt
> scalability patch that has been in question is _not_ in the mix)?

I am testing both the KOTD kernels and the "vanilla" kernels - neither
of which has the dm-crypt patches (as far as I know).

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Jon Nelson's message of 2010-12-07 13:45:14 -0500:
> On Tue, Dec 7, 2010 at 12:22 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> > On Tue, Dec 07 2010 at  1:10pm -0500,
> > Jon Nelson <jnelson@jamponi.net> wrote:
> >
> >> I finally found some time to test this out. With 2.6.37-rc4 (openSUSE
> >> KOTD kernel) I easily encounter the issue.
> >>
> >> Using a virtual machine, I created a stock, minimal openSUSE 11.3 x86_64
> >> install, installed all updates, installed postgresql and the 'KOTD'
> >> (Kernel of the Day)
> >> kernel, and ran the following tests (as postgres user because I'm
> >> lazy).
> >>
> >> 1. create a database (from bash):
> >>
> >> createdb test
> >>
> >> 2. place the following contents in a file (I used 't.sql'):
> >>
> >> begin;
> >> create temporary table foo as select x as a, ARRAY[x] as b FROM
> >> generate_series(1, 10000000 ) AS x;
> >> create index foo_a_idx on foo (a);
> >> create index foo_b_idx on foo USING GIN (b);
> >> rollback;
> >>
> >> 3. execute that sql:
> >>
> >> psql -f t.sql --echo-all test
> >>
> >>
> >> With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
> >> without issue.
> >>
> >> With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
> >> pretty frequently.
> >
> > How does it fail?  postgres errors?  kernel errors?
> 
> postgresql errors. Typically, header corruption but from the limited
> visibility I've had into this via strace, what I see is zeroed pages
> where there shouldn't be.

This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
come from some piece of code explicitly filling a page with zeros, and
that often happens in the corner cases for O_DIRECT and a few other
places in the filesystem.

Have you tried triggering this with a regular block device?

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Jon Nelson's message of 2010-12-07 13:45:14 -0500:
>> On Tue, Dec 7, 2010 at 12:22 PM, Mike Snitzer <snitzer@redhat.com> wrote:
>> > On Tue, Dec 07 2010 at  1:10pm -0500,
>> > Jon Nelson <jnelson@jamponi.net> wrote:
>> >
>> >> I finally found some time to test this out. With 2.6.37-rc4 (openSUSE
>> >> KOTD kernel) I easily encounter the issue.
>> >>
>> >> Using a virtual machine, I created a stock, minimal openSUSE 11.3 x86_64
>> >> install, installed all updates, installed postgresql and the 'KOTD'
>> >> (Kernel of the Day)
>> >> kernel, and ran the following tests (as postgres user because I'm
>> >> lazy).
>> >>
>> >> 1. create a database (from bash):
>> >>
>> >> createdb test
>> >>
>> >> 2. place the following contents in a file (I used 't.sql'):
>> >>
>> >> begin;
>> >> create temporary table foo as select x as a, ARRAY[x] as b FROM
>> >> generate_series(1, 10000000 ) AS x;
>> >> create index foo_a_idx on foo (a);
>> >> create index foo_b_idx on foo USING GIN (b);
>> >> rollback;
>> >>
>> >> 3. execute that sql:
>> >>
>> >> psql -f t.sql --echo-all test
>> >>
>> >>
>> >> With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
>> >> without issue.
>> >>
>> >> With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
>> >> pretty frequently.
>> >
>> > How does it fail?  postgres errors?  kernel errors?
>>
>> postgresql errors. Typically, header corruption but from the limited
>> visibility I've had into this via strace, what I see is zeroed pages
>> where there shouldn't be.
>
> This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
> come from some piece of code explicitly filling a page with zeros, and
> that often happens in the corner cases for O_DIRECT and a few other
> places in the filesystem.
>
> Have you tried triggering this with a regular block device?

I just tried the whole set of tests, but with /dev/sdb directly (as
ext4) without any crypt-y bits.
It takes more iterations but out of 6 tests I had one failure: same
type of thing, 'invalid page header in block ....'.

I can't guarantee that it is a full-page of zeroes, just what I saw
from the (limited) stracing I did.



-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
> >> postgresql errors. Typically, header corruption but from the limited
> >> visibility I've had into this via strace, what I see is zeroed pages
> >> where there shouldn't be.
> >
> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
> > come from some piece of code explicitly filling a page with zeros, and
> > that often happens in the corner cases for O_DIRECT and a few other
> > places in the filesystem.
> >
> > Have you tried triggering this with a regular block device?
> 
> I just tried the whole set of tests, but with /dev/sdb directly (as
> ext4) without any crypt-y bits.
> It takes more iterations but out of 6 tests I had one failure: same
> type of thing, 'invalid page header in block ....'.
> 
> I can't guarantee that it is a full-page of zeroes, just what I saw
> from the (limited) stracing I did.

Fantastic. Now for our usual suspects:

1) Is postgres using O_DIRECT?  If yes, please turn it off
2) Is postgres allocating sparse files?  If yes, please have it fully
allocate the file instead.
3) Is postgres using preallocation (fallocate)?  If yes, please have it
fully allocate the file instead

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
>> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> >> postgresql errors. Typically, header corruption but from the limited
>> >> visibility I've had into this via strace, what I see is zeroed pages
>> >> where there shouldn't be.
>> >
>> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
>> > come from some piece of code explicitly filling a page with zeros, and
>> > that often happens in the corner cases for O_DIRECT and a few other
>> > places in the filesystem.
>> >
>> > Have you tried triggering this with a regular block device?
>>
>> I just tried the whole set of tests, but with /dev/sdb directly (as
>> ext4) without any crypt-y bits.
>> It takes more iterations but out of 6 tests I had one failure: same
>> type of thing, 'invalid page header in block ....'.
>>
>> I can't guarantee that it is a full-page of zeroes, just what I saw
>> from the (limited) stracing I did.
>
> Fantastic. Now for our usual suspects:
>
> 1) Is postgres using O_DIRECT?  If yes, please turn it off

According to strace, O_DIRECT didn't show up once during the test.

> 2) Is postgres allocating sparse files?  If yes, please have it fully
> allocate the file instead.

That's a tough one. I don't think postgresql does that, but I'm not an
expert here.

> 3) Is postgres using preallocation (fallocate)?  If yes, please have it
> fully allocate the file instead

As far as strace is concerned, postgreql is not using fallocate in
this version.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
> >> >> postgresql errors. Typically, header corruption but from the limited
> >> >> visibility I've had into this via strace, what I see is zeroed pages
> >> >> where there shouldn't be.
> >> >
> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
> >> > come from some piece of code explicitly filling a page with zeros, and
> >> > that often happens in the corner cases for O_DIRECT and a few other
> >> > places in the filesystem.
> >> >
> >> > Have you tried triggering this with a regular block device?
> >>
> >> I just tried the whole set of tests, but with /dev/sdb directly (as
> >> ext4) without any crypt-y bits.
> >> It takes more iterations but out of 6 tests I had one failure: same
> >> type of thing, 'invalid page header in block ....'.
> >>
> >> I can't guarantee that it is a full-page of zeroes, just what I saw
> >> from the (limited) stracing I did.
> >
> > Fantastic. Now for our usual suspects:
> >
> > 1) Is postgres using O_DIRECT?  If yes, please turn it off
> 
> According to strace, O_DIRECT didn't show up once during the test.
> 
> > 2) Is postgres allocating sparse files?  If yes, please have it fully
> > allocate the file instead.
> 
> That's a tough one. I don't think postgresql does that, but I'm not an
> expert here.
> 
> > 3) Is postgres using preallocation (fallocate)?  If yes, please have it
> > fully allocate the file instead
> 
> As far as strace is concerned, postgreql is not using fallocate in
> this version.

Well, the only other usual suspect would be mmap.  Does the strace show
that you're using read/write for file IO or is it doing a lot of mmaps
on the files?

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 2:33 PM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
>> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
>> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> >> >> postgresql errors. Typically, header corruption but from the limited
>> >> >> visibility I've had into this via strace, what I see is zeroed pages
>> >> >> where there shouldn't be.
>> >> >
>> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
>> >> > come from some piece of code explicitly filling a page with zeros, and
>> >> > that often happens in the corner cases for O_DIRECT and a few other
>> >> > places in the filesystem.
>> >> >
>> >> > Have you tried triggering this with a regular block device?
>> >>
>> >> I just tried the whole set of tests, but with /dev/sdb directly (as
>> >> ext4) without any crypt-y bits.
>> >> It takes more iterations but out of 6 tests I had one failure: same
>> >> type of thing, 'invalid page header in block ....'.
>> >>
>> >> I can't guarantee that it is a full-page of zeroes, just what I saw
>> >> from the (limited) stracing I did.
>> >
>> > Fantastic. Now for our usual suspects:
>> >
>> > 1) Is postgres using O_DIRECT?  If yes, please turn it off
>>
>> According to strace, O_DIRECT didn't show up once during the test.
>>
>> > 2) Is postgres allocating sparse files?  If yes, please have it fully
>> > allocate the file instead.
>>
>> That's a tough one. I don't think postgresql does that, but I'm not an
>> expert here.
>>
>> > 3) Is postgres using preallocation (fallocate)?  If yes, please have it
>> > fully allocate the file instead
>>
>> As far as strace is concerned, postgreql is not using fallocate in
>> this version.
>
> Well, the only other usual suspect would be mmap.  Does the strace show
> that you're using read/write for file IO or is it doing a lot of mmaps
> on the files?

I'm pretty sure postgresql uses regular file I/O and not mmap.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
> >> >> postgresql errors. Typically, header corruption but from the limited
> >> >> visibility I've had into this via strace, what I see is zeroed pages
> >> >> where there shouldn't be.
> >> >
> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
> >> > come from some piece of code explicitly filling a page with zeros, and
> >> > that often happens in the corner cases for O_DIRECT and a few other
> >> > places in the filesystem.
> >> >
> >> > Have you tried triggering this with a regular block device?
> >>
> >> I just tried the whole set of tests, but with /dev/sdb directly (as
> >> ext4) without any crypt-y bits.
> >> It takes more iterations but out of 6 tests I had one failure: same
> >> type of thing, 'invalid page header in block ....'.
> >>
> >> I can't guarantee that it is a full-page of zeroes, just what I saw
> >> from the (limited) stracing I did.
> >
> > Fantastic. Now for our usual suspects:
> >
> > 1) Is postgres using O_DIRECT?  If yes, please turn it off
> 
> According to strace, O_DIRECT didn't show up once during the test.
> 
> > 2) Is postgres allocating sparse files?  If yes, please have it fully
> > allocate the file instead.
> 
> That's a tough one. I don't think postgresql does that, but I'm not an
> expert here.

Ok, please compare du -k and du -k --apparent-size for each of the
files involved in the postgres run.

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 2:41 PM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
>> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
>> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> >> >> postgresql errors. Typically, header corruption but from the limited
>> >> >> visibility I've had into this via strace, what I see is zeroed pages
>> >> >> where there shouldn't be.
>> >> >
>> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
>> >> > come from some piece of code explicitly filling a page with zeros, and
>> >> > that often happens in the corner cases for O_DIRECT and a few other
>> >> > places in the filesystem.
>> >> >
>> >> > Have you tried triggering this with a regular block device?
>> >>
>> >> I just tried the whole set of tests, but with /dev/sdb directly (as
>> >> ext4) without any crypt-y bits.
>> >> It takes more iterations but out of 6 tests I had one failure: same
>> >> type of thing, 'invalid page header in block ....'.
>> >>
>> >> I can't guarantee that it is a full-page of zeroes, just what I saw
>> >> from the (limited) stracing I did.
>> >
>> > Fantastic. Now for our usual suspects:
>> >
>> > 1) Is postgres using O_DIRECT?  If yes, please turn it off
>>
>> According to strace, O_DIRECT didn't show up once during the test.
>>
>> > 2) Is postgres allocating sparse files?  If yes, please have it fully
>> > allocate the file instead.
>>
>> That's a tough one. I don't think postgresql does that, but I'm not an
>> expert here.
>
> Ok, please compare du -k and du -k --apparent-size for each of the
> files involved in the postgres run.

Because this is all done in a transaction (which fails), and because
the table is a TEMPORARY table, there *are* no files once the
transaction fails because postgresql unlinks them.

I can modify the test to use real tables and do things outside of a
transaction, however.

I was using fdatasync[1] and now I'm using sync. I'm on 9 iterations
without a failure (on ext4 - no crypt). Theoretically, these settings
only make a difference in the event of a crash. However, could they
make a difference in terms of the paths taken in the kernel?


[1] for wal_sync_method

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Jon Nelson's message of 2010-12-07 15:48:58 -0500:
> On Tue, Dec 7, 2010 at 2:41 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
> >> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> >> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
> >> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
> >> >> >> postgresql errors. Typically, header corruption but from the limited
> >> >> >> visibility I've had into this via strace, what I see is zeroed pages
> >> >> >> where there shouldn't be.
> >> >> >
> >> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
> >> >> > come from some piece of code explicitly filling a page with zeros, and
> >> >> > that often happens in the corner cases for O_DIRECT and a few other
> >> >> > places in the filesystem.
> >> >> >
> >> >> > Have you tried triggering this with a regular block device?
> >> >>
> >> >> I just tried the whole set of tests, but with /dev/sdb directly (as
> >> >> ext4) without any crypt-y bits.
> >> >> It takes more iterations but out of 6 tests I had one failure: same
> >> >> type of thing, 'invalid page header in block ....'.
> >> >>
> >> >> I can't guarantee that it is a full-page of zeroes, just what I saw
> >> >> from the (limited) stracing I did.
> >> >
> >> > Fantastic. Now for our usual suspects:
> >> >
> >> > 1) Is postgres using O_DIRECT?  If yes, please turn it off
> >>
> >> According to strace, O_DIRECT didn't show up once during the test.
> >>
> >> > 2) Is postgres allocating sparse files?  If yes, please have it fully
> >> > allocate the file instead.
> >>
> >> That's a tough one. I don't think postgresql does that, but I'm not an
> >> expert here.
> >
> > Ok, please compare du -k and du -k --apparent-size for each of the
> > files involved in the postgres run.
> 
> Because this is all done in a transaction (which fails), and because
> the table is a TEMPORARY table, there *are* no files once the
> transaction fails because postgresql unlinks them.
> 
> I can modify the test to use real tables and do things outside of a
> transaction, however.

That would really help.

> 
> I was using fdatasync[1] and now I'm using sync. I'm on 9 iterations
> without a failure (on ext4 - no crypt). Theoretically, these settings
> only make a difference in the event of a crash. However, could they
> make a difference in terms of the paths taken in the kernel?

Yes, the paths are different but similar.  If ext4 is causing zeros to
get stuffed somewhere, it should happen for both O_SYNC and fdatasync.

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 3:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Jon Nelson's message of 2010-12-07 15:48:58 -0500:
>> On Tue, Dec 7, 2010 at 2:41 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> > Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
>> >> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> >> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
>> >> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> >> >> >> postgresql errors. Typically, header corruption but from the limited
>> >> >> >> visibility I've had into this via strace, what I see is zeroed pages
>> >> >> >> where there shouldn't be.
>> >> >> >
>> >> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
>> >> >> > come from some piece of code explicitly filling a page with zeros, and
>> >> >> > that often happens in the corner cases for O_DIRECT and a few other
>> >> >> > places in the filesystem.
>> >> >> >
>> >> >> > Have you tried triggering this with a regular block device?
>> >> >>
>> >> >> I just tried the whole set of tests, but with /dev/sdb directly (as
>> >> >> ext4) without any crypt-y bits.
>> >> >> It takes more iterations but out of 6 tests I had one failure: same
>> >> >> type of thing, 'invalid page header in block ....'.
>> >> >>
>> >> >> I can't guarantee that it is a full-page of zeroes, just what I saw
>> >> >> from the (limited) stracing I did.
>> >> >
>> >> > Fantastic. Now for our usual suspects:

Maybe not so fantastic. I kept testing and had no more failures. At
all. After 40+ iterations I gave up.
I went back to trying ext4 on a LUKS volume. The 'hit' ratio went to
something like 1 in 3, or better.

I will continue to do testing with and without LUKS. I did /not/
reboot between tests, but I do start with a fresh postgres database.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption?

[Milan Broz]

On 12/08/2010 04:29 AM, Jon Nelson wrote:
> Maybe not so fantastic. I kept testing and had no more failures. At
> all. After 40+ iterations I gave up.
> I went back to trying ext4 on a LUKS volume. The 'hit' ratio went to
> something like 1 in 3, or better.

Encryption usually propagates bit corruption (not sure if it is
in this case). But in principle if there is one bit corrupted,
after decryption the whole sector is corrupted.
(That's why bit media errors have usually more serious impact
with FDE.)

Isn't there random noise instead of zeroes when reading
sparse files?
We should probably write some test focusing on sparse files
handling here...

Milan

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Jon Nelson's message of 2010-12-07 22:29:26 -0500:
> On Tue, Dec 7, 2010 at 3:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > Excerpts from Jon Nelson's message of 2010-12-07 15:48:58 -0500:
> >> On Tue, Dec 7, 2010 at 2:41 PM, Chris Mason <chris.mason@oracle.com> wrote:
> >> > Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
> >> >> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> >> >> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
> >> >> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
> >> >> >> >> postgresql errors. Typically, header corruption but from the limited
> >> >> >> >> visibility I've had into this via strace, what I see is zeroed pages
> >> >> >> >> where there shouldn't be.
> >> >> >> >
> >> >> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
> >> >> >> > come from some piece of code explicitly filling a page with zeros, and
> >> >> >> > that often happens in the corner cases for O_DIRECT and a few other
> >> >> >> > places in the filesystem.
> >> >> >> >
> >> >> >> > Have you tried triggering this with a regular block device?
> >> >> >>
> >> >> >> I just tried the whole set of tests, but with /dev/sdb directly (as
> >> >> >> ext4) without any crypt-y bits.
> >> >> >> It takes more iterations but out of 6 tests I had one failure: same
> >> >> >> type of thing, 'invalid page header in block ....'.
> >> >> >>
> >> >> >> I can't guarantee that it is a full-page of zeroes, just what I saw
> >> >> >> from the (limited) stracing I did.
> >> >> >
> >> >> > Fantastic. Now for our usual suspects:
> 
> Maybe not so fantastic. I kept testing and had no more failures. At
> all. After 40+ iterations I gave up.
> I went back to trying ext4 on a LUKS volume. The 'hit' ratio went to
> something like 1 in 3, or better.
> 
> I will continue to do testing with and without LUKS. I did /not/
> reboot between tests, but I do start with a fresh postgres database.
> 

Once we trigger once without dm-crypt, dm-crypt is off the hook.  Just
to verify, when you say without luks, you mean without any crypto bits
in use at all on the filesystems postgres uses?

Usually the trick to reproducing filesystem corruptions is adding memory
pressure.  The corruption is probably a bad interaction between reads
and writes, and we need to make sure the reads actually happen.

http://oss.oracle.com/~mason/pin_ram.c

gcc -Wall -o pin_ram pin_ram.c

pin_ram -m 80%-of-your-ram-in-mb

The idea is to trigger constant reads without having to swap heavily.
80% might be too much.

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Dave Chinner]

On Wed, Dec 08, 2010 at 07:20:24AM -0500, Chris Mason wrote:
> Excerpts from Jon Nelson's message of 2010-12-07 22:29:26 -0500:
> > On Tue, Dec 7, 2010 at 3:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > > Excerpts from Jon Nelson's message of 2010-12-07 15:48:58 -0500:
> > >> On Tue, Dec 7, 2010 at 2:41 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > >> > Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
> > >> >> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > >> >> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
> > >> >> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > >> >> >> >> postgresql errors. Typically, header corruption but from the limited
> > >> >> >> >> visibility I've had into this via strace, what I see is zeroed pages
> > >> >> >> >> where there shouldn't be.
> > >> >> >> >
> > >> >> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
> > >> >> >> > come from some piece of code explicitly filling a page with zeros, and
> > >> >> >> > that often happens in the corner cases for O_DIRECT and a few other
> > >> >> >> > places in the filesystem.
> > >> >> >> >
> > >> >> >> > Have you tried triggering this with a regular block device?
> > >> >> >>
> > >> >> >> I just tried the whole set of tests, but with /dev/sdb directly (as
> > >> >> >> ext4) without any crypt-y bits.
> > >> >> >> It takes more iterations but out of 6 tests I had one failure: same
> > >> >> >> type of thing, 'invalid page header in block ....'.
> > >> >> >>
> > >> >> >> I can't guarantee that it is a full-page of zeroes, just what I saw
> > >> >> >> from the (limited) stracing I did.
> > >> >> >
> > >> >> > Fantastic. Now for our usual suspects:
> > 
> > Maybe not so fantastic. I kept testing and had no more failures. At
> > all. After 40+ iterations I gave up.
> > I went back to trying ext4 on a LUKS volume. The 'hit' ratio went to
> > something like 1 in 3, or better.
> > 
> > I will continue to do testing with and without LUKS. I did /not/
> > reboot between tests, but I do start with a fresh postgres database.
> > 
> 
> Once we trigger once without dm-crypt, dm-crypt is off the hook.  Just
> to verify, when you say without luks, you mean without any crypto bits
> in use at all on the filesystems postgres uses?
> 
> Usually the trick to reproducing filesystem corruptions is adding memory
> pressure.  The corruption is probably a bad interaction between reads
> and writes, and we need to make sure the reads actually happen.
> 
> http://oss.oracle.com/~mason/pin_ram.c
> 
> gcc -Wall -o pin_ram pin_ram.c
> 
> pin_ram -m 80%-of-your-ram-in-mb

Implemented in xfstests about 10 years ago:

http://git.kernel.org/?p=fs/xfs/xfstests-dev.git;a=blob;f=src/usemem.c;h=b8794a6b209cebf8dbf312a8ef131e2e54b18d29;hb=HEAD

:P

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Dave Chinner's message of 2010-12-15 22:37:18 -0500:
> On Wed, Dec 08, 2010 at 07:20:24AM -0500, Chris Mason wrote:
> > 
> > Usually the trick to reproducing filesystem corruptions is adding memory
> > pressure.  The corruption is probably a bad interaction between reads
> > and writes, and we need to make sure the reads actually happen.
> > 
> > http://oss.oracle.com/~mason/pin_ram.c
> > 
> > gcc -Wall -o pin_ram pin_ram.c
> > 
> > pin_ram -m 80%-of-your-ram-in-mb
> 
> Implemented in xfstests about 10 years ago:
> 
> http://git.kernel.org/?p=fs/xfs/xfstests-dev.git;a=blob;f=src/usemem.c;h=b8794a6b209cebf8dbf312a8ef131e2e54b18d29;hb=HEAD

But mine can use shm! I don't remember adding it, so it must have grown
there while it sat on the oracle servers.  Our own special Christmas
magic.

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 2:41 PM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Jon Nelson's message of 2010-12-07 15:25:47 -0500:
>> On Tue, Dec 7, 2010 at 2:02 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> > Excerpts from Jon Nelson's message of 2010-12-07 14:34:40 -0500:
>> >> On Tue, Dec 7, 2010 at 12:52 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> >> >> postgresql errors. Typically, header corruption but from the limited
>> >> >> visibility I've had into this via strace, what I see is zeroed pages
>> >> >> where there shouldn't be.
>> >> >
>> >> > This sounds a lot like a bug higher up than dm-crypt.  Zeros tend to
>> >> > come from some piece of code explicitly filling a page with zeros, and
>> >> > that often happens in the corner cases for O_DIRECT and a few other
>> >> > places in the filesystem.
>> >> >
>> >> > Have you tried triggering this with a regular block device?
>> >>
>> >> I just tried the whole set of tests, but with /dev/sdb directly (as
>> >> ext4) without any crypt-y bits.
>> >> It takes more iterations but out of 6 tests I had one failure: same
>> >> type of thing, 'invalid page header in block ....'.
>> >>
>> >> I can't guarantee that it is a full-page of zeroes, just what I saw
>> >> from the (limited) stracing I did.
>> >
>> > Fantastic. Now for our usual suspects:
>> >
>> > 1) Is postgres using O_DIRECT?  If yes, please turn it off
>>
>> According to strace, O_DIRECT didn't show up once during the test.
>>
>> > 2) Is postgres allocating sparse files?  If yes, please have it fully
>> > allocate the file instead.
>>
>> That's a tough one. I don't think postgresql does that, but I'm not an
>> expert here.
>
> Ok, please compare du -k and du -k --apparent-size for each of the
> files involved in the postgres run.

One of the files (the table itself) is very slightly sparse:
588240 (apparent) vs 588244

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Ted Ts'o]

On Tue, Dec 07, 2010 at 01:22:43PM -0500, Mike Snitzer wrote:
> > 1. create a database (from bash):
> > 
> > createdb test
> > 
> > 2. place the following contents in a file (I used 't.sql'):
> > 
> > begin;
> > create temporary table foo as select x as a, ARRAY[x] as b FROM
> > generate_series(1, 10000000 ) AS x;
> > create index foo_a_idx on foo (a);
> > create index foo_b_idx on foo USING GIN (b);
> > rollback;
> > 
> > 3. execute that sql:
> > 
> > psql -f t.sql --echo-all test
> > 
> > With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
> > without issue.
> > 
> > With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
> > pretty frequently.

So I just tried to reproduce this on an Ubuntu 10.04 system running
2.6.37-rc5 (completely stock except for a few apparmor patches that I
needed to keep the apparmor userspace from complaining).  I'm using
Postgres 8.4.5-0ubuntu10.04.

Using the above procedure, I wasn't able to reproduce.  Then I
realized this might have been because I was using an SSD root file
system (which is secured using LUKS/dm-crypt, with LVM on top of
dm-crypt).  So I mounted a file system on a 5400 rpm SSD disk, which
is also protected using LUKS/dm-crypt with LVM on top.  I then
executed the PostgresQL commands:

CREATE TABLESPACE test LOCATION '/kbuild/postgres';
SET default_tablespace = test;
COMMIT
\quit

I then re-ran the above proceduing, and verified that all of the I/O
was going to the 5400rpm laptop disk.

I then ran the above procedure a half-dozen times, and I still haven't
been able to reproduce any Postgresql errors or kernel errors.

Jon, can you help me identify what might be different with your run
and mine?  What version of Postgres are you using?

Thanks,

						- Ted

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 1:35 PM, Ted Ts'o <tytso@mit.edu> wrote:
> On Tue, Dec 07, 2010 at 01:22:43PM -0500, Mike Snitzer wrote:
>> > 1. create a database (from bash):
>> >
>> > createdb test
>> >
>> > 2. place the following contents in a file (I used 't.sql'):
>> >
>> > begin;
>> > create temporary table foo as select x as a, ARRAY[x] as b FROM
>> > generate_series(1, 10000000 ) AS x;
>> > create index foo_a_idx on foo (a);
>> > create index foo_b_idx on foo USING GIN (b);
>> > rollback;
>> >
>> > 3. execute that sql:
>> >
>> > psql -f t.sql --echo-all test
>> >
>> > With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
>> > without issue.
>> >
>> > With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
>> > pretty frequently.
>
> So I just tried to reproduce this on an Ubuntu 10.04 system running
> 2.6.37-rc5 (completely stock except for a few apparmor patches that I
> needed to keep the apparmor userspace from complaining).  I'm using
> Postgres 8.4.5-0ubuntu10.04.
>
> Using the above procedure, I wasn't able to reproduce.  Then I
> realized this might have been because I was using an SSD root file
> system (which is secured using LUKS/dm-crypt, with LVM on top of
> dm-crypt).  So I mounted a file system on a 5400 rpm SSD disk, which
> is also protected using LUKS/dm-crypt with LVM on top.  I then
> executed the PostgresQL commands:
>
> CREATE TABLESPACE test LOCATION '/kbuild/postgres';
> SET default_tablespace = test;
> COMMIT
> \quit
>
> I then re-ran the above proceduing, and verified that all of the I/O
> was going to the 5400rpm laptop disk.
>
> I then ran the above procedure a half-dozen times, and I still haven't
> been able to reproduce any Postgresql errors or kernel errors.
>
> Jon, can you help me identify what might be different with your run
> and mine?  What version of Postgres are you using?

I am using postgres 8.4.5 on openSUSE 11.3 x86_64.
The problems were observed on both "real" hardware (thinkpad T61p) and
in virtualbox, where all current testing is taking place. The current
kernel is a "vanilla" (unpatched) kernel. I *did* set wal_sync_method
to fdatasync, however, if that is relevant. Otherwise, the pg config
is stock. With no crypt involved, I did have to iterate the tests to
observe the issue - a half-dozen times or more were necessary.
Typically, when crypt was involved, the issue would manifest much more
rapidly.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 1:35 PM, Ted Ts'o <tytso@mit.edu> wrote:
> On Tue, Dec 07, 2010 at 01:22:43PM -0500, Mike Snitzer wrote:
>> > 1. create a database (from bash):
>> >
>> > createdb test
>> >
>> > 2. place the following contents in a file (I used 't.sql'):
>> >
>> > begin;
>> > create temporary table foo as select x as a, ARRAY[x] as b FROM
>> > generate_series(1, 10000000 ) AS x;
>> > create index foo_a_idx on foo (a);
>> > create index foo_b_idx on foo USING GIN (b);
>> > rollback;
>> >
>> > 3. execute that sql:
>> >
>> > psql -f t.sql --echo-all test
>> >
>> > With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
>> > without issue.
>> >
>> > With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
>> > pretty frequently.
>
> So I just tried to reproduce this on an Ubuntu 10.04 system running
> 2.6.37-rc5 (completely stock except for a few apparmor patches that I
> needed to keep the apparmor userspace from complaining).  I'm using
> Postgres 8.4.5-0ubuntu10.04.
>
> Using the above procedure, I wasn't able to reproduce.  Then I
> realized this might have been because I was using an SSD root file
> system (which is secured using LUKS/dm-crypt, with LVM on top of
> dm-crypt).  So I mounted a file system on a 5400 rpm SSD disk, which
> is also protected using LUKS/dm-crypt with LVM on top.  I then
> executed the PostgresQL commands:
>
> CREATE TABLESPACE test LOCATION '/kbuild/postgres';
> SET default_tablespace = test;
> COMMIT
> \quit
>
> I then re-ran the above proceduing, and verified that all of the I/O
> was going to the 5400rpm laptop disk.
>
> I then ran the above procedure a half-dozen times, and I still haven't
> been able to reproduce any Postgresql errors or kernel errors.
>
> Jon, can you help me identify what might be different with your run
> and mine?  What version of Postgres are you using?

One difference is the location of the transaction logs (pg_xlog). In
my case, /var/lib/pgsql/data *is* mountpoint for the test volume
(actually, it's a symlink to the mount point). In your case, that is
not so. Perhaps that makes a difference?  pgsql_tmp might also be on
two different volumes in your case (I can't be sure).


-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Tue, Dec 7, 2010 at 9:37 PM, Jon Nelson <jnelson@jamponi.net> wrote:
> On Tue, Dec 7, 2010 at 1:35 PM, Ted Ts'o <tytso@mit.edu> wrote:
>> On Tue, Dec 07, 2010 at 01:22:43PM -0500, Mike Snitzer wrote:
>>> > 1. create a database (from bash):
>>> >
>>> > createdb test
>>> >
>>> > 2. place the following contents in a file (I used 't.sql'):
>>> >
>>> > begin;
>>> > create temporary table foo as select x as a, ARRAY[x] as b FROM
>>> > generate_series(1, 10000000 ) AS x;
>>> > create index foo_a_idx on foo (a);
>>> > create index foo_b_idx on foo USING GIN (b);
>>> > rollback;
>>> >
>>> > 3. execute that sql:
>>> >
>>> > psql -f t.sql --echo-all test
>>> >
>>> > With 2.6.34.7 I can re-run [3] all day long, as many times as I want,
>>> > without issue.
>>> >
>>> > With 2.6.37-rc4-13 (the currently-installed KOTD kernel) if tails
>>> > pretty frequently.
>>
>> So I just tried to reproduce this on an Ubuntu 10.04 system running
>> 2.6.37-rc5 (completely stock except for a few apparmor patches that I
>> needed to keep the apparmor userspace from complaining).  I'm using
>> Postgres 8.4.5-0ubuntu10.04.
>>
>> Using the above procedure, I wasn't able to reproduce.  Then I
>> realized this might have been because I was using an SSD root file
>> system (which is secured using LUKS/dm-crypt, with LVM on top of
>> dm-crypt).  So I mounted a file system on a 5400 rpm SSD disk, which
>> is also protected using LUKS/dm-crypt with LVM on top.  I then
>> executed the PostgresQL commands:
>>
>> CREATE TABLESPACE test LOCATION '/kbuild/postgres';
>> SET default_tablespace = test;
>> COMMIT
>> \quit
>>
>> I then re-ran the above proceduing, and verified that all of the I/O
>> was going to the 5400rpm laptop disk.
>>
>> I then ran the above procedure a half-dozen times, and I still haven't
>> been able to reproduce any Postgresql errors or kernel errors.
>>
>> Jon, can you help me identify what might be different with your run
>> and mine?  What version of Postgres are you using?
>
> One difference is the location of the transaction logs (pg_xlog). In
> my case, /var/lib/pgsql/data *is* mountpoint for the test volume
> (actually, it's a symlink to the mount point). In your case, that is
> not so. Perhaps that makes a difference?  pgsql_tmp might also be on
> two different volumes in your case (I can't be sure).


I grabbed a Kubuntu iso and installed Kubuntu 10.10, and then upgraded
to 'natty', and eventually to 2.6.37-8-generic.

With that install, and postgresql's "data" (/var/lib/postgresql/data)
being located on a LUKS+ext4 volume, I easily observe the behavior.

Does this help?

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Ted Ts'o]

On Tue, Dec 07, 2010 at 09:37:20PM -0600, Jon Nelson wrote:
> One difference is the location of the transaction logs (pg_xlog). In
> my case, /var/lib/pgsql/data *is* mountpoint for the test volume
> (actually, it's a symlink to the mount point). In your case, that is
> not so. Perhaps that makes a difference?  pgsql_tmp might also be on
> two different volumes in your case (I can't be sure).

I just tried tried to run t.sql five times with /var/lib/postgres as a
mountpoint for a 5400 rpm disk, and I still haven't been able to
replicate it.

If you can point out how to query pgsql_tmp (I'm using a completely
default postgres install), that would be helpful, but I don't think it
would be going anywhere else.

Still trying....

						- Ted

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Thu, Dec 9, 2010 at 12:01 PM, Ted Ts'o <tytso@mit.edu> wrote:
> On Tue, Dec 07, 2010 at 09:37:20PM -0600, Jon Nelson wrote:
>> One difference is the location of the transaction logs (pg_xlog). In
>> my case, /var/lib/pgsql/data *is* mountpoint for the test volume
>> (actually, it's a symlink to the mount point). In your case, that is
>> not so. Perhaps that makes a difference?  pgsql_tmp might also be on
>> two different volumes in your case (I can't be sure).
>
> I just tried tried to run t.sql five times with /var/lib/postgres as a
> mountpoint for a 5400 rpm disk, and I still haven't been able to
> replicate it.

You should be OK, there. Are you using encryption or no?
I had difficulty replicating the issue without encryption.

> If you can point out how to query pgsql_tmp (I'm using a completely
> default postgres install), that would be helpful, but I don't think it
> would be going anywhere else.

Normally it's /var/lib/pgsql/data/pgsql_tmp (or
/var/lib/postgres/data/pgsql_tmp in your case). By placing
/var/lib/{postgresql,pgsql}/data on the LUKS + ext4 volume, on both
openSUSE 11.3 and Kubuntu, I was able to replicate the problem easily,
in VirtualBox. I can give qemu a try. In both cases I was using a
2.6.37x kernel.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Ted Ts'o]

On Thu, Dec 09, 2010 at 12:10:58PM -0600, Jon Nelson wrote:
> 
> You should be OK, there. Are you using encryption or no?
> I had difficulty replicating the issue without encryption.

Yes, I'm using encryption.  LUKS with aes-xts-plain-sha256, and then
LVM on top of LUKS.

> > If you can point out how to query pgsql_tmp (I'm using a completely
> > default postgres install), that would be helpful, but I don't think it
> > would be going anywhere else.
> 
> Normally it's /var/lib/pgsql/data/pgsql_tmp (or
> /var/lib/postgres/data/pgsql_tmp in your case). By placing
> /var/lib/{postgresql,pgsql}/data on the LUKS + ext4 volume, on both
> openSUSE 11.3 and Kubuntu, I was able to replicate the problem easily,
> in VirtualBox. I can give qemu a try. In both cases I was using a
> 2.6.37x kernel.

Ah, I'm not using virtualization.  I'm running on a X410 laptop, on
raw hardware.  Perhaps virtualization slows things down enough that it
triggers?  Or maybe you're running with a more constrained memory than
I?  How much memory do you have configured in your VM?

    	     	       	   		      - Ted

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Thu, Dec 9, 2010 at 2:13 PM, Ted Ts'o <tytso@mit.edu> wrote:
> On Thu, Dec 09, 2010 at 12:10:58PM -0600, Jon Nelson wrote:
>>
>> You should be OK, there. Are you using encryption or no?
>> I had difficulty replicating the issue without encryption.
>
> Yes, I'm using encryption.  LUKS with aes-xts-plain-sha256, and then
> LVM on top of LUKS.

Hmm.
The cipher is listed as:
aes-cbc-essiv:sha256

>> > If you can point out how to query pgsql_tmp (I'm using a completely
>> > default postgres install), that would be helpful, but I don't think it
>> > would be going anywhere else.
>>
>> Normally it's /var/lib/pgsql/data/pgsql_tmp (or
>> /var/lib/postgres/data/pgsql_tmp in your case). By placing
>> /var/lib/{postgresql,pgsql}/data on the LUKS + ext4 volume, on both
>> openSUSE 11.3 and Kubuntu, I was able to replicate the problem easily,
>> in VirtualBox. I can give qemu a try. In both cases I was using a
>> 2.6.37x kernel.
>
> Ah, I'm not using virtualization.  I'm running on a X410 laptop, on
> raw hardware.  Perhaps virtualization slows things down enough that it
> triggers?  Or maybe you're running with a more constrained memory than
> I?  How much memory do you have configured in your VM?

512MB.

'free' reports 75MB, 419MB free.

I originally noticed the problem on really real hardware (thinkpad
T61p), however.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Andi Kleen]

> 512MB.
> 
> 'free' reports 75MB, 419MB free.
> 
> I originally noticed the problem on really real hardware (thinkpad
> T61p), however.

If you can easily reproduce it could you try a git bisect?

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500:
> > 512MB.
> > 
> > 'free' reports 75MB, 419MB free.
> > 
> > I originally noticed the problem on really real hardware (thinkpad
> > T61p), however.
> 
> If you can easily reproduce it could you try a git bisect?

Do we have a known good kernel?  I looked back through the thread and
didn't see any reports where the postgres test on ext4 passed in this
config.

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Fri, Dec 10, 2010 at 2:38 AM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500:
>> > 512MB.
>> >
>> > 'free' reports 75MB, 419MB free.
>> >
>> > I originally noticed the problem on really real hardware (thinkpad
>> > T61p), however.
>>
>> If you can easily reproduce it could you try a git bisect?
>
> Do we have a known good kernel?  I looked back through the thread and
> didn't see any reports where the postgres test on ext4 passed in this
> config.
>
> -chris
>

Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179

from the tests I've done that one showed the least or no corruption if
you count the empty /etc/env.d/03opengl as an artefact

(I tested 3 commits in total)


1) 5a87b7a5da250c9be6d757758425dfeaf8ed3179

2) 1de3e3df917459422cb2aecac440febc8879d410

3) bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc

1 -> 3 (earlier -> later)

Regards

Matt

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Ted Ts'o]

On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote:
> 
> Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179
> 
> from the tests I've done that one showed the least or no corruption if
> you count the empty /etc/env.d/03opengl as an artefact

Yes, that's a good test.  Also try commit bd2d0210cf.  The patch
series that is most likely to be at fault if there is a regression in
between 5a87b7a5d and bd2d0210cf inclusive.

I did a lot of testing before submitting it, but that wa a tricky
rewrite.  If you can reproduce the problem reliably, it might be good
to try commit 16828088f9 (the commit before 5a87b7a5d) and commit
bd2d0210cf.  If it reliably reproduces on bd2d0210cf, but is clean on
16828088f9, then it's my ext4 block i/o submission patches, and we'll
need to either figure out what's going on or back out that set of
changes.

If that's the case, a bisect of those changes (it's only 6 commits, so
it shouldn't take long) would be most appreciated.

Regards,

						- Ted

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Thu, Dec 9, 2010 at 8:38 PM, Ted Ts'o <tytso@mit.edu> wrote:
> On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote:
>>
>> Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179
>>
>> from the tests I've done that one showed the least or no corruption if
>> you count the empty /etc/env.d/03opengl as an artefact
>
> Yes, that's a good test.  Also try commit bd2d0210cf.  The patch
> series that is most likely to be at fault if there is a regression in
> between 5a87b7a5d and bd2d0210cf inclusive.
>
> I did a lot of testing before submitting it, but that wa a tricky
> rewrite.  If you can reproduce the problem reliably, it might be good
> to try commit 16828088f9 (the commit before 5a87b7a5d) and commit
> bd2d0210cf.  If it reliably reproduces on bd2d0210cf, but is clean on
> 16828088f9, then it's my ext4 block i/o submission patches, and we'll
> need to either figure out what's going on or back out that set of
> changes.
>
> If that's the case, a bisect of those changes (it's only 6 commits, so
> it shouldn't take long) would be most appreciated.

I observed the behavior on bd2d0210cf in a qemu-kvm install of
openSUSE 11.3 (x86_64) on *totally* different host - an AMD quad-core.

I did /not/ observe the behavior on 16828088f9 (yet). I'll run the
test a few more times on 1682..

Additionally, I am building a bisected kernel now (
cb20d5188366f04d96d2e07b1240cc92170ade40 ), but won't be able to get
back at it for a while.

I hope this helps.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Fri, Dec 10, 2010 at 12:52 AM, Jon Nelson <jnelson@jamponi.net> wrote:
> On Thu, Dec 9, 2010 at 8:38 PM, Ted Ts'o <tytso@mit.edu> wrote:
>> On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote:
>>>
>>> Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179
>>>
>>> from the tests I've done that one showed the least or no corruption if
>>> you count the empty /etc/env.d/03opengl as an artefact
>>
>> Yes, that's a good test.  Also try commit bd2d0210cf.  The patch
>> series that is most likely to be at fault if there is a regression in
>> between 5a87b7a5d and bd2d0210cf inclusive.
>>
>> I did a lot of testing before submitting it, but that wa a tricky
>> rewrite.  If you can reproduce the problem reliably, it might be good
>> to try commit 16828088f9 (the commit before 5a87b7a5d) and commit
>> bd2d0210cf.  If it reliably reproduces on bd2d0210cf, but is clean on
>> 16828088f9, then it's my ext4 block i/o submission patches, and we'll
>> need to either figure out what's going on or back out that set of
>> changes.
>>
>> If that's the case, a bisect of those changes (it's only 6 commits, so
>> it shouldn't take long) would be most appreciated.
>
> I observed the behavior on bd2d0210cf in a qemu-kvm install of
> openSUSE 11.3 (x86_64) on *totally* different host - an AMD quad-core.
>
> I did /not/ observe the behavior on 16828088f9 (yet). I'll run the
> test a few more times on 1682..
>
> Additionally, I am building a bisected kernel now (
> cb20d5188366f04d96d2e07b1240cc92170ade40 ), but won't be able to get
> back at it for a while.

cb20d5188366f04d96d2e07b1240cc92170ade40 seems OK so far. I'm going to
try 1de3e3df917459422cb2aecac440febc8879d410 soon.


-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Fri, Dec 10, 2010 at 8:58 AM, Jon Nelson <jnelson@jamponi.net> wrote:
> On Fri, Dec 10, 2010 at 12:52 AM, Jon Nelson <jnelson@jamponi.net> wrote:
>> On Thu, Dec 9, 2010 at 8:38 PM, Ted Ts'o <tytso@mit.edu> wrote:
>>> On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote:
>>>>
>>>> Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179
>>>>
>>>> from the tests I've done that one showed the least or no corruption if
>>>> you count the empty /etc/env.d/03opengl as an artefact
>>>
>>> Yes, that's a good test.  Also try commit bd2d0210cf.  The patch
>>> series that is most likely to be at fault if there is a regression in
>>> between 5a87b7a5d and bd2d0210cf inclusive.
>>>
>>> I did a lot of testing before submitting it, but that wa a tricky
>>> rewrite.  If you can reproduce the problem reliably, it might be good
>>> to try commit 16828088f9 (the commit before 5a87b7a5d) and commit
>>> bd2d0210cf.  If it reliably reproduces on bd2d0210cf, but is clean on
>>> 16828088f9, then it's my ext4 block i/o submission patches, and we'll
>>> need to either figure out what's going on or back out that set of
>>> changes.
>>>
>>> If that's the case, a bisect of those changes (it's only 6 commits, so
>>> it shouldn't take long) would be most appreciated.
>>
>> I observed the behavior on bd2d0210cf in a qemu-kvm install of
>> openSUSE 11.3 (x86_64) on *totally* different host - an AMD quad-core.
>>
>> I did /not/ observe the behavior on 16828088f9 (yet). I'll run the
>> test a few more times on 1682..
>>
>> Additionally, I am building a bisected kernel now (
>> cb20d5188366f04d96d2e07b1240cc92170ade40 ), but won't be able to get
>> back at it for a while.
>
> cb20d5188366f04d96d2e07b1240cc92170ade40 seems OK so far. I'm going to
> try 1de3e3df917459422cb2aecac440febc8879d410 soon.

Barring false negatives, bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
appears to be the culprit (according to git bisect).
I will test bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc again, confirm
the behavior, and work backwards to try to reduce the possibility of
false negatives.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Fri, Dec 10, 2010 at 10:54 AM, Jon Nelson <jnelson@jamponi.net> wrote:
> On Fri, Dec 10, 2010 at 8:58 AM, Jon Nelson <jnelson@jamponi.net> wrote:
>> On Fri, Dec 10, 2010 at 12:52 AM, Jon Nelson <jnelson@jamponi.net> wrote:
>>> On Thu, Dec 9, 2010 at 8:38 PM, Ted Ts'o <tytso@mit.edu> wrote:
>>>> On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote:
>>>>>
>>>>> Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179
>>>>>
>>>>> from the tests I've done that one showed the least or no corruption if
>>>>> you count the empty /etc/env.d/03opengl as an artefact
>>>>
>>>> Yes, that's a good test.  Also try commit bd2d0210cf.  The patch
>>>> series that is most likely to be at fault if there is a regression in
>>>> between 5a87b7a5d and bd2d0210cf inclusive.
>>>>
>>>> I did a lot of testing before submitting it, but that wa a tricky
>>>> rewrite.  If you can reproduce the problem reliably, it might be good
>>>> to try commit 16828088f9 (the commit before 5a87b7a5d) and commit
>>>> bd2d0210cf.  If it reliably reproduces on bd2d0210cf, but is clean on
>>>> 16828088f9, then it's my ext4 block i/o submission patches, and we'll
>>>> need to either figure out what's going on or back out that set of
>>>> changes.
>>>>
>>>> If that's the case, a bisect of those changes (it's only 6 commits, so
>>>> it shouldn't take long) would be most appreciated.
>>>
>>> I observed the behavior on bd2d0210cf in a qemu-kvm install of
>>> openSUSE 11.3 (x86_64) on *totally* different host - an AMD quad-core.
>>>
>>> I did /not/ observe the behavior on 16828088f9 (yet). I'll run the
>>> test a few more times on 1682..
>>>
>>> Additionally, I am building a bisected kernel now (
>>> cb20d5188366f04d96d2e07b1240cc92170ade40 ), but won't be able to get
>>> back at it for a while.
>>
>> cb20d5188366f04d96d2e07b1240cc92170ade40 seems OK so far. I'm going to
>> try 1de3e3df917459422cb2aecac440febc8879d410 soon.
>
> Barring false negatives, bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
> appears to be the culprit (according to git bisect).
> I will test bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc again, confirm
> the behavior, and work backwards to try to reduce the possibility of
> false negatives.

A few additional notes, in no particular order:

- For me, triggering the problem is fairly easy when encryption is involved.
- I'm now 81 iterations into testing
bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc *without* encryption.  Out of
81 iterations, I have 4 failures: #16, 40, 62, and 64.

I will now try 1de3e3df917459422cb2aecac440febc8879d410 much more extensively.

Is this useful information?

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Ted Ts'o]

On Fri, Dec 10, 2010 at 08:14:56PM -0600, Jon Nelson wrote:
> > Barring false negatives, bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
> > appears to be the culprit (according to git bisect).
> > I will test bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc again, confirm
> > the behavior, and work backwards to try to reduce the possibility of
> > false negatives.
> 
> A few additional notes, in no particular order:
> 
> - For me, triggering the problem is fairly easy when encryption is involved.
> - I'm now 81 iterations into testing
> bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc *without* encryption.  Out of
> 81 iterations, I have 4 failures: #16, 40, 62, and 64.
> 
> I will now try 1de3e3df917459422cb2aecac440febc8879d410 much more extensively.
> 
> Is this useful information?

Yes, indeed.  Is this in the virtualized environment or on real
hardware at this point?  And how many CPU's do you have configured in
your virtualized environment, and how memory memory?  Is having a
certain number of CPU's critical for reproducing the problem?  Is
constricting the amount of memory important?

It'll be a lot easier if I can reproduce it locally, which is why I'm
asking all of these questions.

Thanks,

					- Ted

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Ted Ts'o]

One experiment --- can you try this with the file system mounted with
data=writeback, and see if the problem reproduces in that journalling
mode?

I want to rule out (if possible) journal_submit_inode_data_buffers()
racing with mpage_da_submit_io().  I don't think that's the issue, but
I'd prefer to do the experiment to make sure.  So if you can use a
kernel and system configuration which triggers the problem, and then
try changing the mount options to include data=writeback, and then
rerun the test, and let me know if the problem still reproduces, I'd
be really grateful.

Thanks,

					- Ted

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Sat, Dec 11, 2010 at 7:40 PM, Ted Ts'o <tytso@mit.edu> wrote:
> Yes, indeed.  Is this in the virtualized environment or on real
> hardware at this point?  And how many CPU's do you have configured in
> your virtualized environment, and how memory memory?  Is having a
> certain number of CPU's critical for reproducing the problem?  Is
> constricting the amount of memory important?

Originally, I observed the behavior on really real hardware.

Since then, I have been able to reproduce it in VirtualBox and
qemu-kvm, with openSUSE 11.3 and KUbuntu. All of the more recent tests
have been with qemu-kvm.

I have one CPU configured in the environment, 512MB of memory.
I have not done any memory-constriction tests whatsoever.

> It'll be a lot easier if I can reproduce it locally, which is why I'm
> asking all of these questions.

On Sat, Dec 11, 2010 at 8:34 PM, Ted Ts'o <tytso@mit.edu> wrote:
> One experiment --- can you try this with the file system mounted with
> data=writeback, and see if the problem reproduces in that journalling
> mode?

That test is running now, first with encryption. I will report if it
shows problems. If it does, I will wait until I have been able to see
that a few times, and move to a no-encryption test. Typically, I have
to run quite a few more iterations of that test before problems show
up (if they will at all).

> I want to rule out (if possible) journal_submit_inode_data_buffers()
> racing with mpage_da_submit_io().  I don't think that's the issue, but
> I'd prefer to do the experiment to make sure.  So if you can use a
> kernel and system configuration which triggers the problem, and then
> try changing the mount options to include data=writeback, and then
> rerun the test, and let me know if the problem still reproduces, I'd
> be really grateful.


-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Sat, Dec 11, 2010 at 9:16 PM, Jon Nelson <jnelson@jamponi.net> wrote:
> On Sat, Dec 11, 2010 at 7:40 PM, Ted Ts'o <tytso@mit.edu> wrote:
>> Yes, indeed.  Is this in the virtualized environment or on real
>> hardware at this point?  And how many CPU's do you have configured in
>> your virtualized environment, and how memory memory?  Is having a
>> certain number of CPU's critical for reproducing the problem?  Is
>> constricting the amount of memory important?
>
> Originally, I observed the behavior on really real hardware.
>
> Since then, I have been able to reproduce it in VirtualBox and
> qemu-kvm, with openSUSE 11.3 and KUbuntu. All of the more recent tests
> have been with qemu-kvm.
>
> I have one CPU configured in the environment, 512MB of memory.
> I have not done any memory-constriction tests whatsoever.
>
>> It'll be a lot easier if I can reproduce it locally, which is why I'm
>> asking all of these questions.
>
> On Sat, Dec 11, 2010 at 8:34 PM, Ted Ts'o <tytso@mit.edu> wrote:
>> One experiment --- can you try this with the file system mounted with
>> data=writeback, and see if the problem reproduces in that journalling
>> mode?
>
> That test is running now, first with encryption. I will report if it
> shows problems. If it does, I will wait until I have been able to see
> that a few times, and move to a no-encryption test. Typically, I have
> to run quite a few more iterations of that test before problems show
> up (if they will at all).
>
>> I want to rule out (if possible) journal_submit_inode_data_buffers()
>> racing with mpage_da_submit_io().  I don't think that's the issue, but
>> I'd prefer to do the experiment to make sure.  So if you can use a
>> kernel and system configuration which triggers the problem, and then
>> try changing the mount options to include data=writeback, and then
>> rerun the test, and let me know if the problem still reproduces, I'd
>> be really grateful.

Using 2.6.37-rc5 and data=writeback,noatime and LUKS encryption I hit
the problem 71 times out of 173.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Ted Ts'o]

On Sun, Dec 12, 2010 at 04:18:29AM -0600, Jon Nelson wrote:
> > I have one CPU configured in the environment, 512MB of memory.
> > I have not done any memory-constriction tests whatsoever.

I've finally been able to reproduce it myself, on real hardware.  SMP
is not necessary to reproduce it, although having more than one CPU
doesn't hurt.  What I did need to do (on real hardware with 4 gigs of
memory) was to turn off swap and pin enough memory so that free memory
was around 200megs or so before the start of the test.  (This is the
natural amount of free memory that the system would try to reach,
since 200 megs is about 5% of 4 gigs.)

Then, during the test, free memory would drop to 50-70 megabytes,
forcing writeback to run, and then I could trigger it about 1-2 times
out of three.

I'm guessing that when you used 512mb of memory, that was in effect a
memory-constriction test, and if you were to push the memory down a
little further, it might reproduce even more quickly.  My next step is
to try to reproduce this in a VM, and then I can start probing to see
what might be going on.

						- Ted

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Sun, Dec 12, 2010 at 6:43 AM, Ted Ts'o <tytso@mit.edu> wrote:
> On Sun, Dec 12, 2010 at 04:18:29AM -0600, Jon Nelson wrote:
>> > I have one CPU configured in the environment, 512MB of memory.
>> > I have not done any memory-constriction tests whatsoever.
>
> I've finally been able to reproduce it myself, on real hardware.  SMP
> is not necessary to reproduce it, although having more than one CPU
> doesn't hurt.  What I did need to do (on real hardware with 4 gigs of
> memory) was to turn off swap and pin enough memory so that free memory
> was around 200megs or so before the start of the test.  (This is the
> natural amount of free memory that the system would try to reach,
> since 200 megs is about 5% of 4 gigs.)
>
> Then, during the test, free memory would drop to 50-70 megabytes,
> forcing writeback to run, and then I could trigger it about 1-2 times
> out of three.
>
> I'm guessing that when you used 512mb of memory, that was in effect a
> memory-constriction test, and if you were to push the memory down a
> little further, it might reproduce even more quickly.  My next step is
> to try to reproduce this in a VM, and then I can start probing to see
> what might be going on.

I'm glad you've been able to reproduce the problem! If you should need
any further assistance, please do not hesitate to ask.


-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Ted Ts'o]

On Sun, Dec 12, 2010 at 07:11:28AM -0600, Jon Nelson wrote:
> I'm glad you've been able to reproduce the problem! If you should need
> any further assistance, please do not hesitate to ask.

This patch seems to fix the problem for me.  (Unless the partition is
mounted with mblk_io_submit.)

Could you confirm that it fixes it for you as well?

Thanks!

					- Ted

commit a8649d85bd0ab3be6014918bd9eae319a0ffc691
Author: Theodore Ts'o <tytso@mit.edu>
Date:   Sun Dec 12 20:57:19 2010 -0500

    ext4: Turn off multiple page-io submission by default
    
    Jon Nelson has found a test case which causes postgresql to fail with
    the error:
    
    psql:t.sql:4: ERROR: invalid page header in block 38269 of relation base/16384/16581
    
    Under memory pressure, it looks like part of a file can end up getting
    replaced by zero's.  Until we can figure out the cause, we'll roll
    back the change and use block_write_full_page() instead of
    ext4_bio_write_page().  The new, more efficient writing function can
    be used via the mount option mblk_io_submit, so we can test and fix
    the new page I/O code.
    
    To reproduce the problem, install postgres 8.4 or 9.0, and pin enough
    memory such that the system just at the end of triggering writeback
    before running the following sql script:
    
    begin;
    create temporary table foo as select x as a, ARRAY[x] as b FROM
    generate_series(1, 10000000 ) AS x;
    create index foo_a_idx on foo (a);
    create index foo_b_idx on foo USING GIN (b);
    rollback;
    
    If the temporary table is created on a hard drive partition which is
    encrypted using dm_crypt, then under memory pressure, approximately
    30-40% of the time, pgsql will issue the above failure.
    
    This patch should fix this problem, and the problem will come back if
    the file system is mounted with the mblk_io_submit mount option.
    
    Reported-by: Jon Nelson <jnelson@jamponi.net>
    Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 6a5edea..6eb598b 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -910,6 +910,7 @@ struct ext4_inode_info {
 #define EXT4_MOUNT_JOURNAL_CHECKSUM	0x800000 /* Journal checksums */
 #define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT	0x1000000 /* Journal Async Commit */
 #define EXT4_MOUNT_I_VERSION            0x2000000 /* i_version support */
+#define EXT4_MOUNT_MBLK_IO_SUBMIT	0x4000000
 #define EXT4_MOUNT_DELALLOC		0x8000000 /* Delalloc support */
 #define EXT4_MOUNT_DATA_ERR_ABORT	0x10000000 /* Abort on file data write */
 #define EXT4_MOUNT_BLOCK_VALIDITY	0x20000000 /* Block validity checking */
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index bdbe699..e659597 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -2125,9 +2125,12 @@ static int mpage_da_submit_io(struct mpage_da_data *mpd,
 			 */
 			if (unlikely(journal_data && PageChecked(page)))
 				err = __ext4_journalled_writepage(page, len);
-			else
+			else if (test_opt(inode->i_sb, MBLK_IO_SUBMIT))
 				err = ext4_bio_write_page(&io_submit, page,
 							  len, mpd->wbc);
+			else
+				err = block_write_full_page(page,
+					noalloc_get_block_write, mpd->wbc);
 
 			if (!err)
 				mpd->pages_written++;
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index e32195d..fb15c9c 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -1026,6 +1026,8 @@ static int ext4_show_options(struct seq_file *seq, struct vfsmount *vfs)
 	    !(def_mount_opts & EXT4_DEFM_NODELALLOC))
 		seq_puts(seq, ",nodelalloc");
 
+	if (test_opt(sb, MBLK_IO_SUBMIT))
+		seq_puts(seq, ",mblk_io_submit");
 	if (sbi->s_stripe)
 		seq_printf(seq, ",stripe=%lu", sbi->s_stripe);
 	/*
@@ -1239,8 +1241,8 @@ enum {
 	Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota,
 	Opt_noquota, Opt_ignore, Opt_barrier, Opt_nobarrier, Opt_err,
 	Opt_resize, Opt_usrquota, Opt_grpquota, Opt_i_version,
-	Opt_stripe, Opt_delalloc, Opt_nodelalloc,
-	Opt_block_validity, Opt_noblock_validity,
+	Opt_stripe, Opt_delalloc, Opt_nodelalloc, Opt_mblk_io_submit,
+	Opt_nomblk_io_submit, Opt_block_validity, Opt_noblock_validity,
 	Opt_inode_readahead_blks, Opt_journal_ioprio,
 	Opt_dioread_nolock, Opt_dioread_lock,
 	Opt_discard, Opt_nodiscard,
@@ -1304,6 +1306,8 @@ static const match_table_t tokens = {
 	{Opt_resize, "resize"},
 	{Opt_delalloc, "delalloc"},
 	{Opt_nodelalloc, "nodelalloc"},
+	{Opt_mblk_io_submit, "mblk_io_submit"},
+	{Opt_nomblk_io_submit, "nomblk_io_submit"},
 	{Opt_block_validity, "block_validity"},
 	{Opt_noblock_validity, "noblock_validity"},
 	{Opt_inode_readahead_blks, "inode_readahead_blks=%u"},
@@ -1725,6 +1729,12 @@ set_qf_format:
 		case Opt_nodelalloc:
 			clear_opt(sbi->s_mount_opt, DELALLOC);
 			break;
+		case Opt_mblk_io_submit:
+			set_opt(sbi->s_mount_opt, MBLK_IO_SUBMIT);
+			break;
+		case Opt_nomblk_io_submit:
+			clear_opt(sbi->s_mount_opt, MBLK_IO_SUBMIT);
+			break;
 		case Opt_stripe:
 			if (match_int(&args[0], &option))
 				return 0;

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Sun, Dec 12, 2010 at 8:06 PM, Ted Ts'o <tytso@mit.edu> wrote:
> On Sun, Dec 12, 2010 at 07:11:28AM -0600, Jon Nelson wrote:
>> I'm glad you've been able to reproduce the problem! If you should need
>> any further assistance, please do not hesitate to ask.
>
> This patch seems to fix the problem for me.  (Unless the partition is
> mounted with mblk_io_submit.)
>
> Could you confirm that it fixes it for you as well?

I believe I have applied the (relevant) inode.c changes to
bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc, rebuilt and begun testing.
Now at 28 passes without error, I think I can say that the patch
appears to resolve the issue.

-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Mon, Dec 13, 2010 at 7:56 PM, Jon Nelson <jnelson@jamponi.net> wrote:
> On Sun, Dec 12, 2010 at 8:06 PM, Ted Ts'o <tytso@mit.edu> wrote:
>> On Sun, Dec 12, 2010 at 07:11:28AM -0600, Jon Nelson wrote:
>>> I'm glad you've been able to reproduce the problem! If you should need
>>> any further assistance, please do not hesitate to ask.
>>
>> This patch seems to fix the problem for me.  (Unless the partition is
>> mounted with mblk_io_submit.)
>>
>> Could you confirm that it fixes it for you as well?
>
> I believe I have applied the (relevant) inode.c changes to
> bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc, rebuilt and begun testing.
> Now at 28 passes without error, I think I can say that the patch
> appears to resolve the issue.
>
> --
> Jon
>

Confirmed !

I'm running my box for 5+ hours right now with your patch applied in
addition to Andi's/Milan's patch
(http://www.kernel.org/pub/linux/kernel/people/agk/patches/2.6/editing/dm-crypt-scale-to-multiple-CPUs.patch)
, Ted, and can't see any indications of corruptions so far (while
doing an emerge -e system) and doing everyday stuff.
My /home partition (with ext4) is also still intact [which of course
has a backup] so it seems to fix it for me, too

so the corruption I was seeing was similar in a way to that of Jon

You can add a
Tested-by: Matthias Bayer <jackdachef@gmail.com>

Thanks a lot to everyone for your support ! :)


I have a question though: the deactivation of multiple page-io
submission support most likely only would affect bigger systems or
also desktop systems (like mine) ?

Regards

Matt

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Andi Kleen]

> I have a question though: the deactivation of multiple page-io
> submission support most likely only would affect bigger systems or
> also desktop systems (like mine) ?

I think this is not a final fix, just a workaround.
The problem with the other path still really needs to be tracked down.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Wed, Dec 15, 2010 at 8:16 PM, Andi Kleen <andi@firstfloor.org> wrote:
>> I have a question though: the deactivation of multiple page-io
>> submission support most likely only would affect bigger systems or
>> also desktop systems (like mine) ?
>
> I think this is not a final fix, just a workaround.
> The problem with the other path still really needs to be tracked down.
>
> -Andi
>
> --
> ak@linux.intel.com -- Speaking for myself only.
>

ok,

thanks for the clarification

Regards

Matt

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Wed, Dec 15, 2010 at 8:25 PM, Matt <jackdachef@gmail.com> wrote:
> On Wed, Dec 15, 2010 at 8:16 PM, Andi Kleen <andi@firstfloor.org> wrote:
>>> I have a question though: the deactivation of multiple page-io
>>> submission support most likely only would affect bigger systems or
>>> also desktop systems (like mine) ?
>>
>> I think this is not a final fix, just a workaround.
>> The problem with the other path still really needs to be tracked down.
>>
>> -Andi
>>
>> --
>> ak@linux.intel.com -- Speaking for myself only.
>>
>
> ok,
>
> thanks for the clarification
>
> Regards
>
> Matt
>

Sorry to spam the mailing lists again

make that a
Reported-and-Tested-by: Matthias Bayer <jackdachef@gmail.com>

(hope that's the correct way to write it)

Regards

Matt

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Mike Fedyk]

On Thu, Dec 9, 2010 at 5:38 PM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500:
>> > 512MB.
>> >
>> > 'free' reports 75MB, 419MB free.
>> >
>> > I originally noticed the problem on really real hardware (thinkpad
>> > T61p), however.
>>
>> If you can easily reproduce it could you try a git bisect?
>
> Do we have a known good kernel?  I looked back through the thread and
> didn't see any reports where the postgres test on ext4 passed in this
> config.
>

2.6.34.something.  -- Any chance a newer kernel can be tested to be found good?

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Chris Mason]

Excerpts from Mike Fedyk's message of 2010-12-09 20:58:40 -0500:
> On Thu, Dec 9, 2010 at 5:38 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500:
> >> > 512MB.
> >> >
> >> > 'free' reports 75MB, 419MB free.
> >> >
> >> > I originally noticed the problem on really real hardware (thinkpad
> >> > T61p), however.
> >>
> >> If you can easily reproduce it could you try a git bisect?
> >
> > Do we have a known good kernel?  I looked back through the thread and
> > didn't see any reports where the postgres test on ext4 passed in this
> > config.
> >
> 
> 2.6.34.something.  -- Any chance a newer kernel can be tested to be found good?

But he is triggering the ext4 corruption without dm-crypt.  I think
dm-crypt was still used somewhere on the system during the test, just
not on the partitions that actually hit the corruption.

-chris

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Jon Nelson]

On Thu, Dec 9, 2010 at 8:00 PM, Chris Mason <chris.mason@oracle.com> wrote:
> Excerpts from Mike Fedyk's message of 2010-12-09 20:58:40 -0500:
>> On Thu, Dec 9, 2010 at 5:38 PM, Chris Mason <chris.mason@oracle.com> wrote:
>> > Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500:
>> >> > 512MB.
>> >> >
>> >> > 'free' reports 75MB, 419MB free.
>> >> >
>> >> > I originally noticed the problem on really real hardware (thinkpad
>> >> > T61p), however.
>> >>
>> >> If you can easily reproduce it could you try a git bisect?
>> >
>> > Do we have a known good kernel?  I looked back through the thread and
>> > didn't see any reports where the postgres test on ext4 passed in this
>> > config.
>> >
>>
>> 2.6.34.something.  -- Any chance a newer kernel can be tested to be found good?
>
> But he is triggering the ext4 corruption without dm-crypt.  I think
> dm-crypt was still used somewhere on the system during the test, just
> not on the partitions that actually hit the corruption.

I now have my doubts about being able to trigger it without dm-crypt.
I stand by my report, but I'm also unable to reproduce it after after
50 test iterations. I did run 2.6.36 for a few weeks without any issue
that I recall.



-- 
Jon

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Sat, Dec 4, 2010 at 8:38 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Sat, Dec 04 2010 at  2:18pm -0500,
> Matt <jackdachef@gmail.com> wrote:
>
>> On Wed, Dec 1, 2010 at 10:23 PM, Mike Snitzer <snitzer@redhat.com> wrote:
>> > Matt and Jon,
>> >
>> > If you'd be up to it: could you try testing your dm-crypt+ext4
>> > corruption reproducers against the following two 2.6.37-rc commits:
>> >
>> > 1) 1de3e3df917459422cb2aecac440febc8879d410
>> > then
>> > 2) bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
>> >
>> > Then, depending on results of no corruption for those commits, bonus
>> > points for testing the same commits but with Andi and Milan's latest
>> > dm-crypt cpu scalability patch applied too:
>> > https://patchwork.kernel.org/patch/365542/
>> >
>> > Thanks!
>> > Mike
>> >
>>
>> Hi Mike,
>>
>> it seems like there isn't even much testing to do:
>>
>> I tested all 3 commits / checkouts by re-compiling gcc which was/is
>> the 2nd easy way to trigger this "corruption", compiling google's
>> chromium (v9) and looking at the output/existance of gcc, g++ and
>> eselect opengl list
>
> Can you be a bit more precise about what you're doing to reproduce?
> What sequence?  What (if any) builds are going in parallel?  Etc.
>
>> so far everything went fine
>>
>> After that I used the new patch (v6 or pre-v6), before that I had to
>>
>> replace WQ_MEM_RECLAIM with WQ_RESCUER
>>
>> and, re-compiled the kernels
>>
>> shortly after I had booted up the system with the first kernel
>> (http://git.eu.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5a87b7a5da250c9be6d757758425dfeaf8ed3179)
>> the output of 'eselect opengl list' did show no opengl backend
>> selected
>>
>> so it seems to manifest itself even earlier (ext4: call
>> mpage_da_submit_io() from mpage_da_map_blocks()) even if only subtly
>> and over time -
>> I'm still currently running that kernel and posting from it & having tests run
>
> OK.
>
>> I'm not sure if it's even a problem with ext4 - I haven't had the time
>> to test with XFS yet - maybe it's also happening with that so it more
>> likely would be dm-core, like Milan suspected
>> (http://marc.info/?l=linux-kernel&m=129123636223477&w=2) :(
>
> It'd be interesting to try to reproduce with that same kernel but using
> XFS.  I'll check with Milan on what he thinks would be the best next
> steps.  Ideally we'll be able to reproduce your results to aid in
> pinpointing the issue.  I think Milan will be trying to do so shortly
> (if he hasn't started already -- using gentoo emerge, etc).
>
>> even though most of the time it's compiling I don't need to do much -
>> I need the box for work so if my time allows next tests would be next
>> weekend and I'm back to my other partition
>>
>> I really do hope that this bugger can be nailed down ASAP - I like the
>> improvements made in 2.6.37 but without the dm-crypt multi-cpu patch
>> it's only half the "fun" ;)
>
> Sure, we'll need to get to the bottom of this before we can have
> confidence sending the dm-crypt cpu scalability patch upstream.
>
> Thanks for your testing,
> Mike
>

I should have made it clear that the results I get are observed when
using the kernels/checkouts *with* the dm-crypt multi-cpu patch,
without the patch I didn't see that kind of problems (hardlocks, files
missing, etc.)

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Heinz Diehl]

On 05.12.2010, Matt wrote: 

> I should have made it clear that the results I get are observed when
> using the kernels/checkouts *with* the dm-crypt multi-cpu patch,
> without the patch I didn't see that kind of problems (hardlocks, files
> missing, etc.)

I have to take back my other two emails, stating that no corruption
happened with the dm-crypt multi-cpu patch. Today, I encountered
filesystem corruption on one, and a complete hardlock on another machine.
No logfile entries, no m-sysrq, a complete deadlock. Filesystem was
corrupted here too, had to reboot from CD.

The machines with plain 2.6.37-rc4 are up and running...

Re: hunt for 2.6.37 dm-crypt+ext4 corruption?

[Milan Broz]

On 12/05/2010 11:09 AM, Heinz Diehl wrote:
> On 05.12.2010, Matt wrote: 

> I have to take back my other two emails, stating that no corruption
> happened with the dm-crypt multi-cpu patch. Today, I encountered
> filesystem corruption on one, and a complete hardlock on another machine.
> No logfile entries, no m-sysrq, a complete deadlock. Filesystem was
> corrupted here too, had to reboot from CD.

Which kernel? 2.6.37-rc?

Anyone seen this with 2.6.36 and the same dmcrypt patch?
(All info I had is that is is stable with here.)

It still seems to like dmcrypt with its parallel processing is just
trigger to another bug in 37-rc.

Milan

Re: hunt for 2.6.37 dm-crypt+ext4 corruption?

[Heinz Diehl]

On 05.12.2010, Milan Broz wrote: 

> Which kernel? 2.6.37-rc?

2.6.37-rc4 on one and 2.6.37-rc3-git2 on the other machine.

> Anyone seen this with 2.6.36 and the same dmcrypt patch?
> (All info I had is that is is stable with here.)

Both 2.6.36 and 2.6.36.1 with your patch have been running flawlessly.
Not a single problem at all, over several weeks. 24/7.

> It still seems to like dmcrypt with its parallel processing is just
> trigger to another bug in 37-rc.

That's the impression I got, too.

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Theodore Tso]

On Dec 5, 2010, at 5:21 AM, Milan Broz wrote:
> 
> Which kernel? 2.6.37-rc?
> 
> Anyone seen this with 2.6.36 and the same dmcrypt patch?
> (All info I had is that is is stable with here.)
> 
> It still seems to like dmcrypt with its parallel processing is just
> trigger to another bug in 37-rc.

I've been using a kernel which is between 2.6.37-rc2 and -rc3 with a LUKS / dm-crypt / LVM / ext4 setup for my primary file systems, and I haven't observed any corruption for the last two weeks or so.   It's on my todo list to upgrade to top of Linus's tree, but perhaps this is a useful data point.

As another thought, what version of GCC are people using who are having difficulty?   Could this perhaps be a compiler-related issue?

-- Ted

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Matt]

On Sun, Dec 5, 2010 at 2:24 PM, Theodore Tso <tytso@mit.edu> wrote:
>
> On Dec 5, 2010, at 5:21 AM, Milan Broz wrote:
>>
>> Which kernel? 2.6.37-rc?
>>
>> Anyone seen this with 2.6.36 and the same dmcrypt patch?
>> (All info I had is that is is stable with here.)
>>
>> It still seems to like dmcrypt with its parallel processing is just
>> trigger to another bug in 37-rc.
>
> I've been using a kernel which is between 2.6.37-rc2 and -rc3 with a LUKS / dm-crypt / LVM / ext4 setup for my primary file systems, and I haven't observed any corruption for the last two weeks or so.   It's on my todo list to upgrade to top of Linus's tree, but perhaps this is a useful data point.
>
> As another thought, what version of GCC are people using who are having difficulty?   Could this perhaps be a compiler-related issue?
>
> -- Ted
>
>

Hi Ted,

to quote its output:


gcc -v
Using built-in specs.
COLLECT_GCC=/usr/x86_64-pc-linux-gnu/gcc-bin/4.5.1/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/4.5.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with:
/var/tmp/portage/sys-devel/gcc-4.5.1-r1/work/gcc-4.5.1/configure
--prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/4.5.1
--includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.1/include
--datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.5.1
--mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.5.1/man
--infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.5.1/info
--with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.1/include/g++-v4
--host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu
--disable-altivec --disable-fixed-point --with-ppl --with-cloog
--enable-lto --enable-nls --without-included-gettext
--with-system-zlib --disable-werror --enable-secureplt
--enable-multilib --enable-libmudflap --disable-libssp --enable-esp
--enable-libgomp --enable-cld
--with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/4.5.1/python
--enable-checking=release --enable-java-awt=gtk --enable-objc-gc
--enable-languages=c,c++,java,objc,obj-c++,fortran --enable-shared
--enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu
--with-bugurl=http://bugs.gentoo.org/ --with-pkgversion='Gentoo
Hardened 4.5.1-r1 p1.4, pie-0.4.5'
Thread model: posix
gcc version 4.5.1 (Gentoo Hardened 4.5.1-r1 p1.4, pie-0.4.5)


output of emerge -p gcc:

These are the packages that would be merged, in order:

Calculating dependencies                                  ... done!
[ebuild   R   ] sys-devel/gcc-4.5.1-r1  USE="fortran gcj graphite gtk
hardened lto mudflap (multilib) multislot nls nptl objc objc++ objc-gc
openmp (-altivec) -bootstrap -build -doc (-fixed-point) (-libffi)
(-n32) (-n64) -nocxx -nopie -nossp -test -vanilla" 0 kB



and to be precise it's gcc 4.5.1 with some gentoo-specific fixes and
fixes from upstream (4.5.2) [take a look at patchset 1.4],
in my case it also has the --enable-esp functionality [hardened]
which should include something like -D_FORTIFY_SOURCE=2, -fstack-protector-all
and for linking/ldd: -Wl,-z,now -Wl,-z,relro

(I don't know if the part with the linker and the fstack-protector is accurate)

I'm adding below the output of mount of the system-partition of the
system I was running the kernel on - where the [more observable]
corruption was observed (checkout
bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc)
-> this output got generated while I mounted it from my working (no
corruption observed) system with 2.6.36 kernel - I don't know if it's
useful - just in case you might need it
[forgot to post this in my last email]

Thanks & Regards

Matt



[  607.849644] EXT4-fs (dm-7): INFO: recovery required on readonly filesystem
[  607.849651] EXT4-fs (dm-7): write access will be enabled during recovery
[  609.559363] EXT4-fs (dm-7): orphan cleanup on readonly fs
[  609.559375] EXT4-fs (dm-7): ext4_orphan_cleanup: truncating inode
2238873 to 0 bytes
[  609.559493] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231865
[  609.559531] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231870
[  609.559553] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396001
[  609.559588] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396036
[  609.559610] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2395699
[  609.559675] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231859
[  609.559695] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231868
[  609.559715] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396696
[  609.559736] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396697
[  609.559755] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396699
[  609.559775] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2395948
[  609.559809] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231856
[  609.559830] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231866
[  609.559850] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239446
[  609.559872] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239447
[  609.559892] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239454
[  609.559912] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239458
[  609.559933] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239033
[  609.559992] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231858
[  609.560012] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231862
[  609.560033] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393696
[  609.560054] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393697
[  609.560074] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393698
[  609.560094] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2395268
[  609.582087] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231130
[  609.582147] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231860
[  609.582179] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2371247
[  609.595564] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2371250
[  609.605893] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2373715
[  609.605928] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393813
[  609.605958] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231417
[  609.605980] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231854
[  609.605999] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231864
[  609.606019] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239111
[  609.606039] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239113
[  609.606069] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239114
[  609.606099] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239120
[  609.608409] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231602
[  609.608452] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231861
[  609.608483] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239420
[  609.608512] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239421
[  609.608542] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239426
[  609.608572] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239637
[  609.608604] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231603
[  609.608634] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231855
[  609.608664] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255714
[  609.608694] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255715
[  609.608723] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255739
[  609.608753] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255645
[  609.608797] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2141218
[  609.608844] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2140971
[  609.630666] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2141266
[  609.630700] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2141267
[  609.630722] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2394877
[  609.630743] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396476
[  609.630765] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396489
[  609.630794] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2396512
[  609.642390] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2229825
[  609.642433] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231604
[  609.657435] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393858
[  609.657476] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393859
[  609.657505] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393860
[  609.657535] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2394679
[  609.658623] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393493
[  609.659363] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2393462
[  609.659404] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2135731
[  609.684858] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2135728
[  609.684904] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239357
[  609.685239] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239362
[  609.697558] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239396
[  609.697604] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239335
[  609.697703] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 1310848
[  609.710785] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 1310828
[  609.713278] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231128
[  609.713311] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231418
[  609.713342] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256648
[  609.713371] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256649
[  609.713400] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256650
[  609.713429] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2255701
[  609.713481] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2231129
[  609.713511] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239633
[  609.713540] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256006
[  609.713570] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239354
[  609.734696] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118116
[  609.734739] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118050
[  609.734771] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256652
[  609.734797] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256654
[  609.734817] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256655
[  609.734847] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239608
[  609.734893] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118049
[  609.734922] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118048
[  609.734951] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2135750
[  609.734981] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2257151
[  609.738316] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2374676
[  609.738352] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256865
[  609.738379] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118047
[  609.738399] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2118046
[  609.738419] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256098
[  609.738439] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256099
[  609.738458] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256100
[  609.738477] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239408
[  609.738502] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2115691
[  609.742723] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2111912
[  609.742771] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2141299
[  609.753070] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239521
[  609.753105] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239016
[  609.753130] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2111888
[  609.753151] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2111865
[  609.753172] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256464
[  609.753192] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2256500
[  609.753212] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2239402
[  609.753235] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2111910
[  609.753255] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2111900
[  609.762311] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144754
[  609.762353] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144757
[  609.762400] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144771
[  609.762428] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144772
[  609.762447] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144773
[  609.762466] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2144774
[  609.762486] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 1310823
[  609.762507] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 1310900
[  609.763591] EXT4-fs (dm-7): ext4_orphan_cleanup: deleting
unreferenced inode 2145297
[  609.763700] EXT4-fs (dm-7): 122 orphan inodes deleted
[  609.763708] EXT4-fs (dm-7): 1 truncate cleaned up
[  609.763714] EXT4-fs (dm-7): recovery complete
[  610.593272] EXT4-fs (dm-7): mounted filesystem with ordered data
mode. Opts: commit=600,barrier=1

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Ted Ts'o]

On Sun, Dec 05, 2010 at 02:44:14PM +0100, Matt wrote:
> gcc version 4.5.1 (Gentoo Hardened 4.5.1-r1 p1.4, pie-0.4.5)

This is probably just me being paranoid, but it might be worth trying
using a gcc 4.4.x compiler and see if that makes any difference.
There have been some other gcc 4.5-caused probablems with the kernel,
and it might be a good idea to rule those out.  (I'm using gcc 4.4.3,
myself, and I'm deliberately avoiding the use of gcc 4.5 for now.)

       	     	       	       - Ted

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Heinz Diehl]

On 05.12.2010, Theodore Tso wrote: 

> As another thought, what version of GCC are people using who 
> are having difficulty? Could this perhaps be a compiler-related issue?

htd@liesel:~> gcc -v
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info
--mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64
--enable-languages=c,c++,objc,fortran,obj-c++,java,ada
--enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3
--enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/
--with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap
--with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit
--enable-libstdcxx-allocator=new --disable-libstdcxx-pch
--enable-version-specific-runtime-libs --program-suffix=-4.3
--enable-linux-futex --without-system-libunwind --with-cpu=generic
--build=x86_64-suse-linux
Thread model: posix
gcc version 4.3.2 [gcc-4_3-branch revision 141291] (SUSE Linux) 

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Daniel J Blueman]

Hi Heinz,

On 5 December 2010 14:33, Heinz Diehl <htd@fritha.org> wrote:
> On 05.12.2010, Theodore Tso wrote:
>
>> As another thought, what version of GCC are people using who
>> are having difficulty? Could this perhaps be a compiler-related issue?
>
> htd@liesel:~> gcc -v
> Using built-in specs.
> Target: x86_64-suse-linux
> Configured with: ../configure --prefix=/usr --infodir=/usr/share/info
> --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64
> --enable-languages=c,c++,objc,fortran,obj-c++,java,ada
> --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3
> --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/
> --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap
> --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit
> --enable-libstdcxx-allocator=new --disable-libstdcxx-pch
> --enable-version-specific-runtime-libs --program-suffix=-4.3
> --enable-linux-futex --without-system-libunwind --with-cpu=generic
> --build=x86_64-suse-linux
> Thread model: posix
> gcc version 4.3.2 [gcc-4_3-branch revision 141291] (SUSE Linux)

A bit late to the party, but does memtest86 pass over multiple iterations?

Also, can you send an 'sudo lspci -vv', so we can check for
known-buggy controllers and bridges?

Thanks,
  Daniel
-- 
Daniel J Blueman

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Heinz Diehl]

[-- Attachment #1: Type: text/plain, Size: 489 bytes --]

On 06.12.2010, Daniel J Blueman wrote: 

> A bit late to the party, but does memtest86 pass over multiple iterations?

Yes, it does. This machine had not a single fault in several years, it's
absolutely rock-stable. These freezes/corruptions are the first ones ever,
and they vanish when I go down to 2.6.36 or revert the latest dm-crypt
multi-cpu patch with 2.6.37-rc.

> Also, can you send an 'sudo lspci -vv', so we can check for
> known-buggy controllers and bridges?

It's attached.


[-- Attachment #2: lspci.txt --]
[-- Type: text/plain, Size: 21581 bytes --]

00:00.0 Host bridge: ATI Technologies Inc RX780/RX790 Chipset Host Bridge
	Subsystem: ATI Technologies Inc RX780/RX790 Chipset Host Bridge
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
	Latency: 32
	Region 3: Memory at <ignored> (64-bit, non-prefetchable)
	Capabilities: [c4] HyperTransport: Slave or Primary Interface
		Command: BaseUnitID=0 UnitCnt=12 MastHost- DefDir- DUL-
		Link Control 0: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0 IsocEn- LSEn- ExtCTL- 64b-
		Link Config 0: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit DwFcInEn- LWO=16bit DwFcOutEn-
		Link Control 1: CFlE- CST- CFE- <LkFail+ Init- EOC+ TXO+ <CRCErr=0 IsocEn- LSEn- ExtCTL- 64b-
		Link Config 1: MLWI=8bit DwFcIn- MLWO=8bit DwFcOut- LWI=8bit DwFcInEn- LWO=8bit DwFcOutEn-
		Revision ID: 3.00
		Link Frequency 0: [b]
		Link Error 0: <Prot- <Ovfl- <EOC- CTLTm-
		Link Frequency Capability 0: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+ 800MHz+ 1.0GHz+ 1.2GHz+ 1.4GHz- 1.6GHz- Vend-
		Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD-
		Link Frequency 1: 200MHz
		Link Error 1: <Prot- <Ovfl- <EOC- CTLTm-
		Link Frequency Capability 1: 200MHz- 300MHz- 400MHz- 500MHz- 600MHz- 800MHz- 1.0GHz- 1.2GHz- 1.4GHz- 1.6GHz- Vend-
		Error Handling: PFlE- OFlE- PFE- OFE- EOCFE- RFE- CRCFE- SERRFE- CF- RE- PNFE- ONFE- EOCNFE- RNFE- CRCNFE- SERRNFE-
		Prefetchable memory behind bridge Upper: 00-00
		Bus Number: 00
	Capabilities: [40] HyperTransport: Retry Mode
	Capabilities: [54] HyperTransport: UnitID Clumping
	Capabilities: [9c] HyperTransport: #1a

00:02.0 PCI bridge: ATI Technologies Inc RD790 PCI to PCI bridge (external gfx0 port A) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: fa000000-fcffffff
	Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
			ClockPM- Suprise- LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis- ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB
	Capabilities: [a0] Message Signalled Interrupts: Mask- 64bit- Count=1/1 Enable-
		Address: 00000000  Data: 0000
	Capabilities: [b0] Subsystem: ATI Technologies Inc Device 5957
	Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
	Capabilities: [100] Vendor Specific Information <?>
	Capabilities: [110] Virtual Channel <?>

00:0a.0 PCI bridge: ATI Technologies Inc RD790 PCI to PCI bridge (PCI express gpp port F) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: 0000d000-0000dfff
	Memory behind bridge: fdc00000-fdcfffff
	Prefetchable memory behind bridge: 00000000fdf00000-00000000fdffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #5, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <64ns, L1 <1us
			ClockPM- Suprise- LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis- ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB
	Capabilities: [a0] Message Signalled Interrupts: Mask- 64bit- Count=1/1 Enable-
		Address: 00000000  Data: 0000
	Capabilities: [b0] Subsystem: ATI Technologies Inc Device 5957
	Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
	Capabilities: [100] Vendor Specific Information <?>
	Capabilities: [110] Virtual Channel <?>

00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] (prog-if 01 [AHCI 1.0])
	Subsystem: Giga-byte Technology Device b002
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 22
	Region 0: I/O ports at ff00 [size=8]
	Region 1: I/O ports at fe00 [size=4]
	Region 2: I/O ports at fd00 [size=8]
	Region 3: I/O ports at fc00 [size=4]
	Region 4: I/O ports at fb00 [size=16]
	Region 5: Memory at fe02f000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [70] SATA HBA <?>
	Kernel driver in use: ahci
	Kernel modules: ahci

00:12.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller (prog-if 10 [OHCI])
	Subsystem: Giga-byte Technology Device 5004
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at fe02e000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd

00:12.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller (prog-if 10 [OHCI])
	Subsystem: Giga-byte Technology Device 5004
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at fe02d000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd

00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller (prog-if 20 [EHCI])
	Subsystem: Giga-byte Technology Device 5004
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 17
	Region 0: Memory at fe02c000 (32-bit, non-prefetchable) [size=256]
	Capabilities: [c0] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
		Bridge: PM- B3+
	Capabilities: [e4] Debug port: BAR=1 offset=00e0
	Kernel driver in use: ehci_hcd
	Kernel modules: ehci-hcd

00:13.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller (prog-if 10 [OHCI])
	Subsystem: Giga-byte Technology Device 5004
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 18
	Region 0: Memory at fe02b000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd

00:13.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller (prog-if 10 [OHCI])
	Subsystem: Giga-byte Technology Device 5004
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 18
	Region 0: Memory at fe02a000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd

00:13.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller (prog-if 20 [EHCI])
	Subsystem: Giga-byte Technology Device 5004
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 19
	Region 0: Memory at fe029000 (32-bit, non-prefetchable) [size=256]
	Capabilities: [c0] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
		Bridge: PM- B3+
	Capabilities: [e4] Debug port: BAR=1 offset=00e0
	Kernel driver in use: ehci_hcd
	Kernel modules: ehci-hcd

00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3a)
	Subsystem: Giga-byte Technology Device 4385
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Capabilities: [b0] HyperTransport: MSI Mapping Enable- Fixed+
	Kernel modules: i2c-piix4

00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller (prog-if 8a [Master SecP PriP])
	Subsystem: Giga-byte Technology Device 5002
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 16
	Region 0: I/O ports at 01f0 [size=8]
	Region 1: I/O ports at 03f4 [size=1]
	Region 2: I/O ports at 0170 [size=8]
	Region 3: I/O ports at 0374 [size=1]
	Region 4: I/O ports at fa00 [size=16]
	Capabilities: [70] Message Signalled Interrupts: Mask- 64bit- Count=1/1 Enable-
		Address: 00000000  Data: 0000
	Kernel driver in use: pata_atiixp
	Kernel modules: pata_atiixp, ide-pci-generic

00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA)
	Subsystem: Giga-byte Technology Device a022
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin ? routed to IRQ 16
	Region 0: Memory at fe024000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: HDA Intel
	Kernel modules: snd-hda-intel

00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
	Subsystem: ATI Technologies Inc SB700/SB800 LPC host controller
	Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (prog-if 01 [Subtractive decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop+ ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 64
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=64
	I/O behind bridge: 0000c000-0000cfff
	Memory behind bridge: fde00000-fdefffff
	Prefetchable memory behind bridge: fdd00000-fddfffff
	Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-

00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2 Controller (prog-if 10 [OHCI])
	Subsystem: Giga-byte Technology Device 5004
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Interrupt: pin C routed to IRQ 18
	Region 0: Memory at fe028000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd

00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, Athlon64, Sempron] HyperTransport Configuration
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Capabilities: [80] HyperTransport: Host or Secondary Interface
		Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL-
		Link Control: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0 IsocEn- LSEn+ ExtCTL- 64b-
		Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit DwFcInEn- LWO=16bit DwFcOutEn-
		Revision ID: 3.00
		Link Frequency: [b]
		Link Error: <Prot- <Ovfl- <EOC- CTLTm-
		Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+ 800MHz+ 1.0GHz+ 1.2GHz+ 1.4GHz- 1.6GHz- Vend-
		Feature Capability: IsocFC+ LDTSTOP+ CRCTM- ECTLT- 64bA+ UIDRD- ExtRS- UCnfE-

00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, Athlon64, Sempron] Address Map
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, Athlon64, Sempron] DRAM Controller
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, Athlon64, Sempron] Miscellaneous Control
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Capabilities: [f0] Secure device <?>
	Kernel modules: k10temp

00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, Athlon64, Sempron] Link Control
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

01:00.0 VGA compatible controller: nVidia Corporation G70 [GeForce 7600 GS] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: CardExpert Technology Device 1401
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 10
	Region 0: Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
	Region 1: Memory at d0000000 (64-bit, prefetchable) [size=256M]
	Region 3: Memory at fb000000 (64-bit, non-prefetchable) [size=16M]
	Region 5: I/O ports at ef00 [size=128]
	[virtual] Expansion ROM at fc000000 [disabled] [size=128K]
	Capabilities: [60] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] Message Signalled Interrupts: Mask- 64bit+ Count=1/1 Enable-
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v1) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <256ns, L1 <4us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Suprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100] Virtual Channel <?>
	Capabilities: [128] Power Budgeting <?>
	Kernel modules: nvidiafb

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
	Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 40
	Region 0: I/O ports at de00 [size=256]
	Region 2: Memory at fdfff000 (64-bit, prefetchable) [size=4K]
	Region 4: Memory at fdfe0000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at fdf00000 [disabled] [size=64K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Count=1/2 Enable+
		Address: 00000000fee0f00c  Data: 4161
	Capabilities: [70] Express (v1) Endpoint, MSI 01
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <8us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <64us
			ClockPM+ Suprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [b0] MSI-X: Enable- Mask- TabSize=2
		Vector table: BAR=4 offset=00000000
		PBA: BAR=4 offset=00000800
	Capabilities: [d0] Vital Product Data <?>
	Capabilities: [100] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSVoil-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSVoil-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSVoil-
		CESta:	RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [140] Virtual Channel <?>
	Capabilities: [160] Device Serial Number 78-56-34-12-78-56-34-12
	Kernel driver in use: r8169
	Kernel modules: r8169

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Andi Kleen]

> I've been using a kernel which is between 2.6.37-rc2 and -rc3 with a LUKS / dm-crypt / LVM / ext4 setup for my primary file systems, and I haven't observed any corruption for the last two weeks or so.   It's on my todo list to upgrade to top of Linus's tree, but perhaps this is a useful data point.

The reported corruptions are with a .37 forward port someone did of my parallel 
kcryptd patch.  The 2.6.36 (and earlier) versions are rock solid with many users.

The classic dmcrypt is single threaded and very slow.

> As another thought, what version of GCC are people using who are having difficulty?   Could this perhaps be a compiler-related issue?

A compiler problem seems very unlikely here.

What may be an useful experiment would be to test 2.6.37-rc + ext4 over
device mapper, but not dmcrypt. If that fails too then it's likely
some generic device mapper problem.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: hunt for 2.6.37 dm-crypt+ext4 corruption?

[Mike Snitzer]

On Sun, Dec 05 2010 at  3:28pm -0500,
Andi Kleen <andi@firstfloor.org> wrote:

> > As another thought, what version of GCC are people using who are having difficulty?   Could this perhaps be a compiler-related issue?
> 
> A compiler problem seems very unlikely here.
> 
> What may be an useful experiment would be to test 2.6.37-rc + ext4 over
> device mapper, but not dmcrypt. If that fails too then it's likely
> some generic device mapper problem.

That would only prove its not dm-crypt; it would not absolve a non-DM
2.6.37-rc change at all.

Leveraging the fact that 2.6.36 + parallel dm-crypt is reportedly very
solid: maybe these would be more interesting (each would keep the latest
parallel dm-crypt patch overlayed for all tests):

1) Test 2.6.37-rc prior to all flush+fua changes (only changes DM saw
   for 2.6.37-rc so far).

2) Back out all ext4 changes that were merged for 2.6.37-rc
   - though Heinz is using XFS exclussively and still reported
     corruption

3) A full git bisect of good=v2.6.36 bad=v2.6.37-rc4 would be the most
   interesting.  Albeit much more tedious.

Mike

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Milan Broz]

On 12/05/2010 09:28 PM, Andi Kleen wrote:
>> I've been using a kernel which is between 2.6.37-rc2 and -rc3 with
>> a LUKS / dm-crypt / LVM / ext4 setup for my primary file systems,
>> and I haven't observed any corruption for the last two weeks or so.
>> It's on my todo list to upgrade to top of Linus's tree, but perhaps
>> this is a useful data point.
> 
> The reported corruptions are with a .37 forward port someone did of
> my parallel kcryptd patch.  The 2.6.36 (and earlier) versions are
> rock solid with many users.

ok, to not confuse which patch version people are testing,
let's use the version in Alasdair's DM repo now (upstream queue):
http://www.kernel.org/pub/linux/kernel/people/agk/patches/2.6/editing/dm-crypt-scale-to-multiple-CPUs.patch
(it should be the same as v5/v6 here on dm-devel -
Andi's patch + my changes for device stacking deadlock)

The same patch (just with minor define fix) is reported rock solid for 2.6.36.

Milan

Re: [dm-devel] hunt for 2.6.37 dm-crypt+ext4 corruption?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 598 bytes --]

On Sun, 05 Dec 2010 08:24:32 EST, Theodore Tso said:

> I've been using a kernel which is between 2.6.37-rc2 and -rc3 with a LUKS /
> dm-crypt / LVM / ext4 setup for my primary file systems, and I haven't observed
> any corruption for the last two weeks or so.

Pretty much exactly the same setup here (LUKS/dm-crypt/LVM/ext4), pretty
consistently running most-recent mmotm kernel, which tends to be somewhat ahead
of linux-next, and I haven't seen anything either. (I admit I haven't tried
-rc4-mmotm1202 yet).

> As another thought, what version of GCC 

rpm -q says 'gcc-4.5.1-5.fc15.x86_64'.  

[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

Re: hunt for 2.6.37 dm-crypt+ext4 corruption?

[Heinz Diehl]

On 05.12.2010, Milan Broz wrote: 

> It still seems to like dmcrypt with its parallel processing is just
> trigger to another bug in 37-rc.

To come back to this: my 3 systems (XFS filesystem) running the latest 
dm-crypt-scale-to-multiple-cpus patch from Andi Kleen/Milan Broz have 
not showed a single problem since 2.6.37-rc6 and above. No corruption any 
longer, no freezes, nothing. The patch applies cleanly to 2.6.37, too, 
and runs just fine.

I blindly guess that my data corruption problem was related to something else in the
2.6.37-rc series up to -rc4/5.

Since this patch is a significant improvement: any chance that it finally gets 
merged into mainline/stable?

Re: hunt for 2.6.37 dm-crypt+ext4 corruption?

[Matt]

On Thu, Jan 6, 2011 at 4:56 PM, Heinz Diehl <htd@fancy-poultry.org> wrote:
> On 05.12.2010, Milan Broz wrote:
>
>> It still seems to like dmcrypt with its parallel processing is just
>> trigger to another bug in 37-rc.
>
> To come back to this: my 3 systems (XFS filesystem) running the latest
> dm-crypt-scale-to-multiple-cpus patch from Andi Kleen/Milan Broz have
> not showed a single problem since 2.6.37-rc6 and above. No corruption any
> longer, no freezes, nothing. The patch applies cleanly to 2.6.37, too,
> and runs just fine.
>
> I blindly guess that my data corruption problem was related to something else in the
> 2.6.37-rc series up to -rc4/5.
>
> Since this patch is a significant improvement: any chance that it finally gets
> merged into mainline/stable?
>
>

Hi Heinz,

I've been using this patch since 2.6.37-rc6+ with ext4 and xfs
filesystems and haven't seen any corruptions since then
(ext4 got "fixed" since 2.6.37-rc6, xfs showed no problems from the start)

http://git.eu.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=1449032be17abb69116dbc393f67ceb8bd034f92
(is the actual temporary fix for ext4)

Regards

Matt

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Sun, Dec 5, 2010 at 11:09 AM, Heinz Diehl <htd@fritha.org> wrote:
> On 05.12.2010, Matt wrote:
>
>> I should have made it clear that the results I get are observed when
>> using the kernels/checkouts *with* the dm-crypt multi-cpu patch,
>> without the patch I didn't see that kind of problems (hardlocks, files
>> missing, etc.)
>
> I have to take back my other two emails, stating that no corruption
> happened with the dm-crypt multi-cpu patch. Today, I encountered
> filesystem corruption on one, and a complete hardlock on another machine.
> No logfile entries, no m-sysrq, a complete deadlock. Filesystem was
> corrupted here too, had to reboot from CD.
>
> The machines with plain 2.6.37-rc4 are up and running...
>
>

Hi Heinz,

that's bad news :(
hopefully you had a backup available


to complete my last report with the corruption/empty file for
/var/db/pkg/sys-devel/patch-2.6.1/COUNTER

(kernel running was from commit bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc)

I got more and more corruption and the following BUG message in dmesg:

[ 6097.179452] ------------[ cut here ]------------
[ 6097.179456] kernel BUG at fs/ext4/inode.c:2717!
[ 6097.179457] invalid opcode: 0000 [#1] PREEMPT SMP
[ 6097.179459] last sysfs file:
/sys/devices/system/cpu/cpu7/cache/index2/shared_cpu_map
[ 6097.179461] CPU 5
[ 6097.179462] Modules linked in: it87 hwmon_vid coretemp hwmon
fglrx(P) firewire_ohci firewire_core i2c_i801 e1000e wmi shpchp tg3
libphy e1000 scsi_wait_scan sl811_hcd ohci_hcd ssb usb_storage
ehci_hcd
[ 6097.179472]
[ 6097.179475] Pid: 4540, comm: jbd2/dm-2-8 Tainted: P
2.6.36-rc6_bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc+ #1 FMP55/ipower
G3710
[ 6097.179477] RIP: 0010:[<ffffffff8119cc50>]  [<ffffffff8119cc50>]
ext4_writepage+0x270/0x280
[ 6097.179482] RSP: 0018:ffff8801baec3b40  EFLAGS: 00010246
[ 6097.179483] RAX: 8000000000020069 RBX: ffffea0004222088 RCX: 0000000000000030
[ 6097.179485] RDX: 0000000000000a0f RSI: ffff8801baec3cc0 RDI: ffffea0004222088
[ 6097.179486] RBP: ffff8801472ca6a0 R08: ffff880183cc6de8 R09: 0000000000000000
[ 6097.179488] R10: 0000000000000008 R11: 0000000000000010 R12: 0000000000000a0f
[ 6097.179489] R13: 0000000000001000 R14: ffff8801baec3cc0 R15: ffff8801baec3c10
[ 6097.179491] FS:  0000000000000000(0000) GS:ffff880002140000(0000)
knlGS:0000000000000000
[ 6097.179492] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 6097.179494] CR2: 00007f19058591f0 CR3: 0000000001c04000 CR4: 00000000000006e0
[ 6097.179495] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6097.179497] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 6097.179498] Process jbd2/dm-2-8 (pid: 4540, threadinfo
ffff8801baec2000, task ffff8801bfef0040)
[ 6097.179500] Stack:
[ 6097.179500]  ffff8801baec3c10 ffffffff810fce75 ffff8801472ca808
ffff8801472ca808
[ 6097.179503] <0> ffff8801472ca808 0000000000000a0f ffffea0004222088
0000000000000003
[ 6097.179506] <0> ffff8801baec3c10 ffffffff810a261d 000000000000000e
ffffffff810a2ab1
[ 6097.179509] Call Trace:
[ 6097.179513]  [<ffffffff810fce75>] ? __set_page_dirty_buffers+0x85/0xe0
[ 6097.179517]  [<ffffffff810a261d>] ? __writepage+0xd/0x40
[ 6097.179519]  [<ffffffff810a2ab1>] ? write_cache_pages+0x1b1/0x3d0
[ 6097.179521]  [<ffffffff810a2610>] ? __writepage+0x0/0x40
[ 6097.179525]  [<ffffffff811d1869>] ? journal_submit_data_buffers+0x99/0x100
[ 6097.179528]  [<ffffffff811d1db1>] ?
jbd2_journal_commit_transaction+0x331/0x1330
[ 6097.179532]  [<ffffffff8172097b>] ? schedule+0x37b/0x8d0
[ 6097.179534]  [<ffffffff817237f8>] ? _raw_spin_lock_irqsave+0x18/0x20
[ 6097.179538]  [<ffffffff810538c3>] ? lock_timer_base.clone.23+0x33/0x70
[ 6097.179540]  [<ffffffff8105396e>] ? try_to_del_timer_sync+0x6e/0x90
[ 6097.179542]  [<ffffffff811d5d91>] ? kjournald2+0xb1/0x210
[ 6097.179545]  [<ffffffff81062b80>] ? autoremove_wake_function+0x0/0x30
[ 6097.179547]  [<ffffffff811d5ce0>] ? kjournald2+0x0/0x210
[ 6097.179549]  [<ffffffff811d5ce0>] ? kjournald2+0x0/0x210
[ 6097.179551]  [<ffffffff81062706>] ? kthread+0x96/0xa0
[ 6097.179555]  [<ffffffff81003394>] ? kernel_thread_helper+0x4/0x10
[ 6097.179557]  [<ffffffff81062670>] ? kthread+0x0/0xa0
[ 6097.179559]  [<ffffffff81003390>] ? kernel_thread_helper+0x0/0x10
[ 6097.179560] Code: ff f6 c4 40 0f 84 4a fe ff ff e9 77 ff ff ff 48
c7 c6 f0 3f 82 81 48 c7 c7 40 b1 9d 81 31 c0 e8 e0 34 58 00 e9 6e fe
ff ff 0f 0b <0f> 0b 4d 8b 6d 10 e9 96 fe ff ff 0f 1f 44 00 00 48 83 ec
08 ba
[ 6097.179580] RIP  [<ffffffff8119cc50>] ext4_writepage+0x270/0x280
[ 6097.179583]  RSP <ffff8801baec3b40>
[ 6097.179584] ---[ end trace 5199c452c07fe3ec ]---

I saved the output to my boot partition via dmesg > dmesg_bd2d.txt


after that when running eselect (only the command) while it parsed the
available config files - it would say that files are missing for
eselect mesa

------------------------------------------------------------------------------------------------------------------------------
*so it would look like so:*
------------------------------------------------------------------------------------------------------------------------------

eselect
Usage: eselect <global options> <module name> <module options>

Global options:
  --brief                   Make output shorter
  --no-color,--no-colour    Disable coloured output

Built-in modules:
  help                      Display a help message
  usage                     Display a usage message
  version                   Display version information

Extra modules:
  bashcomp                  Manage contributed bash-completion scripts
  binutils                  Manage installed versions of sys-devel/binutils
  boost                     Manage boost installations
  cblas                     Manage installed CBLAS implementations
  ctags                     Manage /usr/bin/ctags implementations
  ecj                       Manage ECJ targets
  editor                    Manage the EDITOR environment variable
  env                       Manage environment variables set in /etc/env.d/
  esd                       Select esound daemon or wrapper
  fontconfig                Manage fontconfig /etc/fonts/conf.d/ symlinks
  java-nsplugin             Manage the Java plugin for Netscape-like Browsers
  java-vm                   Manage the Java system and user VM
  kernel                    Manage the /usr/src/linux symlink
/usr/share/eselect/modules/mesa.eselect: line 15:
/usr/share/mesa/eselect-mesa.conf: No such file or directory
!!! Error: Failed to source config
Call stack:
    * source (mesa.eselect:15)
    * load_config (config.bash:105)
    * do_list (modules.eselect:58)
    * check_do (core.bash:24)
    * do_action (core.bash:89)
    * es_do_list_modules (eselect:122)
    * es_do_help (eselect:99)
    * main (eselect:194)
  mesa                      No description available
  modules                   A module for querying modules. By default, it lists
                            all available modules
  news                      Read Gentoo ("GLEP 42") news items
  opengl                    Manage the OpenGL implementation used by your
                            system
  package-manager           Manage the PACKAGE_MANAGER environment variable
  pager                     Manage the PAGER environment variable
  php                       Manage php installations
  pinentry                  Manage /usr/bin/pinentry symlink
  postgresql                Manage postgresql slots
  profile                   Manage the /etc/make.profile symlink
  python                    Manage Python symlinks
  rc                        Manage /etc/init.d scripts in runlevels
  ruby                      Manage ruby symlinks
  vi                        Manage /usr/bin/vi implementations
  visual                    Manage the VISUAL environment variable
  wxwidgets                 Manage the system default wxWidgets profile.
  xvmc                      Manage the XvMC implementation used by your system
exiting

------------------------------------------------------------------------------------------------------------------------------
*normally it would look like:*
------------------------------------------------------------------------------------------------------------------------------

eselect
Usage: eselect <global options> <module name> <module options>

Global options:
  --brief                   Make output shorter
  --no-color,--no-colour    Disable coloured output

Built-in modules:
  help                      Display a help message
  usage                     Display a usage message
  version                   Display version information

Extra modules:
  bashcomp                  Manage contributed bash-completion scripts
  binutils                  Manage installed versions of sys-devel/binutils
  boost                     Manage boost installations
  cblas                     Manage installed CBLAS implementations
  ctags                     Manage /usr/bin/ctags implementations
  ecj                       Manage ECJ targets
  editor                    Manage the EDITOR environment variable
  env                       Manage environment variables set in /etc/env.d/
  esd                       Select esound daemon or wrapper
  fontconfig                Manage fontconfig /etc/fonts/conf.d/ symlinks
  java-nsplugin             Manage the Java plugin for Netscape-like Browsers
  java-vm                   Manage the Java system and user VM
  kernel                    Manage the /usr/src/linux symlink
  mesa                      Manage the OpenGL driver architecture used by
                            media-libs/mesa
  modules                   A module for querying modules. By default, it lists
                            all available modules
  news                      Read Gentoo ("GLEP 42") news items
  opengl                    Manage the OpenGL implementation used by your
                            system
  package-manager           Manage the PACKAGE_MANAGER environment variable
  pager                     Manage the PAGER environment variable
  php                       Manage php installations
  pinentry                  Manage /usr/bin/pinentry symlink
  postgresql                Manage postgresql slots
  profile                   Manage the /etc/make.profile symlink
  python                    Manage Python symlinks
  rc                        Manage /etc/init.d scripts in runlevels
  ruby                      Manage ruby symlinks
  vi                        Manage /usr/bin/vi implementations
  visual                    Manage the VISUAL environment variable
  wxwidgets                 Manage the system default wxWidgets profile.
  xvmc                      Manage the XvMC implementation used by your system


then I continued some browsing on the web (reading stuff & research)
and letting music (mp3) run

rekonq was fired up, for some minutes it worked, and then it hang
and/or slowed down during loading of websites - it eventually finished
it couldn't really be closed - only via killlall
and it remained in D (daemon ?) state in htop

ps aux | grep rekonq showed:
[rekonq] <defunct>

addr2line didn't reveal much useful - only: inode.c:0

shortly before I wanted to shutdown my box I to tried to sync stuff to
post some info (if you guys need it) but syncing wouldn't work anymore

time sync && sdparm -C sync /dev/sdd

the Magic SYSRQ key still (half-way) worked: closing apps, etc. was
possible but syncing also didn't work anymore

so I did a Alt + Print + REI UO


short summary what happened:

the kernel running is from checkout/commit:
bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
1) emerge -1 sys-devel/gcc
COUNTER file in /var/db/pkg/sys-devel/patch-2.6.1/ got corrupted/empty
2) config files of eselect mesa got corrupted
3) rekonq would hang/slow down for several moments while loading
websites - it would continue loading them eventually but further usage
wasn't possible anymore

4) somewhere between 1 and 3 the BUG message appeared in dmesg
5) syncing wouldn't work anymore

*) often un- and re-mounting is my /boot-partition
*) my portage-tree is on the XFS-partition with delayed logging
*) the other ext4-partition (dm-3) is my /home-partition mounted ro (read-only)

Seems like it's really some issues with ext4 ...

Below you'll find the output of dmesg for the full session

Thanks & Regards

Matt



[    0.000000] Linux version
2.6.36-rc6_bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc+ (root@lupus) (gcc
version 4.5.1 (Gentoo Hardened 4.5.1-r1 p1.4, pie-0.4.5) ) #1 SMP
PREEMPT Thu Dec 2 21:23:25 CET 2010
[    0.000000] Command line: dolvm root=/dev/ram0 init=/linuxrc
ramdisk=8192 crypt_root=/dev/sdd6 real_root=/dev/mapper/XPS-ROOT
noresume noresume2 udev ro elevator=deadline
snd-hda-intel.enable_msi=1 fbcon=scrollback:256K pax_softmode=1
clocksource=tsc usbcore.autosuspend=1 raid=noautodetect
video=vesafb:mtrr:3,ywrap vga=794 nomodeset
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
[    0.000000]  BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000bf780000 (usable)
[    0.000000]  BIOS-e820: 00000000bf780000 - 00000000bf78e000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bf78e000 - 00000000bf7d0000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf7ed000 - 00000000c0000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
[    0.000000]  BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 00000001c0000000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI present.
[    0.000000] AMI BIOS detected: BIOS may corrupt low RAM, working around it.
[    0.000000] e820 update range: 0000000000000000 - 0000000000010000
(usable) ==> (reserved)
[    0.000000] e820 update range: 0000000000000000 - 0000000000001000
(usable) ==> (reserved)
[    0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x1c0000 max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: uncachable
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-BFFFF uncachable
[    0.000000]   C0000-D3FFF write-protect
[    0.000000]   D4000-DFFFF uncachable
[    0.000000]   E0000-E3FFF write-protect
[    0.000000]   E4000-E7FFF write-through
[    0.000000]   E8000-EBFFF write-protect
[    0.000000]   EC000-EFFFF write-through
[    0.000000]   F0000-FFFFF write-protect
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 1C0000000 mask FC0000000 uncachable
[    0.000000]   1 base 000000000 mask E00000000 write-back
[    0.000000]   2 base 0C0000000 mask FC0000000 uncachable
[    0.000000]   3 disabled
[    0.000000]   4 disabled
[    0.000000]   5 disabled
[    0.000000]   6 disabled
[    0.000000]   7 disabled
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] original variable MTRRs
[    0.000000] reg 0, base: 7GB, range: 1GB, type UC
[    0.000000] reg 1, base: 0GB, range: 8GB, type WB
[    0.000000] reg 2, base: 3GB, range: 1GB, type UC
[    0.000000] total RAM covered: 6144M
[    0.000000] Found optimal setting for mtrr clean up
[    0.000000]  gran_size: 64K 	chunk_size: 64K 	num_reg: 4  	lose cover RAM: 0G
[    0.000000] New variable MTRRs
[    0.000000] reg 0, base: 0GB, range: 2GB, type WB
[    0.000000] reg 1, base: 2GB, range: 1GB, type WB
[    0.000000] reg 2, base: 4GB, range: 2GB, type WB
[    0.000000] reg 3, base: 6GB, range: 1GB, type WB
[    0.000000] e820 update range: 00000000c0000000 - 0000000100000000
(usable) ==> (reserved)
[    0.000000] last_pfn = 0xbf780 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] modified physical RAM map:
[    0.000000]  modified: 0000000000000000 - 0000000000010000 (reserved)
[    0.000000]  modified: 0000000000010000 - 000000000009dc00 (usable)
[    0.000000]  modified: 000000000009dc00 - 00000000000a0000 (reserved)
[    0.000000]  modified: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  modified: 0000000000100000 - 00000000bf780000 (usable)
[    0.000000]  modified: 00000000bf780000 - 00000000bf78e000 (ACPI data)
[    0.000000]  modified: 00000000bf78e000 - 00000000bf7d0000 (ACPI NVS)
[    0.000000]  modified: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
[    0.000000]  modified: 00000000bf7ed000 - 00000000c0000000 (reserved)
[    0.000000]  modified: 00000000fee00000 - 00000000fee01000 (reserved)
[    0.000000]  modified: 00000000ffb00000 - 0000000100000000 (reserved)
[    0.000000]  modified: 0000000100000000 - 00000001c0000000 (usable)
[    0.000000] initial memory mapped : 0 - 20000000
[    0.000000] found SMP MP-table at [ffff8800000ff780] ff780
[    0.000000] init_memory_mapping: 0000000000000000-00000000bf780000
[    0.000000]  0000000000 - 00bf600000 page 2M
[    0.000000]  00bf600000 - 00bf780000 page 4k
[    0.000000] kernel direct mapping tables up to bf780000 @ 16000-1b000
[    0.000000] init_memory_mapping: 0000000100000000-00000001c0000000
[    0.000000]  0100000000 - 01c0000000 page 2M
[    0.000000] kernel direct mapping tables up to 1c0000000 @ 19000-21000
[    0.000000] RAMDISK: 37cd6000 - 37ff0000
[    0.000000] ACPI: RSDP 00000000000f9cf0 00024 (v02 ACPIAM)
[    0.000000] ACPI: XSDT 00000000bf780100 0006C (v01 ACRSYS ACRPRDCT
20100329 MSFT 00000097)
[    0.000000] ACPI: FACP 00000000bf780290 000F4 (v04 ACRSYS FACP1137
20100329 MSFT 00000097)
[    0.000000] ACPI: DSDT 00000000bf7805e0 07E42 (v02  926A1 926A1P15
00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf78e000 00040
[    0.000000] ACPI: APIC 00000000bf780390 0008C (v02 ACRSYS APIC1137
20100329 MSFT 00000097)
[    0.000000] ACPI: MCFG 00000000bf780420 0003C (v01 ACRSYS OEMMCFG
20100329 MSFT 00000097)
[    0.000000] ACPI: SLIC 00000000bf780460 00176 (v01 ACRSYS ACRPRDCT
20100329 MSFT 00000097)
[    0.000000] ACPI: OEMB 00000000bf78e040 00072 (v01 ACRSYS OEMB1137
20100329 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bf78a5e0 00038 (v01 ACRSYS OEMHPET
20100329 MSFT 00000097)
[    0.000000] ACPI: GSCI 00000000bf78e0c0 02024 (v01 ACRSYS GMCHSCI
20100329 MSFT 00000097)
[    0.000000] ACPI: AWMI 00000000bf7900f0 0004E (v01 ACRSYS OEMB1137
20100329 MSFT 00000097)
[    0.000000] ACPI: SSDT 00000000bf792c10 00363 (v01 DpgPmm    CpuPm
00000012 INTL 20051117)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000]  [ffffea0000000000-ffffea00061fffff] PMD ->
[ffff880002600000-ffff8800079fffff] on node 0
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x001c0000
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[3] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x000bf780
[    0.000000]     0: 0x00100000 -> 0x001c0000
[    0.000000] On node 0 totalpages: 1570573
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 3925 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 14280 pages used for memmap
[    0.000000]   DMA32 zone: 765880 pages, LIFO batch:31
[    0.000000]   Normal zone: 10752 pages used for memmap
[    0.000000]   Normal zone: 775680 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x808
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
[    0.000000] ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0xffffffff base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] nr_irqs_gsi: 40
[    0.000000] PM: Registered nosave memory: 000000000009d000 - 000000000009e000
[    0.000000] PM: Registered nosave memory: 000000000009e000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
[    0.000000] PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
[    0.000000] PM: Registered nosave memory: 00000000bf780000 - 00000000bf78e000
[    0.000000] PM: Registered nosave memory: 00000000bf78e000 - 00000000bf7d0000
[    0.000000] PM: Registered nosave memory: 00000000bf7d0000 - 00000000bf7e0000
[    0.000000] PM: Registered nosave memory: 00000000bf7e0000 - 00000000bf7ed000
[    0.000000] PM: Registered nosave memory: 00000000bf7ed000 - 00000000c0000000
[    0.000000] PM: Registered nosave memory: 00000000c0000000 - 00000000fee00000
[    0.000000] PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
[    0.000000] PM: Registered nosave memory: 00000000fee01000 - 00000000ffb00000
[    0.000000] PM: Registered nosave memory: 00000000ffb00000 - 0000000100000000
[    0.000000] Allocating PCI resources starting at c0000000 (gap:
c0000000:3ee00000)
[    0.000000] early_res array is doubled to 64 at [1c000 - 1c7ff]
[    0.000000] setup_percpu: NR_CPUS:16 nr_cpumask_bits:16
nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 27 pages/cpu @ffff880002000000 s81088
r8192 d21312 u262144
[    0.000000] pcpu-alloc: s81088 r8192 d21312 u262144 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.
Total pages: 1545485
[    0.000000] Kernel command line: dolvm root=/dev/ram0 init=/linuxrc
ramdisk=8192 crypt_root=/dev/sdd6 real_root=/dev/mapper/XPS-ROOT
noresume noresume2 udev ro elevator=deadline
snd-hda-intel.enable_msi=1 fbcon=scrollback:256K pax_softmode=1
clocksource=tsc usbcore.autosuspend=1 raid=noautodetect
video=vesafb:mtrr:3,ywrap vga=794 nomodeset
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11,
8388608 bytes)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Calgary: detecting Calgary via BIOS EBDA area
[    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
[    0.000000] Subtract (51 early reservations)
[    0.000000]   #1 [0001000000 - 0001ded3cc]   TEXT DATA BSS
[    0.000000]   #2 [0037cd6000 - 0037ff0000]         RAMDISK
[    0.000000]   #3 [0001dee000 - 0001dee243]             BRK
[    0.000000]   #4 [00000ff790 - 0000100000]   BIOS reserved
[    0.000000]   #5 [00000ff780 - 00000ff790]    MP-table mpf
[    0.000000]   #6 [000009dc00 - 00000fced0]   BIOS reserved
[    0.000000]   #7 [00000fd074 - 00000ff780]   BIOS reserved
[    0.000000]   #8 [00000fced0 - 00000fd074]    MP-table mpc
[    0.000000]   #9 [0000010000 - 0000012000]      TRAMPOLINE
[    0.000000]   #10 [0000012000 - 0000016000]     ACPI WAKEUP
[    0.000000]   #11 [0000016000 - 0000019000]         PGTABLE
[    0.000000]   #12 [0000019000 - 000001c000]         PGTABLE
[    0.000000]   #13 [0001dee280 - 0001def280]         BOOTMEM
[    0.000000]   #14 [0001ded400 - 0001ded880]         BOOTMEM
[    0.000000]   #15 [00025f0000 - 00025f1000]         BOOTMEM
[    0.000000]   #16 [00025f1000 - 00025f2000]         BOOTMEM
[    0.000000]   #17 [0002600000 - 0007a00000]        MEMMAP 0
[    0.000000]   #18 [0001ded880 - 0001dedb00]         BOOTMEM
[    0.000000]   #19 [0001def280 - 0001e17280]         BOOTMEM
[    0.000000]   #20 [0001e17280 - 0001e3f280]         BOOTMEM
[    0.000000]   #21 [0001e40000 - 0001e41000]         BOOTMEM
[    0.000000]   #22 [0001dedb00 - 0001dedb41]         BOOTMEM
[    0.000000]   #23 [0001dedb80 - 0001dedbc3]         BOOTMEM
[    0.000000]   #24 [0001dedc00 - 0001dedea0]         BOOTMEM
[    0.000000]   #25 [0001dedec0 - 0001dedee0]         BOOTMEM
[    0.000000]   #26 [0001dedf00 - 0001dedf20]         BOOTMEM
[    0.000000]   #27 [0001e3f280 - 0001e3f3b5]         BOOTMEM
[    0.000000]   #28 [0001e3f3c0 - 0001e3f4f5]         BOOTMEM
[    0.000000]   #29 [0002000000 - 000201b000]         BOOTMEM
[    0.000000]   #30 [0002040000 - 000205b000]         BOOTMEM
[    0.000000]   #31 [0002080000 - 000209b000]         BOOTMEM
[    0.000000]   #32 [00020c0000 - 00020db000]         BOOTMEM
[    0.000000]   #33 [0002100000 - 000211b000]         BOOTMEM
[    0.000000]   #34 [0002140000 - 000215b000]         BOOTMEM
[    0.000000]   #35 [0002180000 - 000219b000]         BOOTMEM
[    0.000000]   #36 [00021c0000 - 00021db000]         BOOTMEM
[    0.000000]   #37 [0001dedf40 - 0001dedf48]         BOOTMEM
[    0.000000]   #38 [0001dedf80 - 0001dedf88]         BOOTMEM
[    0.000000]   #39 [0001dedfc0 - 0001dedfe0]         BOOTMEM
[    0.000000]   #40 [0001e3f500 - 0001e3f540]         BOOTMEM
[    0.000000]   #41 [0001e3f540 - 0001e3f660]         BOOTMEM
[    0.000000]   #42 [0001e3f680 - 0001e3f6c8]         BOOTMEM
[    0.000000]   #43 [0001e3f700 - 0001e3f748]         BOOTMEM
[    0.000000]   #44 [0001e41000 - 0001e49000]         BOOTMEM
[    0.000000]   #45 [0007a00000 - 0008200000]         BOOTMEM
[    0.000000]   #46 [00021db000 - 00025db000]         BOOTMEM
[    0.000000]   #47 [0008200000 - 000c200000]         BOOTMEM
[    0.000000]   #48 [0001e49000 - 0001e69000]         BOOTMEM
[    0.000000]   #49 [0001e69000 - 0001ea9000]         BOOTMEM
[    0.000000]   #50 [000001c800 - 0000024800]         BOOTMEM
[    0.000000] Memory: 6099308k/7340032k available (7321k kernel code,
1057740k absent, 182984k reserved, 5818k data, 548k init)
[    0.000000] Preemptable hierarchical RCU implementation.
[    0.000000] 	RCU-based detection of stalled CPUs is disabled.
[    0.000000] 	Verbose stalled-CPUs detection is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:744
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] ------------------------
[    0.000000] | Locking API testsuite:
[    0.000000] ----------------------------------------------------------------------------
[    0.000000]                                  | spin |wlock |rlock
|mutex | wsem | rsem |
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]                      A-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]                  A-B-B-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]              A-B-B-C-C-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]              A-B-C-A-B-C deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]          A-B-B-C-C-D-D-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]          A-B-C-D-B-D-D-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]          A-B-C-D-B-C-D-A deadlock:failed|failed|  ok
|failed|failed|failed|
[    0.000000]                     double unlock:  ok  |  ok  |failed|
 ok  |failed|failed|
[    0.000000]                   initialize
held:failed|failed|failed|failed|failed|failed|
[    0.000000]                  bad unlock order:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]               recursive read-lock:             |  ok  |
            |failed|
[    0.000000]            recursive read-lock #2:             |  ok  |
            |failed|
[    0.000000]             mixed read-write-lock:             |failed|
            |failed|
[    0.000000]             mixed write-read-lock:             |failed|
            |failed|
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]      hard-irqs-on + irq-safe-A/12:failed|failed|  ok  |
[    0.000000]      soft-irqs-on + irq-safe-A/12:failed|failed|  ok  |
[    0.000000]      hard-irqs-on + irq-safe-A/21:failed|failed|  ok  |
[    0.000000]      soft-irqs-on + irq-safe-A/21:failed|failed|  ok  |
[    0.000000]        sirq-safe-A => hirqs-on/12:failed|failed|  ok  |
[    0.000000]        sirq-safe-A => hirqs-on/21:failed|failed|  ok  |
[    0.000000]          hard-safe-A + irqs-on/12:failed|failed|  ok  |
[    0.000000]          soft-safe-A + irqs-on/12:failed|failed|  ok  |
[    0.000000]          hard-safe-A + irqs-on/21:failed|failed|  ok  |
[    0.000000]          soft-safe-A + irqs-on/21:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/123:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/123:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/132:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/132:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/213:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/213:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/231:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/231:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/312:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/312:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/321:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/321:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/123:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/123:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/132:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/132:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/213:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/213:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/231:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/231:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/312:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/312:failed|failed|  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/321:failed|failed|  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/321:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/123:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/123:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/132:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/132:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/213:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/213:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/231:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/231:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/312:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/312:failed|failed|  ok  |
[    0.000000]       hard-irq lock-inversion/321:failed|failed|  ok  |
[    0.000000]       soft-irq lock-inversion/321:failed|failed|  ok  |
[    0.000000]       hard-irq read-recursion/123:  ok  |
[    0.000000]       soft-irq read-recursion/123:  ok  |
[    0.000000]       hard-irq read-recursion/132:  ok  |
[    0.000000]       soft-irq read-recursion/132:  ok  |
[    0.000000]       hard-irq read-recursion/213:  ok  |
[    0.000000]       soft-irq read-recursion/213:  ok  |
[    0.000000]       hard-irq read-recursion/231:  ok  |
[    0.000000]       soft-irq read-recursion/231:  ok  |
[    0.000000]       hard-irq read-recursion/312:  ok  |
[    0.000000]       soft-irq read-recursion/312:  ok  |
[    0.000000]       hard-irq read-recursion/321:  ok  |
[    0.000000]       soft-irq read-recursion/321:  ok  |
[    0.000000] --------------------------------------------------------
[    0.000000] 142 out of 218 testcases failed, as expected. |
[    0.000000] ----------------------------------------------------
[    0.000000] hpet clockevent registered
[    0.000000] Fast TSC calibration using PIT
[    0.001000] Detected 2792.737 MHz processor.
[    0.000003] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5585.47 BogoMIPS (lpj=2792737)
[    0.000011] pid_max: default: 32768 minimum: 301
[    0.000066] Mount-cache hash table entries: 256
[    0.000161] CPU: Physical Processor ID: 0
[    0.000164] CPU: Processor Core ID: 0
[    0.000169] mce: CPU supports 9 MCE banks
[    0.000182] CPU0: Thermal monitoring enabled (TM1)
[    0.000191] using mwait in idle threads.
[    0.000194] Performance Events: PEBS fmt1+, Nehalem events, Intel PMU driver.
[    0.000203] ... version:                3
[    0.000205] ... bit width:              48
[    0.000208] ... generic registers:      4
[    0.000211] ... value mask:             0000ffffffffffff
[    0.000215] ... max period:             000000007fffffff
[    0.000218] ... fixed-purpose events:   3
[    0.000221] ... event mask:             000000070000000f
[    0.000273] ACPI: Core revision 20100702
[    0.028502] Setting APIC routing to flat
[    0.028842] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.038829] CPU0: Intel(R) Core(TM) i7 CPU         860  @ 2.80GHz stepping 05
[    0.142448] NMI watchdog enabled, takes one hw-pmu counter.
[    0.145357] Booting Node   0, Processors  #1
[    0.236253] NMI watchdog enabled, takes one hw-pmu counter.
[    0.238184]  #2
[    0.329074] NMI watchdog enabled, takes one hw-pmu counter.
[    0.331006]  #3
[    0.421900] NMI watchdog enabled, takes one hw-pmu counter.
[    0.423829]  #4
[    0.514734] NMI watchdog enabled, takes one hw-pmu counter.
[    0.516650]  #5
[    0.607527] NMI watchdog enabled, takes one hw-pmu counter.
[    0.609475]  #6
[    0.700353] NMI watchdog enabled, takes one hw-pmu counter.
[    0.702295]  #7 Ok.
[    0.793176] NMI watchdog enabled, takes one hw-pmu counter.
[    0.794119] Brought up 8 CPUs
[    0.794125] Total of 8 processors activated (44680.64 BogoMIPS).
[    0.797213] xor: automatically using best checksumming function: generic_sse
[    0.802099]    generic_sse: 10436.000 MB/sec
[    0.802102] xor: using function: generic_sse (10436.000 MB/sec)
[    0.802143] NET: Registered protocol family 16
[    0.802312] ACPI FADT declares the system doesn't support PCIe
ASPM, so disable it
[    0.802317] ACPI: bus type pci registered
[    0.802390] dca service started, version 1.12.1
[    0.802429] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem
0xe0000000-0xefffffff] (base 0xe0000000)
[    0.802434] PCI: not using MMCONFIG
[    0.802436] PCI: Using configuration type 1 for base access
[    0.806651] bio: create slab <bio-0> at 0
[    0.823061] raid6: int64x1   2539 MB/s
[    0.840038] raid6: int64x2   3042 MB/s
[    0.857018] raid6: int64x4   2148 MB/s
[    0.873976] raid6: int64x8   2183 MB/s
[    0.890937] raid6: sse2x1    6937 MB/s
[    0.907901] raid6: sse2x2    8132 MB/s
[    0.924865] raid6: sse2x4    9089 MB/s
[    0.924868] raid6: using algorithm sse2x4 (9089 MB/s)
[    0.925935] ACPI: EC: Look up EC in DSDT
[    0.927582] ACPI: Executed 1 blocks of module-level executable AML code
[    0.931492] ACPI: SSDT 00000000bf790140 0244C (v01 DpgPmm  P001Ist
00000011 INTL 20051117)
[    0.931811] ACPI: Dynamic OEM Table Load:
[    0.931815] ACPI: SSDT (null) 0244C (v01 DpgPmm  P001Ist 00000011
INTL 20051117)
[    0.931995] ACPI: SSDT 00000000bf792590 00678 (v01  PmRef  P001Cst
00003001 INTL 20051117)
[    0.932266] ACPI: Dynamic OEM Table Load:
[    0.932270] ACPI: SSDT (null) 00678 (v01  PmRef  P001Cst 00003001
INTL 20051117)
[    0.932338] ACPI: Interpreter enabled
[    0.932341] ACPI: (supports S0 S3 S4 S5)
[    0.932357] ACPI: Using IOAPIC for interrupt routing
[    0.932381] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem
0xe0000000-0xefffffff] (base 0xe0000000)
[    0.934570] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved
in ACPI motherboard resources
[    0.969233] ACPI: No dock devices found.
[    0.969237] PCI: Using host bridge windows from ACPI; if necessary,
use "pci=nocrs" and report a bug
[    0.969433] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.969621] pci_root PNP0A08:00: host bridge window [io  0x0000-0x0cf7]
[    0.969626] pci_root PNP0A08:00: host bridge window [io  0x0d00-0xffff]
[    0.969629] pci_root PNP0A08:00: host bridge window [mem
0x000a0000-0x000bffff]
[    0.969633] pci_root PNP0A08:00: host bridge window [mem
0x000d0000-0x000dffff]
[    0.969638] pci_root PNP0A08:00: host bridge window [mem
0xc0000000-0xdfffffff]
[    0.969641] pci_root PNP0A08:00: host bridge window [mem
0xf0000000-0xfed8ffff]
[    0.969725] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
[    0.969727] pci 0000:00:03.0: PME# disabled
[    0.969946] pci 0000:00:19.0: reg 10: [mem 0xfbcc0000-0xfbcdffff]
[    0.969953] pci 0000:00:19.0: reg 14: [mem 0xfbcfc000-0xfbcfcfff]
[    0.969960] pci 0000:00:19.0: reg 18: [io  0xbc00-0xbc1f]
[    0.970004] pci 0000:00:19.0: PME# supported from D0 D3hot D3cold
[    0.970007] pci 0000:00:19.0: PME# disabled
[    0.970044] pci 0000:00:1a.0: reg 10: [mem 0xfbcfa000-0xfbcfa3ff]
[    0.970110] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold
[    0.970113] pci 0000:00:1a.0: PME# disabled
[    0.970145] pci 0000:00:1b.0: reg 10: [mem 0xfbcf4000-0xfbcf7fff 64bit]
[    0.970192] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    0.970195] pci 0000:00:1b.0: PME# disabled
[    0.970257] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    0.970260] pci 0000:00:1c.0: PME# disabled
[    0.970323] pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
[    0.970326] pci 0000:00:1c.1: PME# disabled
[    0.970388] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold
[    0.970391] pci 0000:00:1c.2: PME# disabled
[    0.970454] pci 0000:00:1c.3: PME# supported from D0 D3hot D3cold
[    0.970457] pci 0000:00:1c.3: PME# disabled
[    0.970520] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold
[    0.970523] pci 0000:00:1c.4: PME# disabled
[    0.970564] pci 0000:00:1d.0: reg 10: [mem 0xfbcf8000-0xfbcf83ff]
[    0.970630] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[    0.970633] pci 0000:00:1d.0: PME# disabled
[    0.970816] pci 0000:00:1f.2: reg 10: [io  0xb880-0xb887]
[    0.970822] pci 0000:00:1f.2: reg 14: [io  0xb800-0xb803]
[    0.970829] pci 0000:00:1f.2: reg 18: [io  0xb480-0xb487]
[    0.970836] pci 0000:00:1f.2: reg 1c: [io  0xb400-0xb403]
[    0.970843] pci 0000:00:1f.2: reg 20: [io  0xb080-0xb09f]
[    0.970849] pci 0000:00:1f.2: reg 24: [mem 0xfbcf2000-0xfbcf27ff]
[    0.970879] pci 0000:00:1f.2: PME# supported from D3hot
[    0.970882] pci 0000:00:1f.2: PME# disabled
[    0.970907] pci 0000:00:1f.3: reg 10: [mem 0xfbcff800-0xfbcff8ff 64bit]
[    0.970926] pci 0000:00:1f.3: reg 20: [io  0x0400-0x041f]
[    0.970991] pci 0000:01:00.0: reg 10: [mem 0xd0000000-0xdfffffff 64bit pref]
[    0.971000] pci 0000:01:00.0: reg 18: [mem 0xfbde0000-0xfbdfffff 64bit]
[    0.971007] pci 0000:01:00.0: reg 20: [io  0xc000-0xc0ff]
[    0.971018] pci 0000:01:00.0: reg 30: [mem 0xfbdc0000-0xfbddffff pref]
[    0.971034] pci 0000:01:00.0: supports D1 D2
[    0.971061] pci 0000:01:00.1: reg 10: [mem 0xfbdbc000-0xfbdbffff 64bit]
[    0.971102] pci 0000:01:00.1: supports D1 D2
[    0.971116] pci 0000:00:03.0: PCI bridge to [bus 01-01]
[    0.971120] pci 0000:00:03.0:   bridge window [io  0xc000-0xcfff]
[    0.971123] pci 0000:00:03.0:   bridge window [mem 0xfbd00000-0xfbdfffff]
[    0.971126] pci 0000:00:03.0:   bridge window [mem
0xd0000000-0xdfffffff 64bit pref]
[    0.971161] pci 0000:00:1c.0: PCI bridge to [bus 02-02]
[    0.971166] pci 0000:00:1c.0:   bridge window [io  0xf000-0x0000] (disabled)
[    0.971169] pci 0000:00:1c.0:   bridge window [mem
0xfff00000-0x000fffff] (disabled)
[    0.971174] pci 0000:00:1c.0:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971305] pci 0000:03:00.0: reg 24: [mem 0xfbefe000-0xfbefffff]
[    0.971346] pci 0000:03:00.0: PME# supported from D3hot
[    0.971350] pci 0000:03:00.0: PME# disabled
[    0.971396] pci 0000:03:00.1: reg 10: [io  0xdc00-0xdc07]
[    0.971409] pci 0000:03:00.1: reg 14: [io  0xd880-0xd883]
[    0.971421] pci 0000:03:00.1: reg 18: [io  0xd800-0xd807]
[    0.971434] pci 0000:03:00.1: reg 1c: [io  0xd480-0xd483]
[    0.971446] pci 0000:03:00.1: reg 20: [io  0xd400-0xd40f]
[    0.971512] pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe
device.  You can enable it with 'pcie_aspm=force'
[    0.971517] pci 0000:00:1c.1: PCI bridge to [bus 03-03]
[    0.971522] pci 0000:00:1c.1:   bridge window [io  0xd000-0xdfff]
[    0.971525] pci 0000:00:1c.1:   bridge window [mem 0xfbe00000-0xfbefffff]
[    0.971529] pci 0000:00:1c.1:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971564] pci 0000:00:1c.2: PCI bridge to [bus 04-04]
[    0.971569] pci 0000:00:1c.2:   bridge window [io  0xf000-0x0000] (disabled)
[    0.971572] pci 0000:00:1c.2:   bridge window [mem
0xfff00000-0x000fffff] (disabled)
[    0.971577] pci 0000:00:1c.2:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971612] pci 0000:00:1c.3: PCI bridge to [bus 05-05]
[    0.971616] pci 0000:00:1c.3:   bridge window [io  0xf000-0x0000] (disabled)
[    0.971619] pci 0000:00:1c.3:   bridge window [mem
0xfff00000-0x000fffff] (disabled)
[    0.971624] pci 0000:00:1c.3:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971659] pci 0000:00:1c.4: PCI bridge to [bus 06-06]
[    0.971664] pci 0000:00:1c.4:   bridge window [io  0xf000-0x0000] (disabled)
[    0.971667] pci 0000:00:1c.4:   bridge window [mem
0xfff00000-0x000fffff] (disabled)
[    0.971672] pci 0000:00:1c.4:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971720] pci 0000:07:06.0: reg 10: [mem 0xfbfff800-0xfbffffff]
[    0.971729] pci 0000:07:06.0: reg 14: [io  0xec00-0xec7f]
[    0.971785] pci 0000:07:06.0: supports D2
[    0.971787] pci 0000:07:06.0: PME# supported from D2 D3hot D3cold
[    0.971790] pci 0000:07:06.0: PME# disabled
[    0.971823] pci 0000:00:1e.0: PCI bridge to [bus 07-07] (subtractive decode)
[    0.971828] pci 0000:00:1e.0:   bridge window [io  0xe000-0xefff]
[    0.971831] pci 0000:00:1e.0:   bridge window [mem 0xfbf00000-0xfbffffff]
[    0.971836] pci 0000:00:1e.0:   bridge window [mem
0xfff00000-0x000fffff pref] (disabled)
[    0.971838] pci 0000:00:1e.0:   bridge window [io  0x0000-0x0cf7]
(subtractive decode)
[    0.971839] pci 0000:00:1e.0:   bridge window [io  0x0d00-0xffff]
(subtractive decode)
[    0.971841] pci 0000:00:1e.0:   bridge window [mem
0x000a0000-0x000bffff] (subtractive decode)
[    0.971843] pci 0000:00:1e.0:   bridge window [mem
0x000d0000-0x000dffff] (subtractive decode)
[    0.971844] pci 0000:00:1e.0:   bridge window [mem
0xc0000000-0xdfffffff] (subtractive decode)
[    0.971846] pci 0000:00:1e.0:   bridge window [mem
0xf0000000-0xfed8ffff] (subtractive decode)
[    0.971871] pci_bus 0000:00: on NUMA node 0
[    0.971873] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[    0.971979] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR1E._PRT]
[    0.972012] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR20._PRT]
[    0.972033] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR21._PRT]
[    0.972060] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR22._PRT]
[    0.972081] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR23._PRT]
[    0.972102] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR24._PRT]
[    0.979937] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15)
[    0.979980] ACPI: PCI Interrupt Link [LNKB] (IRQs *5)
[    0.980018] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 *11 12 14 15)
[    0.980059] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 11 12 14 *15)
[    0.980100] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 *14 15)
[    0.980140] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 7 10 11 12
14 15) *0, disabled.
[    0.980183] ACPI: PCI Interrupt Link [LNKG] (IRQs *3 4 6 7 10 11 12 14 15)
[    0.980224] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 6 *7 10 11 12 14 15)
[    0.980262] HEST: Table is not found!
[    0.980407] SCSI subsystem initialized
[    0.980420] libata version 3.00 loaded.
[    0.980464] usbcore: registered new interface driver usbfs
[    0.980504] usbcore: registered new interface driver hub
[    0.980531] usbcore: registered new device driver usb
[    0.980649] Advanced Linux Sound Architecture Driver Version 1.0.23.
[    0.980652] PCI: Using ACPI for IRQ routing
[    0.980655] PCI: pci_cache_line_size set to 64 bytes
[    0.980718] reserve RAM buffer: 000000000009dc00 - 000000000009ffff
[    0.980720] reserve RAM buffer: 00000000bf780000 - 00000000bfffffff
[    0.980824]   alloc irq_desc for 40 on node 0
[    0.980826]   alloc kstat_irqs on node 0
[    0.980831]   alloc irq_desc for 41 on node 0
[    0.980832]   alloc kstat_irqs on node 0
[    0.980835]   alloc irq_desc for 42 on node 0
[    0.980836]   alloc kstat_irqs on node 0
[    0.980839]   alloc irq_desc for 43 on node 0
[    0.980840]   alloc kstat_irqs on node 0
[    0.980844]   alloc irq_desc for 44 on node 0
[    0.980845]   alloc kstat_irqs on node 0
[    0.980847] HPET: 8 timers in total, 5 timers will be used for per-cpu timer
[    0.980856] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 40, 41, 42, 43, 44, 0
[    0.980863] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[    0.983779] hpet: hpet2 irq 40 for MSI
[    0.983878] hpet: hpet3 irq 41 for MSI
[    0.984888] hpet: hpet4 irq 42 for MSI
[    0.985882] hpet: hpet5 irq 43 for MSI
[    0.986887] hpet: hpet6 irq 44 for MSI
[    0.990871] Switching to clocksource tsc
[    0.990940] pnp: PnP ACPI init
[    0.990948] ACPI: bus type pnp registered
[    0.993389] pnp: PnP ACPI: found 16 devices
[    0.993392] ACPI: ACPI bus type pnp unregistered
[    0.993401] system 00:01: [mem 0xfc000000-0xfcffffff] has been reserved
[    0.993405] system 00:01: [mem 0xfd000000-0xfdffffff] has been reserved
[    0.993409] system 00:01: [mem 0xfe000000-0xfebfffff] has been reserved
[    0.993412] system 00:01: [mem 0xfed14000-0xfed19fff] has been reserved
[    0.993419] system 00:08: [io  0x0a00-0x0a0f] has been reserved
[    0.993422] system 00:08: [io  0x0a10-0x0a1f] has been reserved
[    0.993426] system 00:08: [io  0x0a20-0x0a2f] has been reserved
[    0.993429] system 00:08: [io  0x0a30-0x0a3f] has been reserved
[    0.993434] system 00:09: [io  0x04d0-0x04d1] has been reserved
[    0.993438] system 00:09: [io  0x0800-0x087f] has been reserved
[    0.993441] system 00:09: [io  0x0500-0x057f] has been reserved
[    0.993445] system 00:09: [mem 0xfed1c000-0xfed1ffff] has been reserved
[    0.993449] system 00:09: [mem 0xfed20000-0xfed3ffff] has been reserved
[    0.993452] system 00:09: [mem 0xfed40000-0xfed8ffff] has been reserved
[    0.993458] system 00:0c: [mem 0xffc00000-0xffefffff] has been reserved
[    0.993463] system 00:0d: [mem 0xfec00000-0xfec00fff] could not be reserved
[    0.993467] system 00:0d: [mem 0xfee00000-0xfee00fff] has been reserved
[    0.993472] system 00:0e: [mem 0xe0000000-0xefffffff] has been reserved
[    0.993479] system 00:0f: [mem 0x00000000-0x0009ffff] could not be reserved
[    0.993482] system 00:0f: [mem 0x000c0000-0x000cffff] has been reserved
[    0.993486] system 00:0f: [mem 0x000e0000-0x000fffff] could not be reserved
[    0.993490] system 00:0f: [mem 0x00100000-0xbfffffff] could not be reserved
[    0.993494] system 00:0f: [mem 0xfed90000-0xffffffff] could not be reserved
[    0.999900] pci 0000:00:1c.0: BAR 8: assigned [mem 0xc0000000-0xc01fffff]
[    0.999906] pci 0000:00:1c.0: BAR 9: assigned [mem
0xc0200000-0xc03fffff 64bit pref]
[    0.999911] pci 0000:00:1c.1: BAR 9: assigned [mem
0xc0400000-0xc05fffff 64bit pref]
[    0.999915] pci 0000:00:1c.2: BAR 8: assigned [mem 0xc0600000-0xc07fffff]
[    0.999919] pci 0000:00:1c.2: BAR 9: assigned [mem
0xc0800000-0xc09fffff 64bit pref]
[    0.999923] pci 0000:00:1c.3: BAR 8: assigned [mem 0xc0a00000-0xc0bfffff]
[    0.999927] pci 0000:00:1c.3: BAR 9: assigned [mem
0xc0c00000-0xc0dfffff 64bit pref]
[    0.999931] pci 0000:00:1c.4: BAR 8: assigned [mem 0xc0e00000-0xc0ffffff]
[    0.999935] pci 0000:00:1c.4: BAR 9: assigned [mem
0xc1000000-0xc11fffff 64bit pref]
[    0.999940] pci 0000:00:1c.0: BAR 7: assigned [io  0x1000-0x1fff]
[    0.999943] pci 0000:00:1c.2: BAR 7: assigned [io  0x2000-0x2fff]
[    0.999947] pci 0000:00:1c.3: BAR 7: assigned [io  0x3000-0x3fff]
[    0.999951] pci 0000:00:1c.4: BAR 7: assigned [io  0x4000-0x4fff]
[    0.999954] pci 0000:00:03.0: PCI bridge to [bus 01-01]
[    0.999958] pci 0000:00:03.0:   bridge window [io  0xc000-0xcfff]
[    0.999963] pci 0000:00:03.0:   bridge window [mem 0xfbd00000-0xfbdfffff]
[    0.999967] pci 0000:00:03.0:   bridge window [mem
0xd0000000-0xdfffffff 64bit pref]
[    0.999973] pci 0000:00:1c.0: PCI bridge to [bus 02-02]
[    0.999977] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
[    0.999982] pci 0000:00:1c.0:   bridge window [mem 0xc0000000-0xc01fffff]
[    0.999987] pci 0000:00:1c.0:   bridge window [mem
0xc0200000-0xc03fffff 64bit pref]
[    0.999994] pci 0000:00:1c.1: PCI bridge to [bus 03-03]
[    0.999998] pci 0000:00:1c.1:   bridge window [io  0xd000-0xdfff]
[    1.000004] pci 0000:00:1c.1:   bridge window [mem 0xfbe00000-0xfbefffff]
[    1.000009] pci 0000:00:1c.1:   bridge window [mem
0xc0400000-0xc05fffff 64bit pref]
[    1.000016] pci 0000:00:1c.2: PCI bridge to [bus 04-04]
[    1.000019] pci 0000:00:1c.2:   bridge window [io  0x2000-0x2fff]
[    1.000025] pci 0000:00:1c.2:   bridge window [mem 0xc0600000-0xc07fffff]
[    1.000030] pci 0000:00:1c.2:   bridge window [mem
0xc0800000-0xc09fffff 64bit pref]
[    1.000036] pci 0000:00:1c.3: PCI bridge to [bus 05-05]
[    1.000040] pci 0000:00:1c.3:   bridge window [io  0x3000-0x3fff]
[    1.000045] pci 0000:00:1c.3:   bridge window [mem 0xc0a00000-0xc0bfffff]
[    1.000050] pci 0000:00:1c.3:   bridge window [mem
0xc0c00000-0xc0dfffff 64bit pref]
[    1.000057] pci 0000:00:1c.4: PCI bridge to [bus 06-06]
[    1.000061] pci 0000:00:1c.4:   bridge window [io  0x4000-0x4fff]
[    1.000066] pci 0000:00:1c.4:   bridge window [mem 0xc0e00000-0xc0ffffff]
[    1.000071] pci 0000:00:1c.4:   bridge window [mem
0xc1000000-0xc11fffff 64bit pref]
[    1.000078] pci 0000:00:1e.0: PCI bridge to [bus 07-07]
[    1.000082] pci 0000:00:1e.0:   bridge window [io  0xe000-0xefff]
[    1.000087] pci 0000:00:1e.0:   bridge window [mem 0xfbf00000-0xfbffffff]
[    1.000092] pci 0000:00:1e.0:   bridge window [mem pref disabled]
[    1.000103]   alloc irq_desc for 16 on node -1
[    1.000104]   alloc kstat_irqs on node -1
[    1.000107] pci 0000:00:03.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.000112] pci 0000:00:03.0: setting latency timer to 64
[    1.000118] pci 0000:00:1c.0: enabling device (0104 -> 0107)
[    1.000122]   alloc irq_desc for 17 on node -1
[    1.000123]   alloc kstat_irqs on node -1
[    1.000126] pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.000130] pci 0000:00:1c.0: setting latency timer to 64
[    1.000136] pci 0000:00:1c.1: PCI INT B -> GSI 16 (level, low) -> IRQ 16
[    1.000141] pci 0000:00:1c.1: setting latency timer to 64
[    1.000147] pci 0000:00:1c.2: enabling device (0104 -> 0107)
[    1.000151]   alloc irq_desc for 18 on node -1
[    1.000152]   alloc kstat_irqs on node -1
[    1.000154] pci 0000:00:1c.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    1.000159] pci 0000:00:1c.2: setting latency timer to 64
[    1.000165] pci 0000:00:1c.3: enabling device (0104 -> 0107)
[    1.000168]   alloc irq_desc for 19 on node -1
[    1.000169]   alloc kstat_irqs on node -1
[    1.000172] pci 0000:00:1c.3: PCI INT D -> GSI 19 (level, low) -> IRQ 19
[    1.000177] pci 0000:00:1c.3: setting latency timer to 64
[    1.000182] pci 0000:00:1c.4: enabling device (0104 -> 0107)
[    1.000187] pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.000191] pci 0000:00:1c.4: setting latency timer to 64
[    1.000196] pci 0000:00:1e.0: setting latency timer to 64
[    1.000199] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
[    1.000200] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
[    1.000202] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
[    1.000203] pci_bus 0000:00: resource 7 [mem 0x000d0000-0x000dffff]
[    1.000205] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xdfffffff]
[    1.000206] pci_bus 0000:00: resource 9 [mem 0xf0000000-0xfed8ffff]
[    1.000208] pci_bus 0000:01: resource 0 [io  0xc000-0xcfff]
[    1.000209] pci_bus 0000:01: resource 1 [mem 0xfbd00000-0xfbdfffff]
[    1.000211] pci_bus 0000:01: resource 2 [mem 0xd0000000-0xdfffffff
64bit pref]
[    1.000212] pci_bus 0000:02: resource 0 [io  0x1000-0x1fff]
[    1.000214] pci_bus 0000:02: resource 1 [mem 0xc0000000-0xc01fffff]
[    1.000215] pci_bus 0000:02: resource 2 [mem 0xc0200000-0xc03fffff
64bit pref]
[    1.000217] pci_bus 0000:03: resource 0 [io  0xd000-0xdfff]
[    1.000218] pci_bus 0000:03: resource 1 [mem 0xfbe00000-0xfbefffff]
[    1.000220] pci_bus 0000:03: resource 2 [mem 0xc0400000-0xc05fffff
64bit pref]
[    1.000222] pci_bus 0000:04: resource 0 [io  0x2000-0x2fff]
[    1.000223] pci_bus 0000:04: resource 1 [mem 0xc0600000-0xc07fffff]
[    1.000225] pci_bus 0000:04: resource 2 [mem 0xc0800000-0xc09fffff
64bit pref]
[    1.000226] pci_bus 0000:05: resource 0 [io  0x3000-0x3fff]
[    1.000228] pci_bus 0000:05: resource 1 [mem 0xc0a00000-0xc0bfffff]
[    1.000229] pci_bus 0000:05: resource 2 [mem 0xc0c00000-0xc0dfffff
64bit pref]
[    1.000231] pci_bus 0000:06: resource 0 [io  0x4000-0x4fff]
[    1.000232] pci_bus 0000:06: resource 1 [mem 0xc0e00000-0xc0ffffff]
[    1.000234] pci_bus 0000:06: resource 2 [mem 0xc1000000-0xc11fffff
64bit pref]
[    1.000235] pci_bus 0000:07: resource 0 [io  0xe000-0xefff]
[    1.000237] pci_bus 0000:07: resource 1 [mem 0xfbf00000-0xfbffffff]
[    1.000238] pci_bus 0000:07: resource 4 [io  0x0000-0x0cf7]
[    1.000240] pci_bus 0000:07: resource 5 [io  0x0d00-0xffff]
[    1.000241] pci_bus 0000:07: resource 6 [mem 0x000a0000-0x000bffff]
[    1.000243] pci_bus 0000:07: resource 7 [mem 0x000d0000-0x000dffff]
[    1.000244] pci_bus 0000:07: resource 8 [mem 0xc0000000-0xdfffffff]
[    1.000246] pci_bus 0000:07: resource 9 [mem 0xf0000000-0xfed8ffff]
[    1.000285] NET: Registered protocol family 2
[    1.000335] IP route cache hash table entries: 262144 (order: 9,
2097152 bytes)
[    1.000669] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    1.001759] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes)
[    1.002200] TCP: Hash tables configured (established 262144 bind 65536)
[    1.002204] TCP reno registered
[    1.002211] UDP hash table entries: 4096 (order: 6, 393216 bytes)
[    1.002295] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes)
[    1.002453] NET: Registered protocol family 1
[    1.002529] RPC: Registered udp transport module.
[    1.002532] RPC: Registered tcp transport module.
[    1.002534] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.002606] pci 0000:01:00.0: Boot video device
[    1.002617] PCI: CLS 32 bytes, default 64
[    1.012488] Trying to unpack rootfs image as initramfs...
[    1.063737] Freeing initrd memory: 3176k freed
[    1.064087] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    1.064092] Placing 64MB software IO TLB between ffff880008200000 -
ffff88000c200000
[    1.064096] software IO TLB at phys 0x8200000 - 0xc200000
[    1.065672] microcode: CPU0 sig=0x106e5, pf=0x2, revision=0x3
[    1.065680] microcode: CPU1 sig=0x106e5, pf=0x2, revision=0x3
[    1.065688] microcode: CPU2 sig=0x106e5, pf=0x2, revision=0x3
[    1.065695] microcode: CPU3 sig=0x106e5, pf=0x2, revision=0x3
[    1.065703] microcode: CPU4 sig=0x106e5, pf=0x2, revision=0x3
[    1.065711] microcode: CPU5 sig=0x106e5, pf=0x2, revision=0x3
[    1.065717] microcode: CPU6 sig=0x106e5, pf=0x2, revision=0x3
[    1.065724] microcode: CPU7 sig=0x106e5, pf=0x2, revision=0x3
[    1.065754] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    1.065766] Scanning for low memory corruption every 60 seconds
[    1.065851] Intel AES-NI instructions are not detected.
[    1.065854] Intel PCLMULQDQ-NI instructions are not detected.
[    1.066196] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    1.066364] VFS: Disk quotas dquot_6.5.2
[    1.066393] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.066565] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    1.066675] fuse init (API version 7.15)
[    1.066791] JFS: nTxBlock = 8192, nTxLock = 65536
[    1.067912] SGI XFS with ACLs, security attributes, realtime, large
block/inode numbers, no debug enabled
[    1.068092] SGI XFS Quota Management subsystem
[    1.068192] Btrfs loaded
[    1.068196] msgmni has been set to 11918
[    1.068814] alg: No test for fcrypt (fcrypt-generic)
[    1.070044] alg: No test for stdrng (krng)
[    1.070067] async_tx: api initialized (async)
[    1.070103] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[    1.070108] io scheduler noop registered
[    1.070110] io scheduler deadline registered (default)
[    1.070126] io scheduler cfq registered
[    1.071557] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    1.072222] vesafb: framebuffer at 0xd0000000, mapped to
0xffffc90010780000, using 5120k, total 16384k
[    1.072227] vesafb: mode is 1280x1024x16, linelength=2560, pages=5
[    1.072230] vesafb: scrolling: redraw
[    1.072233] vesafb: Truecolor: size=0:5:6:5, shift=0:11:5:0
[    1.075837] Console: switching to colour frame buffer device 160x64
[    1.077207] fb0: VESA VGA frame buffer device
[    1.077253] intel_idle: MWAIT substates: 0x1120
[    1.077254] intel_idle: v0.4 model 0x1E
[    1.077255] intel_idle: lapic_timer_reliable_states 0x2
[    1.077512] input: Power Button as
/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
[    1.077533] ACPI: Power Button [PWRB]
[    1.077587] input: Power Button as
/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
[    1.077605] ACPI: Power Button [PWRF]
[    1.077791] ACPI: acpi_idle yielding to intel_idle
[    1.080443] ERST: Table is not found!
[    1.080686] Linux agpgart interface v0.103
[    1.080799] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180
seconds, margin is 60 seconds).
[    1.080819] Hangcheck: Using getrawmonotonic().
[    1.080833] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    1.082260] brd: module loaded
[    1.082402] Loading iSCSI transport class v2.0-870.
[    1.082590] SCSI Media Changer driver v0.25
[    1.082647] ahci 0000:00:1f.2: version 3.0
[    1.082657] ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
[    1.082697]   alloc irq_desc for 45 on node -1
[    1.082699]   alloc kstat_irqs on node -1
[    1.082706] ahci 0000:00:1f.2: irq 45 for MSI/MSI-X
[    1.082728] ahci: SSS flag set, parallel bus scan disabled
[    1.093750] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 3
Gbps 0x3f impl SATA mode
[    1.093770] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led
clo pio slum part ems sxs apst
[    1.093791] ahci 0000:00:1f.2: setting latency timer to 64
[    1.103753] scsi0 : ahci
[    1.103849] scsi1 : ahci
[    1.103919] scsi2 : ahci
[    1.103990] scsi3 : ahci
[    1.104061] scsi4 : ahci
[    1.104131] scsi5 : ahci
[    1.104255] ata1: SATA max UDMA/133 abar m2048@0xfbcf2000 port
0xfbcf2100 irq 45
[    1.104274] ata2: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.104292] ata3: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.104310] ata4: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.104329] ata5: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.104347] ata6: SATA max UDMA/133 irq_stat 0x00400040, connection
status changed irq 45
[    1.104387] ahci 0000:03:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.114713] ahci 0000:03:00.0: AHCI 0001.0000 32 slots 2 ports 3
Gbps 0x3 impl SATA mode
[    1.114738] ahci 0000:03:00.0: flags: 64bit ncq led clo pmp pio
[    1.114756] ahci 0000:03:00.0: setting latency timer to 64
[    1.114838] scsi6 : ahci
[    1.114923] scsi7 : ahci
[    1.116223] ata7: SATA max UDMA/133 abar m8192@0xfbefe000 port
0xfbefe100 irq 17
[    1.117457] ata8: SATA max UDMA/133 abar m8192@0xfbefe000 port
0xfbefe180 irq 17
[    1.119341] pata_jmicron 0000:03:00.1: enabling device (0000 -> 0001)
[    1.120566] pata_jmicron 0000:03:00.1: PCI INT B -> GSI 18 (level,
low) -> IRQ 18
[    1.121797] pata_jmicron 0000:03:00.1: setting latency timer to 64
[    1.121843] scsi8 : pata_jmicron
[    1.123148] scsi9 : pata_jmicron
[    1.124710] ata9: PATA max UDMA/100 cmd 0xdc00 ctl 0xd880 bmdma 0xd400 irq 18
[    1.125969] ata10: PATA max UDMA/100 cmd 0xd800 ctl 0xd480 bmdma
0xd408 irq 18
[    1.127755] tun: Universal TUN/TAP device driver, 1.6
[    1.129021] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    1.130387] uhci_hcd: USB Universal Host Controller Interface driver
[    1.131722] usbcore: registered new interface driver wusb-cbaf
[    1.133019] usbcore: registered new interface driver libusual
[    1.134363] PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at
0x60,0x64 irq 1,12
[    1.135988] serio: i8042 KBD port at 0x60,0x64 irq 1
[    1.137223] serio: i8042 AUX port at 0x60,0x64 irq 12
[    1.138549] mice: PS/2 mouse device common for all mice
[    1.139959] rtc_cmos 00:03: RTC can wake from S4
[    1.141247] rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
[    1.142518] rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
[    1.143823] lirc_dev: IR Remote Control driver registered, major 251
[    1.145056] Linux video capture interface: v2.00
[    1.146353] md: linear personality registered for level -1
[    1.147614] md: raid0 personality registered for level 0
[    1.148872] md: raid1 personality registered for level 1
[    1.150114] md: raid10 personality registered for level 10
[    1.151351] md: raid6 personality registered for level 6
[    1.152575] md: raid5 personality registered for level 5
[    1.153800] md: raid4 personality registered for level 4
[    1.155010] md: multipath personality registered for level -4
[    1.156256] device-mapper: uevent: version 1.0.3
[    1.157521] device-mapper: ioctl: 4.18.0-ioctl (2010-06-29)
initialised: dm-devel@redhat.com
[    1.158744] device-mapper: multipath: version 1.1.1 loaded
[    1.159912] device-mapper: multipath round-robin: version 1.0.0 loaded
[    1.161107] EDAC MC: Ver: 2.1.0 Dec  2 2010
[    1.162921] cpuidle: using governor ladder
[    1.165149] cpuidle: using governor menu
[    1.166406] ioatdma: Intel(R) QuickData Technology Driver 4.00
[    1.168373] usbcore: registered new interface driver hiddev
[    1.169624] usbcore: registered new interface driver usbhid
[    1.170856] usbhid: USB HID core driver
[    1.174417]   alloc irq_desc for 22 on node -1
[    1.174419]   alloc kstat_irqs on node -1
[    1.174425] HDA Intel 0000:00:1b.0: PCI INT A -> GSI 22 (level,
low) -> IRQ 22
[    1.175639]   alloc irq_desc for 46 on node -1
[    1.175641]   alloc kstat_irqs on node -1
[    1.175649] HDA Intel 0000:00:1b.0: irq 46 for MSI/MSI-X
[    1.175668] HDA Intel 0000:00:1b.0: setting latency timer to 64
[    1.188725] hda_codec: ALC888: BIOS auto-probing.
[    1.195353] HDA Intel 0000:01:00.1: PCI INT B -> GSI 17 (level,
low) -> IRQ 17
[    1.196518]   alloc irq_desc for 47 on node -1
[    1.196519]   alloc kstat_irqs on node -1
[    1.196530] HDA Intel 0000:01:00.1: irq 47 for MSI/MSI-X
[    1.196545] HDA Intel 0000:01:00.1: setting latency timer to 64
[    1.201174] ALSA device list:
[    1.202347]   #0: HDA Intel at 0xfbcf4000 irq 46
[    1.203534]   #1: HD-Audio Generic at 0xfbdbc000 irq 47
[    1.204934] TCP cubic registered
[    1.206125] Initializing XFRM netlink socket
[    1.207364] NET: Registered protocol family 10
[    1.208809] lo: Disabled Privacy Extensions
[    1.210176] IPv6 over IPv4 tunneling driver
[    1.211645] sit0: Disabled Privacy Extensions
[    1.212977] NET: Registered protocol family 17
[    1.214171] NET: Registered protocol family 15
[    1.215353] lib80211: common routines for IEEE802.11 drivers
[    1.216521] lib80211_crypt: registered algorithm 'NULL'
[    1.216523] Registering the dns_resolver key type
[    1.220032] registered taskstats version 1
[    1.221577] rtc_cmos 00:03: setting system clock to 2010-12-05
01:00:38 UTC (1291510838)
[    1.407270] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    1.410068] ata1.00: ATA-8: WDC WD1001FALS-00J7B0, 05.00K05, max UDMA/133
[    1.411737] ata1.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    1.414711] ata1.00: configured for UDMA/133
[    1.421313] ata7: SATA link down (SStatus 0 SControl 300)
[    1.423054] ata8: SATA link down (SStatus 0 SControl 300)
[    1.427351] scsi 0:0:0:0: Direct-Access     ATA      WDC
WD1001FALS-0 05.0 PQ: 0 ANSI: 5
[    1.429035] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks:
(1.00 TB/931 GiB)
[    1.429639] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    1.432412] sd 0:0:0:0: [sda] Write Protect is off
[    1.434037] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    1.434066] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    1.448255]  sda: sda1 sda2 < sda5 >
[    1.450882] sd 0:0:0:0: [sda] Attached SCSI disk
[    2.150918] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    2.154315] ata2.00: ATAPI: ATAPI   iHAS324   Y, BL1Y, max UDMA/100
[    2.158058] ata2.00: configured for UDMA/100
[    2.172159] scsi 1:0:0:0: CD-ROM            ATAPI    iHAS324   Y
  BL1Y PQ: 0 ANSI: 5
[    2.178984] sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw
xa/form2 cdda tray
[    2.180793] cdrom: Uniform CD-ROM driver Revision: 3.20
[    2.183013] sr 1:0:0:0: Attached scsi CD-ROM sr0
[    2.183263] sr 1:0:0:0: Attached scsi generic sg1 type 5
[    2.906521] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    2.910285] ata3.00: ATA-8: WDC WD10EADS-22M2B0, 01.00A01, max UDMA/133
[    2.912039] ata3.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    2.916802] ata3.00: configured for UDMA/133
[    2.929597] scsi 2:0:0:0: Direct-Access     ATA      WDC
WD10EADS-22M 01.0 PQ: 0 ANSI: 5
[    2.932097] sd 2:0:0:0: [sdb] 1953525168 512-byte logical blocks:
(1.00 TB/931 GiB)
[    2.932315] sd 2:0:0:0: Attached scsi generic sg2 type 0
[    2.935905] sd 2:0:0:0: [sdb] Write Protect is off
[    2.937750] sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    2.937778] sd 2:0:0:0: [sdb] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    2.959664]  sdb: sdb1 sdb2 < sdb5 >
[    2.962264] sd 2:0:0:0: [sdb] Attached SCSI disk
[    3.654159] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    3.659099] ata4.00: ATAPI: HL-DT-ST DVDRAM GH41N, MN01, max UDMA/100
[    3.664512] ata4.00: configured for UDMA/100
[    3.684402] scsi 3:0:0:0: CD-ROM            HL-DT-ST DVDRAM GH41N
  MN01 PQ: 0 ANSI: 5
[    3.708124] sr1: scsi3-mmc drive: 40x/40x writer dvd-ram cd/rw
xa/form2 cdda tray
[    3.710457] sr 3:0:0:0: Attached scsi CD-ROM sr1
[    3.710714] sr 3:0:0:0: Attached scsi generic sg3 type 5
[    4.433762] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    4.442378] ata5.00: ATA-7: SAMSUNG HD154UI, 1AG01118, max UDMA7
[    4.444363] ata5.00: 2930277168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    4.453155] ata5.00: configured for UDMA/133
[    4.465797] scsi 4:0:0:0: Direct-Access     ATA      SAMSUNG
HD154UI  1AG0 PQ: 0 ANSI: 5
[    4.468444] sd 4:0:0:0: [sdc] 2930277168 512-byte logical blocks:
(1.50 TB/1.36 TiB)
[    4.468743] sd 4:0:0:0: Attached scsi generic sg4 type 0
[    4.472475] sd 4:0:0:0: [sdc] Write Protect is off
[    4.474387] sd 4:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[    4.474414] sd 4:0:0:0: [sdc] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    4.578267]  sdc: sdc1 sdc2 sdc3 sdc4 < sdc5 sdc6 sdc7 sdc8 sdc9 >
[    4.581629] sd 4:0:0:0: [sdc] Attached SCSI disk
[    5.190426] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    5.198462] ata6.00: ATA-8: SAMSUNG HD203WI, 1AN10003, max UDMA/133
[    5.200472] ata6.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    5.208528] ata6.00: configured for UDMA/133
[    5.220365] scsi 5:0:0:0: Direct-Access     ATA      SAMSUNG
HD203WI  1AN1 PQ: 0 ANSI: 5
[    5.223019] sd 5:0:0:0: [sdd] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[    5.223153] sd 5:0:0:0: Attached scsi generic sg5 type 0
[    5.227355] sd 5:0:0:0: [sdd] Write Protect is off
[    5.229433] sd 5:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[    5.229461] sd 5:0:0:0: [sdd] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    5.339571]  sdd: sdd1 sdd2 sdd3 < sdd5 sdd6 sdd7 sdd8 sdd9 sdd10 sdd11 >
[    5.342714] sd 5:0:0:0: [sdd] Attached SCSI disk
[    5.385840] Freeing unused kernel memory: 548k freed
[    5.388112] Write protecting the kernel read-only data: 12288k
[    5.390774] Freeing unused kernel memory: 852k freed
[    5.393234] Freeing unused kernel memory: 1472k freed
[    6.177325] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    6.177329] Warning! ehci_hcd should always be loaded before
uhci_hcd and ohci_hcd, not after
[    6.177334] ehci_hcd: block sizes: qh 104 qtd 96 itd 192 sitd 96
[    6.177382] ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    6.177444] ehci_hcd 0000:00:1a.0: setting latency timer to 64
[    6.177449] ehci_hcd 0000:00:1a.0: EHCI Host Controller
[    6.177477] drivers/usb/core/inode.c: creating file 'devices'
[    6.177482] drivers/usb/core/inode.c: creating file '001'
[    6.177754] ehci_hcd 0000:00:1a.0: new USB bus registered, assigned
bus number 1
[    6.177769] ehci_hcd 0000:00:1a.0: reset hcs_params 0x200002 dbg=2
cc=0 pcc=0 ordered !ppc ports=2
[    6.177775] ehci_hcd 0000:00:1a.0: reset hcc_params 36881 caching
frame 1024 64 bit addr
[    6.177797] ehci_hcd 0000:00:1a.0: support lpm
[    6.177810] ehci_hcd 0000:00:1a.0: debug port 2
[    6.177816] ehci_hcd 0000:00:1a.0: reset command 0080002 (park)=0
ithresh=8 period=1024 Reset HALT
[    6.181704] ehci_hcd 0000:00:1a.0: cache line size of 32 is not supported
[    6.181708] ehci_hcd 0000:00:1a.0: supports USB remote wakeup
[    6.181734] ehci_hcd 0000:00:1a.0: irq 16, io mem 0xfbcfa000
[    6.181740] ehci_hcd 0000:00:1a.0: reset command 0080002 (park)=0
ithresh=8 period=1024 Reset HALT
[    6.185618] ehci_hcd 0000:00:1a.0: init command 0010001 (park)=0
ithresh=1 period=1024 RUN
[    6.191604] ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[    6.191644] usb usb1: default language 0x0409
[    6.191655] usb usb1: udev 1, busnum 1, minor = 0
[    6.191658] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    6.191662] usb usb1: New USB device strings: Mfr=3, Product=2,
SerialNumber=1
[    6.191665] usb usb1: Product: EHCI Host Controller
[    6.191668] usb usb1: Manufacturer: Linux
2.6.36-rc6_bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc+ ehci_hcd
[    6.191672] usb usb1: SerialNumber: 0000:00:1a.0
[    6.191916] usb usb1: usb_probe_device
[    6.191921] usb usb1: configuration #1 chosen from 1 choice
[    6.191933] usb usb1: adding 1-0:1.0 (config #1, interface 0)
[    6.192165] hub 1-0:1.0: usb_probe_interface
[    6.192169] hub 1-0:1.0: usb_probe_interface - got id
[    6.192173] hub 1-0:1.0: USB hub found
[    6.192180] hub 1-0:1.0: 2 ports detected
[    6.192183] hub 1-0:1.0: standalone hub
[    6.192185] hub 1-0:1.0: no power switching (usb 1.0)
[    6.192188] hub 1-0:1.0: individual port over-current protection
[    6.192191] hub 1-0:1.0: power on to power good time: 20ms
[    6.192197] hub 1-0:1.0: local power source is good
[    6.192200] hub 1-0:1.0: trying to enable port power on non-switchable hub
[    6.192409] drivers/usb/core/inode.c: creating file '001'
[    6.192486]   alloc irq_desc for 23 on node -1
[    6.192489]   alloc kstat_irqs on node -1
[    6.192497] ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[    6.192516] ehci_hcd 0000:00:1d.0: setting latency timer to 64
[    6.192521] ehci_hcd 0000:00:1d.0: EHCI Host Controller
[    6.192532] drivers/usb/core/inode.c: creating file '002'
[    6.192782] ehci_hcd 0000:00:1d.0: new USB bus registered, assigned
bus number 2
[    6.192796] ehci_hcd 0000:00:1d.0: reset hcs_params 0x200002 dbg=2
cc=0 pcc=0 ordered !ppc ports=2
[    6.192801] ehci_hcd 0000:00:1d.0: reset hcc_params 36881 caching
frame 1024 64 bit addr
[    6.192819] ehci_hcd 0000:00:1d.0: support lpm
[    6.192832] ehci_hcd 0000:00:1d.0: debug port 2
[    6.192838] ehci_hcd 0000:00:1d.0: reset command 0080002 (park)=0
ithresh=8 period=1024 Reset HALT
[    6.196713] ehci_hcd 0000:00:1d.0: cache line size of 32 is not supported
[    6.196716] ehci_hcd 0000:00:1d.0: supports USB remote wakeup
[    6.196737] ehci_hcd 0000:00:1d.0: irq 23, io mem 0xfbcf8000
[    6.196743] ehci_hcd 0000:00:1d.0: reset command 0080002 (park)=0
ithresh=8 period=1024 Reset HALT
[    6.200645] ehci_hcd 0000:00:1d.0: init command 0010001 (park)=0
ithresh=1 period=1024 RUN
[    6.206613] ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[    6.206647] usb usb2: default language 0x0409
[    6.206657] usb usb2: udev 1, busnum 2, minor = 128
[    6.206661] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
[    6.206665] usb usb2: New USB device strings: Mfr=3, Product=2,
SerialNumber=1
[    6.206668] usb usb2: Product: EHCI Host Controller
[    6.206672] usb usb2: Manufacturer: Linux
2.6.36-rc6_bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc+ ehci_hcd
[    6.206675] usb usb2: SerialNumber: 0000:00:1d.0
[    6.206970] usb usb2: usb_probe_device
[    6.206976] usb usb2: configuration #1 chosen from 1 choice
[    6.206989] usb usb2: adding 2-0:1.0 (config #1, interface 0)
[    6.207247] hub 2-0:1.0: usb_probe_interface
[    6.207252] hub 2-0:1.0: usb_probe_interface - got id
[    6.207257] hub 2-0:1.0: USB hub found
[    6.207265] hub 2-0:1.0: 2 ports detected
[    6.207267] hub 2-0:1.0: standalone hub
[    6.207270] hub 2-0:1.0: no power switching (usb 1.0)
[    6.207272] hub 2-0:1.0: individual port over-current protection
[    6.207276] hub 2-0:1.0: power on to power good time: 20ms
[    6.207282] hub 2-0:1.0: local power source is good
[    6.207285] hub 2-0:1.0: trying to enable port power on non-switchable hub
[    6.207543] drivers/usb/core/inode.c: creating file '001'
[    6.239335] Initializing USB Mass Storage driver...
[    6.239500] usbcore: registered new interface driver usb-storage
[    6.239504] USB Mass Storage support registered.
[    6.281096] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    6.281101] ohci_hcd: block sizes: ed 80 td 96
[    6.291277] ehci_hcd 0000:00:1a.0: GetStatus port:1 status 001803 0
 ACK POWER sig=j CSC CONNECT
[    6.291284] hub 1-0:1.0: port 1: status 0501 change 0001
[    6.306450] sl811: driver sl811-hcd, 19 May 2005
[    6.307260] ehci_hcd 0000:00:1d.0: GetStatus port:1 status 001803 0
 ACK POWER sig=j CSC CONNECT
[    6.307267] hub 2-0:1.0: port 1: status 0501 change 0001
[    6.391098] hub 1-0:1.0: state 7 ports 2 chg 0002 evt 0000
[    6.391111] hub 1-0:1.0: port 1, status 0501, change 0000, 480 Mb/s
[    6.409005] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k6-NAPI
[    6.409009] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    6.442327] ehci_hcd 0000:00:1a.0: port 1 high speed
[    6.442336] ehci_hcd 0000:00:1a.0: GetStatus port:1 status 001005 0
 ACK POWER sig=se0 PE CONNECT
[    6.493028] usb 1-1: new high speed USB device using ehci_hcd and address 2
[    6.544151] ehci_hcd 0000:00:1a.0: port 1 high speed
[    6.544160] ehci_hcd 0000:00:1a.0: GetStatus port:1 status 001005 0
 ACK POWER sig=se0 PE CONNECT
[    6.606965] ehci_hcd 0000:00:1a.0: set dev address 2 for port 1
[    6.606971] ehci_hcd 0000:00:1a.0: LPM: no device attached
[    6.607252] usb 1-1: udev 2, busnum 1, minor = 1
[    6.607257] usb 1-1: New USB device found, idVendor=8087, idProduct=0020
[    6.607261] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    6.607534] usb 1-1: usb_probe_device
[    6.607540] usb 1-1: configuration #1 chosen from 1 choice
[    6.607746] usb 1-1: adding 1-1:1.0 (config #1, interface 0)
[    6.607997] hub 1-1:1.0: usb_probe_interface
[    6.608002] hub 1-1:1.0: usb_probe_interface - got id
[    6.608006] hub 1-1:1.0: USB hub found
[    6.608060] hub 1-1:1.0: 6 ports detected
[    6.608063] hub 1-1:1.0: standalone hub
[    6.608066] hub 1-1:1.0: individual port power switching
[    6.608068] hub 1-1:1.0: individual port over-current protection
[    6.608071] hub 1-1:1.0: Single TT
[    6.608074] hub 1-1:1.0: TT requires at most 8 FS bit times (666 ns)
[    6.608077] hub 1-1:1.0: Port indicators are supported
[    6.608080] hub 1-1:1.0: power on to power good time: 100ms
[    6.608337] hub 1-1:1.0: local power source is good
[    6.608341] hub 1-1:1.0: enabling power on all ports
[    6.609369] drivers/usb/core/inode.c: creating file '002'
[    6.609406] hub 2-0:1.0: state 7 ports 2 chg 0002 evt 0000
[    6.609417] hub 2-0:1.0: port 1, status 0501, change 0000, 480 Mb/s
[    6.659951] ehci_hcd 0000:00:1d.0: port 1 high speed
[    6.659958] ehci_hcd 0000:00:1d.0: GetStatus port:1 status 001005 0
 ACK POWER sig=se0 PE CONNECT
[    6.708790] hub 1-1:1.0: port 1: status 0101 change 0001
[    6.709043] hub 1-1:1.0: port 2: status 0101 change 0001
[    6.709412] hub 1-1:1.0: port 4: status 0101 change 0001
[    6.710611] usb 2-1: new high speed USB device using ehci_hcd and address 2
[    6.761776] ehci_hcd 0000:00:1d.0: port 1 high speed
[    6.761785] ehci_hcd 0000:00:1d.0: GetStatus port:1 status 001005 0
 ACK POWER sig=se0 PE CONNECT
[    6.809517] usb 1-1: link qh256-0001/ffff8801bc2679c0 start 1 [1/0 us]
[    6.824464] ehci_hcd 0000:00:1d.0: set dev address 2 for port 1
[    6.824471] ehci_hcd 0000:00:1d.0: LPM: no device attached
[    6.824713] usb 2-1: udev 2, busnum 2, minor = 129
[    6.824717] usb 2-1: New USB device found, idVendor=8087, idProduct=0020
[    6.824721] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    6.825088] usb 2-1: usb_probe_device
[    6.825093] usb 2-1: configuration #1 chosen from 1 choice
[    6.825251] usb 2-1: adding 2-1:1.0 (config #1, interface 0)
[    6.825566] hub 2-1:1.0: usb_probe_interface
[    6.825570] hub 2-1:1.0: usb_probe_interface - got id
[    6.825574] hub 2-1:1.0: USB hub found
[    6.825712] hub 2-1:1.0: 8 ports detected
[    6.825715] hub 2-1:1.0: standalone hub
[    6.825718] hub 2-1:1.0: individual port power switching
[    6.825720] hub 2-1:1.0: individual port over-current protection
[    6.825723] hub 2-1:1.0: Single TT
[    6.825726] hub 2-1:1.0: TT requires at most 8 FS bit times (666 ns)
[    6.825729] hub 2-1:1.0: Port indicators are supported
[    6.825732] hub 2-1:1.0: power on to power good time: 100ms
[    6.825996] hub 2-1:1.0: local power source is good
[    6.826001] hub 2-1:1.0: enabling power on all ports
[    6.827213] drivers/usb/core/inode.c: creating file '002'
[    6.827254] hub 1-1:1.0: state 7 ports 6 chg 0016 evt 0000
[    6.827319] hub 1-1:1.0: port 1, status 0101, change 0000, 12 Mb/s
[    6.889515] usb 1-1.1: new low speed USB device using ehci_hcd and address 3
[    6.901453] hub 1-1:1.0: port 1 not reset yet, waiting 10ms
[    6.927415] hub 2-1:1.0: port 7: status 0101 change 0001
[    6.979124] usb 1-1.1: skipped 1 descriptor after interface
[    6.979130] usb 1-1.1: skipped 1 descriptor after interface
[    6.979728] usb 1-1.1: default language 0x0409
[    6.981744] usb 1-1.1: udev 3, busnum 1, minor = 2
[    6.981749] usb 1-1.1: New USB device found, idVendor=046d, idProduct=c521
[    6.981753] usb 1-1.1: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[    6.981757] usb 1-1.1: Product: USB Receiver
[    6.981760] usb 1-1.1: Manufacturer: Logitech
[    6.982131] usb 1-1.1: usb_probe_device
[    6.982136] usb 1-1.1: configuration #1 chosen from 1 choice
[    6.983971] usb 1-1.1: adding 1-1.1:1.0 (config #1, interface 0)
[    6.984315] usbhid 1-1.1:1.0: usb_probe_interface
[    6.984319] usbhid 1-1.1:1.0: usb_probe_interface - got id
[    6.988522] input: Logitech USB Receiver as
/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.1/1-1.1:1.0/input/input2
[    6.989215] generic-usb 0003:046D:C521.0001: input,hidraw0: USB HID
v1.11 Mouse [Logitech USB Receiver] on usb-0000:00:1a.0-1.1/input0
[    6.989242] usb 1-1.1: adding 1-1.1:1.1 (config #1, interface 1)
[    6.989461] usbhid 1-1.1:1.1: usb_probe_interface
[    6.989465] usbhid 1-1.1:1.1: usb_probe_interface - got id
[    6.996760] input: Logitech USB Receiver as
/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.1/1-1.1:1.1/input/input3
[    6.996792] usb 1-1.1: link qh8-0e01/ffff8801bc37ecc0 start 2 [1/2 us]
[    6.997274] usbhid 1-1.1:1.1: looking for a minor, starting at 0
[    6.997683] generic-usb 0003:046D:C521.0002: input,hiddev0,hidraw1:
USB HID v1.11 Device [Logitech USB Receiver] on
usb-0000:00:1a.0-1.1/input1
[    6.997895] drivers/usb/core/inode.c: creating file '003'
[    6.998069] hub 1-1:1.0: port 2, status 0101, change 0000, 12 Mb/s
[    7.009277] hub 1-1:1.0: port 2 not reset yet, waiting 10ms
[    7.027069] usb 2-1: link qh256-0001/ffff8801bffded40 start 1 [1/0 us]
[    7.071213] usb 1-1.2: new high speed USB device using ehci_hcd and address 4
[    7.082151] hub 1-1:1.0: port 2 not reset yet, waiting 10ms
[    7.156809] usb 1-1.2: default language 0x0409
[    7.157554] usb 1-1.2: udev 4, busnum 1, minor = 3
[    7.157560] usb 1-1.2: New USB device found, idVendor=04f9, idProduct=002a
[    7.157564] usb 1-1.2: New USB device strings: Mfr=1, Product=2,
SerialNumber=3
[    7.157568] usb 1-1.2: Product: HL-5240
[    7.157570] usb 1-1.2: Manufacturer: Brother
[    7.157573] usb 1-1.2: SerialNumber: H7J241026
[    7.157920] usb 1-1.2: usb_probe_device
[    7.157926] usb 1-1.2: configuration #1 chosen from 1 choice
[    7.157994] usb 1-1.2: adding 1-1.2:1.0 (config #1, interface 0)
[    7.158527] drivers/usb/core/inode.c: creating file '004'
[    7.158732] hub 1-1:1.0: port 4, status 0101, change 0000, 12 Mb/s
[    7.220958] usb 1-1.4: new low speed USB device using ehci_hcd and address 5
[    7.233888] hub 1-1:1.0: port 4 not reset yet, waiting 10ms
[    7.319584] usb 1-1.4: skipped 1 descriptor after interface
[    7.319590] usb 1-1.4: skipped 1 descriptor after interface
[    7.320518] usb 1-1.4: default language 0x0409
[    7.330379] usb 1-1.4: udev 5, busnum 1, minor = 4
[    7.330384] usb 1-1.4: New USB device found, idVendor=045e, idProduct=00db
[    7.330389] usb 1-1.4: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[    7.330393] usb 1-1.4: Product: Natural® Ergonomic Keyboard 4000
[    7.330396] usb 1-1.4: Manufacturer: Microsoft
[    7.330772] usb 1-1.4: usb_probe_device
[    7.330778] usb 1-1.4: configuration #1 chosen from 1 choice
[    7.331419] usb 1-1.4: adding 1-1.4:1.0 (config #1, interface 0)
[    7.331721] usbhid 1-1.4:1.0: usb_probe_interface
[    7.331726] usbhid 1-1.4:1.0: usb_probe_interface - got id
[    7.339696] input: Microsoft Natural® Ergonomic Keyboard 4000 as
/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.4/1-1.4:1.0/input/input4
[    7.339729] usb 1-1.4: link qh8-0e01/ffff8801bc37e5c0 start 3 [1/2 us]
[    7.340205] microsoft 0003:045E:00DB.0003: input,hidraw2: USB HID
v1.11 Keyboard [Microsoft Natural® Ergonomic Keyboard 4000] on
usb-0000:00:1a.0-1.4/input0
[    7.340235] usb 1-1.4: adding 1-1.4:1.1 (config #1, interface 1)
[    7.340458] usbhid 1-1.4:1.1: usb_probe_interface
[    7.340462] usbhid 1-1.4:1.1: usb_probe_interface - got id
[    7.351610] input: Microsoft Natural® Ergonomic Keyboard 4000 as
/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.4/1-1.4:1.1/input/input5
[    7.351641] usb 1-1.4: link qh8-0e01/ffff8801bc37e140 start 4 [1/2 us]
[    7.352117] microsoft 0003:045E:00DB.0004: input,hidraw3: USB HID
v1.11 Device [Microsoft Natural® Ergonomic Keyboard 4000] on
usb-0000:00:1a.0-1.4/input1
[    7.352382] drivers/usb/core/inode.c: creating file '005'
[    7.352412] hub 1-1:1.0: state 7 ports 6 chg 0000 evt 0010
[    7.352558] hub 2-1:1.0: state 7 ports 8 chg 0080 evt 0000
[    7.352736] hub 2-1:1.0: port 7, status 0101, change 0000, 12 Mb/s
[    7.363527] hub 2-1:1.0: port 7 not reset yet, waiting 10ms
[    7.425472] usb 2-1.7: new high speed USB device using ehci_hcd and address 3
[    7.438415] hub 2-1:1.0: port 7 not reset yet, waiting 10ms
[    7.523852] usb 2-1.7: skipped 1 descriptor after interface
[    7.524474] usb 2-1.7: default language 0x0409
[    7.779614] usb 2-1.7: udev 3, busnum 2, minor = 130
[    7.779619] usb 2-1.7: New USB device found, idVendor=0bda, idProduct=0182
[    7.779624] usb 2-1.7: New USB device strings: Mfr=1, Product=2,
SerialNumber=3
[    7.779628] usb 2-1.7: Product: USB2.0-CRW
[    7.779631] usb 2-1.7: Manufacturer: Generic
[    7.779633] usb 2-1.7: SerialNumber: 20060413092100000
[    7.780019] usb 2-1.7: usb_probe_device
[    7.780025] usb 2-1.7: configuration #1 chosen from 1 choice
[    7.782659] usb 2-1.7: adding 2-1.7:1.0 (config #1, interface 0)
[    7.786533] libusual 2-1.7:1.0: usb_probe_interface
[    7.786569] libusual 2-1.7:1.0: usb_probe_interface - got id
[    7.786589] usb-storage 2-1.7:1.0: usb_probe_interface
[    7.786596] usb-storage 2-1.7:1.0: usb_probe_interface - got id
[    7.786792] scsi10 : usb-storage 2-1.7:1.0
[    7.787471] usb 2-1.7: adding 2-1.7:1.1 (config #1, interface 1)
[    7.787733] usbhid 2-1.7:1.1: usb_probe_interface
[    7.787737] usbhid 2-1.7:1.1: usb_probe_interface - got id
[    7.794723] generic-usb: probe of 0003:0BDA:0182.0005 failed with error -22
[    7.794976] drivers/usb/core/inode.c: creating file '003'
[    7.795170] hub 2-1:1.0: state 7 ports 8 chg 0000 evt fe80
[    8.789063] scsi 10:0:0:0: Direct-Access     Generic- Compact Flash
   1.00 PQ: 0 ANSI: 0 CCS
[    8.792163] scsi 10:0:0:1: Direct-Access     Generic- xD-Picture
   1.00 PQ: 0 ANSI: 0 CCS
[    8.795530] scsi 10:0:0:2: Direct-Access     Generic- SD/MMC
   1.00 PQ: 0 ANSI: 0 CCS
[    8.798521] scsi 10:0:0:3: Direct-Access     Generic- MS/MS-Pro/HG
   1.00 PQ: 0 ANSI: 0 CCS
[    8.801642] scsi 10:0:0:4: Direct-Access     Generic- MicroSD
   1.00 PQ: 0 ANSI: 0 CCS
[    8.803878] sd 10:0:0:0: Attached scsi generic sg6 type 0
[    8.804720] sd 10:0:0:1: Attached scsi generic sg7 type 0
[    8.805553] sd 10:0:0:2: Attached scsi generic sg8 type 0
[    8.806297] sd 10:0:0:3: Attached scsi generic sg9 type 0
[    8.807057] sd 10:0:0:4: Attached scsi generic sg10 type 0
[    8.813581] sd 10:0:0:0: [sde] Attached SCSI removable disk
[    8.818867] sd 10:0:0:3: [sdh] Attached SCSI removable disk
[    8.821323] sd 10:0:0:2: [sdg] Attached SCSI removable disk
[    8.823809] sd 10:0:0:1: [sdf] Attached SCSI removable disk
[    8.829770] sd 10:0:0:4: [sdi] Attached SCSI removable disk
[   15.076182] alg: No test for xts(twofish) (xts(twofish-asm))
[   17.638932] EXT3-fs (dm-2): error: couldn't mount because of
unsupported optional features (240)
[   17.641941] EXT2-fs (dm-2): error: couldn't mount because of
unsupported optional features (240)
[   17.671951] EXT4-fs (dm-2): mounted filesystem with ordered data
mode. Opts: (null)
[   22.760051] udev[4924]: starting version 164
[   22.977989] usb 1-1.1: link qh8-0e01/ffff8801bc1edf40 start 5 [1/2 us]
[   22.996108] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[   22.997551] ACPI: WMI: Mapper loaded
[   23.005901] e1000e: Intel(R) PRO/1000 Network Driver - 1.2.7-k2
[   23.005904] e1000e: Copyright (c) 1999 - 2010 Intel Corporation.
[   23.005930]   alloc irq_desc for 20 on node -1
[   23.005932]   alloc kstat_irqs on node -1
[   23.005939] e1000e 0000:00:19.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
[   23.005947] e1000e 0000:00:19.0: setting latency timer to 64
[   23.006068]   alloc irq_desc for 48 on node -1
[   23.006070]   alloc kstat_irqs on node -1
[   23.006080] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[   23.089803] usb 1-1.1: unlink qh8-0e01/ffff8801bc1edf40 start 5 [1/2 us]
[   23.101871] firewire_ohci 0000:07:06.0: PCI INT A -> GSI 19 (level,
low) -> IRQ 19
[   23.155845] firewire_ohci: Added fw-ohci device 0000:07:06.0, OHCI
v1.10, 4 IR + 8 IT contexts, quirks 0x1
[   23.233537] e1000e 0000:00:19.0: eth0: (PCI Express:2.5GB/s:Width
x1) 90:fb:a6:46:2d:ec
[   23.233540] e1000e 0000:00:19.0: eth0: Intel(R) PRO/1000 Network Connection
[   23.233574] e1000e 0000:00:19.0: eth0: MAC: 9, PHY: 9, PBA No: ffffff-0ff
[   23.233625] i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level,
low) -> IRQ 18
[   23.562647] fglrx: module license 'Proprietary. (C) 2002 - ATI
Technologies, Starnberg, GERMANY' taints kernel.
[   23.562651] Disabling lock debugging due to kernel taint
[   23.580507] [fglrx] Maximum main memory to use for locked dma
buffers: 5765 MBytes.
[   23.580547] [fglrx]   vendor: 1002 device: 6899 count: 1
[   23.580832] [fglrx] ioport: bar 4, base 0xc000, size: 0x100
[   23.580844] pci 0000:01:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[   23.580848] pci 0000:01:00.0: setting latency timer to 64
[   23.581084] [fglrx] Kernel PAT support is enabled
[   23.581097] [fglrx] module loaded - fglrx 8.79.4 [Oct 26 2010] with 1 minors
[   23.655871] firewire_core: created device fw0: GUID 00016c20013727d2, S400
[   26.168563] EXT4-fs (dm-2): re-mounted. Opts: commit=600
[   26.316906] REISERFS (device sdd1): found reiserfs format "3.6"
with standard journal
[   26.316921] REISERFS (device sdd1): using ordered data mode
[   26.326028] REISERFS (device sdd1): journal params: device sdd1,
size 8192, journal first block 18, max trans len 1024, max batch 900,
max commit age 600, max trans age 600
[   26.326367] REISERFS (device sdd1): checking transaction log (sdd1)
[   26.340422] REISERFS (device sdd1): Using r5 hash to sort names
[   28.565078] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[   28.615910] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[   28.617174] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   30.096385] e1000e: eth0 NIC Link is Up 100 Mbps Full Duplex, Flow
Control: RX
[   30.096392] e1000e 0000:00:19.0: eth0: 10/100 speed: disabling TSO
[   30.096944] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   32.838669] Adding 9420796k swap on /dev/mapper/XPS-SWAP.
Priority:-1 extents:1 across:9420796k
[   40.834555] eth0: no IPv6 routers present
[   45.016971] ehci_hcd 0000:00:1a.0: reused qh ffff8801bc1edf40 schedule
[   45.016981] usb 1-1.1: link qh8-0e01/ffff8801bc1edf40 start 5 [1/2 us]
[   46.251903] coretemp coretemp.0: TjMax is 99 C.
[   46.252070] coretemp coretemp.1: TjMax is 99 C.
[   46.252133] coretemp coretemp.2: TjMax is 99 C.
[   46.252309] coretemp coretemp.3: TjMax is 99 C.
[   46.279441] it87: Found IT8720F chip at 0xa10, revision 8
[   46.279450] it87: VID is disabled (pins used for GPIO)
[   46.279461] it87: Beeping is supported
[  106.621323]   alloc irq_desc for 49 on node -1
[  106.621325]   alloc kstat_irqs on node -1
[  106.621332] fglrx_pci 0000:01:00.0: irq 49 for MSI/MSI-X
[  106.621830] [fglrx] Firegl kernel thread PID: 6637
[  106.622092] [fglrx] IRQ 49 Enabled
[  107.000068] [fglrx] Gart USWC size:1280 M.
[  107.000070] [fglrx] Gart cacheable size:508 M.
[  107.000073] [fglrx] Reserved FB block: Shared offset:0, size:1000000
[  107.000074] [fglrx] Reserved FB block: Unshared offset:f8fd000, size:403000
[  107.000075] [fglrx] Reserved FB block: Unshared offset:3fff4000, size:c000
[  265.523684] EXT4-fs (dm-3): mounted filesystem with ordered data
mode. Opts: commit=600
[  951.897329] Enabling EXPERIMENTAL delayed logging feature - use at
your own risk.
[  951.954027] XFS mounting filesystem dm-4
[  952.157109] Ending clean XFS mount for filesystem: dm-4
[ 2116.735158] conftest[20068]: segfault at 7fff294acf80 ip
00007f61690981ba sp 00007fff294acf40 error 6 in
ld-2.12.1.so[7f6169092000+20000]
[ 3012.610585] hda-intel: IRQ timing workaround is activated for card
#1. Suggest a bigger bdl_pos_adj.
[ 6000.313047] REISERFS (device sdd1): found reiserfs format "3.6"
with standard journal
[ 6000.313066] REISERFS (device sdd1): using ordered data mode
[ 6000.323773] REISERFS (device sdd1): journal params: device sdd1,
size 8192, journal first block 18, max trans len 1024, max batch 900,
max commit age 600, max trans age 600
[ 6000.324033] REISERFS (device sdd1): checking transaction log (sdd1)
[ 6000.338187] REISERFS (device sdd1): Using r5 hash to sort names
[ 6037.155535] REISERFS (device sdd1): found reiserfs format "3.6"
with standard journal
[ 6037.155546] REISERFS (device sdd1): using ordered data mode
[ 6037.155677] REISERFS (device sdd1): journal params: device sdd1,
size 8192, journal first block 18, max trans len 1024, max batch 900,
max commit age 600, max trans age 600
[ 6037.155910] REISERFS (device sdd1): checking transaction log (sdd1)
[ 6037.205634] REISERFS (device sdd1): Using r5 hash to sort names
[ 6097.179452] ------------[ cut here ]------------
[ 6097.179456] kernel BUG at fs/ext4/inode.c:2717!
[ 6097.179457] invalid opcode: 0000 [#1] PREEMPT SMP
[ 6097.179459] last sysfs file:
/sys/devices/system/cpu/cpu7/cache/index2/shared_cpu_map
[ 6097.179461] CPU 5
[ 6097.179462] Modules linked in: it87 hwmon_vid coretemp hwmon
fglrx(P) firewire_ohci firewire_core i2c_i801 e1000e wmi shpchp tg3
libphy e1000 scsi_wait_scan sl811_hcd ohci_hcd ssb usb_storage
ehci_hcd
[ 6097.179472]
[ 6097.179475] Pid: 4540, comm: jbd2/dm-2-8 Tainted: P
2.6.36-rc6_bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc+ #1 FMP55/ipower
G3710
[ 6097.179477] RIP: 0010:[<ffffffff8119cc50>]  [<ffffffff8119cc50>]
ext4_writepage+0x270/0x280
[ 6097.179482] RSP: 0018:ffff8801baec3b40  EFLAGS: 00010246
[ 6097.179483] RAX: 8000000000020069 RBX: ffffea0004222088 RCX: 0000000000000030
[ 6097.179485] RDX: 0000000000000a0f RSI: ffff8801baec3cc0 RDI: ffffea0004222088
[ 6097.179486] RBP: ffff8801472ca6a0 R08: ffff880183cc6de8 R09: 0000000000000000
[ 6097.179488] R10: 0000000000000008 R11: 0000000000000010 R12: 0000000000000a0f
[ 6097.179489] R13: 0000000000001000 R14: ffff8801baec3cc0 R15: ffff8801baec3c10
[ 6097.179491] FS:  0000000000000000(0000) GS:ffff880002140000(0000)
knlGS:0000000000000000
[ 6097.179492] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 6097.179494] CR2: 00007f19058591f0 CR3: 0000000001c04000 CR4: 00000000000006e0
[ 6097.179495] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6097.179497] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 6097.179498] Process jbd2/dm-2-8 (pid: 4540, threadinfo
ffff8801baec2000, task ffff8801bfef0040)
[ 6097.179500] Stack:
[ 6097.179500]  ffff8801baec3c10 ffffffff810fce75 ffff8801472ca808
ffff8801472ca808
[ 6097.179503] <0> ffff8801472ca808 0000000000000a0f ffffea0004222088
0000000000000003
[ 6097.179506] <0> ffff8801baec3c10 ffffffff810a261d 000000000000000e
ffffffff810a2ab1
[ 6097.179509] Call Trace:
[ 6097.179513]  [<ffffffff810fce75>] ? __set_page_dirty_buffers+0x85/0xe0
[ 6097.179517]  [<ffffffff810a261d>] ? __writepage+0xd/0x40
[ 6097.179519]  [<ffffffff810a2ab1>] ? write_cache_pages+0x1b1/0x3d0
[ 6097.179521]  [<ffffffff810a2610>] ? __writepage+0x0/0x40
[ 6097.179525]  [<ffffffff811d1869>] ? journal_submit_data_buffers+0x99/0x100
[ 6097.179528]  [<ffffffff811d1db1>] ?
jbd2_journal_commit_transaction+0x331/0x1330
[ 6097.179532]  [<ffffffff8172097b>] ? schedule+0x37b/0x8d0
[ 6097.179534]  [<ffffffff817237f8>] ? _raw_spin_lock_irqsave+0x18/0x20
[ 6097.179538]  [<ffffffff810538c3>] ? lock_timer_base.clone.23+0x33/0x70
[ 6097.179540]  [<ffffffff8105396e>] ? try_to_del_timer_sync+0x6e/0x90
[ 6097.179542]  [<ffffffff811d5d91>] ? kjournald2+0xb1/0x210
[ 6097.179545]  [<ffffffff81062b80>] ? autoremove_wake_function+0x0/0x30
[ 6097.179547]  [<ffffffff811d5ce0>] ? kjournald2+0x0/0x210
[ 6097.179549]  [<ffffffff811d5ce0>] ? kjournald2+0x0/0x210
[ 6097.179551]  [<ffffffff81062706>] ? kthread+0x96/0xa0
[ 6097.179555]  [<ffffffff81003394>] ? kernel_thread_helper+0x4/0x10
[ 6097.179557]  [<ffffffff81062670>] ? kthread+0x0/0xa0
[ 6097.179559]  [<ffffffff81003390>] ? kernel_thread_helper+0x0/0x10
[ 6097.179560] Code: ff f6 c4 40 0f 84 4a fe ff ff e9 77 ff ff ff 48
c7 c6 f0 3f 82 81 48 c7 c7 40 b1 9d 81 31 c0 e8 e0 34 58 00 e9 6e fe
ff ff 0f 0b <0f> 0b 4d 8b 6d 10 e9 96 fe ff ff 0f 1f 44 00 00 48 83 ec
08 ba
[ 6097.179580] RIP  [<ffffffff8119cc50>] ext4_writepage+0x270/0x280
[ 6097.179583]  RSP <ffff8801baec3b40>
[ 6097.179584] ---[ end trace 5199c452c07fe3ec ]---

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Matt]

On Sat, Dec 4, 2010 at 8:38 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Sat, Dec 04 2010 at  2:18pm -0500,
> Matt <jackdachef@gmail.com> wrote:
>
>> On Wed, Dec 1, 2010 at 10:23 PM, Mike Snitzer <snitzer@redhat.com> wrote:
>> > Matt and Jon,
>> >
>> > If you'd be up to it: could you try testing your dm-crypt+ext4
>> > corruption reproducers against the following two 2.6.37-rc commits:
>> >
>> > 1) 1de3e3df917459422cb2aecac440febc8879d410
>> > then
>> > 2) bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
>> >
>> > Then, depending on results of no corruption for those commits, bonus
>> > points for testing the same commits but with Andi and Milan's latest
>> > dm-crypt cpu scalability patch applied too:
>> > https://patchwork.kernel.org/patch/365542/
>> >
>> > Thanks!
>> > Mike
>> >
>>
>> Hi Mike,
>>
>> it seems like there isn't even much testing to do:
>>
>> I tested all 3 commits / checkouts by re-compiling gcc which was/is
>> the 2nd easy way to trigger this "corruption", compiling google's
>> chromium (v9) and looking at the output/existance of gcc, g++ and
>> eselect opengl list
>
> Can you be a bit more precise about what you're doing to reproduce?
> What sequence?  What (if any) builds are going in parallel?  Etc.
>
>> so far everything went fine
>>
>> After that I used the new patch (v6 or pre-v6), before that I had to
>>
>> replace WQ_MEM_RECLAIM with WQ_RESCUER
>>
>> and, re-compiled the kernels
>>
>> shortly after I had booted up the system with the first kernel
>> (http://git.eu.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5a87b7a5da250c9be6d757758425dfeaf8ed3179)
>> the output of 'eselect opengl list' did show no opengl backend
>> selected
>>
>> so it seems to manifest itself even earlier (ext4: call
>> mpage_da_submit_io() from mpage_da_map_blocks()) even if only subtly
>> and over time -
>> I'm still currently running that kernel and posting from it & having tests run
>
> OK.
>
>> I'm not sure if it's even a problem with ext4 - I haven't had the time
>> to test with XFS yet - maybe it's also happening with that so it more
>> likely would be dm-core, like Milan suspected
>> (http://marc.info/?l=linux-kernel&m=129123636223477&w=2) :(
>
> It'd be interesting to try to reproduce with that same kernel but using
> XFS.  I'll check with Milan on what he thinks would be the best next
> steps.  Ideally we'll be able to reproduce your results to aid in
> pinpointing the issue.  I think Milan will be trying to do so shortly
> (if he hasn't started already -- using gentoo emerge, etc).
>
>> even though most of the time it's compiling I don't need to do much -
>> I need the box for work so if my time allows next tests would be next
>> weekend and I'm back to my other partition
>>
>> I really do hope that this bugger can be nailed down ASAP - I like the
>> improvements made in 2.6.37 but without the dm-crypt multi-cpu patch
>> it's only half the "fun" ;)
>
> Sure, we'll need to get to the bottom of this before we can have
> confidence sending the dm-crypt cpu scalability patch upstream.
>
> Thanks for your testing,
> Mike
>

OK, before bed time I found some kind of corruption:

running kernel is from commit: bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc

the messages might be overseen - so they're difficult to notice:

steps:
1) bootup
2) (might need to re-install graphics driver due to driver switch, in
this case magic properties [or what's its name] didn't change so the
kernel module still worked)
3) firing up 2 xterms, xload, xclock, gksu -> terminal -> firefox,
nautilus --no-desktop, gnome-mplayer (playing mp3)
4) emerge -1 sys-devel/gcc (from one of the xterms)

after emerge -1 sys-devel/gcc
finished it displayed:

>>> Auto-cleaning packages...
portage: COUNTER for sys-devel/patch-2.6.1 was corrupted; resetting to
value of 0
portage: COUNTER for sys-devel/patch-2.6.1 was corrupted; resetting to
value of 0

(the COUNTER file normally should have a value, e.g.:
cat /var/db/pkg/sys-devel/gcc-4.5.1-r1/COUNTER
20560)

in this case it's empty:
cat /var/db/pkg/sys-devel/patch-2.6.1/COUNTER

(shows nothing)

reference thread: http://forums.gentoo.org/viewtopic-t-836605-start-0.html

it's solvable by re-install but in case of not-recoverable files (e.g.
personal files) it would be critical

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

[Heinz Diehl]

On 04.12.2010, Matt wrote: 

> I'm not sure if it's even a problem with ext4 - I haven't had the time
> to test with XFS yet

I can and have run both -rc3 and -rc4 with Milans patch v6, without any problems at
all, under heavy load and disk I/O. I'm using XFS exclusively. The system
is a testing server and contains 3 harddisks, 2 SATA disks and one SSD.
The whole system runs flawlessly, without any corruption...

Re: dm-crypt barrier support is effective

[Heinz Diehl]

On 01.12.2010, Milan Broz wrote: 

> Anyway, I run several tests on 2.6.37-rc3+ and see no integrity
> problems (using xfs,ext3 and ext4 over dmcrypt).

Not that this might help, but just for testing purposes, I have run all 
the -rcX from 2.6.36 on with Milan's patch (XFS filesystem) under heavy 
load and disk i/o, and have not encountered a single problem or corruption.

Re: dm-crypt barrier support is effective

[Heinz Diehl]

On 15.11.2010, Milan Broz wrote: 

> even with v5 I sent on Friday?

Your v5 patch applies cleanly to 2.6.36, but fails to build
on my system:

[....]
LD      fs/xfs/xfs.o
LD      fs/xfs/built-in.o
CC      fs/compat_ioctl.o
drivers/md/dm-crypt.c: In function crypt_ctr':
drivers/md/dm-crypt.c:1408: error: WQ_MEM_RECLAIM' undeclared (first use in this function)
drivers/md/dm-crypt.c:1408: error: (Each undeclared identifier is reported only once
drivers/md/dm-crypt.c:1408: error: for each function it appears in.)
make[2]: *** [drivers/md/dm-crypt.o] Error 1
make[1]: *** [drivers/md] Error 2
make: *** [drivers] Error 2
make: *** Waiting for unfinished jobs....
CC      fs/nfsctl.o
CC      net/netfilter/nf_sockopt.o
[....]

Re: dm-crypt barrier support is effective

[Milan Broz]

On 11/15/2010 08:25 AM, Heinz Diehl wrote:
> On 15.11.2010, Milan Broz wrote: 
> 
> drivers/md/dm-crypt.c: In function crypt_ctr':
> drivers/md/dm-crypt.c:1408: error: WQ_MEM_RECLAIM' undeclared (first use in this function)
> drivers/md/dm-crypt.c:1408: error: (Each undeclared identifier is reported only once
> drivers/md/dm-crypt.c:1408: error: for each function it appears in.)

It should be enough to just replace WQ_MEM_RECLAIM to WQ_RESCUER for 2.6.36.
(that define is new in 2.6.37)

Milan

Re: [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Heinz Diehl]

On 07.11.2010, Matt wrote: 

> noatime,nodiratime,barrier=1

One of my systems runs a squid webcache and has a lot of disk load,
and I've never seen data loss or corruption, all the way up from 2.6.34 to
the latest -git.

(Btw: "noatime" superseeds "nodiratime", you can therefore drop it when
"noatime" is set.)

> I read about barrier-problems and data getting to the partition when
> using dm-crypt and several layers so I don't know if that could be
> related

I would rather guess that you have filesystem / harddisk problems,
but I must admit I've never mounted one of my filesystems with barriers
enabled (I use XFS exclusively).

Re: [PATCH] DM-CRYPT: Scale to multiple CPUs v3 on 2.6.37-rc* ?

[Heinz Diehl]

On 07.11.2010, Matt wrote: 

> [dm-devel] [PATCH] DM-CRYPT: Scale to multiple CPUs v3

I have used it (and the older version of it) over a period of at least 
3 months now, up to the latest -rc-git without any problems.