From: Yinghai Lu <yinghai@kernel.org>
To: paulmck@linux.vnet.ibm.com
Cc: linux-kernel@vger.kernel.org, mingo@redhat.com, hpa@zytor.com,
tglx@linutronix.de, mingo@elte.hu
Subject: Re: [tip:core/rcu] Revert "rcu: Decrease memory-barrier usage based on semi-formal proof"
Date: Mon, 23 May 2011 15:01:24 -0700 [thread overview]
Message-ID: <4DDAD934.9010603@kernel.org> (raw)
In-Reply-To: <20110523212530.GF7428@linux.vnet.ibm.com>
On 05/23/2011 02:25 PM, Paul E. McKenney wrote:
> On Mon, May 23, 2011 at 01:14:22PM -0700, Yinghai Lu wrote:
>> On 05/21/2011 07:08 AM, Paul E. McKenney wrote:
>>> On Sat, May 21, 2011 at 06:18:44AM -0700, Paul E. McKenney wrote:
>>>> On Fri, May 20, 2011 at 05:02:40PM -0700, Yinghai Lu wrote:
>>>>> On 05/20/2011 04:49 PM, Paul E. McKenney wrote:
>>>>>> On Fri, May 20, 2011 at 04:16:28PM -0700, Yinghai Lu wrote:
>>>>> ...
>>>>>>>
>>>>>>> the same one i sent out before, but let DEBUG_LOCKING_API_SELFTESTS disabled.
>>>>>>
>>>>>> OK, just to make sure I understand... You are compiling exactly the
>>>>>> same kernel source tree with exactly the same .config, just with two
>>>>>> different versions of gcc, correct?
>>>>> yes.
>>>>>>
>>>>>> If so, it is quite possible that the slow one is the correct one. :-/
>>>>> yeah, new version always have problem.
>>>>>
>>>>> looks like opensuse11.3 has 4.5.0 and fedora14 has 4.5.1
>>>>
>>>> OK, so fedora14 is the fast one (4.5.1) and opensuse11.3 is the slow
>>>> one (4.5.0), correct?
>>>
>>> And does commit c7a3786030 help? This commit (from Peter Zijlstra)
>>> tidied up RCU kthreads' scheduler interactions. The patch is below,
>>> though it is probably more convenient to pull it from the rcu/next
>>> branch of:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git
>>>
>
> Thank you for testing this!
it's ok. don't want to see our servers have problem with newer kernel.
>
> This is with the same config that you emailed out on May 12th?
yes.
>
> In particular, CONFIG_TREE_RCU=y?
>
>> [ 337.132517] INFO: task rcun0:8 blocked for more than 120 seconds.
>> [ 337.133238] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> [ 337.160396] rcun0 D 0000000000000000 0 8 2 0x00000000
>> [ 337.161232] ffff882070d3fe90 0000000000000046 ffff882070d3e000 0000000000004000
>> [ 337.161291] 00000000001d1f80 ffff882070d3ffd8 00000000001d1f80 ffff882070d3ffd8
>> [ 337.161348] 0000000000004000 00000000001d1f80 ffff882070d18000 ffff882070d422b0
>> [ 337.161404] Call Trace:
>> [ 337.161433] [<ffffffff810afab6>] ? __lock_release+0x166/0x16f
>> [ 337.161459] [<ffffffff81c1dae1>] ? _raw_spin_unlock_irqrestore+0x3f/0x46
>> [ 337.161486] [<ffffffff810ce633>] ? rcu_cpu_kthread_should_stop+0x137/0x137
>> [ 337.161512] [<ffffffff810add8a>] ? trace_hardirqs_on+0xd/0xf
>> [ 337.161533] [<ffffffff810ce633>] ? rcu_cpu_kthread_should_stop+0x137/0x137
>> [ 337.161558] [<ffffffff81099e41>] kthread+0x8c/0xa8
>> [ 337.161584] [<ffffffff81c257d4>] kernel_thread_helper+0x4/0x10
>> [ 337.161606] [<ffffffff81c1dd80>] ? retint_restore_args+0xe/0xe
>> [ 337.161627] [<ffffffff81099db5>] ? __init_kthread_worker+0x5b/0x5b
>> [ 337.161645] [<ffffffff81c257d0>] ? gs_change+0xb/0xb
>> [ 337.161651] no locks held by rcun0/8.
>
> This is quite surprising. The "rcun" kthreads invoke rcu_node_kthread(),
> which does not call rcu_cpu_kthread_should_stop().
>
> But perhaps the stack backtrace got confused.
>
> Could you please try the following diagnostic patch to help me work out
> where the rcun threads are getting stuck?
>
[ 275.679636] RAMDISK: gzip image found at block 0
[ 277.504381] rcun ffffffff82440b00 starting wait for work.
[ 277.504693] rcun ffffffff82440b00 completed wait for work.
[ 277.504951] rcun ffffffff82440b00 initiating boost.
[ 277.515920] rcun ffffffff82440b00 completed boost.
[ 277.516157] rcun ffffffff82440b00 awaking rcuc2.
[ 277.535818] rcun ffffffff82440b00 awakened rcuc2.
[ 277.536075] rcun ffffffff82440b00 starting wait for work.
[ 277.604609] EXT3-fs: barriers not enabled
[ 277.605278] kjournald starting. Commit interval 5 seconds
[ 277.605473] EXT3-fs (ram0): warning: maximal mount count reached, running e2fsck is recommended
[ 277.605493] EXT3-fs (ram0): using internal journal
[ 277.605505] EXT3-fs (ram0): mounted filesystem with writeback data mode
[ 277.605555] VFS: Mounted root (ext3 filesystem) on device 1:0.
[ 277.605600] async_waiting @ 1
[ 277.605604] async_continuing @ 1 after 0 usec
[ 277.669722] Freeing unused kernel memory: 2892k freed
INIT: version 2.86 booting
System Boot Control: Running /etc/init.d/boot
Mounting procfs at /proc done
Mounting sysfs at /sys done
Mounting debugfs at /sys/kernel/debug done
Mounting tmpfs at /dev done
Initializing /dev done
Mounting devpts at /dev/pts done
Boot logging started on /dev/ttyS0(/dev/console) at Wed May 25 13:43:55 2011
FATAL: Could not load /lib/modules/2.6.39-tip-yh-05892-gb7b703b-dirty/modules.dep: No such file or directory
Setting up the hardware clockmodprobe: FATAL: Could not load /lib/modules/2.6.39-tip-yh-05892-gb7b703b-dirty/modules.dep: No such file or directory
hwclock: With --noadjfile, you must specify either --utc or --localtime
^[[8[ 277.836052] udevd (18674): /proc/18674/oom_adj is deprecated, please use /proc/18674/oom_score_adj instead.
fa[ 277.845689] udevd version 128 started
iled
Disabling IP forwarding done
done
Starting udevd: done
Loading drivers, configuring devices: [ 278.019705] rcun ffffffff82441300 starting wait for work.
[ 278.020057] rcun ffffffff82441300 completed wait for work.
[ 278.020314] rcun ffffffff82441300 initiating boost.
[ 278.035052] rcun ffffffff82441300 completed boost.
[ 278.035383] rcun ffffffff82441300 awaking rcuc141.
[ 278.054945] rcun ffffffff82441300 awakened rcuc141.
[ 278.055197] rcun ffffffff82441300 starting wait for work.
[ 278.159361] rcun ffffffff82441100 starting wait for work.
[ 278.159666] rcun ffffffff82441100 completed wait for work.
[ 278.159949] rcun ffffffff82441100 initiating boost.
[ 278.174805] rcun ffffffff82441100 completed boost.
[ 278.175044] rcun ffffffff82441100 awaking rcuc111.
[ 278.194728] rcun ffffffff82441100 awakened rcuc111.
[ 278.194987] rcun ffffffff82441100 starting wait for work.
[ 278.303039] rcun ffffffff82440c00 starting wait for work.
[ 278.303323] rcun ffffffff82440c00 completed wait for work.
[ 278.303568] rcun ffffffff82440c00 initiating boost.
[ 278.314519] rcun ffffffff82440c00 completed boost.
[ 278.314750] rcun ffffffff82440c00 awaking rcuc31.
[ 278.334455] rcun ffffffff82440c00 awakened rcuc31.
[ 278.334687] rcun ffffffff82440c00 starting wait for work.
[ 278.498822] rcun ffffffff82441400 starting wait for work.
[ 278.499131] rcun ffffffff82441400 completed wait for work.
[ 278.499412] rcun ffffffff82441400 initiating boost.
[ 278.514295] rcun ffffffff82441400 completed boost.
[ 278.514632] rcun ffffffff82441400 awaking rcuc151.
[ 278.534082] rcun ffffffff82441400 awakened rcuc151.
[ 278.534338] rcun ffffffff82441400 starting wait for work.
[ 278.686359] rcun ffffffff82440e00 starting wait for work.
[ 278.686670] rcun ffffffff82440e00 completed wait for work.
[ 278.686927] rcun ffffffff82440e00 initiating boost.
[ 278.703910] rcun ffffffff82440e00 completed boost.
[ 278.704148] rcun ffffffff82440e00 awaking rcuc51.
[ 278.723778] rcun ffffffff82440e00 awakened rcuc51.
[ 278.724036] rcun ffffffff82440e00 starting wait for work.
[ 278.762564] rcun ffffffff82440e00 completed wait for work.
[ 278.762863] rcun ffffffff82440e00 initiating boost.
[ 278.763540] rcun ffffffff82440e00 completed boost.
[ 278.763782] rcun ffffffff82440e00 awaking rcuc51.
[ 278.764012] rcun ffffffff82440e00 awakened rcuc51.
[ 278.783768] rcun ffffffff82440e00 starting wait for work.
[ 278.784047] rcun ffffffff82440e00 completed wait for work.
[ 278.803684] rcun ffffffff82440e00 initiating boost.
[ 278.803937] rcun ffffffff82440e00 completed boost.
[ 278.823598] rcun ffffffff82440e00 awaking rcuc51.
[ 278.823851] rcun ffffffff82440e00 awakened rcuc51.
[ 278.843603] rcun ffffffff82440e00 starting wait for work.
[ 278.922171] rcun ffffffff82441300 completed wait for work.
[ 278.922498] rcun ffffffff82441300 initiating boost.
[ 278.922762] rcun ffffffff82441300 completed boost.
[ 278.934066] rcun ffffffff82441300 awaking rcuc131.
[ 278.934376] rcun ffffffff82441300 awakened rcuc131.
[ 278.953529] rcun ffffffff82441300 starting wait for work.
[ 279.017973] rcun ffffffff82440e00 completed wait for work.
[ 279.018288] rcun ffffffff82440e00 initiating boost.
[ 279.018515] rcun ffffffff82440e00 completed boost.
[ 279.033336] rcun ffffffff82440e00 awaking rcuc51.
[ 279.034041] rcun ffffffff82440e00 awakened rcuc51.
[ 279.053188] rcun ffffffff82440e00 starting wait for work.
[ 279.149846] rcun ffffffff82441300 completed wait for work.
[ 279.150185] rcun ffffffff82441300 initiating boost.
[ 279.150438] rcun ffffffff82441300 completed boost.
[ 279.163080] rcun ffffffff82441300 awaking rcuc131.
[ 279.163341] rcun ffffffff82441300 awakened rcuc131.
[ 279.183285] rcun ffffffff82441300 starting wait for work.
[ 279.313608] rcun ffffffff82441300 completed wait for work.
[ 279.313983] rcun ffffffff82441300 initiating boost.
[ 279.314216] rcun ffffffff82441300 completed boost.
[ 279.332841] rcun ffffffff82441300 awaking rcuc131.
[ 279.333359] rcun ffffffff82441300 awakened rcuc131.
[ 279.352563] rcun ffffffff82441300 starting wait for work.
[ 279.409412] rcun ffffffff82441300 completed wait for work.
[ 279.409775] rcun ffffffff82441300 initiating boost.
[ 279.410057] rcun ffffffff82441300 completed boost.
[ 279.422561] rcun ffffffff82441300 awaking rcuc131.
[ 279.422810] rcun ffffffff82441300 awakened rcuc131.
[ 279.442473] rcun ffffffff82441300 starting wait for work.
[ 279.932452] rcun ffffffff82441100 completed wait for work.
[ 279.932806] rcun ffffffff82441100 initiating boost.
[ 279.933047] rcun ffffffff82441100 completed boost.
[ 279.952298] rcun ffffffff82441100 awaking rcuc110.
[ 279.952658] rcun ffffffff82441100 awakened rcuc110.
[ 279.971749] rcun ffffffff82441100 starting wait for work.
[ 279.972249] rcun ffffffff82441100 completed wait for work.
[ 279.991659] rcun ffffffff82441100 initiating boost.
[ 279.992066] rcun ffffffff82441100 completed boost.
[ 280.011403] rcun ffffffff82441100 awaking rcuc110.
[ 280.011658] rcun ffffffff82441100 awakened rcuc110.
[ 280.012070] rcun ffffffff82441100 starting wait for work.
[ 280.112094] rcun ffffffff82440b00 completed wait for work.
[ 280.112427] rcun ffffffff82440b00 initiating boost.
[ 280.112674] rcun ffffffff82440b00 completed boost.
[ 280.131375] rcun ffffffff82440b00 awaking rcuc11.
[ 280.131651] rcun ffffffff82440b00 awakened rcuc11.
[ 280.151524] rcun ffffffff82440b00 starting wait for work.
[ 280.459704] rcun ffffffff82440b00 completed wait for work.
[ 280.459997] rcun ffffffff82440b00 initiating boost.
[ 280.460228] rcun ffffffff82440b00 completed boost.
[ 280.470779] rcun ffffffff82440b00 awaking rcuc0.
[ 280.471062] rcun ffffffff82440b00 awakened rcuc0.
[ 280.490721] rcun ffffffff82440b00 starting wait for work.
[ 280.567316] rcun ffffffff82441400 completed wait for work.
[ 280.567647] rcun ffffffff82441400 initiating boost.
[ 280.567897] rcun ffffffff82441400 completed boost.
[ 280.580815] rcun ffffffff82441400 awaking rcuc151.
[ 280.581116] rcun ffffffff82441400 awakened rcuc151.
[ 280.600382] rcun ffffffff82441400 starting wait for work.
[ 280.695170] rcun ffffffff82440f00 starting wait for work.
[ 280.695506] rcun ffffffff82440f00 completed wait for work.
[ 280.695847] rcun ffffffff82440f00 initiating boost.
[ 280.710661] rcun ffffffff82440f00 completed boost.
[ 280.711207] rcun ffffffff82440f00 awaking rcuc71.
[ 280.730198] rcun ffffffff82440f00 awakened rcuc71.
[ 280.730443] rcun ffffffff82440f00 starting wait for work.
[ 281.601394] rcun ffffffff82440c00 completed wait for work.
[ 281.601753] rcun ffffffff82440c00 initiating boost.
[ 281.602004] rcun ffffffff82440c00 completed boost.
[ 281.618891] rcun ffffffff82440c00 awaking rcuc30.
[ 281.619164] rcun ffffffff82440c00 awakened rcuc30.
[ 281.638755] rcun ffffffff82440c00 starting wait for work.
[ 281.729334] rcun ffffffff82441300 completed wait for work.
[ 281.729661] rcun ffffffff82441300 initiating boost.
[ 281.729920] rcun ffffffff82441300 completed boost.
[ 281.748587] rcun ffffffff82441300 awaking rcuc131.
[ 281.748871] rcun ffffffff82441300 awakened rcuc131.
[ 281.768287] rcun ffffffff82441300 starting wait for work.
[ 281.905078] rcun ffffffff82440b00 completed wait for work.
[ 281.905380] rcun ffffffff82440b00 initiating boost.
[ 281.905623] rcun ffffffff82440b00 completed boost.
[ 281.918170] rcun ffffffff82440b00 awaking rcuc11.
[ 281.918450] rcun ffffffff82440b00 awakened rcuc11.
[ 281.938055] rcun ffffffff82440b00 starting wait for work.
[ 282.240380] rcun ffffffff82441300 completed wait for work.
[ 282.240667] rcun ffffffff82441300 initiating boost.
[ 282.240890] rcun ffffffff82441300 completed boost.
[ 282.257498] rcun ffffffff82441300 awaking rcuc130.
[ 282.257772] rcun ffffffff82441300 awakened rcuc130.
[ 282.277380] rcun ffffffff82441300 starting wait for work.
[ 282.304255] rcun ffffffff82441300 completed wait for work.
[ 282.304551] rcun ffffffff82441300 initiating boost.
[ 282.304792] rcun ffffffff82441300 completed boost.
[ 282.317376] rcun ffffffff82441300 awaking rcuc130.
[ 282.317639] rcun ffffffff82441300 awakened rcuc130.
[ 282.337291] rcun ffffffff82441300 starting wait for work.
[ 282.427834] rcun ffffffff82440e00 completed wait for work.
[ 282.428165] rcun ffffffff82440e00 initiating boost.
[ 282.428404] rcun ffffffff82440e00 completed boost.
[ 282.447168] rcun ffffffff82440e00 awaking rcuc50.
[ 282.447398] rcun ffffffff82440e00 awakened rcuc50.
[ 282.467022] rcun ffffffff82440e00 starting wait for work.
[ 282.543751] rcun ffffffff82441300 completed wait for work.
[ 282.544030] rcun ffffffff82441300 initiating boost.
[ 282.544262] rcun ffffffff82441300 completed boost.
[ 282.556969] rcun ffffffff82441300 awaking rcuc130.
[ 282.557221] rcun ffffffff82441300 awakened rcuc130.
[ 282.576959] rcun ffffffff82441300 starting wait for work.
[ 282.651510] rcun ffffffff82440e00 completed wait for work.
[ 282.651859] rcun ffffffff82440e00 initiating boost.
[ 282.652115] rcun ffffffff82440e00 completed boost.
[ 282.666799] rcun ffffffff82440e00 awaking rcuc50.
[ 282.667062] rcun ffffffff82440e00 awakened rcuc50.
[ 282.686638] rcun ffffffff82440e00 starting wait for work.
[ 283.469957] rcun ffffffff82440c00 completed wait for work.
[ 283.470235] rcun ffffffff82440c00 initiating boost.
[ 283.470457] rcun ffffffff82440c00 completed boost.
[ 283.485312] rcun ffffffff82440c00 awaking rcuc20.
[ 283.485548] rcun ffffffff82440c00 awakened rcuc20.
[ 283.505188] rcun ffffffff82440c00 starting wait for work.
[ 283.513893] rcun ffffffff82440c00 completed wait for work.
[ 283.525161] rcun ffffffff82440c00 initiating boost.
[ 283.525381] rcun ffffffff82440c00 completed boost.
[ 283.545051] rcun ffffffff82440c00 awaking rcuc20.
[ 283.545289] rcun ffffffff82440c00 awakened rcuc20.
[ 283.545497] rcun ffffffff82440c00 starting wait for work.
[ 283.577780] rcun ffffffff82440c00 completed wait for work.
[ 283.578067] rcun ffffffff82440c00 initiating boost.
[ 283.585133] rcun ffffffff82440c00 completed boost.
[ 283.585366] rcun ffffffff82440c00 awaking rcuc20.
[ 283.604979] rcun ffffffff82440c00 awakened rcuc20.
[ 283.605219] rcun ffffffff82440c00 starting wait for work.
[ 283.673621] rcun ffffffff82440c00 completed wait for work.
[ 283.673904] rcun ffffffff82440c00 initiating boost.
[ 283.674126] rcun ffffffff82440c00 completed boost.
[ 283.684951] rcun ffffffff82440c00 awaking rcuc20.
[ 283.685186] rcun ffffffff82440c00 awakened rcuc20.
[ 283.704835] rcun ffffffff82440c00 starting wait for work.
[ 283.721536] rcun ffffffff82440c00 completed wait for work.
[ 283.724733] rcun ffffffff82440c00 initiating boost.
[ 283.724974] rcun ffffffff82440c00 completed boost.
[ 283.744676] rcun ffffffff82440c00 awaking rcuc20.
[ 283.744921] rcun ffffffff82440c00 awakened rcuc20.
[ 283.745142] rcun ffffffff82440c00 starting wait for work.
[ 283.849306] rcun ffffffff82440c00 completed wait for work.
[ 283.849580] rcun ffffffff82440c00 initiating boost.
[ 283.849806] rcun ffffffff82440c00 completed boost.
[ 283.864625] rcun ffffffff82440c00 awaking rcuc20.
[ 283.864859] rcun ffffffff82440c00 awakened rcuc20.
[ 283.884509] rcun ffffffff82440c00 starting wait for work.
[ 283.897233] rcun ffffffff82440c00 completed wait for work.
[ 283.904500] rcun ffffffff82440c00 initiating boost.
[ 283.904740] rcun ffffffff82440c00 completed boost.
[ 283.924388] rcun ffffffff82440c00 awaking rcuc20.
[ 283.924639] rcun ffffffff82440c00 awakened rcuc20.
[ 283.924857] rcun ffffffff82440c00 starting wait for work.
[ 283.961137] rcun ffffffff82440c00 completed wait for work.
[ 283.961412] rcun ffffffff82440c00 initiating boost.
[ 283.964375] rcun ffffffff82440c00 completed boost.
[ 283.964585] rcun ffffffff82440c00 awaking rcuc20.
[ 283.984347] rcun ffffffff82440c00 awakened rcuc20.
[ 283.984580] rcun ffffffff82440c00 starting wait for work.
[ 284.064957] rcun ffffffff82440c00 completed wait for work.
[ 284.065249] rcun ffffffff82440c00 initiating boost.
[ 284.065462] rcun ffffffff82440c00 completed boost.
[ 284.084281] rcun ffffffff82440c00 awaking rcuc20.
[ 284.084506] rcun ffffffff82440c00 awakened rcuc20.
[ 284.104135] rcun ffffffff82440c00 starting wait for work.
[ 284.124914] rcun ffffffff82440c00 completed wait for work.
[ 284.125251] rcun ffffffff82440c00 initiating boost.
[ 284.125488] rcun ffffffff82440c00 completed boost.
[ 284.144394] rcun ffffffff82440c00 awaking rcuc20.
[ 284.144636] rcun ffffffff82440c00 awakened rcuc20.
[ 284.163985] rcun ffffffff82440c00 starting wait for work.
[ 284.352449] rcun ffffffff82440c00 completed wait for work.
[ 284.352722] rcun ffffffff82440c00 initiating boost.
[ 284.352965] rcun ffffffff82440c00 completed boost.
[ 284.363729] rcun ffffffff82440c00 awaking rcuc21.
[ 284.363952] rcun ffffffff82440c00 awakened rcuc21.
[ 284.383609] rcun ffffffff82440c00 starting wait for work.
[ 284.400355] rcun ffffffff82440c00 completed wait for work.
[ 284.403541] rcun ffffffff82440c00 initiating boost.
[ 284.403752] rcun ffffffff82440c00 completed boost.
[ 284.423478] rcun ffffffff82440c00 awaking rcuc21.
[ 284.423730] rcun ffffffff82440c00 awakened rcuc21.
[ 284.423961] rcun ffffffff82440c00 starting wait for work.
[ 284.464264] rcun ffffffff82440c00 completed wait for work.
[ 284.464531] rcun ffffffff82440c00 initiating boost.
[ 284.464739] rcun ffffffff82440c00 completed boost.
[ 284.483510] rcun ffffffff82440c00 awaking rcuc21.
[ 284.483725] rcun ffffffff82440c00 awakened rcuc21.
[ 284.503396] rcun ffffffff82440c00 starting wait for work.
[ 284.524166] rcun ffffffff82440c00 completed wait for work.
[ 284.524430] rcun ffffffff82440c00 initiating boost.
[ 284.524660] rcun ffffffff82440c00 completed boost.
[ 284.543410] rcun ffffffff82440c00 awaking rcuc21.
[ 284.543634] rcun ffffffff82440c00 awakened rcuc21.
[ 284.563295] rcun ffffffff82440c00 starting wait for work.
done
Loading required kernel modules done
Activating device mapper...
FATAL: Could not load /lib/modules/2.6.39-tip-yh-05892-gb7b703b-dirty/modules.dep: No such file or directory
failed
Starting MD Raid unused
Waiting for udev to settle...
Scanning for LVM volume groups...
File descriptor 3 left open
Reading all physical volumes. This may take a while...
Activating LVM volume groups...
File descriptor 3 left open
done
Waiting for /firmware
microcode . no more events
Checking file systems...
fsck 1.41.1 (01-Sep-2008)
Checking all file systems. done
done
Mounting local file systems...
/proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
debugfs on /sys/kernel/debug type debugfs (rw)
udev on /dev type tmpfs (rw)
devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
/firmware on /lib/firmware type tmpfs (rw)
microcode on /usr/lib/microcode type tmpfs (rw) done
Activating remaining swap-devices in /etc/fstab... done
Setting up linker cache (/etc/ld.so.cache) using ldconfig done
Creating /var/log/boot.msg done
Using boot-specified hostname 'lb-g5plus-1t-host'
Setting up hostname 'lb-g5plus-1t-host' done
Setting up loopback interface lo
lo IP address: 127.0.0.1/8
IP address: 127.0.0.2/8
done
System Boot Control: The system has been set up
Skipped features: boot.md
System Boot Control: Running /etc/init.d/boot.local done
INIT: Entering runlevel: 3
Boot logging started on /dev/ttyS0(/dev/console) at Wed May 25 13:44:05 2011
Master Resource Control: previous runlevel: N, switching to runlevel:3
Starting D-Bus daemon done
Initializing random number generator done
[ 287.794818] rcun ffffffff82440f00 completed wait for work.
[ 287.795218] rcun ffffffff82440f00 initiating boost.
[ 287.795440] rcun ffffffff82440f00 completed boost.
[ 287.807922] rcun ffffffff82440f00 awaking rcuc71.
[ 287.808186] rcun ffffffff82440f00 awakened rcuc71.
[ 287.808195] rcun ffffffff82440f00 starting wait for work.
Starting syslog services done
[ 288.693183] rcun ffffffff82440e00 completed wait for work.
[ 288.693586] rcun ffffffff82440e00 initiating boost.
[ 288.693810] rcun ffffffff82440e00 completed boost.
[ 288.706108] rcun ffffffff82440e00 awaking rcuc50.
[ 288.706352] rcun ffffffff82440e00 awakened rcuc50.
[ 288.725895] rcun ffffffff82440e00 starting wait for work.
[ 288.726166] rcun ffffffff82440e00 completed wait for work.
[ 288.745842] rcun ffffffff82440e00 initiating boost.
[ 288.746067] rcun ffffffff82440e00 completed boost.
[ 288.765741] rcun ffffffff82440e00 awaking rcuc50.
[ 288.765967] rcun ffffffff82440e00 awakened rcuc50.
[ 288.766203] rcun ffffffff82440e00 starting wait for work.
Loading CPUFreq modules done
Starting HAL daemon done
Setting up (localfs) network interfaces:
lo
lo IP address: 127.0.0.1/8
IP address: 127.0.0.2/8 done
[ 289.323903] rcun ffffffff82440c00 completed wait for work.
[ 289.324239] rcun ffffffff82440c00 initiating boost.
[ 289.324458] rcun ffffffff82440c00 completed boost.
[ 289.334891] rcun ffffffff82440c00 awaking rcuc20.
[ 289.335112] rcun ffffffff82440c00 awakened rcuc20.
[ 289.354785] rcun ffffffff82440c00 starting wait for work.
eth0 device: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)
No configuration found for eth0 unused
eth1 device: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)
No configuration found for eth1 unused
[ 289.859076] rcun ffffffff82440c00 completed wait for work.
[ 289.859428] rcun ffffffff82440c00 initiating boost.
[ 289.859641] rcun ffffffff82440c00 completed boost.
[ 289.873915] rcun ffffffff82440c00 awaking rcuc31.
[ 289.874127] rcun ffffffff82440c00 awakened rcuc31.
[ 289.893814] rcun ffffffff82440c00 starting wait for work.
eth10 device: Intel Corporation 82576 Gigabit Network Connection (rev 01)
No configuration found for eth10 unused
[ 290.106822] rcun ffffffff82440b00 completed wait for work.
[ 290.107203] rcun ffffffff82440b00 initiating boost.
[ 290.107436] rcun ffffffff82440b00 completed boost.
[ 290.123462] rcun ffffffff82440b00 awaking rcuc11.
[ 290.123677] rcun ffffffff82440b00 awakened rcuc11.
[ 290.143342] rcun ffffffff82440b00 starting wait for work.
eth11 device: Intel Corporation 82576 Gigabit Network Connection (rev 01)
No configuration found for eth11 unused
[ 290.350351] rcun ffffffff82441100 completed wait for work.
[ 290.350744] rcun ffffffff82441100 initiating boost.
[ 290.351016] rcun ffffffff82441100 completed boost.
[ 290.363076] rcun ffffffff82441100 awaking rcuc111.
[ 290.363345] rcun ffffffff82441100 awakened rcuc111.
[ 290.382943] rcun ffffffff82441100 starting wait for work.
eth12 device: Intel Corporation 82599EB 10 Gigabit Network Connection (rev 01)
No configuration found for eth12 unused
[ 290.610137] rcun ffffffff82440b00 completed wait for work.
[ 290.610487] rcun ffffffff82440b00 initiating boost.
[ 290.610719] rcun ffffffff82440b00 completed boost.
[ 290.622562] rcun ffffffff82440b00 awaking rcuc11.
[ 290.622774] rcun ffffffff82440b00 awakened rcuc11.
[ 290.642453] rcun ffffffff82440b00 starting wait for work.
[ 290.642720] rcun ffffffff82440b00 completed wait for work.
[ 290.662423] rcun ffffffff82440b00 initiating boost.
[ 290.662643] rcun ffffffff82440b00 completed boost.
eth13 device: Intel Corporation 82599EB 10 Gigabit Network Connection (rev 01)
[ 290.682708] rcun ffffffff82440b00 awaking rcuc11.
[ 290.702367] rcun ffffffff82440b00 awakened rcuc11.
No configuration f[ 290.702694] rcun ffffffff82440b00 starting wait for work.
ound for eth13 unused
[ 290.865245] rcun ffffffff82440e00 completed wait for work.
[ 290.865586] rcun ffffffff82440e00 initiating boost.
[ 290.865802] rcun ffffffff82440e00 completed boost.
[ 290.882235] rcun ffffffff82440e00 awaking rcuc51.
[ 290.882530] rcun ffffffff82440e00 awakened rcuc51.
[ 290.902004] rcun ffffffff82440e00 starting wait for work.
eth14 device: Intel Corporation 82599EB 10-Gigabit KX4 Network Connection (rev 01)
No configuration found for eth14 unused
[ 291.128769] rcun ffffffff82440c00 completed wait for work.
[ 291.129110] rcun ffffffff82440c00 initiating boost.
[ 291.129333] rcun ffffffff82440c00 completed boost.
[ 291.141675] rcun ffffffff82440c00 awaking rcuc31.
[ 291.141904] rcun ffffffff82440c00 awakened rcuc31.
[ 291.161527] rcun ffffffff82440c00 starting wait for work.
eth15 device: Intel Corporation 82599EB 10-Gigabit KX4 Network Connection (rev 01)
No configuration found for eth15 unused
[ 291.380340] rcun ffffffff82440c00 completed wait for work.
[ 291.380664] rcun ffffffff82440c00 initiating boost.
[ 291.380883] rcun ffffffff82440c00 completed boost.
[ 291.391214] rcun ffffffff82440c00 awaking rcuc21.
[ 291.391431] rcun ffffffff82440c00 awakened rcuc21.
[ 291.411105] rcun ffffffff82440c00 starting wait for work.
[ 291.432271] rcun ffffffff82440c00 completed wait for work.
[ 291.432551] rcun ffffffff82440c00 initiating boost.
[ 291.432785] rcun ffffffff82440c00 completed boost.
[ 291.451161] rcun ffffffff82440c00 awaking rcuc21.
[ 291.451167] rcun ffffffff82440c00 awakened rcuc21.
[ 291.451170] rcun ffffffff82440c00 starting wait for work.
eth16 device: Intel Corporation 82599EB 10-Gigabit KX4 Network Connection (rev 01)
No configuration found for eth16 unused
[ 291.652073] rcun ffffffff82440b00 completed wait for work.
[ 291.652448] rcun ffffffff82440b00 initiating boost.
[ 291.652741] rcun ffffffff82440b00 completed boost.
[ 291.670699] rcun ffffffff82440b00 awaking rcuc1.
[ 291.670945] rcun ffffffff82440b00 awakened rcuc1.
[ 291.690597] rcun ffffffff82440b00 starting wait for work.
eth17 device: Intel Corporation 82599EB 10-Gigabit KX4 Network Connection (rev 01)
No configuration found for eth17 unused
[ 291.931570] rcun ffffffff82441000 starting wait for work.
[ 291.931899] rcun ffffffff82441000 completed wait for work.
[ 291.932142] rcun ffffffff82441000 initiating boost.
[ 291.950219] rcun ffffffff82441000 completed boost.
[ 291.950445] rcun ffffffff82441000 awaking rcuc90.
[ 291.970127] rcun ffffffff82441000 awakened rcuc90.
[ 291.970344] rcun ffffffff82441000 starting wait for work.
[ 291.990092] rcun ffffffff82441000 completed wait for work.
[ 291.990101] rcun ffffffff82441000 initiating boost.
[ 291.990105] rcun ffffffff82441000 completed boost.
[ 291.990109] rcun ffffffff82441000 awaking rcuc90.
[ 291.990119] rcun ffffffff82441000 awakened rcuc90.
[ 291.990123] rcun ffffffff82441000 starting wait for work.
eth18 device: Intel Corporation 82599EB 10 Gigabit Network Connection (rev 01)
No configuration found for eth18 unused
eth19 device: Intel Corporation 82599EB 10 Gigabit Network Connection (rev 01)
No configuration found for eth19 unused
[ 292.470547] rcun ffffffff82440e00 completed wait for work.
[ 292.470936] rcun ffffffff82440e00 initiating boost.
[ 292.471191] rcun ffffffff82440e00 completed boost.
[ 292.489291] rcun ffffffff82440e00 awaking rcuc51.
[ 292.489550] rcun ffffffff82440e00 awakened rcuc51.
[ 292.509226] rcun ffffffff82440e00 starting wait for work.
[ 292.509478] rcun ffffffff82440e00 completed wait for work.
[ 292.529136] rcun ffffffff82440e00 initiating boost.
[ 292.529141] rcun ffffffff82440e00 completed boost.
[ 292.529144] rcun ffffffff82440e00 awaking rcuc51.
[ 292.529151] rcun ffffffff82440e00 awakened rcuc51.
[ 292.529155] rcun ffffffff82440e00 starting wait for work.
eth2 device: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)
No configuration found for eth2 unused
[ 292.753993] rcun ffffffff82440c00 completed wait for work.
[ 292.754348] rcun ffffffff82440c00 initiating boost.
[ 292.754596] rcun ffffffff82440c00 completed boost.
[ 292.768779] rcun ffffffff82440c00 awaking rcuc31.
[ 292.769018] rcun ffffffff82440c00 awakened rcuc31.
[ 292.788632] rcun ffffffff82440c00 starting wait for work.
eth20 device: Intel Corporation 82599EB 10-Gigabit KX4 Network Connection (rev 01)
No configuration found for eth20 unused
[ 292.985585] rcun ffffffff82440c00 completed wait for work.
[ 292.985925] rcun ffffffff82440c00 initiating boost.
[ 292.986150] rcun ffffffff82440c00 completed boost.
[ 292.998422] rcun ffffffff82440c00 awaking rcuc31.
[ 292.998647] rcun ffffffff82440c00 awakened rcuc31.
[ 293.018318] rcun ffffffff82440c00 starting wait for work.
eth21 device: Intel Corporation 82599EB 10-Gigabit KX4 Network Connection (rev 01)
No configuration found for eth21 unused
[ 293.289596] rcun ffffffff82440b00 completed wait for work.
[ 293.289987] rcun ffffffff82440b00 initiating boost.
[ 293.290245] rcun ffffffff82440b00 completed boost.
[ 293.307893] rcun ffffffff82440b00 awaking rcuc1.
[ 293.308159] rcun ffffffff82440b00 awakened rcuc1.
[ 293.327728] rcun ffffffff82440b00 starting wait for work.
[ 293.337111] rcun ffffffff82440b00 completed wait for work.
[ 293.347737] rcun ffffffff82440b00 initiating boost.
[ 293.347958] rcun ffffffff82440b00 completed boost.
eth22 de[ 293.367776] rcun ffffffff82440b00 awaking rcuc1.
vice: Intel Corporation 82599EB [ 293.368186] rcun ffffffff82440b00 awakened rcuc1.
10 Gigabit Netwo[ 293.368191] rcun ffffffff82440b00 starting wait for work.
rk Connection (rev 01)
No configuration found for eth22 unused
eth23 device: Intel Corporation 82599EB 10 Gigabit Network Connection (rev 01)
No configuration found for eth23 unused
eth24 device: Intel Corporation 82599EB 10-Gigabit KX4 Network Connection (rev 01)
No configuration found for eth24 unused
[ 294.075710] rcun ffffffff82440e00 completed wait for work.
[ 294.076063] rcun ffffffff82440e00 initiating boost.
[ 294.076287] rcun ffffffff82440e00 completed boost.
[ 294.086523] rcun ffffffff82440e00 awaking rcuc51.
[ 294.086800] rcun ffffffff82440e00 awakened rcuc51.
[ 294.106385] rcun ffffffff82440e00 starting wait for work.
[ 294.135592] rcun ffffffff82440e00 completed wait for work.
[ 294.135876] rcun ffffffff82440e00 initiating boost.
[ 294.136103] rcun ffffffff82440e00 completed boost.
[ 294.146373] rcun ffffffff82440e00 awaking rcuc51.
[ 294.146643] rcun ffffffff82440e00 awakened rcuc51.
[ 294.166295] rcun ffffffff82440e00 starting wait for work.
eth25 device: Intel Corporation 82599EB 10-Gigabit KX4 Network Connection (rev 01)
No configuration found for eth25 unused
[ 294.383392] rcun ffffffff82440e00 completed wait for work.
[ 294.383728] rcun ffffffff82440e00 initiating boost.
[ 294.383945] rcun ffffffff82440e00 completed boost.
[ 294.395907] rcun ffffffff82440e00 awaking rcuc51.
[ 294.396134] rcun ffffffff82440e00 awakened rcuc51.
[ 294.415794] rcun ffffffff82440e00 starting wait for work.
eth26 device: QLogic Corp. 10GbE Converged Network Adapter (TCP/IP Networking) (rev 02)
No configuration found for eth26 unused
[ 294.658799] rcun ffffffff82440c00 completed wait for work.
[ 294.659130] rcun ffffffff82440c00 initiating boost.
[ 294.659341] rcun ffffffff82440c00 completed boost.
[ 294.675419] rcun ffffffff82440c00 awaking rcuc31.
[ 294.675645] rcun ffffffff82440c00 awakened rcuc31.
[ 294.695468] rcun ffffffff82440c00 starting wait for work.
eth27 device: QLogic Corp. 10GbE Converged Network Adapter (TCP/IP Networking) (rev 02)
No configuration found for eth27 unused
[ 294.918239] rcun ffffffff82440b00 completed wait for work.
[ 294.918602] rcun ffffffff82440b00 initiating boost.
[ 294.918834] rcun ffffffff82440b00 completed boost.
[ 294.934983] rcun ffffffff82440b00 awaking rcuc0.
[ 294.935245] rcun ffffffff82440b00 awakened rcuc0.
[ 294.954854] rcun ffffffff82440b00 starting wait for work.
eth3 device: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)
No configuration found for eth3 unused
[ 295.233919] rcun ffffffff82440d00 starting wait for work.
[ 295.234268] rcun ffffffff82440d00 completed wait for work.
[ 295.234550] rcun ffffffff82440d00 initiating boost.
[ 295.244455] rcun ffffffff82440d00 completed boost.
[ 295.244686] rcun ffffffff82440d00 awaking rcuc40.
eth4 de[ 295.264423] rcun ffffffff82440d00 awakened rcuc40.
vice: Intel Corporation 82576 Gi[ 295.264682] rcun ffffffff82440d00 starting wait for work.
gabit Network Connection (rev 01)
No configuration found for eth4 unused
[ 295.477285] rcun ffffffff82440f00 completed wait for work.
[ 295.477635] rcun ffffffff82440f00 initiating boost.
[ 295.477861] rcun ffffffff82440f00 completed boost.
[ 295.494030] rcun ffffffff82440f00 awaking rcuc70.
[ 295.494278] rcun ffffffff82440f00 awakened rcuc70.
[ 295.513989] rcun ffffffff82440f00 starting wait for work.
eth5 device: Intel Corporation 82576 Gigabit Network Connection (rev 01)
No configuration found for eth5 unused
[ 295.720833] rcun ffffffff82440d00 completed wait for work.
[ 295.721156] rcun ffffffff82440d00 initiating boost.
[ 295.721376] rcun ffffffff82440d00 completed boost.
[ 295.733558] rcun ffffffff82440d00 awaking rcuc41.
[ 295.733799] rcun ffffffff82440d00 awakened rcuc41.
[ 295.753424] rcun ffffffff82440d00 starting wait for work.
eth6 device: Intel Corporation 82576 Gigabit Network Connection (rev 01)
No configuration found for eth6 unused
[ 296.000356] rcun ffffffff82440c00 completed wait for work.
[ 296.000716] rcun ffffffff82440c00 initiating boost.
[ 296.000942] rcun ffffffff82440c00 completed boost.
[ 296.013086] rcun ffffffff82440c00 awaking rcuc31.
[ 296.013305] rcun ffffffff82440c00 awakened rcuc31.
[ 296.032935] rcun ffffffff82440c00 starting wait for work.
eth7 device: Intel Corporation 82576 Gigabit Network Connection (rev 01)
No configuration found for eth7 unused
[ 296.220235] rcun ffffffff82440e00 completed wait for work.
[ 296.220570] rcun ffffffff82440e00 initiating boost.
[ 296.220796] rcun ffffffff82440e00 completed boost.
[ 296.232694] rcun ffffffff82440e00 awaking rcuc51.
[ 296.232934] rcun ffffffff82440e00 awakened rcuc51.
[ 296.252558] rcun ffffffff82440e00 starting wait for work.
[ 296.252841] rcun ffffffff82440e00 completed wait for work.
[ 296.272515] rcun ffffffff82440e00 initiating boost.
[ 296.272784] rcun ffffffff82440e00 completed boost.
[ 296.292387] rcun ffffffff82440e00 awaking rcuc51.
[ 296.292647] rcun ffffffff82440e00 awakened rcuc51.
[ 296.292920] rcun ffffffff82440e00 starting wait for work.
eth8 device: Intel Corporation 82576 Gigabit Network Connection (rev 01)
No configuration found for eth8 unused
[ 296.483877] rcun ffffffff82440b00 completed wait for work.
[ 296.484237] rcun ffffffff82440b00 initiating boost.
[ 296.484456] rcun ffffffff82440b00 completed boost.
[ 296.502204] rcun ffffffff82440b00 awaking rcuc1.
[ 296.502491] rcun ffffffff82440b00 awakened rcuc1.
[ 296.522201] rcun ffffffff82440b00 starting wait for work.
[ 296.522459] rcun ffffffff82440b00 completed wait for work.
[ 296.542024] rcun ffffffff82440b00 initiating boost.
[ 296.542317] rcun ffffffff82440b00 completed boost.
[ 296.561935] rcun ffffffff82440b00 awaking rcuc1.
[ 296.562213] rcun ffffffff82440b00 awakened rcuc1.
[ 296.562431] rcun ffffffff82440b00 starting wait for work.
eth9 device: Intel Corporation 82576 Gigabit Network Connection (rev 01)
No configuration found for eth9 unused
ib0 device: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)
No configuration found for ib0 unused
[ 297.014805] rcun ffffffff82440b00 completed wait for work.
[ 297.015156] rcun ffffffff82440b00 initiating boost.
[ 297.015406] rcun ffffffff82440b00 completed boost.
[ 297.031238] rcun ffffffff82440b00 awaking rcuc1.
[ 297.031490] rcun ffffffff82440b00 awakened rcuc1.
[ 297.051130] rcun ffffffff82440b00 starting wait for work.
ib1 device: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)
No configuration found for ib1 unused
Setting up service (localfs) network . . . . . . . . . . done
Starting RPC portmap daemon done
Setting up (remotefs) network interfaces:
Setting up service (remotefs) network . . . . . . . . . . done
Master Resource Control: runlevel 3 has been reached
[ 298.843747] rcun ffffffff82440e00 completed wait for work.
[ 298.844125] rcun ffffffff82440e00 initiating boost.
[ 298.844380] rcun ffffffff82440e00 completed boost.
[ 298.858327] rcun ffffffff82440e00 awaking rcuc52.
[ 298.858648] rcun ffffffff82440e00 awakened rcuc52.
[ 298.878014] rcun ffffffff82440e00 starting wait for work.
[ 298.887761] rcun ffffffff82440c00 completed wait for work.
[ 298.898179] rcun ffffffff82440c00 initiating boost.
[ 298.898480] rcun ffffffff82440c00 completed boost.
[ 298.917825] rcun ffffffff82440c00 awaking rcuc30.
[ 298.918148] rcun ffffffff82440c00 awakened rcuc30.
[ 298.918399] rcun ffffffff82440c00 starting wait for work.
[ 299.051210] rcun ffffffff82441300 completed wait for work.
[ 299.051271] rcun ffffffff82440d00 completed wait for work.
[ 299.051279] rcun ffffffff82440d00 initiating boost.
[ 299.051283] rcun ffffffff82440d00 completed boost.
[ 299.051288] rcun ffffffff82440d00 awaking rcuc40.
[ 299.051300] rcun ffffffff82440c00 completed wait for work.
[ 299.051314] rcun ffffffff82440c00 initiating boost.
[ 299.051323] rcun ffffffff82440c00 completed boost.
[ 299.051328] rcun ffffffff82440c00 awaking rcuc31.
[ 299.051356] rcun ffffffff82440d00 awakened rcuc40.
[ 299.051360] rcun ffffffff82440d00 starting wait for work.
[ 299.051383] rcun ffffffff82440c00 awakened rcuc31.
[ 299.051388] rcun ffffffff82440c00 starting wait for work.
[ 299.148138] rcun ffffffff82441300 initiating boost.
[ 299.167706] rcun ffffffff82441300 completed boost.
[ 299.167993] rcun ffffffff82441300 awaking rcuc132.
[ 299.187516] rcun ffffffff82441300 awakened rcuc132.
[ 299.187841] rcun ffffffff82441300 starting wait for work.
[ 299.223164] rcun ffffffff82441300 completed wait for work.
[ 299.223485] rcun ffffffff82441300 initiating boost.
[ 299.227367] rcun ffffffff82441300 completed boost.
[ 299.227654] rcun ffffffff82441300 awaking rcuc132.
[ 299.227969] rcun ffffffff82441300 awakened rcuc132.
[ 299.247460] rcun ffffffff82441300 starting wait for work.
[ 299.278920] rcun ffffffff82440d00 completed wait for work.
[ 299.279251] rcun ffffffff82440d00 initiating boost.
[ 299.279497] rcun ffffffff82440d00 completed boost.
[ 299.297867] rcun ffffffff82440d00 awaking rcuc40.
[ 299.298180] rcun ffffffff82440d00 awakened rcuc40.
[ 299.317166] rcun ffffffff82440d00 starting wait for work.
[ 299.334809] rcun ffffffff82441300 completed wait for work.
[ 299.336979] rcun ffffffff82441300 initiating boost.
[ 299.337250] rcun ffffffff82441300 completed boost.
[ 299.338720] rcun ffffffff82441100 completed wait for work.
[ 299.338737] rcun ffffffff82441100 initiating boost.
[ 299.338745] rcun ffffffff82441100 completed boost.
[ 299.338752] rcun ffffffff82441100 awaking rcuc102.
[ 299.338803] rcun ffffffff82441100 awakened rcuc102.
[ 299.338809] rcun ffffffff82441100 starting wait for work.
[ 299.342935] rcun ffffffff82440f00 completed wait for work.
[ 299.342964] rcun ffffffff82440f00 initiating boost.
[ 299.342991] rcun ffffffff82440f00 completed boost.
[ 299.343009] rcun ffffffff82440f00 awaking rcuc70.
[ 299.343064] rcun ffffffff82440f00 awakened rcuc70.
[ 299.343077] rcun ffffffff82440f00 starting wait for work.
[ 299.350627] rcun ffffffff82441100 completed wait for work.
[ 299.350634] rcun ffffffff82441100 initiating boost.
[ 299.350638] rcun ffffffff82441100 completed boost.
[ 299.350641] rcun ffffffff82441100 awaking rcuc102.
[ 299.350652] rcun ffffffff82441100 awakened rcuc102.
[ 299.350655] rcun ffffffff82441100 starting wait for work.
[ 299.350689] rcun ffffffff82440f00 completed wait for work.
[ 299.350701] rcun ffffffff82440f00 initiating boost.
[ 299.350708] rcun ffffffff82440f00 completed boost.
[ 299.350716] rcun ffffffff82440f00 awaking rcuc70.
[ 299.350807] rcun ffffffff82440f00 awakened rcuc70.
[ 299.350815] rcun ffffffff82440f00 starting wait for work.
[ 299.557141] rcun ffffffff82441300 awaking rcuc132.
[ 299.557143] rcun ffffffff82441300 awakened rcuc132.
[ 299.557145] rcun ffffffff82441300 starting wait for work.
lb-g5plus-1t-host login:
lb-g5plus-1t-host login: [ 379.823934] INFO: task rcun0:8 blocked for more than 120 seconds.
[ 379.824295] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 379.843811] rcun0 D 0000000000000000 0 8 2 0x00000000
[ 379.844152] ffff882070d47e90 0000000000000046 ffff882070d46000 0000000000004000
[ 379.844178] 00000000001d1f40 ffff882070d47fd8 00000000001d1f40 ffff882070d47fd8
[ 379.844204] 0000000000004000 00000000001d1f40 ffff882070d18000 ffff882070d4a2b0
[ 379.844232] Call Trace:
[ 379.844255] [<ffffffff810afa0a>] ? __lock_release+0x166/0x16f
[ 379.844273] [<ffffffff81c21de9>] ? _raw_spin_unlock_irqrestore+0x3f/0x46
[ 379.844287] [<ffffffff810cf297>] ? rcu_cpu_kthread_timer+0x44/0x44
[ 379.844298] [<ffffffff810adcde>] ? trace_hardirqs_on+0xd/0xf
[ 379.844309] [<ffffffff810cf297>] ? rcu_cpu_kthread_timer+0x44/0x44
[ 379.844325] [<ffffffff81099e3d>] kthread+0x8c/0xa8
[ 379.844342] [<ffffffff81c29ad4>] kernel_thread_helper+0x4/0x10
[ 379.844353] [<ffffffff81c22080>] ? retint_restore_args+0xe/0xe
[ 379.844364] [<ffffffff81099db1>] ? __init_kthread_worker+0x5b/0x5b
[ 379.844375] [<ffffffff81c29ad0>] ? gs_change+0xb/0xb
[ 379.844379] INFO: lockdep is turned off.
[ 379.844595] INFO: task rcun8:576 blocked for more than 120 seconds.
[ 379.844598] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 379.844602] rcun8 D 0000000000000000 0 576 2 0x00000000
[ 379.844608] ffff881fff98de90 0000000000000046 ffff881fff98c000 0000000000004000
[ 379.844634] 00000000001d1f40 ffff881fff98dfd8 00000000001d1f40 ffff881fff98dfd8
[ 379.844658] 0000000000004000 00000000001d1f40 ffff882070d18000 ffff881fff9822b0
[ 379.844683] Call Trace:
[ 379.844694] [<ffffffff810afa0a>] ? __lock_release+0x166/0x16f
[ 379.844705] [<ffffffff81c21de9>] ? _raw_spin_unlock_irqrestore+0x3f/0x46
[ 379.844715] [<ffffffff810cf297>] ? rcu_cpu_kthread_timer+0x44/0x44
[ 379.844726] [<ffffffff810adcde>] ? trace_hardirqs_on+0xd/0xf
[ 379.844736] [<ffffffff810cf297>] ? rcu_cpu_kthread_timer+0x44/0x44
[ 379.844747] [<ffffffff81099e3d>] kthread+0x8c/0xa8
[ 379.844759] [<ffffffff81c29ad4>] kernel_thread_helper+0x4/0x10
[ 379.844769] [<ffffffff81c22080>] ? retint_restore_args+0xe/0xe
[ 379.844781] [<ffffffff81099db1>] ? __init_kthread_worker+0x5b/0x5b
[ 379.844791] [<ffffffff81c29ad0>] ? gs_change+0xb/0xb
[ 379.844794] INFO: lockdep is turned off.
next prev parent reply other threads:[~2011-05-23 22:02 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <tip-80d02085d99039b3b7f3a73c8896226b0cb1ba07@git.kernel.org>
2011-05-20 21:04 ` [tip:core/rcu] Revert "rcu: Decrease memory-barrier usage based on semi-formal proof" Yinghai Lu
2011-05-20 22:42 ` Paul E. McKenney
2011-05-20 23:09 ` Yinghai Lu
2011-05-20 23:14 ` Paul E. McKenney
2011-05-20 23:16 ` Yinghai Lu
2011-05-20 23:49 ` Paul E. McKenney
2011-05-21 0:02 ` Yinghai Lu
2011-05-21 13:18 ` Paul E. McKenney
2011-05-21 14:08 ` Paul E. McKenney
2011-05-23 20:14 ` Yinghai Lu
2011-05-23 21:25 ` Paul E. McKenney
2011-05-23 22:01 ` Yinghai Lu [this message]
2011-05-23 22:55 ` Yinghai Lu
2011-05-23 22:58 ` Yinghai Lu
2011-05-24 1:18 ` Paul E. McKenney
2011-05-24 1:26 ` Yinghai Lu
2011-05-24 1:35 ` Paul E. McKenney
2011-05-24 21:23 ` Yinghai Lu
2011-05-25 0:05 ` Paul E. McKenney
2011-05-25 0:13 ` Yinghai Lu
2011-05-25 4:46 ` Paul E. McKenney
2011-05-25 7:24 ` Ingo Molnar
2011-05-25 20:48 ` Paul E. McKenney
2011-05-25 7:18 ` Ingo Molnar
2011-05-25 0:16 ` Paul E. McKenney
2011-05-25 0:10 ` Yinghai Lu
2011-05-25 4:52 ` Paul E. McKenney
2011-05-25 7:27 ` Ingo Molnar
2011-05-25 20:47 ` Paul E. McKenney
2011-05-25 20:52 ` Ingo Molnar
2011-05-25 22:15 ` Yinghai Lu
2011-05-25 22:34 ` Paul E. McKenney
2011-05-25 22:49 ` Yinghai Lu
2011-05-26 1:13 ` Paul E. McKenney
2011-05-26 1:30 ` Paul E. McKenney
2011-05-26 6:13 ` Ingo Molnar
2011-05-26 14:25 ` Paul E. McKenney
2011-05-26 17:43 ` Paul E. McKenney
2011-05-26 20:26 ` Ingo Molnar
2011-05-26 15:08 ` Yinghai Lu
2011-05-26 16:28 ` Paul E. McKenney
2011-05-28 1:04 ` Paul E. McKenney
2011-05-28 4:03 ` Yinghai Lu
2011-05-28 6:38 ` Paul E. McKenney
2011-05-24 1:12 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4DDAD934.9010603@kernel.org \
--to=yinghai@kernel.org \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=mingo@redhat.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.