2.6.32-rc4: Reported regressions from 2.6.31

2.6.32-rc4: Reported regressions from 2.6.31

[Rafael J. Wysocki]

[Note:
  We started to get more regression reports for 2.6.32-rc (100%+ jump in the
  last 10 days.  Nevertheless, we're still receiving new reports of regressions
  from 2.6.30.]

This message contains a list of some regressions from 2.6.31, for which there
are no fixes in the mainline I know of.  If any of them have been fixed already,
please let me know.

If you know of any other unresolved regressions from 2.6.31, please let me know
either and I'll add them to the list.  Also, please let me know if any of the
entries below are invalid.

Each entry from the list will be sent additionally in an automatic reply to
this message with CCs to the people involved in reporting and handling the
issue.


Listed regressions statistics:

  Date          Total  Pending  Unresolved
  ----------------------------------------
  2009-10-12       48       31          27
  2009-10-02       22       15           9


Unresolved regressions
----------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14392
Subject		: Touchpad "paste" stops working after suspend to RAM
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-11 16:21 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125527987316493&w=4
Handled-By	: Dmitry Torokhov <dmitry.torokhov@gmail.com>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14390
Subject		: "bind" a device to a driver doesn't not work anymore
Submitter	: Éric Piel <Eric.Piel@tremplin-utc.net>
Date		: 2009-10-11 0:04 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125521979921241&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
Subject		: Build system issue
Submitter	: Peter Zijlstra <peterz@infradead.org>
Date		: 2009-10-09 8:58 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14387
Subject		: deadlock with fallocate
Submitter	: Thomas Neumann <tneumann@users.sourceforge.net>
Date		: 2009-10-07 3:00 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125488495526471&w=4
Handled-By	: Christoph Hellwig <hch@lst.de>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14386
Subject		: GPF in snd_hda_intel
Submitter	: Luca Tettamanti <kronos.it@gmail.com>
Date		: 2009-10-10 13:01 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125517999408019&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14384
Subject		: tbench regression with 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:51 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59abf02644c45f1591e1374ee7bb45dc757fcb88
References	: http://marc.info/?l=linux-kernel&m=125508216713138&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14383
Subject		: hackbench regression with kernel 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:19 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29cd8bae396583a2ee9a3340db8c5102acf9f6fd
References	: http://marc.info/?l=linux-kernel&m=125508007510274&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14381
Subject		: iwlagn lost connection after s2ram (with warnings)
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-07 14:20 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125492569119947&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14380
Subject		: Video tearing/glitching with T400 laptops
Submitter	: Theodore Ts'o <tytso@mit.edu>
Date		: 2009-10-02 22:40 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125452324520623&w=4
Handled-By	: Jesse Barnes <jbarnes@virtuousgeek.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock@gmail.com>
Date		: 2009-10-08 21:46 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra@navynet.it>
Date		: 2009-10-08 14:51 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14376
Subject		: Kernel NULL pointer dereference/ kvm subsystem
Submitter	: Don Dupuis <dondster@gmail.com>
Date		: 2009-10-06 14:38 (6 days old)
References	: http://marc.info/?l=linux-kernel&m=125484025021737&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14374
Subject		: MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea
Submitter	: Nick Piggin <npiggin@suse.de>
Date		: 2009-10-02 7:34 (10 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=db8be50c4307dac2b37305fc59c8dc0f978d09ea
References	: http://marc.info/?l=linux-kernel&m=125446885705223&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14373
Subject		: Task blocked for more than 120 seconds
Submitter	: Zeno Davatz <zdavatz@gmail.com>
Date		: 2009-10-02 10:16 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447858618412&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14372
Subject		: Wireless not working after suspend-resume
Submitter	: Fabio Comolli <fabio.comolli@gmail.com>
Date		: 2009-10-03 15:36 (9 days old)
References	: http://lkml.org/lkml/2009/10/3/91


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
Subject		: ext4 corruptions
Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
Date		: 2009-10-09 19:20 (3 days old)
References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14355
Subject		: USB serial regression after 2.6.31.1 with Huawei E169 GSM modem
Submitter	: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date		: 2009-10-10 03:07 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125513456327542&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
Subject		: Bad corruption with 2.6.32-rc1 and upwards
Submitter	: Holger Freyther <zecke@selfish.org>
Date		: 2009-10-09 15:42 (3 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
Submitter	: Miles Lane <miles.lane@gmail.com>
Date		: 2009-10-05 3:39 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14352
Subject		: WARNING: at net/mac80211/scan.c:267
Submitter	: Maciej Rutecki <maciej.rutecki@gmail.com>
Date		: 2009-10-08 00:30 (4 days old)
References	: http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089#c7


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14334
Subject		: pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m
Submitter	: Jose Marino <braket@hotmail.com>
Date		: 2009-10-06 15:44 (6 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
Subject		: oops in wireless, iwl3945 related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-29 17:12 (13 days old)
References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14298
Subject		: warning at manage.c:361 (set_irq_wake), matrix-keypad related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 20:07 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125434130703538&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14297
Subject		: console resume broken since ba15ab0e8d
Submitter	: Sascha Hauer <s.hauer@pengutronix.de>
Date		: 2009-09-30 15:11 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125432349404060&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14296
Subject		: spitz boots but suspend/resume is broken
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 12:06 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125431244516449&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14279
Subject		: Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:14 (12 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5f68563996e812f9ca35b3939ad2a42e5d254d66


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14277
Subject		: Caught 8-bit read from freed memory in b43 driver at association
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:06 (12 days old)


Regressions with patches
------------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14382
Subject		: Transmit failure in et131x.
Submitter	: Nick Bowler <nbowler@elliptictech.com>
Date		: 2009-10-08 14:08 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501117713744&w=4
Handled-By	: Alan Cox <alan@lxorguk.ukuu.org.uk>
Patch		: http://patchwork.kernel.org/patch/52698/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14375
Subject		: Intel(R) I/OAT DMA Engine init failed
Submitter	: Alexander Beregalov <a.beregalov@gmail.com>
Date		: 2009-10-02 9:46 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447680016160&w=4
Handled-By	: Dan Williams <dan.j.williams@intel.com>
Patch		: http://patchwork.kernel.org/patch/51808/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14302
Subject		: Kernel panic on i386 machine when booting with profile=2
Submitter	: Shi, Alex <alex.shi@intel.com>
Date		: 2009-10-01 3:23 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=125436749607199&w=4
Handled-By	: Alex Shi <alex.shi@intel.com>
Patch		: http://patchwork.kernel.org/patch/50813/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:12 (12 days old)
Handled-By	: Michael Buesch <mb@bu3sch.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220


For details, please visit the bug entries and follow the links given in
references.

As you can see, there is a Bugzilla entry for each of the listed regressions.
There also is a Bugzilla entry used for tracking the regressions from 2.6.31,
unresolved as well as resolved, at:

http://bugzilla.kernel.org/show_bug.cgi?id=14230

Please let me know if there are any Bugzilla entries that should be added to
the list in there.

Thanks,
Rafael

2.6.32-rc4: Reported regressions from 2.6.31

[Rafael J. Wysocki]

[Note:
  We started to get more regression reports for 2.6.32-rc (100%+ jump in the
  last 10 days.  Nevertheless, we're still receiving new reports of regressions
  from 2.6.30.]

This message contains a list of some regressions from 2.6.31, for which there
are no fixes in the mainline I know of.  If any of them have been fixed already,
please let me know.

If you know of any other unresolved regressions from 2.6.31, please let me know
either and I'll add them to the list.  Also, please let me know if any of the
entries below are invalid.

Each entry from the list will be sent additionally in an automatic reply to
this message with CCs to the people involved in reporting and handling the
issue.


Listed regressions statistics:

  Date          Total  Pending  Unresolved
  ----------------------------------------
  2009-10-12       48       31          27
  2009-10-02       22       15           9


Unresolved regressions
----------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14392
Subject		: Touchpad "paste" stops working after suspend to RAM
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-11 16:21 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125527987316493&w=4
Handled-By	: Dmitry Torokhov <dmitry.torokhov@gmail.com>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14390
Subject		: "bind" a device to a driver doesn't not work anymore
Submitter	: Éric Piel <Eric.Piel@tremplin-utc.net>
Date		: 2009-10-11 0:04 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125521979921241&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
Subject		: Build system issue
Submitter	: Peter Zijlstra <peterz@infradead.org>
Date		: 2009-10-09 8:58 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14387
Subject		: deadlock with fallocate
Submitter	: Thomas Neumann <tneumann@users.sourceforge.net>
Date		: 2009-10-07 3:00 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125488495526471&w=4
Handled-By	: Christoph Hellwig <hch@lst.de>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14386
Subject		: GPF in snd_hda_intel
Submitter	: Luca Tettamanti <kronos.it@gmail.com>
Date		: 2009-10-10 13:01 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125517999408019&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14384
Subject		: tbench regression with 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:51 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59abf02644c45f1591e1374ee7bb45dc757fcb88
References	: http://marc.info/?l=linux-kernel&m=125508216713138&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14383
Subject		: hackbench regression with kernel 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:19 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29cd8bae396583a2ee9a3340db8c5102acf9f6fd
References	: http://marc.info/?l=linux-kernel&m=125508007510274&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14381
Subject		: iwlagn lost connection after s2ram (with warnings)
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-07 14:20 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125492569119947&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14380
Subject		: Video tearing/glitching with T400 laptops
Submitter	: Theodore Ts'o <tytso@mit.edu>
Date		: 2009-10-02 22:40 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125452324520623&w=4
Handled-By	: Jesse Barnes <jbarnes@virtuousgeek.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock@gmail.com>
Date		: 2009-10-08 21:46 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra@navynet.it>
Date		: 2009-10-08 14:51 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14376
Subject		: Kernel NULL pointer dereference/ kvm subsystem
Submitter	: Don Dupuis <dondster@gmail.com>
Date		: 2009-10-06 14:38 (6 days old)
References	: http://marc.info/?l=linux-kernel&m=125484025021737&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14374
Subject		: MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea
Submitter	: Nick Piggin <npiggin@suse.de>
Date		: 2009-10-02 7:34 (10 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=db8be50c4307dac2b37305fc59c8dc0f978d09ea
References	: http://marc.info/?l=linux-kernel&m=125446885705223&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14373
Subject		: Task blocked for more than 120 seconds
Submitter	: Zeno Davatz <zdavatz@gmail.com>
Date		: 2009-10-02 10:16 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447858618412&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14372
Subject		: Wireless not working after suspend-resume
Submitter	: Fabio Comolli <fabio.comolli@gmail.com>
Date		: 2009-10-03 15:36 (9 days old)
References	: http://lkml.org/lkml/2009/10/3/91


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
Subject		: ext4 corruptions
Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
Date		: 2009-10-09 19:20 (3 days old)
References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14355
Subject		: USB serial regression after 2.6.31.1 with Huawei E169 GSM modem
Submitter	: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date		: 2009-10-10 03:07 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125513456327542&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
Subject		: Bad corruption with 2.6.32-rc1 and upwards
Submitter	: Holger Freyther <zecke@selfish.org>
Date		: 2009-10-09 15:42 (3 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
Submitter	: Miles Lane <miles.lane@gmail.com>
Date		: 2009-10-05 3:39 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14352
Subject		: WARNING: at net/mac80211/scan.c:267
Submitter	: Maciej Rutecki <maciej.rutecki@gmail.com>
Date		: 2009-10-08 00:30 (4 days old)
References	: http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089#c7


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14334
Subject		: pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m
Submitter	: Jose Marino <braket@hotmail.com>
Date		: 2009-10-06 15:44 (6 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
Subject		: oops in wireless, iwl3945 related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-29 17:12 (13 days old)
References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14298
Subject		: warning at manage.c:361 (set_irq_wake), matrix-keypad related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 20:07 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125434130703538&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14297
Subject		: console resume broken since ba15ab0e8d
Submitter	: Sascha Hauer <s.hauer@pengutronix.de>
Date		: 2009-09-30 15:11 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125432349404060&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14296
Subject		: spitz boots but suspend/resume is broken
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 12:06 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125431244516449&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14279
Subject		: Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:14 (12 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5f68563996e812f9ca35b3939ad2a42e5d254d66


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14277
Subject		: Caught 8-bit read from freed memory in b43 driver at association
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:06 (12 days old)


Regressions with patches
------------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14382
Subject		: Transmit failure in et131x.
Submitter	: Nick Bowler <nbowler@elliptictech.com>
Date		: 2009-10-08 14:08 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501117713744&w=4
Handled-By	: Alan Cox <alan@lxorguk.ukuu.org.uk>
Patch		: http://patchwork.kernel.org/patch/52698/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14375
Subject		: Intel(R) I/OAT DMA Engine init failed
Submitter	: Alexander Beregalov <a.beregalov@gmail.com>
Date		: 2009-10-02 9:46 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447680016160&w=4
Handled-By	: Dan Williams <dan.j.williams@intel.com>
Patch		: http://patchwork.kernel.org/patch/51808/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14302
Subject		: Kernel panic on i386 machine when booting with profile=2
Submitter	: Shi, Alex <alex.shi@intel.com>
Date		: 2009-10-01 3:23 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=125436749607199&w=4
Handled-By	: Alex Shi <alex.shi@intel.com>
Patch		: http://patchwork.kernel.org/patch/50813/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:12 (12 days old)
Handled-By	: Michael Buesch <mb@bu3sch.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220


For details, please visit the bug entries and follow the links given in
references.

As you can see, there is a Bugzilla entry for each of the listed regressions.
There also is a Bugzilla entry used for tracking the regressions from 2.6.31,
unresolved as well as resolved, at:

http://bugzilla.kernel.org/show_bug.cgi?id=14230

Please let me know if there are any Bugzilla entries that should be added to
the list in there.

Thanks,
Rafael

_______________________________________________
linux-pm mailing list
linux-pm@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/linux-pm

2.6.32-rc4: Reported regressions from 2.6.31

[Rafael J. Wysocki]

[Note:
  We started to get more regression reports for 2.6.32-rc (100%+ jump in the
  last 10 days.  Nevertheless, we're still receiving new reports of regressions
  from 2.6.30.]

This message contains a list of some regressions from 2.6.31, for which there
are no fixes in the mainline I know of.  If any of them have been fixed already,
please let me know.

If you know of any other unresolved regressions from 2.6.31, please let me know
either and I'll add them to the list.  Also, please let me know if any of the
entries below are invalid.

Each entry from the list will be sent additionally in an automatic reply to
this message with CCs to the people involved in reporting and handling the
issue.


Listed regressions statistics:

  Date          Total  Pending  Unresolved
  ----------------------------------------
  2009-10-12       48       31          27
  2009-10-02       22       15           9


Unresolved regressions
----------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14392
Subject		: Touchpad "paste" stops working after suspend to RAM
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-11 16:21 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125527987316493&w=4
Handled-By	: Dmitry Torokhov <dmitry.torokhov@gmail.com>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14390
Subject		: "bind" a device to a driver doesn't not work anymore
Submitter	: Éric Piel <Eric.Piel@tremplin-utc.net>
Date		: 2009-10-11 0:04 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125521979921241&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
Subject		: Build system issue
Submitter	: Peter Zijlstra <peterz@infradead.org>
Date		: 2009-10-09 8:58 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14387
Subject		: deadlock with fallocate
Submitter	: Thomas Neumann <tneumann@users.sourceforge.net>
Date		: 2009-10-07 3:00 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125488495526471&w=4
Handled-By	: Christoph Hellwig <hch@lst.de>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14386
Subject		: GPF in snd_hda_intel
Submitter	: Luca Tettamanti <kronos.it@gmail.com>
Date		: 2009-10-10 13:01 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125517999408019&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14384
Subject		: tbench regression with 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:51 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59abf02644c45f1591e1374ee7bb45dc757fcb88
References	: http://marc.info/?l=linux-kernel&m=125508216713138&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14383
Subject		: hackbench regression with kernel 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:19 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29cd8bae396583a2ee9a3340db8c5102acf9f6fd
References	: http://marc.info/?l=linux-kernel&m=125508007510274&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14381
Subject		: iwlagn lost connection after s2ram (with warnings)
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-07 14:20 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125492569119947&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14380
Subject		: Video tearing/glitching with T400 laptops
Submitter	: Theodore Ts'o <tytso@mit.edu>
Date		: 2009-10-02 22:40 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125452324520623&w=4
Handled-By	: Jesse Barnes <jbarnes@virtuousgeek.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock@gmail.com>
Date		: 2009-10-08 21:46 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra@navynet.it>
Date		: 2009-10-08 14:51 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14376
Subject		: Kernel NULL pointer dereference/ kvm subsystem
Submitter	: Don Dupuis <dondster@gmail.com>
Date		: 2009-10-06 14:38 (6 days old)
References	: http://marc.info/?l=linux-kernel&m=125484025021737&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14374
Subject		: MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea
Submitter	: Nick Piggin <npiggin@suse.de>
Date		: 2009-10-02 7:34 (10 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=db8be50c4307dac2b37305fc59c8dc0f978d09ea
References	: http://marc.info/?l=linux-kernel&m=125446885705223&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14373
Subject		: Task blocked for more than 120 seconds
Submitter	: Zeno Davatz <zdavatz@gmail.com>
Date		: 2009-10-02 10:16 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447858618412&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14372
Subject		: Wireless not working after suspend-resume
Submitter	: Fabio Comolli <fabio.comolli@gmail.com>
Date		: 2009-10-03 15:36 (9 days old)
References	: http://lkml.org/lkml/2009/10/3/91


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
Subject		: ext4 corruptions
Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
Date		: 2009-10-09 19:20 (3 days old)
References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14355
Subject		: USB serial regression after 2.6.31.1 with Huawei E169 GSM modem
Submitter	: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date		: 2009-10-10 03:07 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125513456327542&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
Subject		: Bad corruption with 2.6.32-rc1 and upwards
Submitter	: Holger Freyther <zecke@selfish.org>
Date		: 2009-10-09 15:42 (3 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
Submitter	: Miles Lane <miles.lane@gmail.com>
Date		: 2009-10-05 3:39 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14352
Subject		: WARNING: at net/mac80211/scan.c:267
Submitter	: Maciej Rutecki <maciej.rutecki@gmail.com>
Date		: 2009-10-08 00:30 (4 days old)
References	: http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089#c7


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14334
Subject		: pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m
Submitter	: Jose Marino <braket@hotmail.com>
Date		: 2009-10-06 15:44 (6 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
Subject		: oops in wireless, iwl3945 related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-29 17:12 (13 days old)
References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14298
Subject		: warning at manage.c:361 (set_irq_wake), matrix-keypad related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 20:07 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125434130703538&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14297
Subject		: console resume broken since ba15ab0e8d
Submitter	: Sascha Hauer <s.hauer@pengutronix.de>
Date		: 2009-09-30 15:11 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125432349404060&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14296
Subject		: spitz boots but suspend/resume is broken
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 12:06 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125431244516449&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14279
Subject		: Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:14 (12 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5f68563996e812f9ca35b3939ad2a42e5d254d66


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14277
Subject		: Caught 8-bit read from freed memory in b43 driver at association
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:06 (12 days old)


Regressions with patches
------------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14382
Subject		: Transmit failure in et131x.
Submitter	: Nick Bowler <nbowler@elliptictech.com>
Date		: 2009-10-08 14:08 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501117713744&w=4
Handled-By	: Alan Cox <alan@lxorguk.ukuu.org.uk>
Patch		: http://patchwork.kernel.org/patch/52698/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14375
Subject		: Intel(R) I/OAT DMA Engine init failed
Submitter	: Alexander Beregalov <a.beregalov@gmail.com>
Date		: 2009-10-02 9:46 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447680016160&w=4
Handled-By	: Dan Williams <dan.j.williams@intel.com>
Patch		: http://patchwork.kernel.org/patch/51808/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14302
Subject		: Kernel panic on i386 machine when booting with profile=2
Submitter	: Shi, Alex <alex.shi@intel.com>
Date		: 2009-10-01 3:23 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=125436749607199&w=4
Handled-By	: Alex Shi <alex.shi@intel.com>
Patch		: http://patchwork.kernel.org/patch/50813/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:12 (12 days old)
Handled-By	: Michael Buesch <mb@bu3sch.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220


For details, please visit the bug entries and follow the links given in
references.

As you can see, there is a Bugzilla entry for each of the listed regressions.
There also is a Bugzilla entry used for tracking the regressions from 2.6.31,
unresolved as well as resolved, at:

http://bugzilla.kernel.org/show_bug.cgi?id=14230

Please let me know if there are any Bugzilla entries that should be added to
the list in there.

Thanks,
Rafael

_______________________________________________
linux-pm mailing list
linux-pm@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/linux-pm

[Bug #14277] Caught 8-bit read from freed memory in b43 driver at association

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14277
Subject		: Caught 8-bit read from freed memory in b43 driver at association
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:06 (12 days old)

[Bug #14277] Caught 8-bit read from freed memory in b43 driver at association

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14277
Subject		: Caught 8-bit read from freed memory in b43 driver at association
Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
Date		: 2009-09-30 18:06 (12 days old)

[Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:12 (12 days old)
Handled-By	: Michael Buesch <mb@bu3sch.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220

[Bug #14279] Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14279
Subject		: Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:14 (12 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5f68563996e812f9ca35b3939ad2a42e5d254d66

[Bug #14296] spitz boots but suspend/resume is broken

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14296
Subject		: spitz boots but suspend/resume is broken
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 12:06 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125431244516449&w=4

[Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
Date		: 2009-09-30 18:12 (12 days old)
Handled-By	: Michael Buesch <mb-fseUSCV1ubazQB+pC5nmwQ@public.gmane.org>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220

[Bug #14279] Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14279
Subject		: Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop
Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
Date		: 2009-09-30 18:14 (12 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5f68563996e812f9ca35b3939ad2a42e5d254d66

[Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14334
Subject		: pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m
Submitter	: Jose Marino <braket@hotmail.com>
Date		: 2009-10-06 15:44 (6 days old)

[Bug #14297] console resume broken since ba15ab0e8d

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14297
Subject		: console resume broken since ba15ab0e8d
Submitter	: Sascha Hauer <s.hauer@pengutronix.de>
Date		: 2009-09-30 15:11 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125432349404060&w=4

[Bug #14302] Kernel panic on i386 machine when booting with profile=2

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14302
Subject		: Kernel panic on i386 machine when booting with profile=2
Submitter	: Shi, Alex <alex.shi@intel.com>
Date		: 2009-10-01 3:23 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=125436749607199&w=4
Handled-By	: Alex Shi <alex.shi@intel.com>
Patch		: http://patchwork.kernel.org/patch/50813/

[Bug #14352] WARNING: at net/mac80211/scan.c:267

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14352
Subject		: WARNING: at net/mac80211/scan.c:267
Submitter	: Maciej Rutecki <maciej.rutecki@gmail.com>
Date		: 2009-10-08 00:30 (4 days old)
References	: http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089#c7

[Bug #14298] warning at manage.c:361 (set_irq_wake), matrix-keypad related?

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14298
Subject		: warning at manage.c:361 (set_irq_wake), matrix-keypad related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 20:07 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125434130703538&w=4

[Bug #14299] oops in wireless, iwl3945 related?

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
Subject		: oops in wireless, iwl3945 related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-29 17:12 (13 days old)
References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4

[Bug #14297] console resume broken since ba15ab0e8d

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14297
Subject		: console resume broken since ba15ab0e8d
Submitter	: Sascha Hauer <s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ@public.gmane.org>
Date		: 2009-09-30 15:11 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125432349404060&w=4

[Bug #14352] WARNING: at net/mac80211/scan.c:267

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14352
Subject		: WARNING: at net/mac80211/scan.c:267
Submitter	: Maciej Rutecki <maciej.rutecki-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-08 00:30 (4 days old)
References	: http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089#c7

[Bug #14302] Kernel panic on i386 machine when booting with profile=2

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14302
Subject		: Kernel panic on i386 machine when booting with profile=2
Submitter	: Shi, Alex <alex.shi-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-01 3:23 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=125436749607199&w=4
Handled-By	: Alex Shi <alex.shi-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/50813/

[Bug #14353] BUG: sleeping function called from invalid context at kernel/mutex.c:280

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
Submitter	: Miles Lane <miles.lane@gmail.com>
Date		: 2009-10-05 3:39 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4

[Bug #14355] USB serial regression after 2.6.31.1 with Huawei E169 GSM modem

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14355
Subject		: USB serial regression after 2.6.31.1 with Huawei E169 GSM modem
Submitter	: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date		: 2009-10-10 03:07 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125513456327542&w=4

[Bug #14370] ext4 corruptions

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
Subject		: ext4 corruptions
Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
Date		: 2009-10-09 19:20 (3 days old)
References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4

[Bug #14354] Bad corruption with 2.6.32-rc1 and upwards

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
Subject		: Bad corruption with 2.6.32-rc1 and upwards
Submitter	: Holger Freyther <zecke@selfish.org>
Date		: 2009-10-09 15:42 (3 days old)

[Bug #14353] BUG: sleeping function called from invalid context at kernel/mutex.c:280

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-05 3:39 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4

[Bug #14372] Wireless not working after suspend-resume

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14372
Subject		: Wireless not working after suspend-resume
Submitter	: Fabio Comolli <fabio.comolli@gmail.com>
Date		: 2009-10-03 15:36 (9 days old)
References	: http://lkml.org/lkml/2009/10/3/91

[Bug #14374] MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14374
Subject		: MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea
Submitter	: Nick Piggin <npiggin@suse.de>
Date		: 2009-10-02 7:34 (10 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=db8be50c4307dac2b37305fc59c8dc0f978d09ea
References	: http://marc.info/?l=linux-kernel&m=125446885705223&w=4

[Bug #14373] Task blocked for more than 120 seconds

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14373
Subject		: Task blocked for more than 120 seconds
Submitter	: Zeno Davatz <zdavatz@gmail.com>
Date		: 2009-10-02 10:16 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447858618412&w=4

[Bug #14373] Task blocked for more than 120 seconds

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14373
Subject		: Task blocked for more than 120 seconds
Submitter	: Zeno Davatz <zdavatz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-02 10:16 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447858618412&w=4

[Bug #14372] Wireless not working after suspend-resume

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14372
Subject		: Wireless not working after suspend-resume
Submitter	: Fabio Comolli <fabio.comolli-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-03 15:36 (9 days old)
References	: http://lkml.org/lkml/2009/10/3/91

[Bug #14378] Problems with net/core/skbuff.c

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra@navynet.it>
Date		: 2009-10-08 14:51 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4

[Bug #14375] Intel(R) I/OAT DMA Engine init failed

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14375
Subject		: Intel(R) I/OAT DMA Engine init failed
Submitter	: Alexander Beregalov <a.beregalov@gmail.com>
Date		: 2009-10-02 9:46 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447680016160&w=4
Handled-By	: Dan Williams <dan.j.williams@intel.com>
Patch		: http://patchwork.kernel.org/patch/51808/

[Bug #14376] Kernel NULL pointer dereference/ kvm subsystem

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14376
Subject		: Kernel NULL pointer dereference/ kvm subsystem
Submitter	: Don Dupuis <dondster@gmail.com>
Date		: 2009-10-06 14:38 (6 days old)
References	: http://marc.info/?l=linux-kernel&m=125484025021737&w=4

[Bug #14376] Kernel NULL pointer dereference/ kvm subsystem

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14376
Subject		: Kernel NULL pointer dereference/ kvm subsystem
Submitter	: Don Dupuis <dondster-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-06 14:38 (6 days old)
References	: http://marc.info/?l=linux-kernel&m=125484025021737&w=4

[Bug #14375] Intel(R) I/OAT DMA Engine init failed

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14375
Subject		: Intel(R) I/OAT DMA Engine init failed
Submitter	: Alexander Beregalov <a.beregalov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-02 9:46 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447680016160&w=4
Handled-By	: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/51808/

[Bug #14378] Problems with net/core/skbuff.c

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
Date		: 2009-10-08 14:51 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4

[Bug #14380] Video tearing/glitching with T400 laptops

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14380
Subject		: Video tearing/glitching with T400 laptops
Submitter	: Theodore Ts'o <tytso@mit.edu>
Date		: 2009-10-02 22:40 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125452324520623&w=4
Handled-By	: Jesse Barnes <jbarnes@virtuousgeek.org>

[Bug #14382] Transmit failure in et131x.

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14382
Subject		: Transmit failure in et131x.
Submitter	: Nick Bowler <nbowler@elliptictech.com>
Date		: 2009-10-08 14:08 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501117713744&w=4
Handled-By	: Alan Cox <alan@lxorguk.ukuu.org.uk>
Patch		: http://patchwork.kernel.org/patch/52698/

[Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock@gmail.com>
Date		: 2009-10-08 21:46 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4

[Bug #14381] iwlagn lost connection after s2ram (with warnings)

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14381
Subject		: iwlagn lost connection after s2ram (with warnings)
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-07 14:20 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125492569119947&w=4

[Bug #14382] Transmit failure in et131x.

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14382
Subject		: Transmit failure in et131x.
Submitter	: Nick Bowler <nbowler-7BP4RkwGw0uXmMXjJBpWqg@public.gmane.org>
Date		: 2009-10-08 14:08 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501117713744&w=4
Handled-By	: Alan Cox <alan-qBU/x9rampVanCEyBjwyrvXRex20P6io@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/52698/

[Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-08 21:46 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4

[Bug #14380] Video tearing/glitching with T400 laptops

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14380
Subject		: Video tearing/glitching with T400 laptops
Submitter	: Theodore Ts'o <tytso-3s7WtUTddSA@public.gmane.org>
Date		: 2009-10-02 22:40 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125452324520623&w=4
Handled-By	: Jesse Barnes <jbarnes-Y1mF5jBUw70BENJcbMCuUQ@public.gmane.org>

[Bug #14383] hackbench regression with kernel 2.6.32-rc1

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14383
Subject		: hackbench regression with kernel 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:19 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29cd8bae396583a2ee9a3340db8c5102acf9f6fd
References	: http://marc.info/?l=linux-kernel&m=125508007510274&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>

[Bug #14384] tbench regression with 2.6.32-rc1

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14384
Subject		: tbench regression with 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:51 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59abf02644c45f1591e1374ee7bb45dc757fcb88
References	: http://marc.info/?l=linux-kernel&m=125508216713138&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>

[Bug #14386] GPF in snd_hda_intel

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14386
Subject		: GPF in snd_hda_intel
Submitter	: Luca Tettamanti <kronos.it@gmail.com>
Date		: 2009-10-10 13:01 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125517999408019&w=4

[Bug #14386] GPF in snd_hda_intel

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14386
Subject		: GPF in snd_hda_intel
Submitter	: Luca Tettamanti <kronos.it-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-10 13:01 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125517999408019&w=4

[Bug #14383] hackbench regression with kernel 2.6.32-rc1

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14383
Subject		: hackbench regression with kernel 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Date		: 2009-10-09 9:19 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29cd8bae396583a2ee9a3340db8c5102acf9f6fd
References	: http://marc.info/?l=linux-kernel&m=125508007510274&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra-/NLkJaSkS4VmR6Xm/wNWPw@public.gmane.org>

[Bug #14387] deadlock with fallocate

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14387
Subject		: deadlock with fallocate
Submitter	: Thomas Neumann <tneumann@users.sourceforge.net>
Date		: 2009-10-07 3:00 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125488495526471&w=4
Handled-By	: Christoph Hellwig <hch@lst.de>

[Bug #14387] deadlock with fallocate

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14387
Subject		: deadlock with fallocate
Submitter	: Thomas Neumann <tneumann-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f@public.gmane.org>
Date		: 2009-10-07 3:00 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125488495526471&w=4
Handled-By	: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[Bug #14389] Build system issue

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
Subject		: Build system issue
Submitter	: Peter Zijlstra <peterz@infradead.org>
Date		: 2009-10-09 8:58 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4

[Bug #14392] Touchpad "paste" stops working after suspend to RAM

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14392
Subject		: Touchpad "paste" stops working after suspend to RAM
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-11 16:21 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125527987316493&w=4
Handled-By	: Dmitry Torokhov <dmitry.torokhov@gmail.com>

[Bug #14390] "bind" a device to a driver doesn't not work anymore

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14390
Subject		: "bind" a device to a driver doesn't not work anymore
Submitter	: Éric Piel <Eric.Piel@tremplin-utc.net>
Date		: 2009-10-11 0:04 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125521979921241&w=4

[Bug #14392] Touchpad "paste" stops working after suspend to RAM

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14392
Subject		: Touchpad "paste" stops working after suspend to RAM
Submitter	: Carlos R. Mafra <crmafra2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-11 16:21 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125527987316493&w=4
Handled-By	: Dmitry Torokhov <dmitry.torokhov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

[Bug #14390] "bind" a device to a driver doesn't not work anymore

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14390
Subject		: "bind" a device to a driver doesn't not work anymore
Submitter	: Éric Piel <Eric.Piel-VkQ1JFuSMpfAbQlEx87xDw@public.gmane.org>
Date		: 2009-10-11 0:04 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125521979921241&w=4

[Bug #14389] Build system issue

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
Subject		: Build system issue
Submitter	: Peter Zijlstra <peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Date		: 2009-10-09 8:58 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4

Re: [Bug #14382] Transmit failure in et131x.

[Alan Cox]

On Mon, 12 Oct 2009 00:22:05 +0200 (CEST)
"Rafael J. Wysocki" <rjw@sisk.pl> wrote:

> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).

Patch sent for testing, nothing heard back. I think I've fixed it - I
certainly found a bug 8)

Re: [Bug #14382] Transmit failure in et131x.

[Alan Cox]

On Mon, 12 Oct 2009 00:22:05 +0200 (CEST)
"Rafael J. Wysocki" <rjw-KKrjLPT3xs0@public.gmane.org> wrote:

> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).

Patch sent for testing, nothing heard back. I think I've fixed it - I
certainly found a bug 8)

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[Linus Torvalds]

On Mon, 12 Oct 2009, Rafael J. Wysocki wrote:
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14374
> Subject		: MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea
> Submitter	: Nick Piggin <npiggin@suse.de>
> Date		: 2009-10-02 7:34 (10 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=db8be50c4307dac2b37305fc59c8dc0f978d09ea
> References	: http://marc.info/?l=linux-kernel&m=125446885705223&w=4

I think we're going to have to just revert that commit. It's wrong.

It would be nice to do USB handoff earlier, but

 - the correct fix is to make the damn IOMMU code to do it's thing later

 - the USB handoff cannot be done with just PCI config cycles, and will 
   need to do ioremap's etc, which in turn will need the resources to be 
   allocated, which in turn is not possible at FIXUP_HEADER time.

so however much we migth _wish_ to do what that commit does, the commit is 
obvious crap in retrospect.

The IOMMU should not be reprogrammed until _after_ we have handled and 
re-initialized all the devices, so I really think the problem is the IOMMU 
code, not the USB code.

(Of course, the underlying problem is that USB legacy mode handling by 
BIOS with SMM is some seriously insane crap, no question about that. At 
the same time, the IOMMU code is just clearly wrong in removing the legacy 
mappings before we have initialized all devices, so the revert is simply 
the right thing to do)

		Linus

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[Linus Torvalds]

On Mon, 12 Oct 2009, Rafael J. Wysocki wrote:
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
> Subject		: ext4 corruptions
> Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
> Date		: 2009-10-09 19:20 (3 days old)
> References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
> Subject		: Bad corruption with 2.6.32-rc1 and upwards
> Submitter	: Holger Freyther <zecke@selfish.org>
> Date		: 2009-10-09 15:42 (3 days old)

These two look like duplicates to me. No?

			Linus

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[Theodore Tso]

On Sun, Oct 11, 2009 at 04:11:20PM -0700, Linus Torvalds wrote:
> 
> On Mon, 12 Oct 2009, Rafael J. Wysocki wrote:
> >
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
> > Subject		: ext4 corruptions
> > Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
> > Date		: 2009-10-09 19:20 (3 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
> > Subject		: Bad corruption with 2.6.32-rc1 and upwards
> > Submitter	: Holger Freyther <zecke@selfish.org>
> > Date		: 2009-10-09 15:42 (3 days old)
> 
> These two look like duplicates to me. No?

I'm not sure.  I'm not able to replicate either failure.  According to
Holger, his failure happens after a clean shutdown.  Alexey's happens
after a crash/forced power down.  So they could possibly be two
separate failures --- although obviously I hope not!

However, Alexey has been posting his failure information in bug
#14354, so we might as well close #14370.  What I really need is hints
about how to replicate the failure.

						- Ted

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin Mattock]

On Sun, Oct 11, 2009 at 3:22 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject         : ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter       : Justin Mattock <justinmattock@gmail.com>
> Date            : 2009-10-08 21:46 (4 days old)
> References      : http://marc.info/?l=linux-kernel&m=125504031328941&w=4
>
>
>

pulled rc4 today and the warning is still present.
here is the results of git bisect:

d9adc2e031bd22d5d9607a53a8d3b30e0b675f39 is the first bad commit
commit d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
Author: Lin Ming <ming.m.lin@intel.com>
Date:   Mon Jul 27 11:31:10 2009 +0800

    ACPICA: reformat predefined method table, no functional change

    Reformatted the methods that return package objects.

    Signed-off-by: Lin Ming <ming.m.lin@intel.com>
    Signed-off-by: Len Brown <len.brown@intel.com>

:040000 040000 17c30a6e75ce2fafc677b32bdcf3fbecae8458ab
f43e38a2a4034bcc73ef14943e61187f11b08731 M      drivers

I will go and attach the git log to the bug report.

-- 
Justin P. Mattock

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin Mattock]

On Sun, Oct 11, 2009 at 3:22 PM, Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject         : ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter       : Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date            : 2009-10-08 21:46 (4 days old)
> References      : http://marc.info/?l=linux-kernel&m=125504031328941&w=4
>
>
>

pulled rc4 today and the warning is still present.
here is the results of git bisect:

d9adc2e031bd22d5d9607a53a8d3b30e0b675f39 is the first bad commit
commit d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
Author: Lin Ming <ming.m.lin-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Date:   Mon Jul 27 11:31:10 2009 +0800

    ACPICA: reformat predefined method table, no functional change

    Reformatted the methods that return package objects.

    Signed-off-by: Lin Ming <ming.m.lin-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
    Signed-off-by: Len Brown <len.brown-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

:040000 040000 17c30a6e75ce2fafc677b32bdcf3fbecae8458ab
f43e38a2a4034bcc73ef14943e61187f11b08731 M      drivers

I will go and attach the git log to the bug report.

-- 
Justin P. Mattock

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Lin Ming]

On Mon, 2009-10-12 at 11:14 +0800, Justin Mattock wrote:
> On Sun, Oct 11, 2009 at 3:22 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14379
> > Subject         : ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> > Submitter       : Justin Mattock <justinmattock@gmail.com>
> > Date            : 2009-10-08 21:46 (4 days old)
> > References      : http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> >
> >
> >
> 
> pulled rc4 today and the warning is still present.
> here is the results of git bisect:

Patch at http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1 fixes the
warning.

Actually, the _BIF warning was fixed at commit
e899b6485c332aa2d7510739507ab5e5d7b28e59.
See http://bugzilla.kernel.org/show_bug.cgi?id=11822#c14 

I made a mistake at the cleanup of predefined method table so the
warning shows up again.

Lin Ming

> 
> d9adc2e031bd22d5d9607a53a8d3b30e0b675f39 is the first bad commit
> commit d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> Author: Lin Ming <ming.m.lin@intel.com>
> Date:   Mon Jul 27 11:31:10 2009 +0800
> 
>     ACPICA: reformat predefined method table, no functional change
> 
>     Reformatted the methods that return package objects.
> 
>     Signed-off-by: Lin Ming <ming.m.lin@intel.com>
>     Signed-off-by: Len Brown <len.brown@intel.com>
> 
> :040000 040000 17c30a6e75ce2fafc677b32bdcf3fbecae8458ab
> f43e38a2a4034bcc73ef14943e61187f11b08731 M      drivers
> 
> I will go and attach the git log to the bug report.
> 

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Lin Ming]

On Mon, 2009-10-12 at 11:14 +0800, Justin Mattock wrote:
> On Sun, Oct 11, 2009 at 3:22 PM, Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14379
> > Subject         : ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> > Submitter       : Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Date            : 2009-10-08 21:46 (4 days old)
> > References      : http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> >
> >
> >
> 
> pulled rc4 today and the warning is still present.
> here is the results of git bisect:

Patch at http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1 fixes the
warning.

Actually, the _BIF warning was fixed at commit
e899b6485c332aa2d7510739507ab5e5d7b28e59.
See http://bugzilla.kernel.org/show_bug.cgi?id=11822#c14 

I made a mistake at the cleanup of predefined method table so the
warning shows up again.

Lin Ming

> 
> d9adc2e031bd22d5d9607a53a8d3b30e0b675f39 is the first bad commit
> commit d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> Author: Lin Ming <ming.m.lin-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Date:   Mon Jul 27 11:31:10 2009 +0800
> 
>     ACPICA: reformat predefined method table, no functional change
> 
>     Reformatted the methods that return package objects.
> 
>     Signed-off-by: Lin Ming <ming.m.lin-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>     Signed-off-by: Len Brown <len.brown-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> 
> :040000 040000 17c30a6e75ce2fafc677b32bdcf3fbecae8458ab
> f43e38a2a4034bcc73ef14943e61187f11b08731 M      drivers
> 
> I will go and attach the git log to the bug report.
> 

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Lin Ming wrote:
> On Mon, 2009-10-12 at 11:14 +0800, Justin Mattock wrote:
>    
>> On Sun, Oct 11, 2009 at 3:22 PM, Rafael J. Wysocki<rjw@sisk.pl>  wrote:
>>      
>>> This message has been generated automatically as a part of a report
>>> of recent regressions.
>>>
>>> The following bug entry is on the current list of known regressions
>>> from 2.6.31.  Please verify if it still should be listed and let me know
>>> (either way).
>>>
>>>
>>> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14379
>>> Subject         : ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
>>> Submitter       : Justin Mattock<justinmattock@gmail.com>
>>> Date            : 2009-10-08 21:46 (4 days old)
>>> References      : http://marc.info/?l=linux-kernel&m=125504031328941&w=4
>>>
>>>
>>>
>>>        
>> pulled rc4 today and the warning is still present.
>> here is the results of git bisect:
>>      
>
> Patch at http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1 fixes the
> warning.
>
> Actually, the _BIF warning was fixed at commit
> e899b6485c332aa2d7510739507ab5e5d7b28e59.
> See http://bugzilla.kernel.org/show_bug.cgi?id=11822#c14
>
> I made a mistake at the cleanup of predefined method table so the
> warning shows up again.
>
> Lin Ming
>
>    
Cool, I can give that a try.
then I guess just leave this open(bugreport) until this patch
http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1
gets pushed into the main tree then close the bugreport.

Justin P. Mattock

Re: [Bug #14370] ext4 corruptions

[Alexey Fisher]

Am Montag, den 12.10.2009, 00:22 +0200 schrieb Rafael J. Wysocki:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
> Subject		: ext4 corruptions
> Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
> Date		: 2009-10-09 19:20 (3 days old)
> References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kernel-testers" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

This can be removed, it is duplicate of this bug:
http://bugzilla.kernel.org/show_bug.cgi?id=14354

Re: [Bug #14279] Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop

[Jan Beulich]

The issue is still present, albeit a patch (available in the bugzilla entry) had
been posted on Oct 6th. Jan

>>> "Rafael J. Wysocki" <rjw@sisk.pl> 12.10.09 00:22 >>>
This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14279 
Subject		: Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:14 (12 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5f68563996e812f9ca35b3939ad2a42e5d254d66 

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[David Woodhouse]

On Mon, 2009-10-12 at 00:07 +0100, Linus Torvalds wrote:
> The IOMMU should not be reprogrammed until _after_ we have handled and 
> re-initialized all the devices, so I really think the problem is the IOMMU 
> code, not the USB code.

The IOMMU init is happening in the right place, I think. It has to be
before device_initcall(), because it has to be set up before we actually
start initialising devices -- we have to have the IOMMU in place before
any real drivers try to set up DMA mappings.

It's currently fs_initcall(), with an explicit comment saying that it
has to be after PCI initialisation. The only way it could be later is if
we move it to rootfs_initcall().

> (Of course, the underlying problem is that USB legacy mode handling by 
> BIOS with SMM is some seriously insane crap, no question about that. At 
> the same time, the IOMMU code is just clearly wrong in removing the legacy 
> mappings before we have initialized all devices, so the revert is simply 
> the right thing to do)

Well, according to the design, the IOMMU code is doing the right thing¹.

The theory is that the BIOS _tells_ us about the legacy mappings it
needs by putting them in an ACPI table (the RMRR table). Of course, the
reality is that BIOSes are universally shit and often fail to do this.

If the BIOSes were only _slightly_ shit, there'd be a few harmless DMA
faults and the legacy USB would stop of its own accord. But no, a bunch
of them also manage to lock up in SMM mode when their DMA goes AWOL.

The only viable solution (short of an open source BIOS written by sober
people) was to quiesce the USB controllers before enabling the IOMMU.

The final PCI quirks are currently run from pci_init() which is a
device_initcall(), which is too late -- in fact, it could actually be
_after_ some of the real device drivers have taken control of the same
hardware.

So the better fix is probably just to fix that problem -- move the final
PCI quirks so they happen a little earlier. If we move them to
fs_initcall_sync() and then move the IOMMU init to rootfs_initcall(),
then everything ought to work, I think...

(Let's rename 'pci_init()' and move it to quirks.c while we're at it,
since it only actually does that one thing. If/when I submit this
properly, I'll do that in a separate commit.)

diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 64b838e..e0d9199 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -311,7 +311,7 @@ void pci_iommu_shutdown(void)
 	amd_iommu_shutdown();
 }
 /* Must execute after PCI subsystem */
-fs_initcall(pci_iommu_init);
+rootfs_initcall(pci_iommu_init);
 
 #ifdef CONFIG_PCI
 /* Many VIA bridges seem to corrupt data for DAC. Disable it here */
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 6edecff..2b575cf 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2719,17 +2719,6 @@ int __attribute__ ((weak)) pci_ext_cfg_avail(struct pci_dev *dev)
 	return 1;
 }
 
-static int __devinit pci_init(void)
-{
-	struct pci_dev *dev = NULL;
-
-	while ((dev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, dev)) != NULL) {
-		pci_fixup_device(pci_fixup_final, dev);
-	}
-
-	return 0;
-}
-
 static int __init pci_setup(char *str)
 {
 	while (str) {
@@ -2767,8 +2756,6 @@ static int __init pci_setup(char *str)
 }
 early_param("pci", pci_setup);
 
-device_initcall(pci_init);
-
 EXPORT_SYMBOL(pci_reenable_device);
 EXPORT_SYMBOL(pci_enable_device_io);
 EXPORT_SYMBOL(pci_enable_device_mem);
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 6099fac..d99c9b4 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -2572,6 +2572,19 @@ void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev)
 	}
 	pci_do_fixups(dev, start, end);
 }
+
+static int __devinit pci_apply_final_quirks(void)
+{
+	struct pci_dev *dev = NULL;
+
+	while ((dev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, dev)) != NULL) {
+		pci_fixup_device(pci_fixup_final, dev);
+	}
+
+	return 0;
+}
+
+fs_initcall_sync(pci_apply_final_quirks);
 #else
 void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev) {}
 #endif


-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

¹ Yes, the design is completely broken. Anyone with any real-world
  experience would have known that the BIOS would never get this right.
  Personally, I find that it's useful to assume malice on the part of
  BIOS authors. I'm not suggesting that it's actually _true_, but it
  does tend to give a useful predictor of _just_ how badly they're going
  to screw things up.

Re: [Bug #14378] Problems with net/core/skbuff.c

[David Miller]

From: "Rafael J. Wysocki" <rjw@sisk.pl>
Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)

> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
> Subject		: Problems with net/core/skbuff.c
> Submitter	: Massimo Cetra <mcetra@navynet.it>
> Date		: 2009-10-08 14:51 (4 days old)
> References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4

I don't know what to do about this one.

The user indicates that they have the vserver patches applied,
so maybe there is some interaction with that stuff.

[PATCH 1/4] Rename pci_init() to pci_apply_final_quirks(), move it to quirks.c

[David Woodhouse]

This function may have done more in the past, but all it does now is
apply the PCI_FIXUP_FINAL quirks. So name it sensibly and put it where
it belongs.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
---
 drivers/pci/pci.c    |   13 -------------
 drivers/pci/quirks.c |   13 +++++++++++++
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 6edecff..2b575cf 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2719,17 +2719,6 @@ int __attribute__ ((weak)) pci_ext_cfg_avail(struct pci_dev *dev)
 	return 1;
 }
 
-static int __devinit pci_init(void)
-{
-	struct pci_dev *dev = NULL;
-
-	while ((dev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, dev)) != NULL) {
-		pci_fixup_device(pci_fixup_final, dev);
-	}
-
-	return 0;
-}
-
 static int __init pci_setup(char *str)
 {
 	while (str) {
@@ -2767,8 +2756,6 @@ static int __init pci_setup(char *str)
 }
 early_param("pci", pci_setup);
 
-device_initcall(pci_init);
-
 EXPORT_SYMBOL(pci_reenable_device);
 EXPORT_SYMBOL(pci_enable_device_io);
 EXPORT_SYMBOL(pci_enable_device_mem);
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 6099fac..0142275 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -2572,6 +2572,19 @@ void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev)
 	}
 	pci_do_fixups(dev, start, end);
 }
+
+static int __devinit pci_apply_final_quirks(void)
+{
+	struct pci_dev *dev = NULL;
+
+	while ((dev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, dev)) != NULL) {
+		pci_fixup_device(pci_fixup_final, dev);
+	}
+
+	return 0;
+}
+
+device_initcall(pci_apply_final_quirks);
 #else
 void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev) {}
 #endif
-- 
1.6.5.rc2

[PATCH 2/4] Mark pci_apply_final_quirks() __init rather than __devinit

[David Woodhouse]

It doesn't get invoked on hotplug; it can be thrown away after init.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
---
 drivers/pci/quirks.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 0142275..77bf620 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -2573,7 +2573,7 @@ void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev)
 	pci_do_fixups(dev, start, end);
 }
 
-static int __devinit pci_apply_final_quirks(void)
+static int __init pci_apply_final_quirks(void)
 {
 	struct pci_dev *dev = NULL;
 
-- 
1.6.5.rc2

[PATCH 3/4] Run pci_apply_final_quirks() sooner.

[David Woodhouse]

Having this as a device_initcall() means that some real device drivers
can actually initialise _before_ the quirks are run, which is wrong.

We want it to run _before_ device_initcall(), but _after_ fs_initcall(),
since some arch-specific PCI initialisation like pcibios_assign_resources()
is done at fs_initcall().

We could use rootfs_initcall() but I actually want to use that for the
IOMMU initialisation, which has to come after the quirks, but still
before the real devices. So use fs_initcall_sync() instead -- since this
is entirely synchronous, it doesn't hurt that it'll escape the
synchronisation.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
---
 drivers/pci/quirks.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 77bf620..4e1b1b8 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -2584,7 +2584,7 @@ static int __init pci_apply_final_quirks(void)
 	return 0;
 }
 
-device_initcall(pci_apply_final_quirks);
+fs_initcall_sync(pci_apply_final_quirks);
 #else
 void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev) {}
 #endif
-- 
1.6.5.rc2

[PATCH 4/4] x86: Move pci_iommu_init to rootfs_initcall()

[David Woodhouse]

We want this to happen after the PCI quirks, which are now running at
the very end of the fs_initcalls.

This works around the BIOS problems which were originally addressed by
commit db8be50c4307dac2b37305fc59c8dc0f978d09ea ('USB: Work around BIOS
bugs by quiescing USB controllers earlier'), which was reverted in
commit d93a8f829fe1d2f3002f2c6ddb553d12db420412.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
---
 arch/x86/kernel/pci-dma.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index d20009b..b2a71dc 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -311,7 +311,7 @@ void pci_iommu_shutdown(void)
 	amd_iommu_shutdown();
 }
 /* Must execute after PCI subsystem */
-fs_initcall(pci_iommu_init);
+rootfs_initcall(pci_iommu_init);
 
 #ifdef CONFIG_PCI
 /* Many VIA bridges seem to corrupt data for DAC. Disable it here */
-- 
1.6.5.rc2

Re: [Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[Michael Buesch]

On Monday 12 October 2009 00:22:00 Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
> Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
> Submitter	: Christian Casteyde <casteyde.christian@free.fr>
> Date		: 2009-09-30 18:12 (12 days old)
> Handled-By	: Michael Buesch <mb@bu3sch.de>

I think John or Johannes is handling it, as my patch was rejected.

> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220

-- 
Greetings, Michael.

Re: [Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[Michael Buesch]

On Monday 12 October 2009 00:22:00 Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
> Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
> Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
> Date		: 2009-09-30 18:12 (12 days old)
> Handled-By	: Michael Buesch <mb-fseUSCV1ubazQB+pC5nmwQ@public.gmane.org>

I think John or Johannes is handling it, as my patch was rejected.

> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220

-- 
Greetings, Michael.

Re: [Bug #14374] MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea

[Mikael Pettersson]

Rafael J. Wysocki wrote:
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=3D14374
> Subject		: MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea
> Submitter	: Nick Piggin <npiggin@suse.de>
> Date		: 2009-10-02 7:34 (10 days old)
> First-Bad-Commit: http://git.kernel.org/?p=3Dlinux/kernel/git/torvalds/linu=
> x-2.6.git;a=3Dcommit;h=3Ddb8be50c4307dac2b37305fc59c8dc0f978d09ea
> References	: http://marc.info/?l=3Dlinux-kernel&m=3D125446885705223&w=3D4

Fixed for 2.6.32-rc5 by d93a8f829fe1d2f3002f2c6ddb553d12db420412
'Revert "USB: Work around BIOS bugs by quiescing USB controllers earlier"'.

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[David Woodhouse]

On Mon, 2009-10-12 at 11:19 +0100, David Woodhouse wrote:
> So the better fix is probably just to fix that problem -- move the final
> PCI quirks so they happen a little earlier. If we move them to
> fs_initcall_sync() and then move the IOMMU init to rootfs_initcall(),
> then everything ought to work, I think...

I've sent that series of patches for review. Assuming you like it,
please pull those patches and a couple of other fixes -- one cosmetic
and one serious crash fix (another BIOS workaround, in fact), from

git://git.infradead.org/~dwmw2/iommu-2.6.32.git

David Woodhouse (5):
      intel-iommu: Yet another BIOS workaround: Isoch DMAR unit with no TLB space
      Rename pci_init() to pci_apply_final_quirks(), move it to quirks.c
      Mark pci_apply_final_quirks() __init rather than __devinit
      Run pci_apply_final_quirks() sooner.
      x86: Move pci_iommu_init to rootfs_initcall()

Roland Dreier (2):
      intel-iommu: Make "Unknown DMAR structure" message more informative
      intel-iommu: Decode (and ignore) RHSA entries

 arch/x86/kernel/pci-dma.c |    2 +-
 drivers/pci/dmar.c        |   13 ++++++-
 drivers/pci/intel-iommu.c |   82 ++++++++++++++++++++++++++++++++++++++++++---
 drivers/pci/pci.c         |   13 -------
 drivers/pci/quirks.c      |   13 +++++++
 5 files changed, 103 insertions(+), 20 deletions(-)


-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo "CtRiX" Cetra]

David Miller ha scritto:
> From: "Rafael J. Wysocki" <rjw@sisk.pl>
> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
> 
>> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
>> Subject		: Problems with net/core/skbuff.c
>> Submitter	: Massimo Cetra <mcetra@navynet.it>
>> Date		: 2009-10-08 14:51 (4 days old)
>> References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4
> 
> I don't know what to do about this one.
> 
> The user indicates that they have the vserver patches applied,
> so maybe there is some interaction with that stuff.

Actually i don't think that it's Vserver related.

I'm not a kernel hacker but the vserver patch doesn't seems to have much 
relation with networking (for who don't know vserver, it's a sort of 
advanced chroot).

I'm using OCFS2 and drbd. Both of those 2 things make heavy use of 
networking to share data between two hosts.

The backtrace seems to begin in the "world" of OFCS2.
I'm CC'ing ocfs mailing list where someome may have some clues.

Thanks,
Max

[Ocfs2-devel] [Bug #14378] Problems with net/core/skbuff.c

[Massimo CtRiX Cetra]

David Miller ha scritto:
> From: "Rafael J. Wysocki" <rjw@sisk.pl>
> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
> 
>> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
>> Subject		: Problems with net/core/skbuff.c
>> Submitter	: Massimo Cetra <mcetra@navynet.it>
>> Date		: 2009-10-08 14:51 (4 days old)
>> References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4
> 
> I don't know what to do about this one.
> 
> The user indicates that they have the vserver patches applied,
> so maybe there is some interaction with that stuff.

Actually i don't think that it's Vserver related.

I'm not a kernel hacker but the vserver patch doesn't seems to have much 
relation with networking (for who don't know vserver, it's a sort of 
advanced chroot).

I'm using OCFS2 and drbd. Both of those 2 things make heavy use of 
networking to share data between two hosts.

The backtrace seems to begin in the "world" of OFCS2.
I'm CC'ing ocfs mailing list where someome may have some clues.

Thanks,
Max

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[Linus Torvalds]

On Mon, 12 Oct 2009, David Woodhouse wrote:
> 
> Well, according to the design, the IOMMU code is doing the right thing¹.
> 
> The theory is that the BIOS _tells_

There is no "theory". There is only crap BIOSes. Stop living in a dream 
world, and stop making arguments that are only relevant in that dream 
world.

> The only viable solution (short of an open source BIOS written by sober
> people)

Again, you're living in that dream world. Wake up, sheeple.

BIOS writers write crap, because it's a crap job. It's that simple. Yes, 
they're probably drunk or drugged up, but they need it to deal with the 
hand they have been dealt.

There are absolutely _zero_ incentives to write good firmware for any 
particular device (and nothing else matters), because the projects are all 
(a) largely "throw-over-the-fence-and-forget-about-it" for any particular 
machine (ie any fixes are mostly relevant for the _next_ generation of 
machines), and (b) they have no actual user incentives or feedback, since 
nobody really "runs" the firmware anyway - it's supposed to be invisible 
by design and just sets things up.

So outside of the testing that it gets (_before_ it ever hits any users 
hands) there is never _ever_ going to be any QA.  Once it is in user 
hands, it is what it is. Almost nobody upgrades their firmware in 
practice. Yeah, geeks may do. Normal people? Not so much.

And that would _not_ change even with an open source BIOS. Why? Because 
the _code_ generally isn't the problem. The problem tends to be all the 
localization for any particular machine.

Even if the code was open source and perfect, we'd still have trouble with 
firmware - because 99% of it is driven by tables that by design have to be 
changed for each machine (yes, yes, a BIOS is easily a megabyte of 
compressed code too, but a lot of it is the nice GUI for setup etc - the 
actual code that runs on _bootup_ is rather small, and is all about those 
tables).

We might have a few _fewer_ problems if the code was perfect, and maybe 
we'd have missed this particular issue (but I doubt it, it's still a 
localized table). But it wouldn't really change any of the fundamentals in 
the issue.

BTW, in this case, it's not even necessarily the firmware people who are 
to blame. It's a combination of (a) USB is crazy polled DMA and (b) IOMMU 
is a whole new thing and (c) ACPI is too f*cking complex so nobody will 
_ever_ get it right anyway, even if the firmware people didn't have 
everything against them.

So your arguments about RMRR tables and "should do" are POINTLESS. Just 
give it up.

The sane thing to do is to have a legacy IOMMU mapping until all devices 
are initialized, so that things work _despite_ the inevitable BIOS 
crapola. End of story.

So stop blaming the BIOS. We _know_ firmware is crap - there is no point 
in blaming it. The response to "firmware bug" should be "oh, of course - 
and our code was too fragile, since it didn't take that into account".

And stop saying these problems would magically go away with open-source 
firmware. That just shows that you don't understand the realities of the 
situation. Even an open-source bios would end up having buggy tables, and 
even with an opensource bios, users generally wouldn't upgrade it.

"If wishes were horses, beggars would ride"

> was to quiesce the USB controllers before enabling the IOMMU.

The other solution would be to just _enable_ (and do all the setup) of the 
IOMMU early. And then just leave a legacy mapping for the IOMMU, and then 
_after_all_devices_are_set_up_ can you then remove the legacy mapping.

I don't much care. Just as long as the DMA works for the whole bus setup.

> The final PCI quirks are currently run from pci_init() which is a
> device_initcall(), which is too late -- in fact, it could actually be
> _after_ some of the real device drivers have taken control of the same
> hardware.

Now, this I can certainly agree 100% with. We can and should move the 
final quirks up a bit. And yes, I absolutely think we should at least 
_guarantee_ that those quirks are run before any other device_initcalls, 
regardless of any IOMMU issues (ie now it looks like it depends on link 
ordering whether a built-in driver might run before or after pci_init()).

So I agree that we may well be able to fix this by changing the ordering. 
What I disagree with is your continual "I wish things were different". 
I've seen you make that "open source firmware" argument before. It's 
pointless. Stop doing it.

But your patch looks fine.

That said, I think your choice of initcall is odd, even if it is 
understandable. Right now we have, at least on x86:

	fs_initcall(pcibios_assign_resources);

and I assume you picked fs_initcall_sync() so that it happens after that 
one. No?

Which makes sense, but at the same time, it all looks just random. And 
different architectures actually do it in different places (some seem to 
do it inside pcibios_init() at subsys_initcall time). So I'm not even sure 
fs_initcall_sync() will do it for other architectures, although it looks 
like most others do their final PCI setup _earlier_ rather than later.

I'm wondering if we should not get _rid_ of that whole pci_init() (you 
renamed it, and I agree - it's clearly not really "pci_init()"), and move 
it to just be the last thing that pci_subsys_init() does, or perhaps all 
the way into pcibios_resource_survey().

But I guess there could be random ACPI initcalls etc involved too, and 
subtle ordering constraints with _those_. And we have way too many 
arch-specific details here. So your patch may be the simplest one, but I 
wish we could also make some of this be less of a jungle of different 
initcalls.

Basically, it seems silly to have this kind of subtlety for just the final 
quirk, when the _other_ quirks are all handled by generic code in very 
well-defined places.

For example, one of the effects of this mess is that as far as I can tell, 
PCI hotplug (or cardbus, which is a special case of PCI hotplug) will 
never run the pci_fixup_late fixups at all, even though it _will_ run the 
pci_fixup_header/early/enable ones (because those are done by generic 
code: pci_setup_device(), pci_device_add() and pci_enable_device()).

			Linus

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[David Woodhouse]

On Mon, 2009-10-12 at 16:24 +0100, Linus Torvalds wrote:
> 
> On Mon, 12 Oct 2009, David Woodhouse wrote:
> > 
> > Well, according to the design, the IOMMU code is doing the right thing¹.
> > 
> > The theory is that the BIOS _tells_
> 
> There is no "theory". There is only crap BIOSes. Stop living in a dream 
> world, and stop making arguments that are only relevant in that dream 
> world.

You're preaching to the choir, Linus.

> > The only viable solution (short of an open source BIOS written by sober
> > people)
> 
> Again, you're living in that dream world. Wake up, sheeple.

 ... 

> So your arguments about RMRR tables and "should do" are POINTLESS. Just 
> give it up.

I didn't intend that as an argument. That was just a statement of how
the architecture specifies it's _supposed_ to work. I _said_ it was
nonsense, in my own footnote.

> The sane thing to do is to have a legacy IOMMU mapping until all devices 
> are initialized, so that things work _despite_ the inevitable BIOS 
> crapola. End of story.

Damn right.

That's precisely what the original "kill USB DMA early" patch was
intending to do -- and that's why I've been telling the people
responsible for the RMRR design that it was a stupid idea in the first
place.

The problem with the original patch was just that I didn't realise that
some architectures didn't have ioremap() working at the time that
PCI_FIXUP_HEADER() runs.

> So stop blaming the BIOS. We _know_ firmware is crap - there is no point 
> in blaming it. The response to "firmware bug" should be "oh, of course - 
> and our code was too fragile, since it didn't take that into account".

Yes. That's why our code has grown a metric shitload of BIOS
workarounds, and been refactored to be less fragile, since I've taken
over responsibility for it.

You won't stop me hating the BIOS for it though. And although I agree
that it wouldn't be perfect if we had an open source BIOS, it _would_ be
a whole lot better.

> The other solution would be to just _enable_ (and do all the setup) of the 
> IOMMU early. And then just leave a legacy mapping for the IOMMU, and then 
> _after_all_devices_are_set_up_ can you then remove the legacy mapping.

That involves allocating a _shitload_ of page tables for a 1:1 mapping
of all of physical memory. We used to do that to pander to broken
graphics drivers, and the AMD guys asked me to stop because it was
actually quite painful for them to do the same.

> But your patch looks fine.
> 
> That said, I think your choice of initcall is odd, even if it is 
> understandable. Right now we have, at least on x86:
> 
> 	fs_initcall(pcibios_assign_resources);
> 
> and I assume you picked fs_initcall_sync() so that it happens after that 
> one. No?

Right.

> Which makes sense, but at the same time, it all looks just random. 

No more random than the fact that pcibios_assign_resources() is marked
as a FILE SYSTEM initcall in the first place, surely?

We have a very limited method of expressing ordering constraints for
initcalls; we make do with what we have. Some people around here have
been known to argue _very_ vocally against a more comprehensive
framework for ordering them.

Yes, sometimes it looks odd :)

> And 
> different architectures actually do it in different places (some seem to 
> do it inside pcibios_init() at subsys_initcall time). So I'm not even sure 
> fs_initcall_sync() will do it for other architectures, although it looks 
> like most others do their final PCI setup _earlier_ rather than later.

They will do it earlier; never later. It would be completely broken to
do it any later -- fs_initcall_sync() and rootfs_initcall() are new
inventions, so such an architecture would have to be doing it in
device_initcall(). And that would just be totally broken.

> But I guess there could be random ACPI initcalls etc involved too, and 
> subtle ordering constraints with _those_. And we have way too many 
> arch-specific details here. So your patch may be the simplest one, but I 
> wish we could also make some of this be less of a jungle of different 
> initcalls.

Yeah, it's a complete mess of generic and arch-specific initcalls, all
invoked at different times. It would be good to clean that up, perhaps,
but maybe not for 2.6.32.

> Basically, it seems silly to have this kind of subtlety for just the final 
> quirk, when the _other_ quirks are all handled by generic code in very 
> well-defined places.

I did think briefly about adding it to pci_subsys_init(), but moving it
into arch-specific code didn't seem like a great idea. And the
pcibios_assign_resources() initcall happens _after_ that, anyway.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[Alan Cox]

> That involves allocating a _shitload_ of page tables for a 1:1 mapping
> of all of physical memory. We used to do that to pander to broken
> graphics drivers, and the AMD guys asked me to stop because it was
> actually quite painful for them to do the same.

SMM isn't going to be touching high pages. In fact I wouldn't be the
least suprised if simply mapping the low 1MB including EBDA/ISA space and
the ACPI reserved areas worked simply because if it touches any other RAM
in SMM mode its going to be peeing over our memory spaces.

And if I remember rightly the magic SMM remapped memory area is quite
small (64K) and low down too plus not DMAable (because otherwise DMA in
the real OS to the pages SMM aliases over will go kersplat if an SMM
occurs)

Alan

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[Linus Torvalds]

On Mon, 12 Oct 2009, David Woodhouse wrote:
> 
> > The other solution would be to just _enable_ (and do all the setup) of the 
> > IOMMU early. And then just leave a legacy mapping for the IOMMU, and then 
> > _after_all_devices_are_set_up_ can you then remove the legacy mapping.
> 
> That involves allocating a _shitload_ of page tables for a 1:1 mapping
> of all of physical memory.

I don't think that's true.

Nobody cares about "all physical memory". For one thing, we know that 
anything that we consider to be normal memory (ie it's listed in the e820 
tables as RAM) can't be all that interesting, since the BIOS definitely 
released that to us.

That said, as long as the IOMMU is clearly enabled after the quirks have 
run, for this particular case we don't much care.

But I could also imagine something similar happening for some BIOS-enabled 
ethernet device being set up to listen to packets into some BIOS data 
areas (left-overs from whatever bootp thing or other), which doesn't have 
a quirk, and which ends up doing DMA until we actually load the driver.

Of course, we'd hope that the DMA just fails and nothing bad really 
happens (hopefully the driver re-init will clear up any hung device). But 
I can also imagine the hardware simply being really really unhappy, and 
not recovering.

So in many ways it would be safest to leave memory that we don't know 
about and we don't own as DMA'able in the IOMMU.

And no, I don't think it would be a shitload of pages. Quite the reverse. 
It's probably not very many at all.

		Linus

Re: [Bug #14372] Wireless not working after suspend-resume

[Fabio Comolli]

Still present with -rc4.

Error is:

[  690.547660] wlan0: direct probe to AP 00:11:95:09:2c:e0 (try 1)
[  690.550064] wlan0: direct probe responded
[  690.550074] wlan0: authenticate with AP 00:11:95:09:2c:e0 (try 1)
[  690.551629] wlan0: authenticated
[  690.551663] wlan0: associate with AP 00:11:95:09:2c:e0 (try 1)
[  690.553289] wlan0: RX AssocResp from 00:11:95:09:2c:e0 (capab=0x451
status=12 aid=1)
[  690.553298] wlan0: AP denied association (code=12)

rfkill-ing the device off and then on fixes the problem.

Fabio




On Mon, Oct 12, 2009 at 12:22 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14372
> Subject         : Wireless not working after suspend-resume
> Submitter       : Fabio Comolli <fabio.comolli@gmail.com>
> Date            : 2009-10-03 15:36 (9 days old)
> References      : http://lkml.org/lkml/2009/10/3/91
>
>
>

Re: [Bug #14372] Wireless not working after suspend-resume

[Fabio Comolli]

Still present with -rc4.

Error is:

[  690.547660] wlan0: direct probe to AP 00:11:95:09:2c:e0 (try 1)
[  690.550064] wlan0: direct probe responded
[  690.550074] wlan0: authenticate with AP 00:11:95:09:2c:e0 (try 1)
[  690.551629] wlan0: authenticated
[  690.551663] wlan0: associate with AP 00:11:95:09:2c:e0 (try 1)
[  690.553289] wlan0: RX AssocResp from 00:11:95:09:2c:e0 (capab=0x451
status=12 aid=1)
[  690.553298] wlan0: AP denied association (code=12)

rfkill-ing the device off and then on fixes the problem.

Fabio




On Mon, Oct 12, 2009 at 12:22 AM, Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14372
> Subject         : Wireless not working after suspend-resume
> Submitter       : Fabio Comolli <fabio.comolli@gmail.com>
> Date            : 2009-10-03 15:36 (9 days old)
> References      : http://lkml.org/lkml/2009/10/3/91
>
>
>

Re: 2.6.32-rc4: Reported regressions from 2.6.31

[David Woodhouse]

On Mon, 2009-10-12 at 18:35 +0100, Linus Torvalds wrote:
> 
> On Mon, 12 Oct 2009, David Woodhouse wrote:
> > 
> > > The other solution would be to just _enable_ (and do all the setup) of the 
> > > IOMMU early. And then just leave a legacy mapping for the IOMMU, and then 
> > > _after_all_devices_are_set_up_ can you then remove the legacy mapping.
> > 
> > That involves allocating a _shitload_ of page tables for a 1:1 mapping
> > of all of physical memory.
> 
> I don't think that's true.
> 
> Nobody cares about "all physical memory". For one thing, we know that 
> anything that we consider to be normal memory (ie it's listed in the e820 
> tables as RAM) can't be all that interesting, since the BIOS definitely 
> released that to us.

Yeah, that's probably true. I was thinking of the 1:1 mappings we set up
for crappy drivers under Linux, which do need access to all of memory
and don't use the DMA API correctly. This is different, though.

We have talked about dma-mapping all of E820-reserved memory, but that
would allow a rogue device to scribble over arbitrary bits of memory
belonging to the BIOS, which isn't necessarily such a good thing. Even
if we did tear it down as soon as a real driver came along, the TXT
folks would still object -- and rightly so.

As you said, it's sufficient just to enable the IOMMU after the quirks
have run. So the patches I posted earlier should be just fine.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: [Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[Rafael J. Wysocki]

On Monday 12 October 2009, Michael Buesch wrote:
> On Monday 12 October 2009 00:22:00 Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
> > Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
> > Submitter	: Christian Casteyde <casteyde.christian@free.fr>
> > Date		: 2009-09-30 18:12 (12 days old)
> > Handled-By	: Michael Buesch <mb@bu3sch.de>
> 
> I think John or Johannes is handling it, as my patch was rejected.
> 
> > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220

Johannes, what's the status here?

Rafael

Re: [Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[Rafael J. Wysocki]

On Monday 12 October 2009, Michael Buesch wrote:
> On Monday 12 October 2009 00:22:00 Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
> > Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
> > Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
> > Date		: 2009-09-30 18:12 (12 days old)
> > Handled-By	: Michael Buesch <mb-fseUSCV1ubazQB+pC5nmwQ@public.gmane.org>
> 
> I think John or Johannes is handling it, as my patch was rejected.
> 
> > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220

Johannes, what's the status here?

Rafael

Re: [Bug #14279] Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop

[Rafael J. Wysocki]

On Monday 12 October 2009, Jan Beulich wrote:
> The issue is still present, albeit a patch (available in the bugzilla entry) had
> been posted on Oct 6th. Jan

Thanks for the update.  I saw the patch go into the tip tree a few moments ago.

Best,
Rafael

Re: [Bug #14279] Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop

[Rafael J. Wysocki]

On Monday 12 October 2009, Jan Beulich wrote:
> The issue is still present, albeit a patch (available in the bugzilla entry) had
> been posted on Oct 6th. Jan

Thanks for the update.  I saw the patch go into the tip tree a few moments ago.

Best,
Rafael

Re: [Bug #14370] ext4 corruptions

[Rafael J. Wysocki]

On Monday 12 October 2009, Alexey Fisher wrote:
> Am Montag, den 12.10.2009, 00:22 +0200 schrieb Rafael J. Wysocki:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
> > Subject		: ext4 corruptions
> > Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
> > Date		: 2009-10-09 19:20 (3 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe kernel-testers" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> This can be removed, it is duplicate of this bug:
> http://bugzilla.kernel.org/show_bug.cgi?id=14354

Thanks, I closed #14370 as a duplicate of #14354.

Best,
Rafael

Re: [Bug #14370] ext4 corruptions

[Rafael J. Wysocki]

On Monday 12 October 2009, Alexey Fisher wrote:
> Am Montag, den 12.10.2009, 00:22 +0200 schrieb Rafael J. Wysocki:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
> > Subject		: ext4 corruptions
> > Submitter	: Alexey Fisher <bug-track-M18mAb7Tlt0yCq4wW13eYl6hYfS7NtTn@public.gmane.org>
> > Date		: 2009-10-09 19:20 (3 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe kernel-testers" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> This can be removed, it is duplicate of this bug:
> http://bugzilla.kernel.org/show_bug.cgi?id=14354

Thanks, I closed #14370 as a duplicate of #14354.

Best,
Rafael

Re: [Bug #14372] Wireless not working after suspend-resume

[Rafael J. Wysocki]

On Monday 12 October 2009, Fabio Comolli wrote:
> Still present with -rc4.
> 
> Error is:
> 
> [  690.547660] wlan0: direct probe to AP 00:11:95:09:2c:e0 (try 1)
> [  690.550064] wlan0: direct probe responded
> [  690.550074] wlan0: authenticate with AP 00:11:95:09:2c:e0 (try 1)
> [  690.551629] wlan0: authenticated
> [  690.551663] wlan0: associate with AP 00:11:95:09:2c:e0 (try 1)
> [  690.553289] wlan0: RX AssocResp from 00:11:95:09:2c:e0 (capab=0x451
> status=12 aid=1)
> [  690.553298] wlan0: AP denied association (code=12)
> 
> rfkill-ing the device off and then on fixes the problem.

Thanks for the update.

Rafael

Re: [Bug #14374] MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea

[Rafael J. Wysocki]

On Monday 12 October 2009, Mikael Pettersson wrote:
> Rafael J. Wysocki wrote:
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=3D14374
> > Subject		: MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea
> > Submitter	: Nick Piggin <npiggin@suse.de>
> > Date		: 2009-10-02 7:34 (10 days old)
> > First-Bad-Commit: http://git.kernel.org/?p=3Dlinux/kernel/git/torvalds/linu=
> > x-2.6.git;a=3Dcommit;h=3Ddb8be50c4307dac2b37305fc59c8dc0f978d09ea
> > References	: http://marc.info/?l=3Dlinux-kernel&m=3D125446885705223&w=3D4
> 
> Fixed for 2.6.32-rc5 by d93a8f829fe1d2f3002f2c6ddb553d12db420412
> 'Revert "USB: Work around BIOS bugs by quiescing USB controllers earlier"'.

Thanks, closed.

Rafael

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

On Monday 12 October 2009, Justin P. Mattock wrote:
> Lin Ming wrote:
> > On Mon, 2009-10-12 at 11:14 +0800, Justin Mattock wrote:
> >    
> >> On Sun, Oct 11, 2009 at 3:22 PM, Rafael J. Wysocki<rjw@sisk.pl>  wrote:
> >>      
> >>> This message has been generated automatically as a part of a report
> >>> of recent regressions.
> >>>
> >>> The following bug entry is on the current list of known regressions
> >>> from 2.6.31.  Please verify if it still should be listed and let me know
> >>> (either way).
> >>>
> >>>
> >>> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14379
> >>> Subject         : ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> >>> Submitter       : Justin Mattock<justinmattock@gmail.com>
> >>> Date            : 2009-10-08 21:46 (4 days old)
> >>> References      : http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> >>>
> >>>
> >>>
> >>>        
> >> pulled rc4 today and the warning is still present.
> >> here is the results of git bisect:
> >>      
> >
> > Patch at http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1 fixes the
> > warning.
> >
> > Actually, the _BIF warning was fixed at commit
> > e899b6485c332aa2d7510739507ab5e5d7b28e59.
> > See http://bugzilla.kernel.org/show_bug.cgi?id=11822#c14
> >
> > I made a mistake at the cleanup of predefined method table so the
> > warning shows up again.
> >
> > Lin Ming
> >
> >    
> Cool, I can give that a try.
> then I guess just leave this open(bugreport) until this patch
> http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1
> gets pushed into the main tree then close the bugreport.

I updated the bug entry with the patch information.

Thanks,
Rafael

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> On Monday 12 October 2009, Justin P. Mattock wrote:
>    
>> Lin Ming wrote:
>>      
>>> On Mon, 2009-10-12 at 11:14 +0800, Justin Mattock wrote:
>>>
>>>        
>>>> On Sun, Oct 11, 2009 at 3:22 PM, Rafael J. Wysocki<rjw@sisk.pl>   wrote:
>>>>
>>>>          
>>>>> This message has been generated automatically as a part of a report
>>>>> of recent regressions.
>>>>>
>>>>> The following bug entry is on the current list of known regressions
>>>>> from 2.6.31.  Please verify if it still should be listed and let me know
>>>>> (either way).
>>>>>
>>>>>
>>>>> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14379
>>>>> Subject         : ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
>>>>> Submitter       : Justin Mattock<justinmattock@gmail.com>
>>>>> Date            : 2009-10-08 21:46 (4 days old)
>>>>> References      : http://marc.info/?l=linux-kernel&m=125504031328941&w=4
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>            
>>>> pulled rc4 today and the warning is still present.
>>>> here is the results of git bisect:
>>>>
>>>>          
>>> Patch at http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1 fixes the
>>> warning.
>>>
>>> Actually, the _BIF warning was fixed at commit
>>> e899b6485c332aa2d7510739507ab5e5d7b28e59.
>>> See http://bugzilla.kernel.org/show_bug.cgi?id=11822#c14
>>>
>>> I made a mistake at the cleanup of predefined method table so the
>>> warning shows up again.
>>>
>>> Lin Ming
>>>
>>>
>>>        
>> Cool, I can give that a try.
>> then I guess just leave this open(bugreport) until this patch
>> http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1
>> gets pushed into the main tree then close the bugreport.
>>      
>
> I updated the bug entry with the patch information.
>
> Thanks,
> Rafael
>
>    
o.k. after somehow blowing my system up, I'm up and
running. Anyways I applied that patch and yes it works.

Justin P. Mattock

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> On Monday 12 October 2009, Justin P. Mattock wrote:
>    
>> Lin Ming wrote:
>>      
>>> On Mon, 2009-10-12 at 11:14 +0800, Justin Mattock wrote:
>>>
>>>        
>>>> On Sun, Oct 11, 2009 at 3:22 PM, Rafael J. Wysocki<rjw-KKrjLPT3xs0@public.gmane.org>   wrote:
>>>>
>>>>          
>>>>> This message has been generated automatically as a part of a report
>>>>> of recent regressions.
>>>>>
>>>>> The following bug entry is on the current list of known regressions
>>>>> from 2.6.31.  Please verify if it still should be listed and let me know
>>>>> (either way).
>>>>>
>>>>>
>>>>> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14379
>>>>> Subject         : ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
>>>>> Submitter       : Justin Mattock<justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>>>> Date            : 2009-10-08 21:46 (4 days old)
>>>>> References      : http://marc.info/?l=linux-kernel&m=125504031328941&w=4
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>            
>>>> pulled rc4 today and the warning is still present.
>>>> here is the results of git bisect:
>>>>
>>>>          
>>> Patch at http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1 fixes the
>>> warning.
>>>
>>> Actually, the _BIF warning was fixed at commit
>>> e899b6485c332aa2d7510739507ab5e5d7b28e59.
>>> See http://bugzilla.kernel.org/show_bug.cgi?id=11822#c14
>>>
>>> I made a mistake at the cleanup of predefined method table so the
>>> warning shows up again.
>>>
>>> Lin Ming
>>>
>>>
>>>        
>> Cool, I can give that a try.
>> then I guess just leave this open(bugreport) until this patch
>> http://bugzilla.kernel.org/show_bug.cgi?id=14379#c1
>> gets pushed into the main tree then close the bugreport.
>>      
>
> I updated the bug entry with the patch information.
>
> Thanks,
> Rafael
>
>    
o.k. after somehow blowing my system up, I'm up and
running. Anyways I applied that patch and yes it works.

Justin P. Mattock

Re: [Bug #14372] Wireless not working after suspend-resume

[Maciej Rutecki]

CC added

2009/10/12 Fabio Comolli <fabio.comolli@gmail.com>:

[...]
> [  690.553298] wlan0: AP denied association (code=12)
[...]

I have the same code in different situation:
http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089

Regards
-- 
Maciej Rutecki
http://www.maciek.unixy.pl

Re: [Bug #14372] Wireless not working after suspend-resume

[Maciej Rutecki]

CC added

2009/10/12 Fabio Comolli <fabio.comolli@gmail.com>:

[...]
> [  690.553298] wlan0: AP denied association (code=12)
[...]

I have the same code in different situation:
http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089

Regards
-- 
Maciej Rutecki
http://www.maciek.unixy.pl

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

David Miller ha scritto:
> From: "Rafael J. Wysocki" <rjw@sisk.pl>
> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
>
>   
>> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
>> Subject		: Problems with net/core/skbuff.c
>> Submitter	: Massimo Cetra <mcetra@navynet.it>
>> Date		: 2009-10-08 14:51 (4 days old)
>> References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4
>>     
>
> I don't know what to do about this one.
>
> The user indicates that they have the vserver patches applied,
> so maybe there is some interaction with that stuff.
>   
Actually i found another oops which is very similar to the previous one.
Here, vserver is not involved, and the problem starts at drbd which 
lives in kernel space (the other oops started at ocfs2).

Both ocfs2 and drbd make heavy use of network I/O so i guess the problem 
is something in the network layer.

Anything i can do to help to debugging and solving this issue ?

Thanks
Max




[-- Attachment #2: oops2.txt --]
[-- Type: text/plain, Size: 122916 bytes --]

[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f2c ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f2d ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f2e ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f2f ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f30 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f31 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f32 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f33 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f34 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f35 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f36 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f37 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f38 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f39 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f3a ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f3b ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f3c ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f3d ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f3e ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f3f ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f40 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f41 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f42 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f43 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f44 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f45 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f46 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f47 ]---

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

Massimo Cetra ha scritto:
> David Miller ha scritto:
>> From: "Rafael J. Wysocki" <rjw@sisk.pl>
>> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
>>
>>  
>>> Bug-Entry    : http://bugzilla.kernel.org/show_bug.cgi?id=14378
>>> Subject        : Problems with net/core/skbuff.c
>>> Submitter    : Massimo Cetra <mcetra@navynet.it>
>>> Date        : 2009-10-08 14:51 (4 days old)
>>> References    : http://marc.info/?l=linux-kernel&m=125501488220358&w=4
>>>     
>>
>> I don't know what to do about this one.
>>
>> The user indicates that they have the vserver patches applied,
>> so maybe there is some interaction with that stuff.
>>   
> Actually i found another oops which is very similar to the previous one.

And here it is another one, this time triggered by postfix, where mor 
drbd nor vserver are involved.
This is not the same server where the other oopses were grabbed.

Max

Re: [Bug #14378] Problems with net/core/skbuff.c

[Eric Dumazet]

Massimo Cetra a écrit :
> David Miller ha scritto:
>> From: "Rafael J. Wysocki" <rjw@sisk.pl>
>> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
>>
>>  
>>> Bug-Entry    : http://bugzilla.kernel.org/show_bug.cgi?id=14378
>>> Subject        : Problems with net/core/skbuff.c
>>> Submitter    : Massimo Cetra <mcetra@navynet.it>
>>> Date        : 2009-10-08 14:51 (4 days old)
>>> References    : http://marc.info/?l=linux-kernel&m=125501488220358&w=4
>>>     
>>
>> I don't know what to do about this one.
>>
>> The user indicates that they have the vserver patches applied,
>> so maybe there is some interaction with that stuff.
>>   
> Actually i found another oops which is very similar to the previous one.
> Here, vserver is not involved, and the problem starts at drbd which
> lives in kernel space (the other oops started at ocfs2).
> 
> Both ocfs2 and drbd make heavy use of network I/O so i guess the problem
> is something in the network layer.
> 
> Anything i can do to help to debugging and solving this issue ?
> 
> Thanks
> Max
> 

Problem is kfree_skb() is called from irq context, wich is not allowed.

static void skb_release_head_state(struct sk_buff *skb)
{
...
if (skb->destructor) {
	WARN_ON(in_irq());
	skb->destructor();
}
...
}

virtio_net start_xmit() function calls free_old_xmit_skbs() and 
free_old_xmit_skbs() ultimately calls kfree_skb()

Quick fix would be to use dev_kfree_skb_any() instead,
because netpoll can definitly calls start_xmit()
with irq disabled.

Could you please following patch ?

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 8d00976..54bf091 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -454,7 +454,7 @@ static unsigned int free_old_xmit_skbs(struct virtnet_info *vi)
 		vi->dev->stats.tx_bytes += skb->len;
 		vi->dev->stats.tx_packets++;
 		tot_sgs += skb_vnet_hdr(skb)->num_sg;
-		kfree_skb(skb);
+		dev_kfree_skb_any(skb);
 	}
 	return tot_sgs;
 }

Re: [Bug #14378] Problems with net/core/skbuff.c

[David Miller]

From: Massimo Cetra <ctrixk@navynet.it>
Date: Tue, 13 Oct 2009 11:11:17 +0200

> Here, vserver is not involved, and the problem starts at drbd which
> lives in kernel space (the other oops started at ocfs2).
> 
> Both ocfs2 and drbd make heavy use of network I/O so i guess the
> problem is something in the network layer.
> 
> Anything i can do to help to debugging and solving this issue ?

Is all of your traffic going over virtio_net?

Re: [Bug #14378] Problems with net/core/skbuff.c

[David Miller]

From: Massimo Cetra <ctrixk-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
Date: Tue, 13 Oct 2009 11:11:17 +0200

> Here, vserver is not involved, and the problem starts at drbd which
> lives in kernel space (the other oops started at ocfs2).
> 
> Both ocfs2 and drbd make heavy use of network I/O so i guess the
> problem is something in the network layer.
> 
> Anything i can do to help to debugging and solving this issue ?

Is all of your traffic going over virtio_net?

Re: [Bug #14378] Problems with net/core/skbuff.c

[David Miller]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 13 Oct 2009 12:19:51 +0200

> Quick fix would be to use dev_kfree_skb_any() instead,
> because netpoll can definitly calls start_xmit()
> with irq disabled.

Indeed, because of netpoll(), that is something drivers
must be able to cope with.

> Could you please following patch ?

Indeed, let us know how Eric's patch works.

Thanks Eric!

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

Eric Dumazet ha scritto:
> Could you please following patch ?
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 8d00976..54bf091 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -454,7 +454,7 @@ static unsigned int free_old_xmit_skbs(struct virtnet_info *vi)
>  		vi->dev->stats.tx_bytes += skb->len;
>  		vi->dev->stats.tx_packets++;
>  		tot_sgs += skb_vnet_hdr(skb)->num_sg;
> -		kfree_skb(skb);
> +		dev_kfree_skb_any(skb);
>  	}
>  	return tot_sgs;
>  }
>
>
>   
Thank you very much.

Compiling.
It' will be in production in a few minutes.
I'll let you know if the problem arises again.
give me a couple of days because this problem is randomly triggered.
Sometimes it happens multiple times a day, sometimes none.

Max

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

David Miller ha scritto:
> From: Massimo Cetra <ctrixk@navynet.it>
> Date: Tue, 13 Oct 2009 11:11:17 +0200
>
>   
>> Here, vserver is not involved, and the problem starts at drbd which
>> lives in kernel space (the other oops started at ocfs2).
>>
>> Both ocfs2 and drbd make heavy use of network I/O so i guess the
>> problem is something in the network layer.
>>
>> Anything i can do to help to debugging and solving this issue ?
>>     
>
> Is all of your traffic going over virtio_net?
>   
Yes, indeed.
I'm compiling the patch and i'll let you know.

Thanks,
Max

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

David Miller ha scritto:
> From: Massimo Cetra <ctrixk-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
> Date: Tue, 13 Oct 2009 11:11:17 +0200
>
>   
>> Here, vserver is not involved, and the problem starts at drbd which
>> lives in kernel space (the other oops started at ocfs2).
>>
>> Both ocfs2 and drbd make heavy use of network I/O so i guess the
>> problem is something in the network layer.
>>
>> Anything i can do to help to debugging and solving this issue ?
>>     
>
> Is all of your traffic going over virtio_net?
>   
Yes, indeed.
I'm compiling the patch and i'll let you know.

Thanks,
Max

Re: [Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[John W. Linville]

On Mon, Oct 12, 2009 at 11:49:39PM +0200, Rafael J. Wysocki wrote:
> On Monday 12 October 2009, Michael Buesch wrote:
> > On Monday 12 October 2009 00:22:00 Rafael J. Wysocki wrote:
> > > This message has been generated automatically as a part of a report
> > > of recent regressions.
> > > 
> > > The following bug entry is on the current list of known regressions
> > > from 2.6.31.  Please verify if it still should be listed and let me know
> > > (either way).
> > > 
> > > 
> > > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
> > > Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
> > > Submitter	: Christian Casteyde <casteyde.christian@free.fr>
> > > Date		: 2009-09-30 18:12 (12 days old)
> > > Handled-By	: Michael Buesch <mb@bu3sch.de>
> > 
> > I think John or Johannes is handling it, as my patch was rejected.
> > 
> > > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220
> 
> Johannes, what's the status here?

Johannes posted a patch here:

	http://marc.info/?l=linux-wireless&m=125525657120789&w=2

It is on my merge train, but it hasn't gone to Linus (or Dave) just
yet -- probably today.

Hth!

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: [Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[John W. Linville]

On Mon, Oct 12, 2009 at 11:49:39PM +0200, Rafael J. Wysocki wrote:
> On Monday 12 October 2009, Michael Buesch wrote:
> > On Monday 12 October 2009 00:22:00 Rafael J. Wysocki wrote:
> > > This message has been generated automatically as a part of a report
> > > of recent regressions.
> > > 
> > > The following bug entry is on the current list of known regressions
> > > from 2.6.31.  Please verify if it still should be listed and let me know
> > > (either way).
> > > 
> > > 
> > > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
> > > Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
> > > Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
> > > Date		: 2009-09-30 18:12 (12 days old)
> > > Handled-By	: Michael Buesch <mb-fseUSCV1ubazQB+pC5nmwQ@public.gmane.org>
> > 
> > I think John or Johannes is handling it, as my patch was rejected.
> > 
> > > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220
> 
> Johannes, what's the status here?

Johannes posted a patch here:

	http://marc.info/?l=linux-wireless&m=125525657120789&w=2

It is on my merge train, but it hasn't gone to Linus (or Dave) just
yet -- probably today.

Hth!

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org			might be all we have.  Be ready.

Re: [Bug #14389] Build system issue

[Sam Ravnborg]

On Mon, Oct 12, 2009 at 12:22:09AM +0200, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
> Subject		: Build system issue
> Submitter	: Peter Zijlstra <peterz@infradead.org>
> Date		: 2009-10-09 8:58 (3 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
> References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4

It is fixed by: 2331d1a6cd3d6e580bc88b9a160066d9e1177fe1

I did not have the bug-entry when the changelog was written
so no references there.

	Sam

Re: [Bug #14389] Build system issue

[Sam Ravnborg]

On Mon, Oct 12, 2009 at 12:22:09AM +0200, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
> Subject		: Build system issue
> Submitter	: Peter Zijlstra <peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> Date		: 2009-10-09 8:58 (3 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
> References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4

It is fixed by: 2331d1a6cd3d6e580bc88b9a160066d9e1177fe1

I did not have the bug-entry when the changelog was written
so no references there.

	Sam

Re: [Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[Rafael J. Wysocki]

On Tuesday 13 October 2009, John W. Linville wrote:
> On Mon, Oct 12, 2009 at 11:49:39PM +0200, Rafael J. Wysocki wrote:
> > On Monday 12 October 2009, Michael Buesch wrote:
> > > On Monday 12 October 2009 00:22:00 Rafael J. Wysocki wrote:
> > > > This message has been generated automatically as a part of a report
> > > > of recent regressions.
> > > > 
> > > > The following bug entry is on the current list of known regressions
> > > > from 2.6.31.  Please verify if it still should be listed and let me know
> > > > (either way).
> > > > 
> > > > 
> > > > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
> > > > Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
> > > > Submitter	: Christian Casteyde <casteyde.christian@free.fr>
> > > > Date		: 2009-09-30 18:12 (12 days old)
> > > > Handled-By	: Michael Buesch <mb@bu3sch.de>
> > > 
> > > I think John or Johannes is handling it, as my patch was rejected.
> > > 
> > > > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220
> > 
> > Johannes, what's the status here?
> 
> Johannes posted a patch here:
> 
> 	http://marc.info/?l=linux-wireless&m=125525657120789&w=2
> 
> It is on my merge train, but it hasn't gone to Linus (or Dave) just
> yet -- probably today.
> 
> Hth!

Yes, it does, I've updated the bug entry with the link to the Johannes' patch.

Thanks,
Rafael

Re: [Bug #14278] New message "NOHZ: local_softirq_pending 08" at each ping request

[Rafael J. Wysocki]

On Tuesday 13 October 2009, John W. Linville wrote:
> On Mon, Oct 12, 2009 at 11:49:39PM +0200, Rafael J. Wysocki wrote:
> > On Monday 12 October 2009, Michael Buesch wrote:
> > > On Monday 12 October 2009 00:22:00 Rafael J. Wysocki wrote:
> > > > This message has been generated automatically as a part of a report
> > > > of recent regressions.
> > > > 
> > > > The following bug entry is on the current list of known regressions
> > > > from 2.6.31.  Please verify if it still should be listed and let me know
> > > > (either way).
> > > > 
> > > > 
> > > > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
> > > > Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
> > > > Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
> > > > Date		: 2009-09-30 18:12 (12 days old)
> > > > Handled-By	: Michael Buesch <mb-fseUSCV1ubazQB+pC5nmwQ@public.gmane.org>
> > > 
> > > I think John or Johannes is handling it, as my patch was rejected.
> > > 
> > > > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220
> > 
> > Johannes, what's the status here?
> 
> Johannes posted a patch here:
> 
> 	http://marc.info/?l=linux-wireless&m=125525657120789&w=2
> 
> It is on my merge train, but it hasn't gone to Linus (or Dave) just
> yet -- probably today.
> 
> Hth!

Yes, it does, I've updated the bug entry with the link to the Johannes' patch.

Thanks,
Rafael

Re: [Bug #14389] Build system issue

[Rafael J. Wysocki]

On Tuesday 13 October 2009, Sam Ravnborg wrote:
> On Mon, Oct 12, 2009 at 12:22:09AM +0200, Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
> > Subject		: Build system issue
> > Submitter	: Peter Zijlstra <peterz@infradead.org>
> > Date		: 2009-10-09 8:58 (3 days old)
> > First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
> > References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4
> 
> It is fixed by: 2331d1a6cd3d6e580bc88b9a160066d9e1177fe1
> 
> I did not have the bug-entry when the changelog was written
> so no references there.

Thanks, closing.

Rafael

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

Massimo Cetra ha scritto:
> Eric Dumazet ha scritto:
>> Could you please following patch ?
>>
>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>> index 8d00976..54bf091 100644
>> --- a/drivers/net/virtio_net.c
>> +++ b/drivers/net/virtio_net.c
>> @@ -454,7 +454,7 @@ static unsigned int free_old_xmit_skbs(struct 
>> virtnet_info *vi)
>>          vi->dev->stats.tx_bytes += skb->len;
>>          vi->dev->stats.tx_packets++;
>>          tot_sgs += skb_vnet_hdr(skb)->num_sg;
>> -        kfree_skb(skb);
>> +        dev_kfree_skb_any(skb);
>>      }
>>      return tot_sgs;
>>  }
>>
>>
>>   
> Thank you very much.
>
> Compiling.
> It' will be in production in a few minutes.
> I'll let you know if the problem arises again.
> give me a couple of days because this problem is randomly triggered.
> Sometimes it happens multiple times a day, sometimes none.
>
Eric,
thanks for the patch.
The problem didn't arise again and i haven't seen any warning like that 
on both servers where that problem was happening more frequently.

I would say that it's fixed and if it's not, i'll let you know as soon 
as it happens again.

Thanks again,
Max

[PATCH] virtio_net: use dev_kfree_skb_any() in free_old_xmit_skbs()

[Eric Dumazet]

Massimo Cetra a écrit :
> Eric,
> thanks for the patch.
> The problem didn't arise again and i haven't seen any warning like that
> on both servers where that problem was happening more frequently.
> 
> I would say that it's fixed and if it's not, i'll let you know as soon
> as it happens again.
> 

Thanks Massimo, I think patch is reasonably safe and should be taken as is :


[PATCH] virtio_net: use dev_kfree_skb_any() in free_old_xmit_skbs()

Because netpoll can call netdevice start_xmit() method with
irqs disabled, drivers should not call kfree_skb() from
their start_xmit(), but use dev_kfree_skb_any() instead.

Oct  8 11:16:52 172.30.1.31 [113074.791813] ------------[ cut here ]------------
Oct  8 11:16:52 172.30.1.31 [113074.791813] WARNING: at net/core/skbuff.c:398 \
                skb_release_head_state+0x64/0xc8()
Oct  8 11:16:52 172.30.1.31 [113074.791813] Hardware name: 
Oct  8 11:16:52 172.30.1.31 [113074.791813] Modules linked in: netconsole ocfs2 jbd2 quota_tree \
ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs crc32c drbd cn loop \
serio_raw psmouse snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net pcspkr parport_pc parport \
i2c_piix4 i2c_core button processor evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot \
dm_mod ide_cd_mod cdrom ata_generic ata_piix virtio_blk libata scsi_mod piix ide_pci_generic ide_core \
                virtio_pci virtio_ring virtio floppy thermal fan thermal_sys [last unloaded: netconsole]
Oct  8 11:16:52 172.30.1.31 [113074.791813] Pid: 11132, comm: php5-cgi Tainted: G        W  \
                2.6.31.2-vserver #1
Oct  8 11:16:52 172.30.1.31 [113074.791813] Call Trace:
Oct  8 11:16:52 172.30.1.31 [113074.791813] <IRQ>  [<ffffffff81253cd5>] ? \
                skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa01cb139>] ? free_old_xmit_skbs+0x51/0x6e \
                [virtio_net]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa01cbc85>] ? start_xmit+0x26/0xf2 [virtio_net]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa0429216>] ? write_msg+0x90/0xeb [netconsole]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8106b142>] ? vx_update_load+0x18/0x13e
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81308309>] ? printk+0x4e/0x5d
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff810626b7>] ? ktime_get+0xc/0x41
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
Oct  8 11:16:52 172.30.1.31 [113074.791813] <EOI>  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31

Reported-and-tested-by: Massimo Cetra <mcetra@navynet.it>
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Bug-Entry: http://bugzilla.kernel.org/show_bug.cgi?id=14378
---

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 8d00976..54bf091 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -454,7 +454,7 @@ static unsigned int free_old_xmit_skbs(struct virtnet_info *vi)
 		vi->dev->stats.tx_bytes += skb->len;
 		vi->dev->stats.tx_packets++;
 		tot_sgs += skb_vnet_hdr(skb)->num_sg;
-		kfree_skb(skb);
+		dev_kfree_skb_any(skb);
 	}
 	return tot_sgs;
 }

[PATCH] virtio_net: use dev_kfree_skb_any() in free_old_xmit_skbs()

[Eric Dumazet]

Massimo Cetra a écrit :
> Eric,
> thanks for the patch.
> The problem didn't arise again and i haven't seen any warning like that
> on both servers where that problem was happening more frequently.
> 
> I would say that it's fixed and if it's not, i'll let you know as soon
> as it happens again.
> 

Thanks Massimo, I think patch is reasonably safe and should be taken as is :


[PATCH] virtio_net: use dev_kfree_skb_any() in free_old_xmit_skbs()

Because netpoll can call netdevice start_xmit() method with
irqs disabled, drivers should not call kfree_skb() from
their start_xmit(), but use dev_kfree_skb_any() instead.

Oct  8 11:16:52 172.30.1.31 [113074.791813] ------------[ cut here ]------------
Oct  8 11:16:52 172.30.1.31 [113074.791813] WARNING: at net/core/skbuff.c:398 \
                skb_release_head_state+0x64/0xc8()
Oct  8 11:16:52 172.30.1.31 [113074.791813] Hardware name: 
Oct  8 11:16:52 172.30.1.31 [113074.791813] Modules linked in: netconsole ocfs2 jbd2 quota_tree \
ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs crc32c drbd cn loop \
serio_raw psmouse snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net pcspkr parport_pc parport \
i2c_piix4 i2c_core button processor evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot \
dm_mod ide_cd_mod cdrom ata_generic ata_piix virtio_blk libata scsi_mod piix ide_pci_generic ide_core \
                virtio_pci virtio_ring virtio floppy thermal fan thermal_sys [last unloaded: netconsole]
Oct  8 11:16:52 172.30.1.31 [113074.791813] Pid: 11132, comm: php5-cgi Tainted: G        W  \
                2.6.31.2-vserver #1
Oct  8 11:16:52 172.30.1.31 [113074.791813] Call Trace:
Oct  8 11:16:52 172.30.1.31 [113074.791813] <IRQ>  [<ffffffff81253cd5>] ? \
                skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa01cb139>] ? free_old_xmit_skbs+0x51/0x6e \
                [virtio_net]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa01cbc85>] ? start_xmit+0x26/0xf2 [virtio_net]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa0429216>] ? write_msg+0x90/0xeb [netconsole]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8106b142>] ? vx_update_load+0x18/0x13e
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81308309>] ? printk+0x4e/0x5d
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff810626b7>] ? ktime_get+0xc/0x41
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
Oct  8 11:16:52 172.30.1.31 [113074.791813] <EOI>  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31

Reported-and-tested-by: Massimo Cetra <mcetra-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
Signed-off-by: Eric Dumazet <eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Bug-Entry: http://bugzilla.kernel.org/show_bug.cgi?id=14378
---

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 8d00976..54bf091 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -454,7 +454,7 @@ static unsigned int free_old_xmit_skbs(struct virtnet_info *vi)
 		vi->dev->stats.tx_bytes += skb->len;
 		vi->dev->stats.tx_packets++;
 		tot_sgs += skb_vnet_hdr(skb)->num_sg;
-		kfree_skb(skb);
+		dev_kfree_skb_any(skb);
 	}
 	return tot_sgs;
 }

Re: [PATCH] virtio_net: use dev_kfree_skb_any() in free_old_xmit_skbs()

[David Miller]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Thu, 15 Oct 2009 02:36:43 +0200

> Massimo Cetra a écrit :
>> Eric,
>> thanks for the patch.
>> The problem didn't arise again and i haven't seen any warning like that
>> on both servers where that problem was happening more frequently.
>> 
>> I would say that it's fixed and if it's not, i'll let you know as soon
>> as it happens again.
>> 
> 
> Thanks Massimo, I think patch is reasonably safe and should be taken as is :
> 
> 
> [PATCH] virtio_net: use dev_kfree_skb_any() in free_old_xmit_skbs()

Applied, thanks Eric.

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter	: Justin Mattock<justinmattock@gmail.com>
> Date		: 2009-10-08 21:46 (45 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> Handled-By	: Alexey Starikovskiy<astarikovskiy@suse.de>
> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
>
>
>
>    
still there.

Justin P. Mattock

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter	: Justin Mattock<justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date		: 2009-10-08 21:46 (45 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> Handled-By	: Alexey Starikovskiy<astarikovskiy-l3A5Bk7waGM@public.gmane.org>
> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
>
>
>
>    
still there.

Justin P. Mattock

[Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock@gmail.com>
Date		: 2009-10-08 21:46 (45 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
Handled-By	: Alexey Starikovskiy <astarikovskiy@suse.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

On Tuesday 17 November 2009, Justin P. Mattock wrote:
> Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> > Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> > Submitter	: Justin Mattock<justinmattock@gmail.com>
> > Date		: 2009-10-08 21:46 (40 days old)
> > First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> > References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> > Handled-By	: Alexey Starikovskiy<astarikovskiy@suse.de>
> > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
> >
> >
> >
> >    
> o.k. just pulled the latest to see, and
> the warning message is there.
> so yes this bug report should be open
> until this is fixed.

Thanks for the update.

Rafael

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

On Tuesday 17 November 2009, Justin P. Mattock wrote:
> Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> > Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> > Submitter	: Justin Mattock<justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Date		: 2009-10-08 21:46 (40 days old)
> > First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> > References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> > Handled-By	: Alexey Starikovskiy<astarikovskiy-l3A5Bk7waGM@public.gmane.org>
> > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
> >
> >
> >
> >    
> o.k. just pulled the latest to see, and
> the warning message is there.
> so yes this bug report should be open
> until this is fixed.

Thanks for the update.

Rafael

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter	: Justin Mattock<justinmattock@gmail.com>
> Date		: 2009-10-08 21:46 (40 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> Handled-By	: Alexey Starikovskiy<astarikovskiy@suse.de>
> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
>
>
>
>    
o.k. just pulled the latest to see, and
the warning message is there.
so yes this bug report should be open
until this is fixed.

Justin P. Mattock

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter	: Justin Mattock<justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date		: 2009-10-08 21:46 (40 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> Handled-By	: Alexey Starikovskiy<astarikovskiy-l3A5Bk7waGM@public.gmane.org>
> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
>
>
>
>    
o.k. just pulled the latest to see, and
the warning message is there.
so yes this bug report should be open
until this is fixed.

Justin P. Mattock

[Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock@gmail.com>
Date		: 2009-10-08 21:46 (40 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
Handled-By	: Alexey Starikovskiy <astarikovskiy@suse.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347

[Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-08 21:46 (40 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
Handled-By	: Alexey Starikovskiy <astarikovskiy-l3A5Bk7waGM@public.gmane.org>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

On Monday 26 October 2009, Justin P. Mattock wrote:
> Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> > Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> > Submitter	: Justin Mattock<justinmattock@gmail.com>
> > Date		: 2009-10-08 21:46 (19 days old)
> > First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> > References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> > Handled-By	: Alexey Starikovskiy<astarikovskiy@suse.de>
> > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
> >
> >
> >
> >    
> The patch submitted gets rid of the warning
> for people crying wolf!! but probably would not close
> the bugreport until the patch makes it into the main
> kernel.

Yup.

Thanks,
Rafael

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter	: Justin Mattock<justinmattock@gmail.com>
> Date		: 2009-10-08 21:46 (19 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> Handled-By	: Alexey Starikovskiy<astarikovskiy@suse.de>
> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
>
>
>
>    
The patch submitted gets rid of the warning
for people crying wolf!! but probably would not close
the bugreport until the patch makes it into the main
kernel.

Justin P. Mattock

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter	: Justin Mattock<justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date		: 2009-10-08 21:46 (19 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> Handled-By	: Alexey Starikovskiy<astarikovskiy-l3A5Bk7waGM@public.gmane.org>
> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
>
>
>
>    
The patch submitted gets rid of the warning
for people crying wolf!! but probably would not close
the bugreport until the patch makes it into the main
kernel.

Justin P. Mattock

[Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock@gmail.com>
Date		: 2009-10-08 21:46 (19 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
Handled-By	: Alexey Starikovskiy <astarikovskiy@suse.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347

[Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-08 21:46 (19 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
Handled-By	: Alexey Starikovskiy <astarikovskiy-l3A5Bk7waGM@public.gmane.org>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347