Re: frequent softlockups with 3.10rc6.

[Steven Rostedt <rostedt@goodmis.org>]

On Mon, 2013-06-24 at 16:39 +0200, Oleg Nesterov wrote:
> On 06/23, Dave Jones wrote:
> >
> > On Sun, Jun 23, 2013 at 06:04:52PM +0200, Oleg Nesterov wrote:
> >
> >  > Could you please do the following:
> >  >
> >  > 	1. # cd /sys/kernel/debug/tracing
> >  > 	   # echo 0 >> options/function-trace
> >  > 	   # echo preemptirqsoff >> current_tracer
> >
> > dammit.
> >
> > WARNING: at include/linux/list.h:385 rb_head_page_deactivate.isra.39+0x61/0x80()
> 
> Hmmm. which kernel do use use?
> 
>    380  #define list_for_each(pos, head) \
>    381          for (pos = (head)->next; pos != (head); pos = pos->next)
>    382
>    383  /**
>    384   * __list_for_each      -       iterate over a list
>    385   * @pos:        the &struct list_head to use as a loop cursor.
>    386   * @head:       the head for your list.
>    387   *
>    388   * This variant doesn't differ from list_for_each() any more.
>    389   * We don't do prefetching in either case.
>    390   */
>    391  #define __list_for_each(pos, head) \
>    392          for (pos = (head)->next; pos != (head); pos = pos->next)
>    393
>    394  /**
>    395   * list_for_each_prev   -       iterate over a list backwards
>    396   * @pos:        the &struct list_head to use as a loop cursor.
>    397   * @head:       the head for your list.
>    398   */
>    399  #define list_for_each_prev(pos, head) \
>    400          for (pos = (head)->prev; pos != (head); pos = pos->prev)
> 
> On 9e895ace5d8 (Linux 3.10-rc7).

Right, and on 3.10-rc6:

382 
383 /**
384  * __list_for_each      -       iterate over a list
385  * @pos:        the &struct list_head to use as a loop cursor.
386  * @head:       the head for your list.
387  *
388  * This variant doesn't differ from list_for_each() any more.
389  * We don't do prefetching in either case.
390  */
391 #define __list_for_each(pos, head) \
392         for (pos = (head)->next; pos != (head); pos = pos->next)
393 


> 
> >  check_list_nodes corruption. next->prev should be prev (ffff88023b8a1a08), but was 00ffff88023b8a1a. (next=ffff880243288001).
> 
> Can't find "check_list_nodes" in lib/list_debug.c or elsewhere...
> 
> >   [<ffffffff816e467d>] dump_stack+0x19/0x1b
> >   [<ffffffff8104a0c1>] warn_slowpath_common+0x61/0x80
> >   [<ffffffff8104a12c>] warn_slowpath_fmt+0x4c/0x50
> >   [<ffffffff81112c61>] rb_head_page_deactivate.isra.39+0x61/0x80
> 
> How? rb_list_head_clear() just modifies list->next directly.
> 
> > hopefully despite that it'll actually function as intended.
> 
> Yes ;)

I'm curious to what happened.

-- Steve