All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@suse.cz>
To: Glauber Costa <glommer@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Dave Chinner <david@fromorbit.com>,
	linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>
Subject: Re: linux-next: slab shrinkers: BUG at mm/list_lru.c:92
Date: Wed, 26 Jun 2013 10:22:20 +0200	[thread overview]
Message-ID: <20130626082220.GG28748@dhcp22.suse.cz> (raw)
In-Reply-To: <20130623115127.GA7986@localhost.localdomain>

On Sun 23-06-13 15:51:29, Glauber Costa wrote:
> On Fri, Jun 21, 2013 at 11:00:21AM +0200, Michal Hocko wrote:
> > On Thu 20-06-13 17:12:01, Michal Hocko wrote:
> > > I am bisecting it again. It is quite tedious, though, because good case
> > > is hard to be sure about.
> > 
> > OK, so now I converged to 2d4fc052 (inode: convert inode lru list to generic lru
> > list code.) in my tree and I have double checked it matches what is in
> > the linux-next. This doesn't help much to pin point the issue I am
> > afraid :/
> > 
> Can you revert this patch (easiest way ATM is to rewind your tree to a point
> right before it) and apply the following patch?

OK, I am testing it now.

> As Dave has mentioned, it is very likely that this bug was already there, we
> were just not ever checking imbalances. The attached patch would tell us at
> least if the imbalance was there before.

Maybe I wasn't clear before but I have seen mostly hangs (posted
earlier) during bisection. I do not remember BUG_ONs on imbalances after
inode_lru_isolate fix.

> If this is the case, I would suggest
> turning the BUG condition into a WARN_ON_ONCE since we would be officially
> not introducing any regression. It is no less of a bug, though, and we should
> keep looking for it.
> 
> The main change from before / after the patch is that we are now keeping things
> per node. One possibility of having this BUGing would be to have an inode to be
> inserted into one node-lru and removed from another. I cannot see how it could
> happen, because kernel pages are stable in memory and are not moved from node
> to node. We could still have some sort of weird bug in the node calculation
> function. 

> In any case, would it be possible for you to artificially restrict
> your setup to a single node ? Although I have no idea how to do that, we seem
> to have no parameter to disable numa. Maybe booting with less memory, enough to
> fit a single node?

I can play with memmap to use areas on from a single node. Let's see
whether the patch in the follow up email shows something first.
-- 
Michal Hocko
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@suse.cz>
To: Glauber Costa <glommer@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Dave Chinner <david@fromorbit.com>,
	linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>
Subject: Re: linux-next: slab shrinkers: BUG at mm/list_lru.c:92
Date: Wed, 26 Jun 2013 10:22:20 +0200	[thread overview]
Message-ID: <20130626082220.GG28748@dhcp22.suse.cz> (raw)
In-Reply-To: <20130623115127.GA7986@localhost.localdomain>

On Sun 23-06-13 15:51:29, Glauber Costa wrote:
> On Fri, Jun 21, 2013 at 11:00:21AM +0200, Michal Hocko wrote:
> > On Thu 20-06-13 17:12:01, Michal Hocko wrote:
> > > I am bisecting it again. It is quite tedious, though, because good case
> > > is hard to be sure about.
> > 
> > OK, so now I converged to 2d4fc052 (inode: convert inode lru list to generic lru
> > list code.) in my tree and I have double checked it matches what is in
> > the linux-next. This doesn't help much to pin point the issue I am
> > afraid :/
> > 
> Can you revert this patch (easiest way ATM is to rewind your tree to a point
> right before it) and apply the following patch?

OK, I am testing it now.

> As Dave has mentioned, it is very likely that this bug was already there, we
> were just not ever checking imbalances. The attached patch would tell us at
> least if the imbalance was there before.

Maybe I wasn't clear before but I have seen mostly hangs (posted
earlier) during bisection. I do not remember BUG_ONs on imbalances after
inode_lru_isolate fix.

> If this is the case, I would suggest
> turning the BUG condition into a WARN_ON_ONCE since we would be officially
> not introducing any regression. It is no less of a bug, though, and we should
> keep looking for it.
> 
> The main change from before / after the patch is that we are now keeping things
> per node. One possibility of having this BUGing would be to have an inode to be
> inserted into one node-lru and removed from another. I cannot see how it could
> happen, because kernel pages are stable in memory and are not moved from node
> to node. We could still have some sort of weird bug in the node calculation
> function. 

> In any case, would it be possible for you to artificially restrict
> your setup to a single node ? Although I have no idea how to do that, we seem
> to have no parameter to disable numa. Maybe booting with less memory, enough to
> fit a single node?

I can play with memmap to use areas on from a single node. Let's see
whether the patch in the follow up email shows something first.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2013-06-26  8:22 UTC|newest]

Thread overview: 127+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-06-17 14:18 linux-next: slab shrinkers: BUG at mm/list_lru.c:92 Michal Hocko
2013-06-17 14:18 ` Michal Hocko
2013-06-17 15:14 ` Glauber Costa
2013-06-17 15:14   ` Glauber Costa
2013-06-17 15:33   ` Michal Hocko
2013-06-17 15:33     ` Michal Hocko
2013-06-17 16:54     ` Glauber Costa
2013-06-17 16:54       ` Glauber Costa
2013-06-18  7:42       ` Michal Hocko
2013-06-18  7:42         ` Michal Hocko
2013-06-17 21:35   ` Andrew Morton
2013-06-17 21:35     ` Andrew Morton
2013-06-17 22:30     ` Glauber Costa
2013-06-18  2:46       ` Dave Chinner
2013-06-18  2:46         ` Dave Chinner
2013-06-18  6:31         ` Glauber Costa
2013-06-18  6:31           ` Glauber Costa
2013-06-18  8:24           ` Michal Hocko
2013-06-18  8:24             ` Michal Hocko
2013-06-18 10:44             ` Michal Hocko
2013-06-18 10:44               ` Michal Hocko
2013-06-18 13:50               ` Michal Hocko
2013-06-18 13:50                 ` Michal Hocko
2013-06-25  2:27                 ` Dave Chinner
2013-06-25  2:27                   ` Dave Chinner
2013-06-26  8:15                   ` Michal Hocko
2013-06-26  8:15                     ` Michal Hocko
2013-06-26 23:24                     ` Dave Chinner
2013-06-26 23:24                       ` Dave Chinner
2013-06-27 14:54                       ` Michal Hocko
2013-06-27 14:54                         ` Michal Hocko
2013-06-28  8:39                         ` Michal Hocko
2013-06-28  8:39                           ` Michal Hocko
2013-06-28 14:31                           ` Glauber Costa
2013-06-28 14:31                             ` Glauber Costa
2013-06-28 15:12                             ` Michal Hocko
2013-06-28 15:12                               ` Michal Hocko
2013-06-29  2:55                         ` Dave Chinner
2013-06-29  2:55                           ` Dave Chinner
2013-06-30 18:33                           ` Michal Hocko
2013-07-01  1:25                             ` Dave Chinner
2013-07-01  1:25                               ` Dave Chinner
2013-07-01  7:50                               ` Michal Hocko
2013-07-01  7:50                                 ` Michal Hocko
2013-07-01  8:10                                 ` Dave Chinner
2013-07-01  8:10                                   ` Dave Chinner
2013-07-02  9:22                                   ` Michal Hocko
2013-07-02 12:19                                     ` Dave Chinner
2013-07-02 12:19                                       ` Dave Chinner
2013-07-02 12:44                                       ` Michal Hocko
2013-07-02 12:44                                         ` Michal Hocko
2013-07-03 11:24                                         ` Dave Chinner
2013-07-03 11:24                                           ` Dave Chinner
2013-07-03 14:08                                           ` Glauber Costa
2013-07-03 14:08                                             ` Glauber Costa
2013-07-04 16:36                                           ` Michal Hocko
2013-07-04 16:36                                             ` Michal Hocko
2013-07-08 12:53                                             ` Michal Hocko
2013-07-08 21:04                                               ` Andrew Morton
2013-07-08 21:04                                                 ` Andrew Morton
2013-07-09 17:34                                                 ` Glauber Costa
2013-07-09 17:34                                                   ` Glauber Costa
2013-07-09 17:51                                                   ` Andrew Morton
2013-07-09 17:51                                                     ` Andrew Morton
2013-07-09 17:32                                               ` Glauber Costa
2013-07-09 17:32                                                 ` Glauber Costa
2013-07-09 17:50                                                 ` Andrew Morton
2013-07-09 17:50                                                   ` Andrew Morton
2013-07-09 17:57                                                   ` Glauber Costa
2013-07-09 17:57                                                     ` Glauber Costa
2013-07-09 17:57                                                 ` Michal Hocko
2013-07-09 17:57                                                   ` Michal Hocko
2013-07-09 21:39                                                   ` Andrew Morton
2013-07-09 21:39                                                     ` Andrew Morton
2013-07-10  2:31                                               ` Dave Chinner
2013-07-10  2:31                                                 ` Dave Chinner
2013-07-10  7:34                                                 ` Michal Hocko
2013-07-10  7:34                                                   ` Michal Hocko
2013-07-10  8:06                                                 ` Michal Hocko
2013-07-10  8:06                                                   ` Michal Hocko
2013-07-11  2:26                                                   ` Dave Chinner
2013-07-11  2:26                                                     ` Dave Chinner
2013-07-11  3:03                                                     ` Andrew Morton
2013-07-11  3:03                                                       ` Andrew Morton
2013-07-11 13:23                                                     ` Michal Hocko
2013-07-11 13:23                                                       ` Michal Hocko
2013-07-12  1:42                                                       ` Hugh Dickins
2013-07-12  1:42                                                         ` Hugh Dickins
2013-07-13  3:29                                                         ` Dave Chinner
2013-07-13  3:29                                                           ` Dave Chinner
2013-07-15  9:14                                             ` Michal Hocko
2013-07-15  9:14                                               ` Michal Hocko
2013-06-18  6:26       ` Glauber Costa
2013-06-18  8:25         ` Michal Hocko
2013-06-18  8:25           ` Michal Hocko
2013-06-19  7:13         ` Michal Hocko
2013-06-19  7:13           ` Michal Hocko
2013-06-19  7:35           ` Glauber Costa
2013-06-19  7:35             ` Glauber Costa
2013-06-19  8:52             ` Glauber Costa
2013-06-19  8:52               ` Glauber Costa
2013-06-19 13:57             ` Michal Hocko
2013-06-19 13:57               ` Michal Hocko
2013-06-19 14:02               ` Glauber Costa
2013-06-19 14:02                 ` Glauber Costa
2013-06-19 14:28           ` Michal Hocko
2013-06-19 14:28             ` Michal Hocko
2013-06-20 14:11             ` Glauber Costa
2013-06-20 14:11               ` Glauber Costa
2013-06-20 15:12               ` Michal Hocko
2013-06-20 15:16                 ` Michal Hocko
2013-06-20 15:16                   ` Michal Hocko
2013-06-21  9:00                 ` Michal Hocko
2013-06-21  9:00                   ` Michal Hocko
2013-06-23 11:51                   ` Glauber Costa
2013-06-23 11:51                     ` Glauber Costa
2013-06-23 11:55                     ` Glauber Costa
2013-06-25  2:29                     ` Dave Chinner
2013-06-25  2:29                       ` Dave Chinner
2013-06-26  8:22                     ` Michal Hocko [this message]
2013-06-26  8:22                       ` Michal Hocko
2013-06-18  8:19       ` Michal Hocko
2013-06-18  8:19         ` Michal Hocko
2013-06-18  8:21         ` Glauber Costa
2013-06-18  8:21           ` Glauber Costa
2013-06-18  8:26           ` Michal Hocko
2013-06-18  8:26             ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130626082220.GG28748@dhcp22.suse.cz \
    --to=mhocko@suse.cz \
    --cc=akpm@linux-foundation.org \
    --cc=david@fromorbit.com \
    --cc=glommer@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.