All of lore.kernel.org
 help / color / mirror / Atom feed
From: Hugh Dickins <hughd@google.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Hugh Dickins <hughd@google.com>,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org
Subject: BUG_ON(!mapping_empty(&inode->i_data))
Date: Tue, 30 Mar 2021 18:30:22 -0700 (PDT)	[thread overview]
Message-ID: <alpine.LSU.2.11.2103301654520.2648@eggly.anvils> (raw)

Running my usual tmpfs kernel builds swapping load, on Sunday's rc4-mm1
mmotm (I never got to try rc3-mm1 but presume it behaved the same way),
I hit clear_inode()'s BUG_ON(!mapping_empty(&inode->i_data)); on two
machines, within an hour or few, repeatably though not to order.

The stack backtrace has always been clear_inode < ext4_clear_inode <
ext4_evict_inode < evict < dispose_list < prune_icache_sb <
super_cache_scan < do_shrink_slab < shrink_slab_memcg < shrink_slab <
shrink_node_memgs < shrink_node < balance_pgdat < kswapd.

ext4 is the disk filesystem I read the source to build from, and also
the filesystem I use on a loop device on a tmpfs file: I have not tried
with other filesystems, nor checked whether perhaps it happens always on
the loop one or always on the disk one.  I have not seen it happen with
tmpfs - probably because its inodes cannot be evicted by the shrinker
anyway; I have not seen it happen when "rm -rf" evicts ext4 or tmpfs
inodes (but suspect that may be down to timing, or less pressure).
I doubt it's a matter of filesystem: think it's an XArray thing.

Whenever I've looked at the XArray nodes involved, the root node
(shift 6) contained one or three (adjacent) pointers to empty shift
0 nodes, which each had offset and parent and array correctly set.
Is there some way in which empty nodes can get left behind, and so
fail eviction's mapping_empty() check?

I did wonder whether some might get left behind if xas_alloc() fails
(though probably the tree here is too shallow to show that).  Printks
showed that occasionally xas_alloc() did fail while testing (maybe at
memcg limit), but there was no correlation with the BUG_ONs.

I did wonder whether this is a long-standing issue, which your new
BUG_ON is the first to detect: so tried 5.12-rc5 clear_inode() with
a BUG_ON(!xa_empty(&inode->i_data.i_pages)) after its nrpages and
nrexceptional BUG_ONs.  The result there surprised me: I expected
it to behave the same way, but it hits that BUG_ON in a minute or
so, instead of an hour or so.  Was there a fix you made somewhere,
to avoid the BUG_ON(!mapping_empty) most of the time? but needs
more work. I looked around a little, but didn't find any.

I had hoped to work this out myself, and save us both some writing:
but better hand over to you, in the hope that you'll quickly guess
what's up, then I can try patches. I do like the no-nrexceptionals
series, but there's something still to be fixed.

Hugh
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org

WARNING: multiple messages have this Message-ID (diff)
From: Hugh Dickins <hughd@google.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Hugh Dickins <hughd@google.com>,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org
Subject: BUG_ON(!mapping_empty(&inode->i_data))
Date: Tue, 30 Mar 2021 18:30:22 -0700 (PDT)	[thread overview]
Message-ID: <alpine.LSU.2.11.2103301654520.2648@eggly.anvils> (raw)

Running my usual tmpfs kernel builds swapping load, on Sunday's rc4-mm1
mmotm (I never got to try rc3-mm1 but presume it behaved the same way),
I hit clear_inode()'s BUG_ON(!mapping_empty(&inode->i_data)); on two
machines, within an hour or few, repeatably though not to order.

The stack backtrace has always been clear_inode < ext4_clear_inode <
ext4_evict_inode < evict < dispose_list < prune_icache_sb <
super_cache_scan < do_shrink_slab < shrink_slab_memcg < shrink_slab <
shrink_node_memgs < shrink_node < balance_pgdat < kswapd.

ext4 is the disk filesystem I read the source to build from, and also
the filesystem I use on a loop device on a tmpfs file: I have not tried
with other filesystems, nor checked whether perhaps it happens always on
the loop one or always on the disk one.  I have not seen it happen with
tmpfs - probably because its inodes cannot be evicted by the shrinker
anyway; I have not seen it happen when "rm -rf" evicts ext4 or tmpfs
inodes (but suspect that may be down to timing, or less pressure).
I doubt it's a matter of filesystem: think it's an XArray thing.

Whenever I've looked at the XArray nodes involved, the root node
(shift 6) contained one or three (adjacent) pointers to empty shift
0 nodes, which each had offset and parent and array correctly set.
Is there some way in which empty nodes can get left behind, and so
fail eviction's mapping_empty() check?

I did wonder whether some might get left behind if xas_alloc() fails
(though probably the tree here is too shallow to show that).  Printks
showed that occasionally xas_alloc() did fail while testing (maybe at
memcg limit), but there was no correlation with the BUG_ONs.

I did wonder whether this is a long-standing issue, which your new
BUG_ON is the first to detect: so tried 5.12-rc5 clear_inode() with
a BUG_ON(!xa_empty(&inode->i_data.i_pages)) after its nrpages and
nrexceptional BUG_ONs.  The result there surprised me: I expected
it to behave the same way, but it hits that BUG_ON in a minute or
so, instead of an hour or so.  Was there a fix you made somewhere,
to avoid the BUG_ON(!mapping_empty) most of the time? but needs
more work. I looked around a little, but didn't find any.

I had hoped to work this out myself, and save us both some writing:
but better hand over to you, in the hope that you'll quickly guess
what's up, then I can try patches. I do like the no-nrexceptionals
series, but there's something still to be fixed.

Hugh

WARNING: multiple messages have this Message-ID (diff)
From: Hugh Dickins <hughd@google.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Hugh Dickins <hughd@google.com>,
	 linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	 linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org
Subject: BUG_ON(!mapping_empty(&inode->i_data))
Date: Tue, 30 Mar 2021 18:30:22 -0700 (PDT)	[thread overview]
Message-ID: <alpine.LSU.2.11.2103301654520.2648@eggly.anvils> (raw)

Running my usual tmpfs kernel builds swapping load, on Sunday's rc4-mm1
mmotm (I never got to try rc3-mm1 but presume it behaved the same way),
I hit clear_inode()'s BUG_ON(!mapping_empty(&inode->i_data)); on two
machines, within an hour or few, repeatably though not to order.

The stack backtrace has always been clear_inode < ext4_clear_inode <
ext4_evict_inode < evict < dispose_list < prune_icache_sb <
super_cache_scan < do_shrink_slab < shrink_slab_memcg < shrink_slab <
shrink_node_memgs < shrink_node < balance_pgdat < kswapd.

ext4 is the disk filesystem I read the source to build from, and also
the filesystem I use on a loop device on a tmpfs file: I have not tried
with other filesystems, nor checked whether perhaps it happens always on
the loop one or always on the disk one.  I have not seen it happen with
tmpfs - probably because its inodes cannot be evicted by the shrinker
anyway; I have not seen it happen when "rm -rf" evicts ext4 or tmpfs
inodes (but suspect that may be down to timing, or less pressure).
I doubt it's a matter of filesystem: think it's an XArray thing.

Whenever I've looked at the XArray nodes involved, the root node
(shift 6) contained one or three (adjacent) pointers to empty shift
0 nodes, which each had offset and parent and array correctly set.
Is there some way in which empty nodes can get left behind, and so
fail eviction's mapping_empty() check?

I did wonder whether some might get left behind if xas_alloc() fails
(though probably the tree here is too shallow to show that).  Printks
showed that occasionally xas_alloc() did fail while testing (maybe at
memcg limit), but there was no correlation with the BUG_ONs.

I did wonder whether this is a long-standing issue, which your new
BUG_ON is the first to detect: so tried 5.12-rc5 clear_inode() with
a BUG_ON(!xa_empty(&inode->i_data.i_pages)) after its nrpages and
nrexceptional BUG_ONs.  The result there surprised me: I expected
it to behave the same way, but it hits that BUG_ON in a minute or
so, instead of an hour or so.  Was there a fix you made somewhere,
to avoid the BUG_ON(!mapping_empty) most of the time? but needs
more work. I looked around a little, but didn't find any.

I had hoped to work this out myself, and save us both some writing:
but better hand over to you, in the hope that you'll quickly guess
what's up, then I can try patches. I do like the no-nrexceptionals
series, but there's something still to be fixed.

Hugh


             reply	other threads:[~2021-03-31  1:30 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-31  1:30 Hugh Dickins [this message]
2021-03-31  1:30 ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-03-31  1:30 ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-03-31  2:49 ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-03-31  2:49   ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-03-31 21:58   ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-03-31 21:58     ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-03-31 21:58     ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-01 17:06     ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-04-01 17:06       ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-04-02  3:13       ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-04-02  3:13         ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-04-02 13:27         ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-04-02 13:27           ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-04-02 17:04           ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-04-02 17:04             ` BUG_ON(!mapping_empty(&inode->i_data)) Matthew Wilcox
2021-04-02 20:24             ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-02 20:24               ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-02 20:24               ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-02 21:16               ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-02 21:16                 ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-02 21:16                 ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-30  4:16                 ` BUG_ON(!mapping_empty(&inode->i_data)) Andrew Morton
2021-04-30  4:16                   ` BUG_ON(!mapping_empty(&inode->i_data)) Andrew Morton
2021-04-30  5:41                   ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-30  5:41                     ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins
2021-04-30  5:41                     ` BUG_ON(!mapping_empty(&inode->i_data)) Hugh Dickins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.LSU.2.11.2103301654520.2648@eggly.anvils \
    --to=hughd@google.com \
    --cc=akpm@linux-foundation.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-nvdimm@lists.01.org \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.