linux-next.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@kernel.org>
To: Arnd Bergmann <arnd@arndb.de>
Cc: Naresh Kamboju <naresh.kamboju@linaro.org>,
	"Linux F2FS DEV,
	Mailing List"  <linux-f2fs-devel@lists.sourceforge.net>,
	linux-ext4 <linux-ext4@vger.kernel.org>,
	linux-block <linux-block@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	open list <linux-kernel@vger.kernel.org>,
	Linux-Next Mailing List <linux-next@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>,
	Andreas Dilger <adilger.kernel@dilger.ca>,
	Jaegeuk Kim <jaegeuk@kernel.org>, Theodore Ts'o <tytso@mit.edu>,
	Chao Yu <chao@kernel.org>, Hugh Dickins <hughd@google.com>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Matthew Wilcox <willy@infradead.org>,
	Chao Yu <yuchao0@huawei.com>,
	lkft-triage@lists.linaro.org
Subject: Re: mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page
Date: Tue, 19 May 2020 10:45:35 +0200	[thread overview]
Message-ID: <20200519084535.GG32497@dhcp22.suse.cz> (raw)
In-Reply-To: <CAK8P3a2T_j-Ynvhsqe_FCqS2-ZdLbo0oMbHhHChzMbryE0izAQ@mail.gmail.com>

On Tue 19-05-20 10:11:25, Arnd Bergmann wrote:
> On Tue, May 19, 2020 at 9:52 AM Michal Hocko <mhocko@kernel.org> wrote:
> >
> > On Mon 18-05-20 19:40:55, Naresh Kamboju wrote:
> > > Thanks for looking into this problem.
> > >
> > > On Sat, 2 May 2020 at 02:28, Andrew Morton <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Fri, 1 May 2020 18:08:28 +0530 Naresh Kamboju <naresh.kamboju@linaro.org> wrote:
> > > >
> > > > > mkfs -t ext4 invoked oom-killer on i386 kernel running on x86_64 device
> > > > > and started happening on linux -next master branch kernel tag next-20200430
> > > > > and next-20200501. We did not bisect this problem.
> > [...]
> > > Creating journal (131072 blocks): [   31.251333] mkfs.ext4 invoked
> > > oom-killer: gfp_mask=0x101cc0(GFP_USER|__GFP_WRITE), order=0,
> > > oom_score_adj=0
> > [...]
> > > [   31.500943] DMA free:187396kB min:22528kB low:28160kB high:33792kB
> > > reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB
> > > active_file:4736kB inactive_file:431688kB unevictable:0kB
> > > writepending:62020kB present:783360kB managed:668264kB mlocked:0kB
> > > kernel_stack:888kB pagetables:0kB bounce:0kB free_pcp:880kB
> > > local_pcp:216kB free_cma:163840kB
> >
> > This is really unexpected. You are saying this is a regular i386 and DMA
> > should be bottom 16MB while yours is 780MB and the rest of the low mem
> > is in the Normal zone which is completely missing here. How have you got
> > to that configuration? I have to say I haven't seen anything like that
> > on i386.
> 
> I think that line comes from an ARM32 beaglebone-X15 machine showing
> the same symptom. The i386 line from the log file that Naresh linked to at
> https://lkft.validation.linaro.org/scheduler/job/1406110#L1223  is less
> unusual:

OK, that makes more sense! At least for the memory layout.
 
> [   34.931663] Node 0 active_anon:21464kB inactive_anon:8688kB
> active_file:16604kB inactive_file:849976kB unevictable:0kB
> isolated(anon):0kB isolated(file):0kB mapped:25284kB dirty:58952kB
> writeback:27772kB shmem:8944kB writeback_tmp:0kB unstable:0kB
> all_unreclaimable? yes
> [   34.955523] DMA free:3356kB min:68kB low:84kB high:100kB
> reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB
> active_file:0kB inactive_file:11964kB unevictable:0kB
> writepending:11980kB present:15964kB managed:15876kB mlocked:0kB
> kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB
> free_cma:0kB
> [   34.983385] lowmem_reserve[]: 0 825 1947 825
> [   34.987678] Normal free:3948kB min:7732kB low:8640kB high:9548kB
> reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB
> active_file:1096kB inactive_file:786400kB unevictable:0kB
> writepending:65432kB present:884728kB managed:845576kB mlocked:0kB
> kernel_stack:1112kB pagetables:0kB bounce:0kB free_pcp:2908kB
> local_pcp:500kB free_cma:0kB

The lowmem is really low (way below the min watermark so even memory
reserves for high priority and atomic requests are depleted. There is
still 786MB of inactive page cache to be reclaimed. It doesn't seem to
be dirty or under the writeback but it still might be pinned by the
filesystem. I would suggest watching vmscan reclaim tracepoints and
check why the reclaim fails to reclaim anything.
-- 
Michal Hocko
SUSE Labs

  reply	other threads:[~2020-05-19  8:45 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-01 12:38 mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page Naresh Kamboju
2020-05-01 20:58 ` Andrew Morton
2020-05-18 14:10   ` Naresh Kamboju
2020-05-19  7:52     ` Michal Hocko
2020-05-19  8:11       ` Arnd Bergmann
2020-05-19  8:45         ` Michal Hocko [this message]
2020-05-20 11:56           ` Naresh Kamboju
2020-05-20 17:59             ` Naresh Kamboju
2020-05-20 19:09               ` Chris Down
2020-05-21  9:22                 ` Naresh Kamboju
2020-05-21  9:35                   ` Arnd Bergmann
2020-05-21  9:55                 ` Michal Hocko
2020-05-21 10:41                   ` Naresh Kamboju
2020-05-21 10:58                     ` Michal Hocko
2020-05-21 12:24                       ` Hugh Dickins
2020-05-21 12:44                         ` Michal Hocko
2020-05-21 19:17                           ` Johannes Weiner
2020-05-21 20:06                             ` Hugh Dickins
2020-05-21 21:58                               ` Johannes Weiner
2020-05-21 23:35                                 ` Hugh Dickins
2020-05-28 14:59                                 ` Michal Hocko
2020-05-21 16:34                   ` Michal Hocko
2020-05-21 19:00                     ` Naresh Kamboju
2020-05-21 20:53                       ` Naresh Kamboju
2020-05-28 15:03                         ` Michal Hocko
2020-05-28 16:17                           ` Naresh Kamboju
2020-05-28 16:41                             ` Chris Down
2020-05-29  1:50                               ` Yafang Shao
2020-05-29  1:56                                 ` Chris Down
2020-05-29  9:49                                   ` Michal Hocko
2020-06-11  9:55                                     ` Michal Hocko
2020-06-12  9:43                                       ` Naresh Kamboju
2020-06-12 12:09                                         ` Michal Hocko
2020-06-17 13:37                     ` Naresh Kamboju
2020-06-17 13:57                       ` Chris Down
2020-06-17 14:11                         ` Michal Hocko
2020-06-17 15:53                           ` Naresh Kamboju
2020-06-17 16:06                             ` Michal Hocko
2020-06-17 20:13                               ` Naresh Kamboju
2020-06-17 21:09                                 ` Chris Down
2020-06-18  1:43                                   ` Yafang Shao
2020-06-18 12:37                                     ` Chris Down
2020-06-18 12:41                                       ` Michal Hocko
2020-06-18 12:49                                         ` Chris Down
2020-06-18 14:59                                       ` Yafang Shao
2020-06-17 13:59                       ` Michal Hocko
2020-06-17 14:08                         ` Chris Down
2020-05-21  2:39               ` Yafang Shao
2020-05-21  8:58                 ` Naresh Kamboju
2020-05-21  9:47                   ` Yafang Shao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200519084535.GG32497@dhcp22.suse.cz \
    --to=mhocko@kernel.org \
    --cc=aarcange@redhat.com \
    --cc=adilger.kernel@dilger.ca \
    --cc=akpm@linux-foundation.org \
    --cc=arnd@arndb.de \
    --cc=chao@kernel.org \
    --cc=hughd@google.com \
    --cc=jaegeuk@kernel.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-f2fs-devel@lists.sourceforge.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-next@vger.kernel.org \
    --cc=lkft-triage@lists.linaro.org \
    --cc=naresh.kamboju@linaro.org \
    --cc=tytso@mit.edu \
    --cc=willy@infradead.org \
    --cc=yuchao0@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).