From: Sander Eikelenboom <linux@eikelenboom.it>
To: Rasmus Villemoes <linux@rasmusvillemoes.dk>,
Linus Torvalds <torvalds@linux-foundation.org>
Cc: linux-kernel <linux-kernel@vger.kernel.org>,
"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: Linux 5.13-rc6 regression to 5.12.x: kernel OOM and panic during kernel boot in low memory Xen VM's (256MB assigned memory).
Date: Thu, 17 Jun 2021 20:02:27 +0200 [thread overview]
Message-ID: <0b12f27b-1109-b621-c969-10814b2c1c2f@eikelenboom.it> (raw)
In-Reply-To: <9108c22e-3521-9e24-6124-7776d947b788@rasmusvillemoes.dk>
On 17/06/2021 17:37, Rasmus Villemoes wrote:
> On 17/06/2021 17.01, Linus Torvalds wrote:
>> On Thu, Jun 17, 2021 at 2:26 AM Sander Eikelenboom <linux@eikelenboom.it> wrote:
>>>
>>> I just tried to upgrade and test the linux kernel going from the 5.12 kernel series to 5.13-rc6 on my homeserver with Xen, but ran in some trouble.
>>>
>>> Some VM's boot fine (with more than 256MB memory assigned), but the smaller (memory wise) PVH ones crash during kernel boot due to OOM.
>>> Booting VM's with 5.12(.9) kernel still works fine, also when dom0 is running 5.13-rc6 (but it has more memory assigned, so that is not unexpected).
>>
>> Adding Rasmus to the cc, because this looks kind of like the async
>> roofs population thing that caused some other oom issues too.
>
> Yes, that looks like the same issue.
>
>> Rasmus? Original report here:
>>
>> https://lore.kernel.org/lkml/ee8bf04c-6e55-1d9b-7bdb-25e6108e8e1e@eikelenboom.it/
>>
>> I do find it odd that we'd be running out of memory so early..
>
> Indeed. It would be nice to know if these also reproduce with
> initramfs_async=0 on the command line.
>
> But what is even more curious is that in the other report
> (https://lore.kernel.org/lkml/20210607144419.GA23706@xsang-OptiPlex-9020/),
> it seemed to trigger with _more_ memory - though I may be misreading
> what Oliver was telling me:
>
>> please be noted that we use 'vmalloc=512M' for both parent and this
> commit.
>> since it's ok on parent but oom on this commit, we want to send this
> report
>> to show the potential problem of the commit on some cases.
>>
>> we also tested by changing to use 'vmalloc=128M', it will succeed.
>
> Those tests were done in a VM with 16G memory, and then he also wrote
>
>> we also tried to follow exactly above steps to test on
>> some local machine (8G memory), but cannot reproduce.
>
> Are there some special rules for what memory pools PID1 versus the
> kworker threads can dip into?
>
>
> Side note: I also had a ppc64 report with different symptoms (the
> initramfs was corrupted), but that turned out to also reproduce with
> e7cb072eb98 reverted, so that is likely unrelated. But just FTR that
> thread is here:
> https://lore.kernel.org/lkml/CA+QYu4qxf2CYe2gC6EYnOHXPKS-+cEXL=MnUvqRFaN7W1i6ahQ@mail.gmail.com/
>
> Rasmus
>
I choose to first finish the bisection attempt, not so suprising it ends up with:
e7cb072eb988e46295512617c39d004f9e1c26f8 is the first bad commit
So at least that link is confirmed.
I also checked out booting with "initramfs_async=0" and now the guest boots with the 5.13-rc6-ish kernel which fails without that.
--
Sander
next prev parent reply other threads:[~2021-06-17 18:02 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-17 9:26 Linux 5.13-rc6 regression to 5.12.x: kernel OOM and panic during kernel boot in low memory Xen VM's (256MB assigned memory) Sander Eikelenboom
2021-06-17 10:30 ` Juergen Gross
2021-06-17 12:35 ` Sander Eikelenboom
2021-06-17 15:01 ` Linus Torvalds
2021-06-17 15:37 ` Rasmus Villemoes
2021-06-17 15:37 ` Rasmus Villemoes
2021-06-17 18:02 ` Sander Eikelenboom [this message]
2021-06-17 19:39 ` Sander Eikelenboom
2021-06-18 1:06 ` Sander Eikelenboom
2021-06-21 16:54 ` Rasmus Villemoes
2021-06-21 21:36 ` Sander Eikelenboom
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0b12f27b-1109-b621-c969-10814b2c1c2f@eikelenboom.it \
--to=linux@eikelenboom.it \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@rasmusvillemoes.dk \
--cc=torvalds@linux-foundation.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).