From: Bruno Goncalves <bgoncalv@redhat.com>
To: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: linux-kernel@vger.kernel.org, CKI Project <cki-project@redhat.com>
Subject: Re: Panic on ppc64le using kernel 5.13.0-rc3
Date: Thu, 10 Jun 2021 17:14:24 +0200 [thread overview]
Message-ID: <CA+QYu4oFGkP1G+9TqGtqffgOEBycSMKtFbV-1X+kL4NeTyEegg@mail.gmail.com> (raw)
In-Reply-To: <d13db73d-0806-00cd-ff84-5f5b03ffbef6@rasmusvillemoes.dk>
On Thu, Jun 10, 2021 at 3:02 PM Rasmus Villemoes
<linux@rasmusvillemoes.dk> wrote:
>
> On 10/06/2021 13.47, Bruno Goncalves wrote:
> > Hello,
> >
> > We've observed in some cases kernel panic when trying to boot on
> > ppc64le using a kernel based on 5.13.0-rc3. We are not sure if it
> > could be related to patch
> > https://lore.kernel.org/lkml/20210313212528.2956377-2-linux@rasmusvillemoes.dk/
> >
>
> Thanks for the report. It's possible, but I'll need some help from you
> to get more info.
>
> First, can you send me the .config?
The .config is on
https://s3.us-east-1.amazonaws.com/arr-cki-prod-datawarehouse-public/datawarehouse-public/2021/06/09/317881801/build_ppc64le_redhat:1332368174/kernel-block-ppc64le-d3f02e52f5548006f04358d407bbb7fe51255c41.config
>
> >
> > [ 1.516075] wait_for_initramfs() called before rootfs_initcalls
>
> This is likely because you have CONFIG_UEVENT_HELPER_PATH set to some
> non-empty path (/sbin/hotplug perhaps). This did get reported once before:
>
CONFIG_UEVENT_HELPER_PATH is not set. In the .config we have "#
CONFIG_UEVENT_HELPER is not set"
> https://lore.kernel.org/lkml/45556f52-cd2f-5512-ba65-81e4acee21ff@rasmusvillemoes.dk/
>
> I think I should go and prepare a patch that moves the
> usermodehelper_enable() call to after initramfs unpacking has been
> initiated.
>
> But until then, can you check if you do have UEVENT_HELPER_PATH set, and
> if so, does changing it to the empty string make a change wrt this crash?
>
>
> > [ 1.559757] PCI: CLS 128 bytes, default 128
> > [ 1.560150] Trying to unpack rootfs image as initramfs...
>
> OK, so now we got to populate_rootfs() and have kicked off a worker to
> do the unpacking. Meanwhile, PID1 goes on to do other initcalls.
>
> ...
>
> > [ 1.764430] Initramfs unpacking failed: no cpio magic
>
> Whoa, that's not good. Did something scramble over the initramfs memory
> while it was being unpacked? It's been .2 seconds since the start of the
> unpacking, so it's unlikely the very beginning of the initramfs is corrupt.
>
> Can you try booting with initramfs_async=0 on the command line and see
> if the kernel still crashes?
We are not able to reproduce it 100% of the time, but sure I can try
with this option and see what happens.
We've also seen:
Initramfs unpacking failed: junk within compressed archive
This can be seen on the other 2 console logs that I provided the link to.
Bruno
>
> > [ 1.766204] Freeing initrd memory: 18176K
> ...
>
>
> > [ 1.787649] Run /init as init process
> > [ 1.787793] Failed to execute /init (error -2)
> > [ 1.787801] Run /sbin/init as init process
> > [ 1.787842] Run /etc/init as init process
> > [ 1.787880] Run /bin/init as init process
> > [ 1.787921] Run /bin/sh as init process
> > [ 1.787978] Kernel panic - not syncing: No working init found. Try
>
> Yeah, well, this is expected when unpacking the initramfs failed.
>
> So I think the problem is the "no cpio magic", i.e. the initramfs got
> corrupted somehow. But I don't have any idea why that would happen -
> freeing the initramfs memory only happens after unpacking is done
> (naturally...).
>
> Rasmus
>
next prev parent reply other threads:[~2021-06-10 15:14 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-10 11:47 Panic on ppc64le using kernel 5.13.0-rc3 Bruno Goncalves
2021-06-10 13:02 ` Rasmus Villemoes
2021-06-10 15:14 ` Bruno Goncalves [this message]
2021-06-11 7:13 ` Rasmus Villemoes
2021-06-11 15:06 ` Bruno Goncalves
2021-06-11 21:49 ` Rasmus Villemoes
2021-06-14 5:47 ` Bruno Goncalves
2021-06-14 8:03 ` Bruno Goncalves
2021-06-14 8:11 ` Rasmus Villemoes
2021-06-15 3:20 ` Dave Young
2021-06-15 5:13 ` Dave Young
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+QYu4oFGkP1G+9TqGtqffgOEBycSMKtFbV-1X+kL4NeTyEegg@mail.gmail.com \
--to=bgoncalv@redhat.com \
--cc=cki-project@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@rasmusvillemoes.dk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).