From: Andrea Righi <andrea.righi@canonical.com>
To: Ard Biesheuvel <ardb@kernel.org>
Cc: "Jason A. Donenfeld" <Jason@zx2c4.com>,
Paolo Pisati <paolo.pisati@canonical.com>,
linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org,
Darren Hart <darren@os.amperecomputing.com>
Subject: Re: kernel 6.2 stuck at boot (efi_call_rts) on arm64
Date: Thu, 16 Mar 2023 13:49:58 +0100 [thread overview]
Message-ID: <ZBMQdgPepwa+VyAH@righiandr-XPS-13-7390> (raw)
In-Reply-To: <CAMj1kXF=8KoCnRmUyLCZmbfPTeOFQZBeudZuTeA0uHOv-1drFg@mail.gmail.com>
On Thu, Mar 16, 2023 at 01:43:32PM +0100, Ard Biesheuvel wrote:
> On Thu, 16 Mar 2023 at 13:41, Andrea Righi <andrea.righi@canonical.com> wrote:
> >
> > On Thu, Mar 16, 2023 at 01:38:30PM +0100, Ard Biesheuvel wrote:
> > > On Thu, 16 Mar 2023 at 13:21, Ard Biesheuvel <ardb@kernel.org> wrote:
> > > >
> > > > On Thu, 16 Mar 2023 at 12:34, Andrea Righi <andrea.righi@canonical.com> wrote:
> > > > >
> > > > > On Thu, Mar 16, 2023 at 11:18:21AM +0100, Ard Biesheuvel wrote:
> > > > > > On Thu, 16 Mar 2023 at 11:03, Andrea Righi <andrea.righi@canonical.com> wrote:
> > > > > > >
> > > > > > > On Thu, Mar 16, 2023 at 10:55:58AM +0100, Ard Biesheuvel wrote:
> > > > > > > > (cc Darren)
> > > > > > > >
> > > > > > > > On Thu, 16 Mar 2023 at 10:45, Andrea Righi <andrea.righi@canonical.com> wrote:
> > > > > > > > >
> > > > > > > > > On Thu, Mar 16, 2023 at 08:58:20AM +0100, Ard Biesheuvel wrote:
> > > > > > > > > > Hello Andrea,
> > > > > > > > > >
> > > > > > > > > > On Thu, 16 Mar 2023 at 08:54, Andrea Righi <andrea.righi@canonical.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hello,
> > > > > > > > > > >
> > > > > > > > > > > the latest v6.2.6 kernel fails to boot on some arm64 systems, the kernel
> > > > > > > > > > > gets stuck and never completes the boot. On the console I see this:
> > > > > > > > > > >
> > > > > > > > > > > [ 72.043484] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> > > > > > > > > > > [ 72.049571] rcu: 22-...0: (30 GPs behind) idle=b10c/1/0x4000000000000000 softirq=164/164 fqs=6443
> > > > > > > > > > > [ 72.058520] (detected by 28, t=15005 jiffies, g=449, q=174 ncpus=32)
> > > > > > > > > > > [ 72.064949] Task dump for CPU 22:
> > > > > > > > > > > [ 72.068251] task:kworker/u64:5 state:R running task stack:0 pid:447 ppid:2 flags:0x0000000a
> > > > > > > > > > > [ 72.078156] Workqueue: efi_rts_wq efi_call_rts
> > > > > > > > > > > [ 72.082595] Call trace:
> > > > > > > > > > > [ 72.085029] __switch_to+0xbc/0x100
> > > > > > > > > > > [ 72.088508] 0xffff80000fe83d4c
> > > > > > > > > > >
> > > > > > > > > > > After that, as a consequence, I start to get a lot of hung task timeout traces.
> > > > > > > > > > >
> > > > > > > > > > > I tried to bisect the problem and I found that the offending commit is
> > > > > > > > > > > this one:
> > > > > > > > > > >
> > > > > > > > > > > e7b813b32a42 ("efi: random: refresh non-volatile random seed when RNG is initialized")
> > > > > > > > > > >
> > > > > > > > > > > I've reverted this commit for now and everything works just fine, but I
> > > > > > > > > > > was wondering if the problem could be caused by a lack of entropy on
> > > > > > > > > > > these arm64 boxes or something else.
> > > > > > > > > > >
> > > > > > > > > > > Any suggestion? Let me know if you want me to do any specific test.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thanks for the report.
> > > > > > > > > >
> > > > > > > > > > This is most likely the EFI SetVariable() call going off into the
> > > > > > > > > > weeds and never returning.
> > > > > > > > > >
> > > > > > > > > > Is this an Ampere Altra system by any chance? Do you see it on
> > > > > > > > > > different types of hardware?
> > > > > > > > >
> > > > > > > > > This is: Ampere eMAG / Lenovo ThinkSystem HR330a.
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Could you check whether SetVariable works on this system? E.g. by
> > > > > > > > > > updating the EFI boot timeout (sudo efibootmgr -t <n>)?
> > > > > > > > >
> > > > > > > > > ubuntu@kuzzle:~$ sudo efibootmgr -t 10
> > > > > > > > > ^C^C^C^C
> > > > > > > > >
> > > > > > > > > ^ Stuck there, so it really looks like SetVariable is the problem.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Could you please share the output of
> > > > > > > >
> > > > > > > > dmidecode -s bios
> > > > > > > > dmidecode -s system-family
> > > > > > >
> > > > > > > $ sudo dmidecode -s bios-vendor
> > > > > > > LENOVO
> > > > > > > $ sudo dmidecode -s bios-version
> > > > > > > hve104r-1.15
> > > > > > > $ sudo dmidecode -s bios-release-date
> > > > > > > 02/26/2021
> > > > > > > $ sudo dmidecode -s bios-revision
> > > > > > > 1.15
> > > > > > > $ sudo dmidecode -s system-family
> > > > > > > Lenovo ThinkSystem HR330A/HR350A
> > > > > > >
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > Mind checking if this patch fixes your issue as well?
> > > > > >
> > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/commit/?h=altra-fix&id=77fa99dd4741456da85049c13ec31a148f5f5ac0
> > > > >
> > > > > Unfortunately this doesn't seem to be enough, I'm still getting the same
> > > > > problem also with this patch applied.
> > > > >
> > > >
> > > > Thanks for trying.
> > > >
> > > > How about the last 3 patches on this branch?
> > > >
> > > > https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=efi-smbios-altra-fix
> > >
> > > Actually, that may not match your hardware.
> > >
> > > Does your kernel log have a line like
> > >
> > > SMCCC: SOC_ID: ID = jep106:036b:0019 Revision = 0x00000102
> > >
> > > ?
> >
> > $ sudo dmesg | grep "SMCCC: SOC_ID"
> > [ 5.320782] SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
> >
>
> Thanks. Could you share the entire dmidecode output somewhere? Or at
> least the type 4 record(s)?
Sure, here's the full output of dmidecode:
https://pastebin.ubuntu.com/p/4ZmKmP2xTm/
-Andrea
next prev parent reply other threads:[~2023-03-16 12:50 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-16 7:54 kernel 6.2 stuck at boot (efi_call_rts) on arm64 Andrea Righi
2023-03-16 7:58 ` Ard Biesheuvel
2023-03-16 9:45 ` Andrea Righi
2023-03-16 9:55 ` Ard Biesheuvel
2023-03-16 10:03 ` Andrea Righi
2023-03-16 10:18 ` Ard Biesheuvel
2023-03-16 11:33 ` Andrea Righi
2023-03-16 12:21 ` Ard Biesheuvel
2023-03-16 12:38 ` Ard Biesheuvel
2023-03-16 12:41 ` Andrea Righi
2023-03-16 12:43 ` Ard Biesheuvel
2023-03-16 12:49 ` Andrea Righi [this message]
2023-03-16 13:45 ` Ard Biesheuvel
2023-03-16 13:46 ` Ard Biesheuvel
2023-03-16 13:50 ` Andrea Righi
2023-03-16 13:53 ` Ard Biesheuvel
2023-03-16 13:59 ` Andrea Righi
2023-03-16 14:06 ` Ard Biesheuvel
2023-03-16 14:08 ` Ard Biesheuvel
2023-03-16 14:25 ` Andrea Righi
2023-03-16 17:52 ` Andrea Righi
2023-03-16 18:55 ` Ard Biesheuvel
2023-03-16 18:57 ` Andrea Righi
2023-03-16 22:28 ` Darren Hart
2023-03-18 10:35 ` Ard Biesheuvel
2023-03-20 18:00 ` Darren Hart
2023-04-13 20:24 ` Andrea Righi
2023-04-17 22:05 ` Darren Hart
2023-04-18 5:42 ` Andrea Righi
2023-03-16 9:45 ` Linux regression tracking #adding (Thorsten Leemhuis)
2023-04-05 12:50 ` Linux regression tracking #update (Thorsten Leemhuis)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZBMQdgPepwa+VyAH@righiandr-XPS-13-7390 \
--to=andrea.righi@canonical.com \
--cc=Jason@zx2c4.com \
--cc=ardb@kernel.org \
--cc=darren@os.amperecomputing.com \
--cc=linux-efi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=paolo.pisati@canonical.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).