linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Linux regression tracking #update (Thorsten Leemhuis)"  <regressions@leemhuis.info>
To: "Richard W.M. Jones" <rjones@redhat.com>,
	Aaron Thompson <dev@aaront.org>,
	Peter Zijlstra <peterz@infradead.org>
Cc: linux-kernel@vger.kernel.org,
	Linux kernel regressions list <regressions@lists.linux.dev>
Subject: Re: printk.time causes rare kernel boot hangs
Date: Sun, 18 Jun 2023 12:25:35 +0200	[thread overview]
Message-ID: <1413638e-9614-056e-cfc5-5f9c8bf8af00@leemhuis.info> (raw)
In-Reply-To: <172193eb-14cc-549e-2953-6749a3a4c502@leemhuis.info>

On 13.06.23 16:07, Linux regression tracking #adding (Thorsten Leemhuis)
wrote:
> 
> On 13.06.23 15:41, Richard W.M. Jones wrote:
>> [Being tracked in this bug which contains much more detail:
>> https://gitlab.com/qemu-project/qemu/-/issues/1696 ]
>>
>> Recent kernels hang rarely when booted on qemu.  Usually you need to
>> boot 100s or 1,000s of times to see the hang, compared to 292,612 [sic]
>> successful boots which I was able to do before the problematic commit.
>>
>> A reproducer (you'll probably need to use Fedora) is:
>>
>>   $ while guestfish -a /dev/null -v run >& /tmp/log; do echo -n . ; done
>>
>> You will need to leave it running for probably several hours, and
>> examine the /tmp/log file at the end.
>>
>> I tracked this down to the following commit:
>>
>>   commit f31dcb152a3d0816e2f1deab4e64572336da197d
>>   Author: Aaron Thompson <dev@aaront.org>
>>   Date:   Thu Apr 13 17:50:12 2023 +0000
>>
>>     sched/clock: Fix local_clock() before sched_clock_init()
>>     
>>     Have local_clock() return sched_clock() if sched_clock_init() has not
>>     yet run. sched_clock_cpu() has this check but it was not included in the
>>     new noinstr implementation of local_clock().
>>
>>   (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f31dcb152a3d0816e2f1deab4e64572336da197d)
>>
>> Reverting this commit fixes the problem.
>>
>> I don't know _why_ this commit is wrong, but can we revert it as it
>> causes serious problems with libguestfs hanging randomly.
>>
>> Or if there's anything you want me to try out then let me know,
>> because I can reproduce the problem locally quite easily.
> 
> Thanks for the report. To be sure the issue doesn't fall through the
> cracks unnoticed, I'm adding it to regzbot, the Linux kernel regression
> tracking bot:
> 
> #regzbot ^introduced f31dcb152a3d0816e2f1deab4e64572336da197d
> #regzbot title sched/clock: printk.time causes rare kernel boot hangs
> #regzbot ignore-activity

#regzbot fix: tick/common: Align tick period during sched_timer setup
#regzbot monitor:
https://lore.kernel.org/all/12c6f9a3-d087-b824-0d05-0d18c9bc1bf3@amazon.com/
#regzbot ignore-activity

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
That page also explains what to do if mails like this annoy you.

  reply	other threads:[~2023-06-18 10:25 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-06-13 13:41 printk.time causes rare kernel boot hangs Richard W.M. Jones
2023-06-13 14:07 ` Linux regression tracking #adding (Thorsten Leemhuis)
2023-06-18 10:25   ` Linux regression tracking #update (Thorsten Leemhuis) [this message]
2023-06-14  9:21 ` Peter Zijlstra
2023-06-14  9:45   ` Richard W.M. Jones
2023-06-14 10:30     ` Richard W.M. Jones
2023-06-14 10:39       ` Richard W.M. Jones
2023-06-14 11:35         ` Peter Zijlstra
2023-06-14 11:43           ` Richard W.M. Jones
2023-06-14 12:37           ` Richard W.M. Jones
2023-06-14 12:53           ` Peter Zijlstra
2023-06-14 13:03             ` Richard W.M. Jones
2023-06-14 13:09               ` Peter Zijlstra
2023-06-14 14:53                 ` Peter Zijlstra
2023-06-14 15:07                   ` Richard W.M. Jones
2023-06-14 15:19                     ` Peter Zijlstra
2023-06-14 15:22                       ` Richard W.M. Jones
2023-06-14 15:31                       ` Peter Zijlstra
2023-06-14 15:50                         ` Richard W.M. Jones
2023-06-14 17:34                           ` Richard W.M. Jones
2023-06-15  7:40                             ` Alexandre Belloni
2023-06-15  7:48                               ` Richard W.M. Jones
2023-06-14 11:20       ` Peter Zijlstra
2023-06-14 11:16     ` Peter Zijlstra
2023-06-14 11:22       ` Richard W.M. Jones
2023-06-14 11:26         ` Richard W.M. Jones
2023-06-15 11:04           ` YiFei Zhu
2023-06-15 11:29             ` Richard W.M. Jones
2023-06-15 11:31             ` Richard W.M. Jones
2023-06-15 12:20               ` Dr. David Alan Gilbert
2023-06-15 12:21               ` Richard W.M. Jones
2023-06-15 12:23                 ` Richard W.M. Jones

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1413638e-9614-056e-cfc5-5f9c8bf8af00@leemhuis.info \
    --to=regressions@leemhuis.info \
    --cc=dev@aaront.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=regressions@lists.linux.dev \
    --cc=rjones@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).