From: Guenter Roeck <linux@roeck-us.net>
To: Joel Fernandes <joel@joelfernandes.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, patches@lists.linux.dev,
linux-kernel@vger.kernel.org, torvalds@linux-foundation.org,
akpm@linux-foundation.org, shuah@kernel.org,
patches@kernelci.org, lkft-triage@lists.linaro.org,
pavel@denx.de, jonathanh@nvidia.com, f.fainelli@gmail.com,
sudipm.mukherjee@gmail.com, srw@sladewatkins.net, rwarsow@gmx.de,
conor@kernel.org, paulmck@kernel.org
Subject: Re: [PATCH 5.15 00/92] 5.15.126-rc1 review
Date: Wed, 9 Aug 2023 13:38:19 -0700 [thread overview]
Message-ID: <6b05a082-41a7-f0cf-c0a4-1cced8d5a230@roeck-us.net> (raw)
In-Reply-To: <20230809201413.GA3374446@google.com>
On 8/9/23 13:14, Joel Fernandes wrote:
> On Wed, Aug 09, 2023 at 12:25:48PM -0700, Guenter Roeck wrote:
>> On Wed, Aug 09, 2023 at 02:35:59PM -0400, Joel Fernandes wrote:
>>> On Wed, Aug 9, 2023 at 12:18 PM Guenter Roeck <linux@roeck-us.net> wrote:
>>>>
>>>> On 8/9/23 06:53, Joel Fernandes wrote:
>>>>> On Wed, Aug 09, 2023 at 12:40:36PM +0200, Greg Kroah-Hartman wrote:
>>>>>> This is the start of the stable review cycle for the 5.15.126 release.
>>>>>> There are 92 patches in this series, all will be posted as a response
>>>>>> to this one. If anyone has any issues with these being applied, please
>>>>>> let me know.
>>>>>>
>>>>>> Responses should be made by Fri, 11 Aug 2023 10:36:10 +0000.
>>>>>> Anything received after that time might be too late.
>>>>>>
>>>>>> The whole patch series can be found in one patch at:
>>>>>> https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.15.126-rc1.gz
>>>>>> or in the git tree and branch at:
>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.15.y
>>>>>> and the diffstat can be found below.
>>>>>
>>>>> Not necesscarily new with 5.15 stable but 3 of the 19 rcutorture scenarios
>>>>> hang with this -rc: TREE04, TREE07, TASKS03.
>>>>>
>>>>> 5.15 has a known stop machine issue where it hangs after 1.5 hours with cpu
>>>>> hotplug rcutorture testing. Me and tglx are continuing to debug this. The
>>>>> issue does not show up on anything but 5.15 stable kernels and neither on
>>>>> mainline.
>>>>>
>>>>
>>>> Do you by any have a crash pattern that we could possibly use to find the crash
>>>> in ChromeOS crash logs ? No idea if that would help, but it could provide some
>>>> additional data points.
>>>
>>> The pattern shows as a hard hang, the system is unresponsive and all CPUs
>>> are stuck in stop_machine. Sometimes it recovers on its own from the
>>> hang and then RCU immediately gives stall warnings. It takes 1.5 hour
>>> to reproduce and sometimes never happens for several hours.
>>>
>>> It appears related to CPU hotplug since gdb showed me most of the CPUs
>>> are spinning in multi_cpu_stop() / stop machine after the hang.
>>>
>>
>> Hmm, we do see lots of soft lockups with multi_cpu_stop() in the backtrace,
>> but not with v5.15.y but with v5.4.y. The actual hang is in stop_machine_yield().
>
> Interesting. It looks similar as far as the stack dump in gdb goes, here are
> the stacks I dumped with the hang I referred to:
> https://paste.debian.net/1288308/
>
That link gives me "Entry not found".
Guenter
next prev parent reply other threads:[~2023-08-09 20:39 UTC|newest]
Thread overview: 121+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-09 10:40 [PATCH 5.15 00/92] 5.15.126-rc1 review Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 01/92] io_uring: gate iowait schedule on having pending requests Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 02/92] perf: Fix function pointer case Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 03/92] net/mlx5: Free irqs only on shutdown callback Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 04/92] arm64: errata: Add workaround for TSB flush failures Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 05/92] arm64: errata: Add detection for TRBE write to out-of-range Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 06/92] iommu/arm-smmu-v3: Work around MMU-600 erratum 1076982 Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 07/92] iommu/arm-smmu-v3: Document MMU-700 erratum 2812531 Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 08/92] iommu/arm-smmu-v3: Add explicit feature for nesting Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 09/92] iommu/arm-smmu-v3: Document nesting-related errata Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 10/92] arm64: dts: imx8mn-var-som: add missing pull-up for onboard PHY reset pinmux Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 11/92] firmware: arm_scmi: Fix chan_free cleanup on SMC Greg Kroah-Hartman
2023-08-09 19:33 ` Florian Fainelli
2023-08-10 8:45 ` Sudeep Holla
2023-08-09 10:40 ` [PATCH 5.15 12/92] word-at-a-time: use the same return type for has_zero regardless of endianness Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 13/92] KVM: s390: fix sthyi error handling Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 14/92] wifi: cfg80211: Fix return value in scan logic Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 15/92] net/mlx5: DR, fix memory leak in mlx5dr_cmd_create_reformat_ctx Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 16/92] net/mlx5e: fix return value check in mlx5e_ipsec_remove_trailer() Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 17/92] bpf: Add length check for SK_DIAG_BPF_STORAGE_REQ_MAP_FD parsing Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 18/92] rtnetlink: let rtnl_bridge_setlink checks IFLA_BRIDGE_MODE length Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 19/92] net: dsa: fix value check in bcm_sf2_sw_probe() Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 20/92] perf test uprobe_from_different_cu: Skip if there is no gcc Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 21/92] net: sched: cls_u32: Fix match key mis-addressing Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 22/92] mISDN: hfcpci: Fix potential deadlock on &hc->lock Greg Kroah-Hartman
2023-08-09 10:40 ` [PATCH 5.15 23/92] qed: Fix kernel-doc warnings Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 24/92] qed: Fix scheduling in a tasklet while getting stats Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 25/92] net: annotate data-races around sk->sk_max_pacing_rate Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 26/92] net: add missing READ_ONCE(sk->sk_rcvlowat) annotation Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 27/92] net: add missing READ_ONCE(sk->sk_sndbuf) annotation Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 28/92] net: add missing READ_ONCE(sk->sk_rcvbuf) annotation Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 29/92] net: add missing data-race annotations around sk->sk_peek_off Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 30/92] net: add missing data-race annotation for sk_ll_usec Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 31/92] net/sched: taprio: Limit TCA_TAPRIO_ATTR_SCHED_CYCLE_TIME to INT_MAX Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 32/92] bpf, cpumap: Handle skb as well when clean up ptr_ring Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 33/92] net/sched: cls_u32: No longer copy tcf_result on update to avoid use-after-free Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 34/92] net/sched: cls_fw: " Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 35/92] net/sched: cls_route: " Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 36/92] bpf: sockmap: Remove preempt_disable in sock_map_sk_acquire Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 37/92] net: ll_temac: Switch to use dev_err_probe() helper Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 38/92] net: ll_temac: fix error checking of irq_of_parse_and_map() Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 39/92] net: korina: handle clk prepare error in korina_probe() Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 40/92] net: netsec: Ignore phy-mode on SynQuacer in DT mode Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 41/92] net: dcb: choose correct policy to parse DCB_ATTR_BCN Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 42/92] s390/qeth: Dont call dev_close/dev_open (DOWN/UP) Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 43/92] ip6mr: Fix skb_under_panic in ip6mr_cache_report() Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 44/92] vxlan: Fix nexthop hash size Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 45/92] net/mlx5: fs_core: Make find_closest_ft more generic Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 46/92] net/mlx5: fs_core: Skip the FTs in the same FS_TYPE_PRIO_CHAINS fs_prio Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 47/92] prestera: fix fallback to previous version on same major version Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 48/92] tcp_metrics: fix addr_same() helper Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 49/92] tcp_metrics: annotate data-races around tm->tcpm_stamp Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 50/92] tcp_metrics: annotate data-races around tm->tcpm_lock Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 51/92] tcp_metrics: annotate data-races around tm->tcpm_vals[] Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 52/92] tcp_metrics: annotate data-races around tm->tcpm_net Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 53/92] tcp_metrics: fix data-race in tcpm_suck_dst() vs fastopen Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 54/92] scsi: zfcp: Defer fc_rport blocking until after ADISC response Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 55/92] scsi: storvsc: Limit max_sectors for virtual Fibre Channel devices Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 56/92] libceph: fix potential hang in ceph_osdc_notify() Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 57/92] USB: zaurus: Add ID for A-300/B-500/C-700 Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 58/92] ceph: defer stopping mdsc delayed_work Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 59/92] firmware: arm_scmi: Drop OF node reference in the transport channel setup Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 60/92] exfat: use kvmalloc_array/kvfree instead of kmalloc_array/kfree Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 61/92] exfat: release s_lock before calling dir_emit() Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 62/92] mtd: spinand: toshiba: Fix ecc_get_status Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 63/92] mtd: rawnand: meson: fix OOB available bytes for ECC Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 64/92] arm64: dts: stratix10: fix incorrect I2C property for SCL signal Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 65/92] net: tun_chr_open(): set sk_uid from current_fsuid() Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 66/92] net: tap_open(): " Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 67/92] wifi: mt76: mt7615: do not advertise 5 GHz on first phy of MT7615D (DBDC) Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 68/92] rbd: prevent busy loop when requesting exclusive lock Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 69/92] bpf: Disable preemption in bpf_event_output Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 70/92] open: make RESOLVE_CACHED correctly test for O_TMPFILE Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 71/92] drm/ttm: check null pointer before accessing when swapping Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 72/92] bpf, cpumap: Make sure kthread is running before map update returns Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 73/92] file: reinstate f_pos locking optimization for regular files Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 74/92] fs/ntfs3: Use __GFP_NOWARN allocation at ntfs_load_attr_list() Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 75/92] fs/sysv: Null check to prevent null-ptr-deref bug Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 76/92] Bluetooth: L2CAP: Fix use-after-free in l2cap_sock_ready_cb Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 77/92] net: usbnet: Fix WARNING in usbnet_start_xmit/usb_submit_urb Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 78/92] fs: Protect reconfiguration of sb read-write from racing writes Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 79/92] ext2: Drop fragment support Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 80/92] mtd: rawnand: omap_elm: Fix incorrect type in assignment Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 81/92] mtd: rawnand: rockchip: fix oobfree offset and description Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 82/92] mtd: rawnand: rockchip: Align hwecc vs. raw page helper layouts Greg Kroah-Hartman
2023-08-09 10:41 ` [PATCH 5.15 83/92] mtd: rawnand: fsl_upm: Fix an off-by one test in fun_exec_op() Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 84/92] powerpc/mm/altmap: Fix altmap boundary check Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 85/92] drm/fsl-dcu: Use drm_plane_helper_destroy() Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 86/92] drm/imx/ipuv3: Fix front porch adjustment upon hactive aligning Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 87/92] selftests/rseq: check if libc rseq support is registered Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 88/92] selftests/rseq: Play nice with binaries statically linked against glibc 2.35+ Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 89/92] soundwire: bus: pm_runtime_request_resume on peripheral attachment Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 90/92] soundwire: fix enumeration completion Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 91/92] PM / wakeirq: support enabling wake-up irq after runtime_suspend called Greg Kroah-Hartman
2023-08-09 10:42 ` [PATCH 5.15 92/92] PM: sleep: wakeirq: fix wake irq arming Greg Kroah-Hartman
2023-08-09 13:53 ` [PATCH 5.15 00/92] 5.15.126-rc1 review Joel Fernandes
2023-08-09 16:18 ` Guenter Roeck
2023-08-09 18:35 ` Joel Fernandes
2023-08-09 18:39 ` Joel Fernandes
2023-08-09 19:25 ` Guenter Roeck
2023-08-09 20:14 ` Joel Fernandes
2023-08-09 20:38 ` Guenter Roeck [this message]
2023-08-09 20:39 ` Joel Fernandes
2023-08-09 21:45 ` Guenter Roeck
2023-08-10 17:55 ` Paul E. McKenney
2023-08-10 21:54 ` Joel Fernandes
2023-08-10 22:14 ` Joel Fernandes
2023-08-10 22:34 ` Paul E. McKenney
2023-08-10 22:55 ` Guenter Roeck
2023-08-10 23:13 ` Joel Fernandes
2023-08-09 17:05 ` SeongJae Park
2023-08-09 19:38 ` Florian Fainelli
2023-08-10 10:16 ` Harshit Mogalapalli
2023-08-10 10:25 ` Guenter Roeck
2023-08-10 10:24 ` Guenter Roeck
2023-08-10 16:25 ` Florian Fainelli
2023-08-11 10:02 ` Greg Kroah-Hartman
2023-08-10 16:06 ` Guenter Roeck
2023-08-11 10:05 ` Greg Kroah-Hartman
2023-08-10 21:11 ` Ron Economos
2023-08-10 21:54 ` Daniel Díaz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6b05a082-41a7-f0cf-c0a4-1cced8d5a230@roeck-us.net \
--to=linux@roeck-us.net \
--cc=akpm@linux-foundation.org \
--cc=conor@kernel.org \
--cc=f.fainelli@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=joel@joelfernandes.org \
--cc=jonathanh@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lkft-triage@lists.linaro.org \
--cc=patches@kernelci.org \
--cc=patches@lists.linux.dev \
--cc=paulmck@kernel.org \
--cc=pavel@denx.de \
--cc=rwarsow@gmx.de \
--cc=shuah@kernel.org \
--cc=srw@sladewatkins.net \
--cc=stable@vger.kernel.org \
--cc=sudipm.mukherjee@gmail.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).