From: Catalin Marinas <catalin.marinas@arm.com> To: Marek Szyprowski <m.szyprowski@samsung.com> Cc: "Christoph Lameter (Ampere)" <cl@gentwo.org>, Mark Rutland <mark.rutland@arm.com>, "linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>, "Rafael J. Wysocki" <rafael@kernel.org>, Viresh Kumar <vireshk@kernel.org>, Will Deacon <will@kernel.org>, Jonathan.Cameron@huawei.com, Matteo.Carlini@arm.com, Valentin.Schneider@arm.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, Eric Mackay <eric.mackay@oracle.com>, dave.kleikamp@oracle.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux@armlinux.org.uk, robin.murphy@arm.com, vanshikonda@os.amperecomputing.com, yang@os.amperecomputing.com, Nishanth Menon <nm@ti.com>, Stephen Boyd <sboyd@kernel.org> Subject: Re: [PATCH v3] ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512 Date: Fri, 8 Mar 2024 14:51:44 +0000 [thread overview] Message-ID: <ZesmAO1jJfEjTwxd@arm.com> (raw) In-Reply-To: <c1f2902d-cefc-4122-9b86-d1d32911f590@samsung.com> On Fri, Mar 08, 2024 at 03:01:28PM +0100, Marek Szyprowski wrote: > This patch landed in today's linux-next as commit 0499a78369ad ("ARM64: > Dynamically allocate cpumasks and increase supported CPUs to 512"). > Unfortunately it triggers the following warning during boot on most of > my ARM64-based test boards. Here is an example from Odroid-N2 board: > > ------------[ cut here ]------------ > WARNING: CPU: 4 PID: 63 at drivers/opp/core.c:2554 > dev_pm_opp_set_config+0x390/0x710 > Modules linked in: dw_hdmi_i2s_audio meson_gxl smsc onboard_usb_hub(+) > rtc_pcf8563 panfrost snd_soc_meson_axg_sound_card drm_shmem_helper > crct10dif_ce dwmac_generic snd_soc_meson_card_utils gpu_sched > snd_soc_meson_g12a_toacodec snd_soc_meson_g12a_tohdmitx rc_odroid > snd_soc_meson_codec_glue pwm_meson ao_cec_g12a meson_gxbb_wdt meson_ir > snd_soc_meson_axg_frddr snd_soc_meson_axg_toddr snd_soc_meson_axg_tdmin > meson_vdec(C) v4l2_mem2mem videobuf2_dma_contig snd_soc_meson_axg_tdmout > videobuf2_memops axg_audio videobuf2_v4l2 sclk_div videodev > reset_meson_audio_arb snd_soc_meson_axg_fifo clk_phase dwmac_meson8b > stmmac_platform videobuf2_common mdio_mux_meson_g12a meson_drm > meson_dw_hdmi rtc_meson_vrtc stmmac meson_ddr_pmu_g12 mc dw_hdmi > drm_display_helper pcs_xpcs snd_soc_meson_t9015 meson_canvas gpio_fan > display_connector snd_soc_meson_axg_tdm_interface > snd_soc_simple_amplifier snd_soc_meson_axg_tdm_formatter nvmem_meson_efuse > hub 1-1:1.0: USB hub found > CPU: 4 PID: 63 Comm: kworker/u12:5 Tainted: G C 6.8.0-rc3+ #14677 > Hardware name: Hardkernel ODROID-N2 (DT) > Workqueue: events_unbound deferred_probe_work_func > pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) > pc : dev_pm_opp_set_config+0x390/0x710 > lr : dev_pm_opp_set_config+0x5c/0x710 > ... > Call trace: > dev_pm_opp_set_config+0x390/0x710 > dt_cpufreq_probe+0x268/0x480 > platform_probe+0x68/0xd8 > really_probe+0x148/0x2b4 > __driver_probe_device+0x78/0x12c > driver_probe_device+0xdc/0x164 > __device_attach_driver+0xb8/0x138 > bus_for_each_drv+0x84/0xe0 > __device_attach+0xa8/0x1b0 > device_initial_probe+0x14/0x20 > bus_probe_device+0xb0/0xb4 > deferred_probe_work_func+0x8c/0xc8 > process_one_work+0x1ec/0x53c > worker_thread+0x298/0x408 > kthread+0x124/0x128 > ret_from_fork+0x10/0x20 > irq event stamp: 317178 > hardirqs last enabled at (317177): [<ffff8000801788d4>] > ktime_get_coarse_real_ts64+0x10c/0x110 > hardirqs last disabled at (317178): [<ffff800081222030>] el1_dbg+0x24/0x8c > softirqs last enabled at (315802): [<ffff800080010a60>] > __do_softirq+0x4a0/0x4e8 > softirqs last disabled at (315793): [<ffff8000800169b0>] > ____do_softirq+0x10/0x1c > ---[ end trace 0000000000000000 ]--- > cpu cpu2: error -EBUSY: failed to set regulators > cpufreq-dt: probe of cpufreq-dt failed with error -16 > > It looks that cpufreq-dt and/or opp drivers needs some adjustments > related with this change. That's strange. Is this with defconfig? I wonder whether NR_CPUS being larger caused the issue with this specific code. Otherwise CPUMASK_OFFSTACK may not work that well on arm64. > > > > --- > > > > > > Original post: https://www.spinics.net/lists/linux-mm/msg369701.html > > V2: https://lkml.org/lkml/2024/2/7/505 > > > > > > V1->V2 > > > > - Keep quotation marks > > - Remove whiltespace damage > > - Add tested by > > > > V2->V3: > > - Add test results > > - Rework descriptions > > > > > > arch/arm64/Kconfig | 16 +++++++++++++++- > > 1 file changed, 15 insertions(+), 1 deletion(-) > > > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > > index aa7c1d435139..4e767dede47d 100644 > > --- a/arch/arm64/Kconfig > > +++ b/arch/arm64/Kconfig > > @@ -1427,7 +1427,21 @@ config SCHED_SMT > > config NR_CPUS > > int "Maximum number of CPUs (2-4096)" > > range 2 4096 > > - default "256" > > + default "512" > > + > > +# > > +# Determines the placement of cpumasks. > > +# > > +# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated. > > +# Useful for machines with lots of core because it avoids increasing > > +# the size of many of the data structures in the kernel. > > +# > > +# If this is off then the cpumasks have a static sizes and are > > +# embedded within data structures. > > +# > > + config CPUMASK_OFFSTACK > > + def_bool y > > + depends on NR_CPUS > 256 > > > > config HOTPLUG_CPU > > bool "Support for hot-pluggable CPUs" > > Best regards > -- > Marek Szyprowski, PhD > Samsung R&D Institute Poland > -- Catalin
WARNING: multiple messages have this Message-ID (diff)
From: Catalin Marinas <catalin.marinas@arm.com> To: Marek Szyprowski <m.szyprowski@samsung.com> Cc: "Christoph Lameter (Ampere)" <cl@gentwo.org>, Mark Rutland <mark.rutland@arm.com>, "linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>, "Rafael J. Wysocki" <rafael@kernel.org>, Viresh Kumar <vireshk@kernel.org>, Will Deacon <will@kernel.org>, Jonathan.Cameron@huawei.com, Matteo.Carlini@arm.com, Valentin.Schneider@arm.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, Eric Mackay <eric.mackay@oracle.com>, dave.kleikamp@oracle.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux@armlinux.org.uk, robin.murphy@arm.com, vanshikonda@os.amperecomputing.com, yang@os.amperecomputing.com, Nishanth Menon <nm@ti.com>, Stephen Boyd <sboyd@kernel.org> Subject: Re: [PATCH v3] ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512 Date: Fri, 8 Mar 2024 14:51:44 +0000 [thread overview] Message-ID: <ZesmAO1jJfEjTwxd@arm.com> (raw) In-Reply-To: <c1f2902d-cefc-4122-9b86-d1d32911f590@samsung.com> On Fri, Mar 08, 2024 at 03:01:28PM +0100, Marek Szyprowski wrote: > This patch landed in today's linux-next as commit 0499a78369ad ("ARM64: > Dynamically allocate cpumasks and increase supported CPUs to 512"). > Unfortunately it triggers the following warning during boot on most of > my ARM64-based test boards. Here is an example from Odroid-N2 board: > > ------------[ cut here ]------------ > WARNING: CPU: 4 PID: 63 at drivers/opp/core.c:2554 > dev_pm_opp_set_config+0x390/0x710 > Modules linked in: dw_hdmi_i2s_audio meson_gxl smsc onboard_usb_hub(+) > rtc_pcf8563 panfrost snd_soc_meson_axg_sound_card drm_shmem_helper > crct10dif_ce dwmac_generic snd_soc_meson_card_utils gpu_sched > snd_soc_meson_g12a_toacodec snd_soc_meson_g12a_tohdmitx rc_odroid > snd_soc_meson_codec_glue pwm_meson ao_cec_g12a meson_gxbb_wdt meson_ir > snd_soc_meson_axg_frddr snd_soc_meson_axg_toddr snd_soc_meson_axg_tdmin > meson_vdec(C) v4l2_mem2mem videobuf2_dma_contig snd_soc_meson_axg_tdmout > videobuf2_memops axg_audio videobuf2_v4l2 sclk_div videodev > reset_meson_audio_arb snd_soc_meson_axg_fifo clk_phase dwmac_meson8b > stmmac_platform videobuf2_common mdio_mux_meson_g12a meson_drm > meson_dw_hdmi rtc_meson_vrtc stmmac meson_ddr_pmu_g12 mc dw_hdmi > drm_display_helper pcs_xpcs snd_soc_meson_t9015 meson_canvas gpio_fan > display_connector snd_soc_meson_axg_tdm_interface > snd_soc_simple_amplifier snd_soc_meson_axg_tdm_formatter nvmem_meson_efuse > hub 1-1:1.0: USB hub found > CPU: 4 PID: 63 Comm: kworker/u12:5 Tainted: G C 6.8.0-rc3+ #14677 > Hardware name: Hardkernel ODROID-N2 (DT) > Workqueue: events_unbound deferred_probe_work_func > pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) > pc : dev_pm_opp_set_config+0x390/0x710 > lr : dev_pm_opp_set_config+0x5c/0x710 > ... > Call trace: > dev_pm_opp_set_config+0x390/0x710 > dt_cpufreq_probe+0x268/0x480 > platform_probe+0x68/0xd8 > really_probe+0x148/0x2b4 > __driver_probe_device+0x78/0x12c > driver_probe_device+0xdc/0x164 > __device_attach_driver+0xb8/0x138 > bus_for_each_drv+0x84/0xe0 > __device_attach+0xa8/0x1b0 > device_initial_probe+0x14/0x20 > bus_probe_device+0xb0/0xb4 > deferred_probe_work_func+0x8c/0xc8 > process_one_work+0x1ec/0x53c > worker_thread+0x298/0x408 > kthread+0x124/0x128 > ret_from_fork+0x10/0x20 > irq event stamp: 317178 > hardirqs last enabled at (317177): [<ffff8000801788d4>] > ktime_get_coarse_real_ts64+0x10c/0x110 > hardirqs last disabled at (317178): [<ffff800081222030>] el1_dbg+0x24/0x8c > softirqs last enabled at (315802): [<ffff800080010a60>] > __do_softirq+0x4a0/0x4e8 > softirqs last disabled at (315793): [<ffff8000800169b0>] > ____do_softirq+0x10/0x1c > ---[ end trace 0000000000000000 ]--- > cpu cpu2: error -EBUSY: failed to set regulators > cpufreq-dt: probe of cpufreq-dt failed with error -16 > > It looks that cpufreq-dt and/or opp drivers needs some adjustments > related with this change. That's strange. Is this with defconfig? I wonder whether NR_CPUS being larger caused the issue with this specific code. Otherwise CPUMASK_OFFSTACK may not work that well on arm64. > > > > --- > > > > > > Original post: https://www.spinics.net/lists/linux-mm/msg369701.html > > V2: https://lkml.org/lkml/2024/2/7/505 > > > > > > V1->V2 > > > > - Keep quotation marks > > - Remove whiltespace damage > > - Add tested by > > > > V2->V3: > > - Add test results > > - Rework descriptions > > > > > > arch/arm64/Kconfig | 16 +++++++++++++++- > > 1 file changed, 15 insertions(+), 1 deletion(-) > > > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > > index aa7c1d435139..4e767dede47d 100644 > > --- a/arch/arm64/Kconfig > > +++ b/arch/arm64/Kconfig > > @@ -1427,7 +1427,21 @@ config SCHED_SMT > > config NR_CPUS > > int "Maximum number of CPUs (2-4096)" > > range 2 4096 > > - default "256" > > + default "512" > > + > > +# > > +# Determines the placement of cpumasks. > > +# > > +# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated. > > +# Useful for machines with lots of core because it avoids increasing > > +# the size of many of the data structures in the kernel. > > +# > > +# If this is off then the cpumasks have a static sizes and are > > +# embedded within data structures. > > +# > > + config CPUMASK_OFFSTACK > > + def_bool y > > + depends on NR_CPUS > 256 > > > > config HOTPLUG_CPU > > bool "Support for hot-pluggable CPUs" > > Best regards > -- > Marek Szyprowski, PhD > Samsung R&D Institute Poland > -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2024-03-08 14:51 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-03-07 1:45 [PATCH v3] ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512 Christoph Lameter (Ampere) 2024-03-07 1:45 ` Christoph Lameter (Ampere) 2024-03-07 17:49 ` Mark Rutland 2024-03-07 17:49 ` Mark Rutland 2024-03-07 19:07 ` Catalin Marinas 2024-03-07 19:07 ` Catalin Marinas 2024-03-18 18:17 ` Catalin Marinas 2024-03-18 18:17 ` Catalin Marinas [not found] ` <CGME20240308140130eucas1p1259c805a0b6491ce2f69c6fca0264b1f@eucas1p1.samsung.com> 2024-03-08 14:01 ` Marek Szyprowski 2024-03-08 14:01 ` Marek Szyprowski 2024-03-08 14:51 ` Catalin Marinas [this message] 2024-03-08 14:51 ` Catalin Marinas 2024-03-08 16:21 ` Marek Szyprowski 2024-03-08 16:21 ` Marek Szyprowski 2024-03-08 17:08 ` Christoph Lameter (Ampere) 2024-03-08 17:08 ` Christoph Lameter (Ampere) 2024-03-11 12:12 ` Mark Rutland 2024-03-11 12:12 ` Mark Rutland 2024-03-11 14:56 ` Marek Szyprowski 2024-03-11 14:56 ` Marek Szyprowski 2024-03-11 15:22 ` Catalin Marinas 2024-03-11 15:22 ` Catalin Marinas 2024-03-11 16:51 ` Marek Szyprowski 2024-03-11 16:51 ` Marek Szyprowski 2024-03-11 17:08 ` Catalin Marinas 2024-03-11 17:08 ` Catalin Marinas 2024-03-11 18:55 ` Catalin Marinas 2024-03-11 18:55 ` Catalin Marinas 2024-03-11 21:07 ` Christoph Lameter (Ampere) 2024-03-11 21:07 ` Christoph Lameter (Ampere) 2024-03-12 17:06 ` Christoph Lameter (Ampere) 2024-03-12 17:06 ` Christoph Lameter (Ampere) 2024-03-12 17:55 ` Catalin Marinas 2024-03-12 17:55 ` Catalin Marinas 2024-03-13 14:35 ` Sudeep Holla 2024-03-13 14:35 ` Sudeep Holla 2024-03-13 16:22 ` Marek Szyprowski 2024-03-13 16:22 ` Marek Szyprowski 2024-03-13 16:39 ` Christoph Lameter (Ampere) 2024-03-13 16:39 ` Christoph Lameter (Ampere) 2024-03-13 20:18 ` Marek Szyprowski 2024-03-13 20:18 ` Marek Szyprowski 2024-03-13 17:13 ` Russell King (Oracle) 2024-03-13 17:13 ` Russell King (Oracle) 2024-03-14 8:39 ` Catalin Marinas 2024-03-14 8:39 ` Catalin Marinas 2024-03-14 12:28 ` Marek Szyprowski 2024-03-14 12:28 ` Marek Szyprowski 2024-03-14 13:17 ` Russell King (Oracle) 2024-03-14 13:17 ` Russell King (Oracle) 2024-03-14 17:01 ` Christoph Lameter (Ampere) 2024-03-14 17:01 ` Christoph Lameter (Ampere) 2024-03-14 13:57 ` Catalin Marinas 2024-03-14 13:57 ` Catalin Marinas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=ZesmAO1jJfEjTwxd@arm.com \ --to=catalin.marinas@arm.com \ --cc=Jonathan.Cameron@huawei.com \ --cc=Matteo.Carlini@arm.com \ --cc=Valentin.Schneider@arm.com \ --cc=akpm@linux-foundation.org \ --cc=anshuman.khandual@arm.com \ --cc=cl@gentwo.org \ --cc=dave.kleikamp@oracle.com \ --cc=eric.mackay@oracle.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-pm@vger.kernel.org \ --cc=linux@armlinux.org.uk \ --cc=m.szyprowski@samsung.com \ --cc=mark.rutland@arm.com \ --cc=nm@ti.com \ --cc=rafael@kernel.org \ --cc=robin.murphy@arm.com \ --cc=sboyd@kernel.org \ --cc=vanshikonda@os.amperecomputing.com \ --cc=vireshk@kernel.org \ --cc=will@kernel.org \ --cc=yang@os.amperecomputing.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.