* Regression with e428e250fde6 on BeagleBoard Rev C2
@ 2021-09-19 17:41 Jarkko Nikula
2021-09-20 13:34 ` Tony Lindgren
2021-09-20 18:19 ` H. Nikolaus Schaller
0 siblings, 2 replies; 21+ messages in thread
From: Jarkko Nikula @ 2021-09-19 17:41 UTC (permalink / raw)
To: linux-omap; +Cc: Tony Lindgren, Laurent Pinchart
Hi
I found another timer related regression on BeagleBoard Rev C2 which was
present before 6cfcd5563b4f ("clocksource/drivers/timer-ti-dm: Fix
suspend and resume for am3 and am4") and which remains after fix
3ff340e24c9d ("bus: ti-sysc: Fix gpt12 system timer issue with reserved
status") including today's head d4d016caa4b8 ("alpha: move __udiv_qrnnd
library function to arch/alpha/lib/").
Issue occurs when omap3_isp is not loaded and symptoms are the same than
my previous finding [1]. I.e. timer interrupts appear missing and need
to hit keys on serial console in order to let boot (when omap3_isp not
built) or "sleep 1" in shell to continue.
I bisected that regression to commit
e428e250fde6 ("ARM: dts: Configure system timers for omap3")
According to commit and dmesg it seems to switch from 32k timer to 13
MHz timer. Commit does not explain why it marks 32k timer as unusable on
the BeagleBoard. Or was that a copy-paste error from another dts?
Before
[ 0.000000] OMAP clockevent source: timer12 at 32768 Hz
[ 0.000000] OMAP clocksource: 32k_counter at 32768 Hz
[ 0.000000] clocksource: 32k_counter: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 58327039986419 ns
[ 0.000030] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps
every 65535999984741ns
After e428e250fde6
[ 0.000000] TI gptimer clockevent: 13000000 Hz at
/ocp@68000000/target-module@49032000
[ 0.000000] TI gptimer clocksource: always-on
/ocp@68000000/target-module@48304000
[ 0.000091] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps
every 65535999984741ns
[ 0.000183] clocksource: dmtimer: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 58327039986419 ns
I fail to understand how omap3isp affects this since it actually disable
clocks after probe. Does it keep some power domain active which then
keeps the timer active etc?
1. https://marc.info/?l=linux-omap&m=162221018410523&w=2
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-19 17:41 Regression with e428e250fde6 on BeagleBoard Rev C2 Jarkko Nikula @ 2021-09-20 13:34 ` Tony Lindgren 2021-09-20 14:52 ` Andreas Kemnade 2021-09-20 16:35 ` Jarkko Nikula 2021-09-20 18:19 ` H. Nikolaus Schaller 1 sibling, 2 replies; 21+ messages in thread From: Tony Lindgren @ 2021-09-20 13:34 UTC (permalink / raw) To: Jarkko Nikula; +Cc: linux-omap, Laurent Pinchart Hi, * Jarkko Nikula <jarkko.nikula@bitmer.com> [210919 17:42]: > Hi > > I found another timer related regression on BeagleBoard Rev C2 which was > present before 6cfcd5563b4f ("clocksource/drivers/timer-ti-dm: Fix > suspend and resume for am3 and am4") and which remains after fix > 3ff340e24c9d ("bus: ti-sysc: Fix gpt12 system timer issue with reserved > status") including today's head d4d016caa4b8 ("alpha: move __udiv_qrnnd > library function to arch/alpha/lib/"). > > Issue occurs when omap3_isp is not loaded and symptoms are the same than > my previous finding [1]. I.e. timer interrupts appear missing and need > to hit keys on serial console in order to let boot (when omap3_isp not > built) or "sleep 1" in shell to continue. > > I bisected that regression to commit > e428e250fde6 ("ARM: dts: Configure system timers for omap3") OK thanks for bisecting it down :) > According to commit and dmesg it seems to switch from 32k timer to 13 > MHz timer. Commit does not explain why it marks 32k timer as unusable on > the BeagleBoard. Or was that a copy-paste error from another dts? There is a 32k timer errata for beagleboard "Boards revision A and < B5" at https://elinux.org/BeagleBoard_Community, but looks like that is not what we were doing earlier looks like.. > Before > > [ 0.000000] OMAP clockevent source: timer12 at 32768 Hz > [ 0.000000] OMAP clocksource: 32k_counter at 32768 Hz > [ 0.000000] clocksource: 32k_counter: mask: 0xffffffff max_cycles: > 0xffffffff, max_idle_ns: 58327039986419 ns > [ 0.000030] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps > every 65535999984741ns Interesting, so the workaround we've had is still using the 32k clock source instead of the mpu clock source, and also using the 32k_counter. Let's switch back to that then, that sounds much nicer :) > After e428e250fde6 > > [ 0.000000] TI gptimer clockevent: 13000000 Hz at > /ocp@68000000/target-module@49032000 > [ 0.000000] TI gptimer clocksource: always-on > /ocp@68000000/target-module@48304000 > [ 0.000091] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps > every 65535999984741ns > [ 0.000183] clocksource: dmtimer: mask: 0xffffffff max_cycles: > 0xffffffff, max_idle_ns: 58327039986419 ns The patch below should take us back to the earlier configuration, can you please give it a try? > I fail to understand how omap3isp affects this since it actually disable > clocks after probe. Does it keep some power domain active which then > keeps the timer active etc? Sounds like it's because omap3isp never allows the SoC to enter deeper idle states. The mpu clock is stopped for idle states. Regards, Tony 8<--------------- diff --git a/arch/arm/boot/dts/omap3-beagle.dts b/arch/arm/boot/dts/omap3-beagle.dts --- a/arch/arm/boot/dts/omap3-beagle.dts +++ b/arch/arm/boot/dts/omap3-beagle.dts @@ -304,11 +304,6 @@ &usbhsehci { phys = <0 &hsusb2_phy>; }; -/* Unusable as clocksource because of unreliable oscillator */ -&counter32k { - status = "disabled"; -}; - /* Unusable as clockevent because if unreliable oscillator, allow to idle */ &timer1_target { /delete-property/ti,no-reset-on-init; @@ -318,22 +313,13 @@ timer@0 { }; }; -/* Preferred always-on timer for clocksource */ +/* Preferred always-on timer for clockevent */ &timer12_target { ti,no-reset-on-init; ti,no-idle; timer@0 { - /* Always clocked by secure_32k_fck */ - }; -}; - -/* Preferred timer for clockevent */ -&timer2_target { - ti,no-reset-on-init; - ti,no-idle; - timer@0 { - assigned-clocks = <&gpt2_fck>; - assigned-clock-parents = <&sys_ck>; + assigned-clocks = <&gpt12_fck>; + assigned-clock-parents = <&secure_32k_fck>; }; }; diff --git a/arch/arm/boot/dts/omap3-devkit8000.dts b/arch/arm/boot/dts/omap3-devkit8000.dts --- a/arch/arm/boot/dts/omap3-devkit8000.dts +++ b/arch/arm/boot/dts/omap3-devkit8000.dts @@ -15,11 +15,6 @@ aliases { }; }; -/* Unusable as clocksource because of unreliable oscillator */ -&counter32k { - status = "disabled"; -}; - /* Unusable as clockevent because if unreliable oscillator, allow to idle */ &timer1_target { /delete-property/ti,no-reset-on-init; @@ -29,21 +24,12 @@ timer@0 { }; }; -/* Preferred always-on timer for clocksource */ +/* Preferred always-on timer for clockevent */ &timer12_target { ti,no-reset-on-init; ti,no-idle; timer@0 { - /* Always clocked by secure_32k_fck */ - }; -}; - -/* Preferred timer for clockevent */ -&timer2_target { - ti,no-reset-on-init; - ti,no-idle; - timer@0 { - assigned-clocks = <&gpt2_fck>; - assigned-clock-parents = <&sys_ck>; + assigned-clocks = <&gpt12_fck>; + assigned-clock-parents = <&secure_32k_fck>; }; }; -- 2.33.0 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-20 13:34 ` Tony Lindgren @ 2021-09-20 14:52 ` Andreas Kemnade 2021-09-21 7:00 ` Tony Lindgren 2021-09-20 16:35 ` Jarkko Nikula 1 sibling, 1 reply; 21+ messages in thread From: Andreas Kemnade @ 2021-09-20 14:52 UTC (permalink / raw) To: Tony Lindgren; +Cc: Jarkko Nikula, linux-omap, Laurent Pinchart Hi, On Mon, 20 Sep 2021 16:34:59 +0300 Tony Lindgren <tony@atomide.com> wrote: > > I fail to understand how omap3isp affects this since it actually disable > > clocks after probe. Does it keep some power domain active which then > > keeps the timer active etc? > > Sounds like it's because omap3isp never allows the SoC to enter deeper > idle states. The mpu clock is stopped for idle states. On GTA04 I experienced also some strange pm issues with omap3isp: rmmod clears up everything fine. After probing, clocks seems to be turned off, but power consumption does not drop. It only drops when there is an iommu_detach paired to that iommu_attach in the driver. But if I add it, the driver does not work properly. Regards, Andreas ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-20 14:52 ` Andreas Kemnade @ 2021-09-21 7:00 ` Tony Lindgren 2021-09-21 16:25 ` Andreas Kemnade 0 siblings, 1 reply; 21+ messages in thread From: Tony Lindgren @ 2021-09-21 7:00 UTC (permalink / raw) To: Andreas Kemnade; +Cc: Jarkko Nikula, linux-omap, Laurent Pinchart * Andreas Kemnade <andreas@kemnade.info> [210920 15:26]: > Hi, > > On Mon, 20 Sep 2021 16:34:59 +0300 > Tony Lindgren <tony@atomide.com> wrote: > > > > I fail to understand how omap3isp affects this since it actually disable > > > clocks after probe. Does it keep some power domain active which then > > > keeps the timer active etc? > > > > Sounds like it's because omap3isp never allows the SoC to enter deeper > > idle states. The mpu clock is stopped for idle states. > > On GTA04 I experienced also some strange pm issues with omap3isp: > rmmod clears up everything fine. > > After probing, clocks seems to be turned off, but power consumption > does not drop. It only drops when there is an iommu_detach paired to > that iommu_attach in the driver. > But if I add it, the driver does not work properly. Are you also seeing a timer issue? Or just the omap3isp not idling the SoC issue? Regards, Tony ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-21 7:00 ` Tony Lindgren @ 2021-09-21 16:25 ` Andreas Kemnade 0 siblings, 0 replies; 21+ messages in thread From: Andreas Kemnade @ 2021-09-21 16:25 UTC (permalink / raw) To: Tony Lindgren; +Cc: Jarkko Nikula, linux-omap, Laurent Pinchart On Tue, 21 Sep 2021 10:00:59 +0300 Tony Lindgren <tony@atomide.com> wrote: > * Andreas Kemnade <andreas@kemnade.info> [210920 15:26]: > > Hi, > > > > On Mon, 20 Sep 2021 16:34:59 +0300 > > Tony Lindgren <tony@atomide.com> wrote: > > > > > > I fail to understand how omap3isp affects this since it actually disable > > > > clocks after probe. Does it keep some power domain active which then > > > > keeps the timer active etc? > > > > > > Sounds like it's because omap3isp never allows the SoC to enter deeper > > > idle states. The mpu clock is stopped for idle states. > > > > On GTA04 I experienced also some strange pm issues with omap3isp: > > rmmod clears up everything fine. > > > > After probing, clocks seems to be turned off, but power consumption > > does not drop. It only drops when there is an iommu_detach paired to > > that iommu_attach in the driver. > > But if I add it, the driver does not work properly. > > Are you also seeing a timer issue? Or just the omap3isp not idling the > SoC issue? > Just the omap3isp not idling the SoC. That are just general pm experience I had with the omap3isp issue, I have seen on earlier kernel versions but never succeeded in properly fixing them. I have not tried the 5.15-rcX on gta04 yet. Regards, Andreas ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-20 13:34 ` Tony Lindgren 2021-09-20 14:52 ` Andreas Kemnade @ 2021-09-20 16:35 ` Jarkko Nikula 2021-09-21 7:00 ` Tony Lindgren 1 sibling, 1 reply; 21+ messages in thread From: Jarkko Nikula @ 2021-09-20 16:35 UTC (permalink / raw) To: Tony Lindgren; +Cc: linux-omap, Laurent Pinchart On 9/20/21 4:34 PM, Tony Lindgren wrote: > The patch below should take us back to the earlier configuration, > can you please give it a try? > I double checked and unfortunate it doesn't boot at all or give any output when patch is applied: ## Booting kernel from Legacy Image at 80200000 ... Image Name: Linux-5.15.0-rc1-00239-gd4d016ca Image Type: ARM Linux Kernel Image (uncompressed) Data Size: 4756991 Bytes = 4.5 MiB Load Address: 80008000 Entry Point: 80008000 Verifying Checksum ... OK Loading Kernel Image ... OK Starting kernel ... I used the weekend's head d4d016caa4b8 ("alpha: move __udiv_qrnnd library function to arch/alpha/lib/") in testing. Jarkko ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-20 16:35 ` Jarkko Nikula @ 2021-09-21 7:00 ` Tony Lindgren 2021-09-21 17:49 ` Jarkko Nikula 2021-10-20 17:52 ` Jarkko Nikula 0 siblings, 2 replies; 21+ messages in thread From: Tony Lindgren @ 2021-09-21 7:00 UTC (permalink / raw) To: Jarkko Nikula; +Cc: linux-omap, Laurent Pinchart * Jarkko Nikula <jarkko.nikula@bitmer.com> [210920 16:35]: > On 9/20/21 4:34 PM, Tony Lindgren wrote: > > The patch below should take us back to the earlier configuration, > > can you please give it a try? > > > I double checked and unfortunate it doesn't boot at all or give any > output when patch is applied: Hmm I tested it with copying the beagle timer configuration to omap3.dtsi and things worked for me. If you can, please try adding CONFIG_SERIAL_EARLYCON=y and then also add "debug earlyprintk" to kernel command line. It might show something that I'm not able to reproduce. Maybe there is a bootloader dependency or something like that. It should be OK to keep these options enabled, except maybe not for "debug" in the command line. Regards, Tony ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-21 7:00 ` Tony Lindgren @ 2021-09-21 17:49 ` Jarkko Nikula 2021-09-22 10:07 ` Tony Lindgren 2021-10-20 17:52 ` Jarkko Nikula 1 sibling, 1 reply; 21+ messages in thread From: Jarkko Nikula @ 2021-09-21 17:49 UTC (permalink / raw) To: Tony Lindgren; +Cc: linux-omap, Laurent Pinchart On 9/21/21 10:00 AM, Tony Lindgren wrote: > * Jarkko Nikula <jarkko.nikula@bitmer.com> [210920 16:35]: >> On 9/20/21 4:34 PM, Tony Lindgren wrote: >>> The patch below should take us back to the earlier configuration, >>> can you please give it a try? >>> >> I double checked and unfortunate it doesn't boot at all or give any >> output when patch is applied: > > Hmm I tested it with copying the beagle timer configuration to omap3.dtsi > and things worked for me. If you can, please try adding > CONFIG_SERIAL_EARLYCON=y and then also add "debug earlyprintk" to kernel > command line. It might show something that I'm not able to reproduce. > Maybe there is a bootloader dependency or something like that. > Hmm.. no luck on top of d5f6545934c4 ("qnx4: work around gcc false positive warning bug") with the patch. Nothing gets printed after "Starting kernel ...". Without the patch boots fine. Here's the full /proc/cmdline from running system: console=ttyO2,115200n8 debug earlyprintk=ttyO2,115200n8 mpurate=auto buddy=none camera=none vram=12M omapfb.mode=dvi:640x480MR-16@60 omapdss.def_disp=dvi root=/dev/mmcblk0p2 rw rootfstype=ext4 rootwait > It should be OK to keep these options enabled, except maybe not for > "debug" in the command line. > Yeah, systemd outputs some amount of debug stuff, fortunately doesn't render system unusable :-) ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-21 17:49 ` Jarkko Nikula @ 2021-09-22 10:07 ` Tony Lindgren 2021-09-22 17:21 ` Jarkko Nikula 0 siblings, 1 reply; 21+ messages in thread From: Tony Lindgren @ 2021-09-22 10:07 UTC (permalink / raw) To: Jarkko Nikula Cc: linux-omap, Laurent Pinchart, H. Nikolaus Schaller, Andreas Kemnade Hi, * Jarkko Nikula <jarkko.nikula@bitmer.com> [210921 17:50]: > On 9/21/21 10:00 AM, Tony Lindgren wrote: > > * Jarkko Nikula <jarkko.nikula@bitmer.com> [210920 16:35]: > >> On 9/20/21 4:34 PM, Tony Lindgren wrote: > >>> The patch below should take us back to the earlier configuration, > >>> can you please give it a try? > >>> > >> I double checked and unfortunate it doesn't boot at all or give any > >> output when patch is applied: > > > > Hmm I tested it with copying the beagle timer configuration to omap3.dtsi > > and things worked for me. If you can, please try adding > > CONFIG_SERIAL_EARLYCON=y and then also add "debug earlyprintk" to kernel > > command line. It might show something that I'm not able to reproduce. > > Maybe there is a bootloader dependency or something like that. > > > Hmm.. no luck on top of d5f6545934c4 ("qnx4: work around gcc false > positive warning bug") with the patch. Nothing gets printed after > "Starting kernel ...". Without the patch boots fine. OK interesting. Best to keep the booting quirk handling for beagle rev a to b4 then. No reason to keep the timer quirks for rev b5 and c boards though. Can you try the following patch that removes the quirks for omap3-beagle by default? The patch also adds a new omap3-beagle-ab4.dts that keeps the current quirks and also limits the pmic capabilities for the old known buggy boards. I also noticed that omap3-beagle.dts is missing the twl power node probably because of the rev a to b4 quirks. Adding the twl power node with "ti,twl4030-power-idle-osc-off" similar to beagle xm might produce some nice power savings after the quirks are gone :) Needs to be tested with kernel serial console detached and serial port autoidle configured. That's a different patch for sure. Regards, Tony 8< ----------------------- diff --git a/arch/arm/boot/dts/Makefile b/arch/arm/boot/dts/Makefile --- a/arch/arm/boot/dts/Makefile +++ b/arch/arm/boot/dts/Makefile @@ -779,6 +779,7 @@ dtb-$(CONFIG_ARCH_OMAP3) += \ logicpd-som-lv-37xx-devkit.dtb \ omap3430-sdp.dtb \ omap3-beagle.dtb \ + omap3-beagle-ab4.dtb \ omap3-beagle-xm.dtb \ omap3-beagle-xm-ab.dtb \ omap3-cm-t3517.dtb \ diff --git a/arch/arm/boot/dts/omap3-beagle-ab4.dts b/arch/arm/boot/dts/omap3-beagle-ab4.dts new file mode 100644 --- /dev/null +++ b/arch/arm/boot/dts/omap3-beagle-ab4.dts @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0-only +/dts-v1/; + +#include "omap3-beagle.dts" + +/* + * Workaround for capacitor C70 issue, see "Boards revision A and < B5" + * section at https://elinux.org/BeagleBoard_Community + */ + +/* Unusable as clocksource because of unreliable oscillator */ +&counter32k { + status = "disabled"; +}; + +/* Unusable as clockevent because if unreliable oscillator, allow to idle */ +&timer1_target { + /delete-property/ti,no-reset-on-init; + /delete-property/ti,no-idle; + timer@0 { + /delete-property/ti,timer-alwon; + }; +}; + +/* Preferred always-on timer for clocksource */ +&timer12_target { + ti,no-reset-on-init; + ti,no-idle; + timer@0 { + /* Always clocked by secure_32k_fck */ + }; +}; + +/* Preferred timer for clockevent */ +&timer2_target { + ti,no-reset-on-init; + ti,no-idle; + timer@0 { + assigned-clocks = <&gpt2_fck>; + assigned-clock-parents = <&sys_ck>; + }; +}; + +&twl { + twl_power: power { + compatible = "ti,twl4030-power-idle"; + ti,use_poweroff; + }; +}; diff --git a/arch/arm/boot/dts/omap3-beagle.dts b/arch/arm/boot/dts/omap3-beagle.dts --- a/arch/arm/boot/dts/omap3-beagle.dts +++ b/arch/arm/boot/dts/omap3-beagle.dts @@ -304,39 +304,6 @@ &usbhsehci { phys = <0 &hsusb2_phy>; }; -/* Unusable as clocksource because of unreliable oscillator */ -&counter32k { - status = "disabled"; -}; - -/* Unusable as clockevent because if unreliable oscillator, allow to idle */ -&timer1_target { - /delete-property/ti,no-reset-on-init; - /delete-property/ti,no-idle; - timer@0 { - /delete-property/ti,timer-alwon; - }; -}; - -/* Preferred always-on timer for clocksource */ -&timer12_target { - ti,no-reset-on-init; - ti,no-idle; - timer@0 { - /* Always clocked by secure_32k_fck */ - }; -}; - -/* Preferred timer for clockevent */ -&timer2_target { - ti,no-reset-on-init; - ti,no-idle; - timer@0 { - assigned-clocks = <&gpt2_fck>; - assigned-clock-parents = <&sys_ck>; - }; -}; - &twl_gpio { ti,use-leds; /* pullups: BIT(1) */ -- 2.33.0 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-22 10:07 ` Tony Lindgren @ 2021-09-22 17:21 ` Jarkko Nikula 2021-09-23 5:41 ` Tony Lindgren 0 siblings, 1 reply; 21+ messages in thread From: Jarkko Nikula @ 2021-09-22 17:21 UTC (permalink / raw) To: Tony Lindgren Cc: linux-omap, Laurent Pinchart, H. Nikolaus Schaller, Andreas Kemnade [-- Attachment #1: Type: text/plain, Size: 1444 bytes --] On 9/22/21 1:07 PM, Tony Lindgren wrote: >> Hmm.. no luck on top of d5f6545934c4 ("qnx4: work around gcc false >> positive warning bug") with the patch. Nothing gets printed after >> "Starting kernel ...". Without the patch boots fine. > > OK interesting. Best to keep the booting quirk handling for beagle > rev a to b4 then. No reason to keep the timer quirks for rev b5 and c > boards though. > > Can you try the following patch that removes the quirks for omap3-beagle > by default? > > The patch also adds a new omap3-beagle-ab4.dts that keeps the current > quirks and also limits the pmic capabilities for the old known buggy > boards. > Better luck with this one but looks like idling cause "undefined instruction" crash. Cache/memory etc corruption perhaps? Serial console log attached. I used the new omap3-beagle-ab4.dtb: cat arch/arm/boot/dts/omap3-beagle-ab4.dtb >>arch/arm/boot/zImage; ma LOADADDR=0x80008000 uImage > I also noticed that omap3-beagle.dts is missing the twl power node > probably because of the rev a to b4 quirks. > > Adding the twl power node with "ti,twl4030-power-idle-osc-off" similar > to beagle xm might produce some nice power savings after the quirks are > gone :) > > Needs to be tested with kernel serial console detached and serial > port autoidle configured. That's a different patch for sure. > I guess the timer change would be the fix and twl changes for normal development cycle? Jarkko [-- Attachment #2: minicom.cap --] [-- Type: application/vnd.tcpdump.pcap, Size: 18158 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-22 17:21 ` Jarkko Nikula @ 2021-09-23 5:41 ` Tony Lindgren 2021-09-23 18:03 ` Jarkko Nikula 0 siblings, 1 reply; 21+ messages in thread From: Tony Lindgren @ 2021-09-23 5:41 UTC (permalink / raw) To: Jarkko Nikula Cc: linux-omap, Laurent Pinchart, H. Nikolaus Schaller, Andreas Kemnade * Jarkko Nikula <jarkko.nikula@bitmer.com> [210922 17:22]: > Better luck with this one but looks like idling cause "undefined > instruction" crash. Cache/memory etc corruption perhaps? Serial console > log attached. Hmm. If you comment out the twl power node, does the omap3-beagle-ab4.dtb boot normally for you? It should behave the same as current mainline then with the omap3isp issue. > I used the new omap3-beagle-ab4.dtb: OK. So for your rev c2 board, does the new omap3-beagle.dtb boot and work for you? > I guess the timer change would be the fix and twl changes for normal > development cycle? Yeah. Especially if it makes things worse for omap3-beagle-ab4.dtb. Regards, Tony ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-23 5:41 ` Tony Lindgren @ 2021-09-23 18:03 ` Jarkko Nikula 2021-09-24 7:02 ` Tony Lindgren 0 siblings, 1 reply; 21+ messages in thread From: Jarkko Nikula @ 2021-09-23 18:03 UTC (permalink / raw) To: Tony Lindgren Cc: linux-omap, Laurent Pinchart, H. Nikolaus Schaller, Andreas Kemnade On 9/23/21 8:41 AM, Tony Lindgren wrote: > * Jarkko Nikula <jarkko.nikula@bitmer.com> [210922 17:22]: >> Better luck with this one but looks like idling cause "undefined >> instruction" crash. Cache/memory etc corruption perhaps? Serial console >> log attached. > > Hmm. If you comment out the twl power node, does the omap3-beagle-ab4.dtb > boot normally for you? It should behave the same as current mainline then > with the omap3isp issue. > Commenting the twl_power node out or removing the twl section leads to immediately rebooting kernel: Starting kernel ... undefined instruction pc : [<80008000>] lr : [<8ff41c1d>] reloc pc : [<701c7000>] lr : [<80100c1d>] sp : 8df20998 ip : 00000020 fp : 00000001 r10: 0000071f r9 : 8df20ed0 r8 : 8df2e864 r7 : 00000000 r6 : 80008000 r5 : 8ff9f5dc r4 : 00000000 r3 : 8df20fa8 r2 : 80000100 r1 : 0000060a r0 : 00000000 Flags: nZCv IRQs off FIQs off Mode SVC_32 Resetting CPU ... >> I used the new omap3-beagle-ab4.dtb: > > OK. So for your rev c2 board, does the new omap3-beagle.dtb boot and > work for you? > Booting with that alone doesn't output anything (with the patch applied). Jarkko ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-23 18:03 ` Jarkko Nikula @ 2021-09-24 7:02 ` Tony Lindgren 2021-09-25 14:55 ` Jarkko Nikula 0 siblings, 1 reply; 21+ messages in thread From: Tony Lindgren @ 2021-09-24 7:02 UTC (permalink / raw) To: Jarkko Nikula Cc: linux-omap, Laurent Pinchart, H. Nikolaus Schaller, Andreas Kemnade * Jarkko Nikula <jarkko.nikula@bitmer.com> [210923 18:04]: > On 9/23/21 8:41 AM, Tony Lindgren wrote: > > * Jarkko Nikula <jarkko.nikula@bitmer.com> [210922 17:22]: > >> Better luck with this one but looks like idling cause "undefined > >> instruction" crash. Cache/memory etc corruption perhaps? Serial console > >> log attached. > > > > Hmm. If you comment out the twl power node, does the omap3-beagle-ab4.dtb > > boot normally for you? It should behave the same as current mainline then > > with the omap3isp issue. > > > Commenting the twl_power node out or removing the twl section leads to > immediately rebooting kernel: Below is an updated patch to move the timer quirks to omap3-beagle-ab4.dtb with no twl changes. I've verified the resulting omap3-beagle-ab4.dtb matches the current unpatched mainline kernel omap3-beagle.dtb. Then for figuring out why omap3-beagle.dtb with no timer quirks does not boot on rev c, if you could please enable CONFIG_DEBUG_LL=y and CONFIG_EARLY_PRINTK=y and select the related debug uart. You can select the defaults for other configuration options. Also, note that if your u-boot fdtaddr is too low enabling more things in the kernel may cause a situation where the uncompressing kernel overwrites the dtb. Looks like i have setenv fdtaddr 0x84000000. Regards, Tony 8< --------------------- diff --git a/arch/arm/boot/dts/Makefile b/arch/arm/boot/dts/Makefile --- a/arch/arm/boot/dts/Makefile +++ b/arch/arm/boot/dts/Makefile @@ -779,6 +779,7 @@ dtb-$(CONFIG_ARCH_OMAP3) += \ logicpd-som-lv-37xx-devkit.dtb \ omap3430-sdp.dtb \ omap3-beagle.dtb \ + omap3-beagle-ab4.dtb \ omap3-beagle-xm.dtb \ omap3-beagle-xm-ab.dtb \ omap3-cm-t3517.dtb \ diff --git a/arch/arm/boot/dts/omap3-beagle-ab4.dts b/arch/arm/boot/dts/omap3-beagle-ab4.dts new file mode 100644 --- /dev/null +++ b/arch/arm/boot/dts/omap3-beagle-ab4.dts @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0-only +/dts-v1/; + +#include "omap3-beagle.dts" + +/* + * Workaround for capacitor C70 issue, see "Boards revision A and < B5" + * section at https://elinux.org/BeagleBoard_Community + */ + +/* Unusable as clocksource because of unreliable oscillator */ +&counter32k { + status = "disabled"; +}; + +/* Unusable as clockevent because if unreliable oscillator, allow to idle */ +&timer1_target { + /delete-property/ti,no-reset-on-init; + /delete-property/ti,no-idle; + timer@0 { + /delete-property/ti,timer-alwon; + }; +}; + +/* Preferred always-on timer for clocksource */ +&timer12_target { + ti,no-reset-on-init; + ti,no-idle; + timer@0 { + /* Always clocked by secure_32k_fck */ + }; +}; + +/* Preferred timer for clockevent */ +&timer2_target { + ti,no-reset-on-init; + ti,no-idle; + timer@0 { + assigned-clocks = <&gpt2_fck>; + assigned-clock-parents = <&sys_ck>; + }; +}; diff --git a/arch/arm/boot/dts/omap3-beagle.dts b/arch/arm/boot/dts/omap3-beagle.dts --- a/arch/arm/boot/dts/omap3-beagle.dts +++ b/arch/arm/boot/dts/omap3-beagle.dts @@ -304,39 +304,6 @@ &usbhsehci { phys = <0 &hsusb2_phy>; }; -/* Unusable as clocksource because of unreliable oscillator */ -&counter32k { - status = "disabled"; -}; - -/* Unusable as clockevent because if unreliable oscillator, allow to idle */ -&timer1_target { - /delete-property/ti,no-reset-on-init; - /delete-property/ti,no-idle; - timer@0 { - /delete-property/ti,timer-alwon; - }; -}; - -/* Preferred always-on timer for clocksource */ -&timer12_target { - ti,no-reset-on-init; - ti,no-idle; - timer@0 { - /* Always clocked by secure_32k_fck */ - }; -}; - -/* Preferred timer for clockevent */ -&timer2_target { - ti,no-reset-on-init; - ti,no-idle; - timer@0 { - assigned-clocks = <&gpt2_fck>; - assigned-clock-parents = <&sys_ck>; - }; -}; - &twl_gpio { ti,use-leds; /* pullups: BIT(1) */ -- 2.33.0 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-24 7:02 ` Tony Lindgren @ 2021-09-25 14:55 ` Jarkko Nikula 2021-09-30 7:04 ` Tony Lindgren 0 siblings, 1 reply; 21+ messages in thread From: Jarkko Nikula @ 2021-09-25 14:55 UTC (permalink / raw) To: Tony Lindgren Cc: linux-omap, Laurent Pinchart, H. Nikolaus Schaller, Andreas Kemnade On 9/24/21 10:02 AM, Tony Lindgren wrote: > * Jarkko Nikula <jarkko.nikula@bitmer.com> [210923 18:04]: >> On 9/23/21 8:41 AM, Tony Lindgren wrote: >>> * Jarkko Nikula <jarkko.nikula@bitmer.com> [210922 17:22]: >>>> Better luck with this one but looks like idling cause "undefined >>>> instruction" crash. Cache/memory etc corruption perhaps? Serial console >>>> log attached. >>> >>> Hmm. If you comment out the twl power node, does the omap3-beagle-ab4.dtb >>> boot normally for you? It should behave the same as current mainline then >>> with the omap3isp issue. >>> >> Commenting the twl_power node out or removing the twl section leads to >> immediately rebooting kernel: > > Below is an updated patch to move the timer quirks to omap3-beagle-ab4.dtb > with no twl changes. > Sigh, I found the reason for immediately rebooting kernel from bash history: "rm arch/arm/boot/zImage" followed by "cat arch/arm/boot/dts/omap3-beagle-ab4.dtb >>arch/arm/boot/zImage; ma LOADADDR=0x80008000 uImage", i.e. forgot to build the zImage between. (alias ma='nice make -j `getconf _NPROCESSORS_ONLN` ARCH=arm CROSS_COMPILE="ccache arm-linux-gnueabihf-"') So yes, with your latest patch omap3-beagle-ab4.dtb behaves as unpatched omap3-beagle.dtb (boots but timer issues when omap3_isp not loaded). Jarkko ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-25 14:55 ` Jarkko Nikula @ 2021-09-30 7:04 ` Tony Lindgren 0 siblings, 0 replies; 21+ messages in thread From: Tony Lindgren @ 2021-09-30 7:04 UTC (permalink / raw) To: Jarkko Nikula Cc: linux-omap, Laurent Pinchart, H. Nikolaus Schaller, Andreas Kemnade * Jarkko Nikula <jarkko.nikula@bitmer.com> [210925 14:56]: > So yes, with your latest patch omap3-beagle-ab4.dtb behaves as unpatched > omap3-beagle.dtb (boots but timer issues when omap3_isp not loaded). OK great. So now we still need to figure out why rev c is not booting without the quirks for the fix. Regards, Tony ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-21 7:00 ` Tony Lindgren 2021-09-21 17:49 ` Jarkko Nikula @ 2021-10-20 17:52 ` Jarkko Nikula 2021-10-21 13:58 ` Tony Lindgren 1 sibling, 1 reply; 21+ messages in thread From: Jarkko Nikula @ 2021-10-20 17:52 UTC (permalink / raw) To: Tony Lindgren; +Cc: linux-omap, Laurent Pinchart Hi On 9/21/21 10:00 AM, Tony Lindgren wrote: > * Jarkko Nikula <jarkko.nikula@bitmer.com> [210920 16:35]: >> On 9/20/21 4:34 PM, Tony Lindgren wrote: >>> The patch below should take us back to the earlier configuration, >>> can you please give it a try? >>> >> I double checked and unfortunate it doesn't boot at all or give any >> output when patch is applied: > > Hmm I tested it with copying the beagle timer configuration to omap3.dtsi > and things worked for me. If you can, please try adding > CONFIG_SERIAL_EARLYCON=y and then also add "debug earlyprintk" to kernel > command line. It might show something that I'm not able to reproduce. > Maybe there is a bootloader dependency or something like that. > As discussed offline recently I triple checked your first patch in this thread on top of v5.15-rc6 and using omap2plus_defconfig (has CONFIG_SERIAL_EARLYCON=y) and still I'm not able to get any output with earlyprintk. My kernel command line included "console=ttyO2,115200n8 debug earlyprintk=ttyO2,115200n8". Plain v5.15-rc6 boots. Could it be possible the "secure_32k_fck" clock has some system hanging issue on my beagle? How I understand from e428e250fde6 ("ARM: dts: Configure system timers for omap3") that "secure_32k_fck" is not referenced explicitly but your patch here does. Jarkko ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-10-20 17:52 ` Jarkko Nikula @ 2021-10-21 13:58 ` Tony Lindgren 2021-10-21 18:10 ` Jarkko Nikula 0 siblings, 1 reply; 21+ messages in thread From: Tony Lindgren @ 2021-10-21 13:58 UTC (permalink / raw) To: Jarkko Nikula; +Cc: linux-omap, Laurent Pinchart Hi, * Jarkko Nikula <jarkko.nikula@bitmer.com> [211020 17:53]: > Hi > > On 9/21/21 10:00 AM, Tony Lindgren wrote: > > * Jarkko Nikula <jarkko.nikula@bitmer.com> [210920 16:35]: > >> On 9/20/21 4:34 PM, Tony Lindgren wrote: > >>> The patch below should take us back to the earlier configuration, > >>> can you please give it a try? > >>> > >> I double checked and unfortunate it doesn't boot at all or give any > >> output when patch is applied: > > > > Hmm I tested it with copying the beagle timer configuration to omap3.dtsi > > and things worked for me. If you can, please try adding > > CONFIG_SERIAL_EARLYCON=y and then also add "debug earlyprintk" to kernel > > command line. It might show something that I'm not able to reproduce. > > Maybe there is a bootloader dependency or something like that. > > > As discussed offline recently I triple checked your first patch in this > thread on top of v5.15-rc6 and using omap2plus_defconfig (has > CONFIG_SERIAL_EARLYCON=y) and still I'm not able to get any output with > earlyprintk. Oh looks like omap3-beagle variants are missing the stdout-path that allows CONFIG_SERIAL_EARLYCON=y with "earlycon" in the kernel command line to show early serial debug output. Can you try with the patch below? > My kernel command line included "console=ttyO2,115200n8 debug > earlyprintk=ttyO2,115200n8". Plain v5.15-rc6 boots. You also need to add "earlycon" to the command line. You could just add "debug earlycon earlyprintk" if you ever need these and debug is not too noisy for you :) With the stdout-path, you don't need to specify earlyprintk. You should also update your command line to use use ttyS2 instead of ttyO2 to avoid extra warnings while at it. And make sure your .config has CONFIG_SERIAL_8250_OMAP=y. These should not affect the earlycon output AFAIK though. > Could it be possible the "secure_32k_fck" clock has some system hanging > issue on my beagle? How I understand from e428e250fde6 ("ARM: dts: > Configure system timers for omap3") that "secure_32k_fck" is not > referenced explicitly but your patch here does. Yes seems like there may be some extra issue on the omap3-beagle boards. Seeing the earlycon output would be good to see. Regards, Tony 8< ----------------- From tony Mon Sep 17 00:00:00 2001 From: Tony Lindgren <tony@atomide.com> Date: Thu, 21 Oct 2021 16:48:30 +0300 Subject: [PATCH] ARM: dts: Add missing stdout-path for omap3-beagle variants This allows seeing early serial console output as long as the kernel config has CONFIG_SERIAL_EARLYCON=y, and the kernel command line has earlycon. Signed-off-by: Tony Lindgren <tony@atomide.com> --- arch/arm/boot/dts/omap3-beagle-xm.dts | 4 ++++ arch/arm/boot/dts/omap3-beagle.dts | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/arch/arm/boot/dts/omap3-beagle-xm.dts b/arch/arm/boot/dts/omap3-beagle-xm.dts --- a/arch/arm/boot/dts/omap3-beagle-xm.dts +++ b/arch/arm/boot/dts/omap3-beagle-xm.dts @@ -21,6 +21,10 @@ memory@80000000 { reg = <0x80000000 0x20000000>; /* 512 MB */ }; + chosen { + stdout-path = &uart3; + }; + aliases { display0 = &dvi0; display1 = &tv0; diff --git a/arch/arm/boot/dts/omap3-beagle.dts b/arch/arm/boot/dts/omap3-beagle.dts --- a/arch/arm/boot/dts/omap3-beagle.dts +++ b/arch/arm/boot/dts/omap3-beagle.dts @@ -21,6 +21,10 @@ memory@80000000 { reg = <0x80000000 0x10000000>; /* 256 MB */ }; + chosen { + stdout-path = &uart3; + }; + aliases { display0 = &dvi0; display1 = &tv0; -- 2.33.0 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-10-21 13:58 ` Tony Lindgren @ 2021-10-21 18:10 ` Jarkko Nikula 2021-10-24 6:41 ` Tony Lindgren 0 siblings, 1 reply; 21+ messages in thread From: Jarkko Nikula @ 2021-10-21 18:10 UTC (permalink / raw) To: Tony Lindgren; +Cc: linux-omap, Laurent Pinchart On 10/21/21 4:58 PM, Tony Lindgren wrote: >> Could it be possible the "secure_32k_fck" clock has some system hanging >> issue on my beagle? How I understand from e428e250fde6 ("ARM: dts: >> Configure system timers for omap3") that "secure_32k_fck" is not >> referenced explicitly but your patch here does. > > Yes seems like there may be some extra issue on the omap3-beagle > boards. Seeing the earlycon output would be good to see. > Here are the last lines with your earlier patch and patch below applied (my tested-by below): ... [ 0.000000] IRQ: Found an INTC at 0x(ptrval) (revision 4.0) with 96 interrupts [ 0.000000] random: get_random_bytes called from start_kernel+0x464/0x6c4 with crng_init=0 [ 0.000000] Clocking rate (Crystal/Core/MPU): 26.0/332/600 MHz [ 0.000000] OMAP clocksource: 32k_counter at 32768 Hz [ 0.000000] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns [ 0.000000] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns [ 0.014617] TI gptimer clocksource: always-on /ocp@68000000/target-module@48304000 [ 0.022583] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns [ 0.032073] clocksource: dmtimer: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns [ 0.044372] Console: colour dummy device 80x30 > Subject: [PATCH] ARM: dts: Add missing stdout-path for omap3-beagle > variants > > This allows seeing early serial console output as long as the kernel > config has CONFIG_SERIAL_EARLYCON=y, and the kernel command line has > earlycon. > > Signed-off-by: Tony Lindgren <tony@atomide.com> You may add to this patch: Tested-by: Jarkko Nikula <jarkko.nikula@bitmer.com> ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-10-21 18:10 ` Jarkko Nikula @ 2021-10-24 6:41 ` Tony Lindgren 0 siblings, 0 replies; 21+ messages in thread From: Tony Lindgren @ 2021-10-24 6:41 UTC (permalink / raw) To: Jarkko Nikula; +Cc: linux-omap, Laurent Pinchart * Jarkko Nikula <jarkko.nikula@bitmer.com> [211021 18:10]: > On 10/21/21 4:58 PM, Tony Lindgren wrote: > >> Could it be possible the "secure_32k_fck" clock has some system hanging > >> issue on my beagle? How I understand from e428e250fde6 ("ARM: dts: > >> Configure system timers for omap3") that "secure_32k_fck" is not > >> referenced explicitly but your patch here does. > > > > Yes seems like there may be some extra issue on the omap3-beagle > > boards. Seeing the earlycon output would be good to see. > > > Here are the last lines with your earlier patch and patch below applied > (my tested-by below): OK thanks. > ... > [ 0.000000] IRQ: Found an INTC at 0x(ptrval) (revision 4.0) with 96 > interrupts > [ 0.000000] random: get_random_bytes called from > start_kernel+0x464/0x6c4 with crng_init=0 > [ 0.000000] Clocking rate (Crystal/Core/MPU): 26.0/332/600 MHz > [ 0.000000] OMAP clocksource: 32k_counter at 32768 Hz > [ 0.000000] clocksource: 32k_counter: mask: 0xffffffff max_cycles: > 0xffffffff, max_idle_ns: 58327039986419 ns > [ 0.000000] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps > every 65535999984741ns > [ 0.014617] TI gptimer clocksource: always-on > /ocp@68000000/target-module@48304000 > [ 0.022583] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps > every 65535999984741ns > [ 0.032073] clocksource: dmtimer: mask: 0xffffffff max_cycles: > 0xffffffff, max_idle_ns: 58327039986419 ns > [ 0.044372] Console: colour dummy device 80x30 Hmm so this is with the gpt12 that should be only used for the a-b variants. Sure it should still work on later variants too though. Looks like the 32k timer for the gpt12 clocksource is working as the time is increasing, but the clockevent is not working somehow. Regards, Tony ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-19 17:41 Regression with e428e250fde6 on BeagleBoard Rev C2 Jarkko Nikula 2021-09-20 13:34 ` Tony Lindgren @ 2021-09-20 18:19 ` H. Nikolaus Schaller 2021-09-21 7:02 ` Tony Lindgren 1 sibling, 1 reply; 21+ messages in thread From: H. Nikolaus Schaller @ 2021-09-20 18:19 UTC (permalink / raw) To: Jarkko Nikula, Linux-OMAP Cc: Tony Lindgren, Laurent Pinchart, Andreas Kemnade Hi, > Am 19.09.2021 um 19:41 schrieb Jarkko Nikula <jarkko.nikula@bitmer.com>: > > I found another timer related regression on BeagleBoard Rev C2 which was I remembered that I have a BB C2 collecting dust... > present before 6cfcd5563b4f ("clocksource/drivers/timer-ti-dm: Fix > suspend and resume for am3 and am4") and which remains after fix > 3ff340e24c9d ("bus: ti-sysc: Fix gpt12 system timer issue with reserved > status") including today's head d4d016caa4b8 ("alpha: move __udiv_qrnnd > library function to arch/alpha/lib/"). > > Issue occurs when omap3_isp is not loaded and symptoms are the same than > my previous finding [1]. I.e. timer interrupts appear missing and need > to hit keys on serial console in order to let boot (when omap3_isp not > built) or "sleep 1" in shell to continue. After removing dust, I can confirm both effects after doing echo blacklist omap3_isp >/etc/modprobe.d/isp.conf and rebooting. Another effect I have seen is that heartbeat (I think it is a local patch I have in our devicetree) stops and toggles almost each time I press the enter key on the serial console. And, I once got attached log. Hope this helps. BR and thanks, Nikolaus root@letux:~# uname -a Linux letux 5.15.0-rc2-letux+ #6867 SMP PREEMPT Mon Sep 20 19:47:48 CEST 2021 armv7l GNU/Linux root@letux:~# root@letux:~# [ 444.397186] rcu: INFO: rcu_preempt self-detected stall on CPU [ 444.403594] rcu: 0-...!: (1 ticks this GP) idle=869/1/0x40000002 softirq=7514/7514 fqs=0 [ 444.412689] (t=5297 jiffies g=17433 q=5) [ 444.417144] rcu: rcu_preempt kthread timer wakeup didn't happen for 5296 jiffies! g17433 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 444.429595] rcu: Possible timer handling issue on cpu=0 timer-softirq=4616 [ 444.437225] rcu: rcu_preempt kthread starved for 5297 jiffies! g17433 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 444.448638] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 444.458618] rcu: RCU grace-period kthread stack dump: [ 444.464172] task:rcu_preempt state:I stack: 0 pid: 11 ppid: 2 flags:0x00000000 [ 444.473449] [<c089cd68>] (__schedule) from [<c089cfcc>] (schedule+0xdc/0x12c) [ 444.481353] [<c089cfcc>] (schedule) from [<c08a006c>] (schedule_timeout+0xc0/0xec) [ 444.489685] [<c08a006c>] (schedule_timeout) from [<c018e794>] (rcu_gp_fqs_loop+0x108/0x2d4) [ 444.498901] [<c018e794>] (rcu_gp_fqs_loop) from [<c018f940>] (rcu_gp_kthread+0xf0/0x128) [ 444.507812] [<c018f940>] (rcu_gp_kthread) from [<c0150878>] (kthread+0x14c/0x154) [ 444.516082] [<c0150878>] (kthread) from [<c0100150>] (ret_from_fork+0x14/0x24) [ 444.524047] Exception stack(0xc10dbfb0 to 0xc10dbff8) [ 444.529632] bfa0: ???????? ???????? ???????? ???????? [ 444.538574] bfc0: ???????? ???????? ???????? ???????? ???????? ???????? ???????? ???????? [ 444.547515] bfe0: ???????? ???????? ???????? ???????? ???????? ???????? [ 444.554779] rcu: Stack dump where RCU GP kthread last ran: [ 444.560821] NMI backtrace for cpu 0 [ 444.564697] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.0-rc2-letux+ #6867 [ 444.572540] Hardware name: Generic OMAP3-GP (Flattened Device Tree) [ 444.579406] [<c010f1a0>] (unwind_backtrace) from [<c010a47c>] (show_stack+0x10/0x14) [ 444.587951] [<c010a47c>] (show_stack) from [<c0896148>] (dump_stack_lvl+0x40/0x4c) [ 444.596313] [<c0896148>] (dump_stack_lvl) from [<c04cc82c>] (nmi_cpu_backtrace+0xd4/0xe8) [ 444.605316] [<c04cc82c>] (nmi_cpu_backtrace) from [<c04cc8f0>] (nmi_trigger_cpumask_backtrace+0xb0/0x184) [ 444.615814] [<c04cc8f0>] (nmi_trigger_cpumask_backtrace) from [<c0893c54>] (rcu_check_gp_kthread_starvation+0x118/0x150) [ 444.627716] [<c0893c54>] (rcu_check_gp_kthread_starvation) from [<c01926e8>] (rcu_sched_clock_irq+0x360/0xae0) [ 444.638702] [<c01926e8>] (rcu_sched_clock_irq) from [<c019b8f4>] (update_process_times+0x90/0xc4) [ 444.648468] [<c019b8f4>] (update_process_times) from [<c01ac714>] (tick_sched_handle+0x54/0x60) [ 444.658020] [<c01ac714>] (tick_sched_handle) from [<c01acb84>] (tick_sched_timer+0x44/0x9c) [ 444.667205] [<c01acb84>] (tick_sched_timer) from [<c019c520>] (__hrtimer_run_queues+0x17c/0x280) [ 444.676849] [<c019c520>] (__hrtimer_run_queues) from [<c019d25c>] (hrtimer_interrupt+0xb0/0x208) [ 444.686492] [<c019d25c>] (hrtimer_interrupt) from [<c0727df8>] (dmtimer_clockevent_interrupt+0x24/0x2c) [ 444.696807] [<c0727df8>] (dmtimer_clockevent_interrupt) from [<c017ebd4>] (__handle_irq_event_percpu+0xb8/0x200) [ 444.707977] [<c017ebd4>] (__handle_irq_event_percpu) from [<c017ed40>] (handle_irq_event_percpu+0x24/0x70) [ 444.718566] [<c017ed40>] (handle_irq_event_percpu) from [<c017edc4>] (handle_irq_event+0x38/0x5c) [ 444.728302] [<c017edc4>] (handle_irq_event) from [<c0183598>] (handle_level_irq+0x7c/0xb4) [ 444.737396] [<c0183598>] (handle_level_irq) from [<c017def0>] (handle_irq_desc+0x24/0x34) [ 444.746398] [<c017def0>] (handle_irq_desc) from [<c017e574>] (handle_domain_irq+0x3c/0x54) [ 444.755493] [<c017e574>] (handle_domain_irq) from [<c0100b3c>] (__irq_svc+0x5c/0x90) [ 444.764007] Exception stack(0xc0d01ed0 to 0xc0d01f18) [ 444.769561] 1ec0: ???????? ???????? ???????? ???????? [ 444.778503] 1ee0: ???????? ???????? ???????? ???????? ???????? ???????? ???????? ???????? [ 444.787445] 1f00: ???????? ???????? ???????? ???????? ???????? ???????? [ 444.794708] [<c0100b3c>] (__irq_svc) from [<c06f64cc>] (cpuidle_enter_state+0x1c4/0x2f8) [ 444.803619] [<c06f64cc>] (cpuidle_enter_state) from [<c06f6664>] (cpuidle_enter+0x50/0x54) [ 444.812713] [<c06f6664>] (cpuidle_enter) from [<c01603e4>] (do_idle+0x274/0x2a0) [ 444.820861] [<c01603e4>] (do_idle) from [<c0160714>] (cpu_startup_entry+0x18/0x1c) [ 444.829193] [<c0160714>] (cpu_startup_entry) from [<c0c0128c>] (start_kernel+0x770/0x958) [ 444.838409] NMI backtrace for cpu 0 [ 444.842285] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.0-rc2-letux+ #6867 [ 444.850128] Hardware name: Generic OMAP3-GP (Flattened Device Tree) [ 444.857025] [<c010f1a0>] (unwind_backtrace) from [<c010a47c>] (show_stack+0x10/0x14) [ 444.865570] [<c010a47c>] (show_stack) from [<c0896148>] (dump_stack_lvl+0x40/0x4c) [ 444.873901] [<c0896148>] (dump_stack_lvl) from [<c04cc82c>] (nmi_cpu_backtrace+0xd4/0xe8) [ 444.882904] [<c04cc82c>] (nmi_cpu_backtrace) from [<c04cc8f0>] (nmi_trigger_cpumask_backtrace+0xb0/0x184) [ 444.893402] [<c04cc8f0>] (nmi_trigger_cpumask_backtrace) from [<c0893dac>] (rcu_dump_cpu_stacks+0x120/0x144) [ 444.904174] [<c0893dac>] (rcu_dump_cpu_stacks) from [<c01926ec>] (rcu_sched_clock_irq+0x364/0xae0) [ 444.914001] [<c01926ec>] (rcu_sched_clock_irq) from [<c019b8f4>] (update_process_times+0x90/0xc4) [ 444.923736] [<c019b8f4>] (update_process_times) from [<c01ac714>] (tick_sched_handle+0x54/0x60) [ 444.933288] [<c01ac714>] (tick_sched_handle) from [<c01acb84>] (tick_sched_timer+0x44/0x9c) [ 444.942443] [<c01acb84>] (tick_sched_timer) from [<c019c520>] (__hrtimer_run_queues+0x17c/0x280) [ 444.952087] [<c019c520>] (__hrtimer_run_queues) from [<c019d25c>] (hrtimer_interrupt+0xb0/0x208) [ 444.961730] [<c019d25c>] (hrtimer_interrupt) from [<c0727df8>] (dmtimer_clockevent_interrupt+0x24/0x2c) [ 444.972045] [<c0727df8>] (dmtimer_clockevent_interrupt) from [<c017ebd4>] (__handle_irq_event_percpu+0xb8/0x200) [ 444.983215] [<c017ebd4>] (__handle_irq_event_percpu) from [<c017ed40>] (handle_irq_event_percpu+0x24/0x70) [ 444.993804] [<c017ed40>] (handle_irq_event_percpu) from [<c017edc4>] (handle_irq_event+0x38/0x5c) [ 445.003540] [<c017edc4>] (handle_irq_event) from [<c0183598>] (handle_level_irq+0x7c/0xb4) [ 445.012634] [<c0183598>] (handle_level_irq) from [<c017def0>] (handle_irq_desc+0x24/0x34) [ 445.021636] [<c017def0>] (handle_irq_desc) from [<c017e574>] (handle_domain_irq+0x3c/0x54) [ 445.030700] [<c017e574>] (handle_domain_irq) from [<c0100b3c>] (__irq_svc+0x5c/0x90) [ 445.039215] Exception stack(0xc0d01ed0 to 0xc0d01f18) [ 445.044799] 1ec0: ???????? ???????? ???????? ???????? [ 445.053741] 1ee0: ???????? ???????? ???????? ???????? ???????? ???????? ???????? ???????? [ 445.062683] 1f00: ???????? ???????? ???????? ???????? ???????? ???????? [ 445.069946] [<c0100b3c>] (__irq_svc) from [<c06f64cc>] (cpuidle_enter_state+0x1c4/0x2f8) [ 445.078857] [<c06f64cc>] (cpuidle_enter_state) from [<c06f6664>] (cpuidle_enter+0x50/0x54) [ 445.087951] [<c06f6664>] (cpuidle_enter) from [<c01603e4>] (do_idle+0x274/0x2a0) [ 445.096099] [<c01603e4>] (do_idle) from [<c0160714>] (cpu_startup_entry+0x18/0x1c) [ 445.104431] [<c0160714>] (cpu_startup_entry) from [<c0c0128c>] (start_kernel+0x770/0x958) root@letux:~# ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Regression with e428e250fde6 on BeagleBoard Rev C2 2021-09-20 18:19 ` H. Nikolaus Schaller @ 2021-09-21 7:02 ` Tony Lindgren 0 siblings, 0 replies; 21+ messages in thread From: Tony Lindgren @ 2021-09-21 7:02 UTC (permalink / raw) To: H. Nikolaus Schaller Cc: Jarkko Nikula, Linux-OMAP, Laurent Pinchart, Andreas Kemnade * H. Nikolaus Schaller <hns@goldelico.com> [210920 18:20]: > Another effect I have seen is that heartbeat (I think it is a local patch > I have in our devicetree) stops and toggles almost each time I press the enter > key on the serial console. Yeah this would be the case if the clockevent is not working properly. I assume this is with beagleboard rev c2 and not on other boards? Regards, Tony ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2021-10-24 6:41 UTC | newest] Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-09-19 17:41 Regression with e428e250fde6 on BeagleBoard Rev C2 Jarkko Nikula 2021-09-20 13:34 ` Tony Lindgren 2021-09-20 14:52 ` Andreas Kemnade 2021-09-21 7:00 ` Tony Lindgren 2021-09-21 16:25 ` Andreas Kemnade 2021-09-20 16:35 ` Jarkko Nikula 2021-09-21 7:00 ` Tony Lindgren 2021-09-21 17:49 ` Jarkko Nikula 2021-09-22 10:07 ` Tony Lindgren 2021-09-22 17:21 ` Jarkko Nikula 2021-09-23 5:41 ` Tony Lindgren 2021-09-23 18:03 ` Jarkko Nikula 2021-09-24 7:02 ` Tony Lindgren 2021-09-25 14:55 ` Jarkko Nikula 2021-09-30 7:04 ` Tony Lindgren 2021-10-20 17:52 ` Jarkko Nikula 2021-10-21 13:58 ` Tony Lindgren 2021-10-21 18:10 ` Jarkko Nikula 2021-10-24 6:41 ` Tony Lindgren 2021-09-20 18:19 ` H. Nikolaus Schaller 2021-09-21 7:02 ` Tony Lindgren
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.