On 01/11/2019 15:47, Andre Przywara wrote: > On 11/1/19 11:30 AM, Clément Péron wrote: Hi, for the sake of finishing this thread, in case search engines point to it: >> On Thu, 31 Oct 2019 at 21:35, Vasily Khoruzhick >> wrote: >>> >>> On Thu, Oct 31, 2019 at 12:10 PM Clément Péron >>> wrote: >>>> >>>> Hi, >>>> >>>> Just a remark here but the interrupt are from 152 to 155 SPI. >>>> But there is an offset of 32 no (remove SGI/PPI)? >>>> This should be from 120 to 123 >>> >>> I already tried it (and I believe someone already suggested it above), >>> it doesn't fix PMU interrupts though. >> >> Ok thanks for the confirmation. >> >> Made a research about the PMU for A64 and found that Andre Przywara >> made a patch to enable it: >> https://gist.github.com/apritzel/d025abaa1425fcaf5991b5ffcf18a0a3 >> >> Maybe he can confirm or not the issue on A64 ? > > Well, I tried it back then, but couldn't make the interrupts work (and > yes, I tried +/- 32). That's the reason I didn't send it back then. I did the experiment drafted below, and found that the interrupts are not 152-155, as the manual describes, but 148-151 instead. And yes, those are the GIC interrupt IDs used, but the GIC binding requires us to give the SPI numbers, so we need to subtract 32, which was not the case for the original patch. Thanks to Maxime for merging the fix patch: https://archive.armlinux.org.uk/lurker/message/20191105.110651.914455de.en.html In case someone wants to confirm this or in general needs to find/confirm IRQ numbers for some peripherals: > I can't say whether the IRQ lines are not wired or the manual just gives > the wrong numbers. I don't have access to a board before Sunday, but if > someone wants to beat me to it: > - Hack U-Boot to add a command to program one PMU counter to expire > quickly, and enable overflow interrupts. Attaching a U-Boot patch to trigger a PMU cycle counter overflow IRQ. > - Enable *all* SPIs on the GIC distributor level, and enable the > distributor. Keep the GIC CPU interface disabled. The GIC distributor on most Allwinner SoCs is located at 0x01c81000. This is the first address shown in the GIC's DT node. To enable all IRQs in U-Boot, on the U-Boot command prompt: => mw.l 0x01c81100 0xffffffff 0x20 This will set all possible 1024 bits in the GICD_ISENABLERn registers (offset 0x100) to 1. Reading this back will reveal which IRQs the GIC actually is configured for, typically we have much fewer SPIs supported. To enable the distributor (really: group 1 interrupts): => mw.l 0x01c81000 1 As suggested, we don't touch the GIC CPU interface, so no interrupts will actually reach the core. > - Trigger the U-Boot command, and inspect the GICD_ISPENDR registers to > see if any SPI fired. Trigger the IRQ, for instance using the command provided by the patch above. Keep the PMU enabled, because it will deassert the IRQ lines otherwise: => pmuc start => pmuc (check that the counter overflows) => md.l 0x01c81200 0x20 This will print the pending bits (GICD_ISPENDRn at offset 0x200) for each IRQ. In U-Boot we expect no other IRQs to ever fire, so any bit set in here would be due to our experiment. In this case the output looked like: 01c81200: 00000000 00000000 00000000 00000000 01c81210: 00100000 00000000 00000000 00000000 If you do the counting, this is bit 20 in word 4, so 4 * 32 + 20 = 148. Subtract 32 to get the number that goes into the DT node. U-Boot is UP, so this is probably the IRQ on core 0. A reasonable assumption is that the other cores just follow behind this, which you can confirm in Linux, by running some perf command on only a single core and watching the interrupt count increasing: $ grep pmu /proc/interrupts $ taskset -cp 1 $$ $ perf record sleep 3 $ grep pmu /proc/interrupts > - Also double check the PMU overflow status register to verify that the > event triggered. Just calling "pmuc" will print the PMOVSCLR register. Hope that helps! Cheers, Andre >>>> On Wed, 25 Sep 2019 at 13:09, Maxime Ripard wrote: >>>>> >>>>> On Mon, Sep 23, 2019 at 04:55:59PM -0700, Vasily Khoruzhick wrote: >>>>>> On Mon, Sep 23, 2019 at 4:51 PM Vasily Khoruzhick >>>>>> wrote: >>>>>>> >>>>>>> On Mon, Aug 12, 2019 at 10:39 PM Maxime Ripard >>>>>>> wrote: >>>>>>>> >>>>>>>> On Mon, Aug 12, 2019 at 11:01:51AM -0700, Vasily Khoruzhick wrote: >>>>>>>>> On Mon, Aug 12, 2019 at 1:04 AM Maxime Ripard >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> On Thu, Aug 08, 2019 at 12:59:07PM -0700, Vasily Khoruzhick >>>>>>>>>> wrote: >>>>>>>>>>> On Thu, Aug 8, 2019 at 9:26 AM Maxime Ripard >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Aug 07, 2019 at 10:36:08AM -0700, Vasily Khoruzhick >>>>>>>>>>>> wrote: >>>>>>>>>>>>> On Wed, Aug 7, 2019 at 4:56 AM Maxime Ripard >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Aug 06, 2019 at 07:39:26PM -0700, Vasily >>>>>>>>>>>>>> Khoruzhick wrote: >>>>>>>>>>>>>>> On Tue, Aug 6, 2019 at 2:14 PM Robin Murphy >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2019-08-06 9:52 pm, Vasily Khoruzhick wrote: >>>>>>>>>>>>>>>>> On Tue, Aug 6, 2019 at 1:19 PM Harald Geyer >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Vasily Khoruzhick writes: >>>>>>>>>>>>>>>>>>> On Tue, Aug 6, 2019 at 7:35 AM Robin Murphy >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 06/08/2019 15:01, Vasily Khoruzhick wrote: >>>>>>>>>>>>>>>>>>>>> Looks like PMU in A64 is broken, it generates no >>>>>>>>>>>>>>>>>>>>> interrupts at all and >>>>>>>>>>>>>>>>>>>>> as result 'perf top' shows no events. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Does something like 'perf stat sleep 1' at least >>>>>>>>>>>>>>>>>>>> count cycles correctly? >>>>>>>>>>>>>>>>>>>> It could well just be that the interrupt numbers are >>>>>>>>>>>>>>>>>>>> wrong... >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Looks like it does, at least result looks plausible: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm using perf stat regularly (cache benchmarks) and >>>>>>>>>>>>>>>>>> it works fine. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Unfortunately I wasn't aware that perf stat is a poor >>>>>>>>>>>>>>>>>> test for >>>>>>>>>>>>>>>>>> the interrupts part of the node, when I added it. So >>>>>>>>>>>>>>>>>> I'm not too >>>>>>>>>>>>>>>>>> surprised I got it wrong. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> However, it would be unfortunate if the node got >>>>>>>>>>>>>>>>>> removed completely, >>>>>>>>>>>>>>>>>> because perf stat would not work anymore. Maybe we can >>>>>>>>>>>>>>>>>> only remove >>>>>>>>>>>>>>>>>> the interrupts or just fix them even if the HW doesn't >>>>>>>>>>>>>>>>>> work? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I'm not familiar with PMU driver. Is it possible to get >>>>>>>>>>>>>>>>> it working >>>>>>>>>>>>>>>>> without interrupts? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yup - you get a grumpy message from the driver, it will >>>>>>>>>>>>>>>> refuse sampling >>>>>>>>>>>>>>>> events (the ones which weren't working anyway), and if >>>>>>>>>>>>>>>> you measure >>>>>>>>>>>>>>>> anything for long enough that a counter overflows you'll >>>>>>>>>>>>>>>> get wonky >>>>>>>>>>>>>>>> results. But for counting hardware events over >>>>>>>>>>>>>>>> relatively short periods >>>>>>>>>>>>>>>> it'll still do the job. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I tried to drop interrupts completely from the node but >>>>>>>>>>>>>>> 'perf top' is >>>>>>>>>>>>>>> still broken. Though now in different way: it complains >>>>>>>>>>>>>>> "cycles: PMU >>>>>>>>>>>>>>> Hardware doesn't support sampling/overflow-interrupts. >>>>>>>>>>>>>>> Try 'perf >>>>>>>>>>>>>>> stat'" >>>>>>>>>>>>>> >>>>>>>>>>>>>> I have no idea if that's the culprit, but what is the >>>>>>>>>>>>>> state of the >>>>>>>>>>>>>> 0x09010000 register? >>>>>>>>>>>>> >>>>>>>>>>>>> What register is that and how do I check it? >>>>>>>>>>>> >>>>>>>>>>>> It's in the CPUX Configuration block, and the bits are >>>>>>>>>>>> labelled as CPU >>>>>>>>>>>> Debug Reset. >>>>>>>>>>>> >>>>>>>>>>>> And if you have busybox, you can use devmem. >>>>>>>>>>> >>>>>>>>>>> CPUX configuration block is at 0x01700000 according to A64 user >>>>>>>>>>> manual, and particular register you're interested in is at >>>>>>>>>>> 0x01700080, >>>>>>>>>>> its value is 0x1110110F. >>>>>>>>>>> >>>>>>>>>>> Bits 16-19 are not defined in user manual and are not set. >>>>>>>>>> >>>>>>>>>> Sorry, I somehow thought this was for the H6... >>>>>>>>>> >>>>>>>>>> I don't have any idea then :/ >>>>>>>>> >>>>>>>>> OK, so what should we do? 'perf top'/'perf record' work fine if >>>>>>>>> PMU >>>>>>>>> node is dropped, but they don't work if PMU node is present >>>>>>>>> (even with >>>>>>>>> interrupts dropped). I'd prefer to have 'perf top' and 'perf >>>>>>>>> record' >>>>>>>>> working instead of 'perf stat' >>>>>>>> >>>>>>>> Well, it doesn't work so we should just remove the node, and if >>>>>>>> someone wants it back, they should figure it out. >>>>>>> >>>>>>> Hey Maxime, >>>>>>> >>>>>>> So can you merge this patch? >>>>>> >>>>>> Added new Maxime's email to CC >>>>> >>>>> Queued as a fix for 5.4, thanks! >>>>> Maxime >>>>> _______________________________________________ >>>>> linux-arm-kernel mailing list >>>>> linux-arm-kernel@lists.infradead.org >>>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel >