Re: [PATCH] driver/perf: Add PMU driver for the ARM DMC-620 memory controller.

From: Robin Murphy <robin.murphy@arm.com>
To: Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>
Cc: Tuan Phan <tuanphan@os.amperecomputing.com>,
	linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	Tuan Phan <tuanphan@amperemail.onmicrosoft.com>
Subject: Re: [PATCH] driver/perf: Add PMU driver for the ARM DMC-620 memory controller.
Date: Wed, 1 Apr 2020 12:12:23 +0100	[thread overview]
Message-ID: <4d843ec7-ed74-4431-d8c7-d5aa6bd83c18@arm.com> (raw)
In-Reply-To: <20200401102724.GA17575@willie-the-truck>

On 2020-04-01 11:27 am, Will Deacon wrote:
> On Wed, Apr 01, 2020 at 10:52:26AM +0100, Mark Rutland wrote:
>> On Tue, Mar 31, 2020 at 03:14:59PM -0700, Tuan Phan wrote:
>>>> On Mar 20, 2020, at 4:25 AM, Mark Rutland <mark.rutland@arm.com> wrote:
>>>> On Thu, Mar 19, 2020 at 12:03:43PM -0700, Tuan Phan wrote:
>>>>>> On Mar 19, 2020, at 8:16 AM, Mark Rutland <mark.rutland@arm.com> wrote:
>>>>>> On Tue, Mar 17, 2020 at 05:29:38PM -0700, Tuan Phan wrote:
>>>>>>> +static int arm_dmc620_pmu_dev_init(struct arm_dmc620_pmu *dmc620_pmu)
>>>>>>> +{
>>>>>>> +	struct platform_device *pdev = dmc620_pmu->pdev;
>>>>>>> +	int ret;
>>>>>>> +
>>>>>>> +	ret = devm_request_irq(&pdev->dev, dmc620_pmu->irq,
>>>>>>> +				arm_dmc620_pmu_handle_irq,
>>>>>>> +				IRQF_SHARED,
>>>>>>> +				dev_name(&pdev->dev), dmc620_pmu);
>>>>>>
>>>>>> This should have IRQF_NOBALANCING | IRQF_NO_THREAD. I don't think we
>>>>>> should have IRQF_SHARED.
>>>>> => I agree on having IRQF_NOBALANCING and IRQF_NO_THREAD. But
>>>>> IRQF_SHARED is needed. In our platform all DMC620s share same IRQs and
>>>>> any cpus can access the pmu registers.
>>>>
>>>> Linux needs to ensure that the same instance is concistently accessed
>>>> from the same CPU, and needs to migrate the IRQ to handle that. Given we
>>>> do that on a per-instance basis, we cannot share the IRQ with another
>>>> instance.
>>>>
>>>> Please feed back to you HW designers that muxing IRQs like this causes
>>>> significant problems for software.
>>>
>>> I looked at the SMMUv3 PMU driver and it also uses IRQF_SHARED. SMMUv3
>>> PMU and DMC620 PMU are very much similar in which counters can be
>>> accessed by any cores using memory map. Any special reasons
>>> IRQF_SHARED works with SMMUv3 PMU driver?
>>
>> No; I believe that is a bug in the SMMUv3 PMU driver. If the IRQ were
>> shared, and another driver that held the IRQ changed the affinity,
>> things would go very wrong.
> 
> I *think* the idea is that the SMMUv3 PMU driver manages multiple PMCG
> devices, which may all share an irq line, rather than the irq line being
> shared by some other driver that might change the affinity. So I suspect
> dropping IRQF_SHARED will break things.

Each PMCG is conceptually a distinct PMU with its own interrupt - for 
instance, MMU-600 has one PMCG for its TCU and one for each TBU, each 
with a distinct interrupt output signal. Of course, integrators can and 
will mash them all together into a single SPI (particularly since 
they're all part of "the SMMU"), but that boils down to the same case as 
here.

This is going to continue to happen, so we could really do with figuring 
out a way to let MMIO system PMU drivers properly cope with shared 
interrupts in general :/

Robin.