From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758760AbdLROsY (ORCPT ); Mon, 18 Dec 2017 09:48:24 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:51490 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758710AbdLROsS (ORCPT ); Mon, 18 Dec 2017 09:48:18 -0500 Subject: Re: [PATCH 0/2] arm64 SMMUv3 PMU driver with IORT support To: Linu Cherian , Neil Leeder Cc: Will Deacon , Mark Rutland , Mark Langsdorf , Jon Masters , Timur Tabi , linux-kernel@vger.kernel.org, Mark Brown , Mark Salter , linux-arm-kernel@lists.infradead.org, linu.cherian@cavium.com, Sunil.Goutham@cavium.com, ynorov@caviumnetworks.com, Marc Zyngier References: <1501876754-1064-1-git-send-email-nleeder@codeaurora.org> <20171210023549.GA22492@virtx40> From: Robin Murphy Message-ID: <999b7132-2de8-2872-1067-fc7e02cbc57a@arm.com> Date: Mon, 18 Dec 2017 14:48:14 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <20171210023549.GA22492@virtx40> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/12/17 02:35, Linu Cherian wrote: > Hi, > > > On Fri Aug 04, 2017 at 03:59:12PM -0400, Neil Leeder wrote: >> This adds a driver for the SMMUv3 PMU into the perf framework. >> It includes an IORT update to support PM Counter Groups. >> > > In one of Cavium's upcoming SOC, SMMU PMCG implementation is such a way > that platform bus id (Device ID in ITS terminmology)is shared with that of SMMU. > This would be a matter of concern for software if the SMMU and SMMU PMCG blocks > are managed by two independent drivers. > > The problem arises when we want to alloc/free MSIs for these devices > using the APIs, platform_msi_domain_alloc/free_irqs. > Platform bus id being same for these two hardware blocks, they end up sharing the same > ITT(Interrupt Translation Table) in GIC ITS and hence alloc, free and management > of this shared ITT becomes a problem when they are managed by two independent > drivers. What is the problem exactly? IIRC resizing a possibly-live ITT is a right pain in the bum to do - is it just that? > We were looking into the option of keeping the SMMU PMCG nodes as sub nodes under > SMMUv3 node, so that SMMUv3 driver could probe and figure out the total vectors > required for SMMU PMCG devices and make a common platform_msi_domain_alloc/free_irqs > function call for all devices that share the platform bus id. I'm not sure how scalable that approach would be, since it's not entirely obvious how to handle PMCGs associated with named components or root complexes (rather than directly with SMMU instances). We certainly don't want to end up spraying similar PMCG DevID logic around all manner of GPU/accelerator/etc. drivers in future (whilst PMCGs for device TLBs will be expected to have distinct IDs from their host devices, they could reasonably still overlap with other PMCGs/SMMUs). > Would like to know your expert opinion on what would be the right approach > to handle this case ? My gut feeling says the way to deal with this properly is in the ITS code, but I appreciate that that's a lot easier said than done :/ Robin. From mboxrd@z Thu Jan 1 00:00:00 1970 From: robin.murphy@arm.com (Robin Murphy) Date: Mon, 18 Dec 2017 14:48:14 +0000 Subject: [PATCH 0/2] arm64 SMMUv3 PMU driver with IORT support In-Reply-To: <20171210023549.GA22492@virtx40> References: <1501876754-1064-1-git-send-email-nleeder@codeaurora.org> <20171210023549.GA22492@virtx40> Message-ID: <999b7132-2de8-2872-1067-fc7e02cbc57a@arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On 10/12/17 02:35, Linu Cherian wrote: > Hi, > > > On Fri Aug 04, 2017 at 03:59:12PM -0400, Neil Leeder wrote: >> This adds a driver for the SMMUv3 PMU into the perf framework. >> It includes an IORT update to support PM Counter Groups. >> > > In one of Cavium's upcoming SOC, SMMU PMCG implementation is such a way > that platform bus id (Device ID in ITS terminmology)is shared with that of SMMU. > This would be a matter of concern for software if the SMMU and SMMU PMCG blocks > are managed by two independent drivers. > > The problem arises when we want to alloc/free MSIs for these devices > using the APIs, platform_msi_domain_alloc/free_irqs. > Platform bus id being same for these two hardware blocks, they end up sharing the same > ITT(Interrupt Translation Table) in GIC ITS and hence alloc, free and management > of this shared ITT becomes a problem when they are managed by two independent > drivers. What is the problem exactly? IIRC resizing a possibly-live ITT is a right pain in the bum to do - is it just that? > We were looking into the option of keeping the SMMU PMCG nodes as sub nodes under > SMMUv3 node, so that SMMUv3 driver could probe and figure out the total vectors > required for SMMU PMCG devices and make a common platform_msi_domain_alloc/free_irqs > function call for all devices that share the platform bus id. I'm not sure how scalable that approach would be, since it's not entirely obvious how to handle PMCGs associated with named components or root complexes (rather than directly with SMMU instances). We certainly don't want to end up spraying similar PMCG DevID logic around all manner of GPU/accelerator/etc. drivers in future (whilst PMCGs for device TLBs will be expected to have distinct IDs from their host devices, they could reasonably still overlap with other PMCGs/SMMUs). > Would like to know your expert opinion on what would be the right approach > to handle this case ? My gut feeling says the way to deal with this properly is in the ITS code, but I appreciate that that's a lot easier said than done :/ Robin.