From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S937986AbdLSGhZ (ORCPT ); Tue, 19 Dec 2017 01:37:25 -0500 Received: from mail-by2nam03on0058.outbound.protection.outlook.com ([104.47.42.58]:65120 "EHLO NAM03-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S933272AbdLSGhY (ORCPT ); Tue, 19 Dec 2017 01:37:24 -0500 Date: Tue, 19 Dec 2017 12:06:54 +0530 From: Linu Cherian To: Robin Murphy Cc: Linu Cherian , Neil Leeder , Will Deacon , Mark Rutland , Mark Langsdorf , Jon Masters , Timur Tabi , linux-kernel@vger.kernel.org, Mark Brown , Mark Salter , linux-arm-kernel@lists.infradead.org, Sunil.Goutham@cavium.com, ynorov@caviumnetworks.com, Marc Zyngier Subject: Re: [PATCH 0/2] arm64 SMMUv3 PMU driver with IORT support Message-ID: <20171219063654.GA13415@virtx40> References: <1501876754-1064-1-git-send-email-nleeder@codeaurora.org> <20171210023549.GA22492@virtx40> <999b7132-2de8-2872-1067-fc7e02cbc57a@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <999b7132-2de8-2872-1067-fc7e02cbc57a@arm.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: SN4PR0801CA0012.namprd08.prod.outlook.com (10.161.215.150) To DM5PR07MB3611.namprd07.prod.outlook.com (10.164.153.161) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 744223ae-aef5-45e5-b329-08d546aaef43 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(2017052603307);SRVR:DM5PR07MB3611; X-Microsoft-Exchange-Diagnostics: 1;DM5PR07MB3611;3:OUEeNnqHquOGKsfJrMO0sqIuX84ij+Oga1f/yLUL9QNimtaekAI5CHEOoJo+b+R5J1CMorv9aEh5OPPUklfGEwto84UpYBGBHioUMBNWvlRCWxgZmZ06HOQYJN7nj1nnsvphhhOj50Fk/q2dMuDIXryivBTs7NAoZG0qbWKkIhDXrBMwYLuy03ONz7qJAU4/uAV+OPTMYgnjfxRT12OSqvjaSgMkXAkET0/UA3qj/vPozSjEIgqPE7QQq4k30i9i;25:fDq6wcH/kpCmFEQ6uX2PBiMLnBXyLAR76LnbOP3u4hkHTjXtyuqaE2Dyf8RLQXWEwiv4XKczql7WbtGQSQHubfogCQBExE52ur2fj1BhDEfbsdrdiXdYQ872ObE8UjkcUk90lFO20EBflvbKFnjn+RZngqa06tuEcwxfL+Zjb4i5q11SI7VLjUbhlj5ZjytSwNrK7G4wLGC/dvB8fNQvli50QC8N9NukK/m61zRpqDpPTmNFoBRnpg97IqMCdjDj9snRx341kwkW6+BXO5MtY/TFVzTNHHdj47OR67JRWG+flg+D0HYgIZlPevonY/l2zSODdXt7hy5U+m6vJjz44g==;31:EwgFDO6+ukAQc2Y1NkLIqJe3t/NI5/by8le7Vlln0Z29mSCMrCEQeO0RmVen3WT8NjWtds7p4cc2jg0yWm6oc6nV8CMoY5SsfSNxdEPTRJAEitQwfD8QS2PP6PqjrzMpQch/LVn5zZkjRzfg2poTPcjt9PptrYMPyMhrceq1uDXiyRaxINczYD/zFWRcg0/Bp4t0HeNJYigrT217UF1JNlkMK0idmiyq0YU6cgflBWg= X-MS-TrafficTypeDiagnostic: DM5PR07MB3611: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Linu.Cherian@cavium.com; X-Microsoft-Exchange-Diagnostics: 1;DM5PR07MB3611;20:T7VdVoNb4j7VXte8CozHMWCDxtXU7/aC6yYcIQSSSRyW6JZY/xMuhsgZVZU2a35qjTZoqqtF90pV/pcHTF8Dnpgz0NPbgfWgyAFuhJemcCkXdGPuPj9Kabi2amth6MC7LUkNEHzJZlCiPGy9rCaMC1hlllKe+flSCXRCm3HB3ip+yIejjLZlRG2IBqKxIE6BZrnvgpze834wGPwqQdARunodFu/d77fGGmV0rwEDiHqnM6qjMDeLLdmCqoEYXQSrSz5KlCUar+kPIu0GM6Arv6mGtS8gZWi/kUpETlHGRku+0DZ9ZNQzRO+ttEAeziGXXAV07UxVPL/NqklpSEm0OPQ90HI/mYRNafOGlnwny4wbeac8C77ZtahnYbNkbGEW1zzgB6buYJejIGDzwEpFfjWbTw/boZzkS+/SQJrSvl+ODnm7JUCed6n7xQ3Fh5NuV+3a0hpgt0WE7PJcHWOcw4UkNiUqEblts1mgKtnXjsBa6PoNcuega426VoWyYnG4;4:tYC52X6UmWSdojxM4DC2qV/z+pEB/gzM/bqj5z0a9IH42h5cS6kn7sZcJMbyLn8aU/tegy+rShwRxli4IeclW6cSvpDXW81Y6x8EdX4gRE0kffQQDjhpHA47na6PAcD2t9rRgD4aTp+DVxxG6BxMnE/BL5qppV1/OBfv1OflDcqSwAd2zTpnMbZWXC7D4QUa5O1hPbcm8hn0L7edtU+xkKNcWPrYi7XJEbnwXLGazkqtxe91CPF4jQcetafy2BQHKvW0xC5ZKpnebeU2w4qGlA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040450)(2401047)(8121501046)(5005006)(3231023)(3002001)(10201501046)(93006095)(93001095)(6041248)(20161123558100)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123564025)(20161123555025)(6072148)(201708071742011);SRVR:DM5PR07MB3611;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:DM5PR07MB3611; X-Forefront-PRVS: 052670E5A4 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6069001)(366004)(39860400002)(396003)(346002)(376002)(24454002)(199004)(189003)(305945005)(7736002)(50466002)(316002)(5660300001)(23726003)(42186006)(54906003)(16586007)(53936002)(2950100002)(6916009)(58126008)(52116002)(8676002)(81156014)(47776003)(81166006)(9686003)(2906002)(33716001)(83506002)(7416002)(33656002)(6116002)(3846002)(1076002)(33896004)(8936002)(72206003)(122856001)(68736007)(6246003)(39060400002)(386003)(53546011)(59450400001)(4326008)(66066001)(106356001)(105586002)(478600001)(76176011)(97736004)(86362001)(229853002)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:DM5PR07MB3611;H:localhost;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;DM5PR07MB3611;23:9NszeLHF1OK/cZAx6C+gVYD/RWTC+R0qDnNTzsNFJ?= =?us-ascii?Q?zf3QAe2JPZmlA9nm2YKXL5Y8KF/bep9YuhbNcjRF8Jk5nQPhpQyHqyiDpn2q?= =?us-ascii?Q?wH6eQtxxvUBwRGSx0fmdB3cLjG8N1i2v34eGvKxFhv591qTfAkwMtwQ1Avup?= =?us-ascii?Q?5Ch3rdE8KxlggbJe5UD1KEhv9JYZNNoLYTGP+Nq17PmEPev0wwVz0OL9vVGW?= =?us-ascii?Q?+k44fb5msqyJiTZN1QhHIC7UOtHopPH8li5qx5SkyzEh5wYzZs5EwA/WOebZ?= =?us-ascii?Q?O5IGF606EmcK7dDy9RRfuV8ry1KpFpFyqhAx8Nl2dB2n9al72hs87Dk4RJU2?= =?us-ascii?Q?J8iIBQMoRaYt1uTYsucYg/w0EwbnS2MkmAnkTleOtydcAlJ/+hBuEs18N2eo?= =?us-ascii?Q?WHRfc7uHAWZCxOBFIUItWc26YkJ/eaUe1Pq+PwyBAIM/O0abfKsu0TTL3kfk?= =?us-ascii?Q?8L1b8Wj5BtEGbNepxEZECcaI9NHDDei5LmkICDZDF62U2FuPNMfZJTISMhc/?= =?us-ascii?Q?CxuLsw/nnlH7cwi/k9cSWqcOeN+GuLuenJ4khGIgJaNWfuelSuH42za0raJt?= =?us-ascii?Q?YS4pAdw39eCmd7Kd/K/MXcuXxNubpS43AKIee2Yf8YvZA6sntU1WtIqDxK7W?= =?us-ascii?Q?r/p+dVaaxk16rKDHV1Ywy9G0w8i9CooyzUyrnNKoLZuul+gr2CgjB+03R4dQ?= =?us-ascii?Q?LcdSSWpraWfXLLARtcM6aagyVKHnVqLp3dr8TzQ1KlXc+Duilisdog/A1bbL?= =?us-ascii?Q?hixktAjx7fV+SF457ECcfH75U4Km8lra3/Xzs56Orbo9Jo5GH+2C7TYzIx+d?= =?us-ascii?Q?7qz0O+VlC+8Yc4mvTa4bOz/MeANtf6NDR/u+e8YHkPKcvdmmyrgUsRpurwt4?= =?us-ascii?Q?0kVmGNMG/cwIJcYpZLmWtf0W3r2TUo5NCfmQBOH3x6PDlc+6euePPjUtbgvL?= =?us-ascii?Q?qIrHu2us4V3enaO9vICpEjeqSyPKhwJPLaom9BVJmnU/ms5QV230Oz9wOAoa?= =?us-ascii?Q?cPtgQToK1586IySKJs8QB3JWLef68VohXP8CdDBTtUEhu6GJpQTyOkgczMrq?= =?us-ascii?Q?+jKcL6jCAVvq6P3VaZcSJivQ85EVL5Ctm4csaFKZScZzdURACwPCOoHv9Sd3?= =?us-ascii?Q?RhlN3CaXDOPg1RhizPtHPpFAH6CC4gN+Fiv3N0RPRTYyjlKOexV5aLJpN29W?= =?us-ascii?Q?38XPtIv+4dR0rvFpnXbmvKdGHBA7mRXPnuO33cltVL7el9CuH0m2Fv5/ZcJ0?= =?us-ascii?Q?/2ron4EgXLOtHuq7aUZgXFHAlSYhVyBpfcdx7uWWHDx78ANU7UR5ZjGteSWb?= =?us-ascii?B?QT09?= X-Microsoft-Exchange-Diagnostics: 1;DM5PR07MB3611;6:Rd+KW6d5+r9FBJoEPk9Vn5Oce7TiA45V5qscuh2OV0mJDpln+lAPdBvIbXSMjkqfdC4XIpFzyI8N7SUjbynlRDkvtOCbtaFRNvI13bvcUBKmOZH5WsLhBgP/cgE1baYumUxZdAk0p3mdjYjv75UCQrP/xYACziGyHZP2rW3VM5C0Sua7aCQ3HQMZ9rxFceNLchkR1UeR5Cdjd+oRHL9hRo6ka5PfYmrOoEZLqhgqtLyDGCtwjk6h1yd1EzF95qIAlaNh/mFBijCMPEyAZ+1mXMDpxdxFEKWh/BkbS2cGQ9HDO01zLCBe20BBPL41RUlk0Y9uuvszByJDgHRCmyGKv9ojocn6YwW3A3rUChqXgXE=;5:A5HcEtrsrwO2Ctun2TBer4DrEFbROvWXVUC1PpC6hH5bNGocFipvWjouvu6sXV//CZbFTHkuetSDKBXFf4eDSF6VkI10PGDkv9T6hlbjLyy2ASI3zxWEs579qWiDTagXOQ4UhPWod9ioSKM6m5pShvHMt4ARwuIfSIrLcQhlEss=;24:Pxa2aNsqC2v8+/ihaAG+zbFWCc8VaQczDRr1/cIq5cb5NsVfgX4I1oJcwfcaJQ67xja7i7jpFls5bFTxUPJZlU7dG3MH84bdby6AbbXPmq0=;7:4cOm6NvuwLY144YmEJGHoEehdZriANAbBZ/KdDPD9Cq7UFSSeAvPA/zaCIZTGw5CEIyI8jJ1Yhn0qRhREvmriyESDKjLRoiU/wyfoNuF7h7R4XJEzs/sJDKs/kg49Oab/7nfPuu8RgFv4ubrR5/c+yr9hLdbJm15E7LMiwpgfV0XA9NHhVkojMiHvJWoK+Xs+VY8teeMCrfMEt7c5Oxs7UH9RdLBR40xKCtfudtM8E43HIvsNdhr93YAIryY2jMy SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Dec 2017 06:37:06.8181 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 744223ae-aef5-45e5-b329-08d546aaef43 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR07MB3611 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Robin, On Mon Dec 18, 2017 at 02:48:14PM +0000, Robin Murphy wrote: > On 10/12/17 02:35, Linu Cherian wrote: > >Hi, > > > > > >On Fri Aug 04, 2017 at 03:59:12PM -0400, Neil Leeder wrote: > >>This adds a driver for the SMMUv3 PMU into the perf framework. > >>It includes an IORT update to support PM Counter Groups. > >> > > > >In one of Cavium's upcoming SOC, SMMU PMCG implementation is such a way > >that platform bus id (Device ID in ITS terminmology)is shared with that of SMMU. > >This would be a matter of concern for software if the SMMU and SMMU PMCG blocks > >are managed by two independent drivers. > > > >The problem arises when we want to alloc/free MSIs for these devices > >using the APIs, platform_msi_domain_alloc/free_irqs. > >Platform bus id being same for these two hardware blocks, they end up sharing the same > >ITT(Interrupt Translation Table) in GIC ITS and hence alloc, free and management > >of this shared ITT becomes a problem when they are managed by two independent > >drivers. > > What is the problem exactly? IIRC resizing a possibly-live ITT is a > right pain in the bum to do - is it just that? Yes exactly. Resizing ITT was the problem in sharing. > >We were looking into the option of keeping the SMMU PMCG nodes as sub nodes under > >SMMUv3 node, so that SMMUv3 driver could probe and figure out the total vectors > >required for SMMU PMCG devices and make a common platform_msi_domain_alloc/free_irqs > >function call for all devices that share the platform bus id. > > I'm not sure how scalable that approach would be, since it's not > entirely obvious how to handle PMCGs associated with named > components or root complexes (rather than directly with SMMU > instances). We certainly don't want to end up spraying similar PMCG > DevID logic around all manner of GPU/accelerator/etc. drivers in > future (whilst PMCGs for device TLBs will be expected to have > distinct IDs from their host devices, they could reasonably still > overlap with other PMCGs/SMMUs). > OK. While trying the above approach, we also felt that the code will become lot messier than actually thought. > >Would like to know your expert opinion on what would be the right approach > >to handle this case ? > > My gut feeling says the way to deal with this properly is in the ITS > code, but I appreciate that that's a lot easier said than done :/ > Yes Correct. > Robin. -- Linu cherian From mboxrd@z Thu Jan 1 00:00:00 1970 From: linu.cherian@cavium.com (Linu Cherian) Date: Tue, 19 Dec 2017 12:06:54 +0530 Subject: [PATCH 0/2] arm64 SMMUv3 PMU driver with IORT support In-Reply-To: <999b7132-2de8-2872-1067-fc7e02cbc57a@arm.com> References: <1501876754-1064-1-git-send-email-nleeder@codeaurora.org> <20171210023549.GA22492@virtx40> <999b7132-2de8-2872-1067-fc7e02cbc57a@arm.com> Message-ID: <20171219063654.GA13415@virtx40> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org Hi Robin, On Mon Dec 18, 2017 at 02:48:14PM +0000, Robin Murphy wrote: > On 10/12/17 02:35, Linu Cherian wrote: > >Hi, > > > > > >On Fri Aug 04, 2017 at 03:59:12PM -0400, Neil Leeder wrote: > >>This adds a driver for the SMMUv3 PMU into the perf framework. > >>It includes an IORT update to support PM Counter Groups. > >> > > > >In one of Cavium's upcoming SOC, SMMU PMCG implementation is such a way > >that platform bus id (Device ID in ITS terminmology)is shared with that of SMMU. > >This would be a matter of concern for software if the SMMU and SMMU PMCG blocks > >are managed by two independent drivers. > > > >The problem arises when we want to alloc/free MSIs for these devices > >using the APIs, platform_msi_domain_alloc/free_irqs. > >Platform bus id being same for these two hardware blocks, they end up sharing the same > >ITT(Interrupt Translation Table) in GIC ITS and hence alloc, free and management > >of this shared ITT becomes a problem when they are managed by two independent > >drivers. > > What is the problem exactly? IIRC resizing a possibly-live ITT is a > right pain in the bum to do - is it just that? Yes exactly. Resizing ITT was the problem in sharing. > >We were looking into the option of keeping the SMMU PMCG nodes as sub nodes under > >SMMUv3 node, so that SMMUv3 driver could probe and figure out the total vectors > >required for SMMU PMCG devices and make a common platform_msi_domain_alloc/free_irqs > >function call for all devices that share the platform bus id. > > I'm not sure how scalable that approach would be, since it's not > entirely obvious how to handle PMCGs associated with named > components or root complexes (rather than directly with SMMU > instances). We certainly don't want to end up spraying similar PMCG > DevID logic around all manner of GPU/accelerator/etc. drivers in > future (whilst PMCGs for device TLBs will be expected to have > distinct IDs from their host devices, they could reasonably still > overlap with other PMCGs/SMMUs). > OK. While trying the above approach, we also felt that the code will become lot messier than actually thought. > >Would like to know your expert opinion on what would be the right approach > >to handle this case ? > > My gut feeling says the way to deal with this properly is in the ITS > code, but I appreciate that that's a lot easier said than done :/ > Yes Correct. > Robin. -- Linu cherian