From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9817AC433ED for ; Thu, 8 Apr 2021 11:08:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F5FB610E5 for ; Thu, 8 Apr 2021 11:08:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230515AbhDHLIm (ORCPT ); Thu, 8 Apr 2021 07:08:42 -0400 Received: from foss.arm.com ([217.140.110.172]:44076 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229803AbhDHLIl (ORCPT ); Thu, 8 Apr 2021 07:08:41 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3010F106F; Thu, 8 Apr 2021 04:08:30 -0700 (PDT) Received: from C02TD0UTHF1T.local (unknown [10.57.24.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3B5303F73D; Thu, 8 Apr 2021 04:08:26 -0700 (PDT) Date: Thu, 8 Apr 2021 12:08:15 +0100 From: Mark Rutland To: Will Deacon Cc: Rob Herring , Catalin Marinas , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Jiri Olsa , Ian Rogers , Alexander Shishkin , Honnappa Nagarahalli , Zachary.Leaf@arm.com, Raphael Gault , Jonathan Cameron , Namhyung Kim , Itaru Kitayama , linux-arm-kernel , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH v6 02/10] arm64: perf: Enable PMU counter direct access for perf event Message-ID: <20210408110800.GA32792@C02TD0UTHF1T.local> References: <20210311000837.3630499-1-robh@kernel.org> <20210311000837.3630499-3-robh@kernel.org> <20210330153125.GC6567@willie-the-truck> <20210331160059.GD7815@willie-the-truck> <20210407124437.GA15622@willie-the-truck> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210407124437.GA15622@willie-the-truck> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 07, 2021 at 01:44:37PM +0100, Will Deacon wrote: > [Moving Mark to To: since I'd like his view on this] > > On Thu, Apr 01, 2021 at 02:45:21PM -0500, Rob Herring wrote: > > On Wed, Mar 31, 2021 at 11:01 AM Will Deacon wrote: > > > > > > On Tue, Mar 30, 2021 at 12:09:38PM -0500, Rob Herring wrote: > > > > On Tue, Mar 30, 2021 at 10:31 AM Will Deacon wrote: > > > > > > > > > > On Wed, Mar 10, 2021 at 05:08:29PM -0700, Rob Herring wrote: > > > > > > From: Raphael Gault > > > > > > +static void armv8pmu_event_unmapped(struct perf_event *event, struct mm_struct *mm) > > > > > > +{ > > > > > > + struct arm_pmu *armpmu = to_arm_pmu(event->pmu); > > > > > > + > > > > > > + if (!(event->hw.flags & ARMPMU_EL0_RD_CNTR)) > > > > > > + return; > > > > > > + > > > > > > + if (atomic_dec_and_test(&mm->context.pmu_direct_access)) > > > > > > + on_each_cpu_mask(&armpmu->supported_cpus, refresh_pmuserenr, mm, 1); > > > > > > > > > > Given that the pmu_direct_access field is global per-mm, won't this go > > > > > wrong if multiple PMUs are opened by the same process but only a subset > > > > > are exposed to EL0? Perhaps pmu_direct_access should be treated as a mask > > > > > rather than a counter, so that we can 'and' it with the supported_cpus for > > > > > the PMU we're dealing with. > > > > > > > > It needs to be a count to support multiple events on the same PMU. If > > > > the event is not enabled for EL0, then we'd exit out on the > > > > ARMPMU_EL0_RD_CNTR check. So I think we're fine. > > > > > > I'm still not convinced; pmu_direct_access is shared between PMUs, so > > > testing the result of atomic_dec_and_test() just doesn't make sense to > > > me, as another PMU could be playing with the count. > > > > How is that a problem? Let's make a concrete example: > > > > map PMU1:event2 -> pmu_direct_access = 1 -> enable access > > map PMU2:event3 -> pmu_direct_access = 2 > > map PMU1:event4 -> pmu_direct_access = 3 > > unmap PMU2:event3 -> pmu_direct_access = 2 > > unmap PMU1:event2 -> pmu_direct_access = 1 > > unmap PMU1:event4 -> pmu_direct_access = 0 -> disable access > > > > The only issue I can see is PMU2 remains enabled for user access until > > the last unmap. But we're sharing the mm, so who cares? Also, in this > > scenario it is the user's problem to pin themselves to cores sharing a > > PMU. If the user doesn't do that, they get to keep the pieces. > > I guess I'm just worried about exposing the counters to userspace after > the PMU driver (or perf core?) thinks that they're no longer exposed in > case we leak other events. IMO that's not practically different from the single-PMU case (i.e. multi-PMU isn't material, either we have a concern with leaking or we don't); more on that below. While it looks odd to place this on the mm, I don't think it's the end of the world. > However, I'm not sure how this is supposed to work normally: what > happens if e.g. a privileged user has a per-cpu counter for a kernel > event while a task has a counter with direct access -- can that task > read the kernel event out of the PMU registers from userspace? Yes -- userspace could go read any counters even though it isn't supposed to, and could potentially infer information from those. It won't have access to the config registers or kernel data structures, so it isn't guaranteed to know what the even is or when it is context-switched/reprogrammed/etc. If we believe that's a problem, then it's difficult to do anything robust other than denying userspace access entirely, since disabling userspace access while in use would surprise applications, and denying privileged events would need some global state that we consult at event creation time (in addition to being an inversion of privilege). IIRC there was some fuss about this a while back on x86; I'll go dig and see what I can find, unless Peter has a memory... Thanks, Mark. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDAB0C433ED for ; Thu, 8 Apr 2021 11:10:40 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 29BEE610E5 for ; Thu, 8 Apr 2021 11:10:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 29BEE610E5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=WCQOgZhSUOIc+rDVDYm38FbZME35mfQ1Fx0clvATAYE=; b=bWu6Zk8i1wNSEimPavXoFgedV guTAQl70wlzrwOELz0YpHCae5UBPJ79G+mJRfkbg9qfMDvi0hnXQeyE1D5rlibGc7J9oMkUFgvnR0 8RMjoav1RW7vM3dBapXiogJP5lezRPQzMNFsX0ygfb44vbNYoHdibEtJH9mZE2xN6TVru8QhW3ecD 63qW85MumRuSK9ZOC5Mj6Y/dHrVXY+AjoH1JD9W3c2Zu+wvTHUoEsysKbdPj+ZXliqDnRSPj84U39 9ku4kRmwqmXKgLbyG/cKPljTlAKliYAXoTtuJkUWVcqRppyOAIveb9UNTeYtlrfJOMkgh/ZOHR7ij 0d+KLK8VQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUSX0-007l37-7O; Thu, 08 Apr 2021 11:09:02 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUSWW-007ksN-Jx for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 11:08:35 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3010F106F; Thu, 8 Apr 2021 04:08:30 -0700 (PDT) Received: from C02TD0UTHF1T.local (unknown [10.57.24.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3B5303F73D; Thu, 8 Apr 2021 04:08:26 -0700 (PDT) Date: Thu, 8 Apr 2021 12:08:15 +0100 From: Mark Rutland To: Will Deacon Cc: Rob Herring , Catalin Marinas , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Jiri Olsa , Ian Rogers , Alexander Shishkin , Honnappa Nagarahalli , Zachary.Leaf@arm.com, Raphael Gault , Jonathan Cameron , Namhyung Kim , Itaru Kitayama , linux-arm-kernel , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH v6 02/10] arm64: perf: Enable PMU counter direct access for perf event Message-ID: <20210408110800.GA32792@C02TD0UTHF1T.local> References: <20210311000837.3630499-1-robh@kernel.org> <20210311000837.3630499-3-robh@kernel.org> <20210330153125.GC6567@willie-the-truck> <20210331160059.GD7815@willie-the-truck> <20210407124437.GA15622@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210407124437.GA15622@willie-the-truck> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_120833_043787_3E2D184B X-CRM114-Status: GOOD ( 41.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Apr 07, 2021 at 01:44:37PM +0100, Will Deacon wrote: > [Moving Mark to To: since I'd like his view on this] > > On Thu, Apr 01, 2021 at 02:45:21PM -0500, Rob Herring wrote: > > On Wed, Mar 31, 2021 at 11:01 AM Will Deacon wrote: > > > > > > On Tue, Mar 30, 2021 at 12:09:38PM -0500, Rob Herring wrote: > > > > On Tue, Mar 30, 2021 at 10:31 AM Will Deacon wrote: > > > > > > > > > > On Wed, Mar 10, 2021 at 05:08:29PM -0700, Rob Herring wrote: > > > > > > From: Raphael Gault > > > > > > +static void armv8pmu_event_unmapped(struct perf_event *event, struct mm_struct *mm) > > > > > > +{ > > > > > > + struct arm_pmu *armpmu = to_arm_pmu(event->pmu); > > > > > > + > > > > > > + if (!(event->hw.flags & ARMPMU_EL0_RD_CNTR)) > > > > > > + return; > > > > > > + > > > > > > + if (atomic_dec_and_test(&mm->context.pmu_direct_access)) > > > > > > + on_each_cpu_mask(&armpmu->supported_cpus, refresh_pmuserenr, mm, 1); > > > > > > > > > > Given that the pmu_direct_access field is global per-mm, won't this go > > > > > wrong if multiple PMUs are opened by the same process but only a subset > > > > > are exposed to EL0? Perhaps pmu_direct_access should be treated as a mask > > > > > rather than a counter, so that we can 'and' it with the supported_cpus for > > > > > the PMU we're dealing with. > > > > > > > > It needs to be a count to support multiple events on the same PMU. If > > > > the event is not enabled for EL0, then we'd exit out on the > > > > ARMPMU_EL0_RD_CNTR check. So I think we're fine. > > > > > > I'm still not convinced; pmu_direct_access is shared between PMUs, so > > > testing the result of atomic_dec_and_test() just doesn't make sense to > > > me, as another PMU could be playing with the count. > > > > How is that a problem? Let's make a concrete example: > > > > map PMU1:event2 -> pmu_direct_access = 1 -> enable access > > map PMU2:event3 -> pmu_direct_access = 2 > > map PMU1:event4 -> pmu_direct_access = 3 > > unmap PMU2:event3 -> pmu_direct_access = 2 > > unmap PMU1:event2 -> pmu_direct_access = 1 > > unmap PMU1:event4 -> pmu_direct_access = 0 -> disable access > > > > The only issue I can see is PMU2 remains enabled for user access until > > the last unmap. But we're sharing the mm, so who cares? Also, in this > > scenario it is the user's problem to pin themselves to cores sharing a > > PMU. If the user doesn't do that, they get to keep the pieces. > > I guess I'm just worried about exposing the counters to userspace after > the PMU driver (or perf core?) thinks that they're no longer exposed in > case we leak other events. IMO that's not practically different from the single-PMU case (i.e. multi-PMU isn't material, either we have a concern with leaking or we don't); more on that below. While it looks odd to place this on the mm, I don't think it's the end of the world. > However, I'm not sure how this is supposed to work normally: what > happens if e.g. a privileged user has a per-cpu counter for a kernel > event while a task has a counter with direct access -- can that task > read the kernel event out of the PMU registers from userspace? Yes -- userspace could go read any counters even though it isn't supposed to, and could potentially infer information from those. It won't have access to the config registers or kernel data structures, so it isn't guaranteed to know what the even is or when it is context-switched/reprogrammed/etc. If we believe that's a problem, then it's difficult to do anything robust other than denying userspace access entirely, since disabling userspace access while in use would surprise applications, and denying privileged events would need some global state that we consult at event creation time (in addition to being an inversion of privilege). IIRC there was some fuss about this a while back on x86; I'll go dig and see what I can find, unless Peter has a memory... Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel