From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B713C7EE2C for ; Tue, 30 May 2023 12:15:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231549AbjE3MPA (ORCPT ); Tue, 30 May 2023 08:15:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44950 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231735AbjE3MOu (ORCPT ); Tue, 30 May 2023 08:14:50 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0345AB0; Tue, 30 May 2023 05:14:48 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 988B3AB6; Tue, 30 May 2023 05:15:33 -0700 (PDT) Received: from [10.57.83.37] (unknown [10.57.83.37]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5F5DD3F67D; Tue, 30 May 2023 05:14:46 -0700 (PDT) Message-ID: Date: Tue, 30 May 2023 13:14:41 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: [PATCH 2/2] arm64: Notify on pte permission upgrades Content-Language: en-GB To: Jason Gunthorpe , Alistair Popple Cc: Andrew Morton , will@kernel.org, catalin.marinas@arm.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, nicolinc@nvidia.com, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, John Hubbard , zhi.wang.linux@gmail.com, Sean Christopherson References: <3cece716fc09724793aa832e755abfc9d70a8bb3.1684892404.git-series.apopple@nvidia.com> <5d8e1f752051173d2d1b5c3e14b54eb3506ed3ef.1684892404.git-series.apopple@nvidia.com> <87pm6ii6qi.fsf@nvidia.com> From: Robin Murphy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2023-05-30 12:54, Jason Gunthorpe wrote: > On Tue, May 30, 2023 at 06:05:41PM +1000, Alistair Popple wrote: >> >>>> As no notification is sent and the SMMU does not snoop TLB invalidates >>>> it will continue to return read-only entries to a device even though >>>> the CPU page table contains a writable entry. This leads to a >>>> continually faulting device and no way of handling the fault. >>> >>> Doesn't the fault generate a PRI/etc? If we get a PRI maybe we should >>> just have the iommu driver push an iotlb invalidation command before >>> it acks it? PRI is already really slow so I'm not sure a pipelined >>> invalidation is going to be a problem? Does the SMMU architecture >>> permit negative caching which would suggest we need it anyhow? >> >> Yes, SMMU architecture (which matches the ARM architecture in regards to >> TLB maintenance requirements) permits negative caching of some mapping >> attributes including the read-only attribute. Hence without the flushing >> we fault continuously. > > Sounds like a straight up SMMU bug, invalidate the cache after > resolving the PRI event. No, if the IOPF handler calls back into the mm layer to resolve the fault, and the mm layer issues an invalidation in the process of that which isn't propagated back to the SMMU (as it would be if BTM were in use), logically that's the mm layer's failing. The SMMU driver shouldn't have to issue extra mostly-redundant invalidations just because different CPU architectures have different idiosyncracies around caching of permissions. Thanks, Robin. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1B1C9C77B7A for ; Tue, 30 May 2023 12:15:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rhPf+EfX+A2PafGEiM+WiiBo8q2BooQCtIhDr8H0pQE=; b=K5J56QirQ+UnBe 8IssQ+mpb+eQl5nUEDs2vBx+q0soLnDTiFD57/KpXK6ZU/zBZXOZqcZq0J8225ApfKey6cGOs71Ik iVCH/MJSEPIzebIScHMI4fTBzMK28+U/Eq392/0j+NJOd/OMlwgzyUvaDEyY1rT/6j3GtB19Mwvpd uLJVWBKcaR0AzJ6jEqaMovjl7r260+RgWEb5C+mUgUl9oTkcjrw+2EBgK5O6CI/Y8FjFoirYBb1By pOW13exlLR1q3gxbtbRZGiMlxgvlg6FQLmkcdSlWCLU4exVjQ92DbATDtw+xuEBkTlD1F2/6/o86F WcQQ6CGHgF46yPxmRfQA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q3yFi-00DoSU-2y; Tue, 30 May 2023 12:15:02 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q3yFa-00DoMN-0r for linux-arm-kernel@lists.infradead.org; Tue, 30 May 2023 12:14:56 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 988B3AB6; Tue, 30 May 2023 05:15:33 -0700 (PDT) Received: from [10.57.83.37] (unknown [10.57.83.37]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5F5DD3F67D; Tue, 30 May 2023 05:14:46 -0700 (PDT) Message-ID: Date: Tue, 30 May 2023 13:14:41 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: [PATCH 2/2] arm64: Notify on pte permission upgrades Content-Language: en-GB To: Jason Gunthorpe , Alistair Popple Cc: Andrew Morton , will@kernel.org, catalin.marinas@arm.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, nicolinc@nvidia.com, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, John Hubbard , zhi.wang.linux@gmail.com, Sean Christopherson References: <3cece716fc09724793aa832e755abfc9d70a8bb3.1684892404.git-series.apopple@nvidia.com> <5d8e1f752051173d2d1b5c3e14b54eb3506ed3ef.1684892404.git-series.apopple@nvidia.com> <87pm6ii6qi.fsf@nvidia.com> From: Robin Murphy In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230530_051454_427544_B71BA485 X-CRM114-Status: GOOD ( 13.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2023-05-30 12:54, Jason Gunthorpe wrote: > On Tue, May 30, 2023 at 06:05:41PM +1000, Alistair Popple wrote: >> >>>> As no notification is sent and the SMMU does not snoop TLB invalidates >>>> it will continue to return read-only entries to a device even though >>>> the CPU page table contains a writable entry. This leads to a >>>> continually faulting device and no way of handling the fault. >>> >>> Doesn't the fault generate a PRI/etc? If we get a PRI maybe we should >>> just have the iommu driver push an iotlb invalidation command before >>> it acks it? PRI is already really slow so I'm not sure a pipelined >>> invalidation is going to be a problem? Does the SMMU architecture >>> permit negative caching which would suggest we need it anyhow? >> >> Yes, SMMU architecture (which matches the ARM architecture in regards to >> TLB maintenance requirements) permits negative caching of some mapping >> attributes including the read-only attribute. Hence without the flushing >> we fault continuously. > > Sounds like a straight up SMMU bug, invalidate the cache after > resolving the PRI event. No, if the IOPF handler calls back into the mm layer to resolve the fault, and the mm layer issues an invalidation in the process of that which isn't propagated back to the SMMU (as it would be if BTM were in use), logically that's the mm layer's failing. The SMMU driver shouldn't have to issue extra mostly-redundant invalidations just because different CPU architectures have different idiosyncracies around caching of permissions. Thanks, Robin. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel