From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF513C433E0 for ; Tue, 2 Mar 2021 11:22:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 94A0864F0B for ; Tue, 2 Mar 2021 11:22:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1442204AbhCBLUr (ORCPT ); Tue, 2 Mar 2021 06:20:47 -0500 Received: from foss.arm.com ([217.140.110.172]:49570 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1383247AbhCBLIJ (ORCPT ); Tue, 2 Mar 2021 06:08:09 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 27DBAED1; Tue, 2 Mar 2021 03:07:23 -0800 (PST) Received: from [10.57.48.219] (unknown [10.57.48.219]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C9C703F73C; Tue, 2 Mar 2021 03:07:20 -0800 (PST) Subject: Re: [RFC 09/13] iommu/arm-smmu: Make use of dev_64bit_mmio_supported() To: Nicolas Saenz Julienne , linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org Cc: f.fainelli@gmail.com, robh+dt@kernel.org, ardb@kernel.org, hch@infradead.org, narmstrong@baylibre.com, dwmw2@infradead.org, linux@armlinux.org.uk, catalin.marinas@arm.com, arnd@arndb.de, will@kernel.org References: <20210226140305.26356-1-nsaenzjulienne@suse.de> <20210226140305.26356-10-nsaenzjulienne@suse.de> From: Robin Murphy Message-ID: <3a4cf13f-c098-9ff3-6c0e-2c94daae452b@arm.com> Date: Tue, 2 Mar 2021 11:07:19 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <20210226140305.26356-10-nsaenzjulienne@suse.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2021-02-26 14:03, Nicolas Saenz Julienne wrote: > Some arm SMMU implementations might sit on a bus that doesn't support > 64bit memory accesses. In that case default to using hi_lo_{readq, > writeq}() and BUG if such platform tries to use AArch64 formats as they > rely on writeq()'s atomicity. > > Signed-off-by: Nicolas Saenz Julienne > --- > drivers/iommu/arm/arm-smmu/arm-smmu.c | 9 +++++++++ > drivers/iommu/arm/arm-smmu/arm-smmu.h | 9 +++++++-- > 2 files changed, 16 insertions(+), 2 deletions(-) > > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c > index d8c6bfde6a61..239ff42b20c3 100644 > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c > @@ -1889,6 +1889,15 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) > smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_64K; > } > > + /* > + * 64bit accesses not possible through the interconnect, AArch64 > + * formats depend on it. > + */ > + BUG_ON(!dev_64bit_mmio_supported(smmu->dev) && > + smmu->features & (ARM_SMMU_FEAT_FMT_AARCH64_4K | > + ARM_SMMU_FEAT_FMT_AARCH64_16K | > + ARM_SMMU_FEAT_FMT_AARCH64_64K)); No. Crashing the kernel in a probe routine which is free to fail is unacceptable either way, but guaranteeing failure in the case that the workaround *would* be required is doubly so. Basically, this logic is backwards - if you really wanted to handle it generically, this would be the point at which you'd need to actively suppress all the detected hardware features which depend on 64-bit atomicity, not complain about them. > + > if (smmu->impl && smmu->impl->cfg_probe) { > ret = smmu->impl->cfg_probe(smmu); > if (ret) > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h b/drivers/iommu/arm/arm-smmu/arm-smmu.h > index d2a2d1bc58ba..997d13a21717 100644 > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.h > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h > @@ -477,15 +477,20 @@ static inline void arm_smmu_writel(struct arm_smmu_device *smmu, int page, > { > if (smmu->impl && unlikely(smmu->impl->write_reg)) > smmu->impl->write_reg(smmu, page, offset, val); > - else > + else if (dev_64bit_mmio_supported(smmu->dev)) > writel_relaxed(val, arm_smmu_page(smmu, page) + offset); > + else > + hi_lo_writeq_relaxed(val, arm_smmu_page(smmu, page) + offset); As Arnd pointed out, this is in completely the wrong place. Also, in general it doesn't work if the implementation already needs a hook to filter or override register accesses for any other reason. TBH I'm not convinced that this isn't *more* of a mess than handling it on a SoC-specific basis... Robin. > } > > static inline u64 arm_smmu_readq(struct arm_smmu_device *smmu, int page, int offset) > { > if (smmu->impl && unlikely(smmu->impl->read_reg64)) > return smmu->impl->read_reg64(smmu, page, offset); > - return readq_relaxed(arm_smmu_page(smmu, page) + offset); > + else if (dev_64bit_mmio_supported(smmu->dev)) > + return readq_relaxed(arm_smmu_page(smmu, page) + offset); > + else > + return hi_lo_readq_relaxed(arm_smmu_page(smmu, page) + offset); > } > > static inline void arm_smmu_writeq(struct arm_smmu_device *smmu, int page, > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55CABC433DB for ; Wed, 3 Mar 2021 14:59:28 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EFB6260201 for ; Wed, 3 Mar 2021 14:59:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFB6260201 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:Cc:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0pxBYj7gCO9sWdMhRTVVW0xUDaI0MNjL7BeJ4nhHnec=; b=OPGb6zBO/MuIpElnqw1ri5ef+ z/q8M+RaFeL2zFlvyvxBNJGm4ZvevcGH940kN1Gv/0OgD5IOt5NZHIvMEOXttDshf/qWn1WV0UQ2a JaWkZYE24rYfmhbgr2XnvQQAvA7svkBOgX4sguYS3ZZim9c45ReKmHQ88bL0+wKv8RB5jaBJhOToO NLEep1+7RMtmBT5iJkI8BO5Zk0OCHcc6DucCsrUpjZlSzMMRVhB6Eg3GIWU8/q5lhJ+pd+m0B38Ke Eywi4pVD1Uoo6+JR7nC0uRCml3ryguaGCU3e7eY56DbyOUZWQDskKcSSpygruIOprYMzlnt0ojnC/ CIWyo7nag==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHSv7-005FPk-VF; Wed, 03 Mar 2021 14:56:15 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSBO-0054Ma-Ds for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:08:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender :Reply-To:Content-ID:Content-Description; bh=5YIQpF79HXZo7cjcQ5XYLFJO+t04pMUNa1+i/LF6/nM=; b=CqwoRuEHcbpcoOKLHWBUVKAXBk R8mJab6IWURbtgmwXuN4piKW7vQWhi4a12KzF0W++nvvj4AeO5CrSJP+ILTp6IFBUWSXA5kbcawSH mymYThcVwxii0rwp/mDilJydz/Ui6+cWrQPuoDcDV1NuZJBsBvtueafPHVbrFNxxByXM58HcQFVAN M7sxI24NlaTh8F0Ht/ay8Cvm88PocOFdQWg17su1f7twtRuh32fYNZmVdXj7LTncpr9o7Qhstr22k pzd8cZi0CqaVDGeU0oMK5JEbzMDaoYYQ7h/jJjdTVW9UPSXFya/c/0zio+w0BcFE1YE33QZF5JdPP vPY/BIsw==; Received: from foss.arm.com ([217.140.110.172]) by casper.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lH2sE-00GzCN-6y for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 11:07:31 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 27DBAED1; Tue, 2 Mar 2021 03:07:23 -0800 (PST) Received: from [10.57.48.219] (unknown [10.57.48.219]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C9C703F73C; Tue, 2 Mar 2021 03:07:20 -0800 (PST) Subject: Re: [RFC 09/13] iommu/arm-smmu: Make use of dev_64bit_mmio_supported() To: Nicolas Saenz Julienne , linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org Cc: f.fainelli@gmail.com, robh+dt@kernel.org, ardb@kernel.org, hch@infradead.org, narmstrong@baylibre.com, dwmw2@infradead.org, linux@armlinux.org.uk, catalin.marinas@arm.com, arnd@arndb.de, will@kernel.org References: <20210226140305.26356-1-nsaenzjulienne@suse.de> <20210226140305.26356-10-nsaenzjulienne@suse.de> From: Robin Murphy Message-ID: <3a4cf13f-c098-9ff3-6c0e-2c94daae452b@arm.com> Date: Tue, 2 Mar 2021 11:07:19 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <20210226140305.26356-10-nsaenzjulienne@suse.de> Content-Language: en-GB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_110731_554454_74AD5FD8 X-CRM114-Status: GOOD ( 23.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2021-02-26 14:03, Nicolas Saenz Julienne wrote: > Some arm SMMU implementations might sit on a bus that doesn't support > 64bit memory accesses. In that case default to using hi_lo_{readq, > writeq}() and BUG if such platform tries to use AArch64 formats as they > rely on writeq()'s atomicity. > > Signed-off-by: Nicolas Saenz Julienne > --- > drivers/iommu/arm/arm-smmu/arm-smmu.c | 9 +++++++++ > drivers/iommu/arm/arm-smmu/arm-smmu.h | 9 +++++++-- > 2 files changed, 16 insertions(+), 2 deletions(-) > > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c > index d8c6bfde6a61..239ff42b20c3 100644 > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c > @@ -1889,6 +1889,15 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) > smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_64K; > } > > + /* > + * 64bit accesses not possible through the interconnect, AArch64 > + * formats depend on it. > + */ > + BUG_ON(!dev_64bit_mmio_supported(smmu->dev) && > + smmu->features & (ARM_SMMU_FEAT_FMT_AARCH64_4K | > + ARM_SMMU_FEAT_FMT_AARCH64_16K | > + ARM_SMMU_FEAT_FMT_AARCH64_64K)); No. Crashing the kernel in a probe routine which is free to fail is unacceptable either way, but guaranteeing failure in the case that the workaround *would* be required is doubly so. Basically, this logic is backwards - if you really wanted to handle it generically, this would be the point at which you'd need to actively suppress all the detected hardware features which depend on 64-bit atomicity, not complain about them. > + > if (smmu->impl && smmu->impl->cfg_probe) { > ret = smmu->impl->cfg_probe(smmu); > if (ret) > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h b/drivers/iommu/arm/arm-smmu/arm-smmu.h > index d2a2d1bc58ba..997d13a21717 100644 > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.h > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h > @@ -477,15 +477,20 @@ static inline void arm_smmu_writel(struct arm_smmu_device *smmu, int page, > { > if (smmu->impl && unlikely(smmu->impl->write_reg)) > smmu->impl->write_reg(smmu, page, offset, val); > - else > + else if (dev_64bit_mmio_supported(smmu->dev)) > writel_relaxed(val, arm_smmu_page(smmu, page) + offset); > + else > + hi_lo_writeq_relaxed(val, arm_smmu_page(smmu, page) + offset); As Arnd pointed out, this is in completely the wrong place. Also, in general it doesn't work if the implementation already needs a hook to filter or override register accesses for any other reason. TBH I'm not convinced that this isn't *more* of a mess than handling it on a SoC-specific basis... Robin. > } > > static inline u64 arm_smmu_readq(struct arm_smmu_device *smmu, int page, int offset) > { > if (smmu->impl && unlikely(smmu->impl->read_reg64)) > return smmu->impl->read_reg64(smmu, page, offset); > - return readq_relaxed(arm_smmu_page(smmu, page) + offset); > + else if (dev_64bit_mmio_supported(smmu->dev)) > + return readq_relaxed(arm_smmu_page(smmu, page) + offset); > + else > + return hi_lo_readq_relaxed(arm_smmu_page(smmu, page) + offset); > } > > static inline void arm_smmu_writeq(struct arm_smmu_device *smmu, int page, > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel