From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59980C4160E for ; Tue, 2 Mar 2021 16:23:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37C8464F17 for ; Tue, 2 Mar 2021 16:23:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1449211AbhCBQQn (ORCPT ); Tue, 2 Mar 2021 11:16:43 -0500 Received: from mx2.suse.de ([195.135.220.15]:45296 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381426AbhCBNi7 (ORCPT ); Tue, 2 Mar 2021 08:38:59 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id C33C4ACBC; Tue, 2 Mar 2021 13:38:16 +0000 (UTC) Message-ID: Subject: Re: [RFC 09/13] iommu/arm-smmu: Make use of dev_64bit_mmio_supported() From: Nicolas Saenz Julienne To: Robin Murphy , linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org Cc: f.fainelli@gmail.com, robh+dt@kernel.org, ardb@kernel.org, hch@infradead.org, narmstrong@baylibre.com, dwmw2@infradead.org, linux@armlinux.org.uk, catalin.marinas@arm.com, arnd@arndb.de, will@kernel.org Date: Tue, 02 Mar 2021 14:38:14 +0100 In-Reply-To: <3a4cf13f-c098-9ff3-6c0e-2c94daae452b@arm.com> References: <20210226140305.26356-1-nsaenzjulienne@suse.de> <20210226140305.26356-10-nsaenzjulienne@suse.de> <3a4cf13f-c098-9ff3-6c0e-2c94daae452b@arm.com> Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-wN85PY7iiGVDnz4rnxxC" User-Agent: Evolution 3.38.4 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --=-wN85PY7iiGVDnz4rnxxC Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Robin, thanks for taking the time to look at this. On Tue, 2021-03-02 at 11:07 +0000, Robin Murphy wrote: > On 2021-02-26 14:03, Nicolas Saenz Julienne wrote: > > Some arm SMMU implementations might sit on a bus that doesn't support > > 64bit memory accesses. In that case default to using hi_lo_{readq, > > writeq}() and BUG if such platform tries to use AArch64 formats as they > > rely on writeq()'s atomicity. > >=20 > > Signed-off-by: Nicolas Saenz Julienne > > --- > > =C2=A0=C2=A0drivers/iommu/arm/arm-smmu/arm-smmu.c | 9 +++++++++ > > =C2=A0=C2=A0drivers/iommu/arm/arm-smmu/arm-smmu.h | 9 +++++++-- > > =C2=A0=C2=A02 files changed, 16 insertions(+), 2 deletions(-) > >=20 > > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/= arm-smmu/arm-smmu.c > > index d8c6bfde6a61..239ff42b20c3 100644 > > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c > > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c > > @@ -1889,6 +1889,15 @@ static int arm_smmu_device_cfg_probe(struct arm_= smmu_device *smmu) > > =C2=A0=C2=A0 smmu->features |=3D ARM_SMMU_FEAT_FMT_AARCH64_64K; > > =C2=A0=C2=A0 } > > =C2=A0=C2=A0 > >=20 > > + /* > > + * 64bit accesses not possible through the interconnect, AArch64 > > + * formats depend on it. > > + */ > > + BUG_ON(!dev_64bit_mmio_supported(smmu->dev) && > > + smmu->features & (ARM_SMMU_FEAT_FMT_AARCH64_4K | > > + ARM_SMMU_FEAT_FMT_AARCH64_16K | > > + ARM_SMMU_FEAT_FMT_AARCH64_64K)); >=20 > No. Crashing the kernel in a probe routine which is free to fail is=20 > unacceptable either way, but guaranteeing failure in the case that the= =20 > workaround *would* be required is doubly so. >=20 > Basically, this logic is backwards - if you really wanted to handle it= =20 > generically, this would be the point at which you'd need to actively=20 > suppress all the detected hardware features which depend on 64-bit=20 > atomicity, not complain about them. Understood. > > + > > =C2=A0=C2=A0 if (smmu->impl && smmu->impl->cfg_probe) { > > =C2=A0=C2=A0 ret =3D smmu->impl->cfg_probe(smmu); > > =C2=A0=C2=A0 if (ret) > > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h b/drivers/iommu/arm/= arm-smmu/arm-smmu.h > > index d2a2d1bc58ba..997d13a21717 100644 > > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.h > > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h > > @@ -477,15 +477,20 @@ static inline void arm_smmu_writel(struct arm_smm= u_device *smmu, int page, > > =C2=A0=C2=A0{ > > =C2=A0=C2=A0 if (smmu->impl && unlikely(smmu->impl->write_reg)) > > =C2=A0=C2=A0 smmu->impl->write_reg(smmu, page, offset, val); > > - else > > + else if (dev_64bit_mmio_supported(smmu->dev)) > > =C2=A0=C2=A0 writel_relaxed(val, arm_smmu_page(smmu, page) + offset); > > + else > > + hi_lo_writeq_relaxed(val, arm_smmu_page(smmu, page) + offset); >=20 > As Arnd pointed out, this is in completely the wrong place. Also, in=20 Yes, sorry for that, not too proud of it. > general it doesn't work if the implementation already needs a hook to=20 > filter or override register accesses for any other reason. TBH I'm not= =20 I'm not sure I get your point here, 'smmu->impl' has precedence over the MM= IO capability check. Custom implementations would still get their callbacks. > convinced that this isn't *more* of a mess than handling it on a=20 > SoC-specific basis... I see your point. Just to explain why I went to these lengths: my understanding is that the specifics of how to perform 32bit accesses to SMMU's 64bit registers is def= ined in spec. So it made sense to move it into the non implementation dependent = side of the driver. All in all, I'll think of something simpler. Regards, Nicolas --=-wN85PY7iiGVDnz4rnxxC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEErOkkGDHCg2EbPcGjlfZmHno8x/4FAmA+P8YACgkQlfZmHno8 x/4StQgAkLUKE6F/7yFab1Ti1xLuHnbi95BqA747fmb/x/LUnVKSRs05zbtCn10z M9gK+LYnbYRHIXrqL4GHWBJG/R+qF3XVKoNHAk5dfrctCaGnVujGrfxSTB4lkH/z gbbKlXo8UHkTk6QnIE5XgmGz+2RgLP2yCx7XtWvKRpLRgTP4fsbC7M0wxERzBIEn VFnoM2a/eKGP3j2spN6TVy2AdXaBu89kosJ50ytD8AwSTrD9eNx15fBzy5w1sqrw gZNA3lSp0VbqRSb1G2Efa/tSkHW38Rb7YWOdf6sY6fBIT9VXW2QpHxolv3DurPk4 0JbQK4vEzXdARdgCN/rrhB9UgL8FdA== =teKv -----END PGP SIGNATURE----- --=-wN85PY7iiGVDnz4rnxxC-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2756AC433E0 for ; Wed, 3 Mar 2021 17:36:39 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9733264EDB for ; Wed, 3 Mar 2021 17:36:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9733264EDB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: MIME-Version:References:In-Reply-To:Date:Cc:To:From:Subject:Message-ID: Reply-To:Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date :Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gq3RKo6uxLG7AKhOWOEu+Y3ARwEnIjgPd531SIQliAQ=; b=jgAAPt2ME4pAgTWMcvNJ5DmWO bdY2sheHbVdCJtLuuNK7VWfz2MoSPDWZWxYiUQ/L9JPnuhBOwpMsHS/pO6AEyz1e4eyR/ierzG5yk Sqbt37MBoEldOkXvXBtAsDuAoN96oVYvmJGWuL9l/jYlmeG/y6YTlol1aA7AcQ1wpsMeRTbX2Qyrb rJWs/W5OuZEJ1qaMycNU3u4PokqXpz/P8VNg+gmVuij+3lx2gCOo5ihWGnhj/W+XwitXRie6DOlg9 nPLIYyPUBlFNXOCeJpoB16xpAWxpCQdCi1gn0dyp0kzZhe4S5SCANn6BRHguOOF4Fzv9H1xi7kLrm j1G47TO5g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHVO2-005pCW-GJ; Wed, 03 Mar 2021 17:34:15 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSva-005Fau-PE for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:56:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=MIME-Version:Content-Type:References: In-Reply-To:Date:Cc:To:From:Subject:Message-ID:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=ncz7aLBnT2RzJIZ6AfQqswX5mi4vmOFwlyTvQV8ypGA=; b=I59dcbmzUAvyKJUUgDHyHgnDuI EbfXW93isg3bK30tYdh2cdJroD9Ki0QcETy2g3ni149Dwv1XcIiTmRDTKtLwF+oUiw95ObBZnlnJN wEYQYxyKlR+vpS1uvCRap7yQbzNMJbt9JiJO2Cw+uCpxEvE9kPTnz9bvnwfV+LzDECTybcpzhVauO HHHVS5nhnkDpjRwrg9TPNeDPxiJT+KAON+12UmmV4pvxGMkZzj3E0uhfMl0JSFq8z/sWL8BuVLhFj hh5SG+VtEeqMw15AMiayzbyvRev0Nq0fCtJE6mxWcGuThNkXl2khkhnkNlFoh5YHbya1fYc0D6Txt bV7hT8HA==; Received: from mx2.suse.de ([195.135.220.15]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lH5EC-00HC4K-Cc for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 13:38:22 +0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id C33C4ACBC; Tue, 2 Mar 2021 13:38:16 +0000 (UTC) Message-ID: Subject: Re: [RFC 09/13] iommu/arm-smmu: Make use of dev_64bit_mmio_supported() From: Nicolas Saenz Julienne To: Robin Murphy , linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org Cc: f.fainelli@gmail.com, robh+dt@kernel.org, ardb@kernel.org, hch@infradead.org, narmstrong@baylibre.com, dwmw2@infradead.org, linux@armlinux.org.uk, catalin.marinas@arm.com, arnd@arndb.de, will@kernel.org Date: Tue, 02 Mar 2021 14:38:14 +0100 In-Reply-To: <3a4cf13f-c098-9ff3-6c0e-2c94daae452b@arm.com> References: <20210226140305.26356-1-nsaenzjulienne@suse.de> <20210226140305.26356-10-nsaenzjulienne@suse.de> <3a4cf13f-c098-9ff3-6c0e-2c94daae452b@arm.com> User-Agent: Evolution 3.38.4 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_133822_204224_3D4B976E X-CRM114-Status: GOOD ( 25.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============6698466687115655374==" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org --===============6698466687115655374== Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-wN85PY7iiGVDnz4rnxxC" --=-wN85PY7iiGVDnz4rnxxC Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Robin, thanks for taking the time to look at this. On Tue, 2021-03-02 at 11:07 +0000, Robin Murphy wrote: > On 2021-02-26 14:03, Nicolas Saenz Julienne wrote: > > Some arm SMMU implementations might sit on a bus that doesn't support > > 64bit memory accesses. In that case default to using hi_lo_{readq, > > writeq}() and BUG if such platform tries to use AArch64 formats as they > > rely on writeq()'s atomicity. > >=20 > > Signed-off-by: Nicolas Saenz Julienne > > --- > > =C2=A0=C2=A0drivers/iommu/arm/arm-smmu/arm-smmu.c | 9 +++++++++ > > =C2=A0=C2=A0drivers/iommu/arm/arm-smmu/arm-smmu.h | 9 +++++++-- > > =C2=A0=C2=A02 files changed, 16 insertions(+), 2 deletions(-) > >=20 > > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/= arm-smmu/arm-smmu.c > > index d8c6bfde6a61..239ff42b20c3 100644 > > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c > > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c > > @@ -1889,6 +1889,15 @@ static int arm_smmu_device_cfg_probe(struct arm_= smmu_device *smmu) > > =C2=A0=C2=A0 smmu->features |=3D ARM_SMMU_FEAT_FMT_AARCH64_64K; > > =C2=A0=C2=A0 } > > =C2=A0=C2=A0 > >=20 > > + /* > > + * 64bit accesses not possible through the interconnect, AArch64 > > + * formats depend on it. > > + */ > > + BUG_ON(!dev_64bit_mmio_supported(smmu->dev) && > > + smmu->features & (ARM_SMMU_FEAT_FMT_AARCH64_4K | > > + ARM_SMMU_FEAT_FMT_AARCH64_16K | > > + ARM_SMMU_FEAT_FMT_AARCH64_64K)); >=20 > No. Crashing the kernel in a probe routine which is free to fail is=20 > unacceptable either way, but guaranteeing failure in the case that the= =20 > workaround *would* be required is doubly so. >=20 > Basically, this logic is backwards - if you really wanted to handle it= =20 > generically, this would be the point at which you'd need to actively=20 > suppress all the detected hardware features which depend on 64-bit=20 > atomicity, not complain about them. Understood. > > + > > =C2=A0=C2=A0 if (smmu->impl && smmu->impl->cfg_probe) { > > =C2=A0=C2=A0 ret =3D smmu->impl->cfg_probe(smmu); > > =C2=A0=C2=A0 if (ret) > > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h b/drivers/iommu/arm/= arm-smmu/arm-smmu.h > > index d2a2d1bc58ba..997d13a21717 100644 > > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.h > > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h > > @@ -477,15 +477,20 @@ static inline void arm_smmu_writel(struct arm_smm= u_device *smmu, int page, > > =C2=A0=C2=A0{ > > =C2=A0=C2=A0 if (smmu->impl && unlikely(smmu->impl->write_reg)) > > =C2=A0=C2=A0 smmu->impl->write_reg(smmu, page, offset, val); > > - else > > + else if (dev_64bit_mmio_supported(smmu->dev)) > > =C2=A0=C2=A0 writel_relaxed(val, arm_smmu_page(smmu, page) + offset); > > + else > > + hi_lo_writeq_relaxed(val, arm_smmu_page(smmu, page) + offset); >=20 > As Arnd pointed out, this is in completely the wrong place. Also, in=20 Yes, sorry for that, not too proud of it. > general it doesn't work if the implementation already needs a hook to=20 > filter or override register accesses for any other reason. TBH I'm not= =20 I'm not sure I get your point here, 'smmu->impl' has precedence over the MM= IO capability check. Custom implementations would still get their callbacks. > convinced that this isn't *more* of a mess than handling it on a=20 > SoC-specific basis... I see your point. Just to explain why I went to these lengths: my understanding is that the specifics of how to perform 32bit accesses to SMMU's 64bit registers is def= ined in spec. So it made sense to move it into the non implementation dependent = side of the driver. All in all, I'll think of something simpler. Regards, Nicolas --=-wN85PY7iiGVDnz4rnxxC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEErOkkGDHCg2EbPcGjlfZmHno8x/4FAmA+P8YACgkQlfZmHno8 x/4StQgAkLUKE6F/7yFab1Ti1xLuHnbi95BqA747fmb/x/LUnVKSRs05zbtCn10z M9gK+LYnbYRHIXrqL4GHWBJG/R+qF3XVKoNHAk5dfrctCaGnVujGrfxSTB4lkH/z gbbKlXo8UHkTk6QnIE5XgmGz+2RgLP2yCx7XtWvKRpLRgTP4fsbC7M0wxERzBIEn VFnoM2a/eKGP3j2spN6TVy2AdXaBu89kosJ50ytD8AwSTrD9eNx15fBzy5w1sqrw gZNA3lSp0VbqRSb1G2Efa/tSkHW38Rb7YWOdf6sY6fBIT9VXW2QpHxolv3DurPk4 0JbQK4vEzXdARdgCN/rrhB9UgL8FdA== =teKv -----END PGP SIGNATURE----- --=-wN85PY7iiGVDnz4rnxxC-- --===============6698466687115655374== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel --===============6698466687115655374==--