From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20CF7C433B4 for ; Fri, 7 May 2021 09:52:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E43C8613D6 for ; Fri, 7 May 2021 09:52:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236591AbhEGJxr (ORCPT ); Fri, 7 May 2021 05:53:47 -0400 Received: from foss.arm.com ([217.140.110.172]:52862 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236569AbhEGJxq (ORCPT ); Fri, 7 May 2021 05:53:46 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E87AE106F; Fri, 7 May 2021 02:52:46 -0700 (PDT) Received: from [10.57.59.124] (unknown [10.57.59.124]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B85233F718; Fri, 7 May 2021 02:52:44 -0700 (PDT) Subject: Re: [PATCH v3 09/10] iommu/arm-smmu: Get associated RMR info and install bypass SMR To: Steven Price , Shameer Kolothum , linux-arm-kernel@lists.infradead.org, linux-acpi@vger.kernel.org, iommu@lists.linux-foundation.org Cc: linuxarm@huawei.com, lorenzo.pieralisi@arm.com, joro@8bytes.org, wanghuiqiang@huawei.com, guohanjun@huawei.com, Sami.Mujawar@arm.com, jon@solid-run.com, eric.auger@redhat.com References: <20210420082751.1829-1-shameerali.kolothum.thodi@huawei.com> <20210420082751.1829-10-shameerali.kolothum.thodi@huawei.com> <501cd986-7f9c-9aa7-b4e9-f2ef98fb7a95@arm.com> From: Robin Murphy Message-ID: <8c7dad26-b286-3974-9316-78ce1129ebe3@arm.com> Date: Fri, 7 May 2021 10:52:39 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: <501cd986-7f9c-9aa7-b4e9-f2ef98fb7a95@arm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org On 2021-05-06 16:17, Steven Price wrote: > On 20/04/2021 09:27, Shameer Kolothum wrote: >> From: Jon Nettleton >> >> Check if there is any RMR info associated with the devices behind >> the SMMU and if any, install bypass SMRs for them. This is to >> keep any ongoing traffic associated with these devices alive >> when we enable/reset SMMU during probe(). >> >> Signed-off-by: Jon Nettleton >> Signed-off-by: Shameer Kolothum >> --- >>   drivers/iommu/arm/arm-smmu/arm-smmu.c | 42 +++++++++++++++++++++++++++ >>   drivers/iommu/arm/arm-smmu/arm-smmu.h |  2 ++ >>   2 files changed, 44 insertions(+) >> >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c >> b/drivers/iommu/arm/arm-smmu/arm-smmu.c >> index d8c6bfde6a61..4d2f91626d87 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c >> @@ -2102,6 +2102,43 @@ err_reset_platform_ops: __maybe_unused; >>       return err; >>   } >> +static void arm_smmu_rmr_install_bypass_smr(struct arm_smmu_device >> *smmu) >> +{ >> +    struct iommu_rmr *e; >> +    int i, cnt = 0; >> +    u32 smr; >> + >> +    for (i = 0; i < smmu->num_mapping_groups; i++) { >> +        smr = arm_smmu_gr0_read(smmu, ARM_SMMU_GR0_SMR(i)); >> +        if (!FIELD_GET(ARM_SMMU_SMR_VALID, smr)) >> +            continue; >> + >> +        list_for_each_entry(e, &smmu->rmr_list, list) { >> +            if (FIELD_GET(ARM_SMMU_SMR_ID, smr) != e->sid) >> +                continue; >> + >> +            smmu->smrs[i].id = FIELD_GET(ARM_SMMU_SMR_ID, smr); >> +            smmu->smrs[i].mask = FIELD_GET(ARM_SMMU_SMR_MASK, smr); >> +            smmu->smrs[i].valid = true; >> + >> +            smmu->s2crs[i].type = S2CR_TYPE_BYPASS; >> +            smmu->s2crs[i].privcfg = S2CR_PRIVCFG_DEFAULT; >> +            smmu->s2crs[i].cbndx = 0xff; >> + >> +            cnt++; >> +        } >> +    } > > If I understand this correctly - this is looking at the current > (hardware) configuration of the SMMU and attempting to preserve any > bypass SMRs. However from what I can tell it suffers from the following > two problems: > >  (a) Only the ID of the SMR is being checked, not the MASK. So if the > firmware has setup an SMR matching a number of streams this will break. > >  (b) The SMMU might not be enabled at all (CLIENTPD==1) or bypass > enabled for unmatched streams (USFCFG==0). Yes, trying to infer anything from the current SMMU hardware state is bogus - consider what you might find left over after a kexec, for instance. The *only* way to detect the presence and applicability of RMRs is to look at the actual RMR nodes in the IORT. Ignore what we let the Qualcomm ACPI bootloader hack do - that whole implementation is "special". Robin. > Certainly in my test setup case (b) applies and so this doesn't work. > Perhaps something like the below would work better? (It works in the > case of the SMMU not enabled - I've not tested case (a)). > > Steve > > ----8<---- > static void arm_smmu_rmr_install_bypass_smr(struct arm_smmu_device *smmu) > { >     struct iommu_rmr *e; >     int i, cnt = 0; >     u32 smr; >     u32 reg; > >     reg = arm_smmu_gr0_read(smmu, ARM_SMMU_GR0_sCR0); > >     if ((reg & ARM_SMMU_sCR0_USFCFG) && !(reg & ARM_SMMU_sCR0_CLIENTPD)) { >         /* >          * SMMU is already enabled and disallowing bypass, so preserve >          * the existing SMRs >          */ >         for (i = 0; i < smmu->num_mapping_groups; i++) { >             smr = arm_smmu_gr0_read(smmu, ARM_SMMU_GR0_SMR(i)); >             if (!FIELD_GET(ARM_SMMU_SMR_VALID, smr)) >                 continue; >             smmu->smrs[i].id = FIELD_GET(ARM_SMMU_SMR_ID, smr); >             smmu->smrs[i].mask = FIELD_GET(ARM_SMMU_SMR_MASK, smr); >             smmu->smrs[i].valid = true; >         } >     } > >     list_for_each_entry(e, &smmu->rmr_list, list) { >         u32 sid = e->sid; > >         i = arm_smmu_find_sme(smmu, sid, ~0); >         if (i < 0) >             continue; >         if (smmu->s2crs[i].count == 0) { >             smmu->smrs[i].id = sid; >             smmu->smrs[i].mask = ~0; >             smmu->smrs[i].valid = true; >         } >         smmu->s2crs[i].count++; >         smmu->s2crs[i].type = S2CR_TYPE_BYPASS; >         smmu->s2crs[i].privcfg = S2CR_PRIVCFG_DEFAULT; >         smmu->s2crs[i].cbndx = 0xff; > >         cnt++; >     } > >     if ((reg & ARM_SMMU_sCR0_USFCFG) && !(reg & ARM_SMMU_sCR0_CLIENTPD)) { >         /* Remove the valid bit for unused SMRs */ >         for (i = 0; i < smmu->num_mapping_groups; i++) { >             if (smmu->s2crs[i].count == 0) >                 smmu->smrs[i].valid = false; >         } >     } > >     dev_notice(smmu->dev, "\tpreserved %d boot mapping%s\n", cnt, >            cnt == 1 ? "" : "s"); > }