From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F07DC433E0 for ; Fri, 22 Jan 2021 11:58:11 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 511FA22C9F for ; Fri, 22 Jan 2021 11:58:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 511FA22C9F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=y1n+p2DkaPyc8KKeHx+gV04jAX/T+7d61z/gqwPYI3A=; b=1hUKy91kR1D83FZ1y+S0t3Prq 4sAnEr/VuZWD+CHEa1D7B+BvAR8QdmC/DpwTN57Hkh/smsMUmB0zxqoH71BMS05o2ENPFk/PYUSJl DcG+F8txiP03molnxkD7GC1brXhihaELG8fYI6pRTb0GNLJ0Gqj1mUTq2/rLM5+nMZRDWueMlV+fq gnZoL4av2+SrNjZLs/+1lBs+DL2Ky1SU66RrfStxucODFNa1YaxNP11dXQ2aDCorj/yc2vLc4Insa Kct2J+Jr19I51tOg3WY5ktvxEoeoopqs18Pe5jF9e1aXMT8s0sGfKapr6LjoPmHw2qlFnPgYpyxM1 2m/jvNpmQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2v3N-0003TH-1N; Fri, 22 Jan 2021 11:56:37 +0000 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2v3F-0003Qr-QL for linux-arm-kernel@lists.infradead.org; Fri, 22 Jan 2021 11:56:31 +0000 Received: by mail-wr1-x42d.google.com with SMTP id g10so4834821wrx.1 for ; Fri, 22 Jan 2021 03:56:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0SgQac0tiwSnubcxGDua82/tHq+RVHRl87pjSP5itgM=; b=c/+lm4UV7z4yrpevZBw5MXVbSHJfG5GvTYXmR523SsGeYoSjX+jOx59hx0TYoUsns6 Ict0mSd2fZU4yt/FVPoKw/R/mbLQ8F5Wem25jFZQVmCmSll0JDtLuUbhwlD0DVmoitZ1 aj8CSe9+sfkxxiDLIynYlR5PUqgH3yj7sppaM7GwCdms7piu1hXvNh00PX9kvSScWCBQ RwKf/LiIAqjTHqz/PGpJu5oJltalvLPVZfYD0YAGhuGxe9j1+/39N7ejCX5yp47AiScG 6HZ0DppJ5pDNc/r04nIS65fIXwrP1ALy7IRZjxbVA0gVJ/XXXs5wHnrG9SmT/5S9DUxW z/vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0SgQac0tiwSnubcxGDua82/tHq+RVHRl87pjSP5itgM=; b=YpVDG0PjGge8oSbIuGGNtRcs4u+LeFvGbOLkazLoiWH5aUn+4I2Y6A30tjurbtnabZ jaXQZ0v1mSgDA4/PnQt79d9aCLSw6/flFOAM3wMpBOg7vwz/NVJW37t777brOFPW7l9E RwXwiDeNOtq6vlzG5zqTgVeFQ28gi3zyjnxvy4Nlnh/r852Ni/FR39UYn/LgB3zyYhew VR5ZV+fWbzthiRtZ3T/C8mZ24teU2z4UbdeVtNPlvew9ZYSLQO1In1lIW07QFSxTdqfh QwvE6AIXYrsTNfqwjjHEmByvLQojlAooqHNaNpNLTVvjvl/skFpZ+76AwuTTSJ2dvOQM I4hg== X-Gm-Message-State: AOAM533cx7+CbPLFPHLsizc0maQrZ1X7rbimJ/9/de9nzGLG7AouNiCI 2HMi5lqdSu6BmVZptYTZuxMGAw== X-Google-Smtp-Source: ABdhPJwQhy/pUuxgwb7y2QDYJj2WfJIWjTPpeXGb67NOjvpLAu1UsjCylvROLm7OF+CedXFEccK+Pg== X-Received: by 2002:adf:9b92:: with SMTP id d18mr4293974wrc.170.1611316588567; Fri, 22 Jan 2021 03:56:28 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id s25sm13293901wrs.49.2021.01.22.03.56.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 03:56:28 -0800 (PST) From: Jean-Philippe Brucker To: will@kernel.org, robin.murphy@arm.com, joro@8bytes.org Subject: [PATCH 1/3] iommu/arm-smmu-v3: Split arm_smmu_tlb_inv_range() Date: Fri, 22 Jan 2021 12:52:56 +0100 Message-Id: <20210122115257.2502526-2-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210122115257.2502526-1-jean-philippe@linaro.org> References: <20210122115257.2502526-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210122_065630_183131_4F7AAC96 X-CRM114-Status: GOOD ( 18.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jean-Philippe Brucker , vivek.gautam@arm.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, Jonathan.Cameron@huawei.com, zhangfei.gao@linaro.org, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Extract some of the cmd initialization and the ATC invalidation from arm_smmu_tlb_inv_range(), to allow an MMU notifier to invalidate a VA range by ASID. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 62 ++++++++++++--------- 1 file changed, 35 insertions(+), 27 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 740a3d487591..a27b074d5c0c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1934,40 +1934,27 @@ static void arm_smmu_tlb_inv_context(void *cookie) arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); } -static void arm_smmu_tlb_inv_range(unsigned long iova, size_t size, - size_t granule, bool leaf, +static void arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, + unsigned long iova, size_t size, + size_t granule, struct arm_smmu_domain *smmu_domain) { struct arm_smmu_device *smmu = smmu_domain->smmu; - unsigned long start = iova, end = iova + size, num_pages = 0, tg = 0; + unsigned long end = iova + size, num_pages = 0, tg = 0; size_t inv_range = granule; struct arm_smmu_cmdq_batch cmds = {}; - struct arm_smmu_cmdq_ent cmd = { - .tlbi = { - .leaf = leaf, - }, - }; - if (!size) return; - if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - cmd.opcode = CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; - } else { - cmd.opcode = CMDQ_OP_TLBI_S2_IPA; - cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; - } - if (smmu->features & ARM_SMMU_FEAT_RANGE_INV) { /* Get the leaf page size */ tg = __ffs(smmu_domain->domain.pgsize_bitmap); /* Convert page size of 12,14,16 (log2) to 1,2,3 */ - cmd.tlbi.tg = (tg - 10) / 2; + cmd->tlbi.tg = (tg - 10) / 2; /* Determine what level the granule is at */ - cmd.tlbi.ttl = 4 - ((ilog2(granule) - 3) / (tg - 3)); + cmd->tlbi.ttl = 4 - ((ilog2(granule) - 3) / (tg - 3)); num_pages = size >> tg; } @@ -1985,11 +1972,11 @@ static void arm_smmu_tlb_inv_range(unsigned long iova, size_t size, /* Determine the power of 2 multiple number of pages */ scale = __ffs(num_pages); - cmd.tlbi.scale = scale; + cmd->tlbi.scale = scale; /* Determine how many chunks of 2^scale size we have */ num = (num_pages >> scale) & CMDQ_TLBI_RANGE_NUM_MAX; - cmd.tlbi.num = num - 1; + cmd->tlbi.num = num - 1; /* range is num * 2^scale * pgsize */ inv_range = num << (scale + tg); @@ -1998,17 +1985,37 @@ static void arm_smmu_tlb_inv_range(unsigned long iova, size_t size, num_pages -= num << scale; } - cmd.tlbi.addr = iova; - arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); + cmd->tlbi.addr = iova; + arm_smmu_cmdq_batch_add(smmu, &cmds, cmd); iova += inv_range; } arm_smmu_cmdq_batch_submit(smmu, &cmds); +} + +static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, + size_t granule, bool leaf, + struct arm_smmu_domain *smmu_domain) +{ + struct arm_smmu_cmdq_ent cmd = { + .tlbi = { + .leaf = leaf, + }, + }; + + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + cmd.opcode = CMDQ_OP_TLBI_NH_VA; + cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + } else { + cmd.opcode = CMDQ_OP_TLBI_S2_IPA; + cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; + } + arm_smmu_tlb_inv_range(&cmd, iova, size, granule, smmu_domain); /* * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, start, size); + arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); } static void arm_smmu_tlb_inv_page_nosync(struct iommu_iotlb_gather *gather, @@ -2024,7 +2031,7 @@ static void arm_smmu_tlb_inv_page_nosync(struct iommu_iotlb_gather *gather, static void arm_smmu_tlb_inv_walk(unsigned long iova, size_t size, size_t granule, void *cookie) { - arm_smmu_tlb_inv_range(iova, size, granule, false, cookie); + arm_smmu_tlb_inv_range_domain(iova, size, granule, false, cookie); } static const struct iommu_flush_ops arm_smmu_flush_ops = { @@ -2622,8 +2629,9 @@ static void arm_smmu_iotlb_sync(struct iommu_domain *domain, { struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - arm_smmu_tlb_inv_range(gather->start, gather->end - gather->start, - gather->pgsize, true, smmu_domain); + arm_smmu_tlb_inv_range_domain(gather->start, + gather->end - gather->start, + gather->pgsize, true, smmu_domain); } static phys_addr_t -- 2.30.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel