From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49E9CC433ED for ; Thu, 15 Apr 2021 07:15:29 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 957ED611C0 for ; Thu, 15 Apr 2021 07:15:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 957ED611C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Cc:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dBKdMNbnp/9T5oBwcsJtFj+h5hO4oRUA3MgUGRlRaoI=; b=dTM203iYu9VZM5lPHCOXDEXzX yaCSF4KI9AuNq8bIM5Zamv8BonHWc+kNRchl54jXSUJmqxuZu5pcTbyF+Y42lnA8AfEU+kJH6eg5s 8pTfsHiGy9SPHulyJcN4gL52H26JeHaG1tMfMIXbm2QQb897pcdibGgSKQ7qOYsg96Fx4whblhAea LTMohZ06U9NqXBAHCa4fCT2lr5k/4J/tlkF+zx4uqszt8MaceRao3qM4IDTA0umnveXoZMOIF/cBq IVr11/WGY+6CJE0q3/aeUSEznjfvY1QAlZ1KAjjv347zTQXT9ge3Dz24Jbx2tj9Yug6A0n6GgK6UB 1TUo7wuXw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lWwC6-00FADt-Hi; Thu, 15 Apr 2021 07:13:43 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWwC2-00FACA-Sa for linux-arm-kernel@desiato.infradead.org; Thu, 15 Apr 2021 07:13:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:To: Subject:Cc:Sender:Reply-To:Content-ID:Content-Description; bh=jWd6i41c7Tp4ORCqbRQd6ssq1yT4qDwRAQfei7IRloQ=; b=SW4km4rMwbboyL4uPkH+YRJgjZ 31DndpRpw47bZr3//7pRtkdfoku4R6zMk7Mx+0AF7LSxmWg38cre+TgunR+iKjva9aNM7UqAncllr xpYlNC73MzCByJlXTQsK1niNDIRVX8yBRqmSi/0yRqX7ipR7LRdcELWoT3JLO/rTbHCUEofnPnW4d zGkJf5Yv4qgym7NQX3csHCKb/qjxsJizLs9h/gxyfu/PFoPJK6lMJoq5Yg7bP7Siyf9QDLPiGsnE9 OaNbBD8okghgYhncpjz8O0Mt2rUW3it237NCT4JphLBf09yt37satpCDyZCCX6u7kZySvGkPyF5NQ bgvAx5sw==; Received: from mga01.intel.com ([192.55.52.88]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWwBz-008KeZ-6f for linux-arm-kernel@lists.infradead.org; Thu, 15 Apr 2021 07:13:36 +0000 IronPort-SDR: xume94tBFVvsVvTwxE9DVb8sX/WbRBO75aEfUrrReqfZYLWi4Ct6sLTEqMheHT/JUz4o7Nmpcj UZcx02q9AFEw== X-IronPort-AV: E=McAfee;i="6200,9189,9954"; a="215304080" X-IronPort-AV: E=Sophos;i="5.82,223,1613462400"; d="scan'208";a="215304080" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Apr 2021 00:13:33 -0700 IronPort-SDR: N7a/losEaCAlfPsp0RkiF5M5/3RUguWGDB4OlvAUxaL1WE0aXMlAc18uuES1cel2X0mOe1zXJ4 U37IARQwIqgw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,223,1613462400"; d="scan'208";a="444089450" Received: from allen-box.sh.intel.com (HELO [10.239.159.128]) ([10.239.159.128]) by fmsmga004.fm.intel.com with ESMTP; 15 Apr 2021 00:13:29 -0700 Cc: baolu.lu@linux.intel.com, Alex Williamson , Cornelia Huck , Kirti Wankhede , wanghaibin.wang@huawei.com, jiangkunkun@huawei.com, yuzenghui@huawei.com, lushenming@huawei.com Subject: Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework To: Keqian Zhu , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, Robin Murphy , Will Deacon , Joerg Roedel , Yi Sun , Jean-Philippe Brucker , Jonathan Cameron , Tian Kevin References: <20210413085457.25400-1-zhukeqian1@huawei.com> <20210413085457.25400-2-zhukeqian1@huawei.com> From: Lu Baolu Message-ID: <56b001fa-b4fe-c595-dc5e-f362d2f07a19@linux.intel.com> Date: Thu, 15 Apr 2021 15:03:52 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210415_001335_293620_7D92555D X-CRM114-Status: GOOD ( 32.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 4/15/21 2:18 PM, Keqian Zhu wrote: > Hi Baolu, > > Thanks for the review! > > On 2021/4/14 15:00, Lu Baolu wrote: >> Hi Keqian, >> >> On 4/13/21 4:54 PM, Keqian Zhu wrote: >>> Some types of IOMMU are capable of tracking DMA dirty log, such as >>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the >>> dirty log tracking framework in the IOMMU base layer. >>> >>> Three new essential interfaces are added, and we maintaince the status >>> of dirty log tracking in iommu_domain. >>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking >>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap >>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap >>> >>> A new dev feature are added to indicate whether a specific type of >>> iommu hardware supports and its driver realizes them. >>> >>> Signed-off-by: Keqian Zhu >>> Signed-off-by: Kunkun Jiang >>> --- >>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++ >>> include/linux/iommu.h | 53 +++++++++++++++ >>> 2 files changed, 203 insertions(+) >>> >>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c >>> index d0b0a15dba84..667b2d6d2fc0 100644 >>> --- a/drivers/iommu/iommu.c >>> +++ b/drivers/iommu/iommu.c >>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus, >>> domain->type = type; >>> /* Assume all sizes by default; the driver may override this later */ >>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap; >>> + mutex_init(&domain->switch_log_lock); >>> return domain; >>> } >>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain, >>> } >>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr); >>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable, >>> + unsigned long iova, size_t size, int prot) >>> +{ >>> + const struct iommu_ops *ops = domain->ops; >>> + int ret; >>> + >>> + if (unlikely(!ops || !ops->switch_dirty_log)) >>> + return -ENODEV; >>> + >>> + mutex_lock(&domain->switch_log_lock); >>> + if (enable && domain->dirty_log_tracking) { >>> + ret = -EBUSY; >>> + goto out; >>> + } else if (!enable && !domain->dirty_log_tracking) { >>> + ret = -EINVAL; >>> + goto out; >>> + } >>> + >>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot); >>> + if (ret) >>> + goto out; >>> + >>> + domain->dirty_log_tracking = enable; >>> +out: >>> + mutex_unlock(&domain->switch_log_lock); >>> + return ret; >>> +} >>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log); >> >> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the >> difference between >> >> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM) >> >> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM) > Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable > are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these > interfaces for it. > > IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should Previously we had iommu_dev_has_feature() and then was cleaned up due to lack of real users. If you have a real case for it, why not bringing it back? > design it as not switchable. I will modify the commit message of patch#12, thanks! I am not sure that I fully get your point. But I can't see any gaps of using iommu_dev_enable/disable_feature() to switch dirty log on and off. Probably I missed anything. > >> >>> + >>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova, >>> + size_t size, unsigned long *bitmap, >>> + unsigned long base_iova, unsigned long bitmap_pgshift) >>> +{ >>> + const struct iommu_ops *ops = domain->ops; >>> + unsigned int min_pagesz; >>> + size_t pgsize; >>> + int ret = 0; >>> + >>> + if (unlikely(!ops || !ops->sync_dirty_log)) >>> + return -ENODEV; >>> + >>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap); >>> + if (!IS_ALIGNED(iova | size, min_pagesz)) { >>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n", >>> + iova, size, min_pagesz); >>> + return -EINVAL; >>> + } >>> + >>> + mutex_lock(&domain->switch_log_lock); >>> + if (!domain->dirty_log_tracking) { >>> + ret = -EINVAL; >>> + goto out; >>> + } >>> + >>> + while (size) { >>> + pgsize = iommu_pgsize(domain, iova, size); >>> + >>> + ret = ops->sync_dirty_log(domain, iova, pgsize, >>> + bitmap, base_iova, bitmap_pgshift); >> >> Any reason why do you want to do this in a per-4K page manner? This can >> lead to a lot of indirect calls and bad performance. >> >> How about a sync_dirty_pages()? > The function name of iommu_pgsize() is a bit puzzling. Actually it will try to > compute the max size that fit into size, so the pgsize can be a large page size > even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic. This series has some improvement on the iommu_pgsize() helper. https://lore.kernel.org/linux-iommu/20210405191112.28192-1-isaacm@codeaurora.org/ Best regards, baolu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel