From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6249CC433DB for ; Thu, 28 Jan 2021 15:24:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1888F64DF4 for ; Thu, 28 Jan 2021 15:24:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232517AbhA1PYQ (ORCPT ); Thu, 28 Jan 2021 10:24:16 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:11530 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232178AbhA1PWP (ORCPT ); Thu, 28 Jan 2021 10:22:15 -0500 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4DRPKs1ZVgzjFdr; Thu, 28 Jan 2021 23:20:17 +0800 (CST) Received: from [10.174.184.42] (10.174.184.42) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.498.0; Thu, 28 Jan 2021 23:21:20 +0800 Subject: Re: [PATCH v3 2/2] vfio/iommu_type1: Fix some sanity checks in detach group To: Alex Williamson References: <20210122092635.19900-1-zhukeqian1@huawei.com> <20210122092635.19900-3-zhukeqian1@huawei.com> <20210127164641.36e17bf5@omen.home.shazbot.org> CC: , , , , , Kirti Wankhede , Cornelia Huck , Will Deacon , "Marc Zyngier" , Catalin Marinas , "Mark Rutland" , James Morse , "Robin Murphy" , Joerg Roedel , Daniel Lezcano , Thomas Gleixner , "Suzuki K Poulose" , Julien Thierry , Andrew Morton , Alexios Zavras , , From: Keqian Zhu Message-ID: <5093dace-4b8a-d455-ba16-d0c2da755573@huawei.com> Date: Thu, 28 Jan 2021 23:21:20 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: <20210127164641.36e17bf5@omen.home.shazbot.org> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.174.184.42] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2021/1/28 7:46, Alex Williamson wrote: > On Fri, 22 Jan 2021 17:26:35 +0800 > Keqian Zhu wrote: > >> vfio_sanity_check_pfn_list() is used to check whether pfn_list and >> notifier are empty when remove the external domain, so it makes a >> wrong assumption that only external domain will use the pinning >> interface. >> >> Now we apply the pfn_list check when a vfio_dma is removed and apply >> the notifier check when all domains are removed. >> >> Fixes: a54eb55045ae ("vfio iommu type1: Add support for mediated devices") >> Signed-off-by: Keqian Zhu >> --- >> drivers/vfio/vfio_iommu_type1.c | 33 ++++++++++----------------------- >> 1 file changed, 10 insertions(+), 23 deletions(-) >> >> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c >> index 161725395f2f..d8c10f508321 100644 >> --- a/drivers/vfio/vfio_iommu_type1.c >> +++ b/drivers/vfio/vfio_iommu_type1.c >> @@ -957,6 +957,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, >> >> static void vfio_remove_dma(struct vfio_iommu *iommu, struct vfio_dma *dma) >> { >> + WARN_ON(!RB_EMPTY_ROOT(&dma->pfn_list)); >> vfio_unmap_unpin(iommu, dma, true); >> vfio_unlink_dma(iommu, dma); >> put_task_struct(dma->task); >> @@ -2250,23 +2251,6 @@ static void vfio_iommu_unmap_unpin_reaccount(struct vfio_iommu *iommu) >> } >> } >> >> -static void vfio_sanity_check_pfn_list(struct vfio_iommu *iommu) >> -{ >> - struct rb_node *n; >> - >> - n = rb_first(&iommu->dma_list); >> - for (; n; n = rb_next(n)) { >> - struct vfio_dma *dma; >> - >> - dma = rb_entry(n, struct vfio_dma, node); >> - >> - if (WARN_ON(!RB_EMPTY_ROOT(&dma->pfn_list))) >> - break; >> - } >> - /* mdev vendor driver must unregister notifier */ >> - WARN_ON(iommu->notifier.head); >> -} >> - >> /* >> * Called when a domain is removed in detach. It is possible that >> * the removed domain decided the iova aperture window. Modify the >> @@ -2366,10 +2350,10 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, >> kfree(group); >> >> if (list_empty(&iommu->external_domain->group_list)) { >> - vfio_sanity_check_pfn_list(iommu); >> - >> - if (!IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu)) >> + if (!IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu)) { >> + WARN_ON(iommu->notifier.head); >> vfio_iommu_unmap_unpin_all(iommu); >> + } >> >> kfree(iommu->external_domain); >> iommu->external_domain = NULL; >> @@ -2403,10 +2387,12 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, >> */ >> if (list_empty(&domain->group_list)) { >> if (list_is_singular(&iommu->domain_list)) { >> - if (!iommu->external_domain) >> + if (!iommu->external_domain) { >> + WARN_ON(iommu->notifier.head); >> vfio_iommu_unmap_unpin_all(iommu); >> - else >> + } else { >> vfio_iommu_unmap_unpin_reaccount(iommu); >> + } >> } >> iommu_domain_free(domain->domain); >> list_del(&domain->next); >> @@ -2488,9 +2474,10 @@ static void vfio_iommu_type1_release(void *iommu_data) >> struct vfio_iommu *iommu = iommu_data; >> struct vfio_domain *domain, *domain_tmp; >> >> + WARN_ON(iommu->notifier.head); > > I don't see that this does any harm, but isn't it actually redundant? > It seems vfio-core only calls the iommu backend release function after > removing all the groups, so the tests in _detach_group should catch all > cases. We're expecting the vfio bus/mdev driver to remove the notifier > when a device is closed, which necessarily occurs before detaching the > group. Thanks, > > Alex Hi Alex, Sorry that today I was busy at sending the smmu HTTU based dma dirty log tracking. I will reply you tomorrow. Thanks! Keqian. > >> + >> if (iommu->external_domain) { >> vfio_release_domain(iommu->external_domain, true); >> - vfio_sanity_check_pfn_list(iommu); >> kfree(iommu->external_domain); >> } >> > > . > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7A48C433E6 for ; Thu, 28 Jan 2021 15:21:45 +0000 (UTC) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5DC2E64E02 for ; Thu, 28 Jan 2021 15:21:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5DC2E64E02 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 0E11486C0E; Thu, 28 Jan 2021 15:21:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 12KC8sNx52jw; Thu, 28 Jan 2021 15:21:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 53E5C86658; Thu, 28 Jan 2021 15:21:44 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 320FAC0FA7; Thu, 28 Jan 2021 15:21:44 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id C92C7C013A for ; Thu, 28 Jan 2021 15:21:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id AFF5787339 for ; Thu, 28 Jan 2021 15:21:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 976XIeOZskAl for ; Thu, 28 Jan 2021 15:21:41 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by hemlock.osuosl.org (Postfix) with ESMTPS id 9077987329 for ; Thu, 28 Jan 2021 15:21:37 +0000 (UTC) Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4DRPKs1ZVgzjFdr; Thu, 28 Jan 2021 23:20:17 +0800 (CST) Received: from [10.174.184.42] (10.174.184.42) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.498.0; Thu, 28 Jan 2021 23:21:20 +0800 Subject: Re: [PATCH v3 2/2] vfio/iommu_type1: Fix some sanity checks in detach group To: Alex Williamson References: <20210122092635.19900-1-zhukeqian1@huawei.com> <20210122092635.19900-3-zhukeqian1@huawei.com> <20210127164641.36e17bf5@omen.home.shazbot.org> From: Keqian Zhu Message-ID: <5093dace-4b8a-d455-ba16-d0c2da755573@huawei.com> Date: Thu, 28 Jan 2021 23:21:20 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: <20210127164641.36e17bf5@omen.home.shazbot.org> X-Originating-IP: [10.174.184.42] X-CFilter-Loop: Reflected Cc: Mark Rutland , jiangkunkun@huawei.com, kvm@vger.kernel.org, Catalin Marinas , Kirti Wankhede , Will Deacon , kvmarm@lists.cs.columbia.edu, Marc Zyngier , Daniel Lezcano , wanghaibin.wang@huawei.com, Julien Thierry , Suzuki K Poulose , Alexios Zavras , Thomas Gleixner , linux-arm-kernel@lists.infradead.org, Cornelia Huck , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, James Morse , Andrew Morton , Robin Murphy X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On 2021/1/28 7:46, Alex Williamson wrote: > On Fri, 22 Jan 2021 17:26:35 +0800 > Keqian Zhu wrote: > >> vfio_sanity_check_pfn_list() is used to check whether pfn_list and >> notifier are empty when remove the external domain, so it makes a >> wrong assumption that only external domain will use the pinning >> interface. >> >> Now we apply the pfn_list check when a vfio_dma is removed and apply >> the notifier check when all domains are removed. >> >> Fixes: a54eb55045ae ("vfio iommu type1: Add support for mediated devices") >> Signed-off-by: Keqian Zhu >> --- >> drivers/vfio/vfio_iommu_type1.c | 33 ++++++++++----------------------- >> 1 file changed, 10 insertions(+), 23 deletions(-) >> >> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c >> index 161725395f2f..d8c10f508321 100644 >> --- a/drivers/vfio/vfio_iommu_type1.c >> +++ b/drivers/vfio/vfio_iommu_type1.c >> @@ -957,6 +957,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, >> >> static void vfio_remove_dma(struct vfio_iommu *iommu, struct vfio_dma *dma) >> { >> + WARN_ON(!RB_EMPTY_ROOT(&dma->pfn_list)); >> vfio_unmap_unpin(iommu, dma, true); >> vfio_unlink_dma(iommu, dma); >> put_task_struct(dma->task); >> @@ -2250,23 +2251,6 @@ static void vfio_iommu_unmap_unpin_reaccount(struct vfio_iommu *iommu) >> } >> } >> >> -static void vfio_sanity_check_pfn_list(struct vfio_iommu *iommu) >> -{ >> - struct rb_node *n; >> - >> - n = rb_first(&iommu->dma_list); >> - for (; n; n = rb_next(n)) { >> - struct vfio_dma *dma; >> - >> - dma = rb_entry(n, struct vfio_dma, node); >> - >> - if (WARN_ON(!RB_EMPTY_ROOT(&dma->pfn_list))) >> - break; >> - } >> - /* mdev vendor driver must unregister notifier */ >> - WARN_ON(iommu->notifier.head); >> -} >> - >> /* >> * Called when a domain is removed in detach. It is possible that >> * the removed domain decided the iova aperture window. Modify the >> @@ -2366,10 +2350,10 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, >> kfree(group); >> >> if (list_empty(&iommu->external_domain->group_list)) { >> - vfio_sanity_check_pfn_list(iommu); >> - >> - if (!IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu)) >> + if (!IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu)) { >> + WARN_ON(iommu->notifier.head); >> vfio_iommu_unmap_unpin_all(iommu); >> + } >> >> kfree(iommu->external_domain); >> iommu->external_domain = NULL; >> @@ -2403,10 +2387,12 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, >> */ >> if (list_empty(&domain->group_list)) { >> if (list_is_singular(&iommu->domain_list)) { >> - if (!iommu->external_domain) >> + if (!iommu->external_domain) { >> + WARN_ON(iommu->notifier.head); >> vfio_iommu_unmap_unpin_all(iommu); >> - else >> + } else { >> vfio_iommu_unmap_unpin_reaccount(iommu); >> + } >> } >> iommu_domain_free(domain->domain); >> list_del(&domain->next); >> @@ -2488,9 +2474,10 @@ static void vfio_iommu_type1_release(void *iommu_data) >> struct vfio_iommu *iommu = iommu_data; >> struct vfio_domain *domain, *domain_tmp; >> >> + WARN_ON(iommu->notifier.head); > > I don't see that this does any harm, but isn't it actually redundant? > It seems vfio-core only calls the iommu backend release function after > removing all the groups, so the tests in _detach_group should catch all > cases. We're expecting the vfio bus/mdev driver to remove the notifier > when a device is closed, which necessarily occurs before detaching the > group. Thanks, > > Alex Hi Alex, Sorry that today I was busy at sending the smmu HTTU based dma dirty log tracking. I will reply you tomorrow. Thanks! Keqian. > >> + >> if (iommu->external_domain) { >> vfio_release_domain(iommu->external_domain, true); >> - vfio_sanity_check_pfn_list(iommu); >> kfree(iommu->external_domain); >> } >> > > . > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D86FFC433E0 for ; Thu, 28 Jan 2021 15:21:41 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 2932E64DED for ; Thu, 28 Jan 2021 15:21:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2932E64DED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 8393A4B211; Thu, 28 Jan 2021 10:21:40 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oPpj24Z8C8rS; Thu, 28 Jan 2021 10:21:39 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 2A0314B1F9; Thu, 28 Jan 2021 10:21:39 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 67C594B185 for ; Thu, 28 Jan 2021 10:21:38 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PS7xQTsstwqT for ; Thu, 28 Jan 2021 10:21:37 -0500 (EST) Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 7AF434B109 for ; Thu, 28 Jan 2021 10:21:36 -0500 (EST) Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4DRPKs1ZVgzjFdr; Thu, 28 Jan 2021 23:20:17 +0800 (CST) Received: from [10.174.184.42] (10.174.184.42) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.498.0; Thu, 28 Jan 2021 23:21:20 +0800 Subject: Re: [PATCH v3 2/2] vfio/iommu_type1: Fix some sanity checks in detach group To: Alex Williamson References: <20210122092635.19900-1-zhukeqian1@huawei.com> <20210122092635.19900-3-zhukeqian1@huawei.com> <20210127164641.36e17bf5@omen.home.shazbot.org> From: Keqian Zhu Message-ID: <5093dace-4b8a-d455-ba16-d0c2da755573@huawei.com> Date: Thu, 28 Jan 2021 23:21:20 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: <20210127164641.36e17bf5@omen.home.shazbot.org> X-Originating-IP: [10.174.184.42] X-CFilter-Loop: Reflected Cc: kvm@vger.kernel.org, Catalin Marinas , Kirti Wankhede , Will Deacon , kvmarm@lists.cs.columbia.edu, Marc Zyngier , Joerg Roedel , Daniel Lezcano , Alexios Zavras , Thomas Gleixner , linux-arm-kernel@lists.infradead.org, Cornelia Huck , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Andrew Morton , Robin Murphy X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On 2021/1/28 7:46, Alex Williamson wrote: > On Fri, 22 Jan 2021 17:26:35 +0800 > Keqian Zhu wrote: > >> vfio_sanity_check_pfn_list() is used to check whether pfn_list and >> notifier are empty when remove the external domain, so it makes a >> wrong assumption that only external domain will use the pinning >> interface. >> >> Now we apply the pfn_list check when a vfio_dma is removed and apply >> the notifier check when all domains are removed. >> >> Fixes: a54eb55045ae ("vfio iommu type1: Add support for mediated devices") >> Signed-off-by: Keqian Zhu >> --- >> drivers/vfio/vfio_iommu_type1.c | 33 ++++++++++----------------------- >> 1 file changed, 10 insertions(+), 23 deletions(-) >> >> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c >> index 161725395f2f..d8c10f508321 100644 >> --- a/drivers/vfio/vfio_iommu_type1.c >> +++ b/drivers/vfio/vfio_iommu_type1.c >> @@ -957,6 +957,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, >> >> static void vfio_remove_dma(struct vfio_iommu *iommu, struct vfio_dma *dma) >> { >> + WARN_ON(!RB_EMPTY_ROOT(&dma->pfn_list)); >> vfio_unmap_unpin(iommu, dma, true); >> vfio_unlink_dma(iommu, dma); >> put_task_struct(dma->task); >> @@ -2250,23 +2251,6 @@ static void vfio_iommu_unmap_unpin_reaccount(struct vfio_iommu *iommu) >> } >> } >> >> -static void vfio_sanity_check_pfn_list(struct vfio_iommu *iommu) >> -{ >> - struct rb_node *n; >> - >> - n = rb_first(&iommu->dma_list); >> - for (; n; n = rb_next(n)) { >> - struct vfio_dma *dma; >> - >> - dma = rb_entry(n, struct vfio_dma, node); >> - >> - if (WARN_ON(!RB_EMPTY_ROOT(&dma->pfn_list))) >> - break; >> - } >> - /* mdev vendor driver must unregister notifier */ >> - WARN_ON(iommu->notifier.head); >> -} >> - >> /* >> * Called when a domain is removed in detach. It is possible that >> * the removed domain decided the iova aperture window. Modify the >> @@ -2366,10 +2350,10 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, >> kfree(group); >> >> if (list_empty(&iommu->external_domain->group_list)) { >> - vfio_sanity_check_pfn_list(iommu); >> - >> - if (!IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu)) >> + if (!IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu)) { >> + WARN_ON(iommu->notifier.head); >> vfio_iommu_unmap_unpin_all(iommu); >> + } >> >> kfree(iommu->external_domain); >> iommu->external_domain = NULL; >> @@ -2403,10 +2387,12 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, >> */ >> if (list_empty(&domain->group_list)) { >> if (list_is_singular(&iommu->domain_list)) { >> - if (!iommu->external_domain) >> + if (!iommu->external_domain) { >> + WARN_ON(iommu->notifier.head); >> vfio_iommu_unmap_unpin_all(iommu); >> - else >> + } else { >> vfio_iommu_unmap_unpin_reaccount(iommu); >> + } >> } >> iommu_domain_free(domain->domain); >> list_del(&domain->next); >> @@ -2488,9 +2474,10 @@ static void vfio_iommu_type1_release(void *iommu_data) >> struct vfio_iommu *iommu = iommu_data; >> struct vfio_domain *domain, *domain_tmp; >> >> + WARN_ON(iommu->notifier.head); > > I don't see that this does any harm, but isn't it actually redundant? > It seems vfio-core only calls the iommu backend release function after > removing all the groups, so the tests in _detach_group should catch all > cases. We're expecting the vfio bus/mdev driver to remove the notifier > when a device is closed, which necessarily occurs before detaching the > group. Thanks, > > Alex Hi Alex, Sorry that today I was busy at sending the smmu HTTU based dma dirty log tracking. I will reply you tomorrow. Thanks! Keqian. > >> + >> if (iommu->external_domain) { >> vfio_release_domain(iommu->external_domain, true); >> - vfio_sanity_check_pfn_list(iommu); >> kfree(iommu->external_domain); >> } >> > > . > _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CF2FC433E0 for ; Thu, 28 Jan 2021 15:23:24 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2281564DD6 for ; Thu, 28 Jan 2021 15:23:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2281564DD6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Bcnd9GhAiRX3NDV1j6b16JanXjMPbuGRxGDY1QW0Acg=; b=aHG2fUHQazDBwyhsmNrCQFFT5 zhkI8Wvu3PA6I9rSvTBqKKRt56UtG/Jrl0zrBuKGujsPShISXVWUKFJsNMNpXJvlDFfTcPg80GK+v YE5NpO4+YQXWoiKPmCoPc5srVJ+CrbWt/B/L1TBSj+cPzAFZw0OITM/LQ8YKSpVq5Qx1Dh8W1BTLj CBZMivPm4tSSCKTB77eia4mfUwVwKGufoHNXRTadAsAIAQQpP1VGDDUbSNIR8a0ywbzW3b9TjydAz od8FipIJqug1vd4R0sQasn70pfU+FXDENfPfJmo4ukvpKIpxMLyTtUl9/XWy4rdNRWeCYoT19rIRH MHSVK4DGw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l597E-0005h0-FT; Thu, 28 Jan 2021 15:21:48 +0000 Received: from szxga05-in.huawei.com ([45.249.212.191]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l5975-0005aR-6K for linux-arm-kernel@lists.infradead.org; Thu, 28 Jan 2021 15:21:42 +0000 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4DRPKs1ZVgzjFdr; Thu, 28 Jan 2021 23:20:17 +0800 (CST) Received: from [10.174.184.42] (10.174.184.42) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.498.0; Thu, 28 Jan 2021 23:21:20 +0800 Subject: Re: [PATCH v3 2/2] vfio/iommu_type1: Fix some sanity checks in detach group To: Alex Williamson References: <20210122092635.19900-1-zhukeqian1@huawei.com> <20210122092635.19900-3-zhukeqian1@huawei.com> <20210127164641.36e17bf5@omen.home.shazbot.org> From: Keqian Zhu Message-ID: <5093dace-4b8a-d455-ba16-d0c2da755573@huawei.com> Date: Thu, 28 Jan 2021 23:21:20 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: <20210127164641.36e17bf5@omen.home.shazbot.org> X-Originating-IP: [10.174.184.42] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210128_102141_414652_0DAE8F52 X-CRM114-Status: GOOD ( 20.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , jiangkunkun@huawei.com, kvm@vger.kernel.org, Catalin Marinas , Kirti Wankhede , Will Deacon , kvmarm@lists.cs.columbia.edu, Marc Zyngier , Joerg Roedel , Daniel Lezcano , wanghaibin.wang@huawei.com, Julien Thierry , Suzuki K Poulose , Alexios Zavras , Thomas Gleixner , linux-arm-kernel@lists.infradead.org, Cornelia Huck , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, James Morse , Andrew Morton , Robin Murphy Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2021/1/28 7:46, Alex Williamson wrote: > On Fri, 22 Jan 2021 17:26:35 +0800 > Keqian Zhu wrote: > >> vfio_sanity_check_pfn_list() is used to check whether pfn_list and >> notifier are empty when remove the external domain, so it makes a >> wrong assumption that only external domain will use the pinning >> interface. >> >> Now we apply the pfn_list check when a vfio_dma is removed and apply >> the notifier check when all domains are removed. >> >> Fixes: a54eb55045ae ("vfio iommu type1: Add support for mediated devices") >> Signed-off-by: Keqian Zhu >> --- >> drivers/vfio/vfio_iommu_type1.c | 33 ++++++++++----------------------- >> 1 file changed, 10 insertions(+), 23 deletions(-) >> >> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c >> index 161725395f2f..d8c10f508321 100644 >> --- a/drivers/vfio/vfio_iommu_type1.c >> +++ b/drivers/vfio/vfio_iommu_type1.c >> @@ -957,6 +957,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, >> >> static void vfio_remove_dma(struct vfio_iommu *iommu, struct vfio_dma *dma) >> { >> + WARN_ON(!RB_EMPTY_ROOT(&dma->pfn_list)); >> vfio_unmap_unpin(iommu, dma, true); >> vfio_unlink_dma(iommu, dma); >> put_task_struct(dma->task); >> @@ -2250,23 +2251,6 @@ static void vfio_iommu_unmap_unpin_reaccount(struct vfio_iommu *iommu) >> } >> } >> >> -static void vfio_sanity_check_pfn_list(struct vfio_iommu *iommu) >> -{ >> - struct rb_node *n; >> - >> - n = rb_first(&iommu->dma_list); >> - for (; n; n = rb_next(n)) { >> - struct vfio_dma *dma; >> - >> - dma = rb_entry(n, struct vfio_dma, node); >> - >> - if (WARN_ON(!RB_EMPTY_ROOT(&dma->pfn_list))) >> - break; >> - } >> - /* mdev vendor driver must unregister notifier */ >> - WARN_ON(iommu->notifier.head); >> -} >> - >> /* >> * Called when a domain is removed in detach. It is possible that >> * the removed domain decided the iova aperture window. Modify the >> @@ -2366,10 +2350,10 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, >> kfree(group); >> >> if (list_empty(&iommu->external_domain->group_list)) { >> - vfio_sanity_check_pfn_list(iommu); >> - >> - if (!IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu)) >> + if (!IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu)) { >> + WARN_ON(iommu->notifier.head); >> vfio_iommu_unmap_unpin_all(iommu); >> + } >> >> kfree(iommu->external_domain); >> iommu->external_domain = NULL; >> @@ -2403,10 +2387,12 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, >> */ >> if (list_empty(&domain->group_list)) { >> if (list_is_singular(&iommu->domain_list)) { >> - if (!iommu->external_domain) >> + if (!iommu->external_domain) { >> + WARN_ON(iommu->notifier.head); >> vfio_iommu_unmap_unpin_all(iommu); >> - else >> + } else { >> vfio_iommu_unmap_unpin_reaccount(iommu); >> + } >> } >> iommu_domain_free(domain->domain); >> list_del(&domain->next); >> @@ -2488,9 +2474,10 @@ static void vfio_iommu_type1_release(void *iommu_data) >> struct vfio_iommu *iommu = iommu_data; >> struct vfio_domain *domain, *domain_tmp; >> >> + WARN_ON(iommu->notifier.head); > > I don't see that this does any harm, but isn't it actually redundant? > It seems vfio-core only calls the iommu backend release function after > removing all the groups, so the tests in _detach_group should catch all > cases. We're expecting the vfio bus/mdev driver to remove the notifier > when a device is closed, which necessarily occurs before detaching the > group. Thanks, > > Alex Hi Alex, Sorry that today I was busy at sending the smmu HTTU based dma dirty log tracking. I will reply you tomorrow. Thanks! Keqian. > >> + >> if (iommu->external_domain) { >> vfio_release_domain(iommu->external_domain, true); >> - vfio_sanity_check_pfn_list(iommu); >> kfree(iommu->external_domain); >> } >> > > . > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel