From mboxrd@z Thu Jan 1 00:00:00 1970 From: Robin Murphy Subject: Re: [PATCH v3 00/34] iommu: Move iommu_group setup to IOMMU core code Date: Wed, 1 Jul 2020 11:53:07 +0100 Message-ID: <9b0ef27a-f249-a90b-9899-e53b946f83cc@arm.com> References: <20200429133712.31431-1-joro@8bytes.org> <20200701004020.GA6221@lca.pw> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20200701004020.GA6221-J5quhbR+WMc@public.gmane.org> Content-Language: en-GB Sender: linux-tegra-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Qian Cai , Joerg Roedel Cc: Heiko Stuebner , Bjorn Andersson , linux-tegra-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Thierry Reding , Daniel Drake , Will Deacon , Jean-Philippe Brucker , linux-samsung-soc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Krzysztof Kozlowski , Jonathan Hunter , linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, Andy Gross , jonathan.derrick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org, linux-s390-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-arm-msm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, Matthias Brugger , virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Gerald Schaefer , David Woodhouse List-Id: linux-tegra@vger.kernel.org On 2020-07-01 01:40, Qian Cai wrote: > Looks like this patchset introduced an use-after-free on arm-smmu-v3. > > Reproduced using mlx5, > > # echo 1 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs > # echo 0 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs > > The .config, > https://github.com/cailca/linux-mm/blob/master/arm64.config > > Looking at the free stack, > > iommu_release_device->iommu_group_remove_device > > was introduced in 07/34 ("iommu: Add probe_device() and release_device() > call-backs"). Right, iommu_group_remove_device can tear down the group and call ->domain_free before the driver has any knowledge of the last device going away via the ->release_device call. I guess the question is do we simply flip the call order in iommu_release_device() so drivers can easily clean up their internal per-device state first, or do we now want them to be robust against freeing domains with devices still nominally attached? Robin. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 063BBC433E0 for ; Wed, 1 Jul 2020 10:53:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DA5AE20775 for ; Wed, 1 Jul 2020 10:53:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730104AbgGAKxN (ORCPT ); Wed, 1 Jul 2020 06:53:13 -0400 Received: from foss.arm.com ([217.140.110.172]:57020 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729892AbgGAKxN (ORCPT ); Wed, 1 Jul 2020 06:53:13 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B597030E; Wed, 1 Jul 2020 03:53:12 -0700 (PDT) Received: from [10.57.21.32] (unknown [10.57.21.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BB9483F73C; Wed, 1 Jul 2020 03:53:08 -0700 (PDT) Subject: Re: [PATCH v3 00/34] iommu: Move iommu_group setup to IOMMU core code To: Qian Cai , Joerg Roedel Cc: Heiko Stuebner , Bjorn Andersson , linux-tegra@vger.kernel.org, Thierry Reding , Daniel Drake , Will Deacon , Jean-Philippe Brucker , linux-samsung-soc@vger.kernel.org, iommu@lists.linux-foundation.org, Krzysztof Kozlowski , Jonathan Hunter , linux-rockchip@lists.infradead.org, Andy Gross , jonathan.derrick@intel.com, linux-s390@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mediatek@lists.infradead.org, Matthias Brugger , virtualization@lists.linux-foundation.org, Gerald Schaefer , David Woodhouse , linux-kernel@vger.kernel.org, Kukjin Kim References: <20200429133712.31431-1-joro@8bytes.org> <20200701004020.GA6221@lca.pw> From: Robin Murphy Message-ID: <9b0ef27a-f249-a90b-9899-e53b946f83cc@arm.com> Date: Wed, 1 Jul 2020 11:53:07 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20200701004020.GA6221@lca.pw> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On 2020-07-01 01:40, Qian Cai wrote: > Looks like this patchset introduced an use-after-free on arm-smmu-v3. > > Reproduced using mlx5, > > # echo 1 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs > # echo 0 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs > > The .config, > https://github.com/cailca/linux-mm/blob/master/arm64.config > > Looking at the free stack, > > iommu_release_device->iommu_group_remove_device > > was introduced in 07/34 ("iommu: Add probe_device() and release_device() > call-backs"). Right, iommu_group_remove_device can tear down the group and call ->domain_free before the driver has any knowledge of the last device going away via the ->release_device call. I guess the question is do we simply flip the call order in iommu_release_device() so drivers can easily clean up their internal per-device state first, or do we now want them to be robust against freeing domains with devices still nominally attached? Robin. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FB6FC433E1 for ; Wed, 1 Jul 2020 10:53:18 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D38B720747 for ; Wed, 1 Jul 2020 10:53:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D38B720747 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 8F61D2FCDA; Wed, 1 Jul 2020 10:53:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PrhjA49DAOCs; Wed, 1 Jul 2020 10:53:15 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id C4DA926387; Wed, 1 Jul 2020 10:53:15 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9E78FC0865; Wed, 1 Jul 2020 10:53:15 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6B328C0733; Wed, 1 Jul 2020 10:53:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 52D6F8AC04; Wed, 1 Jul 2020 10:53:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 03IVVzt2GxJ8; Wed, 1 Jul 2020 10:53:13 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by hemlock.osuosl.org (Postfix) with ESMTP id 8AA5B8AC02; Wed, 1 Jul 2020 10:53:13 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B597030E; Wed, 1 Jul 2020 03:53:12 -0700 (PDT) Received: from [10.57.21.32] (unknown [10.57.21.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BB9483F73C; Wed, 1 Jul 2020 03:53:08 -0700 (PDT) Subject: Re: [PATCH v3 00/34] iommu: Move iommu_group setup to IOMMU core code To: Qian Cai , Joerg Roedel References: <20200429133712.31431-1-joro@8bytes.org> <20200701004020.GA6221@lca.pw> From: Robin Murphy Message-ID: <9b0ef27a-f249-a90b-9899-e53b946f83cc@arm.com> Date: Wed, 1 Jul 2020 11:53:07 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20200701004020.GA6221@lca.pw> Content-Language: en-GB Cc: Heiko Stuebner , virtualization@lists.linux-foundation.org, Matthias Brugger , Thierry Reding , Daniel Drake , Will Deacon , Jean-Philippe Brucker , linux-samsung-soc@vger.kernel.org, Krzysztof Kozlowski , Jonathan Hunter , linux-rockchip@lists.infradead.org, Andy Gross , jonathan.derrick@intel.com, linux-s390@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-tegra@vger.kernel.org, Bjorn Andersson , Gerald Schaefer , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Kukjin Kim , David Woodhouse X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On 2020-07-01 01:40, Qian Cai wrote: > Looks like this patchset introduced an use-after-free on arm-smmu-v3. > > Reproduced using mlx5, > > # echo 1 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs > # echo 0 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs > > The .config, > https://github.com/cailca/linux-mm/blob/master/arm64.config > > Looking at the free stack, > > iommu_release_device->iommu_group_remove_device > > was introduced in 07/34 ("iommu: Add probe_device() and release_device() > call-backs"). Right, iommu_group_remove_device can tear down the group and call ->domain_free before the driver has any knowledge of the last device going away via the ->release_device call. I guess the question is do we simply flip the call order in iommu_release_device() so drivers can easily clean up their internal per-device state first, or do we now want them to be robust against freeing domains with devices still nominally attached? Robin. _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E016CC433E0 for ; Wed, 1 Jul 2020 10:53:31 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B0BD520747 for ; Wed, 1 Jul 2020 10:53:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="P5ZcytTQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0BD520747 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2cOC480t4gZ0EgcOJse2IObOoNR7OpnOi4I0LIOSpVo=; b=P5ZcytTQljZXbATAqB7ng+X5t 3Q0sXFCid7pbHcnNPhSZsZpdQ7c2i1I2DeSb/tvxSEMzn4RHzv27EiWKicD78Vr8GGMJ7iajK2EoM aks7z0KbAwNuGDCzft6Wf9Hwc5dkl00z8+6vLne/fCs3xaMR4igGMOLlyR0IL4cgDJkQIBkjhZWGa PNBm3gClYNyh/Pp1gdmDpGRDy1vx8bv3aCeWuESEfn/vSDl+sU7BcjOLsLIQk/gV7b271TUqa/yeU 9nvTR8VVzOQEVEP4nrL/6vE5qpXQQ3v66CUDTqmlmzUJ91BOkPPJ4E3F9RbJgRDoz28/CDki/Ry8s VH+iH9xFQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jqaMh-0000ix-Vq; Wed, 01 Jul 2020 10:53:20 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jqaMe-0000h8-MG; Wed, 01 Jul 2020 10:53:18 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B597030E; Wed, 1 Jul 2020 03:53:12 -0700 (PDT) Received: from [10.57.21.32] (unknown [10.57.21.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BB9483F73C; Wed, 1 Jul 2020 03:53:08 -0700 (PDT) Subject: Re: [PATCH v3 00/34] iommu: Move iommu_group setup to IOMMU core code To: Qian Cai , Joerg Roedel References: <20200429133712.31431-1-joro@8bytes.org> <20200701004020.GA6221@lca.pw> From: Robin Murphy Message-ID: <9b0ef27a-f249-a90b-9899-e53b946f83cc@arm.com> Date: Wed, 1 Jul 2020 11:53:07 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20200701004020.GA6221@lca.pw> Content-Language: en-GB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200701_065316_843681_EBD79041 X-CRM114-Status: GOOD ( 17.08 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Heiko Stuebner , virtualization@lists.linux-foundation.org, Matthias Brugger , Thierry Reding , Daniel Drake , Will Deacon , Jean-Philippe Brucker , linux-samsung-soc@vger.kernel.org, Krzysztof Kozlowski , Jonathan Hunter , linux-rockchip@lists.infradead.org, Andy Gross , jonathan.derrick@intel.com, linux-s390@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-tegra@vger.kernel.org, Bjorn Andersson , Gerald Schaefer , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Kukjin Kim , David Woodhouse Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org On 2020-07-01 01:40, Qian Cai wrote: > Looks like this patchset introduced an use-after-free on arm-smmu-v3. > > Reproduced using mlx5, > > # echo 1 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs > # echo 0 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs > > The .config, > https://github.com/cailca/linux-mm/blob/master/arm64.config > > Looking at the free stack, > > iommu_release_device->iommu_group_remove_device > > was introduced in 07/34 ("iommu: Add probe_device() and release_device() > call-backs"). Right, iommu_group_remove_device can tear down the group and call ->domain_free before the driver has any knowledge of the last device going away via the ->release_device call. I guess the question is do we simply flip the call order in iommu_release_device() so drivers can easily clean up their internal per-device state first, or do we now want them to be robust against freeing domains with devices still nominally attached? Robin. _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek