From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25A40C433F5 for ; Mon, 15 Nov 2021 02:10:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0F4CE60F46 for ; Mon, 15 Nov 2021 02:10:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234999AbhKOCN3 (ORCPT ); Sun, 14 Nov 2021 21:13:29 -0500 Received: from mga03.intel.com ([134.134.136.65]:15410 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230395AbhKOCNV (ORCPT ); Sun, 14 Nov 2021 21:13:21 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10168"; a="233307209" X-IronPort-AV: E=Sophos;i="5.87,235,1631602800"; d="scan'208";a="233307209" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2021 18:10:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,235,1631602800"; d="scan'208";a="505714518" Received: from allen-box.sh.intel.com ([10.239.159.118]) by orsmga008.jf.intel.com with ESMTP; 14 Nov 2021 18:10:21 -0800 From: Lu Baolu To: Greg Kroah-Hartman , Joerg Roedel , Alex Williamson , Bjorn Helgaas , Jason Gunthorpe , Kevin Tian , Ashok Raj Cc: Will Deacon , rafael@kernel.org, Diana Craciun , Cornelia Huck , Eric Auger , Liu Yi L , Jacob jun Pan , Chaitanya Kulkarni , iommu@lists.linux-foundation.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH 00/11] Fix BUG_ON in vfio_iommu_group_notifier() Date: Mon, 15 Nov 2021 10:05:41 +0800 Message-Id: <20211115020552.2378167-1-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi folks, The iommu group is the minimal isolation boundary for DMA. Devices in a group can access each other's MMIO registers via peer to peer DMA and also need share the same I/O address space. Once the I/O address space is assigned to user control it is no longer available to the dma_map* API, which effectively makes the DMA API non-working. Second, userspace can use DMA initiated by a device that it controls to access the MMIO spaces of other devices in the group. This allows userspace to indirectly attack any kernel owned device and it's driver. Therefore groups must either be entirely under kernel control or userspace control, never a mixture. Unfortunately some systems have problems with the granularity of groups and there are a couple of important exceptions: - pci_stub allows the admin to block driver binding on a device and make it permanently shared with userspace. Since PCI stub does not do DMA it is safe, however the admin must understand that using pci_stub allows userspace to attack whatever device it was bound it. - PCI bridges are sometimes included in groups. Typically PCI bridges do not use DMA, and generally do not have MMIO regions. Generally any device that does not have any MMIO registers is a possible candidate for an exception. Currently vfio adopts a workaround to detect violations of the above restrictions by monitoring the driver core BOUND event, and hardwiring the above exceptions. Since there is no way for vfio to reject driver binding at this point, BUG_ON() is triggered if a violation is captured (kernel driver BOUND event on a group which already has some devices assigned to userspace). Aside from the bad user experience this opens a way for root userspace to crash the kernel, even in high integrity configurations, by manipulating the module binding and triggering the BUG_ON. This series solves this problem by making the user/kernel ownership a core concept at the IOMMU layer. The driver core enforces kernel ownership while drivers are bound and violations now result in a error codes during probe, not BUG_ON failures. Patch partitions: [PATCH 1-2]: Detect DMA ownership conflicts during driver binding; [PATCH 3-6]: Add security context management for assigned devices; [PATCH 7-11]: Various cleanups. Ideas contributed by: Jason Gunthorpe Kevin Tian Ashok Raj Lu Baolu Review contributors: Jason Gunthorpe Kevin Tian Ashok Raj Liu Yi L Jacob jun Pan Chaitanya Kulkarni This also is part one of three initial series for IOMMUFD: * Move IOMMU Group security into the iommu layer - Generic IOMMUFD implementation - VFIO ability to consume IOMMUFD This is based on v5.16-rc1 and available on github: https://github.com/LuBaolu/intel-iommu/commits/iommu-dma-ownership-v1 Best regards, baolu Jason Gunthorpe (1): vfio: Delete the unbound_list Lu Baolu (10): iommu: Add device dma ownership set/release interfaces driver core: Set DMA ownership during driver bind/unbind PCI: pci_stub: Suppress kernel DMA ownership auto-claiming PCI: portdrv: Suppress kernel DMA ownership auto-claiming iommu: Add security context management for assigned devices iommu: Expose group variants of dma ownership interfaces vfio: Use DMA_OWNER_USER to declaim passthrough devices vfio: Remove use of vfio_group_viable() vfio: Remove iommu group notifier iommu: Remove iommu group changes notifier include/linux/device/driver.h | 7 +- include/linux/iommu.h | 75 ++++--- drivers/base/dd.c | 12 ++ drivers/iommu/iommu.c | 274 ++++++++++++++++++-------- drivers/pci/pci-stub.c | 3 + drivers/pci/pcie/portdrv_pci.c | 2 + drivers/vfio/fsl-mc/vfio_fsl_mc.c | 1 + drivers/vfio/pci/vfio_pci.c | 3 + drivers/vfio/platform/vfio_amba.c | 1 + drivers/vfio/platform/vfio_platform.c | 1 + drivers/vfio/vfio.c | 247 ++--------------------- 11 files changed, 294 insertions(+), 332 deletions(-) -- 2.25.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25703C433FE for ; Mon, 15 Nov 2021 02:10:34 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB17A61156 for ; Mon, 15 Nov 2021 02:10:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BB17A61156 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 79E1180D65; Mon, 15 Nov 2021 02:10:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fUbwZVfLWEl1; Mon, 15 Nov 2021 02:10:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id E6BAA80D5F; Mon, 15 Nov 2021 02:10:29 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B4E68C002E; Mon, 15 Nov 2021 02:10:29 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id A36A0C0012 for ; Mon, 15 Nov 2021 02:10:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6E52740259 for ; Mon, 15 Nov 2021 02:10:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 31_7Wzw_YTKp for ; Mon, 15 Nov 2021 02:10:27 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5512740255 for ; Mon, 15 Nov 2021 02:10:27 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10168"; a="220575011" X-IronPort-AV: E=Sophos;i="5.87,235,1631602800"; d="scan'208";a="220575011" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2021 18:10:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,235,1631602800"; d="scan'208";a="505714518" Received: from allen-box.sh.intel.com ([10.239.159.118]) by orsmga008.jf.intel.com with ESMTP; 14 Nov 2021 18:10:21 -0800 From: Lu Baolu To: Greg Kroah-Hartman , Joerg Roedel , Alex Williamson , Bjorn Helgaas , Jason Gunthorpe , Kevin Tian , Ashok Raj Subject: [PATCH 00/11] Fix BUG_ON in vfio_iommu_group_notifier() Date: Mon, 15 Nov 2021 10:05:41 +0800 Message-Id: <20211115020552.2378167-1-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Cc: Chaitanya Kulkarni , kvm@vger.kernel.org, rafael@kernel.org, linux-pci@vger.kernel.org, Cornelia Huck , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Jacob jun Pan , Diana Craciun , Will Deacon X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" Hi folks, The iommu group is the minimal isolation boundary for DMA. Devices in a group can access each other's MMIO registers via peer to peer DMA and also need share the same I/O address space. Once the I/O address space is assigned to user control it is no longer available to the dma_map* API, which effectively makes the DMA API non-working. Second, userspace can use DMA initiated by a device that it controls to access the MMIO spaces of other devices in the group. This allows userspace to indirectly attack any kernel owned device and it's driver. Therefore groups must either be entirely under kernel control or userspace control, never a mixture. Unfortunately some systems have problems with the granularity of groups and there are a couple of important exceptions: - pci_stub allows the admin to block driver binding on a device and make it permanently shared with userspace. Since PCI stub does not do DMA it is safe, however the admin must understand that using pci_stub allows userspace to attack whatever device it was bound it. - PCI bridges are sometimes included in groups. Typically PCI bridges do not use DMA, and generally do not have MMIO regions. Generally any device that does not have any MMIO registers is a possible candidate for an exception. Currently vfio adopts a workaround to detect violations of the above restrictions by monitoring the driver core BOUND event, and hardwiring the above exceptions. Since there is no way for vfio to reject driver binding at this point, BUG_ON() is triggered if a violation is captured (kernel driver BOUND event on a group which already has some devices assigned to userspace). Aside from the bad user experience this opens a way for root userspace to crash the kernel, even in high integrity configurations, by manipulating the module binding and triggering the BUG_ON. This series solves this problem by making the user/kernel ownership a core concept at the IOMMU layer. The driver core enforces kernel ownership while drivers are bound and violations now result in a error codes during probe, not BUG_ON failures. Patch partitions: [PATCH 1-2]: Detect DMA ownership conflicts during driver binding; [PATCH 3-6]: Add security context management for assigned devices; [PATCH 7-11]: Various cleanups. Ideas contributed by: Jason Gunthorpe Kevin Tian Ashok Raj Lu Baolu Review contributors: Jason Gunthorpe Kevin Tian Ashok Raj Liu Yi L Jacob jun Pan Chaitanya Kulkarni This also is part one of three initial series for IOMMUFD: * Move IOMMU Group security into the iommu layer - Generic IOMMUFD implementation - VFIO ability to consume IOMMUFD This is based on v5.16-rc1 and available on github: https://github.com/LuBaolu/intel-iommu/commits/iommu-dma-ownership-v1 Best regards, baolu Jason Gunthorpe (1): vfio: Delete the unbound_list Lu Baolu (10): iommu: Add device dma ownership set/release interfaces driver core: Set DMA ownership during driver bind/unbind PCI: pci_stub: Suppress kernel DMA ownership auto-claiming PCI: portdrv: Suppress kernel DMA ownership auto-claiming iommu: Add security context management for assigned devices iommu: Expose group variants of dma ownership interfaces vfio: Use DMA_OWNER_USER to declaim passthrough devices vfio: Remove use of vfio_group_viable() vfio: Remove iommu group notifier iommu: Remove iommu group changes notifier include/linux/device/driver.h | 7 +- include/linux/iommu.h | 75 ++++--- drivers/base/dd.c | 12 ++ drivers/iommu/iommu.c | 274 ++++++++++++++++++-------- drivers/pci/pci-stub.c | 3 + drivers/pci/pcie/portdrv_pci.c | 2 + drivers/vfio/fsl-mc/vfio_fsl_mc.c | 1 + drivers/vfio/pci/vfio_pci.c | 3 + drivers/vfio/platform/vfio_amba.c | 1 + drivers/vfio/platform/vfio_platform.c | 1 + drivers/vfio/vfio.c | 247 ++--------------------- 11 files changed, 294 insertions(+), 332 deletions(-) -- 2.25.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu