From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B23ACC433EF for ; Tue, 5 Apr 2022 19:10:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6C31D61044; Tue, 5 Apr 2022 19:10:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OqkfM141pZUG; Tue, 5 Apr 2022 19:10:55 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 502D060B4C; Tue, 5 Apr 2022 19:10:55 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8D800C0012; Tue, 5 Apr 2022 19:10:54 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8FD1AC0012 for ; Tue, 5 Apr 2022 19:10:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6CE8C4151E for ; Tue, 5 Apr 2022 19:10:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vy-0n4qgftrx for ; Tue, 5 Apr 2022 19:10:51 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5FDCA40A03 for ; Tue, 5 Apr 2022 19:10:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649185850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ufYYfNJ31ZxJiptHxZVI0cJDY245GxkfqBFvOLAxMek=; b=i4sxydNzBzlt+RbQ8mFFxcb6emuA4HilpqZDkJBC1AbeKU0V9Kjw7xmgskakUyPnT5zVxc vxqDFrmoTIDOCIN5oayzjMhM3KKLW4g5tNYnh92A1I2xwDwkqC3aYF2pNcsshL3cHZe/nt xvWQ4gOZvAH+9cWlRGrK/RvGUzSChi0= Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-563-5BKK9o0HPxSMqVxE8AVuVg-1; Tue, 05 Apr 2022 15:10:47 -0400 X-MC-Unique: 5BKK9o0HPxSMqVxE8AVuVg-1 Received: by mail-io1-f71.google.com with SMTP id y3-20020a056602178300b00645d25c30c1so143578iox.6 for ; Tue, 05 Apr 2022 12:10:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=ufYYfNJ31ZxJiptHxZVI0cJDY245GxkfqBFvOLAxMek=; b=oZIdgXT/FJouWqtqY9YDomhNVAuf9IuFQx+sYvpo7DBlb8chYyK5asEj3+wgXj2XNH I+I8N5a4FBeTwGjSuKkL3ZnlRVIaUEXHKrltIrWaSD9zDvClq2mTebQa2h32BhKsNAc4 rS6sSexWPYAm+2Uj+WnndriJY9Gg4SwBmz1rjNqpHdqsTDD261JIOVtKgvkFZFsDB9ph RjQrjWFx+oG0+oMamTsz2ERmIch+pmlWPtN5/6G3ucGtUgzqSTTcdL2MmqxSulN6yGHj CMw525r0xRFgRN2mPG+peCny7RJ2DotOxxT/DRyY17hzWfqcoUDs4yYTXz+NLcBAWzmY /JZw== X-Gm-Message-State: AOAM53036KHQVVmTGvBpliYj7HfeQ6nhaQ+k09YU1PatPg4MztXJJT1p SgXsBeOeUrhrtNrIN7qjR58PEY+bR2Al8HSxzdl5HMfH2m6XJq4k1s1RQ2o98FWxo69aFvxb3hS CBl5ATjCVbncpeGARbhQQtZEN61teHQ== X-Received: by 2002:a92:cdad:0:b0:2c6:7b76:a086 with SMTP id g13-20020a92cdad000000b002c67b76a086mr2568134ild.5.1649185846542; Tue, 05 Apr 2022 12:10:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxocs2ZVaXF5YVJwjGBcLRqnK5UBIr3mQlWSQgNmkFdgGBC2rjlpWZLrLl9AI2wrgBUaUJw1w== X-Received: by 2002:a92:cdad:0:b0:2c6:7b76:a086 with SMTP id g13-20020a92cdad000000b002c67b76a086mr2568112ild.5.1649185846282; Tue, 05 Apr 2022 12:10:46 -0700 (PDT) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id x2-20020a056602160200b006463c1977f9sm9314961iow.22.2022.04.05.12.10.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Apr 2022 12:10:45 -0700 (PDT) Date: Tue, 5 Apr 2022 13:10:44 -0600 From: Alex Williamson To: Jason Gunthorpe Subject: Re: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Message-ID: <20220405131044.23910b77.alex.williamson@redhat.com> In-Reply-To: <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> References: <0-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> Organization: Red Hat MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=alex.williamson@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Nelson Escobar , "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org, Will Deacon , Christoph Hellwig , linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux-foundation.org, Gerald Schaefer , David Woodhouse , "Tian, Kevin" , linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org, Cornelia Huck , Christian Benvenuti , Robin Murphy X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Tue, 5 Apr 2022 13:16:01 -0300 Jason Gunthorpe wrote: > dev_is_dma_coherent() is the control to determine if IOMMU_CACHE can be > supported. > > IOMMU_CACHE means that normal DMAs do not require any additional coherency > mechanism and is the basic uAPI that VFIO exposes to userspace. For > instance VFIO applications like DPDK will not work if additional coherency > operations are required. > > Therefore check dev_is_dma_coherent() before allowing a device to join a > domain. This will block device/platform/iommu combinations from using VFIO > that do not support cache coherent DMA. > > Signed-off-by: Jason Gunthorpe > --- > drivers/vfio/vfio.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > index a4555014bd1e72..2a3aa3e742d943 100644 > --- a/drivers/vfio/vfio.c > +++ b/drivers/vfio/vfio.c > @@ -32,6 +32,7 @@ > #include > #include > #include > +#include > #include "vfio.h" > > #define DRIVER_VERSION "0.3" > @@ -1348,6 +1349,11 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) > if (IS_ERR(device)) > return PTR_ERR(device); > > + if (group->type == VFIO_IOMMU && !dev_is_dma_coherent(device->dev)) { > + ret = -ENODEV; > + goto err_device_put; > + } > + Failing at the point where the user is trying to gain access to the device seems a little late in the process and opaque, wouldn't we rather have vfio bus drivers fail to probe such devices? I'd expect this to occur in the vfio_register_group_dev() path. Thanks, Alex > if (!try_module_get(device->dev->driver->owner)) { > ret = -ENODEV; > goto err_device_put; _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 67D0AC433F5 for ; Tue, 5 Apr 2022 19:10:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 04EB940222; Tue, 5 Apr 2022 19:10:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g8pejL3UhAyt; Tue, 5 Apr 2022 19:10:55 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 9988B402F3; Tue, 5 Apr 2022 19:10:54 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 46FA2C002C; Tue, 5 Apr 2022 19:10:54 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2A030C0012 for ; Tue, 5 Apr 2022 19:10:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0C896827CE for ; Tue, 5 Apr 2022 19:10:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VHvTR6GSJ13R for ; Tue, 5 Apr 2022 19:10:49 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id C5F7782591 for ; Tue, 5 Apr 2022 19:10:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649185848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ufYYfNJ31ZxJiptHxZVI0cJDY245GxkfqBFvOLAxMek=; b=Z9xUuWIFR1s5Ev4w2nyOD2v0z3fHci6g13TOV980JOAop8ZJh1B/KGPbUnOLPzwiod2D5n zR9KVBTDx3Z1Fn096cuMlF3ritv9wL0sh6HY7KAWGny3m03LjZYEBUy5itgGqEsLKTFi79 JEkrQ5Axjt92XOV3XQSYNJdJKHT4Qkg= Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-658-37AqgOSgODCoxRRiLZymNA-1; Tue, 05 Apr 2022 15:10:47 -0400 X-MC-Unique: 37AqgOSgODCoxRRiLZymNA-1 Received: by mail-io1-f70.google.com with SMTP id i19-20020a5d9353000000b006495ab76af6so179658ioo.0 for ; Tue, 05 Apr 2022 12:10:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=ufYYfNJ31ZxJiptHxZVI0cJDY245GxkfqBFvOLAxMek=; b=injPos5WqEEd9aeypRJA7pRTRfICYoeJkDjkbXo8DPfsN6PBCLbwaTgTqgPuB0lg3q 7aB4VJnpV/kOq9XVO3EqRbdVsrtKkdwAWYXtyV2iZuxgnSHh0O8qPeAWFHhYmeA1zpjr ZeV0w+UpwGBDZ3GYUAE650YuJ5Xznm042P9ef+hafJSPaUgsMDYP41zTIdFtQ//mqkMs nSrVEVkEZBImqTQG7np39i2Udl35liigZCFkV6AbBtb+kltJc+Rzj4Zx+ixf5DztfMA4 oRVgLPRQUz2cDru6Q6gqNIdq1Ant/VFnmPRX9qN6TvKwdEX6XjJfqqd0azchIiU2iDDN VHIg== X-Gm-Message-State: AOAM533h3QgZjqdhXRD7fODiTKdon4DMx8owW387BmbWkYtazT714ETK F/qQekbcNbyP1/2PxQxA15CccN7wpxoniQ9hAtOzwHpl1XfPVeyfAfMopExphC7CFYvFBhKT37m f0j/7IO0Gd8WNqeD5FLtPq/cxZLKam2xTV0oTM1oI3w== X-Received: by 2002:a92:cdad:0:b0:2c6:7b76:a086 with SMTP id g13-20020a92cdad000000b002c67b76a086mr2568140ild.5.1649185846546; Tue, 05 Apr 2022 12:10:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxocs2ZVaXF5YVJwjGBcLRqnK5UBIr3mQlWSQgNmkFdgGBC2rjlpWZLrLl9AI2wrgBUaUJw1w== X-Received: by 2002:a92:cdad:0:b0:2c6:7b76:a086 with SMTP id g13-20020a92cdad000000b002c67b76a086mr2568112ild.5.1649185846282; Tue, 05 Apr 2022 12:10:46 -0700 (PDT) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id x2-20020a056602160200b006463c1977f9sm9314961iow.22.2022.04.05.12.10.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Apr 2022 12:10:45 -0700 (PDT) Date: Tue, 5 Apr 2022 13:10:44 -0600 From: Alex Williamson To: Jason Gunthorpe Subject: Re: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Message-ID: <20220405131044.23910b77.alex.williamson@redhat.com> In-Reply-To: <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> References: <0-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> Organization: Red Hat MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=alex.williamson@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Nelson Escobar , Matthew Rosato , "Michael S. Tsirkin" , virtualization@lists.linux-foundation.org, Will Deacon , Christoph Hellwig , linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-rdma@vger.kernel.org, Joerg Roedel , iommu@lists.linux-foundation.org, Gerald Schaefer , David Woodhouse , linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org, Cornelia Huck , Rob Clark , Suravee Suthikulpanit , Christian Benvenuti , Robin Murphy , Lu Baolu X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" On Tue, 5 Apr 2022 13:16:01 -0300 Jason Gunthorpe wrote: > dev_is_dma_coherent() is the control to determine if IOMMU_CACHE can be > supported. > > IOMMU_CACHE means that normal DMAs do not require any additional coherency > mechanism and is the basic uAPI that VFIO exposes to userspace. For > instance VFIO applications like DPDK will not work if additional coherency > operations are required. > > Therefore check dev_is_dma_coherent() before allowing a device to join a > domain. This will block device/platform/iommu combinations from using VFIO > that do not support cache coherent DMA. > > Signed-off-by: Jason Gunthorpe > --- > drivers/vfio/vfio.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > index a4555014bd1e72..2a3aa3e742d943 100644 > --- a/drivers/vfio/vfio.c > +++ b/drivers/vfio/vfio.c > @@ -32,6 +32,7 @@ > #include > #include > #include > +#include > #include "vfio.h" > > #define DRIVER_VERSION "0.3" > @@ -1348,6 +1349,11 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) > if (IS_ERR(device)) > return PTR_ERR(device); > > + if (group->type == VFIO_IOMMU && !dev_is_dma_coherent(device->dev)) { > + ret = -ENODEV; > + goto err_device_put; > + } > + Failing at the point where the user is trying to gain access to the device seems a little late in the process and opaque, wouldn't we rather have vfio bus drivers fail to probe such devices? I'd expect this to occur in the vfio_register_group_dev() path. Thanks, Alex > if (!try_module_get(device->dev->driver->owner)) { > ret = -ENODEV; > goto err_device_put; _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F227BC433EF for ; Tue, 5 Apr 2022 19:12:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZXk5iarS1w4nZaBGI2/+DQBLDUsYoDSi1gnf4zgd8HA=; b=z+6x+iAjIBG0U3 XlpaiW6D1ClJI1ZdVmfkn1Cp5CLJL+T70Ho2gatWzXiKeoR86gzXbQUMrNi0SUdWK6nP9+SmBxRkx JvW2Z+clroat13on1x/Bs6fNeC/Hl751k/2QyXMJhDVHaiH/6Bbut52HfdGAYpwyPFVBPMGpQ+hhp kSTMmfMmmx2BY3TVQBZt+gmcDjqCHUsmPRbncP48DdpcZ91JlpED6QqVpCXp1BKtO0OkpYlet2Mwu tL+eGUtQ+ClPQxZra3fxgFg2e1THdgq3207VzcHUK3AeqvdK8ZkWfACQbxmgW2xQLRLIfj3ElSMya WtsNFN6W+kLRVz9Gheeg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nboZu-002IjS-30; Tue, 05 Apr 2022 19:10:58 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nboZq-002Ii8-Tp for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 19:10:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649185848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ufYYfNJ31ZxJiptHxZVI0cJDY245GxkfqBFvOLAxMek=; b=Z9xUuWIFR1s5Ev4w2nyOD2v0z3fHci6g13TOV980JOAop8ZJh1B/KGPbUnOLPzwiod2D5n zR9KVBTDx3Z1Fn096cuMlF3ritv9wL0sh6HY7KAWGny3m03LjZYEBUy5itgGqEsLKTFi79 JEkrQ5Axjt92XOV3XQSYNJdJKHT4Qkg= Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-492-FP1O8nNgN8yMeFu4nfa66A-1; Tue, 05 Apr 2022 15:10:47 -0400 X-MC-Unique: FP1O8nNgN8yMeFu4nfa66A-1 Received: by mail-io1-f72.google.com with SMTP id h14-20020a05660208ce00b00645c339411bso136444ioz.8 for ; Tue, 05 Apr 2022 12:10:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=ufYYfNJ31ZxJiptHxZVI0cJDY245GxkfqBFvOLAxMek=; b=KJ68wUrtpl8dN+z+9cSKCp/Zz2nbyRXyr4Pfr2ScQBu/9yrLoO4MWlRgs7vHANF3Ml kq7/w3hKZDou94fa4wegYqgGFMQWu1szlj2xSPIVvlv1jfkdzqHXIQ7zpOGf+R046jHK pI5jGWdt+AuzHA5SejLtunrqH1N2FG5QdXyiemTqww1zpNVx5UEf8ekG17pJZwMAxyf2 7q/tSSeSj9SHl4+itRjqKmcJ0SwmRru4miPGvBNLYebhBuNfPZOn8HlZMzf5aQQ53zHy PBkB1K+G80qJBscLCrnJDO7WHQ2dDUWjbLzxX8uoBaPaXdhEIMD1pve7jSxPUxGwNriy cx5Q== X-Gm-Message-State: AOAM530m+/FG3Gr+6YYqfdr05+0fC0GlGCnPmINqszeCEbbjvMc2vUcO 8bPqsTIxGUbIt15acuZHe+xKMHarBouEoB13k7IR6RrgdIXmqBgw1PnD01bvhuesz2pqJRkXbMH riX6BcOA50cmPyIguthEcQxkatYmRS5oHagA= X-Received: by 2002:a92:cdad:0:b0:2c6:7b76:a086 with SMTP id g13-20020a92cdad000000b002c67b76a086mr2568128ild.5.1649185846540; Tue, 05 Apr 2022 12:10:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxocs2ZVaXF5YVJwjGBcLRqnK5UBIr3mQlWSQgNmkFdgGBC2rjlpWZLrLl9AI2wrgBUaUJw1w== X-Received: by 2002:a92:cdad:0:b0:2c6:7b76:a086 with SMTP id g13-20020a92cdad000000b002c67b76a086mr2568112ild.5.1649185846282; Tue, 05 Apr 2022 12:10:46 -0700 (PDT) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id x2-20020a056602160200b006463c1977f9sm9314961iow.22.2022.04.05.12.10.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Apr 2022 12:10:45 -0700 (PDT) Date: Tue, 5 Apr 2022 13:10:44 -0600 From: Alex Williamson To: Jason Gunthorpe Cc: Lu Baolu , Christian Benvenuti , Cornelia Huck , David Woodhouse , Gerald Schaefer , iommu@lists.linux-foundation.org, Jason Wang , Joerg Roedel , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arm-msm@vger.kernel.org, linux-rdma@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , "Michael S. Tsirkin" , Nelson Escobar , netdev@vger.kernel.org, Rob Clark , Robin Murphy , Suravee Suthikulpanit , virtualization@lists.linux-foundation.org, Will Deacon , Christoph Hellwig , "Tian, Kevin" Subject: Re: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Message-ID: <20220405131044.23910b77.alex.williamson@redhat.com> In-Reply-To: <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> References: <0-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> Organization: Red Hat MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=alex.williamson@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_121055_084614_D217326A X-CRM114-Status: GOOD ( 22.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 5 Apr 2022 13:16:01 -0300 Jason Gunthorpe wrote: > dev_is_dma_coherent() is the control to determine if IOMMU_CACHE can be > supported. > > IOMMU_CACHE means that normal DMAs do not require any additional coherency > mechanism and is the basic uAPI that VFIO exposes to userspace. For > instance VFIO applications like DPDK will not work if additional coherency > operations are required. > > Therefore check dev_is_dma_coherent() before allowing a device to join a > domain. This will block device/platform/iommu combinations from using VFIO > that do not support cache coherent DMA. > > Signed-off-by: Jason Gunthorpe > --- > drivers/vfio/vfio.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > index a4555014bd1e72..2a3aa3e742d943 100644 > --- a/drivers/vfio/vfio.c > +++ b/drivers/vfio/vfio.c > @@ -32,6 +32,7 @@ > #include > #include > #include > +#include > #include "vfio.h" > > #define DRIVER_VERSION "0.3" > @@ -1348,6 +1349,11 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) > if (IS_ERR(device)) > return PTR_ERR(device); > > + if (group->type == VFIO_IOMMU && !dev_is_dma_coherent(device->dev)) { > + ret = -ENODEV; > + goto err_device_put; > + } > + Failing at the point where the user is trying to gain access to the device seems a little late in the process and opaque, wouldn't we rather have vfio bus drivers fail to probe such devices? I'd expect this to occur in the vfio_register_group_dev() path. Thanks, Alex > if (!try_module_get(device->dev->driver->owner)) { > ret = -ENODEV; > goto err_device_put; _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2716BC4332F for ; Tue, 5 Apr 2022 23:26:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1451005AbiDEXGf (ORCPT ); Tue, 5 Apr 2022 19:06:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1573495AbiDETMt (ORCPT ); Tue, 5 Apr 2022 15:12:49 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5C4FBE8863 for ; Tue, 5 Apr 2022 12:10:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649185849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ufYYfNJ31ZxJiptHxZVI0cJDY245GxkfqBFvOLAxMek=; b=KYU3KkTRmZuyeN3fh2VBrfDUF92xda083Jz/LEHXz/N/9S3KNY8T6oKF9Cwl+EGCY2bMjM wtqV02mVMI7E8XvQRM/EE0PxQgjL2UAKgIa7hzpazB5Z+/GFlcS23Z+fjj+XOrL7Z/KgAS 62T5WvawxCNmiXwodAUQNm1IxAqySmU= Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-13-iaURa4LtOFWV7i6LEnFaqw-1; Tue, 05 Apr 2022 15:10:48 -0400 X-MC-Unique: iaURa4LtOFWV7i6LEnFaqw-1 Received: by mail-io1-f70.google.com with SMTP id f11-20020a056602070b00b00645d08010fcso120544iox.15 for ; Tue, 05 Apr 2022 12:10:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=ufYYfNJ31ZxJiptHxZVI0cJDY245GxkfqBFvOLAxMek=; b=tTNA0w6LpMWjWcE3pnQT51ZKWDJsBpUXkTqqcaRzVxXDR6BRfUsMxsAP6yH9hlqr+c VqvZCSdz+aL77nAD0dfHJtbckyCJq6FxYMFoGvyTzeT/MBxoXW67qBz+r8yOJZnYpqk6 cUO1tIXG2ISHQc1l9+/Ca7im2qCWXOewn54WwgvODd28wRsaWLWtinkxlqjT7T8KHGHt b8CA3X4iTiC0hP0pgj7dfNv/6p+SxF4bOe0VKFPy9/4lLbktkkOhdE+7IfbVAFncrZlo koAhEBP6DnXmbh0kwADP05UTjBFUIAEmsZwOZ8AVyAqwfnWOO11zEdDwUo8yct5cohqR reHA== X-Gm-Message-State: AOAM530ucJeySNeSK2jFL4av2OZgdf5FJPQ/AI/7VdPLeB/YZzPh/wn5 2omsJJEIrhKfAnm/I2wsv6NYifeqw09kXZ5MeP0mzcYyy92zCpJoVc5rnD/UXoVg0QdrWHcmJhl yBv+0Bw0my23iBuGLZv+PSBpC9w== X-Received: by 2002:a92:cdad:0:b0:2c6:7b76:a086 with SMTP id g13-20020a92cdad000000b002c67b76a086mr2568137ild.5.1649185846542; Tue, 05 Apr 2022 12:10:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxocs2ZVaXF5YVJwjGBcLRqnK5UBIr3mQlWSQgNmkFdgGBC2rjlpWZLrLl9AI2wrgBUaUJw1w== X-Received: by 2002:a92:cdad:0:b0:2c6:7b76:a086 with SMTP id g13-20020a92cdad000000b002c67b76a086mr2568112ild.5.1649185846282; Tue, 05 Apr 2022 12:10:46 -0700 (PDT) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id x2-20020a056602160200b006463c1977f9sm9314961iow.22.2022.04.05.12.10.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Apr 2022 12:10:45 -0700 (PDT) Date: Tue, 5 Apr 2022 13:10:44 -0600 From: Alex Williamson To: Jason Gunthorpe Cc: Lu Baolu , Christian Benvenuti , Cornelia Huck , David Woodhouse , Gerald Schaefer , iommu@lists.linux-foundation.org, Jason Wang , Joerg Roedel , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arm-msm@vger.kernel.org, linux-rdma@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , "Michael S. Tsirkin" , Nelson Escobar , netdev@vger.kernel.org, Rob Clark , Robin Murphy , Suravee Suthikulpanit , virtualization@lists.linux-foundation.org, Will Deacon , Christoph Hellwig , "Tian, Kevin" Subject: Re: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Message-ID: <20220405131044.23910b77.alex.williamson@redhat.com> In-Reply-To: <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> References: <0-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> Organization: Red Hat MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Tue, 5 Apr 2022 13:16:01 -0300 Jason Gunthorpe wrote: > dev_is_dma_coherent() is the control to determine if IOMMU_CACHE can be > supported. > > IOMMU_CACHE means that normal DMAs do not require any additional coherency > mechanism and is the basic uAPI that VFIO exposes to userspace. For > instance VFIO applications like DPDK will not work if additional coherency > operations are required. > > Therefore check dev_is_dma_coherent() before allowing a device to join a > domain. This will block device/platform/iommu combinations from using VFIO > that do not support cache coherent DMA. > > Signed-off-by: Jason Gunthorpe > --- > drivers/vfio/vfio.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > index a4555014bd1e72..2a3aa3e742d943 100644 > --- a/drivers/vfio/vfio.c > +++ b/drivers/vfio/vfio.c > @@ -32,6 +32,7 @@ > #include > #include > #include > +#include > #include "vfio.h" > > #define DRIVER_VERSION "0.3" > @@ -1348,6 +1349,11 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) > if (IS_ERR(device)) > return PTR_ERR(device); > > + if (group->type == VFIO_IOMMU && !dev_is_dma_coherent(device->dev)) { > + ret = -ENODEV; > + goto err_device_put; > + } > + Failing at the point where the user is trying to gain access to the device seems a little late in the process and opaque, wouldn't we rather have vfio bus drivers fail to probe such devices? I'd expect this to occur in the vfio_register_group_dev() path. Thanks, Alex > if (!try_module_get(device->dev->driver->owner)) { > ret = -ENODEV; > goto err_device_put;