From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 52D54C433F5 for ; Wed, 6 Apr 2022 07:03:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id F0A8340361; Wed, 6 Apr 2022 07:03:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HDP_2fV_9rkN; Wed, 6 Apr 2022 07:03:00 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 630F74004F; Wed, 6 Apr 2022 07:03:00 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A5E91C0084; Wed, 6 Apr 2022 07:02:59 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7C89FC0012; Wed, 6 Apr 2022 07:02:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 648D6417BF; Wed, 6 Apr 2022 07:02:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kLpNSO6Q2mIQ; Wed, 6 Apr 2022 07:02:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp4.osuosl.org (Postfix) with ESMTPS id 14934417BE; Wed, 6 Apr 2022 07:02:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649228577; x=1680764577; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=9yXnWPFaJPcfSAhgXMQM40zFYivmgfvuHZVyoADMm+c=; b=dFBg8ot5f6UeudbfGh0hVryuENz8e/s7yn0YN+l/ReHmRuGvvLYOqMDu ICmdexfQIdg+TmnJ1GaCbEzyQkm/VnObck0jcJqQSuQE7y+MCNzG3oqE3 kO+yHRpYC/PBiWeyL1bleStfzSprP6n29unYz46MYLd81M7tt29HrpfrW eexuT9N9WU5orqyUQqAwpNArXCA67woAvVbJSqirDglIwYoN0l96hoBdi fNJdwtVCR2pJNtXctWFLZmLN7k1C+Px/F33ZnT5zVmsIxMXgUVIexmXL2 LCOtjsHbHGKQYfQlUIT8nbWn5jVu0V5mleUsfzvDz0LqvkOX2GoBtP86N g==; X-IronPort-AV: E=McAfee;i="6200,9189,10308"; a="240902648" X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="240902648" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Apr 2022 00:02:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="549419306" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga007.jf.intel.com with ESMTP; 06 Apr 2022 00:02:38 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Wed, 6 Apr 2022 00:02:37 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.175) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eBsP97RyNVSYXfzW8F4rHWni1sG/h3uEbWD6PLNUBSygD18SAKz/2QGaWvwkWKOJvIuxtC0TlRXQpImmoftz/2uSqVx3+Ehq8gxCVS6Bf314AcShdPb7pYTcSpmQcbfv3tMnkZ3Xr8f0Qqlc4td0mx6k/JeRDXojsBJz4NDEH3H0NJF0LI2CjMRA14a8ZFLXxHb/3SiyELxUuZVL7S7xq1W/ARyqWTNvXZhKbuVB9su3DqB8Gt9imp80pIbelQDu2HBfhclZTSVEFQ95Gh+uX8IHaDguzDytGDjbyiZKWbDnKY1/F58jeU9hO+TYr7k/jrvRsU3m1ZDF0Ihlm1s0GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MpZ5rpjZ/323Q4FiP3H6XfuAq/WWbmz2u2b2ATpZRiM=; b=I7SefwtHVKGK5w/dzLMRezmQMPcMjak4bDmfbhQOrh9owDU7qceXEvhUx7QP6vTIEl7X3E6wF9kehI3JoQY+9MqI3obUP2+1Ar7gotsixxCrtDuQQdVs055tSrUUgmXSkXeJJ5i9F56+fHIRR4CPi0e+xZF0gIdkjYA6POen9mE23p51vhYiMEcm9qcohDdqIunLz3cF3ev7a/0Sh912/q61+IaM/AK48mtwGcV+8Aci8YqFf5c4LHunoZ6KeDI2W5zqdi6LD5bNC6orRhoSa561uibmqvX0VLUoXnNkyrvTUNl8cozuRWCwqmHO6InLXmo9ebT7NxfMjOSLQb9NJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5276.namprd11.prod.outlook.com (2603:10b6:408:135::18) by PH7PR11MB5885.namprd11.prod.outlook.com (2603:10b6:510:134::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.30; Wed, 6 Apr 2022 07:02:36 +0000 Received: from BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::c4ea:a404:b70b:e54e]) by BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::c4ea:a404:b70b:e54e%7]) with mapi id 15.20.5123.031; Wed, 6 Apr 2022 07:02:36 +0000 From: "Tian, Kevin" To: Jason Gunthorpe , Alex Williamson Subject: RE: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Thread-Topic: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Thread-Index: AQHYSQh+iGQB4aLBFUu6szRllIBFLqzhr6EAgAAFLgCAAMFI0A== Date: Wed, 6 Apr 2022 07:02:36 +0000 Message-ID: References: <0-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <20220405131044.23910b77.alex.williamson@redhat.com> <20220405192916.GT2120790@nvidia.com> In-Reply-To: <20220405192916.GT2120790@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dd1d5368-9447-465b-35fd-08da179b6e30 x-ms-traffictypediagnostic: PH7PR11MB5885:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5Azv5BpEPVMypZ7ZE/68AxddWT9AhaSUULiUIPoVJ43Z/0MKytwAyzwrT8Yrxt5Dp0ZYkFzELg78LbjkYaHMquOq53uKZrUJy+lmTswDcMMmBN5RrXKckYsztRnkdaa/VZWuA+AwRKlgJlFg1ZIDDBLzT0dmUh2RcfBdPFoica0qEeg7hw8dBd0U4jeKHam3gIt1uL0tCJGFvb0lFhehH3qtC83T37r9P8Xokv4xKWU+AwTiKgHf1ft+fi/UNr/U1jPN6Yi7tD+7puHOzL8G/tmS0AbO71A/SVoL1Iew7gA4YaHECbP2FWe9HguZj6UnANprG74YMjjCFcPrr+9NaqKA/qPmwSFDN9QiE2TnHz5ZL0Tr3/HQsaU8fypcKGHtyhLj1LZGavzopgxXAt/RhVtgF2KbJp/m1hZ0xe1IEeSUUc+KNHeMLhjEVjy/7+zSnSVZeskLgwv9e2J/HPH/5rPtcArFwPJDSU+SyeVywNmVYC+tlRUCAD099uQu2LsPLwDEADO60oJTmGLKGKZ+YfdoUho5PgF2aY0Rk3XabBN5HHmPte5iLF76xk5yLYe6C0oEARuyxp2jRv3nuQ33cQdhuXd7bk3btMP60nY2D5i+GqBWYDVTu0KdKJxnsmuA3AlQWBU8ewvOV+wvZd9iGF8n3e8kjFAcIZdrTjy6vShaVMk46DF/w26JYLkrj7TGmkuaV4jyiZINYqWdrhmZIA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR11MB5276.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(316002)(8676002)(71200400001)(66446008)(4326008)(52536014)(66556008)(6506007)(66946007)(110136005)(66476007)(7696005)(54906003)(508600001)(55016003)(76116006)(8936002)(64756008)(7416002)(5660300002)(33656002)(86362001)(9686003)(186003)(26005)(83380400001)(38070700005)(2906002)(38100700002)(82960400001)(122000001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Gmd1F0kaSG5OHjvaZPLWx6NVaVyull7bel9d013fcaw/eZO66M3Nlk0keRtl?= =?us-ascii?Q?qXKs/c15VLZfQqrjKDz+6NRbKj/s4SC4rGViNbMb+NDxKmd8XfE3JFF2R5WY?= =?us-ascii?Q?n0nv4jZg9OFH6soZCsHJDj40gZMxcgWHivfd06apqJE8je4LmIrmlQJ50NER?= =?us-ascii?Q?iNheOoO4UGVfbz2onI8JcCEWyD3gIlHk1PTjMNb3hBMnTsCb9h3Y1kmeLoXW?= =?us-ascii?Q?W8bsQWFHAf2CR5QP4ru9aqvIhhy7+O8QfqGmv2KFvY/Sa8i6PFyIP2XcrZAn?= =?us-ascii?Q?5z7pe3JsIzs6UOOeGWVHdj3nCS0tby7si/rIKV75+ykVgXKL1bQwbMBI/MlI?= =?us-ascii?Q?JSqLhRJmRRwbBHlwFYdRCXI0jr83C25hmhNAGDNf5h0MZ2tTCF2k71l5bGWK?= =?us-ascii?Q?HCbAMCMpMBNnZAulkP3reD46EIRZ3dvHVB/5ZJtKkkUyoTxDzS/ov72nVDhK?= =?us-ascii?Q?2bpHACHXYHosAZGAsGyPboQLRmEzgwRZjsHRNFy2YJSI1YwZXYyuDa90MXqA?= =?us-ascii?Q?Y5mnSY79Hz8eZjZJa0KkSxZD0pgwieACaVwmuPPemBYl8y+oQsQU1FirHxCX?= =?us-ascii?Q?r+qkTzDgJWTjTEiZCQEmOK/rXxpNImDo3L+1oXW55Plrl2bwooUwjL0XgIG1?= =?us-ascii?Q?VjZvEsHaAd2J/s7Esb3cItMM6C2cwnJoCc3VqbGD9F3G3fdGToopNpMcZ3Gi?= =?us-ascii?Q?h/HnFNTktk/hoEy5fuL3rpZFbcCxEO8CNXhZX19IVoYuHI4JnCd3bwpTq04d?= =?us-ascii?Q?izOz6txSbXP/mv0nKwZlJJTqizYnGvh8/2rYx0OQdQZ4dvWMNZxj1dxsG1P5?= =?us-ascii?Q?cNiET/y45qWppgv+SQtmwiB+lgVP2JA8Xb/aJ7BDrDPpL9n2KZ1HUE4mMtoH?= =?us-ascii?Q?dlqg+TJCmtEsp1+Qu01XC7lbMLGTiYM5jXJH6dpPeY1opkE7KYm69HzJiH53?= =?us-ascii?Q?BpntXw4Xk12maVf5vCLMvSLKxso5p8u25XMOEt+2/SYGXq5OQfbAbAGVeGGA?= =?us-ascii?Q?RhEtSFIaE1c9Tc+efkqOVepP6Mx73Fs4Jv0mfHT7kTm9JWRlJVoaQuJ88nm+?= =?us-ascii?Q?HotDkb48k5pfBQFSWVKiWoKOBcp8fpuAR8GYKp85RrPaaBY02r4pMlln7dF+?= =?us-ascii?Q?dBYR1QPq7T66ZezYi1vT69vW0O3iJQyXEjFn+9f+RdATD1YtqwAfjYq/VSkp?= =?us-ascii?Q?HcRqId623/McuWWdEpoUgBrDaQG5YqHsgOLdPevAybSYNFuBK1P9c2T/6uVw?= =?us-ascii?Q?1e3J1hBExHlF6giKAJKQB+rfEXsKvRXXmNIpeGkcbOOpyjoipkef8lQIAnKv?= =?us-ascii?Q?/9cCV8eIXsvhthRjwQI1VY1LAs5jBhjwiMwYwO1vt2UesnLbubAR245RnSMY?= =?us-ascii?Q?jj8ZX4ZxYLod5yLxFgEWVeBZo7NyH9sLp3EAmjWKkzlC3fFaSpK/JudmfxFz?= =?us-ascii?Q?KN1v6PhPUxTtOow9q0i4A7Iu/lJnN6dYvlqFYqgxK2xM11g6XghiIp+VzqfS?= =?us-ascii?Q?1aYELPMwWCcKAtFpurpawVpA0KVaONCKLph89gy9ndocCfOxTrZKOqJy/NY5?= =?us-ascii?Q?fHlvn6vL5ejF6q12tXcVcD9+MuTy4QN/iqfJCgxzLtplTW0k/lsgbk2CQoxG?= =?us-ascii?Q?I8YaJ1mgTJgWBE7LfMvHMhKMxdpXcB1uqjK/AQpZfDvlC+8ryk53WArLR8tV?= =?us-ascii?Q?SKsaJJO4PHb4mDVcpWdb9FYN4t8iQENqWvya8oejN3H0SagMFpS48o7TXnWK?= =?us-ascii?Q?oDDpdNwP7g=3D=3D?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd1d5368-9447-465b-35fd-08da179b6e30 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2022 07:02:36.1106 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bnIj9JeokweneDmdbZpx8OX+uXUvq0hi4EJf212dVk9h+hRopZ0qPTT8ZZAqihWB/l8gdLrsiJoZgLSeSfMKIA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB5885 X-OriginatorOrg: intel.com Cc: Nelson Escobar , "Michael S. Tsirkin" , Jason Wang , "virtualization@lists.linux-foundation.org" , Will Deacon , Christoph Hellwig , "linux-s390@vger.kernel.org" , "kvm@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "iommu@lists.linux-foundation.org" , Gerald Schaefer , David Woodhouse , "linux-arm-msm@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "netdev@vger.kernel.org" , Cornelia Huck , Christian Benvenuti , Robin Murphy X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" > From: Jason Gunthorpe > Sent: Wednesday, April 6, 2022 3:29 AM > > On Tue, Apr 05, 2022 at 01:10:44PM -0600, Alex Williamson wrote: > > On Tue, 5 Apr 2022 13:16:01 -0300 > > Jason Gunthorpe wrote: > > > > > dev_is_dma_coherent() is the control to determine if IOMMU_CACHE can > be > > > supported. > > > > > > IOMMU_CACHE means that normal DMAs do not require any additional > coherency > > > mechanism and is the basic uAPI that VFIO exposes to userspace. For > > > instance VFIO applications like DPDK will not work if additional coherency > > > operations are required. > > > > > > Therefore check dev_is_dma_coherent() before allowing a device to join > a > > > domain. This will block device/platform/iommu combinations from using > VFIO > > > that do not support cache coherent DMA. > > > > > > Signed-off-by: Jason Gunthorpe > > > drivers/vfio/vfio.c | 6 ++++++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > > > index a4555014bd1e72..2a3aa3e742d943 100644 > > > +++ b/drivers/vfio/vfio.c > > > @@ -32,6 +32,7 @@ > > > #include > > > #include > > > #include > > > +#include > > > #include "vfio.h" > > > > > > #define DRIVER_VERSION "0.3" > > > @@ -1348,6 +1349,11 @@ static int vfio_group_get_device_fd(struct > vfio_group *group, char *buf) > > > if (IS_ERR(device)) > > > return PTR_ERR(device); > > > > > > + if (group->type == VFIO_IOMMU && !dev_is_dma_coherent(device- > >dev)) { > > > + ret = -ENODEV; > > > + goto err_device_put; > > > + } > > > + > > > > Failing at the point where the user is trying to gain access to the > > device seems a little late in the process and opaque, wouldn't we > > rather have vfio bus drivers fail to probe such devices? I'd expect > > this to occur in the vfio_register_group_dev() path. Thanks, > > Yes, that is a good point. > > So like this: > > int vfio_register_group_dev(struct vfio_device *device) > { > + if (!dev_is_dma_coherent(device->dev)) > + return -EINVAL; > + > return __vfio_register_dev(device, > vfio_group_find_or_alloc(device->dev)); > } > > I fixed it up. > if that is the case should it also apply to usnic and vdpa in the first patch (i.e. fail the probe)? _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71162C433EF for ; Wed, 6 Apr 2022 07:03:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 16AD581AD7; Wed, 6 Apr 2022 07:03:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CaZbjTxERzz7; Wed, 6 Apr 2022 07:03:00 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 8BFCD819BE; Wed, 6 Apr 2022 07:02:59 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 542BBC002C; Wed, 6 Apr 2022 07:02:59 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7C89FC0012; Wed, 6 Apr 2022 07:02:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 648D6417BF; Wed, 6 Apr 2022 07:02:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kLpNSO6Q2mIQ; Wed, 6 Apr 2022 07:02:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp4.osuosl.org (Postfix) with ESMTPS id 14934417BE; Wed, 6 Apr 2022 07:02:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649228577; x=1680764577; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=9yXnWPFaJPcfSAhgXMQM40zFYivmgfvuHZVyoADMm+c=; b=dFBg8ot5f6UeudbfGh0hVryuENz8e/s7yn0YN+l/ReHmRuGvvLYOqMDu ICmdexfQIdg+TmnJ1GaCbEzyQkm/VnObck0jcJqQSuQE7y+MCNzG3oqE3 kO+yHRpYC/PBiWeyL1bleStfzSprP6n29unYz46MYLd81M7tt29HrpfrW eexuT9N9WU5orqyUQqAwpNArXCA67woAvVbJSqirDglIwYoN0l96hoBdi fNJdwtVCR2pJNtXctWFLZmLN7k1C+Px/F33ZnT5zVmsIxMXgUVIexmXL2 LCOtjsHbHGKQYfQlUIT8nbWn5jVu0V5mleUsfzvDz0LqvkOX2GoBtP86N g==; X-IronPort-AV: E=McAfee;i="6200,9189,10308"; a="240902648" X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="240902648" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Apr 2022 00:02:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="549419306" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga007.jf.intel.com with ESMTP; 06 Apr 2022 00:02:38 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Wed, 6 Apr 2022 00:02:37 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.175) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eBsP97RyNVSYXfzW8F4rHWni1sG/h3uEbWD6PLNUBSygD18SAKz/2QGaWvwkWKOJvIuxtC0TlRXQpImmoftz/2uSqVx3+Ehq8gxCVS6Bf314AcShdPb7pYTcSpmQcbfv3tMnkZ3Xr8f0Qqlc4td0mx6k/JeRDXojsBJz4NDEH3H0NJF0LI2CjMRA14a8ZFLXxHb/3SiyELxUuZVL7S7xq1W/ARyqWTNvXZhKbuVB9su3DqB8Gt9imp80pIbelQDu2HBfhclZTSVEFQ95Gh+uX8IHaDguzDytGDjbyiZKWbDnKY1/F58jeU9hO+TYr7k/jrvRsU3m1ZDF0Ihlm1s0GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MpZ5rpjZ/323Q4FiP3H6XfuAq/WWbmz2u2b2ATpZRiM=; b=I7SefwtHVKGK5w/dzLMRezmQMPcMjak4bDmfbhQOrh9owDU7qceXEvhUx7QP6vTIEl7X3E6wF9kehI3JoQY+9MqI3obUP2+1Ar7gotsixxCrtDuQQdVs055tSrUUgmXSkXeJJ5i9F56+fHIRR4CPi0e+xZF0gIdkjYA6POen9mE23p51vhYiMEcm9qcohDdqIunLz3cF3ev7a/0Sh912/q61+IaM/AK48mtwGcV+8Aci8YqFf5c4LHunoZ6KeDI2W5zqdi6LD5bNC6orRhoSa561uibmqvX0VLUoXnNkyrvTUNl8cozuRWCwqmHO6InLXmo9ebT7NxfMjOSLQb9NJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5276.namprd11.prod.outlook.com (2603:10b6:408:135::18) by PH7PR11MB5885.namprd11.prod.outlook.com (2603:10b6:510:134::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.30; Wed, 6 Apr 2022 07:02:36 +0000 Received: from BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::c4ea:a404:b70b:e54e]) by BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::c4ea:a404:b70b:e54e%7]) with mapi id 15.20.5123.031; Wed, 6 Apr 2022 07:02:36 +0000 From: "Tian, Kevin" To: Jason Gunthorpe , Alex Williamson Subject: RE: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Thread-Topic: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Thread-Index: AQHYSQh+iGQB4aLBFUu6szRllIBFLqzhr6EAgAAFLgCAAMFI0A== Date: Wed, 6 Apr 2022 07:02:36 +0000 Message-ID: References: <0-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <20220405131044.23910b77.alex.williamson@redhat.com> <20220405192916.GT2120790@nvidia.com> In-Reply-To: <20220405192916.GT2120790@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dd1d5368-9447-465b-35fd-08da179b6e30 x-ms-traffictypediagnostic: PH7PR11MB5885:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5Azv5BpEPVMypZ7ZE/68AxddWT9AhaSUULiUIPoVJ43Z/0MKytwAyzwrT8Yrxt5Dp0ZYkFzELg78LbjkYaHMquOq53uKZrUJy+lmTswDcMMmBN5RrXKckYsztRnkdaa/VZWuA+AwRKlgJlFg1ZIDDBLzT0dmUh2RcfBdPFoica0qEeg7hw8dBd0U4jeKHam3gIt1uL0tCJGFvb0lFhehH3qtC83T37r9P8Xokv4xKWU+AwTiKgHf1ft+fi/UNr/U1jPN6Yi7tD+7puHOzL8G/tmS0AbO71A/SVoL1Iew7gA4YaHECbP2FWe9HguZj6UnANprG74YMjjCFcPrr+9NaqKA/qPmwSFDN9QiE2TnHz5ZL0Tr3/HQsaU8fypcKGHtyhLj1LZGavzopgxXAt/RhVtgF2KbJp/m1hZ0xe1IEeSUUc+KNHeMLhjEVjy/7+zSnSVZeskLgwv9e2J/HPH/5rPtcArFwPJDSU+SyeVywNmVYC+tlRUCAD099uQu2LsPLwDEADO60oJTmGLKGKZ+YfdoUho5PgF2aY0Rk3XabBN5HHmPte5iLF76xk5yLYe6C0oEARuyxp2jRv3nuQ33cQdhuXd7bk3btMP60nY2D5i+GqBWYDVTu0KdKJxnsmuA3AlQWBU8ewvOV+wvZd9iGF8n3e8kjFAcIZdrTjy6vShaVMk46DF/w26JYLkrj7TGmkuaV4jyiZINYqWdrhmZIA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR11MB5276.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(316002)(8676002)(71200400001)(66446008)(4326008)(52536014)(66556008)(6506007)(66946007)(110136005)(66476007)(7696005)(54906003)(508600001)(55016003)(76116006)(8936002)(64756008)(7416002)(5660300002)(33656002)(86362001)(9686003)(186003)(26005)(83380400001)(38070700005)(2906002)(38100700002)(82960400001)(122000001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Gmd1F0kaSG5OHjvaZPLWx6NVaVyull7bel9d013fcaw/eZO66M3Nlk0keRtl?= =?us-ascii?Q?qXKs/c15VLZfQqrjKDz+6NRbKj/s4SC4rGViNbMb+NDxKmd8XfE3JFF2R5WY?= =?us-ascii?Q?n0nv4jZg9OFH6soZCsHJDj40gZMxcgWHivfd06apqJE8je4LmIrmlQJ50NER?= =?us-ascii?Q?iNheOoO4UGVfbz2onI8JcCEWyD3gIlHk1PTjMNb3hBMnTsCb9h3Y1kmeLoXW?= =?us-ascii?Q?W8bsQWFHAf2CR5QP4ru9aqvIhhy7+O8QfqGmv2KFvY/Sa8i6PFyIP2XcrZAn?= =?us-ascii?Q?5z7pe3JsIzs6UOOeGWVHdj3nCS0tby7si/rIKV75+ykVgXKL1bQwbMBI/MlI?= =?us-ascii?Q?JSqLhRJmRRwbBHlwFYdRCXI0jr83C25hmhNAGDNf5h0MZ2tTCF2k71l5bGWK?= =?us-ascii?Q?HCbAMCMpMBNnZAulkP3reD46EIRZ3dvHVB/5ZJtKkkUyoTxDzS/ov72nVDhK?= =?us-ascii?Q?2bpHACHXYHosAZGAsGyPboQLRmEzgwRZjsHRNFy2YJSI1YwZXYyuDa90MXqA?= =?us-ascii?Q?Y5mnSY79Hz8eZjZJa0KkSxZD0pgwieACaVwmuPPemBYl8y+oQsQU1FirHxCX?= =?us-ascii?Q?r+qkTzDgJWTjTEiZCQEmOK/rXxpNImDo3L+1oXW55Plrl2bwooUwjL0XgIG1?= =?us-ascii?Q?VjZvEsHaAd2J/s7Esb3cItMM6C2cwnJoCc3VqbGD9F3G3fdGToopNpMcZ3Gi?= =?us-ascii?Q?h/HnFNTktk/hoEy5fuL3rpZFbcCxEO8CNXhZX19IVoYuHI4JnCd3bwpTq04d?= =?us-ascii?Q?izOz6txSbXP/mv0nKwZlJJTqizYnGvh8/2rYx0OQdQZ4dvWMNZxj1dxsG1P5?= =?us-ascii?Q?cNiET/y45qWppgv+SQtmwiB+lgVP2JA8Xb/aJ7BDrDPpL9n2KZ1HUE4mMtoH?= =?us-ascii?Q?dlqg+TJCmtEsp1+Qu01XC7lbMLGTiYM5jXJH6dpPeY1opkE7KYm69HzJiH53?= =?us-ascii?Q?BpntXw4Xk12maVf5vCLMvSLKxso5p8u25XMOEt+2/SYGXq5OQfbAbAGVeGGA?= =?us-ascii?Q?RhEtSFIaE1c9Tc+efkqOVepP6Mx73Fs4Jv0mfHT7kTm9JWRlJVoaQuJ88nm+?= =?us-ascii?Q?HotDkb48k5pfBQFSWVKiWoKOBcp8fpuAR8GYKp85RrPaaBY02r4pMlln7dF+?= =?us-ascii?Q?dBYR1QPq7T66ZezYi1vT69vW0O3iJQyXEjFn+9f+RdATD1YtqwAfjYq/VSkp?= =?us-ascii?Q?HcRqId623/McuWWdEpoUgBrDaQG5YqHsgOLdPevAybSYNFuBK1P9c2T/6uVw?= =?us-ascii?Q?1e3J1hBExHlF6giKAJKQB+rfEXsKvRXXmNIpeGkcbOOpyjoipkef8lQIAnKv?= =?us-ascii?Q?/9cCV8eIXsvhthRjwQI1VY1LAs5jBhjwiMwYwO1vt2UesnLbubAR245RnSMY?= =?us-ascii?Q?jj8ZX4ZxYLod5yLxFgEWVeBZo7NyH9sLp3EAmjWKkzlC3fFaSpK/JudmfxFz?= =?us-ascii?Q?KN1v6PhPUxTtOow9q0i4A7Iu/lJnN6dYvlqFYqgxK2xM11g6XghiIp+VzqfS?= =?us-ascii?Q?1aYELPMwWCcKAtFpurpawVpA0KVaONCKLph89gy9ndocCfOxTrZKOqJy/NY5?= =?us-ascii?Q?fHlvn6vL5ejF6q12tXcVcD9+MuTy4QN/iqfJCgxzLtplTW0k/lsgbk2CQoxG?= =?us-ascii?Q?I8YaJ1mgTJgWBE7LfMvHMhKMxdpXcB1uqjK/AQpZfDvlC+8ryk53WArLR8tV?= =?us-ascii?Q?SKsaJJO4PHb4mDVcpWdb9FYN4t8iQENqWvya8oejN3H0SagMFpS48o7TXnWK?= =?us-ascii?Q?oDDpdNwP7g=3D=3D?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd1d5368-9447-465b-35fd-08da179b6e30 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2022 07:02:36.1106 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bnIj9JeokweneDmdbZpx8OX+uXUvq0hi4EJf212dVk9h+hRopZ0qPTT8ZZAqihWB/l8gdLrsiJoZgLSeSfMKIA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB5885 X-OriginatorOrg: intel.com Cc: Nelson Escobar , Matthew Rosato , "Michael S. Tsirkin" , "virtualization@lists.linux-foundation.org" , Will Deacon , Christoph Hellwig , "linux-s390@vger.kernel.org" , "kvm@vger.kernel.org" , "linux-rdma@vger.kernel.org" , Joerg Roedel , "iommu@lists.linux-foundation.org" , Gerald Schaefer , David Woodhouse , "linux-arm-msm@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "netdev@vger.kernel.org" , Cornelia Huck , Rob Clark , Suravee Suthikulpanit , Christian Benvenuti , Robin Murphy , Lu Baolu X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" > From: Jason Gunthorpe > Sent: Wednesday, April 6, 2022 3:29 AM > > On Tue, Apr 05, 2022 at 01:10:44PM -0600, Alex Williamson wrote: > > On Tue, 5 Apr 2022 13:16:01 -0300 > > Jason Gunthorpe wrote: > > > > > dev_is_dma_coherent() is the control to determine if IOMMU_CACHE can > be > > > supported. > > > > > > IOMMU_CACHE means that normal DMAs do not require any additional > coherency > > > mechanism and is the basic uAPI that VFIO exposes to userspace. For > > > instance VFIO applications like DPDK will not work if additional coherency > > > operations are required. > > > > > > Therefore check dev_is_dma_coherent() before allowing a device to join > a > > > domain. This will block device/platform/iommu combinations from using > VFIO > > > that do not support cache coherent DMA. > > > > > > Signed-off-by: Jason Gunthorpe > > > drivers/vfio/vfio.c | 6 ++++++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > > > index a4555014bd1e72..2a3aa3e742d943 100644 > > > +++ b/drivers/vfio/vfio.c > > > @@ -32,6 +32,7 @@ > > > #include > > > #include > > > #include > > > +#include > > > #include "vfio.h" > > > > > > #define DRIVER_VERSION "0.3" > > > @@ -1348,6 +1349,11 @@ static int vfio_group_get_device_fd(struct > vfio_group *group, char *buf) > > > if (IS_ERR(device)) > > > return PTR_ERR(device); > > > > > > + if (group->type == VFIO_IOMMU && !dev_is_dma_coherent(device- > >dev)) { > > > + ret = -ENODEV; > > > + goto err_device_put; > > > + } > > > + > > > > Failing at the point where the user is trying to gain access to the > > device seems a little late in the process and opaque, wouldn't we > > rather have vfio bus drivers fail to probe such devices? I'd expect > > this to occur in the vfio_register_group_dev() path. Thanks, > > Yes, that is a good point. > > So like this: > > int vfio_register_group_dev(struct vfio_device *device) > { > + if (!dev_is_dma_coherent(device->dev)) > + return -EINVAL; > + > return __vfio_register_dev(device, > vfio_group_find_or_alloc(device->dev)); > } > > I fixed it up. > if that is the case should it also apply to usnic and vdpa in the first patch (i.e. fail the probe)? _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D511C433FE for ; Wed, 6 Apr 2022 07:04:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QQOxyONTlU/PSHD34MNPlP76/32iENgPXAAKt6NSRNc=; b=YuaD9E7ySpCK/1 I/THFIzl6NIPFSsVsC6kKDXXi4MgcbXgCP+5u8cYdlE+iV8GssqJ/k7gQN2l+/CQZ4hj8Fy88avAi 3+C7fyPSxlxUI3oQiTe2Pc5RZOJ61mxJwEpqfAXgNQNH4jp7zZ9hzYtOFP1bHeycCgBRjjEj83qF3 iZSefhsjPbSyF4sp8jLQNzdIQXDaTHpWAi81WnbJO8Fj+9EwBrvrg+l7EhQs5M1X8jSHRy0tuf1Cc c5NVOBLZjWIZuws1fO3YI6hbGGoZdvedlkM5SJ2T6a/g2AtI/C/uqDCU8QxIR0XzRlap9yoSvzfbq 5AYJHzUeTL7TV5r2QF8g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbzh7-004KXO-AL; Wed, 06 Apr 2022 07:03:09 +0000 Received: from mga02.intel.com ([134.134.136.20]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbzh3-004KVB-Px for linux-arm-kernel@lists.infradead.org; Wed, 06 Apr 2022 07:03:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649228585; x=1680764585; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=9yXnWPFaJPcfSAhgXMQM40zFYivmgfvuHZVyoADMm+c=; b=mbsvYEjLHxyxhPLGjp+0MSH2d1Qi/HG2YDxQ8XKlUnvmRk4WPnPRRyW9 doMX+SYVUVuYwTWHoiZAmxdSWMe2mBKjf4YRbL0BbJ7XLPs/5LtOnT0xp SiQEQ5PHmn7wyUfV7HT8K9qrbYTIt85a0qY+bPNIRvhlIZ8wo7VWg+aIc B9RY694pRZaXEwFvPnzw4RCTkRD6h3e8p4w2o7GJm1bgrNndVuwKR0W2k AZo0+PznGTgUvNeTKR5UnAP445dl3BqX7sGRObH89/FaMLCs/fAin5JMp 86wDijGvKA7J3EGNmMoMaKyv2OuPzVrZSjgZRmMTexiDEn7yOPioG14EI w==; X-IronPort-AV: E=McAfee;i="6200,9189,10308"; a="248491674" X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="248491674" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Apr 2022 00:02:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="549419306" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga007.jf.intel.com with ESMTP; 06 Apr 2022 00:02:38 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Wed, 6 Apr 2022 00:02:37 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.175) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eBsP97RyNVSYXfzW8F4rHWni1sG/h3uEbWD6PLNUBSygD18SAKz/2QGaWvwkWKOJvIuxtC0TlRXQpImmoftz/2uSqVx3+Ehq8gxCVS6Bf314AcShdPb7pYTcSpmQcbfv3tMnkZ3Xr8f0Qqlc4td0mx6k/JeRDXojsBJz4NDEH3H0NJF0LI2CjMRA14a8ZFLXxHb/3SiyELxUuZVL7S7xq1W/ARyqWTNvXZhKbuVB9su3DqB8Gt9imp80pIbelQDu2HBfhclZTSVEFQ95Gh+uX8IHaDguzDytGDjbyiZKWbDnKY1/F58jeU9hO+TYr7k/jrvRsU3m1ZDF0Ihlm1s0GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MpZ5rpjZ/323Q4FiP3H6XfuAq/WWbmz2u2b2ATpZRiM=; b=I7SefwtHVKGK5w/dzLMRezmQMPcMjak4bDmfbhQOrh9owDU7qceXEvhUx7QP6vTIEl7X3E6wF9kehI3JoQY+9MqI3obUP2+1Ar7gotsixxCrtDuQQdVs055tSrUUgmXSkXeJJ5i9F56+fHIRR4CPi0e+xZF0gIdkjYA6POen9mE23p51vhYiMEcm9qcohDdqIunLz3cF3ev7a/0Sh912/q61+IaM/AK48mtwGcV+8Aci8YqFf5c4LHunoZ6KeDI2W5zqdi6LD5bNC6orRhoSa561uibmqvX0VLUoXnNkyrvTUNl8cozuRWCwqmHO6InLXmo9ebT7NxfMjOSLQb9NJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5276.namprd11.prod.outlook.com (2603:10b6:408:135::18) by PH7PR11MB5885.namprd11.prod.outlook.com (2603:10b6:510:134::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.30; Wed, 6 Apr 2022 07:02:36 +0000 Received: from BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::c4ea:a404:b70b:e54e]) by BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::c4ea:a404:b70b:e54e%7]) with mapi id 15.20.5123.031; Wed, 6 Apr 2022 07:02:36 +0000 From: "Tian, Kevin" To: Jason Gunthorpe , Alex Williamson CC: Lu Baolu , Christian Benvenuti , Cornelia Huck , David Woodhouse , Gerald Schaefer , "iommu@lists.linux-foundation.org" , "Jason Wang" , Joerg Roedel , "kvm@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-arm-msm@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "linux-s390@vger.kernel.org" , Matthew Rosato , "Michael S. Tsirkin" , Nelson Escobar , "netdev@vger.kernel.org" , Rob Clark , Robin Murphy , "Suravee Suthikulpanit" , "virtualization@lists.linux-foundation.org" , Will Deacon , Christoph Hellwig Subject: RE: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Thread-Topic: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Thread-Index: AQHYSQh+iGQB4aLBFUu6szRllIBFLqzhr6EAgAAFLgCAAMFI0A== Date: Wed, 6 Apr 2022 07:02:36 +0000 Message-ID: References: <0-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <20220405131044.23910b77.alex.williamson@redhat.com> <20220405192916.GT2120790@nvidia.com> In-Reply-To: <20220405192916.GT2120790@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dd1d5368-9447-465b-35fd-08da179b6e30 x-ms-traffictypediagnostic: PH7PR11MB5885:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5Azv5BpEPVMypZ7ZE/68AxddWT9AhaSUULiUIPoVJ43Z/0MKytwAyzwrT8Yrxt5Dp0ZYkFzELg78LbjkYaHMquOq53uKZrUJy+lmTswDcMMmBN5RrXKckYsztRnkdaa/VZWuA+AwRKlgJlFg1ZIDDBLzT0dmUh2RcfBdPFoica0qEeg7hw8dBd0U4jeKHam3gIt1uL0tCJGFvb0lFhehH3qtC83T37r9P8Xokv4xKWU+AwTiKgHf1ft+fi/UNr/U1jPN6Yi7tD+7puHOzL8G/tmS0AbO71A/SVoL1Iew7gA4YaHECbP2FWe9HguZj6UnANprG74YMjjCFcPrr+9NaqKA/qPmwSFDN9QiE2TnHz5ZL0Tr3/HQsaU8fypcKGHtyhLj1LZGavzopgxXAt/RhVtgF2KbJp/m1hZ0xe1IEeSUUc+KNHeMLhjEVjy/7+zSnSVZeskLgwv9e2J/HPH/5rPtcArFwPJDSU+SyeVywNmVYC+tlRUCAD099uQu2LsPLwDEADO60oJTmGLKGKZ+YfdoUho5PgF2aY0Rk3XabBN5HHmPte5iLF76xk5yLYe6C0oEARuyxp2jRv3nuQ33cQdhuXd7bk3btMP60nY2D5i+GqBWYDVTu0KdKJxnsmuA3AlQWBU8ewvOV+wvZd9iGF8n3e8kjFAcIZdrTjy6vShaVMk46DF/w26JYLkrj7TGmkuaV4jyiZINYqWdrhmZIA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR11MB5276.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(316002)(8676002)(71200400001)(66446008)(4326008)(52536014)(66556008)(6506007)(66946007)(110136005)(66476007)(7696005)(54906003)(508600001)(55016003)(76116006)(8936002)(64756008)(7416002)(5660300002)(33656002)(86362001)(9686003)(186003)(26005)(83380400001)(38070700005)(2906002)(38100700002)(82960400001)(122000001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Gmd1F0kaSG5OHjvaZPLWx6NVaVyull7bel9d013fcaw/eZO66M3Nlk0keRtl?= =?us-ascii?Q?qXKs/c15VLZfQqrjKDz+6NRbKj/s4SC4rGViNbMb+NDxKmd8XfE3JFF2R5WY?= =?us-ascii?Q?n0nv4jZg9OFH6soZCsHJDj40gZMxcgWHivfd06apqJE8je4LmIrmlQJ50NER?= =?us-ascii?Q?iNheOoO4UGVfbz2onI8JcCEWyD3gIlHk1PTjMNb3hBMnTsCb9h3Y1kmeLoXW?= =?us-ascii?Q?W8bsQWFHAf2CR5QP4ru9aqvIhhy7+O8QfqGmv2KFvY/Sa8i6PFyIP2XcrZAn?= =?us-ascii?Q?5z7pe3JsIzs6UOOeGWVHdj3nCS0tby7si/rIKV75+ykVgXKL1bQwbMBI/MlI?= =?us-ascii?Q?JSqLhRJmRRwbBHlwFYdRCXI0jr83C25hmhNAGDNf5h0MZ2tTCF2k71l5bGWK?= =?us-ascii?Q?HCbAMCMpMBNnZAulkP3reD46EIRZ3dvHVB/5ZJtKkkUyoTxDzS/ov72nVDhK?= =?us-ascii?Q?2bpHACHXYHosAZGAsGyPboQLRmEzgwRZjsHRNFy2YJSI1YwZXYyuDa90MXqA?= =?us-ascii?Q?Y5mnSY79Hz8eZjZJa0KkSxZD0pgwieACaVwmuPPemBYl8y+oQsQU1FirHxCX?= =?us-ascii?Q?r+qkTzDgJWTjTEiZCQEmOK/rXxpNImDo3L+1oXW55Plrl2bwooUwjL0XgIG1?= =?us-ascii?Q?VjZvEsHaAd2J/s7Esb3cItMM6C2cwnJoCc3VqbGD9F3G3fdGToopNpMcZ3Gi?= =?us-ascii?Q?h/HnFNTktk/hoEy5fuL3rpZFbcCxEO8CNXhZX19IVoYuHI4JnCd3bwpTq04d?= =?us-ascii?Q?izOz6txSbXP/mv0nKwZlJJTqizYnGvh8/2rYx0OQdQZ4dvWMNZxj1dxsG1P5?= =?us-ascii?Q?cNiET/y45qWppgv+SQtmwiB+lgVP2JA8Xb/aJ7BDrDPpL9n2KZ1HUE4mMtoH?= =?us-ascii?Q?dlqg+TJCmtEsp1+Qu01XC7lbMLGTiYM5jXJH6dpPeY1opkE7KYm69HzJiH53?= =?us-ascii?Q?BpntXw4Xk12maVf5vCLMvSLKxso5p8u25XMOEt+2/SYGXq5OQfbAbAGVeGGA?= =?us-ascii?Q?RhEtSFIaE1c9Tc+efkqOVepP6Mx73Fs4Jv0mfHT7kTm9JWRlJVoaQuJ88nm+?= =?us-ascii?Q?HotDkb48k5pfBQFSWVKiWoKOBcp8fpuAR8GYKp85RrPaaBY02r4pMlln7dF+?= =?us-ascii?Q?dBYR1QPq7T66ZezYi1vT69vW0O3iJQyXEjFn+9f+RdATD1YtqwAfjYq/VSkp?= =?us-ascii?Q?HcRqId623/McuWWdEpoUgBrDaQG5YqHsgOLdPevAybSYNFuBK1P9c2T/6uVw?= =?us-ascii?Q?1e3J1hBExHlF6giKAJKQB+rfEXsKvRXXmNIpeGkcbOOpyjoipkef8lQIAnKv?= =?us-ascii?Q?/9cCV8eIXsvhthRjwQI1VY1LAs5jBhjwiMwYwO1vt2UesnLbubAR245RnSMY?= =?us-ascii?Q?jj8ZX4ZxYLod5yLxFgEWVeBZo7NyH9sLp3EAmjWKkzlC3fFaSpK/JudmfxFz?= =?us-ascii?Q?KN1v6PhPUxTtOow9q0i4A7Iu/lJnN6dYvlqFYqgxK2xM11g6XghiIp+VzqfS?= =?us-ascii?Q?1aYELPMwWCcKAtFpurpawVpA0KVaONCKLph89gy9ndocCfOxTrZKOqJy/NY5?= =?us-ascii?Q?fHlvn6vL5ejF6q12tXcVcD9+MuTy4QN/iqfJCgxzLtplTW0k/lsgbk2CQoxG?= =?us-ascii?Q?I8YaJ1mgTJgWBE7LfMvHMhKMxdpXcB1uqjK/AQpZfDvlC+8ryk53WArLR8tV?= =?us-ascii?Q?SKsaJJO4PHb4mDVcpWdb9FYN4t8iQENqWvya8oejN3H0SagMFpS48o7TXnWK?= =?us-ascii?Q?oDDpdNwP7g=3D=3D?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd1d5368-9447-465b-35fd-08da179b6e30 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2022 07:02:36.1106 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bnIj9JeokweneDmdbZpx8OX+uXUvq0hi4EJf212dVk9h+hRopZ0qPTT8ZZAqihWB/l8gdLrsiJoZgLSeSfMKIA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB5885 X-OriginatorOrg: intel.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220406_000305_928491_7BFD09BB X-CRM114-Status: GOOD ( 32.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > From: Jason Gunthorpe > Sent: Wednesday, April 6, 2022 3:29 AM > > On Tue, Apr 05, 2022 at 01:10:44PM -0600, Alex Williamson wrote: > > On Tue, 5 Apr 2022 13:16:01 -0300 > > Jason Gunthorpe wrote: > > > > > dev_is_dma_coherent() is the control to determine if IOMMU_CACHE can > be > > > supported. > > > > > > IOMMU_CACHE means that normal DMAs do not require any additional > coherency > > > mechanism and is the basic uAPI that VFIO exposes to userspace. For > > > instance VFIO applications like DPDK will not work if additional coherency > > > operations are required. > > > > > > Therefore check dev_is_dma_coherent() before allowing a device to join > a > > > domain. This will block device/platform/iommu combinations from using > VFIO > > > that do not support cache coherent DMA. > > > > > > Signed-off-by: Jason Gunthorpe > > > drivers/vfio/vfio.c | 6 ++++++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > > > index a4555014bd1e72..2a3aa3e742d943 100644 > > > +++ b/drivers/vfio/vfio.c > > > @@ -32,6 +32,7 @@ > > > #include > > > #include > > > #include > > > +#include > > > #include "vfio.h" > > > > > > #define DRIVER_VERSION "0.3" > > > @@ -1348,6 +1349,11 @@ static int vfio_group_get_device_fd(struct > vfio_group *group, char *buf) > > > if (IS_ERR(device)) > > > return PTR_ERR(device); > > > > > > + if (group->type == VFIO_IOMMU && !dev_is_dma_coherent(device- > >dev)) { > > > + ret = -ENODEV; > > > + goto err_device_put; > > > + } > > > + > > > > Failing at the point where the user is trying to gain access to the > > device seems a little late in the process and opaque, wouldn't we > > rather have vfio bus drivers fail to probe such devices? I'd expect > > this to occur in the vfio_register_group_dev() path. Thanks, > > Yes, that is a good point. > > So like this: > > int vfio_register_group_dev(struct vfio_device *device) > { > + if (!dev_is_dma_coherent(device->dev)) > + return -EINVAL; > + > return __vfio_register_dev(device, > vfio_group_find_or_alloc(device->dev)); > } > > I fixed it up. > if that is the case should it also apply to usnic and vdpa in the first patch (i.e. fail the probe)? _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD750C433F5 for ; Wed, 6 Apr 2022 11:51:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229585AbiDFLxL (ORCPT ); Wed, 6 Apr 2022 07:53:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230437AbiDFLwY (ORCPT ); Wed, 6 Apr 2022 07:52:24 -0400 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11A4E3D51A6; Wed, 6 Apr 2022 00:03:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649228583; x=1680764583; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=9yXnWPFaJPcfSAhgXMQM40zFYivmgfvuHZVyoADMm+c=; b=QYuIUUTFgKNNUhoLPEjr+MFALjO1/xIRd7ix+BTczNghBfuxHKFeaJyl tsYsy/UmrfyWFRIFBj/gMwotMHJ0W+QuRpImB8pNtTB6wEPHmbtRZyJVL KH5wCOiWpwP4r2jKI/RePMTpPt1c81ODPK6QvRvRES9FyN1LZIBGVWcp5 AkJCqsgPTZWs+RBcJY3Qr8x9sKQFB+W+37sFs4HQIY/U4Ltg0tiOfRHtq x65sluwRXAl5uvRrzOY3pwhtyiLodoZGwm0eOOPLoYCdeKXO8Cxw1hpbh Nu6tCYVdeDE05VQ2VagN8cz3t7oaQSKWB/2Z6gvIZncK0KiyNdkqzSoI8 w==; X-IronPort-AV: E=McAfee;i="6200,9189,10308"; a="248491673" X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="248491673" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Apr 2022 00:02:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="549419306" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga007.jf.intel.com with ESMTP; 06 Apr 2022 00:02:38 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Wed, 6 Apr 2022 00:02:37 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.175) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Wed, 6 Apr 2022 00:02:37 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eBsP97RyNVSYXfzW8F4rHWni1sG/h3uEbWD6PLNUBSygD18SAKz/2QGaWvwkWKOJvIuxtC0TlRXQpImmoftz/2uSqVx3+Ehq8gxCVS6Bf314AcShdPb7pYTcSpmQcbfv3tMnkZ3Xr8f0Qqlc4td0mx6k/JeRDXojsBJz4NDEH3H0NJF0LI2CjMRA14a8ZFLXxHb/3SiyELxUuZVL7S7xq1W/ARyqWTNvXZhKbuVB9su3DqB8Gt9imp80pIbelQDu2HBfhclZTSVEFQ95Gh+uX8IHaDguzDytGDjbyiZKWbDnKY1/F58jeU9hO+TYr7k/jrvRsU3m1ZDF0Ihlm1s0GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MpZ5rpjZ/323Q4FiP3H6XfuAq/WWbmz2u2b2ATpZRiM=; b=I7SefwtHVKGK5w/dzLMRezmQMPcMjak4bDmfbhQOrh9owDU7qceXEvhUx7QP6vTIEl7X3E6wF9kehI3JoQY+9MqI3obUP2+1Ar7gotsixxCrtDuQQdVs055tSrUUgmXSkXeJJ5i9F56+fHIRR4CPi0e+xZF0gIdkjYA6POen9mE23p51vhYiMEcm9qcohDdqIunLz3cF3ev7a/0Sh912/q61+IaM/AK48mtwGcV+8Aci8YqFf5c4LHunoZ6KeDI2W5zqdi6LD5bNC6orRhoSa561uibmqvX0VLUoXnNkyrvTUNl8cozuRWCwqmHO6InLXmo9ebT7NxfMjOSLQb9NJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5276.namprd11.prod.outlook.com (2603:10b6:408:135::18) by PH7PR11MB5885.namprd11.prod.outlook.com (2603:10b6:510:134::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.30; Wed, 6 Apr 2022 07:02:36 +0000 Received: from BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::c4ea:a404:b70b:e54e]) by BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::c4ea:a404:b70b:e54e%7]) with mapi id 15.20.5123.031; Wed, 6 Apr 2022 07:02:36 +0000 From: "Tian, Kevin" To: Jason Gunthorpe , Alex Williamson CC: Lu Baolu , Christian Benvenuti , Cornelia Huck , David Woodhouse , Gerald Schaefer , "iommu@lists.linux-foundation.org" , "Jason Wang" , Joerg Roedel , "kvm@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-arm-msm@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "linux-s390@vger.kernel.org" , Matthew Rosato , "Michael S. Tsirkin" , Nelson Escobar , "netdev@vger.kernel.org" , Rob Clark , Robin Murphy , "Suravee Suthikulpanit" , "virtualization@lists.linux-foundation.org" , Will Deacon , Christoph Hellwig Subject: RE: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Thread-Topic: [PATCH 2/5] vfio: Require that devices support DMA cache coherence Thread-Index: AQHYSQh+iGQB4aLBFUu6szRllIBFLqzhr6EAgAAFLgCAAMFI0A== Date: Wed, 6 Apr 2022 07:02:36 +0000 Message-ID: References: <0-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <2-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <20220405131044.23910b77.alex.williamson@redhat.com> <20220405192916.GT2120790@nvidia.com> In-Reply-To: <20220405192916.GT2120790@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dd1d5368-9447-465b-35fd-08da179b6e30 x-ms-traffictypediagnostic: PH7PR11MB5885:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5Azv5BpEPVMypZ7ZE/68AxddWT9AhaSUULiUIPoVJ43Z/0MKytwAyzwrT8Yrxt5Dp0ZYkFzELg78LbjkYaHMquOq53uKZrUJy+lmTswDcMMmBN5RrXKckYsztRnkdaa/VZWuA+AwRKlgJlFg1ZIDDBLzT0dmUh2RcfBdPFoica0qEeg7hw8dBd0U4jeKHam3gIt1uL0tCJGFvb0lFhehH3qtC83T37r9P8Xokv4xKWU+AwTiKgHf1ft+fi/UNr/U1jPN6Yi7tD+7puHOzL8G/tmS0AbO71A/SVoL1Iew7gA4YaHECbP2FWe9HguZj6UnANprG74YMjjCFcPrr+9NaqKA/qPmwSFDN9QiE2TnHz5ZL0Tr3/HQsaU8fypcKGHtyhLj1LZGavzopgxXAt/RhVtgF2KbJp/m1hZ0xe1IEeSUUc+KNHeMLhjEVjy/7+zSnSVZeskLgwv9e2J/HPH/5rPtcArFwPJDSU+SyeVywNmVYC+tlRUCAD099uQu2LsPLwDEADO60oJTmGLKGKZ+YfdoUho5PgF2aY0Rk3XabBN5HHmPte5iLF76xk5yLYe6C0oEARuyxp2jRv3nuQ33cQdhuXd7bk3btMP60nY2D5i+GqBWYDVTu0KdKJxnsmuA3AlQWBU8ewvOV+wvZd9iGF8n3e8kjFAcIZdrTjy6vShaVMk46DF/w26JYLkrj7TGmkuaV4jyiZINYqWdrhmZIA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5276.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(316002)(8676002)(71200400001)(66446008)(4326008)(52536014)(66556008)(6506007)(66946007)(110136005)(66476007)(7696005)(54906003)(508600001)(55016003)(76116006)(8936002)(64756008)(7416002)(5660300002)(33656002)(86362001)(9686003)(186003)(26005)(83380400001)(38070700005)(2906002)(38100700002)(82960400001)(122000001);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Gmd1F0kaSG5OHjvaZPLWx6NVaVyull7bel9d013fcaw/eZO66M3Nlk0keRtl?= =?us-ascii?Q?qXKs/c15VLZfQqrjKDz+6NRbKj/s4SC4rGViNbMb+NDxKmd8XfE3JFF2R5WY?= =?us-ascii?Q?n0nv4jZg9OFH6soZCsHJDj40gZMxcgWHivfd06apqJE8je4LmIrmlQJ50NER?= =?us-ascii?Q?iNheOoO4UGVfbz2onI8JcCEWyD3gIlHk1PTjMNb3hBMnTsCb9h3Y1kmeLoXW?= =?us-ascii?Q?W8bsQWFHAf2CR5QP4ru9aqvIhhy7+O8QfqGmv2KFvY/Sa8i6PFyIP2XcrZAn?= =?us-ascii?Q?5z7pe3JsIzs6UOOeGWVHdj3nCS0tby7si/rIKV75+ykVgXKL1bQwbMBI/MlI?= =?us-ascii?Q?JSqLhRJmRRwbBHlwFYdRCXI0jr83C25hmhNAGDNf5h0MZ2tTCF2k71l5bGWK?= =?us-ascii?Q?HCbAMCMpMBNnZAulkP3reD46EIRZ3dvHVB/5ZJtKkkUyoTxDzS/ov72nVDhK?= =?us-ascii?Q?2bpHACHXYHosAZGAsGyPboQLRmEzgwRZjsHRNFy2YJSI1YwZXYyuDa90MXqA?= =?us-ascii?Q?Y5mnSY79Hz8eZjZJa0KkSxZD0pgwieACaVwmuPPemBYl8y+oQsQU1FirHxCX?= =?us-ascii?Q?r+qkTzDgJWTjTEiZCQEmOK/rXxpNImDo3L+1oXW55Plrl2bwooUwjL0XgIG1?= =?us-ascii?Q?VjZvEsHaAd2J/s7Esb3cItMM6C2cwnJoCc3VqbGD9F3G3fdGToopNpMcZ3Gi?= =?us-ascii?Q?h/HnFNTktk/hoEy5fuL3rpZFbcCxEO8CNXhZX19IVoYuHI4JnCd3bwpTq04d?= =?us-ascii?Q?izOz6txSbXP/mv0nKwZlJJTqizYnGvh8/2rYx0OQdQZ4dvWMNZxj1dxsG1P5?= =?us-ascii?Q?cNiET/y45qWppgv+SQtmwiB+lgVP2JA8Xb/aJ7BDrDPpL9n2KZ1HUE4mMtoH?= =?us-ascii?Q?dlqg+TJCmtEsp1+Qu01XC7lbMLGTiYM5jXJH6dpPeY1opkE7KYm69HzJiH53?= =?us-ascii?Q?BpntXw4Xk12maVf5vCLMvSLKxso5p8u25XMOEt+2/SYGXq5OQfbAbAGVeGGA?= =?us-ascii?Q?RhEtSFIaE1c9Tc+efkqOVepP6Mx73Fs4Jv0mfHT7kTm9JWRlJVoaQuJ88nm+?= =?us-ascii?Q?HotDkb48k5pfBQFSWVKiWoKOBcp8fpuAR8GYKp85RrPaaBY02r4pMlln7dF+?= =?us-ascii?Q?dBYR1QPq7T66ZezYi1vT69vW0O3iJQyXEjFn+9f+RdATD1YtqwAfjYq/VSkp?= =?us-ascii?Q?HcRqId623/McuWWdEpoUgBrDaQG5YqHsgOLdPevAybSYNFuBK1P9c2T/6uVw?= =?us-ascii?Q?1e3J1hBExHlF6giKAJKQB+rfEXsKvRXXmNIpeGkcbOOpyjoipkef8lQIAnKv?= =?us-ascii?Q?/9cCV8eIXsvhthRjwQI1VY1LAs5jBhjwiMwYwO1vt2UesnLbubAR245RnSMY?= =?us-ascii?Q?jj8ZX4ZxYLod5yLxFgEWVeBZo7NyH9sLp3EAmjWKkzlC3fFaSpK/JudmfxFz?= =?us-ascii?Q?KN1v6PhPUxTtOow9q0i4A7Iu/lJnN6dYvlqFYqgxK2xM11g6XghiIp+VzqfS?= =?us-ascii?Q?1aYELPMwWCcKAtFpurpawVpA0KVaONCKLph89gy9ndocCfOxTrZKOqJy/NY5?= =?us-ascii?Q?fHlvn6vL5ejF6q12tXcVcD9+MuTy4QN/iqfJCgxzLtplTW0k/lsgbk2CQoxG?= =?us-ascii?Q?I8YaJ1mgTJgWBE7LfMvHMhKMxdpXcB1uqjK/AQpZfDvlC+8ryk53WArLR8tV?= =?us-ascii?Q?SKsaJJO4PHb4mDVcpWdb9FYN4t8iQENqWvya8oejN3H0SagMFpS48o7TXnWK?= =?us-ascii?Q?oDDpdNwP7g=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd1d5368-9447-465b-35fd-08da179b6e30 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2022 07:02:36.1106 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bnIj9JeokweneDmdbZpx8OX+uXUvq0hi4EJf212dVk9h+hRopZ0qPTT8ZZAqihWB/l8gdLrsiJoZgLSeSfMKIA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB5885 X-OriginatorOrg: intel.com Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org > From: Jason Gunthorpe > Sent: Wednesday, April 6, 2022 3:29 AM >=20 > On Tue, Apr 05, 2022 at 01:10:44PM -0600, Alex Williamson wrote: > > On Tue, 5 Apr 2022 13:16:01 -0300 > > Jason Gunthorpe wrote: > > > > > dev_is_dma_coherent() is the control to determine if IOMMU_CACHE can > be > > > supported. > > > > > > IOMMU_CACHE means that normal DMAs do not require any additional > coherency > > > mechanism and is the basic uAPI that VFIO exposes to userspace. For > > > instance VFIO applications like DPDK will not work if additional cohe= rency > > > operations are required. > > > > > > Therefore check dev_is_dma_coherent() before allowing a device to joi= n > a > > > domain. This will block device/platform/iommu combinations from using > VFIO > > > that do not support cache coherent DMA. > > > > > > Signed-off-by: Jason Gunthorpe > > > drivers/vfio/vfio.c | 6 ++++++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > > > index a4555014bd1e72..2a3aa3e742d943 100644 > > > +++ b/drivers/vfio/vfio.c > > > @@ -32,6 +32,7 @@ > > > #include > > > #include > > > #include > > > +#include > > > #include "vfio.h" > > > > > > #define DRIVER_VERSION "0.3" > > > @@ -1348,6 +1349,11 @@ static int vfio_group_get_device_fd(struct > vfio_group *group, char *buf) > > > if (IS_ERR(device)) > > > return PTR_ERR(device); > > > > > > + if (group->type =3D=3D VFIO_IOMMU && !dev_is_dma_coherent(device- > >dev)) { > > > + ret =3D -ENODEV; > > > + goto err_device_put; > > > + } > > > + > > > > Failing at the point where the user is trying to gain access to the > > device seems a little late in the process and opaque, wouldn't we > > rather have vfio bus drivers fail to probe such devices? I'd expect > > this to occur in the vfio_register_group_dev() path. Thanks, >=20 > Yes, that is a good point. >=20 > So like this: >=20 > int vfio_register_group_dev(struct vfio_device *device) > { > + if (!dev_is_dma_coherent(device->dev)) > + return -EINVAL; > + > return __vfio_register_dev(device, > vfio_group_find_or_alloc(device->dev)); > } >=20 > I fixed it up. >=20 if that is the case should it also apply to usnic and vdpa in the first patch (i.e. fail the probe)?