From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26A98C4332F for ; Thu, 10 Nov 2022 17:11:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231373AbiKJRLl (ORCPT ); Thu, 10 Nov 2022 12:11:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231586AbiKJRLa (ORCPT ); Thu, 10 Nov 2022 12:11:30 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12FCC4875B for ; Thu, 10 Nov 2022 09:10:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668100229; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3XBNZVfwHOyxrKzEqgSDBaV/kDGeCsLaREPRXxkN8EM=; b=XULxVucSkRIGoY/EN8rhP0fUqRAz3tswTdiEdVuYBewesksd5qBC60MF413K/p5Qg5uyFC TelMqzDSbVV24cBNUc6mUQFinKm0NXsV/BHbL7cz1sVKqfaj1Hsqsv08cay9qG13GZ7HJ0 e1uNlKBGY5Vsajy1edc/kWghC2LupkE= Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-635-SnB2kGhDPKWZ2f4cFrkjaQ-1; Thu, 10 Nov 2022 12:10:27 -0500 X-MC-Unique: SnB2kGhDPKWZ2f4cFrkjaQ-1 Received: by mail-io1-f70.google.com with SMTP id n23-20020a056602341700b00689fc6dbfd6so1433632ioz.8 for ; Thu, 10 Nov 2022 09:10:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3XBNZVfwHOyxrKzEqgSDBaV/kDGeCsLaREPRXxkN8EM=; b=S4WUURcg+wBk8PAaZuktVHdQmKDoZGm1kq2Ph18KwQj83tfVp1+bG5xbBkVIIvnR9F 1nauu6/3m/NJGWXhju2ntP1zmShnuFiMIIWO4sJONScwarP+/pxm1nZW0qxpznzDZJpp ADPb69NOrZZtDkMfuKaj2maOaUIHodt1oa6vsKHN0CCPUKL+qx2w/CqlekA7aEhUu/gl BCHeVJHYQy9gaRieIV/c+/6AsmhVGnDYV3oQo4zjYal5kj8vgXWJCONpEMubP5vhJP0j ugVtLS50wPJ43nI6MPWQ/WI6zs/ctHvCzjTIdqVj7xryW3G1FDLb8dFqIvvSnfXRpzPU Dl0w== X-Gm-Message-State: ACrzQf1HOzp/tfV0UQh0KViUF+IXhxaWtea1nogVTXNjL4u1eu1U6nsP PZAvpSa3dsxJ5IZusdpgDEOn8TbwxcPBOEVLVscfvlTQGBpU/cYPgpVT5ZlNVwT4PBxEwg3Z5m3 QvPD+H+17EP9B898K+AyZ1A== X-Received: by 2002:a05:6638:4413:b0:374:fbbe:2da6 with SMTP id bp19-20020a056638441300b00374fbbe2da6mr3149409jab.163.1668100226563; Thu, 10 Nov 2022 09:10:26 -0800 (PST) X-Google-Smtp-Source: AMsMyM5st2ROvi8y5IPXYSHhk18Mv2oS6m3aLQCRNeAm9L0KyqI2dN1lMnwqPxWnGmIZODwHu3sF6g== X-Received: by 2002:a05:6638:4413:b0:374:fbbe:2da6 with SMTP id bp19-20020a056638441300b00374fbbe2da6mr3149377jab.163.1668100226283; Thu, 10 Nov 2022 09:10:26 -0800 (PST) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id m1-20020a924a01000000b002f9652849f6sm30029ilf.67.2022.11.10.09.10.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Nov 2022 09:10:25 -0800 (PST) Date: Thu, 10 Nov 2022 10:10:23 -0700 From: Alex Williamson To: "Tian, Kevin" Cc: Jason Gunthorpe , Alexander Gordeev , David Airlie , Tony Krowiak , "Christian Borntraeger" , Cornelia Huck , Daniel Vetter , Diana Craciun , "dri-devel@lists.freedesktop.org" , "Eric Auger" , Eric Farman , "Harald Freudenberger" , Vasily Gorbik , Heiko Carstens , "intel-gfx@lists.freedesktop.org" , "intel-gvt-dev@lists.freedesktop.org" , "iommu@lists.linux.dev" , Jani Nikula , "Jason Herne" , Joonas Lahtinen , Joerg Roedel , "kvm@vger.kernel.org" , "linux-s390@vger.kernel.org" , Longfang Liu , "Matthew Rosato" , Peter Oberparleiter , Halil Pasic , Robin Murphy , "Vivi, Rodrigo" , "Shameer Kolothum" , Sven Schnelle , Tvrtko Ursulin , Vineeth Vijayan , Will Deacon , Yishai Hadas , Zhenyu Wang , "Wang, Zhi A" , Lu Baolu , Nicolin Chen , "Liu, Yi L" Subject: Re: [PATCH v2 10/11] vfio: Make vfio_container optionally compiled Message-ID: <20221110101023.28e7a790.alex.williamson@redhat.com> In-Reply-To: References: <0-v2-65016290f146+33e-vfio_iommufd_jgg@nvidia.com> <10-v2-65016290f146+33e-vfio_iommufd_jgg@nvidia.com> <20221108152831.1a2ed3df.alex.williamson@redhat.com> <20221109101809.2ff08303.alex.williamson@redhat.com> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.34; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-s390@vger.kernel.org On Thu, 10 Nov 2022 06:57:57 +0000 "Tian, Kevin" wrote: > > From: Jason Gunthorpe > > Sent: Thursday, November 10, 2022 3:53 AM > > > > On Wed, Nov 09, 2022 at 10:18:09AM -0700, Alex Williamson wrote: > > > > > DPDK supports no-iommu mode. > > > > Er? Huh? How? I thought no-iommu was for applications that didn't do > > DMA? How is DPDK getting packets in/out without DMA? I guess it snoops > > in /proc/ or something to learn PFNs of mlock'd memory? > > iirc dpdk started with UIO plus various tricks (root privilege, hugepage, etc.) > to lock and learn PFN's from pagemap. Then when migrating it to vfio the > no-iommu option was introduced to provide UIO compatibility. IIRC, we essentially introduced no-iommu mode vfio because DPDK started pushing for extending interrupt support in uio-pci-generic. The UIO driver is also only meant for devices that don't do DMA, but obviously DPDK didn't care about that. Rather than extend UIO, we offered this no-iommu mode in vfio since we already had more extensive MSI support, were better able to impose restrictions on access to the device, and using the same device access makes the transition to proper IOMMU backed configurations more seamless. Thanks, Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 481BBC4332F for ; Thu, 10 Nov 2022 17:10:37 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AA28E10E175; Thu, 10 Nov 2022 17:10:35 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by gabe.freedesktop.org (Postfix) with ESMTPS id 141E110E175 for ; Thu, 10 Nov 2022 17:10:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668100228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3XBNZVfwHOyxrKzEqgSDBaV/kDGeCsLaREPRXxkN8EM=; b=bSmu2RAUiPP+uyo1YY4ieRzl9cShG71GbLpnBxH2iT03EesUvtjp48uwtHrt5Mtdb/GbJx 3Bmk3RSSqk5MkKvhFrJaxVIH+zm8UugH4OueDyc3RVnqPvVn6Vday7YjCE1XwOVbwCt/Nc 337UpDRnVVttpGEttl8herIPp31QH1Y= Received: from mail-il1-f198.google.com (mail-il1-f198.google.com [209.85.166.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-635-hJv-SN6EOmu-44yTMIkksA-1; Thu, 10 Nov 2022 12:10:27 -0500 X-MC-Unique: hJv-SN6EOmu-44yTMIkksA-1 Received: by mail-il1-f198.google.com with SMTP id s4-20020a056e02216400b003021b648144so1987529ilv.19 for ; Thu, 10 Nov 2022 09:10:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3XBNZVfwHOyxrKzEqgSDBaV/kDGeCsLaREPRXxkN8EM=; b=WBL4ETyUYS+8j95/VjH9WzfQiCFQ9OkNGHC3OIjC/1crLWIPPkOcus+yZMG+R+mYzX bZBUv2tE+dJ53iW7/ZnBs1cy6lU82FQg2VyBPmeRTDQQG2xQXhWd7M7VVmk/PQeqBaxo vDwDlDGvarmqGyKkAjSvgceJIiYxrTiFHqFsuP2061bN6k5KiyvzFNX1mEL66krdifpY YWctVZn/3XQxH7RZcmIUAoSSECqvAd3epfGhnbRwvMpIMSwHZvuzX2nmtqG6ZPpAaHN0 dT7mSZJEFZMLH6I0raqtPRT1cr5r7D4pYsEs6xREF2ufQTqXGmv15n8A/P8MUv4aR/yY 3deA== X-Gm-Message-State: ACrzQf2soThy9q6plOjnjdP1Be9t1zByLGsGPMRsK9Vg/kL8weZipMas Bs9K5gZsCfqB33lBrSNHjeiwW2Qvr5CAarZrNH0eK3JbwWWfymwA5xibXSj82PjHqMXUUII2KIA RG+33Ymhnv6QuHWIU9qH4eJOn0uS1 X-Received: by 2002:a05:6638:4413:b0:374:fbbe:2da6 with SMTP id bp19-20020a056638441300b00374fbbe2da6mr3149425jab.163.1668100226566; Thu, 10 Nov 2022 09:10:26 -0800 (PST) X-Google-Smtp-Source: AMsMyM5st2ROvi8y5IPXYSHhk18Mv2oS6m3aLQCRNeAm9L0KyqI2dN1lMnwqPxWnGmIZODwHu3sF6g== X-Received: by 2002:a05:6638:4413:b0:374:fbbe:2da6 with SMTP id bp19-20020a056638441300b00374fbbe2da6mr3149377jab.163.1668100226283; Thu, 10 Nov 2022 09:10:26 -0800 (PST) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id m1-20020a924a01000000b002f9652849f6sm30029ilf.67.2022.11.10.09.10.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Nov 2022 09:10:25 -0800 (PST) Date: Thu, 10 Nov 2022 10:10:23 -0700 From: Alex Williamson To: "Tian, Kevin" Subject: Re: [PATCH v2 10/11] vfio: Make vfio_container optionally compiled Message-ID: <20221110101023.28e7a790.alex.williamson@redhat.com> In-Reply-To: References: <0-v2-65016290f146+33e-vfio_iommufd_jgg@nvidia.com> <10-v2-65016290f146+33e-vfio_iommufd_jgg@nvidia.com> <20221108152831.1a2ed3df.alex.williamson@redhat.com> <20221109101809.2ff08303.alex.williamson@redhat.com> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.34; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "kvm@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , Vineeth Vijayan , Diana Craciun , Alexander Gordeev , Longfang Liu , "linux-s390@vger.kernel.org" , "Liu, Yi L" , Matthew Rosato , Will Deacon , Joerg Roedel , Halil Pasic , "iommu@lists.linux.dev" , Nicolin Chen , Jason Gunthorpe , Christian Borntraeger , "intel-gfx@lists.freedesktop.org" , "Wang, Zhi A" , Jason Herne , Eric Farman , Vasily Gorbik , Heiko Carstens , Eric Auger , Harald Freudenberger , "Vivi, Rodrigo" , "intel-gvt-dev@lists.freedesktop.org" , Tony Krowiak , Tvrtko Ursulin , Yishai Hadas , Cornelia Huck , Peter Oberparleiter , Shameer Kolothum , Sven Schnelle , Robin Murphy , Lu Baolu Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Thu, 10 Nov 2022 06:57:57 +0000 "Tian, Kevin" wrote: > > From: Jason Gunthorpe > > Sent: Thursday, November 10, 2022 3:53 AM > > > > On Wed, Nov 09, 2022 at 10:18:09AM -0700, Alex Williamson wrote: > > > > > DPDK supports no-iommu mode. > > > > Er? Huh? How? I thought no-iommu was for applications that didn't do > > DMA? How is DPDK getting packets in/out without DMA? I guess it snoops > > in /proc/ or something to learn PFNs of mlock'd memory? > > iirc dpdk started with UIO plus various tricks (root privilege, hugepage, etc.) > to lock and learn PFN's from pagemap. Then when migrating it to vfio the > no-iommu option was introduced to provide UIO compatibility. IIRC, we essentially introduced no-iommu mode vfio because DPDK started pushing for extending interrupt support in uio-pci-generic. The UIO driver is also only meant for devices that don't do DMA, but obviously DPDK didn't care about that. Rather than extend UIO, we offered this no-iommu mode in vfio since we already had more extensive MSI support, were better able to impose restrictions on access to the device, and using the same device access makes the transition to proper IOMMU backed configurations more seamless. Thanks, Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 203F4C4332F for ; Thu, 10 Nov 2022 17:10:33 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8449910E08C; Thu, 10 Nov 2022 17:10:32 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by gabe.freedesktop.org (Postfix) with ESMTPS id DD8AB10E08C for ; Thu, 10 Nov 2022 17:10:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668100228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3XBNZVfwHOyxrKzEqgSDBaV/kDGeCsLaREPRXxkN8EM=; b=bSmu2RAUiPP+uyo1YY4ieRzl9cShG71GbLpnBxH2iT03EesUvtjp48uwtHrt5Mtdb/GbJx 3Bmk3RSSqk5MkKvhFrJaxVIH+zm8UugH4OueDyc3RVnqPvVn6Vday7YjCE1XwOVbwCt/Nc 337UpDRnVVttpGEttl8herIPp31QH1Y= Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-642-O5Ej5kgZNZOndX5aiB3FyQ-1; Thu, 10 Nov 2022 12:10:27 -0500 X-MC-Unique: O5Ej5kgZNZOndX5aiB3FyQ-1 Received: by mail-io1-f71.google.com with SMTP id f2-20020a5ec602000000b006dc67829888so1453973iok.3 for ; Thu, 10 Nov 2022 09:10:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3XBNZVfwHOyxrKzEqgSDBaV/kDGeCsLaREPRXxkN8EM=; b=cB0HhBXKTFgq3xNrXau0c0E36jJdxE8To702qj46UwmDS26cm06lpBwQ2u53v/52cZ qsZHw5BBHDRbnn5bw6pyyBQ1K/sXsuXRr+zLWb/3EJjtmmrnnHTocloaZRhQ5geDt8dD ME/305j2IiYvqWeMigo1uw1/L8ad0ZMiso02VYo0YcLtXQIqadoCgai9tv5psE+o8m+A JZ0oNF9fQ8k4GxWnZXsXLUpP14DvkurIi0TxfnLZUmOL9UgINtP3tK0i1X97nISGvBPL FZXXvBUVle5d+nfr4sw6eCSIqi9KrSfB20lUMJwCNn2lQ2SjeIToF6965aR8fOha1LiR ChbA== X-Gm-Message-State: ACrzQf0Qlmb/PDELpmIcJ7uPVrjGlUZs1GBraqjppfmqtBKxMSgjT4JK PL5TL13vSXpmlxBTQ73P+i1hHtWbPGBSaH+2ZsVlKJiNw0+fHRO6da3UmskunIRulzTZxB3eWCC KMj7ux2CwrGB27l4cPFke9mNKVE36 X-Received: by 2002:a05:6638:4413:b0:374:fbbe:2da6 with SMTP id bp19-20020a056638441300b00374fbbe2da6mr3149418jab.163.1668100226563; Thu, 10 Nov 2022 09:10:26 -0800 (PST) X-Google-Smtp-Source: AMsMyM5st2ROvi8y5IPXYSHhk18Mv2oS6m3aLQCRNeAm9L0KyqI2dN1lMnwqPxWnGmIZODwHu3sF6g== X-Received: by 2002:a05:6638:4413:b0:374:fbbe:2da6 with SMTP id bp19-20020a056638441300b00374fbbe2da6mr3149377jab.163.1668100226283; Thu, 10 Nov 2022 09:10:26 -0800 (PST) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id m1-20020a924a01000000b002f9652849f6sm30029ilf.67.2022.11.10.09.10.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Nov 2022 09:10:25 -0800 (PST) Date: Thu, 10 Nov 2022 10:10:23 -0700 From: Alex Williamson To: "Tian, Kevin" Message-ID: <20221110101023.28e7a790.alex.williamson@redhat.com> In-Reply-To: References: <0-v2-65016290f146+33e-vfio_iommufd_jgg@nvidia.com> <10-v2-65016290f146+33e-vfio_iommufd_jgg@nvidia.com> <20221108152831.1a2ed3df.alex.williamson@redhat.com> <20221109101809.2ff08303.alex.williamson@redhat.com> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.34; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [Intel-gfx] [PATCH v2 10/11] vfio: Make vfio_container optionally compiled X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "kvm@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , Vineeth Vijayan , Diana Craciun , Alexander Gordeev , David Airlie , Longfang Liu , "linux-s390@vger.kernel.org" , "Liu, Yi L" , Matthew Rosato , Will Deacon , Joerg Roedel , Halil Pasic , "iommu@lists.linux.dev" , Nicolin Chen , Jason Gunthorpe , Christian Borntraeger , "intel-gfx@lists.freedesktop.org" , Jason Herne , Eric Farman , Vasily Gorbik , Heiko Carstens , Eric Auger , Harald Freudenberger , "Vivi, Rodrigo" , "intel-gvt-dev@lists.freedesktop.org" , Tony Krowiak , Yishai Hadas , Cornelia Huck , Peter Oberparleiter , Shameer Kolothum , Sven Schnelle , Daniel Vetter , Robin Murphy , Lu Baolu Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On Thu, 10 Nov 2022 06:57:57 +0000 "Tian, Kevin" wrote: > > From: Jason Gunthorpe > > Sent: Thursday, November 10, 2022 3:53 AM > > > > On Wed, Nov 09, 2022 at 10:18:09AM -0700, Alex Williamson wrote: > > > > > DPDK supports no-iommu mode. > > > > Er? Huh? How? I thought no-iommu was for applications that didn't do > > DMA? How is DPDK getting packets in/out without DMA? I guess it snoops > > in /proc/ or something to learn PFNs of mlock'd memory? > > iirc dpdk started with UIO plus various tricks (root privilege, hugepage, etc.) > to lock and learn PFN's from pagemap. Then when migrating it to vfio the > no-iommu option was introduced to provide UIO compatibility. IIRC, we essentially introduced no-iommu mode vfio because DPDK started pushing for extending interrupt support in uio-pci-generic. The UIO driver is also only meant for devices that don't do DMA, but obviously DPDK didn't care about that. Rather than extend UIO, we offered this no-iommu mode in vfio since we already had more extensive MSI support, were better able to impose restrictions on access to the device, and using the same device access makes the transition to proper IOMMU backed configurations more seamless. Thanks, Alex