From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2364C432C0 for ; Wed, 20 Nov 2019 01:46:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 82F2722440 for ; Wed, 20 Nov 2019 01:46:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="HCjWIOdd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727082AbfKTBq4 (ORCPT ); Tue, 19 Nov 2019 20:46:56 -0500 Received: from mail-qk1-f196.google.com ([209.85.222.196]:38635 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727264AbfKTBq4 (ORCPT ); Tue, 19 Nov 2019 20:46:56 -0500 Received: by mail-qk1-f196.google.com with SMTP id e2so19871472qkn.5 for ; Tue, 19 Nov 2019 17:46:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=5+9Fvs2hHnzXGmSYo5rNZCdiEPrUh/cYU0evbRShhoY=; b=HCjWIOddtnoHmPWRf4FzneGGvTFt4lt0Qnwga3HxP9l9r/953SslWwymUKuceNII1r C8Kxob0b9+5Sjz0axlf4AS/Uc63tSVTvJ1UjuW5tulQheCsX0XvegdE8AR6qW7lZGtju s65UneWZvkeVDcahTklL0F8vASvaxvsLqq+zQGNvFkEFhmhaq5iJSeXfPapYw11HTuCE 3I5OJMBEV/nDeSpfY8MmkQwqsIFet2Jk4Ex8zMkUQY4JQylT737h9l/BJzRfyjtIrG4D Ptf+fgNplP2jFar6Zim8tZBH1+jvsIcfukVoMcQwthWb7HMUJQIm63MNRosLysAUHddb JYHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=5+9Fvs2hHnzXGmSYo5rNZCdiEPrUh/cYU0evbRShhoY=; b=W/z1XUB2qRJdFJiRg0FnCY+UtnNZY5PytYT9Bx2O5O0WhHwUBI0ULkz8BRR2RT+zkO 67leeK8dr77YjB982P2HX89hxFZgiE5R3qek8kdD1FOC8/H7GVeA82VPwhsKuXrERqBC pnhn9+6B7+VbQqA7Dm8KOkowVuVu44DWHOJMokRaV74du78mPxOlqa52WJM6cMYjuAom f/3Eblyg14qDp26DVLCpApZBUG0GkGUraSwPjnB1XNTwD71iX6QjCyN9av9RR6ZQ7Qqz GI9rEbvVImcDdrJG0HyzzrL1DLmmbEdaX01g0aOqT6/WHp/nsOlscDyZsQLkps4y2ZCX CY3A== X-Gm-Message-State: APjAAAVdxMIwt7WgeOHtLS90+KJzdQLVcUbgyAC2UrOpqVC+A8m3dUYN aGDN4gSbdNjq/qqRr1Q0PoXGWw== X-Google-Smtp-Source: APXvYqx8E56BaWuEeGzlaFj2QPwWrGbJ2KjCH4Yd7AqNKfpF9dUI7xm0d+mnVrpQXXqyTrxGBZV10g== X-Received: by 2002:a37:f605:: with SMTP id y5mr270283qkj.288.1574214415103; Tue, 19 Nov 2019 17:46:55 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-113-180.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.113.180]) by smtp.gmail.com with ESMTPSA id k27sm10562991qkj.30.2019.11.19.17.46.54 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 19 Nov 2019 17:46:54 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1iXF53-0003Yu-SS; Tue, 19 Nov 2019 21:46:53 -0400 Date: Tue, 19 Nov 2019 21:46:53 -0400 From: Jason Gunthorpe To: "Michael S. Tsirkin" Cc: Jason Wang , Parav Pandit , Jeff Kirsher , "davem@davemloft.net" , "gregkh@linuxfoundation.org" , Dave Ertman , "netdev@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "nhorman@redhat.com" , "sassmann@redhat.com" , Kiran Patil , Alex Williamson , "Bie, Tiwei" Subject: Re: [net-next v2 1/1] virtual-bus: Implementation of Virtual Bus Message-ID: <20191120014653.GR4991@ziepe.ca> References: <13946106-dab2-6bbe-df79-ca6dfdeb4c51@redhat.com> <20191119164632.GA4991@ziepe.ca> <20191119134822-mutt-send-email-mst@kernel.org> <20191119191547.GL4991@ziepe.ca> <20191119163147-mutt-send-email-mst@kernel.org> <20191119231023.GN4991@ziepe.ca> <20191119191053-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191119191053-mutt-send-email-mst@kernel.org> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On Tue, Nov 19, 2019 at 07:16:21PM -0500, Michael S. Tsirkin wrote: > On Tue, Nov 19, 2019 at 07:10:23PM -0400, Jason Gunthorpe wrote: > > On Tue, Nov 19, 2019 at 04:33:40PM -0500, Michael S. Tsirkin wrote: > > > On Tue, Nov 19, 2019 at 03:15:47PM -0400, Jason Gunthorpe wrote: > > > > On Tue, Nov 19, 2019 at 01:58:42PM -0500, Michael S. Tsirkin wrote: > > > > > On Tue, Nov 19, 2019 at 12:46:32PM -0400, Jason Gunthorpe wrote: > > > > > > As always, this is all very hard to tell without actually seeing real > > > > > > accelerated drivers implement this. > > > > > > > > > > > > Your patch series might be a bit premature in this regard. > > > > > > > > > > Actually drivers implementing this have been posted, haven't they? > > > > > See e.g. https://lwn.net/Articles/804379/ > > > > > > > > Is that a real driver? It looks like another example quality > > > > thing. > > > > > > > > For instance why do we need any of this if it has '#define > > > > IFCVF_MDEV_LIMIT 1' ? > > > > > > > > Surely for this HW just use vfio over the entire PCI function and be > > > > done with it? > > > > > > What this does is allow using it with unmodified virtio drivers > > > within guests. You won't get this with passthrough as it only > > > implements parts of virtio in hardware. > > > > I don't mean use vfio to perform passthrough, I mean to use vfio to > > implement the software parts in userspace while vfio to talk to the > > hardware. > > You repeated vfio twice here, hard to decode what you meant actually. 'while using vifo to talk to the hardware' > > kernel -> vfio -> user space virtio driver -> qemu -> guest > > Exactly what has been implemented for control path. I do not mean the modified mediated vfio this series proposes, I mean vfio-pci, on a full PCI VF, exactly like we have today. > The interface between vfio and userspace is > based on virtio which is IMHO much better than > a vendor specific one. userspace stays vendor agnostic. Why is that even a good thing? It is much easier to provide drivers via qemu/etc in user space then it is to make kernel upgrades. We've learned this lesson many times. This is why we have had the philosophy that if it doesn't need to be in the kernel it should be in userspace. > > Generally we don't want to see things in the kernel that can be done > > in userspace, and to me, at least for this driver, this looks > > completely solvable in userspace. > > I don't think that extends as far as actively encouraging userspace > drivers poking at hardware in a vendor specific way. Yes, it does, if you can implement your user space requirements using vfio then why do you need a kernel driver? The kernel needs to be involved when there are things only the kernel can do. If IFC has such things they should be spelled out to justify using a mediated device. > That has lots of security and portability implications and isn't > appropriate for everyone. This is already using vfio. It doesn't make sense to claim that using vfio properly is somehow less secure or less portable. What I find particularly ugly is that this 'IFC VF NIC' driver pretends to be a mediated vfio device, but actually bypasses all the mediated device ops for managing dma security and just directly plugs the system IOMMU for the underlying PCI device into vfio. I suppose this little hack is what is motivating this abuse of vfio in the first place? Frankly I think a kernel driver touching a PCI function for which vfio is now controlling the system iommu for is a violation of the security model, and I'm very surprised AlexW didn't NAK this idea. Perhaps it is because none of the patches actually describe how the DMA security model for this so-called mediated device works? :( Or perhaps it is because this submission is split up so much it is hard to see what is being proposed? (I note this IFC driver is the first user of the mdev_set_iommu_device() function) > It is kernel's job to abstract hardware away and present a unified > interface as far as possible. Sure, you could create a virtio accelerator driver framework in our new drivers/accel I hear was started. That could make some sense, if we had HW that actually required/benefited from kernel involvement. Jason