From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 182DEC432C0 for ; Wed, 20 Nov 2019 16:45:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D750D2088F for ; Wed, 20 Nov 2019 16:45:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="jN7A5asv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729937AbfKTQp2 (ORCPT ); Wed, 20 Nov 2019 11:45:28 -0500 Received: from mail-qt1-f193.google.com ([209.85.160.193]:39067 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727696AbfKTQp2 (ORCPT ); Wed, 20 Nov 2019 11:45:28 -0500 Received: by mail-qt1-f193.google.com with SMTP id t8so221023qtc.6 for ; Wed, 20 Nov 2019 08:45:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=q1Cjbt1crzEAWmbqF8+hcYqt5jaBaZcCfokyHmlm+ZE=; b=jN7A5asvd6GYVcpqiyvx4PrQ1taaPGc3wWST/UZf8modwzUdyuMV0Ldym4k2MugPls HQGQEGZW00LzrXy5HPuVBjwRuFbWfU05hdBTEbZToMwdUTNdTtQVj9m9puaZqvHH9X+G PpYEvEeWGvR0B2vA2WQxgPP/FveyyT1eaa0ztp0oUEspEXqYKgckX4NMcMjP96niJEAp jyHu2bSeQYhrFisDhP3b6+zkyFisrMbchDyzNS/kCGNg18LHkBSiK3UDoRnjeLP+mec5 UBzUGa3NKhNWggwMX1DpWnhkqtejG3/1lW4JohsNw9UuCU9O0kfLL5EZhGqUyRDFoFtN MpIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=q1Cjbt1crzEAWmbqF8+hcYqt5jaBaZcCfokyHmlm+ZE=; b=PBnPpHVSDDgD8ZQfysqNs+0W4oIjvrvDCEdyPMGN3qRYZn8Xp1bDbxC/jGMKQxkR0o O+uxBEGDgR/y6PxwV4sM54JsxXukMAP2Rt8T/TlcAKqbt/wVtBNBC1t0veK+KtXacmgN wtxEewpiUEkgAYhbFQuYR4E2CN2pEgFCI4MaRZM1YacyuHZU0gjyBqh0UMiTWTJqcFOV S9W5NBIzW6ULfnArmM1A60+7f37Y6v3ohxShIUApL+/7DpFDB2VtNzCn0YhhM78KkM6b VJyeLRUHDJ2g6PyEo9W86LRh9d8lwGWHISc7f8rSIDP0mYlJNR+VZI3xAvwdLl9ArtMH kMHA== X-Gm-Message-State: APjAAAWzu8hAYj5eXggSpSWTkj1UFRsU/2WV+zk5+qZeQgSHE8jCmzhx 1gpWFhcHaLJtejPP7ZgwHar+4g== X-Google-Smtp-Source: APXvYqwbHsjBsBVLwFNKQge8I0L4IlOfNnRWURPlA6eW2epxh/y9TADVEeZ/MKxrIQtLu9b5PHBosA== X-Received: by 2002:ac8:641:: with SMTP id e1mr3584184qth.319.1574268326456; Wed, 20 Nov 2019 08:45:26 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-113-180.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.113.180]) by smtp.gmail.com with ESMTPSA id j89sm14324797qte.72.2019.11.20.08.45.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Nov 2019 08:45:25 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1iXT6b-0007oA-2a; Wed, 20 Nov 2019 12:45:25 -0400 Date: Wed, 20 Nov 2019 12:45:25 -0400 From: Jason Gunthorpe To: "Michael S. Tsirkin" Cc: Jason Wang , Parav Pandit , Jeff Kirsher , "davem@davemloft.net" , "gregkh@linuxfoundation.org" , Dave Ertman , "netdev@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "nhorman@redhat.com" , "sassmann@redhat.com" , Kiran Patil , Alex Williamson , "Bie, Tiwei" Subject: Re: [net-next v2 1/1] virtual-bus: Implementation of Virtual Bus Message-ID: <20191120164525.GH22515@ziepe.ca> References: <20191119191547.GL4991@ziepe.ca> <20191119163147-mutt-send-email-mst@kernel.org> <20191119231023.GN4991@ziepe.ca> <20191119191053-mutt-send-email-mst@kernel.org> <20191120014653.GR4991@ziepe.ca> <20191120022141-mutt-send-email-mst@kernel.org> <20191120130319.GA22515@ziepe.ca> <20191120083908-mutt-send-email-mst@kernel.org> <20191120143054.GF22515@ziepe.ca> <20191120093607-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191120093607-mutt-send-email-mst@kernel.org> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On Wed, Nov 20, 2019 at 09:57:17AM -0500, Michael S. Tsirkin wrote: > On Wed, Nov 20, 2019 at 10:30:54AM -0400, Jason Gunthorpe wrote: > > On Wed, Nov 20, 2019 at 08:43:20AM -0500, Michael S. Tsirkin wrote: > > > On Wed, Nov 20, 2019 at 09:03:19AM -0400, Jason Gunthorpe wrote: > > > > On Wed, Nov 20, 2019 at 02:38:08AM -0500, Michael S. Tsirkin wrote: > > > > > > > I don't think that extends as far as actively encouraging userspace > > > > > > > drivers poking at hardware in a vendor specific way. > > > > > > > > > > > > Yes, it does, if you can implement your user space requirements using > > > > > > vfio then why do you need a kernel driver? > > > > > > > > > > People's requirements differ. You are happy with just pass through a VF > > > > > you can already use it. Case closed. There are enough people who have > > > > > a fixed userspace that people have built virtio accelerators, > > > > > now there's value in supporting that, and a vendor specific > > > > > userspace blob is not supporting that requirement. > > > > > > > > I have no idea what you are trying to explain here. I'm not advocating > > > > for vfio pass through. > > > > > > You seem to come from an RDMA background, used to userspace linking to > > > vendor libraries to do basic things like push bits out on the network, > > > because users live on the performance edge and rebuild their > > > userspace often anyway. > > > > > > Lots of people are not like that, they would rather have the > > > vendor-specific driver live in the kernel, with userspace being > > > portable, thank you very much. > > > > You are actually proposing a very RDMA like approach with a split > > kernel/user driver design. Maybe the virtio user driver will turn out > > to be 'portable'. > > > > Based on the last 20 years of experience, the kernel component has > > proven to be the larger burden and drag than the userspace part. I > > think the high interest in DPDK, SPDK and others show this is a common > > principle. > > And I guess the interest in BPF shows the opposite? There is room for both, I wouldn't discount either approach entirely out of hand. > > At the very least for new approaches like this it makes alot of sense > > to have a user space driver until enough HW is available that a > > proper, well thought out kernel side can be built. > > But hardware is available, driver has been posted by Intel. > Have you looked at that? I'm not sure pointing at that driver is so helpful, it is very small and mostly just reflects virtio ops into some undocumented register pokes. There is no explanation at all for the large scale architecture choices: - Why vfio - Why mdev without providing a device IOMMU - Why use GUID lifecycle management for singlton function PF/VF drivers - Why not use devlink - Why not use vfio-pci with a userspace driver These are legitimate questions and answers like "because we like it this way" or "this is how the drivers are written today" isn't very satisfying at all. > > For instance, this VFIO based approach might be very suitable to the > > intel VF based ICF driver, but we don't yet have an example of non-VF > > HW that might not be well suited to VFIO. > > I don't think we should keep moving the goalposts like this. It is ABI, it should be done as best we can as we have to live with it for a long time. Right now HW is just starting to come to market with VDPA and it feels rushed to design a whole subsystem style ABI around one, quite simplistic, driver example. > If people write drivers and find some infrastruture useful, > and it looks more or less generic on the outset, then I don't > see why it's a bad idea to merge it. Because it is userspace ABI, caution is always justified when defining new ABI. Jason