From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7642BC2D0A3 for ; Fri, 6 Nov 2020 23:47:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2369420882 for ; Fri, 6 Nov 2020 23:47:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=intel-com.20150623.gappssmtp.com header.i=@intel-com.20150623.gappssmtp.com header.b="tkdRkau4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726880AbgKFXrO (ORCPT ); Fri, 6 Nov 2020 18:47:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727247AbgKFXrN (ORCPT ); Fri, 6 Nov 2020 18:47:13 -0500 Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82031C0613D2 for ; Fri, 6 Nov 2020 15:47:13 -0800 (PST) Received: by mail-ed1-x542.google.com with SMTP id v4so2968853edi.0 for ; Fri, 06 Nov 2020 15:47:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Ld/AFos27KlKPC12X38aMGZUocKXd6SaJrBUuNVGmN0=; b=tkdRkau4k4qjSOcL+lCqzMeF+zlmlKnfIhzdIaXGkB93zPzYnQCCykbyAg5/beJ7x1 HgU3JBNk6olMLPuuM02azIWJ//sqvEgCfm3ZkRkYahGBfztBEZIuwB0a+2QXERVQpzZb LQsrQk8z7NgBJNN1hwTtGdI2NqjcbW5QMvB/MXhBjVqRCt4Lugw2PNQV574teL+vJ8Gb Rr47XEeICg3VZuwo9nltfAwQsb+AnZV93sF2wQ1ynN2NooVyHpU1Hje/5WOxFo9IPy+C EKNk1Xh660oMB9+fPeQ1LRRC/oblHCBovC7ZPkhP5ZKEZoASeAS6xILfFyPOg4Y+PM7b zEjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Ld/AFos27KlKPC12X38aMGZUocKXd6SaJrBUuNVGmN0=; b=COpSeAfNihs+csl71qP3iV8wYp+CUYWyNjK6BaLg4dzaYzhyZEk3l6uGxljKCf+rmP A7o8pwAMDaL6Z3U0i41ak1uuYAQTZOOlxW+0qK3LmgtFd2hs3i4TmyBaEcPxqpGcJBar Ac8+roW4c0E9BeLtY0lwelTzP4Gv4bm80AYVvtyaJTxkjY+ngP3Zh0h0uhlQVf9Y9qky +Dkk0W+X/ml2sM3jergRqvHbx6b3FI+tYJJTZZTgQWcoPBZcq5lL0dciSns2vZwVmZQQ 4727yQo9mEz9TxCieiFg81pxhIN3m/zUtGF/GC4GoMMe8a57djqVj6hAVh7B3pOi1aRB N9uA== X-Gm-Message-State: AOAM533jTy+NP9xZ/QO53ug8xatnA6mnSU/bZK1SC7CEeAkDvSxXpYPP WMWLJdXoK5z4mpVPYCU1pu+XjZzJ2H+7Ho2eGfGgSw== X-Google-Smtp-Source: ABdhPJy/RVV3sqTqc4xE+GihNiS1eS50AdCrqwgYd+vRMib4fmp3nZlcH77SGQmU6AM1MQg85qDtVq7mK+6+ZFe4bSI= X-Received: by 2002:aa7:d843:: with SMTP id f3mr4792654eds.354.1604706432202; Fri, 06 Nov 2020 15:47:12 -0800 (PST) MIME-Version: 1.0 References: <20201102132158.GA3352700@nvidia.com> <20201103124351.GM2620339@nvidia.com> <20201104124017.GW2620339@nvidia.com> <20201104135415.GX2620339@nvidia.com> <20201106131415.GT2620339@nvidia.com> <20201106164850.GA85879@otc-nc-03> <20201106175131.GW2620339@nvidia.com> In-Reply-To: <20201106175131.GW2620339@nvidia.com> From: Dan Williams Date: Fri, 6 Nov 2020 15:47:00 -0800 Message-ID: Subject: Re: [PATCH v4 06/17] PCI: add SIOV and IMS capability detection To: Jason Gunthorpe Cc: "Raj, Ashok" , "Tian, Kevin" , "Jiang, Dave" , Bjorn Helgaas , "vkoul@kernel.org" , "Dey, Megha" , "maz@kernel.org" , "bhelgaas@google.com" , "tglx@linutronix.de" , "alex.williamson@redhat.com" , "Pan, Jacob jun" , "Liu, Yi L" , "Lu, Baolu" , "Kumar, Sanjay K" , "Luck, Tony" , "jing.lin@intel.com" , "kwankhede@nvidia.com" , "eric.auger@redhat.com" , "parav@mellanox.com" , "rafael@kernel.org" , "netanelg@mellanox.com" , "shahafs@mellanox.com" , "yan.y.zhao@linux.intel.com" , "pbonzini@redhat.com" , "Ortiz, Samuel" , "Hossain, Mona" , "dmaengine@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-pci@vger.kernel.org" , "kvm@vger.kernel.org" Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Fri, Nov 6, 2020 at 9:51 AM Jason Gunthorpe wrote: [..] > > This is true for IMS as well. But probably not implemented in the kernel as > > such. From a HW point of view (take idxd for instance) the facility is > > available to native OS as well. The early RFC supported this for native. > > I can't follow what you are trying to say here. I'm having a hard time following the technical cruxes of this debate. I grokked your feedback on the original IMS proposal way back at the beginning of this effort (pre-COVID even!), so maybe I can mediate here as well. Although, SIOV is that much harder for me to spell than IMS, so bear with me. > Dave said the IMS cap was to indicate that the VMM supported emulation > of IMS so that the VMM can do the MSI addr/data translation as part of > the emulation. > > I'm saying emulation will be too horrible for our devices that don't > require *any* emulation. This part I think I understand, i.e. why spend any logic emulating IMS as MSI since the IMS capability can be a paravirtualized interface from guest to VMM with none of the compromises that MSI would enforce. Did I get that right? > It is a bad architecture. The platform needs to handle this globally > for all devices, not special hacky emulations things custom made for > every device out there. I confess I don't quite understand the shape of what "platform needs to handle this globally" means, but I understand the desired end result of "no emulation added where not needed". However, would this mean that the bare-metal idxd driver can not be used directly in the guest without modification? For example, as I understand from talking to Ashok, idxd has some device events like error notification hard wired to MSI while data patch interrupts are IMS. So even if the IMS side does not hook up MSI emulation doesn't idxd still need MSI emulation to reuse the bare metal driver directly? > > Native devices can have both MSIx and IMS capability. But as I > > understand this isn't how we have partitioned things in SW today. We > > left IMS only for mdev's. And I agree this would be very useful. > > That split is just some decision idxd did, we are thinking about doing > other things in our devices. Where does the collision happen between what you need for a clean implementation of an IMS-like capability (/me misses his "dev-msi" name that got thrown out in the Thomas rewrite), and emulation needed to not have VF special casing in the idxd driver. Also feel free to straighten me out (Jason or Ashok) if I've botched the understanding of this.