From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84CF1C55178 for ; Fri, 6 Nov 2020 23:47:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31DC720867 for ; Fri, 6 Nov 2020 23:47:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=intel-com.20150623.gappssmtp.com header.i=@intel-com.20150623.gappssmtp.com header.b="tkdRkau4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727916AbgKFXrO (ORCPT ); Fri, 6 Nov 2020 18:47:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727569AbgKFXrN (ORCPT ); Fri, 6 Nov 2020 18:47:13 -0500 Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94B46C0613D4 for ; Fri, 6 Nov 2020 15:47:13 -0800 (PST) Received: by mail-ed1-x542.google.com with SMTP id y15so2793217ede.11 for ; Fri, 06 Nov 2020 15:47:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Ld/AFos27KlKPC12X38aMGZUocKXd6SaJrBUuNVGmN0=; b=tkdRkau4k4qjSOcL+lCqzMeF+zlmlKnfIhzdIaXGkB93zPzYnQCCykbyAg5/beJ7x1 HgU3JBNk6olMLPuuM02azIWJ//sqvEgCfm3ZkRkYahGBfztBEZIuwB0a+2QXERVQpzZb LQsrQk8z7NgBJNN1hwTtGdI2NqjcbW5QMvB/MXhBjVqRCt4Lugw2PNQV574teL+vJ8Gb Rr47XEeICg3VZuwo9nltfAwQsb+AnZV93sF2wQ1ynN2NooVyHpU1Hje/5WOxFo9IPy+C EKNk1Xh660oMB9+fPeQ1LRRC/oblHCBovC7ZPkhP5ZKEZoASeAS6xILfFyPOg4Y+PM7b zEjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Ld/AFos27KlKPC12X38aMGZUocKXd6SaJrBUuNVGmN0=; b=d092luDaU6tA98L/FwqfoEq0whdQEInrAnIkqHupVMQXrSL3FKHJCperXDKZLDhmNf KMfvaPsxK+739+c3FvoVZCqlGjRp0z443Z9Knt+YAznkO+ehe1qoYIkDcCjJ2xfoV8ri Z0WiPoM3X8FEe2JznMEJbUd0Wh76EEHDWFYKzE3huF0+oSca77MSWgxLcIRPmvvbzVZ1 GlxwS/iskWtJ9nBi5oPTqHjGeCl7ibxjDDpezA54tKtwOsxHloQw91C4Rm1yrQYQC9rg 1mrix3QyD+HTiWoudRK7iFFUJhCxxGVjTs8/ZYw6/yvH8lCh62oTdIPXiJE0eGYuXpj7 Kd6Q== X-Gm-Message-State: AOAM533G4oldpt1lbepGzrzrFEbPTP9DSYhRTTFI/pFYE8T6YQ52hJPH tBzBKNegN3OWIFdOQz15t79m7268zJ26s9wdcYQXAg== X-Google-Smtp-Source: ABdhPJy/RVV3sqTqc4xE+GihNiS1eS50AdCrqwgYd+vRMib4fmp3nZlcH77SGQmU6AM1MQg85qDtVq7mK+6+ZFe4bSI= X-Received: by 2002:aa7:d843:: with SMTP id f3mr4792654eds.354.1604706432202; Fri, 06 Nov 2020 15:47:12 -0800 (PST) MIME-Version: 1.0 References: <20201102132158.GA3352700@nvidia.com> <20201103124351.GM2620339@nvidia.com> <20201104124017.GW2620339@nvidia.com> <20201104135415.GX2620339@nvidia.com> <20201106131415.GT2620339@nvidia.com> <20201106164850.GA85879@otc-nc-03> <20201106175131.GW2620339@nvidia.com> In-Reply-To: <20201106175131.GW2620339@nvidia.com> From: Dan Williams Date: Fri, 6 Nov 2020 15:47:00 -0800 Message-ID: Subject: Re: [PATCH v4 06/17] PCI: add SIOV and IMS capability detection To: Jason Gunthorpe Cc: "Raj, Ashok" , "Tian, Kevin" , "Jiang, Dave" , Bjorn Helgaas , "vkoul@kernel.org" , "Dey, Megha" , "maz@kernel.org" , "bhelgaas@google.com" , "tglx@linutronix.de" , "alex.williamson@redhat.com" , "Pan, Jacob jun" , "Liu, Yi L" , "Lu, Baolu" , "Kumar, Sanjay K" , "Luck, Tony" , "jing.lin@intel.com" , "kwankhede@nvidia.com" , "eric.auger@redhat.com" , "parav@mellanox.com" , "rafael@kernel.org" , "netanelg@mellanox.com" , "shahafs@mellanox.com" , "yan.y.zhao@linux.intel.com" , "pbonzini@redhat.com" , "Ortiz, Samuel" , "Hossain, Mona" , "dmaengine@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-pci@vger.kernel.org" , "kvm@vger.kernel.org" Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org On Fri, Nov 6, 2020 at 9:51 AM Jason Gunthorpe wrote: [..] > > This is true for IMS as well. But probably not implemented in the kernel as > > such. From a HW point of view (take idxd for instance) the facility is > > available to native OS as well. The early RFC supported this for native. > > I can't follow what you are trying to say here. I'm having a hard time following the technical cruxes of this debate. I grokked your feedback on the original IMS proposal way back at the beginning of this effort (pre-COVID even!), so maybe I can mediate here as well. Although, SIOV is that much harder for me to spell than IMS, so bear with me. > Dave said the IMS cap was to indicate that the VMM supported emulation > of IMS so that the VMM can do the MSI addr/data translation as part of > the emulation. > > I'm saying emulation will be too horrible for our devices that don't > require *any* emulation. This part I think I understand, i.e. why spend any logic emulating IMS as MSI since the IMS capability can be a paravirtualized interface from guest to VMM with none of the compromises that MSI would enforce. Did I get that right? > It is a bad architecture. The platform needs to handle this globally > for all devices, not special hacky emulations things custom made for > every device out there. I confess I don't quite understand the shape of what "platform needs to handle this globally" means, but I understand the desired end result of "no emulation added where not needed". However, would this mean that the bare-metal idxd driver can not be used directly in the guest without modification? For example, as I understand from talking to Ashok, idxd has some device events like error notification hard wired to MSI while data patch interrupts are IMS. So even if the IMS side does not hook up MSI emulation doesn't idxd still need MSI emulation to reuse the bare metal driver directly? > > Native devices can have both MSIx and IMS capability. But as I > > understand this isn't how we have partitioned things in SW today. We > > left IMS only for mdev's. And I agree this would be very useful. > > That split is just some decision idxd did, we are thinking about doing > other things in our devices. Where does the collision happen between what you need for a clean implementation of an IMS-like capability (/me misses his "dev-msi" name that got thrown out in the Thomas rewrite), and emulation needed to not have VF special casing in the idxd driver. Also feel free to straighten me out (Jason or Ashok) if I've botched the understanding of this.