From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0124FC47258 for ; Mon, 4 May 2020 00:20:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DEEF720735 for ; Mon, 4 May 2020 00:20:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726630AbgEDAUn (ORCPT ); Sun, 3 May 2020 20:20:43 -0400 Received: from mga09.intel.com ([134.134.136.24]:12909 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726415AbgEDAUn (ORCPT ); Sun, 3 May 2020 20:20:43 -0400 IronPort-SDR: /44pAGFr3PEIgO8WOC2a5Hc3VGa4vtr6QMqumRj92P52LdFJFTBU9AA2m7dCmKTdQNYtEZdp6v kiy0I6MkTZ8A== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 May 2020 17:20:42 -0700 IronPort-SDR: /mMz6XCW3lxY5mOEFkrRZiB5zjEgRMPKeWRLpxwqsJAwdgTcPPtFnmz7qtbDpLxM761Ed1asw9 lGsTKmK054Og== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,349,1583222400"; d="scan'208";a="406316584" Received: from meghadey-mobl1.amr.corp.intel.com (HELO [10.212.197.87]) ([10.212.197.87]) by orsmga004.jf.intel.com with ESMTP; 03 May 2020 17:20:41 -0700 Subject: Re: [PATCH RFC 00/15] Add VFIO mediated device support and IMS support for the idxd driver. To: Jason Gunthorpe Cc: Dan Williams , Dave Jiang , Vinod Koul , maz@kernel.org, Bjorn Helgaas , "Rafael J. Wysocki" , Greg KH , Thomas Gleixner , "H. Peter Anvin" , Alex Williamson , Jacob jun Pan , "Raj, Ashok" , Yi L Liu , baolu.lu@intel.com, "Tian, Kevin" , Sanjay K Kumar , "Luck, Tony" , Jing Lin , kwankhede@nvidia.com, eric.auger@redhat.com, parav@mellanox.com, dmaengine@vger.kernel.org, Linux Kernel Mailing List , X86 ML , linux-pci@vger.kernel.org, KVM list References: <158751095889.36773.6009825070990637468.stgit@djiang5-desk3.ch.intel.com> <20200421235442.GO11945@mellanox.com> <20200423191846.GE13640@mellanox.com> <098aef60-35a4-dc44-be07-ea43c1a726c7@linux.intel.com> <20200503222229.GE19158@mellanox.com> <5bc05b74-536f-f72d-c406-18644436f11b@linux.intel.com> <20200503223618.GG19158@mellanox.com> From: "Dey, Megha" Message-ID: <913537f9-5b5a-2334-f3cf-a8417d5208dd@linux.intel.com> Date: Sun, 3 May 2020 17:20:41 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20200503223618.GG19158@mellanox.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org On 5/3/2020 3:36 PM, Jason Gunthorpe wrote: > On Sun, May 03, 2020 at 03:31:39PM -0700, Dey, Megha wrote: >> >> Hi Jason, >> >> On 5/3/2020 3:22 PM, Jason Gunthorpe wrote: >>> On Fri, May 01, 2020 at 03:31:51PM -0700, Dey, Megha wrote: >>>>>> This has been my concern reviewing the implementation. IMS needs more >>>>>> than one in-tree user to validate degrees of freedom in the api. I had >>>>>> been missing a second "in-tree user" to validate the scope of the >>>>>> flexibility that was needed. >>>>> >>>>> IMS is too narrowly specified. >>>>> >>>>> All platforms that support MSI today can support IMS. It is simply a >>>>> way for the platform to give the driver an addr/data pair that triggers >>>>> an interrupt when a posted write is performed to that pair. >>>>> >>>> >>>> Well, yes and no. IMS requires interrupt remapping in addition to the >>>> dynamic nature of IRQ allocation. >>> >>> You've mentioned remapping a few times, but I really can't understand >>> why it has anything to do with platform_msi or IMS.. >> >> So after some internal discussions, we have concluded that IMS has no >> linkage with Interrupt remapping, IR is just a platform concept. IMS is just >> a name Intel came up with, all it really means is device managed addr/data >> writes to generate interrupts. Technically we can call something IMS even if >> device has its own location to store interrupts in non-pci standard >> mechanism, much like platform-msi indeed. We simply need to extend >> platform-msi to its address some of its shortcomings: increase number of >> interrupts to > 2048, enable dynamic allocation of interrupts, add >> mask/unmask callbacks in addition to write_msg etc. > > Sounds right to me > > Persumably you still need a way for the driver, eg vfio, to ensure a > MSI is remappable, but shouldn't that be exactly the same way as done > in normal PCI MSI today? yes exactly, it should be done in the same way as PCI-MSI, if IR is enabled we will have IR_PCI_MSI for platform msi as well. > >> FWIW, even MSI can be IMS with rules on how to manage the addr/data writes >> following pci sig .. its just that. > > Yep, IMHO, our whole handling of MSI is very un-general sometimes.. > > I thought the msi_domain stuff that some platforms are using is a way > to improve on that? You might find that updating x86 to use msi_domain > might be helpful in this project??? yes, we need to take a closer look at this. > > Jason >