From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A8B7C4338F for ; Tue, 10 Aug 2021 13:57:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4F82E606A5 for ; Tue, 10 Aug 2021 13:57:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242113AbhHJN5c (ORCPT ); Tue, 10 Aug 2021 09:57:32 -0400 Received: from mail.kernel.org ([198.145.29.99]:35096 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238296AbhHJN5a (ORCPT ); Tue, 10 Aug 2021 09:57:30 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9812160295; Tue, 10 Aug 2021 13:57:08 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mDSFe-0044bF-L3; Tue, 10 Aug 2021 14:57:06 +0100 Date: Tue, 10 Aug 2021 14:57:06 +0100 Message-ID: <8735rhbdxp.wl-maz@kernel.org> From: Marc Zyngier To: Sunil Muthuswamy Cc: Robin Murphy , Thomas Gleixner , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "catalin.marinas@arm.com" , "will@kernel.org" , Michael Kelley , Boqun Feng , KY Srinivasan , Arnd Bergmann , Lorenzo Pieralisi Subject: Re: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS In-Reply-To: References: <87a6mt2jke.wl-maz@kernel.org> <87tuka24kj.wl-maz@kernel.org> <87r1f9wooc.wl-maz@kernel.org> <87wnp0b86y.wl-maz@kernel.org> <87o8a81boi.wl-maz@kernel.org> <87pmunasik.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: sunilmut@microsoft.com, robin.murphy@arm.com, tglx@linutronix.de, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org, mikelley@microsoft.com, Boqun.Feng@microsoft.com, kys@microsoft.com, arnd@arndb.de, lorenzo.pieralisi@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 10 Aug 2021 02:10:40 +0100, Sunil Muthuswamy wrote: > > On Monday, August 9, 2021 2:15 AM, > Marc Zyngier wrote: > [...] > > If you plug directly into the GICv3 layer, I'd rather you inject SPIs, > > just like any other non-architectural MSI controller. You can directly > > interface with the ACPI GSI layer for that, without any need to mess > > with the GICv3 internals. The SPI space isn't very large, but still > > much larger than the equivalent x86 space (close to 1000). > > > > If time is of the essence, I suggest you go the SPI way. For anything > > involving LPIs, I really want to see a firmware spec that works for > > everyone as opposed to a localised Hyper-V hack. > > > Ok, thanks. Before we commit to anything, I would like to make sure > that I am on the same page in terms of your description. With that in > mind, I have few questions. Hopefully, these should settle the matter. > 1. If we go with the SPI route, then the way I envision it is that the > Hyper-V vPCI driver will implement an IRQ chip, which will take > care of allocating & managing the SPI interrupt for Hyper-V vPCI. > This IRQ chip will parent itself to the architectural GIC IRQ chip for > general interrupt management. Does that match with your > understanding/suggestion as well? Yes. > > 2. In the above, how will Hyper-V vPCI module discover the > architectural GIC IRQ domain generically for virtual devices that > are not firmware enumerated? Today, the GIC v3 IRQ domain is > not exported and the general 'irq_find_xyz' APIs only work for > firmware enumerated devices (i.e. something that has a fwnode > handle). You don't need to discover it with ACPI. You simply instantiate your own irqdomain using acpi_irq_create_hierarchy(), which will do the right thing. Your PCI driver will have to create its own fwnode out of thin air (there is an API for that), and call into this function to plumb everything. > > 3. Longer term, if we implement LPIs (with an ITS or Direct LPI), to > be able to support all scenarios such as Live Migration, the > Hyper-V virtual PCI driver would like to be able to control the > MSI address & data that gets programmed on the device > (i.e. .irq_compose_msi_msg). We can use the architectural > methods for everything else. Does that fit into the realm of > what would be acceptable upstream? I cannot see how this works. The address has to match that of the virtual HW you target (whether this is a redistributor or an ITS), and the data is only meaningful in that context. And it really shouldn't matter at all, as I expect you don't let the guest directly write to the PCI MSI-X table. If you let the guest have access direct to that table (which seems to contradict your "live migration" argument), then your best bet is to use provide a skeletal IOMMU implementation, and get iommu_dma_compose_msi_msg() to do the remapping. But frankly, that's horrible and I fully expect the IOMMU people to push back (and that still doesn't give you any control over the data, only the address). Thanks, M. -- Without deviation from the norm, progress is not possible.