From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29F72C433F5 for ; Tue, 16 Nov 2021 08:33:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 038F761423 for ; Tue, 16 Nov 2021 08:33:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232261AbhKPIgm convert rfc822-to-8bit (ORCPT ); Tue, 16 Nov 2021 03:36:42 -0500 Received: from mail.kernel.org ([198.145.29.99]:41978 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232040AbhKPIgl (ORCPT ); Tue, 16 Nov 2021 03:36:41 -0500 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9AAB861104; Tue, 16 Nov 2021 08:33:44 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mmtuQ-005mYX-D0; Tue, 16 Nov 2021 08:33:42 +0000 Date: Tue, 16 Nov 2021 08:33:42 +0000 Message-ID: <87a6i48pp5.wl-maz@kernel.org> From: Marc Zyngier To: =?UTF-8?B?Q8OpZHJpYw==?= Le Goater Cc: , Thomas Gleixner , Michael Ellerman , PowerPC , Greg Kurz Subject: Re: [PATCH 16/39] irqdomain: Make normal and nomap irqdomains exclusive In-Reply-To: <1fe9d629-0f5f-4807-b97c-77b3b3c7de72@kaod.org> References: <20210520163751.27325-1-maz@kernel.org> <20210520163751.27325-17-maz@kernel.org> <1fe9d629-0f5f-4807-b97c-77b3b3c7de72@kaod.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: clg@kaod.org, linux-kernel@vger.kernel.org, tglx@linutronix.de, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, groug@kaod.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 15 Nov 2021 19:05:17 +0000, Cédric Le Goater wrote: > > Hello Mark, s/k/c/ > > On 5/20/21 18:37, Marc Zyngier wrote: > > Direct mappings are completely exclusive of normal mappings, meaning > > that we can refactor the code slightly so that we can get rid of > > the revmap_direct_max_irq field and use the revmap_size field > > instead, reducing the size of the irqdomain structure. > > > > Signed-off-by: Marc Zyngier > > > This patch is breaking the POWER9/POWER10 XIVE driver (these are not > old PPC systems :) on machines sharing the same LSI HW IRQ. For instance, > a linux KVM guest with a virtio-rng and a virtio-balloon device. In that > case, Linux creates two distinct IRQ mappings which can lead to some > unexpected behavior. Either the irq domain translates, or it doesn't. If the driver creates a nomap domain, and yet expects some sort of translation to happen, then the driver is fundamentally broken. And even without that: how do you end-up with a single HW interrupt having two mappings? > A fix to go forward would be to change the XIVE IRQ domain to use a > 'Tree' domain for reverse mapping and not the 'No Map' domain mapping. > I will keep you updated for XIVE. I bet there is a bit more to it. From what you are saying above, something rather ungodly is happening in the XIVE code. M. -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 763B9C433EF for ; Tue, 16 Nov 2021 08:34:19 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC7AB61504 for ; Tue, 16 Nov 2021 08:34:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org AC7AB61504 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HtfVc6SsKz307L for ; Tue, 16 Nov 2021 19:34:16 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=198.145.29.99; helo=mail.kernel.org; envelope-from=maz@kernel.org; receiver=) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HtfV345Lsz2xCN for ; Tue, 16 Nov 2021 19:33:47 +1100 (AEDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9AAB861104; Tue, 16 Nov 2021 08:33:44 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mmtuQ-005mYX-D0; Tue, 16 Nov 2021 08:33:42 +0000 Date: Tue, 16 Nov 2021 08:33:42 +0000 Message-ID: <87a6i48pp5.wl-maz@kernel.org> From: Marc Zyngier To: =?UTF-8?B?Q8OpZHJpYw==?= Le Goater Subject: Re: [PATCH 16/39] irqdomain: Make normal and nomap irqdomains exclusive In-Reply-To: <1fe9d629-0f5f-4807-b97c-77b3b3c7de72@kaod.org> References: <20210520163751.27325-1-maz@kernel.org> <20210520163751.27325-17-maz@kernel.org> <1fe9d629-0f5f-4807-b97c-77b3b3c7de72@kaod.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: clg@kaod.org, linux-kernel@vger.kernel.org, tglx@linutronix.de, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, groug@kaod.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Gleixner , PowerPC , linux-kernel@vger.kernel.org, Greg Kurz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Mon, 15 Nov 2021 19:05:17 +0000, C=C3=A9dric Le Goater wrote: >=20 > Hello Mark, s/k/c/ >=20 > On 5/20/21 18:37, Marc Zyngier wrote: > > Direct mappings are completely exclusive of normal mappings, meaning > > that we can refactor the code slightly so that we can get rid of > > the revmap_direct_max_irq field and use the revmap_size field > > instead, reducing the size of the irqdomain structure. > >=20 > > Signed-off-by: Marc Zyngier >=20 >=20 > This patch is breaking the POWER9/POWER10 XIVE driver (these are not > old PPC systems :) on machines sharing the same LSI HW IRQ. For instance, > a linux KVM guest with a virtio-rng and a virtio-balloon device. In that > case, Linux creates two distinct IRQ mappings which can lead to some > unexpected behavior. Either the irq domain translates, or it doesn't. If the driver creates a nomap domain, and yet expects some sort of translation to happen, then the driver is fundamentally broken. And even without that: how do you end-up with a single HW interrupt having two mappings? > A fix to go forward would be to change the XIVE IRQ domain to use a > 'Tree' domain for reverse mapping and not the 'No Map' domain mapping. > I will keep you updated for XIVE. I bet there is a bit more to it. From what you are saying above, something rather ungodly is happening in the XIVE code. M. --=20 Without deviation from the norm, progress is not possible.