From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from galois.linutronix.de (Galois.linutronix.de. [2a0a:51c0:0:12e:550::1]) by gmr-mx.google.com with ESMTPS id q28si287321lfb.10.2020.12.11.02.13.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Dec 2020 02:13:33 -0800 (PST) From: Thomas Gleixner Subject: Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts In-Reply-To: References: <20201210192536.118432146@linutronix.de> <20201210194045.250321315@linutronix.de> <7f7af60f-567f-cdef-f8db-8062a44758ce@oracle.com> Date: Fri, 11 Dec 2020 11:13:31 +0100 Message-ID: <874kksiras.fsf@nanos.tec.linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= , boris.ostrovsky@oracle.com, LKML Cc: Peter Zijlstra , Marc Zyngier , Stefano Stabellini , xen-devel@lists.xenproject.org, "James E.J. Bottomley" , Helge Deller , afzal mohammed , linux-parisc@vger.kernel.org, Russell King , linux-arm-kernel@lists.infradead.org, Mark Rutland , Catalin Marinas , Will Deacon , Christian Borntraeger , Heiko Carstens , linux-s390@vger.kernel.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , David Airlie , Daniel Vetter , Pankaj Bharadiya , Chris Wilson , Wambui Karuga , intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, Tvrtko Ursulin , Linus Walleij , linux-gpio@vger.kernel.org, Lee Jones , Jon Mason , Dave Jiang , Allen Hubbe , linux-ntb@googlegroups.com, Lorenzo Pieralisi , Rob Herring , Bjorn Helgaas , Michal Simek , linux-pci@vger.kernel.org, Karthikeyan Mitran , Hou Zhiqiang , Tariq Toukan , "David S. Miller" , Jakub Kicinski , netdev@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed , Leon Romanovsky List-ID: On Fri, Dec 11 2020 at 07:17, J=C3=BCrgen Gro=C3=9F wrote: > On 11.12.20 00:20, boris.ostrovsky@oracle.com wrote: >>=20 >> On 12/10/20 2:26 PM, Thomas Gleixner wrote: >>> All event channel setups bind the interrupt on CPU0 or the target CPU f= or >>> percpu interrupts and overwrite the affinity mask with the corresponding >>> cpumask. That does not make sense. >>> >>> The XEN implementation of irqchip::irq_set_affinity() already picks a >>> single target CPU out of the affinity mask and the actual target is sto= red >>> in the effective CPU mask, so destroying the user chosen affinity mask >>> which might contain more than one CPU is wrong. >>> >>> Change the implementation so that the channel is bound to CPU0 at the X= EN >>> level and leave the affinity mask alone. At startup of the interrupt >>> affinity will be assigned out of the affinity mask and the XEN binding = will >>> be updated. >>=20 >>=20 >> If that's the case then I wonder whether we need this call at all and in= stead bind at startup time. > > This binding to cpu0 was introduced with commit 97253eeeb792d61ed2 > and I have no reason to believe the underlying problem has been > eliminated. "The kernel-side VCPU binding was not being correctly set for newly allocated or bound interdomain events. In ARM guests where 2-level events were used, this would result in no interdomain events being handled because the kernel-side VCPU masks would all be clear. x86 guests would work because the irq affinity was set during irq setup and this would set the correct kernel-side VCPU binding." I'm not convinced that this is really correctly analyzed because affinity setting is done at irq startup. switch (__irq_startup_managed(desc, aff, force)) { case IRQ_STARTUP_NORMAL: ret =3D __irq_startup(desc); irq_setup_affinity(desc); break; which is completely architecture agnostic. So why should this magically work on x86 and not on ARM if both are using the same XEN irqchip with the same irqchip callbacks. Thanks, tglx From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8926C2BB48 for ; Fri, 11 Dec 2020 10:13:36 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7CE2122D72 for ; Fri, 11 Dec 2020 10:13:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7CE2122D72 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linutronix.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F08DF6ED29; Fri, 11 Dec 2020 10:13:35 +0000 (UTC) Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by gabe.freedesktop.org (Postfix) with ESMTPS id 550376ED42; Fri, 11 Dec 2020 10:13:34 +0000 (UTC) From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1607681612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4TDbx5/eRxAHfo/4EOFfTsBBr8/iJ4NkSA5alVnInbw=; b=Ebn4XFcCDWYTM7X6Ch/mHznY5CAClfBbnhfLrO3Y6Kuikr6oFUaWSYQxhfOCU5ekEgEuk5 ABxuPFqFDpUr/6fqEOmv/vOsnXIUuOejIYPyy0GJ0t4RBjN90FMlPlJ+tVDpM1qv0p7R8a xeZwcfSm29wJ/oRRI9rOhGLtzHCBhA3bpzeRWeG0Z24sHj1GQq05/L9FR4mQ32JHOYHSXu dDKuIj780YnhAu3jYewZAvLFsgNKDUqY7qYYXiJp4nkUDKsOaT+FgTS0xQYjkM48kJqr48 89TXlGkoZBhTjLzmOxl42qVBK3QR64G1yObP38Wf6bryGtmLISA8jiMWLoDDjQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1607681612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4TDbx5/eRxAHfo/4EOFfTsBBr8/iJ4NkSA5alVnInbw=; b=xExbb1LasawAWe5q2U6DGt3Co7x6OFVa9wgrktZhoDZT1DZdHSUTeF92DjXx94ng108/hg 9efMO3KmpdkXHQDQ== To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= , boris.ostrovsky@oracle.com, LKML In-Reply-To: References: <20201210192536.118432146@linutronix.de> <20201210194045.250321315@linutronix.de> <7f7af60f-567f-cdef-f8db-8062a44758ce@oracle.com> Date: Fri, 11 Dec 2020 11:13:31 +0100 Message-ID: <874kksiras.fsf@nanos.tec.linutronix.de> MIME-Version: 1.0 Subject: Re: [Intel-gfx] [patch 27/30] xen/events: Only force affinity mask for percpu interrupts X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Karthikeyan Mitran , Peter Zijlstra , Catalin Marinas , Linus Walleij , dri-devel@lists.freedesktop.org, Chris Wilson , "James E.J. Bottomley" , Saeed Mahameed , netdev@vger.kernel.org, Jakub Kicinski , Will Deacon , Michal Simek , linux-s390@vger.kernel.org, afzal mohammed , Stefano Stabellini , Dave Jiang , Leon Romanovsky , linux-rdma@vger.kernel.org, Marc Zyngier , Helge Deller , Russell King , Christian Borntraeger , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org, Heiko Carstens , Wambui Karuga , Allen Hubbe , Rob Herring , David Airlie , linux-gpio@vger.kernel.org, Lorenzo Pieralisi , Bjorn Helgaas , Lee Jones , linux-arm-kernel@lists.infradead.org, linux-parisc@vger.kernel.org, Hou Zhiqiang , Tariq Toukan , Jon Mason , linux-ntb@googlegroups.com, intel-gfx@lists.freedesktop.org, "David S. Miller" Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" T24gRnJpLCBEZWMgMTEgMjAyMCBhdCAwNzoxNywgSsO8cmdlbiBHcm/DnyB3cm90ZToKPiBPbiAx MS4xMi4yMCAwMDoyMCwgYm9yaXMub3N0cm92c2t5QG9yYWNsZS5jb20gd3JvdGU6Cj4+IAo+PiBP biAxMi8xMC8yMCAyOjI2IFBNLCBUaG9tYXMgR2xlaXhuZXIgd3JvdGU6Cj4+PiBBbGwgZXZlbnQg Y2hhbm5lbCBzZXR1cHMgYmluZCB0aGUgaW50ZXJydXB0IG9uIENQVTAgb3IgdGhlIHRhcmdldCBD UFUgZm9yCj4+PiBwZXJjcHUgaW50ZXJydXB0cyBhbmQgb3ZlcndyaXRlIHRoZSBhZmZpbml0eSBt YXNrIHdpdGggdGhlIGNvcnJlc3BvbmRpbmcKPj4+IGNwdW1hc2suIFRoYXQgZG9lcyBub3QgbWFr ZSBzZW5zZS4KPj4+Cj4+PiBUaGUgWEVOIGltcGxlbWVudGF0aW9uIG9mIGlycWNoaXA6OmlycV9z ZXRfYWZmaW5pdHkoKSBhbHJlYWR5IHBpY2tzIGEKPj4+IHNpbmdsZSB0YXJnZXQgQ1BVIG91dCBv ZiB0aGUgYWZmaW5pdHkgbWFzayBhbmQgdGhlIGFjdHVhbCB0YXJnZXQgaXMgc3RvcmVkCj4+PiBp biB0aGUgZWZmZWN0aXZlIENQVSBtYXNrLCBzbyBkZXN0cm95aW5nIHRoZSB1c2VyIGNob3NlbiBh ZmZpbml0eSBtYXNrCj4+PiB3aGljaCBtaWdodCBjb250YWluIG1vcmUgdGhhbiBvbmUgQ1BVIGlz IHdyb25nLgo+Pj4KPj4+IENoYW5nZSB0aGUgaW1wbGVtZW50YXRpb24gc28gdGhhdCB0aGUgY2hh bm5lbCBpcyBib3VuZCB0byBDUFUwIGF0IHRoZSBYRU4KPj4+IGxldmVsIGFuZCBsZWF2ZSB0aGUg YWZmaW5pdHkgbWFzayBhbG9uZS4gQXQgc3RhcnR1cCBvZiB0aGUgaW50ZXJydXB0Cj4+PiBhZmZp bml0eSB3aWxsIGJlIGFzc2lnbmVkIG91dCBvZiB0aGUgYWZmaW5pdHkgbWFzayBhbmQgdGhlIFhF TiBiaW5kaW5nIHdpbGwKPj4+IGJlIHVwZGF0ZWQuCj4+IAo+PiAKPj4gSWYgdGhhdCdzIHRoZSBj YXNlIHRoZW4gSSB3b25kZXIgd2hldGhlciB3ZSBuZWVkIHRoaXMgY2FsbCBhdCBhbGwgYW5kIGlu c3RlYWQgYmluZCBhdCBzdGFydHVwIHRpbWUuCj4KPiBUaGlzIGJpbmRpbmcgdG8gY3B1MCB3YXMg aW50cm9kdWNlZCB3aXRoIGNvbW1pdCA5NzI1M2VlZWI3OTJkNjFlZDIKPiBhbmQgSSBoYXZlIG5v IHJlYXNvbiB0byBiZWxpZXZlIHRoZSB1bmRlcmx5aW5nIHByb2JsZW0gaGFzIGJlZW4KPiBlbGlt aW5hdGVkLgoKICAgICJUaGUga2VybmVsLXNpZGUgVkNQVSBiaW5kaW5nIHdhcyBub3QgYmVpbmcg Y29ycmVjdGx5IHNldCBmb3IgbmV3bHkKICAgICBhbGxvY2F0ZWQgb3IgYm91bmQgaW50ZXJkb21h aW4gZXZlbnRzLiAgSW4gQVJNIGd1ZXN0cyB3aGVyZSAyLWxldmVsCiAgICAgZXZlbnRzIHdlcmUg dXNlZCwgdGhpcyB3b3VsZCByZXN1bHQgaW4gbm8gaW50ZXJkb21haW4gZXZlbnRzIGJlaW5nCiAg ICAgaGFuZGxlZCBiZWNhdXNlIHRoZSBrZXJuZWwtc2lkZSBWQ1BVIG1hc2tzIHdvdWxkIGFsbCBi ZSBjbGVhci4KCiAgICAgeDg2IGd1ZXN0cyB3b3VsZCB3b3JrIGJlY2F1c2UgdGhlIGlycSBhZmZp bml0eSB3YXMgc2V0IGR1cmluZyBpcnEKICAgICBzZXR1cCBhbmQgdGhpcyB3b3VsZCBzZXQgdGhl IGNvcnJlY3Qga2VybmVsLXNpZGUgVkNQVSBiaW5kaW5nLiIKCkknbSBub3QgY29udmluY2VkIHRo YXQgdGhpcyBpcyByZWFsbHkgY29ycmVjdGx5IGFuYWx5emVkIGJlY2F1c2UgYWZmaW5pdHkKc2V0 dGluZyBpcyBkb25lIGF0IGlycSBzdGFydHVwLgoKICAgICAgICAgICAgICAgIHN3aXRjaCAoX19p cnFfc3RhcnR1cF9tYW5hZ2VkKGRlc2MsIGFmZiwgZm9yY2UpKSB7CgkgICAgICAgIGNhc2UgSVJR X1NUQVJUVVBfTk9STUFMOgoJICAgICAgICAgICAgICAgIHJldCA9IF9faXJxX3N0YXJ0dXAoZGVz Yyk7CiAgICAgICAgICAgICAgICAgICAgICAgIGlycV9zZXR1cF9hZmZpbml0eShkZXNjKTsKCQkJ YnJlYWs7Cgp3aGljaCBpcyBjb21wbGV0ZWx5IGFyY2hpdGVjdHVyZSBhZ25vc3RpYy4gU28gd2h5 IHNob3VsZCB0aGlzIG1hZ2ljYWxseQp3b3JrIG9uIHg4NiBhbmQgbm90IG9uIEFSTSBpZiBib3Ro IGFyZSB1c2luZyB0aGUgc2FtZSBYRU4gaXJxY2hpcCB3aXRoCnRoZSBzYW1lIGlycWNoaXAgY2Fs bGJhY2tzLgoKVGhhbmtzLAoKICAgICAgICB0Z2x4CgoKX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KSW50ZWwtZ2Z4IG1haWxpbmcgbGlzdApJbnRlbC1nZnhA bGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxt YW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg==