From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12954C433EF for ; Mon, 24 Jan 2022 11:25:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237483AbiAXLZA (ORCPT ); Mon, 24 Jan 2022 06:25:00 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:48024 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237440AbiAXLY5 (ORCPT ); Mon, 24 Jan 2022 06:24:57 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 0D326B80AE3 for ; Mon, 24 Jan 2022 11:24:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC482C340E1; Mon, 24 Jan 2022 11:24:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1643023494; bh=m/SHnOH+m0JwktM7yHyCtPza4vsWP6iO5cM6+Z32lb8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=j49ucWmDcM79ombzb0danGgRCSyjQlhFJxW0S4NGV8mOUJneicxPx4mO84m6cOvuR KaeYLlAMvtXm3ChK0pgeOsLT1d2YwO07BJM9mWLPifiSkspAT259wuhk3gGeacCph7 Bbz7lz5VgMm6Wr4KFTuAzoN390cHu9EJ+uoB1ed/OGa9u0VfdIHyWN405gJnV50RLJ 66SmSSByYbXysfAmqcJnbWShZA4EzJaQ2FzCZpNXufDYZCY5rlbcXto1meoism3Edc wIgptZZOZFQLJTTzKSGOYshzFdlO9hJWf5bpkYI2auRQUmsR7VwygRhMWt8IODn62A fwVt7ncAfEfkg== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nBxSu-002MoL-TC; Mon, 24 Jan 2022 11:24:53 +0000 Date: Mon, 24 Jan 2022 11:24:52 +0000 Message-ID: <87a6fl8jgb.wl-maz@kernel.org> From: Marc Zyngier To: Xiongfeng Wang Cc: , , , John Garry Subject: Re: [PATCH] irqchip/gic-v3-its: Select housekeeping CPUs preferentially for managed IRQs In-Reply-To: <20220124073440.88598-1-wangxiongfeng2@huawei.com> References: <20220124073440.88598-1-wangxiongfeng2@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: wangxiongfeng2@huawei.com, tglx@linutronix.de, linux-kernel@vger.kernel.org, guohanjun@huawei.com, john.garry@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org + John Garry, as he was reporting issues around the same piece of code[1] On Mon, 24 Jan 2022 07:34:40 +0000, Xiongfeng Wang wrote: > > When using kernel parameter 'isolcpus=managed_irq,xxxx' to bind the > managed IRQs to housekeeping CPUs, the effective_affinity sometimes > still contains the non-housekeeping CPUs. > > irq_do_set_affinity() passes the housekeeping cpumask to > chip->irq_set_affinity(), but ITS driver select CPU according to > irq_common_data->affinity. While 'irq_common_data->affinity' is updated > after chip->irq_set_affinity() is called in irq_do_set_affinity(). Also > 'irq_common_data->affinity' may contains non-housekeeping CPUs. I found > the below link explaining the reason. > https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg2267032.html > > To modify CPU selecting logic to prefer housekeeping CPUs, select CPU > from the input cpumask parameter first. If none of it is online, then > select CPU from 'irq_common_data->affinity'. > > Signed-off-by: Xiongfeng Wang > --- > drivers/irqchip/irq-gic-v3-its.c | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > index d25b7a864bbb..17c15d3b2784 100644 > --- a/drivers/irqchip/irq-gic-v3-its.c > +++ b/drivers/irqchip/irq-gic-v3-its.c > @@ -1624,7 +1624,10 @@ static int its_select_cpu(struct irq_data *d, > > cpu = cpumask_pick_least_loaded(d, tmpmask); > } else { > - cpumask_and(tmpmask, irq_data_get_affinity_mask(d), cpu_online_mask); > + cpumask_and(tmpmask, aff_mask, cpu_online_mask); > + if (cpumask_empty(tmpmask)) > + cpumask_and(tmpmask, irq_data_get_affinity_mask(d), > + cpu_online_mask); I think that the online_cpu_mask logical and is a bit wrong. A managed interrupt should be able to target an offline CPU: diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c index eb0882d15366..0cea46bdaf99 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -1620,7 +1620,7 @@ static int its_select_cpu(struct irq_data *d, cpu = cpumask_pick_least_loaded(d, tmpmask); } else { - cpumask_and(tmpmask, irq_data_get_affinity_mask(d), cpu_online_mask); + cpumask_copy(tmpmask, aff_mask); /* If we cannot cross sockets, limit the search to that node */ if ((its_dev->its->flags & ITS_FLAGS_WORKAROUND_CAVIUM_23144) && We still have an issue when the system hasn't booted with all its CPUs, as the corresponding collections aren't initialised and we end-up in a rather bad place. M. [1] https://lore.kernel.org/r/78615d08-1764-c895-f3b7-bfddfbcbdfb9@huawei.com -- Without deviation from the norm, progress is not possible.