From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BB7CC433E0 for ; Sat, 27 Mar 2021 21:47:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 23911619A9 for ; Sat, 27 Mar 2021 21:47:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231243AbhC0Vq1 convert rfc822-to-8bit (ORCPT ); Sat, 27 Mar 2021 17:46:27 -0400 Received: from mail.kernel.org ([198.145.29.99]:34808 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230478AbhC0Vp7 (ORCPT ); Sat, 27 Mar 2021 17:45:59 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 50C7D601FE; Sat, 27 Mar 2021 21:45:57 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lQGkl-004BpL-9c; Sat, 27 Mar 2021 21:45:55 +0000 Date: Sat, 27 Mar 2021 21:45:54 +0000 Message-ID: <87mtuofey5.wl-maz@kernel.org> From: Marc Zyngier To: Pali =?UTF-8?B?Um9ow6Fy?= Cc: Jianjun Wang , Bjorn Helgaas , Rob Herring , Lorenzo Pieralisi , Ryder Lee , Philipp Zabel , Matthias Brugger , linux-pci@vger.kernel.org, linux-mediatek@lists.infradead.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, youlin.pei@mediatek.com, chuanjia.liu@mediatek.com, qizhong.cheng@mediatek.com, sin_jieyang@mediatek.com, drinkcat@chromium.org, Rex-BC.Chen@mediatek.com, anson.chuang@mediatek.com, Krzysztof Wilczyski Subject: Re: [v9,5/7] PCI: mediatek-gen3: Add MSI support In-Reply-To: <20210327202904.nvn7tfodmc2xw23l@pali> References: <20210324030510.29177-1-jianjun.wang@mediatek.com> <20210324030510.29177-6-jianjun.wang@mediatek.com> <20210327192837.4rr46oeiuokritlc@pali> <87o8f4fkkh.wl-maz@kernel.org> <20210327202904.nvn7tfodmc2xw23l@pali> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: pali@kernel.org, jianjun.wang@mediatek.com, bhelgaas@google.com, robh+dt@kernel.org, lorenzo.pieralisi@arm.com, ryder.lee@mediatek.com, p.zabel@pengutronix.de, matthias.bgg@gmail.com, linux-pci@vger.kernel.org, linux-mediatek@lists.infradead.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, youlin.pei@mediatek.com, chuanjia.liu@mediatek.com, qizhong.cheng@mediatek.com, sin_jieyang@mediatek.com, drinkcat@chromium.org, Rex-BC.Chen@mediatek.com, anson.chuang@mediatek.com, kw@linux.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, 27 Mar 2021 20:29:04 +0000, Pali Rohár wrote: > > On Saturday 27 March 2021 19:44:30 Marc Zyngier wrote: > > On Sat, 27 Mar 2021 19:28:37 +0000, > > Pali Rohár wrote: > > > > > > On Wednesday 24 March 2021 11:05:08 Jianjun Wang wrote: > > > > +static void mtk_pcie_msi_handler(struct mtk_pcie_port *port, int set_idx) > > > > +{ > > > > + struct mtk_msi_set *msi_set = &port->msi_sets[set_idx]; > > > > + unsigned long msi_enable, msi_status; > > > > + unsigned int virq; > > > > + irq_hw_number_t bit, hwirq; > > > > + > > > > + msi_enable = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > > > + > > > > + do { > > > > + msi_status = readl_relaxed(msi_set->base + > > > > + PCIE_MSI_SET_STATUS_OFFSET); > > > > + msi_status &= msi_enable; > > > > + if (!msi_status) > > > > + break; > > > > + > > > > + for_each_set_bit(bit, &msi_status, PCIE_MSI_IRQS_PER_SET) { > > > > + hwirq = bit + set_idx * PCIE_MSI_IRQS_PER_SET; > > > > + virq = irq_find_mapping(port->msi_bottom_domain, hwirq); > > > > + generic_handle_irq(virq); > > > > + } > > > > + } while (true); > > > > > > Hello! > > > > > > Just a question, cannot this while-loop cause block of processing other > > > interrupts? > > > > This is a level interrupt. You don't have much choice but to handle it > > immediately, although an alternative would be to mask it and deal with > > it in a thread. And since Linux doesn't deal with interrupt priority, > > a screaming interrupt is never a good thing. > > I see. Something like "interrupt priority" (which does not exist?) would > be needed to handle it. Interrupt priorities definitely exist, but Linux doesn't use them. Furthermore, This wouldn't be relevant here as you get a bunch of MSI multiplexed onto a single one. Where would you apply the priority? > > > > I have done tests with different HW (aardvark) but with same while(true) > > > loop logic. One XHCI PCIe controller was sending MSI interrupts too fast > > > and interrupt handler with this while(true) logic was in infinite loop. > > > During one IRQ it was calling infinite many times generic_handle_irq() > > > as HW was feeding new and new MSI hwirq into status register. > > > > Define "too fast". > > Fast - next interrupt comes prior checking if while(true)-loop should stop. That's definitely not something you can easily fix at the interrupt handling level. You need to prevent this from happening. That's usually the result of a misprogramming or a HW bug. > > If something in the system is able to program the > > XHCI device in such a way that it causes a screaming interrupt, that's > > the place to look for problems, and probably not in the interrupt > > handling itself, which does what it is supposed to do. > > > > > But this is different HW, so it can have different behavior and does not > > > have to cause above issue. > > > > > > I have just spotted same code pattern for processing MSI interrupts... > > > > This is a common pattern that you will find in pretty much any > > interrupt handling/demuxing, and is done this way when the cost of > > taking the exception is high compared to that of handling it. > > And would not help if while(true)-loop is replaced by loop with upper > limit of iterations? Or just call only one iteration? That wouldn't change much: you would still have the interrupt being pending, and it would fire again at the earliest opportunity. At best, the root interrupt controller is able to present you with another interrupt before forcing you to deal with the one you have ignored again. But you cannot rely on that either. And to be honest, other interrupts are only a part of the problem you are describing. With a screaming interrupt, you can't execute userspace. This is as bad as it gets. M. -- Without deviation from the norm, progress is not possible.