RE: [RFC 0/7] cpuidle: Add poking mechanism to support non-IPI wakeup

From: Aisheng Dong <aisheng.dong@nxp.com>
To: Marc Zyngier <marc.zyngier@arm.com>,
	Leonard Crestez <leonard.crestez@nxp.com>,
	"l.stach@pengutronix.de" <l.stach@pengutronix.de>,
	Richard Zhu <hongxing.zhu@nxp.com>, Jacky Bai <ping.bai@nxp.com>
Cc: Fabio Estevam <fabio.estevam@nxp.com>,
	Cosmin Samoila <cosmin.samoila@nxp.com>,
	Robin Gong <yibin.gong@nxp.com>, Mircea Pop <mircea.pop@nxp.com>,
	Daniel Baluta <daniel.baluta@nxp.com>,
	"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
	"shawnguo@kernel.org" <shawnguo@kernel.org>,
	Robert Chiras <robert.chiras@nxp.com>,
	Anson Huang <anson.huang@nxp.com>, Jun Li <jun.li@nxp.com>,
	Abel Vesa <abel.vesa@nxp.com>,
	"robh@kernel.org" <robh@kernel.org>,
	Zening Wang <zening.wang@nxp.com>,
	dl-linux-imx <linux-imx@nxp.com>, BOUGH CHEN <haibo.chen@nxp.com>,
	Horia Geanta <horia.geanta@nxp.com>,
	Peter Chen <peter.chen@nxp.com>,
	Joakim Zhang <qiangqing.zhang@nxp.com>,
	"rjw@rjwysocki.net" <rjw@rjwysocki.net>,
	Leo Zhang <leo.zhang@nxp.com>,
	Shenwei Wang <shenwei.wang@nxp.com>,
	"linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>,
	Ranjani Vaidyanathan <ranjani.vaidyanathan@nxp.com>,
	Han Xu <han.xu@nxp.com>,
	"will.deacon@arm.com" <will.deacon@arm.com>,
	Iuliana Prodan <iuliana.prodan@nxp.com>,
	"sudeep.holla@arm.com" <sudeep.holla@arm.com>,
	"lorenzo.pieralisi@arm.com" <lorenzo.pieralisi@arm.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"mark.rutland@arm.com" <mark.rutland@arm.com>,
	Peng Fan <peng.fan@nxp.com>,
	"kernel@pengutronix.de" <kernel@pengutronix.de>,
	Viorel Suman <viorel.suman@nxp.com>
Subject: RE: [RFC 0/7] cpuidle: Add poking mechanism to support non-IPI wakeup
Date: Thu, 28 Mar 2019 11:21:00 +0000	[thread overview]
Message-ID: <AM0PR04MB4211216AA60055524D1F46FC80590@AM0PR04MB4211.eurprd04.prod.outlook.com> (raw)
In-Reply-To: <0ac4b527-2f79-4d21-7408-18c87cadb512@arm.com>

> From: Marc Zyngier [mailto:marc.zyngier@arm.com]
> Sent: Thursday, March 28, 2019 2:13 AM
> On 27/03/2019 17:00, Leonard Crestez wrote:
> > On Wed, 2019-03-27 at 17:06 +0100, Lucas Stach wrote:
> >> Am Mittwoch, den 27.03.2019, 15:57 +0000 schrieb Marc Zyngier:
> >>> On 27/03/2019 15:44, Lucas Stach wrote:
> >>>> Am Mittwoch, den 27.03.2019, 13:21 +0000 schrieb Abel Vesa:
> >>>>> This work is a workaround I'm looking into (more as a background
> >>>>> task) in order to add support for cpuidle on i.MX8MQ based platforms.
> >>>>>
> >>>>> The main idea here is getting around the missing GIC wake_request
> >>>>> signal (due to integration design issue) by waking up a each
> >>>>> individual core through some dedicated SW power-up bits inside the
> >>>>> power controller (GPC) right before every IPI is requested for that each
> individual core.
> >>>>
> >>>> Just a general comment, without going into the details of this series:
> >>>> this issue is not only affecting IPIs, but also MSIs terminated at
> >>>> the GIC. Currently MSIs are terminated at the PCIe core, but
> >>>> terminating them at the GIC is clearly preferable, as this allows
> >>>> assigning CPU affinity to individual MSIs and lowers IRQ service overhead.
> >>>>
> >>>> I'm not sure what the consequences are for upstream Linux support
> >>>> yet, but we should keep in mind that having a workaround for IPIs
> >>>> is only solving part of the issue.
> >>>
> >>> If this erratum is affecting more than just IPIs, then indeed I
> >>> don't see how this patch series solves anything.
> >>>
> >>> But the erratum documentation seems to imply that only SGIs are
> >>> affected, and goes as far as suggesting to use an external interrupt
> >>> would solve it. How comes this is not the case? Or is it that
> >>> anything directly routed to a redistributor is also affected? This
> >>> would break LPIs (and thus MSIs) and PPIs (the CPU timer, among others).
> >>>
> >>> What is the *exact* status of this thing? I have the ugly feeling
> >>> that the true workaround is just to disable cpuidle.
> >>
> >> As far as I understand the erratum, the basic issue is that the GIC
> >> wake_request signals are not connected to the GPC (the CPU/peripheral
> >> power sequencer). The SPIs are routed through the GPC and thus are
> >> visible as wakeup sources, which is why the workaround of using an
> >> external SPI as wakeup trigger for the IPI works.
> >
> > We had a kernel workaround for IPIs in our internal tree for a long
> > time and I don't think we do anything special for PCI. Does PCI MSI
> > really bypass the GPC on 8mq?
> 
> If you have an ITS, certainly. If you don't, it depends. MSIs can hit the
> distributor's MBI registers and generate non-wired SPIs, which I assume will
> bypass the GPC altogether.
> 

Richard & Jacky,

Can you double check if this issue affect PCI MSI function?

Regards
Dong Aisheng

> > Adding Richard/Jacky, they might know about this.
> >
> > This seems like something of a corner case to me, don't many imx
> > boards ship without PCI; especially for low-power scenarios? If
> > required it might be reasonable to add an additional workaround to
> > disable all cpuidle if pci msis are used.
> 
> Establishing a link between cpuidle and PCI in the kernel would be pretty
> invasive, and that would come on top of what this series also mandates.
> 
> At that level of apparent brokenness, it is far safer to get cpuidle out of the
> picture altogether, and I'd rather see these patches in a vendor tree (for once).
> 
> Thanks,
> 
> 	M.
> --
> Jazz is not dead. It just smells funny...