From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751843AbdGZKlO (ORCPT ); Wed, 26 Jul 2017 06:41:14 -0400 Received: from mail-pg0-f68.google.com ([74.125.83.68]:36143 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751431AbdGZKlN (ORCPT ); Wed, 26 Jul 2017 06:41:13 -0400 Date: Wed, 26 Jul 2017 20:40:56 +1000 From: Nicholas Piggin To: "Gautham R. Shenoy" Cc: Michael Ellerman , Michael Neuling , Vaidyanathan Srinivasan , Shilpasri G Bhat , Akshay Adiga , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: Re: [RESEND] [v3 PATCH 2/2] powernv/powerpc: Clear PECE1 in LPCR via stop-api only on Hotplug Message-ID: <20170726204056.36233b13@roar.ozlabs.ibm.com> In-Reply-To: <1500634894-18352-1-git-send-email-ego@linux.vnet.ibm.com> References: <1500634894-18352-1-git-send-email-ego@linux.vnet.ibm.com> Organization: IBM X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 21 Jul 2017 16:31:34 +0530 "Gautham R. Shenoy" wrote: > From: "Gautham R. Shenoy" > > Currently we use the stop-api provided by the firmware to program the > SLW engine to restore the values of hypervisor resources that get lost > on deeper idle states (such as winkle). Since the deep states were > only used for CPU-Hotplug on POWER8 systems, we would program the LPCR > to have the PECE1 bit since Hotplugged CPUs shouldn't be spuriously > woken up by decrementer. > > On POWER9, some of the deep platform idle states such as stop4 can be > used in cpuidle as well. In this case, we want the CPU in stop4 to be > woken up by the decrementer when some timer on the CPU expires. > > In this patch, we program the stop-api for LPCR with PECE1 > bit cleared only when we are offlining the CPU and set it > back once the CPU is online. Looks pretty good to me, thanks! Reviewed-by: Nicholas Piggin > > Signed-off-by: Gautham R. Shenoy > --- > v2 --> v3: > - Program the LPCR during platform idle entry/exit on both POWER8 and > POWER9 > > v1 --> v2: > - Move the LPCR manipulations for CPU-Hotplug into idle.c > > arch/powerpc/platforms/powernv/idle.c | 34 +++++++++++++++++++++++++++++++++- > arch/powerpc/platforms/powernv/smp.c | 8 -------- > 2 files changed, 33 insertions(+), 9 deletions(-) > > diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c > index 2abee07..a1296e7 100644 > --- a/arch/powerpc/platforms/powernv/idle.c > +++ b/arch/powerpc/platforms/powernv/idle.c > @@ -68,7 +68,7 @@ static int pnv_save_sprs_for_deep_states(void) > * all cpus at boot. Get these reg values of current cpu and use the > * same across all cpus. > */ > - uint64_t lpcr_val = mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1; > + uint64_t lpcr_val = mfspr(SPRN_LPCR); > uint64_t hid0_val = mfspr(SPRN_HID0); > uint64_t hid1_val = mfspr(SPRN_HID1); > uint64_t hid4_val = mfspr(SPRN_HID4); > @@ -355,6 +355,14 @@ void power9_idle(void) > } > > #ifdef CONFIG_HOTPLUG_CPU > +static void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 lpcr_val) > +{ > + u64 pir = get_hard_smp_processor_id(cpu); > + > + mtspr(SPRN_LPCR, lpcr_val); > + opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val); > +} > + > /* > * pnv_cpu_offline: A function that puts the CPU into the deepest > * available platform idle state on a CPU-Offline. > @@ -364,6 +372,20 @@ unsigned long pnv_cpu_offline(unsigned int cpu) > { > unsigned long srr1; > u32 idle_states = pnv_get_supported_cpuidle_states(); > + u64 lpcr_val; > + > + /* > + * We don't want to take decrementer interrupts while we are > + * offline, so clear LPCR:PECE1. We keep PECE2 (and > + * LPCR_PECE_HVEE on P9) enabled as to let IPIs in. > + * > + * If the CPU gets woken up by a special wakeup, ensure that > + * the SLW engine sets LPCR with decrementer bit cleared, else > + * the CPU will come back to the kernel due to a spurious > + * wakeup. > + */ > + lpcr_val = mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1; > + pnv_program_cpu_hotplug_lpcr(cpu, lpcr_val); > > __ppc64_runlatch_off(); > > @@ -394,6 +416,16 @@ unsigned long pnv_cpu_offline(unsigned int cpu) > > __ppc64_runlatch_on(); > > + /* > + * Re-enable decrementer interrupts in LPCR. > + * > + * Further, we want stop states to be woken up by decrementer > + * for non-hotplug cases. So program the LPCR via stop api as > + * well. > + */ > + lpcr_val = mfspr(SPRN_LPCR) | (u64)LPCR_PECE1; > + pnv_program_cpu_hotplug_lpcr(cpu, lpcr_val); > + > return srr1; > } > #endif > diff --git a/arch/powerpc/platforms/powernv/smp.c b/arch/powerpc/platforms/powernv/smp.c > index 40dae96..536b07b 100644 > --- a/arch/powerpc/platforms/powernv/smp.c > +++ b/arch/powerpc/platforms/powernv/smp.c > @@ -164,12 +164,6 @@ static void pnv_smp_cpu_kill_self(void) > if (cpu_has_feature(CPU_FTR_ARCH_207S)) > wmask = SRR1_WAKEMASK_P8; > > - /* We don't want to take decrementer interrupts while we are offline, > - * so clear LPCR:PECE1. We keep PECE2 (and LPCR_PECE_HVEE on P9) > - * enabled as to let IPIs in. > - */ > - mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1); > - > while (!generic_check_cpu_restart(cpu)) { > /* > * Clear IPI flag, since we don't handle IPIs while > @@ -219,8 +213,6 @@ static void pnv_smp_cpu_kill_self(void) > > } > > - /* Re-enable decrementer interrupts */ > - mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) | LPCR_PECE1); > DBG("CPU%d coming online...\n", cpu); > } >