From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11172C33CB2 for ; Wed, 15 Jan 2020 09:49:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D8450222C3 for ; Wed, 15 Jan 2020 09:49:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1579081774; bh=s7oQ95LUMoNiII/w7YJlT+AHtSuE4O6j/QDVgfxLDyM=; h=References:In-Reply-To:From:Date:Subject:To:Cc:List-ID:From; b=Ieh1OWJA+TUEcJfZrdVdxXMUSkVSEmEHpgZuGkDVKmrTUa0ysDcZ7hQIPvhNlQ7gq wzaorY/Ezgb99e+Oxn6d0D24SeXZXVkC0j3TszlWmGap0roNno3mUyHsYyudYZzUte 4cEPTTVkkeL8WBKpl3nVg1LFeOETPyOoyVvSOuS4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729504AbgAOJta (ORCPT ); Wed, 15 Jan 2020 04:49:30 -0500 Received: from mail-oi1-f193.google.com ([209.85.167.193]:35278 "EHLO mail-oi1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729559AbgAOJta (ORCPT ); Wed, 15 Jan 2020 04:49:30 -0500 Received: by mail-oi1-f193.google.com with SMTP id k4so14830271oik.2; Wed, 15 Jan 2020 01:49:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xlblpk3Q7AFTy8Enno95ZSx0IUH147BofZyxZsTxPpQ=; b=TucKJXUCA3iBOurtQTxVBa/z8C9REkYA7BJC4bLHsU20pWHX8G0CPEON/zgGfgROz8 jwyhbLr5j2JjwsgdL2GIT3EzLxo7Di8fOQrERkdEJ7U7/QiPIC90oZ50fmH92oivcX4w 4nk8FEs52YgRW4roJaXdtPnaDSiUZuO+jsaSeqw5obwUuylTkiKm+kAy0NqZaqaEbJGp HyvO4SXZNtNmMLSezXxOt1GFZUvU8xOr9u0riNPtbG6A2o43vIEf5c94v4ZdcUQ/+rNA 09NbTz5Mgy/U+8mMr6AAl1/vUPkueJt0UHwsbWG3+q7l+/SeT1Za/tB2usl63ImC6Iqf Y4Og== X-Gm-Message-State: APjAAAVX1XL4PXVWExXRv2a3FQ2mHbtqRNbZ2YgU8z0/8nYn4u+xpCf+ 7f32MQtaREYqi5KzILq3sFk0XhhtpRHD3q7EPg8= X-Google-Smtp-Source: APXvYqzvsE9JWkTyA/qGCPe6FC5SfX/eWtD++abCAXNNUKmVhLTpM4TrHUCIWNo6kY2HmIQTdm2QXbcLDAmQBj9bi24= X-Received: by 2002:aca:d6c8:: with SMTP id n191mr21412346oig.103.1579081768509; Wed, 15 Jan 2020 01:49:28 -0800 (PST) MIME-Version: 1.0 References: <20200115063410.131692-1-hsinyi@chromium.org> In-Reply-To: <20200115063410.131692-1-hsinyi@chromium.org> From: "Rafael J. Wysocki" Date: Wed, 15 Jan 2020 10:49:17 +0100 Message-ID: Subject: Re: [PATCH v5] reboot: support offline CPUs before reboot To: Hsin-Yi Wang Cc: Thomas Gleixner , Josh Poimboeuf , Ingo Molnar , Peter Zijlstra , Jiri Kosina , Pavankumar Kondeti , Vitaly Kuznetsov , Aaro Koskinen , Greg Kroah-Hartman , Will Deacon , Fenghua Yu , James Morse , Mark Rutland , Heiko Carstens , Guenter Roeck , Stephen Boyd , Linux Kernel Mailing List , Linux ARM , linux-csky@vger.kernel.org, linux-ia64@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev , linux-s390@vger.kernel.org, Linux-sh list , sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, Linux PM Content-Type: text/plain; charset="UTF-8" Sender: linux-csky-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-csky@vger.kernel.org On Wed, Jan 15, 2020 at 7:35 AM Hsin-Yi Wang wrote: > > Currently system reboots uses architecture specific codes (smp_send_stop) > to offline non reboot CPUs. Most architecture's implementation is looping > through all non reboot online CPUs and call ipi function to each of them. Some > architecture like arm64, arm, and x86... would set offline masks to cpu without > really offline them. This causes some race condition and kernel warning comes > out sometimes when system reboots. > > This patch adds a config ARCH_OFFLINE_CPUS_ON_REBOOT, which would offline cpus in > migrate_to_reboot_cpu(). If non reboot cpus are all offlined here, the loop for > checking online cpus would be an empty loop. If architecture don't enable this > config, or some cpus somehow fails to offline, it would fallback to ipi > function. > > Opt in this config for architectures that support CONFIG_HOTPLUG_CPU. > > Signed-off-by: Hsin-Yi Wang > --- > Change from v4: > * fix a few nits: naming, comments, remove Kconfig text... > > Change from v3: > * Opt in config for architectures that support CONFIG_HOTPLUG_CPU > * Merge function offline_secondary_cpus() and freeze_secondary_cpus() > with an additional flag. This does not seem to be a very good idea, since freeze_secondary_cpus() does much more than you need for reboot. For reboot, you basically only need to do something like this AFAICS: cpu_maps_update_begin(); for_each_online_cpu(i) { if (i != cpu) _cpu_down(i, 1, CPUHP_OFFLINE); } cpu_hotplug_disabled++; cpu_maps_update_done(); And you may put this into a function defined outside of CONFIG_PM_SLEEP. > > Change from v2: > * Add another config instead of configed by CONFIG_HOTPLUG_CPU So why exactly is this new Kconfig option needed? Everybody supporting CPU hotplug seems to opt in anyway. [cut] > > -int freeze_secondary_cpus(int primary) > +int freeze_secondary_cpus(int primary, bool reboot) > { > int cpu, error = 0; > > @@ -1237,11 +1237,13 @@ int freeze_secondary_cpus(int primary) > if (cpu == primary) > continue; > > - if (pm_wakeup_pending()) { > +#ifdef CONFIG_PM_SLEEP > + if (!reboot && pm_wakeup_pending()) { > pr_info("Wakeup pending. Abort CPU freeze\n"); > error = -EBUSY; > break; > } > +#endif Please avoid using #ifdefs in function bodies. This makes the code hard to maintain in the long term. > > trace_suspend_resume(TPS("CPU_OFF"), cpu, true); > error = _cpu_down(cpu, 1, CPUHP_OFFLINE); > @@ -1250,7 +1252,9 @@ int freeze_secondary_cpus(int primary) > cpumask_set_cpu(cpu, frozen_cpus); > else { > pr_err("Error taking CPU%d down: %d\n", cpu, error); > - break; > + /* When rebooting, offline as many CPUs as possible. */ > + if (!reboot) > + break; > } > } > > diff --git a/kernel/reboot.c b/kernel/reboot.c > index c4d472b7f1b4..12f643b66e57 100644 > --- a/kernel/reboot.c > +++ b/kernel/reboot.c > @@ -7,6 +7,7 @@ > > #define pr_fmt(fmt) "reboot: " fmt > > +#include > #include > #include > #include > @@ -220,7 +221,9 @@ void migrate_to_reboot_cpu(void) > /* The boot cpu is always logical cpu 0 */ > int cpu = reboot_cpu; > > +#if !IS_ENABLED(CONFIG_ARCH_OFFLINE_CPUS_ON_REBOOT) > cpu_hotplug_disable(); > +#endif You can write this as if (!IS_ENABLED(CONFIG_ARCH_OFFLINE_CPUS_ON_REBOOT)) cpu_hotplug_disable(); That's what IS_ENABLED() is there for. > > /* Make certain the cpu I'm about to reboot on is online */ > if (!cpu_online(cpu)) > @@ -231,6 +234,11 @@ void migrate_to_reboot_cpu(void) > > /* Make certain I only run on the appropriate processor */ > set_cpus_allowed_ptr(current, cpumask_of(cpu)); > + > +#if IS_ENABLED(CONFIG_ARCH_OFFLINE_CPUS_ON_REBOOT) > + /* Offline other cpus if possible */ > + freeze_secondary_cpus(cpu, true); > +#endif The above comment applies here too. > } > > /** > --