From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42069C33CB1 for ; Thu, 16 Jan 2020 09:26:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 157DE20730 for ; Thu, 16 Jan 2020 09:26:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="YNsZvqMC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730700AbgAPJ0K (ORCPT ); Thu, 16 Jan 2020 04:26:10 -0500 Received: from mail-il1-f194.google.com ([209.85.166.194]:44534 "EHLO mail-il1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730599AbgAPJ0K (ORCPT ); Thu, 16 Jan 2020 04:26:10 -0500 Received: by mail-il1-f194.google.com with SMTP id z12so17515802iln.11 for ; Thu, 16 Jan 2020 01:26:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=5+T2+f3YUqy/mz/g0zqdHuyixNv3uIRSRLqHpYCvIA8=; b=YNsZvqMC3qBwhA390kR+0O1+Z775NXpgorZ1Uj0LTNfhunlCVnboeL+hdbdTD7zPyj cO4YdefQbXLyr/t0viYTQNIs2/R9znJRs+JPdwF+zlywPmjg5MJOT97iGih9buKS3tvr detFCcSpM8To0oU1Jp0ON3jtZma7mcyCUwQck= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=5+T2+f3YUqy/mz/g0zqdHuyixNv3uIRSRLqHpYCvIA8=; b=EN210UzYxUD/rac+HJa1uPrGgX88XfntJv31s0hlf2CXoC/w4jM6yF+pnQCrgRAAdm yhMLegPV6Q9lp6ggAsx8jTmsQ8u6oQcuHZk0jyFdU8O9dIv/xULD1QwDw6IyAsZ2CLcR e4QdF+7DHdlx0Ce+hBFqwIGCDPwLHZBS3Axd68nfRj6zcT8iKlQ2rpYlHNbbOxeqZOHk ygthcF2l7UU+t23Th18H+Teda+4SpF3Le6rx0d3j+AXxkmkz5YvFE+Q82h9Cgm6ImdFt fzVso9L9raL7EoL020+u7eFoGFboJkAtCCUPAGdjHKsnDVELCl0G+PCXEIW++2nttlJA Ih/Q== X-Gm-Message-State: APjAAAWbD8kW9XVu7mrWzxyRYVQ/Wmlv+m79uRqTH2l8uG6faqfQ28tZ eIsamxlOiSG3J4rZC8HFH1Y9xkE5R0KlS7aeUai9Mw== X-Google-Smtp-Source: APXvYqzcJuvwLp0x5+fMsP13ADUUEVQCs/aLETOB1GPnAQGr8kZxelM+YUGAkJZnZhbBbnQ8HdB5j4UwUbclM7dRrHw= X-Received: by 2002:a92:8851:: with SMTP id h78mr2599028ild.308.1579166769584; Thu, 16 Jan 2020 01:26:09 -0800 (PST) MIME-Version: 1.0 References: <20200115063410.131692-1-hsinyi@chromium.org> <20200115114112.GA3663@bogus> In-Reply-To: <20200115114112.GA3663@bogus> From: Hsin-Yi Wang Date: Thu, 16 Jan 2020 17:25:43 +0800 Message-ID: Subject: Re: [PATCH v5] reboot: support offline CPUs before reboot To: Sudeep Holla Cc: Thomas Gleixner , Josh Poimboeuf , Ingo Molnar , Peter Zijlstra , Jiri Kosina , Pavankumar Kondeti , Vitaly Kuznetsov , Aaro Koskinen , Greg Kroah-Hartman , Will Deacon , Fenghua Yu , James Morse , Mark Rutland , Heiko Carstens , Guenter Roeck , Stephen Boyd , lkml , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" , linux-csky@vger.kernel.org, linux-ia64@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev , linux-s390@vger.kernel.org, Linux-sh list , sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, Linux PM Content-Type: text/plain; charset="UTF-8" Sender: linux-csky-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-csky@vger.kernel.org On Wed, Jan 15, 2020 at 7:41 PM Sudeep Holla wrote: > > On Wed, Jan 15, 2020 at 02:34:10PM +0800, Hsin-Yi Wang wrote: > > Currently system reboots uses architecture specific codes (smp_send_stop) > > to offline non reboot CPUs. Most architecture's implementation is looping > > through all non reboot online CPUs and call ipi function to each of them. Some > > architecture like arm64, arm, and x86... would set offline masks to cpu without > > really offline them. This causes some race condition and kernel warning comes > > out sometimes when system reboots. > > > > This patch adds a config ARCH_OFFLINE_CPUS_ON_REBOOT, which would offline cpus in > > migrate_to_reboot_cpu(). If non reboot cpus are all offlined here, the loop for > > checking online cpus would be an empty loop. If architecture don't enable this > > config, or some cpus somehow fails to offline, it would fallback to ipi > > function. > > > > What's the timing impact on systems with large number of CPUs(say 256 or > more) ? I remember we added some change to reduce the wait times for > offlining CPUs in system suspend path on arm64, still not negligible. > This is not the final solution, but I would still provided some data points here: Tested on my arm64 with 4 cpu: 2 a53 and 2 a72. Offlining 3 cpu takes about 60~65 ms Offlining 2 cpu(a53+a72 or a72+a72) takes about 42~47 ms Offlining 1 cpu(a53 or a72) takes about 23~25 ms. It would take longer time for systems with large number of CPUs.