From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751141AbcGLH0Y (ORCPT ); Tue, 12 Jul 2016 03:26:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49228 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750717AbcGLH0X (ORCPT ); Tue, 12 Jul 2016 03:26:23 -0400 Reply-To: xlpang@redhat.com Subject: Re: [V3 PATCH 1/2] x86/panic: Replace smp_send_stop() with kdump friendly version References: <20160705113323.5864.66834.stgit@softrs> <20160705113325.5864.74840.stgit@softrs> <57845FFC.6050308@redhat.com> <04EAB7311EE43145B2D3536183D1A84454C6F9CB@GSjpTKYDCembx31.service.hitachi.net> <578494D4.2020505@redhat.com> <04EAB7311EE43145B2D3536183D1A84454C707D6@GSjpTKYDCembx31.service.hitachi.net> To: =?UTF-8?B?5rKz5ZCI6Iux5a6PIC8gS0FXQUnvvIxISURFSElSTw==?= , "'xlpang@redhat.com'" , Andrew Morton , Dave Young , "Eric W. Biederman" , Baoquan He Cc: Michal Hocko , Toshi Kani , "Peter Zijlstra (Intel)" , Minfei Huang , "H. Peter Anvin" , Daniel Walker , Ingo Molnar , Takao Indoh , "x86@kernel.org" , "Lee, Chun-Yi" , Borislav Petkov , Vivek Goyal , Petr Mladek , Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , "kexec@lists.infradead.org" , "linux-kernel@vger.kernel.org" , HATAYAMA Daisuke , Masami Hiramatsu , Tejun Heo , Vitaly Kuznetsov From: Xunlei Pang Message-ID: <57849B8E.2070007@redhat.com> Date: Tue, 12 Jul 2016 15:26:06 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <04EAB7311EE43145B2D3536183D1A84454C707D6@GSjpTKYDCembx31.service.hitachi.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 12 Jul 2016 07:26:22 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2016/07/12 at 15:12, 河合英宏 / KAWAI，HIDEHIRO wrote: >> From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-owner@vger.kernel.org] On Behalf Of Xunlei Pang >> Sent: Tuesday, July 12, 2016 3:57 PM >> On 2016/07/12 at 11:56, 河合英宏 / KAWAI，HIDEHIRO wrote: >>> Hi Xunlei, >>> >>> Thanks for the review. >>> >>>> From: Xunlei Pang [mailto:xpang@redhat.com] >>>> Sent: Tuesday, July 12, 2016 12:12 PM >>>> On 2016/07/05 at 19:33, Hidehiro Kawai wrote: >>>>> This patch fixes one of the problems reported by Daniel Walker >>>>> (https://lkml.org/lkml/2015/6/24/44). >>>>> >>>>> If crash_kexec_post_notifiers boot option is specified, other CPUs >>>>> are stopped by smp_send_stop() instead of machine_crash_shutdown() >>>>> in crash_kexec() path. This behavior change leads two problems. >>>>> >>>>> Problem 1: >>>>> octeon_generic_shutdown() for MIPS OCTEON assumes that other CPUs are >>>>> still online and try to stop their watchdog timer. If >>>>> smp_send_stop() is called before octeon_generic_shutdown(), stopping >>>>> watchdog timer will fail because other CPUs have been offlined by >>>>> smp_send_stop(). >>>>> >>>>> panic() >>>>> if crash_kexec_post_notifiers == 1 >>>>> smp_send_stop() >>>>> atomic_notifier_call_chain() >>>>> kmsg_dump() >>>>> crash_kexec() >>>>> machine_crash_shutdown() >>>>> octeon_generic_shutdown() // shutdown watchdog for ONLINE CPUs >>>>> >>>>> Problem 2: >>>>> Most of architectures stop other CPUs in machine_crash_shutdown() >>>>> path, and they also do something needed for kdump. For example, >>>>> they save registers, disable virtualization extensions, and so on. >>>>> However, if smp_send_stop() stops other CPUs before >>>>> machine_crash_shutdown(), we miss those operations. >>>>> >>>>> How do we fix these problems? In the first place, we should stop >>>>> other CPUs as soon as possible when panic() was called, otherwise >>>>> other CPUs may wipe out a clue to the cause of the failure. So, we >>>>> replace smp_send_stop() with more suitable one for kdump. >>>>> >>>>> This patch solves Problem 2 by replacing smp_send_stop() in panic() >>>>> with panic_smp_send_stop(). This is a weak function which calls >>>>> smp_send_stop(), and architecture dependent code may override this >>>>> with appropriate one. This patch only provides x86-specific version. >>>>> >>>>> Changes in V3: >>>>> - Revise comments, description, and symbol names >>>>> >>>>> Changes in V2: >>>>> - Replace smp_send_stop() call with crash_kexec version which >>>>> saves cpu states and cleans up VMX/SVM >>>>> - Drop a fix for Problem 1 at this moment >>>>> >>>>> Reported-by: Daniel Walker >>>>> Fixes: f06e5153f4ae (kernel/panic.c: add "crash_kexec_post_notifiers" option) >>>>> Signed-off-by: Hidehiro Kawai >>>>> Cc: Dave Young >>>>> Cc: Baoquan He >>>>> Cc: Vivek Goyal >>>>> Cc: Eric Biederman >>>>> Cc: Masami Hiramatsu >>>>> Cc: Thomas Gleixner >>>>> Cc: Ingo Molnar >>>>> Cc: "H. Peter Anvin" >>>>> Cc: Borislav Petkov >>>>> Cc: Toshi Kani >>>>> Cc: "Peter Zijlstra (Intel)" >>>>> Cc: Takao Indoh >>>>> Cc: "Lee, Chun-Yi" >>>>> Cc: Minfei Huang >>>>> Cc: Andrew Morton >>>>> Cc: Michal Hocko >>>>> Cc: Vitaly Kuznetsov >>>>> Cc: Petr Mladek >>>>> Cc: Tejun Heo >>>>> Cc: Josh Poimboeuf >>>>> --- >>>>> arch/x86/kernel/crash.c | 14 ++++++++++---- >>>>> kernel/panic.c | 43 ++++++++++++++++++++++++++++++++----------- >>>>> 2 files changed, 42 insertions(+), 15 deletions(-) >>>>> >>>>> diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c >>>>> index 9ef978d..3305433 100644 >>>>> --- a/arch/x86/kernel/crash.c >>>>> +++ b/arch/x86/kernel/crash.c >>>>> @@ -133,15 +133,21 @@ static void kdump_nmi_callback(int cpu, struct pt_regs *regs) >>>>> disable_local_APIC(); >>>>> } >>>>> >>>>> -static void kdump_nmi_shootdown_cpus(void) >>>>> +/* Override the weak function in kernel/panic.c */ >>>>> +void panic_smp_send_stop(void) >>>>> { >>>>> - nmi_shootdown_cpus(kdump_nmi_callback); >>>>> + static int cpus_stopped; >>>> Should be atomic_t type? >>> panic_smp_send_stop() can be called by only one panicking CPU >>> (but can be called twice). It is sufficient to be normal variable. >> There are other call sites of __crash_kexec() for oops cases, which can >> call panic_smp_send_stop() concurrently on a different cpu. > In oops cases, crash_kexec() is called first, then __crash_kexec() is > called. crash_kexec() excludes concurrent execution of panic() and > crash_kexec() via panic_cpu, so panic_smp_send_stop() shouldn't be > called concurrently. Right, that's why oops calls crash_kexec() not __crash_kexec(). I have no problem on this. Thanks! Regards, Xunlei