From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDE7AFC6196 for ; Fri, 8 Nov 2019 21:08:46 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 93B7520869 for ; Fri, 8 Nov 2019 21:08:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="fMw8cEZs"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rjflubiq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93B7520869 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+infradead-linux-riscv=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description :Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Q1kpkmeW3WrVBn2JgRi+0JAks1iQTmEdjo4e1WViIyY=; b=fMw8cEZseP4w0r UT8dJI/9m+jnIHIE1/tvkX6DlknRQ1T8TFzDi2JHX4rLuSASaM0LGJkqLAUIEQMnUjNLmPzeStKAU aZ1lHOlftFGKB1jbbmT3WUuUQ9K4hTBQUoBLCnQ5P6paJ3/4b/UE3kvKJVKav8fmo/PQ+8U/2hyqc oBn1heA6RuzWwlihv+j730lTOE1Mjjs3ebqzFiSqZIXF67dcncqbx4Pnmu3lCzVSVaB9OB2DWGbRt bVJBkLcyRqy7UhItadImC/Y1Yu+g2Ls8PzkdyqlOULxxBfTsZvxt+rxod/+GLuG+8ulA9grKV4izh gc68VBiMgxphYp5NTalQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iTBUr-0007ac-8j; Fri, 08 Nov 2019 21:08:45 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iTBUo-0007ZX-0w; Fri, 08 Nov 2019 21:08:43 +0000 Received: by mail-wm1-x344.google.com with SMTP id z19so7580915wmk.3; Fri, 08 Nov 2019 13:08:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=leuvFNJRBbgvPXZJlD4yDG614p9ePBBBJqh6/bi0nWM=; b=rjflubiq8T92r7IsfuflmmOLhAbyeUiuHnf/rQSYhhlSmttRaTaotAvL5xYp4yOydT GvzpvT1wRN/Se5w5Bamn3iuJIAGtHWJ3FQg7uwfFHiBZYXDG6coXsfRkKjBQWYAriMTe 4kTrO4cphBn73YaGY9oVhRABcMWkQkwRmG/gGE9R1eQjZ3xrtTYtO4ekpHSWHoSNgXQt LOIHB3CeUqi5Hx5sAbRW5lNwEUAqSAnAREOPMl/n34UjrP0bXjp7N2IIvVwOE5dHTyoC y/6fNB8lMSjFK19DQSqAbg52NdT8Yh0NvqWS+ZavZPT5hVJ6jwjvtDCEDMZ1djztH8HA lzbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=leuvFNJRBbgvPXZJlD4yDG614p9ePBBBJqh6/bi0nWM=; b=EkxvlN/PHgu+Du4+nXXUXlO6FYK+cIsSjhhvSKhkyLxfGCi842NrWIKRS3rPh7qIQB NOBQkj1/0i17s7rv88avu8xyUaK9EO3TNM9LRbNbetoKgl+v6NYBLkSJOwfx+rVOG70U vMSkcCm94cQSnYNtwIbcUCQ/yRC6NvKy9hwFRyBUIeLxrOyGCKFv4uNkWoCiTqpEcusA lTh7Z0G2AX9jaIPWxlnIgppH/60R+E0J5FDZooxuFZq7damQoZMkMD5aUoyNeLrOt19Q Zcx2bSPvW18pFXWYQ74FQg5qrlgtclqg2R5LTzdlsu+8FrQNOTPjMGy1PZ8V834FAbbg NfYg== X-Gm-Message-State: APjAAAW9Q9T7wImqIPWljXQh8GPcPeg8iOXFIFDUazMbWOT19B6ZRWi6 HqgYbsF91GaVJ04FJrgjfbs= X-Google-Smtp-Source: APXvYqy4dAVQ+ZAcZ+Na7c6XiIr61o8y7Lfg2+bZQEg4XJKlBUyi7kCGw2dZZvY/2MRfFQOLGhdeXA== X-Received: by 2002:a1c:810d:: with SMTP id c13mr6004971wmd.154.1573247319697; Fri, 08 Nov 2019 13:08:39 -0800 (PST) Received: from [10.83.36.153] ([217.173.96.166]) by smtp.gmail.com with ESMTPSA id j63sm9370828wmj.46.2019.11.08.13.08.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 08 Nov 2019 13:08:38 -0800 (PST) Subject: Re: [PATCH 00/50] Add log level to show_stack() To: Russell King - ARM Linux admin , Dmitry Safonov References: <20191106030542.868541-1-dima@arista.com> <20191106092039.GT4131@hirez.programming.kicks-ass.net> <10db6fa1-5b17-ebe6-09e0-6335e09e4db8@arista.com> <20191106203440.GH3079@worktop.programming.kicks-ass.net> <20191108173045.GY25745@shell.armlinux.org.uk> From: Dmitry Safonov <0x7f454c46@gmail.com> Message-ID: <5a411348-be84-f592-ef2d-ffaec99332bd@gmail.com> Date: Fri, 8 Nov 2019 21:08:30 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.0 MIME-Version: 1.0 In-Reply-To: <20191108173045.GY25745@shell.armlinux.org.uk> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191108_130842_111666_BC36BA18 X-CRM114-Status: GOOD ( 18.89 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juri Lelli , linux-sh@vger.kernel.org, Catalin Marinas , Ben Segall , Guo Ren , Pavel Machek , Vincent Guittot , Paul Burton , Michael Ellerman , Geert Uytterhoeven , Mel Gorman , Jiri Slaby , Matt Turner , uclinux-h8-devel@lists.sourceforge.jp, Len Brown , linux-pm@vger.kernel.org, linux-um@lists.infradead.org, Thomas Gleixner , Dietmar Eggemann , Richard Henderson , Greg Kroah-Hartman , "Rafael J. Wysocki" , LKML , Ralf Baechle , Paul Mackerras , Andrew Morton , linux-ia64@vger.kernel.org, Tetsuo Handa , James Hogan , "James E.J. Bottomley" , Max Filippov , Vincent Chen , Ingo Molnar , linux-s390@vger.kernel.org, linux-c6x-dev@linux-c6x.org, Yoshinori Sato , linux-hexagon@vger.kernel.org, Helge Deller , "open list:TENSILICA XTENSA PORT \(xtensa\)" , Vasily Gorbik , Aurelien Jacquiot , linux-m68k@lists.linux-m68k.org, Stafford Horne , linux-arm-kernel@lists.infradead.org, Chris Zankel , Tony Luck , Douglas Anderson , Benjamin Herrenschmidt , Heiko Carstens , Will Deacon , Daniel Thompson , Brian Cain , Christian Borntraeger , kgdb-bugreport@lists.sourceforge.net, linux-snps-arc@lists.infradead.org, Fenghua Yu , Borislav Petkov , Jeff Dike , Steven Rostedt , Ivan Kokshaysky , Greentime Hu , Guan Xuetao , linux-parisc@vger.kernel.org, linux-alpha@vger.kernel.org, Ley Foon Tan , "David S. Miller" , Rich Felker , Petr Mladek , Peter Zijlstra , "H. Peter Anvin" , sparclinux@vger.kernel.org, linux-riscv@lists.infradead.org, Anton Ivanov , Jonas Bonn , Richard Weinberger , X86 ML , clang-built-linux@googlegroups.com, Ingo Molnar , Mark Salter , Albert Ou , Stefan Kristiansson , openrisc@lists.librecores.org, Paul Walmsley , Michal Simek , Vineet Gupta , linux-mips@vger.kernel.org, Sergey Senozhatsky , Palmer Dabbelt , Jason Wessel , nios2-dev@lists.rocketboards.org, linuxppc-dev@lists.ozlabs.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+infradead-linux-riscv=archiver.kernel.org@lists.infradead.org On 11/8/19 5:30 PM, Russell King - ARM Linux admin wrote: > On Fri, Nov 08, 2019 at 04:28:30PM +0000, Dmitry Safonov wrote: [..] >> >> Well, the use-case for lower log-level is that everything goes into logs >> (/var/log/dmesg or /var/log/messages whatever rsyslog has settting). >> >> That has it's value: >> - after a failure (i.e. panic) messages, those were only signs that >> something goes wrong can be seen in logs which can give ideas what has >> happened. > > No they don't. When the kernel panics, userspace generally stops > running, so rsyslog won't be able to write them to /var/log/messages. > > How, by "kernel panics" I mean a real kernel panic, which probably > isn't what you're talking about there. You are probably talking > about the whole shebang of non-fatal kernel oops, kernel warnings > and the like. If so, I'd ask you to stop confuzzilating terminology. > > If you really want to capture such events, then you need to have the > kernel write the panic to (e.g.) flash - see the mtdoops driver. I was talking about things prior the panic: OOMs, MMC write/read warnings, hung tasks, we also have local patches to produce a warning if the mutex is being held for too long or a task is starving on CPU time by hard/soft irqs (I hope I will design something like that for upstream). I've found those warnings useful to: (a) have an early message when the things are starting going bad. (b) analyze contentions or too large scale for a box or faulty hardware for non-reproducible issues just from logs. We use kexec to save the dmesg ringbuffer content after the panic. Thanks, Dmitry _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B3B3C17441 for ; Sat, 9 Nov 2019 01:31:14 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D0808214DA for ; Sat, 9 Nov 2019 01:31:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rjflubiq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D0808214DA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4790336hzJzF7dF for ; Sat, 9 Nov 2019 12:31:11 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2a00:1450:4864:20::341; helo=mail-wm1-x341.google.com; envelope-from=0x7f454c46@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="rjflubiq"; dkim-atps=neutral Received: from mail-wm1-x341.google.com (mail-wm1-x341.google.com [IPv6:2a00:1450:4864:20::341]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 478tDG6LZXzDrJl for ; Sat, 9 Nov 2019 08:08:43 +1100 (AEDT) Received: by mail-wm1-x341.google.com with SMTP id t26so7583069wmi.4 for ; Fri, 08 Nov 2019 13:08:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=leuvFNJRBbgvPXZJlD4yDG614p9ePBBBJqh6/bi0nWM=; b=rjflubiq8T92r7IsfuflmmOLhAbyeUiuHnf/rQSYhhlSmttRaTaotAvL5xYp4yOydT GvzpvT1wRN/Se5w5Bamn3iuJIAGtHWJ3FQg7uwfFHiBZYXDG6coXsfRkKjBQWYAriMTe 4kTrO4cphBn73YaGY9oVhRABcMWkQkwRmG/gGE9R1eQjZ3xrtTYtO4ekpHSWHoSNgXQt LOIHB3CeUqi5Hx5sAbRW5lNwEUAqSAnAREOPMl/n34UjrP0bXjp7N2IIvVwOE5dHTyoC y/6fNB8lMSjFK19DQSqAbg52NdT8Yh0NvqWS+ZavZPT5hVJ6jwjvtDCEDMZ1djztH8HA lzbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=leuvFNJRBbgvPXZJlD4yDG614p9ePBBBJqh6/bi0nWM=; b=aPW/TfTySOBsOqigh8FKoQbGLBZsD1t/WcRKM70ZVNw62APWvccMZsQctis+u21p5L ej8jyr3v8/9iiHLYnGemwT1tzjqJDAlPsR48Tc+dvjuOu2ww4+FFsfwNDiQYMJ6Yrapg fJrKtW/3lqzNXz43oi6I9ZK7j/lOclglDyJfzKkxRNRVpSW2sGTAh3vuPzRzBPM3JtWJ ngCW6zgIRgFYm/K6kuu3Kz9n+tNyoN88kuoHvYjRebtmteriD1b16TNTIr43bU/0OVAX JsHQckO3z+krmIZjFpssyVPwO1IbavtMFDqjN+rdJ6ZbRqANLZieOE46/oqmFmdu4VEg WaFg== X-Gm-Message-State: APjAAAXdYqqBWIefcEM3gMPeuhAQ1gGTkWMALGV7/9cWbNvcfquV+gwq lmw2NvlJ5Uc8Ku3IkiRp8I8= X-Google-Smtp-Source: APXvYqy4dAVQ+ZAcZ+Na7c6XiIr61o8y7Lfg2+bZQEg4XJKlBUyi7kCGw2dZZvY/2MRfFQOLGhdeXA== X-Received: by 2002:a1c:810d:: with SMTP id c13mr6004971wmd.154.1573247319697; Fri, 08 Nov 2019 13:08:39 -0800 (PST) Received: from [10.83.36.153] ([217.173.96.166]) by smtp.gmail.com with ESMTPSA id j63sm9370828wmj.46.2019.11.08.13.08.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 08 Nov 2019 13:08:38 -0800 (PST) Subject: Re: [PATCH 00/50] Add log level to show_stack() To: Russell King - ARM Linux admin , Dmitry Safonov References: <20191106030542.868541-1-dima@arista.com> <20191106092039.GT4131@hirez.programming.kicks-ass.net> <10db6fa1-5b17-ebe6-09e0-6335e09e4db8@arista.com> <20191106203440.GH3079@worktop.programming.kicks-ass.net> <20191108173045.GY25745@shell.armlinux.org.uk> From: Dmitry Safonov <0x7f454c46@gmail.com> Message-ID: <5a411348-be84-f592-ef2d-ffaec99332bd@gmail.com> Date: Fri, 8 Nov 2019 21:08:30 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.0 MIME-Version: 1.0 In-Reply-To: <20191108173045.GY25745@shell.armlinux.org.uk> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 09 Nov 2019 12:19:43 +1100 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juri Lelli , linux-sh@vger.kernel.org, Catalin Marinas , Ben Segall , Guo Ren , Pavel Machek , Vincent Guittot , Paul Burton , Geert Uytterhoeven , Mel Gorman , Jiri Slaby , Matt Turner , uclinux-h8-devel@lists.sourceforge.jp, Len Brown , linux-pm@vger.kernel.org, linux-um@lists.infradead.org, Thomas Gleixner , Dietmar Eggemann , Richard Henderson , Greg Kroah-Hartman , "Rafael J. Wysocki" , LKML , Ralf Baechle , Paul Mackerras , Andrew Morton , linux-ia64@vger.kernel.org, Tetsuo Handa , James Hogan , "James E.J. Bottomley" , Max Filippov , Vincent Chen , Ingo Molnar , linux-s390@vger.kernel.org, linux-c6x-dev@linux-c6x.org, Yoshinori Sato , linux-hexagon@vger.kernel.org, Helge Deller , "open list:TENSILICA XTENSA PORT \(xtensa\)" , Vasily Gorbik , Aurelien Jacquiot , linux-m68k@lists.linux-m68k.org, Stafford Horne , linux-arm-kernel@lists.infradead.org, Chris Zankel , Tony Luck , Douglas Anderson , Heiko Carstens , Will Deacon , Daniel Thompson , Brian Cain , Christian Borntraeger , kgdb-bugreport@lists.sourceforge.net, linux-snps-arc@lists.infradead.org, Fenghua Yu , Borislav Petkov , Jeff Dike , Steven Rostedt , Ivan Kokshaysky , Greentime Hu , Guan Xuetao , linux-parisc@vger.kernel.org, linux-alpha@vger.kernel.org, Ley Foon Tan , "David S. Miller" , Rich Felker , Petr Mladek , Peter Zijlstra , "H. Peter Anvin" , sparclinux@vger.kernel.org, linux-riscv@lists.infradead.org, Anton Ivanov , Jonas Bonn , Richard Weinberger , X86 ML , clang-built-linux@googlegroups.com, Ingo Molnar , Mark Salter , Albert Ou , Stefan Kristiansson , openrisc@lists.librecores.org, Paul Walmsley , Michal Simek , Vineet Gupta , linux-mips@vger.kernel.org, Sergey Senozhatsky , Palmer Dabbelt , Jason Wessel , nios2-dev@lists.rocketboards.org, linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On 11/8/19 5:30 PM, Russell King - ARM Linux admin wrote: > On Fri, Nov 08, 2019 at 04:28:30PM +0000, Dmitry Safonov wrote: [..] >> >> Well, the use-case for lower log-level is that everything goes into logs >> (/var/log/dmesg or /var/log/messages whatever rsyslog has settting). >> >> That has it's value: >> - after a failure (i.e. panic) messages, those were only signs that >> something goes wrong can be seen in logs which can give ideas what has >> happened. > > No they don't. When the kernel panics, userspace generally stops > running, so rsyslog won't be able to write them to /var/log/messages. > > How, by "kernel panics" I mean a real kernel panic, which probably > isn't what you're talking about there. You are probably talking > about the whole shebang of non-fatal kernel oops, kernel warnings > and the like. If so, I'd ask you to stop confuzzilating terminology. > > If you really want to capture such events, then you need to have the > kernel write the panic to (e.g.) flash - see the mtdoops driver. I was talking about things prior the panic: OOMs, MMC write/read warnings, hung tasks, we also have local patches to produce a warning if the mutex is being held for too long or a task is starving on CPU time by hard/soft irqs (I hope I will design something like that for upstream). I've found those warnings useful to: (a) have an early message when the things are starting going bad. (b) analyze contentions or too large scale for a box or faulty hardware for non-reproducible issues just from logs. We use kexec to save the dmesg ringbuffer content after the panic. Thanks, Dmitry From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FF04FC6197 for ; Fri, 8 Nov 2019 21:08:46 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2EB5A20869 for ; Fri, 8 Nov 2019 21:08:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="QbbjxG0h"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rjflubiq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2EB5A20869 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-snps-arc-bounces+linux-snps-arc=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description :Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vwuFpo0UiDnzPFiimc03yYQ0I92lDoFU8ZPsbJmfvdQ=; b=QbbjxG0heFqxkZ XHsH/AccYNT3ZYzwIWV5Y4Do9840af3v/UCZo4UggQfpoScwFiSg5n8xLCuJaCm0/JoGKuWfqgvO1 rne9pxM2hPJyOU2Hbb7tCic1zipJQVIHJC4hnF4b1QHltR97UwRJJ48wIL7aftvtitfESfjccVZkf 07zUfLmRS+Hyd9lM698b5xAgTD9IUrJs3AEzH+myRzrF73tX29f15NjrHmc0opBODxiU1X+TUG2h2 xT9QiWLhGgRm2ZRKo0wZOD1siUqX1SnzDqfSn0/z67ZNpWQ/5Fc7dZgKT+lyUpazbWVGuqgZ+kqqr 9PAoIr7pr5KPW1NCFcCg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iTBUr-0007aO-0U; Fri, 08 Nov 2019 21:08:45 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iTBUo-0007ZX-0w; Fri, 08 Nov 2019 21:08:43 +0000 Received: by mail-wm1-x344.google.com with SMTP id z19so7580915wmk.3; Fri, 08 Nov 2019 13:08:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=leuvFNJRBbgvPXZJlD4yDG614p9ePBBBJqh6/bi0nWM=; b=rjflubiq8T92r7IsfuflmmOLhAbyeUiuHnf/rQSYhhlSmttRaTaotAvL5xYp4yOydT GvzpvT1wRN/Se5w5Bamn3iuJIAGtHWJ3FQg7uwfFHiBZYXDG6coXsfRkKjBQWYAriMTe 4kTrO4cphBn73YaGY9oVhRABcMWkQkwRmG/gGE9R1eQjZ3xrtTYtO4ekpHSWHoSNgXQt LOIHB3CeUqi5Hx5sAbRW5lNwEUAqSAnAREOPMl/n34UjrP0bXjp7N2IIvVwOE5dHTyoC y/6fNB8lMSjFK19DQSqAbg52NdT8Yh0NvqWS+ZavZPT5hVJ6jwjvtDCEDMZ1djztH8HA lzbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=leuvFNJRBbgvPXZJlD4yDG614p9ePBBBJqh6/bi0nWM=; b=EkxvlN/PHgu+Du4+nXXUXlO6FYK+cIsSjhhvSKhkyLxfGCi842NrWIKRS3rPh7qIQB NOBQkj1/0i17s7rv88avu8xyUaK9EO3TNM9LRbNbetoKgl+v6NYBLkSJOwfx+rVOG70U vMSkcCm94cQSnYNtwIbcUCQ/yRC6NvKy9hwFRyBUIeLxrOyGCKFv4uNkWoCiTqpEcusA lTh7Z0G2AX9jaIPWxlnIgppH/60R+E0J5FDZooxuFZq7damQoZMkMD5aUoyNeLrOt19Q Zcx2bSPvW18pFXWYQ74FQg5qrlgtclqg2R5LTzdlsu+8FrQNOTPjMGy1PZ8V834FAbbg NfYg== X-Gm-Message-State: APjAAAW9Q9T7wImqIPWljXQh8GPcPeg8iOXFIFDUazMbWOT19B6ZRWi6 HqgYbsF91GaVJ04FJrgjfbs= X-Google-Smtp-Source: APXvYqy4dAVQ+ZAcZ+Na7c6XiIr61o8y7Lfg2+bZQEg4XJKlBUyi7kCGw2dZZvY/2MRfFQOLGhdeXA== X-Received: by 2002:a1c:810d:: with SMTP id c13mr6004971wmd.154.1573247319697; Fri, 08 Nov 2019 13:08:39 -0800 (PST) Received: from [10.83.36.153] ([217.173.96.166]) by smtp.gmail.com with ESMTPSA id j63sm9370828wmj.46.2019.11.08.13.08.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 08 Nov 2019 13:08:38 -0800 (PST) Subject: Re: [PATCH 00/50] Add log level to show_stack() To: Russell King - ARM Linux admin , Dmitry Safonov References: <20191106030542.868541-1-dima@arista.com> <20191106092039.GT4131@hirez.programming.kicks-ass.net> <10db6fa1-5b17-ebe6-09e0-6335e09e4db8@arista.com> <20191106203440.GH3079@worktop.programming.kicks-ass.net> <20191108173045.GY25745@shell.armlinux.org.uk> From: Dmitry Safonov <0x7f454c46@gmail.com> Message-ID: <5a411348-be84-f592-ef2d-ffaec99332bd@gmail.com> Date: Fri, 8 Nov 2019 21:08:30 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.0 MIME-Version: 1.0 In-Reply-To: <20191108173045.GY25745@shell.armlinux.org.uk> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191108_130842_111666_BC36BA18 X-CRM114-Status: GOOD ( 18.89 ) X-BeenThere: linux-snps-arc@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on Synopsys ARC Processors List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juri Lelli , linux-sh@vger.kernel.org, Catalin Marinas , Ben Segall , Guo Ren , Pavel Machek , Vincent Guittot , Paul Burton , Michael Ellerman , Geert Uytterhoeven , Mel Gorman , Jiri Slaby , Matt Turner , uclinux-h8-devel@lists.sourceforge.jp, Len Brown , linux-pm@vger.kernel.org, linux-um@lists.infradead.org, Thomas Gleixner , Dietmar Eggemann , Richard Henderson , Greg Kroah-Hartman , "Rafael J. Wysocki" , LKML , Ralf Baechle , Paul Mackerras , Andrew Morton , linux-ia64@vger.kernel.org, Tetsuo Handa , James Hogan , "James E.J. Bottomley" , Max Filippov , Vincent Chen , Ingo Molnar , linux-s390@vger.kernel.org, linux-c6x-dev@linux-c6x.org, Yoshinori Sato , linux-hexagon@vger.kernel.org, Helge Deller , "open list:TENSILICA XTENSA PORT \(xtensa\)" , Vasily Gorbik , Aurelien Jacquiot , linux-m68k@lists.linux-m68k.org, Stafford Horne , linux-arm-kernel@lists.infradead.org, Chris Zankel , Tony Luck , Douglas Anderson , Benjamin Herrenschmidt , Heiko Carstens , Will Deacon , Daniel Thompson , Brian Cain , Christian Borntraeger , kgdb-bugreport@lists.sourceforge.net, linux-snps-arc@lists.infradead.org, Fenghua Yu , Borislav Petkov , Jeff Dike , Steven Rostedt , Ivan Kokshaysky , Greentime Hu , Guan Xuetao , linux-parisc@vger.kernel.org, linux-alpha@vger.kernel.org, Ley Foon Tan , "David S. Miller" , Rich Felker , Petr Mladek , Peter Zijlstra , "H. Peter Anvin" , sparclinux@vger.kernel.org, linux-riscv@lists.infradead.org, Anton Ivanov , Jonas Bonn , Richard Weinberger , X86 ML , clang-built-linux@googlegroups.com, Ingo Molnar , Mark Salter , Albert Ou , Stefan Kristiansson , openrisc@lists.librecores.org, Paul Walmsley , Michal Simek , Vineet Gupta , linux-mips@vger.kernel.org, Sergey Senozhatsky , Palmer Dabbelt , Jason Wessel , nios2-dev@lists.rocketboards.org, linuxppc-dev@lists.ozlabs.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-snps-arc" Errors-To: linux-snps-arc-bounces+linux-snps-arc=archiver.kernel.org@lists.infradead.org On 11/8/19 5:30 PM, Russell King - ARM Linux admin wrote: > On Fri, Nov 08, 2019 at 04:28:30PM +0000, Dmitry Safonov wrote: [..] >> >> Well, the use-case for lower log-level is that everything goes into logs >> (/var/log/dmesg or /var/log/messages whatever rsyslog has settting). >> >> That has it's value: >> - after a failure (i.e. panic) messages, those were only signs that >> something goes wrong can be seen in logs which can give ideas what has >> happened. > > No they don't. When the kernel panics, userspace generally stops > running, so rsyslog won't be able to write them to /var/log/messages. > > How, by "kernel panics" I mean a real kernel panic, which probably > isn't what you're talking about there. You are probably talking > about the whole shebang of non-fatal kernel oops, kernel warnings > and the like. If so, I'd ask you to stop confuzzilating terminology. > > If you really want to capture such events, then you need to have the > kernel write the panic to (e.g.) flash - see the mtdoops driver. I was talking about things prior the panic: OOMs, MMC write/read warnings, hung tasks, we also have local patches to produce a warning if the mutex is being held for too long or a task is starving on CPU time by hard/soft irqs (I hope I will design something like that for upstream). I've found those warnings useful to: (a) have an early message when the things are starting going bad. (b) analyze contentions or too large scale for a box or faulty hardware for non-reproducible issues just from logs. We use kexec to save the dmesg ringbuffer content after the panic. Thanks, Dmitry _______________________________________________ linux-snps-arc mailing list linux-snps-arc@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-snps-arc From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dmitry Safonov <0x7f454c46@gmail.com> Date: Fri, 8 Nov 2019 21:08:30 +0000 Subject: [OpenRISC] [PATCH 00/50] Add log level to show_stack() In-Reply-To: <20191108173045.GY25745@shell.armlinux.org.uk> References: <20191106030542.868541-1-dima@arista.com> <20191106092039.GT4131@hirez.programming.kicks-ass.net> <10db6fa1-5b17-ebe6-09e0-6335e09e4db8@arista.com> <20191106203440.GH3079@worktop.programming.kicks-ass.net> <20191108173045.GY25745@shell.armlinux.org.uk> Message-ID: <5a411348-be84-f592-ef2d-ffaec99332bd@gmail.com> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: openrisc@lists.librecores.org On 11/8/19 5:30 PM, Russell King - ARM Linux admin wrote: > On Fri, Nov 08, 2019 at 04:28:30PM +0000, Dmitry Safonov wrote: [..] >> >> Well, the use-case for lower log-level is that everything goes into logs >> (/var/log/dmesg or /var/log/messages whatever rsyslog has settting). >> >> That has it's value: >> - after a failure (i.e. panic) messages, those were only signs that >> something goes wrong can be seen in logs which can give ideas what has >> happened. > > No they don't. When the kernel panics, userspace generally stops > running, so rsyslog won't be able to write them to /var/log/messages. > > How, by "kernel panics" I mean a real kernel panic, which probably > isn't what you're talking about there. You are probably talking > about the whole shebang of non-fatal kernel oops, kernel warnings > and the like. If so, I'd ask you to stop confuzzilating terminology. > > If you really want to capture such events, then you need to have the > kernel write the panic to (e.g.) flash - see the mtdoops driver. I was talking about things prior the panic: OOMs, MMC write/read warnings, hung tasks, we also have local patches to produce a warning if the mutex is being held for too long or a task is starving on CPU time by hard/soft irqs (I hope I will design something like that for upstream). I've found those warnings useful to: (a) have an early message when the things are starting going bad. (b) analyze contentions or too large scale for a box or faulty hardware for non-reproducible issues just from logs. We use kexec to save the dmesg ringbuffer content after the panic. Thanks, Dmitry From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <0x7f454c46@gmail.com> Subject: Re: [PATCH 00/50] Add log level to show_stack() References: <20191106030542.868541-1-dima@arista.com> <20191106092039.GT4131@hirez.programming.kicks-ass.net> <10db6fa1-5b17-ebe6-09e0-6335e09e4db8@arista.com> <20191106203440.GH3079@worktop.programming.kicks-ass.net> <20191108173045.GY25745@shell.armlinux.org.uk> From: Dmitry Safonov <0x7f454c46@gmail.com> Message-ID: <5a411348-be84-f592-ef2d-ffaec99332bd@gmail.com> Date: Fri, 8 Nov 2019 21:08:30 +0000 MIME-Version: 1.0 In-Reply-To: <20191108173045.GY25745@shell.armlinux.org.uk> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit To: Russell King - ARM Linux admin , Dmitry Safonov Cc: Peter Zijlstra , LKML , Andrew Morton , Greg Kroah-Hartman , Ingo Molnar , Jiri Slaby , Petr Mladek , Sergey Senozhatsky , Steven Rostedt , Tetsuo Handa , Albert Ou , Ben Segall , Dietmar Eggemann , Greentime Hu , Ingo Molnar , James Hogan , Juri Lelli , Mel Gorman , Michal Simek , Palmer Dabbelt , Paul Burton , Paul Walmsley , Ralf Baechle , Thomas Gleixner , Vincent Chen , Vincent Guittot , Will Deacon , linux-mips@vger.kernel.org, linux-riscv@lists.infradead.org, Ivan Kokshaysky , Matt Turner , Richard Henderson , linux-alpha@vger.kernel.org, Vineet Gupta , linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, clang-built-linux@googlegroups.com, Catalin Marinas , Aurelien Jacquiot , Mark Salter , linux-c6x-dev@linux-c6x.org, Guo Ren , Yoshinori Sato , uclinux-h8-devel@lists.sourceforge.jp, Brian Cain , linux-hexagon@vger.kernel.org, Fenghua Yu , Tony Luck , linux-ia64@vger.kernel.org, Geert Uytterhoeven , linux-m68k@lists.linux-m68k.org, Ley Foon Tan , nios2-dev@lists.rocketboards.org, Jonas Bonn , Stafford Horne , Stefan Kristiansson , openrisc@lists.librecores.org, Helge Deller , "James E.J. Bottomley" , linux-parisc@vger.kernel.org, Benjamin Herrenschmidt , Michael Ellerman , Paul Mackerras , linuxppc-dev@lists.ozlabs.org, Christian Borntraeger , Heiko Carstens , Vasily Gorbik , linux-s390@vger.kernel.org, Rich Felker , linux-sh@vger.kernel.org, "David S. Miller" , sparclinux@vger.kernel.org, Anton Ivanov , Jeff Dike , Richard Weinberger , linux-um@lists.infradead.org, Guan Xuetao , Borislav Petkov , "H. Peter Anvin" , X86 ML , Chris Zankel , Max Filippov , "open list:TENSILICA XTENSA PORT (xtensa)" , Len Brown , Pavel Machek , "Rafael J. Wysocki" , linux-pm@vger.kernel.org, Daniel Thompson , Douglas Anderson , Jason Wessel , kgdb-bugreport@lists.sourceforge.net List-ID: On 11/8/19 5:30 PM, Russell King - ARM Linux admin wrote: > On Fri, Nov 08, 2019 at 04:28:30PM +0000, Dmitry Safonov wrote: [..] >> >> Well, the use-case for lower log-level is that everything goes into logs >> (/var/log/dmesg or /var/log/messages whatever rsyslog has settting). >> >> That has it's value: >> - after a failure (i.e. panic) messages, those were only signs that >> something goes wrong can be seen in logs which can give ideas what has >> happened. > > No they don't. When the kernel panics, userspace generally stops > running, so rsyslog won't be able to write them to /var/log/messages. > > How, by "kernel panics" I mean a real kernel panic, which probably > isn't what you're talking about there. You are probably talking > about the whole shebang of non-fatal kernel oops, kernel warnings > and the like. If so, I'd ask you to stop confuzzilating terminology. > > If you really want to capture such events, then you need to have the > kernel write the panic to (e.g.) flash - see the mtdoops driver. I was talking about things prior the panic: OOMs, MMC write/read warnings, hung tasks, we also have local patches to produce a warning if the mutex is being held for too long or a task is starving on CPU time by hard/soft irqs (I hope I will design something like that for upstream). I've found those warnings useful to: (a) have an early message when the things are starting going bad. (b) analyze contentions or too large scale for a box or faulty hardware for non-reproducible issues just from logs. We use kexec to save the dmesg ringbuffer content after the panic. Thanks, Dmitry From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dmitry Safonov <0x7f454c46@gmail.com> Subject: Re: [PATCH 00/50] Add log level to show_stack() Date: Fri, 8 Nov 2019 21:08:30 +0000 Message-ID: <5a411348-be84-f592-ef2d-ffaec99332bd@gmail.com> References: <20191106030542.868541-1-dima@arista.com> <20191106092039.GT4131@hirez.programming.kicks-ass.net> <10db6fa1-5b17-ebe6-09e0-6335e09e4db8@arista.com> <20191106203440.GH3079@worktop.programming.kicks-ass.net> <20191108173045.GY25745@shell.armlinux.org.uk> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description :Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vwuFpo0UiDnzPFiimc03yYQ0I92lDoFU8ZPsbJmfvdQ=; b=QbbjxG0heFqxkZ XHsH/AccYNT3ZYzwIWV5Y4Do9840af3v/UCZo4UggQfpoScwFiSg5n8xLCuJaCm0/JoGKuWfqgvO1 rne9pxM2hPJyOU2Hbb7tCic1zipJQVIHJC4hnF4b1QHltR97UwRJJ48wIL7aftvtitfESfjccVZkf 07zUfLmRS+Hyd9lM698b5xAgTD9IUrJs3AEzH+myRzrF73tX29f15NjrHmc0opBODxiU1X+TUG2h2 xT9QiWLhGgRm2ZRKo0wZOD1siUqX1SnzDqfSn0/z67ZNpWQ/5Fc7dZgKT+lyUpazbWVGuqgZ+kqqr 9PAoIr7pr5KPW1NCFcCg==; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=leuvFNJRBbgvPXZJlD4yDG614p9ePBBBJqh6/bi0nWM=; b=rjflubiq8T92r7IsfuflmmOLhAbyeUiuHnf/rQSYhhlSmttRaTaotAvL5xYp4yOydT GvzpvT1wRN/Se5w5Bamn3iuJIAGtHWJ3FQg7uwfFHiBZYXDG6coXsfRkKjBQWYAriMTe 4kTrO4cphBn73YaGY9oVhRABcMWkQkwRmG/gGE9R1eQjZ3xrtTYtO4ekpHSWHoSNgXQt LOIHB3CeUqi5Hx5sAbRW5lNwEUAqSAnAREOPMl/n34UjrP0bXjp7N2IIvVwOE5dHTyoC y/6fNB8lMSjFK19DQSqAbg52NdT8Yh0NvqWS+ZavZPT5hVJ6jwjvtDCEDMZ1djztH8HA lzbQ== In-Reply-To: <20191108173045.GY25745@shell.armlinux.org.uk> Content-Language: en-US List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-snps-arc" Errors-To: linux-snps-arc-bounces+gla-linux-snps-arc=m.gmane.org@lists.infradead.org To: Russell King - ARM Linux admin , Dmitry Safonov Cc: Juri Lelli , linux-sh@vger.kernel.org, Catalin Marinas , Ben Segall , Guo Ren , Pavel Machek , Vincent Guittot , Paul Burton , Michael Ellerman , Geert Uytterhoeven , Mel Gorman , Jiri Slaby , Matt Turner , uclinux-h8-devel@lists.sourceforge.jp, Len Brown , linux-pm@vger.kernel.org, linux-um@lists.infradead.org, Thomas Gleixner , Dietmar Eggemann , Richard Henderson , Greg Kroah-Hartman , "Rafael J. Wysocki" , LKML , Ralf Baechle , Paul Mackerras <> On 11/8/19 5:30 PM, Russell King - ARM Linux admin wrote: > On Fri, Nov 08, 2019 at 04:28:30PM +0000, Dmitry Safonov wrote: [..] >> >> Well, the use-case for lower log-level is that everything goes into logs >> (/var/log/dmesg or /var/log/messages whatever rsyslog has settting). >> >> That has it's value: >> - after a failure (i.e. panic) messages, those were only signs that >> something goes wrong can be seen in logs which can give ideas what has >> happened. > > No they don't. When the kernel panics, userspace generally stops > running, so rsyslog won't be able to write them to /var/log/messages. > > How, by "kernel panics" I mean a real kernel panic, which probably > isn't what you're talking about there. You are probably talking > about the whole shebang of non-fatal kernel oops, kernel warnings > and the like. If so, I'd ask you to stop confuzzilating terminology. > > If you really want to capture such events, then you need to have the > kernel write the panic to (e.g.) flash - see the mtdoops driver. I was talking about things prior the panic: OOMs, MMC write/read warnings, hung tasks, we also have local patches to produce a warning if the mutex is being held for too long or a task is starving on CPU time by hard/soft irqs (I hope I will design something like that for upstream). I've found those warnings useful to: (a) have an early message when the things are starting going bad. (b) analyze contentions or too large scale for a box or faulty hardware for non-reproducible issues just from logs. We use kexec to save the dmesg ringbuffer content after the panic. Thanks, Dmitry