[PATCH 0/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[PATCH 0/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[Seiji Aguchi]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2071 bytes --]

Hi,

This series aims to develop logging facility for enterprise use.

It is important to save kernel messages reliably on enterprise system 
because they are helpful for diagnosing system.

This series add kmsg_dump() to the paths loosing kernel messages.
The use case is the following.

[Use case of reboot/poweroff/halt/emergency_restart]

 My company has often experienced the followings in our support service.
 - Customer's system suddenly reboots.
 - Customers ask us to investigate the reason of the reboot.

 We recognize the fact itself because boot messages remain in /var/log/messages.
 However, we can't investigate the reason why the system rebooted,  because the last messages don't remain.
 And off course we can't explain the reason.
 
 
 We can solve above problem with this patch as follows.
 Case1: reboot with command 
   - We can see "Restarting system with command:" or ""Restarting system.".

 Case2: halt with command
   - We can see "System halted.".

 Case3: poweroff with command
   - We can see " Power down.".

 Case4: emergency_restart with sysrq.
   - We can see "Sysrq:" outputted in __handle_sysrq().

 Case5: emergency_restart with softdog.
   - We can see "Initiating system reboot" in watchdog_fire().

 So, we can distinguish the reason of reboot, poweroff, halt and emergency_restart.

 If customer executed reboot command, you may think the customer should know the fact.
 However, they often claim they don't execute the command when they rebooted system by mistake.

 No evidential message remain on current Linux kernel, so we can't show the proof to the customer.
 This patch improves this situation.


The first patch alters mtdoops and ramoops to perform their actions only for KMSG_DUMP_PANIC,
KMSG_DUMP_OOPS and KMSG_DUMP_KEXEC because they would like to log crashes only.

The latter patch adds kmsg_dump() to reboot, halt, poweroff and emergency_restart path.

ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

[PATCH 1/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[Seiji Aguchi]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1495 bytes --]

This patch alters mtdoops and ramoops to perform their actions only for KMSG_DUMP_PANIC,
KMSG_DUMP_OOPS and KMSG_DUMP_KEXEC because they would like to log crashes only.

 Signed-off-by: Seiji Aguchi <seiji.aguchi@hds.com>

---
 drivers/char/ramoops.c |    5 +++++
 drivers/mtd/mtdoops.c  |    5 +++++
 2 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/drivers/char/ramoops.c b/drivers/char/ramoops.c index 73dcb0e..8998b29 100644
--- a/drivers/char/ramoops.c
+++ b/drivers/char/ramoops.c
@@ -69,6 +69,11 @@ static void ramoops_do_dump(struct kmsg_dumper *dumper,
 	char *buf;
 	struct timeval timestamp;
 
+	if (reason != KMSG_DUMP_OOPS &&
+	    reason != KMSG_DUMP_PANIC &&
+	    reason != KMSG_DUMP_KEXEC)
+		return;
+
 	/* Only dump oopses if dump_oops is set */
 	if (reason == KMSG_DUMP_OOPS && !dump_oops)
 		return;
diff --git a/drivers/mtd/mtdoops.c b/drivers/mtd/mtdoops.c index 1ee72f3..c948150 100644
--- a/drivers/mtd/mtdoops.c
+++ b/drivers/mtd/mtdoops.c
@@ -307,6 +307,11 @@ static void mtdoops_do_dump(struct kmsg_dumper *dumper,
 	unsigned long l1_cpy, l2_cpy;
 	char *dst;
 
+	if (reason != KMSG_DUMP_OOPS &&
+	    reason != KMSG_DUMP_PANIC &&
+	    reason != KMSG_DUMP_KEXEC)
+		return;
+
 	/* Only dump oopses if dump_oops is set */
 	if (reason == KMSG_DUMP_OOPS && !dump_oops)
 		return;
--
1.7.2.2

ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

[PATCH 2/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[Seiji Aguchi]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2748 bytes --]

 We need to know the reason why system rebooted in support service.
 However, we can't inform our customers of the reason because
 final messages are lost on current Linux kernel.
 
 This patch improves the situation above because the final messages 
 are saved by adding kmsg_dump() to reboot, halt, poweroff and 
 emergency_restart path.

 Signed-off-by: Seiji Aguchi <seiji.aguchi@hds.com>

---
 include/linux/kmsg_dump.h |    4 ++++
 kernel/printk.c           |    4 ++++
 kernel/sys.c              |    6 ++++++
 3 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/include/linux/kmsg_dump.h b/include/linux/kmsg_dump.h
index 24b4414..2a0d7d6 100644
--- a/include/linux/kmsg_dump.h
+++ b/include/linux/kmsg_dump.h
@@ -18,6 +18,10 @@ enum kmsg_dump_reason {
 	KMSG_DUMP_OOPS,
 	KMSG_DUMP_PANIC,
 	KMSG_DUMP_KEXEC,
+	KMSG_DUMP_RESTART,
+	KMSG_DUMP_HALT,
+	KMSG_DUMP_POWEROFF,
+	KMSG_DUMP_EMERG,
 };
 
 /**
diff --git a/kernel/printk.c b/kernel/printk.c
index b2ebaee..c5648b4 100644
--- a/kernel/printk.c
+++ b/kernel/printk.c
@@ -1516,6 +1516,10 @@ static const char * const kmsg_reasons[] = {
 	[KMSG_DUMP_OOPS]	= "oops",
 	[KMSG_DUMP_PANIC]	= "panic",
 	[KMSG_DUMP_KEXEC]	= "kexec",
+	[KMSG_DUMP_RESTART]	= "restart",
+	[KMSG_DUMP_HALT]	= "halt",
+	[KMSG_DUMP_POWEROFF]	= "poweroff",
+	[KMSG_DUMP_EMERG]	= "emergency_restart",
 };
 
 static const char *kmsg_to_str(enum kmsg_dump_reason reason)
diff --git a/kernel/sys.c b/kernel/sys.c
index 7f5a0cd..3eab886 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -43,6 +43,8 @@
 #include <linux/kprobes.h>
 #include <linux/user_namespace.h>
 
+#include <linux/kmsg_dump.h>
+
 #include <asm/uaccess.h>
 #include <asm/io.h>
 #include <asm/unistd.h>
@@ -285,6 +287,7 @@ out_unlock:
  */
 void emergency_restart(void)
 {
+	kmsg_dump(KMSG_DUMP_EMERG);
 	machine_emergency_restart();
 }
 EXPORT_SYMBOL_GPL(emergency_restart);
@@ -312,6 +315,7 @@ void kernel_restart(char *cmd)
 		printk(KERN_EMERG "Restarting system.\n");
 	else
 		printk(KERN_EMERG "Restarting system with command '%s'.\n", cmd);
+	kmsg_dump(KMSG_DUMP_RESTART);
 	machine_restart(cmd);
 }
 EXPORT_SYMBOL_GPL(kernel_restart);
@@ -333,6 +337,7 @@ void kernel_halt(void)
 	kernel_shutdown_prepare(SYSTEM_HALT);
 	sysdev_shutdown();
 	printk(KERN_EMERG "System halted.\n");
+	kmsg_dump(KMSG_DUMP_HALT);
 	machine_halt();
 }
 
@@ -351,6 +356,7 @@ void kernel_power_off(void)
 	disable_nonboot_cpus();
 	sysdev_shutdown();
 	printk(KERN_EMERG "Power down.\n");
+	kmsg_dump(KMSG_DUMP_POWEROFF);
 	machine_power_off();
 }
 EXPORT_SYMBOL_GPL(kernel_power_off);
-- 
1.7.2.2
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [PATCH 0/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[Andrew Morton]

On Wed, 17 Nov 2010 09:58:18 -0500
Seiji Aguchi <seiji.aguchi@hds.com> wrote:

> Hi,
> 
> This series aims to develop logging facility for enterprise use.
> 
> It is important to save kernel messages reliably on enterprise system 
> because they are helpful for diagnosing system.
> 
> This series add kmsg_dump() to the paths loosing kernel messages.
> The use case is the following.
> 
> [Use case of reboot/poweroff/halt/emergency_restart]
> 
>  My company has often experienced the followings in our support service.
>  - Customer's system suddenly reboots.
>  - Customers ask us to investigate the reason of the reboot.
> 
>  We recognize the fact itself because boot messages remain in /var/log/messages.
>  However, we can't investigate the reason why the system rebooted,  because the last messages don't remain.
>  And off course we can't explain the reason.
>  
>  
>  We can solve above problem with this patch as follows.
>  Case1: reboot with command 
>    - We can see "Restarting system with command:" or ""Restarting system.".
> 
>  Case2: halt with command
>    - We can see "System halted.".
> 
>  Case3: poweroff with command
>    - We can see " Power down.".
> 
>  Case4: emergency_restart with sysrq.
>    - We can see "Sysrq:" outputted in __handle_sysrq().
> 
>  Case5: emergency_restart with softdog.
>    - We can see "Initiating system reboot" in watchdog_fire().
> 
>  So, we can distinguish the reason of reboot, poweroff, halt and emergency_restart.
> 
>  If customer executed reboot command, you may think the customer should know the fact.
>  However, they often claim they don't execute the command when they rebooted system by mistake.
> 
>  No evidential message remain on current Linux kernel, so we can't show the proof to the customer.
>  This patch improves this situation.
> 
> 
> The first patch alters mtdoops and ramoops to perform their actions only for KMSG_DUMP_PANIC,
> KMSG_DUMP_OOPS and KMSG_DUMP_KEXEC because they would like to log crashes only.
> 
> The latter patch adds kmsg_dump() to reboot, halt, poweroff and emergency_restart path.

Damn, that's a good changelog.  We can actually understand why you
wrote the patch, and see what its value is!

One thing: please don't send multiple patches with the same title. 
See Documentation/SubmittingPatches section 15.  I renamed these two
patches to

kmsg_dump: constrain mtdoops and ramoops to perform their actions only for KMSG_DUMP_PANIC

and

kmsg_dump: add kmsg_dump() calls to the reboot, halt, poweroff and emergency_restart paths

Re: [PATCH 1/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[Artem Bityutskiy]

On Wed, 2010-11-17 at 10:00 -0500, Seiji Aguchi wrote:
> This patch alters mtdoops and ramoops to perform their actions only for KMSG_DUMP_PANIC,
> KMSG_DUMP_OOPS and KMSG_DUMP_KEXEC because they would like to log crashes only.
> 
>  Signed-off-by: Seiji Aguchi <seiji.aguchi@hds.com>

Looks OK,

Reviewed-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [PATCH 2/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[Artem Bityutskiy]

On Wed, 2010-11-17 at 10:03 -0500, Seiji Aguchi wrote:
> We need to know the reason why system rebooted in support service.
>  However, we can't inform our customers of the reason because
>  final messages are lost on current Linux kernel.
>  
>  This patch improves the situation above because the final messages 
>  are saved by adding kmsg_dump() to reboot, halt, poweroff and 
>  emergency_restart path.
> 
>  Signed-off-by: Seiji Aguchi <seiji.aguchi@hds.com>

Looks ok,

Reviewed-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [PATCH 1/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[KOSAKI Motohiro]

> This patch alters mtdoops and ramoops to perform their actions only for KMSG_DUMP_PANIC,
> KMSG_DUMP_OOPS and KMSG_DUMP_KEXEC because they would like to log crashes only.
> 
>  Signed-off-by: Seiji Aguchi <seiji.aguchi@hds.com>
> 
> ---
>  drivers/char/ramoops.c |    5 +++++
>  drivers/mtd/mtdoops.c  |    5 +++++
>  2 files changed, 10 insertions(+), 0 deletions(-)
> 
> diff --git a/drivers/char/ramoops.c b/drivers/char/ramoops.c index 73dcb0e..8998b29 100644
> --- a/drivers/char/ramoops.c
> +++ b/drivers/char/ramoops.c
> @@ -69,6 +69,11 @@ static void ramoops_do_dump(struct kmsg_dumper *dumper,
>  	char *buf;
>  	struct timeval timestamp;
>  
> +	if (reason != KMSG_DUMP_OOPS &&
> +	    reason != KMSG_DUMP_PANIC &&
> +	    reason != KMSG_DUMP_KEXEC)
> +		return;
> +
>  	/* Only dump oopses if dump_oops is set */
>  	if (reason == KMSG_DUMP_OOPS && !dump_oops)
>  		return;
> diff --git a/drivers/mtd/mtdoops.c b/drivers/mtd/mtdoops.c index 1ee72f3..c948150 100644
> --- a/drivers/mtd/mtdoops.c
> +++ b/drivers/mtd/mtdoops.c
> @@ -307,6 +307,11 @@ static void mtdoops_do_dump(struct kmsg_dumper *dumper,
>  	unsigned long l1_cpy, l2_cpy;
>  	char *dst;
>  
> +	if (reason != KMSG_DUMP_OOPS &&
> +	    reason != KMSG_DUMP_PANIC &&
> +	    reason != KMSG_DUMP_KEXEC)
> +		return;
> +
>  	/* Only dump oopses if dump_oops is set */
>  	if (reason == KMSG_DUMP_OOPS && !dump_oops)
>  		return;

Looks good.
	Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>

Re: [PATCH 2/2] kmsg_dump: adding to reboot, halt, poweroff and emergency_restart path

[KOSAKI Motohiro]

>  We need to know the reason why system rebooted in support service.
>  However, we can't inform our customers of the reason because
>  final messages are lost on current Linux kernel.
>  
>  This patch improves the situation above because the final messages 
>  are saved by adding kmsg_dump() to reboot, halt, poweroff and 
>  emergency_restart path.
> 
>  Signed-off-by: Seiji Aguchi <seiji.aguchi@hds.com>

This patch doesn't have any notification consumer. It's too bad.
As far as I know, some poeple are now discussing to make generic
firmware access interface. perhaps your one can be integrate it.
Please see and join following discussion.


From: "Luck, Tony" <tony.luck@intel.com>
Subject: [RFC] persistent store