openbmc.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/3] Enable pstore for Rainier
@ 2020-10-02  6:34 Andrew Jeffery
  2020-10-02  6:34 ` [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops Andrew Jeffery
                   ` (4 more replies)
  0 siblings, 5 replies; 14+ messages in thread
From: Andrew Jeffery @ 2020-10-02  6:34 UTC (permalink / raw)
  To: joel; +Cc: openbmc

Hello,

This series adds pstore support to the Rainier platform for recovery of oopses
and panics.

Patch 3/3 is a minor cleanup. Only patch 1/3 is a requirement as 2/3 is handled
by the config snippet in the bitbake metadata.

Please review!

Andrew

Andrew Jeffery (3):
  ARM: dts: rainier: Add reserved memory for ramoops
  ARM: config: Enable PSTORE in aspeed_g5_defconfig
  ARM: dts: rainier: Don't shout addresses

 arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 12 ++++++++++--
 arch/arm/configs/aspeed_g5_defconfig         |  4 ++++
 2 files changed, 14 insertions(+), 2 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops
  2020-10-02  6:34 [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
@ 2020-10-02  6:34 ` Andrew Jeffery
  2020-10-06  3:22   ` Joel Stanley
  2020-10-06  4:25   ` Milton Miller II
  2020-10-02  6:34 ` [PATCH 2/3] ARM: config: Enable PSTORE in aspeed_g5_defconfig Andrew Jeffery
                   ` (3 subsequent siblings)
  4 siblings, 2 replies; 14+ messages in thread
From: Andrew Jeffery @ 2020-10-02  6:34 UTC (permalink / raw)
  To: joel; +Cc: openbmc

Reserve a 1MiB region of memory to record kmsg dumps and console state
into 16kiB ring-buffer slots. The sizing allows for up to 32 dumps to be
captured and read out.

Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
---
 arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
index e6f422edf454..46a0e95049fd 100644
--- a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
+++ b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
@@ -47,6 +47,14 @@ reserved-memory {
 		#size-cells = <1>;
 		ranges;
 
+		ramoops@b7f00000 {
+			compatible = "ramoops";
+			reg = <0xb7f00000 0x100000>;
+			record-size = <0x4000>;
+			console-size = <0x4000>;
+			pmsg-size = <0x4000>;
+		};
+
 		flash_memory: region@B8000000 {
 			no-map;
 			reg = <0xB8000000 0x04000000>; /* 64M */
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH 2/3] ARM: config: Enable PSTORE in aspeed_g5_defconfig
  2020-10-02  6:34 [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
  2020-10-02  6:34 ` [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops Andrew Jeffery
@ 2020-10-02  6:34 ` Andrew Jeffery
  2020-10-06  3:51   ` Joel Stanley
  2020-10-02  6:34 ` [PATCH 3/3] ARM: dts: rainier: Don't shout addresses Andrew Jeffery
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 14+ messages in thread
From: Andrew Jeffery @ 2020-10-02  6:34 UTC (permalink / raw)
  To: joel; +Cc: openbmc

We're making use of it on IBM's Rainier system.

Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
---
 arch/arm/configs/aspeed_g5_defconfig | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/arm/configs/aspeed_g5_defconfig b/arch/arm/configs/aspeed_g5_defconfig
index 2bacd8c90f4b..c52db992b84e 100644
--- a/arch/arm/configs/aspeed_g5_defconfig
+++ b/arch/arm/configs/aspeed_g5_defconfig
@@ -274,6 +274,10 @@ CONFIG_UBIFS_FS=y
 CONFIG_SQUASHFS=y
 CONFIG_SQUASHFS_XZ=y
 CONFIG_SQUASHFS_ZSTD=y
+CONFIG_PSTORE=y
+CONFIG_PSTORE_CONSOLE=y
+CONFIG_PSTORE_PMSG=y
+CONFIG_PSTORE_RAM=y
 # CONFIG_NETWORK_FILESYSTEMS is not set
 CONFIG_HARDENED_USERCOPY=y
 CONFIG_FORTIFY_SOURCE=y
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH 3/3] ARM: dts: rainier: Don't shout addresses
  2020-10-02  6:34 [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
  2020-10-02  6:34 ` [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops Andrew Jeffery
  2020-10-02  6:34 ` [PATCH 2/3] ARM: config: Enable PSTORE in aspeed_g5_defconfig Andrew Jeffery
@ 2020-10-02  6:34 ` Andrew Jeffery
  2020-10-06  3:51   ` Joel Stanley
  2020-10-02  6:38 ` [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
  2020-10-07  7:21 ` Joel Stanley
  4 siblings, 1 reply; 14+ messages in thread
From: Andrew Jeffery @ 2020-10-02  6:34 UTC (permalink / raw)
  To: joel; +Cc: openbmc

Make them lowercase.

Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
---
 arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
index 46a0e95049fd..2e9206b65883 100644
--- a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
+++ b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
@@ -55,9 +55,9 @@ ramoops@b7f00000 {
 			pmsg-size = <0x4000>;
 		};
 
-		flash_memory: region@B8000000 {
+		flash_memory: region@b8000000 {
 			no-map;
-			reg = <0xB8000000 0x04000000>; /* 64M */
+			reg = <0xb8000000 0x04000000>; /* 64M */
 		};
 
 		vga_memory: region@bf000000 {
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH 0/3] Enable pstore for Rainier
  2020-10-02  6:34 [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
                   ` (2 preceding siblings ...)
  2020-10-02  6:34 ` [PATCH 3/3] ARM: dts: rainier: Don't shout addresses Andrew Jeffery
@ 2020-10-02  6:38 ` Andrew Jeffery
  2020-10-07  7:21 ` Joel Stanley
  4 siblings, 0 replies; 14+ messages in thread
From: Andrew Jeffery @ 2020-10-02  6:38 UTC (permalink / raw)
  To: Joel Stanley; +Cc: openbmc



On Fri, 2 Oct 2020, at 16:04, Andrew Jeffery wrote:
> Hello,
> 
> This series adds pstore support to the Rainier platform for recovery of oopses
> and panics.
> 
> Patch 3/3 is a minor cleanup. Only patch 1/3 is a requirement as 2/3 is handled
> by the config snippet in the bitbake metadata.
> 
> Please review!

`--subject-prefix "PATCH linux dev-5.8"` etc

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops
  2020-10-02  6:34 ` [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops Andrew Jeffery
@ 2020-10-06  3:22   ` Joel Stanley
  2020-10-16  3:38     ` Andrew Jeffery
  2020-10-06  4:25   ` Milton Miller II
  1 sibling, 1 reply; 14+ messages in thread
From: Joel Stanley @ 2020-10-06  3:22 UTC (permalink / raw)
  To: Andrew Jeffery; +Cc: OpenBMC Maillist

On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au> wrote:
>
> Reserve a 1MiB region of memory to record kmsg dumps and console state
> into 16kiB ring-buffer slots. The sizing allows for up to 32 dumps to be
> captured and read out.
>
> Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
> ---
>  arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> index e6f422edf454..46a0e95049fd 100644
> --- a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> +++ b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> @@ -47,6 +47,14 @@ reserved-memory {
>                 #size-cells = <1>;
>                 ranges;
>
> +               ramoops@b7f00000 {
> +                       compatible = "ramoops";
> +                       reg = <0xb7f00000 0x100000>;
> +                       record-size = <0x4000>;
> +                       console-size = <0x4000>;

This is conserative. We've got plenty of space, how about we make it bigger?

$ git grep console-size *.dts* | grep -Po "0x([0-9]+)" | xargs printf
"%x\n" | sort -n
8000
8000
10000
10000
20000
20000
20000
20000
20000
60000
100000

The median is 128KB, which sounds reasonable.

$ git grep record-size *.dts* | grep -Po "0x([0-9]+)" | xargs printf "%x\n"
20000
400
400
20000
20000
20000
10000
10000
10000
10000
20000

64KB is the median record size.

> +                       pmsg-size = <0x4000>;

Do we want to add ftrace too?

Should we also add max-reason = KMSG_DUMP_EMERG?

Logging reboots and shutdowns is informative (you know if a reboot was
intentional or due to a crash that wasn't recorded) and allows for
testing.

Cheers,

Joel

> +               };
> +
>                 flash_memory: region@B8000000 {
>                         no-map;
>                         reg = <0xB8000000 0x04000000>; /* 64M */
> --
> 2.25.1
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 2/3] ARM: config: Enable PSTORE in aspeed_g5_defconfig
  2020-10-02  6:34 ` [PATCH 2/3] ARM: config: Enable PSTORE in aspeed_g5_defconfig Andrew Jeffery
@ 2020-10-06  3:51   ` Joel Stanley
  0 siblings, 0 replies; 14+ messages in thread
From: Joel Stanley @ 2020-10-06  3:51 UTC (permalink / raw)
  To: Andrew Jeffery; +Cc: OpenBMC Maillist

On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au> wrote:
>
> We're making use of it on IBM's Rainier system.
>
> Signed-off-by: Andrew Jeffery <andrew@aj.id.au>

Reviewed-by: Joel Stanley <joel@jms.id.au>

> ---
>  arch/arm/configs/aspeed_g5_defconfig | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/arch/arm/configs/aspeed_g5_defconfig b/arch/arm/configs/aspeed_g5_defconfig
> index 2bacd8c90f4b..c52db992b84e 100644
> --- a/arch/arm/configs/aspeed_g5_defconfig
> +++ b/arch/arm/configs/aspeed_g5_defconfig
> @@ -274,6 +274,10 @@ CONFIG_UBIFS_FS=y
>  CONFIG_SQUASHFS=y
>  CONFIG_SQUASHFS_XZ=y
>  CONFIG_SQUASHFS_ZSTD=y
> +CONFIG_PSTORE=y
> +CONFIG_PSTORE_CONSOLE=y
> +CONFIG_PSTORE_PMSG=y
> +CONFIG_PSTORE_RAM=y
>  # CONFIG_NETWORK_FILESYSTEMS is not set
>  CONFIG_HARDENED_USERCOPY=y
>  CONFIG_FORTIFY_SOURCE=y
> --
> 2.25.1
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 3/3] ARM: dts: rainier: Don't shout addresses
  2020-10-02  6:34 ` [PATCH 3/3] ARM: dts: rainier: Don't shout addresses Andrew Jeffery
@ 2020-10-06  3:51   ` Joel Stanley
  0 siblings, 0 replies; 14+ messages in thread
From: Joel Stanley @ 2020-10-06  3:51 UTC (permalink / raw)
  To: Andrew Jeffery; +Cc: OpenBMC Maillist

On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au> wrote:
>
> Make them lowercase.
>
> Signed-off-by: Andrew Jeffery <andrew@aj.id.au>

Reviewed-by: Joel Stanley <joel@jms.id.au>

> ---
>  arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> index 46a0e95049fd..2e9206b65883 100644
> --- a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> +++ b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> @@ -55,9 +55,9 @@ ramoops@b7f00000 {
>                         pmsg-size = <0x4000>;
>                 };
>
> -               flash_memory: region@B8000000 {
> +               flash_memory: region@b8000000 {
>                         no-map;
> -                       reg = <0xB8000000 0x04000000>; /* 64M */
> +                       reg = <0xb8000000 0x04000000>; /* 64M */
>                 };
>
>                 vga_memory: region@bf000000 {
> --
> 2.25.1
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops
  2020-10-02  6:34 ` [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops Andrew Jeffery
  2020-10-06  3:22   ` Joel Stanley
@ 2020-10-06  4:25   ` Milton Miller II
  2020-10-06  4:29     ` Andrew Jeffery
  2020-10-06  4:44     ` Milton Miller II
  1 sibling, 2 replies; 14+ messages in thread
From: Milton Miller II @ 2020-10-06  4:25 UTC (permalink / raw)
  To: Joel Stanley; +Cc: Andrew Jeffery, OpenBMC Maillist

On October 5, 2020 about 10:23 in some timezone, Joel Stanley wrote:
>Subject: [EXTERNAL] Re: [PATCH 1/3] ARM: dts: rainier: Add reserved
>memory for ramoops
>
>On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au> wrote:
>>
>> Reserve a 1MiB region of memory to record kmsg dumps and console
>state
>> into 16kiB ring-buffer slots. The sizing allows for up to 32 dumps
>to be
>> captured and read out.
>>
>> Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
>> ---
>>  arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>>
>> diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
>b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
>> index e6f422edf454..46a0e95049fd 100644
>> --- a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
>> +++ b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
>> @@ -47,6 +47,14 @@ reserved-memory {
>>                 #size-cells = <1>;
>>                 ranges;
>>
>> +               ramoops@b7f00000 {
>> +                       compatible = "ramoops";
>> +                       reg = <0xb7f00000 0x100000>;
>> +                       record-size = <0x4000>;
>> +                       console-size = <0x4000>;
>
>This is conserative. We've got plenty of space, how about we make it
>bigger?
>
>$ git grep console-size *.dts* | grep -Po "0x([0-9]+)" | xargs printf
>"%x\n" | sort -n
>8000
>8000
>10000
>10000
>20000
>20000
>20000
>20000
>20000
>60000
>100000
>
>The median is 128KB, which sounds reasonable.

How big is the dmesg after we are booted?   How big is the default 
kernel buffer for 2 cpus (the kernel config has a base size, but 
also a dynamic size with a base + n * cpu min at boot).

Having the console space larger than printk buffer will not help.

We could config the buffer up if we are not capturing enough of a 
boot and some runtime.

>
>$ git grep record-size *.dts* | grep -Po "0x([0-9]+)" | xargs printf
>"%x\n"
>20000
>400
>400
>20000
>20000
>20000
>10000
>10000
>10000
>10000
>20000
>
>64KB is the median record size.
>
This probably affects the effective compression, assuming
each block is a seperate compression input.

>> +                       pmsg-size = <0x4000>;
>
>Do we want to add ftrace too?
>
>Should we also add max-reason = KMSG_DUMP_EMERG?
>

The admin guide lists KMSG_DUMP_OOPS and KMSG_DUMP_PANIC ?
https://www.kernel.org/doc/html/latest/admin-guide/ramoops.html

We could have something monitoring for OOPS , copying to a log and 
then unlinking the pstore after committed.

>Logging reboots and shutdowns is informative (you know if a reboot
>was
>intentional or due to a crash that wasn't recorded) and allows for
>testing.
>

That should be exposed from audit logging?
but yes.

milton

>Cheers,
>
>Joel
>
>> +               };
>> +
>>                 flash_memory: region@B8000000 {
>>                         no-map;
>>                         reg = <0xB8000000 0x04000000>; /* 64M */
>> --
>> 2.25.1
>>
>
>


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops
  2020-10-06  4:25   ` Milton Miller II
@ 2020-10-06  4:29     ` Andrew Jeffery
  2020-10-06  4:44     ` Milton Miller II
  1 sibling, 0 replies; 14+ messages in thread
From: Andrew Jeffery @ 2020-10-06  4:29 UTC (permalink / raw)
  To: Milton Miller II, Joel Stanley; +Cc: OpenBMC Maillist



On Tue, 6 Oct 2020, at 14:55, Milton Miller II wrote:
> On October 5, 2020 about 10:23 in some timezone, Joel Stanley wrote:
> >Subject: [EXTERNAL] Re: [PATCH 1/3] ARM: dts: rainier: Add reserved
> >memory for ramoops
> >
> >On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au> wrote:
> >>
> >> Reserve a 1MiB region of memory to record kmsg dumps and console
> >state
> >> into 16kiB ring-buffer slots. The sizing allows for up to 32 dumps
> >to be
> >> captured and read out.
> >>
> >> Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
> >> ---
> >>  arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 8 ++++++++
> >>  1 file changed, 8 insertions(+)
> >>
> >> diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> >b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> >> index e6f422edf454..46a0e95049fd 100644
> >> --- a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> >> +++ b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> >> @@ -47,6 +47,14 @@ reserved-memory {
> >>                 #size-cells = <1>;
> >>                 ranges;
> >>
> >> +               ramoops@b7f00000 {
> >> +                       compatible = "ramoops";
> >> +                       reg = <0xb7f00000 0x100000>;
> >> +                       record-size = <0x4000>;
> >> +                       console-size = <0x4000>;
> >
> >This is conserative. We've got plenty of space, how about we make it
> >bigger?
> >
> >$ git grep console-size *.dts* | grep -Po "0x([0-9]+)" | xargs printf
> >"%x\n" | sort -n
> >8000
> >8000
> >10000
> >10000
> >20000
> >20000
> >20000
> >20000
> >20000
> >60000
> >100000
> >
> >The median is 128KB, which sounds reasonable.
> 
> How big is the dmesg after we are booted?   How big is the default 
> kernel buffer for 2 cpus (the kernel config has a base size, but 
> also a dynamic size with a base + n * cpu min at boot).
> 
> Having the console space larger than printk buffer will not help.
> 
> We could config the buffer up if we are not capturing enough of a 
> boot and some runtime.
> 
> >
> >$ git grep record-size *.dts* | grep -Po "0x([0-9]+)" | xargs printf
> >"%x\n"
> >20000
> >400
> >400
> >20000
> >20000
> >20000
> >10000
> >10000
> >10000
> >10000
> >20000
> >
> >64KB is the median record size.
> >
> This probably affects the effective compression, assuming
> each block is a seperate compression input.
> 
> >> +                       pmsg-size = <0x4000>;
> >
> >Do we want to add ftrace too?
> >
> >Should we also add max-reason = KMSG_DUMP_EMERG?
> >
> 
> The admin guide lists KMSG_DUMP_OOPS and KMSG_DUMP_PANIC ?
> https://www.kernel.org/doc/html/latest/admin-guide/ramoops.html
> 
> We could have something monitoring for OOPS , copying to a log and 
> then unlinking the pstore after committed.

systemd-pstore already does this for us, no further configuration required.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops
  2020-10-06  4:25   ` Milton Miller II
  2020-10-06  4:29     ` Andrew Jeffery
@ 2020-10-06  4:44     ` Milton Miller II
  1 sibling, 0 replies; 14+ messages in thread
From: Milton Miller II @ 2020-10-06  4:44 UTC (permalink / raw)
  To: Andrew Jeffery; +Cc: OpenBMC Maillist

On October 5, 2020 at about 11:30PM in some timezone, Andrew Jeffery wrote:
>On Tue, 6 Oct 2020, at 14:55, Milton Miller II wrote:
>> On October 5, 2020 about 10:23 in some timezone, Joel Stanley
>wrote:
>> >Subject: [EXTERNAL] Re: [PATCH 1/3] ARM: dts: rainier: Add
>reserved
>> >memory for ramoops
>> >
>> >On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au>
>wrote:
>> >>
>> >> Reserve a 1MiB region of memory to record kmsg dumps and console
>> >state
>> >> into 16kiB ring-buffer slots. The sizing allows for up to 32
>dumps
>> >to be
>> >> captured and read out.
>> >>
>> >> Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
...
>> The admin guide lists KMSG_DUMP_OOPS and KMSG_DUMP_PANIC ?>>
>INVALID URI REMOVED
>oc_html_latest_admin-2Dguide_ramoops.html&d=DwIBAg&c=jf_iaSHvJObTbx-s
>iA1ZOg&r=bvv7AJEECoRKBU02rcu4F5DWd-EwX8As2xrXeO9ZSo4&m=223EDL7j0GQPUf
>WYDv8kduEPnexbpo3b00uQAlK8YSo&s=MfYAUnU2h1TdWyBC7tQoG3fVUTBTwXTFurBsK
>oZw34E&e= 
>> 
>> We could have something monitoring for OOPS , copying to a log and 
>> then unlinking the pstore after committed.
>
>systemd-pstore already does this for us, no further configuration
>required.

Do we have something that creates service records for these messages?  
I was hoping for something like a PEL for the bmc software.

Not as part of this kernel series though.

milton



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 0/3] Enable pstore for Rainier
  2020-10-02  6:34 [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
                   ` (3 preceding siblings ...)
  2020-10-02  6:38 ` [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
@ 2020-10-07  7:21 ` Joel Stanley
  2020-10-10  2:50   ` Andrew Jeffery
  4 siblings, 1 reply; 14+ messages in thread
From: Joel Stanley @ 2020-10-07  7:21 UTC (permalink / raw)
  To: Andrew Jeffery; +Cc: OpenBMC Maillist

On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au> wrote:
>
> Hello,
>
> This series adds pstore support to the Rainier platform for recovery of oopses
> and panics.
>
> Patch 3/3 is a minor cleanup. Only patch 1/3 is a requirement as 2/3 is handled
> by the config snippet in the bitbake metadata.

I merged 2 and 3. lmk what you want to do with the configuration.

I suggest we enable it for Tacoma too.

>
> Please review!
>
> Andrew
>
> Andrew Jeffery (3):
>   ARM: dts: rainier: Add reserved memory for ramoops
>   ARM: config: Enable PSTORE in aspeed_g5_defconfig
>   ARM: dts: rainier: Don't shout addresses
>
>  arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 12 ++++++++++--
>  arch/arm/configs/aspeed_g5_defconfig         |  4 ++++
>  2 files changed, 14 insertions(+), 2 deletions(-)
>
> --
> 2.25.1
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 0/3] Enable pstore for Rainier
  2020-10-07  7:21 ` Joel Stanley
@ 2020-10-10  2:50   ` Andrew Jeffery
  0 siblings, 0 replies; 14+ messages in thread
From: Andrew Jeffery @ 2020-10-10  2:50 UTC (permalink / raw)
  To: Joel Stanley; +Cc: OpenBMC Maillist



On Wed, 7 Oct 2020, at 17:51, Joel Stanley wrote:
> On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au> wrote:
> >
> > Hello,
> >
> > This series adds pstore support to the Rainier platform for recovery of oopses
> > and panics.
> >
> > Patch 3/3 is a minor cleanup. Only patch 1/3 is a requirement as 2/3 is handled
> > by the config snippet in the bitbake metadata.
> 
> I merged 2 and 3. lmk what you want to do with the configuration.
> 
> I suggest we enable it for Tacoma too.

Yeah I'll respond and enable it on Tacoma as well

> 
> >
> > Please review!
> >
> > Andrew
> >
> > Andrew Jeffery (3):
> >   ARM: dts: rainier: Add reserved memory for ramoops
> >   ARM: config: Enable PSTORE in aspeed_g5_defconfig
> >   ARM: dts: rainier: Don't shout addresses
> >
> >  arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 12 ++++++++++--
> >  arch/arm/configs/aspeed_g5_defconfig         |  4 ++++
> >  2 files changed, 14 insertions(+), 2 deletions(-)
> >
> > --
> > 2.25.1
> >
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops
  2020-10-06  3:22   ` Joel Stanley
@ 2020-10-16  3:38     ` Andrew Jeffery
  0 siblings, 0 replies; 14+ messages in thread
From: Andrew Jeffery @ 2020-10-16  3:38 UTC (permalink / raw)
  To: Joel Stanley; +Cc: OpenBMC Maillist



On Tue, 6 Oct 2020, at 13:52, Joel Stanley wrote:
> On Fri, 2 Oct 2020 at 06:35, Andrew Jeffery <andrew@aj.id.au> wrote:
> >
> > Reserve a 1MiB region of memory to record kmsg dumps and console state
> > into 16kiB ring-buffer slots. The sizing allows for up to 32 dumps to be
> > captured and read out.
> >
> > Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
> > ---
> >  arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> >
> > diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> > index e6f422edf454..46a0e95049fd 100644
> > --- a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> > +++ b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
> > @@ -47,6 +47,14 @@ reserved-memory {
> >                 #size-cells = <1>;
> >                 ranges;
> >
> > +               ramoops@b7f00000 {
> > +                       compatible = "ramoops";
> > +                       reg = <0xb7f00000 0x100000>;
> > +                       record-size = <0x4000>;
> > +                       console-size = <0x4000>;
> 
> This is conserative. We've got plenty of space, how about we make it bigger?
> 
> $ git grep console-size *.dts* | grep -Po "0x([0-9]+)" | xargs printf
> "%x\n" | sort -n
> 8000
> 8000
> 10000
> 10000
> 20000
> 20000
> 20000
> 20000
> 20000
> 60000
> 100000
> 
> The median is 128KB, which sounds reasonable.

Well, maybe? Why should we basing it on what everyone else has done rather than 
what we need? We're compressing the data before it's written to the pstore 
ring. Uncompressed, 16k is in the order of 200 lines of text. With the default 
DEFLATE compression we can fit 3-4x more:

root@rain15bmc:~# uptime
 14:44:20 up 14:44,  load average: 0.01, 0.01, 0.01
root@rain15bmc:~# dmesg | wc -l
640
root@rain15bmc:~# dmesg  >/tmp/dmesg
root@rain15bmc:~# stat -c "%s" /tmp/dmesg
44032
root@rain15bmc:~# gzip /tmp/dmesg
root@rain15bmc:~# stat -c "%s" /tmp/dmesg.gz
11059

I think 16k is more than enough for now?

> 
> $ git grep record-size *.dts* | grep -Po "0x([0-9]+)" | xargs printf "%x\n"
> 20000
> 400
> 400
> 20000
> 20000
> 20000
> 10000
> 10000
> 10000
> 10000
> 20000
> 
> 64KB is the median record size.
> 
> > +                       pmsg-size = <0x4000>;
> 
> Do we want to add ftrace too?

I figured getting an oops/panic stack dump might be enough for the moment.

> 
> Should we also add max-reason = KMSG_DUMP_EMERG?
> 
> Logging reboots and shutdowns is informative (you know if a reboot was
> intentional or due to a crash that wasn't recorded) and allows for
> testing.

Yeah, that's a good idea.

Thanks.

Andrew

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2020-10-16  3:39 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-02  6:34 [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
2020-10-02  6:34 ` [PATCH 1/3] ARM: dts: rainier: Add reserved memory for ramoops Andrew Jeffery
2020-10-06  3:22   ` Joel Stanley
2020-10-16  3:38     ` Andrew Jeffery
2020-10-06  4:25   ` Milton Miller II
2020-10-06  4:29     ` Andrew Jeffery
2020-10-06  4:44     ` Milton Miller II
2020-10-02  6:34 ` [PATCH 2/3] ARM: config: Enable PSTORE in aspeed_g5_defconfig Andrew Jeffery
2020-10-06  3:51   ` Joel Stanley
2020-10-02  6:34 ` [PATCH 3/3] ARM: dts: rainier: Don't shout addresses Andrew Jeffery
2020-10-06  3:51   ` Joel Stanley
2020-10-02  6:38 ` [PATCH 0/3] Enable pstore for Rainier Andrew Jeffery
2020-10-07  7:21 ` Joel Stanley
2020-10-10  2:50   ` Andrew Jeffery

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).