[PATCH] drivers: hv: log when enabling crash_kexec_post_notifiers

[PATCH] drivers: hv: log when enabling crash_kexec_post_notifiers

[Stephen Brennan]

Recently I went down a rabbit hole looking at a race condition in
panic() on a Hyper-V guest. I assumed, since it was missing from the
command line, that crash_kexec_post_notifiers was disabled. Only after
a rather long reproduction and analysis process did I learn that Hyper-V
actually enables this setting unconditionally.

Users and debuggers alike would like to know when these things happen. I
think it would be good to print a message to the kernel log when this
happens, so that a grep for "crash_kexec_post_notifiers" shows relevant
results.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
---
 drivers/hv/hv_common.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
index 181d16bbf49d..c1dd21d0d7ef 100644
--- a/drivers/hv/hv_common.c
+++ b/drivers/hv/hv_common.c
@@ -79,8 +79,10 @@ int __init hv_common_init(void)
 	 * calling crash enlightment interface before running kdump
 	 * kernel.
 	 */
-	if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
+	if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE) {
 		crash_kexec_post_notifiers = true;
+		pr_info("Hyper-V: enabling crash_kexec_post_notifiers\n");
+	}
 
 	/*
 	 * Allocate the per-CPU state for the hypercall input arg.
-- 
2.30.2

RE: [PATCH] drivers: hv: log when enabling crash_kexec_post_notifiers

[Michael Kelley (LINUX)]

From: Stephen Brennan <stephen.s.brennan@oracle.com> Sent: Monday, February 14, 2022 5:38 PM
> 
> Recently I went down a rabbit hole looking at a race condition in
> panic() on a Hyper-V guest. I assumed, since it was missing from the
> command line, that crash_kexec_post_notifiers was disabled. Only after
> a rather long reproduction and analysis process did I learn that Hyper-V
> actually enables this setting unconditionally.
> 
> Users and debuggers alike would like to know when these things happen. I
> think it would be good to print a message to the kernel log when this
> happens, so that a grep for "crash_kexec_post_notifiers" shows relevant
> results.

I'm OK with adding this output line.  However, you have probably
seen the two other LKML threads [1] and [2] about reorganizing the
panic notifiers to clearly distinguish between notifiers that always run
vs. those controlled by "crash_kexec_post_notifiers".  If the changes
proposed in those threads are submitted and accepted, it is likely that
the kernel log message in this patch would become unnecessary.
But since we don't know when those proposed changes might come
to fruition, adding the message for now makes sense.

Reviewed-by: Michael Kelley <mikelley@microsoft.com>

[1] https://lore.kernel.org/lkml/20220108153451.195121-1-gpiccoli@igalia.com/
[2] https://lore.kernel.org/lkml/20220114183046.428796-1-gpiccoli@igalia.com/

> 
> Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
> ---
>  drivers/hv/hv_common.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
> index 181d16bbf49d..c1dd21d0d7ef 100644
> --- a/drivers/hv/hv_common.c
> +++ b/drivers/hv/hv_common.c
> @@ -79,8 +79,10 @@ int __init hv_common_init(void)
>  	 * calling crash enlightment interface before running kdump
>  	 * kernel.
>  	 */
> -	if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
> +	if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE) {
>  		crash_kexec_post_notifiers = true;
> +		pr_info("Hyper-V: enabling crash_kexec_post_notifiers\n");
> +	}
> 
>  	/*
>  	 * Allocate the per-CPU state for the hypercall input arg.
> --
> 2.30.2

Re: [PATCH] drivers: hv: log when enabling crash_kexec_post_notifiers

[Stephen Brennan]

On 2/17/22 08:44, Michael Kelley (LINUX) wrote:
> From: Stephen Brennan <stephen.s.brennan@oracle.com> Sent: Monday, February 14, 2022 5:38 PM
>>
>> Recently I went down a rabbit hole looking at a race condition in
>> panic() on a Hyper-V guest. I assumed, since it was missing from the
>> command line, that crash_kexec_post_notifiers was disabled. Only after
>> a rather long reproduction and analysis process did I learn that Hyper-V
>> actually enables this setting unconditionally.
>>
>> Users and debuggers alike would like to know when these things happen. I
>> think it would be good to print a message to the kernel log when this
>> happens, so that a grep for "crash_kexec_post_notifiers" shows relevant
>> results.
> 
> I'm OK with adding this output line.  However, you have probably
> seen the two other LKML threads [1] and [2] about reorganizing the
> panic notifiers to clearly distinguish between notifiers that always run
> vs. those controlled by "crash_kexec_post_notifiers". 

Yeah, I fired this off before seeing those patches. I need to get on top 
of the "lore+lei" combination so I can see these discussions quickly, as 
there's no subsystem-specific mailing list for the panic/printk stuff. 
The patches only surface if you're mentioned or if you trawl through 
LKML itself :)

> If the changes
> proposed in those threads are submitted and accepted, it is likely that
> the kernel log message in this patch would become unnecessary.
> But since we don't know when those proposed changes might come
> to fruition, adding the message for now makes sense.
> 
> Reviewed-by: Michael Kelley <mikelley@microsoft.com>

Thank you!
Stephen

> 
> [1] https://lore.kernel.org/lkml/20220108153451.195121-1-gpiccoli@igalia.com/
> [2] https://lore.kernel.org/lkml/20220114183046.428796-1-gpiccoli@igalia.com/
> 
>>
>> Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
>> ---
>>   drivers/hv/hv_common.c | 4 +++-
>>   1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
>> index 181d16bbf49d..c1dd21d0d7ef 100644
>> --- a/drivers/hv/hv_common.c
>> +++ b/drivers/hv/hv_common.c
>> @@ -79,8 +79,10 @@ int __init hv_common_init(void)
>>   	 * calling crash enlightment interface before running kdump
>>   	 * kernel.
>>   	 */
>> -	if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
>> +	if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE) {
>>   		crash_kexec_post_notifiers = true;
>> +		pr_info("Hyper-V: enabling crash_kexec_post_notifiers\n");
>> +	}
>>
>>   	/*
>>   	 * Allocate the per-CPU state for the hypercall input arg.
>> --
>> 2.30.2
> 

Re: [PATCH] drivers: hv: log when enabling crash_kexec_post_notifiers

[Wei Liu]

On Thu, Feb 17, 2022 at 04:44:13PM +0000, Michael Kelley (LINUX) wrote:
> From: Stephen Brennan <stephen.s.brennan@oracle.com> Sent: Monday, February 14, 2022 5:38 PM
> > 
> > Recently I went down a rabbit hole looking at a race condition in
> > panic() on a Hyper-V guest. I assumed, since it was missing from the
> > command line, that crash_kexec_post_notifiers was disabled. Only after
> > a rather long reproduction and analysis process did I learn that Hyper-V
> > actually enables this setting unconditionally.
> > 
> > Users and debuggers alike would like to know when these things happen. I
> > think it would be good to print a message to the kernel log when this
> > happens, so that a grep for "crash_kexec_post_notifiers" shows relevant
> > results.
> 
> I'm OK with adding this output line.  However, you have probably
> seen the two other LKML threads [1] and [2] about reorganizing the
> panic notifiers to clearly distinguish between notifiers that always run
> vs. those controlled by "crash_kexec_post_notifiers".  If the changes
> proposed in those threads are submitted and accepted, it is likely that
> the kernel log message in this patch would become unnecessary.
> But since we don't know when those proposed changes might come
> to fruition, adding the message for now makes sense.
> 
> Reviewed-by: Michael Kelley <mikelley@microsoft.com>

Applied to hyperv-next. Thanks.