[PATCH] x86/split_lock: Sanitize userspace and guest error output

[PATCH] x86/split_lock: Sanitize userspace and guest error output

[Prarit Bhargava]

There are two problems with kernel messages in fatal mode that
were found during testing of guests and userspace programs.

The first is that no kernel message is output when the split lock detector
is triggered with a userspace program.  As a result the userspace process
dies from receiving SIGBUS with no indication to the user of what caused
the process to die.

The second problem is that only the first triggering guest causes a kernel
message to be output because the message is output with pr_warn_once().
This also results in a loss of information to the user.

While fixing these I noticed that the same message was being output
three times so I'm cleaning that up too.

Fix fatal mode output, and use consistent messages for fatal and
warn modes for both userspace and guests.

Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: x86@kernel.org
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>
Cc: Sean Christopherson <sean.j.christopherson@intel.com>
Cc: Rahul Tanwar <rahul.tanwar@linux.intel.com>
Cc: Xiaoyao Li <xiaoyao.li@intel.com>
Cc: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
---
 arch/x86/kernel/cpu/intel.c | 24 ++++++++++--------------
 1 file changed, 10 insertions(+), 14 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 166d7c355896..463022aa9b7a 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -1074,10 +1074,14 @@ static void split_lock_init(void)
 	split_lock_verify_msr(sld_state != sld_off);
 }
 
-static void split_lock_warn(unsigned long ip)
+static bool split_lock_warn(unsigned long ip, int fatal)
 {
-	pr_warn_ratelimited("#AC: %s/%d took a split_lock trap at address: 0x%lx\n",
-			    current->comm, current->pid, ip);
+	pr_warn_ratelimited("#AC: %s/%d %ssplit_lock trap at address: 0x%lx\n",
+			    current->comm, current->pid,
+			    sld_state == sld_fatal ? "fatal " : "", ip);
+
+	if (sld_state == sld_fatal || fatal)
+		return false;
 
 	/*
 	 * Disable the split lock detection for this task so it can make
@@ -1086,18 +1090,13 @@ static void split_lock_warn(unsigned long ip)
 	 */
 	sld_update_msr(false);
 	set_tsk_thread_flag(current, TIF_SLD);
+	return true;
 }
 
 bool handle_guest_split_lock(unsigned long ip)
 {
-	if (sld_state == sld_warn) {
-		split_lock_warn(ip);
+	if (split_lock_warn(ip, 0))
 		return true;
-	}
-
-	pr_warn_once("#AC: %s/%d %s split_lock trap at address: 0x%lx\n",
-		     current->comm, current->pid,
-		     sld_state == sld_fatal ? "fatal" : "bogus", ip);
 
 	current->thread.error_code = 0;
 	current->thread.trap_nr = X86_TRAP_AC;
@@ -1108,10 +1107,7 @@ EXPORT_SYMBOL_GPL(handle_guest_split_lock);
 
 bool handle_user_split_lock(struct pt_regs *regs, long error_code)
 {
-	if ((regs->flags & X86_EFLAGS_AC) || sld_state == sld_fatal)
-		return false;
-	split_lock_warn(regs->ip);
-	return true;
+	return split_lock_warn(regs->ip, regs->flags & X86_EFLAGS_AC);
 }
 
 /*
-- 
2.21.3

Re: [PATCH] x86/split_lock: Sanitize userspace and guest error output

[Xiaoyao Li]

On 6/5/2020 7:44 PM, Prarit Bhargava wrote:
> There are two problems with kernel messages in fatal mode that
> were found during testing of guests and userspace programs.
> 
> The first is that no kernel message is output when the split lock detector
> is triggered with a userspace program.  As a result the userspace process
> dies from receiving SIGBUS with no indication to the user of what caused
> the process to die.
> 
> The second problem is that only the first triggering guest causes a kernel
> message to be output because the message is output with pr_warn_once().
> This also results in a loss of information to the user.
> 
> While fixing these I noticed that the same message was being output
> three times so I'm cleaning that up too.
> 
> Fix fatal mode output, and use consistent messages for fatal and
> warn modes for both userspace and guests.
> 
> Signed-off-by: Prarit Bhargava <prarit@redhat.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: x86@kernel.org
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Tony Luck <tony.luck@intel.com>
> Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>
> Cc: Sean Christopherson <sean.j.christopherson@intel.com>
> Cc: Rahul Tanwar <rahul.tanwar@linux.intel.com>
> Cc: Xiaoyao Li <xiaoyao.li@intel.com>
> Cc: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> ---
>   arch/x86/kernel/cpu/intel.c | 24 ++++++++++--------------
>   1 file changed, 10 insertions(+), 14 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
> index 166d7c355896..463022aa9b7a 100644
> --- a/arch/x86/kernel/cpu/intel.c
> +++ b/arch/x86/kernel/cpu/intel.c
> @@ -1074,10 +1074,14 @@ static void split_lock_init(void)
>   	split_lock_verify_msr(sld_state != sld_off);
>   }
>   
> -static void split_lock_warn(unsigned long ip)
> +static bool split_lock_warn(unsigned long ip, int fatal)
>   {
> -	pr_warn_ratelimited("#AC: %s/%d took a split_lock trap at address: 0x%lx\n",
> -			    current->comm, current->pid, ip);
> +	pr_warn_ratelimited("#AC: %s/%d %ssplit_lock trap at address: 0x%lx\n",
> +			    current->comm, current->pid,
> +			    sld_state == sld_fatal ? "fatal " : "", ip);
> +
> +	if (sld_state == sld_fatal || fatal)
> +		return false;
>   
>   	/*
>   	 * Disable the split lock detection for this task so it can make
> @@ -1086,18 +1090,13 @@ static void split_lock_warn(unsigned long ip)
>   	 */
>   	sld_update_msr(false);
>   	set_tsk_thread_flag(current, TIF_SLD);
> +	return true;
>   }
>   
>   bool handle_guest_split_lock(unsigned long ip)
>   {
> -	if (sld_state == sld_warn) {
> -		split_lock_warn(ip);
> +	if (split_lock_warn(ip, 0))
>   		return true;
> -	}
> -
> -	pr_warn_once("#AC: %s/%d %s split_lock trap at address: 0x%lx\n",
> -		     current->comm, current->pid,
> -		     sld_state == sld_fatal ? "fatal" : "bogus", ip);
>   
>   	current->thread.error_code = 0;
>   	current->thread.trap_nr = X86_TRAP_AC;
> @@ -1108,10 +1107,7 @@ EXPORT_SYMBOL_GPL(handle_guest_split_lock);
>   
>   bool handle_user_split_lock(struct pt_regs *regs, long error_code)
>   {
> -	if ((regs->flags & X86_EFLAGS_AC) || sld_state == sld_fatal)
> -		return false;
> -	split_lock_warn(regs->ip);
> -	return true;
> +	return split_lock_warn(regs->ip, regs->flags & X86_EFLAGS_AC);

It's incorrect. You change the behavior that it will print the split 
lock warning even when CPL 3 Alignment Check is turned on.

>   }
>   
>   /*
> 

Re: [PATCH] x86/split_lock: Sanitize userspace and guest error output

[Prarit Bhargava]

On 6/5/20 11:29 AM, Xiaoyao Li wrote:
> On 6/5/2020 7:44 PM, Prarit Bhargava wrote:
>> There are two problems with kernel messages in fatal mode that
>> were found during testing of guests and userspace programs.
>>
>> The first is that no kernel message is output when the split lock detector
>> is triggered with a userspace program.  As a result the userspace process
>> dies from receiving SIGBUS with no indication to the user of what caused
>> the process to die.
>>
>> The second problem is that only the first triggering guest causes a kernel
>> message to be output because the message is output with pr_warn_once().
>> This also results in a loss of information to the user.
>>
>> While fixing these I noticed that the same message was being output
>> three times so I'm cleaning that up too.
>>
>> Fix fatal mode output, and use consistent messages for fatal and
>> warn modes for both userspace and guests.
>>
>> Signed-off-by: Prarit Bhargava <prarit@redhat.com>
>> Cc: Thomas Gleixner <tglx@linutronix.de>
>> Cc: Ingo Molnar <mingo@redhat.com>
>> Cc: Borislav Petkov <bp@alien8.de>
>> Cc: x86@kernel.org
>> Cc: "H. Peter Anvin" <hpa@zytor.com>
>> Cc: Tony Luck <tony.luck@intel.com>
>> Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>
>> Cc: Sean Christopherson <sean.j.christopherson@intel.com>
>> Cc: Rahul Tanwar <rahul.tanwar@linux.intel.com>
>> Cc: Xiaoyao Li <xiaoyao.li@intel.com>
>> Cc: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
>> Cc: Dave Hansen <dave.hansen@linux.intel.com>
>> ---
>>   arch/x86/kernel/cpu/intel.c | 24 ++++++++++--------------
>>   1 file changed, 10 insertions(+), 14 deletions(-)
>>
>> diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
>> index 166d7c355896..463022aa9b7a 100644
>> --- a/arch/x86/kernel/cpu/intel.c
>> +++ b/arch/x86/kernel/cpu/intel.c
>> @@ -1074,10 +1074,14 @@ static void split_lock_init(void)
>>       split_lock_verify_msr(sld_state != sld_off);
>>   }
>>   -static void split_lock_warn(unsigned long ip)
>> +static bool split_lock_warn(unsigned long ip, int fatal)
>>   {
>> -    pr_warn_ratelimited("#AC: %s/%d took a split_lock trap at address: 0x%lx\n",
>> -                current->comm, current->pid, ip);
>> +    pr_warn_ratelimited("#AC: %s/%d %ssplit_lock trap at address: 0x%lx\n",
>> +                current->comm, current->pid,
>> +                sld_state == sld_fatal ? "fatal " : "", ip);
>> +
>> +    if (sld_state == sld_fatal || fatal)
>> +        return false;
>>         /*
>>        * Disable the split lock detection for this task so it can make
>> @@ -1086,18 +1090,13 @@ static void split_lock_warn(unsigned long ip)
>>        */
>>       sld_update_msr(false);
>>       set_tsk_thread_flag(current, TIF_SLD);
>> +    return true;
>>   }
>>     bool handle_guest_split_lock(unsigned long ip)
>>   {
>> -    if (sld_state == sld_warn) {
>> -        split_lock_warn(ip);
>> +    if (split_lock_warn(ip, 0))
>>           return true;
>> -    }
>> -
>> -    pr_warn_once("#AC: %s/%d %s split_lock trap at address: 0x%lx\n",
>> -             current->comm, current->pid,
>> -             sld_state == sld_fatal ? "fatal" : "bogus", ip);
>>         current->thread.error_code = 0;
>>       current->thread.trap_nr = X86_TRAP_AC;
>> @@ -1108,10 +1107,7 @@ EXPORT_SYMBOL_GPL(handle_guest_split_lock);
>>     bool handle_user_split_lock(struct pt_regs *regs, long error_code)
>>   {
>> -    if ((regs->flags & X86_EFLAGS_AC) || sld_state == sld_fatal)
>> -        return false;
>> -    split_lock_warn(regs->ip);
>> -    return true;
>> +    return split_lock_warn(regs->ip, regs->flags & X86_EFLAGS_AC);
> 
> It's incorrect. You change the behavior that it will print the split lock
> warning even when CPL 3 Alignment Check is turned on.

Do you want the message to be displayed in the fatal case of CPL 3 Alignment check?

P.

Re: [PATCH] x86/split_lock: Sanitize userspace and guest error output

[Xiaoyao Li]

On 6/6/2020 12:42 AM, Prarit Bhargava wrote:
> 
> 
> On 6/5/20 11:29 AM, Xiaoyao Li wrote:
>> On 6/5/2020 7:44 PM, Prarit Bhargava wrote:
>>> There are two problems with kernel messages in fatal mode that
>>> were found during testing of guests and userspace programs.
>>>
>>> The first is that no kernel message is output when the split lock detector
>>> is triggered with a userspace program.  As a result the userspace process
>>> dies from receiving SIGBUS with no indication to the user of what caused
>>> the process to die.
>>>
>>> The second problem is that only the first triggering guest causes a kernel
>>> message to be output because the message is output with pr_warn_once().
>>> This also results in a loss of information to the user.
>>>
>>> While fixing these I noticed that the same message was being output
>>> three times so I'm cleaning that up too.
>>>
>>> Fix fatal mode output, and use consistent messages for fatal and
>>> warn modes for both userspace and guests.
>>>
>>> Signed-off-by: Prarit Bhargava <prarit@redhat.com>
>>> Cc: Thomas Gleixner <tglx@linutronix.de>
>>> Cc: Ingo Molnar <mingo@redhat.com>
>>> Cc: Borislav Petkov <bp@alien8.de>
>>> Cc: x86@kernel.org
>>> Cc: "H. Peter Anvin" <hpa@zytor.com>
>>> Cc: Tony Luck <tony.luck@intel.com>
>>> Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>
>>> Cc: Sean Christopherson <sean.j.christopherson@intel.com>
>>> Cc: Rahul Tanwar <rahul.tanwar@linux.intel.com>
>>> Cc: Xiaoyao Li <xiaoyao.li@intel.com>
>>> Cc: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
>>> Cc: Dave Hansen <dave.hansen@linux.intel.com>
>>> ---
>>>    arch/x86/kernel/cpu/intel.c | 24 ++++++++++--------------
>>>    1 file changed, 10 insertions(+), 14 deletions(-)
>>>
>>> diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
>>> index 166d7c355896..463022aa9b7a 100644
>>> --- a/arch/x86/kernel/cpu/intel.c
>>> +++ b/arch/x86/kernel/cpu/intel.c
>>> @@ -1074,10 +1074,14 @@ static void split_lock_init(void)
>>>        split_lock_verify_msr(sld_state != sld_off);
>>>    }
>>>    -static void split_lock_warn(unsigned long ip)
>>> +static bool split_lock_warn(unsigned long ip, int fatal)
>>>    {
>>> -    pr_warn_ratelimited("#AC: %s/%d took a split_lock trap at address: 0x%lx\n",
>>> -                current->comm, current->pid, ip);
>>> +    pr_warn_ratelimited("#AC: %s/%d %ssplit_lock trap at address: 0x%lx\n",
>>> +                current->comm, current->pid,
>>> +                sld_state == sld_fatal ? "fatal " : "", ip);
>>> +
>>> +    if (sld_state == sld_fatal || fatal)
>>> +        return false;
>>>          /*
>>>         * Disable the split lock detection for this task so it can make
>>> @@ -1086,18 +1090,13 @@ static void split_lock_warn(unsigned long ip)
>>>         */
>>>        sld_update_msr(false);
>>>        set_tsk_thread_flag(current, TIF_SLD);
>>> +    return true;
>>>    }
>>>      bool handle_guest_split_lock(unsigned long ip)
>>>    {
>>> -    if (sld_state == sld_warn) {
>>> -        split_lock_warn(ip);
>>> +    if (split_lock_warn(ip, 0))
>>>            return true;
>>> -    }
>>> -
>>> -    pr_warn_once("#AC: %s/%d %s split_lock trap at address: 0x%lx\n",
>>> -             current->comm, current->pid,
>>> -             sld_state == sld_fatal ? "fatal" : "bogus", ip);
>>>          current->thread.error_code = 0;
>>>        current->thread.trap_nr = X86_TRAP_AC;
>>> @@ -1108,10 +1107,7 @@ EXPORT_SYMBOL_GPL(handle_guest_split_lock);
>>>      bool handle_user_split_lock(struct pt_regs *regs, long error_code)
>>>    {
>>> -    if ((regs->flags & X86_EFLAGS_AC) || sld_state == sld_fatal)
>>> -        return false;
>>> -    split_lock_warn(regs->ip);
>>> -    return true;
>>> +    return split_lock_warn(regs->ip, regs->flags & X86_EFLAGS_AC);
>>
>> It's incorrect. You change the behavior that it will print the split lock
>> warning even when CPL 3 Alignment Check is turned on.
> 
> Do you want the message to be displayed in the fatal case of CPL 3 Alignment check?
> 

No. It should never be displayed if #AC happens in CPL 3 and 
X86_EFLAGS_AC is set. In this case, an unaligned access triggers #AC 
regardless of #LOCK prefix. What's more, even there is a #LOCK prefix, 
we still cannot tell the cause because we don't know the priority of 
legacy alignment check #AC and split lock #AC.

If you do want a message, we can only say "unaligned access at address xxx".