[PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Aravind Gopalakrishnan]

Some of older[Fam10h] systems require that certain number of
applied microcode patch levels should not be overwritten by
the microcode loader. Otherwise, system hangs are known to occur.

The 'final_levels' of patch ids have been obtained empirically.
Refer bug https://bugzilla.suse.com/show_bug.cgi?id=913996
for details of the issue.

The short version is that people have predominantly noticed
system hang issues when trying to update microcode levels
beyond the patch IDs below.
[0x01000098, 0x0100009f, 0x010000af]

>From internal discussions, we gathered that OS/hypervisor
cannot reliably perform microcode updates beyond these levels
due to hardware issues. Therefore, we need to abort microcode
update process if we hit any of these levels.

In this patch, we check for those microcode versions and abort
if the current core has one of those final patch levels applied
by the BIOS

A linux version of the patch has already made it into tip-
http://marc.info/?l=linux-kernel&m=143703405627170

Signed-off-by: Aravind Gopalakrishnan <aravind.gopalakrishnan@amd.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
Changes from V2 (per Boris)
  - introduce family check as it's theoritically possible to have
    same patch level for different family too.
  - Indicate that the check is only on Fam10h too while at it

Changes from V1 (per Andrew)
  - use commit text from linux patch
  - include details about how 'final_levels' are obtaines in comments
    (I have also copied it into commit message. But shall remove if you
     feel it's redundant)
  - use ARRAY_SIZE() and kill NULL terminator
  - use XENLOG_INFO in place of pr_debug()
  - correct unsigned int usage

 xen/arch/x86/microcode_amd.c | 45 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index f79b397..b5f7148 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -347,6 +347,43 @@ static int container_fast_forward(const void *data, size_t size_left, size_t *of
     return 0;
 }
 
+/*
+ * The 'final_levels' of patch ids have been obtained empirically.
+ * Refer bug https://bugzilla.suse.com/show_bug.cgi?id=913996
+ * for details of the issue. The short version is that people
+ * using certain Fam10h systems noticed system hang issues when
+ * trying to update microcode levels beyond the patch IDs below.
+ * From internal discussions, we gathered that OS/hypervisor
+ * cannot reliably perform microcode updates beyond these levels
+ * due to hardware issues. Therefore, we need to abort microcode
+ * update process if we hit any of these levels.
+ */
+static unsigned int final_levels[] = {
+    0x01000098,
+    0x0100009f,
+    0x010000af
+};
+
+static bool_t check_final_patch_levels(int cpu)
+{
+    /*
+     * Check the current patch levels on the cpu. If they are equal to
+     * any of the 'final_levels', then we should not update the microcode
+     * patch on the cpu as system will hang otherwise.
+     */
+    struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+    unsigned int i;
+
+    if ( boot_cpu_data.x86 != 0x10 )
+        return 0;
+
+    for ( i = 0; i < ARRAY_SIZE(final_levels); i++ )
+        if ( uci->cpu_sig.rev == final_levels[i] )
+            return 1;
+
+    return 0;
+}
+
 static int cpu_request_microcode(int cpu, const void *buf, size_t bufsize)
 {
     struct microcode_amd *mc_amd, *mc_old;
@@ -369,6 +406,14 @@ static int cpu_request_microcode(int cpu, const void *buf, size_t bufsize)
         goto out;
     }
 
+    if ( check_final_patch_levels(cpu) )
+    {
+        printk(XENLOG_INFO
+               "microcode: Cannot update microcode patch on the cpu as we hit a final level\n");
+        error = -EPERM;
+        goto out;
+    }
+
     mc_amd = xmalloc(struct microcode_amd);
     if ( !mc_amd )
     {
-- 
1.9.1

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Boris Ostrovsky]

On 08/03/2015 03:34 PM, Aravind Gopalakrishnan wrote:
> Some of older[Fam10h] systems require that certain number of
> applied microcode patch levels should not be overwritten by
> the microcode loader. Otherwise, system hangs are known to occur.
>
> The 'final_levels' of patch ids have been obtained empirically.
> Refer bug https://bugzilla.suse.com/show_bug.cgi?id=913996
> for details of the issue.
>
> The short version is that people have predominantly noticed
> system hang issues when trying to update microcode levels
> beyond the patch IDs below.
> [0x01000098, 0x0100009f, 0x010000af]
>
>  From internal discussions, we gathered that OS/hypervisor
> cannot reliably perform microcode updates beyond these levels
> due to hardware issues. Therefore, we need to abort microcode
> update process if we hit any of these levels.
>
> In this patch, we check for those microcode versions and abort
> if the current core has one of those final patch levels applied
> by the BIOS
>
> A linux version of the patch has already made it into tip-
> http://marc.info/?l=linux-kernel&m=143703405627170
>
> Signed-off-by: Aravind Gopalakrishnan <aravind.gopalakrishnan@amd.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

> ---
> Changes from V2 (per Boris)
>    - introduce family check as it's theoritically possible to have
>      same patch level for different family too.
>    - Indicate that the check is only on Fam10h too while at it
>
> Changes from V1 (per Andrew)
>    - use commit text from linux patch
>    - include details about how 'final_levels' are obtaines in comments
>      (I have also copied it into commit message. But shall remove if you
>       feel it's redundant)
>    - use ARRAY_SIZE() and kill NULL terminator
>    - use XENLOG_INFO in place of pr_debug()
>    - correct unsigned int usage
>
>   xen/arch/x86/microcode_amd.c | 45 ++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 45 insertions(+)
>
> diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
> index f79b397..b5f7148 100644
> --- a/xen/arch/x86/microcode_amd.c
> +++ b/xen/arch/x86/microcode_amd.c
> @@ -347,6 +347,43 @@ static int container_fast_forward(const void *data, size_t size_left, size_t *of
>       return 0;
>   }
>   
> +/*
> + * The 'final_levels' of patch ids have been obtained empirically.
> + * Refer bug https://bugzilla.suse.com/show_bug.cgi?id=913996
> + * for details of the issue. The short version is that people
> + * using certain Fam10h systems noticed system hang issues when
> + * trying to update microcode levels beyond the patch IDs below.
> + * From internal discussions, we gathered that OS/hypervisor
> + * cannot reliably perform microcode updates beyond these levels
> + * due to hardware issues. Therefore, we need to abort microcode
> + * update process if we hit any of these levels.
> + */
> +static unsigned int final_levels[] = {
> +    0x01000098,
> +    0x0100009f,
> +    0x010000af
> +};
> +
> +static bool_t check_final_patch_levels(int cpu)
> +{
> +    /*
> +     * Check the current patch levels on the cpu. If they are equal to
> +     * any of the 'final_levels', then we should not update the microcode
> +     * patch on the cpu as system will hang otherwise.
> +     */
> +    struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
> +    unsigned int i;
> +
> +    if ( boot_cpu_data.x86 != 0x10 )
> +        return 0;
> +
> +    for ( i = 0; i < ARRAY_SIZE(final_levels); i++ )
> +        if ( uci->cpu_sig.rev == final_levels[i] )
> +            return 1;
> +
> +    return 0;
> +}
> +
>   static int cpu_request_microcode(int cpu, const void *buf, size_t bufsize)
>   {
>       struct microcode_amd *mc_amd, *mc_old;
> @@ -369,6 +406,14 @@ static int cpu_request_microcode(int cpu, const void *buf, size_t bufsize)
>           goto out;
>       }
>   
> +    if ( check_final_patch_levels(cpu) )
> +    {
> +        printk(XENLOG_INFO
> +               "microcode: Cannot update microcode patch on the cpu as we hit a final level\n");
> +        error = -EPERM;
> +        goto out;
> +    }
> +
>       mc_amd = xmalloc(struct microcode_amd);
>       if ( !mc_amd )
>       {

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Andrew Cooper]

On 03/08/2015 20:54, Boris Ostrovsky wrote:
> On 08/03/2015 03:34 PM, Aravind Gopalakrishnan wrote:
>> Some of older[Fam10h] systems require that certain number of
>> applied microcode patch levels should not be overwritten by
>> the microcode loader. Otherwise, system hangs are known to occur.
>>
>> The 'final_levels' of patch ids have been obtained empirically.
>> Refer bug https://bugzilla.suse.com/show_bug.cgi?id=913996
>> for details of the issue.
>>
>> The short version is that people have predominantly noticed
>> system hang issues when trying to update microcode levels
>> beyond the patch IDs below.
>> [0x01000098, 0x0100009f, 0x010000af]
>>
>>  From internal discussions, we gathered that OS/hypervisor
>> cannot reliably perform microcode updates beyond these levels
>> due to hardware issues. Therefore, we need to abort microcode
>> update process if we hit any of these levels.
>>
>> In this patch, we check for those microcode versions and abort
>> if the current core has one of those final patch levels applied
>> by the BIOS
>>
>> A linux version of the patch has already made it into tip-
>> http://marc.info/?l=linux-kernel&m=143703405627170
>>
>> Signed-off-by: Aravind Gopalakrishnan <aravind.gopalakrishnan@amd.com>
>> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
>
> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

For the avoidance of any confusion, I am happy for my R-b to stand,
given this addition to the patch.

~Andrew

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Jan Beulich]

>>> On 03.08.15 at 21:34, <aravind.gopalakrishnan@amd.com> wrote:
> --- a/xen/arch/x86/microcode_amd.c
> +++ b/xen/arch/x86/microcode_amd.c
> @@ -347,6 +347,43 @@ static int container_fast_forward(const void *data, size_t size_left, size_t *of
>      return 0;
>  }
>  
> +/*
> + * The 'final_levels' of patch ids have been obtained empirically.
> + * Refer bug https://bugzilla.suse.com/show_bug.cgi?id=913996 
> + * for details of the issue. The short version is that people
> + * using certain Fam10h systems noticed system hang issues when
> + * trying to update microcode levels beyond the patch IDs below.
> + * From internal discussions, we gathered that OS/hypervisor
> + * cannot reliably perform microcode updates beyond these levels
> + * due to hardware issues. Therefore, we need to abort microcode
> + * update process if we hit any of these levels.
> + */
> +static unsigned int final_levels[] = {

const

> +    0x01000098,
> +    0x0100009f,
> +    0x010000af
> +};
> +
> +static bool_t check_final_patch_levels(int cpu)

unsigned int

Jan

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Aravind Gopalakrishnan]

On 8/11/2015 9:53 AM, Jan Beulich wrote:
>>>> On 03.08.15 at 21:34, <aravind.gopalakrishnan@amd.com> wrote:
>> --- a/xen/arch/x86/microcode_amd.c
>> +++ b/xen/arch/x86/microcode_amd.c
>> @@ -347,6 +347,43 @@ static int container_fast_forward(const void *data, size_t size_left, size_t *of
>>       return 0;
>>   }
>>   
>> +/*
>> + * The 'final_levels' of patch ids have been obtained empirically.
>> + * Refer bug https://bugzilla.suse.com/show_bug.cgi?id=913996
>> + * for details of the issue. The short version is that people
>> + * using certain Fam10h systems noticed system hang issues when
>> + * trying to update microcode levels beyond the patch IDs below.
>> + * From internal discussions, we gathered that OS/hypervisor
>> + * cannot reliably perform microcode updates beyond these levels
>> + * due to hardware issues. Therefore, we need to abort microcode
>> + * update process if we hit any of these levels.
>> + */
>> +static unsigned int final_levels[] = {
> const

Okay, will fix this.

>> +    0x01000098,
>> +    0x0100009f,
>> +    0x010000af
>> +};
>> +
>> +static bool_t check_final_patch_levels(int cpu)
> unsigned int
>

I can change this too, but- Any specific reason for this?
The other sanity checker or verification functions like 
verify_patch_size() or microcode_fits() return a bool_t too..

Thanks,
-Aravind.

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Andrew Cooper]

On 11/08/15 16:14, Aravind Gopalakrishnan wrote:
> On 8/11/2015 9:53 AM, Jan Beulich wrote:
>>>>> On 03.08.15 at 21:34, <aravind.gopalakrishnan@amd.com> wrote:
>>> --- a/xen/arch/x86/microcode_amd.c
>>> +++ b/xen/arch/x86/microcode_amd.c
>>> @@ -347,6 +347,43 @@ static int container_fast_forward(const void
>>> *data, size_t size_left, size_t *of
>>>       return 0;
>>>   }
>>>   +/*
>>> + * The 'final_levels' of patch ids have been obtained empirically.
>>> + * Refer bug https://bugzilla.suse.com/show_bug.cgi?id=913996
>>> + * for details of the issue. The short version is that people
>>> + * using certain Fam10h systems noticed system hang issues when
>>> + * trying to update microcode levels beyond the patch IDs below.
>>> + * From internal discussions, we gathered that OS/hypervisor
>>> + * cannot reliably perform microcode updates beyond these levels
>>> + * due to hardware issues. Therefore, we need to abort microcode
>>> + * update process if we hit any of these levels.
>>> + */
>>> +static unsigned int final_levels[] = {
>> const
>
> Okay, will fix this.
>
>>> +    0x01000098,
>>> +    0x0100009f,
>>> +    0x010000af
>>> +};
>>> +
>>> +static bool_t check_final_patch_levels(int cpu)
>> unsigned int
>>
>
> I can change this too, but- Any specific reason for this?
> The other sanity checker or verification functions like
> verify_patch_size() or microcode_fits() return a bool_t too..

"int cpu" is the issue (I am guessing), not the return type.

~Andrew

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Aravind Gopalakrishnan]

On 8/11/2015 10:17 AM, Andrew Cooper wrote:
>>
>>>> +    0x01000098,
>>>> +    0x0100009f,
>>>> +    0x010000af
>>>> +};
>>>> +
>>>> +static bool_t check_final_patch_levels(int cpu)
>>> unsigned int
>>>
>> I can change this too, but- Any specific reason for this?
>> The other sanity checker or verification functions like
>> verify_patch_size() or microcode_fits() return a bool_t too..
> "int cpu" is the issue (I am guessing), not the return type.
>

Thought about that too.. but all the microcode_ops functions accept an 
'int cpu'.
I'm just using the same type in check_final_patch_levels().

Anyway, will wait for Jan's feedback..

Thanks,
-Aravind.

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Jan Beulich]

>>> On 11.08.15 at 17:17, <andrew.cooper3@citrix.com> wrote:
> On 11/08/15 16:14, Aravind Gopalakrishnan wrote:
>> On 8/11/2015 9:53 AM, Jan Beulich wrote:
>>>>>> On 03.08.15 at 21:34, <aravind.gopalakrishnan@amd.com> wrote:
>>>> +static bool_t check_final_patch_levels(int cpu)
>>> unsigned int
>>>
>>
>> I can change this too, but- Any specific reason for this?
>> The other sanity checker or verification functions like
>> verify_patch_size() or microcode_fits() return a bool_t too..
> 
> "int cpu" is the issue (I am guessing), not the return type.

Indeed.

jan

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Jan Beulich]

>>> On 11.08.15 at 17:28, <aravind.gopalakrishnan@amd.com> wrote:
> On 8/11/2015 10:17 AM, Andrew Cooper wrote:
>>>
>>>>> +    0x01000098,
>>>>> +    0x0100009f,
>>>>> +    0x010000af
>>>>> +};
>>>>> +
>>>>> +static bool_t check_final_patch_levels(int cpu)
>>>> unsigned int
>>>>
>>> I can change this too, but- Any specific reason for this?
>>> The other sanity checker or verification functions like
>>> verify_patch_size() or microcode_fits() return a bool_t too..
>> "int cpu" is the issue (I am guessing), not the return type.
>>
> 
> Thought about that too.. but all the microcode_ops functions accept an 
> 'int cpu'.
> I'm just using the same type in check_final_patch_levels().

And ideally all other bad examples would be fixed in a cleanup
patch too - CPU numbers can't be negative. In any event we
should aim at not proliferating such sub-optimal code.

Jan

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Aravind Gopalakrishnan]

On 8/11/2015 10:32 AM, Jan Beulich wrote:
>>>>>> +static bool_t check_final_patch_levels(int cpu)
>>>>> unsigned int
>>>>>
>>>> I can change this too, but- Any specific reason for this?
>>>> The other sanity checker or verification functions like
>>>> verify_patch_size() or microcode_fits() return a bool_t too..
>>> "int cpu" is the issue (I am guessing), not the return type.
>>>
>> Thought about that too.. but all the microcode_ops functions accept an
>> 'int cpu'.
>> I'm just using the same type in check_final_patch_levels().
> And ideally all other bad examples would be fixed in a cleanup
> patch too - CPU numbers can't be negative. In any event we
> should aim at not proliferating such sub-optimal code.
>


Okay, I can clean that up while at it.

Would it be OK if I did a pre-patch to cleanup the 'int cpu' usage in 
the microcode* files and then apply this patch on top of it?

Thanks,
-Aravind.

Re: [PATCH V3] x86, amd_ucode: Skip microcode updates for final levels

[Jan Beulich]

>>> On 11.08.15 at 17:43, <aravind.gopalakrishnan@amd.com> wrote:
> On 8/11/2015 10:32 AM, Jan Beulich wrote:
>>>>>>> +static bool_t check_final_patch_levels(int cpu)
>>>>>> unsigned int
>>>>>>
>>>>> I can change this too, but- Any specific reason for this?
>>>>> The other sanity checker or verification functions like
>>>>> verify_patch_size() or microcode_fits() return a bool_t too..
>>>> "int cpu" is the issue (I am guessing), not the return type.
>>>>
>>> Thought about that too.. but all the microcode_ops functions accept an
>>> 'int cpu'.
>>> I'm just using the same type in check_final_patch_levels().
>> And ideally all other bad examples would be fixed in a cleanup
>> patch too - CPU numbers can't be negative. In any event we
>> should aim at not proliferating such sub-optimal code.
>>
> 
> 
> Okay, I can clean that up while at it.
> 
> Would it be OK if I did a pre-patch to cleanup the 'int cpu' usage in 
> the microcode* files and then apply this patch on top of it?

Sure.

Jan