All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] x86: also disable FSRM if ERMS is disabled
@ 2022-09-23  0:58 Daniel Verkamp
  2022-09-23 11:13 ` Borislav Petkov
  0 siblings, 1 reply; 18+ messages in thread
From: Daniel Verkamp @ 2022-09-23  0:58 UTC (permalink / raw)
  To: x86, linux-kernel; +Cc: Tony Luck, Borislav Petkov, Daniel Verkamp, stable

In the "Fast Short REP MOVSB" path of memmove, if we take the path where
the FSRM flag is enabled but the ERMS flag is not, there is no longer a
check for length >= 0x20 (both alternatives will be replaced with NOPs).
If a memmove() requiring a forward copy of less than 0x20 bytes happens
in this case, the `sub $0x20, %rdx` will cause the length to roll around
to a huge value and the copy will eventually hit a page fault.

This is not intended to happen, as the comment above the alternatives
mentions "FSRM implies ERMS".

However, there is a check in early_init_intel() that can disable ERMS,
so we should also be disabling FSRM in this path to maintain correctness
of the memmove() optimization.

Cc: stable@vger.kernel.org
Fixes: f444a5ff95dc ("x86/cpufeatures: Add support for fast short REP; MOVSB")
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
---
 arch/x86/kernel/cpu/intel.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 2d7ea5480ec3..71b412f820c7 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -328,6 +328,7 @@ static void early_init_intel(struct cpuinfo_x86 *c)
 			pr_info("Disabled fast string operations\n");
 			setup_clear_cpu_cap(X86_FEATURE_REP_GOOD);
 			setup_clear_cpu_cap(X86_FEATURE_ERMS);
+			setup_clear_cpu_cap(X86_FEATURE_FSRM);
 		}
 	}
 
-- 
2.37.3.998.g577e59143f-goog


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-09-23  0:58 [PATCH] x86: also disable FSRM if ERMS is disabled Daniel Verkamp
@ 2022-09-23 11:13 ` Borislav Petkov
  2022-09-23 17:25   ` Daniel Verkamp
  0 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2022-09-23 11:13 UTC (permalink / raw)
  To: Daniel Verkamp; +Cc: x86, linux-kernel, Tony Luck

On Thu, Sep 22, 2022 at 05:58:27PM -0700, Daniel Verkamp wrote:
> In the "Fast Short REP MOVSB" path of memmove, if we take the path where
> the FSRM flag is enabled but the ERMS flag is not, there is no longer a
> check for length >= 0x20 (both alternatives will be replaced with NOPs).
> If a memmove() requiring a forward copy of less than 0x20 bytes happens
> in this case, the `sub $0x20, %rdx` will cause the length to roll around
> to a huge value and the copy will eventually hit a page fault.
> 
> This is not intended to happen, as the comment above the alternatives
> mentions "FSRM implies ERMS".
> 
> However, there is a check in early_init_intel() that can disable ERMS,
> so we should also be disabling FSRM in this path to maintain correctness
> of the memmove() optimization.

Is this something you hit in a real-world scenario? If so, how exactly?

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-09-23 11:13 ` Borislav Petkov
@ 2022-09-23 17:25   ` Daniel Verkamp
  2022-09-23 17:51     ` Borislav Petkov
  0 siblings, 1 reply; 18+ messages in thread
From: Daniel Verkamp @ 2022-09-23 17:25 UTC (permalink / raw)
  To: Borislav Petkov; +Cc: x86, linux-kernel, Tony Luck

On Fri, Sep 23, 2022 at 4:13 AM Borislav Petkov <bp@alien8.de> wrote:
>
> On Thu, Sep 22, 2022 at 05:58:27PM -0700, Daniel Verkamp wrote:
> > In the "Fast Short REP MOVSB" path of memmove, if we take the path where
> > the FSRM flag is enabled but the ERMS flag is not, there is no longer a
> > check for length >= 0x20 (both alternatives will be replaced with NOPs).
> > If a memmove() requiring a forward copy of less than 0x20 bytes happens
> > in this case, the `sub $0x20, %rdx` will cause the length to roll around
> > to a huge value and the copy will eventually hit a page fault.
> >
> > This is not intended to happen, as the comment above the alternatives
> > mentions "FSRM implies ERMS".
> >
> > However, there is a check in early_init_intel() that can disable ERMS,
> > so we should also be disabling FSRM in this path to maintain correctness
> > of the memmove() optimization.
>
> Is this something you hit in a real-world scenario? If so, how exactly?
>
> Thx.

Yes, we hit this in crosvm when booting the guest kernel with either
OVMF or u-boot on an Intel 12th Gen CPU. The guest kernel boots fine
when loaded directly (using the crosvm kernel loader and not running
any firmware setup in the guest), but it crashes when booting with
firmware inside the first forward memmove() after alternatives are set
up (which happens to be in printk). I haven't gotten to the bottom of
why exactly using firmware is causing this to be set up in an
inconsistent way, but this is a real-world situation, not just a
hypothetical.

Now that I look at it with fresh eyes again, maybe we should instead
directly patch the memmove FSRM alternative so that the flag-set
version just does the same jmp as the ERMS one. I can prepare a patch
for that instead of (or in addition to) this one if that sounds
better.

Thanks,
-- Daniel

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-09-23 17:25   ` Daniel Verkamp
@ 2022-09-23 17:51     ` Borislav Petkov
  2022-10-07 18:08       ` Daniel Verkamp
  0 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2022-09-23 17:51 UTC (permalink / raw)
  To: Daniel Verkamp; +Cc: x86, linux-kernel, Tony Luck

On Fri, Sep 23, 2022 at 10:25:05AM -0700, Daniel Verkamp wrote:
> Yes, we hit this in crosvm when booting the guest kernel with either
> OVMF or u-boot on an Intel 12th Gen CPU. The guest kernel boots fine
> when loaded directly (using the crosvm kernel loader and not running
> any firmware setup in the guest), but it crashes when booting with
> firmware inside the first forward memmove() after alternatives are set
> up (which happens to be in printk). I haven't gotten to the bottom of
> why exactly using firmware is causing this to be set up in an
> inconsistent way, but this is a real-world situation, not just a
> hypothetical.

Sounds like broken virt firmware or so. And if that is not an issue on
baremetal, then the virt stack should be fixed - not the kernel.

> Now that I look at it with fresh eyes again, maybe we should instead
> directly patch the memmove FSRM alternative so that the flag-set
> version just does the same jmp as the ERMS one. I can prepare a patch
> for that instead of (or in addition to) this one if that sounds
> better.

So, if the virt firmware deviates from how the real hardware behaves,
then the kernel needs no fixing.

So you'd have to figure out why is the virt firmware causing this and
not baremetal.

Then we can talk about fixes.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-09-23 17:51     ` Borislav Petkov
@ 2022-10-07 18:08       ` Daniel Verkamp
  2022-10-11 11:28         ` Borislav Petkov
  2023-01-04  7:43         ` Jiri Slaby
  0 siblings, 2 replies; 18+ messages in thread
From: Daniel Verkamp @ 2022-10-07 18:08 UTC (permalink / raw)
  To: Borislav Petkov; +Cc: x86, linux-kernel, Tony Luck

On Fri, Sep 23, 2022 at 10:51 AM Borislav Petkov <bp@alien8.de> wrote:
>
> On Fri, Sep 23, 2022 at 10:25:05AM -0700, Daniel Verkamp wrote:
> > Yes, we hit this in crosvm when booting the guest kernel with either
> > OVMF or u-boot on an Intel 12th Gen CPU. The guest kernel boots fine
> > when loaded directly (using the crosvm kernel loader and not running
> > any firmware setup in the guest), but it crashes when booting with
> > firmware inside the first forward memmove() after alternatives are set
> > up (which happens to be in printk). I haven't gotten to the bottom of
> > why exactly using firmware is causing this to be set up in an
> > inconsistent way, but this is a real-world situation, not just a
> > hypothetical.
>
> Sounds like broken virt firmware or so. And if that is not an issue on
> baremetal, then the virt stack should be fixed - not the kernel.
>
> > Now that I look at it with fresh eyes again, maybe we should instead
> > directly patch the memmove FSRM alternative so that the flag-set
> > version just does the same jmp as the ERMS one. I can prepare a patch
> > for that instead of (or in addition to) this one if that sounds
> > better.
>
> So, if the virt firmware deviates from how the real hardware behaves,
> then the kernel needs no fixing.
>
> So you'd have to figure out why is the virt firmware causing this and
> not baremetal.
>
> Then we can talk about fixes.

Hi Borislav,

We found that the IA32_MISC_ENABLE MSR setup was missing in the crosvm
firmware boot path (but not when directly booting a kernel, which is
why it did not get noticed for a while). Setting the fast string bit
in the MSR avoids the issue.

However, I still think it would be appropriate to apply this patch or
something like it, since there could be a CPU, microcode update, BIOS,
etc. that clears this bit while still having the CPUID flags for FSRM
and ERMS. The Intel SDM says: "Software can disable fast-string
operation by clearing the fast-string-enable bit (bit 0) of
IA32_MISC_ENABLE MSR", so it's not an invalid configuration for this
bit to be unset.

Additionally, something like this avoids the problem by making the
FSRM case jump directly to the REP MOVSB rather than falling through
to the ERMS jump in the next instruction, which seems like basically
free insurance (but if the FSRM flag gets used somewhere else in the
future, having it set consistently with ERMS is probably still a good
idea, per the original patch):

diff --git a/arch/x86/lib/memmove_64.S b/arch/x86/lib/memmove_64.S
index 724bbf83eb5b..8ac557409c7d 100644
--- a/arch/x86/lib/memmove_64.S
+++ b/arch/x86/lib/memmove_64.S
@@ -38,7 +38,7 @@ SYM_FUNC_START(__memmove)

         /* FSRM implies ERMS => no length checks, do the copy directly */
 .Lmemmove_begin_forward:
-        ALTERNATIVE "cmp $0x20, %rdx; jb 1f", "", X86_FEATURE_FSRM
+        ALTERNATIVE "cmp $0x20, %rdx; jb 1f", "jmp .Lmemmove_erms",
X86_FEATURE_FSRM
         ALTERNATIVE "", "jmp .Lmemmove_erms", X86_FEATURE_ERMS

And hey, this means one less instruction to execute in the FSRM path. :)

Thanks,
-- Daniel

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-10-07 18:08       ` Daniel Verkamp
@ 2022-10-11 11:28         ` Borislav Petkov
  2022-10-11 17:09           ` Luck, Tony
  2023-01-04  7:43         ` Jiri Slaby
  1 sibling, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2022-10-11 11:28 UTC (permalink / raw)
  To: Daniel Verkamp, Tony Luck; +Cc: x86, linux-kernel

On Fri, Oct 07, 2022 at 11:08:45AM -0700, Daniel Verkamp wrote:
> We found that the IA32_MISC_ENABLE MSR setup was missing in the crosvm
> firmware boot path

Sounds like crazy virtualization stuff.

> However, I still think it would be appropriate to apply this patch or
> something like it, since there could be a CPU, microcode update, BIOS,
> etc. that clears this bit while still having the CPUID flags for FSRM
> and ERMS.

You mean that "something" would be so silly so as to clear the MSR but
leave the CPUID bits set?

That sounds like a bug in that "something".

> The Intel SDM says: "Software can disable fast-string operation by
> clearing the fast-string-enable bit (bit 0) of IA32_MISC_ENABLE MSR",
> so it's not an invalid configuration for this bit to be unset.

Dunno, did Intel folks think about clearing the respective CPUID bits
when exposing IA32_MISC_ENABLE[0] to software? Tony?

> Additionally, something like this avoids the problem by making the
> FSRM case jump directly to the REP MOVSB rather than falling through
> to the ERMS jump in the next instruction, which seems like basically
> free insurance (but if the FSRM flag gets used somewhere else in the
> future,

I can't follow here.

> having it set consistently with ERMS is probably still a good
> idea, per the original patch):
> 
> diff --git a/arch/x86/lib/memmove_64.S b/arch/x86/lib/memmove_64.S
> index 724bbf83eb5b..8ac557409c7d 100644
> --- a/arch/x86/lib/memmove_64.S
> +++ b/arch/x86/lib/memmove_64.S
> @@ -38,7 +38,7 @@ SYM_FUNC_START(__memmove)
> 
>          /* FSRM implies ERMS => no length checks, do the copy directly */
>  .Lmemmove_begin_forward:
> -        ALTERNATIVE "cmp $0x20, %rdx; jb 1f", "", X86_FEATURE_FSRM
> +        ALTERNATIVE "cmp $0x20, %rdx; jb 1f", "jmp .Lmemmove_erms",
> X86_FEATURE_FSRM
>          ALTERNATIVE "", "jmp .Lmemmove_erms", X86_FEATURE_ERMS
> 
> And hey, this means one less instruction to execute in the FSRM path. :)

What one less instruction? After patching and since we assume FSRM =>
ERMS, we have only the JMP there if both flags are set. If ERMS only is
set, then we do the length check.

And you need the second alternative call for !ERMS, i.e., old rust.

So yes, your proposal is to do this because we assume if FSRM, then ERMS
but your diff above doesn't make it more readable but less.

Or I'm missing something ofc.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 18+ messages in thread

* RE: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-10-11 11:28         ` Borislav Petkov
@ 2022-10-11 17:09           ` Luck, Tony
  2022-10-11 17:52             ` Borislav Petkov
  0 siblings, 1 reply; 18+ messages in thread
From: Luck, Tony @ 2022-10-11 17:09 UTC (permalink / raw)
  To: Borislav Petkov, Daniel Verkamp; +Cc: x86, linux-kernel

>> The Intel SDM says: "Software can disable fast-string operation by
>> clearing the fast-string-enable bit (bit 0) of IA32_MISC_ENABLE MSR",
>> so it's not an invalid configuration for this bit to be unset.
>
> Dunno, did Intel folks think about clearing the respective CPUID bits
> when exposing IA32_MISC_ENABLE[0] to software? Tony?

I don't know if it was thought about. Experimentally clearing bit
0 of IA32_MISC_ENABLE does not affect the CPUID bit settings
for either ERMS or FSRM (on the one system I tried that supports
both of these bits).

-Tony



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-10-11 17:09           ` Luck, Tony
@ 2022-10-11 17:52             ` Borislav Petkov
  2022-10-11 19:08               ` Luck, Tony
  0 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2022-10-11 17:52 UTC (permalink / raw)
  To: Luck, Tony; +Cc: Daniel Verkamp, x86, linux-kernel, Andrew Cooper

On Tue, Oct 11, 2022 at 05:09:19PM +0000, Luck, Tony wrote:
> >> The Intel SDM says: "Software can disable fast-string operation by
> >> clearing the fast-string-enable bit (bit 0) of IA32_MISC_ENABLE MSR",
> >> so it's not an invalid configuration for this bit to be unset.
> >
> > Dunno, did Intel folks think about clearing the respective CPUID bits
> > when exposing IA32_MISC_ENABLE[0] to software? Tony?
> 
> I don't know if it was thought about. Experimentally clearing bit
> 0 of IA32_MISC_ENABLE does not affect the CPUID bit settings
> for either ERMS or FSRM (on the one system I tried that supports
> both of these bits).

Confirms my observation on a Coffeelake here too.

Oh well, probably not that important. I mean, I bet luserspace is
looking at those CPUID bits - glibc probably - and then probably selects
the proper memcpy or so routine, based on what's there.

If something clears MSR_IA32_MISC_ENABLE_FAST_STRING_BIT and we go and
clear our feature flags and luserspace still queries CPUID then oh well,
it'll be fun. It all depends on why something has cleared them tho. It
could be some performance thing or something a lot more funky. I guess
if stuff starts exploding left and right, there will soon be a microcode
patch after that. :-)

Might be prudent to check with hw folks just in case...

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 18+ messages in thread

* RE: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-10-11 17:52             ` Borislav Petkov
@ 2022-10-11 19:08               ` Luck, Tony
  2022-10-11 20:56                 ` Borislav Petkov
  0 siblings, 1 reply; 18+ messages in thread
From: Luck, Tony @ 2022-10-11 19:08 UTC (permalink / raw)
  To: Borislav Petkov; +Cc: Daniel Verkamp, x86, linux-kernel, andrew.cooper3

> If something clears MSR_IA32_MISC_ENABLE_FAST_STRING_BIT and we go and
> clear our feature flags and luserspace still queries CPUID then oh well,
> it'll be fun. It all depends on why something has cleared them tho. It
> could be some performance thing or something a lot more funky. I guess
> if stuff starts exploding left and right, there will soon be a microcode
> patch after that. :-)

Yes. ERMS CPUID bit was the early indicator to s/w that REP MOVS would
do some fancy speedups if certain conditions about source, destination and
count are all met. A hint to s/w to decide which memcpy() routine to use.

FSRM is a hint that the byte count restriction had pretty much been removed
so s/w can use REP MOVS in even more places.

I don't think Intel will deliberately release a CPU that has FSRM=1, ERMS=0.

-Tony

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-10-11 19:08               ` Luck, Tony
@ 2022-10-11 20:56                 ` Borislav Petkov
  2022-10-11 22:19                   ` Luck, Tony
  0 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2022-10-11 20:56 UTC (permalink / raw)
  To: Luck, Tony; +Cc: Daniel Verkamp, x86, linux-kernel, andrew.cooper3

On Tue, Oct 11, 2022 at 07:08:51PM +0000, Luck, Tony wrote:
> I don't think Intel will deliberately release a CPU that has FSRM=1, ERMS=0.

Sure, but I don't mean that. Rather: if for some reason the kernel or
BIOS is supposed to fix an erratum related to those enhanced REP moving
routines and goes and clears the MSR bit.

That won't help because userspace will still use them since the CPUID
flags remain set.

I guess such a case is probably not going to happen in real life but if
it happened, that bit clearing is kinda useless.

I'd say.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 18+ messages in thread

* RE: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-10-11 20:56                 ` Borislav Petkov
@ 2022-10-11 22:19                   ` Luck, Tony
  2022-10-11 22:59                     ` Andrew Cooper
  0 siblings, 1 reply; 18+ messages in thread
From: Luck, Tony @ 2022-10-11 22:19 UTC (permalink / raw)
  To: Borislav Petkov; +Cc: Daniel Verkamp, x86, linux-kernel, andrew.cooper3

> That won't help because userspace will still use them since the CPUID
> flags remain set.

Even if writing the MSR did clear the CPUID bits, it wouldn't help with applications
that started before the kernel cleared the bits (assuming this was some run-time
update patch).

Worst case scenario is that the applications don't pick the most efficient memcpy().

-Tony

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-10-11 22:19                   ` Luck, Tony
@ 2022-10-11 22:59                     ` Andrew Cooper
  0 siblings, 0 replies; 18+ messages in thread
From: Andrew Cooper @ 2022-10-11 22:59 UTC (permalink / raw)
  To: Luck, Tony, Borislav Petkov
  Cc: Daniel Verkamp, x86, linux-kernel, Andrew Cooper

On 11/10/2022 23:19, Luck, Tony wrote:
>> That won't help because userspace will still use them since the CPUID
>> flags remain set.
> Even if writing the MSR did clear the CPUID bits, it wouldn't help with applications
> that started before the kernel cleared the bits (assuming this was some run-time
> update patch).
>
> Worst case scenario is that the applications don't pick the most efficient memcpy().

As hint-only bits, the kernel is free to do whatever it wants behind the
scenes.  The only case I'm aware of which dynamically clears fast
strings is the #MC ECC poison overread erratum.

If you want to truly hide the CPUID bits, then you can use CPUID
Faulting (IvyBridge and later, available to userspace), or for a few
generations prior to that, the CPUID mask MSRs, but the settings do want
to be uniform from boot.

~Andrew

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2022-10-07 18:08       ` Daniel Verkamp
  2022-10-11 11:28         ` Borislav Petkov
@ 2023-01-04  7:43         ` Jiri Slaby
  2023-01-04 11:39           ` Borislav Petkov
  1 sibling, 1 reply; 18+ messages in thread
From: Jiri Slaby @ 2023-01-04  7:43 UTC (permalink / raw)
  To: Daniel Verkamp, Borislav Petkov; +Cc: x86, linux-kernel, Tony Luck

On 07. 10. 22, 20:08, Daniel Verkamp wrote:
> On Fri, Sep 23, 2022 at 10:51 AM Borislav Petkov <bp@alien8.de> wrote:
>>
>> On Fri, Sep 23, 2022 at 10:25:05AM -0700, Daniel Verkamp wrote:
>>> Yes, we hit this in crosvm when booting the guest kernel with either
>>> OVMF or u-boot on an Intel 12th Gen CPU. The guest kernel boots fine
>>> when loaded directly (using the crosvm kernel loader and not running
>>> any firmware setup in the guest), but it crashes when booting with
>>> firmware inside the first forward memmove() after alternatives are set
>>> up (which happens to be in printk). I haven't gotten to the bottom of
>>> why exactly using firmware is causing this to be set up in an
>>> inconsistent way, but this is a real-world situation, not just a
>>> hypothetical.
>>
>> Sounds like broken virt firmware or so. And if that is not an issue on
>> baremetal, then the virt stack should be fixed - not the kernel.
>>
>>> Now that I look at it with fresh eyes again, maybe we should instead
>>> directly patch the memmove FSRM alternative so that the flag-set
>>> version just does the same jmp as the ERMS one. I can prepare a patch
>>> for that instead of (or in addition to) this one if that sounds
>>> better.
>>
>> So, if the virt firmware deviates from how the real hardware behaves,
>> then the kernel needs no fixing.
>>
>> So you'd have to figure out why is the virt firmware causing this and
>> not baremetal.
>>
>> Then we can talk about fixes.
> 
> Hi Borislav,
> 
> We found that the IA32_MISC_ENABLE MSR setup was missing in the crosvm
> firmware boot path (but not when directly booting a kernel, which is
> why it did not get noticed for a while). Setting the fast string bit
> in the MSR avoids the issue.
> 
> However, I still think it would be appropriate to apply this patch or
> something like it, since there could be a CPU, microcode update, BIOS,
> etc. that clears this bit while still having the CPUID flags for FSRM
> and ERMS.

Let me resurrect this thread... Our customer has an AMD CPU which has 
indeed both capabilities under normal circumstances. But they have a 
cool UEFI BIOS too. They say:

"""
In AMD platform, while disalbe ERMS(Enhanced Rep MOVSB/STOSB) in UEFI 
(system setup -> processor -> Enhanced Rep MOVSB/STOSB), the OS can't 
boot normally.
"""

That is exactly the case here. So can we have the patch (the original 
one, the one below or a better one) to fix this?

> The Intel SDM says: "Software can disable fast-string
> operation by clearing the fast-string-enable bit (bit 0) of
> IA32_MISC_ENABLE MSR", so it's not an invalid configuration for this
> bit to be unset.
> 
> Additionally, something like this avoids the problem by making the
> FSRM case jump directly to the REP MOVSB rather than falling through
> to the ERMS jump in the next instruction, which seems like basically
> free insurance (but if the FSRM flag gets used somewhere else in the
> future, having it set consistently with ERMS is probably still a good
> idea, per the original patch):
> 
> diff --git a/arch/x86/lib/memmove_64.S b/arch/x86/lib/memmove_64.S
> index 724bbf83eb5b..8ac557409c7d 100644
> --- a/arch/x86/lib/memmove_64.S
> +++ b/arch/x86/lib/memmove_64.S
> @@ -38,7 +38,7 @@ SYM_FUNC_START(__memmove)
> 
>           /* FSRM implies ERMS => no length checks, do the copy directly */
>   .Lmemmove_begin_forward:
> -        ALTERNATIVE "cmp $0x20, %rdx; jb 1f", "", X86_FEATURE_FSRM
> +        ALTERNATIVE "cmp $0x20, %rdx; jb 1f", "jmp .Lmemmove_erms",
> X86_FEATURE_FSRM
>           ALTERNATIVE "", "jmp .Lmemmove_erms", X86_FEATURE_ERMS
> 
> And hey, this means one less instruction to execute in the FSRM path. :)

thanks,
-- 
js
suse labs


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2023-01-04  7:43         ` Jiri Slaby
@ 2023-01-04 11:39           ` Borislav Petkov
  2023-01-14  9:19             ` Ingo Molnar
  2023-01-16  5:26             ` Jiri Slaby
  0 siblings, 2 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-01-04 11:39 UTC (permalink / raw)
  To: Jiri Slaby; +Cc: Daniel Verkamp, x86, linux-kernel, Tony Luck

On Wed, Jan 04, 2023 at 08:43:51AM +0100, Jiri Slaby wrote:
> Let me resurrect this thread... Our customer has an AMD CPU which has indeed
> both capabilities under normal circumstances. But they have a cool UEFI BIOS
> too. They say:
> 
> """
> In AMD platform, while disalbe ERMS(Enhanced Rep MOVSB/STOSB) in UEFI
> (system setup -> processor -> Enhanced Rep MOVSB/STOSB), the OS can't boot
> normally.

Any particular reason they're disabling ERMS?

What do they set FSRM to?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2023-01-04 11:39           ` Borislav Petkov
@ 2023-01-14  9:19             ` Ingo Molnar
  2023-01-14  9:58               ` Borislav Petkov
  2023-01-16  5:26             ` Jiri Slaby
  1 sibling, 1 reply; 18+ messages in thread
From: Ingo Molnar @ 2023-01-14  9:19 UTC (permalink / raw)
  To: Borislav Petkov; +Cc: Jiri Slaby, Daniel Verkamp, x86, linux-kernel, Tony Luck


* Borislav Petkov <bp@alien8.de> wrote:

> On Wed, Jan 04, 2023 at 08:43:51AM +0100, Jiri Slaby wrote:
> > Let me resurrect this thread... Our customer has an AMD CPU which has indeed
> > both capabilities under normal circumstances. But they have a cool UEFI BIOS
> > too. They say:
> > 
> > """
> > In AMD platform, while disalbe ERMS(Enhanced Rep MOVSB/STOSB) in UEFI
> > (system setup -> processor -> Enhanced Rep MOVSB/STOSB), the OS can't boot
> > normally.
> 
> Any particular reason they're disabling ERMS?
> 
> What do they set FSRM to?

Nevertheless both Jiri and Daniel are making a valid argument: our x86 
memcpy routines should not behave in an undefined fashion, *regardless* of 
what CPUID environment we are in.

As practice has shown, both on virtual and on bare metal firmware can screw 
things up enough so that the memcpy routines crash under Linux but under no 
other OS...

So while you are technically correct that these are firmware bugs, I'm in 
favor of robustifying our x86 memcpy routines against these bugs. Silently 
not booting, where no other OS fails to boot, is poor form IMO.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2023-01-14  9:19             ` Ingo Molnar
@ 2023-01-14  9:58               ` Borislav Petkov
  0 siblings, 0 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-01-14  9:58 UTC (permalink / raw)
  To: Ingo Molnar; +Cc: Jiri Slaby, Daniel Verkamp, x86, linux-kernel, Tony Luck

On Sat, Jan 14, 2023 at 10:19:40AM +0100, Ingo Molnar wrote:
> So while you are technically correct that these are firmware bugs, I'm in 
> favor of robustifying our x86 memcpy routines against these bugs. Silently not
> booting, where no other OS fails to boot, is poor form IMO.

Yes, I know.

As I told you yesterday on IRC, I'll take care of it.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2023-01-04 11:39           ` Borislav Petkov
  2023-01-14  9:19             ` Ingo Molnar
@ 2023-01-16  5:26             ` Jiri Slaby
  2023-01-16 21:17               ` Borislav Petkov
  1 sibling, 1 reply; 18+ messages in thread
From: Jiri Slaby @ 2023-01-16  5:26 UTC (permalink / raw)
  To: Borislav Petkov; +Cc: Daniel Verkamp, x86, linux-kernel, Tony Luck

On 04. 01. 23, 12:39, Borislav Petkov wrote:
> On Wed, Jan 04, 2023 at 08:43:51AM +0100, Jiri Slaby wrote:
>> Let me resurrect this thread... Our customer has an AMD CPU which has indeed
>> both capabilities under normal circumstances. But they have a cool UEFI BIOS
>> too. They say:
>>
>> """
>> In AMD platform, while disalbe ERMS(Enhanced Rep MOVSB/STOSB) in UEFI
>> (system setup -> processor -> Enhanced Rep MOVSB/STOSB), the OS can't boot
>> normally.
> 
> Any particular reason they're disabling ERMS?

Just so you know (I see you progressed with Ingo to fix this) -- despite 
I asked the very same question, I received no answer quite yet. I 
suppose it will sound like usual "because we can".

> What do they set FSRM to?

I suppose they keep it "enabled".

thanks,
-- 
js
suse labs


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] x86: also disable FSRM if ERMS is disabled
  2023-01-16  5:26             ` Jiri Slaby
@ 2023-01-16 21:17               ` Borislav Petkov
  0 siblings, 0 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-01-16 21:17 UTC (permalink / raw)
  To: Jiri Slaby; +Cc: Daniel Verkamp, x86, linux-kernel, Tony Luck

On Mon, Jan 16, 2023 at 06:26:50AM +0100, Jiri Slaby wrote:
> Just so you know (I see you progressed with Ingo to fix this) -- despite I
> asked the very same question, I received no answer quite yet. I suppose it
> will sound like usual "because we can".

Yah, tell them it is totally useless to disable ERMS on an AMD system. Just
leave the both settings on.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2023-01-16 21:17 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-23  0:58 [PATCH] x86: also disable FSRM if ERMS is disabled Daniel Verkamp
2022-09-23 11:13 ` Borislav Petkov
2022-09-23 17:25   ` Daniel Verkamp
2022-09-23 17:51     ` Borislav Petkov
2022-10-07 18:08       ` Daniel Verkamp
2022-10-11 11:28         ` Borislav Petkov
2022-10-11 17:09           ` Luck, Tony
2022-10-11 17:52             ` Borislav Petkov
2022-10-11 19:08               ` Luck, Tony
2022-10-11 20:56                 ` Borislav Petkov
2022-10-11 22:19                   ` Luck, Tony
2022-10-11 22:59                     ` Andrew Cooper
2023-01-04  7:43         ` Jiri Slaby
2023-01-04 11:39           ` Borislav Petkov
2023-01-14  9:19             ` Ingo Molnar
2023-01-14  9:58               ` Borislav Petkov
2023-01-16  5:26             ` Jiri Slaby
2023-01-16 21:17               ` Borislav Petkov

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.