[PATCH 0/4] Introduce support for PSF mitigation

[PATCH 0/4] Introduce support for PSF mitigation

[Ramakrishna Saripalli]

From: Ramakrishna Saripalli <rk.saripalli@amd.com>

Predictive Store Forwarding:
AMD Zen3 processors feature a new technology called
Predictive Store Forwarding (PSF).

https://www.amd.com/system/files/documents/security-analysis-predictive-store-forwarding.pdf

PSF is a hardware-based micro-architectural optimization designed
to improve the performance of code execution by predicting address
dependencies between loads and stores.

How PSF works:

It is very common for a CPU to execute a load instruction to an address
that was recently written by a store. Modern CPUs implement a technique
known as Store-To-Load-Forwarding (STLF) to improve performance in such
cases. With STLF, data from the store is forwarded directly to the load
without having to wait for it to be written to memory. In a typical CPU,
STLF occurs after the address of both the load and store are calculated
and determined to match.

PSF expands on this by speculating on the relationship between loads and
stores without waiting for the address calculation to complete. With PSF,
the CPU learns over time the relationship between loads and stores.
If STLF typically occurs between a particular store and load, the CPU will
remember this.

In typical code, PSF provides a performance benefit by speculating on
the load result and allowing later instructions to begin execution
sooner than they otherwise would be able to.

Causes of Incorrect PSF:

Incorrect PSF predictions can occur due to two reasons.

First, it is possible that the store/load pair had a dependency for a
while but later stops having a dependency.  This can occur if the address
of either the store or load changes during the execution of the program.

The second source of incorrect PSF predictions can occur if there is an
alias in the PSF predictor structure.  The PSF predictor tracks
store-load pairs based on portions of their RIP. It is possible that a
store-load pair which does have a dependency may alias in the predictor
with another store-load pair which does not.

This can result in incorrect speculation when the second store/load pair
is executed.

Security Analysis:

Previous research has shown that when CPUs speculate on non-architectural
paths it can lead to the potential of side channel attacks.
In particular, programs that implement isolation, also known as
‘sandboxing’, entirely in software may need to be concerned with incorrect
CPU speculation as they can occur due to bad PSF predictions.

Because PSF speculation is limited to the current program context,
the impact of bad PSF speculation is very similar to that of
Speculative Store Bypass (Spectre v4)

Predictive Store Forwarding controls:
There are two hardware control bits which influence the PSF feature:
- MSR 48h bit 2 – Speculative Store Bypass (SSBD)
- MSR 48h bit 7 – Predictive Store Forwarding Disable (PSFD)

The PSF feature is disabled if either of these bits are set.  These bits
are controllable on a per-thread basis in an SMT system. By default, both
SSBD and PSFD are 0 meaning that the speculation features are enabled.

While the SSBD bit disables PSF and speculative store bypass, PSFD only
disables PSF.

PSFD may be desirable for software which is concerned with the
speculative behavior of PSF but desires a smaller performance impact than
setting SSBD.

Support for PSFD is indicated in CPUID Fn8000_0008 EBX[28].
All processors that support PSF will also support PSFD.

Ramakrishna Saripalli (4):
  x86/cpufeatures: Define feature bits to support mitigation of PSF
  x86/speculation: Introduce SPEC_CTRL_MSR bit for PSFD
  x86/speculation: Implement PSF mitigation support
  x86/speculation: Add PSF mitigation kernel parameters

 .../admin-guide/kernel-parameters.txt         |  5 +++++
 arch/x86/include/asm/cpufeatures.h            |  1 +
 arch/x86/include/asm/msr-index.h              |  2 ++
 arch/x86/kernel/cpu/amd.c                     | 19 +++++++++++++++++++
 4 files changed, 27 insertions(+)


base-commit: 0e16f466004d7f04296b9676a712a32a12367d1f
-- 
2.25.1

[PATCH 1/4] x86/cpufeatures: Define feature bits to support mitigation of PSF

[Ramakrishna Saripalli]

From: Ramakrishna Saripalli <rk.saripalli@amd.com>

Certain AMD processors feature a new technology called Predictive Store
Forwarding (PSF).

PSF is a micro-architectural optimization designed to improve the
performance of code execution by predicting dependencies between
loads and stores.

Incorrect PSF predictions can occur due to two reasons.

- It is possible that the load/store pair may have had dependency for
  a while but the dependency has stopped because the address in the
  load/store pair has changed.

- Second source of incorrect PSF prediction can occur because of an alias
  in the PSF predictor structure stored in the microarchitectural state.
  PSF predictor tracks load/store pair based on portions of instruction
  pointer. It is possible that a load/store pair which does have a
  dependency may be aliased by another load/store pair which does not have
  the same dependency. This can result in incorrect speculation.

  Software may be able to detect this aliasing and perform side-channel
  attacks.

These features are being introduced to support mitigation from these attacks.

All CPUs that implement PSF provide one bit to disable this feature.
If the bit to disable this feature is available, it means that the CPU
implements PSF feature and is therefore vulnerable to PSF risks.

The bits that are introduced

X86_FEATURE_PSFD: CPUID_Fn80000008_EBX[28] ("PSF disable")
	If this bit is 1, CPU implements PSF and PSF mitigation is
	supported.

Signed-off-by: Ramakrishna Saripalli<rk.saripalli@amd.com>
---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index cc96e26d69f7..078f46022293 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -309,6 +309,7 @@
 #define X86_FEATURE_AMD_SSBD		(13*32+24) /* "" Speculative Store Bypass Disable */
 #define X86_FEATURE_VIRT_SSBD		(13*32+25) /* Virtualized Speculative Store Bypass Disable */
 #define X86_FEATURE_AMD_SSB_NO		(13*32+26) /* "" Speculative Store Bypass is fixed in hardware. */
+#define X86_FEATURE_PSFD		(13*32+28) /* Predictive Store Forward Disable */
 
 /* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */
 #define X86_FEATURE_DTHERM		(14*32+ 0) /* Digital Thermal Sensor */
-- 
2.25.1

[PATCH 2/4] x86/speculation: Introduce SPEC_CTRL_MSR bit for PSFD

[Ramakrishna Saripalli]

From: Ramakrishna Saripalli <rk.saripalli@amd.com>

All AMD processors that support PSF implement a bit in
SPEC_CTRL MSR (0x48) to disable or enable Predictive Store
Forwarding.

Signed-off-by: Ramakrishna Saripalli<rk.saripalli@amd.com>
---
 arch/x86/include/asm/msr-index.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 546d6ecf0a35..f569918c8754 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -51,6 +51,8 @@
 #define SPEC_CTRL_STIBP			BIT(SPEC_CTRL_STIBP_SHIFT)	/* STIBP mask */
 #define SPEC_CTRL_SSBD_SHIFT		2	   /* Speculative Store Bypass Disable bit */
 #define SPEC_CTRL_SSBD			BIT(SPEC_CTRL_SSBD_SHIFT)	/* Speculative Store Bypass Disable */
+#define SPEC_CTRL_PSFD_SHIFT		7
+#define SPEC_CTRL_PSFD			BIT(SPEC_CTRL_PSFD_SHIFT)	/* Predictive Store Forwarding Disable */
 
 #define MSR_IA32_PRED_CMD		0x00000049 /* Prediction Command */
 #define PRED_CMD_IBPB			BIT(0)	   /* Indirect Branch Prediction Barrier */
-- 
2.25.1

[PATCH 3/4] x86/speculation: Implement PSF mitigation support

[Ramakrishna Saripalli]

From: Ramakrishna Saripalli <rk.saripalli@amd.com>

Implement support for PSF mitigation via a kernel parameter.

Kernel parameter predict_store_fwd has the following values

- off. PSF mitigation is enabled which means the feature is disabled.

- on. PSF mitigation is not enabled. This is also the default behavior.

Signed-off-by: Ramakrishna Saripalli<rk.saripalli@amd.com>
---
 arch/x86/kernel/cpu/amd.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 347a956f71ca..88aac52eeb1b 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -1170,3 +1170,22 @@ void set_dr_addr_mask(unsigned long mask, int dr)
 		break;
 	}
 }
+
+static int __init psf_cmdline(char *str)
+{
+	if (!boot_cpu_has(X86_FEATURE_PSFD))
+		return 0;
+
+	if (!str)
+		return -EINVAL;
+
+	if (!strcmp(str, "off")) {
+		x86_spec_ctrl_base |= SPEC_CTRL_PSFD;
+		wrmsrl(MSR_IA32_SPEC_CTRL, x86_spec_ctrl_base);
+		setup_clear_cpu_cap(X86_FEATURE_PSFD);
+	}
+
+	return 0;
+}
+
+early_param("predict_store_fwd", psf_cmdline);
-- 
2.25.1

[PATCH 4/4] x86/speculation: Add PSF mitigation kernel parameters

[Ramakrishna Saripalli]

From: Ramakrishna Saripalli <rk.saripalli@amd.com>

PSF mitigation introduces a new kernel parameter called
	predict_store_fwd.

Signed-off-by: Ramakrishna Saripalli<rk.saripalli@amd.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 04545725f187..58f6bd02385b 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3940,6 +3940,11 @@
 			Format: {"off"}
 			Disable Hardware Transactional Memory
 
+	predict_store_fwd	[X86] This option controls PSF mitigation
+			off - Turns on PSF mitigation.
+			on  - Turns off PSF mitigation.
+			default : on.
+
 	preempt=	[KNL]
 			Select preemption mode if you have CONFIG_PREEMPT_DYNAMIC
 			none - Limited to cond_resched() calls
-- 
2.25.1

Re: [PATCH 4/4] x86/speculation: Add PSF mitigation kernel parameters

[Randy Dunlap]

Hi,

On 4/21/21 2:01 AM, Ramakrishna Saripalli wrote:
> From: Ramakrishna Saripalli <rk.saripalli@amd.com>
> 
> PSF mitigation introduces a new kernel parameter called
> 	predict_store_fwd.
> 
> Signed-off-by: Ramakrishna Saripalli<rk.saripalli@amd.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 04545725f187..58f6bd02385b 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -3940,6 +3940,11 @@
>  			Format: {"off"}
>  			Disable Hardware Transactional Memory
>  
> +	predict_store_fwd	[X86] This option controls PSF mitigation
> +			off - Turns on PSF mitigation.
> +			on  - Turns off PSF mitigation.
> +			default : on.

This should be formatted more like:

+	predict_store_fwd=	[X86] This option controls PSF mitigation
+			off - Turns on PSF mitigation.
+			on  - Turns off PSF mitigation.
+			default: on.

But why does "off" turn it on and "on" turn it off?


> +
>  	preempt=	[KNL]
>  			Select preemption mode if you have CONFIG_PREEMPT_DYNAMIC
>  			none - Limited to cond_resched() calls
> 

thanks.
-- 
~Randy

Re: [PATCH 4/4] x86/speculation: Add PSF mitigation kernel parameters

[Bandan Das]

Randy Dunlap <rdunlap@infradead.org> writes:

> Hi,
>
> On 4/21/21 2:01 AM, Ramakrishna Saripalli wrote:
>> From: Ramakrishna Saripalli <rk.saripalli@amd.com>
>> 
>> PSF mitigation introduces a new kernel parameter called
>> 	predict_store_fwd.
>> 
>> Signed-off-by: Ramakrishna Saripalli<rk.saripalli@amd.com>
>> ---
>>  Documentation/admin-guide/kernel-parameters.txt | 5 +++++
>>  1 file changed, 5 insertions(+)
>> 
>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>> index 04545725f187..58f6bd02385b 100644
>> --- a/Documentation/admin-guide/kernel-parameters.txt
>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>> @@ -3940,6 +3940,11 @@
>>  			Format: {"off"}
>>  			Disable Hardware Transactional Memory
>>  
>> +	predict_store_fwd	[X86] This option controls PSF mitigation
>> +			off - Turns on PSF mitigation.
>> +			on  - Turns off PSF mitigation.
>> +			default : on.
>
> This should be formatted more like:
>
> +	predict_store_fwd=	[X86] This option controls PSF mitigation
> +			off - Turns on PSF mitigation.
> +			on  - Turns off PSF mitigation.
> +			default: on.
>
> But why does "off" turn it on and "on" turn it off?
>
Maybe, rename the parameter to something like psfd_disable, then off -> disables mitigation and on -> enables it.
Or just rewriting this to off -> turns off predictive store forwarding is probably ok too.

Bandan

>
>> +
>>  	preempt=	[KNL]
>>  			Select preemption mode if you have CONFIG_PREEMPT_DYNAMIC
>>  			none - Limited to cond_resched() calls
>> 
>
> thanks.

Re: [PATCH 4/4] x86/speculation: Add PSF mitigation kernel parameters

[Saripalli, RK]

Bandan / Randy, this convention was chosen based on review of the earlier patches from Boris.

For now, the functionality will be just on and off.
Later based on interest from community and other factors, I will online the prctl and seccomp variants of the 
mitigation.

Thanks,
RK

On 4/21/2021 1:32 PM, Bandan Das wrote:
> Randy Dunlap <rdunlap@infradead.org> writes:
> 
>> Hi,
>>
>> On 4/21/21 2:01 AM, Ramakrishna Saripalli wrote:
>>> From: Ramakrishna Saripalli <rk.saripalli@amd.com>
>>>
>>> PSF mitigation introduces a new kernel parameter called
>>> 	predict_store_fwd.
>>>
>>> Signed-off-by: Ramakrishna Saripalli<rk.saripalli@amd.com>
>>> ---
>>>  Documentation/admin-guide/kernel-parameters.txt | 5 +++++
>>>  1 file changed, 5 insertions(+)
>>>
>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>>> index 04545725f187..58f6bd02385b 100644
>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>> @@ -3940,6 +3940,11 @@
>>>  			Format: {"off"}
>>>  			Disable Hardware Transactional Memory
>>>  
>>> +	predict_store_fwd	[X86] This option controls PSF mitigation
>>> +			off - Turns on PSF mitigation.
>>> +			on  - Turns off PSF mitigation.
>>> +			default : on.
>>
>> This should be formatted more like:
>>
>> +	predict_store_fwd=	[X86] This option controls PSF mitigation
>> +			off - Turns on PSF mitigation.
>> +			on  - Turns off PSF mitigation.
>> +			default: on.
>>
>> But why does "off" turn it on and "on" turn it off?
>>
> Maybe, rename the parameter to something like psfd_disable, then off -> disables mitigation and on -> enables it.
> Or just rewriting this to off -> turns off predictive store forwarding is probably ok too.
> 
> Bandan
> 
>>
>>> +
>>>  	preempt=	[KNL]
>>>  			Select preemption mode if you have CONFIG_PREEMPT_DYNAMIC
>>>  			none - Limited to cond_resched() calls
>>>
>>
>> thanks.
> 

Re: [PATCH 4/4] x86/speculation: Add PSF mitigation kernel parameters

[Borislav Petkov]

On Wed, Apr 21, 2021 at 02:32:13PM -0400, Bandan Das wrote:
> Maybe, rename the parameter to something like psfd_disable, then off
> -> disables mitigation and on -> enables it. Or just rewriting this to
> off -> turns off predictive store forwarding is probably ok too.

Yes:

	off - Turns off predictive store forwarding.
	on  - Turns on...

Ramakrishna, you don't have to call this a mitigation - this is a flag
which controls the feature.

Also, those 4 patches can be merged into a single one which simply adds
the feature along with the boot-time controls.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH 4/4] x86/speculation: Add PSF mitigation kernel parameters

[Saripalli, RK]

On 4/21/2021 1:48 PM, Borislav Petkov wrote:
> On Wed, Apr 21, 2021 at 02:32:13PM -0400, Bandan Das wrote:
>> Maybe, rename the parameter to something like psfd_disable, then off
>> -> disables mitigation and on -> enables it. Or just rewriting this to
>> off -> turns off predictive store forwarding is probably ok too.
> 
> Yes:
> 
> 	off - Turns off predictive store forwarding.
> 	on  - Turns on...
> 
> Ramakrishna, you don't have to call this a mitigation - this is a flag
> which controls the feature.

Agreed. I will fix it.
> 
> Also, those 4 patches can be merged into a single one which simply adds
> the feature along with the boot-time controls.

I separated them into separate patches because the KVM patch depends on one of the patch.
The corresponding QEMU patch depends on another patch.

By separating them into 4 separate patches, my thinking was I could keep them logically separate.
Yes, I can combine all 4 patches into one patch but would like to get feedback before I do so.
> 
> Thx.
> 

Re: [PATCH 4/4] x86/speculation: Add PSF mitigation kernel parameters

[Borislav Petkov]

On Wed, Apr 21, 2021 at 01:55:03PM -0500, Saripalli, RK wrote:
> I separated them into separate patches because the KVM patch depends on one of the patch.
> The corresponding QEMU patch depends on another patch.
> 
> By separating them into 4 separate patches, my thinking was I could keep them logically separate.

Sure but they all go together through the same tree - why does that
matter for Qemu/KVM?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH 4/4] x86/speculation: Add PSF mitigation kernel parameters

[Saripalli, RK]

On 4/21/2021 1:57 PM, Borislav Petkov wrote:
> On Wed, Apr 21, 2021 at 01:55:03PM -0500, Saripalli, RK wrote:
>> I separated them into separate patches because the KVM patch depends on one of the patch.
>> The corresponding QEMU patch depends on another patch.
>>
>> By separating them into 4 separate patches, my thinking was I could keep them logically separate.
> 
> Sure but they all go together through the same tree - why does that
> matter for Qemu/KVM?

Ok. I will combine all the 4 patches into one patch and resend that one patch.

Thanks,
RK
>