[PATCH V3 0/1] x86/sgx: Add code to inject hwpoison into SGX memory

[PATCH V3 0/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Thomas Tai]

This patch adds code to inject hwpoison into SGX memory. The SGX memory
is processor reserved memory that cannot be directly accessed by system
software.

Changes since v2:
- As suggested by Naoya Horiguchi <naoya.horiguchi@nec.com>, add a 
  new mf_action_page_type MF_MSG_SGX and use action_result() to
  show the page type and the action taken.
- Rewrite hwpoison.rst to show the output from dmesg after injection.
- Add reinette.chatre@intel.com to the email list. Sorry that I
  mistyped the email address in the previous emails.
- v2 Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>

Changes since v1:
- Add a comment in hwpoison_inject as suggested by Miaohe
- v1: Reviewed-by: Tony Luck <tony.luck@intel.com>
- v1: Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>


Thomas Tai (1):
  x86/sgx: Add code to inject hwpoison into SGX memory

 Documentation/mm/hwpoison.rst | 24 ++++++++++++++++++++++++
 include/linux/mm.h            |  1 +
 include/ras/ras_event.h       |  1 +
 mm/hwpoison-inject.c          |  4 ++++
 mm/memory-failure.c           |  5 ++++-
 5 files changed, 34 insertions(+), 1 deletion(-)

-- 
2 .31.1

[PATCH V3 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Thomas Tai]

Inspired by commit c6acb1e7bf46 ("x86/sgx: Add hook to error injection
address validation"), add a similar code in hwpoison_inject function to
check if the address is located in SGX Memory. The error will then be
handled by the arch_memory_failure function in the SGX driver. After
injection, the action_result() will print out the page type and the
action taken.

Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
---
 Documentation/mm/hwpoison.rst | 24 ++++++++++++++++++++++++
 include/linux/mm.h            |  1 +
 include/ras/ras_event.h       |  1 +
 mm/hwpoison-inject.c          |  4 ++++
 mm/memory-failure.c           |  5 ++++-
 5 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/Documentation/mm/hwpoison.rst b/Documentation/mm/hwpoison.rst
index b9d5253c1305..100894bb020c 100644
--- a/Documentation/mm/hwpoison.rst
+++ b/Documentation/mm/hwpoison.rst
@@ -162,6 +162,30 @@ Testing
 
   Some portable hwpoison test programs in mce-test, see below.
 
+* Special notes for injection into SGX enclaves
+
+  1) Determine physical address of enclave page
+
+	dmesg | grep "sgx: EPC"
+
+	sgx: EPC section 0x8000c00000-0x807f7fffff
+	sgx: EPC section 0x10000c00000-0x1007fffffff
+
+  2) Convert the EPC address to page frame number.
+
+	For 4K page size, the page frame number for 0x8000c00000 is
+	0x8000c00000 / 0x1000 = 0x8000c00.
+
+  3) Inject a memory error
+
+	modprobe hwpoison-inject
+	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
+
+  4) Check dmesg output
+
+        dmesg | grep "Memory failure"
+        Memory failure: 0x8000c00: recovery action for sgx page: Recovered
+
 References
 ==========
 
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 21f8b27bd9fd..cdca3ff1418c 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3248,6 +3248,7 @@ enum mf_action_page_type {
 	MF_MSG_BUDDY,
 	MF_MSG_DAX,
 	MF_MSG_UNSPLIT_THP,
+	MF_MSG_SGX,
 	MF_MSG_UNKNOWN,
 };
 
diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h
index cbd3ddd7c33d..ee3a925c1e9d 100644
--- a/include/ras/ras_event.h
+++ b/include/ras/ras_event.h
@@ -373,6 +373,7 @@ TRACE_EVENT(aer_event,
 	EM ( MF_MSG_BUDDY, "free buddy page" )				\
 	EM ( MF_MSG_DAX, "dax page" )					\
 	EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" )			\
+	EM ( MF_MSG_SGX, "sgx page" )					\
 	EMe ( MF_MSG_UNKNOWN, "unknown page" )
 
 /*
diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
index 65e242b5a432..141eeeb793b1 100644
--- a/mm/hwpoison-inject.c
+++ b/mm/hwpoison-inject.c
@@ -21,6 +21,10 @@ static int hwpoison_inject(void *data, u64 val)
 	if (!capable(CAP_SYS_ADMIN))
 		return -EPERM;
 
+	/* Inject the error if the page is part of the processor reserved memory */
+	if (arch_is_platform_page(pfn << PAGE_SHIFT))
+		goto inject;
+
 	if (!pfn_valid(pfn))
 		return -ENXIO;
 
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 14439806b5ef..40a22b23b50a 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -781,6 +781,7 @@ static const char * const action_page_types[] = {
 	[MF_MSG_BUDDY]			= "free buddy page",
 	[MF_MSG_DAX]			= "dax page",
 	[MF_MSG_UNSPLIT_THP]		= "unsplit thp",
+	[MF_MSG_SGX]			= "sgx page",
 	[MF_MSG_UNKNOWN]		= "unknown page",
 };
 
@@ -1990,8 +1991,10 @@ int memory_failure(unsigned long pfn, int flags)
 	p = pfn_to_online_page(pfn);
 	if (!p) {
 		res = arch_memory_failure(pfn, flags);
-		if (res == 0)
+		if (res == 0) {
+			action_result(pfn, MF_MSG_SGX, MF_RECOVERED);
 			goto unlock_mutex;
+		}
 
 		if (pfn_valid(pfn)) {
 			pgmap = get_dev_pagemap(pfn, NULL);
-- 
2.31.1

Re: [PATCH V3 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[HORIGUCHI NAOYA(堀口 直也)]

On Mon, Oct 17, 2022 at 06:33:05PM -0400, Thomas Tai wrote:
> Inspired by commit c6acb1e7bf46 ("x86/sgx: Add hook to error injection
> address validation"), add a similar code in hwpoison_inject function to
> check if the address is located in SGX Memory. The error will then be
> handled by the arch_memory_failure function in the SGX driver. After
> injection, the action_result() will print out the page type and the
> action taken.
> 
> Signed-off-by: Thomas Tai <thomas.tai@oracle.com>

Looks good to me. Thank you for the update.

Acked-by: Naoya Horiguchi <naoya.horiguchi@nec.com>

Re: [PATCH V3 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Miaohe Lin]

On 2022/10/18 6:33, Thomas Tai wrote:
> Inspired by commit c6acb1e7bf46 ("x86/sgx: Add hook to error injection
> address validation"), add a similar code in hwpoison_inject function to
> check if the address is located in SGX Memory. The error will then be
> handled by the arch_memory_failure function in the SGX driver. After
> injection, the action_result() will print out the page type and the
> action taken.
> 
> Signed-off-by: Thomas Tai <thomas.tai@oracle.com>

Many thanks for your work. The patch looks good to me.

Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>

Thanks,
Miaohe Lin

Re: [PATCH V3 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Jarkko Sakkinen]

On Mon, Oct 17, 2022 at 06:33:05PM -0400, Thomas Tai wrote:
> Inspired by commit c6acb1e7bf46 ("x86/sgx: Add hook to error injection
> address validation"), add a similar code in hwpoison_inject function to
> check if the address is located in SGX Memory. The error will then be
> handled by the arch_memory_failure function in the SGX driver. After
> injection, the action_result() will print out the page type and the
> action taken.
> 
> Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> ---
>  Documentation/mm/hwpoison.rst | 24 ++++++++++++++++++++++++
>  include/linux/mm.h            |  1 +
>  include/ras/ras_event.h       |  1 +
>  mm/hwpoison-inject.c          |  4 ++++
>  mm/memory-failure.c           |  5 ++++-
>  5 files changed, 34 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/mm/hwpoison.rst b/Documentation/mm/hwpoison.rst
> index b9d5253c1305..100894bb020c 100644
> --- a/Documentation/mm/hwpoison.rst
> +++ b/Documentation/mm/hwpoison.rst
> @@ -162,6 +162,30 @@ Testing
>  
>    Some portable hwpoison test programs in mce-test, see below.
>  
> +* Special notes for injection into SGX enclaves
> +
> +  1) Determine physical address of enclave page
> +
> +	dmesg | grep "sgx: EPC"
> +
> +	sgx: EPC section 0x8000c00000-0x807f7fffff
> +	sgx: EPC section 0x10000c00000-0x1007fffffff
> +
> +  2) Convert the EPC address to page frame number.
> +
> +	For 4K page size, the page frame number for 0x8000c00000 is
> +	0x8000c00000 / 0x1000 = 0x8000c00.
> +
> +  3) Inject a memory error
> +
> +	modprobe hwpoison-inject
> +	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
> +
> +  4) Check dmesg output
> +
> +        dmesg | grep "Memory failure"
> +        Memory failure: 0x8000c00: recovery action for sgx page: Recovered
> +
>  References
>  ==========
>  
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 21f8b27bd9fd..cdca3ff1418c 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -3248,6 +3248,7 @@ enum mf_action_page_type {
>  	MF_MSG_BUDDY,
>  	MF_MSG_DAX,
>  	MF_MSG_UNSPLIT_THP,
> +	MF_MSG_SGX,
>  	MF_MSG_UNKNOWN,
>  };
>  
> diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h
> index cbd3ddd7c33d..ee3a925c1e9d 100644
> --- a/include/ras/ras_event.h
> +++ b/include/ras/ras_event.h
> @@ -373,6 +373,7 @@ TRACE_EVENT(aer_event,
>  	EM ( MF_MSG_BUDDY, "free buddy page" )				\
>  	EM ( MF_MSG_DAX, "dax page" )					\
>  	EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" )			\
> +	EM ( MF_MSG_SGX, "sgx page" )					\
>  	EMe ( MF_MSG_UNKNOWN, "unknown page" )
>  
>  /*
> diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
> index 65e242b5a432..141eeeb793b1 100644
> --- a/mm/hwpoison-inject.c
> +++ b/mm/hwpoison-inject.c
> @@ -21,6 +21,10 @@ static int hwpoison_inject(void *data, u64 val)
>  	if (!capable(CAP_SYS_ADMIN))
>  		return -EPERM;
>  
> +	/* Inject the error if the page is part of the processor reserved memory */
> +	if (arch_is_platform_page(pfn << PAGE_SHIFT))
> +		goto inject;
> +
>  	if (!pfn_valid(pfn))
>  		return -ENXIO;
>  
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 14439806b5ef..40a22b23b50a 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -781,6 +781,7 @@ static const char * const action_page_types[] = {
>  	[MF_MSG_BUDDY]			= "free buddy page",
>  	[MF_MSG_DAX]			= "dax page",
>  	[MF_MSG_UNSPLIT_THP]		= "unsplit thp",
> +	[MF_MSG_SGX]			= "sgx page",
>  	[MF_MSG_UNKNOWN]		= "unknown page",
>  };
>  
> @@ -1990,8 +1991,10 @@ int memory_failure(unsigned long pfn, int flags)
>  	p = pfn_to_online_page(pfn);
>  	if (!p) {
>  		res = arch_memory_failure(pfn, flags);
> -		if (res == 0)
> +		if (res == 0) {
> +			action_result(pfn, MF_MSG_SGX, MF_RECOVERED);
>  			goto unlock_mutex;
> +		}
>  
>  		if (pfn_valid(pfn)) {
>  			pgmap = get_dev_pagemap(pfn, NULL);
> -- 
> 2.31.1
> 

Acked-by: Jarkko Sakkinen <jarkko.sakkinen@iki.fi>

BR, Jarkko