From: Jarkko Sakkinen <jarkko@kernel.org>
To: linux-sgx@vger.kernel.org
Cc: Haitao Huang <haitao.huang@linux.intel.com>,
Vijay Dhanraj <vijay.dhanraj@intel.com>,
Reinette Chatre <reinette.chatre@intel.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
Paul Menzel <pmenzel@molgen.mpg.de>,
Jarkko Sakkinen <jarkko@kernel.org>,
stable@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
x86@kernel.org (maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)),
"H. Peter Anvin" <hpa@zytor.com>,
linux-kernel@vger.kernel.org (open list:X86 ARCHITECTURE (32-BIT
AND 64-BIT))
Subject: [PATCH v2 2/6] x86/sgx: Do not consider unsanitized pages an error
Date: Wed, 31 Aug 2022 20:38:25 +0300 [thread overview]
Message-ID: <20220831173829.126661-3-jarkko@kernel.org> (raw)
In-Reply-To: <20220831173829.126661-1-jarkko@kernel.org>
In sgx_init(), if misc_register() fails or misc_register() succeeds but
neither sgx_drv_init() nor sgx_vepc_init() succeeds, then ksgxd will be
prematurely stopped. This may leave some unsanitized pages, which does
not matter, because SGX will be disabled for the whole power cycle.
This triggers WARN_ON() because sgx_dirty_page_list ends up being
non-empty, and dumps the call stack:
[ 0.268103] sgx: EPC section 0x40200000-0x45f7ffff
[ 0.268591] ------------[ cut here ]------------
[ 0.268592] WARNING: CPU: 6 PID: 83 at
arch/x86/kernel/cpu/sgx/main.c:401 ksgxd+0x1b7/0x1d0
[ 0.268598] Modules linked in:
[ 0.268600] CPU: 6 PID: 83 Comm: ksgxd Not tainted 6.0.0-rc2 #382
[ 0.268603] Hardware name: Dell Inc. XPS 13 9370/0RMYH9, BIOS 1.21.0
07/06/2022
[ 0.268604] RIP: 0010:ksgxd+0x1b7/0x1d0
[ 0.268607] Code: ff e9 f2 fe ff ff 48 89 df e8 75 07 0e 00 84 c0 0f
84 c3 fe ff ff 31 ff e8 e6 07 0e 00 84 c0 0f 85 94 fe ff ff e9 af fe ff
ff <0f> 0b e9 7f fe ff ff e8 dd 9c 95 00 66 66 2e 0f 1f 84 00 00 00 00
[ 0.268608] RSP: 0000:ffffb6c7404f3ed8 EFLAGS: 00010287
[ 0.268610] RAX: ffffb6c740431a10 RBX: ffff8dcd8117b400 RCX:
0000000000000000
[ 0.268612] RDX: 0000000080000000 RSI: ffffb6c7404319d0 RDI:
00000000ffffffff
[ 0.268613] RBP: ffff8dcd820a4d80 R08: ffff8dcd820a4180 R09:
ffff8dcd820a4180
[ 0.268614] R10: 0000000000000000 R11: 0000000000000006 R12:
ffffb6c74006bce0
[ 0.268615] R13: ffff8dcd80e63880 R14: ffffffffa8a60f10 R15:
0000000000000000
[ 0.268616] FS: 0000000000000000(0000) GS:ffff8dcf25580000(0000)
knlGS:0000000000000000
[ 0.268617] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.268619] CR2: 0000000000000000 CR3: 0000000213410001 CR4:
00000000003706e0
[ 0.268620] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 0.268621] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[ 0.268622] Call Trace:
[ 0.268624] <TASK>
[ 0.268627] ? _raw_spin_lock_irqsave+0x24/0x60
[ 0.268632] ? _raw_spin_unlock_irqrestore+0x23/0x40
[ 0.268634] ? __kthread_parkme+0x36/0x90
[ 0.268637] kthread+0xe5/0x110
[ 0.268639] ? kthread_complete_and_exit+0x20/0x20
[ 0.268642] ret_from_fork+0x1f/0x30
[ 0.268647] </TASK>
[ 0.268648] ---[ end trace 0000000000000000 ]---
Ultimately this can crash the kernel, if the following is set:
/proc/sys/kernel/panic_on_warn
In premature stop, print nothing, as the number is by practical means a
random number. Otherwise, it is an indicator of a bug in the driver, and
therefore print the number of unsanitized pages with pr_err().
Link: https://lore.kernel.org/linux-sgx/20220825051827.246698-1-jarkko@kernel.org/T/#u
Fixes: 51ab30eb2ad4 ("x86/sgx: Replace section->init_laundry_list with sgx_dirty_page_list")
Cc: stable@vger.kernel.org # v5.13+
Reported-by: Paul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
---
v6:
- Address Reinette's feedback:
https://lore.kernel.org/linux-sgx/Yw6%2FiTzSdSw%2FY%2FVO@kernel.org/
v5:
- Add the klog dump and sysctl option to the commit message.
v4:
- Explain expectations for dirty_page_list in the function header, instead
of an inline comment.
- Improve commit message to explain the conditions better.
- Return the number of pages left dirty to ksgxd() and print warning after
the 2nd call, if there are any.
v3:
- Remove WARN_ON().
- Tuned comments and the commit message a bit.
v2:
- Replaced WARN_ON() with optional pr_info() inside
__sgx_sanitize_pages().
- Rewrote the commit message.
- Added the fixes tag.
---
arch/x86/kernel/cpu/sgx/main.c | 42 ++++++++++++++++++++++++++++------
1 file changed, 35 insertions(+), 7 deletions(-)
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 515e2a5f25bb..bcd6b64961bd 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -49,17 +49,20 @@ static LIST_HEAD(sgx_dirty_page_list);
* Reset post-kexec EPC pages to the uninitialized state. The pages are removed
* from the input list, and made available for the page allocator. SECS pages
* prepending their children in the input list are left intact.
+ *
+ * Contents of the @dirty_page_list must be thread-local, i.e.
+ * not shared by multiple threads.
*/
-static void __sgx_sanitize_pages(struct list_head *dirty_page_list)
+static long __sgx_sanitize_pages(struct list_head *dirty_page_list)
{
struct sgx_epc_page *page;
+ long left_dirty = 0;
LIST_HEAD(dirty);
int ret;
- /* dirty_page_list is thread-local, no need for a lock: */
while (!list_empty(dirty_page_list)) {
if (kthread_should_stop())
- return;
+ return -ECANCELED;
page = list_first_entry(dirty_page_list, struct sgx_epc_page, list);
@@ -92,12 +95,14 @@ static void __sgx_sanitize_pages(struct list_head *dirty_page_list)
} else {
/* The page is not yet clean - move to the dirty list. */
list_move_tail(&page->list, &dirty);
+ left_dirty++;
}
cond_resched();
}
list_splice(&dirty, dirty_page_list);
+ return left_dirty;
}
static bool sgx_reclaimer_age(struct sgx_epc_page *epc_page)
@@ -388,17 +393,40 @@ void sgx_reclaim_direct(void)
static int ksgxd(void *p)
{
+ long ret;
+
set_freezable();
/*
* Sanitize pages in order to recover from kexec(). The 2nd pass is
* required for SECS pages, whose child pages blocked EREMOVE.
*/
- __sgx_sanitize_pages(&sgx_dirty_page_list);
- __sgx_sanitize_pages(&sgx_dirty_page_list);
+ ret = __sgx_sanitize_pages(&sgx_dirty_page_list);
+ if (ret == -ECANCELED)
+ /* kthread stopped */
+ return 0;
- /* sanity check: */
- WARN_ON(!list_empty(&sgx_dirty_page_list));
+ ret = __sgx_sanitize_pages(&sgx_dirty_page_list);
+ switch (ret) {
+ case 0:
+ /* success, no unsanitized pages */
+ break;
+
+ case -ECANCELED:
+ /* kthread stopped */
+ return 0;
+
+ default:
+ /*
+ * Never expected to happen in a working driver. If it happens
+ * the bug is expected to be in the sanitization process, but
+ * successfully sanitized pages are still valid and driver can
+ * be used and most importantly debugged without issues. To put
+ * short, the global state of kernel is not corrupted so no
+ * reason to do any more complicated rollback.
+ */
+ pr_err("%ld unsanitized pages\n", ret);
+ }
while (!kthread_should_stop()) {
if (try_to_freeze())
--
2.37.2
next prev parent reply other threads:[~2022-08-31 17:38 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-31 17:38 [PATCH v2 0/6] x86/sgx: A collection of tests and fixes Jarkko Sakkinen
2022-08-31 17:38 ` [PATCH v2 1/6] selftests/sgx: Ignore OpenSSL 3.0 deprecated functions warning Jarkko Sakkinen
2022-08-31 17:38 ` Jarkko Sakkinen [this message]
2022-08-31 20:39 ` [PATCH v2 2/6] x86/sgx: Do not consider unsanitized pages an error Reinette Chatre
2022-09-01 10:50 ` Huang, Kai
2022-09-01 21:47 ` jarkko
2022-09-01 21:53 ` Jarkko Sakkinen
2022-09-01 21:56 ` Jarkko Sakkinen
2022-09-01 22:01 ` Jarkko Sakkinen
2022-09-01 22:34 ` Reinette Chatre
2022-09-01 23:56 ` Jarkko Sakkinen
2022-09-02 13:26 ` Jarkko Sakkinen
2022-09-02 15:53 ` Jarkko Sakkinen
2022-09-02 16:08 ` Reinette Chatre
2022-09-02 16:30 ` Jarkko Sakkinen
2022-09-02 17:38 ` Reinette Chatre
2022-09-02 19:20 ` Jarkko Sakkinen
2022-08-31 17:38 ` [PATCH v2 3/6] x86/sgx: Handle VA page allocation failure for EAUG on PF Jarkko Sakkinen
2022-08-31 18:08 ` Reinette Chatre
2022-08-31 18:21 ` Jarkko Sakkinen
2022-08-31 18:33 ` Reinette Chatre
2022-08-31 18:46 ` Jarkko Sakkinen
2022-08-31 17:38 ` [PATCH v2 4/6] selftests/sgx: Add SGX selftest augment_via_eaccept_long Jarkko Sakkinen
2022-08-31 20:07 ` Reinette Chatre
2022-09-01 22:22 ` Jarkko Sakkinen
2022-09-01 23:12 ` Reinette Chatre
2022-09-02 0:03 ` Jarkko Sakkinen
2022-09-04 4:02 ` Jarkko Sakkinen
2022-09-04 4:21 ` Jarkko Sakkinen
2022-08-31 17:38 ` [PATCH v2 5/6] selftests/sgx: retry the ioctls returned with EAGAIN Jarkko Sakkinen
2022-08-31 20:08 ` Reinette Chatre
2022-08-31 17:38 ` [PATCH v2 6/6] selftests/sgx: Add a bpftrace script for tracking allocation errors Jarkko Sakkinen
2022-08-31 20:09 ` Reinette Chatre
2022-09-01 22:24 ` Jarkko Sakkinen
2022-08-31 17:43 ` [PATCH v2 0/6] x86/sgx: A collection of tests and fixes Dave Hansen
2022-08-31 18:11 ` Jarkko Sakkinen
2022-08-31 18:24 ` Dave Hansen
2022-08-31 18:47 ` Jarkko Sakkinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220831173829.126661-3-jarkko@kernel.org \
--to=jarkko@kernel.org \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=haitao.huang@linux.intel.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=pmenzel@molgen.mpg.de \
--cc=reinette.chatre@intel.com \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=vijay.dhanraj@intel.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).