All of lore.kernel.org
 help / color / mirror / Atom feed
From: Haitao Huang <haitao.huang@linux.intel.com>
To: jarkko@kernel.org, dave.hansen@linux.intel.com, tj@kernel.org,
	linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org,
	cgroups@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>
Cc: kai.huang@intel.com, reinette.chatre@intel.com,
	Sean Christopherson <sean.j.christopherson@intel.com>,
	zhiquan1.li@intel.com, kristen@linux.intel.com,
	seanjc@google.com
Subject: [PATCH v3 10/28] x86/sgx: Allow reclaiming up to 32 pages, but scan 16 by default
Date: Wed, 12 Jul 2023 16:01:44 -0700	[thread overview]
Message-ID: <20230712230202.47929-11-haitao.huang@linux.intel.com> (raw)
In-Reply-To: <20230712230202.47929-1-haitao.huang@linux.intel.com>

From: Sean Christopherson <sean.j.christopherson@intel.com>

Modify sgx_reclaim_pages() to take a parameter that specifies the
number of pages to scan for reclaiming. Specify a max value of
32, but scan 16 in the usual case. This allows the number of pages
sgx_reclaim_pages() scans to be specified by the caller, and adjusted
in future patches.

Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Kristen Carlson Accardi <kristen@linux.intel.com>
Signed-off-by: Haitao Huang <haitao.huang@linux.intel.com>
Cc: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kernel/cpu/sgx/main.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index f3a3ed894616..cd5e5517866a 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -17,6 +17,10 @@
 #include "driver.h"
 #include "encl.h"
 #include "encls.h"
+/**
+ * Maximum number of pages to scan for reclaiming.
+ */
+#define SGX_NR_TO_SCAN_MAX	32
 
 struct sgx_epc_section sgx_epc_sections[SGX_MAX_EPC_SECTIONS];
 static int sgx_nr_epc_sections;
@@ -279,7 +283,10 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
 	mutex_unlock(&encl->lock);
 }
 
-/*
+/**
+ * sgx_reclaim_pages() - Reclaim EPC pages from the consumers
+ * @nr_to_scan:		 Number of EPC pages to scan for reclaim
+ *
  * Take a fixed number of pages from the head of the active page pool and
  * reclaim them to the enclave's private shmem files. Skip the pages, which have
  * been accessed since the last scan. Move those pages to the tail of active
@@ -292,9 +299,9 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
  * problematic as it would increase the lock contention too much, which would
  * halt forward progress.
  */
-static void sgx_reclaim_pages(void)
+static void sgx_reclaim_pages(int nr_to_scan)
 {
-	struct sgx_backing backing[SGX_NR_TO_SCAN];
+	struct sgx_backing backing[SGX_NR_TO_SCAN_MAX];
 	struct sgx_epc_page *epc_page, *tmp;
 	struct sgx_encl_page *encl_page;
 	pgoff_t page_index;
@@ -332,7 +339,7 @@ static void sgx_reclaim_pages(void)
 	list_for_each_entry_safe(epc_page, tmp, &iso, list) {
 		encl_page = epc_page->encl_page;
 
-		if (!sgx_reclaimer_age(epc_page))
+		if (i == SGX_NR_TO_SCAN_MAX || !sgx_reclaimer_age(epc_page))
 			goto skip;
 
 		page_index = PFN_DOWN(encl_page->desc - encl_page->encl->base);
@@ -387,7 +394,7 @@ static bool sgx_should_reclaim(unsigned long watermark)
 void sgx_reclaim_direct(void)
 {
 	if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
-		sgx_reclaim_pages();
+		sgx_reclaim_pages(SGX_NR_TO_SCAN);
 }
 
 static int ksgxd(void *p)
@@ -410,7 +417,7 @@ static int ksgxd(void *p)
 				     sgx_should_reclaim(SGX_NR_HIGH_PAGES));
 
 		if (sgx_should_reclaim(SGX_NR_HIGH_PAGES))
-			sgx_reclaim_pages();
+			sgx_reclaim_pages(SGX_NR_TO_SCAN);
 
 		cond_resched();
 	}
@@ -582,7 +589,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
 			break;
 		}
 
-		sgx_reclaim_pages();
+		sgx_reclaim_pages(SGX_NR_TO_SCAN);
 		cond_resched();
 	}
 
-- 
2.25.1


WARNING: multiple messages have this Message-ID (diff)
From: Haitao Huang <haitao.huang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
To: jarkko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
	dave.hansen-VuQAYsv1563Yd54FQh9/CA@public.gmane.org,
	tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-sgx-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	Thomas Gleixner <tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org>,
	Ingo Molnar <mingo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	Borislav Petkov <bp-Gina5bIWoIWzQB+pC5nmwQ@public.gmane.org>,
	x86-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
	"H. Peter Anvin" <hpa-YMNOUZJC4hwAvxtiuMwx3w@public.gmane.org>
Cc: kai.huang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	reinette.chatre-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	Sean Christopherson
	<sean.j.christopherson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
	zhiquan1.li-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	kristen-VuQAYsv1563Yd54FQh9/CA@public.gmane.org,
	seanjc-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org
Subject: [PATCH v3 10/28] x86/sgx: Allow reclaiming up to 32 pages, but scan 16 by default
Date: Wed, 12 Jul 2023 16:01:44 -0700	[thread overview]
Message-ID: <20230712230202.47929-11-haitao.huang@linux.intel.com> (raw)
In-Reply-To: <20230712230202.47929-1-haitao.huang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>

From: Sean Christopherson <sean.j.christopherson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

Modify sgx_reclaim_pages() to take a parameter that specifies the
number of pages to scan for reclaiming. Specify a max value of
32, but scan 16 in the usual case. This allows the number of pages
sgx_reclaim_pages() scans to be specified by the caller, and adjusted
in future patches.

Signed-off-by: Sean Christopherson <sean.j.christopherson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Kristen Carlson Accardi <kristen-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Signed-off-by: Haitao Huang <haitao.huang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Cc: Sean Christopherson <seanjc-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
---
 arch/x86/kernel/cpu/sgx/main.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index f3a3ed894616..cd5e5517866a 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -17,6 +17,10 @@
 #include "driver.h"
 #include "encl.h"
 #include "encls.h"
+/**
+ * Maximum number of pages to scan for reclaiming.
+ */
+#define SGX_NR_TO_SCAN_MAX	32
 
 struct sgx_epc_section sgx_epc_sections[SGX_MAX_EPC_SECTIONS];
 static int sgx_nr_epc_sections;
@@ -279,7 +283,10 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
 	mutex_unlock(&encl->lock);
 }
 
-/*
+/**
+ * sgx_reclaim_pages() - Reclaim EPC pages from the consumers
+ * @nr_to_scan:		 Number of EPC pages to scan for reclaim
+ *
  * Take a fixed number of pages from the head of the active page pool and
  * reclaim them to the enclave's private shmem files. Skip the pages, which have
  * been accessed since the last scan. Move those pages to the tail of active
@@ -292,9 +299,9 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
  * problematic as it would increase the lock contention too much, which would
  * halt forward progress.
  */
-static void sgx_reclaim_pages(void)
+static void sgx_reclaim_pages(int nr_to_scan)
 {
-	struct sgx_backing backing[SGX_NR_TO_SCAN];
+	struct sgx_backing backing[SGX_NR_TO_SCAN_MAX];
 	struct sgx_epc_page *epc_page, *tmp;
 	struct sgx_encl_page *encl_page;
 	pgoff_t page_index;
@@ -332,7 +339,7 @@ static void sgx_reclaim_pages(void)
 	list_for_each_entry_safe(epc_page, tmp, &iso, list) {
 		encl_page = epc_page->encl_page;
 
-		if (!sgx_reclaimer_age(epc_page))
+		if (i == SGX_NR_TO_SCAN_MAX || !sgx_reclaimer_age(epc_page))
 			goto skip;
 
 		page_index = PFN_DOWN(encl_page->desc - encl_page->encl->base);
@@ -387,7 +394,7 @@ static bool sgx_should_reclaim(unsigned long watermark)
 void sgx_reclaim_direct(void)
 {
 	if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
-		sgx_reclaim_pages();
+		sgx_reclaim_pages(SGX_NR_TO_SCAN);
 }
 
 static int ksgxd(void *p)
@@ -410,7 +417,7 @@ static int ksgxd(void *p)
 				     sgx_should_reclaim(SGX_NR_HIGH_PAGES));
 
 		if (sgx_should_reclaim(SGX_NR_HIGH_PAGES))
-			sgx_reclaim_pages();
+			sgx_reclaim_pages(SGX_NR_TO_SCAN);
 
 		cond_resched();
 	}
@@ -582,7 +589,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
 			break;
 		}
 
-		sgx_reclaim_pages();
+		sgx_reclaim_pages(SGX_NR_TO_SCAN);
 		cond_resched();
 	}
 
-- 
2.25.1


  parent reply	other threads:[~2023-07-12 23:03 UTC|newest]

Thread overview: 102+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-07-12 23:01 [PATCH v3 00/28] Add Cgroup support for SGX EPC memory Haitao Huang
2023-07-12 23:01 ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 01/28] x86/sgx: Store struct sgx_encl when allocating new VA pages Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-17 11:14   ` Jarkko Sakkinen
2023-07-17 11:14     ` Jarkko Sakkinen
2023-07-12 23:01 ` [PATCH v3 02/28] x86/sgx: Add EPC page flags to identify owner type Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-17 12:41   ` Jarkko Sakkinen
2023-07-17 12:41     ` Jarkko Sakkinen
2023-07-17 12:43     ` Jarkko Sakkinen
2023-07-17 12:43       ` Jarkko Sakkinen
2023-07-12 23:01 ` [PATCH v3 03/28] x86/sgx: Add 'struct sgx_epc_lru_lists' to encapsulate lru list(s) Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-17 12:45   ` Jarkko Sakkinen
2023-07-17 12:45     ` Jarkko Sakkinen
2023-07-17 13:23     ` Haitao Huang
2023-07-17 13:23       ` Haitao Huang
2023-07-17 14:39       ` Jarkko Sakkinen
2023-07-17 14:39         ` Jarkko Sakkinen
2023-07-24 10:04       ` Huang, Kai
2023-07-24 10:04         ` Huang, Kai
2023-07-24 14:55         ` Haitao Huang
2023-07-24 14:55           ` Haitao Huang
2023-07-24 23:31           ` Huang, Kai
2023-07-31 20:35             ` Haitao Huang
2023-07-31 20:35               ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 04/28] x86/sgx: Use sgx_epc_lru_lists for existing active page list Haitao Huang
2023-07-17 12:47   ` Jarkko Sakkinen
2023-07-17 12:47     ` Jarkko Sakkinen
2023-07-31 20:43     ` Haitao Huang
2023-07-31 20:43       ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 05/28] x86/sgx: Store reclaimable epc pages in sgx_epc_lru_lists Haitao Huang
2023-07-12 23:01 ` [PATCH v3 06/28] x86/sgx: store unreclaimable EPC " Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 07/28] x86/sgx: Introduce EPC page states Haitao Huang
2023-07-12 23:01 ` [PATCH v3 08/28] x86/sgx: Introduce RECLAIM_IN_PROGRESS state Haitao Huang
2023-07-12 23:01 ` [PATCH v3 09/28] x86/sgx: Use a list to track to-be-reclaimed pages Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-12 23:01 ` Haitao Huang [this message]
2023-07-12 23:01   ` [PATCH v3 10/28] x86/sgx: Allow reclaiming up to 32 pages, but scan 16 by default Haitao Huang
2023-07-12 23:01 ` [PATCH v3 11/28] x85/sgx: Return the number of EPC pages that were successfully reclaimed Haitao Huang
2023-07-29 12:47   ` Pavel Machek
2023-07-31 11:10     ` Jarkko Sakkinen
2023-07-31 11:10       ` Jarkko Sakkinen
2023-07-12 23:01 ` [PATCH v3 12/28] x86/sgx: Add option to ignore age of page during EPC reclaim Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 13/28] x86/sgx: Prepare for multiple LRUs Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 14/28] x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 15/28] x86/sgx: Add helper to grab pages from an arbitrary EPC LRU Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 16/28] x86/sgx: Add EPC OOM path to forcefully reclaim EPC Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 17/28] x86/sgx: fix a NULL pointer Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-17 12:48   ` Jarkko Sakkinen
2023-07-17 12:48     ` Jarkko Sakkinen
2023-07-17 12:49     ` Jarkko Sakkinen
2023-07-17 12:49       ` Jarkko Sakkinen
2023-07-17 13:14       ` Haitao Huang
2023-07-17 14:33         ` Jarkko Sakkinen
2023-07-17 14:33           ` Jarkko Sakkinen
2023-07-17 15:49     ` Dave Hansen
2023-07-17 15:49       ` Dave Hansen
2023-07-17 18:49       ` Haitao Huang
2023-07-17 18:52       ` Jarkko Sakkinen
2023-07-17 18:52         ` Jarkko Sakkinen
2023-07-12 23:01 ` [PATCH v3 18/28] cgroup/misc: Fix an overflow Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-17 13:15   ` Jarkko Sakkinen
2023-07-17 13:15     ` Jarkko Sakkinen
2023-07-12 23:01 ` [PATCH v3 19/28] cgroup/misc: Add per resource callbacks for CSS events Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-17 13:16   ` Jarkko Sakkinen
2023-07-17 13:16     ` Jarkko Sakkinen
2023-07-12 23:01 ` [PATCH v3 20/28] cgroup/misc: Add SGX EPC resource type and export APIs for SGX driver Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-12 23:01 ` [PATCH v3 21/28] x86/sgx: Limit process EPC usage with misc cgroup controller Haitao Huang
2023-07-12 23:01   ` Haitao Huang
2023-07-13  0:03   ` Randy Dunlap
2023-07-13  0:03     ` Randy Dunlap
2023-08-17 15:12   ` Mikko Ylinen
2023-07-12 23:01 ` [PATCH v3 22/28] Docs/x86/sgx: Add description for cgroup support Haitao Huang
2023-07-13  0:10   ` Randy Dunlap
2023-07-14 20:01     ` Haitao Huang
2023-07-14 20:26   ` Haitao Huang
2023-08-17 15:18   ` Mikko Ylinen
2023-08-17 15:18     ` Mikko Ylinen
2023-07-12 23:01 ` [PATCH v3 23/28] selftests/sgx: Retry the ioctl()'s returned with EAGAIN Haitao Huang
2023-07-12 23:01 ` [PATCH v3 24/28] selftests/sgx: Move ENCL_HEAP_SIZE_DEFAULT to main.c Haitao Huang
2023-07-12 23:01 ` [PATCH v3 25/28] selftests/sgx: Use encl->encl_size in sigstruct.c Haitao Huang
2023-07-12 23:02 ` [PATCH v3 26/28] selftests/sgx: Include the dynamic heap size to the ELRANGE calculation Haitao Huang
2023-07-12 23:02 ` [PATCH v3 27/28] selftests/sgx: Add SGX selftest augment_via_eaccept_long Haitao Huang
2023-07-12 23:02 ` [PATCH v3 28/28] selftests/sgx: Add scripts for epc cgroup testing Haitao Huang
2023-07-17 11:02 ` [PATCH v3 00/28] Add Cgroup support for SGX EPC memory Jarkko Sakkinen
2023-07-17 11:02   ` Jarkko Sakkinen
2023-07-24 19:09 ` Sohil Mehta
2023-07-24 19:09   ` Sohil Mehta
2023-07-25 17:16   ` Haitao Huang
2023-08-17 15:04 ` Mikko Ylinen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230712230202.47929-11-haitao.huang@linux.intel.com \
    --to=haitao.huang@linux.intel.com \
    --cc=bp@alien8.de \
    --cc=cgroups@vger.kernel.org \
    --cc=dave.hansen@linux.intel.com \
    --cc=hpa@zytor.com \
    --cc=jarkko@kernel.org \
    --cc=kai.huang@intel.com \
    --cc=kristen@linux.intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-sgx@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=reinette.chatre@intel.com \
    --cc=sean.j.christopherson@intel.com \
    --cc=seanjc@google.com \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    --cc=x86@kernel.org \
    --cc=zhiquan1.li@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.