From: "Huang, Kai" <kai.huang@intel.com>
To: Haitao Huang <haitao.huang@linux.intel.com>,
"Mehta, Sohil" <sohil.mehta@intel.com>,
"mingo@redhat.com" <mingo@redhat.com>,
"jarkko@kernel.org" <jarkko@kernel.org>,
"x86@kernel.org" <x86@kernel.org>,
"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
"cgroups@vger.kernel.org" <cgroups@vger.kernel.org>,
"hpa@zytor.com" <hpa@zytor.com>,
"tim.c.chen@linux.intel.com" <tim.c.chen@linux.intel.com>,
"linux-sgx@vger.kernel.org" <linux-sgx@vger.kernel.org>,
"mkoutny@suse.com" <mkoutny@suse.com>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"tj@kernel.org" <tj@kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"bp@alien8.de" <bp@alien8.de>
Cc: "mikko.ylinen@linux.intel.com" <mikko.ylinen@linux.intel.com>,
"seanjc@google.com" <seanjc@google.com>,
"anakrish@microsoft.com" <anakrish@microsoft.com>,
"Zhang, Bo" <zhanb@microsoft.com>,
"kristen@linux.intel.com" <kristen@linux.intel.com>,
"yangjie@microsoft.com" <yangjie@microsoft.com>,
"Li, Zhiquan1" <zhiquan1.li@intel.com>,
"chrisyan@microsoft.com" <chrisyan@microsoft.com>
Subject: Re: [PATCH v9 08/15] x86/sgx: Implement EPC reclamation flows for cgroup
Date: Fri, 23 Feb 2024 11:31:18 +1300 [thread overview]
Message-ID: <f242cfb2-e8ab-49f3-9b8e-1236dd361b64@intel.com> (raw)
In-Reply-To: <op.2jjzsnz0wjvjmi@hhuan26-mobl.amr.corp.intel.com>
On 23/02/2024 6:20 am, Haitao Huang wrote:
> On Wed, 21 Feb 2024 05:00:27 -0600, Huang, Kai <kai.huang@intel.com> wrote:
>
>> On Wed, 2024-02-21 at 00:44 -0600, Haitao Huang wrote:
>>> [...]
>>> >
>>> > Here the @nr_to_scan is reduced by the number of pages that are
>>> > isolated, but
>>> > not actually reclaimed (which is reflected by @cnt).
>>> >
>>> > IIUC, looks you want to make this function do "each cycle" as what you
>>> > mentioned
>>> > in the v8 [1]:
>>> >
>>> > I tested with that approach and found we can only target number of
>>> > pages
>>> > attempted to reclaim not pages actually reclaimed due to the
>>> > uncertainty
>>> > of how long it takes to reclaim pages. Besides targeting number of
>>> > scanned pages for each cycle is also what the ksgxd does.
>>> >
>>> > If we target actual number of pages, sometimes it just takes
>>> too long.
>>> > I
>>> > saw more timeouts with the default time limit when running
>>> parallel
>>> > selftests.
>>> >
>>> > I am not sure what does "sometimes it just takes too long" mean, but
>>> > what I am
>>> > thinking is you are trying to do some perfect but yet complicated code
>>> > here.
>>>
>>> I think what I observed was that the try_charge() would block too long
>>> before getting chance of schedule() to yield, causing more timeouts than
>>> necessary.
>>> I'll do some re-test to be sure.
>>
>> Looks this is a valid information that can be used to justify whatever
>> you are
>> implementing in the EPC cgroup reclaiming function(s).
>>
> I'll add some comments. Was assuming this is just following the old
> design as ksgxd.
> There were some comments at the beginning of
> sgx_epc_cgrooup_reclaim_page().
> /*
> * Attempting to reclaim only a few pages will often fail and is
> * inefficient, while reclaiming a huge number of pages can
> result in
> * soft lockups due to holding various locks for an extended
> duration.
> */
> unsigned int nr_to_scan = SGX_NR_TO_SCAN;
>
> I think it can be improved to emphasize we only "attempt" to finish
> scanning fixed number of pages for reclamation, not enforce number of
> pages successfully reclaimed.
Not sure need to be this comment, but at somewhere just state you are
trying to follow the ksgxd() (the current sgx_reclaim_pages()), but
trying to do it "_across_ given cgroup and all the descendants".
That's the reason you made @nr_to_scan as a pointer.
And also some text to explain why to follow ksgxd() -- not wanting to
block longer due to loop over descendants etc -- so we can focus on
discussing whether such justification is reasonable.
next prev parent reply other threads:[~2024-02-22 22:31 UTC|newest]
Thread overview: 106+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-05 21:06 [PATCH v9 00/15] Add Cgroup support for SGX EPC memory Haitao Huang
2024-02-05 21:06 ` [PATCH v9 01/15] cgroup/misc: Add per resource callbacks for CSS events Haitao Huang
2024-02-05 21:06 ` [PATCH v9 02/15] cgroup/misc: Export APIs for SGX driver Haitao Huang
2024-02-05 21:06 ` [PATCH v9 03/15] cgroup/misc: Add SGX EPC resource type Haitao Huang
2024-02-05 21:06 ` [PATCH v9 04/15] x86/sgx: Implement basic EPC misc cgroup functionality Haitao Huang
2024-02-19 12:47 ` Huang, Kai
2024-02-26 18:25 ` Michal Koutný
2024-02-27 21:35 ` Haitao Huang
2024-03-09 21:10 ` Haitao Huang
2024-02-05 21:06 ` [PATCH v9 05/15] x86/sgx: Add sgx_epc_lru_list to encapsulate LRU list Haitao Huang
2024-02-05 21:06 ` [PATCH v9 06/15] x86/sgx: Abstract tracking reclaimable pages in LRU Haitao Huang
2024-02-05 21:06 ` [PATCH v9 07/15] x86/sgx: Expose sgx_reclaim_pages() for cgroup Haitao Huang
2024-02-20 9:26 ` Huang, Kai
2024-02-05 21:06 ` [PATCH v9 08/15] x86/sgx: Implement EPC reclamation flows " Haitao Huang
2024-02-12 19:35 ` Jarkko Sakkinen
2024-02-20 9:52 ` Huang, Kai
2024-02-20 13:18 ` Michal Koutný
2024-02-20 20:09 ` Huang, Kai
2024-02-21 6:23 ` Haitao Huang
2024-02-21 10:48 ` Huang, Kai
2024-02-22 20:12 ` Haitao Huang
2024-02-22 22:24 ` Huang, Kai
2024-03-28 0:24 ` Haitao Huang
2024-02-21 6:44 ` Haitao Huang
2024-02-21 11:00 ` Huang, Kai
2024-02-22 17:20 ` Haitao Huang
2024-02-22 22:31 ` Huang, Kai [this message]
2024-02-22 18:09 ` Haitao Huang
2024-02-05 21:06 ` [PATCH v9 09/15] x86/sgx: Charge mem_cgroup for per-cgroup reclamation Haitao Huang
2024-02-12 19:46 ` Jarkko Sakkinen
2024-02-13 3:21 ` Haitao Huang
2024-02-15 23:43 ` Dave Hansen
2024-02-16 6:07 ` Haitao Huang
2024-02-16 15:15 ` Dave Hansen
2024-02-16 21:38 ` Haitao Huang
2024-02-16 21:55 ` Dave Hansen
2024-02-16 23:33 ` Haitao Huang
2024-02-05 21:06 ` [PATCH v9 10/15] x86/sgx: Add EPC reclamation in cgroup try_charge() Haitao Huang
2024-02-12 19:55 ` Jarkko Sakkinen
2024-02-12 23:15 ` Haitao Huang
2024-02-14 1:52 ` Jarkko Sakkinen
2024-02-19 15:12 ` Haitao Huang
2024-02-19 20:20 ` Jarkko Sakkinen
2024-02-19 15:39 ` [RFC PATCH] x86/sgx: Remove 'reclaim' boolean parameters Haitao Huang
2024-02-19 15:56 ` Dave Hansen
2024-02-19 20:42 ` Jarkko Sakkinen
2024-02-19 22:25 ` Haitao Huang
2024-02-19 22:43 ` Jarkko Sakkinen
2024-02-19 20:23 ` Jarkko Sakkinen
2024-02-21 11:06 ` [PATCH v9 10/15] x86/sgx: Add EPC reclamation in cgroup try_charge() Huang, Kai
2024-02-22 17:09 ` Haitao Huang
2024-02-22 21:26 ` Huang, Kai
2024-02-22 22:57 ` Haitao Huang
2024-02-23 10:18 ` Huang, Kai
2024-02-23 17:00 ` Haitao Huang
2024-02-26 1:38 ` Huang, Kai
2024-02-26 4:03 ` Haitao Huang
2024-02-26 11:36 ` Huang, Kai
2024-02-26 14:04 ` Dave Hansen
2024-02-26 21:48 ` Haitao Huang
2024-02-26 21:56 ` Dave Hansen
2024-02-26 22:34 ` Huang, Kai
2024-02-26 22:38 ` Dave Hansen
2024-02-26 22:46 ` Huang, Kai
2024-02-27 20:41 ` Jarkko Sakkinen
2024-02-27 9:26 ` Michal Koutný
2024-02-26 21:18 ` Haitao Huang
2024-02-26 22:24 ` Huang, Kai
2024-02-26 22:31 ` Dave Hansen
2024-02-26 22:38 ` Huang, Kai
2024-02-05 21:06 ` [PATCH v9 11/15] x86/sgx: Abstract check for global reclaimable pages Haitao Huang
2024-02-12 19:56 ` Jarkko Sakkinen
2024-02-21 11:34 ` Huang, Kai
2024-02-05 21:06 ` [PATCH v9 12/15] x86/sgx: Expose sgx_epc_cgroup_reclaim_pages() for global reclaimer Haitao Huang
2024-02-12 19:58 ` Jarkko Sakkinen
2024-02-21 11:10 ` Huang, Kai
2024-02-22 16:35 ` Haitao Huang
2024-02-05 21:06 ` [PATCH v9 13/15] x86/sgx: Turn on per-cgroup EPC reclamation Haitao Huang
2024-02-21 11:23 ` Huang, Kai
2024-02-22 16:36 ` Haitao Huang
2024-02-22 22:44 ` Huang, Kai
2024-02-23 18:46 ` Haitao Huang
2024-02-05 21:06 ` [PATCH v9 14/15] Docs/x86/sgx: Add description for cgroup support Haitao Huang
2024-02-05 21:06 ` [PATCH v9 15/15] selftests/sgx: Add scripts for EPC cgroup testing Haitao Huang
2024-03-27 12:55 ` Jarkko Sakkinen
2024-03-27 16:56 ` Jarkko Sakkinen
2024-03-28 0:57 ` Haitao Huang
2024-03-28 3:05 ` Haitao Huang
2024-03-30 11:23 ` Jarkko Sakkinen
2024-03-30 11:26 ` Jarkko Sakkinen
2024-04-02 11:23 ` Michal Koutný
2024-04-02 11:58 ` Jarkko Sakkinen
2024-04-02 16:20 ` Haitao Huang
2024-04-02 17:40 ` Michal Koutný
2024-04-02 18:20 ` Haitao Huang
2024-04-03 16:46 ` Jarkko Sakkinen
2024-04-03 15:33 ` Jarkko Sakkinen
2024-04-02 15:42 ` Dave Hansen
2024-04-03 15:16 ` Jarkko Sakkinen
2024-03-28 3:54 ` Haitao Huang
2024-03-30 11:15 ` Jarkko Sakkinen
2024-03-30 15:32 ` Haitao Huang
2024-03-31 16:19 ` Jarkko Sakkinen
2024-03-31 17:35 ` Haitao Huang
2024-04-01 14:10 ` Jarkko Sakkinen
2024-02-08 8:43 ` [PATCH v9 00/15] Add Cgroup support for SGX EPC memory Mikko Ylinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f242cfb2-e8ab-49f3-9b8e-1236dd361b64@intel.com \
--to=kai.huang@intel.com \
--cc=anakrish@microsoft.com \
--cc=bp@alien8.de \
--cc=cgroups@vger.kernel.org \
--cc=chrisyan@microsoft.com \
--cc=dave.hansen@linux.intel.com \
--cc=haitao.huang@linux.intel.com \
--cc=hpa@zytor.com \
--cc=jarkko@kernel.org \
--cc=kristen@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=mikko.ylinen@linux.intel.com \
--cc=mingo@redhat.com \
--cc=mkoutny@suse.com \
--cc=seanjc@google.com \
--cc=sohil.mehta@intel.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=tj@kernel.org \
--cc=x86@kernel.org \
--cc=yangjie@microsoft.com \
--cc=zhanb@microsoft.com \
--cc=zhiquan1.li@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).