From: Dave Hansen <dave.hansen@intel.com>
To: Tejun Heo <tj@kernel.org>,
Kristen Carlson Accardi <kristen@linux.intel.com>
Cc: linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org,
cgroups@vger.kernel.org, Johannes Weiner <hannes@cmpxchg.org>,
Michal Hocko <mhocko@kernel.org>,
Roman Gushchin <roman.gushchin@linux.dev>,
Shakeel Butt <shakeelb@google.com>,
Muchun Song <songmuchun@bytedance.com>
Subject: Re: [RFC PATCH 00/20] Add Cgroup support for SGX EPC memory
Date: Thu, 22 Sep 2022 14:03:52 -0700 [thread overview]
Message-ID: <7ff6d114-a6cc-e3c5-5edb-8ac0e527d8a9@intel.com> (raw)
In-Reply-To: <YyyykUJQtYbPVctn@slm.duckdns.org>
On 9/22/22 12:08, Tejun Heo wrote:
> Can you please give more concrete examples? I'd love to hear how the SGX EPC
> memory is typically used in what amounts and what's the performance
> implications when they get reclaimed and so on. ie. Please describe a
> realistic usage scenario of contention with sufficient details on how the
> system is set up, what the applications are using the SGX EPC memory for and
> how much, how the contention on memory affects the users and so on.
One wrinkle is that the apps that use SGX EPC memory are *normal* apps.
There are frameworks that some folks are very excited about that allow
you to run mostly unmodified app stacks inside SGX. For example:
https://github.com/gramineproject/graphene
In fact, Gramine users are the troublesome ones for overcommit. Most
explicitly-written SGX applications are quite austere in their SGX
memory use; they're probably never going to see overcommit. These
Gramine-wrapped apps are (relative) pigs. They've been the ones finding
bugs in the existing SGX overcommit code.
So, where does all the SGX memory go? It's the usual suspects:
memcached and redis. ;)
WARNING: multiple messages have this Message-ID (diff)
From: Dave Hansen <dave.hansen-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
To: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
Kristen Carlson Accardi
<kristen-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-sgx-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
Roman Gushchin
<roman.gushchin-fxUVXftIFDnyG1zEObXtfA@public.gmane.org>,
Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
Muchun Song <songmuchun-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org>
Subject: Re: [RFC PATCH 00/20] Add Cgroup support for SGX EPC memory
Date: Thu, 22 Sep 2022 14:03:52 -0700 [thread overview]
Message-ID: <7ff6d114-a6cc-e3c5-5edb-8ac0e527d8a9@intel.com> (raw)
In-Reply-To: <YyyykUJQtYbPVctn-NiLfg/pYEd1N0TnZuCh8vA@public.gmane.org>
On 9/22/22 12:08, Tejun Heo wrote:
> Can you please give more concrete examples? I'd love to hear how the SGX EPC
> memory is typically used in what amounts and what's the performance
> implications when they get reclaimed and so on. ie. Please describe a
> realistic usage scenario of contention with sufficient details on how the
> system is set up, what the applications are using the SGX EPC memory for and
> how much, how the contention on memory affects the users and so on.
One wrinkle is that the apps that use SGX EPC memory are *normal* apps.
There are frameworks that some folks are very excited about that allow
you to run mostly unmodified app stacks inside SGX. For example:
https://github.com/gramineproject/graphene
In fact, Gramine users are the troublesome ones for overcommit. Most
explicitly-written SGX applications are quite austere in their SGX
memory use; they're probably never going to see overcommit. These
Gramine-wrapped apps are (relative) pigs. They've been the ones finding
bugs in the existing SGX overcommit code.
So, where does all the SGX memory go? It's the usual suspects:
memcached and redis. ;)
next prev parent reply other threads:[~2022-09-22 21:04 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-22 17:10 [RFC PATCH 00/20] Add Cgroup support for SGX EPC memory Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 01/20] x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages() Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-23 12:32 ` Jarkko Sakkinen
2022-09-23 12:32 ` Jarkko Sakkinen
2022-09-23 12:35 ` Jarkko Sakkinen
2022-09-23 12:35 ` Jarkko Sakkinen
2022-09-23 12:37 ` Jarkko Sakkinen
2022-09-23 12:37 ` Jarkko Sakkinen
2022-09-22 17:10 ` [RFC PATCH 02/20] x86/sgx: Store EPC page owner as a 'void *' to handle multiple users Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 18:54 ` Dave Hansen
2022-09-22 18:54 ` Dave Hansen
2022-09-23 12:49 ` Jarkko Sakkinen
2022-09-23 12:49 ` Jarkko Sakkinen
2022-09-22 17:10 ` [RFC PATCH 03/20] x86/sgx: Track owning enclave in VA EPC pages Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 18:55 ` Dave Hansen
2022-09-22 18:55 ` Dave Hansen
2022-09-22 20:04 ` Kristen Carlson Accardi
2022-09-22 20:04 ` Kristen Carlson Accardi
2022-09-22 21:39 ` Dave Hansen
2022-09-22 21:39 ` Dave Hansen
2022-09-23 12:52 ` Jarkko Sakkinen
2022-09-23 12:52 ` Jarkko Sakkinen
2022-09-22 17:10 ` [RFC PATCH 04/20] x86/sgx: Add 'struct sgx_epc_lru' to encapsulate lru list(s) Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-23 13:20 ` Jarkko Sakkinen
2022-09-23 13:20 ` Jarkko Sakkinen
2022-09-29 23:04 ` Kristen Carlson Accardi
2022-09-29 23:04 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 05/20] x86/sgx: Introduce unreclaimable EPC page lists Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-23 13:29 ` Jarkko Sakkinen
2022-09-23 13:29 ` Jarkko Sakkinen
2022-09-22 17:10 ` [RFC PATCH 06/20] x86/sgx: Introduce RECLAIM_IN_PROGRESS flag for EPC pages Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 07/20] x86/sgx: Use a list to track to-be-reclaimed pages during reclaim Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 08/20] x86/sgx: Add EPC page flags to identify type of page Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 09/20] x86/sgx: Allow reclaiming up to 32 pages, but scan 16 by default Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 10/20] x86/sgx: Return the number of EPC pages that were successfully reclaimed Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 11/20] x86/sgx: Add option to ignore age of page during EPC reclaim Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 12/20] x86/sgx: Add helper to retrieve SGX EPC LRU given an EPC page Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 13/20] x86/sgx: Prepare for multiple LRUs Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 14/20] x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 15/20] x86/sgx: Add helper to grab pages from an arbitrary EPC LRU Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 16/20] x86/sgx: Add EPC OOM path to forcefully reclaim EPC Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 17/20] cgroup, x86/sgx: Add SGX EPC cgroup controller Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 18/20] x86/sgx: Enable EPC cgroup controller in SGX core Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 19/20] x86/sgx: Add stats and events interfaces to EPC cgroup controller Kristen Carlson Accardi
2022-09-22 17:10 ` Kristen Carlson Accardi
2022-09-22 17:10 ` [RFC PATCH 20/20] docs, cgroup, x86/sgx: Add SGX EPC cgroup controller documentation Kristen Carlson Accardi
2022-09-22 17:41 ` [RFC PATCH 00/20] Add Cgroup support for SGX EPC memory Tejun Heo
2022-09-22 17:41 ` Tejun Heo
2022-09-22 18:59 ` Kristen Carlson Accardi
2022-09-22 18:59 ` Kristen Carlson Accardi
2022-09-22 19:08 ` Tejun Heo
2022-09-22 19:08 ` Tejun Heo
2022-09-22 21:03 ` Dave Hansen [this message]
2022-09-22 21:03 ` Dave Hansen
2022-09-24 0:09 ` Tejun Heo
2022-09-24 0:09 ` Tejun Heo
2022-09-26 18:30 ` Kristen Carlson Accardi
2022-09-26 18:30 ` Kristen Carlson Accardi
2022-10-07 16:39 ` Kristen Carlson Accardi
2022-10-07 16:39 ` Kristen Carlson Accardi
2022-10-07 16:42 ` Tejun Heo
2022-10-07 16:42 ` Tejun Heo
2022-10-07 16:46 ` Kristen Carlson Accardi
2022-10-07 16:46 ` Kristen Carlson Accardi
2022-09-23 12:24 ` Jarkko Sakkinen
2022-09-23 12:24 ` Jarkko Sakkinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7ff6d114-a6cc-e3c5-5edb-8ac0e527d8a9@intel.com \
--to=dave.hansen@intel.com \
--cc=cgroups@vger.kernel.org \
--cc=hannes@cmpxchg.org \
--cc=kristen@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=mhocko@kernel.org \
--cc=roman.gushchin@linux.dev \
--cc=shakeelb@google.com \
--cc=songmuchun@bytedance.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.