From: "Fontenot, Nathan" <ndfont@gmail.com>
To: David Hildenbrand <david@redhat.com>, linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, Michal Hocko <mhocko@kernel.org>,
Michal Hocko <mhocko@suse.com>,
Dan Williams <dan.j.williams@intel.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
powerpc-utils-devel@googlegroups.com, util-linux@vger.kernel.org,
Badari Pulavarty <pbadari@us.ibm.com>,
Nathan Fontenot <nfont@linux.vnet.ibm.com>,
Robert Jennings <rcj@linux.vnet.ibm.com>,
Heiko Carstens <heiko.carstens@de.ibm.com>,
Karel Zak <kzak@redhat.com>
Subject: Re: [PATCH v1] drivers/base/memory.c: indicate all memory blocks as removable
Date: Fri, 31 Jan 2020 07:41:02 -0600 [thread overview]
Message-ID: <5bc61566-c701-ebc1-5ab8-0d411e9ac705@gmail.com> (raw)
In-Reply-To: <20200128093542.6908-1-david@redhat.com>
On 1/28/2020 3:35 AM, David Hildenbrand wrote:
> We see multiple issues with the implementation/interface to compute
> whether a memory block can be offlined (exposed via
> /sys/devices/system/memory/memoryX/removable) and would like to simplify
> it (remove the implementation).
>
> 1. It runs basically lockless. While this might be good for performance,
> we see possible races with memory offlining that will require at least
> some sort of locking to fix.
>
> 2. Nowadays, more false positives are possible. No arch-specific checks
> are performed that validate if memory offlining will not be denied
> right away (and such check will require locking). For example, arm64
> won't allow to offline any memory block that was added during boot -
> which will imply a very high error rate. Other archs have other
> constraints.
>
> 3. The interface is inherently racy. E.g., if a memory block is
> detected to be removable (and was not a false positive at that time),
> there is still no guarantee that offlining will actually succeed. So
> any caller already has to deal with false positives.
>
> 4. It is unclear which performance benefit this interface actually
> provides. The introducing commit 5c755e9fd813 ("memory-hotplug: add
> sysfs removable attribute for hotplug memory remove") mentioned
> "A user-level agent must be able to identify which sections of
> memory are likely to be removable before attempting the
> potentially expensive operation."
> However, no actual performance comparison was included.
>
> Known users:
> - lsmem: Will group memory blocks based on the "removable" property. [1]
> - chmem: Indirect user. It has a RANGE mode where one can specify
> removable ranges identified via lsmem to be offlined. However, it
> also has a "SIZE" mode, which allows a sysadmin to skip the manual
> "identify removable blocks" step. [2]
> - powerpc-utils: Uses the "removable" attribute to skip some memory
> blocks right away when trying to find some to
> offline+remove. However, with ballooning enabled, it
> already skips this information completely (because it
> once resulted in many false negatives). Therefore, the
> implementation can deal with false positives properly
> already. [3]
>
> According to Nathan Fontenot, DLPAR on powerpc is nowadays no longer
> driven from userspace via the drmgr command (powerpc-utils). Nowadays
> it's managed in the kernel - including onlining/offlining of memory
> blocks - triggered by drmgr writing to /sys/kernel/dlpar. So the
> affected legacy userspace handling is only active on old kernels. Only very
> old versions of drmgr on a new kernel (unlikely) might execute slower -
> totally acceptable.
>
> With CONFIG_MEMORY_HOTREMOVE, always indicating "removable" should not
> break any user space tool. We implement a very bad heuristic now. Without
> CONFIG_MEMORY_HOTREMOVE we cannot offline anything, so report
> "not removable" as before.
>
> Original discussion can be found in [4] ("[PATCH RFC v1] mm:
> is_mem_section_removable() overhaul").
>
> Other users of is_mem_section_removable() will be removed next, so that
> we can remove is_mem_section_removable() completely.
>
> [1] http://man7.org/linux/man-pages/man1/lsmem.1.html
> [2] http://man7.org/linux/man-pages/man8/chmem.8.html
> [3] https://github.com/ibm-power-utilities/powerpc-utils
> [4] https://lkml.kernel.org/r/20200117105759.27905-1-david@redhat.com
>
> Suggested-by: Michal Hocko <mhocko@kernel.org>
> Acked-by: Michal Hocko <mhocko@suse.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: "Rafael J. Wysocki" <rafael@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: powerpc-utils-devel@googlegroups.com
> Cc: util-linux@vger.kernel.org
> Cc: Badari Pulavarty <pbadari@us.ibm.com>
> Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> Cc: Robert Jennings <rcj@linux.vnet.ibm.com>
> Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
> Cc: Karel Zak <kzak@redhat.com>
> Signed-off-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Nathan Fontenot <ndfont@gmail.com>
> ---
>
> RFC -> v1:
> - Use IS_ENABLED() instead of ifdefs
> - Add information from Nathan (thanks!)
>
> ---
> drivers/base/memory.c | 23 +++--------------------
> 1 file changed, 3 insertions(+), 20 deletions(-)
>
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 6503f5d0b749..9664be00a4de 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -105,30 +105,13 @@ static ssize_t phys_index_show(struct device *dev,
> }
>
> /*
> - * Show whether the memory block is likely to be offlineable (or is already
> - * offline). Once offline, the memory block could be removed. The return
> - * value does, however, not indicate that there is a way to remove the
> - * memory block.
> + * Legacy interface that we cannot remove. Always indicate "removable"
> + * with CONFIG_MEMORY_HOTREMOVE - bad heuristic.
> */
> static ssize_t removable_show(struct device *dev, struct device_attribute *attr,
> char *buf)
> {
> - struct memory_block *mem = to_memory_block(dev);
> - unsigned long pfn;
> - int ret = 1, i;
> -
> - if (mem->state != MEM_ONLINE)
> - goto out;
> -
> - for (i = 0; i < sections_per_block; i++) {
> - if (!present_section_nr(mem->start_section_nr + i))
> - continue;
> - pfn = section_nr_to_pfn(mem->start_section_nr + i);
> - ret &= is_mem_section_removable(pfn, PAGES_PER_SECTION);
> - }
> -
> -out:
> - return sprintf(buf, "%d\n", ret);
> + return sprintf(buf, "%d\n", (int)IS_ENABLED(CONFIG_MEMORY_HOTREMOVE));
> }
>
> /*
>
next prev parent reply other threads:[~2020-01-31 13:41 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-28 9:35 [PATCH v1] drivers/base/memory.c: indicate all memory blocks as removable David Hildenbrand
2020-01-31 13:41 ` Fontenot, Nathan [this message]
2020-03-27 6:24 ` Dan Williams
2020-03-27 7:47 ` Michal Hocko
2020-03-27 9:00 ` David Hildenbrand
2020-03-27 16:28 ` Dan Williams
2020-03-27 16:50 ` David Hildenbrand
2020-03-27 22:13 ` Dan Williams
2020-03-27 22:42 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5bc61566-c701-ebc1-5ab8-0d411e9ac705@gmail.com \
--to=ndfont@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=dan.j.williams@intel.com \
--cc=david@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=heiko.carstens@de.ibm.com \
--cc=kzak@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=mhocko@suse.com \
--cc=nfont@linux.vnet.ibm.com \
--cc=pbadari@us.ibm.com \
--cc=powerpc-utils-devel@googlegroups.com \
--cc=rafael@kernel.org \
--cc=rcj@linux.vnet.ibm.com \
--cc=util-linux@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).