From: David Hildenbrand <david@redhat.com>
To: Michal Hocko <mhocko@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Dan Williams <dan.j.williams@intel.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
powerpc-utils-devel@googlegroups.com, util-linux@vger.kernel.org,
Badari Pulavarty <pbadari@us.ibm.com>,
Nathan Fontenot <nfont@linux.vnet.ibm.com>,
Robert Jennings <rcj@linux.vnet.ibm.com>,
Heiko Carstens <heiko.carstens@de.ibm.com>,
Karel Zak <kzak@redhat.com>
Subject: Re: [PATCH RFC] drivers/base/memory.c: indicate all memory blocks as removable
Date: Mon, 27 Jan 2020 14:34:15 +0100
Message-ID: <548fbb8f-9c05-fbf7-4413-b48eeb7e1c8e@redhat.com> (raw)
In-Reply-To: <20200127132950.GH1183@dhcp22.suse.cz>
On 27.01.20 14:29, Michal Hocko wrote:
> On Fri 24-01-20 16:53:36, David Hildenbrand wrote:
>> We see multiple issues with the implementation/interface to compute
>> whether a memory block can be offlined (exposed via
>> /sys/devices/system/memory/memoryX/removable) and would like to simplify
>> it (remove the implementation).
>>
>> 1. It runs basically lockless. While this might be good for performance,
>> we see possible races with memory offlining/unplug that will require
>> at least some sort of locking to fix.
>>
>> 2. Nowadays, more false positives are possible. No arch-specific checks
>> are performed that validate if memory offlining will not be denied
>> right away (and such check will require locking). For example, arm64
>> won't allow to offline any memory block that was added during boot -
>> which will imply a very high error rate. Other archs have other
>> constraints.
>>
>> 3. The interface is inherently racy. E.g., if a memory block is
>> detected to be removable (and was not a false positive at that time),
>> there is still no guarantee that offlining will actually succeed. So
>> any caller already has to deal with false positives.
>>
>> 4. It is unclear which performance benefit this interface actually
>> provides. The introducing commit 5c755e9fd813 ("memory-hotplug: add
>> sysfs removable attribute for hotplug memory remove") mentioned
>> "A user-level agent must be able to identify which sections of
>> memory are likely to be removable before attempting the
>> potentially expensive operation."
>> However, no actual performance comparison was included.
>>
>> Known users:
>> - lsmem: Will group memory blocks based on the "removable" property. [1]
>> - chmem: Indirect user. It has a RANGE mode where one can specify
>> removable ranges identified via lsmem to be offlined. However, it
>> also has a "SIZE" mode, which allows a sysadmin to skip the manual
>> "identify removable blocks" step. [2]
>> - powerpc-utils: Uses the "removable" attribute to skip some memory
>> blocks right away when trying to find some to
>> offline+remove. However, with ballooning enabled, it
>> already skips this information completely (because it
>> once resulted in many false negatives). Therefore, the
>> implementation can deal with false positives properly
>> already. [3]
>>
>> With CONFIG_MEMORY_HOTREMOVE, always indicating "removable" should not
>> break any user space tool. We implement a very bad heuristic now. (in
>> contrast: always returning "not removable" would at least affect
>> powerpc-utils)
>>
>> Without CONFIG_MEMORY_HOTREMOVE we cannot offline anything, so report
>> "not removable" as before.
>>
>> Original discussion can be found in [4] ("[PATCH RFC v1] mm:
>> is_mem_section_removable() overhaul").
>>
>> Other users of is_mem_section_removable() will be removed next, so that
>> we can remove is_mem_section_removable() completely.
>>
>> [1] http://man7.org/linux/man-pages/man1/lsmem.1.html
>> [2] http://man7.org/linux/man-pages/man8/chmem.8.html
>> [3] https://github.com/ibm-power-utilities/powerpc-utils
>> [4] https://lkml.kernel.org/r/20200117105759.27905-1-david@redhat.com
>>
>> Suggested-by: Michal Hocko <mhocko@kernel.org>
>> Cc: Dan Williams <dan.j.williams@intel.com>
>> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>> Cc: "Rafael J. Wysocki" <rafael@kernel.org>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: powerpc-utils-devel@googlegroups.com
>> Cc: util-linux@vger.kernel.org
>> Cc: Badari Pulavarty <pbadari@us.ibm.com>
>> Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com>
>> Cc: Robert Jennings <rcj@linux.vnet.ibm.com>
>> Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
>> Cc: Karel Zak <kzak@redhat.com>
>> Signed-off-by: David Hildenbrand <david@redhat.com>
>
> Please add information provided by Nathan.
> Acked-by: Michal Hocko <mhocko@suse.com>
>
> Minor nit below.
>
>> +#ifdef CONFIG_MEMORY_HOTREMOVE
>> + return sprintf(buf, "1\n");
>> +#else
>> + return sprintf(buf, "0\n");
>> +#endif
> int ret = IS_ENABLED(CONFIG_MEMORY_HOTREMOVE);
>
> return sprintf(buf, "%d\n", ret)
>
> would be slightly nicer than explicit ifdefs.
>
Indeed, thanks!
--
Thanks,
David / dhildenb
prev parent reply index
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-24 15:53 David Hildenbrand
2020-01-24 19:10 ` Fontenot, Nathan
2020-01-27 9:23 ` Michal Hocko
2020-01-27 9:33 ` David Hildenbrand
2020-01-27 13:25 ` Michal Hocko
2020-01-27 15:23 ` Fontenot, Nathan
2020-01-27 13:29 ` Michal Hocko
2020-01-27 13:34 ` David Hildenbrand [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=548fbb8f-9c05-fbf7-4413-b48eeb7e1c8e@redhat.com \
--to=david@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=dan.j.williams@intel.com \
--cc=gregkh@linuxfoundation.org \
--cc=heiko.carstens@de.ibm.com \
--cc=kzak@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=nfont@linux.vnet.ibm.com \
--cc=pbadari@us.ibm.com \
--cc=powerpc-utils-devel@googlegroups.com \
--cc=rafael@kernel.org \
--cc=rcj@linux.vnet.ibm.com \
--cc=util-linux@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Util-Linux Archive on lore.kernel.org
Archives are clonable:
git clone --mirror https://lore.kernel.org/util-linux/0 util-linux/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 util-linux util-linux/ https://lore.kernel.org/util-linux \
util-linux@vger.kernel.org
public-inbox-index util-linux
Example config snippet for mirrors
Newsgroup available over NNTP:
nntp://nntp.lore.kernel.org/org.kernel.vger.util-linux
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git