All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@kernel.org>
To: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Kate Stewart <kstewart@linuxfoundation.org>,
	Rich Felker <dalias@libc.org>,
	linux-ia64@vger.kernel.org, linux-sh@vger.kernel.org,
	Peter Zijlstra <peterz@infradead.org>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Balbir Singh <bsingharora@gmail.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Heiko Carstens <heiko.carstens@de.ibm.com>,
	linux-mm@kvack.org, Paul Mackerras <paulus@samba.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	Rashmica Gupta <rashmica.g@gmail.com>,
	Dan Williams <dan.j.williams@intel.com>,
	linux-s390@vger.kernel.org, Michael Neuling <mikey@neuling.org>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Michael Ellerman <mpe@ellerman.id.au>,
	David Hildenbrand <david@redhat.com>,
	linux-acpi@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
	xen-devel@lists.xenproject.org, Len
Subject: Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types
Date: Wed, 3 Oct 2018 15:44:44 +0200	[thread overview]
Message-ID: <20181003134444.GH4714@dhcp22.suse.cz> (raw)
In-Reply-To: <8736tndubn.fsf@vitty.brq.redhat.com>

On Wed 03-10-18 15:38:04, Vitaly Kuznetsov wrote:
> David Hildenbrand <david@redhat.com> writes:
> 
> > On 02/10/2018 15:47, Michal Hocko wrote:
> ...
> >> 
> >> Why do you need a generic hotplug rule in the first place? Why don't you
> >> simply provide different set of rules for different usecases? Let users
> >> decide which usecase they prefer rather than try to be clever which
> >> almost always hits weird corner cases.
> >> 
> >
> > Memory hotplug has to work as reliable as we can out of the box. Letting
> > the user make simple decisions like "oh, I am on hyper-V, I want to
> > online memory to the normal zone" does not feel right. But yes, we
> > should definitely allow to make modifications.
> 
> Last time I was thinking about the imperfectness of the auto-online
> solution we have and any other solution we're able to suggest an idea
> came to my mind - what if we add an eBPF attach point to the
> auto-onlining mechanism effecively offloading decision-making to
> userspace. We'll of couse need to provide all required data (e.g. how
> memory blocks are aligned with physical DIMMs as it makes no sense to
> online part of DIMM as normal and the rest as movable as it's going to
> be impossible to unplug such DIMM anyways).

And how does that differ from the notification mechanism we have? Just
by not relying on the process scheduling? If yes then this revolves
around the implementation detail that you care about time-to-hot-add
vs. time-to-online. And that is a solveable problem - just allocate
memmaps from the hot-added memory.

As David said some of the memory cannot be onlined without further steps
(e.g. when it is standby as David called it) and then I fail to see how
eBPF help in any way.
-- 
Michal Hocko
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org>
To: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: "David Hildenbrand" <david@redhat.com>,
	"Kate Stewart" <kstewart@linuxfoundation.org>,
	"Rich Felker" <dalias@libc.org>,
	linux-ia64@vger.kernel.org, linux-sh@vger.kernel.org,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Benjamin Herrenschmidt" <benh@kernel.crashing.org>,
	"Balbir Singh" <bsingharora@gmail.com>,
	"Dave Hansen" <dave.hansen@linux.intel.com>,
	"Heiko Carstens" <heiko.carstens@de.ibm.com>,
	linux-mm@kvack.org,
	"Pavel Tatashin" <pavel.tatashin@microsoft.com>,
	"Paul Mackerras" <paulus@samba.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	"Rashmica Gupta" <rashmica.g@gmail.com>,
	"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
	linux-s390@vger.kernel.org, "Michael Neuling" <mikey@neuling.org>,
	"Stephen Hemminger" <sthemmin@microsoft.com>,
	"Yoshinori Sato" <ysato@users.sourceforge.jp>,
	"Michael Ellerman" <mpe@ellerman.id.au>,
	linux-acpi@vger.kernel.org, "Ingo Molnar" <mingo@redhat.com>,
	xen-devel@lists.xenproject.org, "Rob Herring" <robh@kernel.org>,
	"Len Brown" <lenb@kernel.org>,
	"Fenghua Yu" <fenghua.yu@intel.com>,
	"Stephen Rothwell" <sfr@canb.auug.org.au>,
	"mike.travis@hpe.com" <mike.travis@hpe.com>,
	"Haiyang Zhang" <haiyangz@microsoft.com>,
	"Dan Williams" <dan.j.williams@intel.com>,
	"Jonathan Neuschäfer" <j.neuschaefer@gmx.net>,
	"Nicholas Piggin" <npiggin@gmail.com>,
	"Joe Perches" <joe@perches.com>,
	"Jérôme Glisse" <jglisse@redhat.com>,
	"Mike Rapoport" <rppt@linux.vnet.ibm.com>,
	"Borislav Petkov" <bp@alien8.de>,
	"Andy Lutomirski" <luto@kernel.org>,
	"Thomas Gleixner" <tglx@linutronix.de>,
	"Joonsoo Kim" <iamjoonsoo.kim@lge.com>,
	"Oscar Salvador" <osalvador@suse.de>,
	"Juergen Gross" <jgross@suse.com>,
	"Tony Luck" <tony.luck@intel.com>,
	"Mathieu Malaterre" <malat@debian.org>,
	"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	linux-kernel@vger.kernel.org,
	"Mauricio Faria de Oliveira" <mauricfo@linux.vnet.ibm.com>,
	"Philippe Ombredanne" <pombredanne@nexb.com>,
	"Martin Schwidefsky" <schwidefsky@de.ibm.com>,
	devel@linuxdriverproject.org,
	"Andrew Morton" <akpm@linux-foundation.org>,
	linuxppc-dev@lists.ozlabs.org,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Subject: Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types
Date: Wed, 3 Oct 2018 15:44:44 +0200	[thread overview]
Message-ID: <20181003134444.GH4714@dhcp22.suse.cz> (raw)
In-Reply-To: <8736tndubn.fsf@vitty.brq.redhat.com>

On Wed 03-10-18 15:38:04, Vitaly Kuznetsov wrote:
> David Hildenbrand <david@redhat.com> writes:
> 
> > On 02/10/2018 15:47, Michal Hocko wrote:
> ...
> >> 
> >> Why do you need a generic hotplug rule in the first place? Why don't you
> >> simply provide different set of rules for different usecases? Let users
> >> decide which usecase they prefer rather than try to be clever which
> >> almost always hits weird corner cases.
> >> 
> >
> > Memory hotplug has to work as reliable as we can out of the box. Letting
> > the user make simple decisions like "oh, I am on hyper-V, I want to
> > online memory to the normal zone" does not feel right. But yes, we
> > should definitely allow to make modifications.
> 
> Last time I was thinking about the imperfectness of the auto-online
> solution we have and any other solution we're able to suggest an idea
> came to my mind - what if we add an eBPF attach point to the
> auto-onlining mechanism effecively offloading decision-making to
> userspace. We'll of couse need to provide all required data (e.g. how
> memory blocks are aligned with physical DIMMs as it makes no sense to
> online part of DIMM as normal and the rest as movable as it's going to
> be impossible to unplug such DIMM anyways).

And how does that differ from the notification mechanism we have? Just
by not relying on the process scheduling? If yes then this revolves
around the implementation detail that you care about time-to-hot-add
vs. time-to-online. And that is a solveable problem - just allocate
memmaps from the hot-added memory.

As David said some of the memory cannot be onlined without further steps
(e.g. when it is standby as David called it) and then I fail to see how
eBPF help in any way.
-- 
Michal Hocko
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org>
To: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: "Kate Stewart" <kstewart@linuxfoundation.org>,
	"Rich Felker" <dalias@libc.org>,
	linux-ia64@vger.kernel.org, linux-sh@vger.kernel.org,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Dave Hansen" <dave.hansen@linux.intel.com>,
	"Heiko Carstens" <heiko.carstens@de.ibm.com>,
	linux-mm@kvack.org, "Paul Mackerras" <paulus@samba.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	"Stephen Rothwell" <sfr@canb.auug.org.au>,
	"Rashmica Gupta" <rashmica.g@gmail.com>,
	"Dan Williams" <dan.j.williams@intel.com>,
	linux-s390@vger.kernel.org, "Michael Neuling" <mikey@neuling.org>,
	"Stephen Hemminger" <sthemmin@microsoft.com>,
	"Yoshinori Sato" <ysato@users.sourceforge.jp>,
	"David Hildenbrand" <david@redhat.com>,
	linux-acpi@vger.kernel.org, "Ingo Molnar" <mingo@redhat.com>,
	xen-devel@lists.xenproject.org, "Len Brown" <lenb@kernel.org>,
	"Pavel Tatashin" <pavel.tatashin@microsoft.com>,
	"Rob Herring" <robh@kernel.org>,
	"mike.travis@hpe.com" <mike.travis@hpe.com>,
	"Haiyang Zhang" <haiyangz@microsoft.com>,
	"Jonathan Neuschäfer" <j.neuschaefer@gmx.net>,
	"Nicholas Piggin" <npiggin@gmail.com>,
	"Martin Schwidefsky" <schwidefsky@de.ibm.com>,
	"Jérôme Glisse" <jglisse@redhat.com>,
	"Mike Rapoport" <rppt@linux.vnet.ibm.com>,
	"Borislav Petkov" <bp@alien8.de>,
	"Andy Lutomirski" <luto@kernel.org>,
	"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
	"Andrew Morton" <akpm@linux-foundation.org>,
	"Oscar Salvador" <osalvador@suse.de>,
	"Juergen Gross" <jgross@suse.com>,
	"Tony Luck" <tony.luck@intel.com>,
	"Mathieu Malaterre" <malat@debian.org>,
	"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	linux-kernel@vger.kernel.org, "Fenghua Yu" <fenghua.yu@intel.com>,
	"Mauricio Faria de Oliveira" <mauricfo@linux.vnet.ibm.com>,
	"Thomas Gleixner" <tglx@linutronix.de>,
	"Philippe Ombredanne" <pombredanne@nexb.com>,
	"Joe Perches" <joe@perches.com>,
	devel@linuxdriverproject.org,
	"Joonsoo Kim" <iamjoonsoo.kim@lge.com>,
	linuxppc-dev@lists.ozlabs.org,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Subject: Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types
Date: Wed, 3 Oct 2018 15:44:44 +0200	[thread overview]
Message-ID: <20181003134444.GH4714@dhcp22.suse.cz> (raw)
In-Reply-To: <8736tndubn.fsf@vitty.brq.redhat.com>

On Wed 03-10-18 15:38:04, Vitaly Kuznetsov wrote:
> David Hildenbrand <david@redhat.com> writes:
> 
> > On 02/10/2018 15:47, Michal Hocko wrote:
> ...
> >> 
> >> Why do you need a generic hotplug rule in the first place? Why don't you
> >> simply provide different set of rules for different usecases? Let users
> >> decide which usecase they prefer rather than try to be clever which
> >> almost always hits weird corner cases.
> >> 
> >
> > Memory hotplug has to work as reliable as we can out of the box. Letting
> > the user make simple decisions like "oh, I am on hyper-V, I want to
> > online memory to the normal zone" does not feel right. But yes, we
> > should definitely allow to make modifications.
> 
> Last time I was thinking about the imperfectness of the auto-online
> solution we have and any other solution we're able to suggest an idea
> came to my mind - what if we add an eBPF attach point to the
> auto-onlining mechanism effecively offloading decision-making to
> userspace. We'll of couse need to provide all required data (e.g. how
> memory blocks are aligned with physical DIMMs as it makes no sense to
> online part of DIMM as normal and the rest as movable as it's going to
> be impossible to unplug such DIMM anyways).

And how does that differ from the notification mechanism we have? Just
by not relying on the process scheduling? If yes then this revolves
around the implementation detail that you care about time-to-hot-add
vs. time-to-online. And that is a solveable problem - just allocate
memmaps from the hot-added memory.

As David said some of the memory cannot be onlined without further steps
(e.g. when it is standby as David called it) and then I fail to see how
eBPF help in any way.
-- 
Michal Hocko
SUSE Labs

  parent reply	other threads:[~2018-10-03 13:44 UTC|newest]

Thread overview: 144+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-09-28 15:03 [PATCH RFC] mm/memory_hotplug: Introduce memory block types David Hildenbrand
2018-09-28 15:03 ` David Hildenbrand
2018-09-28 15:03 ` David Hildenbrand
2018-09-28 17:02 ` Dave Hansen
2018-09-28 17:02   ` Dave Hansen
2018-09-28 17:02   ` Dave Hansen
2018-10-01  9:13   ` David Hildenbrand
2018-10-01  9:13   ` David Hildenbrand
2018-10-01  9:13     ` David Hildenbrand
2018-10-01  9:13     ` David Hildenbrand
2018-10-01 16:24     ` Dave Hansen
2018-10-01 16:24     ` Dave Hansen
2018-10-01 16:24       ` Dave Hansen
2018-10-01 16:24       ` Dave Hansen
2018-10-04  7:48       ` David Hildenbrand
2018-10-04  7:48       ` David Hildenbrand
2018-10-04  7:48         ` David Hildenbrand
2018-10-04  7:48         ` David Hildenbrand
2018-09-28 17:02 ` Dave Hansen
2018-10-01  8:40 ` Michal Hocko
2018-10-01  8:40   ` Michal Hocko
2018-10-01  8:40   ` Michal Hocko
2018-10-01  9:34   ` David Hildenbrand
2018-10-01  9:34     ` David Hildenbrand
2018-10-01  9:34     ` David Hildenbrand
2018-10-02 13:47     ` Michal Hocko
2018-10-02 13:47     ` Michal Hocko
2018-10-02 13:47       ` Michal Hocko
2018-10-02 13:47       ` Michal Hocko
2018-10-02 15:25       ` David Hildenbrand
2018-10-02 15:25       ` David Hildenbrand
2018-10-02 15:25         ` David Hildenbrand
2018-10-02 15:25         ` David Hildenbrand
2018-10-03 13:38         ` Vitaly Kuznetsov
2018-10-03 13:38         ` Vitaly Kuznetsov
2018-10-03 13:38           ` Vitaly Kuznetsov
2018-10-03 13:38           ` Vitaly Kuznetsov
2018-10-03 13:44           ` Michal Hocko
2018-10-03 13:44           ` Michal Hocko [this message]
2018-10-03 13:44             ` Michal Hocko
2018-10-03 13:44             ` Michal Hocko
2018-10-03 13:52             ` Vitaly Kuznetsov
2018-10-03 13:52               ` Vitaly Kuznetsov
2018-10-03 13:52               ` Vitaly Kuznetsov
2018-10-03 14:07               ` Dave Hansen
2018-10-03 14:07                 ` Dave Hansen
2018-10-03 14:07                 ` Dave Hansen
2018-10-03 14:34                 ` Vitaly Kuznetsov
2018-10-03 14:34                 ` Vitaly Kuznetsov
2018-10-03 14:34                   ` Vitaly Kuznetsov
2018-10-03 14:34                   ` Vitaly Kuznetsov
2018-10-03 17:14                   ` David Hildenbrand
2018-10-03 17:14                     ` David Hildenbrand
2018-10-03 17:14                     ` David Hildenbrand
2018-10-04  6:19                     ` Michal Hocko
2018-10-04  6:19                       ` Michal Hocko
2018-10-04  6:19                       ` Michal Hocko
2018-10-04  8:13                       ` David Hildenbrand
2018-10-04  8:13                       ` David Hildenbrand
2018-10-04  8:13                         ` David Hildenbrand
2018-10-04  8:13                         ` David Hildenbrand
2018-10-04 15:28                         ` Michal Suchánek
2018-10-04 15:28                         ` Michal Suchánek
2018-10-04 15:28                           ` Michal Suchánek
2018-10-04 15:28                           ` Michal Suchánek
2018-10-04 15:45                           ` David Hildenbrand
2018-10-04 15:45                           ` David Hildenbrand
2018-10-04 15:45                             ` David Hildenbrand
2018-10-04 15:45                             ` David Hildenbrand
2018-10-04 17:50                             ` Michal Suchánek
2018-10-04 17:50                               ` Michal Suchánek
2018-10-04 17:50                               ` Michal Suchánek
2018-10-05  7:37                               ` David Hildenbrand
2018-10-05  7:37                                 ` David Hildenbrand
2018-10-05  7:37                                 ` David Hildenbrand
2018-10-05  7:37                               ` David Hildenbrand
2018-10-04 17:50                             ` Michal Suchánek
2018-10-04  6:19                     ` Michal Hocko
2018-10-03 17:14                   ` David Hildenbrand
2018-10-03 14:07               ` Dave Hansen
2018-10-03 14:24               ` Michal Hocko
2018-10-03 14:24               ` Michal Hocko
2018-10-03 14:24                 ` Michal Hocko
2018-10-03 14:24                 ` Michal Hocko
2018-10-03 17:06                 ` David Hildenbrand
2018-10-03 17:06                 ` David Hildenbrand
2018-10-03 17:06                   ` David Hildenbrand
2018-10-03 17:06                   ` David Hildenbrand
2018-10-04  8:12                 ` David Hildenbrand
2018-10-04  8:12                 ` David Hildenbrand
2018-10-04  8:12                   ` David Hildenbrand
2018-10-04  8:12                   ` David Hildenbrand
2018-10-03 13:52             ` Vitaly Kuznetsov
2018-10-03 13:54         ` Michal Hocko
2018-10-03 13:54         ` Michal Hocko
2018-10-03 13:54           ` Michal Hocko
2018-10-03 13:54           ` Michal Hocko
2018-10-03 17:00           ` David Hildenbrand
2018-10-03 17:00             ` David Hildenbrand
2018-10-03 17:00             ` David Hildenbrand
2018-10-04  6:28             ` Michal Hocko
2018-10-04  6:28               ` Michal Hocko
2018-10-04  6:28               ` Michal Hocko
2018-10-04  7:40               ` David Hildenbrand
2018-10-04  7:40                 ` David Hildenbrand
2018-10-04  7:40                 ` David Hildenbrand
2018-10-04  7:40               ` David Hildenbrand
2018-10-04  6:28             ` Michal Hocko
2018-10-03 17:00           ` David Hildenbrand
2018-10-01  9:34   ` David Hildenbrand
2018-10-01  8:40 ` Michal Hocko
2018-11-23 11:13 ` David Hildenbrand
2018-11-23 11:13   ` David Hildenbrand
2018-11-23 11:13   ` David Hildenbrand
2018-11-23 18:06   ` Michal Suchánek
2018-11-23 18:06   ` Michal Suchánek
2018-11-23 18:06     ` Michal Suchánek
2018-11-23 18:06     ` Michal Suchánek
2018-11-26 12:30     ` David Hildenbrand
2018-11-26 12:30       ` David Hildenbrand
2018-11-26 12:30       ` David Hildenbrand
2018-11-26 13:33       ` David Hildenbrand
2018-11-26 13:33         ` David Hildenbrand
2018-11-26 13:33         ` David Hildenbrand
2018-11-26 14:20         ` Michal Suchánek
2018-11-26 14:20           ` Michal Suchánek
2018-11-26 14:20           ` Michal Suchánek
2018-11-26 15:59           ` David Hildenbrand
2018-11-26 15:59           ` David Hildenbrand
2018-11-26 15:59             ` David Hildenbrand
2018-11-26 15:59             ` David Hildenbrand
2018-11-27 16:32             ` Michal Suchánek
2018-11-27 16:32             ` Michal Suchánek
2018-11-27 16:32               ` Michal Suchánek
2018-11-27 16:32               ` Michal Suchánek
2018-11-27 16:47               ` David Hildenbrand
2018-11-27 16:47               ` David Hildenbrand
2018-11-27 16:47                 ` David Hildenbrand
2018-11-27 16:47                 ` David Hildenbrand
2018-11-26 14:20         ` Michal Suchánek
2018-11-26 13:33       ` David Hildenbrand
2018-11-26 12:30     ` David Hildenbrand
2018-11-23 11:13 ` David Hildenbrand
  -- strict thread matches above, loose matches on Subject: below --
2018-09-28 15:03 David Hildenbrand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181003134444.GH4714@dhcp22.suse.cz \
    --to=mhocko@kernel.org \
    --cc=benh@kernel.crashing.org \
    --cc=bsingharora@gmail.com \
    --cc=dalias@libc.org \
    --cc=dan.j.williams@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=david@redhat.com \
    --cc=heiko.carstens@de.ibm.com \
    --cc=hpa@zytor.com \
    --cc=kstewart@linuxfoundation.org \
    --cc=linux-acpi@vger.kernel.org \
    --cc=linux-ia64@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-s390@vger.kernel.org \
    --cc=linux-sh@vger.kernel.org \
    --cc=mikey@neuling.org \
    --cc=mingo@redhat.com \
    --cc=mpe@ellerman.id.au \
    --cc=paulus@samba.org \
    --cc=peterz@infradead.org \
    --cc=rashmica.g@gmail.com \
    --cc=sfr@canb.auug.org.au \
    --cc=sthemmin@microsoft.com \
    --cc=vkuznets@redhat.com \
    --cc=xen-devel@lists.xenproject.org \
    --cc=ysato@users.sourceforge.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.