From: David Hildenbrand <david@redhat.com>
To: Rashmica Gupta <rashmica.g@gmail.com>,
Oscar Salvador <osalvador@suse.de>
Cc: akpm@linux-foundation.org, mhocko@suse.com,
dan.j.williams@intel.com, pasha.tatashin@soleen.com,
Jonathan.Cameron@huawei.com, anshuman.khandual@arm.com,
vbabka@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 0/5] Allocate memmap from hotadded memory
Date: Mon, 29 Jul 2019 10:06:23 +0200 [thread overview]
Message-ID: <b3fd1177-45ef-fd9e-78c8-d05138c647da@redhat.com> (raw)
In-Reply-To: <b7de7d9d84e9dd47358a254d36f6a24dd48da963.camel@gmail.com>
>> Of course, other interfaces might make sense.
>>
>> You can then start using these memory blocks and hinder them from
>> getting onlined (as a safety net) via memory notifiers.
>>
>> That would at least avoid you having to call
>> add_memory/remove_memory/offline_pages/device_online/modifying
>> memblock
>> states manually.
>
> I see what you're saying and that definitely sounds safer.
>
> We would still need to call remove_memory and add_memory from memtrace
> as
> just offlining memory doesn't remove it from the linear page tables
> (if
> it's still in the page tables then hardware can prefetch it and if
> hardware tracing is using it then the box checkstops).
That prefetching part is interesting (and nasty as well). If we could at
least get rid of the manual onlining/offlining, I would be able to sleep
better at night ;) One step at a time.
>
>>
>> (binding the memory block devices to a driver would be nicer, but the
>> infrastructure is not really there yet - we have no such drivers in
>> place yet)
>>
>>> I don't know the mm code nor how the notifiers work very well so I
>>> can't quite see how the above would work. I'm assuming memtrace
>>> would
>>> register a hotplug notifier and when memory is offlined from
>>> userspace,
>>> the callback func in memtrace would be called if the priority was
>>> high
>>> enough? But how do we know that the memory being offlined is
>>> intended
>>> for usto touch? Is there a way to offline memory from userspace not
>>> using sysfs or have I missed something in the sysfs interface?
>>
>> The notifier would really only be used to hinder onlining as a safety
>> net. User space prepares (offlines) the memory blocks and then tells
>> the
>> drivers which memory blocks to use.
>>
>>> On a second read, perhaps you are assuming that memtrace is used
>>> after
>>> adding new memory at runtime? If so, that is not the case. If not,
>>> then
>>> would you be able to clarify what I'm not seeing?
>>
>> The main problem I see is that you are calling
>> add_memory/remove_memory() on memory your device driver doesn't own.
>> It
>> could reside on a DIMM if I am not mistaking (or later on
>> paravirtualized memory devices like virtio-mem if I ever get to
>> implement them ;) ).
>
> This is just for baremetal/powernv so shouldn't affect virtual memory
> devices.
Good to now.
>
>>
>> How is it guaranteed that the memory you are allocating does not
>> reside
>> on a DIMM for example added via add_memory() by the ACPI driver?
>
> Good point. We don't have ACPI on powernv but currently this would try
> to remove memory from any online memory node, not just the ones that
> are backed by RAM. oops.
Okay, so essentially no memory hotplug/unplug along with memtrace. (can
we document that somewhere?). I think add_memory()/try_remove_memory()
could be tolerable in these environments (as it's only boot memory).
--
Thanks,
David / dhildenb
next prev parent reply other threads:[~2019-07-29 8:06 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-25 7:52 [PATCH v2 0/5] Allocate memmap from hotadded memory Oscar Salvador
2019-06-25 7:52 ` [PATCH v2 1/5] drivers/base/memory: Remove unneeded check in remove_memory_block_devices Oscar Salvador
2019-06-25 8:01 ` David Hildenbrand
2019-06-25 8:03 ` David Hildenbrand
2019-06-25 8:09 ` Oscar Salvador
2019-06-25 8:27 ` David Hildenbrand
2019-06-25 7:52 ` [PATCH v2 2/5] mm,memory_hotplug: Introduce MHP_VMEMMAP_FLAGS Oscar Salvador
2019-06-25 8:31 ` David Hildenbrand
2019-07-24 20:11 ` Dan Williams
2019-07-24 21:36 ` osalvador
2019-07-25 9:27 ` Oscar Salvador
2019-07-25 9:30 ` David Hildenbrand
2019-07-25 9:40 ` Oscar Salvador
2019-07-25 10:04 ` David Hildenbrand
2019-07-25 10:13 ` Oscar Salvador
2019-07-25 10:15 ` David Hildenbrand
2019-06-25 7:52 ` [PATCH v2 3/5] mm,memory_hotplug: Introduce Vmemmap page helpers Oscar Salvador
2019-06-25 10:28 ` David Hildenbrand
2019-06-26 9:48 ` Oscar Salvador
2019-06-25 7:52 ` [PATCH v2 4/5] mm,memory_hotplug: allocate memmap from the added memory range for sparse-vmemmap Oscar Salvador
2019-06-25 8:49 ` David Hildenbrand
2019-06-26 8:13 ` Oscar Salvador
2019-06-26 8:15 ` David Hildenbrand
2019-06-26 8:17 ` Anshuman Khandual
2019-06-26 8:28 ` Oscar Salvador
2019-07-24 21:49 ` Dan Williams
2019-06-25 7:52 ` [PATCH v2 5/5] mm,memory_hotplug: Allow userspace to enable/disable vmemmap Oscar Salvador
2019-06-25 8:25 ` [PATCH v2 0/5] Allocate memmap from hotadded memory David Hildenbrand
2019-06-25 8:33 ` David Hildenbrand
2019-06-26 8:03 ` Oscar Salvador
2019-06-26 8:11 ` David Hildenbrand
2019-06-26 8:15 ` Oscar Salvador
2019-06-26 8:27 ` Oscar Salvador
2019-06-26 8:37 ` David Hildenbrand
2019-06-26 8:28 ` David Hildenbrand
2019-07-02 6:42 ` Rashmica Gupta
2019-07-02 7:48 ` Oscar Salvador
2019-07-02 8:52 ` Rashmica Gupta
2019-07-10 1:14 ` Rashmica Gupta
2019-07-31 12:08 ` Michal Hocko
2019-07-31 23:06 ` Rashmica Gupta
2019-08-01 7:17 ` Michal Hocko
2019-08-01 7:18 ` David Hildenbrand
2019-08-01 7:24 ` Michal Hocko
2019-08-01 7:26 ` David Hildenbrand
2019-08-01 7:31 ` David Hildenbrand
2019-08-01 7:39 ` Michal Hocko
2019-08-01 7:48 ` Michal Hocko
2019-08-01 9:18 ` David Hildenbrand
2019-08-01 7:34 ` Michal Hocko
2019-08-01 7:50 ` David Hildenbrand
2019-08-01 8:04 ` Michal Hocko
2019-07-16 12:28 ` David Hildenbrand
2019-07-29 5:42 ` Rashmica Gupta
2019-07-29 8:06 ` David Hildenbrand [this message]
2019-07-30 7:08 ` Rashmica Gupta
2019-07-31 2:21 ` Rashmica Gupta
2019-07-31 9:39 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b3fd1177-45ef-fd9e-78c8-d05138c647da@redhat.com \
--to=david@redhat.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=dan.j.williams@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=osalvador@suse.de \
--cc=pasha.tatashin@soleen.com \
--cc=rashmica.g@gmail.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).