From: David Hildenbrand <david@redhat.com>
To: Dan Williams <dan.j.williams@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
"Eric W. Biederman" <ebiederm@xmission.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Linux MM <linux-mm@kvack.org>,
virtio-dev@lists.oasis-open.org,
virtualization@lists.linux-foundation.org,
linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
Linux ACPI <linux-acpi@vger.kernel.org>,
linux-nvdimm <linux-nvdimm@lists.01.org>,
linux-hyperv@vger.kernel.org,
linux-s390 <linux-s390@vger.kernel.org>,
xen-devel <xen-devel@lists.xenproject.org>,
Michal Hocko <mhocko@kernel.org>,
"Michael S . Tsirkin" <mst@redhat.com>,
Michal Hocko <mhocko@suse.com>,
Pankaj Gupta <pankaj.gupta.linux@gmail.com>,
Wei Yang <richard.weiyang@gmail.com>, Baoquan He <bhe@redhat.com>
Subject: Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP
Date: Fri, 1 May 2020 19:45:08 +0200 [thread overview]
Message-ID: <62dd4ce2-86cc-5b85-734f-ec8766528a1b@redhat.com> (raw)
In-Reply-To: <CAPcyv4iOqS0Wbfa2KPfE1axQFGXoRB4mmPRP__Lmqpw6Qpr_ig@mail.gmail.com>
On 01.05.20 19:39, Dan Williams wrote:
> On Fri, May 1, 2020 at 10:21 AM David Hildenbrand <david@redhat.com> wrote:
>>
>> On 01.05.20 18:56, Dan Williams wrote:
>>> On Fri, May 1, 2020 at 2:34 AM David Hildenbrand <david@redhat.com> wrote:
>>>>
>>>> On 01.05.20 00:24, Andrew Morton wrote:
>>>>> On Thu, 30 Apr 2020 20:43:39 +0200 David Hildenbrand <david@redhat.com> wrote:
>>>>>
>>>>>>>
>>>>>>> Why does the firmware map support hotplug entries?
>>>>>>
>>>>>> I assume:
>>>>>>
>>>>>> The firmware memmap was added primarily for x86-64 kexec (and still, is
>>>>>> mostly used on x86-64 only IIRC). There, we had ACPI hotplug. When DIMMs
>>>>>> get hotplugged on real HW, they get added to e820. Same applies to
>>>>>> memory added via HyperV balloon (unless memory is unplugged via
>>>>>> ballooning and you reboot ... the the e820 is changed as well). I assume
>>>>>> we wanted to be able to reflect that, to make kexec look like a real reboot.
>>>>>>
>>>>>> This worked for a while. Then came dax/kmem. Now comes virtio-mem.
>>>>>>
>>>>>>
>>>>>> But I assume only Andrew can enlighten us.
>>>>>>
>>>>>> @Andrew, any guidance here? Should we really add all memory to the
>>>>>> firmware memmap, even if this contradicts with the existing
>>>>>> documentation? (especially, if the actual firmware memmap will *not*
>>>>>> contain that memory after a reboot)
>>>>>
>>>>> For some reason that patch is misattributed - it was authored by
>>>>> Shaohui Zheng <shaohui.zheng@intel.com>, who hasn't been heard from in
>>>>> a decade. I looked through the email discussion from that time and I'm
>>>>> not seeing anything useful. But I wasn't able to locate Dave Hansen's
>>>>> review comments.
>>>>
>>>> Okay, thanks for checking. I think the documentation from 2008 is pretty
>>>> clear what has to be done here. I will add some of these details to the
>>>> patch description.
>>>>
>>>> Also, now that I know that esp. kexec-tools already don't consider
>>>> dax/kmem memory properly (memory will not get dumped via kdump) and
>>>> won't really suffer from a name change in /proc/iomem, I will go back to
>>>> the MHP_DRIVER_MANAGED approach and
>>>> 1. Don't create firmware memmap entries
>>>> 2. Name the resource "System RAM (driver managed)"
>>>> 3. Flag the resource via something like IORESOURCE_MEM_DRIVER_MANAGED.
>>>>
>>>> This way, kernel users and user space can figure out that this memory
>>>> has different semantics and handle it accordingly - I think that was
>>>> what Eric was asking for.
>>>>
>>>> Of course, open for suggestions.
>>>
>>> I'm still more of a fan of this being communicated by "System RAM"
>>
>> I was mentioning somewhere in this thread that "System RAM" inside a
>> hierarchy (like dax/kmem) will already be basically ignored by
>> kexec-tools. So, placing it inside a hierarchy already makes it look
>> special already.
>>
>> But after all, as we have to change kexec-tools either way, we can
>> directly go ahead and flag it properly as special (in case there will
>> ever be other cases where we could no longer distinguish it).
>>
>>> being parented especially because that tells you something about how
>>> the memory is driver-managed and which mechanism might be in play.
>>
>> The could be communicated to some degree via the resource hierarchy.
>>
>> E.g.,
>>
>> [root@localhost ~]# cat /proc/iomem
>> ...
>> 140000000-33fffffff : Persistent Memory
>> 140000000-1481fffff : namespace0.0
>> 150000000-33fffffff : dax0.0
>> 150000000-33fffffff : System RAM (driver managed)
>>
>> vs.
>>
>> :/# cat /proc/iomem
>> [...]
>> 140000000-333ffffff : virtio-mem (virtio0)
>> 140000000-147ffffff : System RAM (driver managed)
>> 148000000-14fffffff : System RAM (driver managed)
>> 150000000-157ffffff : System RAM (driver managed)
>>
>> Good enough for my taste.
>>
>>> What about adding an optional /sys/firmware/memmap/X/parent attribute.
>>
>> I really don't want any firmware memmap entries for something that is
>> not part of the firmware provided memmap. In addition,
>> /sys/firmware/memmap/ is still a fairly x86_64 specific thing. Only mips
>> and two arm configs enable it at all.
>>
>> So, IMHO, /sys/firmware/memmap/ is definitely not the way to go.
>
> I think that's a policy decision and policy decisions do not belong in
> the kernel. Give the tooling the opportunity to decide whether System
> RAM stays that way over a kexec. The parenthetical reference otherwise
> looks out of place to me in the /proc/iomem output. What makes it
> "driver managed" is how the kernel handles it, not how the kernel
> names it.
At least, virtio-mem is different. It really *has to be handled* by the
driver. This is not a policy. It's how it works.
--
Thanks,
David / dhildenb
next prev parent reply other threads:[~2020-05-01 17:45 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-30 10:29 [PATCH v2 0/3] mm/memory_hotplug: Allow to not create firmware memmap entries David Hildenbrand
2020-04-30 10:29 ` [PATCH v2 1/3] mm/memory_hotplug: Prepare passing flags to add_memory() and friends David Hildenbrand
2020-04-30 10:29 ` [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP David Hildenbrand
2020-04-30 15:38 ` Eric W. Biederman
2020-04-30 15:52 ` David Hildenbrand
2020-04-30 16:04 ` Dave Hansen
2020-04-30 16:33 ` Eric W. Biederman
2020-04-30 16:49 ` David Hildenbrand
2020-04-30 18:06 ` Eric W. Biederman
2020-04-30 18:43 ` David Hildenbrand
2020-04-30 18:58 ` Dan Williams
2020-04-30 22:24 ` Andrew Morton
2020-05-01 9:34 ` David Hildenbrand
2020-05-01 16:56 ` Dan Williams
2020-05-01 17:21 ` David Hildenbrand
2020-05-01 17:39 ` Dan Williams
2020-05-01 17:45 ` David Hildenbrand [this message]
2020-05-01 17:51 ` David Hildenbrand
2020-05-01 18:03 ` Dan Williams
2020-05-01 18:14 ` David Hildenbrand
2020-05-01 18:43 ` Dan Williams
2020-05-01 19:17 ` David Hildenbrand
2020-05-01 20:12 ` Dan Williams
2020-05-01 21:10 ` David Hildenbrand
2020-05-01 21:52 ` Dan Williams
2020-05-02 9:26 ` David Hildenbrand
2020-05-02 18:03 ` Dan Williams
2020-04-30 10:29 ` [PATCH v2 3/3] device-dax: Add system ram (add_memory()) with MHP_NO_FIRMWARE_MEMMAP David Hildenbrand
2020-04-30 11:23 ` Dave Hansen
2020-04-30 15:28 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=62dd4ce2-86cc-5b85-734f-ec8766528a1b@redhat.com \
--to=david@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=bhe@redhat.com \
--cc=dan.j.williams@intel.com \
--cc=ebiederm@xmission.com \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-s390@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mhocko@kernel.org \
--cc=mhocko@suse.com \
--cc=mst@redhat.com \
--cc=pankaj.gupta.linux@gmail.com \
--cc=richard.weiyang@gmail.com \
--cc=virtio-dev@lists.oasis-open.org \
--cc=virtualization@lists.linux-foundation.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).