From: David Hildenbrand <david@redhat.com>
To: Baoquan He <bhe@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
"Eric W. Biederman" <ebiederm@xmission.com>,
Russell King - ARM Linux admin <linux@armlinux.org.uk>,
Anshuman Khandual <anshuman.khandual@arm.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Bhupesh Sharma <bhsharma@redhat.com>,
kexec@lists.infradead.org, linux-mm@kvack.org,
James Morse <james.morse@arm.com>, Will Deacon <will@kernel.org>,
linux-arm-kernel@lists.infradead.org,
linuxppc-dev@lists.ozlabs.org, piliu@redhat.com
Subject: Re: [PATCH 1/3] kexec: Prevent removal of memory in use by a loaded kexec image
Date: Wed, 22 Apr 2020 12:05:40 +0200 [thread overview]
Message-ID: <b0e7b7e8-01d0-e03e-83a8-4763ccbede43@redhat.com> (raw)
In-Reply-To: <20200422095733.GU4247@MiWiFi-R3L-srv>
On 22.04.20 11:57, Baoquan He wrote:
> On 04/22/20 at 11:24am, David Hildenbrand wrote:
>> On 22.04.20 11:17, Baoquan He wrote:
>>> On 04/21/20 at 03:29pm, David Hildenbrand wrote:
>>>>>> ACPI SRAT is embeded into efi, need read out the rsdp pointer. If we don't
>>>>>> pass the efi, it won't get the SRAT table correctly, if I remember
>>>>>> correctly. Yeah, I remeber kvm guest can get memory hotplugged with
>>>>>> ACPI only, this won't happen on bare metal though. Need check carefully.
>>>>>> I have been using kvm guest with uefi firmwire recently.
>>>>>
>>>>> Yeah, I can imagine that bare metal is different. kvm only uses ACPI.
>>>>>
>>>>> I'm also asking because of virtio-mem. Memory added via virtio-mem is
>>>>> not part of any efi tables or whatsoever. So I assume the kexec kernel
>>>>> will not detect it automatically (good!), instead load the virtio-mem
>>>>> driver and let it add memory back to the system.
>>>>>
>>>>> I should probably play with kexec and virtio-mem once I have some spare
>>>>> cycles ... to find out what's broken and needs to be addressed :)
>>>>
>>>> FWIW, I just gave virtio-mem and kexec/kdump a try.
>>>>
>>>> a) kdump seems to work. Memory added by virtio-mem is getting dumped.
>>>> The kexec kernel only uses memory in the crash region. The virtio-mem
>>>> driver properly bails out due to is_kdump_kernel().
>>>
>>> Right, kdump is not impacted later added memory.
>>>
>>>>
>>>> b) "kexec -s -l" seems to work fine. For now, the kernel does not seem
>>>> to get placed on virtio-mem memory (pure luck due to the left-to-right
>>>> search). Memory added by virtio-mem is not getting added to the e820
>>>> map. Once the virtio-mem driver comes back up in the kexec kernel, the
>>>> right memory is readded.
>>>
>>> kexec_file_load just behaves as you tested. It doesn't collect later
>>> added memory to e820 because it uses e820_table_kexec directly to pass
>>> e820 to kexec-ed kernel. However, this e820_table_kexec is only updated
>>> during boot stage. I tried hot adding DIMM after boot, kexec-ed kernel
>>> doesn't have it in e820 during bootup, but it's recoginized and added
>>> when ACPI scanning. I think we should update e820_table_kexec when hot
>>> add/remove memory, at least for DIMM. Not sure if DLPAR, virtio-mem,
>>> balloon will need be added into e820_table_kexec too, and if this is
>>> expected behaviour.
>>>
>>> But whatever we do, it won't impact the kexec file_loading, because of
>>> the searching strategy bottom up. Just adding them into e820_table_kexec
>>> will make it consistent with cold reboot which get recognizes and get
>>> them into e820 during bootup.
>>
>> Yeah, I think whatever a cold-booted kernel will see is what kexec-ed
>> kernel should see. Not more, not less.
>>
>> Regarding virtio-mem: Not in e820 on cold-boot.
>> Regarding DIMMs: DIMMs under KVM will never show up in the e820 map
>> IIRC. I think on real HW it can be different.
>
> Yeah, DIMMs under KVM won't show up in e820 map. While this is not feature
> of QEMU/KVM, but a defect of it. I ever asked Igor who is developer of
> QEMU/KVM guest in this area, why we don't make kvm guest recognize
> hotpluggable DIMM and add it into e820 map, he said he had tried to make
> it, but this will corrupt guest on HyperV. So he had to revert the
Yeah, I remember that this had to be reverted due to something breaking.
But OTOH, it allows us to online coldplugged DIMMs online_movable
easily, so I'd say it's even a feature (although, does not behave like
real HW we have).
I use this extensively when testing memory hot(un)plug via coldplugged
DIMMs.
I do wonder if there is real HW, where this is also the case.
> commit on qemu. So I think we can leave it for now for both real HW and
> kvm, or update the e820_table_kexec to include added DIMM for both real
> HW and KVM. I hope one day KVM dev will find a way to conquer the defect
> on HyperV and make the e820map consistent with bare metal. After all,
> kvm guest is trying to imitate real HW for the most part.
>
> Anyway, I will think about the e820_table_kexec updating. See if we can
> do something about it.
Yeah, for DIMMs on real HW it might definitely make sense. We might be
able to hook into updates of /sys/firmware/memmap on memory add/remove.
--
Thanks,
David / dhildenb
next prev parent reply other threads:[~2020-04-22 10:05 UTC|newest]
Thread overview: 92+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-26 18:07 [PATCH 0/3] kexec/memory_hotplug: Prevent removal and accidental use James Morse
2020-03-26 18:07 ` [PATCH 1/3] kexec: Prevent removal of memory in use by a loaded kexec image James Morse
2020-03-27 0:43 ` Anshuman Khandual
2020-03-27 2:54 ` Baoquan He
2020-03-27 15:46 ` James Morse
2020-03-27 2:34 ` Baoquan He
2020-03-27 9:30 ` David Hildenbrand
2020-03-27 16:56 ` James Morse
2020-03-27 17:06 ` David Hildenbrand
2020-03-27 18:07 ` James Morse
2020-03-27 18:52 ` David Hildenbrand
2020-03-30 13:00 ` James Morse
2020-03-30 13:13 ` David Hildenbrand
2020-03-30 17:17 ` James Morse
2020-03-30 18:14 ` David Hildenbrand
2020-04-10 19:10 ` Andrew Morton
2020-04-11 3:44 ` Baoquan He
2020-04-11 9:30 ` Russell King - ARM Linux admin
2020-04-11 9:58 ` David Hildenbrand
2020-04-12 5:35 ` Baoquan He
2020-04-12 8:08 ` Russell King - ARM Linux admin
2020-04-12 19:52 ` Eric W. Biederman
2020-04-12 20:37 ` Bhupesh SHARMA
2020-04-13 2:37 ` Baoquan He
2020-04-13 13:15 ` Eric W. Biederman
2020-04-13 23:01 ` Andrew Morton
2020-04-14 6:13 ` Eric W. Biederman
2020-04-14 6:40 ` Baoquan He
2020-04-14 6:51 ` Baoquan He
2020-04-14 8:00 ` David Hildenbrand
2020-04-14 9:22 ` Baoquan He
2020-04-14 9:37 ` David Hildenbrand
2020-04-14 14:39 ` Baoquan He
2020-04-14 14:49 ` David Hildenbrand
2020-04-15 2:35 ` Baoquan He
2020-04-16 13:31 ` David Hildenbrand
2020-04-16 14:02 ` Baoquan He
2020-04-16 14:09 ` David Hildenbrand
2020-04-16 14:36 ` Baoquan He
2020-04-16 14:47 ` David Hildenbrand
2020-04-21 13:29 ` David Hildenbrand
2020-04-21 13:57 ` David Hildenbrand
2020-04-21 13:59 ` Eric W. Biederman
2020-04-21 14:30 ` David Hildenbrand
2020-04-22 9:17 ` Baoquan He
2020-04-22 9:24 ` David Hildenbrand
2020-04-22 9:57 ` Baoquan He
2020-04-22 10:05 ` David Hildenbrand [this message]
2020-04-22 10:36 ` Baoquan He
2020-04-14 9:16 ` Dave Young
2020-04-14 9:38 ` Dave Young
2020-04-14 7:05 ` David Hildenbrand
2020-04-14 16:55 ` James Morse
2020-04-14 17:41 ` David Hildenbrand
2020-04-15 20:33 ` Eric W. Biederman
2020-04-22 12:28 ` James Morse
2020-04-22 15:25 ` Eric W. Biederman
2020-04-22 16:40 ` David Hildenbrand
2020-04-23 16:29 ` Eric W. Biederman
2020-04-24 7:39 ` David Hildenbrand
2020-04-24 7:41 ` David Hildenbrand
2020-05-01 16:55 ` James Morse
2020-03-26 18:07 ` [PATCH 2/3] mm/memory_hotplug: Allow arch override of non boot memory resource names James Morse
2020-03-27 9:59 ` David Hildenbrand
2020-03-27 15:39 ` James Morse
2020-03-30 13:23 ` David Hildenbrand
2020-03-30 17:17 ` James Morse
2020-04-02 5:49 ` Dave Young
2020-04-02 6:12 ` piliu
2020-04-14 17:21 ` James Morse
2020-04-15 20:36 ` Eric W. Biederman
2020-04-22 12:14 ` James Morse
2020-05-09 0:45 ` Andrew Morton
2020-05-11 8:35 ` David Hildenbrand
2020-03-26 18:07 ` [PATCH 3/3] arm64: memory: Give hotplug memory a different resource name James Morse
2020-03-30 19:01 ` David Hildenbrand
2020-04-15 20:37 ` Eric W. Biederman
2020-04-22 12:14 ` James Morse
2020-03-27 2:11 ` [PATCH 0/3] kexec/memory_hotplug: Prevent removal and accidental use Baoquan He
2020-03-27 15:40 ` James Morse
2020-03-27 9:27 ` David Hildenbrand
2020-03-27 15:42 ` James Morse
2020-03-30 13:18 ` David Hildenbrand
2020-03-30 13:55 ` Baoquan He
2020-03-30 17:17 ` James Morse
2020-03-31 3:46 ` Dave Young
2020-04-14 17:31 ` James Morse
2020-03-31 3:38 ` Dave Young
2020-04-15 20:29 ` Eric W. Biederman
2020-04-22 12:14 ` James Morse
2020-04-22 13:04 ` Eric W. Biederman
2020-04-22 15:40 ` James Morse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b0e7b7e8-01d0-e03e-83a8-4763ccbede43@redhat.com \
--to=david@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=bhe@redhat.com \
--cc=bhsharma@redhat.com \
--cc=catalin.marinas@arm.com \
--cc=ebiederm@xmission.com \
--cc=james.morse@arm.com \
--cc=kexec@lists.infradead.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-mm@kvack.org \
--cc=linux@armlinux.org.uk \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=piliu@redhat.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).