From: David Hildenbrand <david@redhat.com>
To: Anshuman Khandual <anshuman.khandual@arm.com>,
linux-mm@kvack.org, akpm@linux-foundation.org, hca@linux.ibm.com,
catalin.marinas@arm.com
Cc: Oscar Salvador <osalvador@suse.de>,
Vasily Gorbik <gor@linux.ibm.com>, Will Deacon <will@kernel.org>,
Ard Biesheuvel <ardb@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org,
linux-kernel@vger.kernel.org,
"Michael S. Tsirkin" <mst@redhat.com>,
Jason Wang <jasowang@redhat.com>,
Pankaj Gupta <pankaj.gupta.linux@gmail.com>,
Wei Yang <richard.weiyang@linux.alibaba.com>,
teawater <teawaterz@linux.alibaba.com>,
Pankaj Gupta <pankaj.gupta@cloud.ionos.com>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH V4 4/4] virtio-mem: check against mhp_get_pluggable_range() which memory we can hotplug
Date: Mon, 25 Jan 2021 13:01:14 +0100 [thread overview]
Message-ID: <a62e9474-ada3-ab80-90a7-14db87c90aa6@redhat.com> (raw)
In-Reply-To: <1611543532-18698-5-git-send-email-anshuman.khandual@arm.com>
On 25.01.21 03:58, Anshuman Khandual wrote:
> From: David Hildenbrand <david@redhat.com>
>
> Right now, we only check against MAX_PHYSMEM_BITS - but turns out there
> are more restrictions of which memory we can actually hotplug, especially
> om arm64 or s390x once we support them: we might receive something like
> -E2BIG or -ERANGE from add_memory_driver_managed(), stopping device
> operation.
>
> So, check right when initializing the device which memory we can add,
> warning the user. Try only adding actually pluggable ranges: in the worst
> case, no memory provided by our device is pluggable.
>
> In the usual case, we expect all device memory to be pluggable, and in
> corner cases only some memory at the end of the device-managed memory
> region to not be pluggable.
>
> Cc: "Michael S. Tsirkin" <mst@redhat.com>
> Cc: Jason Wang <jasowang@redhat.com>
> Cc: Pankaj Gupta <pankaj.gupta.linux@gmail.com>
> Cc: Oscar Salvador <osalvador@suse.de>
> Cc: Wei Yang <richard.weiyang@linux.alibaba.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: catalin.marinas@arm.com
> Cc: teawater <teawaterz@linux.alibaba.com>
> Cc: Anshuman Khandual <anshuman.khandual@arm.com>
> Cc: Pankaj Gupta <pankaj.gupta@cloud.ionos.com>
> Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Cc: hca@linux.ibm.com
> Cc: Vasily Gorbik <gor@linux.ibm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: Ard Biesheuvel <ardb@kernel.org>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Heiko Carstens <hca@linux.ibm.com>
> Cc: Michal Hocko <mhocko@kernel.org>
> Signed-off-by: David Hildenbrand <david@redhat.com>
> Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
> ---
> drivers/virtio/virtio_mem.c | 40 +++++++++++++++++++++++++------------
> 1 file changed, 27 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c
> index 9fc9ec4a25f5..14c17c5c1695 100644
> --- a/drivers/virtio/virtio_mem.c
> +++ b/drivers/virtio/virtio_mem.c
> @@ -2222,7 +2222,7 @@ static int virtio_mem_unplug_pending_mb(struct virtio_mem *vm)
> */
> static void virtio_mem_refresh_config(struct virtio_mem *vm)
> {
> - const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS;
> + const struct range pluggable_range = mhp_get_pluggable_range(true);
> uint64_t new_plugged_size, usable_region_size, end_addr;
>
> /* the plugged_size is just a reflection of what _we_ did previously */
> @@ -2234,15 +2234,25 @@ static void virtio_mem_refresh_config(struct virtio_mem *vm)
> /* calculate the last usable memory block id */
> virtio_cread_le(vm->vdev, struct virtio_mem_config,
> usable_region_size, &usable_region_size);
> - end_addr = vm->addr + usable_region_size;
> - end_addr = min(end_addr, phys_limit);
> + end_addr = min(vm->addr + usable_region_size - 1,
> + pluggable_range.end);
>
> - if (vm->in_sbm)
> - vm->sbm.last_usable_mb_id =
> - virtio_mem_phys_to_mb_id(end_addr) - 1;
> - else
> - vm->bbm.last_usable_bb_id =
> - virtio_mem_phys_to_bb_id(vm, end_addr) - 1;
> + if (vm->in_sbm) {
> + vm->sbm.last_usable_mb_id = virtio_mem_phys_to_mb_id(end_addr);
> + if (!IS_ALIGNED(end_addr + 1, memory_block_size_bytes()))
> + vm->sbm.last_usable_mb_id--;
> + } else {
> + vm->bbm.last_usable_bb_id = virtio_mem_phys_to_bb_id(vm,
> + end_addr);
> + if (!IS_ALIGNED(end_addr + 1, vm->bbm.bb_size))
> + vm->bbm.last_usable_bb_id--;
> + }
> + /*
> + * If we cannot plug any of our device memory (e.g., nothing in the
> + * usable region is addressable), the last usable memory block id will
> + * be smaller than the first usable memory block id. We'll stop
> + * attempting to add memory with -ENOSPC from our main loop.
> + */
>
> /* see if there is a request to change the size */
> virtio_cread_le(vm->vdev, struct virtio_mem_config, requested_size,
> @@ -2364,6 +2374,7 @@ static int virtio_mem_init_vq(struct virtio_mem *vm)
>
> static int virtio_mem_init(struct virtio_mem *vm)
> {
> + const struct range pluggable_range = mhp_get_pluggable_range(true);
> const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS;
Sorry, forgot to drop ^ (phys_limit), otherwise ther is a friendly
warning from the compiler. We have to drop that line.
Apart from that, at least on x86-64 it does what it's supposed to do. I
temporarily changed MAX_PHYSMEM_BITS to 35 bits and added a virtio-mem
device that crosses the 32 GiB address limit.
[ 0.572084] virtio_mem virtio1: Some device memory is not
addressable/pluggable. This can make some memory unusable.
[ 0.573013] virtio_mem virtio1: start address: 0x740000000
[ 0.573497] virtio_mem virtio1: region size: 0x500000000
And virtio-mem won't add any memory exceeding that:
(qemu) qom-set vmem0 requested-size 20G
(qemu) info memory-devices
Memory device [virtio-mem]: "vmem0"
[...]
Memory device [virtio-mem]: "vmem1"
memaddr: 0x740000000
node: 1
requested-size: 21474836480
size: 3221225472
max-size: 21474836480
block-size: 2097152
memdev: /objects/mem1
I adds all memory up to the 32GiB address limit (35 bits) and stops.
LGTM (arm64 to be tested in the future once supported).
--
Thanks,
David / dhildenb
next prev parent reply other threads:[~2021-01-25 12:22 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-25 2:58 [PATCH V4 0/4] mm/memory_hotplug: Pre-validate the address range with platform Anshuman Khandual
2021-01-25 2:58 ` [PATCH V4 1/4] mm/memory_hotplug: Prevalidate the address range being added " Anshuman Khandual
2021-01-25 9:21 ` David Hildenbrand
2021-01-25 2:58 ` [PATCH V4 2/4] arm64/mm: Define arch_get_mappable_range() Anshuman Khandual
2021-01-25 2:58 ` [PATCH V4 3/4] s390/mm: " Anshuman Khandual
2021-01-25 2:58 ` [PATCH V4 4/4] virtio-mem: check against mhp_get_pluggable_range() which memory we can hotplug Anshuman Khandual
2021-01-25 12:01 ` David Hildenbrand [this message]
2021-01-27 3:42 ` Anshuman Khandual
2021-01-25 9:25 ` [PATCH V4 0/4] mm/memory_hotplug: Pre-validate the address range with platform David Hildenbrand
2021-01-25 9:52 ` Anshuman Khandual
2021-01-25 9:53 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a62e9474-ada3-ab80-90a7-14db87c90aa6@redhat.com \
--to=david@redhat.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=ardb@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=jasowang@redhat.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-s390@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mhocko@kernel.org \
--cc=mst@redhat.com \
--cc=osalvador@suse.de \
--cc=pankaj.gupta.linux@gmail.com \
--cc=pankaj.gupta@cloud.ionos.com \
--cc=richard.weiyang@linux.alibaba.com \
--cc=teawaterz@linux.alibaba.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).