From: Wei Yang <richard.weiyang@linux.alibaba.com>
To: David Hildenbrand <david@redhat.com>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
virtualization@lists.linux-foundation.org,
Andrew Morton <akpm@linux-foundation.org>,
"Michael S . Tsirkin" <mst@redhat.com>,
Jason Wang <jasowang@redhat.com>,
Pankaj Gupta <pankaj.gupta.linux@gmail.com>
Subject: Re: [PATCH v1 23/29] virtio-mem: factor out adding/removing memory from Linux
Date: Fri, 16 Oct 2020 16:59:53 +0800 [thread overview]
Message-ID: <20201016085953.GF44269@L-31X9LVDL-1304.local> (raw)
In-Reply-To: <20201012125323.17509-24-david@redhat.com>
On Mon, Oct 12, 2020 at 02:53:17PM +0200, David Hildenbrand wrote:
>Let's use wrappers for the low-level functions that dev_dbg/dev_warn
>and work on addr + size, such that we can reuse them for adding/removing
>in other granularity.
>
>We only warn when adding memory failed, because that's something to pay
>attention to. We won't warn when removing failed, we'll reuse that in
>racy context soon (and we do have proper BUG_ON() statements in the
>current cases where it must never happen).
>
>Cc: "Michael S. Tsirkin" <mst@redhat.com>
>Cc: Jason Wang <jasowang@redhat.com>
>Cc: Pankaj Gupta <pankaj.gupta.linux@gmail.com>
>Signed-off-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Wei Yang <richard.weiyang@linux.alibaba.com>
>---
> drivers/virtio/virtio_mem.c | 107 ++++++++++++++++++++++++------------
> 1 file changed, 73 insertions(+), 34 deletions(-)
>
>diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c
>index d3ab04f655ee..eb2ad31a8d8a 100644
>--- a/drivers/virtio/virtio_mem.c
>+++ b/drivers/virtio/virtio_mem.c
>@@ -453,18 +453,16 @@ static bool virtio_mem_could_add_memory(struct virtio_mem *vm, uint64_t size)
> }
>
> /*
>- * Try to add a memory block to Linux. This will usually only fail
>- * if out of memory.
>+ * Try adding memory to Linux. Will usually only fail if out of memory.
> *
> * Must not be called with the vm->hotplug_mutex held (possible deadlock with
> * onlining code).
> *
>- * Will not modify the state of the memory block.
>+ * Will not modify the state of memory blocks in virtio-mem.
> */
>-static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id)
>+static int virtio_mem_add_memory(struct virtio_mem *vm, uint64_t addr,
>+ uint64_t size)
> {
>- const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id);
>- const uint64_t size = memory_block_size_bytes();
> int rc;
>
> /*
>@@ -478,32 +476,50 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id)
> return -ENOMEM;
> }
>
>- dev_dbg(&vm->vdev->dev, "adding memory block: %lu\n", mb_id);
>+ dev_dbg(&vm->vdev->dev, "adding memory: 0x%llx - 0x%llx\n", addr,
>+ addr + size - 1);
> /* Memory might get onlined immediately. */
> atomic64_add(size, &vm->offline_size);
> rc = add_memory_driver_managed(vm->nid, addr, size, vm->resource_name,
> MEMHP_MERGE_RESOURCE);
>- if (rc)
>+ if (rc) {
> atomic64_sub(size, &vm->offline_size);
>+ dev_warn(&vm->vdev->dev, "adding memory failed: %d\n", rc);
>+ /*
>+ * TODO: Linux MM does not properly clean up yet in all cases
>+ * where adding of memory failed - especially on -ENOMEM.
>+ */
>+ }
> return rc;
> }
>
> /*
>- * Try to remove a memory block from Linux. Will only fail if the memory block
>- * is not offline.
>+ * See virtio_mem_add_memory(): Try adding a single Linux memory block.
>+ */
>+static int virtio_mem_sbm_add_mb(struct virtio_mem *vm, unsigned long mb_id)
>+{
>+ const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id);
>+ const uint64_t size = memory_block_size_bytes();
>+
>+ return virtio_mem_add_memory(vm, addr, size);
>+}
>+
>+/*
>+ * Try removing memory from Linux. Will only fail if memory blocks aren't
>+ * offline.
> *
> * Must not be called with the vm->hotplug_mutex held (possible deadlock with
> * onlining code).
> *
>- * Will not modify the state of the memory block.
>+ * Will not modify the state of memory blocks in virtio-mem.
> */
>-static int virtio_mem_mb_remove(struct virtio_mem *vm, unsigned long mb_id)
>+static int virtio_mem_remove_memory(struct virtio_mem *vm, uint64_t addr,
>+ uint64_t size)
> {
>- const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id);
>- const uint64_t size = memory_block_size_bytes();
> int rc;
>
>- dev_dbg(&vm->vdev->dev, "removing memory block: %lu\n", mb_id);
>+ dev_dbg(&vm->vdev->dev, "removing memory: 0x%llx - 0x%llx\n", addr,
>+ addr + size - 1);
> rc = remove_memory(vm->nid, addr, size);
> if (!rc) {
> atomic64_sub(size, &vm->offline_size);
>@@ -512,27 +528,41 @@ static int virtio_mem_mb_remove(struct virtio_mem *vm, unsigned long mb_id)
> * immediately instead of waiting.
> */
> virtio_mem_retry(vm);
>+ } else {
>+ dev_dbg(&vm->vdev->dev, "removing memory failed: %d\n", rc);
> }
> return rc;
> }
>
> /*
>- * Try to offline and remove a memory block from Linux.
>+ * See virtio_mem_remove_memory(): Try removing a single Linux memory block.
>+ */
>+static int virtio_mem_sbm_remove_mb(struct virtio_mem *vm, unsigned long mb_id)
>+{
>+ const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id);
>+ const uint64_t size = memory_block_size_bytes();
>+
>+ return virtio_mem_remove_memory(vm, addr, size);
>+}
>+
>+/*
>+ * Try offlining and removing memory from Linux.
> *
> * Must not be called with the vm->hotplug_mutex held (possible deadlock with
> * onlining code).
> *
>- * Will not modify the state of the memory block.
>+ * Will not modify the state of memory blocks in virtio-mem.
> */
>-static int virtio_mem_mb_offline_and_remove(struct virtio_mem *vm,
>- unsigned long mb_id)
>+static int virtio_mem_offline_and_remove_memory(struct virtio_mem *vm,
>+ uint64_t addr,
>+ uint64_t size)
> {
>- const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id);
>- const uint64_t size = memory_block_size_bytes();
> int rc;
>
>- dev_dbg(&vm->vdev->dev, "offlining and removing memory block: %lu\n",
>- mb_id);
>+ dev_dbg(&vm->vdev->dev,
>+ "offlining and removing memory: 0x%llx - 0x%llx\n", addr,
>+ addr + size - 1);
>+
> rc = offline_and_remove_memory(vm->nid, addr, size);
> if (!rc) {
> atomic64_sub(size, &vm->offline_size);
>@@ -541,10 +571,26 @@ static int virtio_mem_mb_offline_and_remove(struct virtio_mem *vm,
> * immediately instead of waiting.
> */
> virtio_mem_retry(vm);
>+ } else {
>+ dev_dbg(&vm->vdev->dev,
>+ "offlining and removing memory failed: %d\n", rc);
> }
> return rc;
> }
>
>+/*
>+ * See virtio_mem_offline_and_remove_memory(): Try offlining and removing
>+ * a single Linux memory block.
>+ */
>+static int virtio_mem_sbm_offline_and_remove_mb(struct virtio_mem *vm,
>+ unsigned long mb_id)
>+{
>+ const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id);
>+ const uint64_t size = memory_block_size_bytes();
>+
>+ return virtio_mem_offline_and_remove_memory(vm, addr, size);
>+}
>+
> /*
> * Trigger the workqueue so the device can perform its magic.
> */
>@@ -1230,17 +1276,10 @@ static int virtio_mem_sbm_plug_and_add_mb(struct virtio_mem *vm,
> VIRTIO_MEM_SBM_MB_OFFLINE_PARTIAL);
>
> /* Add the memory block to linux - if that fails, try to unplug. */
>- rc = virtio_mem_mb_add(vm, mb_id);
>+ rc = virtio_mem_sbm_add_mb(vm, mb_id);
> if (rc) {
> int new_state = VIRTIO_MEM_SBM_MB_UNUSED;
>
>- dev_err(&vm->vdev->dev,
>- "adding memory block %lu failed with %d\n", mb_id, rc);
>-
>- /*
>- * TODO: Linux MM does not properly clean up yet in all cases
>- * where adding of memory failed - especially on -ENOMEM.
>- */
> if (virtio_mem_sbm_unplug_sb(vm, mb_id, 0, count))
> new_state = VIRTIO_MEM_SBM_MB_PLUGGED;
> virtio_mem_sbm_set_mb_state(vm, mb_id, new_state);
>@@ -1411,7 +1450,7 @@ static int virtio_mem_sbm_unplug_any_sb_offline(struct virtio_mem *vm,
> VIRTIO_MEM_SBM_MB_UNUSED);
>
> mutex_unlock(&vm->hotplug_mutex);
>- rc = virtio_mem_mb_remove(vm, mb_id);
>+ rc = virtio_mem_sbm_remove_mb(vm, mb_id);
> BUG_ON(rc);
> mutex_lock(&vm->hotplug_mutex);
> }
>@@ -1504,7 +1543,7 @@ static int virtio_mem_sbm_unplug_any_sb_online(struct virtio_mem *vm,
> */
> if (virtio_mem_sbm_test_sb_unplugged(vm, mb_id, 0, vm->sbm.sbs_per_mb)) {
> mutex_unlock(&vm->hotplug_mutex);
>- rc = virtio_mem_mb_offline_and_remove(vm, mb_id);
>+ rc = virtio_mem_sbm_offline_and_remove_mb(vm, mb_id);
> mutex_lock(&vm->hotplug_mutex);
> if (!rc)
> virtio_mem_sbm_set_mb_state(vm, mb_id,
>@@ -1991,7 +2030,7 @@ static void virtio_mem_remove(struct virtio_device *vdev)
> */
> virtio_mem_sbm_for_each_mb(vm, mb_id,
> VIRTIO_MEM_SBM_MB_OFFLINE_PARTIAL) {
>- rc = virtio_mem_mb_remove(vm, mb_id);
>+ rc = virtio_mem_sbm_remove_mb(vm, mb_id);
> BUG_ON(rc);
> virtio_mem_sbm_set_mb_state(vm, mb_id,
> VIRTIO_MEM_SBM_MB_UNUSED);
>--
>2.26.2
--
Wei Yang
Help you, Help me
next prev parent reply other threads:[~2020-10-16 9:00 UTC|newest]
Thread overview: 109+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-12 12:52 [PATCH v1 00/29] virtio-mem: Big Block Mode (BBM) David Hildenbrand
2020-10-12 12:52 ` [PATCH v1 01/29] virtio-mem: determine nid only once using memory_add_physaddr_to_nid() David Hildenbrand
2020-10-15 3:56 ` Wei Yang
2020-10-15 19:26 ` Pankaj Gupta
2020-10-12 12:52 ` [PATCH v1 02/29] virtio-mem: simplify calculation in virtio_mem_mb_state_prepare_next_mb() David Hildenbrand
2020-10-15 4:02 ` Wei Yang
2020-10-15 8:00 ` David Hildenbrand
2020-10-15 10:00 ` Wei Yang
2020-10-15 10:01 ` David Hildenbrand
2020-10-15 20:24 ` Pankaj Gupta
2020-10-16 9:00 ` David Hildenbrand
2020-10-12 12:52 ` [PATCH v1 03/29] virtio-mem: simplify MAX_ORDER - 1 / pageblock_order handling David Hildenbrand
2020-10-15 7:06 ` Wei Yang
2020-10-12 12:52 ` [PATCH v1 04/29] virtio-mem: drop rc2 in virtio_mem_mb_plug_and_add() David Hildenbrand
2020-10-12 13:09 ` Pankaj Gupta
2020-10-15 7:14 ` Wei Yang
2020-10-12 12:52 ` [PATCH v1 05/29] virtio-mem: generalize check for added memory David Hildenbrand
2020-10-15 8:28 ` Wei Yang
2020-10-15 8:50 ` David Hildenbrand
2020-10-16 2:16 ` Wei Yang
2020-10-16 9:11 ` David Hildenbrand
2020-10-16 10:02 ` Wei Yang
2020-10-16 10:32 ` David Hildenbrand
2020-10-16 22:38 ` Wei Yang
2020-10-17 7:39 ` David Hildenbrand
2020-10-18 12:27 ` Wei Yang
2020-10-16 22:39 ` Wei Yang
2020-10-12 12:53 ` [PATCH v1 06/29] virtio-mem: generalize virtio_mem_owned_mb() David Hildenbrand
2020-10-15 8:32 ` Wei Yang
2020-10-15 8:37 ` David Hildenbrand
2020-10-15 20:30 ` Pankaj Gupta
2020-10-12 12:53 ` [PATCH v1 07/29] virtio-mem: generalize virtio_mem_overlaps_range() David Hildenbrand
2020-10-20 9:22 ` Pankaj Gupta
2020-10-12 12:53 ` [PATCH v1 08/29] virtio-mem: drop last_mb_id David Hildenbrand
2020-10-15 8:35 ` Wei Yang
2020-10-15 20:32 ` Pankaj Gupta
2020-10-12 12:53 ` [PATCH v1 09/29] virtio-mem: don't always trigger the workqueue when offlining memory David Hildenbrand
2020-10-16 4:03 ` Wei Yang
2020-10-16 9:18 ` David Hildenbrand
2020-10-18 3:57 ` Wei Yang
2020-10-19 9:04 ` David Hildenbrand
2020-10-20 0:41 ` Wei Yang
2020-10-20 9:09 ` David Hildenbrand
2020-10-12 12:53 ` [PATCH v1 10/29] virtio-mem: generalize handling when memory is getting onlined deferred David Hildenbrand
2020-10-12 12:53 ` [PATCH v1 11/29] virtio-mem: use "unsigned long" for nr_pages when fake onlining/offlining David Hildenbrand
2020-10-15 20:31 ` Pankaj Gupta
2020-10-16 6:11 ` Wei Yang
2020-10-12 12:53 ` [PATCH v1 12/29] virtio-mem: factor out fake-offlining into virtio_mem_fake_offline() David Hildenbrand
2020-10-16 6:24 ` Wei Yang
2020-10-20 9:31 ` Pankaj Gupta
2020-10-12 12:53 ` [PATCH v1 13/29] virtio-mem: factor out handling of fake-offline pages in memory notifier David Hildenbrand
2020-10-16 7:15 ` Wei Yang
2020-10-16 8:00 ` Wei Yang
2020-10-16 8:57 ` David Hildenbrand
2020-10-18 12:37 ` Wei Yang
2020-10-18 12:38 ` Wei Yang
2020-10-12 12:53 ` [PATCH v1 14/29] virtio-mem: retry fake-offlining via alloc_contig_range() on ZONE_MOVABLE David Hildenbrand
2020-10-12 12:53 ` [PATCH v1 15/29] virito-mem: document Sub Block Mode (SBM) David Hildenbrand
2020-10-15 9:33 ` David Hildenbrand
2020-10-20 9:38 ` Pankaj Gupta
2020-10-16 8:03 ` Wei Yang
2020-10-12 12:53 ` [PATCH v1 16/29] virtio-mem: memory block states are specific to " David Hildenbrand
2020-10-16 8:40 ` Wei Yang
2020-10-16 8:43 ` Wei Yang
2020-10-20 9:48 ` Pankaj Gupta
2020-10-12 12:53 ` [PATCH v1 17/29] virito-mem: subblock " David Hildenbrand
2020-10-16 8:43 ` Wei Yang
2020-10-20 9:54 ` Pankaj Gupta
2020-10-12 12:53 ` [PATCH v1 18/29] virtio-mem: factor out calculation of the bit number within the sb_states bitmap David Hildenbrand
2020-10-16 8:46 ` Wei Yang
2020-10-20 9:58 ` Pankaj Gupta
2020-10-12 12:53 ` [PATCH v1 19/29] virito-mem: existing (un)plug functions are specific to Sub Block Mode (SBM) David Hildenbrand
2020-10-16 8:49 ` Wei Yang
2020-10-12 12:53 ` [PATCH v1 20/29] virtio-mem: nb_sb_per_mb and subblock_size " David Hildenbrand
2020-10-16 8:51 ` Wei Yang
2020-10-16 8:53 ` Wei Yang
2020-10-16 13:17 ` David Hildenbrand
2020-10-18 12:41 ` Wei Yang
2020-10-19 11:57 ` David Hildenbrand
2020-10-12 12:53 ` [PATCH v1 21/29] virtio-mem: memory notifier callbacks " David Hildenbrand
2020-10-19 1:57 ` Wei Yang
2020-10-19 10:22 ` David Hildenbrand
2020-10-12 12:53 ` [PATCH v1 22/29] virtio-mem: memory block ids " David Hildenbrand
2020-10-16 8:54 ` Wei Yang
2020-10-12 12:53 ` [PATCH v1 23/29] virtio-mem: factor out adding/removing memory from Linux David Hildenbrand
2020-10-16 8:59 ` Wei Yang [this message]
2020-10-12 12:53 ` [PATCH v1 24/29] virtio-mem: print debug messages from virtio_mem_send_*_request() David Hildenbrand
2020-10-16 9:07 ` Wei Yang
2020-10-12 12:53 ` [PATCH v1 25/29] virtio-mem: Big Block Mode (BBM) memory hotplug David Hildenbrand
2020-10-16 9:38 ` Wei Yang
2020-10-16 13:13 ` David Hildenbrand
2020-10-19 2:26 ` Wei Yang
2020-10-19 9:15 ` David Hildenbrand
2020-10-12 12:53 ` [PATCH v1 26/29] virtio-mem: allow to force Big Block Mode (BBM) and set the big block size David Hildenbrand
2020-10-12 12:53 ` [PATCH v1 27/29] mm/memory_hotplug: extend offline_and_remove_memory() to handle more than one memory block David Hildenbrand
2020-10-15 13:08 ` Michael S. Tsirkin
2020-10-19 3:22 ` Wei Yang
2020-10-12 12:53 ` [PATCH v1 28/29] virtio-mem: Big Block Mode (BBM) - basic memory hotunplug David Hildenbrand
2020-10-19 3:48 ` Wei Yang
2020-10-19 9:12 ` David Hildenbrand
2020-10-12 12:53 ` [PATCH v1 29/29] virtio-mem: Big Block Mode (BBM) - safe " David Hildenbrand
2020-10-19 7:54 ` Wei Yang
2020-10-19 8:50 ` David Hildenbrand
2020-10-20 0:23 ` Wei Yang
2020-10-20 0:24 ` Wei Yang
2020-10-18 12:49 ` [PATCH v1 00/29] virtio-mem: Big Block Mode (BBM) Wei Yang
2020-10-18 16:35 ` David Hildenbrand
2020-10-18 15:29 ` Michael S. Tsirkin
2020-10-18 16:34 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201016085953.GF44269@L-31X9LVDL-1304.local \
--to=richard.weiyang@linux.alibaba.com \
--cc=akpm@linux-foundation.org \
--cc=david@redhat.com \
--cc=jasowang@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mst@redhat.com \
--cc=pankaj.gupta.linux@gmail.com \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).