From: David Hildenbrand <david@redhat.com> To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtio-dev@lists.oasis-open.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, Michal Hocko <mhocko@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, "Michael S . Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>, Alexander Viro <viro@zeniv.linux.org.uk>, Jason Wang <jasowang@redhat.com>, Oscar Salvador <osalvador@suse.de>, Igor Mammedov <imammedo@redhat.com>, Dave Young <dyoung@redhat.com>, Dan Williams <dan.j.williams@intel.com>, Pavel Tatashin <pasha.tatashin@soleen.com>, Stefan Hajnoczi <stefanha@redhat.com>, Vlastimil Babka <vbabka@suse.cz>, linux-fsdevel@vger.kernel.org Subject: [PATCH RFC v4 13/13] virtio-mem: Drop slab objects when unplug continues to fail Date: Thu, 12 Dec 2019 18:11:37 +0100 [thread overview] Message-ID: <20191212171137.13872-14-david@redhat.com> (raw) In-Reply-To: <20191212171137.13872-1-david@redhat.com> Start dropping slab objects after 30 minutes and repeat every 30 minutes in case we can't unplug more memory using alloc_contig_range(). Log messages and make it configurable. Enable dropping slab objects as default (especially, reclaimable slab objects that are not movable). In the future, we might want to implement+use drop_slab_range(), which will also come in handy for other users (e.g., offlining, gigantic huge pages). Cc: Alexander Viro <viro@zeniv.linux.org.uk> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Oscar Salvador <osalvador@suse.de> Cc: Igor Mammedov <imammedo@redhat.com> Cc: Dave Young <dyoung@redhat.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Pavel Tatashin <pasha.tatashin@soleen.com> Cc: Stefan Hajnoczi <stefanha@redhat.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: linux-fsdevel@vger.kernel.org Signed-off-by: David Hildenbrand <david@redhat.com> --- drivers/virtio/virtio_mem.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 3a57434f92ed..8f25f7453a08 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -25,6 +25,11 @@ static bool unplug_online = true; module_param(unplug_online, bool, 0644); MODULE_PARM_DESC(unplug_online, "Try to unplug online memory"); +static bool drop_slab_objects = true; +module_param(drop_slab_objects, bool, 0644); +MODULE_PARM_DESC(drop_slab_objects, + "Drop slab objects when unplug continues to fail"); + enum virtio_mem_mb_state { /* Unplugged, not added to Linux. Can be reused later. */ VIRTIO_MEM_MB_STATE_UNUSED = 0, @@ -1384,6 +1389,7 @@ static int virtio_mem_mb_unplug_any_sb_online(struct virtio_mem *vm, static int virtio_mem_unplug_request(struct virtio_mem *vm, uint64_t diff) { uint64_t nb_sb = diff / vm->subblock_size; + bool retried = false; unsigned long mb_id; int rc; @@ -1421,6 +1427,7 @@ static int virtio_mem_unplug_request(struct virtio_mem *vm, uint64_t diff) return 0; } +retry_locked: /* Try to unplug subblocks of partially plugged online blocks. */ virtio_mem_for_each_mb_state(vm, mb_id, VIRTIO_MEM_MB_STATE_ONLINE_PARTIAL) { @@ -1445,6 +1452,29 @@ static int virtio_mem_unplug_request(struct virtio_mem *vm, uint64_t diff) } mutex_unlock(&vm->hotplug_mutex); + + /* + * If we can't unplug the requested amount of memory for a long time, + * start freeing up memory in caches. This might harm performance, + * is configurable, and we log a message. Retry imemdiately a second + * time - then wait another VIRTIO_MEM_RETRY_TIMER_MAX_MS. + */ + if (nb_sb && !retried && drop_slab_objects && + vm->retry_timer_ms == VIRTIO_MEM_RETRY_TIMER_MAX_MS) { + if (vm->nid == NUMA_NO_NODE) { + dev_info(&vm->vdev->dev, "dropping all slab objects\n"); + drop_slab(); + } else { + dev_info(&vm->vdev->dev, + "dropping all slab objects on node=%d\n", + vm->nid); + drop_slab_node(vm->nid); + } + retried = true; + mutex_lock(&vm->hotplug_mutex); + goto retry_locked; + } + return nb_sb ? -EBUSY : 0; out_unlock: mutex_unlock(&vm->hotplug_mutex); -- 2.23.0
WARNING: multiple messages have this Message-ID (diff)
From: David Hildenbrand <david@redhat.com> To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtio-dev@lists.oasis-open.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, Michal Hocko <mhocko@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, "Michael S . Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>, Alexander Viro <viro@zeniv.linux.org.uk>, Jason Wang <jasowang@redhat.com>, Oscar Salvador <osalvador@suse.de>, Igor Mammedov <imammedo@redhat.com>, Dave Young <dyoung@redhat.com>, Dan Williams <dan.j.williams@intel.com>, Pavel Tatashin <pasha.tatashin@soleen.com>, Stefan Hajnoczi <stefanha@redhat.com>, Vlastimil Babka <vbabka@suse.cz>, linux-fsdevel@vger.kernel.org Subject: [virtio-dev] [PATCH RFC v4 13/13] virtio-mem: Drop slab objects when unplug continues to fail Date: Thu, 12 Dec 2019 18:11:37 +0100 [thread overview] Message-ID: <20191212171137.13872-14-david@redhat.com> (raw) In-Reply-To: <20191212171137.13872-1-david@redhat.com> Start dropping slab objects after 30 minutes and repeat every 30 minutes in case we can't unplug more memory using alloc_contig_range(). Log messages and make it configurable. Enable dropping slab objects as default (especially, reclaimable slab objects that are not movable). In the future, we might want to implement+use drop_slab_range(), which will also come in handy for other users (e.g., offlining, gigantic huge pages). Cc: Alexander Viro <viro@zeniv.linux.org.uk> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Oscar Salvador <osalvador@suse.de> Cc: Igor Mammedov <imammedo@redhat.com> Cc: Dave Young <dyoung@redhat.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Pavel Tatashin <pasha.tatashin@soleen.com> Cc: Stefan Hajnoczi <stefanha@redhat.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: linux-fsdevel@vger.kernel.org Signed-off-by: David Hildenbrand <david@redhat.com> --- drivers/virtio/virtio_mem.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 3a57434f92ed..8f25f7453a08 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -25,6 +25,11 @@ static bool unplug_online = true; module_param(unplug_online, bool, 0644); MODULE_PARM_DESC(unplug_online, "Try to unplug online memory"); +static bool drop_slab_objects = true; +module_param(drop_slab_objects, bool, 0644); +MODULE_PARM_DESC(drop_slab_objects, + "Drop slab objects when unplug continues to fail"); + enum virtio_mem_mb_state { /* Unplugged, not added to Linux. Can be reused later. */ VIRTIO_MEM_MB_STATE_UNUSED = 0, @@ -1384,6 +1389,7 @@ static int virtio_mem_mb_unplug_any_sb_online(struct virtio_mem *vm, static int virtio_mem_unplug_request(struct virtio_mem *vm, uint64_t diff) { uint64_t nb_sb = diff / vm->subblock_size; + bool retried = false; unsigned long mb_id; int rc; @@ -1421,6 +1427,7 @@ static int virtio_mem_unplug_request(struct virtio_mem *vm, uint64_t diff) return 0; } +retry_locked: /* Try to unplug subblocks of partially plugged online blocks. */ virtio_mem_for_each_mb_state(vm, mb_id, VIRTIO_MEM_MB_STATE_ONLINE_PARTIAL) { @@ -1445,6 +1452,29 @@ static int virtio_mem_unplug_request(struct virtio_mem *vm, uint64_t diff) } mutex_unlock(&vm->hotplug_mutex); + + /* + * If we can't unplug the requested amount of memory for a long time, + * start freeing up memory in caches. This might harm performance, + * is configurable, and we log a message. Retry imemdiately a second + * time - then wait another VIRTIO_MEM_RETRY_TIMER_MAX_MS. + */ + if (nb_sb && !retried && drop_slab_objects && + vm->retry_timer_ms == VIRTIO_MEM_RETRY_TIMER_MAX_MS) { + if (vm->nid == NUMA_NO_NODE) { + dev_info(&vm->vdev->dev, "dropping all slab objects\n"); + drop_slab(); + } else { + dev_info(&vm->vdev->dev, + "dropping all slab objects on node=%d\n", + vm->nid); + drop_slab_node(vm->nid); + } + retried = true; + mutex_lock(&vm->hotplug_mutex); + goto retry_locked; + } + return nb_sb ? -EBUSY : 0; out_unlock: mutex_unlock(&vm->hotplug_mutex); -- 2.23.0 --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
next prev parent reply other threads:[~2019-12-12 17:13 UTC|newest] Thread overview: 116+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-12-12 17:11 [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 01/13] ACPI: NUMA: export pxm_to_node David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 21:43 ` Rafael J. Wysocki 2019-12-13 9:41 ` David Hildenbrand 2019-12-13 9:41 ` David Hildenbrand 2019-12-13 9:41 ` [virtio-dev] " David Hildenbrand 2019-12-13 9:47 ` Rafael J. Wysocki 2019-12-13 9:47 ` Rafael J. Wysocki 2019-12-13 9:47 ` Rafael J. Wysocki 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 02/13] virtio-mem: Paravirtualized memory hotplug David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 03/13] virtio-mem: Paravirtualized memory hotunplug part 1 David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 04/13] mm: Export alloc_contig_range() / free_contig_range() David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 05/13] virtio-mem: Paravirtualized memory hotunplug part 2 David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2020-02-25 18:26 ` Alexander Duyck 2020-02-25 18:26 ` [virtio-dev] " Alexander Duyck 2020-02-25 18:26 ` Alexander Duyck 2020-02-25 18:49 ` David Hildenbrand 2020-02-25 18:49 ` [virtio-dev] " David Hildenbrand 2020-02-25 18:49 ` David Hildenbrand 2020-02-25 21:46 ` Alexander Duyck 2020-02-25 21:46 ` [virtio-dev] " Alexander Duyck 2020-02-25 21:46 ` Alexander Duyck 2020-02-25 22:19 ` David Hildenbrand 2020-02-25 22:19 ` [virtio-dev] " David Hildenbrand 2020-02-25 22:19 ` David Hildenbrand 2020-02-26 16:27 ` Alexander Duyck 2020-02-26 16:27 ` [virtio-dev] " Alexander Duyck 2020-02-26 16:27 ` Alexander Duyck 2019-12-12 17:11 ` [PATCH RFC v4 07/13] virtio-mem: Allow to offline partially unplugged memory blocks David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 08/13] mm/memory_hotplug: Introduce offline_and_remove_memory() David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2020-02-25 14:11 ` Michal Hocko 2020-02-25 14:27 ` David Hildenbrand 2020-02-25 14:27 ` [virtio-dev] " David Hildenbrand 2020-03-02 12:48 ` Michal Hocko 2020-03-02 12:53 ` David Hildenbrand 2020-03-02 12:53 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 09/13] virtio-mem: Offline and remove completely unplugged memory blocks David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 10/13] virtio-mem: Better retry handling David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` [PATCH RFC v4 11/13] mm/vmscan: Move count_vm_event(DROP_SLAB) into drop_slab() David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2020-02-25 14:13 ` Michal Hocko 2019-12-12 17:11 ` [PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node() David Hildenbrand 2019-12-12 17:11 ` [virtio-dev] " David Hildenbrand 2020-02-25 14:58 ` Michal Hocko 2020-02-25 15:09 ` David Hildenbrand 2020-02-25 15:09 ` [virtio-dev] " David Hildenbrand 2020-02-25 17:06 ` Michal Hocko 2020-02-25 17:23 ` David Hildenbrand 2020-02-25 17:23 ` [virtio-dev] " David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand [this message] 2019-12-12 17:11 ` [virtio-dev] [PATCH RFC v4 13/13] virtio-mem: Drop slab objects when unplug continues to fail David Hildenbrand 2019-12-12 17:11 ` David Hildenbrand 2019-12-13 20:15 ` [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory Konrad Rzeszutek Wilk 2019-12-13 20:15 ` Konrad Rzeszutek Wilk 2019-12-16 11:03 ` David Hildenbrand 2019-12-16 11:03 ` David Hildenbrand 2019-12-16 11:03 ` [virtio-dev] " David Hildenbrand 2019-12-24 6:58 ` teawater 2019-12-24 9:28 ` David Hildenbrand 2019-12-24 9:28 ` [virtio-dev] " David Hildenbrand 2019-12-24 9:28 ` David Hildenbrand 2020-01-09 13:48 ` David Hildenbrand 2020-01-09 13:48 ` [virtio-dev] " David Hildenbrand 2020-01-29 9:41 ` David Hildenbrand 2020-01-29 9:41 ` [virtio-dev] " David Hildenbrand 2020-01-29 9:41 ` David Hildenbrand 2020-02-25 9:58 ` David Hildenbrand 2020-02-25 9:58 ` [virtio-dev] " David Hildenbrand 2020-02-25 9:58 ` David Hildenbrand 2020-06-05 8:55 ` Alex Shi 2020-06-05 8:55 ` Alex Shi 2020-06-05 9:08 ` David Hildenbrand 2020-06-05 9:08 ` [virtio-dev] " David Hildenbrand 2020-06-05 9:08 ` David Hildenbrand 2020-06-05 9:36 ` David Hildenbrand 2020-06-05 9:36 ` [virtio-dev] " David Hildenbrand 2020-06-05 9:36 ` David Hildenbrand 2020-06-05 10:05 ` David Hildenbrand 2020-06-05 10:05 ` [virtio-dev] " David Hildenbrand 2020-06-05 10:05 ` David Hildenbrand 2020-06-05 10:46 ` Alex Shi 2020-06-05 10:46 ` Alex Shi 2020-06-05 12:18 ` David Hildenbrand 2020-06-05 12:18 ` [virtio-dev] " David Hildenbrand 2020-06-05 12:18 ` David Hildenbrand 2020-06-09 3:05 ` Alex Shi 2020-06-09 3:05 ` Alex Shi 2020-06-05 10:08 ` Alex Shi 2020-06-05 10:08 ` Alex Shi 2020-06-05 10:06 ` Alex Shi 2020-06-05 10:06 ` Alex Shi 2020-01-09 13:48 ` David Hildenbrand
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191212171137.13872-14-david@redhat.com \ --to=david@redhat.com \ --cc=akpm@linux-foundation.org \ --cc=dan.j.williams@intel.com \ --cc=dyoung@redhat.com \ --cc=imammedo@redhat.com \ --cc=jasowang@redhat.com \ --cc=kvm@vger.kernel.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mhocko@kernel.org \ --cc=mst@redhat.com \ --cc=osalvador@suse.de \ --cc=pasha.tatashin@soleen.com \ --cc=stefanha@redhat.com \ --cc=vbabka@suse.cz \ --cc=viro@zeniv.linux.org.uk \ --cc=virtio-dev@lists.oasis-open.org \ --cc=virtualization@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.