From: David Hildenbrand <david@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Pankaj Gupta" <pankaj.gupta.linux@gmail.com>,
"Daniel P . Berrangé" <berrange@redhat.com>,
"Eduardo Habkost" <ehabkost@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"David Hildenbrand" <david@redhat.com>,
"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
"Pankaj Gupta" <pankaj.gupta@ionos.com>,
"Igor Mammedov" <imammedo@redhat.com>,
"Paolo Bonzini" <pbonzini@redhat.com>
Subject: [PATCH v4 4/7] util/oslib-posix: Don't create too many threads with small memory or little pages
Date: Mon, 4 Oct 2021 14:02:05 +0200 [thread overview]
Message-ID: <20211004120208.7409-5-david@redhat.com> (raw)
In-Reply-To: <20211004120208.7409-1-david@redhat.com>
Let's limit the number of threads to something sane, especially that
- We don't have more threads than the number of pages we have
- We don't have threads that initialize small (< 64 MiB) memory
Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
util/oslib-posix.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index cf2ead54ad..67c08a425e 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -40,6 +40,7 @@
#include <libgen.h>
#include "qemu/cutils.h"
#include "qemu/compiler.h"
+#include "qemu/units.h"
#ifdef CONFIG_LINUX
#include <sys/syscall.h>
@@ -525,7 +526,8 @@ static void *do_madv_populate_write_pages(void *arg)
return (void *)(uintptr_t)ret;
}
-static inline int get_memset_num_threads(int smp_cpus)
+static inline int get_memset_num_threads(size_t hpagesize, size_t numpages,
+ int smp_cpus)
{
long host_procs = sysconf(_SC_NPROCESSORS_ONLN);
int ret = 1;
@@ -533,6 +535,12 @@ static inline int get_memset_num_threads(int smp_cpus)
if (host_procs > 0) {
ret = MIN(MIN(host_procs, MAX_MEM_PREALLOC_THREAD_COUNT), smp_cpus);
}
+
+ /* Especially with gigantic pages, don't create more threads than pages. */
+ ret = MIN(ret, numpages);
+ /* Don't start threads to prealloc comparatively little memory. */
+ ret = MIN(ret, MAX(1, hpagesize * numpages / (64 * MiB)));
+
/* In case sysconf() fails, we fall back to single threaded */
return ret;
}
@@ -542,7 +550,7 @@ static int touch_all_pages(char *area, size_t hpagesize, size_t numpages,
{
static gsize initialized = 0;
MemsetContext context = {
- .num_threads = get_memset_num_threads(smp_cpus),
+ .num_threads = get_memset_num_threads(hpagesize, numpages, smp_cpus),
};
size_t numpages_per_thread, leftover;
void *(*touch_fn)(void *);
--
2.31.1
next prev parent reply other threads:[~2021-10-04 12:12 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-04 12:02 [PATCH v4 0/7] util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc() David Hildenbrand
2021-10-04 12:02 ` [PATCH v4 1/7] util/oslib-posix: Let touch_all_pages() return an error David Hildenbrand
2021-10-04 12:02 ` [PATCH v4 2/7] util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc() David Hildenbrand
2021-10-04 12:02 ` [PATCH v4 3/7] util/oslib-posix: Introduce and use MemsetContext for touch_all_pages() David Hildenbrand
2021-10-07 10:05 ` Dr. David Alan Gilbert
2021-10-07 10:12 ` David Hildenbrand
2021-10-04 12:02 ` David Hildenbrand [this message]
2021-10-04 12:02 ` [PATCH v4 5/7] util/oslib-posix: Avoid creating a single thread with MADV_POPULATE_WRITE David Hildenbrand
2021-10-04 12:02 ` [PATCH v4 6/7] util/oslib-posix: Support concurrent os_mem_prealloc() invocation David Hildenbrand
2021-10-04 12:02 ` [PATCH v4 7/7] util/oslib-posix: Forward SIGBUS to MCE handler under Linux David Hildenbrand
2021-10-11 16:46 ` [PATCH v4 0/7] util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc() David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211004120208.7409-5-david@redhat.com \
--to=david@redhat.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=ehabkost@redhat.com \
--cc=imammedo@redhat.com \
--cc=mst@redhat.com \
--cc=pankaj.gupta.linux@gmail.com \
--cc=pankaj.gupta@ionos.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.