From: David Hildenbrand <david@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Pankaj Gupta" <pankaj.gupta.linux@gmail.com>,
"Daniel P . Berrangé" <berrange@redhat.com>,
"Eduardo Habkost" <ehabkost@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"David Hildenbrand" <david@redhat.com>,
"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
"Pankaj Gupta" <pankaj.gupta@ionos.com>,
"Igor Mammedov" <imammedo@redhat.com>,
"Paolo Bonzini" <pbonzini@redhat.com>
Subject: [PATCH v3 4/7] util/oslib-posix: Don't create too many threads with small memory or little pages
Date: Mon, 16 Aug 2021 11:47:36 +0200 [thread overview]
Message-ID: <20210816094739.21970-5-david@redhat.com> (raw)
In-Reply-To: <20210816094739.21970-1-david@redhat.com>
Let's limit the number of threads to something sane, especially that
- We don't have more threads than the number of pages we have
- We don't have threads that initialize small (< 64 MiB) memory
Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
util/oslib-posix.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index cf2ead54ad..67c08a425e 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -40,6 +40,7 @@
#include <libgen.h>
#include "qemu/cutils.h"
#include "qemu/compiler.h"
+#include "qemu/units.h"
#ifdef CONFIG_LINUX
#include <sys/syscall.h>
@@ -525,7 +526,8 @@ static void *do_madv_populate_write_pages(void *arg)
return (void *)(uintptr_t)ret;
}
-static inline int get_memset_num_threads(int smp_cpus)
+static inline int get_memset_num_threads(size_t hpagesize, size_t numpages,
+ int smp_cpus)
{
long host_procs = sysconf(_SC_NPROCESSORS_ONLN);
int ret = 1;
@@ -533,6 +535,12 @@ static inline int get_memset_num_threads(int smp_cpus)
if (host_procs > 0) {
ret = MIN(MIN(host_procs, MAX_MEM_PREALLOC_THREAD_COUNT), smp_cpus);
}
+
+ /* Especially with gigantic pages, don't create more threads than pages. */
+ ret = MIN(ret, numpages);
+ /* Don't start threads to prealloc comparatively little memory. */
+ ret = MIN(ret, MAX(1, hpagesize * numpages / (64 * MiB)));
+
/* In case sysconf() fails, we fall back to single threaded */
return ret;
}
@@ -542,7 +550,7 @@ static int touch_all_pages(char *area, size_t hpagesize, size_t numpages,
{
static gsize initialized = 0;
MemsetContext context = {
- .num_threads = get_memset_num_threads(smp_cpus),
+ .num_threads = get_memset_num_threads(hpagesize, numpages, smp_cpus),
};
size_t numpages_per_thread, leftover;
void *(*touch_fn)(void *);
--
2.31.1
next prev parent reply other threads:[~2021-08-16 9:49 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-16 9:47 [PATCH v3 0/7] util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc() David Hildenbrand
2021-08-16 9:47 ` [PATCH v3 1/7] util/oslib-posix: Let touch_all_pages() return an error David Hildenbrand
2021-09-28 16:21 ` Daniel P. Berrangé
2021-08-16 9:47 ` [PATCH v3 2/7] util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc() David Hildenbrand
2021-09-28 16:45 ` Daniel P. Berrangé
2021-08-16 9:47 ` [PATCH v3 3/7] util/oslib-posix: Introduce and use MemsetContext for touch_all_pages() David Hildenbrand
2021-08-16 9:47 ` David Hildenbrand [this message]
2021-09-28 16:47 ` [PATCH v3 4/7] util/oslib-posix: Don't create too many threads with small memory or little pages Daniel P. Berrangé
2021-08-16 9:47 ` [PATCH v3 5/7] util/oslib-posix: Avoid creating a single thread with MADV_POPULATE_WRITE David Hildenbrand
2021-09-28 16:52 ` Daniel P. Berrangé
2021-08-16 9:47 ` [PATCH v3 6/7] util/oslib-posix: Support concurrent os_mem_prealloc() invocation David Hildenbrand
2021-09-28 16:56 ` Daniel P. Berrangé
2021-09-28 17:01 ` Daniel P. Berrangé
2021-09-28 17:17 ` David Hildenbrand
2021-08-16 9:47 ` [PATCH v3 7/7] util/oslib-posix: Forward SIGBUS to MCE handler under Linux David Hildenbrand
2021-09-28 16:59 ` Daniel P. Berrangé
2021-09-28 17:13 ` David Hildenbrand
2021-09-28 14:01 ` [PATCH v3 0/7] util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc() David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210816094739.21970-5-david@redhat.com \
--to=david@redhat.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=ehabkost@redhat.com \
--cc=imammedo@redhat.com \
--cc=mst@redhat.com \
--cc=pankaj.gupta.linux@gmail.com \
--cc=pankaj.gupta@ionos.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).