From: Jason Gunthorpe <jgg@ziepe.ca>
To: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: lsf-pc@lists.linuxfoundation.org, linux-mm@kvack.org,
Dan Williams <dan.j.williams@intel.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
Tim Chen <tim.c.chen@linux.intel.com>,
Mike Kravetz <mike.kravetz@oracle.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
Steffen Klassert <steffen.klassert@secunet.com>,
Tejun Heo <tj@kernel.org>, Peter Zijlstra <peterz@infradead.org>,
Alex Williamson <alex.williamson@redhat.com>
Subject: Re: [LSF/MM/BPF TOPIC] kernel multithreading with padata
Date: Wed, 12 Feb 2020 19:31:00 -0400 [thread overview]
Message-ID: <20200212233100.GF31668@ziepe.ca> (raw)
In-Reply-To: <20200212224731.kmss6o6agekkg3mw@ca-dmjordan1.us.oracle.com>
On Wed, Feb 12, 2020 at 05:47:31PM -0500, Daniel Jordan wrote:
> padata has been undergoing some surgery over the last year[0] and now seems
> ready for another enhancement: splitting up and multithreading CPU-intensive
> kernel work.
>
> Quoting from an earlier series[1], the problem I'm trying to solve is
>
> A single CPU can spend an excessive amount of time in the kernel operating
> on large amounts of data. Often these situations arise during initialization-
> and destruction-related tasks, where the data involved scales with system
> size. These long-running jobs can slow startup and shutdown of applications
> and the system itself while extra CPUs sit idle.
>
> Here are the current consumers:
>
> - struct page init (boot, hotplug, pmem)
> - VFIO page pinning (kvm guest init)
> - fallocating a hugetlb file (database shared memory init)
>
> On a large-memory server, DRAM page init is ~23% of kernel boot (3.5s/15.2s),
> and it takes over a minute to start a VFIO-enabled kvm guest or fallocate a
> hugetlb file that occupy a significant fraction of memory. This work results
> in 7-20x speedups and is currently increasing the uptime of our production
> kernels.
>
> Future areas include munmap/exit, umount, and __ib_umem_release. Some of these
> need coarse locks broken up for multithreading (zone->lock, lru_lock).
I'm aware of this ib_umem_release request, it would be interesting to
see, the main workload here is put_page and dma_unmap
Jason
next prev parent reply other threads:[~2020-02-12 23:31 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-12 22:47 [LSF/MM/BPF TOPIC] kernel multithreading with padata Daniel Jordan
2020-02-12 23:31 ` Jason Gunthorpe [this message]
2020-02-13 16:13 ` Daniel Jordan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200212233100.GF31668@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=alex.williamson@redhat.com \
--cc=dan.j.williams@intel.com \
--cc=daniel.m.jordan@oracle.com \
--cc=dave.hansen@linux.intel.com \
--cc=herbert@gondor.apana.org.au \
--cc=linux-mm@kvack.org \
--cc=lsf-pc@lists.linuxfoundation.org \
--cc=mike.kravetz@oracle.com \
--cc=peterz@infradead.org \
--cc=steffen.klassert@secunet.com \
--cc=tim.c.chen@linux.intel.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).