From: David Rientjes <rientjes@google.com>
To: Hillf Danton <hdanton@sina.com>
Cc: Christoph Hellwig <hch@lst.de>,
Tom Lendacky <thomas.lendacky@amd.com>,
"Singh, Brijesh" <brijesh.singh@amd.com>,
"Grimm, Jon" <jon.grimm@amd.com>, Joerg Roedel <joro@8bytes.org>,
linux <linux-kernel@vger.kernel.org>,
iommu <iommu@lists.linux-foundation.org>
Subject: Re: [rfc v2 3/6] dma-pool: dynamically expanding atomic pools
Date: Fri, 10 Apr 2020 12:37:20 -0700 (PDT) [thread overview]
Message-ID: <alpine.DEB.2.21.2004101231240.249689@chino.kir.corp.google.com> (raw)
In-Reply-To: <20200410145520.17864-1-hdanton@sina.com>
On Fri, 10 Apr 2020, Hillf Danton wrote:
>
> On Wed, 8 Apr 2020 14:21:06 -0700 (PDT) David Rientjes wrote:
> >
> > When an atomic pool becomes fully depleted because it is now relied upon
> > for all non-blocking allocations through the DMA API, allow background
> > expansion of each pool by a kworker.
> >
> > When an atomic pool has less than the default size of memory left, kick
> > off a kworker to dynamically expand the pool in the background. The pool
> > is doubled in size, up to MAX_ORDER-1. If memory cannot be allocated at
> > the requested order, smaller allocation(s) are attempted.
> >
> What is proposed looks like a path of single lane without how to
> dynamically shrink the pool taken into account. Thus the risk may
> rise in corner cases where pools are over-expanded in long run
> after one-off peak allocation requests.
>
To us, this is actually a benefit: we prefer the peak size to be
maintained so that we do not need to dynamic resize the pool later at the
cost of throughput. Genpool also does not have great support for
scavenging and freeing unused chunks.
Perhaps we could enforce a maximum size on the pools just as we allow the
default size to be defined by coherent_size= on the command line. Our use
case would not set this, however, since we have not seen egregious genpool
sizes as the result of non-blockable DMA allocations (perhaps the drivers
we use just play friendlier and you have seen excessive usage?).
I'll rely on Christoph to determine whether it makes sense to add some
periodic scavening of the atomic pools, whether that's needed for this to
be merged, or wheter we should enforce some maximum pool size.
next prev parent reply other threads:[~2020-04-10 19:37 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-01 1:54 [rfc] dma-mapping: preallocate unencrypted DMA atomic pool David Rientjes
2020-01-06 17:34 ` Robin Murphy
2020-01-07 10:54 ` Christoph Hellwig
2020-01-07 19:57 ` David Rientjes
2020-01-09 14:31 ` Christoph Hellwig
2020-01-09 18:58 ` David Rientjes
2020-01-17 15:28 ` Tom Lendacky
2020-02-28 9:27 ` David Rientjes
2020-03-02 0:05 ` [rfc 0/6] unencrypted atomic DMA pools with dynamic expansion David Rientjes
2020-03-02 0:05 ` [rfc 1/6] dma-mapping: pass device to atomic allocation functions David Rientjes
2020-03-02 0:05 ` [rfc 2/6] dma-remap: add additional atomic pools to map to gfp mask David Rientjes
2020-03-05 15:42 ` Christoph Hellwig
2020-03-02 0:05 ` [rfc 3/6] dma-remap: wire up the atomic pools depending on " David Rientjes
2020-03-05 15:43 ` Christoph Hellwig
2020-03-02 0:05 ` [rfc 4/6] dma-remap: dynamically expanding atomic pools David Rientjes
2020-03-03 22:29 ` David Rientjes
2020-03-02 0:05 ` [rfc 5/6] dma-direct: atomic allocations must come from unencrypted pools David Rientjes
2020-03-05 15:44 ` Christoph Hellwig
2020-03-07 0:36 ` David Rientjes
2020-03-10 18:47 ` Christoph Hellwig
2020-03-02 0:05 ` [rfc 6/6] dma-remap: double the default DMA coherent pool size David Rientjes
2020-03-05 15:45 ` Christoph Hellwig
2020-04-08 21:21 ` [rfc v2 0/6] unencrypted atomic DMA pools with dynamic expansion David Rientjes
2020-04-08 21:21 ` [rfc v2 1/6] dma-remap: separate DMA atomic pools from direct remap code David Rientjes
2020-04-14 6:35 ` Christoph Hellwig
2020-04-08 21:21 ` [rfc v2 2/6] dma-pool: add additional coherent pools to map to gfp mask David Rientjes
2020-04-08 21:21 ` [rfc v2 3/6] dma-pool: dynamically expanding atomic pools David Rientjes
2020-04-08 21:21 ` [rfc v2 4/6] dma-direct: atomic allocations must come from atomic coherent pools David Rientjes
2020-04-09 21:24 ` Tom Lendacky
2020-04-14 19:18 ` David Rientjes
2020-04-14 6:43 ` Christoph Hellwig
2020-04-14 19:20 ` David Rientjes
2020-04-08 21:21 ` [rfc v2 5/6] x86/mm: unencrypted non-blocking DMA allocations use " David Rientjes
2020-04-08 21:21 ` [rfc v2 6/6] dma-pool: scale the default DMA coherent pool size with memory capacity David Rientjes
[not found] ` <20200410145520.17864-1-hdanton@sina.com>
2020-04-10 19:37 ` David Rientjes [this message]
2020-04-14 6:44 ` [rfc v2 3/6] dma-pool: dynamically expanding atomic pools Christoph Hellwig
2020-04-14 19:23 ` David Rientjes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.21.2004101231240.249689@chino.kir.corp.google.com \
--to=rientjes@google.com \
--cc=brijesh.singh@amd.com \
--cc=hch@lst.de \
--cc=hdanton@sina.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jon.grimm@amd.com \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=thomas.lendacky@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).