Re: [RFC PATCH 04/12] s390/cio: introduce cio DMA pool

From: Halil Pasic <pasic@linux.ibm.com>
To: Sebastian Ott <sebott@linux.ibm.com>
Cc: kvm@vger.kernel.org, linux-s390@vger.kernel.org,
	Cornelia Huck <cohuck@redhat.com>,
	Martin Schwidefsky <schwidefsky@de.ibm.com>,
	virtualization@lists.linux-foundation.org,
	Christian Borntraeger <borntraeger@de.ibm.com>,
	Viktor Mihajlovski <mihajlov@linux.ibm.com>,
	Vasily Gorbik <gor@linux.ibm.com>,
	Janosch Frank <frankja@linux.ibm.com>,
	Claudio Imbrenda <imbrenda@linux.ibm.com>,
	Farhan Ali <alifm@linux.ibm.com>,
	Eric Farman <farman@linux.ibm.com>
Subject: Re: [RFC PATCH 04/12] s390/cio: introduce cio DMA pool
Date: Tue, 16 Apr 2019 15:31:30 +0200	[thread overview]
Message-ID: <20190416153130.2b430744@oc2783563651> (raw)
In-Reply-To: <alpine.LFD.2.21.1904161436400.2054@schleppi>

On Tue, 16 Apr 2019 14:50:14 +0200 (CEST)
Sebastian Ott <sebott@linux.ibm.com> wrote:

> On Fri, 12 Apr 2019, Halil Pasic wrote:
> > On Fri, 12 Apr 2019 14:12:31 +0200 (CEST)
> > Sebastian Ott <sebott@linux.ibm.com> wrote:
> > > On Fri, 12 Apr 2019, Halil Pasic wrote:
> > > > On Thu, 11 Apr 2019 20:25:01 +0200 (CEST)
> > > > Sebastian Ott <sebott@linux.ibm.com> wrote:
> > > > > I don't think we should use this global DMA pool. I guess it's OK for
> > > > > stuff like airq (where we don't have a struct device at hand) but for
> > > > > CCW we should use the device we have. Yes, this way we waste some memory
> > > > > but all dma memory a device uses should fit in a page - so the wastage
> > > > > is not too much.
> > 
> > Regarding the wastage. Let us do the math together in search for an
> > upper (wastage) limit.
> [...]
> > Currently we need at least 224 bytes per device that is ~ 6%
> > of a PAGE_SIZE.
> 
> Yes, we basically waste the whole page. I'm ok with that if the benefit is
> to play nice with the kernel APIs.
> 
> > > For practical
> > > matters: DMA debugging will complain about misuse of a specific device or
> > > driver.
> > > 
> > 
> > Do you mean CONFIG_DMA_API_DEBUG and CONFIG_DMA_API_DEBUG_SG? I've been
> > running with those and did not see any complaints. Maybe we should
> > clarify this one offline...
> 
> I didn't mean to imply that there are bugs already - just that when used
> as intended the DMA_DEBUG_API can complain about stuff like "your device
> is gone but you have still DMA memory set up for it" which will not work
> if you don't use the correct device...
> 

Right. In fact the 'real' allocations happen using gen_pool, and the pool
never shrinks.

IMHO as so often in software engineering we have a trade-off here. I'm
still not convinced I traded badly here, but I will take the request of
yours to tie the dma allocations to a more the device requiring the dma
as a maintainers request, and accommodate it in v1.

Are you fine with having a similar gen_pool backed with dma_pages on a
per struct io_subchannel_private basis?

Based on our offline chat I think you are, but but better safe than sorry.

Regards,
Halil