Re: [RFC PATCH 2/4] kvm: Add host side support for free memory hints

From: "Michael S. Tsirkin" <mst@redhat.com>
To: Alexander Duyck <alexander.h.duyck@linux.intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>,
	Alexander Duyck <alexander.duyck@gmail.com>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	kvm@vger.kernel.org, rkrcmar@redhat.com, x86@kernel.org,
	mingo@redhat.com, bp@alien8.de, hpa@zytor.com,
	pbonzini@redhat.com, tglx@linutronix.de,
	akpm@linux-foundation.org
Subject: Re: [RFC PATCH 2/4] kvm: Add host side support for free memory hints
Date: Mon, 11 Feb 2019 14:24:58 -0500	[thread overview]
Message-ID: <20190211141811-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <58e57acd628f2d6535fc45a028af50855158fda6.camel@linux.intel.com>

On Mon, Feb 11, 2019 at 10:30:10AM -0800, Alexander Duyck wrote:
> On Mon, 2019-02-11 at 12:48 -0500, Michael S. Tsirkin wrote:
> > On Mon, Feb 11, 2019 at 09:41:19AM -0800, Dave Hansen wrote:
> > > On 2/9/19 4:44 PM, Michael S. Tsirkin wrote:
> > > > So the policy should not leak into host/guest interface.
> > > > Instead it is better to just keep the pages pinned and
> > > > ignore the hint for now.
> > > 
> > > It does seems a bit silly to have guests forever hinting about freed
> > > memory when the host never has a hope of doing anything about it.
> > > 
> > > Is that part fixable?
> > 
> > 
> > Yes just not with existing IOMMU APIs.
> > 
> > It's in the paragraph just above that you cut out:
> > 	Yes right now assignment is not smart enough but generally
> > 	you can protect the unused page in the IOMMU and that's it,
> > 	it's safe.
> > 
> > So e.g.
> > 	extern int iommu_remap(struct iommu_domain *domain, unsigned long iova,
> > 				     phys_addr_t paddr, size_t size, int prot);
> > 
> > 
> > I can elaborate if you like but generally we would need an API that
> > allows you to atomically update a mapping for a specific page without
> > perturbing the mapping for other pages.
> > 
> 
> I still don't see how this would solve anything unless you have the
> guest somehow hinting on what pages it is providing to the devices. 
>
> You would have to have the host invalidating the pages when the hint is
> provided, and have a new hint tied to arch_alloc_page that would
> rebuild the IOMMU mapping when a page is allocated.
> 
> I'm pretty certain that the added cost of that would make the hinting
> pretty pointless as my experience has been that the IOMMU is too much
> of a bottleneck to have multiple CPUs trying to create and invalidate
> mappings simultaneously.

I agree it's a concern.

Another option would involve passing these hints in the DMA API.

How about the option of removing the device by hotplug when
host needs overcommit? That would involve either buffering
on host, or requesting free pages after device is removed
along the lines of existing balloon code. That btw seems to
be an argument for making this hinting part of balloon.

-- 
MST