Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

From: Christoph Hellwig <hch@lst.de>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Christoph Hellwig <hch@lst.de>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Bernard Metzler <bmt@zurich.ibm.com>,
	Zhu Yanjun <yanjunz@nvidia.com>,
	Logan Gunthorpe <logang@deltatee.com>,
	Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>,
	Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>,
	linux-rdma@vger.kernel.org, linux-pci@vger.kernel.org,
	iommu@lists.linux-foundation.org
Subject: Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks
Date: Thu, 5 Nov 2020 18:29:21 +0100	[thread overview]
Message-ID: <20201105172921.GA9537@lst.de> (raw)
In-Reply-To: <20201105172357.GE36674@ziepe.ca>

On Thu, Nov 05, 2020 at 01:23:57PM -0400, Jason Gunthorpe wrote:
> But that depends on the calling driver doing this properly, and we
> don't expose an API to get the PCI device of the struct ib_device
> .. how does nvme even work here?

The PCI p2pdma APIs walk the parent chains of a struct device until
they find a PCI device.  And the ib_device eventually ends up there.

> 
> If we can't get here then why did you add the check to the unmap side?

Because I added them to the map and unmap side, but forgot to commit
the map side.  Mostly to be prepared for the case where we could
end up there.  And thinking out loud I actually need to double check
rdmavt if that is true there as well.  It certainly is for rxe and
siw as I checked it on a live system.

> The SW drivers can't handle PCI pages at all, they are going to try to
> memcpy them or something else not __iomem, so we really do need to
> prevent P2P pages going into them.

Ok, let's prevent it for now.  And if someone wants to do it there
they have to do all the work.