Re: Re: [RFC v4 06/11] vduse: Implement an MMU-based IOMMU driver

From: Yongji Xie <xieyongji@bytedance.com>
To: Jason Wang <jasowang@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Stefano Garzarella <sgarzare@redhat.com>,
	Parav Pandit <parav@nvidia.com>, Bob Liu <bob.liu@oracle.com>,
	Christoph Hellwig <hch@infradead.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	Matthew Wilcox <willy@infradead.org>,
	viro@zeniv.linux.org.uk, Jens Axboe <axboe@kernel.dk>,
	bcrl@kvack.org, Jonathan Corbet <corbet@lwn.net>,
	virtualization@lists.linux-foundation.org,
	netdev@vger.kernel.org, kvm@vger.kernel.org, linux-aio@kvack.org,
	linux-fsdevel@vger.kernel.org
Subject: Re: Re: [RFC v4 06/11] vduse: Implement an MMU-based IOMMU driver
Date: Fri, 5 Mar 2021 15:59:02 +0800	[thread overview]
Message-ID: <CACycT3uMV9wg5yVKmEJpbZrs3x0b4+b9eNcUTh3+CjxsG7x2LA@mail.gmail.com> (raw)
In-Reply-To: <2652f696-faf7-26eb-a8b2-c4cfe3aaed15@redhat.com>

On Fri, Mar 5, 2021 at 3:27 PM Jason Wang <jasowang@redhat.com> wrote:
>
>
> On 2021/3/5 3:13 下午, Yongji Xie wrote:
> > On Fri, Mar 5, 2021 at 2:52 PM Jason Wang <jasowang@redhat.com> wrote:
> >>
> >> On 2021/3/5 2:15 下午, Yongji Xie wrote:
> >>
> >> Sorry if I've asked this before.
> >>
> >> But what's the reason for maintaing a dedicated IOTLB here? I think we
> >> could reuse vduse_dev->iommu since the device can not be used by both
> >> virtio and vhost in the same time or use vduse_iova_domain->iotlb for
> >> set_map().
> >>
> >> The main difference between domain->iotlb and dev->iotlb is the way to
> >> deal with bounce buffer. In the domain->iotlb case, bounce buffer
> >> needs to be mapped each DMA transfer because we need to get the bounce
> >> pages by an IOVA during DMA unmapping. In the dev->iotlb case, bounce
> >> buffer only needs to be mapped once during initialization, which will
> >> be used to tell userspace how to do mmap().
> >>
> >> Also, since vhost IOTLB support per mapping token (opauqe), can we use
> >> that instead of the bounce_pages *?
> >>
> >> Sorry, I didn't get you here. Which value do you mean to store in the
> >> opaque pointer？
> >>
> >> So I would like to have a way to use a single IOTLB for manage all kinds
> >> of mappings. Two possible ideas:
> >>
> >> 1) map bounce page one by one in vduse_dev_map_page(), in
> >> VDUSE_IOTLB_GET_FD, try to merge the result if we had the same fd. Then
> >> for bounce pages, userspace still only need to map it once and we can
> >> maintain the actual mapping by storing the page or pa in the opaque
> >> field of IOTLB entry.
> >>
> >> Looks like userspace still needs to unmap the old region and map a new
> >> region (size is changed) with the fd in each VDUSE_IOTLB_GET_FD ioctl.
> >>
> >>
> >> I don't get here. Can you give an example?
> >>
> > For example, userspace needs to process two I/O requests (one page per
> > request). To process the first request, userspace uses
> > VDUSE_IOTLB_GET_FD ioctl to query the iova region (0 ~ 4096) and mmap
> > it.
>
>
> I think in this case we should let VDUSE_IOTLB_GET_FD return the maximum
> range as far as they are backed by the same fd.
>

But now the bounce page is mapped one by one. The second page (4096 ~
8192) might not be mapped when userspace is processing the first
request. So the maximum range is 0 ~ 4096 at that time.

Thanks,
Yongji