[PATCH v3 0/5] VDUSE: Support registering userspace memory as bounce buffer

[Xie Yongji <xieyongji@bytedance.com>]

Hi all,

This series introduces some new ioctls: VDUSE_IOTLB_GET_INFO,
VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM to support
registering and de-registering userspace memory for IOTLB
as bounce buffer in virtio-vdpa case.

The VDUSE_IOTLB_GET_INFO ioctl can help user to query whether
one IOVA region support userspace memory registration. Then
user can use VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM
ioctls to register and de-register userspace memory for this
region.

During registering and de-registering, the DMA data in use
would be copied from kernel bounce pages to userspace bounce
pages and back.

With this feature, some existing application such as SPDK
and DPDK can leverage the datapath of VDUSE directly and
efficiently as discussed before [1][2]. They can register
some preallocated hugepages to VDUSE to avoid an extra
memcpy from bounce-buffer to hugepages.

The kernel and userspace codes could be found in github:

https://github.com/bytedance/linux/tree/vduse-umem
https://github.com/bytedance/qemu/tree/vduse-umem

To test it with qemu-storage-daemon:

$ qemu-storage-daemon \
    --chardev socket,id=charmonitor,path=/tmp/qmp.sock,server=on,wait=off \
    --monitor chardev=charmonitor \
    --blockdev driver=host_device,cache.direct=on,aio=native,filename=/dev/nullb0,node-name=disk0 \
    --export type=vduse-blk,id=vduse-test,name=vduse-test,node-name=disk0,writable=on

[1] https://lkml.org/lkml/2021/6/27/318
[2] https://lkml.org/lkml/2022/7/4/246

Please review, thanks!

V2 to V3:
- Check if the reserved fields is zero [MST]
- Fix some compile errors
- Rework the VDUSE_IOTLB_GET_INFO ioctl to hide some
  internal implementation (bounce buffer) [Jason]
- Add more commit logs for patch 1 [MST]

V1 to V2:
- Drop the patch that updating API version [MST]
- Replace unpin_user_pages() with unpin_user_pages_dirty_lock() [MST]
- Use __vmalloc(__GFP_ACCOUNT) for memory accounting [MST]

Xie Yongji (5):
  vduse: Remove unnecessary spin lock protection
  vduse: Use memcpy_{to,from}_page() in do_bounce()
  vduse: Support using userspace pages as bounce buffer
  vduse: Support registering userspace memory for IOVA regions
  vduse: Support querying information of IOVA regions

 drivers/vdpa/vdpa_user/iova_domain.c | 102 +++++++++++++--
 drivers/vdpa/vdpa_user/iova_domain.h |   8 ++
 drivers/vdpa/vdpa_user/vduse_dev.c   | 180 +++++++++++++++++++++++++++
 include/uapi/linux/vduse.h           |  48 +++++++
 4 files changed, 325 insertions(+), 13 deletions(-)

-- 
2.20.1