[RFC PATCH 0/5] Add Xen PVH dom0 support for GPU

From: Huang Rui <ray.huang@amd.com>
To: "Juergen Gross" <jgross@suse.com>,
	"Stefano Stabellini" <sstabellini@kernel.org>,
	"Oleksandr Tyshchenko" <oleksandr_tyshchenko@epam.com>,
	"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
	"Roger Pau Monné" <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
	dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org
Cc: "Honglei Huang" <honglei1.huang@amd.com>,
	"Stewart Hildebrand" <Stewart.Hildebrand@amd.com>,
	"Huang Rui" <ray.huang@amd.com>,
	"Chen Jiqian" <Jiqian.Chen@amd.com>,
	"Xenia Ragiadakou" <burzalodowa@gmail.com>,
	"Alex Deucher" <alexander.deucher@amd.com>,
	"Julia Zhang" <julia.zhang@amd.com>,
	"Christian König" <christian.koenig@amd.com>
Subject: [RFC PATCH 0/5] Add Xen PVH dom0 support for GPU
Date: Sun, 12 Mar 2023 20:01:52 +0800	[thread overview]
Message-ID: <20230312120157.452859-1-ray.huang@amd.com> (raw)

Hi all,

Currently, we are working to add VirtIO GPU and Passthrough GPU support on
Xen. We expected to use HVM on domU and PVH on dom0. The x86 PVH dom0
support needs a few modifications on our APU platform. These functions
requires multiple software components support including kernel, xen, qemu,
mesa, and virglrenderer. Please see the patch series on Xen and QEMU bleow.

Xen: https://lists.xenproject.org/archives/html/xen-devel/2023-03/msg00714.html
QEMU: https://lists.nongnu.org/archive/html/qemu-devel/2023-03/msg03972.html

Kernel part mainly adds the PVH dom0 support:

1) Enable Xen PVH dom0 for AMDGPU

Please check patch 1 to 3, that enable Xen PVH dom0 on amdgpu. Because we
would like to use hardware IOMMU instead of swiotlb for buffer copy, PV
dom0 only supported swiotlb.

There still some workarounds in the kernel need to dig it out like below
https://git.kernel.org/pub/scm/linux/kernel/git/rui/linux.git/commit/?h=upstream-fox-xen&id=9bee65dd3498dfc6aad283d22ff641198b5c91ed

2) Add PCIe Passthrough (GPU) on Xen PVH dom0

Please check patch 4 to 5, this implements acpi_register_gsi_xen_pvh API to
register GSI for guest domU, amd make a new privcmd to handle the GSI from
the IRQ.

Below are the screenshot of these functions, please take a look.

Passthrough GPU: https://drive.google.com/file/d/17onr5gvDK8KM_LniHTSQEI2hGJZlI09L/view?usp=share_link
Venus: https://drive.google.com/file/d/1_lPq6DMwHu1JQv7LUUVRx31dBj0HJYcL/view?usp=share_link
Zink: https://drive.google.com/file/d/1FxLmKu6X7uJOxx1ZzwOm1yA6IL5WMGzd/view?usp=share_link

Repositories
Kernel: https://git.kernel.org/pub/scm/linux/kernel/git/rui/linux.git/log/?h=upstream-fox-xen
Xen: https://gitlab.com/huangrui123/xen/-/commits/upstream-for-xen
QEMU: https://gitlab.com/huangrui123/qemu/-/commits/upstream-for-xen
Mesa: https://gitlab.freedesktop.org/rui/mesa/-/commits/upstream-for-xen
Virglrenderer: https://gitlab.freedesktop.org/rui/virglrenderer/-/commits/upstream-for-xen

We are writting the documentation on xen wiki page, and will update it in
feature version.

Thanks,
Ray

Chen Jiqian (2):
  x86/xen: acpi registers gsi for xen pvh
  xen/privcmd: add IOCTL_PRIVCMD_GSI_FROM_IRQ

Huang Rui (3):
  x86/xen: disable swiotlb for xen pvh
  xen/grants: update initialization order of xen grant table
  drm/amdgpu: set passthrough mode for xen pvh/hvm

 arch/x86/include/asm/apic.h              |  7 ++++
 arch/x86/include/asm/xen/pci.h           |  5 +++
 arch/x86/kernel/acpi/boot.c              |  2 +-
 arch/x86/kernel/pci-dma.c                |  8 ++++-
 arch/x86/pci/xen.c                       | 43 ++++++++++++++++++++++++
 arch/x86/xen/grant-table.c               |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c |  3 +-
 drivers/xen/events/events_base.c         | 39 +++++++++++++++++++++
 drivers/xen/grant-table.c                |  2 +-
 drivers/xen/privcmd.c                    | 20 +++++++++++
 include/uapi/xen/privcmd.h               |  7 ++++
 include/xen/events.h                     |  5 +++
 12 files changed, 138 insertions(+), 5 deletions(-)

-- 
2.25.1