linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* refactor the i915 GVT support
@ 2021-07-21 15:53 Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 01/21] drm/i915/gvt: integrate into the main Makefile Christoph Hellwig
                   ` (22 more replies)
  0 siblings, 23 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Hi all,

the GVT code in the i915 is a bit of a mess right now due to strange
abstractions and lots of indirect calls.  This series refactors various
bits to clean that up.  The main user visible change is that almost all
of the GVT code moves out of the main i915 driver and into the kvmgt
module.

Tested on my Thinkpad with a Kaby Lake CPU and integrated graphics.

Git tree:

    git://git.infradead.org/users/hch/misc.git i915-gvt

Gitweb:

    http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/i915-gvt

Diffstat:
 b/drivers/gpu/drm/i915/Kconfig                         |   31 
 b/drivers/gpu/drm/i915/Makefile                        |   30 
 b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c |    4 
 b/drivers/gpu/drm/i915/gvt/cfg_space.c                 |   89 --
 b/drivers/gpu/drm/i915/gvt/cmd_parser.c                |    4 
 b/drivers/gpu/drm/i915/gvt/dmabuf.c                    |   36 
 b/drivers/gpu/drm/i915/gvt/execlist.c                  |   12 
 b/drivers/gpu/drm/i915/gvt/gtt.c                       |   55 -
 b/drivers/gpu/drm/i915/gvt/gvt.c                       |  100 --
 b/drivers/gpu/drm/i915/gvt/gvt.h                       |  132 ++-
 b/drivers/gpu/drm/i915/gvt/interrupt.c                 |   38 -
 b/drivers/gpu/drm/i915/gvt/kvmgt.c                     |  634 ++++-------------
 b/drivers/gpu/drm/i915/gvt/mmio.c                      |    4 
 b/drivers/gpu/drm/i915/gvt/opregion.c                  |  148 ---
 b/drivers/gpu/drm/i915/gvt/page_track.c                |    8 
 b/drivers/gpu/drm/i915/gvt/scheduler.c                 |   37 
 b/drivers/gpu/drm/i915/gvt/trace.h                     |    2 
 b/drivers/gpu/drm/i915/gvt/vgpu.c                      |   22 
 b/drivers/gpu/drm/i915/i915_drv.h                      |    7 
 b/drivers/gpu/drm/i915/i915_params.c                   |    2 
 b/drivers/gpu/drm/i915/intel_gvt.c                     |   64 +
 b/drivers/gpu/drm/i915/intel_gvt.h                     |    4 
 drivers/gpu/drm/i915/gvt/Makefile                      |    9 
 drivers/gpu/drm/i915/gvt/hypercall.h                   |   82 --
 drivers/gpu/drm/i915/gvt/mpt.h                         |  400 ----------
 25 files changed, 541 insertions(+), 1413 deletions(-)

^ permalink raw reply	[flat|nested] 60+ messages in thread

* [PATCH 01/21] drm/i915/gvt: integrate into the main Makefile
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-08-03  9:27   ` Zhenyu Wang
  2021-07-21 15:53 ` [PATCH 02/21] drm/i915/gvt: remove module refcounting in intel_gvt_{,un}register_hypervisor Christoph Hellwig
                   ` (21 subsequent siblings)
  22 siblings, 1 reply; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Remove the separately included Makefile and just use the relative
reference from the main i915 Makefile as for source files in other
subdirectories.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/Makefile     | 29 ++++++++++++++++++++++++-----
 drivers/gpu/drm/i915/gvt/Makefile |  9 ---------
 drivers/gpu/drm/i915/gvt/trace.h  |  2 +-
 3 files changed, 25 insertions(+), 15 deletions(-)
 delete mode 100644 drivers/gpu/drm/i915/gvt/Makefile

diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index 4f22cac1c49b..2153f67705b8 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -289,11 +289,30 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \
 
 # virtual gpu code
 i915-y += i915_vgpu.o
-
-ifeq ($(CONFIG_DRM_I915_GVT),y)
-i915-y += intel_gvt.o
-include $(src)/gvt/Makefile
-endif
+i915-$(CONFIG_DRM_I915_GVT) += \
+	intel_gvt.o \
+	gvt/gvt.o \
+	gvt/aperture_gm.o \
+	gvt/handlers.o \
+	gvt/vgpu.o \
+	gvt/trace_points.o \
+	gvt/firmware.o \
+	gvt/interrupt.o \
+	gvt/gtt.o \
+	gvt/cfg_space.o \
+	gvt/opregion.o \
+	gvt/mmio.o \
+	gvt/display.o \
+	gvt/edid.o \
+	gvt/execlist.o \
+	gvt/scheduler.o \
+	gvt/sched_policy.o \
+	gvt/mmio_context.o \
+	gvt/cmd_parser.o \
+	gvt/debugfs.o \
+	gvt/fb_decoder.o \
+	gvt/dmabuf.o \
+	gvt/page_track.o
 
 obj-$(CONFIG_DRM_I915) += i915.o
 obj-$(CONFIG_DRM_I915_GVT_KVMGT) += gvt/kvmgt.o
diff --git a/drivers/gpu/drm/i915/gvt/Makefile b/drivers/gpu/drm/i915/gvt/Makefile
deleted file mode 100644
index ea8324abc784..000000000000
--- a/drivers/gpu/drm/i915/gvt/Makefile
+++ /dev/null
@@ -1,9 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-GVT_DIR := gvt
-GVT_SOURCE := gvt.o aperture_gm.o handlers.o vgpu.o trace_points.o firmware.o \
-	interrupt.o gtt.o cfg_space.o opregion.o mmio.o display.o edid.o \
-	execlist.o scheduler.o sched_policy.o mmio_context.o cmd_parser.o debugfs.o \
-	fb_decoder.o dmabuf.o page_track.o
-
-ccflags-y				+= -I $(srctree)/$(src) -I $(srctree)/$(src)/$(GVT_DIR)/
-i915-y					+= $(addprefix $(GVT_DIR)/, $(GVT_SOURCE))
diff --git a/drivers/gpu/drm/i915/gvt/trace.h b/drivers/gpu/drm/i915/gvt/trace.h
index 6d787750d279..348f57f8301d 100644
--- a/drivers/gpu/drm/i915/gvt/trace.h
+++ b/drivers/gpu/drm/i915/gvt/trace.h
@@ -379,5 +379,5 @@ TRACE_EVENT(render_mmio,
 #undef TRACE_INCLUDE_PATH
 #define TRACE_INCLUDE_PATH .
 #undef TRACE_INCLUDE_FILE
-#define TRACE_INCLUDE_FILE trace
+#define TRACE_INCLUDE_FILE gvt/trace
 #include <trace/define_trace.h>
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 02/21] drm/i915/gvt: remove module refcounting in intel_gvt_{,un}register_hypervisor
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 01/21] drm/i915/gvt: integrate into the main Makefile Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 03/21] drm/i915/gvt: remove enum hypervisor_type Christoph Hellwig
                   ` (20 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

THIS_MODULE always is reference when a symbol called by it is used, so
don't bother with the additional reference.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.c b/drivers/gpu/drm/i915/gvt/gvt.c
index cbac409f6c8a..6d7938aacca4 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.c
+++ b/drivers/gpu/drm/i915/gvt/gvt.c
@@ -308,10 +308,6 @@ intel_gvt_register_hypervisor(const struct intel_gvt_mpt *m)
 	    m->type != INTEL_GVT_HYPERVISOR_XEN)
 		return -EINVAL;
 
-	/* Get a reference for device model module */
-	if (!try_module_get(THIS_MODULE))
-		return -ENODEV;
-
 	intel_gvt_host.mpt = m;
 	intel_gvt_host.hypervisor_type = m->type;
 	gvt = (void *)kdev_to_i915(intel_gvt_host.dev)->gvt;
@@ -321,7 +317,6 @@ intel_gvt_register_hypervisor(const struct intel_gvt_mpt *m)
 	if (ret < 0) {
 		gvt_err("Failed to init %s hypervisor module\n",
 			supported_hypervisors[intel_gvt_host.hypervisor_type]);
-		module_put(THIS_MODULE);
 		return -ENODEV;
 	}
 	gvt_dbg_core("Running with hypervisor %s in host mode\n",
@@ -335,6 +330,5 @@ intel_gvt_unregister_hypervisor(void)
 {
 	void *gvt = (void *)kdev_to_i915(intel_gvt_host.dev)->gvt;
 	intel_gvt_hypervisor_host_exit(intel_gvt_host.dev, gvt);
-	module_put(THIS_MODULE);
 }
 EXPORT_SYMBOL_GPL(intel_gvt_unregister_hypervisor);
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 03/21] drm/i915/gvt: remove enum hypervisor_type
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 01/21] drm/i915/gvt: integrate into the main Makefile Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 02/21] drm/i915/gvt: remove module refcounting in intel_gvt_{,un}register_hypervisor Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 04/21] drm/i915/gvt: move the gvt code into kvmgt.ko Christoph Hellwig
                   ` (19 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

The only supported hypervisor is KVM, so don't bother with dead code
enumerating hypervisors.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.c       |  17 +--
 drivers/gpu/drm/i915/gvt/gvt.h       |   1 -
 drivers/gpu/drm/i915/gvt/hypercall.h |   6 --
 drivers/gpu/drm/i915/gvt/kvmgt.c     |   1 -
 drivers/gpu/drm/i915/gvt/opregion.c  | 150 ++++++---------------------
 5 files changed, 34 insertions(+), 141 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.c b/drivers/gpu/drm/i915/gvt/gvt.c
index 6d7938aacca4..ee09bba970af 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.c
+++ b/drivers/gpu/drm/i915/gvt/gvt.c
@@ -41,11 +41,6 @@
 
 struct intel_gvt_host intel_gvt_host;
 
-static const char * const supported_hypervisors[] = {
-	[INTEL_GVT_HYPERVISOR_XEN] = "XEN",
-	[INTEL_GVT_HYPERVISOR_KVM] = "KVM",
-};
-
 static const struct intel_gvt_ops intel_gvt_ops = {
 	.emulate_cfg_read = intel_vgpu_emulate_cfg_read,
 	.emulate_cfg_write = intel_vgpu_emulate_cfg_write,
@@ -304,23 +299,13 @@ intel_gvt_register_hypervisor(const struct intel_gvt_mpt *m)
 	if (!intel_gvt_host.initialized)
 		return -ENODEV;
 
-	if (m->type != INTEL_GVT_HYPERVISOR_KVM &&
-	    m->type != INTEL_GVT_HYPERVISOR_XEN)
-		return -EINVAL;
-
 	intel_gvt_host.mpt = m;
-	intel_gvt_host.hypervisor_type = m->type;
 	gvt = (void *)kdev_to_i915(intel_gvt_host.dev)->gvt;
 
 	ret = intel_gvt_hypervisor_host_init(intel_gvt_host.dev, gvt,
 					     &intel_gvt_ops);
-	if (ret < 0) {
-		gvt_err("Failed to init %s hypervisor module\n",
-			supported_hypervisors[intel_gvt_host.hypervisor_type]);
+	if (ret < 0)
 		return -ENODEV;
-	}
-	gvt_dbg_core("Running with hypervisor %s in host mode\n",
-		     supported_hypervisors[intel_gvt_host.hypervisor_type]);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(intel_gvt_register_hypervisor);
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 0c0615602343..9b9fdbf9df10 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -59,7 +59,6 @@
 struct intel_gvt_host {
 	struct device *dev;
 	bool initialized;
-	int hypervisor_type;
 	const struct intel_gvt_mpt *mpt;
 };
 
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index f33e3cbd0439..317983153645 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -37,17 +37,11 @@
 
 struct device;
 
-enum hypervisor_type {
-	INTEL_GVT_HYPERVISOR_XEN = 0,
-	INTEL_GVT_HYPERVISOR_KVM,
-};
-
 /*
  * Specific GVT-g MPT modules function collections. Currently GVT-g supports
  * both Xen and KVM by providing dedicated hypervisor-related MPT modules.
  */
 struct intel_gvt_mpt {
-	enum hypervisor_type type;
 	int (*host_init)(struct device *dev, void *gvt, const void *ops);
 	void (*host_exit)(struct device *dev, void *gvt);
 	int (*attach_vgpu)(void *vgpu, unsigned long *handle);
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 1ac98f8aba31..0d0902d7405d 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -2204,7 +2204,6 @@ static bool kvmgt_is_valid_gfn(unsigned long handle, unsigned long gfn)
 }
 
 static const struct intel_gvt_mpt kvmgt_mpt = {
-	.type = INTEL_GVT_HYPERVISOR_KVM,
 	.host_init = kvmgt_host_init,
 	.host_exit = kvmgt_host_exit,
 	.attach_vgpu = kvmgt_attach_vgpu,
diff --git a/drivers/gpu/drm/i915/gvt/opregion.c b/drivers/gpu/drm/i915/gvt/opregion.c
index 33569b910ed5..286ac6d7c6ce 100644
--- a/drivers/gpu/drm/i915/gvt/opregion.c
+++ b/drivers/gpu/drm/i915/gvt/opregion.c
@@ -255,33 +255,6 @@ int intel_vgpu_init_opregion(struct intel_vgpu *vgpu)
 	return 0;
 }
 
-static int map_vgpu_opregion(struct intel_vgpu *vgpu, bool map)
-{
-	u64 mfn;
-	int i, ret;
-
-	for (i = 0; i < INTEL_GVT_OPREGION_PAGES; i++) {
-		mfn = intel_gvt_hypervisor_virt_to_mfn(vgpu_opregion(vgpu)->va
-			+ i * PAGE_SIZE);
-		if (mfn == INTEL_GVT_INVALID_ADDR) {
-			gvt_vgpu_err("fail to get MFN from VA\n");
-			return -EINVAL;
-		}
-		ret = intel_gvt_hypervisor_map_gfn_to_mfn(vgpu,
-				vgpu_opregion(vgpu)->gfn[i],
-				mfn, 1, map);
-		if (ret) {
-			gvt_vgpu_err("fail to map GFN to MFN, errno: %d\n",
-				ret);
-			return ret;
-		}
-	}
-
-	vgpu_opregion(vgpu)->mapped = map;
-
-	return 0;
-}
-
 /**
  * intel_vgpu_opregion_base_write_handler - Opregion base register write handler
  *
@@ -294,34 +267,13 @@ static int map_vgpu_opregion(struct intel_vgpu *vgpu, bool map)
 int intel_vgpu_opregion_base_write_handler(struct intel_vgpu *vgpu, u32 gpa)
 {
 
-	int i, ret = 0;
+	int i;
 
 	gvt_dbg_core("emulate opregion from kernel\n");
 
-	switch (intel_gvt_host.hypervisor_type) {
-	case INTEL_GVT_HYPERVISOR_KVM:
-		for (i = 0; i < INTEL_GVT_OPREGION_PAGES; i++)
-			vgpu_opregion(vgpu)->gfn[i] = (gpa >> PAGE_SHIFT) + i;
-		break;
-	case INTEL_GVT_HYPERVISOR_XEN:
-		/**
-		 * Wins guest on Xengt will write this register twice: xen
-		 * hvmloader and windows graphic driver.
-		 */
-		if (vgpu_opregion(vgpu)->mapped)
-			map_vgpu_opregion(vgpu, false);
-
-		for (i = 0; i < INTEL_GVT_OPREGION_PAGES; i++)
-			vgpu_opregion(vgpu)->gfn[i] = (gpa >> PAGE_SHIFT) + i;
-
-		ret = map_vgpu_opregion(vgpu, true);
-		break;
-	default:
-		ret = -EINVAL;
-		gvt_vgpu_err("not supported hypervisor\n");
-	}
-
-	return ret;
+	for (i = 0; i < INTEL_GVT_OPREGION_PAGES; i++)
+		vgpu_opregion(vgpu)->gfn[i] = (gpa >> PAGE_SHIFT) + i;
+	return 0;
 }
 
 /**
@@ -336,12 +288,7 @@ void intel_vgpu_clean_opregion(struct intel_vgpu *vgpu)
 	if (!vgpu_opregion(vgpu)->va)
 		return;
 
-	if (intel_gvt_host.hypervisor_type == INTEL_GVT_HYPERVISOR_XEN) {
-		if (vgpu_opregion(vgpu)->mapped)
-			map_vgpu_opregion(vgpu, false);
-	} else if (intel_gvt_host.hypervisor_type == INTEL_GVT_HYPERVISOR_KVM) {
-		/* Guest opregion is released by VFIO */
-	}
+	/* Guest opregion is released by VFIO */
 	free_pages((unsigned long)vgpu_opregion(vgpu)->va,
 		   get_order(INTEL_GVT_OPREGION_SIZE));
 
@@ -470,39 +417,22 @@ int intel_vgpu_emulate_opregion_request(struct intel_vgpu *vgpu, u32 swsci)
 	u64 scic_pa = 0, parm_pa = 0;
 	int ret;
 
-	switch (intel_gvt_host.hypervisor_type) {
-	case INTEL_GVT_HYPERVISOR_XEN:
-		scic = *((u32 *)vgpu_opregion(vgpu)->va +
-					INTEL_GVT_OPREGION_SCIC);
-		parm = *((u32 *)vgpu_opregion(vgpu)->va +
-					INTEL_GVT_OPREGION_PARM);
-		break;
-	case INTEL_GVT_HYPERVISOR_KVM:
-		scic_pa = (vgpu_opregion(vgpu)->gfn[0] << PAGE_SHIFT) +
-					INTEL_GVT_OPREGION_SCIC;
-		parm_pa = (vgpu_opregion(vgpu)->gfn[0] << PAGE_SHIFT) +
-					INTEL_GVT_OPREGION_PARM;
-
-		ret = intel_gvt_hypervisor_read_gpa(vgpu, scic_pa,
-						    &scic, sizeof(scic));
-		if (ret) {
-			gvt_vgpu_err("guest opregion read error %d, gpa 0x%llx, len %lu\n",
-				ret, scic_pa, sizeof(scic));
-			return ret;
-		}
-
-		ret = intel_gvt_hypervisor_read_gpa(vgpu, parm_pa,
-						    &parm, sizeof(parm));
-		if (ret) {
-			gvt_vgpu_err("guest opregion read error %d, gpa 0x%llx, len %lu\n",
-				ret, scic_pa, sizeof(scic));
-			return ret;
-		}
+	scic_pa = (vgpu_opregion(vgpu)->gfn[0] << PAGE_SHIFT) +
+				INTEL_GVT_OPREGION_SCIC;
+	parm_pa = (vgpu_opregion(vgpu)->gfn[0] << PAGE_SHIFT) +
+				INTEL_GVT_OPREGION_PARM;
+	ret = intel_gvt_hypervisor_read_gpa(vgpu, scic_pa, &scic, sizeof(scic));
+	if (ret) {
+		gvt_vgpu_err("guest opregion read error %d, gpa 0x%llx, len %lu\n",
+			ret, scic_pa, sizeof(scic));
+		return ret;
+	}
 
-		break;
-	default:
-		gvt_vgpu_err("not supported hypervisor\n");
-		return -EINVAL;
+	ret = intel_gvt_hypervisor_read_gpa(vgpu, parm_pa, &parm, sizeof(parm));
+	if (ret) {
+		gvt_vgpu_err("guest opregion read error %d, gpa 0x%llx, len %lu\n",
+			ret, scic_pa, sizeof(scic));
+		return ret;
 	}
 
 	if (!(swsci & SWSCI_SCI_SELECT)) {
@@ -535,34 +465,20 @@ int intel_vgpu_emulate_opregion_request(struct intel_vgpu *vgpu, u32 swsci)
 	parm = 0;
 
 out:
-	switch (intel_gvt_host.hypervisor_type) {
-	case INTEL_GVT_HYPERVISOR_XEN:
-		*((u32 *)vgpu_opregion(vgpu)->va +
-					INTEL_GVT_OPREGION_SCIC) = scic;
-		*((u32 *)vgpu_opregion(vgpu)->va +
-					INTEL_GVT_OPREGION_PARM) = parm;
-		break;
-	case INTEL_GVT_HYPERVISOR_KVM:
-		ret = intel_gvt_hypervisor_write_gpa(vgpu, scic_pa,
-						    &scic, sizeof(scic));
-		if (ret) {
-			gvt_vgpu_err("guest opregion write error %d, gpa 0x%llx, len %lu\n",
-				ret, scic_pa, sizeof(scic));
-			return ret;
-		}
-
-		ret = intel_gvt_hypervisor_write_gpa(vgpu, parm_pa,
-						    &parm, sizeof(parm));
-		if (ret) {
-			gvt_vgpu_err("guest opregion write error %d, gpa 0x%llx, len %lu\n",
-				ret, scic_pa, sizeof(scic));
-			return ret;
-		}
+	ret = intel_gvt_hypervisor_write_gpa(vgpu, scic_pa, &scic,
+					     sizeof(scic));
+	if (ret) {
+		gvt_vgpu_err("guest opregion write error %d, gpa 0x%llx, len %lu\n",
+			ret, scic_pa, sizeof(scic));
+		return ret;
+	}
 
-		break;
-	default:
-		gvt_vgpu_err("not supported hypervisor\n");
-		return -EINVAL;
+	ret = intel_gvt_hypervisor_write_gpa(vgpu, parm_pa, &parm,
+					     sizeof(parm));
+	if (ret) {
+		gvt_vgpu_err("guest opregion write error %d, gpa 0x%llx, len %lu\n",
+			ret, scic_pa, sizeof(scic));
+		return ret;
 	}
 
 	return 0;
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 04/21] drm/i915/gvt: move the gvt code into kvmgt.ko
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (2 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 03/21] drm/i915/gvt: remove enum hypervisor_type Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-08-09 11:29   ` Joonas Lahtinen
  2021-07-21 15:53 ` [PATCH 05/21] drm/i915/gvt: remove intel_gvt_ops Christoph Hellwig
                   ` (18 subsequent siblings)
  22 siblings, 1 reply; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Instead of having an option to build the gvt code into the main i915
module, just move it into the kvmgt.ko module.  This only requires
a new struct with three entries that the main i915 module needs to
request before enabling VGPU passthrough operations.

This also conveniently streamlines the GVT initialization and avoids
the need for the global device pointer.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/Kconfig                  | 31 +++------
 drivers/gpu/drm/i915/Makefile                 |  7 +-
 .../drm/i915/gt/intel_execlists_submission.c  |  4 +-
 drivers/gpu/drm/i915/gvt/gvt.c                | 56 ++++++----------
 drivers/gpu/drm/i915/gvt/gvt.h                |  6 +-
 drivers/gpu/drm/i915/gvt/kvmgt.c              | 19 ++----
 drivers/gpu/drm/i915/gvt/mpt.h                |  3 -
 drivers/gpu/drm/i915/i915_drv.h               |  7 ++
 drivers/gpu/drm/i915/i915_params.c            |  2 +-
 drivers/gpu/drm/i915/intel_gvt.c              | 64 +++++++++++++++++--
 drivers/gpu/drm/i915/intel_gvt.h              |  4 +-
 11 files changed, 107 insertions(+), 96 deletions(-)

diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig
index f960f5d7664e..46e37238b2e4 100644
--- a/drivers/gpu/drm/i915/Kconfig
+++ b/drivers/gpu/drm/i915/Kconfig
@@ -98,39 +98,24 @@ config DRM_I915_USERPTR
 
 	  If in doubt, say "Y".
 
-config DRM_I915_GVT
-	bool "Enable Intel GVT-g graphics virtualization host support"
+config DRM_I915_GVT_KVMGT
+	tristate "Enable KVM host support Intel GVT-g graphics virtualization"
 	depends on DRM_I915
 	depends on 64BIT
-	default n
+	depends on KVM
+	depends on VFIO_MDEV
 	help
 	  Choose this option if you want to enable Intel GVT-g graphics
 	  virtualization technology host support with integrated graphics.
 	  With GVT-g, it's possible to have one integrated graphics
-	  device shared by multiple VMs under different hypervisors.
+	  device shared by multiple VMs under KVM.
 
-	  Note that at least one hypervisor like Xen or KVM is required for
-	  this driver to work, and it only supports newer device from
-	  Broadwell+. For further information and setup guide, you can
-	  visit: http://01.org/igvt-g.
-
-	  Now it's just a stub to support the modifications of i915 for
-	  GVT device model. It requires at least one MPT modules for Xen/KVM
-	  and other components of GVT device model to work. Use it under
-	  you own risk.
+	  Note that this driver only supports newer device from Broadwell on.
+	  For further information and setup guide, you can visit:
+	  http://01.org/igvt-g.
 
 	  If in doubt, say "N".
 
-config DRM_I915_GVT_KVMGT
-	tristate "Enable KVM/VFIO support for Intel GVT-g"
-	depends on DRM_I915_GVT
-	depends on KVM
-	depends on VFIO_MDEV
-	default n
-	help
-	  Choose this option if you want to enable KVMGT support for
-	  Intel GVT-g.
-
 menu "drm/i915 Debugging"
 depends on DRM_I915
 depends on EXPERT
diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index 2153f67705b8..45cdb94363c7 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -289,8 +289,9 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \
 
 # virtual gpu code
 i915-y += i915_vgpu.o
-i915-$(CONFIG_DRM_I915_GVT) += \
-	intel_gvt.o \
+i915-$(CONFIG_DRM_I915_GVT_KVMGT) += intel_gvt.o
+
+kvmgt-y += gvt/kvmgt.o \
 	gvt/gvt.o \
 	gvt/aperture_gm.o \
 	gvt/handlers.o \
@@ -315,7 +316,7 @@ i915-$(CONFIG_DRM_I915_GVT) += \
 	gvt/page_track.o
 
 obj-$(CONFIG_DRM_I915) += i915.o
-obj-$(CONFIG_DRM_I915_GVT_KVMGT) += gvt/kvmgt.o
+obj-$(CONFIG_DRM_I915_GVT_KVMGT) += kvmgt.o
 
 # header test
 
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index fc77592d88a9..8b23c63db7ba 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -417,7 +417,7 @@ execlists_context_status_change(struct i915_request *rq, unsigned long status)
 	 * Only used when GVT-g is enabled now. When GVT-g is disabled,
 	 * The compiler should eliminate this function as dead-code.
 	 */
-	if (!IS_ENABLED(CONFIG_DRM_I915_GVT))
+	if (!IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT))
 		return;
 
 	atomic_notifier_call_chain(&rq->engine->context_status_notifier,
@@ -911,7 +911,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine)
 
 static bool ctx_single_port_submission(const struct intel_context *ce)
 {
-	return (IS_ENABLED(CONFIG_DRM_I915_GVT) &&
+	return (IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT) &&
 		intel_context_force_single_submission(ce));
 }
 
diff --git a/drivers/gpu/drm/i915/gvt/gvt.c b/drivers/gpu/drm/i915/gvt/gvt.c
index ee09bba970af..3671a076bd7a 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.c
+++ b/drivers/gpu/drm/i915/gvt/gvt.c
@@ -39,8 +39,6 @@
 #include <linux/vfio.h>
 #include <linux/mdev.h>
 
-struct intel_gvt_host intel_gvt_host;
-
 static const struct intel_gvt_ops intel_gvt_ops = {
 	.emulate_cfg_read = intel_vgpu_emulate_cfg_read,
 	.emulate_cfg_write = intel_vgpu_emulate_cfg_write,
@@ -147,13 +145,14 @@ static int init_service_thread(struct intel_gvt *gvt)
  * resources owned by a GVT device.
  *
  */
-void intel_gvt_clean_device(struct drm_i915_private *i915)
+static void intel_gvt_clean_device(struct drm_i915_private *i915)
 {
 	struct intel_gvt *gvt = fetch_and_zero(&i915->gvt);
 
 	if (drm_WARN_ON(&i915->drm, !gvt))
 		return;
 
+	intel_gvt_hypervisor_host_exit(i915->drm.dev, gvt);
 	intel_gvt_destroy_idle_vgpu(gvt->idle_vgpu);
 	intel_gvt_clean_vgpu_types(gvt);
 
@@ -181,7 +180,7 @@ void intel_gvt_clean_device(struct drm_i915_private *i915)
  * Zero on success, negative error code if failed.
  *
  */
-int intel_gvt_init_device(struct drm_i915_private *i915)
+static int intel_gvt_init_device(struct drm_i915_private *i915)
 {
 	struct intel_gvt *gvt;
 	struct intel_vgpu *vgpu;
@@ -253,11 +252,17 @@ int intel_gvt_init_device(struct drm_i915_private *i915)
 
 	intel_gvt_debugfs_init(gvt);
 
+	ret = intel_gvt_hypervisor_host_init(i915->drm.dev, gvt,
+					     &intel_gvt_ops);
+	if (ret)
+		goto out_destroy_idle_vgpu;
+
 	gvt_dbg_core("gvt device initialization is done\n");
-	intel_gvt_host.dev = i915->drm.dev;
-	intel_gvt_host.initialized = true;
 	return 0;
 
+out_destroy_idle_vgpu:
+	intel_gvt_destroy_idle_vgpu(gvt->idle_vgpu);
+	intel_gvt_debugfs_clean(gvt);
 out_clean_types:
 	intel_gvt_clean_vgpu_types(gvt);
 out_clean_thread:
@@ -281,39 +286,18 @@ int intel_gvt_init_device(struct drm_i915_private *i915)
 	return ret;
 }
 
-int
-intel_gvt_pm_resume(struct intel_gvt *gvt)
+static void intel_gvt_pm_resume(struct drm_i915_private *i915)
 {
+	struct intel_gvt *gvt = i915->gvt;
+
 	intel_gvt_restore_fence(gvt);
 	intel_gvt_restore_mmio(gvt);
 	intel_gvt_restore_ggtt(gvt);
-	return 0;
-}
-
-int
-intel_gvt_register_hypervisor(const struct intel_gvt_mpt *m)
-{
-	int ret;
-	void *gvt;
-
-	if (!intel_gvt_host.initialized)
-		return -ENODEV;
-
-	intel_gvt_host.mpt = m;
-	gvt = (void *)kdev_to_i915(intel_gvt_host.dev)->gvt;
-
-	ret = intel_gvt_hypervisor_host_init(intel_gvt_host.dev, gvt,
-					     &intel_gvt_ops);
-	if (ret < 0)
-		return -ENODEV;
-	return 0;
 }
-EXPORT_SYMBOL_GPL(intel_gvt_register_hypervisor);
 
-void
-intel_gvt_unregister_hypervisor(void)
-{
-	void *gvt = (void *)kdev_to_i915(intel_gvt_host.dev)->gvt;
-	intel_gvt_hypervisor_host_exit(intel_gvt_host.dev, gvt);
-}
-EXPORT_SYMBOL_GPL(intel_gvt_unregister_hypervisor);
+const struct i915_virtual_gpu_ops intel_gvt_vgpu_ops = {
+	.init_device	= intel_gvt_init_device,
+	.clean_device	= intel_gvt_clean_device,
+	.pm_resume	= intel_gvt_pm_resume,
+};
+EXPORT_SYMBOL_GPL(intel_gvt_vgpu_ops);
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 9b9fdbf9df10..c346111f0300 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -57,8 +57,6 @@
 #define GVT_MAX_VGPU 8
 
 struct intel_gvt_host {
-	struct device *dev;
-	bool initialized;
 	const struct intel_gvt_mpt *mpt;
 };
 
@@ -727,9 +725,9 @@ void intel_gvt_debugfs_remove_vgpu(struct intel_vgpu *vgpu);
 void intel_gvt_debugfs_init(struct intel_gvt *gvt);
 void intel_gvt_debugfs_clean(struct intel_gvt *gvt);
 
-int intel_gvt_pm_resume(struct intel_gvt *gvt);
-
 #include "trace.h"
 #include "mpt.h"
 
+extern const struct i915_virtual_gpu_ops intel_gvt_vgpu_ops;
+
 #endif
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 0d0902d7405d..65659094eee6 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -49,6 +49,8 @@
 #include "i915_drv.h"
 #include "gvt.h"
 
+MODULE_IMPORT_NS(I915_GVT);
+
 static const struct intel_gvt_ops *intel_gvt_ops;
 
 /* helper macros copied from vfio-pci */
@@ -2225,20 +2227,9 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
 	.is_valid_gfn = kvmgt_is_valid_gfn,
 };
 
-static int __init kvmgt_init(void)
-{
-	if (intel_gvt_register_hypervisor(&kvmgt_mpt) < 0)
-		return -ENODEV;
-	return 0;
-}
-
-static void __exit kvmgt_exit(void)
-{
-	intel_gvt_unregister_hypervisor();
-}
-
-module_init(kvmgt_init);
-module_exit(kvmgt_exit);
+struct intel_gvt_host intel_gvt_host = {
+	.mpt		= &kvmgt_mpt,
+};
 
 MODULE_LICENSE("GPL and additional rights");
 MODULE_AUTHOR("Intel Corporation");
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index e6c5a792a49a..1b5617bb2745 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -394,7 +394,4 @@ static inline bool intel_gvt_hypervisor_is_valid_gfn(
 	return intel_gvt_host.mpt->is_valid_gfn(vgpu->handle, gfn);
 }
 
-int intel_gvt_register_hypervisor(const struct intel_gvt_mpt *);
-void intel_gvt_unregister_hypervisor(void);
-
 #endif /* _GVT_MPT_H_ */
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 38ff2fb89744..37afbe2d9563 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -750,10 +750,17 @@ struct i915_frontbuffer_tracking {
 	unsigned flip_bits;
 };
 
+struct i915_virtual_gpu_ops {
+	int (*init_device)(struct drm_i915_private *dev_priv);
+	void (*clean_device)(struct drm_i915_private *dev_priv);
+	void (*pm_resume)(struct drm_i915_private *i915);
+};
+
 struct i915_virtual_gpu {
 	struct mutex lock; /* serialises sending of g2v_notify command pkts */
 	bool active;
 	u32 caps;
+	const struct i915_virtual_gpu_ops *ops;
 };
 
 struct intel_cdclk_config {
diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
index e07f4cfea63a..a7394294c010 100644
--- a/drivers/gpu/drm/i915/i915_params.c
+++ b/drivers/gpu/drm/i915/i915_params.c
@@ -187,7 +187,7 @@ i915_param_named(enable_dpcd_backlight, int, 0400,
 	"Enable support for DPCD backlight control"
 	"(-1=use per-VBT LFP backlight type setting [default], 0=disabled, 1=enable, 2=force VESA interface, 3=force Intel interface)");
 
-#if IS_ENABLED(CONFIG_DRM_I915_GVT)
+#if IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT)
 i915_param_named(enable_gvt, bool, 0400,
 	"Enable support for Intel GVT-g graphics virtualization host support(default:false)");
 #endif
diff --git a/drivers/gpu/drm/i915/intel_gvt.c b/drivers/gpu/drm/i915/intel_gvt.c
index 4e70c1a9ef2e..315707624d9f 100644
--- a/drivers/gpu/drm/i915/intel_gvt.c
+++ b/drivers/gpu/drm/i915/intel_gvt.c
@@ -25,6 +25,9 @@
 #include "i915_vgpu.h"
 #include "intel_gvt.h"
 #include "gvt/gvt.h"
+#include "gt/intel_context.h"
+#include "gt/intel_ring.h"
+#include "gt/shmem_utils.h"
 
 /**
  * DOC: Intel GVT-g host support
@@ -115,14 +118,23 @@ int intel_gvt_init(struct drm_i915_private *dev_priv)
 		return -EIO;
 	}
 
-	ret = intel_gvt_init_device(dev_priv);
+	request_module("kvmgt");
+	dev_priv->vgpu.ops = symbol_get(intel_gvt_vgpu_ops);
+	if (!dev_priv->vgpu.ops) {
+		drm_dbg(&dev_priv->drm, "Fail to find GVT ops\n");
+		goto bail;
+	}
+
+	ret = dev_priv->vgpu.ops->init_device(dev_priv);
 	if (ret) {
 		drm_dbg(&dev_priv->drm, "Fail to init GVT device\n");
-		goto bail;
+		goto bail_put_ops;
 	}
 
 	return 0;
 
+bail_put_ops:
+	symbol_put(gvt_ops);
 bail:
 	dev_priv->params.enable_gvt = 0;
 	return 0;
@@ -143,10 +155,10 @@ static inline bool intel_gvt_active(struct drm_i915_private *dev_priv)
  */
 void intel_gvt_driver_remove(struct drm_i915_private *dev_priv)
 {
-	if (!intel_gvt_active(dev_priv))
-		return;
-
-	intel_gvt_clean_device(dev_priv);
+	if (intel_gvt_active(dev_priv)) {
+		dev_priv->vgpu.ops->clean_device(dev_priv);
+		symbol_put(dev_priv->vgpu.ops);
+	}
 }
 
 /**
@@ -160,5 +172,43 @@ void intel_gvt_driver_remove(struct drm_i915_private *dev_priv)
 void intel_gvt_resume(struct drm_i915_private *dev_priv)
 {
 	if (intel_gvt_active(dev_priv))
-		intel_gvt_pm_resume(dev_priv->gvt);
+		dev_priv->vgpu.ops->pm_resume(dev_priv);
 }
+
+/*
+ * Exported here so that the exports only get created when GVT support is
+ * actually enabled.
+ */
+EXPORT_SYMBOL_NS_GPL(i915_gem_object_alloc, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_object_create_shmem, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_object_init, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_object_ggtt_pin_ww, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_object_pin_map, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_object_set_to_cpu_domain, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(__i915_gem_object_flush_map, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(__i915_gem_object_set_pages, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_gtt_insert, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_prime_export, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_ww_ctx_init, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_ww_ctx_backoff, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_gem_ww_ctx_fini, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_ppgtt_create, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_request_add, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_request_create, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_request_wait, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_reserve_fence, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_unreserve_fence, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_vm_release, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(i915_vma_move_to_active, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(intel_context_create, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(intel_context_unpin, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(__intel_context_do_pin, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(intel_ring_begin, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(intel_runtime_pm_get, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(intel_runtime_pm_put_unchecked, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(intel_uncore_forcewake_for_reg, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(intel_uncore_forcewake_get, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(intel_uncore_forcewake_put, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(shmem_pin_map, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(shmem_unpin_map, I915_GVT);
+EXPORT_SYMBOL_NS_GPL(__px_dma, I915_GVT);
diff --git a/drivers/gpu/drm/i915/intel_gvt.h b/drivers/gpu/drm/i915/intel_gvt.h
index d7d3fb6186fd..389589f006ad 100644
--- a/drivers/gpu/drm/i915/intel_gvt.h
+++ b/drivers/gpu/drm/i915/intel_gvt.h
@@ -26,11 +26,9 @@
 
 struct drm_i915_private;
 
-#ifdef CONFIG_DRM_I915_GVT
+#if IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT)
 int intel_gvt_init(struct drm_i915_private *dev_priv);
 void intel_gvt_driver_remove(struct drm_i915_private *dev_priv);
-int intel_gvt_init_device(struct drm_i915_private *dev_priv);
-void intel_gvt_clean_device(struct drm_i915_private *dev_priv);
 int intel_gvt_init_host(void);
 void intel_gvt_sanitize_options(struct drm_i915_private *dev_priv);
 void intel_gvt_resume(struct drm_i915_private *dev_priv);
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 05/21] drm/i915/gvt: remove intel_gvt_ops
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (3 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 04/21] drm/i915/gvt: move the gvt code into kvmgt.ko Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 06/21] drm/i915/gvt: remove the map_gfn_to_mfn and set_trap_area ops Christoph Hellwig
                   ` (17 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Remove these pointless indirect alls by just calling the only instance
of each method directly.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.c       | 20 +--------------
 drivers/gpu/drm/i915/gvt/gvt.h       | 24 ------------------
 drivers/gpu/drm/i915/gvt/hypercall.h |  2 +-
 drivers/gpu/drm/i915/gvt/kvmgt.c     | 37 +++++++++++-----------------
 drivers/gpu/drm/i915/gvt/mpt.h       |  5 ++--
 5 files changed, 19 insertions(+), 69 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.c b/drivers/gpu/drm/i915/gvt/gvt.c
index 3671a076bd7a..2b59e79f5e3b 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.c
+++ b/drivers/gpu/drm/i915/gvt/gvt.c
@@ -39,23 +39,6 @@
 #include <linux/vfio.h>
 #include <linux/mdev.h>
 
-static const struct intel_gvt_ops intel_gvt_ops = {
-	.emulate_cfg_read = intel_vgpu_emulate_cfg_read,
-	.emulate_cfg_write = intel_vgpu_emulate_cfg_write,
-	.emulate_mmio_read = intel_vgpu_emulate_mmio_read,
-	.emulate_mmio_write = intel_vgpu_emulate_mmio_write,
-	.vgpu_create = intel_gvt_create_vgpu,
-	.vgpu_destroy = intel_gvt_destroy_vgpu,
-	.vgpu_release = intel_gvt_release_vgpu,
-	.vgpu_reset = intel_gvt_reset_vgpu,
-	.vgpu_activate = intel_gvt_activate_vgpu,
-	.vgpu_deactivate = intel_gvt_deactivate_vgpu,
-	.vgpu_query_plane = intel_vgpu_query_plane,
-	.vgpu_get_dmabuf = intel_vgpu_get_dmabuf,
-	.write_protect_handler = intel_vgpu_page_track_handler,
-	.emulate_hotplug = intel_vgpu_emulate_hotplug,
-};
-
 static void init_device_info(struct intel_gvt *gvt)
 {
 	struct intel_gvt_device_info *info = &gvt->device_info;
@@ -252,8 +235,7 @@ static int intel_gvt_init_device(struct drm_i915_private *i915)
 
 	intel_gvt_debugfs_init(gvt);
 
-	ret = intel_gvt_hypervisor_host_init(i915->drm.dev, gvt,
-					     &intel_gvt_ops);
+	ret = intel_gvt_hypervisor_host_init(i915->drm.dev, gvt);
 	if (ret)
 		goto out_destroy_idle_vgpu;
 
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index c346111f0300..40643fe14383 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -555,30 +555,6 @@ void populate_pvinfo_page(struct intel_vgpu *vgpu);
 int intel_gvt_scan_and_shadow_workload(struct intel_vgpu_workload *workload);
 void enter_failsafe_mode(struct intel_vgpu *vgpu, int reason);
 
-struct intel_gvt_ops {
-	int (*emulate_cfg_read)(struct intel_vgpu *, unsigned int, void *,
-				unsigned int);
-	int (*emulate_cfg_write)(struct intel_vgpu *, unsigned int, void *,
-				unsigned int);
-	int (*emulate_mmio_read)(struct intel_vgpu *, u64, void *,
-				unsigned int);
-	int (*emulate_mmio_write)(struct intel_vgpu *, u64, void *,
-				unsigned int);
-	struct intel_vgpu *(*vgpu_create)(struct intel_gvt *,
-				struct intel_vgpu_type *);
-	void (*vgpu_destroy)(struct intel_vgpu *vgpu);
-	void (*vgpu_release)(struct intel_vgpu *vgpu);
-	void (*vgpu_reset)(struct intel_vgpu *);
-	void (*vgpu_activate)(struct intel_vgpu *);
-	void (*vgpu_deactivate)(struct intel_vgpu *);
-	int (*vgpu_query_plane)(struct intel_vgpu *vgpu, void *);
-	int (*vgpu_get_dmabuf)(struct intel_vgpu *vgpu, unsigned int);
-	int (*write_protect_handler)(struct intel_vgpu *, u64, void *,
-				     unsigned int);
-	void (*emulate_hotplug)(struct intel_vgpu *vgpu, bool connected);
-};
-
-
 enum {
 	GVT_FAILSAFE_UNSUPPORTED_GUEST,
 	GVT_FAILSAFE_INSUFFICIENT_RESOURCE,
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index 317983153645..395bce9633fa 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -42,7 +42,7 @@ struct device;
  * both Xen and KVM by providing dedicated hypervisor-related MPT modules.
  */
 struct intel_gvt_mpt {
-	int (*host_init)(struct device *dev, void *gvt, const void *ops);
+	int (*host_init)(struct device *dev, void *gvt);
 	void (*host_exit)(struct device *dev, void *gvt);
 	int (*attach_vgpu)(void *vgpu, unsigned long *handle);
 	void (*detach_vgpu)(void *vgpu);
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 65659094eee6..935d1c8287a4 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -51,8 +51,6 @@
 
 MODULE_IMPORT_NS(I915_GVT);
 
-static const struct intel_gvt_ops *intel_gvt_ops;
-
 /* helper macros copied from vfio-pci */
 #define VFIO_PCI_OFFSET_SHIFT   40
 #define VFIO_PCI_OFFSET_TO_INDEX(off)   (off >> VFIO_PCI_OFFSET_SHIFT)
@@ -602,9 +600,9 @@ static int handle_edid_regs(struct intel_vgpu *vgpu,
 					gvt_vgpu_err("invalid EDID blob\n");
 					return -EINVAL;
 				}
-				intel_gvt_ops->emulate_hotplug(vgpu, true);
+				intel_vgpu_emulate_hotplug(vgpu, true);
 			} else if (data == VFIO_DEVICE_GFX_LINK_STATE_DOWN)
-				intel_gvt_ops->emulate_hotplug(vgpu, false);
+				intel_vgpu_emulate_hotplug(vgpu, false);
 			else {
 				gvt_vgpu_err("invalid EDID link state %d\n",
 					regs->link_state);
@@ -806,7 +804,7 @@ static int intel_vgpu_create(struct mdev_device *mdev)
 		goto out;
 	}
 
-	vgpu = intel_gvt_ops->vgpu_create(gvt, type);
+	vgpu = intel_gvt_create_vgpu(gvt, type);
 	if (IS_ERR_OR_NULL(vgpu)) {
 		ret = vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
 		gvt_err("failed to create intel vgpu: %d\n", ret);
@@ -833,7 +831,7 @@ static int intel_vgpu_remove(struct mdev_device *mdev)
 	if (handle_valid(vgpu->handle))
 		return -EBUSY;
 
-	intel_gvt_ops->vgpu_destroy(vgpu);
+	intel_gvt_destroy_vgpu(vgpu);
 	return 0;
 }
 
@@ -936,7 +934,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
 	if (ret)
 		goto undo_group;
 
-	intel_gvt_ops->vgpu_activate(vgpu);
+	intel_gvt_activate_vgpu(vgpu);
 
 	atomic_set(&vdev->released, 0);
 	return ret;
@@ -981,7 +979,7 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
 	if (atomic_cmpxchg(&vdev->released, 0, 1))
 		return;
 
-	intel_gvt_ops->vgpu_release(vgpu);
+	intel_gvt_release_vgpu(vgpu);
 
 	ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_IOMMU_NOTIFY,
 					&vdev->iommu_notifier);
@@ -1055,10 +1053,10 @@ static int intel_vgpu_bar_rw(struct intel_vgpu *vgpu, int bar, u64 off,
 	int ret;
 
 	if (is_write)
-		ret = intel_gvt_ops->emulate_mmio_write(vgpu,
+		ret = intel_vgpu_emulate_mmio_write(vgpu,
 					bar_start + off, buf, count);
 	else
-		ret = intel_gvt_ops->emulate_mmio_read(vgpu,
+		ret = intel_vgpu_emulate_mmio_read(vgpu,
 					bar_start + off, buf, count);
 	return ret;
 }
@@ -1114,10 +1112,10 @@ static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
 	switch (index) {
 	case VFIO_PCI_CONFIG_REGION_INDEX:
 		if (is_write)
-			ret = intel_gvt_ops->emulate_cfg_write(vgpu, pos,
+			ret = intel_vgpu_emulate_cfg_write(vgpu, pos,
 						buf, count);
 		else
-			ret = intel_gvt_ops->emulate_cfg_read(vgpu, pos,
+			ret = intel_vgpu_emulate_cfg_read(vgpu, pos,
 						buf, count);
 		break;
 	case VFIO_PCI_BAR0_REGION_INDEX:
@@ -1685,7 +1683,7 @@ static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
 
 		return ret;
 	} else if (cmd == VFIO_DEVICE_RESET) {
-		intel_gvt_ops->vgpu_reset(vgpu);
+		intel_gvt_reset_vgpu(vgpu);
 		return 0;
 	} else if (cmd == VFIO_DEVICE_QUERY_GFX_PLANE) {
 		struct vfio_device_gfx_plane_info dmabuf;
@@ -1698,7 +1696,7 @@ static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
 		if (dmabuf.argsz < minsz)
 			return -EINVAL;
 
-		ret = intel_gvt_ops->vgpu_query_plane(vgpu, &dmabuf);
+		ret = intel_vgpu_query_plane(vgpu, &dmabuf);
 		if (ret != 0)
 			return ret;
 
@@ -1706,14 +1704,10 @@ static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
 								-EFAULT : 0;
 	} else if (cmd == VFIO_DEVICE_GET_GFX_DMABUF) {
 		__u32 dmabuf_id;
-		__s32 dmabuf_fd;
 
 		if (get_user(dmabuf_id, (__u32 __user *)arg))
 			return -EFAULT;
-
-		dmabuf_fd = intel_gvt_ops->vgpu_get_dmabuf(vgpu, dmabuf_id);
-		return dmabuf_fd;
-
+		return intel_vgpu_get_dmabuf(vgpu, dmabuf_id);
 	}
 
 	return -ENOTTY;
@@ -1764,7 +1758,7 @@ static struct mdev_parent_ops intel_vgpu_ops = {
 	.ioctl			= intel_vgpu_ioctl,
 };
 
-static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
+static int kvmgt_host_init(struct device *dev, void *gvt)
 {
 	int ret;
 
@@ -1772,7 +1766,6 @@ static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
 	if (ret)
 		return ret;
 
-	intel_gvt_ops = ops;
 	intel_vgpu_ops.supported_type_groups = gvt_vgpu_type_groups;
 
 	ret = mdev_register_device(dev, &intel_vgpu_ops);
@@ -1864,7 +1857,7 @@ static void kvmgt_page_track_write(struct kvm_vcpu *vcpu, gpa_t gpa,
 					struct kvmgt_guest_info, track_node);
 
 	if (kvmgt_gfn_is_write_protected(info, gpa_to_gfn(gpa)))
-		intel_gvt_ops->write_protect_handler(info->vgpu, gpa,
+		intel_vgpu_page_track_handler(info->vgpu, gpa,
 						     (void *)val, len);
 }
 
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 1b5617bb2745..0e3966e1fec8 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -51,13 +51,12 @@
  * Returns:
  * Zero on success, negative error code if failed
  */
-static inline int intel_gvt_hypervisor_host_init(struct device *dev,
-						 void *gvt, const void *ops)
+static inline int intel_gvt_hypervisor_host_init(struct device *dev, void *gvt)
 {
 	if (!intel_gvt_host.mpt->host_init)
 		return -ENODEV;
 
-	return intel_gvt_host.mpt->host_init(dev, gvt, ops);
+	return intel_gvt_host.mpt->host_init(dev, gvt);
 }
 
 /**
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 06/21] drm/i915/gvt: remove the map_gfn_to_mfn and set_trap_area ops
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (4 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 05/21] drm/i915/gvt: remove intel_gvt_ops Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 07/21] drm/i915/gvt: remove the unused from_virt_to_mfn op Christoph Hellwig
                   ` (16 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

The map_gfn_to_mfn and set_trap_area ops are never defined, so remove
them and clean up code that depends on them in the callers.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/cfg_space.c | 89 ++++++----------------------
 drivers/gpu/drm/i915/gvt/hypercall.h |  4 --
 drivers/gpu/drm/i915/gvt/mpt.h       | 44 --------------
 3 files changed, 17 insertions(+), 120 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/cfg_space.c b/drivers/gpu/drm/i915/gvt/cfg_space.c
index b490e3db2e38..dad3a6054335 100644
--- a/drivers/gpu/drm/i915/gvt/cfg_space.c
+++ b/drivers/gpu/drm/i915/gvt/cfg_space.c
@@ -129,60 +129,16 @@ int intel_vgpu_emulate_cfg_read(struct intel_vgpu *vgpu, unsigned int offset,
 	return 0;
 }
 
-static int map_aperture(struct intel_vgpu *vgpu, bool map)
+static void map_aperture(struct intel_vgpu *vgpu, bool map)
 {
-	phys_addr_t aperture_pa = vgpu_aperture_pa_base(vgpu);
-	unsigned long aperture_sz = vgpu_aperture_sz(vgpu);
-	u64 first_gfn;
-	u64 val;
-	int ret;
-
-	if (map == vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_APERTURE].tracked)
-		return 0;
-
-	val = vgpu_cfg_space(vgpu)[PCI_BASE_ADDRESS_2];
-	if (val & PCI_BASE_ADDRESS_MEM_TYPE_64)
-		val = *(u64 *)(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_2);
-	else
-		val = *(u32 *)(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_2);
-
-	first_gfn = (val + vgpu_aperture_offset(vgpu)) >> PAGE_SHIFT;
-
-	ret = intel_gvt_hypervisor_map_gfn_to_mfn(vgpu, first_gfn,
-						  aperture_pa >> PAGE_SHIFT,
-						  aperture_sz >> PAGE_SHIFT,
-						  map);
-	if (ret)
-		return ret;
-
-	vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_APERTURE].tracked = map;
-	return 0;
+	if (map != vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_APERTURE].tracked)
+		vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_APERTURE].tracked = map;
 }
 
-static int trap_gttmmio(struct intel_vgpu *vgpu, bool trap)
+static void trap_gttmmio(struct intel_vgpu *vgpu, bool trap)
 {
-	u64 start, end;
-	u64 val;
-	int ret;
-
-	if (trap == vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_GTTMMIO].tracked)
-		return 0;
-
-	val = vgpu_cfg_space(vgpu)[PCI_BASE_ADDRESS_0];
-	if (val & PCI_BASE_ADDRESS_MEM_TYPE_64)
-		start = *(u64 *)(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_0);
-	else
-		start = *(u32 *)(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_0);
-
-	start &= ~GENMASK(3, 0);
-	end = start + vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_GTTMMIO].size - 1;
-
-	ret = intel_gvt_hypervisor_set_trap_area(vgpu, start, end, trap);
-	if (ret)
-		return ret;
-
-	vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_GTTMMIO].tracked = trap;
-	return 0;
+	if (trap != vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_GTTMMIO].tracked)
+		vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_GTTMMIO].tracked = trap;
 }
 
 static int emulate_pci_command_write(struct intel_vgpu *vgpu,
@@ -191,26 +147,17 @@ static int emulate_pci_command_write(struct intel_vgpu *vgpu,
 	u8 old = vgpu_cfg_space(vgpu)[offset];
 	u8 new = *(u8 *)p_data;
 	u8 changed = old ^ new;
-	int ret;
 
 	vgpu_pci_cfg_mem_write(vgpu, offset, p_data, bytes);
 	if (!(changed & PCI_COMMAND_MEMORY))
 		return 0;
 
 	if (old & PCI_COMMAND_MEMORY) {
-		ret = trap_gttmmio(vgpu, false);
-		if (ret)
-			return ret;
-		ret = map_aperture(vgpu, false);
-		if (ret)
-			return ret;
+		trap_gttmmio(vgpu, false);
+		map_aperture(vgpu, false);
 	} else {
-		ret = trap_gttmmio(vgpu, true);
-		if (ret)
-			return ret;
-		ret = map_aperture(vgpu, true);
-		if (ret)
-			return ret;
+		trap_gttmmio(vgpu, true);
+		map_aperture(vgpu, true);
 	}
 
 	return 0;
@@ -230,13 +177,12 @@ static int emulate_pci_rom_bar_write(struct intel_vgpu *vgpu,
 	return 0;
 }
 
-static int emulate_pci_bar_write(struct intel_vgpu *vgpu, unsigned int offset,
+static void emulate_pci_bar_write(struct intel_vgpu *vgpu, unsigned int offset,
 	void *p_data, unsigned int bytes)
 {
 	u32 new = *(u32 *)(p_data);
 	bool lo = IS_ALIGNED(offset, 8);
 	u64 size;
-	int ret = 0;
 	bool mmio_enabled =
 		vgpu_cfg_space(vgpu)[PCI_COMMAND] & PCI_COMMAND_MEMORY;
 	struct intel_vgpu_pci_bar *bars = vgpu->cfg_space.bar;
@@ -259,14 +205,14 @@ static int emulate_pci_bar_write(struct intel_vgpu *vgpu, unsigned int offset,
 			 * Untrap the BAR, since guest hasn't configured a
 			 * valid GPA
 			 */
-			ret = trap_gttmmio(vgpu, false);
+			trap_gttmmio(vgpu, false);
 			break;
 		case PCI_BASE_ADDRESS_2:
 		case PCI_BASE_ADDRESS_3:
 			size = ~(bars[INTEL_GVT_PCI_BAR_APERTURE].size -1);
 			intel_vgpu_write_pci_bar(vgpu, offset,
 						size >> (lo ? 0 : 32), lo);
-			ret = map_aperture(vgpu, false);
+			map_aperture(vgpu, false);
 			break;
 		default:
 			/* Unimplemented BARs */
@@ -282,19 +228,18 @@ static int emulate_pci_bar_write(struct intel_vgpu *vgpu, unsigned int offset,
 			 */
 			trap_gttmmio(vgpu, false);
 			intel_vgpu_write_pci_bar(vgpu, offset, new, lo);
-			ret = trap_gttmmio(vgpu, mmio_enabled);
+			trap_gttmmio(vgpu, mmio_enabled);
 			break;
 		case PCI_BASE_ADDRESS_2:
 		case PCI_BASE_ADDRESS_3:
 			map_aperture(vgpu, false);
 			intel_vgpu_write_pci_bar(vgpu, offset, new, lo);
-			ret = map_aperture(vgpu, mmio_enabled);
+			map_aperture(vgpu, mmio_enabled);
 			break;
 		default:
 			intel_vgpu_write_pci_bar(vgpu, offset, new, lo);
 		}
 	}
-	return ret;
 }
 
 /**
@@ -336,8 +281,8 @@ int intel_vgpu_emulate_cfg_write(struct intel_vgpu *vgpu, unsigned int offset,
 	case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_5:
 		if (drm_WARN_ON(&i915->drm, !IS_ALIGNED(offset, 4)))
 			return -EINVAL;
-		return emulate_pci_bar_write(vgpu, offset, p_data, bytes);
-
+		emulate_pci_bar_write(vgpu, offset, p_data, bytes);
+		break;
 	case INTEL_GVT_PCI_SWSCI:
 		if (drm_WARN_ON(&i915->drm, !IS_ALIGNED(offset, 4)))
 			return -EINVAL;
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index 395bce9633fa..f1a4926f6f1b 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -62,10 +62,6 @@ struct intel_gvt_mpt {
 
 	int (*dma_pin_guest_page)(unsigned long handle, dma_addr_t dma_addr);
 
-	int (*map_gfn_to_mfn)(unsigned long handle, unsigned long gfn,
-			      unsigned long mfn, unsigned int nr, bool map);
-	int (*set_trap_area)(unsigned long handle, u64 start, u64 end,
-			     bool map);
 	int (*set_opregion)(void *vgpu);
 	int (*set_edid)(void *vgpu, int port_num);
 	int (*get_vfio_device)(void *vgpu);
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 0e3966e1fec8..bb0e9e71d13e 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -270,50 +270,6 @@ intel_gvt_hypervisor_dma_pin_guest_page(struct intel_vgpu *vgpu,
 	return intel_gvt_host.mpt->dma_pin_guest_page(vgpu->handle, dma_addr);
 }
 
-/**
- * intel_gvt_hypervisor_map_gfn_to_mfn - map a GFN region to MFN
- * @vgpu: a vGPU
- * @gfn: guest PFN
- * @mfn: host PFN
- * @nr: amount of PFNs
- * @map: map or unmap
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_map_gfn_to_mfn(
-		struct intel_vgpu *vgpu, unsigned long gfn,
-		unsigned long mfn, unsigned int nr,
-		bool map)
-{
-	/* a MPT implementation could have MMIO mapped elsewhere */
-	if (!intel_gvt_host.mpt->map_gfn_to_mfn)
-		return 0;
-
-	return intel_gvt_host.mpt->map_gfn_to_mfn(vgpu->handle, gfn, mfn, nr,
-						  map);
-}
-
-/**
- * intel_gvt_hypervisor_set_trap_area - Trap a guest PA region
- * @vgpu: a vGPU
- * @start: the beginning of the guest physical address region
- * @end: the end of the guest physical address region
- * @map: map or unmap
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_set_trap_area(
-		struct intel_vgpu *vgpu, u64 start, u64 end, bool map)
-{
-	/* a MPT implementation could have MMIO trapped elsewhere */
-	if (!intel_gvt_host.mpt->set_trap_area)
-		return 0;
-
-	return intel_gvt_host.mpt->set_trap_area(vgpu->handle, start, end, map);
-}
-
 /**
  * intel_gvt_hypervisor_set_opregion - Set opregion for guest
  * @vgpu: a vGPU
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 07/21] drm/i915/gvt: remove the unused from_virt_to_mfn op
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (5 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 06/21] drm/i915/gvt: remove the map_gfn_to_mfn and set_trap_area ops Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 08/21] drm/i915/gvt: merge struct kvmgt_vdev into struct intel_vgpu Christoph Hellwig
                   ` (15 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/hypercall.h |  1 -
 drivers/gpu/drm/i915/gvt/kvmgt.c     |  6 ------
 drivers/gpu/drm/i915/gvt/mpt.h       | 12 ------------
 3 files changed, 19 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index f1a4926f6f1b..27890a5e2d82 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -47,7 +47,6 @@ struct intel_gvt_mpt {
 	int (*attach_vgpu)(void *vgpu, unsigned long *handle);
 	void (*detach_vgpu)(void *vgpu);
 	int (*inject_msi)(unsigned long handle, u32 addr, u16 data);
-	unsigned long (*from_virt_to_mfn)(void *p);
 	int (*enable_page_track)(unsigned long handle, u64 gfn);
 	int (*disable_page_track)(unsigned long handle, u64 gfn);
 	int (*read_gpa)(unsigned long handle, unsigned long gpa, void *buf,
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 935d1c8287a4..d1f3f1253306 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -2173,11 +2173,6 @@ static int kvmgt_write_gpa(unsigned long handle, unsigned long gpa,
 	return kvmgt_rw_gpa(handle, gpa, buf, len, true);
 }
 
-static unsigned long kvmgt_virt_to_pfn(void *addr)
-{
-	return PFN_DOWN(__pa(addr));
-}
-
 static bool kvmgt_is_valid_gfn(unsigned long handle, unsigned long gfn)
 {
 	struct kvmgt_guest_info *info;
@@ -2204,7 +2199,6 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
 	.attach_vgpu = kvmgt_attach_vgpu,
 	.detach_vgpu = kvmgt_detach_vgpu,
 	.inject_msi = kvmgt_inject_msi,
-	.from_virt_to_mfn = kvmgt_virt_to_pfn,
 	.enable_page_track = kvmgt_page_track_add,
 	.disable_page_track = kvmgt_page_track_remove,
 	.read_gpa = kvmgt_read_gpa,
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index bb0e9e71d13e..6d062cf71de9 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -140,18 +140,6 @@ static inline int intel_gvt_hypervisor_inject_msi(struct intel_vgpu *vgpu)
 	return 0;
 }
 
-/**
- * intel_gvt_hypervisor_set_wp_page - translate a host VA into MFN
- * @p: host kernel virtual address
- *
- * Returns:
- * MFN on success, INTEL_GVT_INVALID_ADDR if failed.
- */
-static inline unsigned long intel_gvt_hypervisor_virt_to_mfn(void *p)
-{
-	return intel_gvt_host.mpt->from_virt_to_mfn(p);
-}
-
 /**
  * intel_gvt_hypervisor_enable_page_track - track a guest page
  * @vgpu: a vGPU
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 08/21] drm/i915/gvt: merge struct kvmgt_vdev into struct intel_vgpu
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (6 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 07/21] drm/i915/gvt: remove the unused from_virt_to_mfn op Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 09/21] drm/i915/gvt: merge struct kvmgt_guest_info into strut intel_vgpu Christoph Hellwig
                   ` (14 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Move towards having only a single structure for the per-VGPU state.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.h       |  31 ++-
 drivers/gpu/drm/i915/gvt/hypercall.h |   1 -
 drivers/gpu/drm/i915/gvt/kvmgt.c     | 288 ++++++++++-----------------
 drivers/gpu/drm/i915/gvt/mpt.h       |  16 --
 drivers/gpu/drm/i915/gvt/vgpu.c      |   8 +-
 5 files changed, 128 insertions(+), 216 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 40643fe14383..9f419e7f7a1b 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -206,21 +206,36 @@ struct intel_vgpu {
 
 	struct dentry *debugfs;
 
-	/* Hypervisor-specific device state. */
-	void *vdev;
-
 	struct list_head dmabuf_obj_list_head;
 	struct mutex dmabuf_lock;
 	struct idr object_idr;
 	struct intel_vgpu_vblank_timer vblank_timer;
 
 	u32 scan_nonprivbb;
-};
 
-static inline void *intel_vgpu_vdev(struct intel_vgpu *vgpu)
-{
-	return vgpu->vdev;
-}
+	struct mdev_device *mdev;
+	struct vfio_region *region;
+	int num_regions;
+	struct eventfd_ctx *intx_trigger;
+	struct eventfd_ctx *msi_trigger;
+
+	/*
+	 * Two caches are used to avoid mapping duplicated pages (eg.
+	 * scratch pages). This help to reduce dma setup overhead.
+	 */
+	struct rb_root gfn_cache;
+	struct rb_root dma_addr_cache;
+	unsigned long nr_cache_entries;
+	struct mutex cache_lock;
+
+	struct notifier_block iommu_notifier;
+	struct notifier_block group_notifier;
+	struct kvm *kvm;
+	struct work_struct release_work;
+	atomic_t released;
+	struct vfio_device *vfio_device;
+	struct vfio_group *vfio_group;
+};
 
 /* validating GM healthy status*/
 #define vgpu_is_vm_unhealthy(ret_val) \
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index 27890a5e2d82..eacee6f41f9c 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -44,7 +44,6 @@ struct device;
 struct intel_gvt_mpt {
 	int (*host_init)(struct device *dev, void *gvt);
 	void (*host_exit)(struct device *dev, void *gvt);
-	int (*attach_vgpu)(void *vgpu, unsigned long *handle);
 	void (*detach_vgpu)(void *vgpu);
 	int (*inject_msi)(unsigned long handle, u32 addr, u16 data);
 	int (*enable_page_track)(unsigned long handle, u64 gfn);
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index d1f3f1253306..bb536463a499 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -108,37 +108,6 @@ struct gvt_dma {
 	struct kref ref;
 };
 
-struct kvmgt_vdev {
-	struct intel_vgpu *vgpu;
-	struct mdev_device *mdev;
-	struct vfio_region *region;
-	int num_regions;
-	struct eventfd_ctx *intx_trigger;
-	struct eventfd_ctx *msi_trigger;
-
-	/*
-	 * Two caches are used to avoid mapping duplicated pages (eg.
-	 * scratch pages). This help to reduce dma setup overhead.
-	 */
-	struct rb_root gfn_cache;
-	struct rb_root dma_addr_cache;
-	unsigned long nr_cache_entries;
-	struct mutex cache_lock;
-
-	struct notifier_block iommu_notifier;
-	struct notifier_block group_notifier;
-	struct kvm *kvm;
-	struct work_struct release_work;
-	atomic_t released;
-	struct vfio_device *vfio_device;
-	struct vfio_group *vfio_group;
-};
-
-static inline struct kvmgt_vdev *kvmgt_vdev(struct intel_vgpu *vgpu)
-{
-	return intel_vgpu_vdev(vgpu);
-}
-
 static inline bool handle_valid(unsigned long handle)
 {
 	return !!(handle & ~0xff);
@@ -250,7 +219,6 @@ static void gvt_unpin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		unsigned long size)
 {
 	struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	int total_pages;
 	int npage;
 	int ret;
@@ -260,7 +228,7 @@ static void gvt_unpin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 	for (npage = 0; npage < total_pages; npage++) {
 		unsigned long cur_gfn = gfn + npage;
 
-		ret = vfio_group_unpin_pages(vdev->vfio_group, &cur_gfn, 1);
+		ret = vfio_group_unpin_pages(vgpu->vfio_group, &cur_gfn, 1);
 		drm_WARN_ON(&i915->drm, ret != 1);
 	}
 }
@@ -269,7 +237,6 @@ static void gvt_unpin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 static int gvt_pin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		unsigned long size, struct page **page)
 {
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	unsigned long base_pfn = 0;
 	int total_pages;
 	int npage;
@@ -284,7 +251,7 @@ static int gvt_pin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		unsigned long cur_gfn = gfn + npage;
 		unsigned long pfn;
 
-		ret = vfio_group_pin_pages(vdev->vfio_group, &cur_gfn, 1,
+		ret = vfio_group_pin_pages(vgpu->vfio_group, &cur_gfn, 1,
 					   IOMMU_READ | IOMMU_WRITE, &pfn);
 		if (ret != 1) {
 			gvt_vgpu_err("vfio_pin_pages failed for gfn 0x%lx, ret %d\n",
@@ -351,7 +318,7 @@ static void gvt_dma_unmap_page(struct intel_vgpu *vgpu, unsigned long gfn,
 static struct gvt_dma *__gvt_cache_find_dma_addr(struct intel_vgpu *vgpu,
 		dma_addr_t dma_addr)
 {
-	struct rb_node *node = kvmgt_vdev(vgpu)->dma_addr_cache.rb_node;
+	struct rb_node *node = vgpu->dma_addr_cache.rb_node;
 	struct gvt_dma *itr;
 
 	while (node) {
@@ -369,7 +336,7 @@ static struct gvt_dma *__gvt_cache_find_dma_addr(struct intel_vgpu *vgpu,
 
 static struct gvt_dma *__gvt_cache_find_gfn(struct intel_vgpu *vgpu, gfn_t gfn)
 {
-	struct rb_node *node = kvmgt_vdev(vgpu)->gfn_cache.rb_node;
+	struct rb_node *node = vgpu->gfn_cache.rb_node;
 	struct gvt_dma *itr;
 
 	while (node) {
@@ -390,7 +357,6 @@ static int __gvt_cache_add(struct intel_vgpu *vgpu, gfn_t gfn,
 {
 	struct gvt_dma *new, *itr;
 	struct rb_node **link, *parent = NULL;
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 
 	new = kzalloc(sizeof(struct gvt_dma), GFP_KERNEL);
 	if (!new)
@@ -403,7 +369,7 @@ static int __gvt_cache_add(struct intel_vgpu *vgpu, gfn_t gfn,
 	kref_init(&new->ref);
 
 	/* gfn_cache maps gfn to struct gvt_dma. */
-	link = &vdev->gfn_cache.rb_node;
+	link = &vgpu->gfn_cache.rb_node;
 	while (*link) {
 		parent = *link;
 		itr = rb_entry(parent, struct gvt_dma, gfn_node);
@@ -414,11 +380,11 @@ static int __gvt_cache_add(struct intel_vgpu *vgpu, gfn_t gfn,
 			link = &parent->rb_right;
 	}
 	rb_link_node(&new->gfn_node, parent, link);
-	rb_insert_color(&new->gfn_node, &vdev->gfn_cache);
+	rb_insert_color(&new->gfn_node, &vgpu->gfn_cache);
 
 	/* dma_addr_cache maps dma addr to struct gvt_dma. */
 	parent = NULL;
-	link = &vdev->dma_addr_cache.rb_node;
+	link = &vgpu->dma_addr_cache.rb_node;
 	while (*link) {
 		parent = *link;
 		itr = rb_entry(parent, struct gvt_dma, dma_addr_node);
@@ -429,51 +395,46 @@ static int __gvt_cache_add(struct intel_vgpu *vgpu, gfn_t gfn,
 			link = &parent->rb_right;
 	}
 	rb_link_node(&new->dma_addr_node, parent, link);
-	rb_insert_color(&new->dma_addr_node, &vdev->dma_addr_cache);
+	rb_insert_color(&new->dma_addr_node, &vgpu->dma_addr_cache);
 
-	vdev->nr_cache_entries++;
+	vgpu->nr_cache_entries++;
 	return 0;
 }
 
 static void __gvt_cache_remove_entry(struct intel_vgpu *vgpu,
 				struct gvt_dma *entry)
 {
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
-
-	rb_erase(&entry->gfn_node, &vdev->gfn_cache);
-	rb_erase(&entry->dma_addr_node, &vdev->dma_addr_cache);
+	rb_erase(&entry->gfn_node, &vgpu->gfn_cache);
+	rb_erase(&entry->dma_addr_node, &vgpu->dma_addr_cache);
 	kfree(entry);
-	vdev->nr_cache_entries--;
+	vgpu->nr_cache_entries--;
 }
 
 static void gvt_cache_destroy(struct intel_vgpu *vgpu)
 {
 	struct gvt_dma *dma;
 	struct rb_node *node = NULL;
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 
 	for (;;) {
-		mutex_lock(&vdev->cache_lock);
-		node = rb_first(&vdev->gfn_cache);
+		mutex_lock(&vgpu->cache_lock);
+		node = rb_first(&vgpu->gfn_cache);
 		if (!node) {
-			mutex_unlock(&vdev->cache_lock);
+			mutex_unlock(&vgpu->cache_lock);
 			break;
 		}
 		dma = rb_entry(node, struct gvt_dma, gfn_node);
 		gvt_dma_unmap_page(vgpu, dma->gfn, dma->dma_addr, dma->size);
 		__gvt_cache_remove_entry(vgpu, dma);
-		mutex_unlock(&vdev->cache_lock);
+		mutex_unlock(&vgpu->cache_lock);
 	}
 }
 
 static void gvt_cache_init(struct intel_vgpu *vgpu)
 {
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
-
-	vdev->gfn_cache = RB_ROOT;
-	vdev->dma_addr_cache = RB_ROOT;
-	vdev->nr_cache_entries = 0;
-	mutex_init(&vdev->cache_lock);
+	vgpu->gfn_cache = RB_ROOT;
+	vgpu->dma_addr_cache = RB_ROOT;
+	vgpu->nr_cache_entries = 0;
+	mutex_init(&vgpu->cache_lock);
 }
 
 static void kvmgt_protect_table_init(struct kvmgt_guest_info *info)
@@ -547,18 +508,17 @@ static void kvmgt_protect_table_del(struct kvmgt_guest_info *info,
 static size_t intel_vgpu_reg_rw_opregion(struct intel_vgpu *vgpu, char *buf,
 		size_t count, loff_t *ppos, bool iswrite)
 {
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) -
 			VFIO_PCI_NUM_REGIONS;
-	void *base = vdev->region[i].data;
+	void *base = vgpu->region[i].data;
 	loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK;
 
 
-	if (pos >= vdev->region[i].size || iswrite) {
+	if (pos >= vgpu->region[i].size || iswrite) {
 		gvt_vgpu_err("invalid op or offset for Intel vgpu OpRegion\n");
 		return -EINVAL;
 	}
-	count = min(count, (size_t)(vdev->region[i].size - pos));
+	count = min(count, (size_t)(vgpu->region[i].size - pos));
 	memcpy(buf, base + pos, count);
 
 	return count;
@@ -651,8 +611,7 @@ static size_t intel_vgpu_reg_rw_edid(struct intel_vgpu *vgpu, char *buf,
 	int ret;
 	unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) -
 			VFIO_PCI_NUM_REGIONS;
-	struct vfio_edid_region *region =
-		(struct vfio_edid_region *)kvmgt_vdev(vgpu)->region[i].data;
+	struct vfio_edid_region *region = vgpu->region[i].data;
 	loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK;
 
 	if (pos < region->vfio_edid_regs.edid_offset) {
@@ -684,34 +643,32 @@ static int intel_vgpu_register_reg(struct intel_vgpu *vgpu,
 		const struct intel_vgpu_regops *ops,
 		size_t size, u32 flags, void *data)
 {
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	struct vfio_region *region;
 
-	region = krealloc(vdev->region,
-			(vdev->num_regions + 1) * sizeof(*region),
+	region = krealloc(vgpu->region,
+			(vgpu->num_regions + 1) * sizeof(*region),
 			GFP_KERNEL);
 	if (!region)
 		return -ENOMEM;
 
-	vdev->region = region;
-	vdev->region[vdev->num_regions].type = type;
-	vdev->region[vdev->num_regions].subtype = subtype;
-	vdev->region[vdev->num_regions].ops = ops;
-	vdev->region[vdev->num_regions].size = size;
-	vdev->region[vdev->num_regions].flags = flags;
-	vdev->region[vdev->num_regions].data = data;
-	vdev->num_regions++;
+	vgpu->region = region;
+	vgpu->region[vgpu->num_regions].type = type;
+	vgpu->region[vgpu->num_regions].subtype = subtype;
+	vgpu->region[vgpu->num_regions].ops = ops;
+	vgpu->region[vgpu->num_regions].size = size;
+	vgpu->region[vgpu->num_regions].flags = flags;
+	vgpu->region[vgpu->num_regions].data = data;
+	vgpu->num_regions++;
 	return 0;
 }
 
 static int kvmgt_get_vfio_device(void *p_vgpu)
 {
 	struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 
-	vdev->vfio_device = vfio_device_get_from_dev(
-		mdev_dev(vdev->mdev));
-	if (!vdev->vfio_device) {
+	vgpu->vfio_device = vfio_device_get_from_dev(
+		mdev_dev(vgpu->mdev));
+	if (!vgpu->vfio_device) {
 		gvt_vgpu_err("failed to get vfio device\n");
 		return -ENODEV;
 	}
@@ -777,14 +734,14 @@ static int kvmgt_set_edid(void *p_vgpu, int port_num)
 	return ret;
 }
 
-static void kvmgt_put_vfio_device(void *vgpu)
+static void kvmgt_put_vfio_device(void *data)
 {
-	struct kvmgt_vdev *vdev = kvmgt_vdev((struct intel_vgpu *)vgpu);
+	struct intel_vgpu *vgpu = data;
 
-	if (WARN_ON(!vdev->vfio_device))
+	if (WARN_ON(!vgpu->vfio_device))
 		return;
 
-	vfio_device_put(vdev->vfio_device);
+	vfio_device_put(vgpu->vfio_device);
 }
 
 static int intel_vgpu_create(struct mdev_device *mdev)
@@ -811,9 +768,9 @@ static int intel_vgpu_create(struct mdev_device *mdev)
 		goto out;
 	}
 
-	INIT_WORK(&kvmgt_vdev(vgpu)->release_work, intel_vgpu_release_work);
+	INIT_WORK(&vgpu->release_work, intel_vgpu_release_work);
 
-	kvmgt_vdev(vgpu)->mdev = mdev;
+	vgpu->mdev = mdev;
 	mdev_set_drvdata(mdev, vgpu);
 
 	gvt_dbg_core("intel_vgpu_create succeeded for mdev: %s\n",
@@ -838,10 +795,8 @@ static int intel_vgpu_remove(struct mdev_device *mdev)
 static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
 				     unsigned long action, void *data)
 {
-	struct kvmgt_vdev *vdev = container_of(nb,
-					       struct kvmgt_vdev,
-					       iommu_notifier);
-	struct intel_vgpu *vgpu = vdev->vgpu;
+	struct intel_vgpu *vgpu =
+		container_of(nb, struct intel_vgpu, iommu_notifier);
 
 	if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
 		struct vfio_iommu_type1_dma_unmap *unmap = data;
@@ -851,7 +806,7 @@ static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
 		iov_pfn = unmap->iova >> PAGE_SHIFT;
 		end_iov_pfn = iov_pfn + unmap->size / PAGE_SIZE;
 
-		mutex_lock(&vdev->cache_lock);
+		mutex_lock(&vgpu->cache_lock);
 		for (; iov_pfn < end_iov_pfn; iov_pfn++) {
 			entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
 			if (!entry)
@@ -861,7 +816,7 @@ static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
 					   entry->size);
 			__gvt_cache_remove_entry(vgpu, entry);
 		}
-		mutex_unlock(&vdev->cache_lock);
+		mutex_unlock(&vgpu->cache_lock);
 	}
 
 	return NOTIFY_OK;
@@ -870,16 +825,15 @@ static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
 static int intel_vgpu_group_notifier(struct notifier_block *nb,
 				     unsigned long action, void *data)
 {
-	struct kvmgt_vdev *vdev = container_of(nb,
-					       struct kvmgt_vdev,
-					       group_notifier);
+	struct intel_vgpu *vgpu =
+		container_of(nb, struct intel_vgpu, group_notifier);
 
 	/* the only action we care about */
 	if (action == VFIO_GROUP_NOTIFY_SET_KVM) {
-		vdev->kvm = data;
+		vgpu->kvm = data;
 
 		if (!data)
-			schedule_work(&vdev->release_work);
+			schedule_work(&vgpu->release_work);
 	}
 
 	return NOTIFY_OK;
@@ -888,17 +842,16 @@ static int intel_vgpu_group_notifier(struct notifier_block *nb,
 static int intel_vgpu_open(struct mdev_device *mdev)
 {
 	struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	unsigned long events;
 	int ret;
 	struct vfio_group *vfio_group;
 
-	vdev->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier;
-	vdev->group_notifier.notifier_call = intel_vgpu_group_notifier;
+	vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier;
+	vgpu->group_notifier.notifier_call = intel_vgpu_group_notifier;
 
 	events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
 	ret = vfio_register_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, &events,
-				&vdev->iommu_notifier);
+				&vgpu->iommu_notifier);
 	if (ret != 0) {
 		gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
 			ret);
@@ -907,7 +860,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
 
 	events = VFIO_GROUP_NOTIFY_SET_KVM;
 	ret = vfio_register_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &events,
-				&vdev->group_notifier);
+				&vgpu->group_notifier);
 	if (ret != 0) {
 		gvt_vgpu_err("vfio_register_notifier for group failed: %d\n",
 			ret);
@@ -920,7 +873,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
 		gvt_vgpu_err("vfio_group_get_external_user_from_dev failed\n");
 		goto undo_register;
 	}
-	vdev->vfio_group = vfio_group;
+	vgpu->vfio_group = vfio_group;
 
 	/* Take a module reference as mdev core doesn't take
 	 * a reference for vendor driver.
@@ -936,39 +889,37 @@ static int intel_vgpu_open(struct mdev_device *mdev)
 
 	intel_gvt_activate_vgpu(vgpu);
 
-	atomic_set(&vdev->released, 0);
+	atomic_set(&vgpu->released, 0);
 	return ret;
 
 undo_group:
-	vfio_group_put_external_user(vdev->vfio_group);
-	vdev->vfio_group = NULL;
+	vfio_group_put_external_user(vgpu->vfio_group);
+	vgpu->vfio_group = NULL;
 
 undo_register:
 	vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY,
-					&vdev->group_notifier);
+					&vgpu->group_notifier);
 
 undo_iommu:
 	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
-					&vdev->iommu_notifier);
+					&vgpu->iommu_notifier);
 out:
 	return ret;
 }
 
 static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
 {
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	struct eventfd_ctx *trigger;
 
-	trigger = vdev->msi_trigger;
+	trigger = vgpu->msi_trigger;
 	if (trigger) {
 		eventfd_ctx_put(trigger);
-		vdev->msi_trigger = NULL;
+		vgpu->msi_trigger = NULL;
 	}
 }
 
 static void __intel_vgpu_release(struct intel_vgpu *vgpu)
 {
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
 	struct kvmgt_guest_info *info;
 	int ret;
@@ -976,18 +927,18 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
 	if (!handle_valid(vgpu->handle))
 		return;
 
-	if (atomic_cmpxchg(&vdev->released, 0, 1))
+	if (atomic_cmpxchg(&vgpu->released, 0, 1))
 		return;
 
 	intel_gvt_release_vgpu(vgpu);
 
-	ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_IOMMU_NOTIFY,
-					&vdev->iommu_notifier);
+	ret = vfio_unregister_notifier(mdev_dev(vgpu->mdev), VFIO_IOMMU_NOTIFY,
+					&vgpu->iommu_notifier);
 	drm_WARN(&i915->drm, ret,
 		 "vfio_unregister_notifier for iommu failed: %d\n", ret);
 
-	ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_GROUP_NOTIFY,
-					&vdev->group_notifier);
+	ret = vfio_unregister_notifier(mdev_dev(vgpu->mdev), VFIO_GROUP_NOTIFY,
+					&vgpu->group_notifier);
 	drm_WARN(&i915->drm, ret,
 		 "vfio_unregister_notifier for group failed: %d\n", ret);
 
@@ -998,9 +949,9 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
 	kvmgt_guest_exit(info);
 
 	intel_vgpu_release_msi_eventfd_ctx(vgpu);
-	vfio_group_put_external_user(vdev->vfio_group);
+	vfio_group_put_external_user(vgpu->vfio_group);
 
-	vdev->kvm = NULL;
+	vgpu->kvm = NULL;
 	vgpu->handle = 0;
 }
 
@@ -1013,10 +964,10 @@ static void intel_vgpu_release(struct mdev_device *mdev)
 
 static void intel_vgpu_release_work(struct work_struct *work)
 {
-	struct kvmgt_vdev *vdev = container_of(work, struct kvmgt_vdev,
-					       release_work);
+	struct intel_vgpu *vgpu =
+		container_of(work, struct intel_vgpu, release_work);
 
-	__intel_vgpu_release(vdev->vgpu);
+	__intel_vgpu_release(vgpu);
 }
 
 static u64 intel_vgpu_get_bar_addr(struct intel_vgpu *vgpu, int bar)
@@ -1098,13 +1049,12 @@ static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
 			size_t count, loff_t *ppos, bool is_write)
 {
 	struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
 	u64 pos = *ppos & VFIO_PCI_OFFSET_MASK;
 	int ret = -EINVAL;
 
 
-	if (index >= VFIO_PCI_NUM_REGIONS + vdev->num_regions) {
+	if (index >= VFIO_PCI_NUM_REGIONS + vgpu->num_regions) {
 		gvt_vgpu_err("invalid index: %u\n", index);
 		return -EINVAL;
 	}
@@ -1133,11 +1083,11 @@ static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
 	case VFIO_PCI_ROM_REGION_INDEX:
 		break;
 	default:
-		if (index >= VFIO_PCI_NUM_REGIONS + vdev->num_regions)
+		if (index >= VFIO_PCI_NUM_REGIONS + vgpu->num_regions)
 			return -EINVAL;
 
 		index -= VFIO_PCI_NUM_REGIONS;
-		return vdev->region[index].ops->rw(vgpu, buf, count,
+		return vgpu->region[index].ops->rw(vgpu, buf, count,
 				ppos, is_write);
 	}
 
@@ -1390,7 +1340,7 @@ static int intel_vgpu_set_msi_trigger(struct intel_vgpu *vgpu,
 			gvt_vgpu_err("eventfd_ctx_fdget failed\n");
 			return PTR_ERR(trigger);
 		}
-		kvmgt_vdev(vgpu)->msi_trigger = trigger;
+		vgpu->msi_trigger = trigger;
 	} else if ((flags & VFIO_IRQ_SET_DATA_NONE) && !count)
 		intel_vgpu_release_msi_eventfd_ctx(vgpu);
 
@@ -1442,7 +1392,6 @@ static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
 			     unsigned long arg)
 {
 	struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 	unsigned long minsz;
 
 	gvt_dbg_core("vgpu%d ioctl, cmd: %d\n", vgpu->id, cmd);
@@ -1461,7 +1410,7 @@ static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
 		info.flags = VFIO_DEVICE_FLAGS_PCI;
 		info.flags |= VFIO_DEVICE_FLAGS_RESET;
 		info.num_regions = VFIO_PCI_NUM_REGIONS +
-				vdev->num_regions;
+				vgpu->num_regions;
 		info.num_irqs = VFIO_PCI_NUM_IRQS;
 
 		return copy_to_user((void __user *)arg, &info, minsz) ?
@@ -1552,22 +1501,22 @@ static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
 					.header.version = 1 };
 
 				if (info.index >= VFIO_PCI_NUM_REGIONS +
-						vdev->num_regions)
+						vgpu->num_regions)
 					return -EINVAL;
 				info.index =
 					array_index_nospec(info.index,
 							VFIO_PCI_NUM_REGIONS +
-							vdev->num_regions);
+							vgpu->num_regions);
 
 				i = info.index - VFIO_PCI_NUM_REGIONS;
 
 				info.offset =
 					VFIO_PCI_INDEX_TO_OFFSET(info.index);
-				info.size = vdev->region[i].size;
-				info.flags = vdev->region[i].flags;
+				info.size = vgpu->region[i].size;
+				info.flags = vgpu->region[i].flags;
 
-				cap_type.type = vdev->region[i].type;
-				cap_type.subtype = vdev->region[i].subtype;
+				cap_type.type = vgpu->region[i].type;
+				cap_type.subtype = vgpu->region[i].subtype;
 
 				ret = vfio_info_add_capability(&caps,
 							&cap_type.header,
@@ -1909,15 +1858,13 @@ static int kvmgt_guest_init(struct mdev_device *mdev)
 {
 	struct kvmgt_guest_info *info;
 	struct intel_vgpu *vgpu;
-	struct kvmgt_vdev *vdev;
 	struct kvm *kvm;
 
 	vgpu = mdev_get_drvdata(mdev);
 	if (handle_valid(vgpu->handle))
 		return -EEXIST;
 
-	vdev = kvmgt_vdev(vgpu);
-	kvm = vdev->kvm;
+	kvm = vgpu->kvm;
 	if (!kvm || kvm->mm != current->mm) {
 		gvt_vgpu_err("KVM is required to use Intel vGPU\n");
 		return -ESRCH;
@@ -1943,7 +1890,7 @@ static int kvmgt_guest_init(struct mdev_device *mdev)
 	kvm_page_track_register_notifier(kvm, &info->track_node);
 
 	debugfs_create_ulong(KVMGT_DEBUGFS_FILENAME, 0444, vgpu->debugfs,
-			     &vdev->nr_cache_entries);
+			     &vgpu->nr_cache_entries);
 	return 0;
 }
 
@@ -1961,52 +1908,33 @@ static bool kvmgt_guest_exit(struct kvmgt_guest_info *info)
 	return true;
 }
 
-static int kvmgt_attach_vgpu(void *p_vgpu, unsigned long *handle)
-{
-	struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
-
-	vgpu->vdev = kzalloc(sizeof(struct kvmgt_vdev), GFP_KERNEL);
-
-	if (!vgpu->vdev)
-		return -ENOMEM;
-
-	kvmgt_vdev(vgpu)->vgpu = vgpu;
-
-	return 0;
-}
-
 static void kvmgt_detach_vgpu(void *p_vgpu)
 {
 	int i;
 	struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
-	struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
 
-	if (!vdev->region)
+	if (!vgpu->region)
 		return;
 
-	for (i = 0; i < vdev->num_regions; i++)
-		if (vdev->region[i].ops->release)
-			vdev->region[i].ops->release(vgpu,
-					&vdev->region[i]);
-	vdev->num_regions = 0;
-	kfree(vdev->region);
-	vdev->region = NULL;
-
-	kfree(vdev);
+	for (i = 0; i < vgpu->num_regions; i++)
+		if (vgpu->region[i].ops->release)
+			vgpu->region[i].ops->release(vgpu,
+					&vgpu->region[i]);
+	vgpu->num_regions = 0;
+	kfree(vgpu->region);
+	vgpu->region = NULL;
 }
 
 static int kvmgt_inject_msi(unsigned long handle, u32 addr, u16 data)
 {
 	struct kvmgt_guest_info *info;
 	struct intel_vgpu *vgpu;
-	struct kvmgt_vdev *vdev;
 
 	if (!handle_valid(handle))
 		return -ESRCH;
 
 	info = (struct kvmgt_guest_info *)handle;
 	vgpu = info->vgpu;
-	vdev = kvmgt_vdev(vgpu);
 
 	/*
 	 * When guest is poweroff, msi_trigger is set to NULL, but vgpu's
@@ -2017,10 +1945,10 @@ static int kvmgt_inject_msi(unsigned long handle, u32 addr, u16 data)
 	 * enabled by guest. so if msi_trigger is null, success is still
 	 * returned and don't inject interrupt into guest.
 	 */
-	if (vdev->msi_trigger == NULL)
+	if (vgpu->msi_trigger == NULL)
 		return 0;
 
-	if (eventfd_signal(vdev->msi_trigger, 1) == 1)
+	if (eventfd_signal(vgpu->msi_trigger, 1) == 1)
 		return 0;
 
 	return -EFAULT;
@@ -2047,7 +1975,6 @@ static int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,
 		unsigned long size, dma_addr_t *dma_addr)
 {
 	struct intel_vgpu *vgpu;
-	struct kvmgt_vdev *vdev;
 	struct gvt_dma *entry;
 	int ret;
 
@@ -2055,9 +1982,8 @@ static int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,
 		return -EINVAL;
 
 	vgpu = ((struct kvmgt_guest_info *)handle)->vgpu;
-	vdev = kvmgt_vdev(vgpu);
 
-	mutex_lock(&vdev->cache_lock);
+	mutex_lock(&vgpu->cache_lock);
 
 	entry = __gvt_cache_find_gfn(vgpu, gfn);
 	if (!entry) {
@@ -2085,20 +2011,19 @@ static int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,
 		*dma_addr = entry->dma_addr;
 	}
 
-	mutex_unlock(&vdev->cache_lock);
+	mutex_unlock(&vgpu->cache_lock);
 	return 0;
 
 err_unmap:
 	gvt_dma_unmap_page(vgpu, gfn, *dma_addr, size);
 err_unlock:
-	mutex_unlock(&vdev->cache_lock);
+	mutex_unlock(&vgpu->cache_lock);
 	return ret;
 }
 
 static int kvmgt_dma_pin_guest_page(unsigned long handle, dma_addr_t dma_addr)
 {
 	struct kvmgt_guest_info *info;
-	struct kvmgt_vdev *vdev;
 	struct gvt_dma *entry;
 	int ret = 0;
 
@@ -2106,15 +2031,14 @@ static int kvmgt_dma_pin_guest_page(unsigned long handle, dma_addr_t dma_addr)
 		return -ENODEV;
 
 	info = (struct kvmgt_guest_info *)handle;
-	vdev = kvmgt_vdev(info->vgpu);
 
-	mutex_lock(&vdev->cache_lock);
+	mutex_lock(&info->vgpu->cache_lock);
 	entry = __gvt_cache_find_dma_addr(info->vgpu, dma_addr);
 	if (entry)
 		kref_get(&entry->ref);
 	else
 		ret = -ENOMEM;
-	mutex_unlock(&vdev->cache_lock);
+	mutex_unlock(&info->vgpu->cache_lock);
 
 	return ret;
 }
@@ -2131,20 +2055,18 @@ static void __gvt_dma_release(struct kref *ref)
 static void kvmgt_dma_unmap_guest_page(unsigned long handle, dma_addr_t dma_addr)
 {
 	struct intel_vgpu *vgpu;
-	struct kvmgt_vdev *vdev;
 	struct gvt_dma *entry;
 
 	if (!handle_valid(handle))
 		return;
 
 	vgpu = ((struct kvmgt_guest_info *)handle)->vgpu;
-	vdev = kvmgt_vdev(vgpu);
 
-	mutex_lock(&vdev->cache_lock);
+	mutex_lock(&vgpu->cache_lock);
 	entry = __gvt_cache_find_dma_addr(vgpu, dma_addr);
 	if (entry)
 		kref_put(&entry->ref, __gvt_dma_release);
-	mutex_unlock(&vdev->cache_lock);
+	mutex_unlock(&vgpu->cache_lock);
 }
 
 static int kvmgt_rw_gpa(unsigned long handle, unsigned long gpa,
@@ -2157,8 +2079,7 @@ static int kvmgt_rw_gpa(unsigned long handle, unsigned long gpa,
 
 	info = (struct kvmgt_guest_info *)handle;
 
-	return vfio_dma_rw(kvmgt_vdev(info->vgpu)->vfio_group,
-			   gpa, buf, len, write);
+	return vfio_dma_rw(info->vgpu->vfio_group, gpa, buf, len, write);
 }
 
 static int kvmgt_read_gpa(unsigned long handle, unsigned long gpa,
@@ -2196,7 +2117,6 @@ static bool kvmgt_is_valid_gfn(unsigned long handle, unsigned long gfn)
 static const struct intel_gvt_mpt kvmgt_mpt = {
 	.host_init = kvmgt_host_init,
 	.host_exit = kvmgt_host_exit,
-	.attach_vgpu = kvmgt_attach_vgpu,
 	.detach_vgpu = kvmgt_detach_vgpu,
 	.inject_msi = kvmgt_inject_msi,
 	.enable_page_track = kvmgt_page_track_add,
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 6d062cf71de9..8a659301d78b 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -71,22 +71,6 @@ static inline void intel_gvt_hypervisor_host_exit(struct device *dev, void *gvt)
 	intel_gvt_host.mpt->host_exit(dev, gvt);
 }
 
-/**
- * intel_gvt_hypervisor_attach_vgpu - call hypervisor to initialize vGPU
- * related stuffs inside hypervisor.
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_attach_vgpu(struct intel_vgpu *vgpu)
-{
-	/* optional to provide */
-	if (!intel_gvt_host.mpt->attach_vgpu)
-		return 0;
-
-	return intel_gvt_host.mpt->attach_vgpu(vgpu, &vgpu->handle);
-}
-
 /**
  * intel_gvt_hypervisor_detach_vgpu - call hypervisor to release vGPU
  * related stuffs inside hypervisor.
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index fa6b92615799..7b9a571c0dc9 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -405,13 +405,9 @@ static struct intel_vgpu *__intel_gvt_create_vgpu(struct intel_gvt *gvt,
 
 	populate_pvinfo_page(vgpu);
 
-	ret = intel_gvt_hypervisor_attach_vgpu(vgpu);
-	if (ret)
-		goto out_clean_vgpu_resource;
-
 	ret = intel_vgpu_init_gtt(vgpu);
 	if (ret)
-		goto out_detach_hypervisor_vgpu;
+		goto out_clean_vgpu_resource;
 
 	ret = intel_vgpu_init_opregion(vgpu);
 	if (ret)
@@ -454,8 +450,6 @@ static struct intel_vgpu *__intel_gvt_create_vgpu(struct intel_gvt *gvt,
 	intel_vgpu_clean_opregion(vgpu);
 out_clean_gtt:
 	intel_vgpu_clean_gtt(vgpu);
-out_detach_hypervisor_vgpu:
-	intel_gvt_hypervisor_detach_vgpu(vgpu);
 out_clean_vgpu_resource:
 	intel_vgpu_free_resource(vgpu);
 out_clean_vgpu_mmio:
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 09/21] drm/i915/gvt: merge struct kvmgt_guest_info into strut intel_vgpu
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (7 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 08/21] drm/i915/gvt: merge struct kvmgt_vdev into struct intel_vgpu Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 10/21] drm/i915/gvt: remove vgpu->handle Christoph Hellwig
                   ` (13 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Consolidate the per-VGPU structures into a single one.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.h   |   8 +++
 drivers/gpu/drm/i915/gvt/kvmgt.c | 117 ++++++++++++-------------------
 2 files changed, 52 insertions(+), 73 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 9f419e7f7a1b..a4691f0607e6 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -34,6 +34,7 @@
 #define _GVT_H_
 
 #include <uapi/linux/pci_regs.h>
+#include <linux/kvm_host.h>
 
 #include "i915_drv.h"
 
@@ -173,6 +174,8 @@ struct intel_vgpu_submission {
 	} last_ctx[I915_NUM_ENGINES];
 };
 
+#define KVMGT_DEBUGFS_FILENAME		"kvmgt_nr_cache_entries"
+
 struct intel_vgpu {
 	struct intel_gvt *gvt;
 	struct mutex vgpu_lock;
@@ -235,6 +238,11 @@ struct intel_vgpu {
 	atomic_t released;
 	struct vfio_device *vfio_device;
 	struct vfio_group *vfio_group;
+
+	struct kvm_page_track_notifier_node track_node;
+#define NR_BKT (1 << 18)
+	struct hlist_head ptable[NR_BKT];
+#undef NR_BKT
 };
 
 /* validating GM healthy status*/
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index bb536463a499..785647e776d2 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -39,7 +39,6 @@
 #include <linux/spinlock.h>
 #include <linux/eventfd.h>
 #include <linux/uuid.h>
-#include <linux/kvm_host.h>
 #include <linux/vfio.h>
 #include <linux/mdev.h>
 #include <linux/debugfs.h>
@@ -88,16 +87,6 @@ struct kvmgt_pgfn {
 	struct hlist_node hnode;
 };
 
-#define KVMGT_DEBUGFS_FILENAME "kvmgt_nr_cache_entries"
-struct kvmgt_guest_info {
-	struct kvm *kvm;
-	struct intel_vgpu *vgpu;
-	struct kvm_page_track_notifier_node track_node;
-#define NR_BKT (1 << 18)
-	struct hlist_head ptable[NR_BKT];
-#undef NR_BKT
-};
-
 struct gvt_dma {
 	struct intel_vgpu *vgpu;
 	struct rb_node gfn_node;
@@ -213,7 +202,7 @@ static void intel_gvt_cleanup_vgpu_type_groups(struct intel_gvt *gvt)
 
 static int kvmgt_guest_init(struct mdev_device *mdev);
 static void intel_vgpu_release_work(struct work_struct *work);
-static bool kvmgt_guest_exit(struct kvmgt_guest_info *info);
+static bool kvmgt_guest_exit(struct intel_vgpu *info);
 
 static void gvt_unpin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		unsigned long size)
@@ -437,12 +426,12 @@ static void gvt_cache_init(struct intel_vgpu *vgpu)
 	mutex_init(&vgpu->cache_lock);
 }
 
-static void kvmgt_protect_table_init(struct kvmgt_guest_info *info)
+static void kvmgt_protect_table_init(struct intel_vgpu *info)
 {
 	hash_init(info->ptable);
 }
 
-static void kvmgt_protect_table_destroy(struct kvmgt_guest_info *info)
+static void kvmgt_protect_table_destroy(struct intel_vgpu *info)
 {
 	struct kvmgt_pgfn *p;
 	struct hlist_node *tmp;
@@ -455,7 +444,7 @@ static void kvmgt_protect_table_destroy(struct kvmgt_guest_info *info)
 }
 
 static struct kvmgt_pgfn *
-__kvmgt_protect_table_find(struct kvmgt_guest_info *info, gfn_t gfn)
+__kvmgt_protect_table_find(struct intel_vgpu *info, gfn_t gfn)
 {
 	struct kvmgt_pgfn *p, *res = NULL;
 
@@ -469,8 +458,7 @@ __kvmgt_protect_table_find(struct kvmgt_guest_info *info, gfn_t gfn)
 	return res;
 }
 
-static bool kvmgt_gfn_is_write_protected(struct kvmgt_guest_info *info,
-				gfn_t gfn)
+static bool kvmgt_gfn_is_write_protected(struct intel_vgpu *info, gfn_t gfn)
 {
 	struct kvmgt_pgfn *p;
 
@@ -478,7 +466,7 @@ static bool kvmgt_gfn_is_write_protected(struct kvmgt_guest_info *info,
 	return !!p;
 }
 
-static void kvmgt_protect_table_add(struct kvmgt_guest_info *info, gfn_t gfn)
+static void kvmgt_protect_table_add(struct intel_vgpu *info, gfn_t gfn)
 {
 	struct kvmgt_pgfn *p;
 
@@ -493,8 +481,7 @@ static void kvmgt_protect_table_add(struct kvmgt_guest_info *info, gfn_t gfn)
 	hash_add(info->ptable, &p->hnode, gfn);
 }
 
-static void kvmgt_protect_table_del(struct kvmgt_guest_info *info,
-				gfn_t gfn)
+static void kvmgt_protect_table_del(struct intel_vgpu *info, gfn_t gfn)
 {
 	struct kvmgt_pgfn *p;
 
@@ -921,7 +908,6 @@ static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
 static void __intel_vgpu_release(struct intel_vgpu *vgpu)
 {
 	struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
-	struct kvmgt_guest_info *info;
 	int ret;
 
 	if (!handle_valid(vgpu->handle))
@@ -945,8 +931,7 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
 	/* dereference module reference taken at open */
 	module_put(THIS_MODULE);
 
-	info = (struct kvmgt_guest_info *)vgpu->handle;
-	kvmgt_guest_exit(info);
+	kvmgt_guest_exit(vgpu);
 
 	intel_vgpu_release_msi_eventfd_ctx(vgpu);
 	vfio_group_put_external_user(vgpu->vfio_group);
@@ -1732,7 +1717,7 @@ static void kvmgt_host_exit(struct device *dev, void *gvt)
 
 static int kvmgt_page_track_add(unsigned long handle, u64 gfn)
 {
-	struct kvmgt_guest_info *info;
+	struct intel_vgpu *info;
 	struct kvm *kvm;
 	struct kvm_memory_slot *slot;
 	int idx;
@@ -1740,7 +1725,7 @@ static int kvmgt_page_track_add(unsigned long handle, u64 gfn)
 	if (!handle_valid(handle))
 		return -ESRCH;
 
-	info = (struct kvmgt_guest_info *)handle;
+	info = (struct intel_vgpu *)handle;
 	kvm = info->kvm;
 
 	idx = srcu_read_lock(&kvm->srcu);
@@ -1766,7 +1751,7 @@ static int kvmgt_page_track_add(unsigned long handle, u64 gfn)
 
 static int kvmgt_page_track_remove(unsigned long handle, u64 gfn)
 {
-	struct kvmgt_guest_info *info;
+	struct intel_vgpu *info;
 	struct kvm *kvm;
 	struct kvm_memory_slot *slot;
 	int idx;
@@ -1774,7 +1759,7 @@ static int kvmgt_page_track_remove(unsigned long handle, u64 gfn)
 	if (!handle_valid(handle))
 		return 0;
 
-	info = (struct kvmgt_guest_info *)handle;
+	info = (struct intel_vgpu *)handle;
 	kvm = info->kvm;
 
 	idx = srcu_read_lock(&kvm->srcu);
@@ -1802,11 +1787,11 @@ static void kvmgt_page_track_write(struct kvm_vcpu *vcpu, gpa_t gpa,
 		const u8 *val, int len,
 		struct kvm_page_track_notifier_node *node)
 {
-	struct kvmgt_guest_info *info = container_of(node,
-					struct kvmgt_guest_info, track_node);
+	struct intel_vgpu *info =
+		container_of(node, struct intel_vgpu, track_node);
 
 	if (kvmgt_gfn_is_write_protected(info, gpa_to_gfn(gpa)))
-		intel_vgpu_page_track_handler(info->vgpu, gpa,
+		intel_vgpu_page_track_handler(info, gpa,
 						     (void *)val, len);
 }
 
@@ -1816,8 +1801,8 @@ static void kvmgt_page_track_flush_slot(struct kvm *kvm,
 {
 	int i;
 	gfn_t gfn;
-	struct kvmgt_guest_info *info = container_of(node,
-					struct kvmgt_guest_info, track_node);
+	struct intel_vgpu *info =
+		container_of(node, struct intel_vgpu, track_node);
 
 	write_lock(&kvm->mmu_lock);
 	for (i = 0; i < slot->npages; i++) {
@@ -1834,7 +1819,6 @@ static void kvmgt_page_track_flush_slot(struct kvm *kvm,
 static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu, struct kvm *kvm)
 {
 	struct intel_vgpu *itr;
-	struct kvmgt_guest_info *info;
 	int id;
 	bool ret = false;
 
@@ -1843,8 +1827,7 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu, struct kvm *kvm)
 		if (!handle_valid(itr->handle))
 			continue;
 
-		info = (struct kvmgt_guest_info *)itr->handle;
-		if (kvm && kvm == info->kvm) {
+		if (kvm && kvm == itr->kvm) {
 			ret = true;
 			goto out;
 		}
@@ -1856,7 +1839,6 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu, struct kvm *kvm)
 
 static int kvmgt_guest_init(struct mdev_device *mdev)
 {
-	struct kvmgt_guest_info *info;
 	struct intel_vgpu *vgpu;
 	struct kvm *kvm;
 
@@ -1873,38 +1855,29 @@ static int kvmgt_guest_init(struct mdev_device *mdev)
 	if (__kvmgt_vgpu_exist(vgpu, kvm))
 		return -EEXIST;
 
-	info = vzalloc(sizeof(struct kvmgt_guest_info));
-	if (!info)
-		return -ENOMEM;
-
-	vgpu->handle = (unsigned long)info;
-	info->vgpu = vgpu;
-	info->kvm = kvm;
-	kvm_get_kvm(info->kvm);
+	vgpu->handle = (unsigned long)vgpu;
+	kvm_get_kvm(vgpu->kvm);
 
-	kvmgt_protect_table_init(info);
+	kvmgt_protect_table_init(vgpu);
 	gvt_cache_init(vgpu);
 
-	info->track_node.track_write = kvmgt_page_track_write;
-	info->track_node.track_flush_slot = kvmgt_page_track_flush_slot;
-	kvm_page_track_register_notifier(kvm, &info->track_node);
+	vgpu->track_node.track_write = kvmgt_page_track_write;
+	vgpu->track_node.track_flush_slot = kvmgt_page_track_flush_slot;
+	kvm_page_track_register_notifier(kvm, &vgpu->track_node);
 
 	debugfs_create_ulong(KVMGT_DEBUGFS_FILENAME, 0444, vgpu->debugfs,
 			     &vgpu->nr_cache_entries);
 	return 0;
 }
 
-static bool kvmgt_guest_exit(struct kvmgt_guest_info *info)
+static bool kvmgt_guest_exit(struct intel_vgpu *info)
 {
-	debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME,
-				      info->vgpu->debugfs));
+	debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME, info->debugfs));
 
 	kvm_page_track_unregister_notifier(info->kvm, &info->track_node);
 	kvm_put_kvm(info->kvm);
 	kvmgt_protect_table_destroy(info);
-	gvt_cache_destroy(info->vgpu);
-	vfree(info);
-
+	gvt_cache_destroy(info);
 	return true;
 }
 
@@ -1927,14 +1900,12 @@ static void kvmgt_detach_vgpu(void *p_vgpu)
 
 static int kvmgt_inject_msi(unsigned long handle, u32 addr, u16 data)
 {
-	struct kvmgt_guest_info *info;
 	struct intel_vgpu *vgpu;
 
 	if (!handle_valid(handle))
 		return -ESRCH;
 
-	info = (struct kvmgt_guest_info *)handle;
-	vgpu = info->vgpu;
+	vgpu = (struct intel_vgpu *)handle;
 
 	/*
 	 * When guest is poweroff, msi_trigger is set to NULL, but vgpu's
@@ -1956,15 +1927,15 @@ static int kvmgt_inject_msi(unsigned long handle, u32 addr, u16 data)
 
 static unsigned long kvmgt_gfn_to_pfn(unsigned long handle, unsigned long gfn)
 {
-	struct kvmgt_guest_info *info;
+	struct intel_vgpu *vgpu;
 	kvm_pfn_t pfn;
 
 	if (!handle_valid(handle))
 		return INTEL_GVT_INVALID_ADDR;
 
-	info = (struct kvmgt_guest_info *)handle;
+	vgpu = (struct intel_vgpu *)handle;
 
-	pfn = gfn_to_pfn(info->kvm, gfn);
+	pfn = gfn_to_pfn(vgpu->kvm, gfn);
 	if (is_error_noslot_pfn(pfn))
 		return INTEL_GVT_INVALID_ADDR;
 
@@ -1981,7 +1952,7 @@ static int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,
 	if (!handle_valid(handle))
 		return -EINVAL;
 
-	vgpu = ((struct kvmgt_guest_info *)handle)->vgpu;
+	vgpu = (struct intel_vgpu *)handle;
 
 	mutex_lock(&vgpu->cache_lock);
 
@@ -2023,22 +1994,22 @@ static int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,
 
 static int kvmgt_dma_pin_guest_page(unsigned long handle, dma_addr_t dma_addr)
 {
-	struct kvmgt_guest_info *info;
+	struct intel_vgpu *vgpu;
 	struct gvt_dma *entry;
 	int ret = 0;
 
 	if (!handle_valid(handle))
 		return -ENODEV;
 
-	info = (struct kvmgt_guest_info *)handle;
+	vgpu = (struct intel_vgpu *)handle;
 
-	mutex_lock(&info->vgpu->cache_lock);
-	entry = __gvt_cache_find_dma_addr(info->vgpu, dma_addr);
+	mutex_lock(&vgpu->cache_lock);
+	entry = __gvt_cache_find_dma_addr(vgpu, dma_addr);
 	if (entry)
 		kref_get(&entry->ref);
 	else
 		ret = -ENOMEM;
-	mutex_unlock(&info->vgpu->cache_lock);
+	mutex_unlock(&vgpu->cache_lock);
 
 	return ret;
 }
@@ -2060,7 +2031,7 @@ static void kvmgt_dma_unmap_guest_page(unsigned long handle, dma_addr_t dma_addr
 	if (!handle_valid(handle))
 		return;
 
-	vgpu = ((struct kvmgt_guest_info *)handle)->vgpu;
+	vgpu = (struct intel_vgpu *)handle;
 
 	mutex_lock(&vgpu->cache_lock);
 	entry = __gvt_cache_find_dma_addr(vgpu, dma_addr);
@@ -2072,14 +2043,14 @@ static void kvmgt_dma_unmap_guest_page(unsigned long handle, dma_addr_t dma_addr
 static int kvmgt_rw_gpa(unsigned long handle, unsigned long gpa,
 			void *buf, unsigned long len, bool write)
 {
-	struct kvmgt_guest_info *info;
+	struct intel_vgpu *vgpu;
 
 	if (!handle_valid(handle))
 		return -ESRCH;
 
-	info = (struct kvmgt_guest_info *)handle;
+	vgpu = (struct intel_vgpu *)handle;
 
-	return vfio_dma_rw(info->vgpu->vfio_group, gpa, buf, len, write);
+	return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, write);
 }
 
 static int kvmgt_read_gpa(unsigned long handle, unsigned long gpa,
@@ -2096,7 +2067,7 @@ static int kvmgt_write_gpa(unsigned long handle, unsigned long gpa,
 
 static bool kvmgt_is_valid_gfn(unsigned long handle, unsigned long gfn)
 {
-	struct kvmgt_guest_info *info;
+	struct intel_vgpu *vgpu;
 	struct kvm *kvm;
 	int idx;
 	bool ret;
@@ -2104,8 +2075,8 @@ static bool kvmgt_is_valid_gfn(unsigned long handle, unsigned long gfn)
 	if (!handle_valid(handle))
 		return false;
 
-	info = (struct kvmgt_guest_info *)handle;
-	kvm = info->kvm;
+	vgpu = (struct intel_vgpu *)handle;
+	kvm = vgpu->kvm;
 
 	idx = srcu_read_lock(&kvm->srcu);
 	ret = kvm_is_visible_gfn(kvm, gfn);
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 10/21] drm/i915/gvt: remove vgpu->handle
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (8 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 09/21] drm/i915/gvt: merge struct kvmgt_guest_info into strut intel_vgpu Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 11/21] drm/i915/gvt: devirtualize ->{read,write}_gpa Christoph Hellwig
                   ` (12 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Always pass the actual vgpu structure instead of encoding it as a
"handle" and add a bool flag to denote if a VGPU is attached.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.h       |   3 +-
 drivers/gpu/drm/i915/gvt/hypercall.h |  32 +++----
 drivers/gpu/drm/i915/gvt/kvmgt.c     | 126 +++++++++------------------
 drivers/gpu/drm/i915/gvt/mpt.h       |  20 ++---
 drivers/gpu/drm/i915/gvt/vgpu.c      |   6 +-
 5 files changed, 71 insertions(+), 116 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index a4691f0607e6..ad10ef2377e7 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -180,8 +180,8 @@ struct intel_vgpu {
 	struct intel_gvt *gvt;
 	struct mutex vgpu_lock;
 	int id;
-	unsigned long handle; /* vGPU handle used by hypervisor MPT modules */
 	bool active;
+	bool attached;
 	bool pv_notified;
 	bool failsafe;
 	unsigned int resetting_eng;
@@ -448,7 +448,6 @@ int intel_gvt_load_firmware(struct intel_gvt *gvt);
 #define RING_CTX_SIZE 320
 
 struct intel_vgpu_creation_params {
-	__u64 handle;
 	__u64 low_gm_sz;  /* in MB */
 	__u64 high_gm_sz; /* in MB */
 	__u64 fence_sz;
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index eacee6f41f9c..9f0475759825 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -36,6 +36,7 @@
 #include <linux/types.h>
 
 struct device;
+struct intel_vgpu;
 
 /*
  * Specific GVT-g MPT modules function collections. Currently GVT-g supports
@@ -44,27 +45,28 @@ struct device;
 struct intel_gvt_mpt {
 	int (*host_init)(struct device *dev, void *gvt);
 	void (*host_exit)(struct device *dev, void *gvt);
-	void (*detach_vgpu)(void *vgpu);
-	int (*inject_msi)(unsigned long handle, u32 addr, u16 data);
-	int (*enable_page_track)(unsigned long handle, u64 gfn);
-	int (*disable_page_track)(unsigned long handle, u64 gfn);
-	int (*read_gpa)(unsigned long handle, unsigned long gpa, void *buf,
+	void (*detach_vgpu)(struct intel_vgpu *vgpu);
+	int (*inject_msi)(struct intel_vgpu *vgpu, u32 addr, u16 data);
+	int (*enable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
+	int (*disable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
+	int (*read_gpa)(struct intel_vgpu *vgpu, unsigned long gpa, void *buf,
 			unsigned long len);
-	int (*write_gpa)(unsigned long handle, unsigned long gpa, void *buf,
+	int (*write_gpa)(struct intel_vgpu *vgpu, unsigned long gpa, void *buf,
 			 unsigned long len);
-	unsigned long (*gfn_to_mfn)(unsigned long handle, unsigned long gfn);
+	unsigned long (*gfn_to_mfn)(struct intel_vgpu *vgpu, unsigned long gfn);
 
-	int (*dma_map_guest_page)(unsigned long handle, unsigned long gfn,
+	int (*dma_map_guest_page)(struct intel_vgpu *vgpu, unsigned long gfn,
 				  unsigned long size, dma_addr_t *dma_addr);
-	void (*dma_unmap_guest_page)(unsigned long handle, dma_addr_t dma_addr);
+	void (*dma_unmap_guest_page)(struct intel_vgpu *vgpu,
+				dma_addr_t dma_addr);
 
-	int (*dma_pin_guest_page)(unsigned long handle, dma_addr_t dma_addr);
+	int (*dma_pin_guest_page)(struct intel_vgpu *vgpu, dma_addr_t dma_addr);
 
-	int (*set_opregion)(void *vgpu);
-	int (*set_edid)(void *vgpu, int port_num);
-	int (*get_vfio_device)(void *vgpu);
-	void (*put_vfio_device)(void *vgpu);
-	bool (*is_valid_gfn)(unsigned long handle, unsigned long gfn);
+	int (*set_opregion)(struct intel_vgpu *vgpu);
+	int (*set_edid)(struct intel_vgpu *vgpu, int port_num);
+	int (*get_vfio_device)(struct intel_vgpu *vgpu);
+	void (*put_vfio_device)(struct intel_vgpu *vgpu);
+	bool (*is_valid_gfn)(struct intel_vgpu *vgpu, unsigned long gfn);
 };
 
 #endif /* _GVT_HYPERCALL_H_ */
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 785647e776d2..bd08c2927c75 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -97,11 +97,6 @@ struct gvt_dma {
 	struct kref ref;
 };
 
-static inline bool handle_valid(unsigned long handle)
-{
-	return !!(handle & ~0xff);
-}
-
 static ssize_t available_instances_show(struct mdev_type *mtype,
 					struct mdev_type_attribute *attr,
 					char *buf)
@@ -649,10 +644,8 @@ static int intel_vgpu_register_reg(struct intel_vgpu *vgpu,
 	return 0;
 }
 
-static int kvmgt_get_vfio_device(void *p_vgpu)
+static int kvmgt_get_vfio_device(struct intel_vgpu *vgpu)
 {
-	struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
-
 	vgpu->vfio_device = vfio_device_get_from_dev(
 		mdev_dev(vgpu->mdev));
 	if (!vgpu->vfio_device) {
@@ -663,9 +656,8 @@ static int kvmgt_get_vfio_device(void *p_vgpu)
 }
 
 
-static int kvmgt_set_opregion(void *p_vgpu)
+static int kvmgt_set_opregion(struct intel_vgpu *vgpu)
 {
-	struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
 	void *base;
 	int ret;
 
@@ -691,9 +683,8 @@ static int kvmgt_set_opregion(void *p_vgpu)
 	return ret;
 }
 
-static int kvmgt_set_edid(void *p_vgpu, int port_num)
+static int kvmgt_set_edid(struct intel_vgpu *vgpu, int port_num)
 {
-	struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
 	struct intel_vgpu_port *port = intel_vgpu_port(vgpu, port_num);
 	struct vfio_edid_region *base;
 	int ret;
@@ -721,10 +712,8 @@ static int kvmgt_set_edid(void *p_vgpu, int port_num)
 	return ret;
 }
 
-static void kvmgt_put_vfio_device(void *data)
+static void kvmgt_put_vfio_device(struct intel_vgpu *vgpu)
 {
-	struct intel_vgpu *vgpu = data;
-
 	if (WARN_ON(!vgpu->vfio_device))
 		return;
 
@@ -772,7 +761,7 @@ static int intel_vgpu_remove(struct mdev_device *mdev)
 {
 	struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
 
-	if (handle_valid(vgpu->handle))
+	if (vgpu->attached)
 		return -EBUSY;
 
 	intel_gvt_destroy_vgpu(vgpu);
@@ -910,7 +899,7 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
 	struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
 	int ret;
 
-	if (!handle_valid(vgpu->handle))
+	if (!vgpu->attached)
 		return;
 
 	if (atomic_cmpxchg(&vgpu->released, 0, 1))
@@ -937,7 +926,7 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
 	vfio_group_put_external_user(vgpu->vfio_group);
 
 	vgpu->kvm = NULL;
-	vgpu->handle = 0;
+	vgpu->attached = false;
 }
 
 static void intel_vgpu_release(struct mdev_device *mdev)
@@ -1715,19 +1704,15 @@ static void kvmgt_host_exit(struct device *dev, void *gvt)
 	intel_gvt_cleanup_vgpu_type_groups((struct intel_gvt *)gvt);
 }
 
-static int kvmgt_page_track_add(unsigned long handle, u64 gfn)
+static int kvmgt_page_track_add(struct intel_vgpu *info, u64 gfn)
 {
-	struct intel_vgpu *info;
-	struct kvm *kvm;
+	struct kvm *kvm = info->kvm;
 	struct kvm_memory_slot *slot;
 	int idx;
 
-	if (!handle_valid(handle))
+	if (!info->attached)
 		return -ESRCH;
 
-	info = (struct intel_vgpu *)handle;
-	kvm = info->kvm;
-
 	idx = srcu_read_lock(&kvm->srcu);
 	slot = gfn_to_memslot(kvm, gfn);
 	if (!slot) {
@@ -1749,19 +1734,15 @@ static int kvmgt_page_track_add(unsigned long handle, u64 gfn)
 	return 0;
 }
 
-static int kvmgt_page_track_remove(unsigned long handle, u64 gfn)
+static int kvmgt_page_track_remove(struct intel_vgpu *info, u64 gfn)
 {
-	struct intel_vgpu *info;
-	struct kvm *kvm;
+	struct kvm *kvm = info->kvm;
 	struct kvm_memory_slot *slot;
 	int idx;
 
-	if (!handle_valid(handle))
+	if (!info->attached)
 		return 0;
 
-	info = (struct intel_vgpu *)handle;
-	kvm = info->kvm;
-
 	idx = srcu_read_lock(&kvm->srcu);
 	slot = gfn_to_memslot(kvm, gfn);
 	if (!slot) {
@@ -1824,7 +1805,7 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu, struct kvm *kvm)
 
 	mutex_lock(&vgpu->gvt->lock);
 	for_each_active_vgpu(vgpu->gvt, itr, id) {
-		if (!handle_valid(itr->handle))
+		if (!itr->attached)
 			continue;
 
 		if (kvm && kvm == itr->kvm) {
@@ -1839,14 +1820,12 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu, struct kvm *kvm)
 
 static int kvmgt_guest_init(struct mdev_device *mdev)
 {
-	struct intel_vgpu *vgpu;
-	struct kvm *kvm;
+	struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+	struct kvm *kvm = vgpu->kvm;
 
-	vgpu = mdev_get_drvdata(mdev);
-	if (handle_valid(vgpu->handle))
+	if (vgpu->attached)
 		return -EEXIST;
 
-	kvm = vgpu->kvm;
 	if (!kvm || kvm->mm != current->mm) {
 		gvt_vgpu_err("KVM is required to use Intel vGPU\n");
 		return -ESRCH;
@@ -1855,7 +1834,7 @@ static int kvmgt_guest_init(struct mdev_device *mdev)
 	if (__kvmgt_vgpu_exist(vgpu, kvm))
 		return -EEXIST;
 
-	vgpu->handle = (unsigned long)vgpu;
+	vgpu->attached = true;
 	kvm_get_kvm(vgpu->kvm);
 
 	kvmgt_protect_table_init(vgpu);
@@ -1881,10 +1860,9 @@ static bool kvmgt_guest_exit(struct intel_vgpu *info)
 	return true;
 }
 
-static void kvmgt_detach_vgpu(void *p_vgpu)
+static void kvmgt_detach_vgpu(struct intel_vgpu *vgpu)
 {
 	int i;
-	struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
 
 	if (!vgpu->region)
 		return;
@@ -1898,15 +1876,11 @@ static void kvmgt_detach_vgpu(void *p_vgpu)
 	vgpu->region = NULL;
 }
 
-static int kvmgt_inject_msi(unsigned long handle, u32 addr, u16 data)
+static int kvmgt_inject_msi(struct intel_vgpu *vgpu, u32 addr, u16 data)
 {
-	struct intel_vgpu *vgpu;
-
-	if (!handle_valid(handle))
+	if (!vgpu->attached)
 		return -ESRCH;
 
-	vgpu = (struct intel_vgpu *)handle;
-
 	/*
 	 * When guest is poweroff, msi_trigger is set to NULL, but vgpu's
 	 * config and mmio register isn't restored to default during guest
@@ -1925,16 +1899,14 @@ static int kvmgt_inject_msi(unsigned long handle, u32 addr, u16 data)
 	return -EFAULT;
 }
 
-static unsigned long kvmgt_gfn_to_pfn(unsigned long handle, unsigned long gfn)
+static unsigned long kvmgt_gfn_to_pfn(struct intel_vgpu *vgpu,
+		unsigned long gfn)
 {
-	struct intel_vgpu *vgpu;
 	kvm_pfn_t pfn;
 
-	if (!handle_valid(handle))
+	if (!vgpu->attached)
 		return INTEL_GVT_INVALID_ADDR;
 
-	vgpu = (struct intel_vgpu *)handle;
-
 	pfn = gfn_to_pfn(vgpu->kvm, gfn);
 	if (is_error_noslot_pfn(pfn))
 		return INTEL_GVT_INVALID_ADDR;
@@ -1942,18 +1914,15 @@ static unsigned long kvmgt_gfn_to_pfn(unsigned long handle, unsigned long gfn)
 	return pfn;
 }
 
-static int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,
+static int kvmgt_dma_map_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		unsigned long size, dma_addr_t *dma_addr)
 {
-	struct intel_vgpu *vgpu;
 	struct gvt_dma *entry;
 	int ret;
 
-	if (!handle_valid(handle))
+	if (!vgpu->attached)
 		return -EINVAL;
 
-	vgpu = (struct intel_vgpu *)handle;
-
 	mutex_lock(&vgpu->cache_lock);
 
 	entry = __gvt_cache_find_gfn(vgpu, gfn);
@@ -1992,17 +1961,15 @@ static int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,
 	return ret;
 }
 
-static int kvmgt_dma_pin_guest_page(unsigned long handle, dma_addr_t dma_addr)
+static int kvmgt_dma_pin_guest_page(struct intel_vgpu *vgpu,
+		dma_addr_t dma_addr)
 {
-	struct intel_vgpu *vgpu;
 	struct gvt_dma *entry;
 	int ret = 0;
 
-	if (!handle_valid(handle))
+	if (!vgpu->attached)
 		return -ENODEV;
 
-	vgpu = (struct intel_vgpu *)handle;
-
 	mutex_lock(&vgpu->cache_lock);
 	entry = __gvt_cache_find_dma_addr(vgpu, dma_addr);
 	if (entry)
@@ -2023,16 +1990,14 @@ static void __gvt_dma_release(struct kref *ref)
 	__gvt_cache_remove_entry(entry->vgpu, entry);
 }
 
-static void kvmgt_dma_unmap_guest_page(unsigned long handle, dma_addr_t dma_addr)
+static void kvmgt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
+		dma_addr_t dma_addr)
 {
-	struct intel_vgpu *vgpu;
 	struct gvt_dma *entry;
 
-	if (!handle_valid(handle))
+	if (!vgpu->attached)
 		return;
 
-	vgpu = (struct intel_vgpu *)handle;
-
 	mutex_lock(&vgpu->cache_lock);
 	entry = __gvt_cache_find_dma_addr(vgpu, dma_addr);
 	if (entry)
@@ -2040,44 +2005,35 @@ static void kvmgt_dma_unmap_guest_page(unsigned long handle, dma_addr_t dma_addr
 	mutex_unlock(&vgpu->cache_lock);
 }
 
-static int kvmgt_rw_gpa(unsigned long handle, unsigned long gpa,
+static int kvmgt_rw_gpa(struct intel_vgpu *vgpu, unsigned long gpa,
 			void *buf, unsigned long len, bool write)
 {
-	struct intel_vgpu *vgpu;
-
-	if (!handle_valid(handle))
+	if (!vgpu->attached)
 		return -ESRCH;
-
-	vgpu = (struct intel_vgpu *)handle;
-
 	return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, write);
 }
 
-static int kvmgt_read_gpa(unsigned long handle, unsigned long gpa,
+static int kvmgt_read_gpa(struct intel_vgpu *vgpu, unsigned long gpa,
 			void *buf, unsigned long len)
 {
-	return kvmgt_rw_gpa(handle, gpa, buf, len, false);
+	return kvmgt_rw_gpa(vgpu, gpa, buf, len, false);
 }
 
-static int kvmgt_write_gpa(unsigned long handle, unsigned long gpa,
+static int kvmgt_write_gpa(struct intel_vgpu *vgpu, unsigned long gpa,
 			void *buf, unsigned long len)
 {
-	return kvmgt_rw_gpa(handle, gpa, buf, len, true);
+	return kvmgt_rw_gpa(vgpu, gpa, buf, len, true);
 }
 
-static bool kvmgt_is_valid_gfn(unsigned long handle, unsigned long gfn)
+static bool kvmgt_is_valid_gfn(struct intel_vgpu *vgpu, unsigned long gfn)
 {
-	struct intel_vgpu *vgpu;
-	struct kvm *kvm;
+	struct kvm *kvm = vgpu->kvm;
 	int idx;
 	bool ret;
 
-	if (!handle_valid(handle))
+	if (!vgpu->attached)
 		return false;
 
-	vgpu = (struct intel_vgpu *)handle;
-	kvm = vgpu->kvm;
-
 	idx = srcu_read_lock(&kvm->srcu);
 	ret = kvm_is_visible_gfn(kvm, gfn);
 	srcu_read_unlock(&kvm->srcu, idx);
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 8a659301d78b..ba0c31c4a705 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -118,7 +118,7 @@ static inline int intel_gvt_hypervisor_inject_msi(struct intel_vgpu *vgpu)
 
 	trace_inject_msi(vgpu->id, addr, data);
 
-	ret = intel_gvt_host.mpt->inject_msi(vgpu->handle, addr, data);
+	ret = intel_gvt_host.mpt->inject_msi(vgpu, addr, data);
 	if (ret)
 		return ret;
 	return 0;
@@ -135,7 +135,7 @@ static inline int intel_gvt_hypervisor_inject_msi(struct intel_vgpu *vgpu)
 static inline int intel_gvt_hypervisor_enable_page_track(
 		struct intel_vgpu *vgpu, unsigned long gfn)
 {
-	return intel_gvt_host.mpt->enable_page_track(vgpu->handle, gfn);
+	return intel_gvt_host.mpt->enable_page_track(vgpu, gfn);
 }
 
 /**
@@ -149,7 +149,7 @@ static inline int intel_gvt_hypervisor_enable_page_track(
 static inline int intel_gvt_hypervisor_disable_page_track(
 		struct intel_vgpu *vgpu, unsigned long gfn)
 {
-	return intel_gvt_host.mpt->disable_page_track(vgpu->handle, gfn);
+	return intel_gvt_host.mpt->disable_page_track(vgpu, gfn);
 }
 
 /**
@@ -165,7 +165,7 @@ static inline int intel_gvt_hypervisor_disable_page_track(
 static inline int intel_gvt_hypervisor_read_gpa(struct intel_vgpu *vgpu,
 		unsigned long gpa, void *buf, unsigned long len)
 {
-	return intel_gvt_host.mpt->read_gpa(vgpu->handle, gpa, buf, len);
+	return intel_gvt_host.mpt->read_gpa(vgpu, gpa, buf, len);
 }
 
 /**
@@ -181,7 +181,7 @@ static inline int intel_gvt_hypervisor_read_gpa(struct intel_vgpu *vgpu,
 static inline int intel_gvt_hypervisor_write_gpa(struct intel_vgpu *vgpu,
 		unsigned long gpa, void *buf, unsigned long len)
 {
-	return intel_gvt_host.mpt->write_gpa(vgpu->handle, gpa, buf, len);
+	return intel_gvt_host.mpt->write_gpa(vgpu, gpa, buf, len);
 }
 
 /**
@@ -195,7 +195,7 @@ static inline int intel_gvt_hypervisor_write_gpa(struct intel_vgpu *vgpu,
 static inline unsigned long intel_gvt_hypervisor_gfn_to_mfn(
 		struct intel_vgpu *vgpu, unsigned long gfn)
 {
-	return intel_gvt_host.mpt->gfn_to_mfn(vgpu->handle, gfn);
+	return intel_gvt_host.mpt->gfn_to_mfn(vgpu, gfn);
 }
 
 /**
@@ -212,7 +212,7 @@ static inline int intel_gvt_hypervisor_dma_map_guest_page(
 		struct intel_vgpu *vgpu, unsigned long gfn, unsigned long size,
 		dma_addr_t *dma_addr)
 {
-	return intel_gvt_host.mpt->dma_map_guest_page(vgpu->handle, gfn, size,
+	return intel_gvt_host.mpt->dma_map_guest_page(vgpu, gfn, size,
 						      dma_addr);
 }
 
@@ -224,7 +224,7 @@ static inline int intel_gvt_hypervisor_dma_map_guest_page(
 static inline void intel_gvt_hypervisor_dma_unmap_guest_page(
 		struct intel_vgpu *vgpu, dma_addr_t dma_addr)
 {
-	intel_gvt_host.mpt->dma_unmap_guest_page(vgpu->handle, dma_addr);
+	intel_gvt_host.mpt->dma_unmap_guest_page(vgpu, dma_addr);
 }
 
 /**
@@ -239,7 +239,7 @@ static inline int
 intel_gvt_hypervisor_dma_pin_guest_page(struct intel_vgpu *vgpu,
 					dma_addr_t dma_addr)
 {
-	return intel_gvt_host.mpt->dma_pin_guest_page(vgpu->handle, dma_addr);
+	return intel_gvt_host.mpt->dma_pin_guest_page(vgpu, dma_addr);
 }
 
 /**
@@ -318,7 +318,7 @@ static inline bool intel_gvt_hypervisor_is_valid_gfn(
 	if (!intel_gvt_host.mpt->is_valid_gfn)
 		return true;
 
-	return intel_gvt_host.mpt->is_valid_gfn(vgpu->handle, gfn);
+	return intel_gvt_host.mpt->is_valid_gfn(vgpu, gfn);
 }
 
 #endif /* _GVT_MPT_H_ */
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index 7b9a571c0dc9..9e8fa9f388eb 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -370,8 +370,8 @@ static struct intel_vgpu *__intel_gvt_create_vgpu(struct intel_gvt *gvt,
 	struct intel_vgpu *vgpu;
 	int ret;
 
-	gvt_dbg_core("handle %llu low %llu MB high %llu MB fence %llu\n",
-			param->handle, param->low_gm_sz, param->high_gm_sz,
+	gvt_dbg_core("low %llu MB high %llu MB fence %llu\n",
+			param->low_gm_sz, param->high_gm_sz,
 			param->fence_sz);
 
 	vgpu = vzalloc(sizeof(*vgpu));
@@ -384,7 +384,6 @@ static struct intel_vgpu *__intel_gvt_create_vgpu(struct intel_gvt *gvt,
 		goto out_free_vgpu;
 
 	vgpu->id = ret;
-	vgpu->handle = param->handle;
 	vgpu->gvt = gvt;
 	vgpu->sched_ctl.weight = param->weight;
 	mutex_init(&vgpu->vgpu_lock);
@@ -477,7 +476,6 @@ struct intel_vgpu *intel_gvt_create_vgpu(struct intel_gvt *gvt,
 	struct intel_vgpu_creation_params param;
 	struct intel_vgpu *vgpu;
 
-	param.handle = 0;
 	param.primary = 1;
 	param.low_gm_sz = type->low_gm_size;
 	param.high_gm_sz = type->high_gm_size;
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 11/21] drm/i915/gvt: devirtualize ->{read,write}_gpa
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (9 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 10/21] drm/i915/gvt: remove vgpu->handle Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 12/21] drm/i915/gvt: devirtualize ->{get,put}_vfio_device Christoph Hellwig
                   ` (11 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just call the VFIO functions directly instead of through the method
table.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/cmd_parser.c |  4 +--
 drivers/gpu/drm/i915/gvt/execlist.c   | 12 ++++-----
 drivers/gpu/drm/i915/gvt/gtt.c        |  6 ++---
 drivers/gpu/drm/i915/gvt/gvt.h        | 37 +++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/hypercall.h  |  4 ---
 drivers/gpu/drm/i915/gvt/kvmgt.c      | 23 -----------------
 drivers/gpu/drm/i915/gvt/mmio.c       |  4 +--
 drivers/gpu/drm/i915/gvt/mpt.h        | 32 -----------------------
 drivers/gpu/drm/i915/gvt/opregion.c   | 10 +++-----
 drivers/gpu/drm/i915/gvt/scheduler.c  | 37 +++++++++++++--------------
 10 files changed, 72 insertions(+), 97 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/cmd_parser.c b/drivers/gpu/drm/i915/gvt/cmd_parser.c
index c4118b808268..c694d2c5efdd 100644
--- a/drivers/gpu/drm/i915/gvt/cmd_parser.c
+++ b/drivers/gpu/drm/i915/gvt/cmd_parser.c
@@ -1009,7 +1009,7 @@ static int cmd_reg_handler(struct parser_exec_state *s,
 	if (GRAPHICS_VER(s->engine->i915) == 9 &&
 	    intel_gvt_mmio_is_sr_in_ctx(gvt, offset) &&
 	    !strncmp(cmd, "lri", 3)) {
-		intel_gvt_hypervisor_read_gpa(s->vgpu,
+		intel_gvt_read_gpa(s->vgpu,
 			s->workload->ring_context_gpa + 12, &ctx_sr_ctl, 4);
 		/* check inhibit context */
 		if (ctx_sr_ctl & 1) {
@@ -1773,7 +1773,7 @@ static int copy_gma_to_hva(struct intel_vgpu *vgpu, struct intel_vgpu_mm *mm,
 		copy_len = (end_gma - gma) >= (I915_GTT_PAGE_SIZE - offset) ?
 			I915_GTT_PAGE_SIZE - offset : end_gma - gma;
 
-		intel_gvt_hypervisor_read_gpa(vgpu, gpa, va + len, copy_len);
+		intel_gvt_read_gpa(vgpu, gpa, va + len, copy_len);
 
 		len += copy_len;
 		gma += copy_len;
diff --git a/drivers/gpu/drm/i915/gvt/execlist.c b/drivers/gpu/drm/i915/gvt/execlist.c
index c8dcda6d4f0d..a3e15e7a8375 100644
--- a/drivers/gpu/drm/i915/gvt/execlist.c
+++ b/drivers/gpu/drm/i915/gvt/execlist.c
@@ -159,12 +159,12 @@ static void emulate_csb_update(struct intel_vgpu_execlist *execlist,
 	hwsp_gpa = intel_vgpu_gma_to_gpa(vgpu->gtt.ggtt_mm,
 					 vgpu->hws_pga[execlist->engine->id]);
 	if (hwsp_gpa != INTEL_GVT_INVALID_ADDR) {
-		intel_gvt_hypervisor_write_gpa(vgpu,
-					       hwsp_gpa + I915_HWS_CSB_BUF0_INDEX * 4 + write_pointer * 8,
-					       status, 8);
-		intel_gvt_hypervisor_write_gpa(vgpu,
-					       hwsp_gpa + intel_hws_csb_write_index(execlist->engine->i915) * 4,
-					       &write_pointer, 4);
+		intel_gvt_write_gpa(vgpu,
+			hwsp_gpa + I915_HWS_CSB_BUF0_INDEX * 4 + write_pointer * 8,
+			status, 8);
+		intel_gvt_write_gpa(vgpu,
+			hwsp_gpa + intel_hws_csb_write_index(execlist->engine->i915) * 4,
+			&write_pointer, 4);
 	}
 
 	gvt_dbg_el("vgpu%d: w pointer %u reg %x csb l %x csb h %x\n",
diff --git a/drivers/gpu/drm/i915/gvt/gtt.c b/drivers/gpu/drm/i915/gvt/gtt.c
index cc2c05e18206..0a953ab7a38b 100644
--- a/drivers/gpu/drm/i915/gvt/gtt.c
+++ b/drivers/gpu/drm/i915/gvt/gtt.c
@@ -312,7 +312,7 @@ static inline int gtt_get_entry64(void *pt,
 		return -EINVAL;
 
 	if (hypervisor_access) {
-		ret = intel_gvt_hypervisor_read_gpa(vgpu, gpa +
+		ret = intel_gvt_read_gpa(vgpu, gpa +
 				(index << info->gtt_entry_size_shift),
 				&e->val64, 8);
 		if (WARN_ON(ret))
@@ -337,7 +337,7 @@ static inline int gtt_set_entry64(void *pt,
 		return -EINVAL;
 
 	if (hypervisor_access) {
-		ret = intel_gvt_hypervisor_write_gpa(vgpu, gpa +
+		ret = intel_gvt_write_gpa(vgpu, gpa +
 				(index << info->gtt_entry_size_shift),
 				&e->val64, 8);
 		if (WARN_ON(ret))
@@ -1495,7 +1495,7 @@ static int attach_oos_page(struct intel_vgpu_oos_page *oos_page,
 	struct intel_gvt *gvt = spt->vgpu->gvt;
 	int ret;
 
-	ret = intel_gvt_hypervisor_read_gpa(spt->vgpu,
+	ret = intel_gvt_read_gpa(spt->vgpu,
 			spt->guest_page.gfn << I915_GTT_PAGE_SHIFT,
 			oos_page->mem, I915_GTT_PAGE_SIZE);
 	if (ret)
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index ad10ef2377e7..e604d6f6d06a 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -35,6 +35,7 @@
 
 #include <uapi/linux/pci_regs.h>
 #include <linux/kvm_host.h>
+#include <linux/vfio.h>
 
 #include "i915_drv.h"
 
@@ -719,6 +720,42 @@ static inline bool intel_gvt_mmio_is_cmd_write_patch(
 	return gvt->mmio.mmio_attribute[offset >> 2] & F_CMD_WRITE_PATCH;
 }
 
+/**
+ * intel_gvt_read_gpa - copy data from GPA to host data buffer
+ * @vgpu: a vGPU
+ * @gpa: guest physical address
+ * @buf: host data buffer
+ * @len: data length
+ *
+ * Returns:
+ * Zero on success, negative error code if failed.
+ */
+static inline int intel_gvt_read_gpa(struct intel_vgpu *vgpu, unsigned long gpa,
+		void *buf, unsigned long len)
+{
+	if (!vgpu->attached)
+		return -ESRCH;
+	return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, false);
+}
+
+/**
+ * intel_gvt_write_gpa - copy data from host data buffer to GPA
+ * @vgpu: a vGPU
+ * @gpa: guest physical address
+ * @buf: host data buffer
+ * @len: data length
+ *
+ * Returns:
+ * Zero on success, negative error code if failed.
+ */
+static inline int intel_gvt_write_gpa(struct intel_vgpu *vgpu,
+		unsigned long gpa, void *buf, unsigned long len)
+{
+	if (!vgpu->attached)
+		return -ESRCH;
+	return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, true);
+}
+
 void intel_gvt_debugfs_remove_vgpu(struct intel_vgpu *vgpu);
 void intel_gvt_debugfs_init(struct intel_gvt *gvt);
 void intel_gvt_debugfs_clean(struct intel_gvt *gvt);
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index 9f0475759825..61e493e2de85 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -49,10 +49,6 @@ struct intel_gvt_mpt {
 	int (*inject_msi)(struct intel_vgpu *vgpu, u32 addr, u16 data);
 	int (*enable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
 	int (*disable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
-	int (*read_gpa)(struct intel_vgpu *vgpu, unsigned long gpa, void *buf,
-			unsigned long len);
-	int (*write_gpa)(struct intel_vgpu *vgpu, unsigned long gpa, void *buf,
-			 unsigned long len);
 	unsigned long (*gfn_to_mfn)(struct intel_vgpu *vgpu, unsigned long gfn);
 
 	int (*dma_map_guest_page)(struct intel_vgpu *vgpu, unsigned long gfn,
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index bd08c2927c75..7b588a371adb 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -39,7 +39,6 @@
 #include <linux/spinlock.h>
 #include <linux/eventfd.h>
 #include <linux/uuid.h>
-#include <linux/vfio.h>
 #include <linux/mdev.h>
 #include <linux/debugfs.h>
 
@@ -2005,26 +2004,6 @@ static void kvmgt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
 	mutex_unlock(&vgpu->cache_lock);
 }
 
-static int kvmgt_rw_gpa(struct intel_vgpu *vgpu, unsigned long gpa,
-			void *buf, unsigned long len, bool write)
-{
-	if (!vgpu->attached)
-		return -ESRCH;
-	return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, write);
-}
-
-static int kvmgt_read_gpa(struct intel_vgpu *vgpu, unsigned long gpa,
-			void *buf, unsigned long len)
-{
-	return kvmgt_rw_gpa(vgpu, gpa, buf, len, false);
-}
-
-static int kvmgt_write_gpa(struct intel_vgpu *vgpu, unsigned long gpa,
-			void *buf, unsigned long len)
-{
-	return kvmgt_rw_gpa(vgpu, gpa, buf, len, true);
-}
-
 static bool kvmgt_is_valid_gfn(struct intel_vgpu *vgpu, unsigned long gfn)
 {
 	struct kvm *kvm = vgpu->kvm;
@@ -2048,8 +2027,6 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
 	.inject_msi = kvmgt_inject_msi,
 	.enable_page_track = kvmgt_page_track_add,
 	.disable_page_track = kvmgt_page_track_remove,
-	.read_gpa = kvmgt_read_gpa,
-	.write_gpa = kvmgt_write_gpa,
 	.gfn_to_mfn = kvmgt_gfn_to_pfn,
 	.dma_map_guest_page = kvmgt_dma_map_guest_page,
 	.dma_unmap_guest_page = kvmgt_dma_unmap_guest_page,
diff --git a/drivers/gpu/drm/i915/gvt/mmio.c b/drivers/gpu/drm/i915/gvt/mmio.c
index 24210b1eaec5..464a2ae1a83c 100644
--- a/drivers/gpu/drm/i915/gvt/mmio.c
+++ b/drivers/gpu/drm/i915/gvt/mmio.c
@@ -136,7 +136,7 @@ int intel_vgpu_emulate_mmio_read(struct intel_vgpu *vgpu, u64 pa,
 	}
 
 	if (drm_WARN_ON_ONCE(&i915->drm, !reg_is_mmio(gvt, offset))) {
-		ret = intel_gvt_hypervisor_read_gpa(vgpu, pa, p_data, bytes);
+		ret = intel_gvt_read_gpa(vgpu, pa, p_data, bytes);
 		goto out;
 	}
 
@@ -212,7 +212,7 @@ int intel_vgpu_emulate_mmio_write(struct intel_vgpu *vgpu, u64 pa,
 	}
 
 	if (drm_WARN_ON_ONCE(&i915->drm, !reg_is_mmio(gvt, offset))) {
-		ret = intel_gvt_hypervisor_write_gpa(vgpu, pa, p_data, bytes);
+		ret = intel_gvt_write_gpa(vgpu, pa, p_data, bytes);
 		goto out;
 	}
 
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index ba0c31c4a705..72388ceec596 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -152,38 +152,6 @@ static inline int intel_gvt_hypervisor_disable_page_track(
 	return intel_gvt_host.mpt->disable_page_track(vgpu, gfn);
 }
 
-/**
- * intel_gvt_hypervisor_read_gpa - copy data from GPA to host data buffer
- * @vgpu: a vGPU
- * @gpa: guest physical address
- * @buf: host data buffer
- * @len: data length
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_read_gpa(struct intel_vgpu *vgpu,
-		unsigned long gpa, void *buf, unsigned long len)
-{
-	return intel_gvt_host.mpt->read_gpa(vgpu, gpa, buf, len);
-}
-
-/**
- * intel_gvt_hypervisor_write_gpa - copy data from host data buffer to GPA
- * @vgpu: a vGPU
- * @gpa: guest physical address
- * @buf: host data buffer
- * @len: data length
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_write_gpa(struct intel_vgpu *vgpu,
-		unsigned long gpa, void *buf, unsigned long len)
-{
-	return intel_gvt_host.mpt->write_gpa(vgpu, gpa, buf, len);
-}
-
 /**
  * intel_gvt_hypervisor_gfn_to_mfn - translate a GFN to MFN
  * @vgpu: a vGPU
diff --git a/drivers/gpu/drm/i915/gvt/opregion.c b/drivers/gpu/drm/i915/gvt/opregion.c
index 286ac6d7c6ce..d2bed466540a 100644
--- a/drivers/gpu/drm/i915/gvt/opregion.c
+++ b/drivers/gpu/drm/i915/gvt/opregion.c
@@ -421,14 +421,14 @@ int intel_vgpu_emulate_opregion_request(struct intel_vgpu *vgpu, u32 swsci)
 				INTEL_GVT_OPREGION_SCIC;
 	parm_pa = (vgpu_opregion(vgpu)->gfn[0] << PAGE_SHIFT) +
 				INTEL_GVT_OPREGION_PARM;
-	ret = intel_gvt_hypervisor_read_gpa(vgpu, scic_pa, &scic, sizeof(scic));
+	ret = intel_gvt_read_gpa(vgpu, scic_pa, &scic, sizeof(scic));
 	if (ret) {
 		gvt_vgpu_err("guest opregion read error %d, gpa 0x%llx, len %lu\n",
 			ret, scic_pa, sizeof(scic));
 		return ret;
 	}
 
-	ret = intel_gvt_hypervisor_read_gpa(vgpu, parm_pa, &parm, sizeof(parm));
+	ret = intel_gvt_read_gpa(vgpu, parm_pa, &parm, sizeof(parm));
 	if (ret) {
 		gvt_vgpu_err("guest opregion read error %d, gpa 0x%llx, len %lu\n",
 			ret, scic_pa, sizeof(scic));
@@ -465,16 +465,14 @@ int intel_vgpu_emulate_opregion_request(struct intel_vgpu *vgpu, u32 swsci)
 	parm = 0;
 
 out:
-	ret = intel_gvt_hypervisor_write_gpa(vgpu, scic_pa, &scic,
-					     sizeof(scic));
+	ret = intel_gvt_write_gpa(vgpu, scic_pa, &scic, sizeof(scic));
 	if (ret) {
 		gvt_vgpu_err("guest opregion write error %d, gpa 0x%llx, len %lu\n",
 			ret, scic_pa, sizeof(scic));
 		return ret;
 	}
 
-	ret = intel_gvt_hypervisor_write_gpa(vgpu, parm_pa, &parm,
-					     sizeof(parm));
+	ret = intel_gvt_write_gpa(vgpu, parm_pa, &parm, sizeof(parm));
 	if (ret) {
 		gvt_vgpu_err("guest opregion write error %d, gpa 0x%llx, len %lu\n",
 			ret, scic_pa, sizeof(scic));
diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c
index 734c37c5e347..801b9168feb5 100644
--- a/drivers/gpu/drm/i915/gvt/scheduler.c
+++ b/drivers/gpu/drm/i915/gvt/scheduler.c
@@ -148,10 +148,10 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload)
 
 	sr_oa_regs(workload, (u32 *)shadow_ring_context, true);
 #define COPY_REG(name) \
-	intel_gvt_hypervisor_read_gpa(vgpu, workload->ring_context_gpa \
+	intel_gvt_read_gpa(vgpu, workload->ring_context_gpa \
 		+ RING_CTX_OFF(name.val), &shadow_ring_context->name.val, 4)
 #define COPY_REG_MASKED(name) {\
-		intel_gvt_hypervisor_read_gpa(vgpu, workload->ring_context_gpa \
+		intel_gvt_read_gpa(vgpu, workload->ring_context_gpa \
 					      + RING_CTX_OFF(name.val),\
 					      &shadow_ring_context->name.val, 4);\
 		shadow_ring_context->name.val |= 0xffff << 16;\
@@ -165,7 +165,7 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload)
 		COPY_REG(rcs_indirect_ctx);
 		COPY_REG(rcs_indirect_ctx_offset);
 	} else if (workload->engine->id == BCS0)
-		intel_gvt_hypervisor_read_gpa(vgpu,
+		intel_gvt_read_gpa(vgpu,
 				workload->ring_context_gpa +
 				BCS_TILE_REGISTER_VAL_OFFSET,
 				(void *)shadow_ring_context +
@@ -176,7 +176,7 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload)
 	/* don't copy Ring Context (the first 0x50 dwords),
 	 * only copy the Engine Context part from guest
 	 */
-	intel_gvt_hypervisor_read_gpa(vgpu,
+	intel_gvt_read_gpa(vgpu,
 			workload->ring_context_gpa +
 			RING_CTX_SIZE,
 			(void *)shadow_ring_context +
@@ -243,7 +243,7 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload)
 		continue;
 
 read:
-		intel_gvt_hypervisor_read_gpa(vgpu, gpa_base, dst, gpa_size);
+		intel_gvt_read_gpa(vgpu, gpa_base, dst, gpa_size);
 		gpa_base = context_gpa;
 		gpa_size = I915_GTT_PAGE_SIZE;
 		dst = context_base + (i << I915_GTT_PAGE_SHIFT);
@@ -909,8 +909,7 @@ static void update_guest_pdps(struct intel_vgpu *vgpu,
 	gpa = ring_context_gpa + RING_CTX_OFF(pdps[0].val);
 
 	for (i = 0; i < 8; i++)
-		intel_gvt_hypervisor_write_gpa(vgpu,
-				gpa + i * 8, &pdp[7 - i], 4);
+		intel_gvt_write_gpa(vgpu, gpa + i * 8, &pdp[7 - i], 4);
 }
 
 static __maybe_unused bool
@@ -1005,13 +1004,13 @@ static void update_guest_context(struct intel_vgpu_workload *workload)
 		continue;
 
 write:
-		intel_gvt_hypervisor_write_gpa(vgpu, gpa_base, src, gpa_size);
+		intel_gvt_write_gpa(vgpu, gpa_base, src, gpa_size);
 		gpa_base = context_gpa;
 		gpa_size = I915_GTT_PAGE_SIZE;
 		src = context_base + (i << I915_GTT_PAGE_SHIFT);
 	}
 
-	intel_gvt_hypervisor_write_gpa(vgpu, workload->ring_context_gpa +
+	intel_gvt_write_gpa(vgpu, workload->ring_context_gpa +
 		RING_CTX_OFF(ring_header.val), &workload->rb_tail, 4);
 
 	shadow_ring_context = (void *) ctx->lrc_reg_state;
@@ -1026,7 +1025,7 @@ static void update_guest_context(struct intel_vgpu_workload *workload)
 	}
 
 #define COPY_REG(name) \
-	intel_gvt_hypervisor_write_gpa(vgpu, workload->ring_context_gpa + \
+	intel_gvt_write_gpa(vgpu, workload->ring_context_gpa + \
 		RING_CTX_OFF(name.val), &shadow_ring_context->name.val, 4)
 
 	COPY_REG(ctx_ctrl);
@@ -1034,7 +1033,7 @@ static void update_guest_context(struct intel_vgpu_workload *workload)
 
 #undef COPY_REG
 
-	intel_gvt_hypervisor_write_gpa(vgpu,
+	intel_gvt_write_gpa(vgpu,
 			workload->ring_context_gpa +
 			sizeof(*shadow_ring_context),
 			(void *)shadow_ring_context +
@@ -1574,7 +1573,7 @@ static void read_guest_pdps(struct intel_vgpu *vgpu,
 	gpa = ring_context_gpa + RING_CTX_OFF(pdps[0].val);
 
 	for (i = 0; i < 8; i++)
-		intel_gvt_hypervisor_read_gpa(vgpu,
+		intel_gvt_read_gpa(vgpu,
 				gpa + i * 8, &pdp[7 - i], 4);
 }
 
@@ -1645,10 +1644,10 @@ intel_vgpu_create_workload(struct intel_vgpu *vgpu,
 		return ERR_PTR(-EINVAL);
 	}
 
-	intel_gvt_hypervisor_read_gpa(vgpu, ring_context_gpa +
+	intel_gvt_read_gpa(vgpu, ring_context_gpa +
 			RING_CTX_OFF(ring_header.val), &head, 4);
 
-	intel_gvt_hypervisor_read_gpa(vgpu, ring_context_gpa +
+	intel_gvt_read_gpa(vgpu, ring_context_gpa +
 			RING_CTX_OFF(ring_tail.val), &tail, 4);
 
 	guest_head = head;
@@ -1675,11 +1674,11 @@ intel_vgpu_create_workload(struct intel_vgpu *vgpu,
 	gvt_dbg_el("ring %s begin a new workload\n", engine->name);
 
 	/* record some ring buffer register values for scan and shadow */
-	intel_gvt_hypervisor_read_gpa(vgpu, ring_context_gpa +
+	intel_gvt_read_gpa(vgpu, ring_context_gpa +
 			RING_CTX_OFF(rb_start.val), &start, 4);
-	intel_gvt_hypervisor_read_gpa(vgpu, ring_context_gpa +
+	intel_gvt_read_gpa(vgpu, ring_context_gpa +
 			RING_CTX_OFF(rb_ctrl.val), &ctl, 4);
-	intel_gvt_hypervisor_read_gpa(vgpu, ring_context_gpa +
+	intel_gvt_read_gpa(vgpu, ring_context_gpa +
 			RING_CTX_OFF(ctx_ctrl.val), &ctx_ctl, 4);
 
 	if (!intel_gvt_ggtt_validate_range(vgpu, start,
@@ -1702,9 +1701,9 @@ intel_vgpu_create_workload(struct intel_vgpu *vgpu,
 	workload->rb_ctl = ctl;
 
 	if (engine->id == RCS0) {
-		intel_gvt_hypervisor_read_gpa(vgpu, ring_context_gpa +
+		intel_gvt_read_gpa(vgpu, ring_context_gpa +
 			RING_CTX_OFF(bb_per_ctx_ptr.val), &per_ctx, 4);
-		intel_gvt_hypervisor_read_gpa(vgpu, ring_context_gpa +
+		intel_gvt_read_gpa(vgpu, ring_context_gpa +
 			RING_CTX_OFF(rcs_indirect_ctx.val), &indirect_ctx, 4);
 
 		workload->wa_ctx.indirect_ctx.guest_gma =
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 12/21] drm/i915/gvt: devirtualize ->{get,put}_vfio_device
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (10 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 11/21] drm/i915/gvt: devirtualize ->{read,write}_gpa Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 13/21] drm/i915/gvt: devirtualize ->set_edid and ->set_opregion Christoph Hellwig
                   ` (10 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just open code the calls to the VFIO APIs.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/dmabuf.c    | 12 ++++++-----
 drivers/gpu/drm/i915/gvt/hypercall.h |  2 --
 drivers/gpu/drm/i915/gvt/kvmgt.c     | 22 --------------------
 drivers/gpu/drm/i915/gvt/mpt.h       | 30 ----------------------------
 4 files changed, 7 insertions(+), 59 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/dmabuf.c b/drivers/gpu/drm/i915/gvt/dmabuf.c
index 8e65cd8258b9..bc53a0c60b44 100644
--- a/drivers/gpu/drm/i915/gvt/dmabuf.c
+++ b/drivers/gpu/drm/i915/gvt/dmabuf.c
@@ -29,7 +29,7 @@
  */
 
 #include <linux/dma-buf.h>
-#include <linux/vfio.h>
+#include <linux/mdev.h>
 
 #include "i915_drv.h"
 #include "gvt.h"
@@ -152,7 +152,7 @@ static void dmabuf_gem_object_free(struct kref *kref)
 					struct intel_vgpu_dmabuf_obj, list);
 			if (dmabuf_obj == obj) {
 				list_del(pos);
-				intel_gvt_hypervisor_put_vfio_device(vgpu);
+				vfio_device_put(vgpu->vfio_device);
 				idr_remove(&vgpu->object_idr,
 					   dmabuf_obj->dmabuf_id);
 				kfree(dmabuf_obj->info);
@@ -493,9 +493,11 @@ int intel_vgpu_query_plane(struct intel_vgpu *vgpu, void *args)
 	kref_init(&dmabuf_obj->kref);
 
 	mutex_lock(&vgpu->dmabuf_lock);
-	if (intel_gvt_hypervisor_get_vfio_device(vgpu)) {
-		gvt_vgpu_err("get vfio device failed\n");
+	vgpu->vfio_device = vfio_device_get_from_dev(mdev_dev(vgpu->mdev));
+	if (!vgpu->vfio_device) {
+		gvt_vgpu_err("failed to get vfio device\n");
 		mutex_unlock(&vgpu->dmabuf_lock);
+		ret = -ENODEV;
 		goto out_free_info;
 	}
 	mutex_unlock(&vgpu->dmabuf_lock);
@@ -605,7 +607,7 @@ void intel_vgpu_dmabuf_cleanup(struct intel_vgpu *vgpu)
 		dmabuf_obj->vgpu = NULL;
 
 		idr_remove(&vgpu->object_idr, dmabuf_obj->dmabuf_id);
-		intel_gvt_hypervisor_put_vfio_device(vgpu);
+		vfio_device_put(vgpu->vfio_device);
 		list_del(pos);
 
 		/* dmabuf_obj might be freed in dmabuf_obj_put */
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index 61e493e2de85..fd903d52f431 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -60,8 +60,6 @@ struct intel_gvt_mpt {
 
 	int (*set_opregion)(struct intel_vgpu *vgpu);
 	int (*set_edid)(struct intel_vgpu *vgpu, int port_num);
-	int (*get_vfio_device)(struct intel_vgpu *vgpu);
-	void (*put_vfio_device)(struct intel_vgpu *vgpu);
 	bool (*is_valid_gfn)(struct intel_vgpu *vgpu, unsigned long gfn);
 };
 
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 7b588a371adb..a74fd4c78f35 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -643,18 +643,6 @@ static int intel_vgpu_register_reg(struct intel_vgpu *vgpu,
 	return 0;
 }
 
-static int kvmgt_get_vfio_device(struct intel_vgpu *vgpu)
-{
-	vgpu->vfio_device = vfio_device_get_from_dev(
-		mdev_dev(vgpu->mdev));
-	if (!vgpu->vfio_device) {
-		gvt_vgpu_err("failed to get vfio device\n");
-		return -ENODEV;
-	}
-	return 0;
-}
-
-
 static int kvmgt_set_opregion(struct intel_vgpu *vgpu)
 {
 	void *base;
@@ -711,14 +699,6 @@ static int kvmgt_set_edid(struct intel_vgpu *vgpu, int port_num)
 	return ret;
 }
 
-static void kvmgt_put_vfio_device(struct intel_vgpu *vgpu)
-{
-	if (WARN_ON(!vgpu->vfio_device))
-		return;
-
-	vfio_device_put(vgpu->vfio_device);
-}
-
 static int intel_vgpu_create(struct mdev_device *mdev)
 {
 	struct intel_vgpu *vgpu = NULL;
@@ -2033,8 +2013,6 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
 	.dma_pin_guest_page = kvmgt_dma_pin_guest_page,
 	.set_opregion = kvmgt_set_opregion,
 	.set_edid = kvmgt_set_edid,
-	.get_vfio_device = kvmgt_get_vfio_device,
-	.put_vfio_device = kvmgt_put_vfio_device,
 	.is_valid_gfn = kvmgt_is_valid_gfn,
 };
 
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 72388ceec596..2196187203af 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -242,36 +242,6 @@ static inline int intel_gvt_hypervisor_set_edid(struct intel_vgpu *vgpu,
 	return intel_gvt_host.mpt->set_edid(vgpu, port_num);
 }
 
-/**
- * intel_gvt_hypervisor_get_vfio_device - increase vfio device ref count
- * @vgpu: a vGPU
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_get_vfio_device(struct intel_vgpu *vgpu)
-{
-	if (!intel_gvt_host.mpt->get_vfio_device)
-		return 0;
-
-	return intel_gvt_host.mpt->get_vfio_device(vgpu);
-}
-
-/**
- * intel_gvt_hypervisor_put_vfio_device - decrease vfio device ref count
- * @vgpu: a vGPU
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline void intel_gvt_hypervisor_put_vfio_device(struct intel_vgpu *vgpu)
-{
-	if (!intel_gvt_host.mpt->put_vfio_device)
-		return;
-
-	intel_gvt_host.mpt->put_vfio_device(vgpu);
-}
-
 /**
  * intel_gvt_hypervisor_is_valid_gfn - check if a visible gfn
  * @vgpu: a vGPU
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 13/21] drm/i915/gvt: devirtualize ->set_edid and ->set_opregion
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (11 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 12/21] drm/i915/gvt: devirtualize ->{get,put}_vfio_device Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 14/21] drm/i915/gvt: devirtualize ->detach_vgpu Christoph Hellwig
                   ` (9 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just call the code to setup the opregions and EDID data directly.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.h       |  3 +++
 drivers/gpu/drm/i915/gvt/hypercall.h |  3 ---
 drivers/gpu/drm/i915/gvt/kvmgt.c     |  6 ++----
 drivers/gpu/drm/i915/gvt/mpt.h       | 32 ----------------------------
 drivers/gpu/drm/i915/gvt/vgpu.c      |  6 +++---
 5 files changed, 8 insertions(+), 42 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index e604d6f6d06a..1cb4b48d9a25 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -516,6 +516,9 @@ void intel_gvt_reset_vgpu(struct intel_vgpu *vgpu);
 void intel_gvt_activate_vgpu(struct intel_vgpu *vgpu);
 void intel_gvt_deactivate_vgpu(struct intel_vgpu *vgpu);
 
+int intel_gvt_set_opregion(struct intel_vgpu *vgpu);
+int intel_gvt_set_edid(struct intel_vgpu *vgpu, int port_num);
+
 /* validating GM functions */
 #define vgpu_gmadr_is_aperture(vgpu, gmadr) \
 	((gmadr >= vgpu_aperture_gmadr_base(vgpu)) && \
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index fd903d52f431..091249a924a8 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -57,9 +57,6 @@ struct intel_gvt_mpt {
 				dma_addr_t dma_addr);
 
 	int (*dma_pin_guest_page)(struct intel_vgpu *vgpu, dma_addr_t dma_addr);
-
-	int (*set_opregion)(struct intel_vgpu *vgpu);
-	int (*set_edid)(struct intel_vgpu *vgpu, int port_num);
 	bool (*is_valid_gfn)(struct intel_vgpu *vgpu, unsigned long gfn);
 };
 
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index a74fd4c78f35..572d51950abb 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -643,7 +643,7 @@ static int intel_vgpu_register_reg(struct intel_vgpu *vgpu,
 	return 0;
 }
 
-static int kvmgt_set_opregion(struct intel_vgpu *vgpu)
+int intel_gvt_set_opregion(struct intel_vgpu *vgpu)
 {
 	void *base;
 	int ret;
@@ -670,7 +670,7 @@ static int kvmgt_set_opregion(struct intel_vgpu *vgpu)
 	return ret;
 }
 
-static int kvmgt_set_edid(struct intel_vgpu *vgpu, int port_num)
+int intel_gvt_set_edid(struct intel_vgpu *vgpu, int port_num)
 {
 	struct intel_vgpu_port *port = intel_vgpu_port(vgpu, port_num);
 	struct vfio_edid_region *base;
@@ -2011,8 +2011,6 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
 	.dma_map_guest_page = kvmgt_dma_map_guest_page,
 	.dma_unmap_guest_page = kvmgt_dma_unmap_guest_page,
 	.dma_pin_guest_page = kvmgt_dma_pin_guest_page,
-	.set_opregion = kvmgt_set_opregion,
-	.set_edid = kvmgt_set_edid,
 	.is_valid_gfn = kvmgt_is_valid_gfn,
 };
 
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 2196187203af..9738aa3377b4 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -210,38 +210,6 @@ intel_gvt_hypervisor_dma_pin_guest_page(struct intel_vgpu *vgpu,
 	return intel_gvt_host.mpt->dma_pin_guest_page(vgpu, dma_addr);
 }
 
-/**
- * intel_gvt_hypervisor_set_opregion - Set opregion for guest
- * @vgpu: a vGPU
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_set_opregion(struct intel_vgpu *vgpu)
-{
-	if (!intel_gvt_host.mpt->set_opregion)
-		return 0;
-
-	return intel_gvt_host.mpt->set_opregion(vgpu);
-}
-
-/**
- * intel_gvt_hypervisor_set_edid - Set EDID region for guest
- * @vgpu: a vGPU
- * @port_num: display port number
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_set_edid(struct intel_vgpu *vgpu,
-						int port_num)
-{
-	if (!intel_gvt_host.mpt->set_edid)
-		return 0;
-
-	return intel_gvt_host.mpt->set_edid(vgpu, port_num);
-}
-
 /**
  * intel_gvt_hypervisor_is_valid_gfn - check if a visible gfn
  * @vgpu: a vGPU
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index 9e8fa9f388eb..2fadd6fea5ac 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -426,14 +426,14 @@ static struct intel_vgpu *__intel_gvt_create_vgpu(struct intel_gvt *gvt,
 
 	intel_gvt_debugfs_add_vgpu(vgpu);
 
-	ret = intel_gvt_hypervisor_set_opregion(vgpu);
+	ret = intel_gvt_set_opregion(vgpu);
 	if (ret)
 		goto out_clean_sched_policy;
 
 	if (IS_BROADWELL(dev_priv) || IS_BROXTON(dev_priv))
-		ret = intel_gvt_hypervisor_set_edid(vgpu, PORT_B);
+		ret = intel_gvt_set_edid(vgpu, PORT_B);
 	else
-		ret = intel_gvt_hypervisor_set_edid(vgpu, PORT_D);
+		ret = intel_gvt_set_edid(vgpu, PORT_D);
 	if (ret)
 		goto out_clean_sched_policy;
 
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 14/21] drm/i915/gvt: devirtualize ->detach_vgpu
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (12 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 13/21] drm/i915/gvt: devirtualize ->set_edid and ->set_opregion Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 15/21] drm/i915/gvt: devirtualize ->inject_msi Christoph Hellwig
                   ` (8 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just call the function directly.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.h       |  1 +
 drivers/gpu/drm/i915/gvt/hypercall.h |  1 -
 drivers/gpu/drm/i915/gvt/kvmgt.c     |  3 +--
 drivers/gpu/drm/i915/gvt/mpt.h       | 16 ----------------
 drivers/gpu/drm/i915/gvt/vgpu.c      |  2 +-
 5 files changed, 3 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 1cb4b48d9a25..82cd00a5b904 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -580,6 +580,7 @@ void populate_pvinfo_page(struct intel_vgpu *vgpu);
 
 int intel_gvt_scan_and_shadow_workload(struct intel_vgpu_workload *workload);
 void enter_failsafe_mode(struct intel_vgpu *vgpu, int reason);
+void intel_vgpu_detach_regions(struct intel_vgpu *vgpu);
 
 enum {
 	GVT_FAILSAFE_UNSUPPORTED_GUEST,
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index 091249a924a8..08c622c4079b 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -45,7 +45,6 @@ struct intel_vgpu;
 struct intel_gvt_mpt {
 	int (*host_init)(struct device *dev, void *gvt);
 	void (*host_exit)(struct device *dev, void *gvt);
-	void (*detach_vgpu)(struct intel_vgpu *vgpu);
 	int (*inject_msi)(struct intel_vgpu *vgpu, u32 addr, u16 data);
 	int (*enable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
 	int (*disable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 572d51950abb..218c3deaf2b4 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1839,7 +1839,7 @@ static bool kvmgt_guest_exit(struct intel_vgpu *info)
 	return true;
 }
 
-static void kvmgt_detach_vgpu(struct intel_vgpu *vgpu)
+void intel_vgpu_detach_regions(struct intel_vgpu *vgpu)
 {
 	int i;
 
@@ -2003,7 +2003,6 @@ static bool kvmgt_is_valid_gfn(struct intel_vgpu *vgpu, unsigned long gfn)
 static const struct intel_gvt_mpt kvmgt_mpt = {
 	.host_init = kvmgt_host_init,
 	.host_exit = kvmgt_host_exit,
-	.detach_vgpu = kvmgt_detach_vgpu,
 	.inject_msi = kvmgt_inject_msi,
 	.enable_page_track = kvmgt_page_track_add,
 	.disable_page_track = kvmgt_page_track_remove,
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 9738aa3377b4..78efcf1e6946 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -71,22 +71,6 @@ static inline void intel_gvt_hypervisor_host_exit(struct device *dev, void *gvt)
 	intel_gvt_host.mpt->host_exit(dev, gvt);
 }
 
-/**
- * intel_gvt_hypervisor_detach_vgpu - call hypervisor to release vGPU
- * related stuffs inside hypervisor.
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline void intel_gvt_hypervisor_detach_vgpu(struct intel_vgpu *vgpu)
-{
-	/* optional to provide */
-	if (!intel_gvt_host.mpt->detach_vgpu)
-		return;
-
-	intel_gvt_host.mpt->detach_vgpu(vgpu);
-}
-
 #define MSI_CAP_CONTROL(offset) (offset + 2)
 #define MSI_CAP_ADDRESS(offset) (offset + 4)
 #define MSI_CAP_DATA(offset) (offset + 8)
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index 2fadd6fea5ac..7a7f768083d2 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -293,7 +293,7 @@ void intel_gvt_destroy_vgpu(struct intel_vgpu *vgpu)
 	intel_vgpu_clean_opregion(vgpu);
 	intel_vgpu_reset_ggtt(vgpu, true);
 	intel_vgpu_clean_gtt(vgpu);
-	intel_gvt_hypervisor_detach_vgpu(vgpu);
+	intel_vgpu_detach_regions(vgpu);
 	intel_vgpu_free_resource(vgpu);
 	intel_vgpu_clean_mmio(vgpu);
 	intel_vgpu_dmabuf_cleanup(vgpu);
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 15/21] drm/i915/gvt: devirtualize ->inject_msi
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (13 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 14/21] drm/i915/gvt: devirtualize ->detach_vgpu Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 16/21] drm/i915/gvt: devirtualize ->is_valid_gfn Christoph Hellwig
                   ` (7 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just open code the MSI injection in a single place instead of going
through the method table.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/hypercall.h |  1 -
 drivers/gpu/drm/i915/gvt/interrupt.c | 38 +++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/gvt/kvmgt.c     | 24 ------------------
 drivers/gpu/drm/i915/gvt/mpt.h       | 37 ---------------------------
 4 files changed, 37 insertions(+), 63 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index 08c622c4079b..de63bd8dd05b 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -45,7 +45,6 @@ struct intel_vgpu;
 struct intel_gvt_mpt {
 	int (*host_init)(struct device *dev, void *gvt);
 	void (*host_exit)(struct device *dev, void *gvt);
-	int (*inject_msi)(struct intel_vgpu *vgpu, u32 addr, u16 data);
 	int (*enable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
 	int (*disable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
 	unsigned long (*gfn_to_mfn)(struct intel_vgpu *vgpu, unsigned long gfn);
diff --git a/drivers/gpu/drm/i915/gvt/interrupt.c b/drivers/gpu/drm/i915/gvt/interrupt.c
index 614b951d919f..f3c350a9a80f 100644
--- a/drivers/gpu/drm/i915/gvt/interrupt.c
+++ b/drivers/gpu/drm/i915/gvt/interrupt.c
@@ -396,9 +396,45 @@ static void init_irq_map(struct intel_gvt_irq *irq)
 }
 
 /* =======================vEvent injection===================== */
+
+#define MSI_CAP_CONTROL(offset) (offset + 2)
+#define MSI_CAP_ADDRESS(offset) (offset + 4)
+#define MSI_CAP_DATA(offset) (offset + 8)
+#define MSI_CAP_EN 0x1
+
 static int inject_virtual_interrupt(struct intel_vgpu *vgpu)
 {
-	return intel_gvt_hypervisor_inject_msi(vgpu);
+	unsigned long offset = vgpu->gvt->device_info.msi_cap_offset;
+	u16 control, data;
+	u32 addr;
+
+	control = *(u16 *)(vgpu_cfg_space(vgpu) + MSI_CAP_CONTROL(offset));
+	addr = *(u32 *)(vgpu_cfg_space(vgpu) + MSI_CAP_ADDRESS(offset));
+	data = *(u16 *)(vgpu_cfg_space(vgpu) + MSI_CAP_DATA(offset));
+
+	/* Do not generate MSI if MSIEN is disabled */
+	if (!(control & MSI_CAP_EN))
+		return 0;
+
+	if (WARN(control & GENMASK(15, 1), "only support one MSI format\n"))
+		return -EINVAL;
+
+	trace_inject_msi(vgpu->id, addr, data);
+
+	/*
+	 * When guest is powered off, msi_trigger is set to NULL, but vgpu's
+	 * config and mmio register isn't restored to default during guest
+	 * poweroff. If this vgpu is still used in next vm, this vgpu's pipe
+	 * may be enabled, then once this vgpu is active, it will get inject
+	 * vblank interrupt request. But msi_trigger is null until msi is
+	 * enabled by guest. so if msi_trigger is null, success is still
+	 * returned and don't inject interrupt into guest.
+	 */
+	if (!vgpu->attached)
+		return -ESRCH;
+	if (vgpu->msi_trigger && eventfd_signal(vgpu->msi_trigger, 1) != 1)
+		return -EFAULT;
+	return 0;
 }
 
 static void propagate_event(struct intel_gvt_irq *irq,
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 218c3deaf2b4..3a035802e4f0 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1855,29 +1855,6 @@ void intel_vgpu_detach_regions(struct intel_vgpu *vgpu)
 	vgpu->region = NULL;
 }
 
-static int kvmgt_inject_msi(struct intel_vgpu *vgpu, u32 addr, u16 data)
-{
-	if (!vgpu->attached)
-		return -ESRCH;
-
-	/*
-	 * When guest is poweroff, msi_trigger is set to NULL, but vgpu's
-	 * config and mmio register isn't restored to default during guest
-	 * poweroff. If this vgpu is still used in next vm, this vgpu's pipe
-	 * may be enabled, then once this vgpu is active, it will get inject
-	 * vblank interrupt request. But msi_trigger is null until msi is
-	 * enabled by guest. so if msi_trigger is null, success is still
-	 * returned and don't inject interrupt into guest.
-	 */
-	if (vgpu->msi_trigger == NULL)
-		return 0;
-
-	if (eventfd_signal(vgpu->msi_trigger, 1) == 1)
-		return 0;
-
-	return -EFAULT;
-}
-
 static unsigned long kvmgt_gfn_to_pfn(struct intel_vgpu *vgpu,
 		unsigned long gfn)
 {
@@ -2003,7 +1980,6 @@ static bool kvmgt_is_valid_gfn(struct intel_vgpu *vgpu, unsigned long gfn)
 static const struct intel_gvt_mpt kvmgt_mpt = {
 	.host_init = kvmgt_host_init,
 	.host_exit = kvmgt_host_exit,
-	.inject_msi = kvmgt_inject_msi,
 	.enable_page_track = kvmgt_page_track_add,
 	.disable_page_track = kvmgt_page_track_remove,
 	.gfn_to_mfn = kvmgt_gfn_to_pfn,
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 78efcf1e6946..59369e8b3b69 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -71,43 +71,6 @@ static inline void intel_gvt_hypervisor_host_exit(struct device *dev, void *gvt)
 	intel_gvt_host.mpt->host_exit(dev, gvt);
 }
 
-#define MSI_CAP_CONTROL(offset) (offset + 2)
-#define MSI_CAP_ADDRESS(offset) (offset + 4)
-#define MSI_CAP_DATA(offset) (offset + 8)
-#define MSI_CAP_EN 0x1
-
-/**
- * intel_gvt_hypervisor_inject_msi - inject a MSI interrupt into vGPU
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_inject_msi(struct intel_vgpu *vgpu)
-{
-	unsigned long offset = vgpu->gvt->device_info.msi_cap_offset;
-	u16 control, data;
-	u32 addr;
-	int ret;
-
-	control = *(u16 *)(vgpu_cfg_space(vgpu) + MSI_CAP_CONTROL(offset));
-	addr = *(u32 *)(vgpu_cfg_space(vgpu) + MSI_CAP_ADDRESS(offset));
-	data = *(u16 *)(vgpu_cfg_space(vgpu) + MSI_CAP_DATA(offset));
-
-	/* Do not generate MSI if MSIEN is disable */
-	if (!(control & MSI_CAP_EN))
-		return 0;
-
-	if (WARN(control & GENMASK(15, 1), "only support one MSI format\n"))
-		return -EINVAL;
-
-	trace_inject_msi(vgpu->id, addr, data);
-
-	ret = intel_gvt_host.mpt->inject_msi(vgpu, addr, data);
-	if (ret)
-		return ret;
-	return 0;
-}
-
 /**
  * intel_gvt_hypervisor_enable_page_track - track a guest page
  * @vgpu: a vGPU
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 16/21] drm/i915/gvt: devirtualize ->is_valid_gfn
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (14 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 15/21] drm/i915/gvt: devirtualize ->inject_msi Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 17/21] drm/i915/gvt: devirtualize ->gfn_to_mfn Christoph Hellwig
                   ` (6 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just call the code directly and move towards the callers.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gtt.c       | 20 ++++++++++++++++++--
 drivers/gpu/drm/i915/gvt/hypercall.h |  1 -
 drivers/gpu/drm/i915/gvt/kvmgt.c     | 17 -----------------
 drivers/gpu/drm/i915/gvt/mpt.h       | 17 -----------------
 4 files changed, 18 insertions(+), 37 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gtt.c b/drivers/gpu/drm/i915/gvt/gtt.c
index 0a953ab7a38b..5783642d4d79 100644
--- a/drivers/gpu/drm/i915/gvt/gtt.c
+++ b/drivers/gpu/drm/i915/gvt/gtt.c
@@ -47,6 +47,22 @@
 static bool enable_out_of_sync = false;
 static int preallocated_oos_pages = 8192;
 
+static bool intel_gvt_is_valid_gfn(struct intel_vgpu *vgpu, unsigned long gfn)
+{
+	struct kvm *kvm = vgpu->kvm;
+	int idx;
+	bool ret;
+
+	if (!vgpu->attached)
+		return false;
+
+	idx = srcu_read_lock(&kvm->srcu);
+	ret = kvm_is_visible_gfn(kvm, gfn);
+	srcu_read_unlock(&kvm->srcu, idx);
+
+	return ret;
+}
+
 /*
  * validate a gm address and related range size,
  * translate it to host gm address
@@ -1329,7 +1345,7 @@ static int ppgtt_populate_spt(struct intel_vgpu_ppgtt_spt *spt)
 			ppgtt_set_shadow_entry(spt, &se, i);
 		} else {
 			gfn = ops->get_pfn(&ge);
-			if (!intel_gvt_hypervisor_is_valid_gfn(vgpu, gfn)) {
+			if (!intel_gvt_is_valid_gfn(vgpu, gfn)) {
 				ops->set_pfn(&se, gvt->gtt.scratch_mfn);
 				ppgtt_set_shadow_entry(spt, &se, i);
 				continue;
@@ -2313,7 +2329,7 @@ static int emulate_ggtt_mmio_write(struct intel_vgpu *vgpu, unsigned int off,
 		/* one PTE update may be issued in multiple writes and the
 		 * first write may not construct a valid gfn
 		 */
-		if (!intel_gvt_hypervisor_is_valid_gfn(vgpu, gfn)) {
+		if (!intel_gvt_is_valid_gfn(vgpu, gfn)) {
 			ops->set_pfn(&m, gvt->gtt.scratch_mfn);
 			goto out;
 		}
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index de63bd8dd05b..c1a9eeed0460 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -55,7 +55,6 @@ struct intel_gvt_mpt {
 				dma_addr_t dma_addr);
 
 	int (*dma_pin_guest_page)(struct intel_vgpu *vgpu, dma_addr_t dma_addr);
-	bool (*is_valid_gfn)(struct intel_vgpu *vgpu, unsigned long gfn);
 };
 
 #endif /* _GVT_HYPERCALL_H_ */
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 3a035802e4f0..2d3ef59e7227 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1961,22 +1961,6 @@ static void kvmgt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
 	mutex_unlock(&vgpu->cache_lock);
 }
 
-static bool kvmgt_is_valid_gfn(struct intel_vgpu *vgpu, unsigned long gfn)
-{
-	struct kvm *kvm = vgpu->kvm;
-	int idx;
-	bool ret;
-
-	if (!vgpu->attached)
-		return false;
-
-	idx = srcu_read_lock(&kvm->srcu);
-	ret = kvm_is_visible_gfn(kvm, gfn);
-	srcu_read_unlock(&kvm->srcu, idx);
-
-	return ret;
-}
-
 static const struct intel_gvt_mpt kvmgt_mpt = {
 	.host_init = kvmgt_host_init,
 	.host_exit = kvmgt_host_exit,
@@ -1986,7 +1970,6 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
 	.dma_map_guest_page = kvmgt_dma_map_guest_page,
 	.dma_unmap_guest_page = kvmgt_dma_unmap_guest_page,
 	.dma_pin_guest_page = kvmgt_dma_pin_guest_page,
-	.is_valid_gfn = kvmgt_is_valid_gfn,
 };
 
 struct intel_gvt_host intel_gvt_host = {
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 59369e8b3b69..1a796f2181ba 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -157,21 +157,4 @@ intel_gvt_hypervisor_dma_pin_guest_page(struct intel_vgpu *vgpu,
 	return intel_gvt_host.mpt->dma_pin_guest_page(vgpu, dma_addr);
 }
 
-/**
- * intel_gvt_hypervisor_is_valid_gfn - check if a visible gfn
- * @vgpu: a vGPU
- * @gfn: guest PFN
- *
- * Returns:
- * true on valid gfn, false on not.
- */
-static inline bool intel_gvt_hypervisor_is_valid_gfn(
-		struct intel_vgpu *vgpu, unsigned long gfn)
-{
-	if (!intel_gvt_host.mpt->is_valid_gfn)
-		return true;
-
-	return intel_gvt_host.mpt->is_valid_gfn(vgpu, gfn);
-}
-
 #endif /* _GVT_MPT_H_ */
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 17/21] drm/i915/gvt: devirtualize ->gfn_to_mfn
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (15 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 16/21] drm/i915/gvt: devirtualize ->is_valid_gfn Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 18/21] drm/i915/gvt: devirtualize ->{enable,disable}_page_track Christoph Hellwig
                   ` (5 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just open code it in the only caller.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gtt.c       |  9 +++++----
 drivers/gpu/drm/i915/gvt/hypercall.h |  1 -
 drivers/gpu/drm/i915/gvt/kvmgt.c     | 16 ----------------
 drivers/gpu/drm/i915/gvt/mpt.h       | 14 --------------
 4 files changed, 5 insertions(+), 35 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gtt.c b/drivers/gpu/drm/i915/gvt/gtt.c
index 5783642d4d79..75782f59df8e 100644
--- a/drivers/gpu/drm/i915/gvt/gtt.c
+++ b/drivers/gpu/drm/i915/gvt/gtt.c
@@ -1176,15 +1176,16 @@ static int is_2MB_gtt_possible(struct intel_vgpu *vgpu,
 	struct intel_gvt_gtt_entry *entry)
 {
 	struct intel_gvt_gtt_pte_ops *ops = vgpu->gvt->gtt.pte_ops;
-	unsigned long pfn;
+	kvm_pfn_t pfn;
 
 	if (!HAS_PAGE_SIZES(vgpu->gvt->gt->i915, I915_GTT_PAGE_SIZE_2M))
 		return 0;
 
-	pfn = intel_gvt_hypervisor_gfn_to_mfn(vgpu, ops->get_pfn(entry));
-	if (pfn == INTEL_GVT_INVALID_ADDR)
+	if (!vgpu->attached)
+		return -EINVAL;
+	pfn = gfn_to_pfn(vgpu->kvm, ops->get_pfn(entry));
+	if (is_error_noslot_pfn(pfn))
 		return -EINVAL;
-
 	return PageTransHuge(pfn_to_page(pfn));
 }
 
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index c1a9eeed0460..dbde492cafc8 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -47,7 +47,6 @@ struct intel_gvt_mpt {
 	void (*host_exit)(struct device *dev, void *gvt);
 	int (*enable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
 	int (*disable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
-	unsigned long (*gfn_to_mfn)(struct intel_vgpu *vgpu, unsigned long gfn);
 
 	int (*dma_map_guest_page)(struct intel_vgpu *vgpu, unsigned long gfn,
 				  unsigned long size, dma_addr_t *dma_addr);
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 2d3ef59e7227..226e3f3c41df 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1855,21 +1855,6 @@ void intel_vgpu_detach_regions(struct intel_vgpu *vgpu)
 	vgpu->region = NULL;
 }
 
-static unsigned long kvmgt_gfn_to_pfn(struct intel_vgpu *vgpu,
-		unsigned long gfn)
-{
-	kvm_pfn_t pfn;
-
-	if (!vgpu->attached)
-		return INTEL_GVT_INVALID_ADDR;
-
-	pfn = gfn_to_pfn(vgpu->kvm, gfn);
-	if (is_error_noslot_pfn(pfn))
-		return INTEL_GVT_INVALID_ADDR;
-
-	return pfn;
-}
-
 static int kvmgt_dma_map_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		unsigned long size, dma_addr_t *dma_addr)
 {
@@ -1966,7 +1951,6 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
 	.host_exit = kvmgt_host_exit,
 	.enable_page_track = kvmgt_page_track_add,
 	.disable_page_track = kvmgt_page_track_remove,
-	.gfn_to_mfn = kvmgt_gfn_to_pfn,
 	.dma_map_guest_page = kvmgt_dma_map_guest_page,
 	.dma_unmap_guest_page = kvmgt_dma_unmap_guest_page,
 	.dma_pin_guest_page = kvmgt_dma_pin_guest_page,
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 1a796f2181ba..2d4bb6eaa08e 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -99,20 +99,6 @@ static inline int intel_gvt_hypervisor_disable_page_track(
 	return intel_gvt_host.mpt->disable_page_track(vgpu, gfn);
 }
 
-/**
- * intel_gvt_hypervisor_gfn_to_mfn - translate a GFN to MFN
- * @vgpu: a vGPU
- * @gpfn: guest pfn
- *
- * Returns:
- * MFN on success, INTEL_GVT_INVALID_ADDR if failed.
- */
-static inline unsigned long intel_gvt_hypervisor_gfn_to_mfn(
-		struct intel_vgpu *vgpu, unsigned long gfn)
-{
-	return intel_gvt_host.mpt->gfn_to_mfn(vgpu, gfn);
-}
-
 /**
  * intel_gvt_hypervisor_dma_map_guest_page - setup dma map for guest page
  * @vgpu: a vGPU
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 18/21] drm/i915/gvt: devirtualize ->{enable,disable}_page_track
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (16 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 17/21] drm/i915/gvt: devirtualize ->gfn_to_mfn Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 19/21] drm/i915/gvt: devirtualize ->dma_{,un}map_guest_page Christoph Hellwig
                   ` (4 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just call the kvmgt functions directly.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.h        |  3 +++
 drivers/gpu/drm/i915/gvt/hypercall.h  |  2 --
 drivers/gpu/drm/i915/gvt/kvmgt.c      |  6 ++----
 drivers/gpu/drm/i915/gvt/mpt.h        | 28 ---------------------------
 drivers/gpu/drm/i915/gvt/page_track.c |  8 ++++----
 5 files changed, 9 insertions(+), 38 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 82cd00a5b904..e186ded7ec68 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -764,6 +764,9 @@ void intel_gvt_debugfs_remove_vgpu(struct intel_vgpu *vgpu);
 void intel_gvt_debugfs_init(struct intel_gvt *gvt);
 void intel_gvt_debugfs_clean(struct intel_gvt *gvt);
 
+int intel_gvt_page_track_add(struct intel_vgpu *info, u64 gfn);
+int intel_gvt_page_track_remove(struct intel_vgpu *info, u64 gfn);
+
 #include "trace.h"
 #include "mpt.h"
 
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index dbde492cafc8..ded13a63ab66 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -45,8 +45,6 @@ struct intel_vgpu;
 struct intel_gvt_mpt {
 	int (*host_init)(struct device *dev, void *gvt);
 	void (*host_exit)(struct device *dev, void *gvt);
-	int (*enable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
-	int (*disable_page_track)(struct intel_vgpu *vgpu, u64 gfn);
 
 	int (*dma_map_guest_page)(struct intel_vgpu *vgpu, unsigned long gfn,
 				  unsigned long size, dma_addr_t *dma_addr);
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 226e3f3c41df..72e33332c412 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1683,7 +1683,7 @@ static void kvmgt_host_exit(struct device *dev, void *gvt)
 	intel_gvt_cleanup_vgpu_type_groups((struct intel_gvt *)gvt);
 }
 
-static int kvmgt_page_track_add(struct intel_vgpu *info, u64 gfn)
+int intel_gvt_page_track_add(struct intel_vgpu *info, u64 gfn)
 {
 	struct kvm *kvm = info->kvm;
 	struct kvm_memory_slot *slot;
@@ -1713,7 +1713,7 @@ static int kvmgt_page_track_add(struct intel_vgpu *info, u64 gfn)
 	return 0;
 }
 
-static int kvmgt_page_track_remove(struct intel_vgpu *info, u64 gfn)
+int intel_gvt_page_track_remove(struct intel_vgpu *info, u64 gfn)
 {
 	struct kvm *kvm = info->kvm;
 	struct kvm_memory_slot *slot;
@@ -1949,8 +1949,6 @@ static void kvmgt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
 static const struct intel_gvt_mpt kvmgt_mpt = {
 	.host_init = kvmgt_host_init,
 	.host_exit = kvmgt_host_exit,
-	.enable_page_track = kvmgt_page_track_add,
-	.disable_page_track = kvmgt_page_track_remove,
 	.dma_map_guest_page = kvmgt_dma_map_guest_page,
 	.dma_unmap_guest_page = kvmgt_dma_unmap_guest_page,
 	.dma_pin_guest_page = kvmgt_dma_pin_guest_page,
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 2d4bb6eaa08e..d2723ac8bb04 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -71,34 +71,6 @@ static inline void intel_gvt_hypervisor_host_exit(struct device *dev, void *gvt)
 	intel_gvt_host.mpt->host_exit(dev, gvt);
 }
 
-/**
- * intel_gvt_hypervisor_enable_page_track - track a guest page
- * @vgpu: a vGPU
- * @gfn: the gfn of guest
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_enable_page_track(
-		struct intel_vgpu *vgpu, unsigned long gfn)
-{
-	return intel_gvt_host.mpt->enable_page_track(vgpu, gfn);
-}
-
-/**
- * intel_gvt_hypervisor_disable_page_track - untrack a guest page
- * @vgpu: a vGPU
- * @gfn: the gfn of guest
- *
- * Returns:
- * Zero on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_disable_page_track(
-		struct intel_vgpu *vgpu, unsigned long gfn)
-{
-	return intel_gvt_host.mpt->disable_page_track(vgpu, gfn);
-}
-
 /**
  * intel_gvt_hypervisor_dma_map_guest_page - setup dma map for guest page
  * @vgpu: a vGPU
diff --git a/drivers/gpu/drm/i915/gvt/page_track.c b/drivers/gpu/drm/i915/gvt/page_track.c
index 84856022528e..3375b51c75f1 100644
--- a/drivers/gpu/drm/i915/gvt/page_track.c
+++ b/drivers/gpu/drm/i915/gvt/page_track.c
@@ -87,7 +87,7 @@ void intel_vgpu_unregister_page_track(struct intel_vgpu *vgpu,
 	track = radix_tree_delete(&vgpu->page_track_tree, gfn);
 	if (track) {
 		if (track->tracked)
-			intel_gvt_hypervisor_disable_page_track(vgpu, gfn);
+			intel_gvt_page_track_remove(vgpu, gfn);
 		kfree(track);
 	}
 }
@@ -112,7 +112,7 @@ int intel_vgpu_enable_page_track(struct intel_vgpu *vgpu, unsigned long gfn)
 	if (track->tracked)
 		return 0;
 
-	ret = intel_gvt_hypervisor_enable_page_track(vgpu, gfn);
+	ret = intel_gvt_page_track_add(vgpu, gfn);
 	if (ret)
 		return ret;
 	track->tracked = true;
@@ -139,7 +139,7 @@ int intel_vgpu_disable_page_track(struct intel_vgpu *vgpu, unsigned long gfn)
 	if (!track->tracked)
 		return 0;
 
-	ret = intel_gvt_hypervisor_disable_page_track(vgpu, gfn);
+	ret = intel_gvt_page_track_remove(vgpu, gfn);
 	if (ret)
 		return ret;
 	track->tracked = false;
@@ -172,7 +172,7 @@ int intel_vgpu_page_track_handler(struct intel_vgpu *vgpu, u64 gpa,
 
 	if (unlikely(vgpu->failsafe)) {
 		/* Remove write protection to prevent furture traps. */
-		intel_vgpu_disable_page_track(vgpu, gpa >> PAGE_SHIFT);
+		intel_gvt_page_track_remove(vgpu, gpa >> PAGE_SHIFT);
 	} else {
 		ret = page_track->handler(page_track, gpa, data, bytes);
 		if (ret)
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 19/21] drm/i915/gvt: devirtualize ->dma_{,un}map_guest_page
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (17 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 18/21] drm/i915/gvt: devirtualize ->{enable,disable}_page_track Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 20/21] drm/i915/gvt: devirtualize dma_pin_guest_page Christoph Hellwig
                   ` (3 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just call the functions directly.  Also remove a pointless wrapper.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/dmabuf.c    | 10 ++--------
 drivers/gpu/drm/i915/gvt/gtt.c       | 20 +++++++++----------
 drivers/gpu/drm/i915/gvt/gvt.h       |  4 ++++
 drivers/gpu/drm/i915/gvt/hypercall.h |  5 -----
 drivers/gpu/drm/i915/gvt/kvmgt.c     |  6 ++----
 drivers/gpu/drm/i915/gvt/mpt.h       | 29 ----------------------------
 6 files changed, 17 insertions(+), 57 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/dmabuf.c b/drivers/gpu/drm/i915/gvt/dmabuf.c
index bc53a0c60b44..03d82f4f9aa2 100644
--- a/drivers/gpu/drm/i915/gvt/dmabuf.c
+++ b/drivers/gpu/drm/i915/gvt/dmabuf.c
@@ -48,12 +48,6 @@ static int vgpu_pin_dma_address(struct intel_vgpu *vgpu,
 	return ret;
 }
 
-static void vgpu_unpin_dma_address(struct intel_vgpu *vgpu,
-				   dma_addr_t dma_addr)
-{
-	intel_gvt_hypervisor_dma_unmap_guest_page(vgpu, dma_addr);
-}
-
 static int vgpu_gem_get_pages(
 		struct drm_i915_gem_object *obj)
 {
@@ -108,7 +102,7 @@ static int vgpu_gem_get_pages(
 		for_each_sg(st->sgl, sg, i, j) {
 			dma_addr = sg_dma_address(sg);
 			if (dma_addr)
-				vgpu_unpin_dma_address(vgpu, dma_addr);
+				intel_gvt_dma_unmap_guest_page(vgpu, dma_addr);
 		}
 		sg_free_table(st);
 		kfree(st);
@@ -130,7 +124,7 @@ static void vgpu_gem_put_pages(struct drm_i915_gem_object *obj,
 		int i;
 
 		for_each_sg(pages->sgl, sg, fb_info->size, i)
-			vgpu_unpin_dma_address(vgpu,
+			intel_gvt_dma_unmap_guest_page(vgpu,
 					       sg_dma_address(sg));
 	}
 
diff --git a/drivers/gpu/drm/i915/gvt/gtt.c b/drivers/gpu/drm/i915/gvt/gtt.c
index 75782f59df8e..0ae28b7f5c07 100644
--- a/drivers/gpu/drm/i915/gvt/gtt.c
+++ b/drivers/gpu/drm/i915/gvt/gtt.c
@@ -1011,7 +1011,7 @@ static inline void ppgtt_invalidate_pte(struct intel_vgpu_ppgtt_spt *spt,
 	if (!pfn || pfn == vgpu->gtt.scratch_pt[type].page_mfn)
 		return;
 
-	intel_gvt_hypervisor_dma_unmap_guest_page(vgpu, pfn << PAGE_SHIFT);
+	intel_gvt_dma_unmap_guest_page(vgpu, pfn << PAGE_SHIFT);
 }
 
 static int ppgtt_invalidate_spt(struct intel_vgpu_ppgtt_spt *spt)
@@ -1210,8 +1210,8 @@ static int split_2MB_gtt_entry(struct intel_vgpu *vgpu,
 		return PTR_ERR(sub_spt);
 
 	for_each_shadow_entry(sub_spt, &sub_se, sub_index) {
-		ret = intel_gvt_hypervisor_dma_map_guest_page(vgpu,
-				start_gfn + sub_index, PAGE_SIZE, &dma_addr);
+		ret = intel_gvt_dma_map_guest_page(vgpu, start_gfn + sub_index,
+						   PAGE_SIZE, &dma_addr);
 		if (ret) {
 			ppgtt_invalidate_spt(spt);
 			return ret;
@@ -1256,8 +1256,8 @@ static int split_64KB_gtt_entry(struct intel_vgpu *vgpu,
 	ops->set_64k_splited(&entry);
 
 	for (i = 0; i < GTT_64K_PTE_STRIDE; i++) {
-		ret = intel_gvt_hypervisor_dma_map_guest_page(vgpu,
-					start_gfn + i, PAGE_SIZE, &dma_addr);
+		ret = intel_gvt_dma_map_guest_page(vgpu, start_gfn + i,
+						   PAGE_SIZE, &dma_addr);
 		if (ret)
 			return ret;
 
@@ -1311,8 +1311,7 @@ static int ppgtt_populate_shadow_entry(struct intel_vgpu *vgpu,
 	}
 
 	/* direct shadow */
-	ret = intel_gvt_hypervisor_dma_map_guest_page(vgpu, gfn, page_size,
-						      &dma_addr);
+	ret = intel_gvt_dma_map_guest_page(vgpu, gfn, page_size, &dma_addr);
 	if (ret)
 		return -ENXIO;
 
@@ -2243,8 +2242,7 @@ static void ggtt_invalidate_pte(struct intel_vgpu *vgpu,
 
 	pfn = pte_ops->get_pfn(entry);
 	if (pfn != vgpu->gvt->gtt.scratch_mfn)
-		intel_gvt_hypervisor_dma_unmap_guest_page(vgpu,
-						pfn << PAGE_SHIFT);
+		intel_gvt_dma_unmap_guest_page(vgpu, pfn << PAGE_SHIFT);
 }
 
 static int emulate_ggtt_mmio_write(struct intel_vgpu *vgpu, unsigned int off,
@@ -2335,8 +2333,8 @@ static int emulate_ggtt_mmio_write(struct intel_vgpu *vgpu, unsigned int off,
 			goto out;
 		}
 
-		ret = intel_gvt_hypervisor_dma_map_guest_page(vgpu, gfn,
-							PAGE_SIZE, &dma_addr);
+		ret = intel_gvt_dma_map_guest_page(vgpu, gfn, PAGE_SIZE,
+						   &dma_addr);
 		if (ret) {
 			gvt_vgpu_err("fail to populate guest ggtt entry\n");
 			/* guest driver may read/write the entry when partial
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index e186ded7ec68..9d5a286e1269 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -766,6 +766,10 @@ void intel_gvt_debugfs_clean(struct intel_gvt *gvt);
 
 int intel_gvt_page_track_add(struct intel_vgpu *info, u64 gfn);
 int intel_gvt_page_track_remove(struct intel_vgpu *info, u64 gfn);
+int intel_gvt_dma_map_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
+		unsigned long size, dma_addr_t *dma_addr);
+void intel_gvt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
+		dma_addr_t dma_addr);
 
 #include "trace.h"
 #include "mpt.h"
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index ded13a63ab66..ba03b3368a95 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -46,11 +46,6 @@ struct intel_gvt_mpt {
 	int (*host_init)(struct device *dev, void *gvt);
 	void (*host_exit)(struct device *dev, void *gvt);
 
-	int (*dma_map_guest_page)(struct intel_vgpu *vgpu, unsigned long gfn,
-				  unsigned long size, dma_addr_t *dma_addr);
-	void (*dma_unmap_guest_page)(struct intel_vgpu *vgpu,
-				dma_addr_t dma_addr);
-
 	int (*dma_pin_guest_page)(struct intel_vgpu *vgpu, dma_addr_t dma_addr);
 };
 
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 72e33332c412..8101c1485f12 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1855,7 +1855,7 @@ void intel_vgpu_detach_regions(struct intel_vgpu *vgpu)
 	vgpu->region = NULL;
 }
 
-static int kvmgt_dma_map_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
+int intel_gvt_dma_map_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		unsigned long size, dma_addr_t *dma_addr)
 {
 	struct gvt_dma *entry;
@@ -1931,7 +1931,7 @@ static void __gvt_dma_release(struct kref *ref)
 	__gvt_cache_remove_entry(entry->vgpu, entry);
 }
 
-static void kvmgt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
+void intel_gvt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
 		dma_addr_t dma_addr)
 {
 	struct gvt_dma *entry;
@@ -1949,8 +1949,6 @@ static void kvmgt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
 static const struct intel_gvt_mpt kvmgt_mpt = {
 	.host_init = kvmgt_host_init,
 	.host_exit = kvmgt_host_exit,
-	.dma_map_guest_page = kvmgt_dma_map_guest_page,
-	.dma_unmap_guest_page = kvmgt_dma_unmap_guest_page,
 	.dma_pin_guest_page = kvmgt_dma_pin_guest_page,
 };
 
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index d2723ac8bb04..26c1ee690f7e 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -71,35 +71,6 @@ static inline void intel_gvt_hypervisor_host_exit(struct device *dev, void *gvt)
 	intel_gvt_host.mpt->host_exit(dev, gvt);
 }
 
-/**
- * intel_gvt_hypervisor_dma_map_guest_page - setup dma map for guest page
- * @vgpu: a vGPU
- * @gfn: guest pfn
- * @size: page size
- * @dma_addr: retrieve allocated dma addr
- *
- * Returns:
- * 0 on success, negative error code if failed.
- */
-static inline int intel_gvt_hypervisor_dma_map_guest_page(
-		struct intel_vgpu *vgpu, unsigned long gfn, unsigned long size,
-		dma_addr_t *dma_addr)
-{
-	return intel_gvt_host.mpt->dma_map_guest_page(vgpu, gfn, size,
-						      dma_addr);
-}
-
-/**
- * intel_gvt_hypervisor_dma_unmap_guest_page - cancel dma map for guest page
- * @vgpu: a vGPU
- * @dma_addr: the mapped dma addr
- */
-static inline void intel_gvt_hypervisor_dma_unmap_guest_page(
-		struct intel_vgpu *vgpu, dma_addr_t dma_addr)
-{
-	intel_gvt_host.mpt->dma_unmap_guest_page(vgpu, dma_addr);
-}
-
 /**
  * intel_gvt_hypervisor_dma_pin_guest_page - pin guest dma buf
  * @vgpu: a vGPU
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 20/21] drm/i915/gvt: devirtualize dma_pin_guest_page
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (18 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 19/21] drm/i915/gvt: devirtualize ->dma_{,un}map_guest_page Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-21 15:53 ` [PATCH 21/21] drm/i915/gvt: remove struct intel_gvt_mpt Christoph Hellwig
                   ` (2 subsequent siblings)
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just call the function directly and remove a pointless wrapper.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/dmabuf.c    | 14 +-------------
 drivers/gpu/drm/i915/gvt/gvt.h       |  1 +
 drivers/gpu/drm/i915/gvt/hypercall.h |  2 --
 drivers/gpu/drm/i915/gvt/kvmgt.c     |  4 +---
 drivers/gpu/drm/i915/gvt/mpt.h       | 15 ---------------
 5 files changed, 3 insertions(+), 33 deletions(-)

diff --git a/drivers/gpu/drm/i915/gvt/dmabuf.c b/drivers/gpu/drm/i915/gvt/dmabuf.c
index 03d82f4f9aa2..eacfe7beaaf6 100644
--- a/drivers/gpu/drm/i915/gvt/dmabuf.c
+++ b/drivers/gpu/drm/i915/gvt/dmabuf.c
@@ -36,18 +36,6 @@
 
 #define GEN8_DECODE_PTE(pte) (pte & GENMASK_ULL(63, 12))
 
-static int vgpu_pin_dma_address(struct intel_vgpu *vgpu,
-				unsigned long size,
-				dma_addr_t dma_addr)
-{
-	int ret = 0;
-
-	if (intel_gvt_hypervisor_dma_pin_guest_page(vgpu, dma_addr))
-		ret = -EINVAL;
-
-	return ret;
-}
-
 static int vgpu_gem_get_pages(
 		struct drm_i915_gem_object *obj)
 {
@@ -83,7 +71,7 @@ static int vgpu_gem_get_pages(
 	for_each_sg(st->sgl, sg, page_num, i) {
 		dma_addr_t dma_addr =
 			GEN8_DECODE_PTE(readq(&gtt_entries[i]));
-		if (vgpu_pin_dma_address(vgpu, PAGE_SIZE, dma_addr)) {
+		if (intel_gvt_dma_pin_guest_page(vgpu, dma_addr)) {
 			ret = -EINVAL;
 			goto out;
 		}
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 9d5a286e1269..a8fc381f558e 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -766,6 +766,7 @@ void intel_gvt_debugfs_clean(struct intel_gvt *gvt);
 
 int intel_gvt_page_track_add(struct intel_vgpu *info, u64 gfn);
 int intel_gvt_page_track_remove(struct intel_vgpu *info, u64 gfn);
+int intel_gvt_dma_pin_guest_page(struct intel_vgpu *vgpu, dma_addr_t dma_addr);
 int intel_gvt_dma_map_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		unsigned long size, dma_addr_t *dma_addr);
 void intel_gvt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
index ba03b3368a95..d49437aeabac 100644
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ b/drivers/gpu/drm/i915/gvt/hypercall.h
@@ -45,8 +45,6 @@ struct intel_vgpu;
 struct intel_gvt_mpt {
 	int (*host_init)(struct device *dev, void *gvt);
 	void (*host_exit)(struct device *dev, void *gvt);
-
-	int (*dma_pin_guest_page)(struct intel_vgpu *vgpu, dma_addr_t dma_addr);
 };
 
 #endif /* _GVT_HYPERCALL_H_ */
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 8101c1485f12..5f10ed5eccde 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1902,8 +1902,7 @@ int intel_gvt_dma_map_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 	return ret;
 }
 
-static int kvmgt_dma_pin_guest_page(struct intel_vgpu *vgpu,
-		dma_addr_t dma_addr)
+int intel_gvt_dma_pin_guest_page(struct intel_vgpu *vgpu, dma_addr_t dma_addr)
 {
 	struct gvt_dma *entry;
 	int ret = 0;
@@ -1949,7 +1948,6 @@ void intel_gvt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
 static const struct intel_gvt_mpt kvmgt_mpt = {
 	.host_init = kvmgt_host_init,
 	.host_exit = kvmgt_host_exit,
-	.dma_pin_guest_page = kvmgt_dma_pin_guest_page,
 };
 
 struct intel_gvt_host intel_gvt_host = {
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
index 26c1ee690f7e..3be602a3f764 100644
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ b/drivers/gpu/drm/i915/gvt/mpt.h
@@ -71,19 +71,4 @@ static inline void intel_gvt_hypervisor_host_exit(struct device *dev, void *gvt)
 	intel_gvt_host.mpt->host_exit(dev, gvt);
 }
 
-/**
- * intel_gvt_hypervisor_dma_pin_guest_page - pin guest dma buf
- * @vgpu: a vGPU
- * @dma_addr: guest dma addr
- *
- * Returns:
- * 0 on success, negative error code if failed.
- */
-static inline int
-intel_gvt_hypervisor_dma_pin_guest_page(struct intel_vgpu *vgpu,
-					dma_addr_t dma_addr)
-{
-	return intel_gvt_host.mpt->dma_pin_guest_page(vgpu, dma_addr);
-}
-
 #endif /* _GVT_MPT_H_ */
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [PATCH 21/21] drm/i915/gvt: remove struct intel_gvt_mpt
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (19 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 20/21] drm/i915/gvt: devirtualize dma_pin_guest_page Christoph Hellwig
@ 2021-07-21 15:53 ` Christoph Hellwig
  2021-07-22  9:45 ` refactor the i915 GVT support Zhenyu Wang
  2021-07-22 10:49 ` Wang, Zhi A
  22 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-21 15:53 UTC (permalink / raw)
  To: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Just call the initializion and exit functions directly and remove
this abstraction entirely.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/gpu/drm/i915/gvt/gvt.c       | 11 ++++-
 drivers/gpu/drm/i915/gvt/gvt.h       | 12 ++---
 drivers/gpu/drm/i915/gvt/hypercall.h | 50 -------------------
 drivers/gpu/drm/i915/gvt/kvmgt.c     | 39 ++-------------
 drivers/gpu/drm/i915/gvt/mpt.h       | 74 ----------------------------
 5 files changed, 17 insertions(+), 169 deletions(-)
 delete mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h
 delete mode 100644 drivers/gpu/drm/i915/gvt/mpt.h

diff --git a/drivers/gpu/drm/i915/gvt/gvt.c b/drivers/gpu/drm/i915/gvt/gvt.c
index 2b59e79f5e3b..c55c542e3c75 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.c
+++ b/drivers/gpu/drm/i915/gvt/gvt.c
@@ -135,7 +135,8 @@ static void intel_gvt_clean_device(struct drm_i915_private *i915)
 	if (drm_WARN_ON(&i915->drm, !gvt))
 		return;
 
-	intel_gvt_hypervisor_host_exit(i915->drm.dev, gvt);
+	mdev_unregister_device(i915->drm.dev);
+	intel_gvt_cleanup_vgpu_type_groups(gvt);
 	intel_gvt_destroy_idle_vgpu(gvt->idle_vgpu);
 	intel_gvt_clean_vgpu_types(gvt);
 
@@ -235,13 +236,19 @@ static int intel_gvt_init_device(struct drm_i915_private *i915)
 
 	intel_gvt_debugfs_init(gvt);
 
-	ret = intel_gvt_hypervisor_host_init(i915->drm.dev, gvt);
+	ret = intel_gvt_init_vgpu_type_groups(gvt);
 	if (ret)
 		goto out_destroy_idle_vgpu;
 
+	ret = mdev_register_device(i915->drm.dev, &intel_vgpu_ops);
+	if (ret)
+		goto out_cleanup_vgpu_type_groups;
+
 	gvt_dbg_core("gvt device initialization is done\n");
 	return 0;
 
+out_cleanup_vgpu_type_groups:
+	intel_gvt_cleanup_vgpu_type_groups(gvt);
 out_destroy_idle_vgpu:
 	intel_gvt_destroy_idle_vgpu(gvt->idle_vgpu);
 	intel_gvt_debugfs_clean(gvt);
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index a8fc381f558e..a1fba27462b7 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -40,7 +40,6 @@
 #include "i915_drv.h"
 
 #include "debug.h"
-#include "hypercall.h"
 #include "mmio.h"
 #include "reg.h"
 #include "interrupt.h"
@@ -58,12 +57,6 @@
 
 #define GVT_MAX_VGPU 8
 
-struct intel_gvt_host {
-	const struct intel_gvt_mpt *mpt;
-};
-
-extern struct intel_gvt_host intel_gvt_host;
-
 /* Describe per-platform limitations. */
 struct intel_gvt_device_info {
 	u32 max_support_vgpus;
@@ -772,9 +765,12 @@ int intel_gvt_dma_map_guest_page(struct intel_vgpu *vgpu, unsigned long gfn,
 void intel_gvt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
 		dma_addr_t dma_addr);
 
+int intel_gvt_init_vgpu_type_groups(struct intel_gvt *gvt);
+void intel_gvt_cleanup_vgpu_type_groups(struct intel_gvt *gvt);
+
 #include "trace.h"
-#include "mpt.h"
 
+extern const struct mdev_parent_ops intel_vgpu_ops;
 extern const struct i915_virtual_gpu_ops intel_gvt_vgpu_ops;
 
 #endif
diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h b/drivers/gpu/drm/i915/gvt/hypercall.h
deleted file mode 100644
index d49437aeabac..000000000000
--- a/drivers/gpu/drm/i915/gvt/hypercall.h
+++ /dev/null
@@ -1,50 +0,0 @@
-/*
- * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- *
- * Authors:
- *    Eddie Dong <eddie.dong@intel.com>
- *    Dexuan Cui
- *    Jike Song <jike.song@intel.com>
- *
- * Contributors:
- *    Zhi Wang <zhi.a.wang@intel.com>
- *
- */
-
-#ifndef _GVT_HYPERCALL_H_
-#define _GVT_HYPERCALL_H_
-
-#include <linux/types.h>
-
-struct device;
-struct intel_vgpu;
-
-/*
- * Specific GVT-g MPT modules function collections. Currently GVT-g supports
- * both Xen and KVM by providing dedicated hypervisor-related MPT modules.
- */
-struct intel_gvt_mpt {
-	int (*host_init)(struct device *dev, void *gvt);
-	void (*host_exit)(struct device *dev, void *gvt);
-};
-
-#endif /* _GVT_HYPERCALL_H_ */
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 5f10ed5eccde..5c72dcf205fb 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -153,7 +153,7 @@ static struct attribute_group *gvt_vgpu_type_groups[] = {
 	[0 ... NR_MAX_INTEL_VGPU_TYPES - 1] = NULL,
 };
 
-static int intel_gvt_init_vgpu_type_groups(struct intel_gvt *gvt)
+int intel_gvt_init_vgpu_type_groups(struct intel_gvt *gvt)
 {
 	int i, j;
 	struct intel_vgpu_type *type;
@@ -182,7 +182,7 @@ static int intel_gvt_init_vgpu_type_groups(struct intel_gvt *gvt)
 	return -ENOMEM;
 }
 
-static void intel_gvt_cleanup_vgpu_type_groups(struct intel_gvt *gvt)
+void intel_gvt_cleanup_vgpu_type_groups(struct intel_gvt *gvt)
 {
 	int i;
 	struct attribute_group *group;
@@ -1646,8 +1646,9 @@ static const struct attribute_group *intel_vgpu_groups[] = {
 	NULL,
 };
 
-static struct mdev_parent_ops intel_vgpu_ops = {
+const struct mdev_parent_ops intel_vgpu_ops = {
 	.mdev_attr_groups       = intel_vgpu_groups,
+	.supported_type_groups	= gvt_vgpu_type_groups,
 	.create			= intel_vgpu_create,
 	.remove			= intel_vgpu_remove,
 
@@ -1660,29 +1661,6 @@ static struct mdev_parent_ops intel_vgpu_ops = {
 	.ioctl			= intel_vgpu_ioctl,
 };
 
-static int kvmgt_host_init(struct device *dev, void *gvt)
-{
-	int ret;
-
-	ret = intel_gvt_init_vgpu_type_groups((struct intel_gvt *)gvt);
-	if (ret)
-		return ret;
-
-	intel_vgpu_ops.supported_type_groups = gvt_vgpu_type_groups;
-
-	ret = mdev_register_device(dev, &intel_vgpu_ops);
-	if (ret)
-		intel_gvt_cleanup_vgpu_type_groups((struct intel_gvt *)gvt);
-
-	return ret;
-}
-
-static void kvmgt_host_exit(struct device *dev, void *gvt)
-{
-	mdev_unregister_device(dev);
-	intel_gvt_cleanup_vgpu_type_groups((struct intel_gvt *)gvt);
-}
-
 int intel_gvt_page_track_add(struct intel_vgpu *info, u64 gfn)
 {
 	struct kvm *kvm = info->kvm;
@@ -1945,14 +1923,5 @@ void intel_gvt_dma_unmap_guest_page(struct intel_vgpu *vgpu,
 	mutex_unlock(&vgpu->cache_lock);
 }
 
-static const struct intel_gvt_mpt kvmgt_mpt = {
-	.host_init = kvmgt_host_init,
-	.host_exit = kvmgt_host_exit,
-};
-
-struct intel_gvt_host intel_gvt_host = {
-	.mpt		= &kvmgt_mpt,
-};
-
 MODULE_LICENSE("GPL and additional rights");
 MODULE_AUTHOR("Intel Corporation");
diff --git a/drivers/gpu/drm/i915/gvt/mpt.h b/drivers/gpu/drm/i915/gvt/mpt.h
deleted file mode 100644
index 3be602a3f764..000000000000
--- a/drivers/gpu/drm/i915/gvt/mpt.h
+++ /dev/null
@@ -1,74 +0,0 @@
-/*
- * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- *
- * Authors:
- *    Eddie Dong <eddie.dong@intel.com>
- *    Dexuan Cui
- *    Jike Song <jike.song@intel.com>
- *
- * Contributors:
- *    Zhi Wang <zhi.a.wang@intel.com>
- *
- */
-
-#ifndef _GVT_MPT_H_
-#define _GVT_MPT_H_
-
-#include "gvt.h"
-
-/**
- * DOC: Hypervisor Service APIs for GVT-g Core Logic
- *
- * This is the glue layer between specific hypervisor MPT modules and GVT-g core
- * logic. Each kind of hypervisor MPT module provides a collection of function
- * callbacks and will be attached to GVT host when the driver is loading.
- * GVT-g core logic will call these APIs to request specific services from
- * hypervisor.
- */
-
-/**
- * intel_gvt_hypervisor_host_init - init GVT-g host side
- *
- * Returns:
- * Zero on success, negative error code if failed
- */
-static inline int intel_gvt_hypervisor_host_init(struct device *dev, void *gvt)
-{
-	if (!intel_gvt_host.mpt->host_init)
-		return -ENODEV;
-
-	return intel_gvt_host.mpt->host_init(dev, gvt);
-}
-
-/**
- * intel_gvt_hypervisor_host_exit - exit GVT-g host side
- */
-static inline void intel_gvt_hypervisor_host_exit(struct device *dev, void *gvt)
-{
-	/* optional to provide */
-	if (!intel_gvt_host.mpt->host_exit)
-		return;
-
-	intel_gvt_host.mpt->host_exit(dev, gvt);
-}
-
-#endif /* _GVT_MPT_H_ */
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (20 preceding siblings ...)
  2021-07-21 15:53 ` [PATCH 21/21] drm/i915/gvt: remove struct intel_gvt_mpt Christoph Hellwig
@ 2021-07-22  9:45 ` Zhenyu Wang
  2021-07-22 12:55   ` Christoph Hellwig
  2021-07-22 10:49 ` Wang, Zhi A
  22 siblings, 1 reply; 60+ messages in thread
From: Zhenyu Wang @ 2021-07-22  9:45 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang,
	Zhi Wang, intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 2889 bytes --]

On 2021.07.21 17:53:34 +0200, Christoph Hellwig wrote:
> Hi all,
> 
> the GVT code in the i915 is a bit of a mess right now due to strange
> abstractions and lots of indirect calls.  This series refactors various
> bits to clean that up.  The main user visible change is that almost all
> of the GVT code moves out of the main i915 driver and into the kvmgt
> module.
>

The reason we isolated hypervisor specific code from core vgpu
emulation is to make multiple hypervisor support possible. Yes, we do
have Xen support but never got way into upstream...And we also have
third party hypervisors which leverage gvt function through current
hypervisor interface.

Sorry I may not have more time to check in details for now, but some
of them look fine to me. I'll review more after vacation or let Zhi check details.

Thanks!

> Tested on my Thinkpad with a Kaby Lake CPU and integrated graphics.
> 
> Git tree:
> 
>     git://git.infradead.org/users/hch/misc.git i915-gvt
> 
> Gitweb:
> 
>     http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/i915-gvt
> 
> Diffstat:
>  b/drivers/gpu/drm/i915/Kconfig                         |   31 
>  b/drivers/gpu/drm/i915/Makefile                        |   30 
>  b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c |    4 
>  b/drivers/gpu/drm/i915/gvt/cfg_space.c                 |   89 --
>  b/drivers/gpu/drm/i915/gvt/cmd_parser.c                |    4 
>  b/drivers/gpu/drm/i915/gvt/dmabuf.c                    |   36 
>  b/drivers/gpu/drm/i915/gvt/execlist.c                  |   12 
>  b/drivers/gpu/drm/i915/gvt/gtt.c                       |   55 -
>  b/drivers/gpu/drm/i915/gvt/gvt.c                       |  100 --
>  b/drivers/gpu/drm/i915/gvt/gvt.h                       |  132 ++-
>  b/drivers/gpu/drm/i915/gvt/interrupt.c                 |   38 -
>  b/drivers/gpu/drm/i915/gvt/kvmgt.c                     |  634 ++++-------------
>  b/drivers/gpu/drm/i915/gvt/mmio.c                      |    4 
>  b/drivers/gpu/drm/i915/gvt/opregion.c                  |  148 ---
>  b/drivers/gpu/drm/i915/gvt/page_track.c                |    8 
>  b/drivers/gpu/drm/i915/gvt/scheduler.c                 |   37 
>  b/drivers/gpu/drm/i915/gvt/trace.h                     |    2 
>  b/drivers/gpu/drm/i915/gvt/vgpu.c                      |   22 
>  b/drivers/gpu/drm/i915/i915_drv.h                      |    7 
>  b/drivers/gpu/drm/i915/i915_params.c                   |    2 
>  b/drivers/gpu/drm/i915/intel_gvt.c                     |   64 +
>  b/drivers/gpu/drm/i915/intel_gvt.h                     |    4 
>  drivers/gpu/drm/i915/gvt/Makefile                      |    9 
>  drivers/gpu/drm/i915/gvt/hypercall.h                   |   82 --
>  drivers/gpu/drm/i915/gvt/mpt.h                         |  400 ----------
>  25 files changed, 541 insertions(+), 1413 deletions(-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* RE: refactor the i915 GVT support
  2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
                   ` (21 preceding siblings ...)
  2021-07-22  9:45 ` refactor the i915 GVT support Zhenyu Wang
@ 2021-07-22 10:49 ` Wang, Zhi A
  2021-07-22 11:26   ` Gerd Hoffmann
  2021-07-22 13:16   ` Greg KH
  22 siblings, 2 replies; 60+ messages in thread
From: Wang, Zhi A @ 2021-07-22 10:49 UTC (permalink / raw)
  To: Christoph Hellwig, Jani Nikula, Joonas Lahtinen, Vivi, Rodrigo,
	Zhenyu Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Hi Christoph:

Thanks so much for the patches and the testing.

The abstraction between i915 and KVMGT module is for our customers who can easily port GVT-g into their own hypervisors. As you can see, all the hypervisor related functions were put in "hypervisor" module. The GVT-g module talks with the "hypervisor" module through the MPT layer. The customers just need to focus on their "hypervisor" module, implement and attach their own "hypervisor" modules to the MPT layer without touching the GVT-g core logic, which reduce great efforts during the porting as the core logic of GVT-g in i915 is pretty vendor-specific and customers aren't motivated to touch it unless they have to.

The boundary between GVT-g core logic and "hypervisor" module also helps a lot on narrowing down the problems when supporting our customers. According to our experience during the support, the less a customer touches the core logic, the less problem will be introduced.

We get many customers who are using commercial hypervisors like QNX or their private hypervisors in many areas in the industry. An reference implementation of "Xen hypervisor" module to demonstrate our customer how to port GVT-g to a type-1 hypervisor instead of KVM can be found here:
https://github.com/intel/gvt-linux/blob/topic/gvt-xengt/drivers/gpu/drm/i915/gvt/xengt.c

But it's hard for some customers to contribute their own "hypervisor" module to the upstream Linux kernel. I am thinking what would be a better solution here? The MPT layer in the kernel helps a lot for customers, but only one open-source "hypervisor" module is there in the kernel. That can confuse people which don't know the story. One thing I was thinking is to put a document about the background and more description in the MPT headers. So it won't confuse more people. 

Feel free to drop more comments. 😊

Thanks,
Zhi.

-----Original Message-----
From: Christoph Hellwig <hch@lst.de> 
Sent: Wednesday, July 21, 2021 6:54 PM
To: Jani Nikula <jani.nikula@linux.intel.com>; Joonas Lahtinen <joonas.lahtinen@linux.intel.com>; Vivi, Rodrigo <rodrigo.vivi@intel.com>; Zhenyu Wang <zhenyuw@linux.intel.com>; Wang, Zhi A <zhi.a.wang@intel.com>
Cc: intel-gfx@lists.freedesktop.org; intel-gvt-dev@lists.freedesktop.org; dri-devel@lists.freedesktop.org; linux-kernel@vger.kernel.org
Subject: refactor the i915 GVT support

Hi all,

the GVT code in the i915 is a bit of a mess right now due to strange abstractions and lots of indirect calls.  This series refactors various bits to clean that up.  The main user visible change is that almost all of the GVT code moves out of the main i915 driver and into the kvmgt module.

Tested on my Thinkpad with a Kaby Lake CPU and integrated graphics.

Git tree:

    git://git.infradead.org/users/hch/misc.git i915-gvt

Gitweb:

    http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/i915-gvt

Diffstat:
 b/drivers/gpu/drm/i915/Kconfig                         |   31 
 b/drivers/gpu/drm/i915/Makefile                        |   30 
 b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c |    4 
 b/drivers/gpu/drm/i915/gvt/cfg_space.c                 |   89 --
 b/drivers/gpu/drm/i915/gvt/cmd_parser.c                |    4 
 b/drivers/gpu/drm/i915/gvt/dmabuf.c                    |   36 
 b/drivers/gpu/drm/i915/gvt/execlist.c                  |   12 
 b/drivers/gpu/drm/i915/gvt/gtt.c                       |   55 -
 b/drivers/gpu/drm/i915/gvt/gvt.c                       |  100 --
 b/drivers/gpu/drm/i915/gvt/gvt.h                       |  132 ++-
 b/drivers/gpu/drm/i915/gvt/interrupt.c                 |   38 -
 b/drivers/gpu/drm/i915/gvt/kvmgt.c                     |  634 ++++-------------
 b/drivers/gpu/drm/i915/gvt/mmio.c                      |    4 
 b/drivers/gpu/drm/i915/gvt/opregion.c                  |  148 ---
 b/drivers/gpu/drm/i915/gvt/page_track.c                |    8 
 b/drivers/gpu/drm/i915/gvt/scheduler.c                 |   37 
 b/drivers/gpu/drm/i915/gvt/trace.h                     |    2 
 b/drivers/gpu/drm/i915/gvt/vgpu.c                      |   22 
 b/drivers/gpu/drm/i915/i915_drv.h                      |    7 
 b/drivers/gpu/drm/i915/i915_params.c                   |    2 
 b/drivers/gpu/drm/i915/intel_gvt.c                     |   64 +
 b/drivers/gpu/drm/i915/intel_gvt.h                     |    4 
 drivers/gpu/drm/i915/gvt/Makefile                      |    9 
 drivers/gpu/drm/i915/gvt/hypercall.h                   |   82 --
 drivers/gpu/drm/i915/gvt/mpt.h                         |  400 ----------
 25 files changed, 541 insertions(+), 1413 deletions(-)

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-22 10:49 ` Wang, Zhi A
@ 2021-07-22 11:26   ` Gerd Hoffmann
  2021-07-27 12:12     ` Jason Gunthorpe
  2021-07-22 13:16   ` Greg KH
  1 sibling, 1 reply; 60+ messages in thread
From: Gerd Hoffmann @ 2021-07-22 11:26 UTC (permalink / raw)
  To: Wang, Zhi A
  Cc: Christoph Hellwig, Jani Nikula, Joonas Lahtinen, Vivi, Rodrigo,
	Zhenyu Wang, intel-gfx, intel-gvt-dev, linux-kernel, dri-devel

  Hi,

> https://github.com/intel/gvt-linux/blob/topic/gvt-xengt/drivers/gpu/drm/i915/gvt/xengt.c

> But it's hard for some customers to contribute their own "hypervisor"
> module to the upstream Linux kernel. I am thinking what would be a
> better solution here? The MPT layer in the kernel helps a lot for
> customers, but only one open-source "hypervisor" module is there in
> the kernel. That can confuse people which don't know the story.  One
> thing I was thinking is to put a document about the background and
> more description in the MPT headers. So it won't confuse more people. 

Getting the xengt module linked above merged into mainline
would also nicely explain why there are hypervisor modules.

take care,
  Gerd


^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-22  9:45 ` refactor the i915 GVT support Zhenyu Wang
@ 2021-07-22 12:55   ` Christoph Hellwig
  0 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-22 12:55 UTC (permalink / raw)
  To: Zhenyu Wang
  Cc: Christoph Hellwig, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	Zhi Wang, intel-gfx, intel-gvt-dev, dri-devel, linux-kernel,
	Greg Kroah-Hartman

On Thu, Jul 22, 2021 at 05:45:16PM +0800, Zhenyu Wang wrote:
> The reason we isolated hypervisor specific code from core vgpu
> emulation is to make multiple hypervisor support possible. Yes, we do
> have Xen support but never got way into upstream...And we also have
> third party hypervisors which leverage gvt function through current
> hypervisor interface.

Out of tree stuff simply does not matter for the upstream kernel,
especially when it creates such a huge burden.

As a background:  I started this refactoring as a lot of the code did
not make much sense when reviewing the mdev integration in preparation
to switching them to the new mdev interface from Jason.

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-22 10:49 ` Wang, Zhi A
  2021-07-22 11:26   ` Gerd Hoffmann
@ 2021-07-22 13:16   ` Greg KH
  1 sibling, 0 replies; 60+ messages in thread
From: Greg KH @ 2021-07-22 13:16 UTC (permalink / raw)
  To: Wang, Zhi A
  Cc: Christoph Hellwig, Jani Nikula, Joonas Lahtinen, Vivi, Rodrigo,
	Zhenyu Wang, intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

On Thu, Jul 22, 2021 at 10:49:47AM +0000, Wang, Zhi A wrote:
> But it's hard for some customers to contribute their own "hypervisor"
> module to the upstream Linux kernel.

What prevents them from doing this?  We will take any code that meets
our standards, what format is this external code in?

> I am thinking what would be a
> better solution here? The MPT layer in the kernel helps a lot for
> customers, but only one open-source "hypervisor" module is there in
> the kernel. That can confuse people which don't know the story. One
> thing I was thinking is to put a document about the background and
> more description in the MPT headers. So it won't confuse more people. 

If no one is using it in the kernel, it needs to be removed.  No
abstractions should be added that are not required by the in-tree code.

So this series should be accepted, _or_ the external code needs to be
submitted for inclusion.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-22 11:26   ` Gerd Hoffmann
@ 2021-07-27 12:12     ` Jason Gunthorpe
  2021-07-28 13:38       ` Wang, Zhi A
  0 siblings, 1 reply; 60+ messages in thread
From: Jason Gunthorpe @ 2021-07-27 12:12 UTC (permalink / raw)
  To: Gerd Hoffmann
  Cc: Wang, Zhi A, Christoph Hellwig, Jani Nikula, Joonas Lahtinen,
	Vivi, Rodrigo, Zhenyu Wang, intel-gfx, intel-gvt-dev,
	linux-kernel, dri-devel

On Thu, Jul 22, 2021 at 01:26:36PM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> > https://github.com/intel/gvt-linux/blob/topic/gvt-xengt/drivers/gpu/drm/i915/gvt/xengt.c
> 
> > But it's hard for some customers to contribute their own "hypervisor"
> > module to the upstream Linux kernel. I am thinking what would be a
> > better solution here? The MPT layer in the kernel helps a lot for
> > customers, but only one open-source "hypervisor" module is there in
> > the kernel. That can confuse people which don't know the story.  One
> > thing I was thinking is to put a document about the background and
> > more description in the MPT headers. So it won't confuse more people. 
> 
> Getting the xengt module linked above merged into mainline
> would also nicely explain why there are hypervisor modules.

It would also be nice to explain why a GPU driver needs a hypervisor
specific shim like this in the first place.

        enum hypervisor_type type;
        int (*host_init)(struct device *dev, void *gvt, const void *ops);
        void (*host_exit)(struct device *dev, void *gvt);
        int (*attach_vgpu)(void *vgpu, unsigned long *handle);
        void (*detach_vgpu)(void *vgpu);

Doesn't vfio provide all this generically with notifiers?

        int (*inject_msi)(unsigned long handle, u32 addr, u16 data);

Isn't this one just an eventfd?

        unsigned long (*from_virt_to_mfn)(void *p);
        int (*read_gpa)(unsigned long handle, unsigned long gpa, void *buf,
                        unsigned long len);
        int (*write_gpa)(unsigned long handle, unsigned long gpa, void *buf,
                         unsigned long len);
        unsigned long (*gfn_to_mfn)(unsigned long handle, unsigned long gfn);

        int (*dma_map_guest_page)(unsigned long handle, unsigned long gfn,
                                  unsigned long size, dma_addr_t *dma_addr);
        void (*dma_unmap_guest_page)(unsigned long handle, dma_addr_t dma_addr);

        int (*dma_pin_guest_page)(unsigned long handle, dma_addr_t dma_addr);

        int (*map_gfn_to_mfn)(unsigned long handle, unsigned long gfn,
                              unsigned long mfn, unsigned int nr, bool map);
        bool (*is_valid_gfn)(unsigned long handle, unsigned long gfn);

Shouldn't the vfio page SW IOMMU do all of this generically?

        int (*enable_page_track)(unsigned long handle, u64 gfn);
        int (*disable_page_track)(unsigned long handle, u64 gfn);
        int (*set_trap_area)(unsigned long handle, u64 start, u64 end,
                             bool map);
        int (*set_opregion)(void *vgpu);
        int (*set_edid)(void *vgpu, int port_num);

edid depends on hypervisor??

        int (*get_vfio_device)(void *vgpu);
        void (*put_vfio_device)(void *vgpu);

Jason

^ permalink raw reply	[flat|nested] 60+ messages in thread

* RE: refactor the i915 GVT support
  2021-07-27 12:12     ` Jason Gunthorpe
@ 2021-07-28 13:38       ` Wang, Zhi A
  2021-07-28 13:43         ` Greg KH
  2021-07-28 17:59         ` Jason Gunthorpe
  0 siblings, 2 replies; 60+ messages in thread
From: Wang, Zhi A @ 2021-07-28 13:38 UTC (permalink / raw)
  To: Jason Gunthorpe, Gerd Hoffmann, Greg KH
  Cc: Christoph Hellwig, Jani Nikula, Joonas Lahtinen, Vivi, Rodrigo,
	Zhenyu Wang, intel-gfx, intel-gvt-dev, linux-kernel, dri-devel

Hi Jason:

I guess those APIs you were talking about are KVM-only. For other hypervisors, e.g. Xen, ARCN cannot use the APIs you mentioned. Not sure if you have already noticed that VFIO is KVM-only right now.

GVT-g is designed for many hypervisors not only KVM. In the design, we implemented an abstraction layer for different hypervisors. You can check the link in the previous email which has an example of how the MPT module "xengt" supports GVT-g running under Xen. 
For example, injecting a msi in VFIO/KVM is via playing with eventfd. But in Xen, we need to issue a hypercall from Dom0. So does others, like querying mappings between GFN and HFN. Some GPU related emulation logic might be implemented differently under different hypervisors because different hypervisors might provide not exact the APIs we want. That's the reason why they get a callback in the MPT (yet not perfect.)  

As you can see, to survive from this situation, we have to rely on an abstraction layer so that we can prevent introducing coding blocks like in the core logic:

If (in_hypervisor_xen)
	Issue hypercalls
else if (in_hypervisor_kvm)
	Play with eventfds.
Else if (in_hypervisor_other)
	xxxx

Thus some of the APIs have to be wrapped in the MPT module in GVT-g design.

Sadly, not all customers are motivated or allowed to get their hypervisor-specific modules into the kernel. We have a customer who runs GVT-g with their private hypervisor. In this case, they don't want to get their "xxxgt" MPT module into upstream since their hypervisor has been in the kernel yet. Also, we have customers who ported the GVT-g to QNX which is another widely used commercial hypervisor in the industry. They can't get the "qnxgt" MPT module into upstream since it's not allowed.

We do understand the situation and try to figure out a solution that can fulfill expectations from different people in the community and also customers. 

According to Greg KH's comments, we are collecting the requirements of MPT modules from other open-source hypervisors in the kernel, e.g. ACRN, to see if they can get another MPT module into the kernel, which will show an example that how the MPT abstraction can benefit. Also, we are evaluating the impact on our customers if we have to remove MPT abstraction in the kernel because there is only one MPT module. 

Thanks so much for the comments.

Thanks,
Zhi.

-----Original Message-----
From: Jason Gunthorpe <jgg@nvidia.com> 
Sent: Tuesday, July 27, 2021 3:12 PM
To: Gerd Hoffmann <kraxel@redhat.com>
Cc: Wang, Zhi A <zhi.a.wang@intel.com>; Christoph Hellwig <hch@lst.de>; Jani Nikula <jani.nikula@linux.intel.com>; Joonas Lahtinen <joonas.lahtinen@linux.intel.com>; Vivi, Rodrigo <rodrigo.vivi@intel.com>; Zhenyu Wang <zhenyuw@linux.intel.com>; intel-gfx@lists.freedesktop.org; intel-gvt-dev@lists.freedesktop.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org
Subject: Re: refactor the i915 GVT support

On Thu, Jul 22, 2021 at 01:26:36PM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> > https://github.com/intel/gvt-linux/blob/topic/gvt-xengt/drivers/gpu/
> > drm/i915/gvt/xengt.c
> 
> > But it's hard for some customers to contribute their own "hypervisor"
> > module to the upstream Linux kernel. I am thinking what would be a 
> > better solution here? The MPT layer in the kernel helps a lot for 
> > customers, but only one open-source "hypervisor" module is there in 
> > the kernel. That can confuse people which don't know the story.  One 
> > thing I was thinking is to put a document about the background and 
> > more description in the MPT headers. So it won't confuse more people.
> 
> Getting the xengt module linked above merged into mainline would also 
> nicely explain why there are hypervisor modules.

It would also be nice to explain why a GPU driver needs a hypervisor specific shim like this in the first place.

        enum hypervisor_type type;
        int (*host_init)(struct device *dev, void *gvt, const void *ops);
        void (*host_exit)(struct device *dev, void *gvt);
        int (*attach_vgpu)(void *vgpu, unsigned long *handle);
        void (*detach_vgpu)(void *vgpu);

Doesn't vfio provide all this generically with notifiers?

        int (*inject_msi)(unsigned long handle, u32 addr, u16 data);

Isn't this one just an eventfd?

        unsigned long (*from_virt_to_mfn)(void *p);
        int (*read_gpa)(unsigned long handle, unsigned long gpa, void *buf,
                        unsigned long len);
        int (*write_gpa)(unsigned long handle, unsigned long gpa, void *buf,
                         unsigned long len);
        unsigned long (*gfn_to_mfn)(unsigned long handle, unsigned long gfn);

        int (*dma_map_guest_page)(unsigned long handle, unsigned long gfn,
                                  unsigned long size, dma_addr_t *dma_addr);
        void (*dma_unmap_guest_page)(unsigned long handle, dma_addr_t dma_addr);

        int (*dma_pin_guest_page)(unsigned long handle, dma_addr_t dma_addr);

        int (*map_gfn_to_mfn)(unsigned long handle, unsigned long gfn,
                              unsigned long mfn, unsigned int nr, bool map);
        bool (*is_valid_gfn)(unsigned long handle, unsigned long gfn);

Shouldn't the vfio page SW IOMMU do all of this generically?

        int (*enable_page_track)(unsigned long handle, u64 gfn);
        int (*disable_page_track)(unsigned long handle, u64 gfn);
        int (*set_trap_area)(unsigned long handle, u64 start, u64 end,
                             bool map);
        int (*set_opregion)(void *vgpu);
        int (*set_edid)(void *vgpu, int port_num);

edid depends on hypervisor??

        int (*get_vfio_device)(void *vgpu);
        void (*put_vfio_device)(void *vgpu);

Jason

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-28 13:38       ` Wang, Zhi A
@ 2021-07-28 13:43         ` Greg KH
  2021-07-28 17:59         ` Jason Gunthorpe
  1 sibling, 0 replies; 60+ messages in thread
From: Greg KH @ 2021-07-28 13:43 UTC (permalink / raw)
  To: Wang, Zhi A
  Cc: Jason Gunthorpe, Gerd Hoffmann, Christoph Hellwig, Jani Nikula,
	Joonas Lahtinen, Vivi, Rodrigo, Zhenyu Wang, intel-gfx,
	intel-gvt-dev, linux-kernel, dri-devel

A: http://en.wikipedia.org/wiki/Top_post
Q: Were do I find info about this thing called top-posting?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

A: No.
Q: Should I include quotations after my reply?

http://daringfireball.net/2007/07/on_top

On Wed, Jul 28, 2021 at 01:38:58PM +0000, Wang, Zhi A wrote:
> Hi Jason:
> 
> I guess those APIs you were talking about are KVM-only. For other hypervisors, e.g. Xen, ARCN cannot use the APIs you mentioned. Not sure if you have already noticed that VFIO is KVM-only right now.

Please wrap your lines properly :(

> GVT-g is designed for many hypervisors not only KVM. In the design, we implemented an abstraction layer for different hypervisors. You can check the link in the previous email which has an example of how the MPT module "xengt" supports GVT-g running under Xen. 
> For example, injecting a msi in VFIO/KVM is via playing with eventfd. But in Xen, we need to issue a hypercall from Dom0. So does others, like querying mappings between GFN and HFN. Some GPU related emulation logic might be implemented differently under different hypervisors because different hypervisors might provide not exact the APIs we want. That's the reason why they get a callback in the MPT (yet not perfect.)  
> 
> As you can see, to survive from this situation, we have to rely on an abstraction layer so that we can prevent introducing coding blocks like in the core logic:
> 
> If (in_hypervisor_xen)
> 	Issue hypercalls
> else if (in_hypervisor_kvm)
> 	Play with eventfds.
> Else if (in_hypervisor_other)
> 	xxxx

That's horrid, and slow, please do this properly.

> Thus some of the APIs have to be wrapped in the MPT module in GVT-g design.
> 
> Sadly, not all customers are motivated or allowed to get their hypervisor-specific modules into the kernel. We have a customer who runs GVT-g with their private hypervisor. In this case, they don't want to get their "xxxgt" MPT module into upstream since their hypervisor has been in the kernel yet. Also, we have customers who ported the GVT-g to QNX which is another widely used commercial hypervisor in the industry. They can't get the "qnxgt" MPT module into upstream since it's not allowed.

Why is it not allowed?

> We do understand the situation and try to figure out a solution that can fulfill expectations from different people in the community and also customers. 
> 
> According to Greg KH's comments, we are collecting the requirements of MPT modules from other open-source hypervisors in the kernel, e.g. ACRN, to see if they can get another MPT module into the kernel, which will show an example that how the MPT abstraction can benefit. Also, we are evaluating the impact on our customers if we have to remove MPT abstraction in the kernel because there is only one MPT module. 

Until that happens, can we please just remove the unneeded layer here as
no one is using it?  Then, when you have a real user for this type of
middle-layer, you can add it back?  We have no need for it now.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-28 13:38       ` Wang, Zhi A
  2021-07-28 13:43         ` Greg KH
@ 2021-07-28 17:59         ` Jason Gunthorpe
  2021-07-29  7:20           ` Christoph Hellwig
  2021-07-29  8:19           ` Wang, Zhi A
  1 sibling, 2 replies; 60+ messages in thread
From: Jason Gunthorpe @ 2021-07-28 17:59 UTC (permalink / raw)
  To: Wang, Zhi A
  Cc: Gerd Hoffmann, Greg KH, Christoph Hellwig, Jani Nikula,
	Joonas Lahtinen, Vivi, Rodrigo, Zhenyu Wang, intel-gfx,
	intel-gvt-dev, linux-kernel, dri-devel

On Wed, Jul 28, 2021 at 01:38:58PM +0000, Wang, Zhi A wrote:

> I guess those APIs you were talking about are KVM-only. For other
> hypervisors, e.g. Xen, ARCN cannot use the APIs you mentioned. Not
> sure if you have already noticed that VFIO is KVM-only right now.

There is very little hard connection between VFIO and KVM, so no, I
don't think that is completely true.

In an event, an in-tree version of other hypervisor support for GVT
needs to go through enabling VFIO support so that the existing API
multiplexers we have can be used properly, not adding a shim layer
trying to recreate VFIO inside a GPU driver.

> GVT-g is designed for many hypervisors not only KVM. In the design,
> we implemented an abstraction layer for different hypervisors. You
> can check the link in the previous email which has an example of how
> the MPT module "xengt" supports GVT-g running under Xen.  For
> example, injecting a msi in VFIO/KVM is via playing with
> eventfd. But in Xen, we need to issue a hypercall from Dom0. 

This is obviously bad design, Xen should plug into the standardized
eventfd scheme as well and trigger its hypercall this way. Then it can
integrate with the existing VFIO interrupt abstraction infrastructure.

> others, like querying mappings between GFN and HFN. 

This should be done through VFIO containers, there is nothing KVM
specific there.

> As you can see, to survive from this situation, we have to rely on
> an abstraction layer so that we can prevent introducing coding
> blocks like in the core logic:

No, you have to fix the abstractions we already have to support the
matrix of things you care about. If this can't be done then maybe we
can add new abstractions, but abstractions like this absoultely should
not be done inside drivers.

Jason

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-28 17:59         ` Jason Gunthorpe
@ 2021-07-29  7:20           ` Christoph Hellwig
  2021-07-29  7:30             ` [Intel-gfx] " Daniel Vetter
  2021-08-03  9:43             ` Zhenyu Wang
  2021-07-29  8:19           ` Wang, Zhi A
  1 sibling, 2 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-07-29  7:20 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Wang, Zhi A, Gerd Hoffmann, Greg KH, Christoph Hellwig,
	Jani Nikula, Joonas Lahtinen, Vivi, Rodrigo, Zhenyu Wang,
	intel-gfx, intel-gvt-dev, linux-kernel, dri-devel

On Wed, Jul 28, 2021 at 02:59:25PM -0300, Jason Gunthorpe wrote:
> On Wed, Jul 28, 2021 at 01:38:58PM +0000, Wang, Zhi A wrote:
> 
> > I guess those APIs you were talking about are KVM-only. For other
> > hypervisors, e.g. Xen, ARCN cannot use the APIs you mentioned. Not
> > sure if you have already noticed that VFIO is KVM-only right now.
> 
> There is very little hard connection between VFIO and KVM, so no, I
> don't think that is completely true.

The only connection is the SET_KVM notifier as far as I can tell.
Which is used by a total of two drivers, including i915/gvt.  That
being said gvt does not only use vfio, but also does quite a few
direct cals to KVM.

> In an event, an in-tree version of other hypervisor support for GVT
> needs to go through enabling VFIO support so that the existing API
> multiplexers we have can be used properly, not adding a shim layer
> trying to recreate VFIO inside a GPU driver.

Yes.  And if we go back to actually looking at the series a lot of
it just removes entirely pointless indirect calls that go to generic
code and not even the kvm code, or questionable data structure designs.
If we were to support another upstream hypervisor we'd just need to
union a few fields in struct intel_gpu and maybe introduce a few
methods.  Preferably in a way that avoids expensive indirect calls
in the fast path.

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Intel-gfx] refactor the i915 GVT support
  2021-07-29  7:20           ` Christoph Hellwig
@ 2021-07-29  7:30             ` Daniel Vetter
  2021-08-03  9:43             ` Zhenyu Wang
  1 sibling, 0 replies; 60+ messages in thread
From: Daniel Vetter @ 2021-07-29  7:30 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jason Gunthorpe, dri-devel, Greg KH, intel-gfx, linux-kernel,
	Gerd Hoffmann, intel-gvt-dev

On Thu, Jul 29, 2021 at 09:20:22AM +0200, Christoph Hellwig wrote:
> On Wed, Jul 28, 2021 at 02:59:25PM -0300, Jason Gunthorpe wrote:
> > On Wed, Jul 28, 2021 at 01:38:58PM +0000, Wang, Zhi A wrote:
> > 
> > > I guess those APIs you were talking about are KVM-only. For other
> > > hypervisors, e.g. Xen, ARCN cannot use the APIs you mentioned. Not
> > > sure if you have already noticed that VFIO is KVM-only right now.
> > 
> > There is very little hard connection between VFIO and KVM, so no, I
> > don't think that is completely true.
> 
> The only connection is the SET_KVM notifier as far as I can tell.
> Which is used by a total of two drivers, including i915/gvt.  That
> being said gvt does not only use vfio, but also does quite a few
> direct cals to KVM.
> 
> > In an event, an in-tree version of other hypervisor support for GVT
> > needs to go through enabling VFIO support so that the existing API
> > multiplexers we have can be used properly, not adding a shim layer
> > trying to recreate VFIO inside a GPU driver.
> 
> Yes.  And if we go back to actually looking at the series a lot of
> it just removes entirely pointless indirect calls that go to generic
> code and not even the kvm code, or questionable data structure designs.
> If we were to support another upstream hypervisor we'd just need to
> union a few fields in struct intel_gpu and maybe introduce a few
> methods.  Preferably in a way that avoids expensive indirect calls
> in the fast path.

fwiw I concur with the direction of this series. gvt landed 5 years ago,
that should have been plenty of time to merge at least one of the other
backends that float around. If it didn't happen in 5 years it aint
suddenly happening in the next few, and the abstraction layer should be
sunset.

Also yes structuring it more as a helper layer with some
unions/subclassing than full blown backend abstractor layer would be a
good idea too I guess (it usually is the right thing to do).
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-28 17:59         ` Jason Gunthorpe
  2021-07-29  7:20           ` Christoph Hellwig
@ 2021-07-29  8:19           ` Wang, Zhi A
  1 sibling, 0 replies; 60+ messages in thread
From: Wang, Zhi A @ 2021-07-29  8:19 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Gerd Hoffmann, Greg KH, Christoph Hellwig, Jani Nikula,
	Joonas Lahtinen, Vivi, Rodrigo, Zhenyu Wang, intel-gfx,
	intel-gvt-dev, linux-kernel, dri-devel

On 7/28/2021 8:59 PM, Jason Gunthorpe wrote:
> On Wed, Jul 28, 2021 at 01:38:58PM +0000, Wang, Zhi A wrote:
>
>> I guess those APIs you were talking about are KVM-only. For other
>> hypervisors, e.g. Xen, ARCN cannot use the APIs you mentioned. Not
>> sure if you have already noticed that VFIO is KVM-only right now.
> There is very little hard connection between VFIO and KVM, so no, I
> don't think that is completely true.
>
> In an event, an in-tree version of other hypervisor support for GVT
> needs to go through enabling VFIO support so that the existing API
> multiplexers we have can be used properly, not adding a shim layer
> trying to recreate VFIO inside a GPU driver.

We were delivering the presentation of GVT-g in Xen summit 2018 and we 
were thinking and talking about supporting VFIO in Xen during the 
presentation (the video can be found from Youtube). But we didn't see 
any motiviation from the Xen community to adopt it.

If people take a look into the code in QEMU, in the PCI-passthrough 
part, Xen is actually not using VFIO even nowadays. We would be glad to 
see someone can influence on that part, especically making all the 
in-kernel hypervisor to use VFIO in PCI-passthrough and supporting mdev. 
That would be a huge benefit for all the users.

>> GVT-g is designed for many hypervisors not only KVM. In the design,
>> we implemented an abstraction layer for different hypervisors. You
>> can check the link in the previous email which has an example of how
>> the MPT module "xengt" supports GVT-g running under Xen.  For
>> example, injecting a msi in VFIO/KVM is via playing with
>> eventfd. But in Xen, we need to issue a hypercall from Dom0.
> This is obviously bad design, Xen should plug into the standardized
> eventfd scheme as well and trigger its hypercall this way. Then it can
> integrate with the existing VFIO interrupt abstraction infrastructure.
>
>> others, like querying mappings between GFN and HFN.
> This should be done through VFIO containers, there is nothing KVM
> specific there.
>
>> As you can see, to survive from this situation, we have to rely on
>> an abstraction layer so that we can prevent introducing coding
>> blocks like in the core logic:
> No, you have to fix the abstractions we already have to support the
> matrix of things you care about. If this can't be done then maybe we
> can add new abstractions, but abstractions like this absoultely should
> not be done inside drivers.
>
> Jason

That's a good point and we were actually thinking about this before and 
I believe that's the correct direction. But just like the situation 
mentioned above, it would be nice if people can really put a great 
influence on all in-kernel hypervisors to use VFIO which can really 
benefit all the users.

For now, we are just going to take christoph's patches.

Zhi


^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [PATCH 01/21] drm/i915/gvt: integrate into the main Makefile
  2021-07-21 15:53 ` [PATCH 01/21] drm/i915/gvt: integrate into the main Makefile Christoph Hellwig
@ 2021-08-03  9:27   ` Zhenyu Wang
  0 siblings, 0 replies; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-03  9:27 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Zhenyu Wang,
	Zhi Wang, intel-gfx, intel-gvt-dev, linux-kernel, dri-devel

[-- Attachment #1: Type: text/plain, Size: 3186 bytes --]

On 2021.07.21 17:53:35 +0200, Christoph Hellwig wrote:
> Remove the separately included Makefile and just use the relative
> reference from the main i915 Makefile as for source files in other
> subdirectories.
>

We agreed to make gvt mostly self-contained in its own directory
before. Although I don't think we would change files much later, but
still need to get ack from i915 maintainers.

Thanks

> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  drivers/gpu/drm/i915/Makefile     | 29 ++++++++++++++++++++++++-----
>  drivers/gpu/drm/i915/gvt/Makefile |  9 ---------
>  drivers/gpu/drm/i915/gvt/trace.h  |  2 +-
>  3 files changed, 25 insertions(+), 15 deletions(-)
>  delete mode 100644 drivers/gpu/drm/i915/gvt/Makefile
> 
> diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
> index 4f22cac1c49b..2153f67705b8 100644
> --- a/drivers/gpu/drm/i915/Makefile
> +++ b/drivers/gpu/drm/i915/Makefile
> @@ -289,11 +289,30 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \
>  
>  # virtual gpu code
>  i915-y += i915_vgpu.o
> -
> -ifeq ($(CONFIG_DRM_I915_GVT),y)
> -i915-y += intel_gvt.o
> -include $(src)/gvt/Makefile
> -endif
> +i915-$(CONFIG_DRM_I915_GVT) += \
> +	intel_gvt.o \
> +	gvt/gvt.o \
> +	gvt/aperture_gm.o \
> +	gvt/handlers.o \
> +	gvt/vgpu.o \
> +	gvt/trace_points.o \
> +	gvt/firmware.o \
> +	gvt/interrupt.o \
> +	gvt/gtt.o \
> +	gvt/cfg_space.o \
> +	gvt/opregion.o \
> +	gvt/mmio.o \
> +	gvt/display.o \
> +	gvt/edid.o \
> +	gvt/execlist.o \
> +	gvt/scheduler.o \
> +	gvt/sched_policy.o \
> +	gvt/mmio_context.o \
> +	gvt/cmd_parser.o \
> +	gvt/debugfs.o \
> +	gvt/fb_decoder.o \
> +	gvt/dmabuf.o \
> +	gvt/page_track.o
>  
>  obj-$(CONFIG_DRM_I915) += i915.o
>  obj-$(CONFIG_DRM_I915_GVT_KVMGT) += gvt/kvmgt.o
> diff --git a/drivers/gpu/drm/i915/gvt/Makefile b/drivers/gpu/drm/i915/gvt/Makefile
> deleted file mode 100644
> index ea8324abc784..000000000000
> --- a/drivers/gpu/drm/i915/gvt/Makefile
> +++ /dev/null
> @@ -1,9 +0,0 @@
> -# SPDX-License-Identifier: GPL-2.0
> -GVT_DIR := gvt
> -GVT_SOURCE := gvt.o aperture_gm.o handlers.o vgpu.o trace_points.o firmware.o \
> -	interrupt.o gtt.o cfg_space.o opregion.o mmio.o display.o edid.o \
> -	execlist.o scheduler.o sched_policy.o mmio_context.o cmd_parser.o debugfs.o \
> -	fb_decoder.o dmabuf.o page_track.o
> -
> -ccflags-y				+= -I $(srctree)/$(src) -I $(srctree)/$(src)/$(GVT_DIR)/
> -i915-y					+= $(addprefix $(GVT_DIR)/, $(GVT_SOURCE))
> diff --git a/drivers/gpu/drm/i915/gvt/trace.h b/drivers/gpu/drm/i915/gvt/trace.h
> index 6d787750d279..348f57f8301d 100644
> --- a/drivers/gpu/drm/i915/gvt/trace.h
> +++ b/drivers/gpu/drm/i915/gvt/trace.h
> @@ -379,5 +379,5 @@ TRACE_EVENT(render_mmio,
>  #undef TRACE_INCLUDE_PATH
>  #define TRACE_INCLUDE_PATH .
>  #undef TRACE_INCLUDE_FILE
> -#define TRACE_INCLUDE_FILE trace
> +#define TRACE_INCLUDE_FILE gvt/trace
>  #include <trace/define_trace.h>
> -- 
> 2.30.2
> 
> _______________________________________________
> intel-gvt-dev mailing list
> intel-gvt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-07-29  7:20           ` Christoph Hellwig
  2021-07-29  7:30             ` [Intel-gfx] " Daniel Vetter
@ 2021-08-03  9:43             ` Zhenyu Wang
  2021-08-03 14:30               ` Jason Gunthorpe
  1 sibling, 1 reply; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-03  9:43 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jason Gunthorpe, dri-devel, Greg KH, intel-gfx, Joonas Lahtinen,
	linux-kernel, Jani Nikula, Zhenyu Wang, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Wang, Zhi A

[-- Attachment #1: Type: text/plain, Size: 1742 bytes --]

On 2021.07.29 09:20:22 +0200, Christoph Hellwig wrote:
> On Wed, Jul 28, 2021 at 02:59:25PM -0300, Jason Gunthorpe wrote:
> > On Wed, Jul 28, 2021 at 01:38:58PM +0000, Wang, Zhi A wrote:
> > 
> > > I guess those APIs you were talking about are KVM-only. For other
> > > hypervisors, e.g. Xen, ARCN cannot use the APIs you mentioned. Not
> > > sure if you have already noticed that VFIO is KVM-only right now.
> > 
> > There is very little hard connection between VFIO and KVM, so no, I
> > don't think that is completely true.
> 
> The only connection is the SET_KVM notifier as far as I can tell.
> Which is used by a total of two drivers, including i915/gvt.  That
> being said gvt does not only use vfio, but also does quite a few
> direct cals to KVM.

yeah, we mostly combined VFIO into hypervisor specific thing before,
e.g interface to set vgpu edid, etc. along with kvm specific calls.

> 
> > In an event, an in-tree version of other hypervisor support for GVT
> > needs to go through enabling VFIO support so that the existing API
> > multiplexers we have can be used properly, not adding a shim layer
> > trying to recreate VFIO inside a GPU driver.
> 
> Yes.  And if we go back to actually looking at the series a lot of
> it just removes entirely pointless indirect calls that go to generic
> code and not even the kvm code, or questionable data structure designs.
> If we were to support another upstream hypervisor we'd just need to
> union a few fields in struct intel_gpu and maybe introduce a few
> methods.  Preferably in a way that avoids expensive indirect calls
> in the fast path.

ok, agree on that.

Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Thanks a lot for this effort!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-03  9:43             ` Zhenyu Wang
@ 2021-08-03 14:30               ` Jason Gunthorpe
  2021-08-04  5:26                 ` Zhenyu Wang
  2021-08-04  5:40                 ` Christoph Hellwig
  0 siblings, 2 replies; 60+ messages in thread
From: Jason Gunthorpe @ 2021-08-03 14:30 UTC (permalink / raw)
  To: Zhenyu Wang
  Cc: Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Wang, Zhi A

On Tue, Aug 03, 2021 at 05:43:15PM +0800, Zhenyu Wang wrote:
> Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>
> 
> Thanks a lot for this effort!

Great, do we have a submission plan for this? how much does it clash
with my open_device/etc patch? ie does the whole thing have to go
through the vfio tree?

Thanks,
Jason

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-03 14:30               ` Jason Gunthorpe
@ 2021-08-04  5:26                 ` Zhenyu Wang
  2021-08-16 17:34                   ` Christoph Hellwig
  2021-08-04  5:40                 ` Christoph Hellwig
  1 sibling, 1 reply; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-04  5:26 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Wang, Zhi A

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]

On 2021.08.03 11:30:58 -0300, Jason Gunthorpe wrote:
> On Tue, Aug 03, 2021 at 05:43:15PM +0800, Zhenyu Wang wrote:
> > Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>
> > 
> > Thanks a lot for this effort!
> 
> Great, do we have a submission plan for this? how much does it clash
> with my open_device/etc patch? ie does the whole thing have to go
> through the vfio tree?
> 

I think Alex would determine when to merge open_device series, gvt part
can be through vfio tree without problem. For this refactor, I would first
merge for gvt staging to do more regression testing before sending through
i915 tree.

Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-03 14:30               ` Jason Gunthorpe
  2021-08-04  5:26                 ` Zhenyu Wang
@ 2021-08-04  5:40                 ` Christoph Hellwig
  1 sibling, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-08-04  5:40 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Zhenyu Wang, Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Wang, Zhi A

On Tue, Aug 03, 2021 at 11:30:58AM -0300, Jason Gunthorpe wrote:
> On Tue, Aug 03, 2021 at 05:43:15PM +0800, Zhenyu Wang wrote:
> > Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>
> > 
> > Thanks a lot for this effort!
> 
> Great, do we have a submission plan for this? how much does it clash
> with my open_device/etc patch? ie does the whole thing have to go
> through the vfio tree?

The two series merge cleanly and build fine.  So if we don't need
the actual conversion of the i915 gvt code to the new mdev registration
scheme this merge window there is no need to synchronize.

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [PATCH 04/21] drm/i915/gvt: move the gvt code into kvmgt.ko
  2021-07-21 15:53 ` [PATCH 04/21] drm/i915/gvt: move the gvt code into kvmgt.ko Christoph Hellwig
@ 2021-08-09 11:29   ` Joonas Lahtinen
  2021-08-09 14:29     ` Christoph Hellwig
  0 siblings, 1 reply; 60+ messages in thread
From: Joonas Lahtinen @ 2021-08-09 11:29 UTC (permalink / raw)
  To: Christoph Hellwig, Jani Nikula, Rodrigo Vivi, Zhenyu Wang, Zhi Wang
  Cc: intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

Quoting Christoph Hellwig (2021-07-21 18:53:38)
> Instead of having an option to build the gvt code into the main i915
> module, just move it into the kvmgt.ko module.  This only requires
> a new struct with three entries that the main i915 module needs to
> request before enabling VGPU passthrough operations.
> 
> This also conveniently streamlines the GVT initialization and avoids
> the need for the global device pointer.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Hi,

Thanks for putting the work into this. This conversion has been
requested for a long time. For clarity, should we call the module
i915_kvmgt?

How far would we be from dynamically modprobing/rmmoding the kvmgt
module in order to eliminate the enable_gvt parameter?

<SNIP>

> +
> +/*
> + * Exported here so that the exports only get created when GVT support is
> + * actually enabled.
> + */
> +EXPORT_SYMBOL_NS_GPL(i915_gem_object_alloc, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_object_create_shmem, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_object_init, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_object_ggtt_pin_ww, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_object_pin_map, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_object_set_to_cpu_domain, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(__i915_gem_object_flush_map, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(__i915_gem_object_set_pages, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_gtt_insert, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_prime_export, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_ww_ctx_init, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_ww_ctx_backoff, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_gem_ww_ctx_fini, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_ppgtt_create, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_request_add, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_request_create, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_request_wait, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_reserve_fence, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_unreserve_fence, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_vm_release, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(i915_vma_move_to_active, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(intel_context_create, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(intel_context_unpin, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(__intel_context_do_pin, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(intel_ring_begin, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(intel_runtime_pm_get, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(intel_runtime_pm_put_unchecked, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(intel_uncore_forcewake_for_reg, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(intel_uncore_forcewake_get, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(intel_uncore_forcewake_put, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(shmem_pin_map, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(shmem_unpin_map, I915_GVT);
> +EXPORT_SYMBOL_NS_GPL(__px_dma, I915_GVT);

This list is also a concern. At the least the double underscore
functions should be eliminated from being exported.

Zhi and Zhenyu, can we have some further patches to clean that up?

Regards, Joonas

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [PATCH 04/21] drm/i915/gvt: move the gvt code into kvmgt.ko
  2021-08-09 11:29   ` Joonas Lahtinen
@ 2021-08-09 14:29     ` Christoph Hellwig
  0 siblings, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-08-09 14:29 UTC (permalink / raw)
  To: Joonas Lahtinen
  Cc: Christoph Hellwig, Jani Nikula, Rodrigo Vivi, Zhenyu Wang,
	Zhi Wang, intel-gfx, intel-gvt-dev, dri-devel, linux-kernel

On Mon, Aug 09, 2021 at 02:29:39PM +0300, Joonas Lahtinen wrote:
> Thanks for putting the work into this. This conversion has been
> requested for a long time. For clarity, should we call the module
> i915_kvmgt?

If this was a new module that would be my preferance.  But the
kvmgt module already exists, so I thought it might be a good idea
to keep the name.

> How far would we be from dynamically modprobing/rmmoding the kvmgt
> module in order to eliminate the enable_gvt parameter?

Hmm.  I suspect it could be done fairly easily, but from what I see
it would still require to disable GUC at i915 module load time.

If you want I could look into that for an incremental patch, but
to unblock the vfio/mdev work I'd like to do that in a separate
stage.

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-04  5:26                 ` Zhenyu Wang
@ 2021-08-16 17:34                   ` Christoph Hellwig
  2021-08-17  1:08                     ` Zhenyu Wang
  0 siblings, 1 reply; 60+ messages in thread
From: Christoph Hellwig @ 2021-08-16 17:34 UTC (permalink / raw)
  To: Zhenyu Wang
  Cc: Jason Gunthorpe, Christoph Hellwig, dri-devel, Greg KH,
	intel-gfx, Joonas Lahtinen, linux-kernel, Jani Nikula,
	Gerd Hoffmann, Vivi, Rodrigo, intel-gvt-dev, Wang, Zhi A

On Wed, Aug 04, 2021 at 01:26:06PM +0800, Zhenyu Wang wrote:
> On 2021.08.03 11:30:58 -0300, Jason Gunthorpe wrote:
> > On Tue, Aug 03, 2021 at 05:43:15PM +0800, Zhenyu Wang wrote:
> > > Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>
> > > 
> > > Thanks a lot for this effort!
> > 
> > Great, do we have a submission plan for this? how much does it clash
> > with my open_device/etc patch? ie does the whole thing have to go
> > through the vfio tree?
> > 
> 
> I think Alex would determine when to merge open_device series, gvt part
> can be through vfio tree without problem. For this refactor, I would first
> merge for gvt staging to do more regression testing before sending through
> i915 tree.

Any updates on this?  I'd really hate to miss this merge window.

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-16 17:34                   ` Christoph Hellwig
@ 2021-08-17  1:08                     ` Zhenyu Wang
  2021-08-17  5:22                       ` Zhenyu Wang
  0 siblings, 1 reply; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-17  1:08 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jason Gunthorpe, dri-devel, Greg KH, intel-gfx, Joonas Lahtinen,
	linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi, Rodrigo,
	intel-gvt-dev, Wang, Zhi A, Jani Nikula

[-- Attachment #1: Type: text/plain, Size: 1117 bytes --]

On 2021.08.16 19:34:58 +0200, Christoph Hellwig wrote:
> On Wed, Aug 04, 2021 at 01:26:06PM +0800, Zhenyu Wang wrote:
> > On 2021.08.03 11:30:58 -0300, Jason Gunthorpe wrote:
> > > On Tue, Aug 03, 2021 at 05:43:15PM +0800, Zhenyu Wang wrote:
> > > > Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>
> > > > 
> > > > Thanks a lot for this effort!
> > > 
> > > Great, do we have a submission plan for this? how much does it clash
> > > with my open_device/etc patch? ie does the whole thing have to go
> > > through the vfio tree?
> > > 
> > 
> > I think Alex would determine when to merge open_device series, gvt part
> > can be through vfio tree without problem. For this refactor, I would first
> > merge for gvt staging to do more regression testing before sending through
> > i915 tree.
> 
> Any updates on this?  I'd really hate to miss this merge window.

I'm still waiting for our validation team's report on this. I'm afraid
it might be missing for next version as i915 merge window is mostly
till rc5...and for any change outside of gvt, it still needs to be
acked by i915 maintainers.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-17  1:08                     ` Zhenyu Wang
@ 2021-08-17  5:22                       ` Zhenyu Wang
  2021-08-19  8:29                         ` Zhenyu Wang
  0 siblings, 1 reply; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-17  5:22 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jason Gunthorpe, dri-devel, Greg KH, intel-gfx, Joonas Lahtinen,
	linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi, Rodrigo,
	intel-gvt-dev, Wang, Zhi A, Jani Nikula

[-- Attachment #1: Type: text/plain, Size: 2118 bytes --]

On 2021.08.17 09:08:55 +0800, Zhenyu Wang wrote:
> On 2021.08.16 19:34:58 +0200, Christoph Hellwig wrote:
> > On Wed, Aug 04, 2021 at 01:26:06PM +0800, Zhenyu Wang wrote:
> > > On 2021.08.03 11:30:58 -0300, Jason Gunthorpe wrote:
> > > > On Tue, Aug 03, 2021 at 05:43:15PM +0800, Zhenyu Wang wrote:
> > > > > Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>
> > > > > 
> > > > > Thanks a lot for this effort!
> > > > 
> > > > Great, do we have a submission plan for this? how much does it clash
> > > > with my open_device/etc patch? ie does the whole thing have to go
> > > > through the vfio tree?
> > > > 
> > > 
> > > I think Alex would determine when to merge open_device series, gvt part
> > > can be through vfio tree without problem. For this refactor, I would first
> > > merge for gvt staging to do more regression testing before sending through
> > > i915 tree.
> > 
> > Any updates on this?  I'd really hate to miss this merge window.
> 
> I'm still waiting for our validation team's report on this. I'm afraid
> it might be missing for next version as i915 merge window is mostly
> till rc5...and for any change outside of gvt, it still needs to be
> acked by i915 maintainers.

Looks our validation team did have problem against recent i915 change.
If you like to try, we have a gvt-staging branch on
https://github.com/intel/gvt-linux which is generated against drm-tip
with gvt changes for testing, currently it's broken.

One issue is with i915 export that intel_context_unpin has been
changed into static inline function. Another is that intel_gvt.c
should be part of i915 for gvt interface instead of depending on KVMGT
config.

But the problem I see is that after moving gvt device model (gvt/*.c
except kvmgt.c) into kvmgt module, we'll have issue with initial mmio
state which current gvt relies on, that is in design supposed to get
initial HW state before i915 driver has taken any operation.  Before
we can ensure that, I think we may only remove MPT part first but
still keep gvt device model as part of i915 with config. I'll try to
split that out.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-17  5:22                       ` Zhenyu Wang
@ 2021-08-19  8:29                         ` Zhenyu Wang
  2021-08-19 14:43                           ` Joonas Lahtinen
  2021-08-20 14:17                           ` Christoph Hellwig
  0 siblings, 2 replies; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-19  8:29 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jason Gunthorpe, dri-devel, Greg KH, intel-gfx, Joonas Lahtinen,
	linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi, Rodrigo,
	intel-gvt-dev, Wang, Zhi A, Jani Nikula

[-- Attachment #1: Type: text/plain, Size: 4715 bytes --]

On 2021.08.17 13:22:03 +0800, Zhenyu Wang wrote:
> > On 2021.08.16 19:34:58 +0200, Christoph Hellwig wrote:
> > > Any updates on this?  I'd really hate to miss this merge window.
> > 
> > I'm still waiting for our validation team's report on this. I'm afraid
> > it might be missing for next version as i915 merge window is mostly
> > till rc5...and for any change outside of gvt, it still needs to be
> > acked by i915 maintainers.
> 
> Looks our validation team did have problem against recent i915 change.
> If you like to try, we have a gvt-staging branch on
> https://github.com/intel/gvt-linux which is generated against drm-tip
> with gvt changes for testing, currently it's broken.
> 
> One issue is with i915 export that intel_context_unpin has been
> changed into static inline function. Another is that intel_gvt.c
> should be part of i915 for gvt interface instead of depending on KVMGT
> config.

I'm working on below patch to resolve this. But I met a weird issue in
case when building i915 as module and also kvmgt module, it caused
busy wait on request_module("kvmgt") when boot, it doesn't happen if
building i915 into kernel. I'm not sure what could be the reason?

> But the problem I see is that after moving gvt device model (gvt/*.c
> except kvmgt.c) into kvmgt module, we'll have issue with initial mmio
> state which current gvt relies on, that is in design supposed to get
> initial HW state before i915 driver has taken any operation.  Before
> we can ensure that, I think we may only remove MPT part first but
> still keep gvt device model as part of i915 with config. I'll try to
> split that out.

Sorry I misread the code that as we always request kvmgt module when
gvt init, so it should still apply original method that this isn't a
problem. Our current validation result has shown no regression as well.

---8<---
From 58ff84572f1a0f9d79ca1d7ec0cff5ecbe78d280 Mon Sep 17 00:00:00 2001
From: Zhenyu Wang <zhenyuw@linux.intel.com>
Date: Thu, 19 Aug 2021 16:36:33 +0800
Subject: [PATCH] TESTONLY:drm/i915/gvt: potential fix for refactor against
 current tip

---
 drivers/gpu/drm/i915/Makefile           | 4 +++-
 drivers/gpu/drm/i915/gt/intel_context.c | 5 +++++
 drivers/gpu/drm/i915/gt/intel_context.h | 3 ++-
 drivers/gpu/drm/i915/i915_trace.h       | 1 +
 4 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index c4f953837f72..2248574428a1 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -296,7 +296,9 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \
 
 # virtual gpu code
 i915-y += i915_vgpu.o
-i915-$(CONFIG_DRM_I915_GVT_KVMGT) += intel_gvt.o
+ifneq ($(CONFIG_DRM_I915_GVT_KVMGT),)
+i915-y += intel_gvt.o
+endif
 
 kvmgt-y += gvt/kvmgt.o \
 	gvt/gvt.o \
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 745e84c72c90..20e7522fed84 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -328,6 +328,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub)
 	intel_context_put(ce);
 }
 
+void intel_context_unpin(struct intel_context *ce)
+{
+	_intel_context_unpin(ce);
+}
+
 static void __intel_context_retire(struct i915_active *active)
 {
 	struct intel_context *ce = container_of(active, typeof(*ce), active);
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index c41098950746..f942cbf6300a 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -131,7 +131,7 @@ static inline void intel_context_sched_disable_unpin(struct intel_context *ce)
 	__intel_context_do_unpin(ce, 2);
 }
 
-static inline void intel_context_unpin(struct intel_context *ce)
+static inline void _intel_context_unpin(struct intel_context *ce)
 {
 	if (!ce->ops->sched_disable) {
 		__intel_context_do_unpin(ce, 1);
@@ -150,6 +150,7 @@ static inline void intel_context_unpin(struct intel_context *ce)
 		}
 	}
 }
+void intel_context_unpin(struct intel_context *ce);
 
 void intel_context_enter_engine(struct intel_context *ce);
 void intel_context_exit_engine(struct intel_context *ce);
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index 806ad688274b..2c6a8bcef7c1 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -17,6 +17,7 @@
 
 #undef TRACE_SYSTEM
 #define TRACE_SYSTEM i915
+#undef TRACE_INCLUDE_FILE
 #define TRACE_INCLUDE_FILE i915_trace
 
 /* watermark/fifo updates */
-- 
2.32.0
---8<---



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-19  8:29                         ` Zhenyu Wang
@ 2021-08-19 14:43                           ` Joonas Lahtinen
  2021-08-26  6:04                             ` Zhenyu Wang
  2021-08-20 14:17                           ` Christoph Hellwig
  1 sibling, 1 reply; 60+ messages in thread
From: Joonas Lahtinen @ 2021-08-19 14:43 UTC (permalink / raw)
  To: Christoph Hellwig, Zhenyu Wang
  Cc: Jason Gunthorpe, dri-devel, Greg KH, intel-gfx, linux-kernel,
	Jani Nikula, Gerd Hoffmann, Vivi, Rodrigo, intel-gvt-dev, Wang,
	Zhi A, Jani Nikula

Quoting Zhenyu Wang (2021-08-19 11:29:29)
> On 2021.08.17 13:22:03 +0800, Zhenyu Wang wrote:
> > > On 2021.08.16 19:34:58 +0200, Christoph Hellwig wrote:
> > > > Any updates on this?  I'd really hate to miss this merge window.
> > > 
> > > I'm still waiting for our validation team's report on this. I'm afraid
> > > it might be missing for next version as i915 merge window is mostly
> > > till rc5...and for any change outside of gvt, it still needs to be
> > > acked by i915 maintainers.
> > 
> > Looks our validation team did have problem against recent i915 change.
> > If you like to try, we have a gvt-staging branch on
> > https://github.com/intel/gvt-linux which is generated against drm-tip
> > with gvt changes for testing, currently it's broken.
> > 
> > One issue is with i915 export that intel_context_unpin has been
> > changed into static inline function. Another is that intel_gvt.c
> > should be part of i915 for gvt interface instead of depending on KVMGT
> > config.
> 
> I'm working on below patch to resolve this. But I met a weird issue in
> case when building i915 as module and also kvmgt module, it caused
> busy wait on request_module("kvmgt") when boot, it doesn't happen if
> building i915 into kernel. I'm not sure what could be the reason?
> 
> > But the problem I see is that after moving gvt device model (gvt/*.c
> > except kvmgt.c) into kvmgt module, we'll have issue with initial mmio
> > state which current gvt relies on, that is in design supposed to get
> > initial HW state before i915 driver has taken any operation.

As mentioned in some past discussions, I think it would be best rely on
golden MMIO located in /lib/firmware or elsewhere. This way we will better
isolate the guest system from host system updates/changes.

This should also hopefully allow enabling kvmgt module after i915 has
already loaded, as the initialization would not be conditional to
capture the MMIO.

Regards, Joonas

> > Before
> > we can ensure that, I think we may only remove MPT part first but
> > still keep gvt device model as part of i915 with config. I'll try to
> > split that out.
> 
> Sorry I misread the code that as we always request kvmgt module when
> gvt init, so it should still apply original method that this isn't a
> problem. Our current validation result has shown no regression as well.
> 
> ---8<---
> From 58ff84572f1a0f9d79ca1d7ec0cff5ecbe78d280 Mon Sep 17 00:00:00 2001
> From: Zhenyu Wang <zhenyuw@linux.intel.com>
> Date: Thu, 19 Aug 2021 16:36:33 +0800
> Subject: [PATCH] TESTONLY:drm/i915/gvt: potential fix for refactor against
>  current tip
> 
> ---
>  drivers/gpu/drm/i915/Makefile           | 4 +++-
>  drivers/gpu/drm/i915/gt/intel_context.c | 5 +++++
>  drivers/gpu/drm/i915/gt/intel_context.h | 3 ++-
>  drivers/gpu/drm/i915/i915_trace.h       | 1 +
>  4 files changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
> index c4f953837f72..2248574428a1 100644
> --- a/drivers/gpu/drm/i915/Makefile
> +++ b/drivers/gpu/drm/i915/Makefile
> @@ -296,7 +296,9 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \
>  
>  # virtual gpu code
>  i915-y += i915_vgpu.o
> -i915-$(CONFIG_DRM_I915_GVT_KVMGT) += intel_gvt.o
> +ifneq ($(CONFIG_DRM_I915_GVT_KVMGT),)
> +i915-y += intel_gvt.o
> +endif
>  
>  kvmgt-y += gvt/kvmgt.o \
>         gvt/gvt.o \
> diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
> index 745e84c72c90..20e7522fed84 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context.c
> +++ b/drivers/gpu/drm/i915/gt/intel_context.c
> @@ -328,6 +328,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub)
>         intel_context_put(ce);
>  }
>  
> +void intel_context_unpin(struct intel_context *ce)
> +{
> +       _intel_context_unpin(ce);
> +}
> +
>  static void __intel_context_retire(struct i915_active *active)
>  {
>         struct intel_context *ce = container_of(active, typeof(*ce), active);
> diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
> index c41098950746..f942cbf6300a 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context.h
> @@ -131,7 +131,7 @@ static inline void intel_context_sched_disable_unpin(struct intel_context *ce)
>         __intel_context_do_unpin(ce, 2);
>  }
>  
> -static inline void intel_context_unpin(struct intel_context *ce)
> +static inline void _intel_context_unpin(struct intel_context *ce)
>  {
>         if (!ce->ops->sched_disable) {
>                 __intel_context_do_unpin(ce, 1);
> @@ -150,6 +150,7 @@ static inline void intel_context_unpin(struct intel_context *ce)
>                 }
>         }
>  }
> +void intel_context_unpin(struct intel_context *ce);
>  
>  void intel_context_enter_engine(struct intel_context *ce);
>  void intel_context_exit_engine(struct intel_context *ce);
> diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
> index 806ad688274b..2c6a8bcef7c1 100644
> --- a/drivers/gpu/drm/i915/i915_trace.h
> +++ b/drivers/gpu/drm/i915/i915_trace.h
> @@ -17,6 +17,7 @@
>  
>  #undef TRACE_SYSTEM
>  #define TRACE_SYSTEM i915
> +#undef TRACE_INCLUDE_FILE
>  #define TRACE_INCLUDE_FILE i915_trace
>  
>  /* watermark/fifo updates */
> -- 
> 2.32.0
> ---8<---
> 
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-19  8:29                         ` Zhenyu Wang
  2021-08-19 14:43                           ` Joonas Lahtinen
@ 2021-08-20 14:17                           ` Christoph Hellwig
  2021-08-20 19:56                             ` Luis Chamberlain
  2021-08-26  6:08                             ` Zhenyu Wang
  1 sibling, 2 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-08-20 14:17 UTC (permalink / raw)
  To: Zhenyu Wang
  Cc: Christoph Hellwig, Jason Gunthorpe, dri-devel, Greg KH,
	intel-gfx, Joonas Lahtinen, linux-kernel, Jani Nikula,
	Gerd Hoffmann, Vivi, Rodrigo, intel-gvt-dev, Wang, Zhi A,
	Jani Nikula, Luis Chamberlain

On Thu, Aug 19, 2021 at 04:29:29PM +0800, Zhenyu Wang wrote:
> I'm working on below patch to resolve this. But I met a weird issue in
> case when building i915 as module and also kvmgt module, it caused
> busy wait on request_module("kvmgt") when boot, it doesn't happen if
> building i915 into kernel. I'm not sure what could be the reason?

Luis, do you know if there is a problem with a request_module from
a driver ->probe routine that is probably called by a module_init
function itself?

In the meantime I'll try to reproduce it locally, but I always had a
hard time getting useful results out of a modular i915, especially
when combined with module paramters. (no blame on i915, just the problem
with modules needed early on).

> 
> > But the problem I see is that after moving gvt device model (gvt/*.c
> > except kvmgt.c) into kvmgt module, we'll have issue with initial mmio
> > state which current gvt relies on, that is in design supposed to get
> > initial HW state before i915 driver has taken any operation.  Before
> > we can ensure that, I think we may only remove MPT part first but
> > still keep gvt device model as part of i915 with config. I'll try to
> > split that out.
> 
> Sorry I misread the code that as we always request kvmgt module when
> gvt init, so it should still apply original method that this isn't a
> problem. Our current validation result has shown no regression as well.

What does initial mmio state mean?  This is something new to me.  But
as you said in this mail unless I missed something very big it should
work the same as before.

> -static inline void intel_context_unpin(struct intel_context *ce)
> +static inline void _intel_context_unpin(struct intel_context *ce)
>  {
>  	if (!ce->ops->sched_disable) {
>  		__intel_context_do_unpin(ce, 1);
> @@ -150,6 +150,7 @@ static inline void intel_context_unpin(struct intel_context *ce)
>  		}
>  	}
>  }
> +void intel_context_unpin(struct intel_context *ce);

Looking at intel_context_unpin/_intel_context_unpin is there really
a need to have this inline to start with?  It don't see much the compiler
could optimize by inlining it.

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-20 14:17                           ` Christoph Hellwig
@ 2021-08-20 19:56                             ` Luis Chamberlain
  2021-08-26  6:12                               ` Zhenyu Wang
  2021-08-26  6:08                             ` Zhenyu Wang
  1 sibling, 1 reply; 60+ messages in thread
From: Luis Chamberlain @ 2021-08-20 19:56 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Zhenyu Wang, Jason Gunthorpe, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Wang, Zhi A, Jani Nikula

On Fri, Aug 20, 2021 at 04:17:24PM +0200, Christoph Hellwig wrote:
> On Thu, Aug 19, 2021 at 04:29:29PM +0800, Zhenyu Wang wrote:
> > I'm working on below patch to resolve this. But I met a weird issue in
> > case when building i915 as module and also kvmgt module, it caused
> > busy wait on request_module("kvmgt") when boot, it doesn't happen if
> > building i915 into kernel. I'm not sure what could be the reason?
> 
> Luis, do you know if there is a problem with a request_module from
> a driver ->probe routine that is probably called by a module_init
> function itself?

Generally no, but you can easily foot yourself in the feet by creating
cross dependencies and not dealing with them properly. I'd make sure
to keep module initialization as simple as possible, and run whatever
takes more time asynchronously, then use a state machine to allow
you to verify where you are in the initialization phase or query it
or wait for a completion with a timeout.

It seems the code in question is getting some spring cleaning, and its
unclear where the code is I can inspect. If there's a tree somewhere I
can take a peak I'd be happy to review possible oddities that may stick
out.

My goto model for these sorts of problems is to abstract the issue
*outside* of the driver in question and implement new selftests to
try to reproduce. This serves two purposes, 1) helps with testing
2) may allow you to see the problem more clearly.

  Luis

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-19 14:43                           ` Joonas Lahtinen
@ 2021-08-26  6:04                             ` Zhenyu Wang
  0 siblings, 0 replies; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-26  6:04 UTC (permalink / raw)
  To: Joonas Lahtinen
  Cc: Christoph Hellwig, Jason Gunthorpe, dri-devel, Greg KH,
	intel-gfx, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Wang, Zhi A, Jani Nikula

[-- Attachment #1: Type: text/plain, Size: 5943 bytes --]

On 2021.08.19 17:43:43 +0300, Joonas Lahtinen wrote:
> Quoting Zhenyu Wang (2021-08-19 11:29:29)
> > On 2021.08.17 13:22:03 +0800, Zhenyu Wang wrote:
> > > > On 2021.08.16 19:34:58 +0200, Christoph Hellwig wrote:
> > > > > Any updates on this?  I'd really hate to miss this merge window.
> > > > 
> > > > I'm still waiting for our validation team's report on this. I'm afraid
> > > > it might be missing for next version as i915 merge window is mostly
> > > > till rc5...and for any change outside of gvt, it still needs to be
> > > > acked by i915 maintainers.
> > > 
> > > Looks our validation team did have problem against recent i915 change.
> > > If you like to try, we have a gvt-staging branch on
> > > https://github.com/intel/gvt-linux which is generated against drm-tip
> > > with gvt changes for testing, currently it's broken.
> > > 
> > > One issue is with i915 export that intel_context_unpin has been
> > > changed into static inline function. Another is that intel_gvt.c
> > > should be part of i915 for gvt interface instead of depending on KVMGT
> > > config.
> > 
> > I'm working on below patch to resolve this. But I met a weird issue in
> > case when building i915 as module and also kvmgt module, it caused
> > busy wait on request_module("kvmgt") when boot, it doesn't happen if
> > building i915 into kernel. I'm not sure what could be the reason?
> > 
> > > But the problem I see is that after moving gvt device model (gvt/*.c
> > > except kvmgt.c) into kvmgt module, we'll have issue with initial mmio
> > > state which current gvt relies on, that is in design supposed to get
> > > initial HW state before i915 driver has taken any operation.
> 
> As mentioned in some past discussions, I think it would be best rely on
> golden MMIO located in /lib/firmware or elsewhere. This way we will better
> isolate the guest system from host system updates/changes.
> 
> This should also hopefully allow enabling kvmgt module after i915 has
> already loaded, as the initialization would not be conditional to
> capture the MMIO.
> 

I think the concern is that even for same GEN hw there could be many
variant platforms e.g APL with gen9, etc. To verify golden states for
them all might take too much effort...

> 
> > > Before
> > > we can ensure that, I think we may only remove MPT part first but
> > > still keep gvt device model as part of i915 with config. I'll try to
> > > split that out.
> > 
> > Sorry I misread the code that as we always request kvmgt module when
> > gvt init, so it should still apply original method that this isn't a
> > problem. Our current validation result has shown no regression as well.
> > 
> > ---8<---
> > From 58ff84572f1a0f9d79ca1d7ec0cff5ecbe78d280 Mon Sep 17 00:00:00 2001
> > From: Zhenyu Wang <zhenyuw@linux.intel.com>
> > Date: Thu, 19 Aug 2021 16:36:33 +0800
> > Subject: [PATCH] TESTONLY:drm/i915/gvt: potential fix for refactor against
> >  current tip
> > 
> > ---
> >  drivers/gpu/drm/i915/Makefile           | 4 +++-
> >  drivers/gpu/drm/i915/gt/intel_context.c | 5 +++++
> >  drivers/gpu/drm/i915/gt/intel_context.h | 3 ++-
> >  drivers/gpu/drm/i915/i915_trace.h       | 1 +
> >  4 files changed, 11 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
> > index c4f953837f72..2248574428a1 100644
> > --- a/drivers/gpu/drm/i915/Makefile
> > +++ b/drivers/gpu/drm/i915/Makefile
> > @@ -296,7 +296,9 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \
> >  
> >  # virtual gpu code
> >  i915-y += i915_vgpu.o
> > -i915-$(CONFIG_DRM_I915_GVT_KVMGT) += intel_gvt.o
> > +ifneq ($(CONFIG_DRM_I915_GVT_KVMGT),)
> > +i915-y += intel_gvt.o
> > +endif
> >  
> >  kvmgt-y += gvt/kvmgt.o \
> >         gvt/gvt.o \
> > diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
> > index 745e84c72c90..20e7522fed84 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_context.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_context.c
> > @@ -328,6 +328,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub)
> >         intel_context_put(ce);
> >  }
> >  
> > +void intel_context_unpin(struct intel_context *ce)
> > +{
> > +       _intel_context_unpin(ce);
> > +}
> > +
> >  static void __intel_context_retire(struct i915_active *active)
> >  {
> >         struct intel_context *ce = container_of(active, typeof(*ce), active);
> > diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
> > index c41098950746..f942cbf6300a 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_context.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_context.h
> > @@ -131,7 +131,7 @@ static inline void intel_context_sched_disable_unpin(struct intel_context *ce)
> >         __intel_context_do_unpin(ce, 2);
> >  }
> >  
> > -static inline void intel_context_unpin(struct intel_context *ce)
> > +static inline void _intel_context_unpin(struct intel_context *ce)
> >  {
> >         if (!ce->ops->sched_disable) {
> >                 __intel_context_do_unpin(ce, 1);
> > @@ -150,6 +150,7 @@ static inline void intel_context_unpin(struct intel_context *ce)
> >                 }
> >         }
> >  }
> > +void intel_context_unpin(struct intel_context *ce);
> >  
> >  void intel_context_enter_engine(struct intel_context *ce);
> >  void intel_context_exit_engine(struct intel_context *ce);
> > diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
> > index 806ad688274b..2c6a8bcef7c1 100644
> > --- a/drivers/gpu/drm/i915/i915_trace.h
> > +++ b/drivers/gpu/drm/i915/i915_trace.h
> > @@ -17,6 +17,7 @@
> >  
> >  #undef TRACE_SYSTEM
> >  #define TRACE_SYSTEM i915
> > +#undef TRACE_INCLUDE_FILE
> >  #define TRACE_INCLUDE_FILE i915_trace
> >  
> >  /* watermark/fifo updates */
> > -- 
> > 2.32.0
> > ---8<---
> > 
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-20 14:17                           ` Christoph Hellwig
  2021-08-20 19:56                             ` Luis Chamberlain
@ 2021-08-26  6:08                             ` Zhenyu Wang
  1 sibling, 0 replies; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-26  6:08 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jason Gunthorpe, dri-devel, Greg KH, intel-gfx, Joonas Lahtinen,
	linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi, Rodrigo,
	intel-gvt-dev, Wang, Zhi A, Jani Nikula, Luis Chamberlain

[-- Attachment #1: Type: text/plain, Size: 2517 bytes --]

On 2021.08.20 16:17:24 +0200, Christoph Hellwig wrote:
> On Thu, Aug 19, 2021 at 04:29:29PM +0800, Zhenyu Wang wrote:
> > I'm working on below patch to resolve this. But I met a weird issue in
> > case when building i915 as module and also kvmgt module, it caused
> > busy wait on request_module("kvmgt") when boot, it doesn't happen if
> > building i915 into kernel. I'm not sure what could be the reason?
> 
> Luis, do you know if there is a problem with a request_module from
> a driver ->probe routine that is probably called by a module_init
> function itself?
> 
> In the meantime I'll try to reproduce it locally, but I always had a
> hard time getting useful results out of a modular i915, especially
> when combined with module paramters. (no blame on i915, just the problem
> with modules needed early on).
> 
> > 
> > > But the problem I see is that after moving gvt device model (gvt/*.c
> > > except kvmgt.c) into kvmgt module, we'll have issue with initial mmio
> > > state which current gvt relies on, that is in design supposed to get
> > > initial HW state before i915 driver has taken any operation.  Before
> > > we can ensure that, I think we may only remove MPT part first but
> > > still keep gvt device model as part of i915 with config. I'll try to
> > > split that out.
> > 
> > Sorry I misread the code that as we always request kvmgt module when
> > gvt init, so it should still apply original method that this isn't a
> > problem. Our current validation result has shown no regression as well.
> 
> What does initial mmio state mean?  This is something new to me.  But
> as you said in this mail unless I missed something very big it should
> work the same as before.
>

It's gvt internal track for all gfx mmio state, and yes with your current
change it should still work as before.

> > -static inline void intel_context_unpin(struct intel_context *ce)
> > +static inline void _intel_context_unpin(struct intel_context *ce)
> >  {
> >  	if (!ce->ops->sched_disable) {
> >  		__intel_context_do_unpin(ce, 1);
> > @@ -150,6 +150,7 @@ static inline void intel_context_unpin(struct intel_context *ce)
> >  		}
> >  	}
> >  }
> > +void intel_context_unpin(struct intel_context *ce);
> 
> Looking at intel_context_unpin/_intel_context_unpin is there really
> a need to have this inline to start with?  It don't see much the compiler
> could optimize by inlining it.

I'll send patch to i915 for this, and get more comments there.

thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-20 19:56                             ` Luis Chamberlain
@ 2021-08-26  6:12                               ` Zhenyu Wang
  2021-09-28  7:41                                 ` Wang, Zhi A
  0 siblings, 1 reply; 60+ messages in thread
From: Zhenyu Wang @ 2021-08-26  6:12 UTC (permalink / raw)
  To: Luis Chamberlain
  Cc: Christoph Hellwig, Jason Gunthorpe, dri-devel, Greg KH,
	intel-gfx, Joonas Lahtinen, linux-kernel, Jani Nikula,
	Gerd Hoffmann, Vivi, Rodrigo, intel-gvt-dev, Wang, Zhi A,
	Jani Nikula

[-- Attachment #1: Type: text/plain, Size: 1814 bytes --]

On 2021.08.20 12:56:34 -0700, Luis Chamberlain wrote:
> On Fri, Aug 20, 2021 at 04:17:24PM +0200, Christoph Hellwig wrote:
> > On Thu, Aug 19, 2021 at 04:29:29PM +0800, Zhenyu Wang wrote:
> > > I'm working on below patch to resolve this. But I met a weird issue in
> > > case when building i915 as module and also kvmgt module, it caused
> > > busy wait on request_module("kvmgt") when boot, it doesn't happen if
> > > building i915 into kernel. I'm not sure what could be the reason?
> > 
> > Luis, do you know if there is a problem with a request_module from
> > a driver ->probe routine that is probably called by a module_init
> > function itself?
> 
> Generally no, but you can easily foot yourself in the feet by creating
> cross dependencies and not dealing with them properly. I'd make sure
> to keep module initialization as simple as possible, and run whatever
> takes more time asynchronously, then use a state machine to allow
> you to verify where you are in the initialization phase or query it
> or wait for a completion with a timeout.
> 
> It seems the code in question is getting some spring cleaning, and its
> unclear where the code is I can inspect. If there's a tree somewhere I
> can take a peak I'd be happy to review possible oddities that may stick
> out.

I tried to put current patches under test here: https://github.com/intel/gvt-linux/tree/gvt-staging
The issue can be produced with CONFIG_DRM_I915=m and CONFIG_DRM_I915_GVT_KVMGT=m.

> 
> My goto model for these sorts of problems is to abstract the issue
> *outside* of the driver in question and implement new selftests to
> try to reproduce. This serves two purposes, 1) helps with testing
> 2) may allow you to see the problem more clearly.
> 

I'll see if can abstract that.

Thanks, Luis.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-08-26  6:12                               ` Zhenyu Wang
@ 2021-09-28  7:41                                 ` Wang, Zhi A
  2021-09-28 14:00                                   ` Luis Chamberlain
  0 siblings, 1 reply; 60+ messages in thread
From: Wang, Zhi A @ 2021-09-28  7:41 UTC (permalink / raw)
  To: Zhenyu Wang, Luis Chamberlain
  Cc: Christoph Hellwig, Jason Gunthorpe, dri-devel, Greg KH,
	intel-gfx, Joonas Lahtinen, linux-kernel, Jani Nikula,
	Gerd Hoffmann, Vivi, Rodrigo, intel-gvt-dev, Nikula, Jani

Hey guys:

After some investigation, I found the root cause this problem ("i915" 
module loading will be stuck with Christoph's refactor patches), which 
can be reproduced by building both i915 and kvmgt as kernel module and 
the loading i915.

The root cause is: in Linux kernel loading, before a kernel module 
loading is finished, its symbols can not be reached by other module when 
resolving the symbols (even they can be found in /proc/kallsyms). 
Because the status of the kernel module is MODULE_STATE_COMING and 
resolve_symbol() from another kernel module will check this and return a 
-EBUSY.

In this case, before i915 loading is finished, the requested module 
"kvmgt" cannot reach the symbols in module i915. Thus it kept waiting 
and left message like below in the dmesg:

[  644.152021] kvmgt: gave up waiting for init of module i915.
[  644.152039] kvmgt: Unknown symbol i915_gem_object_set_to_cpu_domain 
(err -16)
[  674.871409] kvmgt: gave up waiting for init of module i915.
[  674.871427] kvmgt: Unknown symbol intel_ring_begin (err -16)
[  705.590586] kvmgt: gave up waiting for init of module i915.
[  705.590604] kvmgt: Unknown symbol i915_vma_move_to_active (err -16)
[  736.310230] kvmgt: gave up waiting for init of module i915.
[  736.310248] kvmgt: Unknown symbol shmem_unpin_map (err -16)
...

The error message is from execution path below:

kernel/module.c:

[i915 module loading] -> 
request_module("kvmgt")->[modprobe]->init_module("kvmgt")->load_module()->simplify_symbols()->resolve_symbol_wait():

static const struct kernel_symbol *
resolve_symbol_wait(struct module *mod,
             const struct load_info *info,
             const char *name)
{
     const struct kernel_symbol *ksym;
     char owner[MODULE_NAME_LEN];

     if (wait_event_interruptible_timeout(module_wq,
             !IS_ERR(ksym = resolve_symbol(mod, info, name, owner))
             || PTR_ERR(ksym) != -EBUSY,
                          30 * HZ) <= 0) {
         pr_warn("%s: gave up waiting for init of module %s.\n",
             mod->name, owner);

}

code: 
https://github.com/intel/gvt-linux/blob/bd950a66c7919d7121d2530f30984351534a96dc/kernel/module.c#L1452

In resolve_symbol_wait(), it calls resolve_symbol() to resolve the 
symbols in "i915". In resolve_symbol() -> ref_module() -> 
strong_try_module_get(), it will check the status of the module which 
owns the symbol.

static inline int strong_try_module_get(struct module *mod)
{
     BUG_ON(mod && mod->state == MODULE_STATE_UNFORMED);
     if (mod && mod->state == MODULE_STATE_COMING)
         return -EBUSY;
     if (try_module_get(mod))
         return 0;
     else
         return -ENOENT;
}

code:https://github.com/intel/gvt-linux/blob/bd950a66c7919d7121d2530f30984351534a96dc/kernel/module.c#L318

But unfortunately, this execution path begins in i915 module loading, at 
this time, the status of kernel module "i915" is MODULE_STATE_COMING 
until loading of "kvmgt" is finished. Thus a -EBUSY is always returned 
when kernel is trying to resolve symbols for "kvmgt".

This patch below might need re-work:

Author: Christoph Hellwig <hch@lst.de>
Date:   Wed Jul 21 17:53:38 2021 +0200

     drm/i915/gvt: move the gvt code into kvmgt.ko

     Instead of having an option to build the gvt code into the main i915
     module, just move it into the kvmgt.ko module.  This only requires
     a new struct with three entries that the main i915 module needs to
     request before enabling VGPU passthrough operations.

     This also conveniently streamlines the GVT initialization and avoids
     the need for the global device pointer.

     Signed-off-by: Christoph Hellwig <hch@lst.de>
     Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
     Link: 
http://patchwork.freedesktop.org/patch/msgid/20210721155355.173183-5-hch@lst.de
     Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>

On 8/26/21 6:12 AM, Zhenyu Wang wrote:
> On 2021.08.20 12:56:34 -0700, Luis Chamberlain wrote:
>> On Fri, Aug 20, 2021 at 04:17:24PM +0200, Christoph Hellwig wrote:
>>> On Thu, Aug 19, 2021 at 04:29:29PM +0800, Zhenyu Wang wrote:
>>>> I'm working on below patch to resolve this. But I met a weird issue in
>>>> case when building i915 as module and also kvmgt module, it caused
>>>> busy wait on request_module("kvmgt") when boot, it doesn't happen if
>>>> building i915 into kernel. I'm not sure what could be the reason?
>>> Luis, do you know if there is a problem with a request_module from
>>> a driver ->probe routine that is probably called by a module_init
>>> function itself?
>> Generally no, but you can easily foot yourself in the feet by creating
>> cross dependencies and not dealing with them properly. I'd make sure
>> to keep module initialization as simple as possible, and run whatever
>> takes more time asynchronously, then use a state machine to allow
>> you to verify where you are in the initialization phase or query it
>> or wait for a completion with a timeout.
>>
>> It seems the code in question is getting some spring cleaning, and its
>> unclear where the code is I can inspect. If there's a tree somewhere I
>> can take a peak I'd be happy to review possible oddities that may stick
>> out.
> I tried to put current patches under test here: https://github.com/intel/gvt-linux/tree/gvt-staging
> The issue can be produced with CONFIG_DRM_I915=m and CONFIG_DRM_I915_GVT_KVMGT=m.
>
>> My goto model for these sorts of problems is to abstract the issue
>> *outside* of the driver in question and implement new selftests to
>> try to reproduce. This serves two purposes, 1) helps with testing
>> 2) may allow you to see the problem more clearly.
>>
> I'll see if can abstract that.
>
> Thanks, Luis.



^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-09-28  7:41                                 ` Wang, Zhi A
@ 2021-09-28 14:00                                   ` Luis Chamberlain
  2021-09-28 14:35                                     ` Wang, Zhi A
  0 siblings, 1 reply; 60+ messages in thread
From: Luis Chamberlain @ 2021-09-28 14:00 UTC (permalink / raw)
  To: Wang, Zhi A, Jessica Yu, Lucas De Marchi
  Cc: Zhenyu Wang, Christoph Hellwig, Jason Gunthorpe, dri-devel,
	Greg KH, intel-gfx, Joonas Lahtinen, linux-kernel, Jani Nikula,
	Gerd Hoffmann, Vivi, Rodrigo, intel-gvt-dev, Nikula, Jani

On Tue, Sep 28, 2021 at 07:41:00AM +0000, Wang, Zhi A wrote:
> Hey guys:
> 
> After some investigation, I found the root cause this problem ("i915" 
> module loading will be stuck with Christoph's refactor patches), which 
> can be reproduced by building both i915 and kvmgt as kernel module and 
> the loading i915.

Thanks for looking into this!

> The root cause is: in Linux kernel loading, before a kernel module 
> loading is finished, its symbols can not be reached by other module when 
> resolving the symbols (even they can be found in /proc/kallsyms). 
> Because the status of the kernel module is MODULE_STATE_COMING and 
> resolve_symbol() from another kernel module will check this and return a 
> -EBUSY.

Well, it would seem that way but...

> In this case, before i915 loading is finished, the requested module 
> "kvmgt" cannot reach the symbols in module i915. Thus it kept waiting 
> and left message like below in the dmesg:
> 
> [  644.152021] kvmgt: gave up waiting for init of module i915.
> [  644.152039] kvmgt: Unknown symbol i915_gem_object_set_to_cpu_domain 
> (err -16)
> [  674.871409] kvmgt: gave up waiting for init of module i915.
> [  674.871427] kvmgt: Unknown symbol intel_ring_begin (err -16)
> [  705.590586] kvmgt: gave up waiting for init of module i915.
> [  705.590604] kvmgt: Unknown symbol i915_vma_move_to_active (err -16)
> [  736.310230] kvmgt: gave up waiting for init of module i915.
> [  736.310248] kvmgt: Unknown symbol shmem_unpin_map (err -16)
> ...
> 
> The error message is from execution path below:
> 
> kernel/module.c:
> 
> [i915 module loading] -> 
> request_module("kvmgt")->[modprobe]->init_module("kvmgt")->load_module()->simplify_symbols()->resolve_symbol_wait():
> 
> static const struct kernel_symbol *
> resolve_symbol_wait(struct module *mod,
>              const struct load_info *info,
>              const char *name)
> {
>      const struct kernel_symbol *ksym;
>      char owner[MODULE_NAME_LEN];
> 
>      if (wait_event_interruptible_timeout(module_wq,
>              !IS_ERR(ksym = resolve_symbol(mod, info, name, owner))
>              || PTR_ERR(ksym) != -EBUSY,
>                           30 * HZ) <= 0) {
>          pr_warn("%s: gave up waiting for init of module %s.\n",
>              mod->name, owner);
> 
> }

Commit 9bea7f23952d5 ("module: fix bne2 "gave up waiting for init of
module libcrc32c") is worth reviewing. It dealt with a similar issue,
and in particular it addressed the issue with -EBUSY being returned
by ref_module().

And so, in theory that case should be dealt with in resolve_symbol_wait()
already. And so can you try this just to verify something:

diff --git a/kernel/module.c b/kernel/module.c
index 40ec9a030eec..98f87cbb37de 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -1459,7 +1459,7 @@ resolve_symbol_wait(struct module *mod,
 	if (wait_event_interruptible_timeout(module_wq,
 			!IS_ERR(ksym = resolve_symbol(mod, info, name, owner))
 			|| PTR_ERR(ksym) != -EBUSY,
-					     30 * HZ) <= 0) {
+					     160 * HZ) <= 0) {
 		pr_warn("%s: gave up waiting for init of module %s.\n",
 			mod->name, owner);
 	}

> code: 
> https://github.com/intel/gvt-linux/blob/bd950a66c7919d7121d2530f30984351534a96dc/kernel/module.c#L1452
> 
> In resolve_symbol_wait(), it calls resolve_symbol() to resolve the 
> symbols in "i915". In resolve_symbol() -> ref_module() -> 
> strong_try_module_get(), it will check the status of the module which 
> owns the symbol.
> 
> static inline int strong_try_module_get(struct module *mod)
> {
>      BUG_ON(mod && mod->state == MODULE_STATE_UNFORMED);
>      if (mod && mod->state == MODULE_STATE_COMING)
>          return -EBUSY;
>      if (try_module_get(mod))
>          return 0;
>      else
>          return -ENOENT;
> }
> 
> code:https://github.com/intel/gvt-linux/blob/bd950a66c7919d7121d2530f30984351534a96dc/kernel/module.c#L318
> 
> But unfortunately, this execution path begins in i915 module loading, at 
> this time, the status of kernel module "i915" is MODULE_STATE_COMING 
> until loading of "kvmgt" is finished. Thus a -EBUSY is always returned 
> when kernel is trying to resolve symbols for "kvmgt".
>
> 
> This patch below might need re-work:

If the above test patch still fails, well.. that might be telling of
another issue which is perhaps difficult to see at first glance. If
resolve_symbol_wait() won't succeed until request_module("kvmgt")
completes and if this means having kvmgt's init routine complete, that
could end up in some longer chain or in the worst case a sort of
circular dependency which is only implicated by module loading. It'd be
really odd... but I cannot rule it out.

This is one reason I hinted that you should strive to not do much on a
module's init. If you can punt work off for later that's best.

  Luis

> 
> Author: Christoph Hellwig <hch@lst.de>
> Date:   Wed Jul 21 17:53:38 2021 +0200
> 
>      drm/i915/gvt: move the gvt code into kvmgt.ko
> 
>      Instead of having an option to build the gvt code into the main i915
>      module, just move it into the kvmgt.ko module.  This only requires
>      a new struct with three entries that the main i915 module needs to
>      request before enabling VGPU passthrough operations.
> 
>      This also conveniently streamlines the GVT initialization and avoids
>      the need for the global device pointer.
> 
>      Signed-off-by: Christoph Hellwig <hch@lst.de>
>      Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
>      Link: 
> http://patchwork.freedesktop.org/patch/msgid/20210721155355.173183-5-hch@lst.de
>      Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>
> 
> On 8/26/21 6:12 AM, Zhenyu Wang wrote:
> > On 2021.08.20 12:56:34 -0700, Luis Chamberlain wrote:
> >> On Fri, Aug 20, 2021 at 04:17:24PM +0200, Christoph Hellwig wrote:
> >>> On Thu, Aug 19, 2021 at 04:29:29PM +0800, Zhenyu Wang wrote:
> >>>> I'm working on below patch to resolve this. But I met a weird issue in
> >>>> case when building i915 as module and also kvmgt module, it caused
> >>>> busy wait on request_module("kvmgt") when boot, it doesn't happen if
> >>>> building i915 into kernel. I'm not sure what could be the reason?
> >>> Luis, do you know if there is a problem with a request_module from
> >>> a driver ->probe routine that is probably called by a module_init
> >>> function itself?
> >> Generally no, but you can easily foot yourself in the feet by creating
> >> cross dependencies and not dealing with them properly. I'd make sure
> >> to keep module initialization as simple as possible, and run whatever
> >> takes more time asynchronously, then use a state machine to allow
> >> you to verify where you are in the initialization phase or query it
> >> or wait for a completion with a timeout.
> >>
> >> It seems the code in question is getting some spring cleaning, and its
> >> unclear where the code is I can inspect. If there's a tree somewhere I
> >> can take a peak I'd be happy to review possible oddities that may stick
> >> out.
> > I tried to put current patches under test here: https://github.com/intel/gvt-linux/tree/gvt-staging
> > The issue can be produced with CONFIG_DRM_I915=m and CONFIG_DRM_I915_GVT_KVMGT=m.
> >
> >> My goto model for these sorts of problems is to abstract the issue
> >> *outside* of the driver in question and implement new selftests to
> >> try to reproduce. This serves two purposes, 1) helps with testing
> >> 2) may allow you to see the problem more clearly.
> >>
> > I'll see if can abstract that.
> >
> > Thanks, Luis.
> 
> 

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-09-28 14:00                                   ` Luis Chamberlain
@ 2021-09-28 14:35                                     ` Wang, Zhi A
  2021-09-28 15:05                                       ` Jason Gunthorpe
  0 siblings, 1 reply; 60+ messages in thread
From: Wang, Zhi A @ 2021-09-28 14:35 UTC (permalink / raw)
  To: Luis Chamberlain, Jessica Yu, De Marchi, Lucas
  Cc: Zhenyu Wang, Christoph Hellwig, Jason Gunthorpe, dri-devel,
	Greg KH, intel-gfx, Joonas Lahtinen, linux-kernel, Jani Nikula,
	Gerd Hoffmann, Vivi, Rodrigo, intel-gvt-dev, Nikula, Jani

On 9/28/21 2:00 PM, Luis Chamberlain wrote:
> On Tue, Sep 28, 2021 at 07:41:00AM +0000, Wang, Zhi A wrote:
>> Hey guys:
>>
>> After some investigation, I found the root cause this problem ("i915"
>> module loading will be stuck with Christoph's refactor patches), which
>> can be reproduced by building both i915 and kvmgt as kernel module and
>> the loading i915.
> Thanks for looking into this!
>
>> The root cause is: in Linux kernel loading, before a kernel module
>> loading is finished, its symbols can not be reached by other module when
>> resolving the symbols (even they can be found in /proc/kallsyms).
>> Because the status of the kernel module is MODULE_STATE_COMING and
>> resolve_symbol() from another kernel module will check this and return a
>> -EBUSY.
> Well, it would seem that way but...
>
>> In this case, before i915 loading is finished, the requested module
>> "kvmgt" cannot reach the symbols in module i915. Thus it kept waiting
>> and left message like below in the dmesg:
>>
>> [  644.152021] kvmgt: gave up waiting for init of module i915.
>> [  644.152039] kvmgt: Unknown symbol i915_gem_object_set_to_cpu_domain
>> (err -16)
>> [  674.871409] kvmgt: gave up waiting for init of module i915.
>> [  674.871427] kvmgt: Unknown symbol intel_ring_begin (err -16)
>> [  705.590586] kvmgt: gave up waiting for init of module i915.
>> [  705.590604] kvmgt: Unknown symbol i915_vma_move_to_active (err -16)
>> [  736.310230] kvmgt: gave up waiting for init of module i915.
>> [  736.310248] kvmgt: Unknown symbol shmem_unpin_map (err -16)
>> ...
>>
>> The error message is from execution path below:
>>
>> kernel/module.c:
>>
>> [i915 module loading] ->
>> request_module("kvmgt")->[modprobe]->init_module("kvmgt")->load_module()->simplify_symbols()->resolve_symbol_wait():
>>
>> static const struct kernel_symbol *
>> resolve_symbol_wait(struct module *mod,
>>               const struct load_info *info,
>>               const char *name)
>> {
>>       const struct kernel_symbol *ksym;
>>       char owner[MODULE_NAME_LEN];
>>
>>       if (wait_event_interruptible_timeout(module_wq,
>>               !IS_ERR(ksym = resolve_symbol(mod, info, name, owner))
>>               || PTR_ERR(ksym) != -EBUSY,
>>                            30 * HZ) <= 0) {
>>           pr_warn("%s: gave up waiting for init of module %s.\n",
>>               mod->name, owner);
>>
>> }
> Commit 9bea7f23952d5 ("module: fix bne2 "gave up waiting for init of
> module libcrc32c") is worth reviewing. It dealt with a similar issue,
> and in particular it addressed the issue with -EBUSY being returned
> by ref_module().
>
> And so, in theory that case should be dealt with in resolve_symbol_wait()
> already. And so can you try this just to verify something:
>
> diff --git a/kernel/module.c b/kernel/module.c
> index 40ec9a030eec..98f87cbb37de 100644
> --- a/kernel/module.c
> +++ b/kernel/module.c
> @@ -1459,7 +1459,7 @@ resolve_symbol_wait(struct module *mod,
>   	if (wait_event_interruptible_timeout(module_wq,
>   			!IS_ERR(ksym = resolve_symbol(mod, info, name, owner))
>   			|| PTR_ERR(ksym) != -EBUSY,
> -					     30 * HZ) <= 0) {
> +					     160 * HZ) <= 0) {
>   		pr_warn("%s: gave up waiting for init of module %s.\n",
>   			mod->name, owner);
>   	}
>
Hi Luis:

Thanks so much for the reply and patch.:)

I am afraid that this patch wouldn't work in this case as the 
request_module("kvmgt") happens in the init_module of i915. Before the 
initialization path is finished in i915, the i915 symbols are not 
available to be referenced. Unfortunately, It's matter of sequence, not 
of delay. :(

>> code:
>> https://github.com/intel/gvt-linux/blob/bd950a66c7919d7121d2530f30984351534a96dc/kernel/module.c#L1452
>>
>> In resolve_symbol_wait(), it calls resolve_symbol() to resolve the
>> symbols in "i915". In resolve_symbol() -> ref_module() ->
>> strong_try_module_get(), it will check the status of the module which
>> owns the symbol.
>>
>> static inline int strong_try_module_get(struct module *mod)
>> {
>>       BUG_ON(mod && mod->state == MODULE_STATE_UNFORMED);
>>       if (mod && mod->state == MODULE_STATE_COMING)
>>           return -EBUSY;
>>       if (try_module_get(mod))
>>           return 0;
>>       else
>>           return -ENOENT;
>> }
>>
>> code:https://github.com/intel/gvt-linux/blob/bd950a66c7919d7121d2530f30984351534a96dc/kernel/module.c#L318
>>
>> But unfortunately, this execution path begins in i915 module loading, at
>> this time, the status of kernel module "i915" is MODULE_STATE_COMING
>> until loading of "kvmgt" is finished. Thus a -EBUSY is always returned
>> when kernel is trying to resolve symbols for "kvmgt".
>>
>>
>> This patch below might need re-work:
> If the above test patch still fails, well.. that might be telling of
> another issue which is perhaps difficult to see at first glance. If
> resolve_symbol_wait() won't succeed until request_module("kvmgt")
> completes and if this means having kvmgt's init routine complete, that
> could end up in some longer chain or in the worst case a sort of
> circular dependency which is only implicated by module loading. It'd be
> really odd... but I cannot rule it out.
>
> This is one reason I hinted that you should strive to not do much on a
> module's init. If you can punt work off for later that's best.

Yes. I was thinking of the possibility of putting off some work later so 
that we don't need to make a lot of changes. GVT-g needs to take a 
snapshot of GPU registers as the initial virtual states for other vGPUs, 
which requires the initialization happens at a certain early time of 
initialization of i915. I was thinking maybe we can take other patches 
from Christoph like "de-virtualize*" except this one because currently 
we have to maintain a TEST-ONLY patch on our tree to prevent i915 built 
as kernel module.

Zhi.

>    Luis
>
>> Author: Christoph Hellwig <hch@lst.de>
>> Date:   Wed Jul 21 17:53:38 2021 +0200
>>
>>       drm/i915/gvt: move the gvt code into kvmgt.ko
>>
>>       Instead of having an option to build the gvt code into the main i915
>>       module, just move it into the kvmgt.ko module.  This only requires
>>       a new struct with three entries that the main i915 module needs to
>>       request before enabling VGPU passthrough operations.
>>
>>       This also conveniently streamlines the GVT initialization and avoids
>>       the need for the global device pointer.
>>
>>       Signed-off-by: Christoph Hellwig <hch@lst.de>
>>       Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
>>       Link:
>> http://patchwork.freedesktop.org/patch/msgid/20210721155355.173183-5-hch@lst.de
>>       Acked-by: Zhenyu Wang <zhenyuw@linux.intel.com>
>>
>> On 8/26/21 6:12 AM, Zhenyu Wang wrote:
>>> On 2021.08.20 12:56:34 -0700, Luis Chamberlain wrote:
>>>> On Fri, Aug 20, 2021 at 04:17:24PM +0200, Christoph Hellwig wrote:
>>>>> On Thu, Aug 19, 2021 at 04:29:29PM +0800, Zhenyu Wang wrote:
>>>>>> I'm working on below patch to resolve this. But I met a weird issue in
>>>>>> case when building i915 as module and also kvmgt module, it caused
>>>>>> busy wait on request_module("kvmgt") when boot, it doesn't happen if
>>>>>> building i915 into kernel. I'm not sure what could be the reason?
>>>>> Luis, do you know if there is a problem with a request_module from
>>>>> a driver ->probe routine that is probably called by a module_init
>>>>> function itself?
>>>> Generally no, but you can easily foot yourself in the feet by creating
>>>> cross dependencies and not dealing with them properly. I'd make sure
>>>> to keep module initialization as simple as possible, and run whatever
>>>> takes more time asynchronously, then use a state machine to allow
>>>> you to verify where you are in the initialization phase or query it
>>>> or wait for a completion with a timeout.
>>>>
>>>> It seems the code in question is getting some spring cleaning, and its
>>>> unclear where the code is I can inspect. If there's a tree somewhere I
>>>> can take a peak I'd be happy to review possible oddities that may stick
>>>> out.
>>> I tried to put current patches under test here: https://github.com/intel/gvt-linux/tree/gvt-staging
>>> The issue can be produced with CONFIG_DRM_I915=m and CONFIG_DRM_I915_GVT_KVMGT=m.
>>>
>>>> My goto model for these sorts of problems is to abstract the issue
>>>> *outside* of the driver in question and implement new selftests to
>>>> try to reproduce. This serves two purposes, 1) helps with testing
>>>> 2) may allow you to see the problem more clearly.
>>>>
>>> I'll see if can abstract that.
>>>
>>> Thanks, Luis.
>>


^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-09-28 14:35                                     ` Wang, Zhi A
@ 2021-09-28 15:05                                       ` Jason Gunthorpe
  2021-09-29 18:27                                         ` Wang, Zhi A
  0 siblings, 1 reply; 60+ messages in thread
From: Jason Gunthorpe @ 2021-09-28 15:05 UTC (permalink / raw)
  To: Wang, Zhi A
  Cc: Luis Chamberlain, Jessica Yu, De Marchi, Lucas, Zhenyu Wang,
	Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Nikula, Jani

On Tue, Sep 28, 2021 at 02:35:06PM +0000, Wang, Zhi A wrote:

> Yes. I was thinking of the possibility of putting off some work later so 
> that we don't need to make a lot of changes. GVT-g needs to take a 
> snapshot of GPU registers as the initial virtual states for other vGPUs, 
> which requires the initialization happens at a certain early time of 
> initialization of i915. I was thinking maybe we can take other patches 
> from Christoph like "de-virtualize*" except this one because currently 
> we have to maintain a TEST-ONLY patch on our tree to prevent i915 built 
> as kernel module.

How about just capture these registers in the main module/device and
not try so hard to isolate it to the gvt stuff?

Jason

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-09-28 15:05                                       ` Jason Gunthorpe
@ 2021-09-29 18:27                                         ` Wang, Zhi A
  2021-09-29 18:55                                           ` Jason Gunthorpe
  2021-09-30  5:24                                           ` Christoph Hellwig
  0 siblings, 2 replies; 60+ messages in thread
From: Wang, Zhi A @ 2021-09-29 18:27 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Luis Chamberlain, Jessica Yu, De Marchi, Lucas, Zhenyu Wang,
	Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Nikula, Jani

On 9/28/21 3:05 PM, Jason Gunthorpe wrote:
> On Tue, Sep 28, 2021 at 02:35:06PM +0000, Wang, Zhi A wrote:
>
>> Yes. I was thinking of the possibility of putting off some work later so
>> that we don't need to make a lot of changes. GVT-g needs to take a
>> snapshot of GPU registers as the initial virtual states for other vGPUs,
>> which requires the initialization happens at a certain early time of
>> initialization of i915. I was thinking maybe we can take other patches
>> from Christoph like "de-virtualize*" except this one because currently
>> we have to maintain a TEST-ONLY patch on our tree to prevent i915 built
>> as kernel module.
> How about just capture these registers in the main module/device and
> not try so hard to isolate it to the gvt stuff?

Hi Jason:

Thanks for the idea. I am not sure i915 guys would take this idea since 
that it's only for GVT-g, i915 doesn't use this at all. We need to take 
a snapshot of both PCI configuration space and MMIO registers before 
i915 driver starts to touch the HW.

One idea is we capture the registers in intel_gvt.c during the early 
initialization and do everything else when i915.ko is fully loaded. But 
how about dependence between i915.ko and kvmgt.ko? We cannot do 
request_module("kvmgt") in i915.ko.

Maybe Christoph can give more input on this and how we can deal with 
this? Before we figure out an solution, we have to pick that patch out 
since it blocks our pull request schedule.

Also I was thinking if moving gvt into kvmgt.ko is the right direction. 
It seems the module loading system in kernel is not designed for "module 
A loading module B, which needs symbols from module A, in the 
initialization path of module A".

Thanks,

Zhi.

> Jason



^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-09-29 18:27                                         ` Wang, Zhi A
@ 2021-09-29 18:55                                           ` Jason Gunthorpe
  2021-10-01 13:01                                             ` Wang, Zhi A
  2021-09-30  5:24                                           ` Christoph Hellwig
  1 sibling, 1 reply; 60+ messages in thread
From: Jason Gunthorpe @ 2021-09-29 18:55 UTC (permalink / raw)
  To: Wang, Zhi A
  Cc: Luis Chamberlain, Jessica Yu, De Marchi, Lucas, Zhenyu Wang,
	Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Nikula, Jani

On Wed, Sep 29, 2021 at 06:27:16PM +0000, Wang, Zhi A wrote:
> On 9/28/21 3:05 PM, Jason Gunthorpe wrote:
> > On Tue, Sep 28, 2021 at 02:35:06PM +0000, Wang, Zhi A wrote:
> >
> >> Yes. I was thinking of the possibility of putting off some work later so
> >> that we don't need to make a lot of changes. GVT-g needs to take a
> >> snapshot of GPU registers as the initial virtual states for other vGPUs,
> >> which requires the initialization happens at a certain early time of
> >> initialization of i915. I was thinking maybe we can take other patches
> >> from Christoph like "de-virtualize*" except this one because currently
> >> we have to maintain a TEST-ONLY patch on our tree to prevent i915 built
> >> as kernel module.
> > How about just capture these registers in the main module/device and
> > not try so hard to isolate it to the gvt stuff?
> 
> Hi Jason:
> 
> Thanks for the idea. I am not sure i915 guys would take this idea since 
> that it's only for GVT-g, i915 doesn't use this at all. We need to take 
> a snapshot of both PCI configuration space and MMIO registers before 
> i915 driver starts to touch the HW.

Given the code is already linked into i915 I don't see there is much
to object to here. It can remain conditional on the kernel parameter
as today.

As a general philosophy this would all be much less strange if the
mdev .ko is truely optional. It should be cleanly seperate from its
base device and never request_module'd..

In this case auxiliary device might be a good option, have i915 create
one and the mdev module be loaded against it.

In the mean time is there some shortcut to get this series to move
ahead? Is patch 4 essential to the rest of the series?

A really awful hack would be to push the pci_driver_register into a
WQ so that the request_module is guarenteed to not be part of the
module_init callchain.

> Also I was thinking if moving gvt into kvmgt.ko is the right direction. 
> It seems the module loading system in kernel is not designed for "module 
> A loading module B, which needs symbols from module A, in the 
> initialization path of module A".

Of course not, that is a circular module dependency, it should not be
that way. The SW layers need to be clean and orderly - meaning the
i915 module needs to have the minimal amount of code to support the
mdev module.

Jason

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-09-29 18:27                                         ` Wang, Zhi A
  2021-09-29 18:55                                           ` Jason Gunthorpe
@ 2021-09-30  5:24                                           ` Christoph Hellwig
  1 sibling, 0 replies; 60+ messages in thread
From: Christoph Hellwig @ 2021-09-30  5:24 UTC (permalink / raw)
  To: Wang, Zhi A
  Cc: Jason Gunthorpe, Luis Chamberlain, Jessica Yu, De Marchi, Lucas,
	Zhenyu Wang, Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Nikula, Jani

On Wed, Sep 29, 2021 at 06:27:16PM +0000, Wang, Zhi A wrote:
> Thanks for the idea. I am not sure i915 guys would take this idea since 
> that it's only for GVT-g, i915 doesn't use this at all. We need to take 
> a snapshot of both PCI configuration space and MMIO registers before 
> i915 driver starts to touch the HW.
> 
> One idea is we capture the registers in intel_gvt.c during the early 
> initialization and do everything else when i915.ko is fully loaded. But 
> how about dependence between i915.ko and kvmgt.ko? We cannot do 
> request_module("kvmgt") in i915.ko.
> 
> Maybe Christoph can give more input on this and how we can deal with 
> this? Before we figure out an solution, we have to pick that patch out 
> since it blocks our pull request schedule.
> 
> Also I was thinking if moving gvt into kvmgt.ko is the right direction. 
> It seems the module loading system in kernel is not designed for "module 
> A loading module B, which needs symbols from module A, in the 
> initialization path of module A".

I have a few idea on how to sort this "deadlock" out.  But I'm completely
swamped until the end of next week, so I won't get to anything before
that.

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-09-29 18:55                                           ` Jason Gunthorpe
@ 2021-10-01 13:01                                             ` Wang, Zhi A
  2021-10-05  7:33                                               ` Wang, Zhi A
  0 siblings, 1 reply; 60+ messages in thread
From: Wang, Zhi A @ 2021-10-01 13:01 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Luis Chamberlain, Jessica Yu, De Marchi, Lucas, Zhenyu Wang,
	Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Nikula, Jani

On 9/29/21 6:55 PM, Jason Gunthorpe wrote:
> On Wed, Sep 29, 2021 at 06:27:16PM +0000, Wang, Zhi A wrote:
>> On 9/28/21 3:05 PM, Jason Gunthorpe wrote:
>>> On Tue, Sep 28, 2021 at 02:35:06PM +0000, Wang, Zhi A wrote:
>>>
>>>> Yes. I was thinking of the possibility of putting off some work later so
>>>> that we don't need to make a lot of changes. GVT-g needs to take a
>>>> snapshot of GPU registers as the initial virtual states for other vGPUs,
>>>> which requires the initialization happens at a certain early time of
>>>> initialization of i915. I was thinking maybe we can take other patches
>>>> from Christoph like "de-virtualize*" except this one because currently
>>>> we have to maintain a TEST-ONLY patch on our tree to prevent i915 built
>>>> as kernel module.
>>> How about just capture these registers in the main module/device and
>>> not try so hard to isolate it to the gvt stuff?
>> Hi Jason:
>>
>> Thanks for the idea. I am not sure i915 guys would take this idea since
>> that it's only for GVT-g, i915 doesn't use this at all. We need to take
>> a snapshot of both PCI configuration space and MMIO registers before
>> i915 driver starts to touch the HW.
> Given the code is already linked into i915 I don't see there is much
> to object to here. It can remain conditional on the kernel parameter
> as today.
>
> As a general philosophy this would all be much less strange if the
> mdev .ko is truely optional. It should be cleanly seperate from its
> base device and never request_module'd..
>
> In this case auxiliary device might be a good option, have i915 create
> one and the mdev module be loaded against it.
>
> In the mean time is there some shortcut to get this series to move
> ahead? Is patch 4 essential to the rest of the series?
>
> A really awful hack would be to push the pci_driver_register into a
> WQ so that the request_module is guarenteed to not be part of the
> module_init callchain.

Hi Jason and folks:

Thanks so much for the ideas. That sounds great and I was keeping 
thinking how to make progress on this. How about we do like this: We 
don't do request_module("kvmgt") in i915.ko, which resolves the circular 
module dependency. We keep the code of doing snapshot of registers in 
intel_gvt.c. When i915.enable_gvt=1, we do the snapshot. Then we export 
functions for kvmgt.ko in intel_gvt.c to check if gvt in i915 is enabled 
or not and get the snapshots.

How does that sounds? I just need to write another patch and put it on 
top of Christoph's series.

Thanks,

Zhi.

>> Also I was thinking if moving gvt into kvmgt.ko is the right direction.
>> It seems the module loading system in kernel is not designed for "module
>> A loading module B, which needs symbols from module A, in the
>> initialization path of module A".
> Of course not, that is a circular module dependency, it should not be
> that way. The SW layers need to be clean and orderly - meaning the
> i915 module needs to have the minimal amount of code to support the
> mdev module.
>
> Jason



^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: refactor the i915 GVT support
  2021-10-01 13:01                                             ` Wang, Zhi A
@ 2021-10-05  7:33                                               ` Wang, Zhi A
  0 siblings, 0 replies; 60+ messages in thread
From: Wang, Zhi A @ 2021-10-05  7:33 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Luis Chamberlain, Jessica Yu, De Marchi, Lucas, Zhenyu Wang,
	Christoph Hellwig, dri-devel, Greg KH, intel-gfx,
	Joonas Lahtinen, linux-kernel, Jani Nikula, Gerd Hoffmann, Vivi,
	Rodrigo, intel-gvt-dev, Nikula, Jani

Hi folks:

It seems we haven't reached a possible solution of this refactor patch 
series. The current patch series needs to be re-worked because of the 
module/symbol dependency(The root cause has been discussed in another 
email). I have to get them off from our gvt-next repo so that we can 
continue our development and pull-request to upstream. Thanks so much 
for the patch and the discussion.

Thanks,
Zhi.

On 10/1/21 1:01 PM, Wang, Zhi A wrote:
> On 9/29/21 6:55 PM, Jason Gunthorpe wrote:
>> On Wed, Sep 29, 2021 at 06:27:16PM +0000, Wang, Zhi A wrote:
>>> On 9/28/21 3:05 PM, Jason Gunthorpe wrote:
>>>> On Tue, Sep 28, 2021 at 02:35:06PM +0000, Wang, Zhi A wrote:
>>>>
>>>>> Yes. I was thinking of the possibility of putting off some work 
>>>>> later so
>>>>> that we don't need to make a lot of changes. GVT-g needs to take a
>>>>> snapshot of GPU registers as the initial virtual states for other 
>>>>> vGPUs,
>>>>> which requires the initialization happens at a certain early time of
>>>>> initialization of i915. I was thinking maybe we can take other 
>>>>> patches
>>>>> from Christoph like "de-virtualize*" except this one because 
>>>>> currently
>>>>> we have to maintain a TEST-ONLY patch on our tree to prevent i915 
>>>>> built
>>>>> as kernel module.
>>>> How about just capture these registers in the main module/device and
>>>> not try so hard to isolate it to the gvt stuff?
>>> Hi Jason:
>>>
>>> Thanks for the idea. I am not sure i915 guys would take this idea since
>>> that it's only for GVT-g, i915 doesn't use this at all. We need to take
>>> a snapshot of both PCI configuration space and MMIO registers before
>>> i915 driver starts to touch the HW.
>> Given the code is already linked into i915 I don't see there is much
>> to object to here. It can remain conditional on the kernel parameter
>> as today.
>>
>> As a general philosophy this would all be much less strange if the
>> mdev .ko is truely optional. It should be cleanly seperate from its
>> base device and never request_module'd..
>>
>> In this case auxiliary device might be a good option, have i915 create
>> one and the mdev module be loaded against it.
>>
>> In the mean time is there some shortcut to get this series to move
>> ahead? Is patch 4 essential to the rest of the series?
>>
>> A really awful hack would be to push the pci_driver_register into a
>> WQ so that the request_module is guarenteed to not be part of the
>> module_init callchain.
>
> Hi Jason and folks:
>
> Thanks so much for the ideas. That sounds great and I was keeping 
> thinking how to make progress on this. How about we do like this: We 
> don't do request_module("kvmgt") in i915.ko, which resolves the 
> circular module dependency. We keep the code of doing snapshot of 
> registers in intel_gvt.c. When i915.enable_gvt=1, we do the snapshot. 
> Then we export functions for kvmgt.ko in intel_gvt.c to check if gvt 
> in i915 is enabled or not and get the snapshots.
>
> How does that sounds? I just need to write another patch and put it on 
> top of Christoph's series.
>
> Thanks,
>
> Zhi.
>
>>> Also I was thinking if moving gvt into kvmgt.ko is the right direction.
>>> It seems the module loading system in kernel is not designed for 
>>> "module
>>> A loading module B, which needs symbols from module A, in the
>>> initialization path of module A".
>> Of course not, that is a circular module dependency, it should not be
>> that way. The SW layers need to be clean and orderly - meaning the
>> i915 module needs to have the minimal amount of code to support the
>> mdev module.
>>
>> Jason
>
>


^ permalink raw reply	[flat|nested] 60+ messages in thread

end of thread, other threads:[~2021-10-05  7:33 UTC | newest]

Thread overview: 60+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-21 15:53 refactor the i915 GVT support Christoph Hellwig
2021-07-21 15:53 ` [PATCH 01/21] drm/i915/gvt: integrate into the main Makefile Christoph Hellwig
2021-08-03  9:27   ` Zhenyu Wang
2021-07-21 15:53 ` [PATCH 02/21] drm/i915/gvt: remove module refcounting in intel_gvt_{,un}register_hypervisor Christoph Hellwig
2021-07-21 15:53 ` [PATCH 03/21] drm/i915/gvt: remove enum hypervisor_type Christoph Hellwig
2021-07-21 15:53 ` [PATCH 04/21] drm/i915/gvt: move the gvt code into kvmgt.ko Christoph Hellwig
2021-08-09 11:29   ` Joonas Lahtinen
2021-08-09 14:29     ` Christoph Hellwig
2021-07-21 15:53 ` [PATCH 05/21] drm/i915/gvt: remove intel_gvt_ops Christoph Hellwig
2021-07-21 15:53 ` [PATCH 06/21] drm/i915/gvt: remove the map_gfn_to_mfn and set_trap_area ops Christoph Hellwig
2021-07-21 15:53 ` [PATCH 07/21] drm/i915/gvt: remove the unused from_virt_to_mfn op Christoph Hellwig
2021-07-21 15:53 ` [PATCH 08/21] drm/i915/gvt: merge struct kvmgt_vdev into struct intel_vgpu Christoph Hellwig
2021-07-21 15:53 ` [PATCH 09/21] drm/i915/gvt: merge struct kvmgt_guest_info into strut intel_vgpu Christoph Hellwig
2021-07-21 15:53 ` [PATCH 10/21] drm/i915/gvt: remove vgpu->handle Christoph Hellwig
2021-07-21 15:53 ` [PATCH 11/21] drm/i915/gvt: devirtualize ->{read,write}_gpa Christoph Hellwig
2021-07-21 15:53 ` [PATCH 12/21] drm/i915/gvt: devirtualize ->{get,put}_vfio_device Christoph Hellwig
2021-07-21 15:53 ` [PATCH 13/21] drm/i915/gvt: devirtualize ->set_edid and ->set_opregion Christoph Hellwig
2021-07-21 15:53 ` [PATCH 14/21] drm/i915/gvt: devirtualize ->detach_vgpu Christoph Hellwig
2021-07-21 15:53 ` [PATCH 15/21] drm/i915/gvt: devirtualize ->inject_msi Christoph Hellwig
2021-07-21 15:53 ` [PATCH 16/21] drm/i915/gvt: devirtualize ->is_valid_gfn Christoph Hellwig
2021-07-21 15:53 ` [PATCH 17/21] drm/i915/gvt: devirtualize ->gfn_to_mfn Christoph Hellwig
2021-07-21 15:53 ` [PATCH 18/21] drm/i915/gvt: devirtualize ->{enable,disable}_page_track Christoph Hellwig
2021-07-21 15:53 ` [PATCH 19/21] drm/i915/gvt: devirtualize ->dma_{,un}map_guest_page Christoph Hellwig
2021-07-21 15:53 ` [PATCH 20/21] drm/i915/gvt: devirtualize dma_pin_guest_page Christoph Hellwig
2021-07-21 15:53 ` [PATCH 21/21] drm/i915/gvt: remove struct intel_gvt_mpt Christoph Hellwig
2021-07-22  9:45 ` refactor the i915 GVT support Zhenyu Wang
2021-07-22 12:55   ` Christoph Hellwig
2021-07-22 10:49 ` Wang, Zhi A
2021-07-22 11:26   ` Gerd Hoffmann
2021-07-27 12:12     ` Jason Gunthorpe
2021-07-28 13:38       ` Wang, Zhi A
2021-07-28 13:43         ` Greg KH
2021-07-28 17:59         ` Jason Gunthorpe
2021-07-29  7:20           ` Christoph Hellwig
2021-07-29  7:30             ` [Intel-gfx] " Daniel Vetter
2021-08-03  9:43             ` Zhenyu Wang
2021-08-03 14:30               ` Jason Gunthorpe
2021-08-04  5:26                 ` Zhenyu Wang
2021-08-16 17:34                   ` Christoph Hellwig
2021-08-17  1:08                     ` Zhenyu Wang
2021-08-17  5:22                       ` Zhenyu Wang
2021-08-19  8:29                         ` Zhenyu Wang
2021-08-19 14:43                           ` Joonas Lahtinen
2021-08-26  6:04                             ` Zhenyu Wang
2021-08-20 14:17                           ` Christoph Hellwig
2021-08-20 19:56                             ` Luis Chamberlain
2021-08-26  6:12                               ` Zhenyu Wang
2021-09-28  7:41                                 ` Wang, Zhi A
2021-09-28 14:00                                   ` Luis Chamberlain
2021-09-28 14:35                                     ` Wang, Zhi A
2021-09-28 15:05                                       ` Jason Gunthorpe
2021-09-29 18:27                                         ` Wang, Zhi A
2021-09-29 18:55                                           ` Jason Gunthorpe
2021-10-01 13:01                                             ` Wang, Zhi A
2021-10-05  7:33                                               ` Wang, Zhi A
2021-09-30  5:24                                           ` Christoph Hellwig
2021-08-26  6:08                             ` Zhenyu Wang
2021-08-04  5:40                 ` Christoph Hellwig
2021-07-29  8:19           ` Wang, Zhi A
2021-07-22 13:16   ` Greg KH

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).