[PATCH v10 0/9] i915 vgpu PV to improve vgpu performance

[PATCH v10 0/9] i915 vgpu PV to improve vgpu performance

[Xiaolin Zhang]

To improve vgpu performance, it could implement some PV optimization
such as to reduce the mmio access trap numbers or eliminate certain piece
of HW emulation within guest driver to reduce vm exit/vm enter cost.

the solutions in this patch set are implemented two PV optimizations based
on the shared memory region between guest and GVTg for data communication.
The shared memory region is allocated by guest driver and this
region's memory guest physical address will be passed to GVTg through
PVINFO register and later GVTg can access this region directly without
trap cost to achieve data exchange purpose between guest and GVTg.

in this patch set, 2 kind of PV optimization implemented controlled by
pv_caps PVINO register with different pv bit.
1. workload PV submission (context submission): reduce 4 traps to 1 trap
and eliminated execlists HW behaviour emulation.
2. ppgtt PV update: eliminate the cost of ppgtt write protection.

based on the experiment, for small workloads, specifally, glxgears with
vblank_mode off, the average performance gain on single vgpu is 30~50%.
for large workload such as media and 3D, the average performance gain
is about 4%. 

based on the PV mechanism, it could achive more vgpu feature optimization
such as globle GTT update, display plane and water mark update.

v0: RFC patch set
v1: addressed RFC review comments
v2: addressed v1 review comments, added pv callbacks for pv operations
v3:
1. addressed v2 review comments, removed pv callbacks code duplication in
v2 and unified pv calls under g2v notification register. different g2v pv
notifications defined.
2. dropped pv master irq feature due to hard conflict with recnet i915
change and take time to rework.
v4:
1. addressed v3 review comments.
2. extended workload PV submission by skip execlists HW behaviour emulation
and context switch interrupt injection.  
v5:
1. addressed v4 review comments from Chris for pv submission.
2. per-engine communication between PV guest and host.
v6:
1. addressed v5 review comments from Chris for pv submission.
2. addressed v5 review comments from Zhenyu for PV version support.
v7:
1. addessed v6 review comments from Chris starting to use pv cmd buffer
communication between pv guest and GVT.
v8: addressed v7 review comments from Chris to refine pv submssion code.
v9: refined PV user-facing message and to free PV shared memory
in i915_destroy_vgpu.
v10: rebased to 5.3.0-rc8 and conducted the IGT BAT test and i915 live test
with PV feature. No regression introduced by PV features. for IGT BAT test
(fast-feedback.testlist), there are 225 cases in total (162 pass, 12 fail,
42 skip) for both vGPU with/without PV features. for i915 live test, PV
features do not bring regression as well. there is a number of cases failed
for vGPU live test which include intel_workarounds_live_selftests,
intel_hangcheck_live_selftests, igt_vm_isolation,
i915_gem_object_blt_live_selftests, preempt, live_virtual_* and required
further analysis, not contributed by the PV patches.

Xiaolin Zhang (9):
  drm/i915: introduced vgpu pv capability
  drm/i915: vgpu shared memory setup for pv optimization
  drm/i915: vgpu pv command buffer support
  drm/i915: vgpu ppgtt update pv optimization
  drm/i915: vgpu context submission pv optimization
  drm/i915/gvt: GVTg handle pv_caps PVINFO register
  drm/i915/gvt: GVTg handle shared_page setup
  drm/i915/gvt: GVTg support ppgtt pv optimization
  drm/i915/gvt: GVTg support context submission pv optimization

 drivers/gpu/drm/i915/Makefile              |   2 +-
 drivers/gpu/drm/i915/gt/intel_lrc.c        |  12 +-
 drivers/gpu/drm/i915/gvt/execlist.c        |   6 +
 drivers/gpu/drm/i915/gvt/gtt.c             | 298 ++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/gtt.h             |  11 +
 drivers/gpu/drm/i915/gvt/gvt.h             |  12 +-
 drivers/gpu/drm/i915/gvt/handlers.c        | 181 +++++++++++-
 drivers/gpu/drm/i915/gvt/vgpu.c            |  50 ++++
 drivers/gpu/drm/i915/i915_debugfs.c        |   2 +
 drivers/gpu/drm/i915/i915_drv.c            |   2 +
 drivers/gpu/drm/i915/i915_drv.h            |   6 +-
 drivers/gpu/drm/i915/i915_gem.c            |   3 +-
 drivers/gpu/drm/i915/i915_gem_gtt.c        |  10 +-
 drivers/gpu/drm/i915/i915_gem_gtt.h        |   8 +
 drivers/gpu/drm/i915/i915_pvinfo.h         |   9 +-
 drivers/gpu/drm/i915/i915_vgpu.c           | 426 ++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_vgpu.h           | 133 +++++++++
 drivers/gpu/drm/i915/intel_pv_submission.c | 300 ++++++++++++++++++++
 18 files changed, 1455 insertions(+), 16 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/intel_pv_submission.c

-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 1/9] drm/i915: introduced vgpu pv capability

[Xiaolin Zhang]

pv capability for vgpu was introduced by pv_caps in struct
i915_virtual_gpu and a new pv_caps register for host GVT
was defined in struct vgt_if for vgpu pv optimization.

both of them are used to control different feature pv optimization
supported and implemented by both guest and host.

These fields are default zero, no any pv feature enabled.

it also adds VGT_CAPS_PV capability BIT for guest to check GVTg
can support PV feature or not.

v0: RFC, introudced enable_pvmmio module parameter.
v1: addressed RFC comment to remove enable_pvmmio module parameter
by pv capability check.
v2: rebase.
v3: distinct pv caps from guest and host. renamed enable_pvmmio to
pvmmio_caps which is used for host pv caps.
v4: consolidated all pv related functons into a single file i915_vgpu.c
and renamed pvmmio to pv_caps.
v5: refined PV user-facing message and print pv_caps under debugfs.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/i915_debugfs.c |  2 ++
 drivers/gpu/drm/i915/i915_drv.h     |  2 ++
 drivers/gpu/drm/i915/i915_pvinfo.h  |  5 ++++-
 drivers/gpu/drm/i915/i915_vgpu.c    | 43 ++++++++++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_vgpu.h    |  9 ++++++++
 5 files changed, 59 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 43db500..e404c3c 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -66,6 +66,8 @@ static int i915_capabilities(struct seq_file *m, void *data)
 	seq_printf(m, "gen: %d\n", INTEL_GEN(dev_priv));
 	seq_printf(m, "platform: %s\n", intel_platform_name(info->platform));
 	seq_printf(m, "pch: %d\n", INTEL_PCH_TYPE(dev_priv));
+	if (intel_vgpu_active(dev_priv))
+		seq_printf(m, "vgpu pv_caps: 0x%x\n", dev_priv->vgpu.pv_caps);
 
 	msg = "n/a";
 #ifdef CONFIG_INTEL_IOMMU
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index bf60088..19fbbef 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -99,6 +99,7 @@
 #include "i915_irq.h"
 
 #include "intel_gvt.h"
+#include "i915_pvinfo.h"
 
 /* General customization:
  */
@@ -971,6 +972,7 @@ struct i915_virtual_gpu {
 	struct mutex lock; /* serialises sending of g2v_notify command pkts */
 	bool active;
 	u32 caps;
+	u32 pv_caps;
 };
 
 /* used in computing the new watermarks state */
diff --git a/drivers/gpu/drm/i915/i915_pvinfo.h b/drivers/gpu/drm/i915/i915_pvinfo.h
index 683e97a..ad398b4 100644
--- a/drivers/gpu/drm/i915/i915_pvinfo.h
+++ b/drivers/gpu/drm/i915/i915_pvinfo.h
@@ -57,6 +57,7 @@ enum vgt_g2v_type {
 #define VGT_CAPS_FULL_PPGTT		BIT(2)
 #define VGT_CAPS_HWSP_EMULATION		BIT(3)
 #define VGT_CAPS_HUGE_GTT		BIT(4)
+#define VGT_CAPS_PV		BIT(5)
 
 struct vgt_if {
 	u64 magic;		/* VGT_MAGIC */
@@ -109,7 +110,9 @@ struct vgt_if {
 	u32 execlist_context_descriptor_lo;
 	u32 execlist_context_descriptor_hi;
 
-	u32  rsv7[0x200 - 24];    /* pad to one page */
+	u32 pv_caps;
+
+	u32  rsv7[0x200 - 25];    /* pad to one page */
 } __packed;
 
 #define vgtif_offset(x) (offsetof(struct vgt_if, x))
diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
index 968be26..981959b 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.c
+++ b/drivers/gpu/drm/i915/i915_vgpu.c
@@ -95,7 +95,13 @@ void i915_detect_vgpu(struct drm_i915_private *dev_priv)
 
 	dev_priv->vgpu.active = true;
 	mutex_init(&dev_priv->vgpu.lock);
-	DRM_INFO("Virtual GPU for Intel GVT-g detected.\n");
+
+	if (!intel_vgpu_check_pv_caps(dev_priv, shared_area)) {
+		DRM_INFO("Virtual GPU for Intel GVT-g detected.\n");
+		goto out;
+	}
+
+	DRM_INFO("Virtual GPU for Intel GVT-g detected with PV Optimized.\n");
 
 out:
 	pci_iounmap(pdev, shared_area);
@@ -300,3 +306,38 @@ int intel_vgt_balloon(struct i915_ggtt *ggtt)
 	DRM_ERROR("VGT balloon fail\n");
 	return ret;
 }
+
+/*
+ * i915 vgpu PV support for Linux
+ */
+
+/**
+ * intel_vgpu_check_pv_caps - detect virtual GPU PV capabilities
+ * @dev_priv: i915 device private
+ *
+ * This function is called at the initialization stage, to detect VGPU
+ * PV capabilities
+ *
+ * If guest wants to enable pv_caps, it needs to config it explicitly
+ * through vgt_if interface from gvt layer.
+ */
+bool intel_vgpu_check_pv_caps(struct drm_i915_private *dev_priv,
+		void __iomem *shared_area)
+{
+	u32 gvt_pvcaps;
+	u32 pvcaps = 0;
+
+	if (!intel_vgpu_has_pv_caps(dev_priv))
+		return false;
+
+	/* PV capability negotiation between PV guest and GVT */
+	gvt_pvcaps = readl(shared_area + vgtif_offset(pv_caps));
+	pvcaps = dev_priv->vgpu.pv_caps & gvt_pvcaps;
+	dev_priv->vgpu.pv_caps = pvcaps;
+	writel(pvcaps, shared_area + vgtif_offset(pv_caps));
+
+	if (!pvcaps)
+		return false;
+
+	return true;
+}
diff --git a/drivers/gpu/drm/i915/i915_vgpu.h b/drivers/gpu/drm/i915/i915_vgpu.h
index 8b3663d..bbe56b5 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.h
+++ b/drivers/gpu/drm/i915/i915_vgpu.h
@@ -43,7 +43,16 @@
 	return dev_priv->vgpu.caps & VGT_CAPS_HUGE_GTT;
 }
 
+static inline bool
+intel_vgpu_has_pv_caps(struct drm_i915_private *dev_priv)
+{
+	return dev_priv->vgpu.caps & VGT_CAPS_PV;
+}
+
 int intel_vgt_balloon(struct i915_ggtt *ggtt);
 void intel_vgt_deballoon(struct i915_ggtt *ggtt);
 
+/* i915 vgpu pv related functions */
+bool intel_vgpu_check_pv_caps(struct drm_i915_private *dev_priv,
+		void __iomem *shared_area);
 #endif /* _I915_VGPU_H_ */
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 2/9] drm/i915: vgpu shared memory setup for pv optimization

[Xiaolin Zhang]

To enable vgpu pv features, we need to setup a shared memory page
which will be used for data exchange directly accessed between both
guest and backend i915 driver to avoid emulation trap cost.

guest i915 will allocate this page memory and then pass it's physical
address to backend i915 driver through PVINFO register so that backend i915
driver can access this shared page meory without any trap cost with the
help form hyperviser's read guest gpa functionality.

guest i915 will send VGT_G2V_SHARED_PAGE_SETUP notification to host GVT
once shared memory setup finished.

the layout of the shared_page also defined as well in this patch which
is used for pv features implementation.

v0: RFC.
v1: addressed RFC comment to move both shared_page_lock and shared_page
to i915_virtual_gpu structure.
v2: packed i915_virtual_gpu structure.
v3: added SHARED_PAGE_SETUP g2v notification for pv shared_page setup
v4: added intel_vgpu_setup_shared_page() in i915_vgpu_pv.c.
v5: per engine desc data in shared memory.
v6: added version support in shared memory (Zhenyu).
v7: added new function i915_destroy_vgpu to free shared memory.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/i915_drv.c    |  2 +
 drivers/gpu/drm/i915/i915_drv.h    |  4 +-
 drivers/gpu/drm/i915/i915_pvinfo.h |  5 +-
 drivers/gpu/drm/i915/i915_vgpu.c   | 95 ++++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_vgpu.h   | 18 ++++++++
 5 files changed, 122 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index 0dfcb40..b1f95d0 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -1637,6 +1637,8 @@ static void i915_driver_release(struct drm_device *dev)
 
 	disable_rpm_wakeref_asserts(rpm);
 
+	i915_destroy_vgpu(dev_priv);
+
 	i915_gem_driver_release(dev_priv);
 
 	i915_ggtt_driver_release(dev_priv);
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 19fbbef..b3c0849 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -973,7 +973,9 @@ struct i915_virtual_gpu {
 	bool active;
 	u32 caps;
 	u32 pv_caps;
-};
+
+	struct i915_virtual_gpu_pv *pv;
+} __packed;
 
 /* used in computing the new watermarks state */
 struct intel_wm_config {
diff --git a/drivers/gpu/drm/i915/i915_pvinfo.h b/drivers/gpu/drm/i915/i915_pvinfo.h
index ad398b4..3c63603 100644
--- a/drivers/gpu/drm/i915/i915_pvinfo.h
+++ b/drivers/gpu/drm/i915/i915_pvinfo.h
@@ -48,6 +48,7 @@ enum vgt_g2v_type {
 	VGT_G2V_PPGTT_L4_PAGE_TABLE_DESTROY,
 	VGT_G2V_EXECLIST_CONTEXT_CREATE,
 	VGT_G2V_EXECLIST_CONTEXT_DESTROY,
+	VGT_G2V_SHARED_PAGE_SETUP,
 	VGT_G2V_MAX,
 };
 
@@ -112,7 +113,9 @@ struct vgt_if {
 
 	u32 pv_caps;
 
-	u32  rsv7[0x200 - 25];    /* pad to one page */
+	u64 shared_page_gpa;
+
+	u32  rsv7[0x200 - 27];    /* pad to one page */
 } __packed;
 
 #define vgtif_offset(x) (offsetof(struct vgt_if, x))
diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
index 981959b..68fecfd 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.c
+++ b/drivers/gpu/drm/i915/i915_vgpu.c
@@ -107,6 +107,17 @@ void i915_detect_vgpu(struct drm_i915_private *dev_priv)
 	pci_iounmap(pdev, shared_area);
 }
 
+void i915_destroy_vgpu(struct drm_i915_private *dev_priv)
+{
+	struct i915_virtual_gpu_pv *pv = dev_priv->vgpu.pv;
+
+	if (!intel_vgpu_active(dev_priv) || !pv)
+		return;
+
+	__free_page(virt_to_page(pv->shared_page));
+	kfree(pv);
+}
+
 bool intel_vgpu_has_full_ppgtt(struct drm_i915_private *dev_priv)
 {
 	return dev_priv->vgpu.caps & VGT_CAPS_FULL_PPGTT;
@@ -311,6 +322,84 @@ int intel_vgt_balloon(struct i915_ggtt *ggtt)
  * i915 vgpu PV support for Linux
  */
 
+/*
+ * shared_page setup for VGPU PV features
+ */
+static int intel_vgpu_setup_shared_page(struct drm_i915_private *dev_priv,
+		void __iomem *shared_area)
+{
+	void __iomem *addr;
+	struct i915_virtual_gpu_pv *pv;
+	struct gvt_shared_page *base;
+	u64 gpa;
+	u16 ver_maj, ver_min;
+	int ret = 0;
+
+	/* We allocate 1 page shared between guest and GVT for data exchange.
+	 *       ___________.....................
+	 *      |head       |                   |
+	 *      |___________|.................. PAGE/8
+	 *      |PV ELSP                        |
+	 *      :___________....................PAGE/4
+	 *      |desc (SEND)                    |
+	 *      |				|
+	 *      :_______________________________PAGE/2
+	 *      |cmds (SEND)                    |
+	 *      |                               |
+	 *      |                               |
+	 *      |                               |
+	 *      |                               |
+	 *      |_______________________________|
+	 *
+	 * 0 offset: PV version area
+	 * PAGE/8 offset: per engine workload submission data area
+	 * PAGE/4 offset: PV command buffer command descriptor area
+	 * PAGE/2 offset: PV command buffer command data area
+	 */
+
+	base =  (struct gvt_shared_page *)get_zeroed_page(GFP_KERNEL);
+	if (!base) {
+		dev_info(dev_priv->drm.dev, "out of memory for shared memory\n");
+		return -ENOMEM;
+	}
+
+	/* pass guest memory pa address to GVT and then read back to verify */
+	gpa = __pa(base);
+	addr = shared_area + vgtif_offset(shared_page_gpa);
+	writeq(gpa, addr);
+	if (gpa != readq(addr)) {
+		dev_info(dev_priv->drm.dev, "passed shared_page_gpa failed\n");
+		ret = -EIO;
+		goto err;
+	}
+
+	addr = shared_area + vgtif_offset(g2v_notify);
+	writel(VGT_G2V_SHARED_PAGE_SETUP, addr);
+
+	ver_maj = base->ver_major;
+	ver_min = base->ver_minor;
+	if (ver_maj != PV_MAJOR || ver_min != PV_MINOR) {
+		dev_info(dev_priv->drm.dev, "VGPU PV version incompatible\n");
+		ret = -EIO;
+		goto err;
+	}
+
+	pv = kzalloc(sizeof(struct i915_virtual_gpu_pv), GFP_KERNEL);
+	if (!pv) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
+	DRM_INFO("vgpu PV ver major %d and minor %d\n", ver_maj, ver_min);
+	dev_priv->vgpu.pv = pv;
+	pv->shared_page = base;
+	pv->enabled = true;
+	return ret;
+err:
+	__free_page(virt_to_page(base));
+	return ret;
+}
+
 /**
  * intel_vgpu_check_pv_caps - detect virtual GPU PV capabilities
  * @dev_priv: i915 device private
@@ -339,5 +428,11 @@ bool intel_vgpu_check_pv_caps(struct drm_i915_private *dev_priv,
 	if (!pvcaps)
 		return false;
 
+	if (intel_vgpu_setup_shared_page(dev_priv, shared_area)) {
+		dev_priv->vgpu.pv_caps = 0;
+		writel(0, shared_area + vgtif_offset(pv_caps));
+		return false;
+	}
+
 	return true;
 }
diff --git a/drivers/gpu/drm/i915/i915_vgpu.h b/drivers/gpu/drm/i915/i915_vgpu.h
index bbe56b5..401af24 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.h
+++ b/drivers/gpu/drm/i915/i915_vgpu.h
@@ -27,7 +27,25 @@
 #include "i915_drv.h"
 #include "i915_pvinfo.h"
 
+#define PV_MAJOR		1
+#define PV_MINOR		0
+
+/*
+ * A shared page(4KB) between gvt and VM, could be allocated by guest driver
+ * or a fixed location in PCI bar 0 region
+ */
+struct gvt_shared_page {
+	u16 ver_major;
+	u16 ver_minor;
+};
+
+struct i915_virtual_gpu_pv {
+	struct gvt_shared_page *shared_page;
+	bool enabled;
+};
+
 void i915_detect_vgpu(struct drm_i915_private *dev_priv);
+void i915_destroy_vgpu(struct drm_i915_private *dev_priv);
 
 bool intel_vgpu_has_full_ppgtt(struct drm_i915_private *dev_priv);
 
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 3/9] drm/i915: vgpu pv command buffer support

[Xiaolin Zhang]

based on the shared memory setup between guest and GVT, the simple
pv command buffer ring was introduced by this patch used to perform
guest-2-gvt single direction communication.

v1: initial support, added to address i915 PV v6 patch set comment.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/i915_pvinfo.h |   1 +
 drivers/gpu/drm/i915/i915_vgpu.c   | 193 +++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_vgpu.h   |  66 +++++++++++++
 3 files changed, 260 insertions(+)

diff --git a/drivers/gpu/drm/i915/i915_pvinfo.h b/drivers/gpu/drm/i915/i915_pvinfo.h
index 3c63603..db9eebb 100644
--- a/drivers/gpu/drm/i915/i915_pvinfo.h
+++ b/drivers/gpu/drm/i915/i915_pvinfo.h
@@ -49,6 +49,7 @@ enum vgt_g2v_type {
 	VGT_G2V_EXECLIST_CONTEXT_CREATE,
 	VGT_G2V_EXECLIST_CONTEXT_DESTROY,
 	VGT_G2V_SHARED_PAGE_SETUP,
+	VGT_G2V_PV_SEND_TRIGGER,
 	VGT_G2V_MAX,
 };
 
diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
index 68fecfd..6e29a52 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.c
+++ b/drivers/gpu/drm/i915/i915_vgpu.c
@@ -322,6 +322,187 @@ int intel_vgt_balloon(struct i915_ggtt *ggtt)
  * i915 vgpu PV support for Linux
  */
 
+/**
+ * wait_for_desc_update - Wait for the command buffer descriptor update.
+ * @desc:	buffer descriptor
+ * @fence:	response fence
+ * @status:	placeholder for status
+ *
+ * GVT will update command buffer descriptor with new fence and status
+ * after processing the command identified by the fence. Wait for
+ * specified fence and then read from the descriptor status of the
+ * command.
+ *
+ * Return:
+ * *	0 response received (status is valid)
+ * *	-ETIMEDOUT no response within hardcoded timeout
+ * *	-EPROTO no response, CT buffer is in error
+ */
+static int wait_for_desc_update(struct vgpu_pv_ct_buffer_desc *desc,
+		u32 fence, u32 *status)
+{
+	int err;
+
+#define done (READ_ONCE(desc->fence) == fence)
+	err = wait_for_us(done, 5);
+	if (err)
+		err = wait_for(done, 10);
+#undef done
+
+	if (unlikely(err)) {
+		DRM_ERROR("CT: fence %u failed; reported fence=%u\n",
+				fence, desc->fence);
+	}
+
+	*status = desc->status;
+	return err;
+}
+
+/**
+ * DOC: CTB Guest to GVT request
+ *
+ * Format of the CTB Guest to GVT request message is as follows::
+ *
+ *      +------------+---------+---------+---------+---------+
+ *      |   msg[0]   |   [1]   |   [2]   |   ...   |  [n-1]  |
+ *      +------------+---------+---------+---------+---------+
+ *      |   MESSAGE  |       MESSAGE PAYLOAD                 |
+ *      +   HEADER   +---------+---------+---------+---------+
+ *      |            |    0    |    1    |   ...   |    n    |
+ *      +============+=========+=========+=========+=========+
+ *      |  len >= 1  |  FENCE  |     request specific data   |
+ *      +------+-----+---------+---------+---------+---------+
+ *
+ *                   ^-----------------len-------------------^
+ */
+static int pv_command_buffer_write(struct i915_virtual_gpu_pv *pv,
+		const u32 *action, u32 len /* in dwords */, u32 fence)
+{
+	struct vgpu_pv_ct_buffer_desc *desc = pv->ctb.desc;
+	u32 head = desc->head / 4;	/* in dwords */
+	u32 tail = desc->tail / 4;	/* in dwords */
+	u32 size = desc->size / 4;	/* in dwords */
+	u32 used;			/* in dwords */
+	u32 header;
+	u32 *cmds = pv->ctb.cmds;
+	unsigned int i;
+
+	GEM_BUG_ON(desc->size % 4);
+	GEM_BUG_ON(desc->head % 4);
+	GEM_BUG_ON(desc->tail % 4);
+	GEM_BUG_ON(tail >= size);
+
+	/*
+	 * tail == head condition indicates empty.
+	 */
+	if (tail < head)
+		used = (size - head) + tail;
+	else
+		used = tail - head;
+
+	/* make sure there is a space including extra dw for the fence */
+	if (unlikely(used + len + 1 >= size))
+		return -ENOSPC;
+
+	/*
+	 * Write the message. The format is the following:
+	 * DW0: header (including action code)
+	 * DW1: fence
+	 * DW2+: action data
+	 */
+	header = (len << PV_CT_MSG_LEN_SHIFT) |
+		 (PV_CT_MSG_WRITE_FENCE_TO_DESC) |
+		 (action[0] << PV_CT_MSG_ACTION_SHIFT);
+
+	cmds[tail] = header;
+	tail = (tail + 1) % size;
+
+	cmds[tail] = fence;
+	tail = (tail + 1) % size;
+
+	for (i = 1; i < len; i++) {
+		cmds[tail] = action[i];
+		tail = (tail + 1) % size;
+	}
+
+	/* now update desc tail (back in bytes) */
+	desc->tail = tail * 4;
+	GEM_BUG_ON(desc->tail > desc->size);
+
+	return 0;
+}
+
+static u32 pv_get_next_fence(struct i915_virtual_gpu_pv *pv)
+{
+	/* For now it's trivial */
+	return ++pv->next_fence;
+}
+
+static int pv_send(struct drm_i915_private *dev_priv,
+		const u32 *action, u32 len, u32 *status)
+{
+	struct i915_virtual_gpu *vgpu = &dev_priv->vgpu;
+	struct i915_virtual_gpu_pv *pv = vgpu->pv;
+
+	struct vgpu_pv_ct_buffer_desc *desc = pv->ctb.desc;
+
+	u32 fence;
+	int err;
+
+	GEM_BUG_ON(!pv->enabled);
+	GEM_BUG_ON(!len);
+	GEM_BUG_ON(len & ~PV_CT_MSG_LEN_MASK);
+
+	fence = pv_get_next_fence(pv);
+	err = pv_command_buffer_write(pv, action, len, fence);
+	if (unlikely(err))
+		goto unlink;
+
+	intel_vgpu_pv_notify(dev_priv);
+
+	err = wait_for_desc_update(desc, fence, status);
+	if (unlikely(err))
+		goto unlink;
+
+	if ((*status)) {
+		err = -EIO;
+		goto unlink;
+	}
+
+	err = (*status);
+unlink:
+	return err;
+}
+
+static int intel_vgpu_pv_send_command_buffer(
+		struct drm_i915_private *dev_priv,
+		u32 *action, u32 len)
+{
+	struct i915_virtual_gpu *vgpu = &dev_priv->vgpu;
+
+	u32 status = ~0; /* undefined */
+	int ret;
+
+	mutex_lock(&vgpu->pv->send_mutex);
+
+	ret = pv_send(dev_priv, action, len, &status);
+	if (unlikely(ret < 0)) {
+		DRM_ERROR("PV: send action %#X failed; err=%d status=%#X\n",
+			  action[0], ret, status);
+	} else if (unlikely(ret)) {
+		DRM_ERROR("PV: send action %#x returned %d (%#x)\n",
+				action[0], ret, ret);
+	}
+
+	mutex_unlock(&vgpu->pv->send_mutex);
+	return ret;
+}
+
+static void intel_vgpu_pv_notify_mmio(struct drm_i915_private *dev_priv)
+{
+	I915_WRITE(vgtif_reg(g2v_notify), VGT_G2V_PV_SEND_TRIGGER);
+}
+
 /*
  * shared_page setup for VGPU PV features
  */
@@ -394,6 +575,18 @@ static int intel_vgpu_setup_shared_page(struct drm_i915_private *dev_priv,
 	dev_priv->vgpu.pv = pv;
 	pv->shared_page = base;
 	pv->enabled = true;
+
+	/* setup PV command buffer ptr */
+	pv->ctb.cmds = (void *)base + PV_CMD_OFF;
+	pv->ctb.desc = (void *)base + PV_DESC_OFF;
+	pv->ctb.desc->size = PAGE_SIZE/2;
+	pv->ctb.desc->addr = PV_CMD_OFF;
+
+	/* setup PV command buffer callback */
+	pv->send = intel_vgpu_pv_send_command_buffer;
+	pv->notify = intel_vgpu_pv_notify_mmio;
+	mutex_init(&pv->send_mutex);
+
 	return ret;
 err:
 	__free_page(virt_to_page(base));
diff --git a/drivers/gpu/drm/i915/i915_vgpu.h b/drivers/gpu/drm/i915/i915_vgpu.h
index 401af24..c494beb 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.h
+++ b/drivers/gpu/drm/i915/i915_vgpu.h
@@ -29,6 +29,8 @@
 
 #define PV_MAJOR		1
 #define PV_MINOR		0
+#define PV_DESC_OFF		(PAGE_SIZE/4)
+#define PV_CMD_OFF		(PAGE_SIZE/2)
 
 /*
  * A shared page(4KB) between gvt and VM, could be allocated by guest driver
@@ -39,9 +41,60 @@ struct gvt_shared_page {
 	u16 ver_minor;
 };
 
+/*
+ * Definition of the command transport message header (DW0)
+ *
+ * bit[4..0]	message len (in dwords)
+ * bit[7..5]	reserved
+ * bit[8]		write fence to desc
+ * bit[9..11]	reserved
+ * bit[31..16]	action code
+ */
+#define PV_CT_MSG_LEN_SHIFT				0
+#define PV_CT_MSG_LEN_MASK				0x1F
+#define PV_CT_MSG_WRITE_FENCE_TO_DESC	(1 << 8)
+#define PV_CT_MSG_ACTION_SHIFT			16
+#define PV_CT_MSG_ACTION_MASK			0xFFFF
+
+/* PV command transport buffer descriptor */
+struct vgpu_pv_ct_buffer_desc {
+	u32 addr;		/* gfx address */
+	u32 size;		/* size in bytes */
+	u32 head;		/* offset updated by GVT */
+	u32 tail;		/* offset updated by owner */
+
+	u32 fence;		/* fence updated by GVT */
+	u32 status;		/* status updated by GVT */
+} __packed;
+
+/** PV single command transport buffer.
+ *
+ * A single command transport buffer consists of two parts, the header
+ * record (command transport buffer descriptor) and the actual buffer which
+ * holds the commands.
+ *
+ * @desc: pointer to the buffer descriptor
+ * @cmds: pointer to the commands buffer
+ */
+struct vgpu_pv_ct_buffer {
+	struct vgpu_pv_ct_buffer_desc *desc;
+	u32 *cmds;
+};
+
 struct i915_virtual_gpu_pv {
 	struct gvt_shared_page *shared_page;
 	bool enabled;
+
+	/* PV command buffer support */
+	struct vgpu_pv_ct_buffer ctb;
+	u32 next_fence;
+
+	/* To serialize the vgpu PV send actions */
+	struct mutex send_mutex;
+
+	/* VGPU's PV specific send function */
+	int (*send)(struct drm_i915_private *dev_priv, u32 *data, u32 len);
+	void (*notify)(struct drm_i915_private *dev_priv);
 };
 
 void i915_detect_vgpu(struct drm_i915_private *dev_priv);
@@ -67,6 +120,19 @@ struct i915_virtual_gpu_pv {
 	return dev_priv->vgpu.caps & VGT_CAPS_PV;
 }
 
+static inline void
+intel_vgpu_pv_notify(struct drm_i915_private *dev_priv)
+{
+	dev_priv->vgpu.pv->notify(dev_priv);
+}
+
+static inline int
+intel_vgpu_pv_send(struct drm_i915_private *dev_priv,
+		u32 *action, u32 len)
+{
+	return dev_priv->vgpu.pv->send(dev_priv, action, len);
+}
+
 int intel_vgt_balloon(struct i915_ggtt *ggtt);
 void intel_vgt_deballoon(struct i915_ggtt *ggtt);
 
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 4/9] drm/i915: vgpu ppgtt update pv optimization

[Xiaolin Zhang]

This patch extends vgpu ppgtt g2v notification to notify host
GVT-g of ppgtt update from guest including alloc_4lvl, clear_4lv4
and insert_4lvl.

These updates use the shared memory page to pass struct pv_ppgtt_update
from guest to GVT which is used for pv optimiation implemeation within
host GVT side.

This patch also add one new pv_caps level to control ppgtt update.

Use PV_PPGTT_UPDATE to control this level of pv optimization.

v0: RFC.
v1: rebased.
v2: added pv callbacks for vm.{allocate_va_range, insert_entries,
clear_range} within ppgtt.
v3: rebased, disable huge page ppgtt support when using PVMMIO ppgtt
update due to complex and performance impact.
v4: moved alloc/insert/clear_4lvl pv callbacks into i915_vgpu_pv.c and
added a single intel_vgpu_config_pv_caps() for vgpu pv callbacks setup.
v5: rebase.
v6: rebase.
v7: rebase.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/i915_gem.c     |  3 +-
 drivers/gpu/drm/i915/i915_gem_gtt.c | 10 +++--
 drivers/gpu/drm/i915/i915_gem_gtt.h |  8 ++++
 drivers/gpu/drm/i915/i915_vgpu.c    | 79 +++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_vgpu.h    | 25 ++++++++++++
 5 files changed, 120 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 2da9544..1ade8e0 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1309,7 +1309,8 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
 	int ret;
 
 	/* We need to fallback to 4K pages if host doesn't support huge gtt. */
-	if (intel_vgpu_active(dev_priv) && !intel_vgpu_has_huge_gtt(dev_priv))
+	if ((intel_vgpu_active(dev_priv) && !intel_vgpu_has_huge_gtt(dev_priv))
+		|| intel_vgpu_enabled_pv_caps(dev_priv, PV_PPGTT_UPDATE))
 		mkwrite_device_info(dev_priv)->page_sizes =
 			I915_GTT_PAGE_SIZE_4K;
 
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index e62e9d1..00b187a 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -1015,7 +1015,7 @@ static u64 __gen8_ppgtt_clear(struct i915_address_space * const vm,
 	return start;
 }
 
-static void gen8_ppgtt_clear(struct i915_address_space *vm,
+void gen8_ppgtt_clear(struct i915_address_space *vm,
 			     u64 start, u64 length)
 {
 	GEM_BUG_ON(!IS_ALIGNED(start, BIT_ULL(GEN8_PTE_SHIFT)));
@@ -1126,7 +1126,7 @@ static int __gen8_ppgtt_alloc(struct i915_address_space * const vm,
 	return ret;
 }
 
-static int gen8_ppgtt_alloc(struct i915_address_space *vm,
+int gen8_ppgtt_alloc(struct i915_address_space *vm,
 			    u64 start, u64 length)
 {
 	u64 from;
@@ -1326,7 +1326,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 	} while (iter->sg);
 }
 
-static void gen8_ppgtt_insert(struct i915_address_space *vm,
+void gen8_ppgtt_insert(struct i915_address_space *vm,
 			      struct i915_vma *vma,
 			      enum i915_cache_level cache_level,
 			      u32 flags)
@@ -1517,8 +1517,10 @@ static struct i915_ppgtt *gen8_ppgtt_create(struct drm_i915_private *i915)
 	ppgtt->vm.allocate_va_range = gen8_ppgtt_alloc;
 	ppgtt->vm.clear_range = gen8_ppgtt_clear;
 
-	if (intel_vgpu_active(i915))
+	if (intel_vgpu_active(i915)) {
+		intel_vgpu_config_pv_caps(i915, PV_PPGTT_UPDATE, ppgtt);
 		gen8_ppgtt_notify_vgt(ppgtt, true);
+	}
 
 	ppgtt->vm.cleanup = gen8_ppgtt_cleanup;
 
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index 8fd2234..f5536bf 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -588,6 +588,14 @@ static inline void i915_vm_put(struct i915_address_space *vm)
 void gen6_ppgtt_unpin(struct i915_ppgtt *base);
 void gen6_ppgtt_unpin_all(struct i915_ppgtt *base);
 
+void gen8_ppgtt_clear(struct i915_address_space *vm,
+		u64 start, u64 length);
+void gen8_ppgtt_insert(struct i915_address_space *vm,
+		struct i915_vma *vma,
+		enum i915_cache_level cache_level, u32 flags);
+int gen8_ppgtt_alloc(struct i915_address_space *vm,
+		u64 start, u64 length);
+
 void i915_gem_suspend_gtt_mappings(struct drm_i915_private *dev_priv);
 void i915_gem_restore_gtt_mappings(struct drm_i915_private *dev_priv);
 
diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
index 6e29a52..e458892 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.c
+++ b/drivers/gpu/drm/i915/i915_vgpu.c
@@ -96,6 +96,9 @@ void i915_detect_vgpu(struct drm_i915_private *dev_priv)
 	dev_priv->vgpu.active = true;
 	mutex_init(&dev_priv->vgpu.lock);
 
+	/* guest driver PV capability */
+	dev_priv->vgpu.pv_caps = PV_PPGTT_UPDATE;
+
 	if (!intel_vgpu_check_pv_caps(dev_priv, shared_area)) {
 		DRM_INFO("Virtual GPU for Intel GVT-g detected.\n");
 		goto out;
@@ -322,6 +325,82 @@ int intel_vgt_balloon(struct i915_ggtt *ggtt)
  * i915 vgpu PV support for Linux
  */
 
+static int vgpu_ppgtt_pv_update(struct drm_i915_private *dev_priv,
+		u32 action, u64 pdp, u64 start, u64 length, u32 cache_level)
+{
+	u32 data[8];
+
+	data[0] = action;
+	data[1] = lower_32_bits(pdp);
+	data[2] = upper_32_bits(pdp);
+	data[3] = lower_32_bits(start);
+	data[4] = upper_32_bits(start);
+	data[5] = lower_32_bits(length);
+	data[6] = upper_32_bits(length);
+	data[7] = cache_level;
+
+	return intel_vgpu_pv_send(dev_priv, data, ARRAY_SIZE(data));
+}
+
+static void gen8_ppgtt_clear_4lvl_pv(struct i915_address_space *vm,
+		u64 start, u64 length)
+{
+	struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
+	struct drm_i915_private *dev_priv = vm->i915;
+
+	gen8_ppgtt_clear(vm, start, length);
+	vgpu_ppgtt_pv_update(dev_priv, PV_ACTION_PPGTT_L4_CLEAR,
+		px_dma(ppgtt->pd), start, length, 0);
+}
+
+static void gen8_ppgtt_insert_4lvl_pv(struct i915_address_space *vm,
+		struct i915_vma *vma,
+		enum i915_cache_level cache_level, u32 flags)
+{
+	struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
+	struct drm_i915_private *dev_priv = vm->i915;
+	u64 start = vma->node.start;
+	u64 length = vma->node.size;
+
+	gen8_ppgtt_insert(vm, vma, cache_level, flags);
+	vgpu_ppgtt_pv_update(dev_priv, PV_ACTION_PPGTT_L4_INSERT,
+		px_dma(ppgtt->pd), start, length, cache_level);
+}
+
+static int gen8_ppgtt_alloc_4lvl_pv(struct i915_address_space *vm,
+		u64 start, u64 length)
+{
+	struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
+	struct drm_i915_private *dev_priv = vm->i915;
+	int ret;
+
+	ret = gen8_ppgtt_alloc(vm, start, length);
+	if (ret)
+		return ret;
+
+	return vgpu_ppgtt_pv_update(dev_priv, PV_ACTION_PPGTT_L4_ALLOC,
+		px_dma(ppgtt->pd), start, length, 0);
+}
+
+/*
+ * config guest driver PV ops for different PV features
+ */
+void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
+		enum pv_caps cap, void *data)
+{
+	struct i915_ppgtt *ppgtt;
+
+	if (!intel_vgpu_enabled_pv_caps(dev_priv, cap))
+		return;
+
+	if (cap == PV_PPGTT_UPDATE) {
+		ppgtt = (struct i915_ppgtt *)data;
+		ppgtt->vm.allocate_va_range = gen8_ppgtt_alloc_4lvl_pv;
+		ppgtt->vm.insert_entries = gen8_ppgtt_insert_4lvl_pv;
+		ppgtt->vm.clear_range = gen8_ppgtt_clear_4lvl_pv;
+	}
+}
+
 /**
  * wait_for_desc_update - Wait for the command buffer descriptor update.
  * @desc:	buffer descriptor
diff --git a/drivers/gpu/drm/i915/i915_vgpu.h b/drivers/gpu/drm/i915/i915_vgpu.h
index c494beb..b0fee5b 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.h
+++ b/drivers/gpu/drm/i915/i915_vgpu.h
@@ -33,6 +33,21 @@
 #define PV_CMD_OFF		(PAGE_SIZE/2)
 
 /*
+ * define different capabilities of PV optimization
+ */
+enum pv_caps {
+	PV_PPGTT_UPDATE = 0x1,
+};
+
+/* PV actions */
+enum intel_vgpu_pv_action {
+	PV_ACTION_DEFAULT = 0x0,
+	PV_ACTION_PPGTT_L4_ALLOC,
+	PV_ACTION_PPGTT_L4_CLEAR,
+	PV_ACTION_PPGTT_L4_INSERT,
+};
+
+/*
  * A shared page(4KB) between gvt and VM, could be allocated by guest driver
  * or a fixed location in PCI bar 0 region
  */
@@ -120,6 +135,14 @@ struct i915_virtual_gpu_pv {
 	return dev_priv->vgpu.caps & VGT_CAPS_PV;
 }
 
+static inline bool
+intel_vgpu_enabled_pv_caps(struct drm_i915_private *dev_priv,
+		enum pv_caps cap)
+{
+	return (dev_priv->vgpu.active) && intel_vgpu_has_pv_caps(dev_priv)
+			&& (dev_priv->vgpu.pv_caps & cap);
+}
+
 static inline void
 intel_vgpu_pv_notify(struct drm_i915_private *dev_priv)
 {
@@ -139,4 +162,6 @@ struct i915_virtual_gpu_pv {
 /* i915 vgpu pv related functions */
 bool intel_vgpu_check_pv_caps(struct drm_i915_private *dev_priv,
 		void __iomem *shared_area);
+void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
+		enum pv_caps cap, void *data);
 #endif /* _I915_VGPU_H_ */
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 5/9] drm/i915: vgpu context submission pv optimization

[Xiaolin Zhang]

It is performance optimization to override the actual submisison backend
in order to eliminate execlists csb process and reduce mmio trap numbers
for workload submission without context switch interrupt by talking with
GVT via PV submisison notification mechanism between guest and GVT.

Use PV_SUBMISSION to control this level of pv optimization.

v0: RFC.
v1: rebase.
v2: added pv ops for pv context submission. to maximize code resuse,
introduced 2 more ops (submit_ports & preempt_context) instead of 1 op
(set_default_submission) in engine structure. pv version of
submit_ports and preempt_context implemented.
v3:
1. to reduce more code duplication, code refactor and replaced 2 ops
"submit_ports & preempt_contex" from v2 by 1 ops "write_desc"
in engine structure. pv version of write_des implemented.
2. added VGT_G2V_ELSP_SUBMIT for g2v pv notification.
v4: implemented pv elsp submission tasklet as the backend workload
submisison by talking to GVT with PV notificaiton mechanism and renamed
VGT_G2V_ELSP_SUBMIT to VGT_G2V_PV_SUBMISIION.
v5: addressed v4 comments from Chris, intel_pv_submission.c added.
v6: addressed v5 comments from Chris, replaced engine id by hw_id.
v7: rebase.
v8: addressed v7 comments from Chris.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/Makefile              |   2 +-
 drivers/gpu/drm/i915/gt/intel_lrc.c        |  12 +-
 drivers/gpu/drm/i915/i915_vgpu.c           |  18 +-
 drivers/gpu/drm/i915/i915_vgpu.h           |  15 ++
 drivers/gpu/drm/i915/intel_pv_submission.c | 300 +++++++++++++++++++++++++++++
 5 files changed, 341 insertions(+), 6 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/intel_pv_submission.c

diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index 658b930..f500cf1 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -250,7 +250,7 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \
 	selftests/igt_spinner.o
 
 # virtual gpu code
-i915-y += i915_vgpu.o
+i915-y += i915_vgpu.o intel_pv_submission.o
 
 ifeq ($(CONFIG_DRM_I915_GVT),y)
 i915-y += intel_gvt.o
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index a3f0e49..5587aff 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -2933,10 +2933,14 @@ void intel_execlists_set_default_submission(struct intel_engine_cs *engine)
 	engine->unpark = NULL;
 
 	engine->flags |= I915_ENGINE_SUPPORTS_STATS;
-	if (!intel_vgpu_active(engine->i915)) {
-		engine->flags |= I915_ENGINE_HAS_SEMAPHORES;
-		if (HAS_LOGICAL_RING_PREEMPTION(engine->i915))
-			engine->flags |= I915_ENGINE_HAS_PREEMPTION;
+	engine->flags |= I915_ENGINE_HAS_SEMAPHORES;
+	if (HAS_LOGICAL_RING_PREEMPTION(engine->i915))
+		engine->flags |= I915_ENGINE_HAS_PREEMPTION;
+
+	if (intel_vgpu_active(engine->i915)) {
+		engine->flags &= ~I915_ENGINE_HAS_SEMAPHORES;
+		engine->flags &= ~I915_ENGINE_HAS_PREEMPTION;
+		intel_vgpu_config_pv_caps(engine->i915,	PV_SUBMISSION, engine);
 	}
 
 	if (INTEL_GEN(engine->i915) >= 12) /* XXX disabled for debugging */
diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
index e458892..a488b68 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.c
+++ b/drivers/gpu/drm/i915/i915_vgpu.c
@@ -97,7 +97,7 @@ void i915_detect_vgpu(struct drm_i915_private *dev_priv)
 	mutex_init(&dev_priv->vgpu.lock);
 
 	/* guest driver PV capability */
-	dev_priv->vgpu.pv_caps = PV_PPGTT_UPDATE;
+	dev_priv->vgpu.pv_caps = PV_PPGTT_UPDATE | PV_SUBMISSION;
 
 	if (!intel_vgpu_check_pv_caps(dev_priv, shared_area)) {
 		DRM_INFO("Virtual GPU for Intel GVT-g detected.\n");
@@ -389,6 +389,7 @@ void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
 		enum pv_caps cap, void *data)
 {
 	struct i915_ppgtt *ppgtt;
+	struct intel_engine_cs *engine;
 
 	if (!intel_vgpu_enabled_pv_caps(dev_priv, cap))
 		return;
@@ -399,6 +400,11 @@ void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
 		ppgtt->vm.insert_entries = gen8_ppgtt_insert_4lvl_pv;
 		ppgtt->vm.clear_range = gen8_ppgtt_clear_4lvl_pv;
 	}
+
+	if (cap == PV_SUBMISSION) {
+		engine = (struct intel_engine_cs *)data;
+		vgpu_set_pv_submission(engine);
+	}
 }
 
 /**
@@ -594,6 +600,8 @@ static int intel_vgpu_setup_shared_page(struct drm_i915_private *dev_priv,
 	u64 gpa;
 	u16 ver_maj, ver_min;
 	int ret = 0;
+	int i;
+	u32 size;
 
 	/* We allocate 1 page shared between guest and GVT for data exchange.
 	 *       ___________.....................
@@ -666,6 +674,14 @@ static int intel_vgpu_setup_shared_page(struct drm_i915_private *dev_priv,
 	pv->notify = intel_vgpu_pv_notify_mmio;
 	mutex_init(&pv->send_mutex);
 
+	/* setup PV per engine data exchange structure */
+	size = sizeof(struct pv_submission);
+	for (i = 0; i < PV_MAX_ENGINES_NUM; i++) {
+		pv->pv_elsp[i] = (void *)base + PV_ELSP_OFF +  size * i;
+		pv->pv_elsp[i]->submitted = false;
+		spin_lock_init(&pv->pv_elsp[i]->lock);
+	}
+
 	return ret;
 err:
 	__free_page(virt_to_page(base));
diff --git a/drivers/gpu/drm/i915/i915_vgpu.h b/drivers/gpu/drm/i915/i915_vgpu.h
index b0fee5b..c33cb05 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.h
+++ b/drivers/gpu/drm/i915/i915_vgpu.h
@@ -29,6 +29,8 @@
 
 #define PV_MAJOR		1
 #define PV_MINOR		0
+#define PV_MAX_ENGINES_NUM	(VECS1_HW + 1)
+#define PV_ELSP_OFF		(PAGE_SIZE/8)
 #define PV_DESC_OFF		(PAGE_SIZE/4)
 #define PV_CMD_OFF		(PAGE_SIZE/2)
 
@@ -37,6 +39,7 @@
  */
 enum pv_caps {
 	PV_PPGTT_UPDATE = 0x1,
+	PV_SUBMISSION = 0x2,
 };
 
 /* PV actions */
@@ -45,6 +48,7 @@ enum intel_vgpu_pv_action {
 	PV_ACTION_PPGTT_L4_ALLOC,
 	PV_ACTION_PPGTT_L4_CLEAR,
 	PV_ACTION_PPGTT_L4_INSERT,
+	PV_ACTION_ELSP_SUBMISSION,
 };
 
 /*
@@ -56,6 +60,13 @@ struct gvt_shared_page {
 	u16 ver_minor;
 };
 
+/* workload submission pv support data structure */
+struct pv_submission {
+	u64 descs[EXECLIST_MAX_PORTS];
+	bool submitted;
+	spinlock_t lock;
+};
+
 /*
  * Definition of the command transport message header (DW0)
  *
@@ -100,6 +111,9 @@ struct i915_virtual_gpu_pv {
 	struct gvt_shared_page *shared_page;
 	bool enabled;
 
+	/* per engine PV workload submission data */
+	struct pv_submission *pv_elsp[PV_MAX_ENGINES_NUM];
+
 	/* PV command buffer support */
 	struct vgpu_pv_ct_buffer ctb;
 	u32 next_fence;
@@ -164,4 +178,5 @@ bool intel_vgpu_check_pv_caps(struct drm_i915_private *dev_priv,
 		void __iomem *shared_area);
 void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
 		enum pv_caps cap, void *data);
+void vgpu_set_pv_submission(struct intel_engine_cs *engine);
 #endif /* _I915_VGPU_H_ */
diff --git a/drivers/gpu/drm/i915/intel_pv_submission.c b/drivers/gpu/drm/i915/intel_pv_submission.c
new file mode 100644
index 0000000..1083d56
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_pv_submission.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2018 Intel Corporation
+ */
+
+#include "i915_vgpu.h"
+#include "gt/intel_lrc_reg.h"
+#include "gt/intel_gt_pm.h"
+#include "i915_trace.h"
+
+#define CTX_DESC_FORCE_RESTORE BIT_ULL(2)
+
+static u64 execlists_update_context(struct i915_request *rq)
+{
+	struct intel_context *ce = rq->hw_context;
+	u32 *reg_state = ce->lrc_reg_state;
+
+	reg_state[CTX_RING_TAIL + 1] = intel_ring_set_tail(rq->ring, rq->tail);
+
+	return ce->lrc_desc;
+}
+
+static inline struct i915_priolist *to_priolist(struct rb_node *rb)
+{
+	return rb_entry(rb, struct i915_priolist, node);
+}
+
+static void pv_submit(struct intel_engine_cs *engine,
+	       struct i915_request **out,
+	       struct i915_request **end)
+{
+	struct intel_engine_execlists * const execlists = &engine->execlists;
+	struct i915_virtual_gpu_pv *pv = engine->i915->vgpu.pv;
+	struct pv_submission *pv_elsp = pv->pv_elsp[engine->hw_id];
+	struct i915_request *rq;
+	int n, err;
+
+	memset(pv_elsp->descs, 0, sizeof(pv_elsp->descs));
+	n = 0;
+
+	do {
+		rq = *out++;
+		pv_elsp->descs[n++] = execlists_update_context(rq);
+	} while (out != end);
+
+	spin_lock(&pv_elsp->lock);
+	pv_elsp->submitted = true;
+	writel(PV_ACTION_ELSP_SUBMISSION, execlists->submit_reg);
+
+#define done (READ_ONCE(pv_elsp->submitted) == false)
+	err = wait_for_atomic_us(done, 1000);
+#undef done
+	spin_unlock(&pv_elsp->lock);
+
+	if (unlikely(err))
+		DRM_ERROR("PV (%s) workload submission failed\n", engine->name);
+
+}
+
+static struct i915_request *schedule_in(struct i915_request *rq, int idx)
+{
+	trace_i915_request_in(rq, idx);
+
+	intel_gt_pm_get(rq->engine->gt);
+	return i915_request_get(rq);
+}
+
+static void pv_dequeue(struct intel_engine_cs *engine)
+{
+	struct intel_engine_execlists * const execlists = &engine->execlists;
+	struct i915_request **first = execlists->inflight;
+	struct i915_request ** const last_port = first + execlists->port_mask;
+	struct i915_request *last = first[0];
+	struct i915_request **port;
+	bool submit = false;
+	struct rb_node *rb;
+
+	lockdep_assert_held(&engine->active.lock);
+
+	if (last) {
+		if (*++first)
+			return;
+		last = NULL;
+	}
+
+	port = first;
+	while ((rb = rb_first_cached(&execlists->queue))) {
+		struct i915_priolist *p = to_priolist(rb);
+		struct i915_request *rq, *rn;
+		int i;
+
+		priolist_for_each_request_consume(rq, rn, p, i) {
+			if (last && rq->hw_context != last->hw_context) {
+				if (port == last_port)
+					goto done;
+
+				*port = schedule_in(last,
+						    port - execlists->inflight);
+				port++;
+			}
+
+
+			list_del_init(&rq->sched.link);
+			__i915_request_submit(rq);
+			submit = true;
+			last = rq;
+		}
+
+		rb_erase_cached(&p->node, &execlists->queue);
+		i915_priolist_free(p);
+	}
+done:
+	execlists->queue_priority_hint =
+		rb ? to_priolist(rb)->priority : INT_MIN;
+	if (submit) {
+		*port = schedule_in(last, port - execlists->inflight);
+		*++port = NULL;
+		pv_submit(engine, first, port);
+	}
+	execlists->active = execlists->inflight;
+}
+
+static void schedule_out(struct i915_request *rq)
+{
+	trace_i915_request_out(rq);
+
+	intel_gt_pm_put(rq->engine->gt);
+	i915_request_put(rq);
+}
+
+static void vgpu_pv_submission_tasklet(unsigned long data)
+{
+	struct intel_engine_cs * const engine = (struct intel_engine_cs *)data;
+	struct intel_engine_execlists * const execlists = &engine->execlists;
+	struct i915_request **port, *rq;
+	unsigned long flags;
+	struct i915_virtual_gpu_pv *pv = engine->i915->vgpu.pv;
+	struct pv_submission *pv_elsp = pv->pv_elsp[engine->hw_id];
+
+	spin_lock_irqsave(&engine->active.lock, flags);
+
+	for (port = execlists->inflight; (rq = *port); port++) {
+		if (!i915_request_completed(rq))
+			break;
+
+		schedule_out(rq);
+	}
+
+	if (port != execlists->inflight) {
+		int idx = port - execlists->inflight;
+		int rem = ARRAY_SIZE(execlists->inflight) - idx;
+
+		memmove(execlists->inflight, port, rem * sizeof(*port));
+	}
+
+	if (!pv_elsp->submitted)
+		pv_dequeue(engine);
+
+	spin_unlock_irqrestore(&engine->active.lock, flags);
+}
+
+static void pv_reset_prepare(struct intel_engine_cs *engine)
+{
+	struct intel_engine_execlists * const execlists = &engine->execlists;
+
+	GEM_TRACE("%s\n", engine->name);
+
+	/*
+	 * Prevent request submission to the hardware until we have
+	 * completed the reset in i915_gem_reset_finish(). If a request
+	 * is completed by one engine, it may then queue a request
+	 * to a second via its execlists->tasklet *just* as we are
+	 * calling engine->init_hw() and also writing the ELSP.
+	 * Turning off the execlists->tasklet until the reset is over
+	 * prevents the race.
+	 */
+	__tasklet_disable_sync_once(&execlists->tasklet);
+}
+
+static void
+cancel_port_requests(struct intel_engine_execlists * const execlists)
+{
+	struct i915_request * const *port, *rq;
+
+	/* Note we are only using the inflight and not the pending queue */
+	for (port = execlists->active; (rq = *port); port++)
+		schedule_out(rq);
+	execlists->active =
+		memset(execlists->inflight, 0, sizeof(execlists->inflight));
+}
+
+static void pv_reset(struct intel_engine_cs *engine, bool stalled)
+{
+	struct intel_engine_execlists * const execlists = &engine->execlists;
+	struct i915_request *rq;
+	unsigned long flags;
+
+	spin_lock_irqsave(&engine->active.lock, flags);
+
+	cancel_port_requests(execlists);
+
+	/* Push back any incomplete requests for replay after the reset. */
+	rq = execlists_unwind_incomplete_requests(execlists);
+	if (!rq)
+		goto out_unlock;
+
+	if (!i915_request_started(rq))
+		stalled = false;
+
+	__i915_request_reset(rq, stalled);
+	intel_lr_context_reset(engine, rq->hw_context, rq->head, stalled);
+
+out_unlock:
+	spin_unlock_irqrestore(&engine->active.lock, flags);
+}
+
+static void pv_reset_finish(struct intel_engine_cs *engine)
+{
+	struct intel_engine_execlists * const execlists = &engine->execlists;
+
+	if (__tasklet_enable(&execlists->tasklet))
+		/* And kick in case we missed a new request submission. */
+		tasklet_hi_schedule(&execlists->tasklet);
+
+	GEM_TRACE("%s: depth->%d\n", engine->name,
+		  atomic_read(&execlists->tasklet.count));
+}
+
+static void pv_cancel_requests(struct intel_engine_cs *engine)
+{
+	struct intel_engine_execlists * const execlists = &engine->execlists;
+	struct i915_request *rq, *rn;
+	struct rb_node *rb;
+	unsigned long flags;
+
+	GEM_TRACE("%s\n", engine->name);
+
+	spin_lock_irqsave(&engine->active.lock, flags);
+
+	/* Cancel the requests on the HW and clear the ELSP tracker. */
+	cancel_port_requests(execlists);
+
+	/* Mark all executing requests as skipped. */
+	list_for_each_entry(rq, &engine->active.requests, sched.link) {
+		if (!i915_request_signaled(rq))
+			dma_fence_set_error(&rq->fence, -EIO);
+
+		i915_request_mark_complete(rq);
+	}
+
+	/* Flush the queued requests to the timeline list (for retiring). */
+	while ((rb = rb_first_cached(&execlists->queue))) {
+		struct i915_priolist *p = to_priolist(rb);
+		int i;
+
+		priolist_for_each_request_consume(rq, rn, p, i) {
+			list_del_init(&rq->sched.link);
+			__i915_request_submit(rq);
+			dma_fence_set_error(&rq->fence, -EIO);
+			i915_request_mark_complete(rq);
+		}
+
+		rb_erase_cached(&p->node, &execlists->queue);
+		i915_priolist_free(p);
+	}
+
+	execlists->queue_priority_hint = INT_MIN;
+	execlists->queue = RB_ROOT_CACHED;
+
+	spin_unlock_irqrestore(&engine->active.lock, flags);
+}
+
+void vgpu_set_pv_submission(struct intel_engine_cs *engine)
+{
+	/*
+	 * We inherit a bunch of functions from execlists that we'd like
+	 * to keep using:
+	 *
+	 *    engine->submit_request = execlists_submit_request;
+	 *    engine->cancel_requests = execlists_cancel_requests;
+	 *    engine->schedule = execlists_schedule;
+	 *
+	 * But we need to override the actual submission backend in order
+	 * to talk to the GVT with PV notification message.
+	 */
+
+	engine->execlists.tasklet.func = vgpu_pv_submission_tasklet;
+
+	/* do not use execlists park/unpark */
+	engine->park = engine->unpark = NULL;
+
+	engine->reset.prepare = pv_reset_prepare;
+	engine->reset.reset = pv_reset;
+	engine->reset.finish = pv_reset_finish;
+
+	engine->cancel_requests = pv_cancel_requests;
+
+	engine->flags &= ~I915_ENGINE_SUPPORTS_STATS;
+	engine->flags |= I915_ENGINE_NEEDS_BREADCRUMB_TASKLET;
+}
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 6/9] drm/i915/gvt: GVTg handle pv_caps PVINFO register

[Xiaolin Zhang]

implement pv_caps PVINFO register handler in GVTg to
control different level pv optimization within guest.

report VGT_CAPS_PV capability in pvinfo page for guest.

v0: RFC.
v1: rebase.
v2: rebase.
v3: renamed enable_pvmmio to pvmmio_caps which is used for host
pv caps.
v4: renamed pvmmio_caps to pv_caps.
v5: rebase.
v6: rebase.
v7: rebase.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/gvt/handlers.c | 4 ++++
 drivers/gpu/drm/i915/gvt/vgpu.c     | 3 +++
 2 files changed, 7 insertions(+)

diff --git a/drivers/gpu/drm/i915/gvt/handlers.c b/drivers/gpu/drm/i915/gvt/handlers.c
index 45a9124..aceb16f 100644
--- a/drivers/gpu/drm/i915/gvt/handlers.c
+++ b/drivers/gpu/drm/i915/gvt/handlers.c
@@ -1194,6 +1194,7 @@ static int pvinfo_mmio_read(struct intel_vgpu *vgpu, unsigned int offset,
 		break;
 	case 0x78010:	/* vgt_caps */
 	case 0x7881c:
+	case _vgtif_reg(pv_caps):
 		break;
 	default:
 		invalid_read = true;
@@ -1264,6 +1265,9 @@ static int pvinfo_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
 	case _vgtif_reg(g2v_notify):
 		handle_g2v_notification(vgpu, data);
 		break;
+	case _vgtif_reg(pv_caps):
+		DRM_INFO("vgpu id=%d pv caps =0x%x\n", vgpu->id, data);
+		break;
 	/* add xhot and yhot to handled list to avoid error log */
 	case _vgtif_reg(cursor_x_hot):
 	case _vgtif_reg(cursor_y_hot):
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index d5a6e4e..9e00979 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -47,6 +47,7 @@ void populate_pvinfo_page(struct intel_vgpu *vgpu)
 	vgpu_vreg_t(vgpu, vgtif_reg(vgt_caps)) = VGT_CAPS_FULL_PPGTT;
 	vgpu_vreg_t(vgpu, vgtif_reg(vgt_caps)) |= VGT_CAPS_HWSP_EMULATION;
 	vgpu_vreg_t(vgpu, vgtif_reg(vgt_caps)) |= VGT_CAPS_HUGE_GTT;
+	vgpu_vreg_t(vgpu, vgtif_reg(vgt_caps)) |= VGT_CAPS_PV;
 
 	vgpu_vreg_t(vgpu, vgtif_reg(avail_rs.mappable_gmadr.base)) =
 		vgpu_aperture_gmadr_base(vgpu);
@@ -529,6 +530,7 @@ void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr,
 	struct intel_gvt *gvt = vgpu->gvt;
 	struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler;
 	intel_engine_mask_t resetting_eng = dmlr ? ALL_ENGINES : engine_mask;
+	int pv_caps = vgpu_vreg_t(vgpu, vgtif_reg(pv_caps));
 
 	gvt_dbg_core("------------------------------------------\n");
 	gvt_dbg_core("resseting vgpu%d, dmlr %d, engine_mask %08x\n",
@@ -560,6 +562,7 @@ void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr,
 
 		intel_vgpu_reset_mmio(vgpu, dmlr);
 		populate_pvinfo_page(vgpu);
+		vgpu_vreg_t(vgpu, vgtif_reg(pv_caps)) = pv_caps;
 		intel_vgpu_reset_display(vgpu);
 
 		if (dmlr) {
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 7/9] drm/i915/gvt: GVTg handle shared_page setup

[Xiaolin Zhang]

GVTg implemented shared_page setup operation and read_shared_page
functionality based on hypervisor_read_gpa().

the shared_page_gpa was passed from guest driver through PVINFO
shared_page_gpa register.

v0: RFC.
v1: rebase.
v2: rebase.
v3: added shared_page_gpa check and if read_gpa failure, return zero
memory and handle VGT_G2V_SHARED_PAGE_SETUP g2v notification
v4: rebase.
v5: rebase.
v6: rebase, added PV version support.
v7: rebase.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/gvt/gvt.h      |  8 ++++++-
 drivers/gpu/drm/i915/gvt/handlers.c | 20 +++++++++++++++++
 drivers/gpu/drm/i915/gvt/vgpu.c     | 43 +++++++++++++++++++++++++++++++++++++
 3 files changed, 70 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index b47c6ac..71213e0 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -49,6 +49,7 @@
 #include "fb_decoder.h"
 #include "dmabuf.h"
 #include "page_track.h"
+#include "i915_vgpu.h"
 
 #define GVT_MAX_VGPU 8
 
@@ -229,6 +230,8 @@ struct intel_vgpu {
 	struct completion vblank_done;
 
 	u32 scan_nonprivbb;
+	u64 shared_page_gpa;
+	bool shared_page_enabled;
 };
 
 /* validating GM healthy status*/
@@ -690,7 +693,10 @@ static inline void intel_gvt_mmio_set_in_ctx(
 void intel_gvt_debugfs_remove_vgpu(struct intel_vgpu *vgpu);
 void intel_gvt_debugfs_init(struct intel_gvt *gvt);
 void intel_gvt_debugfs_clean(struct intel_gvt *gvt);
-
+int intel_gvt_read_shared_page(struct intel_vgpu *vgpu,
+		unsigned int offset, void *buf, unsigned long len);
+int intel_gvt_write_shared_page(struct intel_vgpu *vgpu,
+		unsigned int offset, void *buf, unsigned long len);
 
 #include "trace.h"
 #include "mpt.h"
diff --git a/drivers/gpu/drm/i915/gvt/handlers.c b/drivers/gpu/drm/i915/gvt/handlers.c
index aceb16f..eb09003 100644
--- a/drivers/gpu/drm/i915/gvt/handlers.c
+++ b/drivers/gpu/drm/i915/gvt/handlers.c
@@ -1195,6 +1195,8 @@ static int pvinfo_mmio_read(struct intel_vgpu *vgpu, unsigned int offset,
 	case 0x78010:	/* vgt_caps */
 	case 0x7881c:
 	case _vgtif_reg(pv_caps):
+	case _vgtif_reg(shared_page_gpa):
+	case _vgtif_reg(shared_page_gpa) + 4:
 		break;
 	default:
 		invalid_read = true;
@@ -1212,6 +1214,9 @@ static int handle_g2v_notification(struct intel_vgpu *vgpu, int notification)
 	enum intel_gvt_gtt_type root_entry_type = GTT_TYPE_PPGTT_ROOT_L4_ENTRY;
 	struct intel_vgpu_mm *mm;
 	u64 *pdps;
+	unsigned long gpa, gfn;
+	u16 ver_major = PV_MAJOR;
+	u16 ver_minor = PV_MINOR;
 
 	pdps = (u64 *)&vgpu_vreg64_t(vgpu, vgtif_reg(pdp[0]));
 
@@ -1225,6 +1230,19 @@ static int handle_g2v_notification(struct intel_vgpu *vgpu, int notification)
 	case VGT_G2V_PPGTT_L3_PAGE_TABLE_DESTROY:
 	case VGT_G2V_PPGTT_L4_PAGE_TABLE_DESTROY:
 		return intel_vgpu_put_ppgtt_mm(vgpu, pdps);
+	case VGT_G2V_SHARED_PAGE_SETUP:
+		gpa = vgpu_vreg64_t(vgpu, vgtif_reg(shared_page_gpa));
+		gfn = gpa >> PAGE_SHIFT;
+		if (!intel_gvt_hypervisor_is_valid_gfn(vgpu, gfn)) {
+			vgpu_vreg_t(vgpu, vgtif_reg(pv_caps)) = 0;
+			return 0;
+		}
+		vgpu->shared_page_gpa = gpa;
+		vgpu->shared_page_enabled = true;
+
+		intel_gvt_write_shared_page(vgpu, 0, &ver_major, 2);
+		intel_gvt_write_shared_page(vgpu, 2, &ver_minor, 2);
+		break;
 	case VGT_G2V_EXECLIST_CONTEXT_CREATE:
 	case VGT_G2V_EXECLIST_CONTEXT_DESTROY:
 	case 1:	/* Remove this in guest driver. */
@@ -1281,6 +1299,8 @@ static int pvinfo_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
 	case _vgtif_reg(pdp[3].hi):
 	case _vgtif_reg(execlist_context_descriptor_lo):
 	case _vgtif_reg(execlist_context_descriptor_hi):
+	case _vgtif_reg(shared_page_gpa):
+	case _vgtif_reg(shared_page_gpa) + 4:
 		break;
 	case _vgtif_reg(rsv5[0])..._vgtif_reg(rsv5[3]):
 		invalid_write = true;
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index 9e00979..811edbb 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -63,6 +63,8 @@ void populate_pvinfo_page(struct intel_vgpu *vgpu)
 	vgpu_vreg_t(vgpu, vgtif_reg(cursor_x_hot)) = UINT_MAX;
 	vgpu_vreg_t(vgpu, vgtif_reg(cursor_y_hot)) = UINT_MAX;
 
+	vgpu_vreg64_t(vgpu, vgtif_reg(shared_page_gpa)) = 0;
+
 	gvt_dbg_core("Populate PVINFO PAGE for vGPU %d\n", vgpu->id);
 	gvt_dbg_core("aperture base [GMADR] 0x%llx size 0x%llx\n",
 		vgpu_aperture_gmadr_base(vgpu), vgpu_aperture_sz(vgpu));
@@ -591,3 +593,44 @@ void intel_gvt_reset_vgpu(struct intel_vgpu *vgpu)
 	intel_gvt_reset_vgpu_locked(vgpu, true, 0);
 	mutex_unlock(&vgpu->vgpu_lock);
 }
+
+/**
+ * intel_gvt_read_shared_page - read content from shared page
+ */
+int intel_gvt_read_shared_page(struct intel_vgpu *vgpu,
+		unsigned int offset, void *buf, unsigned long len)
+{
+	int ret = -EINVAL;
+	unsigned long gpa;
+
+	if (offset >= PAGE_SIZE)
+		goto err;
+
+	gpa = vgpu->shared_page_gpa + offset;
+	ret = intel_gvt_hypervisor_read_gpa(vgpu, gpa, buf, len);
+	if (!ret)
+		return ret;
+err:
+	gvt_vgpu_err("read shared page (offset %x) failed", offset);
+	memset(buf, 0, len);
+	return ret;
+}
+
+int intel_gvt_write_shared_page(struct intel_vgpu *vgpu,
+		unsigned int offset, void *buf, unsigned long len)
+{
+	int ret = -EINVAL;
+	unsigned long gpa;
+
+	if (offset >= PAGE_SIZE)
+		goto err;
+
+	gpa = vgpu->shared_page_gpa + offset;
+	ret = intel_gvt_hypervisor_write_gpa(vgpu, gpa, buf, len);
+	if (!ret)
+		return ret;
+err:
+	gvt_vgpu_err("write shared page (offset %x) failed", offset);
+	memset(buf, 0, len);
+	return ret;
+}
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 8/9] drm/i915/gvt: GVTg support ppgtt pv optimization

[Xiaolin Zhang]

This patch handles ppgtt update from g2v notification.

It read out ppgtt pte entries from guest pte tables page and
convert them to host pfns.

It creates local ppgtt tables and insert the content pages
into the local ppgtt tables directly, which does not track
the usage of guest page table and removes the cost of write
protection from the original shadow page mechansim.

v0: RFC.
v1: rebase.
v2: rebase.
v3: report pv pggtt cap to guest.
v4: renamed VGPU_PVMMIO with VGPU_PVCAP for name consistance, no PV
support if gfx vtd enabled.
v5: rebase.
v6: rebase.
v7: added command transport buffer support.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/gvt/gtt.c      | 298 ++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/gtt.h      |  11 ++
 drivers/gpu/drm/i915/gvt/gvt.h      |   4 +
 drivers/gpu/drm/i915/gvt/handlers.c | 127 ++++++++++++++-
 drivers/gpu/drm/i915/gvt/vgpu.c     |   2 +
 5 files changed, 441 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/gvt/gtt.c b/drivers/gpu/drm/i915/gvt/gtt.c
index 4b04af5..c944ac2 100644
--- a/drivers/gpu/drm/i915/gvt/gtt.c
+++ b/drivers/gpu/drm/i915/gvt/gtt.c
@@ -1771,6 +1771,25 @@ static int ppgtt_handle_guest_write_page_table_bytes(
 	return 0;
 }
 
+static void invalidate_mm_pv(struct intel_vgpu_mm *mm)
+{
+	struct intel_vgpu *vgpu = mm->vgpu;
+	struct intel_gvt *gvt = vgpu->gvt;
+	struct intel_gvt_gtt *gtt = &gvt->gtt;
+	struct intel_gvt_gtt_pte_ops *ops = gtt->pte_ops;
+	struct intel_gvt_gtt_entry se;
+
+	i915_vm_put(&mm->ppgtt->vm);
+
+	ppgtt_get_shadow_root_entry(mm, &se, 0);
+	if (!ops->test_present(&se))
+		return;
+	se.val64 = 0;
+	ppgtt_set_shadow_root_entry(mm, &se, 0);
+
+	mm->ppgtt_mm.shadowed  = false;
+}
+
 static void invalidate_ppgtt_mm(struct intel_vgpu_mm *mm)
 {
 	struct intel_vgpu *vgpu = mm->vgpu;
@@ -1783,6 +1802,11 @@ static void invalidate_ppgtt_mm(struct intel_vgpu_mm *mm)
 	if (!mm->ppgtt_mm.shadowed)
 		return;
 
+	if (VGPU_PVCAP(mm->vgpu, PV_PPGTT_UPDATE)) {
+		invalidate_mm_pv(mm);
+		return;
+	}
+
 	for (index = 0; index < ARRAY_SIZE(mm->ppgtt_mm.shadow_pdps); index++) {
 		ppgtt_get_shadow_root_entry(mm, &se, index);
 
@@ -1800,6 +1824,26 @@ static void invalidate_ppgtt_mm(struct intel_vgpu_mm *mm)
 	mm->ppgtt_mm.shadowed = false;
 }
 
+static int shadow_mm_pv(struct intel_vgpu_mm *mm)
+{
+	struct intel_vgpu *vgpu = mm->vgpu;
+	struct intel_gvt *gvt = vgpu->gvt;
+	struct intel_gvt_gtt_entry se;
+
+	mm->ppgtt = i915_ppgtt_create(gvt->dev_priv);
+	if (IS_ERR(mm->ppgtt)) {
+		gvt_vgpu_err("fail to create ppgtt for pdp 0x%llx\n",
+				px_dma(mm->ppgtt->pd));
+		return PTR_ERR(mm->ppgtt);
+	}
+
+	se.type = GTT_TYPE_PPGTT_ROOT_L4_ENTRY;
+	se.val64 = px_dma(mm->ppgtt->pd);
+	ppgtt_set_shadow_root_entry(mm, &se, 0);
+	mm->ppgtt_mm.shadowed  = true;
+
+	return 0;
+}
 
 static int shadow_ppgtt_mm(struct intel_vgpu_mm *mm)
 {
@@ -1814,6 +1858,9 @@ static int shadow_ppgtt_mm(struct intel_vgpu_mm *mm)
 	if (mm->ppgtt_mm.shadowed)
 		return 0;
 
+	if (VGPU_PVCAP(mm->vgpu, PV_PPGTT_UPDATE))
+		return shadow_mm_pv(mm);
+
 	mm->ppgtt_mm.shadowed = true;
 
 	for (index = 0; index < ARRAY_SIZE(mm->ppgtt_mm.guest_pdps); index++) {
@@ -2825,3 +2872,254 @@ void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu)
 	intel_vgpu_destroy_all_ppgtt_mm(vgpu);
 	intel_vgpu_reset_ggtt(vgpu, true);
 }
+
+#define GEN8_PDE_SHIFT		21
+#define GEN8_PML4E_SHIFT	39
+#define GEN8_PDPE_SHIFT	30
+#define GEN8_PML4E_SIZE	(1UL << GEN8_PML4E_SHIFT)
+#define GEN8_PML4E_SIZE_MASK	(~(GEN8_PML4E_SIZE - 1))
+#define GEN8_PDPE_SIZE		(1UL << GEN8_PDPE_SHIFT)
+#define GEN8_PDPE_SIZE_MASK	(~(GEN8_PDPE_SIZE - 1))
+#define GEN8_PDE_SIZE		(1UL << GEN8_PDE_SHIFT)
+#define GEN8_PDE_SIZE_MASK	(~(GEN8_PDE_SIZE - 1))
+
+#define pml4_addr_end(addr, end)					\
+({	unsigned long __boundary = \
+			((addr) + GEN8_PML4E_SIZE) & GEN8_PML4E_SIZE_MASK; \
+	(__boundary < (end)) ? __boundary : (end);		\
+})
+
+#define pdp_addr_end(addr, end)						\
+({	unsigned long __boundary = \
+			((addr) + GEN8_PDPE_SIZE) & GEN8_PDPE_SIZE_MASK; \
+	(__boundary < (end)) ? __boundary : (end);		\
+})
+
+#define pd_addr_end(addr, end)						\
+({	unsigned long __boundary = \
+			((addr) + GEN8_PDE_SIZE) & GEN8_PDE_SIZE_MASK;	\
+	(__boundary < (end)) ? __boundary : (end);		\
+})
+
+struct ppgtt_walk {
+	unsigned long *mfns;
+	int mfn_index;
+	unsigned long *pt;
+};
+
+static int walk_pt_range(struct intel_vgpu *vgpu, u64 pt,
+				u64 start, u64 end, struct ppgtt_walk *walk)
+{
+	const struct intel_gvt_device_info *info = &vgpu->gvt->device_info;
+	struct intel_gvt_gtt_gma_ops *gma_ops = vgpu->gvt->gtt.gma_ops;
+	unsigned long start_index, end_index;
+	int ret;
+	int i;
+	unsigned long mfn, gfn;
+
+	start_index = gma_ops->gma_to_pte_index(start);
+	end_index = ((end - start) >> PAGE_SHIFT) + start_index;
+
+	ret = intel_gvt_hypervisor_read_gpa(vgpu,
+		(pt & PAGE_MASK) + (start_index << info->gtt_entry_size_shift),
+		walk->pt + start_index,
+		(end_index - start_index) << info->gtt_entry_size_shift);
+	if (ret) {
+		gvt_vgpu_err("fail to read gpa %llx\n", pt);
+		return ret;
+	}
+
+	for (i = start_index; i < end_index; i++) {
+		gfn = walk->pt[i] >> PAGE_SHIFT;
+		mfn = intel_gvt_hypervisor_gfn_to_mfn(vgpu, gfn);
+		if (mfn == INTEL_GVT_INVALID_ADDR) {
+			gvt_vgpu_err("fail to translate gfn: 0x%lx\n", gfn);
+			return -ENXIO;
+		}
+		walk->mfns[walk->mfn_index++] = mfn << PAGE_SHIFT;
+	}
+
+	return 0;
+}
+
+
+static int walk_pd_range(struct intel_vgpu *vgpu, u64 pd,
+				u64 start, u64 end, struct ppgtt_walk *walk)
+{
+	const struct intel_gvt_device_info *info = &vgpu->gvt->device_info;
+	struct intel_gvt_gtt_gma_ops *gma_ops = vgpu->gvt->gtt.gma_ops;
+	unsigned long index;
+	u64 pt, next;
+	int ret  = 0;
+
+	do {
+		index = gma_ops->gma_to_pde_index(start);
+
+		ret = intel_gvt_hypervisor_read_gpa(vgpu,
+			(pd & PAGE_MASK) + (index <<
+			info->gtt_entry_size_shift), &pt, 8);
+		if (ret)
+			return ret;
+		next = pd_addr_end(start, end);
+		walk_pt_range(vgpu, pt, start, next, walk);
+
+		start = next;
+	} while (start != end);
+
+	return ret;
+}
+
+
+static int walk_pdp_range(struct intel_vgpu *vgpu, u64 pdp,
+				  u64 start, u64 end, struct ppgtt_walk *walk)
+{
+	const struct intel_gvt_device_info *info = &vgpu->gvt->device_info;
+	struct intel_gvt_gtt_gma_ops *gma_ops = vgpu->gvt->gtt.gma_ops;
+	unsigned long index;
+	u64 pd, next;
+	int ret  = 0;
+
+	do {
+		index = gma_ops->gma_to_l4_pdp_index(start);
+
+		ret = intel_gvt_hypervisor_read_gpa(vgpu,
+			(pdp & PAGE_MASK) + (index <<
+			info->gtt_entry_size_shift), &pd, 8);
+		if (ret)
+			return ret;
+		next = pdp_addr_end(start, end);
+		walk_pd_range(vgpu, pd, start, next, walk);
+		start = next;
+	} while (start != end);
+
+	return ret;
+}
+
+
+static int walk_pml4_range(struct intel_vgpu *vgpu, u64 pml4,
+				u64 start, u64 end, struct ppgtt_walk *walk)
+{
+	const struct intel_gvt_device_info *info = &vgpu->gvt->device_info;
+	struct intel_gvt_gtt_gma_ops *gma_ops = vgpu->gvt->gtt.gma_ops;
+	unsigned long index;
+	u64 pdp, next;
+	int ret  = 0;
+
+	do {
+		index = gma_ops->gma_to_pml4_index(start);
+		ret = intel_gvt_hypervisor_read_gpa(vgpu,
+			(pml4 & PAGE_MASK) + (index <<
+			info->gtt_entry_size_shift), &pdp, 8);
+		if (ret)
+			return ret;
+		next = pml4_addr_end(start, end);
+		walk_pdp_range(vgpu, pdp, start, next, walk);
+		start = next;
+	} while (start != end);
+
+	return ret;
+}
+
+static int intel_vgpu_pv_ppgtt_insert_4lvl(struct intel_vgpu *vgpu,
+		struct intel_vgpu_mm *mm,
+		u64 pml4, u64 start, u64 length, u32 cache_level)
+{
+	int ret = 0;
+	struct sg_table st;
+	struct scatterlist *sg = NULL;
+	int num_pages;
+	struct i915_vma vma;
+	struct ppgtt_walk walk;
+	int i;
+
+	num_pages = length >> PAGE_SHIFT;
+
+	walk.mfn_index = 0;
+	walk.mfns = NULL;
+	walk.pt = NULL;
+
+	walk.mfns = kmalloc_array(num_pages,
+			sizeof(unsigned long), GFP_KERNEL);
+	if (!walk.mfns) {
+		ret = -ENOMEM;
+		goto fail;
+	}
+
+	walk.pt = (unsigned long *)__get_free_pages(GFP_KERNEL, 0);
+	if (!walk.pt) {
+		ret = -ENOMEM;
+		goto fail;
+	}
+
+	if (sg_alloc_table(&st, num_pages, GFP_KERNEL)) {
+		ret = -ENOMEM;
+		goto fail;
+	}
+
+	ret = walk_pml4_range(vgpu, pml4, start, start + length, &walk);
+	if (ret)
+		goto fail_free_sg;
+
+	WARN_ON(num_pages != walk.mfn_index);
+
+	for_each_sg(st.sgl, sg, num_pages, i) {
+		sg->offset = 0;
+		sg->length = PAGE_SIZE;
+		sg_dma_address(sg) = walk.mfns[i];
+		sg_dma_len(sg) = PAGE_SIZE;
+	}
+
+	memset(&vma, 0, sizeof(vma));
+	vma.node.start = start;
+	vma.pages = &st;
+	mm->ppgtt->vm.insert_entries(&mm->ppgtt->vm, &vma, cache_level, 0);
+
+fail_free_sg:
+	sg_free_table(&st);
+fail:
+	kfree(walk.mfns);
+	free_page((unsigned long)walk.pt);
+
+	return ret;
+}
+
+int intel_vgpu_handle_pv_ppgtt_update(struct intel_vgpu *vgpu,
+		u32 action, struct pv_ppgtt_update *pv_ppgtt)
+{
+	struct intel_vgpu_mm *mm;
+	u64 pdp, start, length;
+	u32 cache_level;
+	int ret = 0;
+
+	pdp = pv_ppgtt->pdp;
+	start = pv_ppgtt->start;
+	length = pv_ppgtt->length;
+	cache_level = pv_ppgtt->cache_level;
+
+	mm = intel_vgpu_find_ppgtt_mm(vgpu, &pdp);
+	if (!mm) {
+		gvt_vgpu_err("failed to find pdp 0x%llx\n", pdp);
+		ret = -EINVAL;
+	}
+
+	if (action == PV_ACTION_PPGTT_L4_ALLOC) {
+		ret = mm->ppgtt->vm.allocate_va_range(&mm->ppgtt->vm,
+				start, length);
+		if (ret)
+			gvt_vgpu_err("failed to alloc %llx\n", pdp);
+	}
+
+	if (action == PV_ACTION_PPGTT_L4_CLEAR) {
+		mm->ppgtt->vm.clear_range(&mm->ppgtt->vm,
+				start, length);
+	}
+
+	if (action ==  PV_ACTION_PPGTT_L4_INSERT) {
+		ret = intel_vgpu_pv_ppgtt_insert_4lvl(vgpu, mm,
+				pdp, start, length, cache_level);
+		if (ret)
+			gvt_vgpu_err("failed to insert %llx\n", pdp);
+	}
+
+	return ret;
+}
diff --git a/drivers/gpu/drm/i915/gvt/gtt.h b/drivers/gpu/drm/i915/gvt/gtt.h
index 8878931..a969331 100644
--- a/drivers/gpu/drm/i915/gvt/gtt.h
+++ b/drivers/gpu/drm/i915/gvt/gtt.h
@@ -141,6 +141,7 @@ struct intel_gvt_partial_pte {
 
 struct intel_vgpu_mm {
 	enum intel_gvt_mm_type type;
+	struct i915_ppgtt *ppgtt;
 	struct intel_vgpu *vgpu;
 
 	struct kref ref;
@@ -253,6 +254,14 @@ struct intel_vgpu_ppgtt_spt {
 	struct list_head post_shadow_list;
 };
 
+/* ppgtt pv support data structure */
+struct pv_ppgtt_update {
+	u64 pdp;
+	u64 start;
+	u64 length;
+	u32 cache_level;
+};
+
 int intel_vgpu_sync_oos_pages(struct intel_vgpu *vgpu);
 
 int intel_vgpu_flush_post_shadow(struct intel_vgpu *vgpu);
@@ -278,4 +287,6 @@ int intel_vgpu_emulate_ggtt_mmio_read(struct intel_vgpu *vgpu,
 int intel_vgpu_emulate_ggtt_mmio_write(struct intel_vgpu *vgpu,
 	unsigned int off, void *p_data, unsigned int bytes);
 
+int intel_vgpu_handle_pv_ppgtt_update(struct intel_vgpu *vgpu,
+		u32 action, struct pv_ppgtt_update *pv_ppgtt);
 #endif /* _GVT_GTT_H_ */
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 71213e0..4e658a5 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -53,6 +53,10 @@
 
 #define GVT_MAX_VGPU 8
 
+#define VGPU_PVCAP(vgpu, cap)	\
+	((vgpu_vreg_t(vgpu, vgtif_reg(pv_caps)) & (cap)) \
+			&& vgpu->shared_page_enabled)
+
 struct intel_gvt_host {
 	struct device *dev;
 	bool initialized;
diff --git a/drivers/gpu/drm/i915/gvt/handlers.c b/drivers/gpu/drm/i915/gvt/handlers.c
index eb09003..7176831 100644
--- a/drivers/gpu/drm/i915/gvt/handlers.c
+++ b/drivers/gpu/drm/i915/gvt/handlers.c
@@ -1209,6 +1209,127 @@ static int pvinfo_mmio_read(struct intel_vgpu *vgpu, unsigned int offset,
 	return 0;
 }
 
+static inline unsigned int ct_header_get_len(u32 header)
+{
+	return (header >> PV_CT_MSG_LEN_SHIFT) & PV_CT_MSG_LEN_MASK;
+}
+
+static inline unsigned int ct_header_get_action(u32 header)
+{
+	return (header >> PV_CT_MSG_ACTION_SHIFT) & PV_CT_MSG_ACTION_MASK;
+}
+
+static int fetch_pv_command_buffer(struct intel_vgpu *vgpu,
+		struct vgpu_pv_ct_buffer_desc *desc,
+		u32 *fence, u32 *action, u32 *data)
+{
+	u32 head, tail, len, size, off;
+	u32 cmd_head;
+	u32 avail;
+	u32 ret;
+
+	/* fetch command descriptor */
+	off = PV_DESC_OFF;
+	ret = intel_gvt_read_shared_page(vgpu, off, desc, sizeof(*desc));
+	if (ret)
+		return ret;
+
+	GEM_BUG_ON(desc->size % 4);
+	GEM_BUG_ON(desc->head % 4);
+	GEM_BUG_ON(desc->tail % 4);
+	GEM_BUG_ON(tail >= size);
+	GEM_BUG_ON(head >= size);
+
+	/* tail == head condition indicates empty */
+	head = desc->head/4;
+	tail = desc->tail/4;
+	size = desc->size/4;
+
+	if (unlikely((tail - head) == 0))
+		return -ENODATA;
+
+	/* fetch command head */
+	off = desc->addr + head * 4;
+	ret = intel_gvt_read_shared_page(vgpu, off, &cmd_head, 4);
+	head = (head + 1) % size;
+	if (ret)
+		goto err;
+
+	len = ct_header_get_len(cmd_head) - 1;
+	*action = ct_header_get_action(cmd_head);
+
+	/* fetch command fence */
+	off = desc->addr + head * 4;
+	ret = intel_gvt_read_shared_page(vgpu, off, fence, 4);
+	head = (head + 1) % size;
+	if (ret)
+		goto err;
+
+	/* no command data */
+	if (len == 0)
+		goto err;
+
+	/* fetch command data */
+	avail = size - head;
+	if (len <= avail) {
+		off =  desc->addr + head * 4;
+		ret = intel_gvt_read_shared_page(vgpu, off, data, len * 4);
+		head = (head + len) % size;
+		if (ret)
+			goto err;
+	} else {
+		/* swap case */
+		off =  desc->addr + head * 4;
+		ret = intel_gvt_read_shared_page(vgpu, off, data, avail * 4);
+		head = (head + avail) % size;
+		if (ret)
+			goto err;
+
+		off = desc->addr;
+		ret = intel_gvt_read_shared_page(vgpu, off, &data[avail],
+				(len - avail) * 4);
+		head = (head + len - avail) % size;
+		if (ret)
+			goto err;
+	}
+
+err:
+	desc->head = head * 4;
+	return ret;
+}
+
+static int handle_pv_actions(struct intel_vgpu *vgpu)
+{
+	struct vgpu_pv_ct_buffer_desc desc;
+	u32 fence, action;
+	u32 data[32];
+	int ret;
+	struct pv_ppgtt_update *ppgtt;
+
+	ret = fetch_pv_command_buffer(vgpu, &desc, &fence, &action, data);
+	if (ret)
+		return ret;
+
+	switch (action) {
+	case PV_ACTION_PPGTT_L4_ALLOC:
+	case PV_ACTION_PPGTT_L4_CLEAR:
+	case PV_ACTION_PPGTT_L4_INSERT:
+		ppgtt = (struct pv_ppgtt_update *)data;
+		ret = intel_vgpu_handle_pv_ppgtt_update(vgpu, action, ppgtt);
+		break;
+	default:
+		break;
+	}
+
+	/* write command descriptor back */
+	desc.fence = fence;
+	desc.status = ret;
+
+	ret = intel_gvt_write_shared_page(vgpu, PV_DESC_OFF,
+			&desc, sizeof(desc));
+	return ret;
+}
+
 static int handle_g2v_notification(struct intel_vgpu *vgpu, int notification)
 {
 	enum intel_gvt_gtt_type root_entry_type = GTT_TYPE_PPGTT_ROOT_L4_ENTRY;
@@ -1217,6 +1338,7 @@ static int handle_g2v_notification(struct intel_vgpu *vgpu, int notification)
 	unsigned long gpa, gfn;
 	u16 ver_major = PV_MAJOR;
 	u16 ver_minor = PV_MINOR;
+	int ret = 0;
 
 	pdps = (u64 *)&vgpu_vreg64_t(vgpu, vgtif_reg(pdp[0]));
 
@@ -1243,6 +1365,9 @@ static int handle_g2v_notification(struct intel_vgpu *vgpu, int notification)
 		intel_gvt_write_shared_page(vgpu, 0, &ver_major, 2);
 		intel_gvt_write_shared_page(vgpu, 2, &ver_minor, 2);
 		break;
+	case VGT_G2V_PV_SEND_TRIGGER:
+		ret = handle_pv_actions(vgpu);
+		break;
 	case VGT_G2V_EXECLIST_CONTEXT_CREATE:
 	case VGT_G2V_EXECLIST_CONTEXT_DESTROY:
 	case 1:	/* Remove this in guest driver. */
@@ -1250,7 +1375,7 @@ static int handle_g2v_notification(struct intel_vgpu *vgpu, int notification)
 	default:
 		gvt_vgpu_err("Invalid PV notification %d\n", notification);
 	}
-	return 0;
+	return ret;
 }
 
 static int send_display_ready_uevent(struct intel_vgpu *vgpu, int ready)
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index 811edbb..e8a957a 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -49,6 +49,8 @@ void populate_pvinfo_page(struct intel_vgpu *vgpu)
 	vgpu_vreg_t(vgpu, vgtif_reg(vgt_caps)) |= VGT_CAPS_HUGE_GTT;
 	vgpu_vreg_t(vgpu, vgtif_reg(vgt_caps)) |= VGT_CAPS_PV;
 
+	if (!intel_vtd_active())
+		vgpu_vreg_t(vgpu, vgtif_reg(pv_caps)) = PV_PPGTT_UPDATE;
 	vgpu_vreg_t(vgpu, vgtif_reg(avail_rs.mappable_gmadr.base)) =
 		vgpu_aperture_gmadr_base(vgpu);
 	vgpu_vreg_t(vgpu, vgtif_reg(avail_rs.mappable_gmadr.size)) =
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH v10 9/9] drm/i915/gvt: GVTg support context submission pv optimization

[Xiaolin Zhang]

implemented context submission pv optimizaiton within GVTg.

GVTg to read context submission data (elsp_data) from the shared_page
directly without trap cost and eliminate execlist HW behavior emulation
without injecting context switch interrupt to guest under PV
submisison mechanism.

v0: RFC.
v1: rebase.
v2: rebase.
v3: report pv context submission cap and handle VGT_G2V_ELSP_SUBMIT
g2v pv notification.
v4: eliminate execlist HW emulation and don't inject context switch
interrupt to guest under PV submisison mechanism.
v5: rebase.
v6: rebase.
v7: rebase.
v8: addressed v7 pv submission comments.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/gvt/execlist.c |  6 ++++++
 drivers/gpu/drm/i915/gvt/handlers.c | 30 +++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/gvt/vgpu.c     |  2 ++
 3 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/gvt/execlist.c b/drivers/gpu/drm/i915/gvt/execlist.c
index f21b8fb..e52bfd6 100644
--- a/drivers/gpu/drm/i915/gvt/execlist.c
+++ b/drivers/gpu/drm/i915/gvt/execlist.c
@@ -382,6 +382,9 @@ static int prepare_execlist_workload(struct intel_vgpu_workload *workload)
 	int ring_id = workload->ring_id;
 	int ret;
 
+	if (VGPU_PVCAP(vgpu, PV_SUBMISSION))
+		return 0;
+
 	if (!workload->emulate_schedule_in)
 		return 0;
 
@@ -429,6 +432,9 @@ static int complete_execlist_workload(struct intel_vgpu_workload *workload)
 		goto out;
 	}
 
+	if (VGPU_PVCAP(vgpu, PV_SUBMISSION))
+		goto out;
+
 	ret = emulate_execlist_ctx_schedule_out(execlist, &workload->ctx_desc);
 out:
 	intel_vgpu_unpin_mm(workload->shadow_mm);
diff --git a/drivers/gpu/drm/i915/gvt/handlers.c b/drivers/gpu/drm/i915/gvt/handlers.c
index 7176831..5dc37e8 100644
--- a/drivers/gpu/drm/i915/gvt/handlers.c
+++ b/drivers/gpu/drm/i915/gvt/handlers.c
@@ -1810,6 +1810,31 @@ static int mmio_read_from_hw(struct intel_vgpu *vgpu,
 	return intel_vgpu_default_mmio_read(vgpu, offset, p_data, bytes);
 }
 
+static int handle_pv_submission(struct intel_vgpu *vgpu, int ring_id)
+{
+	struct intel_vgpu_execlist *execlist;
+	u32 hw_id = vgpu->gvt->dev_priv->engine[ring_id]->hw_id;
+	u32 base = PV_ELSP_OFF + hw_id * sizeof(struct pv_submission);
+	u32 desc_off = offsetof(struct pv_submission, descs);
+	u32 submitted_off = offsetof(struct pv_submission, submitted);
+	bool submitted = false;
+	int ret;
+
+	execlist = &vgpu->submission.execlist[ring_id];
+	desc_off += base;
+	if (intel_gvt_read_shared_page(vgpu, desc_off,
+			&execlist->elsp_dwords.data, 16))
+		return -EINVAL;
+
+	ret = intel_vgpu_submit_execlist(vgpu, ring_id);
+	if (ret)
+		gvt_vgpu_err("fail submit workload on ring %d\n", ring_id);
+
+	submitted_off += base;
+	ret = intel_gvt_write_shared_page(vgpu, submitted_off, &submitted, 1);
+	return ret;
+}
+
 static int elsp_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
 		void *p_data, unsigned int bytes)
 {
@@ -1821,8 +1846,11 @@ static int elsp_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
 	if (WARN_ON(ring_id < 0 || ring_id >= I915_NUM_ENGINES))
 		return -EINVAL;
 
-	execlist = &vgpu->submission.execlist[ring_id];
+	if (VGPU_PVCAP(vgpu, PV_SUBMISSION) &&
+			data == PV_ACTION_ELSP_SUBMISSION)
+		return handle_pv_submission(vgpu, ring_id);
 
+	execlist = &vgpu->submission.execlist[ring_id];
 	execlist->elsp_dwords.data[3 - execlist->elsp_dwords.index] = data;
 	if (execlist->elsp_dwords.index == 3) {
 		ret = intel_vgpu_submit_execlist(vgpu, ring_id);
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index e8a957a..236ba04 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -51,6 +51,8 @@ void populate_pvinfo_page(struct intel_vgpu *vgpu)
 
 	if (!intel_vtd_active())
 		vgpu_vreg_t(vgpu, vgtif_reg(pv_caps)) = PV_PPGTT_UPDATE;
+	vgpu_vreg_t(vgpu, vgtif_reg(pv_caps)) |= PV_SUBMISSION;
+
 	vgpu_vreg_t(vgpu, vgtif_reg(avail_rs.mappable_gmadr.base)) =
 		vgpu_aperture_gmadr_base(vgpu);
 	vgpu_vreg_t(vgpu, vgtif_reg(avail_rs.mappable_gmadr.size)) =
-- 
1.8.3.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✗ Fi.CI.CHECKPATCH: warning for i915 vgpu PV to improve vgpu performance

[Patchwork]

== Series Details ==

Series: i915 vgpu PV to improve vgpu performance
URL   : https://patchwork.freedesktop.org/series/66787/
State : warning

== Summary ==

$ dim checkpatch origin/drm-tip
16a1cd5cf965 drm/i915: introduced vgpu pv capability
-:125: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#125: FILE: drivers/gpu/drm/i915/i915_vgpu.c:325:
+bool intel_vgpu_check_pv_caps(struct drm_i915_private *dev_priv,
+		void __iomem *shared_area)

-:163: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#163: FILE: drivers/gpu/drm/i915/i915_vgpu.h:57:
+bool intel_vgpu_check_pv_caps(struct drm_i915_private *dev_priv,
+		void __iomem *shared_area);

total: 0 errors, 0 warnings, 2 checks, 107 lines checked
a176f82a8fcd drm/i915: vgpu shared memory setup for pv optimization
-:114: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#114: FILE: drivers/gpu/drm/i915/i915_vgpu.c:329:
+static int intel_vgpu_setup_shared_page(struct drm_i915_private *dev_priv,
+		void __iomem *shared_area)

-:172: CHECK:ALLOC_SIZEOF_STRUCT: Prefer kzalloc(sizeof(*pv)...) over kzalloc(sizeof(struct i915_virtual_gpu_pv)...)
#172: FILE: drivers/gpu/drm/i915/i915_vgpu.c:387:
+	pv = kzalloc(sizeof(struct i915_virtual_gpu_pv), GFP_KERNEL);

total: 0 errors, 0 warnings, 2 checks, 172 lines checked
afd40d2ba3c1 drm/i915: vgpu pv command buffer support
-:51: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#51: FILE: drivers/gpu/drm/i915/i915_vgpu.c:342:
+static int wait_for_desc_update(struct vgpu_pv_ct_buffer_desc *desc,
+		u32 fence, u32 *status)

-:63: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#63: FILE: drivers/gpu/drm/i915/i915_vgpu.c:354:
+		DRM_ERROR("CT: fence %u failed; reported fence=%u\n",
+				fence, desc->fence);

-:88: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#88: FILE: drivers/gpu/drm/i915/i915_vgpu.c:379:
+static int pv_command_buffer_write(struct i915_virtual_gpu_pv *pv,
+		const u32 *action, u32 len /* in dwords */, u32 fence)

-:151: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#151: FILE: drivers/gpu/drm/i915/i915_vgpu.c:442:
+static int pv_send(struct drm_i915_private *dev_priv,
+		const u32 *action, u32 len, u32 *status)

-:186: CHECK:OPEN_ENDED_LINE: Lines should not end with a '('
#186: FILE: drivers/gpu/drm/i915/i915_vgpu.c:477:
+static int intel_vgpu_pv_send_command_buffer(

-:203: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#203: FILE: drivers/gpu/drm/i915/i915_vgpu.c:494:
+		DRM_ERROR("PV: send action %#x returned %d (%#x)\n",
+				action[0], ret, ret);

-:226: CHECK:SPACING: spaces preferred around that '/' (ctx:VxV)
#226: FILE: drivers/gpu/drm/i915/i915_vgpu.c:582:
+	pv->ctb.desc->size = PAGE_SIZE/2;
 	                              ^

-:245: CHECK:SPACING: spaces preferred around that '/' (ctx:VxV)
#245: FILE: drivers/gpu/drm/i915/i915_vgpu.h:32:
+#define PV_DESC_OFF		(PAGE_SIZE/4)
                    		          ^

-:246: CHECK:SPACING: spaces preferred around that '/' (ctx:VxV)
#246: FILE: drivers/gpu/drm/i915/i915_vgpu.h:33:
+#define PV_CMD_OFF		(PAGE_SIZE/2)
                   		          ^

-:323: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#323: FILE: drivers/gpu/drm/i915/i915_vgpu.h:131:
+intel_vgpu_pv_send(struct drm_i915_private *dev_priv,
+		u32 *action, u32 len)

total: 0 errors, 0 warnings, 10 checks, 299 lines checked
73862271b37e drm/i915: vgpu ppgtt update pv optimization
-:42: CHECK:LOGICAL_CONTINUATIONS: Logical continuations should be on the previous line
#42: FILE: drivers/gpu/drm/i915/i915_gem.c:1313:
+	if ((intel_vgpu_active(dev_priv) && !intel_vgpu_has_huge_gtt(dev_priv))
+		|| intel_vgpu_enabled_pv_caps(dev_priv, PV_PPGTT_UPDATE))

-:56: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#56: FILE: drivers/gpu/drm/i915/i915_gem_gtt.c:1019:
+void gen8_ppgtt_clear(struct i915_address_space *vm,
 			     u64 start, u64 length)

-:65: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#65: FILE: drivers/gpu/drm/i915/i915_gem_gtt.c:1130:
+int gen8_ppgtt_alloc(struct i915_address_space *vm,
 			    u64 start, u64 length)

-:74: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#74: FILE: drivers/gpu/drm/i915/i915_gem_gtt.c:1330:
+void gen8_ppgtt_insert(struct i915_address_space *vm,
 			      struct i915_vma *vma,

-:98: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#98: FILE: drivers/gpu/drm/i915/i915_gem_gtt.h:592:
+void gen8_ppgtt_clear(struct i915_address_space *vm,
+		u64 start, u64 length);

-:100: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#100: FILE: drivers/gpu/drm/i915/i915_gem_gtt.h:594:
+void gen8_ppgtt_insert(struct i915_address_space *vm,
+		struct i915_vma *vma,

-:103: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#103: FILE: drivers/gpu/drm/i915/i915_gem_gtt.h:597:
+int gen8_ppgtt_alloc(struct i915_address_space *vm,
+		u64 start, u64 length);

-:127: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#127: FILE: drivers/gpu/drm/i915/i915_vgpu.c:329:
+static int vgpu_ppgtt_pv_update(struct drm_i915_private *dev_priv,
+		u32 action, u64 pdp, u64 start, u64 length, u32 cache_level)

-:144: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#144: FILE: drivers/gpu/drm/i915/i915_vgpu.c:346:
+static void gen8_ppgtt_clear_4lvl_pv(struct i915_address_space *vm,
+		u64 start, u64 length)

-:151: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#151: FILE: drivers/gpu/drm/i915/i915_vgpu.c:353:
+	vgpu_ppgtt_pv_update(dev_priv, PV_ACTION_PPGTT_L4_CLEAR,
+		px_dma(ppgtt->pd), start, length, 0);

-:155: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#155: FILE: drivers/gpu/drm/i915/i915_vgpu.c:357:
+static void gen8_ppgtt_insert_4lvl_pv(struct i915_address_space *vm,
+		struct i915_vma *vma,

-:165: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#165: FILE: drivers/gpu/drm/i915/i915_vgpu.c:367:
+	vgpu_ppgtt_pv_update(dev_priv, PV_ACTION_PPGTT_L4_INSERT,
+		px_dma(ppgtt->pd), start, length, cache_level);

-:169: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#169: FILE: drivers/gpu/drm/i915/i915_vgpu.c:371:
+static int gen8_ppgtt_alloc_4lvl_pv(struct i915_address_space *vm,
+		u64 start, u64 length)

-:187: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#187: FILE: drivers/gpu/drm/i915/i915_vgpu.c:389:
+void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
+		enum pv_caps cap, void *data)

-:237: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#237: FILE: drivers/gpu/drm/i915/i915_vgpu.h:140:
+intel_vgpu_enabled_pv_caps(struct drm_i915_private *dev_priv,
+		enum pv_caps cap)

-:240: CHECK:LOGICAL_CONTINUATIONS: Logical continuations should be on the previous line
#240: FILE: drivers/gpu/drm/i915/i915_vgpu.h:143:
+	return (dev_priv->vgpu.active) && intel_vgpu_has_pv_caps(dev_priv)
+			&& (dev_priv->vgpu.pv_caps & cap);

-:251: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#251: FILE: drivers/gpu/drm/i915/i915_vgpu.h:166:
+void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
+		enum pv_caps cap, void *data);

total: 0 errors, 0 warnings, 17 checks, 190 lines checked
2c3dc08988b7 drm/i915: vgpu context submission pv optimization
-:136: CHECK:SPACING: spaces preferred around that '/' (ctx:VxV)
#136: FILE: drivers/gpu/drm/i915/i915_vgpu.h:33:
+#define PV_ELSP_OFF		(PAGE_SIZE/8)
                    		          ^

-:164: CHECK:UNCOMMENTED_DEFINITION: spinlock_t definition without comment
#164: FILE: drivers/gpu/drm/i915/i915_vgpu.h:67:
+	spinlock_t lock;

-:187: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#187: 
new file mode 100644

-:220: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#220: FILE: drivers/gpu/drm/i915/intel_pv_submission.c:29:
+static void pv_submit(struct intel_engine_cs *engine,
+	       struct i915_request **out,

-:249: CHECK:BRACES: Blank lines aren't necessary before a close brace '}'
#249: FILE: drivers/gpu/drm/i915/intel_pv_submission.c:58:
+
+}

-:293: CHECK:LINE_SPACING: Please don't use multiple blank lines
#293: FILE: drivers/gpu/drm/i915/intel_pv_submission.c:102:
+
+

-:481: CHECK:MULTIPLE_ASSIGNMENTS: multiple assignments should be avoided
#481: FILE: drivers/gpu/drm/i915/intel_pv_submission.c:290:
+	engine->park = engine->unpark = NULL;

total: 0 errors, 1 warnings, 6 checks, 423 lines checked
5dc5073bd1dc drm/i915/gvt: GVTg handle pv_caps PVINFO register
46ef167fec9f drm/i915/gvt: GVTg handle shared_page setup
-:51: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#51: FILE: drivers/gpu/drm/i915/gvt/gvt.h:697:
+int intel_gvt_read_shared_page(struct intel_vgpu *vgpu,
+		unsigned int offset, void *buf, unsigned long len);

-:53: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#53: FILE: drivers/gpu/drm/i915/gvt/gvt.h:699:
+int intel_gvt_write_shared_page(struct intel_vgpu *vgpu,
+		unsigned int offset, void *buf, unsigned long len);

-:131: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#131: FILE: drivers/gpu/drm/i915/gvt/vgpu.c:601:
+int intel_gvt_read_shared_page(struct intel_vgpu *vgpu,
+		unsigned int offset, void *buf, unsigned long len)

-:150: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#150: FILE: drivers/gpu/drm/i915/gvt/vgpu.c:620:
+int intel_gvt_write_shared_page(struct intel_vgpu *vgpu,
+		unsigned int offset, void *buf, unsigned long len)

total: 0 errors, 0 warnings, 4 checks, 122 lines checked
ac05391ac0b1 drm/i915/gvt: GVTg support ppgtt pv optimization
-:83: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#83: FILE: drivers/gpu/drm/i915/gvt/gtt.c:1836:
+		gvt_vgpu_err("fail to create ppgtt for pdp 0x%llx\n",
+				px_dma(mm->ppgtt->pd));

-:122: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'end' - possible side-effects?
#122: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2886:
+#define pml4_addr_end(addr, end)					\
+({	unsigned long __boundary = \
+			((addr) + GEN8_PML4E_SIZE) & GEN8_PML4E_SIZE_MASK; \
+	(__boundary < (end)) ? __boundary : (end);		\
+})

-:128: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'end' - possible side-effects?
#128: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2892:
+#define pdp_addr_end(addr, end)						\
+({	unsigned long __boundary = \
+			((addr) + GEN8_PDPE_SIZE) & GEN8_PDPE_SIZE_MASK; \
+	(__boundary < (end)) ? __boundary : (end);		\
+})

-:134: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'end' - possible side-effects?
#134: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2898:
+#define pd_addr_end(addr, end)						\
+({	unsigned long __boundary = \
+			((addr) + GEN8_PDE_SIZE) & GEN8_PDE_SIZE_MASK;	\
+	(__boundary < (end)) ? __boundary : (end);		\
+})

-:147: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#147: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2911:
+static int walk_pt_range(struct intel_vgpu *vgpu, u64 pt,
+				u64 start, u64 end, struct ppgtt_walk *walk)

-:160: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#160: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2924:
+	ret = intel_gvt_hypervisor_read_gpa(vgpu,
+		(pt & PAGE_MASK) + (start_index << info->gtt_entry_size_shift),

-:181: CHECK:LINE_SPACING: Please don't use multiple blank lines
#181: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2945:
+
+

-:183: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#183: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2947:
+static int walk_pd_range(struct intel_vgpu *vgpu, u64 pd,
+				u64 start, u64 end, struct ppgtt_walk *walk)

-:195: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#195: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2959:
+		ret = intel_gvt_hypervisor_read_gpa(vgpu,
+			(pd & PAGE_MASK) + (index <<

-:208: CHECK:LINE_SPACING: Please don't use multiple blank lines
#208: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2972:
+
+

-:210: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#210: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2974:
+static int walk_pdp_range(struct intel_vgpu *vgpu, u64 pdp,
+				  u64 start, u64 end, struct ppgtt_walk *walk)

-:222: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#222: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2986:
+		ret = intel_gvt_hypervisor_read_gpa(vgpu,
+			(pdp & PAGE_MASK) + (index <<

-:234: CHECK:LINE_SPACING: Please don't use multiple blank lines
#234: FILE: drivers/gpu/drm/i915/gvt/gtt.c:2998:
+
+

-:236: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#236: FILE: drivers/gpu/drm/i915/gvt/gtt.c:3000:
+static int walk_pml4_range(struct intel_vgpu *vgpu, u64 pml4,
+				u64 start, u64 end, struct ppgtt_walk *walk)

-:247: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#247: FILE: drivers/gpu/drm/i915/gvt/gtt.c:3011:
+		ret = intel_gvt_hypervisor_read_gpa(vgpu,
+			(pml4 & PAGE_MASK) + (index <<

-:260: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#260: FILE: drivers/gpu/drm/i915/gvt/gtt.c:3024:
+static int intel_vgpu_pv_ppgtt_insert_4lvl(struct intel_vgpu *vgpu,
+		struct intel_vgpu_mm *mm,

-:278: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#278: FILE: drivers/gpu/drm/i915/gvt/gtt.c:3042:
+	walk.mfns = kmalloc_array(num_pages,
+			sizeof(unsigned long), GFP_KERNEL);

-:323: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#323: FILE: drivers/gpu/drm/i915/gvt/gtt.c:3087:
+int intel_vgpu_handle_pv_ppgtt_update(struct intel_vgpu *vgpu,
+		u32 action, struct pv_ppgtt_update *pv_ppgtt)

-:355: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#355: FILE: drivers/gpu/drm/i915/gvt/gtt.c:3119:
+		ret = intel_vgpu_pv_ppgtt_insert_4lvl(vgpu, mm,
+				pdp, start, length, cache_level);

-:394: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#394: FILE: drivers/gpu/drm/i915/gvt/gtt.h:291:
+int intel_vgpu_handle_pv_ppgtt_update(struct intel_vgpu *vgpu,
+		u32 action, struct pv_ppgtt_update *pv_ppgtt);

-:404: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'vgpu' - possible side-effects?
#404: FILE: drivers/gpu/drm/i915/gvt/gvt.h:56:
+#define VGPU_PVCAP(vgpu, cap)	\
+	((vgpu_vreg_t(vgpu, vgtif_reg(pv_caps)) & (cap)) \
+			&& vgpu->shared_page_enabled)

-:406: CHECK:LOGICAL_CONTINUATIONS: Logical continuations should be on the previous line
#406: FILE: drivers/gpu/drm/i915/gvt/gvt.h:58:
+	((vgpu_vreg_t(vgpu, vgtif_reg(pv_caps)) & (cap)) \
+			&& vgpu->shared_page_enabled)

-:430: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#430: FILE: drivers/gpu/drm/i915/gvt/handlers.c:1223:
+static int fetch_pv_command_buffer(struct intel_vgpu *vgpu,
+		struct vgpu_pv_ct_buffer_desc *desc,

-:451: CHECK:SPACING: spaces preferred around that '/' (ctx:VxV)
#451: FILE: drivers/gpu/drm/i915/gvt/handlers.c:1244:
+	head = desc->head/4;
 	                 ^

-:452: CHECK:SPACING: spaces preferred around that '/' (ctx:VxV)
#452: FILE: drivers/gpu/drm/i915/gvt/handlers.c:1245:
+	tail = desc->tail/4;
 	                 ^

-:453: CHECK:SPACING: spaces preferred around that '/' (ctx:VxV)
#453: FILE: drivers/gpu/drm/i915/gvt/handlers.c:1246:
+	size = desc->size/4;
 	                 ^

-:497: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#497: FILE: drivers/gpu/drm/i915/gvt/handlers.c:1290:
+		ret = intel_gvt_read_shared_page(vgpu, off, &data[avail],
+				(len - avail) * 4);

-:536: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#536: FILE: drivers/gpu/drm/i915/gvt/handlers.c:1329:
+	ret = intel_gvt_write_shared_page(vgpu, PV_DESC_OFF,
+			&desc, sizeof(desc));

total: 0 errors, 0 warnings, 28 checks, 521 lines checked
070b7a4248c9 drm/i915/gvt: GVTg support context submission pv optimization
-:72: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#72: FILE: drivers/gpu/drm/i915/gvt/handlers.c:1826:
+	if (intel_gvt_read_shared_page(vgpu, desc_off,
+			&execlist->elsp_dwords.data, 16))

-:93: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#93: FILE: drivers/gpu/drm/i915/gvt/handlers.c:1850:
+	if (VGPU_PVCAP(vgpu, PV_SUBMISSION) &&
+			data == PV_ACTION_ELSP_SUBMISSION)

total: 0 errors, 0 warnings, 2 checks, 69 lines checked

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✓ Fi.CI.BAT: success for i915 vgpu PV to improve vgpu performance

[Patchwork]

== Series Details ==

Series: i915 vgpu PV to improve vgpu performance
URL   : https://patchwork.freedesktop.org/series/66787/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_6905 -> Patchwork_14425
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/

Known issues
------------

  Here are the changes found in Patchwork_14425 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_ctx_create@basic-files:
    - fi-bxt-dsi:         [PASS][1] -> [INCOMPLETE][2] ([fdo#103927])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-bxt-dsi/igt@gem_ctx_create@basic-files.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-bxt-dsi/igt@gem_ctx_create@basic-files.html

  * igt@gem_flink_basic@double-flink:
    - fi-icl-u3:          [PASS][3] -> [DMESG-WARN][4] ([fdo#107724])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-icl-u3/igt@gem_flink_basic@double-flink.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-icl-u3/igt@gem_flink_basic@double-flink.html

  * igt@i915_selftest@live_hangcheck:
    - fi-bsw-kefka:       [PASS][5] -> [INCOMPLETE][6] ([fdo#105876])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-bsw-kefka/igt@i915_selftest@live_hangcheck.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-bsw-kefka/igt@i915_selftest@live_hangcheck.html

  
#### Possible fixes ####

  * igt@gem_ctx_switch@rcs0:
    - fi-icl-u2:          [INCOMPLETE][7] ([fdo#107713]) -> [PASS][8]
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-icl-u2/igt@gem_ctx_switch@rcs0.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-icl-u2/igt@gem_ctx_switch@rcs0.html

  * igt@gem_exec_suspend@basic-s3:
    - fi-blb-e6850:       [INCOMPLETE][9] ([fdo#107718]) -> [PASS][10]
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-blb-e6850/igt@gem_exec_suspend@basic-s3.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-blb-e6850/igt@gem_exec_suspend@basic-s3.html

  * igt@i915_module_load@reload:
    - fi-icl-u3:          [DMESG-WARN][11] ([fdo#107724] / [fdo#111214]) -> [PASS][12]
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-icl-u3/igt@i915_module_load@reload.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-icl-u3/igt@i915_module_load@reload.html

  * igt@i915_selftest@live_reset:
    - {fi-icl-dsi}:       [INCOMPLETE][13] ([fdo#107713]) -> [PASS][14]
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-icl-dsi/igt@i915_selftest@live_reset.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-icl-dsi/igt@i915_selftest@live_reset.html

  * igt@kms_chamelium@hdmi-hpd-fast:
    - fi-kbl-7500u:       [FAIL][15] ([fdo#111407]) -> [PASS][16]
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-kbl-7500u/igt@kms_chamelium@hdmi-hpd-fast.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-kbl-7500u/igt@kms_chamelium@hdmi-hpd-fast.html

  * igt@prime_vgem@basic-fence-flip:
    - fi-ilk-650:         [DMESG-WARN][17] ([fdo#106387]) -> [PASS][18] +1 similar issue
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/fi-ilk-650/igt@prime_vgem@basic-fence-flip.html
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/fi-ilk-650/igt@prime_vgem@basic-fence-flip.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
  [fdo#103927]: https://bugs.freedesktop.org/show_bug.cgi?id=103927
  [fdo#105602]: https://bugs.freedesktop.org/show_bug.cgi?id=105602
  [fdo#105876]: https://bugs.freedesktop.org/show_bug.cgi?id=105876
  [fdo#106107]: https://bugs.freedesktop.org/show_bug.cgi?id=106107
  [fdo#106350]: https://bugs.freedesktop.org/show_bug.cgi?id=106350
  [fdo#106387]: https://bugs.freedesktop.org/show_bug.cgi?id=106387
  [fdo#107713]: https://bugs.freedesktop.org/show_bug.cgi?id=107713
  [fdo#107718]: https://bugs.freedesktop.org/show_bug.cgi?id=107718
  [fdo#107724]: https://bugs.freedesktop.org/show_bug.cgi?id=107724
  [fdo#111214]: https://bugs.freedesktop.org/show_bug.cgi?id=111214
  [fdo#111407]: https://bugs.freedesktop.org/show_bug.cgi?id=111407
  [fdo#111562]: https://bugs.freedesktop.org/show_bug.cgi?id=111562
  [fdo#111597]: https://bugs.freedesktop.org/show_bug.cgi?id=111597


Participating hosts (55 -> 48)
------------------------------

  Additional (1): fi-apl-guc 
  Missing    (8): fi-ilk-m540 fi-cml-h fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-icl-y fi-byt-clapper fi-bdw-samus 


Build changes
-------------

  * CI: CI-20190529 -> None
  * Linux: CI_DRM_6905 -> Patchwork_14425

  CI-20190529: 20190529
  CI_DRM_6905: bd6c56f50d15b22e2348488769580e8a6a378f6b @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_5186: 0008b3e1b2cf7a73b1e995031c9a73fc97b35aad @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
  Patchwork_14425: 070b7a4248c94766767a59256daae6a0daee282d @ git://anongit.freedesktop.org/gfx-ci/linux


== Linux commits ==

070b7a4248c9 drm/i915/gvt: GVTg support context submission pv optimization
ac05391ac0b1 drm/i915/gvt: GVTg support ppgtt pv optimization
46ef167fec9f drm/i915/gvt: GVTg handle shared_page setup
5dc5073bd1dc drm/i915/gvt: GVTg handle pv_caps PVINFO register
2c3dc08988b7 drm/i915: vgpu context submission pv optimization
73862271b37e drm/i915: vgpu ppgtt update pv optimization
afd40d2ba3c1 drm/i915: vgpu pv command buffer support
a176f82a8fcd drm/i915: vgpu shared memory setup for pv optimization
16a1cd5cf965 drm/i915: introduced vgpu pv capability

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/index.html
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH v10 4/9] drm/i915: vgpu ppgtt update pv optimization

[Chris Wilson]

Quoting Xiaolin Zhang (2019-09-17 06:48:15)
> diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
> index e62e9d1..00b187a 100644
> --- a/drivers/gpu/drm/i915/i915_gem_gtt.c
> +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
> @@ -1015,7 +1015,7 @@ static u64 __gen8_ppgtt_clear(struct i915_address_space * const vm,
>         return start;
>  }
>  
> -static void gen8_ppgtt_clear(struct i915_address_space *vm,
> +void gen8_ppgtt_clear(struct i915_address_space *vm,
>                              u64 start, u64 length)
>  {
>         GEM_BUG_ON(!IS_ALIGNED(start, BIT_ULL(GEN8_PTE_SHIFT)));
> @@ -1126,7 +1126,7 @@ static int __gen8_ppgtt_alloc(struct i915_address_space * const vm,
>         return ret;
>  }
>  
> -static int gen8_ppgtt_alloc(struct i915_address_space *vm,
> +int gen8_ppgtt_alloc(struct i915_address_space *vm,
>                             u64 start, u64 length)
>  {
>         u64 from;
> @@ -1326,7 +1326,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
>         } while (iter->sg);
>  }
>  
> -static void gen8_ppgtt_insert(struct i915_address_space *vm,
> +void gen8_ppgtt_insert(struct i915_address_space *vm,
>                               struct i915_vma *vma,
>                               enum i915_cache_level cache_level,
>                               u32 flags)

I think it is unwise to export these. I would suggest focusing on
wrapping the function pointers, for which we need to improve the
constructors.

> diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
> index 6e29a52..e458892 100644
> --- a/drivers/gpu/drm/i915/i915_vgpu.c
> +++ b/drivers/gpu/drm/i915/i915_vgpu.c
> @@ -96,6 +96,9 @@ void i915_detect_vgpu(struct drm_i915_private *dev_priv)
>         dev_priv->vgpu.active = true;
>         mutex_init(&dev_priv->vgpu.lock);
>  
> +       /* guest driver PV capability */
> +       dev_priv->vgpu.pv_caps = PV_PPGTT_UPDATE;
> +
>         if (!intel_vgpu_check_pv_caps(dev_priv, shared_area)) {
>                 DRM_INFO("Virtual GPU for Intel GVT-g detected.\n");
>                 goto out;
> @@ -322,6 +325,82 @@ int intel_vgt_balloon(struct i915_ggtt *ggtt)
>   * i915 vgpu PV support for Linux
>   */
>  
> +static int vgpu_ppgtt_pv_update(struct drm_i915_private *dev_priv,
> +               u32 action, u64 pdp, u64 start, u64 length, u32 cache_level)
> +{
> +       u32 data[8];
> +
> +       data[0] = action;
> +       data[1] = lower_32_bits(pdp);
> +       data[2] = upper_32_bits(pdp);
> +       data[3] = lower_32_bits(start);
> +       data[4] = upper_32_bits(start);
> +       data[5] = lower_32_bits(length);
> +       data[6] = upper_32_bits(length);
> +       data[7] = cache_level;
> +
> +       return intel_vgpu_pv_send(dev_priv, data, ARRAY_SIZE(data));
> +}
> +
> +static void gen8_ppgtt_clear_4lvl_pv(struct i915_address_space *vm,
> +               u64 start, u64 length)
> +{
> +       struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
> +       struct drm_i915_private *dev_priv = vm->i915;
> +
> +       gen8_ppgtt_clear(vm, start, length);
> +       vgpu_ppgtt_pv_update(dev_priv, PV_ACTION_PPGTT_L4_CLEAR,
> +               px_dma(ppgtt->pd), start, length, 0);
> +}
> +
> +static void gen8_ppgtt_insert_4lvl_pv(struct i915_address_space *vm,
> +               struct i915_vma *vma,
> +               enum i915_cache_level cache_level, u32 flags)
> +{
> +       struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
> +       struct drm_i915_private *dev_priv = vm->i915;
> +       u64 start = vma->node.start;
> +       u64 length = vma->node.size;
> +
> +       gen8_ppgtt_insert(vm, vma, cache_level, flags);
> +       vgpu_ppgtt_pv_update(dev_priv, PV_ACTION_PPGTT_L4_INSERT,
> +               px_dma(ppgtt->pd), start, length, cache_level);
> +}
> +
> +static int gen8_ppgtt_alloc_4lvl_pv(struct i915_address_space *vm,
> +               u64 start, u64 length)
> +{
> +       struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
> +       struct drm_i915_private *dev_priv = vm->i915;
> +       int ret;
> +
> +       ret = gen8_ppgtt_alloc(vm, start, length);
> +       if (ret)
> +               return ret;
> +
> +       return vgpu_ppgtt_pv_update(dev_priv, PV_ACTION_PPGTT_L4_ALLOC,
> +               px_dma(ppgtt->pd), start, length, 0);

Unbalanced error path.

> +}
> +
> +/*
> + * config guest driver PV ops for different PV features
> + */
> +void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
> +               enum pv_caps cap, void *data)
> +{
> +       struct i915_ppgtt *ppgtt;
> +
> +       if (!intel_vgpu_enabled_pv_caps(dev_priv, cap))
> +               return;
> +
> +       if (cap == PV_PPGTT_UPDATE) {
> +               ppgtt = (struct i915_ppgtt *)data;
> +               ppgtt->vm.allocate_va_range = gen8_ppgtt_alloc_4lvl_pv;
> +               ppgtt->vm.insert_entries = gen8_ppgtt_insert_4lvl_pv;
> +               ppgtt->vm.clear_range = gen8_ppgtt_clear_4lvl_pv;
> +       }

I'd rather we just augment the factory than have an invasive multiplexed
function.
-Chris
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH v10 3/9] drm/i915: vgpu pv command buffer support

[Chris Wilson]

Quoting Xiaolin Zhang (2019-09-17 06:48:14)
> based on the shared memory setup between guest and GVT, the simple
> pv command buffer ring was introduced by this patch used to perform
> guest-2-gvt single direction communication.
> 
> v1: initial support, added to address i915 PV v6 patch set comment.
> 
> Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
> ---
>  drivers/gpu/drm/i915/i915_pvinfo.h |   1 +
>  drivers/gpu/drm/i915/i915_vgpu.c   | 193 +++++++++++++++++++++++++++++++++++++
>  drivers/gpu/drm/i915/i915_vgpu.h   |  66 +++++++++++++
>  3 files changed, 260 insertions(+)
> 
> diff --git a/drivers/gpu/drm/i915/i915_pvinfo.h b/drivers/gpu/drm/i915/i915_pvinfo.h
> index 3c63603..db9eebb 100644
> --- a/drivers/gpu/drm/i915/i915_pvinfo.h
> +++ b/drivers/gpu/drm/i915/i915_pvinfo.h
> @@ -49,6 +49,7 @@ enum vgt_g2v_type {
>         VGT_G2V_EXECLIST_CONTEXT_CREATE,
>         VGT_G2V_EXECLIST_CONTEXT_DESTROY,
>         VGT_G2V_SHARED_PAGE_SETUP,
> +       VGT_G2V_PV_SEND_TRIGGER,
>         VGT_G2V_MAX,
>  };
>  
> diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
> index 68fecfd..6e29a52 100644
> --- a/drivers/gpu/drm/i915/i915_vgpu.c
> +++ b/drivers/gpu/drm/i915/i915_vgpu.c
> @@ -322,6 +322,187 @@ int intel_vgt_balloon(struct i915_ggtt *ggtt)
>   * i915 vgpu PV support for Linux
>   */
>  
> +/**
> + * wait_for_desc_update - Wait for the command buffer descriptor update.
> + * @desc:      buffer descriptor
> + * @fence:     response fence
> + * @status:    placeholder for status
> + *
> + * GVT will update command buffer descriptor with new fence and status
> + * after processing the command identified by the fence. Wait for
> + * specified fence and then read from the descriptor status of the
> + * command.
> + *
> + * Return:
> + * *   0 response received (status is valid)
> + * *   -ETIMEDOUT no response within hardcoded timeout
> + * *   -EPROTO no response, CT buffer is in error
> + */
> +static int wait_for_desc_update(struct vgpu_pv_ct_buffer_desc *desc,
> +               u32 fence, u32 *status)
> +{
> +       int err;
> +
> +#define done (READ_ONCE(desc->fence) == fence)
> +       err = wait_for_us(done, 5);
> +       if (err)
> +               err = wait_for(done, 10);
> +#undef done
> +
> +       if (unlikely(err)) {
> +               DRM_ERROR("CT: fence %u failed; reported fence=%u\n",
> +                               fence, desc->fence);
> +       }
> +
> +       *status = desc->status;
> +       return err;
> +}
> +
> +/**
> + * DOC: CTB Guest to GVT request
> + *
> + * Format of the CTB Guest to GVT request message is as follows::
> + *
> + *      +------------+---------+---------+---------+---------+
> + *      |   msg[0]   |   [1]   |   [2]   |   ...   |  [n-1]  |
> + *      +------------+---------+---------+---------+---------+
> + *      |   MESSAGE  |       MESSAGE PAYLOAD                 |
> + *      +   HEADER   +---------+---------+---------+---------+
> + *      |            |    0    |    1    |   ...   |    n    |
> + *      +============+=========+=========+=========+=========+
> + *      |  len >= 1  |  FENCE  |     request specific data   |
> + *      +------+-----+---------+---------+---------+---------+
> + *
> + *                   ^-----------------len-------------------^
> + */
> +static int pv_command_buffer_write(struct i915_virtual_gpu_pv *pv,
> +               const u32 *action, u32 len /* in dwords */, u32 fence)
> +{
> +       struct vgpu_pv_ct_buffer_desc *desc = pv->ctb.desc;
> +       u32 head = desc->head / 4;      /* in dwords */
> +       u32 tail = desc->tail / 4;      /* in dwords */
> +       u32 size = desc->size / 4;      /* in dwords */
> +       u32 used;                       /* in dwords */
> +       u32 header;
> +       u32 *cmds = pv->ctb.cmds;
> +       unsigned int i;
> +
> +       GEM_BUG_ON(desc->size % 4);
> +       GEM_BUG_ON(desc->head % 4);
> +       GEM_BUG_ON(desc->tail % 4);
> +       GEM_BUG_ON(tail >= size);
> +
> +       /*
> +        * tail == head condition indicates empty.
> +        */
> +       if (tail < head)
> +               used = (size - head) + tail;
> +       else
> +               used = tail - head;
> +
> +       /* make sure there is a space including extra dw for the fence */
> +       if (unlikely(used + len + 1 >= size))
> +               return -ENOSPC;
> +
> +       /*
> +        * Write the message. The format is the following:
> +        * DW0: header (including action code)
> +        * DW1: fence
> +        * DW2+: action data
> +        */
> +       header = (len << PV_CT_MSG_LEN_SHIFT) |
> +                (PV_CT_MSG_WRITE_FENCE_TO_DESC) |
> +                (action[0] << PV_CT_MSG_ACTION_SHIFT);
> +
> +       cmds[tail] = header;
> +       tail = (tail + 1) % size;
> +
> +       cmds[tail] = fence;
> +       tail = (tail + 1) % size;
> +
> +       for (i = 1; i < len; i++) {
> +               cmds[tail] = action[i];
> +               tail = (tail + 1) % size;
> +       }
> +
> +       /* now update desc tail (back in bytes) */

wmb();

Answers on a postcard as to why this is required and where the
complementary rmb/mb should be.

> +       desc->tail = tail * 4;
> +       GEM_BUG_ON(desc->tail > desc->size);
> +
> +       return 0;
> +}
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH v10 5/9] drm/i915: vgpu context submission pv optimization

[Chris Wilson]

Quoting Xiaolin Zhang (2019-09-17 06:48:16)
> It is performance optimization to override the actual submisison backend
> in order to eliminate execlists csb process and reduce mmio trap numbers
> for workload submission without context switch interrupt by talking with
> GVT via PV submisison notification mechanism between guest and GVT.
> 
> Use PV_SUBMISSION to control this level of pv optimization.
> 
> v0: RFC.
> v1: rebase.
> v2: added pv ops for pv context submission. to maximize code resuse,
> introduced 2 more ops (submit_ports & preempt_context) instead of 1 op
> (set_default_submission) in engine structure. pv version of
> submit_ports and preempt_context implemented.
> v3:
> 1. to reduce more code duplication, code refactor and replaced 2 ops
> "submit_ports & preempt_contex" from v2 by 1 ops "write_desc"
> in engine structure. pv version of write_des implemented.
> 2. added VGT_G2V_ELSP_SUBMIT for g2v pv notification.
> v4: implemented pv elsp submission tasklet as the backend workload
> submisison by talking to GVT with PV notificaiton mechanism and renamed
> VGT_G2V_ELSP_SUBMIT to VGT_G2V_PV_SUBMISIION.
> v5: addressed v4 comments from Chris, intel_pv_submission.c added.
> v6: addressed v5 comments from Chris, replaced engine id by hw_id.
> v7: rebase.
> v8: addressed v7 comments from Chris.
> 
> Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
> ---
>  drivers/gpu/drm/i915/Makefile              |   2 +-
>  drivers/gpu/drm/i915/gt/intel_lrc.c        |  12 +-
>  drivers/gpu/drm/i915/i915_vgpu.c           |  18 +-
>  drivers/gpu/drm/i915/i915_vgpu.h           |  15 ++
>  drivers/gpu/drm/i915/intel_pv_submission.c | 300 +++++++++++++++++++++++++++++
>  5 files changed, 341 insertions(+), 6 deletions(-)
>  create mode 100644 drivers/gpu/drm/i915/intel_pv_submission.c
> 
> diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
> index 658b930..f500cf1 100644
> --- a/drivers/gpu/drm/i915/Makefile
> +++ b/drivers/gpu/drm/i915/Makefile
> @@ -250,7 +250,7 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \
>         selftests/igt_spinner.o
>  
>  # virtual gpu code
> -i915-y += i915_vgpu.o
> +i915-y += i915_vgpu.o intel_pv_submission.o
>  
>  ifeq ($(CONFIG_DRM_I915_GVT),y)
>  i915-y += intel_gvt.o
> diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
> index a3f0e49..5587aff 100644
> --- a/drivers/gpu/drm/i915/gt/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
> @@ -2933,10 +2933,14 @@ void intel_execlists_set_default_submission(struct intel_engine_cs *engine)
>         engine->unpark = NULL;
>  
>         engine->flags |= I915_ENGINE_SUPPORTS_STATS;
> -       if (!intel_vgpu_active(engine->i915)) {
> -               engine->flags |= I915_ENGINE_HAS_SEMAPHORES;
> -               if (HAS_LOGICAL_RING_PREEMPTION(engine->i915))
> -                       engine->flags |= I915_ENGINE_HAS_PREEMPTION;
> +       engine->flags |= I915_ENGINE_HAS_SEMAPHORES;
> +       if (HAS_LOGICAL_RING_PREEMPTION(engine->i915))
> +               engine->flags |= I915_ENGINE_HAS_PREEMPTION;
> +
> +       if (intel_vgpu_active(engine->i915)) {
> +               engine->flags &= ~I915_ENGINE_HAS_SEMAPHORES;
> +               engine->flags &= ~I915_ENGINE_HAS_PREEMPTION;
> +               intel_vgpu_config_pv_caps(engine->i915, PV_SUBMISSION, engine);
>         }
>  
>         if (INTEL_GEN(engine->i915) >= 12) /* XXX disabled for debugging */
> diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
> index e458892..a488b68 100644
> --- a/drivers/gpu/drm/i915/i915_vgpu.c
> +++ b/drivers/gpu/drm/i915/i915_vgpu.c
> @@ -97,7 +97,7 @@ void i915_detect_vgpu(struct drm_i915_private *dev_priv)
>         mutex_init(&dev_priv->vgpu.lock);
>  
>         /* guest driver PV capability */
> -       dev_priv->vgpu.pv_caps = PV_PPGTT_UPDATE;
> +       dev_priv->vgpu.pv_caps = PV_PPGTT_UPDATE | PV_SUBMISSION;
>  
>         if (!intel_vgpu_check_pv_caps(dev_priv, shared_area)) {
>                 DRM_INFO("Virtual GPU for Intel GVT-g detected.\n");
> @@ -389,6 +389,7 @@ void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
>                 enum pv_caps cap, void *data)
>  {
>         struct i915_ppgtt *ppgtt;
> +       struct intel_engine_cs *engine;
>  
>         if (!intel_vgpu_enabled_pv_caps(dev_priv, cap))
>                 return;
> @@ -399,6 +400,11 @@ void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
>                 ppgtt->vm.insert_entries = gen8_ppgtt_insert_4lvl_pv;
>                 ppgtt->vm.clear_range = gen8_ppgtt_clear_4lvl_pv;
>         }
> +
> +       if (cap == PV_SUBMISSION) {
> +               engine = (struct intel_engine_cs *)data;
> +               vgpu_set_pv_submission(engine);
> +       }
>  }
>  
>  /**
> @@ -594,6 +600,8 @@ static int intel_vgpu_setup_shared_page(struct drm_i915_private *dev_priv,
>         u64 gpa;
>         u16 ver_maj, ver_min;
>         int ret = 0;
> +       int i;
> +       u32 size;
>  
>         /* We allocate 1 page shared between guest and GVT for data exchange.
>          *       ___________.....................
> @@ -666,6 +674,14 @@ static int intel_vgpu_setup_shared_page(struct drm_i915_private *dev_priv,
>         pv->notify = intel_vgpu_pv_notify_mmio;
>         mutex_init(&pv->send_mutex);
>  
> +       /* setup PV per engine data exchange structure */
> +       size = sizeof(struct pv_submission);
> +       for (i = 0; i < PV_MAX_ENGINES_NUM; i++) {
> +               pv->pv_elsp[i] = (void *)base + PV_ELSP_OFF +  size * i;
> +               pv->pv_elsp[i]->submitted = false;
> +               spin_lock_init(&pv->pv_elsp[i]->lock);
> +       }
> +
>         return ret;
>  err:
>         __free_page(virt_to_page(base));
> diff --git a/drivers/gpu/drm/i915/i915_vgpu.h b/drivers/gpu/drm/i915/i915_vgpu.h
> index b0fee5b..c33cb05 100644
> --- a/drivers/gpu/drm/i915/i915_vgpu.h
> +++ b/drivers/gpu/drm/i915/i915_vgpu.h
> @@ -29,6 +29,8 @@
>  
>  #define PV_MAJOR               1
>  #define PV_MINOR               0
> +#define PV_MAX_ENGINES_NUM     (VECS1_HW + 1)
> +#define PV_ELSP_OFF            (PAGE_SIZE/8)
>  #define PV_DESC_OFF            (PAGE_SIZE/4)
>  #define PV_CMD_OFF             (PAGE_SIZE/2)
>  
> @@ -37,6 +39,7 @@
>   */
>  enum pv_caps {
>         PV_PPGTT_UPDATE = 0x1,
> +       PV_SUBMISSION = 0x2,
>  };
>  
>  /* PV actions */
> @@ -45,6 +48,7 @@ enum intel_vgpu_pv_action {
>         PV_ACTION_PPGTT_L4_ALLOC,
>         PV_ACTION_PPGTT_L4_CLEAR,
>         PV_ACTION_PPGTT_L4_INSERT,
> +       PV_ACTION_ELSP_SUBMISSION,
>  };
>  
>  /*
> @@ -56,6 +60,13 @@ struct gvt_shared_page {
>         u16 ver_minor;
>  };
>  
> +/* workload submission pv support data structure */
> +struct pv_submission {
> +       u64 descs[EXECLIST_MAX_PORTS];
> +       bool submitted;
> +       spinlock_t lock;
> +};
> +
>  /*
>   * Definition of the command transport message header (DW0)
>   *
> @@ -100,6 +111,9 @@ struct i915_virtual_gpu_pv {
>         struct gvt_shared_page *shared_page;
>         bool enabled;
>  
> +       /* per engine PV workload submission data */
> +       struct pv_submission *pv_elsp[PV_MAX_ENGINES_NUM];
> +
>         /* PV command buffer support */
>         struct vgpu_pv_ct_buffer ctb;
>         u32 next_fence;
> @@ -164,4 +178,5 @@ bool intel_vgpu_check_pv_caps(struct drm_i915_private *dev_priv,
>                 void __iomem *shared_area);
>  void intel_vgpu_config_pv_caps(struct drm_i915_private *dev_priv,
>                 enum pv_caps cap, void *data);
> +void vgpu_set_pv_submission(struct intel_engine_cs *engine);
>  #endif /* _I915_VGPU_H_ */
> diff --git a/drivers/gpu/drm/i915/intel_pv_submission.c b/drivers/gpu/drm/i915/intel_pv_submission.c
> new file mode 100644
> index 0000000..1083d56
> --- /dev/null
> +++ b/drivers/gpu/drm/i915/intel_pv_submission.c
> @@ -0,0 +1,300 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2018 Intel Corporation
> + */
> +
> +#include "i915_vgpu.h"
> +#include "gt/intel_lrc_reg.h"
> +#include "gt/intel_gt_pm.h"
> +#include "i915_trace.h"
> +
> +#define CTX_DESC_FORCE_RESTORE BIT_ULL(2)
> +
> +static u64 execlists_update_context(struct i915_request *rq)
> +{
> +       struct intel_context *ce = rq->hw_context;
> +       u32 *reg_state = ce->lrc_reg_state;
> +
> +       reg_state[CTX_RING_TAIL + 1] = intel_ring_set_tail(rq->ring, rq->tail);
> +
> +       return ce->lrc_desc;
> +}
> +
> +static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> +{
> +       return rb_entry(rb, struct i915_priolist, node);
> +}
> +
> +static void pv_submit(struct intel_engine_cs *engine,
> +              struct i915_request **out,
> +              struct i915_request **end)
> +{
> +       struct intel_engine_execlists * const execlists = &engine->execlists;
> +       struct i915_virtual_gpu_pv *pv = engine->i915->vgpu.pv;
> +       struct pv_submission *pv_elsp = pv->pv_elsp[engine->hw_id];
> +       struct i915_request *rq;
> +       int n, err;
> +
> +       memset(pv_elsp->descs, 0, sizeof(pv_elsp->descs));
> +       n = 0;
> +
> +       do {
> +               rq = *out++;
> +               pv_elsp->descs[n++] = execlists_update_context(rq);
> +       } while (out != end);
> +
> +       spin_lock(&pv_elsp->lock);
> +       pv_elsp->submitted = true;
> +       writel(PV_ACTION_ELSP_SUBMISSION, execlists->submit_reg);
> +
> +#define done (READ_ONCE(pv_elsp->submitted) == false)
> +       err = wait_for_atomic_us(done, 1000);

Please tell me you have a better plan than adding a 1ms wait with
interrupts off.

> +#undef done
> +       spin_unlock(&pv_elsp->lock);
> +
> +       if (unlikely(err))
> +               DRM_ERROR("PV (%s) workload submission failed\n", engine->name);
> +
> +}


> +void vgpu_set_pv_submission(struct intel_engine_cs *engine)
> +{
> +       /*
> +        * We inherit a bunch of functions from execlists that we'd like
> +        * to keep using:
> +        *
> +        *    engine->submit_request = execlists_submit_request;
> +        *    engine->cancel_requests = execlists_cancel_requests;
> +        *    engine->schedule = execlists_schedule;
> +        *
> +        * But we need to override the actual submission backend in order
> +        * to talk to the GVT with PV notification message.
> +        */
> +
> +       engine->execlists.tasklet.func = vgpu_pv_submission_tasklet;
> +
> +       /* do not use execlists park/unpark */
> +       engine->park = engine->unpark = NULL;

But this was used to enable the interrupts so that your tasklet was
called. guc dropped it because they only plan to run on HW that always
sends the interrupts; your supported HW needs the interrupts enabled on
demand.

Please go and add a test to demonstrate this.
-Chris
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✓ Fi.CI.IGT: success for i915 vgpu PV to improve vgpu performance

[Patchwork]

== Series Details ==

Series: i915 vgpu PV to improve vgpu performance
URL   : https://patchwork.freedesktop.org/series/66787/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_6905_full -> Patchwork_14425_full
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  

Known issues
------------

  Here are the changes found in Patchwork_14425_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_ctx_isolation@bcs0-s3:
    - shard-skl:          [PASS][1] -> [INCOMPLETE][2] ([fdo#104108])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-skl1/igt@gem_ctx_isolation@bcs0-s3.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-skl7/igt@gem_ctx_isolation@bcs0-s3.html

  * igt@gem_ctx_shared@exec-single-timeline-bsd:
    - shard-iclb:         [PASS][3] -> [SKIP][4] ([fdo#110841])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb8/igt@gem_ctx_shared@exec-single-timeline-bsd.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb4/igt@gem_ctx_shared@exec-single-timeline-bsd.html

  * igt@gem_exec_balancer@smoke:
    - shard-iclb:         [PASS][5] -> [SKIP][6] ([fdo#110854])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb4/igt@gem_exec_balancer@smoke.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb5/igt@gem_exec_balancer@smoke.html

  * igt@gem_exec_schedule@preempt-other-chain-bsd:
    - shard-iclb:         [PASS][7] -> [SKIP][8] ([fdo#111325]) +6 similar issues
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb3/igt@gem_exec_schedule@preempt-other-chain-bsd.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb4/igt@gem_exec_schedule@preempt-other-chain-bsd.html

  * igt@gem_exec_schedule@promotion-bsd1:
    - shard-iclb:         [PASS][9] -> [SKIP][10] ([fdo#109276]) +10 similar issues
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb4/igt@gem_exec_schedule@promotion-bsd1.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb6/igt@gem_exec_schedule@promotion-bsd1.html

  * igt@gem_tiled_swapping@non-threaded:
    - shard-glk:          [PASS][11] -> [DMESG-WARN][12] ([fdo#108686])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-glk9/igt@gem_tiled_swapping@non-threaded.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-glk9/igt@gem_tiled_swapping@non-threaded.html

  * igt@i915_pm_rps@reset:
    - shard-kbl:          [PASS][13] -> [FAIL][14] ([fdo#102250])
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-kbl1/igt@i915_pm_rps@reset.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-kbl4/igt@i915_pm_rps@reset.html

  * igt@kms_flip@2x-flip-vs-expired-vblank-interruptible:
    - shard-glk:          [PASS][15] -> [FAIL][16] ([fdo#105363])
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-glk9/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-glk4/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible.html

  * igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-render:
    - shard-iclb:         [PASS][17] -> [FAIL][18] ([fdo#103167])
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb1/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-render.html
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb2/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-render.html

  * igt@kms_plane@plane-panning-bottom-right-suspend-pipe-a-planes:
    - shard-apl:          [PASS][19] -> [DMESG-WARN][20] ([fdo#108566]) +4 similar issues
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-apl7/igt@kms_plane@plane-panning-bottom-right-suspend-pipe-a-planes.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-apl2/igt@kms_plane@plane-panning-bottom-right-suspend-pipe-a-planes.html

  * igt@kms_plane_alpha_blend@pipe-c-coverage-7efc:
    - shard-skl:          [PASS][21] -> [FAIL][22] ([fdo#108145] / [fdo#110403])
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-skl9/igt@kms_plane_alpha_blend@pipe-c-coverage-7efc.html
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-skl6/igt@kms_plane_alpha_blend@pipe-c-coverage-7efc.html

  * igt@kms_plane_lowres@pipe-a-tiling-x:
    - shard-iclb:         [PASS][23] -> [FAIL][24] ([fdo#103166])
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb4/igt@kms_plane_lowres@pipe-a-tiling-x.html
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb6/igt@kms_plane_lowres@pipe-a-tiling-x.html

  * igt@kms_psr@cursor_plane_onoff:
    - shard-iclb:         [PASS][25] -> [INCOMPLETE][26] ([fdo#107713])
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb5/igt@kms_psr@cursor_plane_onoff.html
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb1/igt@kms_psr@cursor_plane_onoff.html

  * igt@kms_psr@psr2_cursor_render:
    - shard-iclb:         [PASS][27] -> [SKIP][28] ([fdo#109441]) +2 similar issues
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb2/igt@kms_psr@psr2_cursor_render.html
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb3/igt@kms_psr@psr2_cursor_render.html

  * igt@perf@polling:
    - shard-skl:          [PASS][29] -> [FAIL][30] ([fdo#110728])
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-skl9/igt@perf@polling.html
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-skl9/igt@perf@polling.html

  
#### Possible fixes ####

  * igt@gem_ctx_isolation@rcs0-s3:
    - shard-apl:          [DMESG-WARN][31] ([fdo#108566]) -> [PASS][32] +4 similar issues
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-apl8/igt@gem_ctx_isolation@rcs0-s3.html
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-apl4/igt@gem_ctx_isolation@rcs0-s3.html

  * igt@gem_exec_schedule@independent-bsd2:
    - shard-iclb:         [SKIP][33] ([fdo#109276]) -> [PASS][34] +18 similar issues
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb3/igt@gem_exec_schedule@independent-bsd2.html
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb4/igt@gem_exec_schedule@independent-bsd2.html

  * igt@gem_exec_schedule@preempt-other-bsd:
    - shard-iclb:         [SKIP][35] ([fdo#111325]) -> [PASS][36] +2 similar issues
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb4/igt@gem_exec_schedule@preempt-other-bsd.html
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb6/igt@gem_exec_schedule@preempt-other-bsd.html

  * igt@kms_draw_crc@draw-method-xrgb8888-pwrite-ytiled:
    - shard-skl:          [FAIL][37] ([fdo#103184] / [fdo#103232] / [fdo#108222]) -> [PASS][38]
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-skl1/igt@kms_draw_crc@draw-method-xrgb8888-pwrite-ytiled.html
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-skl9/igt@kms_draw_crc@draw-method-xrgb8888-pwrite-ytiled.html

  * igt@kms_flip@dpms-vs-vblank-race:
    - shard-glk:          [FAIL][39] ([fdo#111609]) -> [PASS][40]
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-glk2/igt@kms_flip@dpms-vs-vblank-race.html
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-glk4/igt@kms_flip@dpms-vs-vblank-race.html

  * igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-blt:
    - shard-iclb:         [FAIL][41] ([fdo#103167]) -> [PASS][42] +3 similar issues
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb4/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-blt.html
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb5/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-blt.html

  * igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-render:
    - shard-kbl:          [DMESG-WARN][43] ([fdo#103558] / [fdo#105602]) -> [PASS][44] +31 similar issues
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-kbl6/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-render.html
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-kbl4/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-render.html

  * igt@kms_plane@plane-panning-bottom-right-suspend-pipe-c-planes:
    - shard-kbl:          [DMESG-WARN][45] ([fdo#103558] / [fdo#105079] / [fdo#105602]) -> [PASS][46]
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-kbl6/igt@kms_plane@plane-panning-bottom-right-suspend-pipe-c-planes.html
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-kbl1/igt@kms_plane@plane-panning-bottom-right-suspend-pipe-c-planes.html

  * igt@kms_plane_alpha_blend@pipe-b-coverage-7efc:
    - shard-skl:          [FAIL][47] ([fdo#108145] / [fdo#110403]) -> [PASS][48]
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-skl5/igt@kms_plane_alpha_blend@pipe-b-coverage-7efc.html
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-skl2/igt@kms_plane_alpha_blend@pipe-b-coverage-7efc.html

  * igt@kms_plane_lowres@pipe-a-tiling-y:
    - shard-iclb:         [FAIL][49] ([fdo#103166]) -> [PASS][50]
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb8/igt@kms_plane_lowres@pipe-a-tiling-y.html
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb4/igt@kms_plane_lowres@pipe-a-tiling-y.html

  * igt@kms_psr@psr2_suspend:
    - shard-iclb:         [SKIP][51] ([fdo#109441]) -> [PASS][52] +2 similar issues
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb1/igt@kms_psr@psr2_suspend.html
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb2/igt@kms_psr@psr2_suspend.html

  * igt@kms_rotation_crc@multiplane-rotation:
    - shard-apl:          [INCOMPLETE][53] ([fdo#103927]) -> [PASS][54]
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-apl6/igt@kms_rotation_crc@multiplane-rotation.html
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-apl1/igt@kms_rotation_crc@multiplane-rotation.html

  
#### Warnings ####

  * igt@gem_mocs_settings@mocs-reset-bsd2:
    - shard-iclb:         [FAIL][55] ([fdo#111330]) -> [SKIP][56] ([fdo#109276])
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb2/igt@gem_mocs_settings@mocs-reset-bsd2.html
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb5/igt@gem_mocs_settings@mocs-reset-bsd2.html

  * igt@gem_mocs_settings@mocs-settings-bsd2:
    - shard-iclb:         [SKIP][57] ([fdo#109276]) -> [FAIL][58] ([fdo#111330])
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb8/igt@gem_mocs_settings@mocs-settings-bsd2.html
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb4/igt@gem_mocs_settings@mocs-settings-bsd2.html

  * igt@kms_atomic_transition@6x-modeset-transitions-nonblocking-fencing:
    - shard-kbl:          [SKIP][59] ([fdo#105602] / [fdo#109271] / [fdo#109278]) -> [SKIP][60] ([fdo#109271] / [fdo#109278]) +1 similar issue
   [59]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-kbl6/igt@kms_atomic_transition@6x-modeset-transitions-nonblocking-fencing.html
   [60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-kbl4/igt@kms_atomic_transition@6x-modeset-transitions-nonblocking-fencing.html

  * igt@kms_dp_dsc@basic-dsc-enable-edp:
    - shard-iclb:         [DMESG-WARN][61] ([fdo#107724]) -> [SKIP][62] ([fdo#109349])
   [61]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-iclb2/igt@kms_dp_dsc@basic-dsc-enable-edp.html
   [62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-iclb5/igt@kms_dp_dsc@basic-dsc-enable-edp.html

  * igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-cur-indfb-draw-pwrite:
    - shard-kbl:          [SKIP][63] ([fdo#105602] / [fdo#109271]) -> [SKIP][64] ([fdo#109271]) +14 similar issues
   [63]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-kbl6/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-cur-indfb-draw-pwrite.html
   [64]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-kbl4/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-cur-indfb-draw-pwrite.html

  * igt@kms_plane_alpha_blend@pipe-b-alpha-transparant-fb:
    - shard-kbl:          [DMESG-FAIL][65] ([fdo#103558] / [fdo#105602] / [fdo#108145]) -> [FAIL][66] ([fdo#108145]) +1 similar issue
   [65]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6905/shard-kbl6/igt@kms_plane_alpha_blend@pipe-b-alpha-transparant-fb.html
   [66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/shard-kbl4/igt@kms_plane_alpha_blend@pipe-b-alpha-transparant-fb.html

  
  [fdo#102250]: https://bugs.freedesktop.org/show_bug.cgi?id=102250
  [fdo#103166]: https://bugs.freedesktop.org/show_bug.cgi?id=103166
  [fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
  [fdo#103184]: https://bugs.freedesktop.org/show_bug.cgi?id=103184
  [fdo#103232]: https://bugs.freedesktop.org/show_bug.cgi?id=103232
  [fdo#103558]: https://bugs.freedesktop.org/show_bug.cgi?id=103558
  [fdo#103927]: https://bugs.freedesktop.org/show_bug.cgi?id=103927
  [fdo#104108]: https://bugs.freedesktop.org/show_bug.cgi?id=104108
  [fdo#105079]: https://bugs.freedesktop.org/show_bug.cgi?id=105079
  [fdo#105363]: https://bugs.freedesktop.org/show_bug.cgi?id=105363
  [fdo#105602]: https://bugs.freedesktop.org/show_bug.cgi?id=105602
  [fdo#107713]: https://bugs.freedesktop.org/show_bug.cgi?id=107713
  [fdo#107724]: https://bugs.freedesktop.org/show_bug.cgi?id=107724
  [fdo#108145]: https://bugs.freedesktop.org/show_bug.cgi?id=108145
  [fdo#108222]: https://bugs.freedesktop.org/show_bug.cgi?id=108222
  [fdo#108566]: https://bugs.freedesktop.org/show_bug.cgi?id=108566
  [fdo#108686]: https://bugs.freedesktop.org/show_bug.cgi?id=108686
  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109276]: https://bugs.freedesktop.org/show_bug.cgi?id=109276
  [fdo#109278]: https://bugs.freedesktop.org/show_bug.cgi?id=109278
  [fdo#109349]: https://bugs.freedesktop.org/show_bug.cgi?id=109349
  [fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
  [fdo#110403]: https://bugs.freedesktop.org/show_bug.cgi?id=110403
  [fdo#110728]: https://bugs.freedesktop.org/show_bug.cgi?id=110728
  [fdo#110841]: https://bugs.freedesktop.org/show_bug.cgi?id=110841
  [fdo#110854]: https://bugs.freedesktop.org/show_bug.cgi?id=110854
  [fdo#111325]: https://bugs.freedesktop.org/show_bug.cgi?id=111325
  [fdo#111330]: https://bugs.freedesktop.org/show_bug.cgi?id=111330
  [fdo#111609]: https://bugs.freedesktop.org/show_bug.cgi?id=111609


Participating hosts (9 -> 8)
------------------------------

  Missing    (1): pig-glk-j5005 


Build changes
-------------

  * CI: CI-20190529 -> None
  * Linux: CI_DRM_6905 -> Patchwork_14425

  CI-20190529: 20190529
  CI_DRM_6905: bd6c56f50d15b22e2348488769580e8a6a378f6b @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_5186: 0008b3e1b2cf7a73b1e995031c9a73fc97b35aad @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
  Patchwork_14425: 070b7a4248c94766767a59256daae6a0daee282d @ git://anongit.freedesktop.org/gfx-ci/linux
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14425/
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx